AI Boardroom Insights

Blog archive

AI Q&A: 5 Questions with Microsoft Research's Dr. James McCaffrey

Microsoft Research's Dr. James McCaffrey is a well known technology and AI expert, and author of the popular column "The Data Science Lab," among others. And while he comes from the tech side, he also regularly thinks, writes and speaks about future of AI and its practical implications.

We here at AI Boardroom recently got a chance to ask Dr. McCaffrey about his thoughts on how AI is changing business and what advice he has for executives to stay ahead. Here's what he shared with us:

AI BOARDROOM: AI has always been used by businesses to help them innovate. Do you have any favorite examples?
DR. MCCAFFREY: AI, as we think of the term today, is new when it comes to business innovation. But machine learning, a subset of AI that focuses on predictions made by using data plus an algorithm like a neural network classifier, has been around for many years. I was involved with a couple of projects that predicted the results of American football games with 70 percent accuracy against the Las Vegas point spread. With the rise of legalized gambling, I'm sure there are similar efforts under way.

How do you see AI changing businesses and the future of work?
The rise of AI resembles the birth of the Internet in some ways. Some effects were obvious and immediate. The death of traditional newspapers and print magazines is one example. Other changes were longer in coming and not nearly as obvious. For example, the profound psychological impacts of social media on literally billions of people was not really anticipated. The rise of AI is the same. The world's first trillionaire will likely be someone who has a long-term vision for an AI application that others don't see.

You've seen a lot of AI implementations. Do you see common threads among  people that have successful AI projects? Or, conversely, unsuccessful projects?
Most of the successful AI projects I've seen have been relatively small scale -- ones that didn't bite off more than they can chew. These small projects typically involve only about three people.

What AI-related technology are you most excited about right now?
I'm closely following efforts to fine-tune large language models. A large language model like GPT-4 is somewhat like a high school senior who knows English grammar and has a Wikipedia-level knowledge of basic facts. A fine-tuned large language model is somewhat like a PhD graduate who has deeper knowledge of a specific area. As recently as 12 months ago, fine-tuning a large language model wasn't feasible except by large tech companies. But tools are being rapidly developed and the ability of an ordinary human to create a custom fine-tuned large language model likely isn't far off. A related development I'm watching closely is how these fine-tuned agents can communicate with each other.

How deep of an understanding of AI technology does, for example, a CEO need in today's world?
History is littered with stories of businesses that missed key opportunities or threats. For example, in 1990, the average length of time that a Fortune 500 company stayed on the index was only 20 years. It's predicted that only half of the current 500 companies will be on the index 10 years from now. CEOs must have a grasp of AI at a high level so that they can hire and understand technical advisors -- and separate hype from practicality.

BONUS QUESTION: What is one thing about AI you want AI Boardroom's readers to know?
AI is both over-hyped and under-hyped. Successful businesses will be the ones that can distinguish unrealistic claims and goals from practical opportunities. Put another way, don't believe everything you read about AI -- including this interview!

You can read more from Dr. McCaffrey on AI via his column "The Data Science Lab" on and his regular contributions to He can be found on LinkedIn here.

Posted by Becky Nagel on May 14, 2024