AI BASICS 101: What Are "AI Chips" and Why Do They Matter (or Do They)?
You're probably hearing a lot lately about AI chips or AI processors and how important they are for the coming AI age (and you've seen NVIDIA's stock reap the benefits!). What, exactly, makes an AI chip different, and what are the actual use cases? For example, are they only important for companies such as OpenAI that makes large LLMs? Can they help your company with its internal AI projects? Do you need them in your laptop if you're running software that has AI features? We'll look at all this and more and offer you links for further reading.
One caveat before we start: Any company can call any chip an "AI" chip -- there is no official standard. All major chip companies are reputable in their manufacturing, but we bring this up because, just like anything labeled "AI" these days, you'll always want to do your own research, especially when making end-user purchases.
What Exactly Is an AI Chip?
Without getting too far into the technical weeds, AI chips are much like other computer chips except that certain features have been optimized to process AI calculations more quickly and efficiently (especially when several chips are combined -- known as multi-core processing).
When it comes to AI, the biggest of these features is parallel processing, which, in its simplest form, means that the chip(s) can simultaneously process many tasks instead of one. Of course, parallel processing has been around for a while, and it's not just used for AI. However, in AI chips, the parallel processing (and other features) in the chip are tailored for AI operations, often at the expense of other operations. If you're running a deep neural network (DNN), for example, which is a type of machine learning algorithm, you're probably running the same kind of mathematical calculations over and over again. Knowing this, the manufacturer can customize the parallel processing features specifically to these, significantly boosting the output, and saving the companies that run these machines (particularly at large scale) everything from time to electricity costs.
It's not just parallel computing architecture that's important. The chip manufacturers can (and do) optimize other aspects of their chips for these kinds of calculations as well. For example, NVIDIA's tensor core graphical processing units are specifically designed to "accelerate the matrix computations involved in neural networks," according to the company.
When Do You Need AI Chips?
Because AI chips are customized for the processes needed for most machine learning, they are very valuable to any company running large AI projects. However, they can also help smaller companies that are running projects that have similar calculations -- it's going to take less time if the chips used for the processes you're running are designed for that process. That's really what we're talking about here with AI chips: They're customized to run AI processes (which are pretty intense, so a lot of work went into this!), although it can mean there could be a downside when used on a large scale for general computing. Specifically, the companies may want processors customized to other types of processing if AI isn't what they're doing daily.
What about AI chips on your laptop or desktop? Can they really help you?
According to this paper from the Center for Security and Emerging Technology (CSET), it's not really the "AI" part of the chips that can help consumers but rather the benefits of all the engineering that has gone into these chips. The Center notes that because of all the re-engineering it took to make these chips so efficient (and using smaller and smaller surfaces), AI chips are now often "more cost-effective than general-purpose chips." Although you might not need the AI customization these processors offer in your day-to-day work, the power you're getting for the price you're paying often make a computer powered by "AI chips" worth it, even if you never dive into machine-learning algorithms or natural language processing (such as chatbots).
Further Reading
This article is aimed at a general audience, so much of the technical information here is greatly (frankly, overly) simplified. To learn more about this topic, we suggest starting with these resources:
- "AI Chips: What They Are and Why They Matter" Whitepaper by Saif M. Khan
- "AI Chips Supply Chain" Web Site
- AI Chip Vendor Breakdown on Github
Posted by Becky Nagel on April 18, 2024