First, we had CPU and GPU. Then got here AMD’s Accelerated Processing Unit (APU, in the event you’re nasty). And with the appearance of the AI PC and smartphones, we have welcomed the NPU (Neural Processing Unit). However now, due to Google, we’re getting yet one more chip: the TPU. First internally deployed by Google again in 2015, the TPU, or Tensor Processing Unit, is the corporate’s proprietary AI accelerator designed to deal with all of the complicated mathematical calculations vital for synthetic intelligence and machine studying (ML) workloads. Designed for cloud-based purposes, the TPU is used primarily for knowledge facilities. Nevertheless, this time period has not too long ago popped up within the new Pixel 11 smartphone by the use of its Google Tensor G6 chip, which the corporate says packs “50 % extra TPU compute.”
So which is it? The reply is confusingly each. Within the Pixel 11 sequence smartphones, the TPU is principally Google’s fancy-schmancy strategy to say NPU. This model of a TPU handles digicam and picture processing in addition to native AI duties. However the knowledge heart TPU is decidedly totally different from a smartphone model, a conventional NPU and a GPU, though there are a couple of similarities. With a view to assist hold your chipset lingo straight, we’re right here to formally introduce you to the actual TPU and clear up a few of the misconceptions.
What the heck is a TPU?
Nice query. A TPU is a chip Google created to cope with cloud-based AI and machine studying duties within the firm’s knowledge facilities. Often known as an AI accelerator, a TPU is optimized to carry out hundreds of thousands of mathematical calculations on the fly in AI fashions, one thing that NPUs and GPUs do as nicely.
We have been speaking so much about AI and ML, which for many people will name into play an NPU. Because it ought to, since we have been crushed over the pinnacle with them advert nauseum due to Apple, the varied PC and smartphone makers and the chip producers. Should you’re unfamiliar, NPUs are the chips you will discover in all of your trendy smartphones, Macs and PCs. They do all of the heavy lifting to your generative and agentic AI duties, whether or not it is erasing an ex from a photograph or utilizing ChatGPT to construct out a three-day journey itinerary. For the Pixel 11 telephones, the TPU replaces the NPU, with Google claiming “as much as 3.5 occasions quicker AI processing whereas utilizing as much as 3.5 occasions much less vitality.”
GPUs are acquainted to many people, particularly in the event you’re a gamer or work with 3D fashions. Earlier than AI, we wished a robust GPU to run graphically demanding video games like Crysis 3 on the highest settings with blistering body charges and buttery easy rendering. Submit-AI, GPUs nonetheless do this in addition to deal with AI coaching and crypto mining, therefore why one of the best ones price an arm, a leg and a firstborn. A jack-of-all-trades, a GPU can do many issues nicely relying on how highly effective it’s.
So what is the distinction between TPU, NPU and GPU?
The issues that set these chipsets aside are largely use instances and scale. Whereas TPUs are used for cloud-based AI operating in Google knowledge facilities, NPUs, on-device TPUs and GPUs may be present in your smartphones and laptops.
TPUs have the most important scale by far. Because of its specialised structure, a TPU is employed in accelerating large computations utilized in LLM (Giant Language Fashions) and big deep studying (DL) duties. TPUs make the most of a {hardware} structure known as a systolic array that enables knowledge to go from one unit to the following in a 2D grid of multipliers, which assist remove potential bottlenecks. Meaning the calculation output turns into the enter for the following unit with out having to write down again to reminiscence, not like GPUs, that are caught juggling knowledge between their compute models and their high-bandwidth reminiscence.
When servicing large AI firms similar to Anthropic and Midjourney, which serve billions of AI requests day by day, counting on a GPU-based resolution may be pricey when it comes to bottlenecks, latency and energy effectivity. Since a TPU is not biking backwards and forwards to learn and write reminiscence, it is a greater match for coaching large-scale neural networks and different large workloads.
Dwelling in our smartphones, IoT and laptops, NPUs and Google’s on-device TPUs are relegated to the shallow finish of the AI/ML pool. They’re optimized for low-power duties, which is why they’re good for issues like digicam results and real-time translation. A TPU can do an LLM, however it must be a small and actually well-defined native one.
Which one is best?
“Higher” is relative. Every AI accelerator is greatest in a selected use case. NPUs and on-device TPUs are higher for dealing with the smaller mainstream AI duties. Their low energy consumption means they will not be an excessive amount of of a draw in your smartphone, laptop computer or smartwatch. Nevertheless, it is obtained sufficient kick to do real-time translation, blur your background throughout a name or enable you keep in mind that obscure present you swore you noticed again within the ’90s.
TPUs are the cost- and energy-efficient resolution that helps run the AI/ML/DL present globally. Processing hundreds of thousands of mathematical calculations whereas coaching ML and DL fashions at large scale means TPUs are purely for large knowledge facilities. With their innate flexibility, GPUs can tackle each small- and large-scale AI workloads in addition to run video games, edit video and run simulations. If you wish to run native LLMs, a beefy GPU is the way in which to go.
