GPU
In plain wordsA computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
A processor built for massively parallel arithmetic: thousands of simple cores, grouped into streaming multiprocessors, apply the same instructions to many data elements at once and are fed by very high memory bandwidthmemory bandwidth, n.: How fast data can be moved between a chip's memory and its calculators. For chatbots, this often decides how quickly words appear, more than raw calculating speed.. Designed to render graphics, GPUs became general-purpose processors in the 2000s (GPGPUGPGPU, abbr.: Using a graphics chip for ordinary number-crunching instead of drawing pictures — the trick that, years later, made modern AI training practical.) and, with CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI., the default hardware for deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract., whose workloads are dominated by dense matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing.. Data-center GPUs add tensor corestensor core, n.: A small, specialized part of an AI chip that does one job — multiplying small blocks of numbers together — far faster than ordinary circuits can. for low-precision matrix math, high-bandwidth memoryhigh-bandwidth memory, n.: Very fast memory made by stacking memory chips on top of each other and placing them right next to the processor, so AI chips can be fed data quickly enough. packaged beside the processor and fast links such as NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip., so that thousands can work as one machine in distributed trainingdistributed training, n.: Training one AI model on many computer chips at once, splitting up the data or the model itself among them and keeping them in sync, so that jobs that would take a single chip thousands of years finish in weeks.. As of 2026 Nvidia supplies most AI GPUs (the H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power., Blackwell and Rubin families), with AMD's Instinct line the main merchant alternative; Google's TPUTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud. and other custom ASICsASIC, abbr.: A chip whose circuits are laid out to do one kind of work extremely well instead of running any program. AI companies design their own to run their models more cheaply than with off-the-shelf chips. compete as purpose-built accelerators.
We rented eight GPUs by the hour to fine-tune the model overnight.
Origin
Graphics processors predate the term, and Sony called the PlayStation's graphics chip a 'GPU' in 1994, but Nvidia popularized it by marketing the GeForce 256 (October 1999) as the first GPU. Neural networks ran on GPUs from the mid-2000s; Rajat Raina, Anand Madhavan and Andrew Ng (2009) reported large speedups for deep learning, and AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward. (2012), trained on two consumer GTX 580 cards, made GPUs the engine of the deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems..
Usage note
In AI usage a 'GPU' is often a data-center accelerator with no display output at all. GPU counts are comparable only within a generation, since each generation multiplies per-chip throughput.
Related
- GPGPUGPGPU, abbr.: Using a graphics chip for ordinary number-crunching instead of drawing pictures — the trick that, years later, made modern AI training practical.
- CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI.
- tensor coretensor core, n.: A small, specialized part of an AI chip that does one job — multiplying small blocks of numbers together — far faster than ordinary circuits can.
- TPUTPU, abbr.: Google's own family of chips built specifically for AI, used to train and run its Gemini models and rented to other companies through its cloud.
- AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models.
- H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power.
- high-bandwidth memoryhigh-bandwidth memory, n.: Very fast memory made by stacking memory chips on top of each other and placing them right next to the processor, so AI chips can be fed data quickly enough.
- VRAMVRAM, abbr.: The memory on a graphics card. For running AI at home it is the key number: the model has to fit in it, so more memory means bigger models.
- CPUCPU, abbr.: The main general-purpose chip in a computer, good at running many different programs step by step. In AI systems it usually coordinates the work while specialized chips do the heavy math.
Sources
- Raina, R., Madhavan, A. & Ng, A. Y. (2009). Large-scale Deep Unsupervised Learning using Graphics Processors. Proceedings of ICML 2009.
Mentioned in
AI acceleratorAI accelerator, n.: Any chip built or tuned to speed up AI — graphics chips, Google's TPUs, phone 'neural engines' and many startup designs. Such chips do the heavy math of training and running models., computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress., deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract., Jensen HuangJensen Huang, prop. n.: The Taiwanese-American co-founder and chief executive of Nvidia, whose graphics chips became the main engine for training and running modern AI., NvidiaNvidia, prop. n.: An American chip company whose graphics processors became the main hardware for training and running AI, making it one of the most valuable companies in the world., AI stackAI stack, n.: The layers of technology an AI product depends on, from chips and data centers at the bottom, through the models, up to the apps people use, each built on the layer below., AlexNetAlexNet, prop. n.: The neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward., AMDAMD, prop. n.: An American chip company that makes computer processors and graphics chips, and Nvidia's main rival in the chips used for artificial intelligence., Andrew NgAndrew Ng, prop. n.: A British-born American computer scientist whose online courses have taught machine learning to millions; he also co-founded Google's deep learning team, Google Brain, and the online-course company Coursera., ASICASIC, abbr.: A chip whose circuits are laid out to do one kind of work extremely well instead of running any program. AI companies design their own to run their models more cheaply than with off-the-shelf chips., Attention Is All You NeedAttention Is All You Need, prop. n.: The 2017 Google research paper that introduced the transformer, the neural-network design behind ChatGPT, Claude, Gemini and nearly every other modern AI language system., big O notationbig O notation, n.: Shorthand for how the cost of a computation grows as the problem gets bigger. Saying attention costs 'O(n²)' means doubling the length of the text roughly quadruples the work., confidential computingconfidential computing, n.: Running computations inside a hardware-sealed compartment of a chip, so that data and AI models stay protected even from the cloud company or administrator operating the machine., CPUCPU, abbr.: The main general-purpose chip in a computer, good at running many different programs step by step. In AI systems it usually coordinates the work while specialized chips do the heavy math., CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI., deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems., FAISSFAISS, prop. n.: A free software library from Meta for searching huge collections of embeddings very fast, on ordinary processors or graphics cards. Many AI search systems use it under the hood., GPU-hourGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour., H100H100, prop. n.: Nvidia's flagship AI chip of 2022–2024, so scarce and sought-after during the generative-AI boom that it became a unit of measure for AI computing power., inference providerinference provider, n.: A company that runs AI models on its own computers and sells access to them over the internet, charging by use, so customers need not buy or operate the hardware themselves., kernelkernel, n.: A word with several AI meanings: a similarity measure used by some learning methods; a small grid of numbers that scans an image inside a neural network; or a small program that runs on a GPU., Liang WenfengLiang Wenfeng, prop. n.: A Chinese entrepreneur who founded DeepSeek, whose low-cost, openly released AI models shocked global markets in January 2025., liquid coolingliquid cooling, n.: Cooling computer chips by running liquid past them instead of blowing air. AI chips now run so hot and are packed so tightly that air alone can no longer keep them cool., matrixmatrix, n.: A rectangular grid of numbers arranged in rows and columns. Most of what a neural network has learned is stored in large grids like this.