rack-scale system
In plain wordsAn AI computer designed as one whole cabinet of chips, cooling and wiring, rather than separate servers, so all the chips inside can work together like one big chip.
A computer designed and sold as an entire rack, in which dozens of accelerators, their host CPUs, switches, power delivery and usually liquid coolingliquid cooling, n.: Cooling computer chips by running liquid past them instead of blowing air. AI chips now run so hot and are packed so tightly that air alone can no longer keep them cool. are integrated so that every accelerator in the rack shares one high-bandwidth scale-upscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out). domain. Nvidia's GB200 NVL72 (shipping from late 2024) is the archetype: 72 Blackwell GPUs and 36 Grace CPUs linked by NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip. switches with 130 TB/s of aggregate bandwidth and 13.4 TB of HBMhigh-bandwidth memory, n.: Very fast memory made by stacking memory chips on top of each other and placing them right next to the processor, so AI chips can be fed data quickly enough., drawing on the order of 120 kW. Other examples include Huawei's CloudMatrix 384 (2025) and AMD's 72-GPU Helios (announced in 2025 for 2026 availability). The rack, rather than the chip or server, becomes the unit of design, purchase and deployment, which concentrates power density (often over 100 kW per rack) and forces redesign of data-centerAI data center, n.: A building or campus packed with AI chips, power equipment and cooling, built to train and run AI models. The biggest use as much electricity as a city. power and cooling.
Origin
Intel promoted a 'Rack Scale Architecture' for disaggregated data-center hardware from 2013, but the term took its current AI meaning with Nvidia's GB200 NVL72, announced in March 2024 and marketed as acting like a single giant GPU.
Related
- scale-up and scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).
- NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip.
- liquid coolingliquid cooling, n.: Cooling computer chips by running liquid past them instead of blowing air. AI chips now run so hot and are packed so tightly that air alone can no longer keep them cool.
- AI supercomputerAI supercomputer, n.: A huge network of AI chips — often tens or hundreds of thousands — wired together in a data center to train the largest AI models.
- AI data centerAI data center, n.: A building or campus packed with AI chips, power equipment and cooling, built to train and run AI models. The biggest use as much electricity as a city.
- GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
Mentioned in
AI data centerAI data center, n.: A building or campus packed with AI chips, power equipment and cooling, built to train and run AI models. The biggest use as much electricity as a city., AI supercomputerAI supercomputer, n.: A huge network of AI chips — often tens or hundreds of thousands — wired together in a data center to train the largest AI models., FLOPSFLOPS, abbr.: Floating-point operations per second: how many arithmetic steps a computer can do each second. It is the standard speed rating for supercomputers and AI chips., liquid coolingliquid cooling, n.: Cooling computer chips by running liquid past them instead of blowing air. AI chips now run so hot and are packed so tightly that air alone can no longer keep them cool., NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip., power usage effectivenesspower usage effectiveness, n.: A score showing how much of a data center's electricity actually reaches its computers: total power drawn divided by the computers' share. A perfect 1.0 would mean no waste; the extra goes mainly to cooling., scale-up and scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).