InfiniBand
In plain wordsA very fast networking technology used to connect the thousands of servers in supercomputers and AI training clusters, designed so data moves with minimal delay.
A switched networking technology, defined by an industry specification, built for supercomputer-style workloads: links of very high bandwidth with latency around a microsecond, native RDMARDMA, abbr.: A way for one computer to read or write another computer's memory directly over the network, skipping the usual software middlemen, so data moves faster with less delay., lossless credit-based flow control, and adapters that take networking work off the CPU. Per-port speeds (four lanes) have risen from 8 Gb/s (SDR, early 2000s) through 200 Gb/s (HDR, 2018) and 400 Gb/s (NDR, 2022) to 800 Gb/s (XDR, 2024). Nvidia's Quantum switches and ConnectX adapters, inherited from Mellanox, dominate the market, and in-network computing such as SHARP performs parts of all-reduceall-reduce, n.: A group operation in which every chip contributes a set of numbers, the sets are added together, and every chip ends up with the same total. It is how parallel training keeps model copies in sync. inside the switches. InfiniBand was long the default scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out). fabric for GPU training clusters, but Ethernet with RDMA (RoCE) and the Ultra Ethernet specification now compete strongly; Meta, for example, built twin 24,576-GPU clusters in 2024, one on each.
Origin
Formed in 1999 by merging two rival server-I/O proposals, Future I/O and Next Generation I/O; the InfiniBand Trade Association released version 1.0 of the specification in 2000. Mellanox (founded 1999) became its main supplier and was acquired by Nvidia for $6.9 billion (announced March 2019, completed 2020).
Usage note
Originally meant to replace PCI as a general server I/O bus; it found its lasting niche in supercomputing and AI clusters instead.
Related
- RDMARDMA, abbr.: A way for one computer to read or write another computer's memory directly over the network, skipping the usual software middlemen, so data moves faster with less delay.
- interconnectinterconnect, n.: The wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one.
- scale-up and scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).
- NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip.
- AI supercomputerAI supercomputer, n.: A huge network of AI chips — often tens or hundreds of thousands — wired together in a data center to train the largest AI models.
- all-reduceall-reduce, n.: A group operation in which every chip contributes a set of numbers, the sets are added together, and every chip ends up with the same total. It is how parallel training keeps model copies in sync.
- NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip.
Mentioned in
NvidiaNvidia, prop. n.: An American chip company whose graphics processors became the main hardware for training and running AI, making it one of the most valuable companies in the world., distributed trainingdistributed training, n.: Training one AI model on many computer chips at once, splitting up the data or the model itself among them and keeping them in sync, so that jobs that would take a single chip thousands of years finish in weeks., interconnectinterconnect, n.: The wiring and networking that connect chips to each other — inside a server, across a data center, or within a single chip — so they can share data and work as one., NCCLNCCL, prop. n.: NVIDIA's software library that lets its graphics chips exchange data with each other quickly — inside one computer and across networks — and that nearly all large-scale AI training on NVIDIA hardware relies on., NVLinkNVLink, prop. n.: Nvidia's private high-speed connection between its AI chips, much faster than ordinary computer connections, letting dozens of chips share memory and act almost like one giant chip., RDMARDMA, abbr.: A way for one computer to read or write another computer's memory directly over the network, skipping the usual software middlemen, so data moves faster with less delay., scale-up and scale-outscale-up and scale-out, phr.: Two ways to get more computing power: make each machine bigger and more tightly linked (scale up), or add more machines connected by a network (scale out).