petaflop/s-day
In plain wordsA unit of computing work: what a computer doing a thousand trillion calculations per second gets done in one day. It is used to compare the cost of training AI models.
A unit of total computation equal to sustaining 10^15 operations per second for one day: 10^15 × 86,400 ≈ 8.64 × 10^19, conventionally rounded to about 10^20 FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules.. It expressed training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules. in manageable numbers before raw FLOP counts in scientific notation became standard. The GPT-3 paper (2020), for example, reported about 3,640 petaflop/s-days for its 175-billion-parameter model (≈3.14 × 10^23 FLOP). OpenAI's convention counted the operations performed by the neural network itself, not the hardware's peak capacity over the period.
Origin
Used as the headline unit of Dario Amodei and Danny Hernandez's OpenAI post 'AI and Compute' (May 2018), which charted landmark systems in petaflop/s-days, and adopted in OpenAI papers such as the GPT-3 report (2020).
Usage note
Largely superseded by plain FLOP counts (e.g., 3 × 10^23 FLOP), which regulators also use.
Related
- FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules.
- training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules.
- computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.
- GPU-hourGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour.
Sources
- Amodei, D. & Hernandez, D. (2018). AI and Compute. OpenAI blog. openai.com/index/ai-and-compute
- Brown, T. B. et al. (2020). Language Models are Few-Shot Learners. arXiv:2005.14165.
Mentioned in
FLOPFLOP, n.: A single arithmetic step, such as multiplying or adding two numbers. Counting them measures how much computing went into an AI model, and laws use such counts to decide which models face extra rules., GPU-hourGPU-hour, n.: One graphics chip working for one hour — the everyday unit for how much computing a job used and for renting AI chips by the hour.