Epoch AI
In plain wordsA research nonprofit that gathers data on how fast AI is advancing — the computing power, data and money behind it — and publishes forecasts and difficult tests for AI.
A research nonprofit that measures and forecasts the drivers of AI progress. Its public databases of notable machine learning models, AI hardware and AI data centersAI data center, n.: A building or campus packed with AI chips, power equipment and cooling, built to train and run AI models. The biggest use as much electricity as a city. are standard sources for estimates of training computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.; its analyses have found that the compute used to train frontier models grew roughly four- to fivefold per year from 2010, examined when developers might exhaust the stock of human-written text, and estimated the pace of algorithmic progress in language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.. It created the FrontierMathFrontierMath, prop. n.: Several hundred new math problems, kept unpublished so AI systems cannot have trained on them, written by professional mathematicians to check whether machines can do advanced, even research-level, mathematics. benchmark (2024) of unpublished research-level mathematics problems — whose funding by OpenAI, disclosed only after OpenAI's o3 results were announced, raised questions about independence — and the Epoch Capabilities Index, and it builds economic models of AI-driven growth such as GATE. Its figures are widely cited in policy debates over scalingscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments. and compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI..
According to Epoch's database, that run used about 5 × 10^25 FLOP.
Origin
Grew out of a volunteer group that from 2021 compiled data on machine learning systems; after its 2022 paper on compute trends across three eras of machine learning it was organized as Epoch AI, directed by Jaime Sevilla, with staff working remotely in several countries.
Related
- FrontierMathFrontierMath, prop. n.: Several hundred new math problems, kept unpublished so AI systems cannot have trained on them, written by professional mathematicians to check whether machines can do advanced, even research-level, mathematics.
- computecompute, n.: The raw computing power used to train and run AI, usually measured by how many specialized chips are used and for how long. With data and algorithms, it is a main driver of AI progress.
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
- AI timelinesAI timelines, n.: Forecasts of when AI will reach major milestones, such as matching humans at most work. People with short timelines expect this within years; others expect decades, or never.
- compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI.
- METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own.
Sources
- Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M. & Villalobos, P. (2022). Compute Trends Across Three Eras of Machine Learning.
Mentioned in
AI timelinesAI timelines, n.: Forecasts of when AI will reach major milestones, such as matching humans at most work. People with short timelines expect this within years; others expect decades, or never., algorithmic progressalgorithmic progress, n.: Improvements in AI methods — not faster chips — that let a model reach the same ability with less computing power. Researchers estimate this has cut the computing needed by half every eight to sixteen months., Chinchilla scaling lawsChinchilla scaling laws, n.: A 2022 DeepMind finding that, for a fixed amount of computing, language models had been made too big and trained on too little text; the best balance is roughly 20 word-pieces of text for every adjustable number inside the model., Stanford HAIStanford HAI, prop. n.: An artificial intelligence institute at Stanford University that funds research, advises policymakers and publishes the AI Index, a widely cited yearly report on the state of AI., training computetraining compute, n.: The total number of arithmetic operations performed to train a model. It is the standard yardstick for the size of an AI training effort, and some laws use it to decide which models face extra rules., transformative AItransformative AI, n.: AI that changes the world as deeply as farming or the Industrial Revolution did. It is defined by its effects on society and the economy, not by how humanlike or clever it is., AI ImpactsAI Impacts, prop. n.: A small research project that studies how and when human-level artificial intelligence might arrive, best known for large surveys asking AI researchers to predict their field's future., data-constrained scalingdata-constrained scaling, n.: How to keep improving AI models when the supply of fresh, high-quality training text runs out — for example by reusing the same data several times, which works well at first and then stops helping., explosive growthexplosive growth, n.: A hypothetical jump in the world economy's growth rate to 30% a year or more, about ten times today's pace, which some economists think AI that replaces human workers could cause.