METR time horizon
In plain wordsA way of measuring AI progress by asking how long a task — timed by how long it takes a skilled human — an AI agent can complete with even odds of success.
A metric that expresses an AI agent's capability as the length of tasks it can complete with a given probability of success, where length is the time skilled human professionals take. The 50% time horizon is estimated by running an agent many times on a suite of software-engineering, machine-learning, cybersecurity and reasoning tasks with measured human completion times (from seconds to many hours), fitting a logistic curve of success probability against the logarithm of human time, and reading off the duration at which predicted success is 50%; the 80% horizon comes from the same curve. METR's March 2025 study found that the frontier 50% horizon had doubled roughly every seven months from 2019 to 2024, with Claude 3.7 Sonnet at about 50 minutes. An update using a larger task suite (Time Horizon 1.1, January 2026) put the doubling time since 2023 at about four months. As of May 2026 the highest estimate, for an early version of Claude Mythos Preview, was at least 16 hours, the limit beyond which METR says its task suite cannot measure reliably; by September 2026 METR had stopped actively updating its public tracker.
If the time-horizon trend holds, agents will manage week-long tasks at 50% reliability within a few years.
Origin
Introduced by Thomas Kwa, Ben West, Joel Becker and some two dozen colleagues at METR (Model Evaluation and Threat Research) in 'Measuring AI Ability to Complete Long Tasks', posted in March 2025. Its exponential chart became one of the most cited graphs in AI forecasting and policy debates of 2025–2026.
Usage note
A horizon measures task difficulty in human time, not how long an agent can run unattended. Critics cite wide confidence intervals, a narrow, mostly software task suite, tasks cleaner than real work, and extrapolations sensitive to the doubling rate; METR itself has published notes on these limits and on cost-aware alternatives (2026).
Related
- RE-BenchRE-Bench, prop. n.: A test from the research group METR that pits AI agents against human experts on seven open-ended machine-learning engineering projects, comparing what each achieves in the same time.
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
- long-horizon tasklong-horizon task, n.: A job that takes many steps and a long time — hours or days of work rather than a single answer — so an AI must stay on track and recover from its own mistakes.
- AI timelinesAI timelines, n.: Forecasts of when AI will reach major milestones, such as matching humans at most work. People with short timelines expect this within years; others expect decades, or never.
- automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors.
- superhuman codersuperhuman coder, n.: A milestone used by AI forecasters: an AI that can do the coding work of a company's best programmer, far faster and in many copies at once.
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
Sources
- Kwa, T., West, B., Becker, J. et al. (2025). Measuring AI Ability to Complete Long Tasks. arXiv:2503.14499. arxiv.org/abs/2503.14499v1
- METR. Time horizons of frontier AI models (tracker). metr.org/time-horizons
Mentioned in
coding agentcoding agent, n.: An AI agent that writes and fixes software by itself: it reads a codebase, edits files, runs commands and tests, and keeps working until a feature is built or a bug is fixed., AI timelinesAI timelines, n.: Forecasts of when AI will reach major milestones, such as matching humans at most work. People with short timelines expect this within years; others expect decades, or never., automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors., long-horizon tasklong-horizon task, n.: A job that takes many steps and a long time — hours or days of work rather than a single answer — so an AI must stay on track and recover from its own mistakes., METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own., task (machine learning)task (machine learning), n.: The job an AI model is meant to do, such as translating sentences, spotting spam or answering questions, defined by its inputs, its expected outputs and how success is scored., RE-BenchRE-Bench, prop. n.: A test from the research group METR that pits AI agents against human experts on seven open-ended machine-learning engineering projects, comparing what each achieves in the same time., Remote Labor IndexRemote Labor Index, prop. n.: A test that gives AI agents real paid freelance projects — design, video, data work and more — and checks how many they finish to a standard a client would accept. Early agents managed very few., superhuman codersuperhuman coder, n.: A milestone used by AI forecasters: an AI that can do the coding work of a company's best programmer, far faster and in many copies at once., Vending-BenchVending-Bench, prop. n.: A test in which an AI agent runs a simulated vending-machine business for months — ordering stock, setting prices, paying fees — to see whether it stays sensible and profitable over a long time.