levels of AGI
In plain wordsA scheme from Google DeepMind researchers for grading AI by how well it performs and how broad its skills are, from emerging to superhuman, much as self-driving cars are graded by levels.
A framework for classifying AI systems by depth of performance and breadth of generality, proposed so that progress toward AGIartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive. can be measured rather than argued about. Performance levels run from 0 (no AI) through 1 Emerging (about as good as, or a little better than, an unskilled person), 2 Competent (at least the 50th percentile of skilled adults), 3 Expert (90th percentile) and 4 Virtuoso (99th percentile) to 5 Superhuman (better than all humans). Each level applies to narrow or general systems, so a chess engine is narrow superhuman while the chatbots of 2023 were classed as emerging AGI. A separate scale of autonomy, from AI as a tool through consultant, collaborator and expert to autonomous agent, distinguishes what a system can do from how it is deployed. The paper also proposes principles for defining AGI, such as focusing on capabilities rather than mechanisms and on potential rather than deployment. OpenAI was reported in 2024 to use its own five-stage scale: chatbots, reasoners, agents, innovators and organizations.
On the levels of AGI scale, the model might be competent general but expert only in coding.
Origin
Proposed by Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Shane Legg and four other Google DeepMind researchers in 'Levels of AGI: Operationalizing Progress on the Path to AGI' (arXiv, November 2023), published as an ICML 2024 position paper and modeled on the SAE levels of driving automationlevels of driving automation, n.: A six-step scale, from Level 0 to Level 5, describing how much of the driving a car's automation does and how much the human must still watch the road and be ready to take over..
Related
- artificial general intelligenceartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive.
- levels of autonomylevels of autonomy, n.: Ladders that grade how independently an AI system acts — from a tool a person fully controls, through suggesting and asking for approval, up to acting on its own while people merely watch.
- levels of driving automationlevels of driving automation, n.: A six-step scale, from Level 0 to Level 5, describing how much of the driving a car's automation does and how much the human must still watch the road and be ready to take over.
- human-level AIhuman-level AI, n.: AI that can match an ordinary adult across the whole range of thinking tasks, such as talking, learning, reasoning and planning. An older, more intuitive name for the goal now usually called general AI.
- superintelligencesuperintelligence, n.: A mind, usually imagined as an AI, that would leave even the cleverest people far behind in nearly every kind of thinking, not just in one game or skill. None is known to exist.
- narrow AInarrow AI, n.: AI built to do one specific job, or a small set of related jobs, such as playing chess or spotting spam, without being able to turn its skill to unrelated problems.
Sources
- Morris, M. R., Sohl-Dickstein, J., Fiedel, N. et al. (2024). Levels of AGI for Operationalizing Progress on the Path to AGI. ICML 2024 position paper; arXiv:2311.02462. arxiv.org/abs/2311.02462
Mentioned in
artificial general intelligenceartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive., intelligenceintelligence, n.: The ability to learn, understand, reason and solve problems — especially new ones — in order to reach goals. Experts disagree about how to define it precisely, for people and even more for machines., human-level AIhuman-level AI, n.: AI that can match an ordinary adult across the whole range of thinking tasks, such as talking, learning, reasoning and planning. An older, more intuitive name for the goal now usually called general AI., levels of autonomylevels of autonomy, n.: Ladders that grade how independently an AI system acts — from a tool a person fully controls, through suggesting and asking for approval, up to acting on its own while people merely watch., narrow AInarrow AI, n.: AI built to do one specific job, or a small set of related jobs, such as playing chess or spotting spam, without being able to turn its skill to unrelated problems., Shane LeggShane Legg, prop. n.: A New Zealand-born AI researcher who co-founded DeepMind and helped make 'artificial general intelligence' a mainstream term and goal., AGI has been achieved internallyAGI has been achieved internally, phr.: A meme claiming, half-jokingly, that an AI company has secretly built human-level AI and is keeping it hidden; used to tease rumors and hype from AI insiders., Jascha Sohl-DicksteinJascha Sohl-Dickstein, prop. n.: An American scientist who in 2015, with colleagues, introduced diffusion models, the technique behind most modern AI image and video generators, by borrowing ideas from physics.