FOOM
In plain wordsSlang for a sudden, explosive jump in an AI's abilities, the AI going 'foom' and racing from about human level to vastly superhuman in a very short time.
A very fast takeofffast takeoff, n.: A scenario in which AI goes from about human level to far beyond it very quickly, in days, weeks or a few months, leaving people little time to react., especially one driven by a single AI system's recursive self-improvementrecursive self-improvement, n.: An AI improving its own design or training, so that each improved version is better at making the next improvement. It is the engine usually imagined behind an intelligence explosion. rather than by economy-wide progress; the word imitates the sound of something bursting into flame. It carries assumptions associated with Eliezer Yudkowsky's position: that one project could go from modest capability to decisive strategic advantagedecisive strategic advantage, n.: A lead in technology and power so large that whoever holds it, whether a country, a company or an AI, could impose its will on the whole world. within days or weeks, mainly through software improvements and with little warning. Opponents, notably the economist Robin Hanson, argued that progress comes from many accumulated contributions spread across many actors, so no single system would outgrow the rest of the world. The word survives as shorthand on both sides, as in 'foom' and 'no-foom' worldviews, and is also used as a verb.
His whole threat model assumes the first system to automate research will foom.
Origin
Earliest written use not established. Popularized by the late-2008 exchange of blog posts between Robin Hanson and Eliezer Yudkowsky on Overcoming Bias, later collected by the Machine Intelligence Research Institute as The Hanson-Yudkowsky AI-Foom Debate (2013) together with the transcript of their 2011 in-person debate.
Related
- fast takeofffast takeoff, n.: A scenario in which AI goes from about human level to far beyond it very quickly, in days, weeks or a few months, leaving people little time to react.
- takeofftakeoff, n.: The stretch of time between AI reaching roughly human level and AI becoming vastly superhuman. People argue over whether it would take decades, years, months or just days.
- intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time.
- recursive self-improvementrecursive self-improvement, n.: An AI improving its own design or training, so that each improved version is better at making the next improvement. It is the engine usually imagined behind an intelligence explosion.
- Eliezer YudkowskyEliezer Yudkowsky, prop. n.: An American writer and AI theorist who, from around 2000, argued that superhuman AI would likely destroy humanity unless its goals were made safe, and who founded an online community devoted to rational thinking.
- LessWrongLessWrong, prop. n.: An online forum, founded in 2009, about thinking clearly and avoiding mental biases; it became a main gathering place for people worried about the dangers of advanced AI.
- slow takeoffslow takeoff, n.: A scenario in which AI becomes superhuman step by step, over years or decades, as it spreads through the economy, giving society time to spot problems and respond.
Sources
- Hanson, R. & Yudkowsky, E. (2013). The Hanson-Yudkowsky AI-Foom Debate. Machine Intelligence Research Institute. intelligence.org/ai-foom-debate
Mentioned in
Eliezer YudkowskyEliezer Yudkowsky, prop. n.: An American writer and AI theorist who, from around 2000, argued that superhuman AI would likely destroy humanity unless its goals were made safe, and who founded an online community devoted to rational thinking., fast takeofffast takeoff, n.: A scenario in which AI goes from about human level to far beyond it very quickly, in days, weeks or a few months, leaving people little time to react., intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time., takeofftakeoff, n.: The stretch of time between AI reaching roughly human level and AI becoming vastly superhuman. People argue over whether it would take decades, years, months or just days., Robin HansonRobin Hanson, prop. n.: An American economist who debated Eliezer Yudkowsky about whether AI would take off suddenly or gradually, and who imagined an economy run by digital copies of human minds.