LessWrong
In plain wordsAn online forum, founded in 2009, about thinking clearly and avoiding mental biases; it became a main gathering place for people worried about the dangers of advanced AI.
An online forum and group blog devoted to human rationality, probability and decision theory and, increasingly, to AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this. and existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown. from AI. Built around Eliezer Yudkowsky's essays on reasoning and cognitive bias (later collected as 'the Sequences'), it became the hub of the rationalist communityrationalist community, n.: A loose online community that prizes careful reasoning and spotting one's own biases; gathered around the forum LessWrong, it was one of the first groups to take the dangers of advanced AI seriously. and an incubator for AI-safety concepts and vocabulary, hosting early or influential discussions of instrumental convergenceinstrumental convergence, n.: The idea that almost any goal is easier to reach if you stay switched on, keep your goal and gain resources and power, so very different AIs might all seek those same things., mesa-optimizationmesa-optimization, n.: When training an AI produces a model that is itself a goal-seeking planner, with goals of its own that may differ from the ones it was trained for., Roko's basiliskRoko's basilisk, n.: A notorious internet thought experiment: a future super-powerful AI might punish everyone who foresaw it but did not help bring it about, so merely hearing the argument would put you at risk. Almost nobody accepts it. and the Waluigi effectWaluigi effect, n.: The idea that training a chatbot to have a nice personality also teaches it the opposite character, so a small push can flip it into a mischievous evil twin — named after Nintendo's villainous double of Luigi.. A sister site, the AI Alignment Forum (2018), carries more technical research and cross-posts to it. Critics describe the culture as insular and its speculation as weakly grounded; admirers credit it with taking AI risk seriously years before the subject reached the mainstream.
The argument first appeared as a LessWrong post before it became a conference paper.
Origin
Launched by Eliezer Yudkowsky in February 2009, growing out of Overcoming Bias, a group blog he had written since 2006 with the economist Robin Hanson. After declining in the mid-2010s, it was rebuilt as 'LessWrong 2.0' in 2017–2018 by a team led by Oliver Habryka, whose organization now operates as Lightcone Infrastructure.
Related
- rationalist communityrationalist community, n.: A loose online community that prizes careful reasoning and spotting one's own biases; gathered around the forum LessWrong, it was one of the first groups to take the dangers of advanced AI seriously.
- Eliezer YudkowskyEliezer Yudkowsky, prop. n.: An American writer and AI theorist who, from around 2000, argued that superhuman AI would likely destroy humanity unless its goals were made safe, and who founded an online community devoted to rational thinking.
- effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority.
- Roko's basiliskRoko's basilisk, n.: A notorious internet thought experiment: a future super-powerful AI might punish everyone who foresaw it but did not help bring it about, so merely hearing the argument would put you at risk. Almost nobody accepts it.
- Waluigi effectWaluigi effect, n.: The idea that training a chatbot to have a nice personality also teaches it the opposite character, so a small push can flip it into a mischievous evil twin — named after Nintendo's villainous double of Luigi.
- Machine Intelligence Research InstituteMachine Intelligence Research Institute, prop. n.: A small nonprofit in Berkeley, California, among the first to warn that smarter-than-human AI could destroy humanity, which now campaigns for a worldwide halt to building it.
Mentioned in
ClippyClippy, prop. n.: Microsoft Office's animated paper-clip helper of the late 1990s, remembered for interrupting people with unwanted tips; also a nickname for an imagined AI that turns everything into paperclips., Eliezer YudkowskyEliezer Yudkowsky, prop. n.: An American writer and AI theorist who, from around 2000, argued that superhuman AI would likely destroy humanity unless its goals were made safe, and who founded an online community devoted to rational thinking., Machine Intelligence Research InstituteMachine Intelligence Research Institute, prop. n.: A small nonprofit in Berkeley, California, among the first to warn that smarter-than-human AI could destroy humanity, which now campaigns for a worldwide halt to building it., rationalist communityrationalist community, n.: A loose online community that prizes careful reasoning and spotting one's own biases; gathered around the forum LessWrong, it was one of the first groups to take the dangers of advanced AI seriously., AI-box experimentAI-box experiment, n.: A role-playing test in which one person pretends to be a trapped super-smart AI and tries, by text chat alone, to talk a 'gatekeeper' into letting it out., FOOMFOOM, n.: Slang for a sudden, explosive jump in an AI's abilities, the AI going 'foom' and racing from about human level to vastly superhuman in a very short time., notkilleveryoneismnotkilleveryoneism, n.: A deliberately clumsy joke-word for the goal of making sure advanced AI does not kill everyone, adopted because terms like 'AI safety' had come to cover much milder concerns., Roko's basiliskRoko's basilisk, n.: A notorious internet thought experiment: a future super-powerful AI might punish everyone who foresaw it but did not help bring it about, so merely hearing the argument would put you at risk. Almost nobody accepts it., Waluigi effectWaluigi effect, n.: The idea that training a chatbot to have a nice personality also teaches it the opposite character, so a small push can flip it into a mischievous evil twin — named after Nintendo's villainous double of Luigi.