Future of Humanity Institute
In plain wordsNick Bostrom's research center at Oxford University (2005–2024), where philosophers and scientists studied threats that could permanently derail humanity's future, advanced artificial intelligence among them.
Nick Bostrom's institute at the University of Oxford (2005–2024), which drew researchers from several disciplines and turned existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown. and the long-run future of humanity into subjects of academic research. Its researchers developed or popularized many ideas now central to AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable. debates, including the orthogonality thesisorthogonality thesis, n.: The idea that how smart an AI is and what it wants are separate matters: a very intelligent machine could pursue almost any goal, including ones that seem pointless or harmful to us., instrumental convergenceinstrumental convergence, n.: The idea that almost any goal is easier to reach if you stay switched on, keep your goal and gain resources and power, so very different AIs might all seek those same things., information hazards and the decisive strategic advantagedecisive strategic advantage, n.: A lead in technology and power so large that whoever holds it, whether a country, a company or an AI, could impose its will on the whole world. of a first mover, brought together in Nick Bostrom's book Superintelligence (2014). FHI also produced a technical roadmap for whole brain emulationwhole brain emulation, n.: Copying a particular person's brain into a computer in such detail, down to individual neurons and their connections, that the simulation would think and behave like that person. (2008) and Toby Ord's The Precipice (2020), hosted the program that became the Centre for the Governance of AICentre for the Governance of AI, prop. n.: A research organization, based in Britain, that studies how governments and companies should handle the dangers of advanced AI and the large economic and political changes it could cause. (2018–2021), and trained many people who went on to work at AI laboratories, think tanks and governments. Its close ties to effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority. and longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived. made it both influential and controversial.
Origin
Founded in 2005 by philosopher Nick Bostrom within Oxford's Faculty of Philosophy and the James Martin 21st Century School (later the Oxford Martin School). After the Faculty of Philosophy froze its fundraising and hiring from 2020 and declined to renew staff contracts, the institute closed on 16 April 2024.
Usage note
Not to be confused with the Future of Life InstituteFuture of Life Institute, prop. n.: An advocacy group that warns about the dangers of advanced AI and other powerful technologies, funds research and lobbies governments. It organized the 2023 open letter calling for a pause on giant AI experiments., a separate nonprofit with overlapping concerns.
Related
- existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown.
- Nick BostromNick Bostrom, prop. n.: A Swedish philosopher whose 2014 book 'Superintelligence' argued that AI smarter than humans could be catastrophic unless its goals are carefully controlled, and helped make AI risk a mainstream concern.
- orthogonality thesisorthogonality thesis, n.: The idea that how smart an AI is and what it wants are separate matters: a very intelligent machine could pursue almost any goal, including ones that seem pointless or harmful to us.
- longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived.
- effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority.
- Machine Intelligence Research InstituteMachine Intelligence Research Institute, prop. n.: A small nonprofit in Berkeley, California, among the first to warn that smarter-than-human AI could destroy humanity, which now campaigns for a worldwide halt to building it.
- Future of Life InstituteFuture of Life Institute, prop. n.: An advocacy group that warns about the dangers of advanced AI and other powerful technologies, funds research and lobbies governments. It organized the 2023 open letter calling for a pause on giant AI experiments.
Sources
- Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
Mentioned in
Future of Life InstituteFuture of Life Institute, prop. n.: An advocacy group that warns about the dangers of advanced AI and other powerful technologies, funds research and lobbies governments. It organized the 2023 open letter calling for a pause on giant AI experiments., Nick BostromNick Bostrom, prop. n.: A Swedish philosopher whose 2014 book 'Superintelligence' argued that AI smarter than humans could be catastrophic unless its goals are carefully controlled, and helped make AI risk a mainstream concern., Centre for the Governance of AICentre for the Governance of AI, prop. n.: A research organization, based in Britain, that studies how governments and companies should handle the dangers of advanced AI and the large economic and political changes it could cause., global catastrophic riskglobal catastrophic risk, n.: A risk of disaster big enough to badly damage human civilization worldwide, such as killing a large share of humanity, even if humanity eventually recovers., Owain EvansOwain Evans, prop. n.: A British AI-safety researcher who created the TruthfulQA test and discovered surprising ways language models generalize, such as narrow training making a model broadly malicious., Toby OrdToby Ord, prop. n.: An Australian philosopher at Oxford who helped found effective altruism and estimated, in his book 'The Precipice', that unaligned AI is the largest threat to humanity's future this century., windfall clausewindfall clause, n.: A proposed promise by AI companies to give away a large share of any enormous profits they make if their AI transforms the economy, so that the gains are widely shared.