vulnerable world hypothesis
In plain wordsThe idea that some future invention could make mass destruction so easy that civilization would be wrecked by default, unless it had unusually strong ways of preventing misuse.
The hypothesis, set out by Nick Bostrom, that some level of technological capability would almost certainly bring the devastation of civilization unless the world had left what he calls the semi-anarchic default condition, in which states cannot reliably stop individuals or small groups in advance, there is no effective worldwide governance, and enough people have destructive motives. Bostrom pictures invention as drawing balls from an urn: most are white (beneficial) or gray (mixed), but a black ball, such as a cheap and easy method of mass destruction, would devastate any civilization that drew it. Advanced AI appears in the argument as a possible black ball, as an amplifier of others, for instance by lowering barriers to biological weaponsbiosecurity, n.: Protecting people from the deliberate or careless spread of dangerous germs and toxins; in AI, making sure AI tools do not make it easier to create or release biological weapons., and as a possible means of stabilization through monitoring or governance. The hypothesis is controversial because the remedies Bostrom examines, including pervasive surveillance and strong global governance, carry grave risks of their own.
If AI makes pathogen design trivial, we will have drawn a black ball from the urn.
Origin
Proposed by Nick Bostrom in 'The Vulnerable World Hypothesis', circulated as a working paper in 2018 and published in Global Policy in 2019.
Related
- existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown.
- global catastrophic riskglobal catastrophic risk, n.: A risk of disaster big enough to badly damage human civilization worldwide, such as killing a large share of humanity, even if humanity eventually recovers.
- CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone.
- differential technological developmentdifferential technological development, n.: A strategy of speeding up technologies that make us safer and slowing down risky ones, so that protections arrive before the dangers they guard against.
- AI governanceAI governance, n.: The rules, institutions and habits that shape how AI is built and used — from laws and treaties to technical standards and a company's own review processes.
- biosecuritybiosecurity, n.: Protecting people from the deliberate or careless spread of dangerous germs and toxins; in AI, making sure AI tools do not make it easier to create or release biological weapons.
Sources
- Bostrom, N. (2019). The Vulnerable World Hypothesis. Global Policy 10(4).
Mentioned in
Nick BostromNick Bostrom, prop. n.: A Swedish philosopher whose 2014 book 'Superintelligence' argued that AI smarter than humans could be catastrophic unless its goals are carefully controlled, and helped make AI risk a mainstream concern., differential technological developmentdifferential technological development, n.: A strategy of speeding up technologies that make us safer and slowing down risky ones, so that protections arrive before the dangers they guard against.