differential technological development
In plain wordsA strategy of speeding up technologies that make us safer and slowing down risky ones, so that protections arrive before the dangers they guard against.
A policy of steering which technologies come first, rather than how fast technology advances overall: in Nick Bostrom's 2002 formulation, slowing those that would raise existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown. and speeding those that would reduce it, protective technologies above all. Applied to AI, it favors advancing alignment, interpretability, evaluation, cybersecurity and biodefense ahead of raw capabilities, so that dangerous capabilities arrive in a world already prepared for them. Because the overall direction of technology is hard to steer, the strategy emphasizes relative timing and the choices of individual researchers, funders and governments. In 2023 the Ethereum co-founder Vitalik Buterin popularized a related program, d/acc (defensive, decentralized and differential acceleration), positioned between effective accelerationismeffective accelerationism, n.: An online movement, known as e/acc, that wants AI and other technology developed as fast as possible, arguing that progress is good and that safety-driven regulation does more harm than good. and calls for a pause, which favors technologies that make defense easier than attack.
Funding interpretability before scaling the next model is differential technological development in practice.
Origin
Proposed by Nick Bostrom in 'Existential Risks' (2002) as a principle for reducing existential risk and elaborated in his later work; Vitalik Buterin's November 2023 essay 'My techno-optimism' introduced d/acc.
Related
- vulnerable world hypothesisvulnerable world hypothesis, n.: The idea that some future invention could make mass destruction so easy that civilization would be wrecked by default, unless it had unusually strong ways of preventing misuse.
- existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown.
- effective accelerationismeffective accelerationism, n.: An online movement, known as e/acc, that wants AI and other technology developed as fast as possible, arguing that progress is good and that safety-driven regulation does more harm than good.
- AI pauseAI pause, n.: A proposed halt, temporary or indefinite, on building AI systems more powerful than today's, to give safety research and regulation time to catch up.
- biosecuritybiosecurity, n.: Protecting people from the deliberate or careless spread of dangerous germs and toxins; in AI, making sure AI tools do not make it easier to create or release biological weapons.
- AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable.
- effective accelerationismeffective accelerationism, n.: An online movement, known as e/acc, that wants AI and other technology developed as fast as possible, arguing that progress is good and that safety-driven regulation does more harm than good.
- AI pauseAI pause, n.: A proposed halt, temporary or indefinite, on building AI systems more powerful than today's, to give safety research and regulation time to catch up.
Sources
- Bostrom, N. (2002). Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards. Journal of Evolution and Technology 9.
Mentioned in
effective accelerationismeffective accelerationism, n.: An online movement, known as e/acc, that wants AI and other technology developed as fast as possible, arguing that progress is good and that safety-driven regulation does more harm than good., deceldecel, n.: A dismissive label, used mainly by AI accelerationists, for anyone who wants AI development slowed down, paused or heavily regulated., vulnerable world hypothesisvulnerable world hypothesis, n.: The idea that some future invention could make mass destruction so easy that civilization would be wrecked by default, unless it had unusually strong ways of preventing misuse.