value lock-in
In plain wordsThe possibility that powerful AI could freeze one set of values or one power structure in place for an extremely long time, so that later generations could never change course.
The persistence of a particular set of values, institutions or distribution of power for an extremely long period, potentially indefinitely, so that future agents cannot revise them. Advanced AI is thought to make lock-in newly feasible: unlike human institutions, AI systems can copy their goals exactly, need not die or be replaced by successors with different views, and could enforce rules perfectly, so a singletonsingleton, n.: A possible future world run by a single decision-maker at the top, such as one government, one alliance or one AI, with no rival able to challenge it. or a dominant AI-backed regime might entrench whatever values it held when it formed. Lock-in is dangerous if those values are mistaken, since moral progress would stop; the concern motivates arguments for preserving pluralism, keeping options open and building AI that supports reflection rather than fixing answers early. Some authors also see an opportunity, since locking in good values could secure a long and flourishing future. Analysts disagree about how early in the AI transition lock-in becomes possible.
Writing today's norms into a superintelligence's goals risks value lock-in of our current moral mistakes.
Origin
Earliest use in this sense not established. Discussed in Nick Bostrom's work on singletons and in Superintelligence (2014); Will MacAskill's What We Owe the Future (2022) devoted a chapter to value lock-in, and Lukas Finnveden, Jess Riedel and Carl Shulman's report 'AGI and Lock-in' (2022) analyzed its mechanisms.
Related
- singletonsingleton, n.: A possible future world run by a single decision-maker at the top, such as one government, one alliance or one AI, with no rival able to challenge it.
- longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived.
- existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown.
- coherent extrapolated volitioncoherent extrapolated volition, n.: A proposal that a superintelligent AI should serve not what people want today but what humanity would agree on wanting after far more knowledge and reflection.
- concentration of powerconcentration of power, n.: The worry that AI will put enormous economic, political or military power in the hands of a few companies, governments or individuals, beyond the reach of ordinary checks and balances.
- suffering risksuffering risk, n.: A risk that the future contains suffering on a vast scale, far beyond anything in history, for example if powerful AI created enormous numbers of suffering beings.
Sources
- MacAskill, W. (2022). What We Owe the Future. Basic Books.
- Finnveden, L., Riedel, J. & Shulman, C. (2022). AGI and Lock-in. Online report.
Mentioned in
existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown., concentration of powerconcentration of power, n.: The worry that AI will put enormous economic, political or military power in the hands of a few companies, governments or individuals, beyond the reach of ordinary checks and balances., longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived., coherent extrapolated volitioncoherent extrapolated volition, n.: A proposal that a superintelligent AI should serve not what people want today but what humanity would agree on wanting after far more knowledge and reflection., singletonsingleton, n.: A possible future world run by a single decision-maker at the top, such as one government, one alliance or one AI, with no rival able to challenge it., suffering risksuffering risk, n.: A risk that the future contains suffering on a vast scale, far beyond anything in history, for example if powerful AI created enormous numbers of suffering beings.