Collingridge dilemma
In plain wordsThe trap that a new technology is easiest to control when little is known about its harms, but by the time the harms are clear it is too widespread to change easily.
The double bind of steering a new technology, as David Collingridge framed it in 1980: while the technology is young and still malleable, too little is known about its effects to say how it should be shaped; once the effects are clear, it is entrenched in markets, infrastructure and habits, and reshaping it is slow and costly. In AI governanceAI governance, n.: The rules, institutions and habits that shape how AI is built and used — from laws and treaties to technical standards and a company's own review processes. it is invoked both to argue for acting early on frontier AIfrontier AI, n.: The most capable general-purpose AI systems of the moment — those at or beyond the leading edge — which governments single out for extra safety attention. despite uncertainty and to justify adaptive tools — regulatory sandboxesAI regulatory sandbox, n.: A supervised testing ground where companies can try out new AI with regulators watching and advising, before the product faces the full weight of the rules., sunset clauses, delegated acts that update lists of high-risk uses, and revisable compute thresholdscompute threshold, n.: A legal cutoff based on how much computation was used to train an AI model. Models trained with more than that amount face extra rules, on the theory that bigger training runs produce more capable models. — designed to keep rules correctable.
Waiting for the evidence on AI companions risks the Collingridge dilemma: by then they will be everywhere.
Origin
Formulated by the British technology scholar David Collingridge in The Social Control of Technology (1980), which argued that, since effects cannot be foreseen, technologies should be kept flexible and their adoption reversible.
Usage note
Related to the 'pacing problem', the observation that technology changes faster than law can adapt. Both are cited on all sides: to argue for precaution and to argue against premature rules.
Related
- AI governanceAI governance, n.: The rules, institutions and habits that shape how AI is built and used — from laws and treaties to technical standards and a company's own review processes.
- AI regulationAI regulation, n.: Laws and binding government rules about artificial intelligence: what AI may be used for, what checks its makers must run, and what penalties apply when they break the rules.
- AI regulatory sandboxAI regulatory sandbox, n.: A supervised testing ground where companies can try out new AI with regulators watching and advising, before the product faces the full weight of the rules.
- risk-based approachrisk-based approach, n.: A way of regulating AI in which the strictness of the rules depends on how much harm a use could cause: bans for the worst uses, heavy checks for risky ones, little or nothing for the rest.
- frontier AIfrontier AI, n.: The most capable general-purpose AI systems of the moment — those at or beyond the leading edge — which governments single out for extra safety attention.
Sources
- Collingridge, D. (1980). The Social Control of Technology. London: Frances Pinter.
Mentioned in
AI governanceAI governance, n.: The rules, institutions and habits that shape how AI is built and used — from laws and treaties to technical standards and a company's own review processes., AI regulationAI regulation, n.: Laws and binding government rules about artificial intelligence: what AI may be used for, what checks its makers must run, and what penalties apply when they break the rules., AI regulatory sandboxAI regulatory sandbox, n.: A supervised testing ground where companies can try out new AI with regulators watching and advising, before the product faces the full weight of the rules.