responsibility gap
In plain wordsThe problem that when an AI system that learns or acts on its own causes harm, it may be unclear whether anyone (its makers, users or operators) can fairly be held responsible.
A situation in which an outcome caused by an autonomous or learning system seems to call for someone to be held responsible, yet no human agent meets the usual conditions for responsibility, such as control over the action and the ability to foresee it. Because learning systems behave in ways their designers did not explicitly program and operators cannot fully predict, harm can occur with no one appropriately blamed or held to account. Philosophers distinguish gaps in culpability, moral accountability, public accountability and active responsibility. Some deny that genuine gaps exist, arguing that responsibility lies with whoever chose to deploy the system; others propose meaningful human control, strict liability or new institutional roles. The concept is central to debates about autonomous weapons, self-driving vehicles and AI liabilityAI liability, n.: Who has to pay or answer in court when an AI system causes harm — the company that built it, the business that used it or the person who relied on it — and under which laws..
If no engineer could have foreseen the drone's choice of target, who answers for it? That is the responsibility gap.
Origin
Introduced by philosopher Andreas Matthias in 'The responsibility gap: Ascribing responsibility for the actions of learning automata' (Ethics and Information Technology, 2004). Robert Sparrow's 'Killer Robots' (2007) applied it to autonomous weapons, and Filippo Santoni de Sio and Giulio Mecacci distinguished four kinds of gap (2021).
Usage note
Related to the problem of many hands (Dennis Thompson, 1980), in which responsibility diffuses among the many contributors to an outcome, and to the moral crumple zonemoral crumple zone, n.: When a largely automated system goes wrong and the human who was supposed to supervise it gets blamed, even though they had little real say; named after the car part that soaks up a crash., in which blame is misplaced rather than missing.
Related
- moral crumple zonemoral crumple zone, n.: When a largely automated system goes wrong and the human who was supposed to supervise it gets blamed, even though they had little real say; named after the car part that soaks up a crash.
- AI liabilityAI liability, n.: Who has to pay or answer in court when an AI system causes harm — the company that built it, the business that used it or the person who relied on it — and under which laws.
- algorithmic accountabilityalgorithmic accountability, n.: The idea that the people and organizations behind automated decision systems must be able to explain and justify those systems' effects, answer for the harm they cause, and fix it.
- machine ethicsmachine ethics, n.: The effort to build AI systems that can tell right from wrong and act ethically themselves, rather than only being used ethically by people.
- human oversighthuman oversight, n.: Keeping people able to watch, understand, question and, if necessary, stop or overrule an AI system, rather than letting it act unchecked.
- AI personhoodAI personhood, n.: The idea of treating an AI system as a 'person' in law or ethics, with rights or duties of its own — for example, to own property, to be sued, or not to be deleted.
Sources
- Matthias, A. (2004). The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology 6(3), 175–183.
- Santoni de Sio, F. & Mecacci, G. (2021). Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them. Philosophy & Technology 34, 1057–1084.
Mentioned in
AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect., machine ethicsmachine ethics, n.: The effort to build AI systems that can tell right from wrong and act ethically themselves, rather than only being used ethically by people., trolley problemtrolley problem, n.: A thought experiment asking whether you should divert a runaway tram so that it kills one person instead of five, often used to ask how self-driving cars should choose in an unavoidable crash., moral crumple zonemoral crumple zone, n.: When a largely automated system goes wrong and the human who was supposed to supervise it gets blamed, even though they had little real say; named after the car part that soaks up a crash.