moral agency
In plain wordsThe capacity to act for moral reasons and be held responsible for what one does. Researchers ask whether AI systems could ever be moral agents, or only tools whose makers and users bear responsibility.
The capacity to act on the basis of moral reasons and to be held morally responsible — praised or blamed — for one's actions; an entity with this capacity is a moral agent. Standard conditions include grasping moral considerations, the ability to deliberate and act on them, and some form of autonomy. James Moor distinguished ethical-impact agents (any technology with moral consequences), implicit ethical agents (designed to behave safely), explicit ethical agents (which represent and reason about ethics) and full ethical agents (with judgment and responsibility comparable to humans'). Machine ethics aims to build artificial moral agents of the explicit kind, and AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this. and constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand. pursue related goals. Critics hold that AI systems cannot be responsible because they lack consciousness or genuine autonomy, and warn that calling them moral agents can obscure the responsibility of developers and deployers.
Origin
The notion of a moral agent is long established in ethics. The phrase 'artificial moral agent' was put into circulation by Colin Allen, Gary Varner and Jason Zinser's 'Prolegomena to Any Future Artificial Moral Agent' (Journal of Experimental and Theoretical Artificial Intelligence, 2000). James Moor's taxonomy appeared in 2006, and Wendell Wallach and Allen's book Moral Machines in 2009.
Usage note
Moral agency and moral patienthoodmoral patienthood, n.: Being the kind of being whose interests matter morally for its own sake, so that it can be wronged. Babies and animals count, even though they are not responsible for what they do. can come apart: a system could be an agent without being a patient, or the reverse. The 'responsibility gap' debate asks who is accountable when an autonomous system causes harm that no person intended or could foresee.
Related
- moral patienthoodmoral patienthood, n.: Being the kind of being whose interests matter morally for its own sake, so that it can be wronged. Babies and animals count, even though they are not responsible for what they do.
- AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this.
- Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand.
- AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect.
- AI personhoodAI personhood, n.: The idea of treating an AI system as a 'person' in law or ethics, with rights or duties of its own — for example, to own property, to be sued, or not to be deleted.
- goal-directednessgoal-directedness, n.: How much a system behaves as if it is pursuing a goal — steering toward particular outcomes and finding new ways around obstacles put in its path.
- moral patienthoodmoral patienthood, n.: Being the kind of being whose interests matter morally for its own sake, so that it can be wronged. Babies and animals count, even though they are not responsible for what they do.
Sources
- Allen, C., Varner, G. & Zinser, J. (2000). Prolegomena to Any Future Artificial Moral Agent. Journal of Experimental and Theoretical Artificial Intelligence 12(3), 251–261.
- Moor, J. H. (2006). The Nature, Importance, and Difficulty of Machine Ethics. IEEE Intelligent Systems 21(4), 18–21.
- Wallach, W. & Allen, C. (2009). Moral Machines: Teaching Robots Right from Wrong. Oxford University Press.
Mentioned in
AI personhoodAI personhood, n.: The idea of treating an AI system as a 'person' in law or ethics, with rights or duties of its own — for example, to own property, to be sued, or not to be deleted., moral patienthoodmoral patienthood, n.: Being the kind of being whose interests matter morally for its own sake, so that it can be wronged. Babies and animals count, even though they are not responsible for what they do.