intent alignment
In plain wordsAn AI is intent-aligned if it is genuinely trying to do what its operator wants — even if it sometimes fails through ignorance or clumsiness.
Paul Christiano's deliberately narrow notion of alignment, which looks only at an AI system's aim: the system counts as aligned with its operator when it is attempting to act on the operator's wishes. What matters is motive rather than result: an intent-aligned system can still err through lack of knowledge or skill, while a system that behaves well by accident is not thereby intent-aligned. Separating motive from competence lets researchers treat alignment as a problem distinct both from capability and from deciding what operators ought to want. Critics note that intent alignment to a single operator says nothing about harms to third parties, and that 'what the operator wants' is ambiguous between literal instructions, underlying intentions and considered preferences.
An intent-aligned assistant that misreads the request is incompetent, not misaligned.
Origin
Paul Christiano proposed the definition in his 2018 essay 'Clarifying "AI alignment"', summarized as 'A is trying to do what H wants it to do'. The label 'intent alignment' became standard soon afterward in his writing and that of Rohin Shah and others.
Related
- AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this.
- value alignmentvalue alignment, n.: Building AI whose goals reflect human values — what people care about and consider right — rather than one that merely follows orders or chases a narrow target.
- corrigibilitycorrigibility, n.: An AI's willingness to let people correct it — to accept being paused, retrained, overruled or shut down — rather than resisting or quietly working around them.
- scalable oversightscalable oversight, n.: The problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking.
- Paul ChristianoPaul Christiano, prop. n.: An American AI-alignment researcher who led the work that made training AI from human preferences practical, the basis of how chatbots are taught to be helpful.
- value alignmentvalue alignment, n.: Building AI whose goals reflect human values — what people care about and consider right — rather than one that merely follows orders or chases a narrow target.
Sources
- Christiano, P. (2018). Clarifying "AI alignment". AI Alignment Forum.
Mentioned in
AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this., corrigibilitycorrigibility, n.: An AI's willingness to let people correct it — to accept being paused, retrained, overruled or shut down — rather than resisting or quietly working around them., value alignmentvalue alignment, n.: Building AI whose goals reflect human values — what people care about and consider right — rather than one that merely follows orders or chases a narrow target.