trusted editing
In plain wordsA safety setup in which a weaker, trusted AI rewrites any suspicious-looking part of a stronger AI's work before it is used, removing possible sabotage while keeping most of the value.
An AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest. protocol in which a trusted, less capable model reviews each output of an untrusted, more capable model and, when the output looks moderately suspicious, rewrites it to remove anything that might be an attack — for example, editing submitted code to delete logic that resembles a hidden backdoor — while preserving the useful work. The most suspicious outputs still go to human auditors; the least suspicious are used unchanged. Because editing keeps more of the strong model's contribution than discarding it in favor of the weak model's own answer ('defer to trusted'), it can achieve a better trade-off between safety and usefulness. Related protocols resample the untrusted model several times, using the least suspicious sample or treating the pattern of suspicion across samples as evidence.
Trusted editing stripped the odd special case from the patch, and the tests still passed.
Origin
Introduced by Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan and Fabien Roger of Redwood Research (2023), who found it among the best-performing protocols in their code-backdooring testbed. Resampling protocols were developed in 'Ctrl-Z: Controlling AI Agents via Resampling' (2025).
Related
- AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest.
- trusted monitoringtrusted monitoring, n.: A safety setup in which a weaker AI that we trust reviews each action of a stronger AI we do not fully trust, flagging the most suspicious actions for a human to check.
- untrusted monitoringuntrusted monitoring, n.: Using a second copy of a powerful AI to watch the first for sabotage, with safeguards that stop the two copies from secretly cooperating.
- control evaluationcontrol evaluation, n.: A test of whether an AI's safety measures would hold if the AI were secretly trying to cause harm: experts play the scheming AI and try to beat the safeguards.
- backdoor attackbackdoor attack, n.: Secretly training an AI model to misbehave whenever it sees a hidden signal — a sticker, a word, a date — while it acts perfectly normally the rest of the time.
- trusted monitoringtrusted monitoring, n.: A safety setup in which a weaker AI that we trust reviews each action of a stronger AI we do not fully trust, flagging the most suspicious actions for a human to check.
Sources
- Greenblatt, R., Shlegeris, B., Sachan, K. & Roger, F. (2023). AI Control: Improving Safety Despite Intentional Subversion. arXiv:2312.06942. arxiv.org/abs/2312.06942
Mentioned in
AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest., Buck ShlegerisBuck Shlegeris, prop. n.: An AI-safety researcher who runs Redwood Research and helped found 'AI control', the study of keeping powerful AI safe even if it is secretly trying to cause harm., trusted monitoringtrusted monitoring, n.: A safety setup in which a weaker AI that we trust reviews each action of a stronger AI we do not fully trust, flagging the most suspicious actions for a human to check.