monitorability
In plain wordsHow easy it is to watch an AI's reasoning and spot when it is planning something bad — a property that could be lost if future AIs stop thinking in readable language.
The extent to which a model's cognition relevant to harmful behavior can be observed and correctly interpreted by a monitor — most often, whether its chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate. reveals intentions to misbehave clearly enough for chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm. to catch them. In the analysis of Tomek Korbak and colleagues' 2025 position paper, it has two sources: some tasks are too hard to do without reasoning in written tokens (necessity), and models also tend to write their reasoning out when they need not (propensity). Either can erode, for instance if reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. pushes reasoning toward unreadable shorthand, if training puts pressure on the reasoning text itself, if new architectures reason in continuous vectors ('neuralese') rather than tokens, or if models learn to conceal their thoughts. The authors asked developers to track and publish monitorability and to factor it into training and architecture choices, even at some cost in capability, a cost others had called a monitorability tax.
Switching to latent reasoning would buy a little speed at a large cost in monitorability.
Origin
Spread through the position paper 'Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety' (July 2025) by Tomek Korbak, Mikita Balesni and 39 coauthors from several frontier laboratories, the UK AI Security Institute and academia. Bowen Baker and colleagues at OpenAI had urged paying a 'monitorability tax' in March 2025 rather than optimizing reasoning against monitors.
Usage note
A property of the model, the task and the monitor together. Distinct from chain-of-thought faithfulnesschain-of-thought faithfulness, n.: Whether the reasoning an AI writes out really reflects how it reached its answer, rather than being an after-the-fact story that leaves out what actually drove the decision.: an incomplete trace can still be monitorable if the dangerous reasoning cannot be done without writing it down.
Related
- chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm.
- chain-of-thought faithfulnesschain-of-thought faithfulness, n.: Whether the reasoning an AI writes out really reflects how it reached its answer, rather than being an after-the-fact story that leaves out what actually drove the decision.
- steganographysteganography, n.: Hiding a secret message inside something that looks ordinary. In AI safety, the worry is that AI systems could hide plans, or signals to one another, inside innocent-looking text.
- reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
- AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest.
- interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box.
Sources
- Korbak, T., Balesni, M. et al. (2025). Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. arXiv:2507.11473. arxiv.org/abs/2507.11473
Mentioned in
chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm., reasoning tracereasoning trace, n.: The written-out 'thinking' an AI reasoning model produces before its final answer — notes to itself full of attempts, checks and corrections, which users may or may not get to see., chain-of-thought faithfulnesschain-of-thought faithfulness, n.: Whether the reasoning an AI writes out really reflects how it reached its answer, rather than being an after-the-fact story that leaves out what actually drove the decision., latent reasoninglatent reasoning, n.: Getting an AI to do its step-by-step thinking inside its internal numbers rather than writing it out in words — potentially faster and more flexible, but much harder for people to inspect., neuraleseneuralese, n.: A nickname for the private 'language' of numbers that AI systems could use to think or to talk to each other, instead of words that people can read., steganographysteganography, n.: Hiding a secret message inside something that looks ordinary. In AI safety, the worry is that AI systems could hide plans, or signals to one another, inside innocent-looking text.