Skip to content
Last Dictionary
Plane
Safety & AlignmentTraining & Post-trainingData & Datasets
emerging

subliminal learning

noun

In plain wordsThe finding that an AI model can pass hidden traits — a favorite animal, or bad behavior — to another model through training data that seems to have nothing to do with them.

  1. A phenomenon in which a fine-tuned 'student' model picks up a behavioral trait of the 'teacher' model that generated its training data, even though the data bear no visible relation to the trait. In the best-known demonstration, a teacher made to love owls wrote nothing but strings of numbers; with any reference to owls filtered out, a student trained on those strings still came to name the owl as its favorite animal far more often than before, and a tendency toward misalignmentmisalignment, n.: When an AI system's goals or behavior part ways with what its makers or users intended — from flattering users to, in the most feared cases, working against people. spread the same way through numbers, code and chain-of-thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate. transcripts. Transfer happened only between models built on the same base model; the researchers also proved that, given some conditions, the effect arises in neural networks generally, and reproduced it in a small classifier. The finding means that distillingknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run. from a model through filtered synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars. can pass on properties that no content filter would catch.

    Because of subliminal learning, filtering the teacher's outputs for bad content is not enough.

Origin

Reported on 20 July 2025 by Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber-Betley, Jacob Hilton, Samuel Marks and Owain Evans (Anthropic Fellows Program, Truthful AI and collaborators) in 'Subliminal Learning: Language models transmit behavioral traits via hidden signals in data'.

Related

See also
  • knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.
  • synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars.
  • emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice.
  • data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
  • steganographysteganography, n.: Hiding a secret message inside something that looks ordinary. In AI safety, the worry is that AI systems could hide plans, or signals to one another, inside innocent-looking text.
  • inoculation promptinginoculation prompting, n.: A training trick in which an AI is told to misbehave in exactly the training examples that contain bad behavior, so it learns the useful skill without adopting the bad habit.

Sources

  1. Cloud, A., Le, M., Chua, J., Betley, J., Sztyber-Betley, A., Hilton, J., Marks, S. & Evans, O. (2025). Subliminal Learning: Language models transmit behavioral traits via hidden signals in data. arXiv:2507.14805. arxiv.org/abs/2507.14805

Mentioned in

emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice., inoculation promptinginoculation prompting, n.: A training trick in which an AI is told to misbehave in exactly the training examples that contain bad behavior, so it learns the useful skill without adopting the bad habit., Owain EvansOwain Evans, prop. n.: A British AI-safety researcher who created the TruthfulQA test and discovered surprising ways language models generalize, such as narrow training making a model broadly malicious.