machine unlearning
In plain wordsErasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
Methods for removing the influence of specific training data or knowledge from a trained model, so that it behaves as if the information had never been learned. Exact unlearning guarantees equivalence to retraining without the data, for example by training separate models on data shards so that only the affected shard must be retrained (SISA). Approximate unlearning instead updates the trained model — gradient ascent on the data to forget, fine-tuning toward alternative answers, or perturbing internal representations — while preserving performance on the rest. Motivations include privacy rights such as erasure under data-protection law, copyright, removing poisoned or erroneous data, and suppressing hazardous capabilities for CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone. mitigation, as with the RMU method evaluated on WMDPWMDP, prop. n.: A multiple-choice test of AI knowledge that could help someone make biological, chemical or cyber weapons, built from questions near the dangerous line but not over it, and used to test ways of erasing such knowledge.. The central difficulty is verification: approximately unlearned knowledge can often be recovered by fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way., probing or rephrased prompts, so forgetting must be tested, for example with membership inferencemembership inference attack, n.: An attack that tries to tell whether a particular person's record, or a particular document, was part of an AI model's training data — which can itself reveal private facts. and relearning attacks.
After machine unlearning the model failed the hazardous-virology questions, but it relearned them within a few hundred fine-tuning steps.
Origin
Coined by Yinzhi Cao and Junfeng Yang in 'Towards Making Systems Forget with Machine Unlearning' (IEEE Symposium on Security and Privacy 2015). Lucas Bourtoule and colleagues' SISA training (2021) offered a practical exact method; for language models, Ronen Eldan and Mark Russinovich's 'Who's Harry Potter?' (2023) and the WMDP benchmark (2024) spurred approximate unlearning research.
Related
- WMDPWMDP, prop. n.: A multiple-choice test of AI knowledge that could help someone make biological, chemical or cyber weapons, built from questions near the dangerous line but not over it, and used to test ways of erasing such knowledge.
- memorizationmemorization, n.: When an AI model stores specific pieces of its training data — a paragraph, a phone number, an image — closely enough to reproduce them, instead of only learning general patterns.
- differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
- fine-tuning attackfine-tuning attack, n.: Removing an AI model's safety training by training it a little more on a small set of harmful examples — cheap to do when the model's files are public or a company offers custom training.
- membership inference attackmembership inference attack, n.: An attack that tries to tell whether a particular person's record, or a particular document, was part of an AI model's training data — which can itself reveal private facts.
- CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone.
Sources
- Cao, Y. & Yang, J. (2015). Towards Making Systems Forget with Machine Unlearning. IEEE S&P 2015.
- Bourtoule, L. et al. (2021). Machine Unlearning. IEEE S&P 2021. arXiv:1912.03817.
Mentioned in
privacyprivacy, n.: Your ability to control who knows what about you. AI raises new privacy questions because it learns from huge amounts of personal data and can infer things about people they never shared., catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on., CBRN riskCBRN risk, n.: The danger that AI could help someone make or use chemical, biological, radiological or nuclear weapons — for example, by explaining hard technical steps an attacker could not work out alone., Center for AI SafetyCenter for AI Safety, prop. n.: A San Francisco nonprofit that studies how advanced AI could lead to disaster and how to prevent it. In 2023 it gathered signatures for a one-sentence warning that AI could threaten human extinction., data protectiondata protection, n.: Laws and practices that control how organizations collect, use, store and share information about people, and that give people rights over it, such as seeing it or having it deleted., memorizationmemorization, n.: When an AI model stores specific pieces of its training data — a paragraph, a phone number, an image — closely enough to reproduce them, instead of only learning general patterns., model editingmodel editing, n.: Changing one specific fact or behavior inside a trained AI model — for example, who leads a country — by directly adjusting a small part of it, without retraining the whole thing., personally identifiable informationpersonally identifiable information, n.: Information that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person., abliterationabliteration, n.: A technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining., concept erasureconcept erasure, n.: Removing a specific idea — such as a person's gender, or an artist's style — from what an AI model can represent or produce, so the model can no longer use or show it., data attributiondata attribution, n.: Working out which training examples were responsible for a particular behavior or answer of an AI model — useful for explaining, debugging and crediting the data behind it., fine-tuning attackfine-tuning attack, n.: Removing an AI model's safety training by training it a little more on a small set of harmful examples — cheap to do when the model's files are public or a company offers custom training., influence functioninfluence function, n.: A mathematical shortcut for estimating how much a single training example shaped a model's behavior, for instance how a prediction would change if that example had been left out, without retraining the model., latent adversarial traininglatent adversarial training, n.: A safety training method that attacks a model's internal signals, rather than its inputs, during training, so the model learns to behave safely even when its inner workings are pushed toward bad behavior., MEMITMEMIT, abbr.: A method for rewriting thousands of facts stored in a language model at once, by spreading small, calculated weight changes across several of its layers., task arithmetictask arithmetic, n.: Editing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed., Virology Capabilities TestVirology Capabilities Test, prop. n.: A hard test of practical laboratory virology know-how, used to check whether AI models know enough about working with viruses that they might help someone misuse biology., WMDPWMDP, prop. n.: A multiple-choice test of AI knowledge that could help someone make biological, chemical or cyber weapons, built from questions near the dangerous line but not over it, and used to test ways of erasing such knowledge.