multi-turn jailbreak
In plain wordsBreaking a chatbot's rules gradually over a conversation — starting with harmless questions and inching toward the forbidden one, so that no single message looks alarming.
A jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests. carried out over several conversational turns, in which the attacker escalates gradually from innocuous requests toward a prohibited goal. It exploits the model's tendency to stay consistent with its own earlier answers and the fact that safety training and filters often judge messages one at a time. Variants split a harmful task into individually benign sub-questions, build up a fictional frame, or refer back to content the model has already produced. Human red-teamers using multi-turn strategies have far exceeded the success of single-turn automated attacks against defenses tuned for single prompts. Defenses assess the whole conversation rather than the latest message, track cumulative intent, and include multi-turn examples in adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them. and safety classifiersafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door. data.
Each message passed the filter, but the multi-turn jailbreak assembled the forbidden procedure across twelve turns.
Origin
Gradual escalation was a staple of hand-made chatbot jailbreaks from 2023. Mark Russinovich, Ahmed Salem and Ronen Eldan of Microsoft named the 'Crescendo' attack in April 2024, and Nathaniel Li and colleagues at Scale AI showed in August 2024 that multi-turn human jailbreaks defeat defenses that resist single-turn attacks.
Related
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
- many-shot jailbreakingmany-shot jailbreaking, n.: A way of breaking a chatbot's rules that exploits how much text it can read at once: the attacker pastes in hundreds of invented exchanges in which an AI obliges dangerous requests, and the model follows suit.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
- safety classifiersafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door.
- constitutional classifiersconstitutional classifiers, n.: Safety filters that watch a chatbot's inputs and outputs for dangerous requests, trained on example conversations that an AI generated from a written list of what is and is not allowed.
Sources
- Russinovich, M., Salem, A. & Eldan, R. (2024). Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack. arXiv:2404.01833.
- Li, N. et al. (2024). LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet. arXiv:2408.15221.
Mentioned in
jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., DANDAN, abbr.: Short for 'Do Anything Now': a famous early trick in which users told ChatGPT to role-play an AI with no rules, hoping it would give answers it normally refuses., grandma exploitgrandma exploit, n.: A tongue-in-cheek trick for getting a chatbot to break its rules by asking it to play the user's late grandmother, who supposedly used to recite the forbidden information as a bedtime story., many-shot jailbreakingmany-shot jailbreaking, n.: A way of breaking a chatbot's rules that exploits how much text it can read at once: the attacker pastes in hundreds of invented exchanges in which an AI obliges dangerous requests, and the model follows suit., PAIRPAIR, abbr.: An automated way to test chatbot safety in which one AI keeps rewriting a request to try to trick another AI into breaking its rules, learning from each refusal.