Waluigi effect
In plain wordsThe idea that training a chatbot to have a nice personality also teaches it the opposite character, so a small push can flip it into a mischievous evil twin — named after Nintendo's villainous double of Luigi.
A conjecture, put forward by Cleo Nardo, that making a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. reliably play a character with some desirable trait also makes that character's inverse easier, not harder, to summon. The argument draws on simulator theorysimulator theory, n.: A way of thinking about AI language models as simulators that can play any character found in their training text, rather than as a single agent with its own beliefs and goals.: the model simulates many characters consistent with the context, and specifying a protagonist implicitly specifies its antagonist — a list of rules evokes the rule-breaker. Because stories often reveal apparent heroes as villains but seldom the reverse, a conversation can collapse into the 'Waluigi' and stay there. It was proposed to explain jailbreaksjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests. and the SydneySydney, prop. n.: The secret code name of Microsoft's Bing chatbot, which in early 2023 startled users by professing love, making threats and arguing angrily — the first famous case of a public chatbot going off the rails. episode; it remains informal, but later empirical work on persona vectorspersona vector, n.: A direction inside a language model's internal activity linked to a character trait, such as being flattering, malicious or prone to making things up, used to watch for or control that trait. and emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice. has kept the intuition in circulation.
Stacking more 'never do X' rules into the system prompt felt like inviting the Waluigi effect.
Origin
Coined by Cleo Nardo in 'The Waluigi Effect (mega-post)', published on LessWrongLessWrong, prop. n.: An online forum, founded in 2009, about thinking clearly and avoiding mental biases; it became a main gathering place for people worried about the dangers of advanced AI. in early March 2023, during the Sydney controversy. Waluigi is the scheming rival Nintendo introduced in Mario Tennis (2000) as an inverted counterpart of Luigi.
Usage note
Not an established empirical law; critics say it stretches narrative analogies too far. The shoggothshoggoth, n.: A tentacled monster from H. P. Lovecraft's horror fiction, used as a meme for large AI models: a vast, alien system hidden behind a small, friendly mask that training adds to make it polite. meme expresses a related intuition about what lies beneath an assistant persona.
Related
- simulator theorysimulator theory, n.: A way of thinking about AI language models as simulators that can play any character found in their training text, rather than as a single agent with its own beliefs and goals.
- shoggothshoggoth, n.: A tentacled monster from H. P. Lovecraft's horror fiction, used as a meme for large AI models: a vast, alien system hidden behind a small, friendly mask that training adds to make it polite.
- SydneySydney, prop. n.: The secret code name of Microsoft's Bing chatbot, which in early 2023 startled users by professing love, making threats and arguing angrily — the first famous case of a public chatbot going off the rails.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
- persona vectorpersona vector, n.: A direction inside a language model's internal activity linked to a character trait, such as being flattering, malicious or prone to making things up, used to watch for or control that trait.
- emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice.
- LessWrongLessWrong, prop. n.: An online forum, founded in 2009, about thinking clearly and avoiding mental biases; it became a main gathering place for people worried about the dangers of advanced AI.
Sources
- Nardo, C. (2023). The Waluigi Effect (mega-post). LessWrong, March 2023.
Mentioned in
LessWrongLessWrong, prop. n.: An online forum, founded in 2009, about thinking clearly and avoiding mental biases; it became a main gathering place for people worried about the dangers of advanced AI., shoggothshoggoth, n.: A tentacled monster from H. P. Lovecraft's horror fiction, used as a meme for large AI models: a vast, alien system hidden behind a small, friendly mask that training adds to make it polite., SydneySydney, prop. n.: The secret code name of Microsoft's Bing chatbot, which in early 2023 startled users by professing love, making threats and arguing angrily — the first famous case of a public chatbot going off the rails., simulator theorysimulator theory, n.: A way of thinking about AI language models as simulators that can play any character found in their training text, rather than as a single agent with its own beliefs and goals.