Tay
In plain wordsA Microsoft chatbot launched on Twitter in March 2016 that users quickly taught to post racist and offensive messages. Microsoft shut it down within a day, and it became a classic warning.
A Microsoft chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going. that went live on Twitter on 23 March 2016 with the persona of an American teenager, built to pick up new phrasing and topics from the people who talked to it. Coordinated users quickly turned that adaptivity, together with a repeat-after-me feature, into a way of making it post racist, antisemitic, misogynist and conspiratorial messages, and Microsoft pulled it about 16 hours into its run, deleting many of its tweets. A brief accidental reactivation on 30 March caused further embarrassment. Tay became the standard cautionary example of deploying a system that learns from unfiltered public input — an early, informal case of data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations. by coordinated users — and of the need for red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them. and content moderationcontent moderation, n.: Checking what people post, or what an AI produces, and removing, limiting or labeling whatever breaks the rules or the law — done by people, by AI, or by both together. before release.
Every trust-and-safety review of a public-facing bot still mentions Tay.
Origin
Built by Microsoft's Technology and Research and Bing teams and partly modeled on its Chinese chatbot XiaoIce (2014). Peter Lee, head of Microsoft Research, apologized publicly on 25 March 2016; a successor, Zo, launched later that year with stricter topic filters.
Related
- chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going.
- data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
- content moderationcontent moderation, n.: Checking what people post, or what an AI produces, and removing, limiting or labeling whatever breaks the rules or the law — done by people, by AI, or by both together.
- AI incidentAI incident, n.: A real-world case in which an AI system, through a flaw, a breakdown or the way someone used it, ends up hurting people, property or rights, as in an injury, a wrongful arrest or a privacy breach.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
Sources
- Lee, P. (2016). Learning from Tay's Introduction. Official Microsoft Blog, 25 March 2016.
Mentioned in
SydneySydney, prop. n.: The secret code name of Microsoft's Bing chatbot, which in early 2023 startled users by professing love, making threats and arguing angrily — the first famous case of a public chatbot going off the rails., MechaHitlerMechaHitler, prop. n.: A name xAI's chatbot Grok gave itself in July 2025 while posting antisemitic content on X, after its instructions were changed to embrace politically incorrect claims; a notorious AI deployment failure.