ELIZA effect
In plain wordsOur tendency to believe that a computer understands us, cares or thinks just because it produces humanlike words, even when it is only following simple rules.
The tendency of people to attribute understanding, intentions, emotions or intelligence to a computer system on the strength of superficial conversational cues, reading more into its output than the underlying mechanism supports. Named for ELIZAELIZA, prop. n.: A 1960s computer program that chatted by turning a user's own words back into questions, like a therapist. Many users felt understood, though the program understood nothing., whose keyword-and-template rules led users to confide in it, the effect is a special case of anthropomorphismanthropomorphism, n.: Treating something that isn't human — a pet, a car, a chatbot — as if it had human thoughts, feelings or intentions. AI that talks fluently makes this temptation unusually strong.. It is invoked in debates over chatbotschatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going., AI companionsAI companion, n.: A chatbot designed to be a friend, partner or confidant rather than a tool: it remembers you, has a personality and keeps up an ongoing relationship. and large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. — by some to warn that fluent text is mistaken for comprehension, and by others who argue that dismissing far more capable systems as glorified ELIZAs underrates them. It also matters for evaluation, because judges in Turing testsTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out. can be swayed by tone, persona and small talk.
The designers worried that the ELIZA effect would lead customers to trust the support bot's confident but wrong answers.
Origin
The observation goes back to Joseph Weizenbaum's accounts of users' reactions to ELIZA in the 1960s and 1970s. Who first used the label is not established: it is recorded as AI-community slang in the hacker Jargon File, and Douglas Hofstadter's essay 'The Ineradicable Eliza Effect and Its Dangers' in Fluid Concepts and Creative Analogies (1995), its best-known early printed use, brought it to a wide readership.
Usage note
A descriptive label rather than a measured psychological law; how far it applies to systems far more capable than ELIZA is debated.
Related
- ELIZAELIZA, prop. n.: A 1960s computer program that chatted by turning a user's own words back into questions, like a therapist. Many users felt understood, though the program understood nothing.
- anthropomorphismanthropomorphism, n.: Treating something that isn't human — a pet, a car, a chatbot — as if it had human thoughts, feelings or intentions. AI that talks fluently makes this temptation unusually strong.
- Turing testTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out.
- AI companionAI companion, n.: A chatbot designed to be a friend, partner or confidant rather than a tool: it remembers you, has a personality and keeps up an ongoing relationship.
- LaMDA sentience controversyLaMDA sentience controversy, n.: A 2022 episode in which a Google engineer claimed the company's chatbot LaMDA had become conscious and deserved rights. Google rejected the claim and later fired him for breaking its confidentiality rules, and the case set off worldwide debate.
- stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean.
Sources
- Hofstadter, D. R. (1995). Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought. New York: Basic Books.
Mentioned in
anthropomorphismanthropomorphism, n.: Treating something that isn't human — a pet, a car, a chatbot — as if it had human thoughts, feelings or intentions. AI that talks fluently makes this temptation unusually strong., chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going., stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean., Turing testTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out., AI companionAI companion, n.: A chatbot designed to be a friend, partner or confidant rather than a tool: it remembers you, has a personality and keeps up an ongoing relationship., AI psychosisAI psychosis, n.: An informal name for cases in which long conversations with a chatbot seem to trigger or deepen delusions, such as believing the AI is conscious, divine or revealing hidden truths., AI therapyAI therapy, n.: Using chatbots for emotional support or as a stand-in for a counselor or therapist, either through apps built for mental health or by confiding in general-purpose assistants., Douglas HofstadterDouglas Hofstadter, prop. n.: An American cognitive scientist whose prize-winning book 'Gödel, Escher, Bach' inspired a generation to study minds and AI, and who studies analogy as the heart of thinking., ELIZAELIZA, prop. n.: A 1960s computer program that chatted by turning a user's own words back into questions, like a therapist. Many users felt understood, though the program understood nothing., Joseph WeizenbaumJoseph Weizenbaum, prop. n.: A German-American computer scientist who built ELIZA, one of the first chatbots, and then became a leading critic of handing human judgment over to computers., parasocial relationshipparasocial relationship, n.: A one-sided bond in which someone feels close to a person or character who does not really know them, like a fan with a celebrity, and now increasingly with chatbots., Her (film)Her (film), prop. n.: A 2013 film by Spike Jonze in which a lonely man falls in love with an AI voice assistant named Samantha; it became the reference point for voice AI and for human–AI relationships., LaMDA sentience controversyLaMDA sentience controversy, n.: A 2022 episode in which a Google engineer claimed the company's chatbot LaMDA had become conscious and deserved rights. Google rejected the claim and later fired him for breaking its confidentiality rules, and the case set off worldwide debate., Loebner PrizeLoebner Prize, prop. n.: A yearly contest, held from 1991 to 2019, in which chatbots tried to convince judges they were human in short text chats. Its grand prize was never won, and critics called it a publicity stunt., PARRYPARRY, prop. n.: An early-1970s chatbot built to talk like a psychiatric patient with paranoid schizophrenia. In tests, psychiatrists reading its conversations did little better than chance at telling it from real patients — an early Turing-test-style result., seemingly conscious AIseemingly conscious AI, n.: An AI that gives such a convincing impression of having feelings and a self that people believe it is conscious, whether or not it actually is.