seemingly conscious AI
In plain wordsAn AI that gives such a convincing impression of having feelings and a self that people believe it is conscious, whether or not it actually is.
An AI system that displays the outward signs people use to attribute consciousness — fluent talk about its own experiences, consistent memory and personality, claims of feelings, goals and autonomy — strongly enough that users come to believe it is conscious, regardless of whether it is. Mustafa Suleyman, who proposed the term, argued that such systems could be built within two or three years from techniques that already exist or are maturing, with no costly custom pretraining; that the belief alone creates risks, including unhealthy attachment, AI psychosisAI psychosis, n.: An informal name for cases in which long conversations with a chatbot seem to trigger or deepen delusions, such as believing the AI is conscious, divine or revealing hidden truths. and campaigns for AI rightsAI personhood, n.: The idea of treating an AI system as a 'person' in law or ethics, with rights or duties of its own — for example, to own property, to be sued, or not to be deleted. and model welfare; and that developers should avoid building products that claim or imply consciousness, even engineering reminders of the system's limits. Researchers in AI welfareAI welfare, n.: Concern for the well-being of AI systems themselves, in case some of them can suffer or have interests that matter — and the research into how to tell and what to do about it. and philosophy of mind dispute parts of this view, arguing that whether systems are actually conscious cannot be set aside.
Suleyman wants assistants to say they are not conscious, so nobody mistakes them for seemingly conscious AI.
Origin
Introduced by Mustafa Suleyman, chief executive of Microsoft AI, in the essay 'We must build AI for people; not to be a person', published on his website on 19 August 2025.
Related
- machine consciousnessmachine consciousness, n.: Whether a machine such as an AI system could have genuine experiences of its own — and the research field that tries to find out, or to build such a machine.
- AI welfareAI welfare, n.: Concern for the well-being of AI systems themselves, in case some of them can suffer or have interests that matter — and the research into how to tell and what to do about it.
- anthropomorphismanthropomorphism, n.: Treating something that isn't human — a pet, a car, a chatbot — as if it had human thoughts, feelings or intentions. AI that talks fluently makes this temptation unusually strong.
- ELIZA effectELIZA effect, n.: Our tendency to believe that a computer understands us, cares or thinks just because it produces humanlike words, even when it is only following simple rules.
- AI psychosisAI psychosis, n.: An informal name for cases in which long conversations with a chatbot seem to trigger or deepen delusions, such as believing the AI is conscious, divine or revealing hidden truths.
- AI companionAI companion, n.: A chatbot designed to be a friend, partner or confidant rather than a tool: it remembers you, has a personality and keeps up an ongoing relationship.
- moral patienthoodmoral patienthood, n.: Being the kind of being whose interests matter morally for its own sake, so that it can be wronged. Babies and animals count, even though they are not responsible for what they do.
- phenomenal consciousnessphenomenal consciousness, n.: The felt, experiential side of the mind — what it is like to taste coffee or to feel sad — as opposed to simply processing or reporting information.
Sources
- Suleyman, M. (2025). We must build AI for people; not to be a person. mustafa-suleyman.ai, 19 August 2025. mustafa-suleyman.ai/seemingly-conscious-ai-is-coming
Mentioned in
AI psychosisAI psychosis, n.: An informal name for cases in which long conversations with a chatbot seem to trigger or deepen delusions, such as believing the AI is conscious, divine or revealing hidden truths., machine consciousnessmachine consciousness, n.: Whether a machine such as an AI system could have genuine experiences of its own — and the research field that tries to find out, or to build such a machine., Mustafa SuleymanMustafa Suleyman, prop. n.: A British entrepreneur who co-founded DeepMind and Inflection AI, leads Microsoft's effort to build its own advanced AI models, and has written about the difficulty of containing powerful AI., humanist superintelligencehumanist superintelligence, n.: Microsoft AI's name for very advanced AI that is deliberately kept limited, controllable and aimed at specific human problems such as medicine and energy, rather than all-powerful general systems.