large language model
In plain wordsA computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.
A language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. of very large scale — billions to trillions of parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned., today almost always arranged as a decoder-only transformerdecoder-only transformer, n.: The kind of transformer behind most chatbots: a single stack that reads text left to right and continues it, without a separate part that first encodes an input. — whose core skill, next-token predictionnext-token prediction, n.: The simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers., is learned during pretrainingpretraining, n.: The first and usually the most expensive stage of building a modern AI model, in which it learns general patterns from enormous amounts of raw text, images or code before being specialized for particular uses. on trillions of tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. of text, often with code, images and audio, the text itself supplying the training targets (self-supervised learningself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image.). Pretraining yields a base modelbase model, n.: A language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering. that continues text; post-trainingpost-training, n.: Everything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner. such as instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text. and reinforcement learning from human feedbackreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well. turns it into a chat modelchat model, n.: A language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps. or reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. that follows instructions. Because a single model can perform many tasks described in a promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., through in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round. rather than task-specific training, LLMs displaced most specialized natural language processingnatural language processing, n.: The field that teaches computers to handle human language — to read, interpret, translate, summarize and produce text and speech. It is one of the oldest and largest areas of artificial intelligence. systems after 2020 and became the core of chat assistants and AI agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.. Capabilities improve predictably with parameters, data and compute (scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.), though particular skills can appear abruptly. There is no agreed size threshold: GPT-2's 1.5 billion parameters counted as large in 2019, while several models marketed as small in 2025 exceed it. Most LLMs are foundation modelsfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools., and most frontier LLMs are now multimodal.
We route simple lookups to a small model and send multi-step questions to the large language model.
Origin
An early use of the phrase is the title of Thorsten Brants, Ashok Popat, Peng Xu, Franz Och and Jeffrey Dean's 2007 paper 'Large Language Models in Machine Translation', whose n-gram models were trained on up to 2 trillion tokens. The neural sense took hold around GPT-3GPT-3, prop. n.: A huge 2020 OpenAI language model that could perform new tasks simply from instructions or a few examples written into its prompt, without retraining. It made prompting a new way of programming. (2020), and 'LLM' became an everyday word after ChatGPTChatGPT, prop. n.: OpenAI's AI chatbot, released free to the public in November 2022, which answers questions, writes and converses in everyday language. Its sudden popularity began the current AI boom.'s release in November 2022.
Usage note
Often used loosely for any chatbot or generative-AI product. Strictly, the LLM is the model; products add system promptssystem prompt, n.: Background instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply., tools, retrieval and safety layers around it. Whether LLMs understand language is debated: see machine understandingmachine understanding, n.: Whether, and in what sense, an AI system grasps what words, situations or problems mean — rather than just producing answers that look as though it does. and stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean..
Related
- language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own.
- transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
- foundation modelfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools.
- pretrainingpretraining, n.: The first and usually the most expensive stage of building a modern AI model, in which it learns general patterns from enormous amounts of raw text, images or code before being specialized for particular uses.
- post-trainingpost-training, n.: Everything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner.
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
- small language modelsmall language model, n.: A language model built to be compact enough to run cheaply and quickly, even on a laptop or phone, trading some knowledge and skill for speed, privacy and lower cost.
- reasoning modelreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
- small language modelsmall language model, n.: A language model built to be compact enough to run cheaply and quickly, even on a laptop or phone, trading some knowledge and skill for speed, privacy and lower cost.
- n-gram modeln-gram model, n.: An early type of language model that predicts the next word from just the few words before it, using counts of how often each word followed those words in a large collection of text.
Sources
- Brants, T., Popat, A. C., Xu, P., Och, F. J. & Dean, J. (2007). Large Language Models in Machine Translation. Proceedings of EMNLP-CoNLL 2007, 858–867. aclanthology.org/D07-1090
- Brown, T. B. et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020. arxiv.org/abs/2005.14165v2
Mentioned in
adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon., agent memoryagent memory, n.: The ways an AI assistant or agent keeps information — your preferences, past conversations, lessons from earlier tasks — and brings it back later, beyond what fits in a single conversation., AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done., AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this., AnthropicAnthropic, prop. n.: An American AI company that makes the Claude chatbot and models. It was founded in 2021 by former OpenAI staff and puts particular emphasis on AI safety research., artificial general intelligenceartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive., artificial intelligenceartificial intelligence, n.: Computer systems that do things we would normally credit to a thinking mind, such as holding a conversation, reading an X-ray or planning a route, and the field of research that builds them., chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going., ChatGPTChatGPT, prop. n.: OpenAI's AI chatbot, released free to the public in November 2022, which answers questions, writes and converses in everyday language. Its sudden popularity began the current AI boom., ClaudeClaude, prop. n.: A family of AI assistants and language models made by the company Anthropic since 2023, widely used for writing, analysis and computer programming., copilotcopilot, n.: An AI assistant built into software that works alongside a person — suggesting code, drafting text or answering questions — while the person stays in charge, like a co-pilot beside the captain., DeepSeekDeepSeek, prop. n.: A Chinese artificial intelligence company whose free, cheaply trained models startled the world in January 2025 by nearly matching the best American systems., embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance., few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice., frontier modelfrontier model, n.: One of the most capable AI models in existence at a given time, built by a handful of leading labs at great cost, and the main target of AI safety rules., GeminiGemini, prop. n.: Google's family of AI models and its chatbot assistant, launched in December 2023. Gemini models handle text, images, audio and video and power many Google products., generative AIgenerative AI, n.: AI that produces something new on request, such as an essay, a picture, a song or a piece of code, having absorbed from enormous numbers of existing examples what such things typically look like., GitHub CopilotGitHub Copilot, prop. n.: An AI assistant from GitHub, built into programmers' code editors, that suggests code as they type, answers questions about code and can carry out coding tasks on its own., GPTGPT, abbr.: Short for 'generative pre-trained transformer', the family of OpenAI models behind ChatGPT. The letters are also used loosely for any similar chatbot model., GPT-5GPT-5, prop. n.: OpenAI's flagship AI model from August 2025, which decides by itself whether to answer quickly or think longer. It replaced earlier models in ChatGPT and was followed by frequent upgrades., inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text., interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box., knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run., language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own.