fancy autocomplete
In plain wordsA dismissive description of AI chatbots as nothing more than a souped-up version of the feature on a phone keyboard that suggests your next word.
A deflationary characterization of large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. as elaborate versions of a phone keyboard's next-word suggestions. The literal part is accurate: a language model is trained by next-token predictionnext-token prediction, n.: The simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers. and generates text one tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. at a time. The disputed part is the implied 'merely'. Critics of the phrase argue that predicting text well at scale requires internal representations of facts, reasoning steps and other people's beliefs, and that post-trainingpost-training, n.: Everything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner., reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. and tool usetool use, n.: An AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work. make current systems more than predictors of internet text; users of the phrase answer that claims about capability should be tested against the mechanism rather than against the impression fluency creates. Often used interchangeably with stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean., though it stresses the training objective rather than the absence of meaning.
If it's fancy autocomplete, it's autocomplete that just found a bug our whole team missed.
Origin
Comparisons of language models to predictive text circulated around GPT-2 (2019) and spread widely after ChatGPT's release in November 2022, in variants such as 'autocomplete on steroids' and 'glorified autocomplete'. No single coiner is known.
Usage note
Geoffrey Hinton and Ilya Sutskever have argued the reverse — that predicting the next word very well requires understanding what is being said — while linguists such as Emily M. Bender reject that inference.
Related
- stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean.
- next-token predictionnext-token prediction, n.: The simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers.
- language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own.
- Potemkin understandingPotemkin understanding, n.: When an AI can correctly explain a concept yet cannot use it properly, revealing that its apparent understanding is a facade — like the fake village fronts said to have been built to impress a Russian empress.
- world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.
- base modelbase model, n.: A language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering.
Mentioned in
next-token predictionnext-token prediction, n.: The simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers., stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean., blurry JPEG of the webblurry JPEG of the web, phr.: Writer Ted Chiang's 2023 comparison of ChatGPT to a low-quality compressed photo of the internet: it keeps the gist of what it learned but blurs the details and fills gaps with plausible guesses., Potemkin understandingPotemkin understanding, n.: When an AI can correctly explain a concept yet cannot use it properly, revealing that its apparent understanding is a facade — like the fake village fronts said to have been built to impress a Russian empress.