maximum likelihood estimation
In plain wordsTuning a model's settings until the real data it learns from looks as likely, and so as unsurprising, to the model as possible. Most AI training, including the pretraining of chatbots, is a version of this.
A rule for fitting a model's parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. to data: score every candidate setting θ by the likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen. p(data | θ) it assigns to the observations and keep the highest-scoring one, in practice by minimizing the negative log-likelihoodlog-likelihood, n.: The logarithm of how probable a model finds the observed data. Taking logs turns a product of many tiny probabilities into a manageable sum; training usually minimizes its negative. analytically or with gradient methods. Under broad regularity conditions the estimate is consistent and asymptotically efficient, attaining the variance bound set by the Fisher informationFisher information, n.: A number, or table of numbers, saying how sharply the probability of the data reacts when a model's settings are nudged; the sharper the reaction, the more precisely the data can pin those settings down.. Many familiar losses are maximum likelihood in disguise: mean squared errormean squared error, n.: A way of scoring numerical predictions: square each prediction's error, then average them, so big misses count much more than small ones. Lower is better. corresponds to Gaussian noise, log losslog loss, n.: A score for probability predictions that punishes confident mistakes severely: saying '99% sure' and being wrong costs far more than saying '60% sure' and being wrong. Lower is better. to a Bernoulli model, and the cross-entropycross-entropy, n.: A score of how surprised a model's predictions are by what actually happens; lower is better. Language models are trained mainly by driving this number down on real text. used to pretrain language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. to maximizing the probability of the training text under an autoregressive modelautoregressive model, n.: A model that produces a sequence one piece at a time, each new piece depending on the pieces already produced — like writing a sentence word by word without going back.. Minimizing cross-entropy on data is the same as minimizing the Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right. KL(p_data ‖ p_θ) between the data distribution and the model. MLE can overfitoverfitting, n.: When a model learns its training examples too well — quirks and noise included — so it does great on data it has seen but poorly on anything new, like a student who memorizes answers without understanding. small datasets and yields point estimates without uncertainty; maximum a posteriori estimationmaximum a posteriori estimation, n.: Picking the single most believable setting of a model after combining the evidence with what was assumed beforehand. In practice it often amounts to ordinary training plus a penalty that keeps settings modest. and Bayesian inferenceBayesian inference, n.: Reasoning that treats unknowns as uncertain beliefs and updates them with evidence using Bayes' theorem. The result is not one answer but a range of answers, each with a probability. bring in a priorprior, n.: What you believe, and how strongly, before looking at the evidence in front of you. In AI it also means built-in assumptions that steer what a system learns..
Pretraining is maximum likelihood estimation on trillions of tokens.
Origin
Daniel Bernoulli (1778) and Carl Friedrich Gauss (1809) reasoned by choosing the most probable value. Ronald Fisher proposed an 'absolute criterion' in 1912 and introduced the term 'maximum likelihood', with its theory, in 'On the Mathematical Foundations of Theoretical Statistics' (1922).
Related
- likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen.
- log-likelihoodlog-likelihood, n.: The logarithm of how probable a model finds the observed data. Taking logs turns a product of many tiny probabilities into a manageable sum; training usually minimizes its negative.
- cross-entropycross-entropy, n.: A score of how surprised a model's predictions are by what actually happens; lower is better. Language models are trained mainly by driving this number down on real text.
- maximum a posteriori estimationmaximum a posteriori estimation, n.: Picking the single most believable setting of a model after combining the evidence with what was assumed beforehand. In practice it often amounts to ordinary training plus a penalty that keeps settings modest.
- Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right.
- Fisher informationFisher information, n.: A number, or table of numbers, saying how sharply the probability of the data reacts when a model's settings are nudged; the sharper the reaction, the more precisely the data can pin those settings down.
Sources
- Fisher, R. A. (1922). On the Mathematical Foundations of Theoretical Statistics. Philosophical Transactions of the Royal Society A, 222.
Mentioned in
Bayesian inferenceBayesian inference, n.: Reasoning that treats unknowns as uncertain beliefs and updates them with evidence using Bayes' theorem. The result is not one answer but a range of answers, each with a probability., cross-entropycross-entropy, n.: A score of how surprised a model's predictions are by what actually happens; lower is better. Language models are trained mainly by driving this number down on real text., density estimationdensity estimation, n.: Working out from a sample of data how likely different values are, by drawing a smooth curve or surface showing where data points bunch up and where they are rare., expectation–maximization algorithmexpectation–maximization algorithm, n.: A method for fitting a model when some information is hidden: guess the hidden parts using the current model, refit the model as if those guesses were right, and repeat until nothing changes., Gaussian mixture modelGaussian mixture model, n.: A model that explains data as coming from several overlapping bell-shaped clouds, working out where each cloud sits, how spread out it is, and how likely each point is to belong to it., generalized linear modelgeneralized linear model, n.: A family of statistical models that extends straight-line regression to outcomes such as yes/no answers, counts or waiting times, by linking the average outcome to a weighted sum of inputs through a suitable curve., Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right., likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen., linear regressionlinear regression, n.: Fitting a straight line (or flat surface) through data points to predict one number from others — say, a house's price from its floor area — choosing the line that keeps prediction errors smallest., log losslog loss, n.: A score for probability predictions that punishes confident mistakes severely: saying '99% sure' and being wrong costs far more than saying '60% sure' and being wrong. Lower is better., logistic regressionlogistic regression, n.: A simple, widely used method for yes-or-no predictions that outputs a probability — say, a 70% chance a customer cancels — by passing a weighted sum of inputs through an S-shaped curve., log-likelihoodlog-likelihood, n.: The logarithm of how probable a model finds the observed data. Taking logs turns a product of many tiny probabilities into a manageable sum; training usually minimizes its negative., mean squared errormean squared error, n.: A way of scoring numerical predictions: square each prediction's error, then average them, so big misses count much more than small ones. Lower is better., normal distributionnormal distribution, n.: The familiar bell-shaped curve: most values cluster near the average, and values become rapidly rarer the farther they lie from it. Measurement errors and adult heights roughly follow it., ordinary least squaresordinary least squares, n.: The standard way to fit a line through data: choose the line that makes the total of the squared vertical distances from the points to the line as small as possible., Bradley–Terry modelBradley–Terry model, n.: A simple statistical rule for turning head-to-head comparisons into scores: each item gets a strength, and the chance one beats another depends only on the difference in their strengths., Fisher informationFisher information, n.: A number, or table of numbers, saying how sharply the probability of the data reacts when a model's settings are nudged; the sharper the reaction, the more precisely the data can pin those settings down., maximum a posteriori estimationmaximum a posteriori estimation, n.: Picking the single most believable setting of a model after combining the evidence with what was assumed beforehand. In practice it often amounts to ordinary training plus a penalty that keeps settings modest.