probability

A measure of uncertainty that assigns each event a number between 0 and 1, giving certain events 1 and making the probabilities of mutually exclusive events add. Andrey Kolmogorov's axioms (1933) define probability as a normalized measure on a space of outcomes, and the resulting theory — random variablesrandom variable, n.: A quantity whose value depends on chance, such as the number rolled on a die or the next word a chatbot will write., distributionsprobability distribution, n.: A complete description of how likely each possible outcome is — for a fair die, one-sixth for each face. A language model produces one of these over its whole vocabulary at every step., expectationsexpected value, n.: The long-run average you would get if a chancy situation could be repeated many times — 3.5 for a fair die, even though no single roll shows 3.5., conditioningconditional probability, n.: The chance of something once you already know that something else is true — for example, the chance of rain given that the sky is cloudy. and limit theorems — holds whatever probability is taken to mean. Interpretations differ: frequentists read it as long-run relative frequency in repeated trials, Bayesians as a rational degree of belief updated by Bayes' theoremBayes' theorem, n.: A formula for updating how much you believe something after seeing new evidence: weigh your earlier belief by how well that explanation predicts what you actually observed.. Machine learning uses both readings and rests on probability throughout: classifiers output class probabilities, language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. define probabilities over sequences of tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens., training often maximizes likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen., and uncertainty, calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time. and samplingsampling, n.: Picking some items at random from a larger set or from a range of possibilities — surveying 1,000 voters, say, or letting an AI choose its next word by a weighted lottery. are probabilistic notions.

The model gives 'Paris' a probability of 0.92, but that number means little unless the model is calibrated.