precision
In plain wordsOf all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder.
- 1
Classification and retrieval
The fraction of predicted positives that are truly positive: TP/(TP + FP), where TP counts true positives and FP false positivesfalse positive, n.: A false alarm: a test or AI system raises a flag, for example for illness, fraud or danger, when nothing of the kind is really there.. In retrieval it is the fraction of returned documents that are relevant (precision@k for the top k results). Precision answers 'when the model says yes, how often is it right?' and matters most when false alarms are costly, as in spam filtering, content takedowns or recommending a medical intervention. It trades off against recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer.: raising a classifier's decision threshold usually increases precision and decreases recall, a trade-off traced by the precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare. and summarized by the F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good.. Unlike accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others., precision ignores true negatives, which makes it informative under class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters., but it depends on prevalence: the same classifier shows lower precision in a population where positives are rarer. In medicine it is called positive predictive value.
At a 0.9 threshold the moderation classifier's precision is 97%, but it catches only 40% of the abuse.
- 2
Numerical computing
The number of bits, and hence significant digits, used to represent numbers in a floating-point formatfloating-point format, n.: A way of storing numbers in a computer like scientific notation — a few digits plus a power of two — trading accuracy for range. AI uses ever-smaller formats to compute faster., as in single (FP32), half (FP16), bfloat16bfloat16, n.: A 16-bit number format invented at Google for AI. It keeps the huge range of the standard 32-bit format but drops precision, which neural networks can usually tolerate. or 8-bit precision. Lower precision cuts memory use and speeds computation at the cost of rounding error and a narrower range, the trade-off exploited by mixed-precision trainingmixed-precision training, n.: Training a model with shorter, less exact numbers for most of the arithmetic and full-length numbers only where accuracy really matters, which saves a great deal of memory and lets modern chips train far faster. and quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy..
- 3
Statistics
The reciprocal of a distribution's variance, or for a multivariate Gaussian the inverse of its covariance matrix (the precision matrix). Bayesian statisticians often work with precisions because the precisions of independent Gaussian sources of evidence simply add when the sources are combined.
Origin
The classification sense comes from information retrieval. Allen Kent and colleagues proposed in 1955 to judge retrieval systems both by the proportion of retrieved documents that are relevant and by the proportion of relevant documents retrieved, and Cyril Cleverdon's Cranfield experiments of the late 1950s and 1960s helped make precision and recall the field's standard pair. Machine learning adopted both for classification.
Usage note
Three unrelated senses share the word, so check whether 'higher precision' means fewer false positives or more bits per number. In metrology, precision means repeatability and is contrasted with accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others..
Related
- recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer.
- F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good.
- precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare.
- confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up.
- false positivefalse positive, n.: A false alarm: a test or AI system raises a flag, for example for illness, fraud or danger, when nothing of the kind is really there.
- floating-point formatfloating-point format, n.: A way of storing numbers in a computer like scientific notation — a few digits plus a power of two — trading accuracy for range. AI uses ever-smaller formats to compute faster.
- mixed-precision trainingmixed-precision training, n.: Training a model with shorter, less exact numbers for most of the arithmetic and full-length numbers only where accuracy really matters, which saves a great deal of memory and lets modern chips train far faster.
- quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.
- recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer.
- accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others.
Sources
- Kent, A., Berry, M. M., Luehrs, F. U. & Perry, J. W. (1955). Machine Literature Searching VIII. Operational Criteria for Designing Information Retrieval Systems. American Documentation 6(2).
- van Rijsbergen, C. J. (1979). Information Retrieval, 2nd ed. London: Butterworths.
Mentioned in
accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others., classificationclassification, n.: Sorting things into categories — spam or not spam, which animal is in a photo, which language a text is written in — by learning from examples that are already labeled., false positivefalse positive, n.: A false alarm: a test or AI system raises a flag, for example for illness, fraud or danger, when nothing of the kind is really there., binary classificationbinary classification, n.: Sorting things into exactly two groups, such as spam versus not spam or sick versus healthy. Most yes-or-no decisions made by AI systems are of this kind., class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters., confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up., decision thresholddecision threshold, n.: The cut-off score above which a model's prediction counts as 'yes'; moving it trades missed cases against false alarms, like setting how sensitive a smoke alarm should be., F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good., information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up., mean average precisionmean average precision, n.: The standard score for object detectors: it rewards finding all the objects while raising few false alarms, averaged over every type of object. Higher is better, up to 100., mixed-precision trainingmixed-precision training, n.: Training a model with shorter, less exact numbers for most of the arithmetic and full-length numbers only where accuracy really matters, which saves a great deal of memory and lets modern chips train far faster., multilabel classificationmultilabel classification, n.: Tagging each item with any number of labels at once — a photo can show a beach, a dog and a sunset — instead of picking just one category., named-entity recognitionnamed-entity recognition, n.: Software that spots names and similar expressions in text and labels each by type: in 'Maria flew from Lagos to Paris on Monday', it marks a person, two cities and a date., precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare., recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer., recall@krecall@k, n.: A score for search systems: of all the right answers that exist, what fraction showed up in the top k results? Higher means fewer relevant items were missed., specificityspecificity, n.: Of all the cases that really were negative, the share a test correctly cleared; a highly specific test rarely raises false alarms on healthy people., Message Understanding ConferenceMessage Understanding Conference, prop. n.: A series of US government-funded contests held from 1987 to 1998 in which research teams built programs to pull key facts, such as who did what, where and when, out of news and military messages.