false positive
In plain wordsA false alarm: a test or AI system raises a flag, for example for illness, fraud or danger, when nothing of the kind is really there.
A case that a classifier or test labels positive although its true label is negative: a legitimate email sent to spam, a healthy patient flagged as ill, an innocent face 'matched' to a watch list. In statistical hypothesis testing it corresponds to a type I error, rejecting a null hypothesis that is true. False positives are counted in the confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up. and drive two distinct rates: the false positive rate, FP/(FP + TN), the share of negatives wrongly flagged (the x-axis of the ROC curveROC curve, n.: A graph showing how a detector's hit rate and false-alarm rate change together as it is made more or less trigger-happy; the closer the curve hugs the top-left corner, the better., equal to 1 − specificityspecificity, n.: Of all the cases that really were negative, the share a test correctly cleared; a highly specific test rarely raises false alarms on healthy people.); and the false discovery rate, FP/(FP + TP), the share of flagged cases that are wrong (equal to 1 − precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder.). When positives are rare, even a small false positive rate can mean that most alarms are false, the base-rate effect that makes mass screening and surveillance error-prone. Lowering the decision threshold trades fewer false negativesfalse negative, n.: A miss: a test or AI system gives the all-clear, for example no illness, no fraud or no danger, when the problem is in fact there. for more false positives; the right balance depends on the relative costs of the two errors.
With a 1% false positive rate and one fraudster per 10,000 customers, about 99 of every 100 alerts will be false positives.
Origin
The language of positive and negative results comes from medical and laboratory testing in the first half of the 20th century. Jerzy Neyman and Egon Pearson formalized the corresponding errors of the first and second kind in hypothesis testing between 1928 and 1933, and radar detection theory of the 1940s, later generalized as signal detection theory, supplied 'false alarm'.
Usage note
Error rates of facial recognitionfacial recognition, n.: Technology that identifies or checks who someone is by comparing a picture of their face with stored photos, used by phones, passport gates and police databases. systems, including false positives, have been found to vary across demographic groups, which is central to debates about algorithmic biasalgorithmic bias, n.: Unfairness in software that judges or serves people: it consistently produces worse results for some groups, such as women, older people or ethnic minorities, often because it learned from skewed or incomplete records..
Related
- false negativefalse negative, n.: A miss: a test or AI system gives the all-clear, for example no illness, no fraud or no danger, when the problem is in fact there.
- confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up.
- precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder.
- specificityspecificity, n.: Of all the cases that really were negative, the share a test correctly cleared; a highly specific test rarely raises false alarms on healthy people.
- ROC curveROC curve, n.: A graph showing how a detector's hit rate and false-alarm rate change together as it is made more or less trigger-happy; the closer the curve hugs the top-left corner, the better.
- p-valuep-value, n.: A number saying how surprising your data would be if nothing interesting were going on. A small p-value hints at a real effect, but it is not the probability that the effect is real.
- class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters.
- algorithmic biasalgorithmic bias, n.: Unfairness in software that judges or serves people: it consistently produces worse results for some groups, such as women, older people or ethnic minorities, often because it learned from skewed or incomplete records.
- false negativefalse negative, n.: A miss: a test or AI system gives the all-clear, for example no illness, no fraud or no danger, when the problem is in fact there.
Sources
- Neyman, J. & Pearson, E. S. (1933). On the Problem of the Most Efficient Tests of Statistical Hypotheses. Philosophical Transactions of the Royal Society A 231.
- Grother, P., Ngan, M. & Hanaoka, K. (2019). Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects. NIST Interagency Report 8280.
Mentioned in
binary classificationbinary classification, n.: Sorting things into exactly two groups, such as spam versus not spam or sick versus healthy. Most yes-or-no decisions made by AI systems are of this kind., confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up., false negativefalse negative, n.: A miss: a test or AI system gives the all-clear, for example no illness, no fraud or no danger, when the problem is in fact there., precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder., recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer., ROC curveROC curve, n.: A graph showing how a detector's hit rate and false-alarm rate change together as it is made more or less trigger-happy; the closer the curve hugs the top-left corner, the better., specificityspecificity, n.: Of all the cases that really were negative, the share a test correctly cleared; a highly specific test rarely raises false alarms on healthy people.