label noise
In plain wordsMistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems.
Errors in the labelslabel, n.: The correct answer attached to an example in a dataset — 'cat' on a photo of a cat, 'positive' on an upbeat review — that a learning system is trained to predict. of a dataset: examples whose recorded label differs from the correct one. Models of label noise distinguish random noise that flips labels uniformly, class-conditional noise that depends on the true class (confusing similar dog breeds, say) and instance-dependent noise concentrated on hard or ambiguous examples. Sources include annotator mistakes, vague guidelines, automated or weakly supervisedweak supervision, n.: Training AI with labels that are cheap but imperfect, such as rough rules of thumb, partial tags or noisy guesses, instead of carefully hand-checked answers. labeling and deliberate tampering (data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.). Large networks can fit noisy labels perfectly, which harms generalizationgeneralization, n.: How well a model copes with new examples it has never seen, rather than just memorizing its training data, like a student who understands the method, not just the answers.; remedies include noise-robust losses, sample reweighting, training schemes in which two networks filter each other's data, 'confident learning' to flag likely errors, and relabeling. Noise in test sets is especially damaging: a 2021 audit estimated an average label error rate of at least 3.3% across ten common benchmarks and at least 6% in ImageNet's validation set, enough to change model rankings.
After relabeling the 2% of examples flagged by confident learning, validation accuracy rose three points.
Origin
Studied in learning theory since Dana Angluin and Philip Laird's 'Learning from Noisy Examples' (1988), which introduced the random classification noise model. It became a major deep-learning topic after Chiyuan Zhang and colleagues showed in 2016 that neural networks can memorize even randomly assigned labels.
Usage note
Not all disagreement is noise: for subjective tasks such as toxicity or sentiment, variation among annotators can carry real information about ambiguity.
Related
- labellabel, n.: The correct answer attached to an example in a dataset — 'cat' on a photo of a cat, 'positive' on an upbeat review — that a learning system is trained to predict.
- inter-annotator agreementinter-annotator agreement, n.: A measure of how often different people labeling the same data give the same answer. Low agreement warns that the task is unclear or the labels are unreliable.
- data cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results.
- data poisoningdata poisoning, n.: Sneaking bad examples into the data an AI learns from, so the finished model behaves as the attacker wants — badly overall, or wrongly only in special situations.
- ground truthground truth, n.: The reference answers treated as correct when checking or training an AI system — for example, the confirmed diagnoses that a medical-imaging model's guesses are compared against.
- data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills.
Sources
- Angluin, D. & Laird, P. (1988). Learning from Noisy Examples. Machine Learning 2(4).
- Northcutt, C. G., Athalye, A. & Mueller, J. (2021). Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. NeurIPS 2021 Datasets and Benchmarks Track. arXiv:2103.14749. arxiv.org/abs/2103.14749
Mentioned in
labellabel, n.: The correct answer attached to an example in a dataset — 'cat' on a photo of a cat, 'positive' on an upbeat review — that a learning system is trained to predict., data cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results., data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills., data-centric AIdata-centric AI, n.: An approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model., ground truthground truth, n.: The reference answers treated as correct when checking or training an AI system — for example, the confirmed diagnoses that a medical-imaging model's guesses are compared against., Bayes error rateBayes error rate, n.: The lowest error any classifier could ever achieve on a task, because some cases are genuinely ambiguous: even a perfect model that knew the true odds would still sometimes guess wrong., Cohen's kappaCohen's kappa, n.: A score for how much two people labeling the same items agree beyond what lucky guessing would produce: 1 means perfect agreement, 0 means no better than chance., implicit feedbackimplicit feedback, n.: Clues about what people like that come from what they do, such as clicking, buying, watching or skipping, rather than from ratings or reviews they deliberately give., inter-annotator agreementinter-annotator agreement, n.: A measure of how often different people labeling the same data give the same answer. Low agreement warns that the task is unclear or the labels are unreliable., sharpness-aware minimizationsharpness-aware minimization, n.: A training method that looks for settings where the error stays low even if the settings are nudged a little, rather than settings that are merely low at one precise point., weak supervisionweak supervision, n.: Training AI with labels that are cheap but imperfect, such as rough rules of thumb, partial tags or noisy guesses, instead of carefully hand-checked answers.