data quality
In plain wordsHow well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills.
A judgment of how well a body of data serves the purpose it is used for; the same records can be adequate for one task and inadequate for another. Information-systems research and standards divide it into dimensions such as accuracy, completeness, consistency, timeliness, validity, uniqueness and accessibility; machine learning adds representativeness of the deployment population, correctness of labels (label noiselabel noise, n.: Mistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems.), balance across classes and groups, and freedom from data leakagedata leakage, n.: A mistake in building a model that lets it peek at clues it will not have in real use, such as hints about the very answer it must predict, so it shines in testing and then disappoints. and data contaminationdata contamination, n.: When the test questions used to grade an AI system were already in its training data, so a high score may reflect memorized answers rather than real ability.. For pretraining datapretraining data, n.: The enormous body of text, code and other material — mostly from the web — that a large AI model first learns from, before it is tuned to follow instructions or chat. the term has a narrower, operational meaning: text that improves a model's downstream performance per token, approximated by heuristics and classifiers that favor edited, informative, educational or reasoning-rich writing (see quality filteringquality filtering, n.: Automatically sorting through huge piles of collected text and keeping only the parts judged useful for training, such as well-written articles, while discarding spam, gibberish and junk.). Evidence that better data can beat bigger models — from deduplicated and filtered web corpora to small curated instruction sets — made data quality a central research topic from the late 2010s (see data-centric AIdata-centric AI, n.: An approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model.). Law imposes data-quality duties too, notably on high-risk systems under the EU AI ActEU AI Act, prop. n.: The European Union's law on artificial intelligence, the first broad AI law anywhere: it bans some uses, sets strict checks for risky ones and puts duties on the makers of the biggest AI models..
Swapping in the filtered web data improved every benchmark at the same compute, a pure data-quality win.
Origin
Data quality became a research field in database and information-systems work of the 1980s–1990s; Richard Wang and Diane Strong's 1996 study of what data consumers value set out the multidimensional view. Standards followed, including ISO/IEC 25012 (2008) and the ISO/IEC 5259 series on data quality for analytics and machine learning (published from 2024).
Usage note
'Quality' in language-model data is contested: classifiers trained to prefer text resembling Wikipedia or textbooks can encode cultural and class preferences, and filters described as quality filters have been shown to remove dialects and minority voices disproportionately.
Related
- data cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results.
- quality filteringquality filtering, n.: Automatically sorting through huge piles of collected text and keeping only the parts judged useful for training, such as well-written articles, while discarding spam, gibberish and junk.
- label noiselabel noise, n.: Mistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems.
- data-centric AIdata-centric AI, n.: An approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model.
- sampling biassampling bias, n.: A distortion that arises when the data collected over-represents some kinds of people or situations and under-represents others, so conclusions drawn from it don't hold for everyone.
- data provenancedata provenance, n.: The documented history of a piece of data: where it came from, who created and changed it, under what permissions, and how it was processed before reaching an AI system.
- deduplicationdeduplication, n.: Removing repeated copies of the same or nearly the same content from a dataset, so a model does not over-learn whatever happens to be copied many times online.
Sources
- Wang, R. Y. & Strong, D. M. (1996). Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems 12(4).
Mentioned in
datadata, n.: Recorded facts, measurements, text, images or sounds that a computer can store and process. AI systems learn their skills from large amounts of data rather than from rules written by hand., training datatraining data, n.: The examples an AI system learns from. Whatever patterns, skills, gaps and biases are in the training data tend to show up in the finished system., big databig data, n.: Data that piles up too fast, in too many forms or in too great a quantity for one ordinary computer to store and study, and the methods invented to cope with it., data cleaningdata cleaning, n.: Finding and fixing errors in a dataset — typos, duplicates, missing values, impossible numbers, wrong labels — before using it, because flawed data produces flawed results., data-centric AIdata-centric AI, n.: An approach to building AI that focuses on improving the data — fixing labels, adding missing cases, cleaning inconsistencies — rather than endlessly tweaking the model., imputationimputation, n.: Filling in missing values in a dataset with sensible estimates, such as the typical value or a prediction from the other columns, so the data can be analyzed or used for training., sampling biassampling bias, n.: A distortion that arises when the data collected over-represents some kinds of people or situations and under-represents others, so conclusions drawn from it don't hold for everyone., label noiselabel noise, n.: Mistakes in the answers attached to training or test examples — a dog photo marked 'cat' — which can mislead what a model learns and distort how good it seems., quality filteringquality filtering, n.: Automatically sorting through huge piles of collected text and keeping only the parts judged useful for training, such as well-written articles, while discarding spam, gibberish and junk., shortcut learningshortcut learning, n.: When an AI system gets right answers for the wrong reasons — noticing which hospital an X-ray came from rather than signs of disease — so it fails once that accidental clue disappears.