Skip to content
Last Dictionary
Plane
Data & DatasetsMathematics & StatisticsSociety, Ethics & Economy

sampling bias

nounAlso: sample bias, sample selection bias, unrepresentative sample

In plain wordsA distortion that arises when the data collected over-represents some kinds of people or situations and under-represents others, so conclusions drawn from it don't hold for everyone.

  1. Systematic error introduced by the way data are gathered, when the collection method favors some parts of the target population and neglects others, so that the sample's makeup no longer mirrors the population's. Unlike random sampling error, it does not shrink as the sample grows. In AI, sampling bias enters through convenience sampling (images from photo-sharing sites, text from the English-language web), through who participates (smartphone owners, patients at one hospital) and through how data are filtered. Models then perform worse for groups and conditions underrepresented in training: face-analysis systems trained mostly on lighter-skinned faces, speech recognizers tuned to some accents, medical models built on one population. Remedies include deliberate sampling design and stratification, reweighting or resampling toward the target distribution, targeted collection of missing data, and reporting performance by subgroup.

    Because the app's users skewed young and urban, the training data had a strong sampling bias.

Origin

A classic concern of survey statistics. The best-known cautionary tale is the 1936 Literary Digest poll, which mailed ballots to people found on lists such as telephone directories and car registrations and wrongly predicted that Alf Landon would defeat Franklin Roosevelt; later analysis showed that non-response — who chose to mail ballots back — also skewed the result. In AI the issue gained prominence through audits of face-analysis systems from 2018.

Usage note

Sampling bias is one form of selection bias, the broader term for any non-random selection of data or subjects. It is a common cause of algorithmic biasalgorithmic bias, n.: Unfairness in software that judges or serves people: it consistently produces worse results for some groups, such as women, older people or ethnic minorities, often because it learned from skewed or incomplete records. and of distribution shiftdistribution shift, n.: When the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested. between training and deployment.

Related

See also
  • algorithmic biasalgorithmic bias, n.: Unfairness in software that judges or serves people: it consistently produces worse results for some groups, such as women, older people or ethnic minorities, often because it learned from skewed or incomplete records.
  • distribution shiftdistribution shift, n.: When the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested.
  • class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters.
  • Gender ShadesGender Shades, prop. n.: A 2018 study showing that commercial face-analysis software guessed the gender of darker-skinned women far less accurately than that of lighter-skinned men.
  • data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills.
  • long taillong tail, n.: The huge number of rare things — uncommon words, unusual road situations, obscure facts — that each appear seldom but together make up a large share of what an AI system meets.
  • biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system.

Mentioned in

biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system., class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters., data qualitydata quality, n.: How well a set of data does its job: whether it is accurate, complete, up to date, consistent and representative — and, for AI training, informative enough to teach useful skills., long taillong tail, n.: The huge number of rare things — uncommon words, unusual road situations, obscure facts — that each appear seldom but together make up a large share of what an AI system meets., low-resource languagelow-resource language, n.: A language with little digital text, recorded speech or language tools available — often one spoken by millions — which makes AI systems much weaker in it than in English., samplingsampling, n.: Picking some items at random from a larger set or from a range of possibilities — surveying 1,000 voters, say, or letting an AI choose its next word by a weighted lottery., covariate shiftcovariate shift, n.: A kind of mismatch where the inputs an AI system sees change — say, photos from a new camera — while the right answer for any given input stays the same.