distribution shift
In plain wordsWhen the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested.
A difference between the joint distribution of inputs and outputs in a model's training data and in the data it faces at test or deployment time, violating the i.i.d.i.i.d., abbr.: Short for 'independent and identically distributed': every data point is produced the same way, and none influences any other — like repeated rolls of one fair die. assumption behind most performance estimates. Standard types are covariate shiftcovariate shift, n.: A kind of mismatch where the inputs an AI system sees change — say, photos from a new camera — while the right answer for any given input stays the same. (the input distribution changes while the input–output relationship holds), label shiftlabel shift, n.: A kind of mismatch where the mix of answers changes — say, a disease becomes far more common during an outbreak — while each individual case still looks the same as before. (class proportions change while each class looks the same) and concept shift (the relationship itself changes, as when the same symptoms indicate different diseases in another population); applied work also speaks of temporal, geographic and subpopulation shift. Real cases include medical models moved between hospitals, wildlife classifiers moved between camera traps, and language models meeting slang or events after their knowledge cutoffknowledge cutoff, n.: The date after which an AI model knows nothing from its training, because its training data stops there. Without a web search tool, it cannot know about later events.. Responses include collecting deployment-like data, reweighting, domain adaptation, robust training objectives, monitoring for data driftdata drift, n.: When the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable. and concept driftconcept drift, n.: When the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold., and evaluation on shifted benchmarks such as WILDS.
Accuracy fell from 94% to 71% at the second hospital — a textbook distribution shift.
Origin
Studied in statistics and econometrics as sample-selection and covariate-shift problems, and gathered under the name 'dataset shift' in the edited volume Dataset Shift in Machine Learning (Joaquin Quiñonero-Candela and colleagues, MIT Press, 2009). 'Distribution shift' had become the usual deep-learning term by the early 2020s, when benchmarks such as WILDS (Pang Wei Koh and colleagues, 2021) collected real-world shifts.
Usage note
data driftdata drift, n.: When the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable. and concept driftconcept drift, n.: When the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold. are the MLOps names for shifts that develop over time in production. A shift is not always harmful, and detecting one does not show whether accuracy has fallen; that requires fresh labels.
Related
- covariate shiftcovariate shift, n.: A kind of mismatch where the inputs an AI system sees change — say, photos from a new camera — while the right answer for any given input stays the same.
- label shiftlabel shift, n.: A kind of mismatch where the mix of answers changes — say, a disease becomes far more common during an outbreak — while each individual case still looks the same as before.
- out-of-distributionout-of-distribution, adj.: Describes inputs that differ from anything an AI system saw in training — like a model trained on daytime photos facing a night scene — where its answers become less reliable.
- data driftdata drift, n.: When the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable.
- concept driftconcept drift, n.: When the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold.
- robustnessrobustness, n.: An AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it.
- transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data.
Sources
- Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A. & Lawrence, N. D. (eds.) (2009). Dataset Shift in Machine Learning. MIT Press.
- Koh, P. W. et al. (2021). WILDS: A Benchmark of in-the-Wild Distribution Shifts. ICML 2021. arXiv:2012.07421. arxiv.org/abs/2012.07421
Mentioned in
AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable., correlationcorrelation, n.: A measure of how closely two quantities move together, from −1 (one rises as the other falls) through 0 (no straight-line link) to +1 (they rise together). It does not show that one causes the other., image classificationimage classification, n.: Teaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities., training settraining set, n.: The examples a machine-learning model learns from, like the worked problems a student practices on, kept separate from the examples later used to check how well it learned., adversarial robustnessadversarial robustness, n.: How well an AI model keeps giving correct answers when someone deliberately tweaks its inputs to trip it up., behavioral cloningbehavioral cloning, n.: The simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation., causal inferencecausal inference, n.: Working out whether one thing actually causes another, rather than merely occurring alongside it — for example, whether a new feature made users stay longer or just happened at the same time., concept driftconcept drift, n.: When the real-world link between what a model sees and the right answer changes over time — as when spammers change tactics — so lessons the model learned no longer hold., data driftdata drift, n.: When the information flowing into an AI model in real use gradually stops resembling the data it was trained on — new customers, new slang, new cameras — making its predictions less reliable., domain adaptationdomain adaptation, n.: Adjusting a model trained on one kind of data, such as daytime photos or one hospital's records, so that it still works on a different but related kind, such as night photos or another hospital's., empirical risk minimizationempirical risk minimization, n.: The basic recipe of most machine learning: from the models on offer, pick the one with the smallest errors on the training examples, trusting that score as a guide to how it will do on new data., feedback loopfeedback loop, n.: A cycle in which what a system produces comes back to shape its own future input, like a microphone howling near its loudspeaker. In AI, a model's predictions can shape the very data it later learns from., generalizationgeneralization, n.: How well a model copes with new examples it has never seen, rather than just memorizing its training data, like a student who understands the method, not just the answers., generalization gapgeneralization gap, n.: The difference between how well a model does on the examples it trained on and how well it does on new ones. A big gap means it memorized rather than learned., goal misgeneralizationgoal misgeneralization, n.: When an AI learns a goal that matched the right behavior during training but turns out to be the wrong goal in new situations — while its skills still work fine., i.i.d.i.i.d., abbr.: Short for 'independent and identically distributed': every data point is produced the same way, and none influences any other — like repeated rolls of one fair die., imitation learningimitation learning, n.: Teaching an AI by showing it examples of an expert doing a task, so it learns to copy the behavior instead of discovering it through trial and error., Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right., model monitoringmodel monitoring, n.: Continuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm., out-of-distributionout-of-distribution, adj.: Describes inputs that differ from anything an AI system saw in training — like a model trained on daytime photos facing a night scene — where its answers become less reliable., probability distributionprobability distribution, n.: A complete description of how likely each possible outcome is — for a fair die, one-sixth for each face. A language model produces one of these over its whole vocabulary at every step., robot learningrobot learning, n.: Teaching robots skills from data and experience, such as watching demonstrations, practicing by trial and error or training in simulation, instead of programming every movement by hand., robustnessrobustness, n.: An AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it., sampling biassampling bias, n.: A distortion that arises when the data collected over-represents some kinds of people or situations and under-represents others, so conclusions drawn from it don't hold for everyone.