domain adaptation

A form of transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data. in which a model trained on labeled data from a source domain is adapted to perform well on a target domain whose data distribution differs while the task stays the same: from synthetic to real images, from one hospital's scanners to another's, or from product reviews to social-media posts. In unsupervised domain adaptation the target data are unlabeled; semi-supervised and supervised variants have some or many target labels. Approaches include reweighting source examples by estimated density ratios to correct covariate shiftcovariate shift, n.: A kind of mismatch where the inputs an AI system sees change — say, photos from a new camera — while the right answer for any given input stays the same.; learning domain-invariant features, for instance by adversarial training against a domain classifier (gradient reversal) or by matching feature statistics across domains; self-training with pseudo-labels on target data; and re-estimating normalization statistics or fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. on target data. Theory by Shai Ben-David and colleagues bounds target error by source error plus a measure of divergence between the domains, explaining when adaptation can succeed. Domain generalization is the harder setting in which no target data are available during training.

Domain adaptation with a gradient-reversal layer recovered most of the accuracy lost moving from synthetic to real street scenes.