surrogate model

Explainable AI

A transparent stand-in for an opaque model: a shallow decision treedecision tree, n.: A model that makes a prediction by asking a series of yes-or-no questions about the input, like a flowchart, with each answer leading to the next question until it reaches a final answer., rule list or sparse linear model fitted not to the true labels but to a black-box model's own predictions, so that reading the stand-in gives an approximate account of what the black box does. A global surrogate is fitted to the black box's outputs across a dataset; a local surrogate approximates it only near one input, as in LIMELIME, abbr.: A method that explains one prediction of any AI model by testing slightly altered versions of the input and fitting a very simple model that copies the AI's behavior near that example.. A surrogate explains the black box only as far as it agrees with it, so its fidelity — the share of held-out inputs on which it matches the black box — should be reported with it: a surrogate with 80% fidelity is silent about the decisions on which it disagrees.

A depth-four tree reproduced the neural network's decisions with 93% fidelity, so we used it as a global surrogate.

Optimization and simulation

A cheap approximation of an expensive function — a physical simulation, a laboratory experiment, or a model's validation loss as a function of its hyperparameters — built from a limited number of evaluations and used to decide where to evaluate next or to replace the expensive function. Gaussian processesGaussian process, n.: A way of predicting an unknown curve or surface that gives not just a best guess at each point but an honest error bar, narrow near measured points and wide far from them. are the classic surrogates in Bayesian optimizationBayesian optimization, n.: A smart way to tune settings when each test is expensive: build a statistical guess of how settings affect results, then try whichever setting that guess says is most promising next.; neural-network surrogates, or emulators, accelerate climate and engineering simulations.

Security

A local copy trained to imitate a target model from its input–output behavior, typically by querying the target's API and recording its answers. Attackers use such copies to craft adversarial examplesadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon. that transfer to the target, or keep them as the product of model extractionmodel extraction, n.: Copying an AI model without permission by asking it huge numbers of questions and using the answers to build a look-alike, or to work out hidden details of how it was made.; in this sense 'substitute model' is also common.