Mathematics & StatisticsMachine LearningInterpretability & Steering

causal inference

nounAlso: causal analysis, causality, causal reasoning, causal effect, potential outcomes, Rubin causal model, average treatment effect, ATE

In plain wordsWorking out whether one thing actually causes another, rather than merely occurring alongside it — for example, whether a new feature made users stay longer or just happened at the same time.

The use of data plus assumptions to estimate cause-and-effect relationships: what would happen to an outcome if a variable were changed by intervention, as opposed to how variables are associated when passively observed. Randomized experiments, including A/B testsA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better., identify causal effects by design; with observational data, effects can be estimated only under explicit assumptions about confoundingconfounding variable, n.: A hidden third factor that influences two things at once and makes them look linked. Ice-cream sales and drownings both rise in hot weather; neither causes the other., using matching, propensity-score weighting, instrumental variables, regression discontinuity or difference-in-differences. Two formal frameworks dominate and are largely equivalent: potential outcomes (Neyman–Rubin), which defines effects as contrasts between counterfactualcounterfactual, n.: A statement about what would have happened if something had been different — 'had I left earlier, I would have caught the train'. AI uses counterfactuals to explain decisions and to measure causes. outcomes, and Judea Pearl's structural causal modelsstructural causal model, n.: A precise description of what causes what in a system — a diagram of arrows plus equations — that predicts what would happen if you intervened, not just what tends to occur together. with causal graphs and the do-calculusdo-calculus, n.: A set of mathematical rules, devised by Judea Pearl, for working out the effect of deliberately changing something using only observational data plus a diagram of what causes what.. In AI, causal inference informs robustness to spurious correlationsspurious correlation, n.: A link in data that fools you: two things move together, yet neither drives the other, because a hidden shared cause, the way the data were collected or luck produced the pattern. AI models readily latch onto such links. and distribution shiftdistribution shift, n.: When the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested., counterfactual fairnesscounterfactual fairness, n.: A fairness test that asks: had this person been of another race or sex, with everything that would have followed from that, would the system have decided the same way?, off-policy evaluation in reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time., and mechanistic interpretabilitymechanistic interpretability, n.: A branch of AI research that tries to reverse-engineer a neural network — working out, piece by piece, which internal parts represent which concepts and how they combine to produce behavior., whose activation patchingactivation patching, n.: An experiment that copies part of a model's internal state from its run on one input into its run on another, to see which internal parts actually cause a change in the answer. and causal abstractioncausal abstraction, n.: A mathematical way to check whether a simple, human-readable recipe really describes what a neural network does inside, by testing whether matching internal parts behave the same way when swapped. are interventions on a network's internals. Whether language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. reason causally is an open research question.
The dashboard shows a correlation; we need causal inference, or a proper experiment, before crediting the new model.

\mathrm{ATE} = \mathbb{E}\big[Y \mid do(X=1)\big] - \mathbb{E}\big[Y \mid do(X=0)\big]

Origin

Sewall Wright's path analysis (1921) introduced causal diagrams, Jerzy Neyman's 1923 potential-outcomes notation and Ronald Fisher's randomized experiments (1925–1935) laid statistical foundations. Donald Rubin extended potential outcomes to observational studies from 1974; Peter Spirtes, Clark Glymour and Richard Scheines pioneered causal discovery (1993); and Judea Pearl's graphical approach culminated in Causality (2000).

Sources

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5).
Pearl, J. (2000; 2nd ed. 2009). Causality: Models, Reasoning, and Inference. Cambridge University Press.

Mentioned in

correlationcorrelation, n.: A measure of how closely two quantities move together, from −1 (one rises as the other falls) through 0 (no straight-line link) to +1 (they rise together). It does not show that one causes the other., Bayesian networkBayesian network, n.: A diagram of causes and effects with probabilities attached — say, how rain and a sprinkler can each make grass wet — that lets a computer update its beliefs about hidden causes when it sees evidence., causal maskcausal mask, n.: A rule inside text-generating AI that stops each word from seeing the words after it, so the model learns to predict what comes next without being able to cheat., confounding variableconfounding variable, n.: A hidden third factor that influences two things at once and makes them look linked. Ice-cream sales and drownings both rise in hot weather; neither causes the other., counterfactualcounterfactual, n.: A statement about what would have happened if something had been different — 'had I left earlier, I would have caught the train'. AI uses counterfactuals to explain decisions and to measure causes., data sciencedata science, n.: The practice of collecting, cleaning, analyzing and interpreting data, using statistics, programming and machine learning, to answer questions and guide decisions., Judea PearlJudea Pearl, prop. n.: An Israeli-American computer scientist who gave AI the mathematics for reasoning under uncertainty and, later, for telling cause and effect apart from mere correlation., linear regressionlinear regression, n.: Fitting a straight line (or flat surface) through data points to predict one number from others — say, a house's price from its floor area — choosing the line that keeps prediction errors smallest., predictionprediction, n.: A model's output for a given input: its best guess at an unknown answer. In AI it need not be about the future; labeling a photo 'cat' counts as a prediction., spurious correlationspurious correlation, n.: A link in data that fools you: two things move together, yet neither drives the other, because a hidden shared cause, the way the data were collected or luck produced the pattern. AI models readily latch onto such links., causal abstractioncausal abstraction, n.: A mathematical way to check whether a simple, human-readable recipe really describes what a neural network does inside, by testing whether matching internal parts behave the same way when swapped., causal mediation analysiscausal mediation analysis, n.: A statistical method for splitting a cause's effect into the part that flows through some go-between and the part that does not; in AI it is used to find which internal parts carry a behavior., counterfactual fairnesscounterfactual fairness, n.: A fairness test that asks: had this person been of another race or sex, with everything that would have followed from that, would the system have decided the same way?, do-calculusdo-calculus, n.: A set of mathematical rules, devised by Judea Pearl, for working out the effect of deliberately changing something using only observational data plus a diagram of what causes what., Simpson's paradoxSimpson's paradox, n.: A statistics trap in which combining groups reverses what each group shows: a treatment can do better than another for men and for women separately, yet worse when everyone is counted together., structural causal modelstructural causal model, n.: A precise description of what causes what in a system — a diagram of arrows plus equations — that predicts what would happen if you intervened, not just what tends to occur together., uplift modelinguplift modeling, n.: Predicting not who will buy, recover or respond, but whose behavior will actually change because of an action such as a discount or a treatment, so that it is offered only where it makes a difference.

causal inference

Origin

Related

Sources

Mentioned in