Rashomon effect
In plain wordsThe common situation in which many quite different models fit the same data almost equally well — like witnesses telling different but equally believable stories about the same event.
The phenomenon that a learning problem often admits many models with nearly equal accuracy that differ substantially in structure, in the features they rely on and in their individual predictions. The set of all models within a small tolerance of the best achievable loss is called the Rashomon set. The effect has several consequences: an explanation of any one model, such as its feature importances, may not describe the data, since equally good models rely on different features; individuals can receive conflicting predictions from equally accurate models (predictive multiplicity); and when the Rashomon set is large it is likely to contain a simple, interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed. as accurate as a black box — an argument Cynthia Rudin and colleagues use for preferring interpretable models. Measures such as model class reliance report the range of a feature's importance across the whole set.
Because of the Rashomon effect, a sparse scoring rule matched the deep model's accuracy to within half a percentage point.
Origin
Named by Leo Breiman in 'Statistical Modeling: The Two Cultures' (Statistical Science, 2001), after Akira Kurosawa's 1950 film Rashomon, in which witnesses give contradictory accounts of one crime. The Rashomon set was formalized by Aaron Fisher, Cynthia Rudin and Francesca Dominici and by Lesia Semenova, Rudin and Ronald Parr (both 2019).
Usage note
Related to underspecification (Alexander D'Amour and colleagues, 2020): models that look equally good in testing can behave very differently after deployment.
Related
- interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.
- feature importancefeature importance, n.: A score for how much each input — age, income, location — matters to a model's predictions. It helps people understand, debug and simplify models, but can mislead if read as cause and effect.
- explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application.
- overfittingoverfitting, n.: When a model learns its training examples too well — quirks and noise included — so it does great on data it has seen but poorly on anything new, like a student who memorizes answers without understanding.
- Leo BreimanLeo Breiman, prop. n.: An American statistician who invented random forests and bagging, decision-tree methods still in wide use, and urged statisticians to judge models by how well they predict.
Sources
- Breiman, L. (2001). Statistical Modeling: The Two Cultures. Statistical Science 16(3), 199–231.
- Fisher, A., Rudin, C. & Dominici, F. (2019). All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously. Journal of Machine Learning Research 20(177).
Mentioned in
interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.