Bayesian information criterion

A criterion for model selectionmodel selection, n.: Choosing, from several candidate models or settings, the one most likely to work best on new data, rather than the one that merely fits the existing data best. derived as a large-sample approximation to the Bayesian model evidence (log marginal likelihood). For a model with k parameters fitted to n observations with maximized likelihoodlikelihood, n.: A score for a possible explanation of some data: how probable that explanation, or that setting of a model, would have made the data actually seen. L̂, BIC = k ln n − 2 ln L̂; the model with the lowest BIC is preferred, and BIC differences approximate log Bayes factors. Its penalty per parameter grows with sample size (ln n, against 2 for the Akaike information criterionAkaike information criterion, n.: A score for comparing statistical models that rewards fitting the data well but charges a penalty for every extra adjustable number, to avoid picking needlessly complicated models. Lower is better.), so BIC favors simpler models than AIC once n exceeds about 7. It is consistent: if the true model is among the candidates, BIC picks it with probability approaching one as data grow, whereas AIC aims at the best predictor. BIC is widely used to choose the number of components of a Gaussian mixture modelGaussian mixture model, n.: A model that explains data as coming from several overlapping bell-shaped clouds, working out where each cloud sits, how spread out it is, and how likely each point is to belong to it., the orders of ARIMAARIMA, n.: A classic statistical recipe for forecasting a series of numbers over time: remove any steady trend, then predict the next value from recent values and recent prediction errors. models and the number of clusters or factors. Like AIC, it requires well-defined parameter counts and likelihoods and can mislead for singular or heavily overparameterized models such as neural networks.

BIC bottomed out at five mixture components, so we stopped there.