Fisher information

The expected valueexpected value, n.: The long-run average you would get if a chancy situation could be repeated many times — 3.5 for a fair die, even though no single roll shows 3.5. of the squared gradientgradient, n.: A list of slopes showing which way, and how steeply, a quantity rises as each of its inputs changes. Training nudges an AI's settings in the opposite direction, downhill. of the log-likelihoodlog-likelihood, n.: The logarithm of how probable a model finds the observed data. Taking logs turns a product of many tiny probabilities into a manageable sum; training usually minimizes its negative. with respect to a model's parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. (the 'score'), or equivalently, under regularity conditions, the negative expected HessianHessian, n.: A table of 'slopes of slopes' that describes how a surface curves: whether a point is the bottom of a bowl, the top of a hill or a mountain pass. of the log-likelihood; with many parameters it is a positive semidefinite matrix. It measures how sensitive the data distribution is to each parameter: the Cramér–Rao bound says no unbiased estimator can have variance below its inverse, and maximum likelihood estimationmaximum likelihood estimation, n.: Tuning a model's settings until the real data it learns from looks as likely, and so as unsurprising, to the model as possible. Most AI training, including the pretraining of chatbots, is a version of this. attains this asymptotically. Geometrically it is the local curvature of the Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right. between nearby models, the metric of information geometry. In deep learning it appears in the natural gradient (preconditioning updates by the inverse Fisher, approximated by K-FAC and related second-order optimizerssecond-order optimization, n.: Training methods that use not only the slope of the error but also how that slope is changing — its curvature — to choose better-sized steps. They can need far fewer steps, but each costs more.), in elastic weight consolidation against catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on., in Laplace approximations of the posteriorposterior, n.: What you believe after weighing new evidence against what you believed before. It is the updated belief that Bayes' theorem produces., and in parameter-importance scores for pruningpruning, n.: Cutting away the parts of an AI model that contribute least, as a gardener trims branches, so the model becomes smaller or faster while still working nearly as well. and model mergingmodel merging, n.: Combining two or more trained AI models into one by blending their internal numbers directly, so the result shares their skills without any further training..

We weighted the merge by each parameter's diagonal Fisher information on its source task.