Neel Nanda

British mechanistic interpretabilitymechanistic interpretability, n.: A branch of AI research that tries to reverse-engineer a neural network — working out, piece by piece, which internal parts represent which concepts and how they combine to produce behavior. researcher who leads the mechanistic interpretability team at Google DeepMindGoogle DeepMind, prop. n.: Google's main artificial intelligence laboratory, formed in 2023 by joining DeepMind and Google Brain. It builds the Gemini models and scientific AI systems such as AlphaFold.. After working on Anthropic's early transformer-circuits papers (2021–2022), including the study of induction headsinduction head, n.: A pattern-completing mechanism inside language models: having seen 'Harry Potter' once, it spots the next 'Harry' and predicts 'Potter' — a simple trick behind much learning from examples in a prompt., he created TransformerLensTransformerLens, prop. n.: A free software library that lets researchers open up language models and record, inspect or alter what happens inside them at every step while the model runs. (2022), an open-source library for inspecting and intervening on the internals of transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it. language models that became a standard research tool. His reverse-engineering of a small transformer that had grokkedgrokking, n.: When a model that has merely memorized its training examples suddenly, long afterward, figures out the underlying rule and starts getting new examples right. modular addition showed that it had learned an algorithm based on trigonometric identities and supplied progress measures that track the hidden change (2023). He popularized activation patchingactivation patching, n.: An experiment that copies part of a model's internal state from its run on one input into its run on another, to see which internal parts actually cause a change in the answer., proposed attribution patchingattribution patching, n.: A fast shortcut that estimates the results of thousands of patching experiments at once, using the slope of the model's output instead of rerunning the model for every part. (2023), and wrote widely used tutorials and lists of open problems that, with his mentoring, drew many newcomers into the field. At Google DeepMind he was senior author of Gemma ScopeGemma Scope, prop. n.: A free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves. (2024), open sparse autoencoderssparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. for Gemma 2 models; in 2025 his team reported disappointing results for sparse autoencoders on practical tasks and shifted toward more pragmatic methods.