active parameters
In plain wordsThe part of a model's internal numbers actually used to process each word. Some big models use only a small slice of themselves at a time, making them cheaper to run than their size suggests.
The number of a model's parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. that take part in computing each tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens., as opposed to its total parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them.. In a dense transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it. the two are equal; in a mixture of expertsmixture of experts, n.: A way to build very large AI models cheaply: the model contains many specialist sub-networks, and for each word only a few are switched on, picked by a small dispatcher network. model a routerrouter, n.: In a mixture-of-experts model, the small component that decides which specialist sub-networks should handle each word, like a dispatcher assigning jobs to workers. sends each token through only a few expert sub-networks per layer, so the active count can be a small fraction of the total: Mixtral 8x7B uses about 12.9 billion of its 46.7 billion parameters per token, and DeepSeek-V3 about 37 billion of 671 billion. Compute per token, and with it prefillprefill, n.: The first stage of an AI's reply, in which it reads your whole prompt in one go before it starts writing. Long prompts make this stage take longer. speed and much of inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text., scales with active parameters (roughly two floating-point operations per active parameter per token). Memory capacity, however, depends on the total, since every expert must be stored, and at large batch sizes most experts are read at every decoding step. Releases therefore quote both, sometimes in the name, as in 'A3B' for three billion active.
It is a 30-billion-parameter model with only 3 billion active, so it decodes about as fast as a small dense model.
Origin
The distinction dates to sparsely gated mixture-of-experts layers (Noam Shazeer and colleagues, 2017) and the Switch Transformer (2021), which decoupled parameter count from compute. 'Active parameters' became everyday vocabulary with Mistral AI's Mixtral 8x7B (December 2023) and the open mixture-of-experts models that followed, such as DeepSeek-V3 (December 2024).
Usage note
Memory-limited users care about total parameters, because the whole model must fit; compute-limited services care about active ones. Comparing a mixture-of-experts model with a dense model by either number alone misleads.
Related
- mixture of expertsmixture of experts, n.: A way to build very large AI models cheaply: the model contains many specialist sub-networks, and for each word only a few are switched on, picked by a small dispatcher network.
- parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them.
- routerrouter, n.: In a mixture-of-experts model, the small component that decides which specialist sub-networks should handle each word, like a dispatcher assigning jobs to workers.
- expert parallelismexpert parallelism, n.: A way of running a 'mixture of experts' model — one made of many specialist sub-networks — by giving each chip its own set of specialists and sending each piece of text to whichever chip holds the specialists it needs.
- inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text.
- sparsitysparsity, n.: The property of being mostly zeros or mostly switched off. AI systems exploit it by skipping the empty parts, so they need less memory and computation.
- parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them.
Mentioned in
inference costinference cost, n.: What it costs to use an AI model after it has been built: the computing power, electricity and hardware needed every time it answers, usually billed per chunk of text., mixture of expertsmixture of experts, n.: A way to build very large AI models cheaply: the model contains many specialist sub-networks, and for each word only a few are switched on, picked by a small dispatcher network., parameter countparameter count, n.: How many adjustable numbers an AI model contains — the usual measure of its size. Modern language models have billions to trillions of them., sparsitysparsity, n.: The property of being mostly zeros or mostly switched off. AI systems exploit it by skipping the empty parts, so they need less memory and computation., routerrouter, n.: In a mixture-of-experts model, the small component that decides which specialist sub-networks should handle each word, like a dispatcher assigning jobs to workers.