FP4
In plain wordsA tiny 4-bit number format that can represent only sixteen different values. With clever scaling tricks it lets the newest AI chips run, and even train, models faster and with less memory.
A 4-bit floating-point formatfloating-point format, n.: A way of storing numbers in a computer like scientific notation — a few digits plus a power of two — trading accuracy for range. AI uses ever-smaller formats to compute faster., normally E2M1 (1 sign, 2 exponent and 1 significand bit), whose sixteen codes represent 0, ±0.5, ±1, ±1.5, ±2, ±3, ±4 and ±6 (with a signed zero). Because so few values are available, FP4 is used only with block scaling, in which small groups of values share a higher-precision scale factor: MXFP4 uses 32-element blocks with a power-of-two scale (from the microscaling formatsmicroscaling formats, n.: Number formats that store small groups of numbers with one shared scale factor, so each number can use very few bits while the group as a whole stays accurate. specification), and Nvidia's NVFP4 uses 16-element blocks with an FP8FP8, abbr.: An 8-bit number format that uses a quarter of the memory of the standard 32-bit format, letting new AI chips train and run models roughly twice as fast as with 16 bits. scale plus a per-tensor scale. Hardware support arrived with Nvidia's Blackwell GPUs (2024) and AMD's MI350 series (2025), roughly doubling throughput over FP8. FP4 is used mainly for inferenceinference, n.: Drawing a conclusion from evidence. In today's AI it most often means running an already-trained model to get an answer, as opposed to training it. — OpenAI released its gpt-oss open-weight models (August 2025) with mixture-of-experts weights in MXFP4 — and research has demonstrated large-scale pretraining in NVFP4.
Origin
Four-bit integer and floating-point quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy. of weights was studied from the late 2010s; FP4 became a standard hardware format with the OCP microscaling specification (September 2023) and Nvidia's Blackwell architecture (announced March 2024). Nvidia detailed its NVFP4 variant in a technical blog post in June 2025.
Usage note
Not the same as INT4, an integer format with evenly spaced values; FP4's values cluster near zero, which better matches the bell-shaped distribution of neural-network weights.
Related
- FP8FP8, abbr.: An 8-bit number format that uses a quarter of the memory of the standard 32-bit format, letting new AI chips train and run models roughly twice as fast as with 16 bits.
- microscaling formatsmicroscaling formats, n.: Number formats that store small groups of numbers with one shared scale factor, so each number can use very few bits while the group as a whole stays accurate.
- quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.
- floating-point formatfloating-point format, n.: A way of storing numbers in a computer like scientific notation — a few digits plus a power of two — trading accuracy for range. AI uses ever-smaller formats to compute faster.
- INT8INT8, abbr.: Storing numbers as whole numbers from −128 to 127 using 8 bits. AI models converted to this format run faster and need a quarter of the memory of 32-bit versions.
- post-training quantizationpost-training quantization, n.: Shrinking an already-trained AI model by storing its numbers in fewer bits, without training it again; a small sample of example inputs is usually enough to fine-tune the rounding.
Sources
- NVIDIA (2025). Introducing NVFP4 for Efficient and Accurate Low-Precision Inference. NVIDIA Technical Blog. developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference
Mentioned in
floating-point formatfloating-point format, n.: A way of storing numbers in a computer like scientific notation — a few digits plus a power of two — trading accuracy for range. AI uses ever-smaller formats to compute faster., FP8FP8, abbr.: An 8-bit number format that uses a quarter of the memory of the standard 32-bit format, letting new AI chips train and run models roughly twice as fast as with 16 bits., gpt-ossgpt-oss, prop. n.: Two free-to-download reasoning models released by OpenAI in August 2025 — its first downloadable language models since 2019 — that anyone may run, modify and use commercially on their own hardware., INT8INT8, abbr.: Storing numbers as whole numbers from −128 to 127 using 8 bits. AI models converted to this format run faster and need a quarter of the memory of 32-bit versions., quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy., tensor coretensor core, n.: A small, specialized part of an AI chip that does one job — multiplying small blocks of numbers together — far faster than ordinary circuits can., Huang's lawHuang's law, n.: A claim, named after Nvidia's chief executive, that AI chip performance improves much faster than Moore's law because chips, software and number formats all improve together., microscaling formatsmicroscaling formats, n.: Number formats that store small groups of numbers with one shared scale factor, so each number can use very few bits while the group as a whole stays accurate., TensorRTTensorRT, prop. n.: NVIDIA's software for making a trained neural network run as fast as possible on NVIDIA graphics chips, by merging and simplifying its calculations and testing which way of doing each one is quickest., TensorRT-LLMTensorRT-LLM, prop. n.: NVIDIA's free software library for running large language models as fast as possible on NVIDIA's own graphics chips, used by many companies that host AI models.