MLX
In plain wordsA free Apple software toolkit for running and training AI models efficiently on Mac computers with Apple's own chips, popular for running language models locally on a laptop.
Apple's open-source array framework for machine learning on computers with Apple's own chips (Apple silicon), created by the company's machine-learning research group. Programmers who know NumPyNumPy, prop. n.: The basic free Python library for working quickly with large grids of numbers; nearly every data-science and machine-learning tool in Python is built on it or copies its style. and PyTorchPyTorch, prop. n.: The most widely used free software toolkit for building and training neural networks, especially in research; programmers write ordinary Python and it handles the calculus and the graphics chips. find its Python interface and its neural-network and optimizer packages familiar, and Swift, C and C++ bindings are also provided. Three design choices set it apart: the CPU and GPU share one pool of memory, so an array never has to be copied to move between them; computation is deferred until a result is actually requested; and, as in JAXJAX, prop. n.: A free Google library that lets researchers write mathematics in familiar Python and automatically get derivatives and very fast code for graphics chips and Google's AI chips., functions can be transformed to compute gradients (automatic differentiationautomatic differentiation, n.: A technique that lets a computer work out exactly how a program's output would change if its inputs changed slightly, by tracking every arithmetic step. It lets AI software compute training updates automatically.), to batch automatically or to compile into optimized graphs. GPU work runs through Apple's Metal interface. The companion package mlx-lm and community model conversions made MLX a common way to run and fine-tunefine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. quantizedquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy. open-weight language models on Macs, whose large unified memory lets them hold models too big for typical consumer graphics cards.
The 4-bit MLX build of the model runs at about 40 tokens per second on my laptop.
Origin
Released as open source by Apple in December 2023, created by machine-learning researchers including Awni Hannun, Jagrit Digani, Angelos Katharopoulos and Ronan Collobert; Apple later promoted it to developers at its 2025 Worldwide Developers Conference.
Usage note
MLX is a framework, not a model format: models for it are usually distributed as safetensorssafetensors, n.: A file format for storing an AI model's learned numbers that cannot hide harmful programs and loads very quickly, replacing older formats that could run code when opened. files on Hugging FaceHugging Face, prop. n.: A company whose website is the main public library for sharing AI models and datasets — often called the GitHub of machine learning — and which makes widely used free AI software.. Contrast Core MLCore ML, prop. n.: Apple's software framework that lets apps on iPhones, iPads and Macs run AI models directly on the device, using its own chips instead of sending data to the cloud., Apple's runtime for shipping models inside apps.
Related
- on-device AIon-device AI, n.: AI that runs directly on your own phone, laptop, car or headset rather than on a company's distant servers, so it can work offline, respond instantly and keep your data on the device.
- Core MLCore ML, prop. n.: Apple's software framework that lets apps on iPhones, iPads and Macs run AI models directly on the device, using its own chips instead of sending data to the cloud.
- llama.cppllama.cpp, prop. n.: A free, open-source program that runs AI language models on ordinary computers and phones, without needing a powerful graphics card, by using compact low-precision versions of the models.
- quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.
- NumPyNumPy, prop. n.: The basic free Python library for working quickly with large grids of numbers; nearly every data-science and machine-learning tool in Python is built on it or copies its style.
- JAXJAX, prop. n.: A free Google library that lets researchers write mathematics in familiar Python and automatically get derivatives and very fast code for graphics chips and Google's AI chips.
- Core MLCore ML, prop. n.: Apple's software framework that lets apps on iPhones, iPads and Macs run AI models directly on the device, using its own chips instead of sending data to the cloud.
Mentioned in
Core MLCore ML, prop. n.: Apple's software framework that lets apps on iPhones, iPads and Macs run AI models directly on the device, using its own chips instead of sending data to the cloud.