QLoRA
In plain wordsA memory-saving version of LoRA that squeezes the big model into a compact 4-bit form while training small add-ons, letting very large models be customized on a single graphics card.
A fine-tuning method that pairs low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves. with a frozen base model compressed to 4-bit precision (quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.): the 4-bit weights are expanded to 16-bit on the fly for each forward and backward computation, and only the LoRA adapters, held in higher precision, receive updates. Three techniques made this work without measurable quality loss in the authors' tests: 4-bit NormalFloat (NF4), a data type whose levels are spaced for normally distributed weights; double quantization, which also quantizes the per-block scaling constants; and paged optimizers, which spill optimizer state into CPU memory when GPU memory briefly runs short. QLoRA brought fine-tuning of a 65-billion-parameter model within a single 48 GB GPU, and its Guanaco models reached 99.3 percent of ChatGPT's score on the Vicuna benchmark after 24 hours of training. It made fine-tuning large open-weight models practical for individuals and small laboratories.
With QLoRA we fine-tuned the 70B model on one 80 GB card.
Origin
Introduced by Tim Dettmers, Artidoro Pagnoni, Ari Holtzman and Luke Zettlemoyer of the University of Washington in May 2023 (NeurIPS 2023), building on Dettmers's bitsandbytes quantization library.
Usage note
The Vicuna-benchmark comparison relied on automated (GPT-4) and human judgments of chat quality. Adapters trained with QLoRA can afterwards be merged into a 16-bit copy of the base model.
Related
- low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves.
- quantizationquantization, n.: Storing an AI model's numbers with fewer digits, like rounding prices to the nearest dollar, so the model takes less memory and runs faster, usually with only a small loss of accuracy.
- parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money.
- fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
- GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
- open-weight modelopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private.
- post-training quantizationpost-training quantization, n.: Shrinking an already-trained AI model by storing its numbers in fewer bits, without training it again; a small sample of example inputs is usually enough to fine-tune the rounding.
- full fine-tuningfull fine-tuning, n.: Fine-tuning in which every one of a model's internal numbers is allowed to change — the most flexible option, but also the most expensive in memory and storage.
Sources
- Dettmers, T., Pagnoni, A., Holtzman, A. & Zettlemoyer, L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023. arXiv:2305.14314. arxiv.org/abs/2305.14314
Mentioned in
low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves., parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money., LLM.int8()LLM.int8(), prop. n.: An early, widely used method for running very large language models with 8-bit numbers, roughly halving their memory needs, by keeping a few unusually large values at higher precision.