Explainer
QLoRA
Train a LoRA adapter while storing the unchanged base weights in 4-bit form.
QLoRA saves more memory by storing the frozen base model in 4-bit form while training a LoRA adapter. Bit width is the space used to represent each number; a bit holds a 0 or a 1, and eight bits make a byte. Quantization represents weights with fewer bits. The adapter and training calculations use higher precision, so the whole process is not simply 4-bit. Its authors fine-tuned a 65 billion parameter model on one GPU with 48 GB of memory while matching 16-bit fine-tuning on their tested tasks (Dettmers et al. 2023). That is an attributed experiment, not a promise that any 65-billion-parameter model or training job fits. Training a 16-bit LoRA, merging it, then quantizing the merged model to 8 bits for serving is not QLoRA. QLoRA trains against a base stored in 4-bit form. Quantizing after the merge is inference compression.
Illustration: 7 billion weights at 4 bits (half a byte each) use about 3.5 GB for the weight values, compared with 14 GB at 16 bits. Both estimates exclude adapters, quantization metadata, optimizer records and working memory. The memory guide shows why the weight file size is only the beginning.
Going deeper
QLoRA combines a 4-bit format designed for bell-shaped weight distributions (NF4), double quantization of the scaling constants, and paged optimizers that spill to CPU memory during spikes. As of 28 October 2026, Unsloth's guide says QLoRA uses about 4 times less memory than 16-bit LoRA and is slightly slower and marginally less accurate; that is a vendor figure (unsloth.ai).
In the long read: Full fine-tuning, LoRA and QLoRA. Every related term is in Fine-tuning, every concept.