AI Dojo Open on the typewriterThe typewriter

Explainer

Adapters, and merging them

1 min read
Assumes: you know what LoRA is.
First published 29 October 2026. Last edited 16 November 2026.

Keep LoRA corrections separate to share a base, or merge one into the weights.

If you trained a LoRA adapter, you have two ways to serve it. Merging adds the correction into the base weights, leaving one model with no separate adapter calculation (Hu et al. 2021). Keeping the adapters separate lets a compatible server share one base model across variants. Choose for your workload and test the speed, rather than treating either arrangement as always faster. An adapter needs the base model it was trained for because it stores a correction to those weights. Its configuration records the base model name and, when set, revision (PEFT). A changed base model requires retraining or validating a new adapter; do not assume the old correction transfers.

Going deeper

S-LoRA reports serving thousands of adapters on a single GPU with a small overhead (Sheng et al. 2023, arxiv.org). In the PEFT library, merging is not in place, and on a quantized base unmerging is not exact (huggingface.co). Older bottleneck adapters add small layers inside the model, which adds latency at inference (Houlsby et al. 2019, arxiv.org). Re-run your tests on the merged artefact, since that is what users get. If you then quantize, merge first and test the quantized file, because quantization can shift quality.

In the long read: Shipping it and keeping it working. Every related term is in Fine-tuning, every concept.