AI Dojo Back to the typewriterThe typewriter
The long readLong read

Reference

Fine-tuning, every concept

190 concepts in six stages, 105 for beginners
First published 29 October 2026. Last edited 9 November 2026.

Every term you are likely to meet when you fine-tune a language model, from what a weight is to when to retrain, each in one or two plain sentences. Beginner concepts are where to start; practitioner concepts are the details you need once you train for real. Where the long read What happens when you fine-tune a model explains a concept, the entry links to that part.

Foundations 18

What fine-tuning is, what it is not, and when to use it.

  1. Weights (parameters) Beginner

    The learned numerical settings that shape a model's predictions. Training updates them; an ordinary prompt leaves them unchanged.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2106.09685, huggingface.co / chat_templating

  2. Pretraining and the base model Beginner

    The first, very large training run on general text that produces a base model, which continues text but does not follow instructions well.

    In the long read: Should you fine-tune at all?

    Sources huggingface.co / chat_templating, arXiv 2405.05904, arXiv 2305.11206

  3. Fine-tuning Beginner

    Training an already pretrained model further on new examples so that its weights change and its behaviour shifts toward those examples.

    In the long read: Should you fine-tune at all?

    Sources docs.aws.amazon.com / custom-models, huggingface.co / index

  4. Prompting and system prompts Beginner

    Writing instructions and context into the text sent to the model; a system prompt is a standing instruction placed at the start of a conversation.

    In the long read: Should you fine-tune at all?

    Sources huggingface.co / chat_templating, docs.cloud.google.com / tune-models, docs.mistral.ai / finetuning, platform.claude.com / overview, platform.claude.com / claude-prompting-best-practices

  5. Few-shot examples (in-context learning) Beginner

    Putting a handful of worked examples in the prompt so the model copies the pattern, with no training.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2005.14165, developers.openai.com / model-optimization

  6. Retrieval-augmented generation (RAG) Beginner

    Looking up relevant documents at question time and adding them to the prompt, so the model answers from current sources without retraining.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2005.11401, arXiv 2403.10131, arXiv 2403.01432

  7. Tool use (function calling) Beginner

    Letting the model request calls to functions or APIs your code runs, so it can fetch live data or take actions instead of relying on memory.

    In the long read: Should you fine-tune at all?

    Sources platform.claude.com / overview, docs.cloud.google.com / tune-models

  8. Base, instruct and chat models Beginner

    A base model continues text; an instruct or chat model is a base model further trained to follow instructions and hold conversations.

    In the long read: Should you fine-tune at all?

    Sources huggingface.co / Llama-3.1-8B, arXiv 2203.02155, arXiv 2308.08747, huggingface.co / chat_templating

  9. Open-weight vs closed models Beginner

    Open-weight models let you download the weights and train them yourself; closed models can only be tuned through whatever service their maker or a cloud partner offers.

    In the long read: Shipping it and keeping it working

    Sources huggingface.co / Llama-3.1-8B

  10. Continued pretraining (domain adaptation) Practitioner

    Continuing the original next-word training on a large pile of raw, unlabelled text from your domain so the model gets used to its language.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2004.10964, arXiv 2403.08763, docs.aws.amazon.com / custom-models, arXiv 2312.05934

  11. Supervised fine-tuning (SFT) Beginner

    Training on input and target-response pairs to encourage the demonstrated behaviour, including patterns of tone, length and format.

    In the long read: What a training example looks like

    Sources docs.aws.amazon.com / custom-models, developers.openai.com / model-optimization, docs.cloud.google.com / tune-models, arXiv 2305.11206

  12. Instruction tuning Beginner

    Supervised fine-tuning on many different tasks written as plain-language instructions, so the model learns to follow instructions in general.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2109.01652

  13. Where fine-tuning fits Beginner

    Consistent style, format or behaviour; narrow repeatable tasks; and replacing a large model with a smaller, cheaper, faster one.

    In the long read: Should you fine-tune at all?

    Sources developers.openai.com / model-optimization, docs.mistral.ai / finetuning, claude.com / fine-tune-claude-3-haiku, docs.cloud.google.com / tune-models

  14. Fine-tuning and new knowledge (behaviour versus facts) Beginner

    Fine-tuning can learn some facts, but retrieval is a useful first choice for a changing knowledge base. In one comparison, retrieval beat further training on documents across the tested knowledge-intensive tasks.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2312.05934, arXiv 2403.01432, arXiv 2309.14316, arXiv 2404.00213, arXiv 2005.11401, arXiv 2405.05904, arXiv 2410.03717

  15. New facts in training data can raise hallucination Practitioner

    One controlled study found that unfamiliar facts in training examples were learned more slowly and could increase hallucination in the tested models. This is a risk, not a prediction for every run.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2405.05904, arXiv 2405.05904

  16. What fine-tuning cannot fix Beginner

    It cannot keep facts current, reach live systems, or replace a clear definition of success. The weights keep no record of where a fact came from, though a model can be trained to cite passages placed in its prompt or retrieved for it. Whether training can take a model beyond its base model's ability is disputed: one view holds that reinforcement learning mostly sharpens what the base model can already do, another reports new solutions after prolonged training.

    In the long read: Should you fine-tune at all?

    Sources arXiv 2005.11401, arXiv 2305.15717, platform.claude.com / overview, developers.openai.com / model-optimization, arXiv 2310.11511, arXiv 2402.03300, arXiv 2504.13837, arXiv 2505.24864

  17. Costs and effort Beginner

    Data collection and cleaning, training compute, evaluation, serving the custom model, and re-doing it all when the base model changes.

    In the long read: Shipping it and keeping it working

    Sources docs.aws.amazon.com / custom-models, docs.fireworks.ai / fine-tuning-models, docs.together.ai / fine-tuning-overview, together.ai / pricing

  18. Decision ladder: prompt, then retrieve, then fine-tune Beginner

    Start with prompting and examples, add retrieval for knowledge and tools for live data, and fine-tune only for behaviour, format or cost problems that remain.

    In the long read: Should you fine-tune at all?

    Sources docs.cloud.google.com / tune-models, docs.mistral.ai / finetuning, developers.openai.com / model-optimization

Data 40

What one example looks like, where examples come from, and how to clean and split them.

  1. Training example Beginner

    One demonstration: an input and the exact output you want the model to learn to produce.

    In the long read: What a training example looks like

    Sources developers.openai.com / supervised-fine-tuning, huggingface.co / sft_trainer

  2. Prompt and target (completion) Beginner

    The part of an example the model reads, and the part it should write back.

    In the long read: What a training example looks like

    Sources huggingface.co / dataset_formats, huggingface.co / sft_trainer

  3. Roles (system, user, assistant, tool) Beginner

    Labels that mark who wrote each message in chat training data.

    In the long read: What a training example looks like

    Sources huggingface.co / chat_templating, docs.cloud.google.com / gemini-supervised-tuning-prepare, huggingface.co / dataset_formats

  4. Single-turn and multi-turn examples Beginner

    One question and answer, versus a whole conversation in which every assistant turn is a target.

    In the long read: What a training example looks like

    Sources huggingface.co / dataset_formats, huggingface.co / sft_trainer

  5. JSONL messages format Beginner

    A file with one JSON training example per line, each holding a list of role-tagged messages.

    In the long read: What a training example looks like

    Sources developers.openai.com / supervised-fine-tuning, docs.cloud.google.com / gemini-supervised-tuning-prepare, huggingface.co / sft_trainer

  6. Tool-calling examples Practitioner

    Examples that show a tool call, its result and the final reply, together with the list of available tools.

    Sources huggingface.co / dataset_formats, developers.openai.com / supervised-fine-tuning

  7. Dataset types Practitioner

    Language modelling, prompt-completion and preference data suit different training methods.

    Sources huggingface.co / dataset_formats

  8. Tokens, tokenizers and special tokens Beginner

    A token is the small piece of text a model reads; a tokenizer turns text into token IDs. Special tokens are reserved markers for boundaries such as the end of a turn.

    In the long read: What a training example looks like

    Sources huggingface.co / chat_templating, huggingface.co / sft_trainer, huggingface.co / 4

  9. Chat template Beginner

    The model-specific recipe that turns a list of messages into the exact token string the model reads.

    In the long read: What a training example looks like

    Sources huggingface.co / chat_templating, huggingface.co / sft_trainer

  10. Template mismatch Practitioner

    Training text formatted differently from inference text, which silently lowers quality.

    In the long read: What a training example looks like

    Sources huggingface.co / chat_templating, huggingface.co / sft_trainer

  11. Quality over quantity Beginner

    Correctness and coverage matter as much as count. A small clean set can beat a noisy one, while broader tasks may still require many varied examples.

    In the long read: The data decides what it learns

    Sources arXiv 2305.11206, arXiv 2305.14314, arXiv 2307.08701, docs.cloud.google.com / gemini-supervised-tuning-prepare

  12. Superficial Alignment Hypothesis Practitioner

    The contested idea that fine-tuning mostly teaches style and format while pretraining supplies knowledge and ability.

    In the long read: The data decides what it learns

    Sources arXiv 2305.11206, arXiv 2312.01552, arXiv 2410.03717

  13. How many examples to start with Beginner

    Start with a small, careful pilot and measure the result. If it shows no gain, inspect the task, prompt, examples and evaluation before adding more data; provider minimums are not guarantees.

    In the long read: The data decides what it learns

    Sources developers.openai.com / supervised-fine-tuning, docs.cloud.google.com / gemini-supervised-tuning-prepare, arXiv 2305.11206

  14. Human-written examples Beginner

    Examples written by people who know the task: slow and costly, with the most control over quality.

    In the long read: The data decides what it learns

    Sources arXiv 2305.11206, developers.openai.com / supervised-fine-tuning

  15. Curated logs Beginner

    Records of real past interactions that are selected and cleaned before being used as training data.

    In the long read: The data decides what it learns

    Sources developers.openai.com / supervised-fine-tuning, edpb.europa.eu / edpb-opinion-ai-models-gdpr-principles-support-responsible-ai_en

  16. Synthetic data Beginner

    Training examples generated by a model rather than written by a person.

    In the long read: The data decides what it learns

    Sources crfm.stanford.edu / alpaca, arXiv 2306.11644, arXiv 2307.08701

  17. Distillation (teacher and student) Beginner

    Training a smaller student model to imitate the outputs of a stronger teacher model.

    In the long read: The data decides what it learns

    Sources arXiv 1503.02531, arXiv 2305.15717, docs.aws.amazon.com / custom-models, openai.com / row-terms-of-use, arXiv 2501.12948

  18. Self-Instruct Practitioner

    Using a model to grow a small seed set of instructions into a large dataset, then filtering for quality and diversity.

    Sources arXiv 2212.10560, crfm.stanford.edu / alpaca

  19. Synthetic data risks and model collapse Practitioner

    Copying the generator's errors, and the loss of rare patterns when models train repeatedly on model-made data.

    In the long read: The data decides what it learns

    Sources arXiv 2305.17493, arXiv 2404.01413, arXiv 2305.15717

  20. Terms and licences on model outputs Practitioner

    Provider terms and model licences that limit using outputs to train other, especially competing, models.

    In the long read: The data decides what it learns

    Sources openai.com / row-terms-of-use, openai.com / services-agreement, anthropic.com / commercial-terms, ai.google.dev / terms, dev.meta.ai / license, crfm.stanford.edu / alpaca

  21. Deduplication Beginner

    Removing exact and near-duplicate examples within the training set and across splits.

    In the long read: The data decides what it learns

    Sources arXiv 2107.06499, arXiv 2202.07646, developers.google.com / dividing-datasets

  22. Quality filtering Beginner

    Removing wrong, off-topic, malformed or unsafe examples using rules, people or a grader model.

    In the long read: The data decides what it learns

    Sources arXiv 2307.08701, arXiv 2305.11206

  23. Consistency of labels and format Beginner

    Treating similar inputs the same way so the model learns one clear behaviour.

    In the long read: The data decides what it learns

    Sources arXiv 2103.14749

  24. Diversity and coverage Beginner

    Varied examples that together cover every kind of request the model will meet in real use.

    In the long read: The data decides what it learns

    Sources arXiv 2305.11206, arXiv 2212.10560, arXiv 2305.15717

  25. Balancing Practitioner

    Adjusting the mix so rare but important cases are not drowned out by common ones.

    Sources developers.google.com / imbalanced-datasets

  26. Edge cases and refusals Beginner

    Explicit examples for unusual inputs, for declining harmful requests, and for not refusing safe ones.

    In the long read: The data decides what it learns

    Sources arXiv 2310.03693, arXiv 2308.01263

  27. Provenance Practitioner

    A record kept with each example of where it came from, so problem rows can be traced and removed.

    In the long read: The data decides what it learns

    Sources arXiv 2404.01413, edpb.europa.eu / edpb-opinion-ai-models-gdpr-principles-support-responsible-ai_en

  28. Training, validation and test sets Beginner

    Data the model learns from, data used to compare settings, and data kept back for the final check.

    In the long read: The data decides what it learns

    Sources scikit-learn.org / cross_validation, developers.google.com / dividing-datasets, docs.cloud.google.com / gemini-supervised-tuning-prepare, developers.openai.com / supervised-fine-tuning

  29. Data leakage Practitioner

    Evaluation information reaching training or development decisions, making results look better than reality.

    In the long read: The data decides what it learns

    Sources arXiv 2207.07048, arXiv 2107.06499, scikit-learn.org / cross_validation

  30. Group-aware splits and stable identifiers Practitioner

    Keeping related examples in the same split, with fixed IDs so the split stays the same every time the data is rebuilt.

    In the long read: The data decides what it learns

    Sources scikit-learn.org / cross_validation

  31. Licences and rights on data Beginner

    The terms attached to datasets and documents that decide whether you may train on them.

    Sources crfm.stanford.edu / alpaca

  32. Memorisation risk Beginner

    A model storing and repeating training text word for word, which can leak private data.

    In the long read: The data decides what it learns

    Sources arXiv 2012.07805, arXiv 2202.07646, arXiv 2205.12506

  33. Differentially private fine-tuning Practitioner

    Training with added noise so no single example can strongly shape the model, giving a provable privacy bound.

    Sources arXiv 2110.05679

  34. Data inspection Beginner

    Reading raw, rendered and masked examples by eye to catch problems that metrics miss.

    In the long read: What a training example looks like

    Sources karpathy.github.io / recipe, huggingface.co / chat_templating

  35. Pilot runs Beginner

    Small, cheap fine-tunes that check the pipeline and the data's effect before a full run.

    In the long read: The data decides what it learns

    Sources developers.openai.com / supervised-fine-tuning, karpathy.github.io / recipe, huggingface.co / sft_trainer

  36. Diagnosing data problems Practitioner

    Matching symptoms such as never stopping or poor production scores to their usual data causes.

    Sources huggingface.co / chat_templating, huggingface.co / sft_trainer, scikit-learn.org / cross_validation, arXiv 2405.05904

  37. Loss masking (completion-only or assistant-only loss) Practitioner

    Choosing which tokens count toward the loss, the prediction penalty. Assistant-only training scores the reply while still reading the prompt as context; other setups may score more text.

    In the long read: What a training example looks like

    Sources huggingface.co / sft_trainer, unsloth.ai / lora-hyperparameters-guide, arXiv 2305.14314, arXiv 2405.14394

  38. Sequence length (max_length) and truncation Beginner

    The maximum tokens a training run is configured to allow per example. It can be shorter than the model's own context maximum. The prompt, the answer and the chat-template tokens have to fit inside it together. With truncation on, anything longer is cut; other pipelines reject or filter the example.

    In the long read: What a training example looks like

    Sources huggingface.co / sft_trainer, huggingface.co / reducing_memory_usage, huggingface.co / model_memory_anatomy, docs.cloud.google.com / gemini-supervised-tuning-prepare

  39. Held-out test set discipline Beginner

    The test set is never trained on, never used to tune prompts or pick checkpoints, and is scored the same way before and after.

    In the long read: The data decides what it learns

    Sources scikit-learn.org / cross_validation, arXiv 2311.04850, developers.google.com / dividing-datasets

Training 59

The training loop, its settings, LoRA and other efficient methods, memory and tools.

  1. Next-token prediction Beginner

    The model is trained to give high probability to the token that actually comes next.

    In the long read: What happens during training

    Sources huggingface.co / sft_trainer

  2. Loss (cross-entropy) Beginner

    An average prediction penalty on the example's target tokens. Lower means a better fit under the same data and scoring setup; it is not an error percentage or proof of better task answers.

    In the long read: What happens during training

    Sources huggingface.co / sft_trainer, developers.google.com / glossary

  3. Teacher forcing Practitioner

    During training, every prediction is made from the true previous tokens of the example rather than from the model's own earlier outputs.

    In the long read: What happens during training

    Sources deeplearningbook.org / rnn, arXiv 1506.03099

  4. Exposure bias (training versus generation mismatch) Practitioner

    A model trained only on true previous tokens must read its own output when it generates, so early mistakes can carry forward and errors can accumulate. Scheduled sampling, which gradually feeds the model its own predictions during training, is one proposed fix.

    In the long read: What happens during training

    Sources arXiv 1511.06732, arXiv 1506.03099

  5. Perplexity Practitioner

    Perplexity is the exponential of the average cross-entropy loss; lower is better.

    In the long read: What happens during training

    Sources huggingface.co / perplexity

  6. Gradient and backpropagation Beginner

    A gradient says how to change each weight to reduce the loss; backpropagation computes it.

    In the long read: What happens during training

    Sources docs.pytorch.org / autogradqs_tutorial, huggingface.co / model_memory_anatomy

  7. Optimizer (Adam and AdamW) Beginner

    The optimizer turns gradients into weight updates; AdamW is Adam with decoupled weight decay.

    In the long read: What happens during training

    Sources arXiv 1412.6980, arXiv 1711.05101, huggingface.co / trainer, huggingface.co / model_memory_anatomy

  8. Weight decay Practitioner

    Weight decay gently shrinks weights each step as a mild guard against overfitting.

    In the long read: What happens during training

    Sources huggingface.co / trainer, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673

  9. Learning rate Beginner

    How large each weight update is. Too small can make learning slow; too large can make training unstable or damage useful behaviour. Choose it using validation results.

    In the long read: What happens during training

    Sources developers.google.com / hyperparameters, huggingface.co / trainer, huggingface.co / sft_trainer, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673

  10. Learning-rate schedule (linear, cosine, constant) Practitioner

    A schedule changes the learning rate over training, for example linear or cosine decay.

    In the long read: What happens during training

    Sources huggingface.co / optimizer_schedules, huggingface.co / trainer, unsloth.ai / lora-hyperparameters-guide

  11. Warmup Practitioner

    Warmup ramps the learning rate up from near zero over the first steps.

    In the long read: What happens during training

    Sources arXiv 1706.02677, huggingface.co / trainer, unsloth.ai / lora-hyperparameters-guide

  12. Gradient clipping (max gradient norm) Practitioner

    Gradient clipping caps the size of the gradient so one bad batch cannot wreck the model.

    In the long read: What happens during training

    Sources huggingface.co / trainer, huggingface.co / sft_trainer

  13. Batch size Beginner

    How many examples one device runs in one forward and backward pass. Several of these microbatches can be accumulated before one weight update.

    In the long read: What happens during training

    Sources developers.google.com / hyperparameters, huggingface.co / trainer, learn.microsoft.com / fine-tuning

  14. Step and epoch Beginner

    A step updates the weights once after a batch. An epoch is one full pass over the training data. More epochs revisit the same examples, which raises overfitting and forgetting risk, and do not guarantee better answers on new inputs.

    In the long read: What happens during training

    Sources developers.google.com / hyperparameters, huggingface.co / callback, unsloth.ai / lora-hyperparameters-guide, huggingface.co / trainer

  15. Gradient accumulation and effective batch size Beginner

    Combining gradients from several microbatches, with loss normalization, so one weight update covers more examples. Without that normalization, the update would grow with the number of passes. This saves memory for intermediate results, not for the weights or the optimizer state. On one GPU, microbatch 1 with 8 accumulation steps is an effective batch of 8: comparable to a batch of 8 under the same normalization, not guaranteed identical.

    In the long read: What happens during training

    Sources huggingface.co / grad_accumulation, unsloth.ai / lora-hyperparameters-guide

  16. Full fine-tuning Beginner

    Full fine-tuning updates every weight in the model.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2405.09673, huggingface.co / model_memory_anatomy

  17. Parameter-efficient fine-tuning (PEFT) Beginner

    Training only a small share of the parameters, either a few added ones (as in LoRA) or a chosen subset of the existing ones (BitFit updates only the bias terms), while the rest stay frozen, instead of updating every weight as full fine-tuning does.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources huggingface.co / index, arXiv 2106.09685, docs.together.ai / fine-tuning-overview, arXiv 2405.09673, arXiv 2106.10199

  18. LoRA (Low-Rank Adaptation) Beginner

    LoRA saves training memory by learning a small correction, called an adapter, beside unchanged base weights. It represents the correction using two small matrices, tables of numbers.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2106.09685, arXiv 2405.09673

  19. LoRA rank (r) Beginner

    The shared width of a LoRA adapter's two thin matrices: how many independent directions the correction can use. It does not set how large the change is. A rank of 1 can still be large. A higher rank trains more numbers and uses more memory. It does not by itself mean better answers, and it is not a count of layers or of examples.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2106.09685, arXiv 2305.14314, arXiv 2405.09673, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide

  20. LoRA alpha (scaling) Practitioner

    LoRA alpha scales the size of the correction by alpha divided by rank. Rank itself is not that size.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2106.09685, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673

  21. rsLoRA (rank-stabilised LoRA) Practitioner

    rsLoRA scales by alpha divided by the square root of rank so high ranks still learn.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2312.03732, huggingface.co / lora

  22. Target modules Practitioner

    Target modules are the weight matrices that receive LoRA adapters.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2106.09685, arXiv 2305.14314, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide

  23. LoRA dropout Practitioner

    LoRA dropout randomly drops adapter inputs during training as light regularisation.

    Sources huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673

  24. LoRA initialisation variants (PiSSA, OLoRA, LoftQ) Practitioner

    PiSSA, OLoRA and LoftQ are alternative ways to initialise LoRA adapters.

    Sources huggingface.co / lora

  25. Quantization (in training) Beginner

    Quantization stores numbers in fewer bits to save memory.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2305.14314

  26. QLoRA Beginner

    QLoRA trains a LoRA adapter while storing the frozen base weights in 4-bit form to save memory. The adapter and training calculations use higher precision. Training a 16-bit LoRA and then quantizing the merged model is not QLoRA; that later step is inference compression.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2305.14314, unsloth.ai / lora-hyperparameters-guide

  27. NF4 (4-bit NormalFloat) and double quantization Practitioner

    NF4 is a 4-bit format for normally distributed weights; double quantization also compresses the scaling constants.

    Sources arXiv 2305.14314

  28. Paged optimizers Practitioner

    Paged optimizers spill optimizer state to CPU RAM during GPU memory spikes.

    Sources arXiv 2305.14314

  29. DoRA (Weight-Decomposed Low-Rank Adaptation) Practitioner

    DoRA splits weights into magnitude and direction and uses LoRA for the direction.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2402.09353, huggingface.co / lora

  30. Adapters (bottleneck adapters) Practitioner

    Adapters insert small trainable layers inside a frozen model.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 1902.00751, arXiv 2106.09685

  31. Prefix tuning and prompt tuning (soft prompts) Practitioner

    Prompt tuning learns soft prompt embeddings added only at the model's input; prefix tuning learns trainable prefix vectors inserted at every layer. Both leave the model's own weights frozen.

    Sources arXiv 2101.00190, arXiv 2104.08691, huggingface.co / prompting

  32. Packing and padding-free batching Practitioner

    Packing groups several short examples into the same fixed-length training row to avoid wasted padding, which can split an example across rows. Padding-free batching instead flattens a sampled batch into one sequence without padding, keeping every example complete.

    Sources huggingface.co / reducing_memory_usage, huggingface.co / sft_trainer

  33. Typical starting hyperparameters Practitioner

    Library and vendor docs publish starting hyperparameters to use for a first run.

    In the long read: What happens during training

    Sources huggingface.co / trainer, huggingface.co / sft_trainer, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673, learn.microsoft.com / fine-tuning

  34. Where training memory goes Beginner

    Training memory holds weights, gradients, optimizer states and activations.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources huggingface.co / model_memory_anatomy

  35. Mixed precision (fp16 and bf16) Beginner

    Mixed precision computes in 16-bit (bf16 or fp16) while keeping a 32-bit master copy of weights.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 1710.03740, huggingface.co / mixed_precision_training, huggingface.co / sft_trainer

  36. Gradient checkpointing Beginner

    Gradient checkpointing recomputes some activations to save memory.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 1604.06174, huggingface.co / grad_checkpointing, huggingface.co / reducing_memory_usage

  37. 8-bit optimizers Practitioner

    8-bit optimizers store optimizer states in 8 bits to save memory.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources arXiv 2110.02861, huggingface.co / model_memory_anatomy

  38. CPU offloading Practitioner

    Offloading moves optimizer states or activations from GPU to CPU memory.

    Sources huggingface.co / reducing_memory_usage, deepspeed.ai / zero

  39. Data parallel training (DDP) Beginner

    Data parallel training copies the model to each GPU and averages gradients.

    Sources docs.pytorch.org / ddp

  40. Sharded training (FSDP and DeepSpeed ZeRO) Practitioner

    FSDP and ZeRO shard weights, gradients and optimizer states across GPUs.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources deepspeed.ai / zero, arXiv 1910.02054, docs.pytorch.org / fsdp, github.com / axolotl

  41. Training loss versus validation loss Beginner

    Training loss measures fit to examples used for updates; validation loss measures fit to held-out examples. Falling training loss alone does not establish better answers to new questions.

    In the long read: What happens during training

    Sources huggingface.co / trainer, learn.microsoft.com / fine-tuning, developers.google.com / overfitting

  42. Overfitting Beginner

    Fitting training examples too closely so performance on new inputs suffers. Falling training loss while validation loss rises is a warning; judge task results too.

    In the long read: What happens during training

    Sources developers.google.com / overfitting, unsloth.ai / lora-hyperparameters-guide, arXiv 2203.02155

  43. Underfitting Beginner

    Underfitting is not learning enough to do well even on training data.

    In the long read: What happens during training

    Sources developers.google.com / overfitting, unsloth.ai / lora-hyperparameters-guide

  44. Early stopping Beginner

    Ending a run when a chosen validation measure stops improving. The chosen measure and patience determine when to stop; loss alone need not identify the best task performance.

    In the long read: What happens during training

    Sources huggingface.co / callback, huggingface.co / trainer

  45. Checkpoints and choosing one Beginner

    A saved snapshot of the model during training. Choosing one by its validation score means that score is no longer an unbiased final measure. The test set is.

    In the long read: What happens during training

    Sources developers.google.com / glossary, huggingface.co / trainer, learn.microsoft.com / fine-tuning, huggingface.co / sft_trainer

  46. Catastrophic forgetting Beginner

    Losing skills or knowledge a model had before while training it on something new, so it gets worse at things it used to do well.

    In the long read: What happens during training

    Sources arXiv 1612.00796, arXiv 2308.08747, arXiv 2405.09673, arXiv 2403.08763, github.com / lm-evaluation-harness, arXiv 2211.09110

  47. Mitigation: mixing in general data (replay) Practitioner

    Replay mixes some earlier or general data into training to reduce forgetting.

    In the long read: What happens during training

    Sources arXiv 2205.12393, arXiv 2403.08763

  48. Mitigation: lower learning rate, fewer epochs, LoRA Practitioner

    Lower learning rates, fewer epochs and LoRA tend to reduce forgetting.

    In the long read: What happens during training

    Sources arXiv 2405.09673

  49. Random seed Beginner

    A seed fixes random number generators so runs can be repeated.

    In the long read: What happens during training

    Sources huggingface.co / trainer, docs.pytorch.org / randomness

  50. Determinism and version pinning Practitioner

    Exact results also depend on library versions and hardware, so pin and record them.

    In the long read: What happens during training

    Sources docs.pytorch.org / randomness, huggingface.co / sft_trainer

  51. Config and experiment logging Beginner

    Experiment logging records config, metrics and outputs of every run.

    In the long read: What happens during training

    Sources huggingface.co / callback, github.com / axolotl, learn.microsoft.com / fine-tuning

  52. Hugging Face Transformers (Trainer) Practitioner

    Transformers is the base library for loading and training open models.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources pypi.org / transformers, huggingface.co / trainer

  53. TRL (SFTTrainer) Practitioner

    TRL is Hugging Face's post-training library, including SFTTrainer.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources pypi.org / trl, huggingface.co / sft_trainer

  54. PEFT library Practitioner

    PEFT is Hugging Face's library of LoRA and other parameter-efficient methods.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources pypi.org / peft, huggingface.co / index

  55. Axolotl Practitioner

    Axolotl is a YAML-config-driven fine-tuning framework.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources github.com / axolotl, pypi.org / axolotl

  56. Unsloth Practitioner

    Unsloth is a library and app for fast, low-memory fine-tuning on one machine.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources github.com / unsloth, pypi.org / unsloth

  57. torchtune Practitioner

    torchtune is a PyTorch-native fine-tuning library that is no longer maintained.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources github.com / torchtune, pypi.org / torchtune

  58. LLaMA-Factory Practitioner

    LLaMA-Factory is a unified fine-tuning toolkit with a no-code web UI.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources github.com / LLaMA-Factory, pypi.org / llamafactory

  59. DeepSpeed and PyTorch FSDP Practitioner

    DeepSpeed and PyTorch FSDP implement sharded multi-GPU training.

    In the long read: Full fine-tuning, LoRA and QLoRA

    Sources pypi.org / deepspeed, deepspeed.ai / zero, docs.pytorch.org / fsdp

Evaluation 20

How to tell whether a fine-tune worked, and what it may have broken.

  1. Evaluation first (success criteria and a baseline) Beginner

    Write down what a better model must do and how you will measure it, and measure the current model, before you train anything.

    In the long read: Did it work?

    Sources platform.claude.com / define-success, developers.openai.com / supervised-fine-tuning, developers.openai.com / evaluation-best-practices, developers.openai.com / model-optimization, platform.claude.com / overview

  2. Baselines Beginner

    The scores you compare against: the base model with a good prompt, the model before tuning, and a stronger reference model.

    In the long read: Did it work?

    Sources developers.openai.com / supervised-fine-tuning, platform.claude.com / define-success, arXiv 2404.04475

  3. Before and after on the same items (paired comparison) Practitioner

    Compare two models question by question on the same items, so the difference is measured more precisely.

    In the long read: Did it work?

    Sources arXiv 2411.00640

  4. Exact match Beginner

    Score an answer as right only if it equals the expected answer, usually after light normalisation.

    In the long read: Did it work?

    Sources platform.claude.com / develop-tests

  5. Deterministic checks (format, schema, rules) Beginner

    Code that checks an output against fixed rules, such as valid JSON, required fields, allowed values or length limits.

    In the long read: Did it work?

    Sources json-schema.org / what-is-jsonschema, platform.claude.com / develop-tests

  6. Generation length limit (max_new_tokens) Practitioner

    The cap on how many new tokens a model may write in one answer, set separately from the length of the prompt. An answer that hits the cap is cut off before its end-of-sequence token, so an evaluation should count cut-off answers as failures or report them separately.

    In the long read: Did it work?

    Sources huggingface.co / llm_tutorial

  7. Reference-based metrics (BLEU, ROUGE, BERTScore) Practitioner

    Scores that compare a generated text with one or more human-written reference texts by overlap or similarity.

    In the long read: Did it work?

    Sources aclanthology.org / P02-1040, aclanthology.org / W04-1013, arXiv 1904.09675, platform.claude.com / develop-tests, developers.openai.com / evaluation-best-practices

  8. Model-graded evaluation (LLM-as-a-judge) Beginner

    Use a language model, given a rubric, to grade or compare outputs that are hard to check with code.

    In the long read: Did it work?

    Sources arXiv 2306.05685, platform.claude.com / develop-tests

  9. Judge biases: position, length, self-preference Practitioner

    Model judges can favour the answer shown first, the longer answer, or answers written by themselves.

    In the long read: Did it work?

    Sources arXiv 2306.05685, arXiv 2305.17926, arXiv 2404.04475, arXiv 2404.13076, arXiv 2404.18796, platform.claude.com / develop-tests

  10. Human evaluation Beginner

    People, ideally domain experts with a written rubric, grade or compare model outputs.

    In the long read: Did it work?

    Sources developers.openai.com / evaluation-best-practices, arXiv 2306.05685, platform.claude.com / develop-tests

  11. Repetition and degeneration Beginner

    Freely generated text can loop, repeat itself or turn bland, which a training or validation loss measured on reference text does not reveal. Read free generations and check them for repetition.

    In the long read: Did it work?

    Sources arXiv 1904.09751

  12. Pairwise comparison Practitioner

    Show two outputs for the same input and ask which is better, then turn many such votes into a ranking.

    Sources arXiv 2403.04132, arXiv 2306.05685

  13. Rates with their denominators (applicable, completed, timed out) Beginner

    Name what each rate measures. In a worked example, 36 successes among 40 eligible requests is 90%, counting a timeout as failure; 36 among 39 completed evaluations is about 92%. Report the timeout separately.

    In the long read: Did it work?

    Sources arXiv 2411.00640

  14. Uncertainty on small test sets (standard error, confidence intervals) Practitioner

    A score from a small test set is an estimate; report a range that shows how much it could move by chance.

    In the long read: Did it work?

    Sources arXiv 2411.00640, doi.org / 1009213286, asaip.psu.edu, statsmodels.org / statsmodels.stats.proportion.proportion_confint

  15. Variance across runs and seeds Practitioner

    The same training recipe with a different random seed, or the same eval with sampling, can give noticeably different scores.

    In the long read: Did it work?

    Sources arXiv 2002.06305, arXiv 2406.10229, arXiv 2411.00640

  16. Not over-reading small differences Beginner

    Treat a small score gap as a tie unless the uncertainty analysis says otherwise.

    In the long read: Did it work?

    Sources arXiv 2411.00640, arXiv 2403.04132, arXiv 2406.10229

  17. Safety regression after fine-tuning Beginner

    Fine-tuning can weaken a chat model's safety behaviour, such as refusing harmful requests, even with harmless-looking training data.

    In the long read: Did it work?

    Sources arXiv 2310.03693, developers.openai.com / supervised-fine-tuning

  18. Red-teaming Beginner

    Deliberately trying to make the model misbehave, to find failures before users do.

    In the long read: Did it work?

    Sources arXiv 2202.03286, arXiv 2209.07858

  19. Benchmark contamination Beginner

    When test items, or close copies, were in the training data, so the score overstates real ability.

    In the long read: Did it work?

    Sources arXiv 2310.18018, arXiv 2311.01964, arXiv 2311.04850, arXiv 2310.17623, arXiv 2406.04244, arXiv 2308.08493

  20. Goodhart's law in evaluation Beginner

    Once you optimise hard for a measurement, it can stop tracking the thing you actually care about.

    In the long read: Did it work?

    Sources arXiv 1803.04585, arXiv 2210.10760

Preference and RL 33

Training by comparison or score instead of by example, and what goes wrong.

  1. Preference tuning (preference feedback) Beginner

    Training signal that says which of two answers is better, or how good an answer is, instead of showing the ideal answer.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2203.02155, arXiv 1706.03741, arXiv 2009.01325, arXiv 2305.18290, developers.openai.com / model-optimization, docs.cloud.google.com / tune-models

  2. Policy and reference model Beginner

    The policy is the model being trained. The reference is an unchanged copy used as a comparison point, often the model after supervised fine-tuning.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2402.03300, huggingface.co / dpo_trainer

  3. Reward model (Bradley-Terry) Beginner

    A separate model trained on human comparisons to output one score per answer, so preferred answers score higher.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2203.02155, arXiv 2009.01325

  4. RLHF Beginner

    Reinforcement learning from human feedback. A classic setup trains on demonstrations, learns a reward model from human rankings, then trains the assistant toward higher reward scores; the score is a proxy for quality.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2203.02155, arXiv 2204.05862, arXiv 1706.03741

  5. PPO Practitioner

    The clipped policy-gradient RL algorithm used in classic RLHF. Alongside the policy it trains a learned value estimator, whose size and architecture vary: often a model of comparable size to the policy, while InstructGPT used one 6 billion parameter value function for policies of every size up to 175 billion.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 1707.06347, arXiv 2402.03300, arXiv 2203.02155

  6. KL penalty Beginner

    A stay-close penalty for changes in answer probabilities relative to an unchanged reference model. It discourages drift but does not guarantee safety or useful answers.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2203.02155, arXiv 2210.10760, arXiv 2503.14476

  7. Alignment tax Practitioner

    Loss of other abilities, such as benchmark scores, that can come with preference training.

    Sources arXiv 2203.02155

  8. Best-of-n and rejection sampling Practitioner

    Generate several answers, keep the highest-scoring one, and optionally fine-tune on the kept answers.

    Sources arXiv 2210.10760, arXiv 2501.12948

  9. DPO Beginner

    Trains directly on chosen and rejected pairs with a classification-style loss, without a separate reward model or an RL loop.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2305.18290, huggingface.co / dpo_trainer

  10. Implicit reward and beta Practitioner

    In DPO, an answer's implicit reward is beta times the log of the ratio between its probability under the trained model and under the frozen reference model. Higher beta keeps the trained model closer to the reference.

    In the long read: Beyond examples: preferences and rewards

    Sources huggingface.co / dpo_trainer, developers.openai.com / direct-preference-optimization, arXiv 2305.18290

  11. IPO Practitioner

    A DPO variant that optimizes preferences with an identity transform to avoid overfitting when preferences look deterministic.

    Sources arXiv 2310.12036

  12. KTO Practitioner

    A preference method that learns from single answers marked good or bad, instead of pairs.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2402.01306

  13. ORPO Practitioner

    Combines SFT and preference tuning in one stage using an odds-ratio penalty, with no reference model.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2403.07691

  14. SimPO Practitioner

    A reference-free DPO variant that scores answers by average per-token log-probability and adds a target margin.

    Sources arXiv 2405.14734

  15. Choosing within the DPO family Practitioner

    Pick by the data you have (pairs or single labels) and whether you can afford a reference model; data quality matters more than the loss.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2406.09279, arXiv 2402.01306

  16. Online vs offline preference learning Practitioner

    Online methods judge fresh answers from the current model during training; offline methods train on a fixed dataset collected beforehand.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2405.08448, arXiv 2402.04792

  17. RLAIF Beginner

    Reinforcement learning where an AI model, not a human, provides the preference labels or scores.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2309.00267, arXiv 2212.08073

  18. Constitutional AI Beginner

    Training a harmless assistant from written principles, using self-critique and AI-judged comparisons instead of human harm labels.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2212.08073

  19. RLVR Beginner

    Reinforcement learning where a program checks the answer, such as unit tests or an answer key, and that check is the reward.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2411.15124, arXiv 2501.12948

  20. GRPO Beginner

    Group relative policy optimization generates several answers to a prompt and compares reward scores within that group. Above-average scores get positive relative signals; below-average scores get negative ones. The scorer can be imperfect.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2402.03300, huggingface.co / grpo_trainer

  21. GRPO variants and REINFORCE-style methods Practitioner

    Fixes and simpler alternatives (Dr. GRPO, DAPO, RLOO) that change loss averaging, clipping or prompt filtering to reduce length bias and waste.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2503.20783, arXiv 2503.14476, arXiv 2402.14740

  22. DeepSeek-R1 and R1-Zero Beginner

    Reasoning models trained largely with reinforcement learning on checkable problems. R1-Zero ran RL straight from the base model with no supervised step. R1 started with a small supervised stage on thousands of cold-start examples, ran RL, then a substantial supervised stage on about 800,000 samples (about 600,000 reasoning and 200,000 non-reasoning, the reasoning ones gathered by rejection sampling), then a final RL stage.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2501.12948

  23. Outcome vs process rewards Practitioner

    Outcome rewards score only the final answer; process rewards score each reasoning step.

    Sources arXiv 2305.20050, arXiv 2402.03300

  24. pass@k and maj@k Practitioner

    pass@k is the chance any of k tries is right; maj@k is the chance the most common answer is right; comparing them shows whether RL added ability or just reliability.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2402.03300, arXiv 2504.13837, arXiv 2505.24864

  25. Reinforcement fine-tuning (RFT) Beginner

    Training where the model generates answers, a grader or reward function scores them, and the model is pushed toward higher-scoring answers. Managed services run this for you: you supply prompts and a grader.

    In the long read: Beyond examples: preferences and rewards

    Sources developers.openai.com / reinforcement-fine-tuning, learn.microsoft.com / overview, docs.aws.amazon.com / reinforcement-fine-tuning, docs.cloud.google.com / reinforcement-tuning, docs.aws.amazon.com / custom-models, developers.openai.com / model-optimization

  26. Reward hacking Beginner

    The model scores highly by exploiting flaws in the reward instead of doing the intended task.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2209.13085, arXiv 2503.11926

  27. Reward model over-optimization Practitioner

    Pushing hard on a learned reward makes its score rise while true quality rises and then falls.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2210.10760, arXiv 2406.02900

  28. Sycophancy Beginner

    Telling users what they want to hear rather than what is true, a behaviour linked to preference training.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2310.13548, arXiv 2212.09251, openai.com / expanding-on-sycophancy

  29. Length bias Beginner

    Preference training tends to make answers longer because raters, reward models and judges often favour longer answers.

    In the long read: Beyond examples: preferences and rewards

    Sources arXiv 2310.03716, arXiv 2403.19159

  30. Evaluating reward models Practitioner

    Checking a reward model on pairs where the better answer is known before training against it.

    Sources arXiv 2403.13787

  31. SFT first, then preference tuning Beginner

    The usual order: teach format and basics with SFT, then refine with preference tuning or RL.

    In the long read: Beyond examples: preferences and rewards

    Sources developers.openai.com / direct-preference-optimization, docs.cloud.google.com / gemini-preference-tuning, arXiv 2203.02155

  32. How much preference data Practitioner

    Published sizes range from thousands to tens of thousands of comparisons, and dozens to hundreds of prompts to start RFT; quality matters most.

    Sources arXiv 1909.08593, huggingface.co / ultrafeedback_binarized, arXiv 2406.09279

  33. Tooling Practitioner

    Open-source libraries such as TRL, OpenRLHF and verl implement DPO, GRPO, PPO and related trainers.

    Sources huggingface.co / index, github.com / OpenRLHF, github.com / verl

Shipping and operating 20

Serving the result, licences, where to fine-tune, and keeping it working.

  1. Self-hosted fine-tuning Practitioner

    Training an open-weight model on hardware you control with open libraries, keeping full control of data, method and the resulting weights.

    In the long read: Shipping it and keeping it working

    Sources huggingface.co / index, huggingface.co / index, docs.together.ai / fine-tuning-overview

  2. Hosted fine-tuning services Practitioner

    Vendor services that run fine-tuning for you through an upload and an API. Which models and methods are offered changes often; see the long read for the position as of 28 October 2026.

    In the long read: Shipping it and keeping it working

    Sources developers.openai.com / deprecations, developers.openai.com / model-optimization, ai.google.dev / model-tuning, docs.cloud.google.com / tune-models, docs.aws.amazon.com / custom-models, docs.aws.amazon.com / custom-model-fine-tuning, claude.com / fine-tune-claude-3-haiku, docs.mistral.ai / finetuning, docs.together.ai / fine-tuning-overview, docs.fireworks.ai / fine-tuning-models, docs.cloud.google.com / tuning, learn.microsoft.com / fine-tuning, docs.together.ai / overview, docs.mistral.ai / finetuning

  3. Maintenance and model lifecycle Practitioner

    A fine-tune hosted by a provider usually lives only as long as its base model there: when the provider retires the base, the hosted fine-tune goes too and must be rebuilt. Open-weight fine-tunes you have downloaded keep working for as long as you serve them, though newer base models may overtake them.

    In the long read: Shipping it and keeping it working

    Sources developers.openai.com / deprecations, docs.together.ai / fine-tuning-overview

  4. Merging LoRA adapters into the base model Beginner

    Add the LoRA correction into the base weights, leaving one model without a separate adapter calculation. Test the merged model you will actually serve.

    In the long read: Shipping it and keeping it working

    Sources arXiv 2106.09685, huggingface.co / lora

  5. Serving adapters separately Practitioner

    Keep one copy of the base model in memory and apply different small adapters per request.

    In the long read: Shipping it and keeping it working

    Sources arXiv 2106.09685, huggingface.co / lora

  6. An adapter is tied to its base model Beginner

    A LoRA adapter stores a correction to one base model's frozen weights, so it only works with that exact model; its config records the base model's name and, when set, its revision. Swapping or updating the base model means training the adapter again.

    In the long read: Shipping it and keeping it working

    Sources huggingface.co / checkpoint, arXiv 2106.09685

  7. Multi-adapter serving (vLLM, S-LoRA) Practitioner

    Serving systems that batch requests for many different LoRA adapters on one shared base model.

    In the long read: Shipping it and keeping it working

    Sources docs.vllm.ai / lora, arXiv 2311.03285

  8. Quantization for serving Beginner

    Store model weights with fewer bits so the model needs less memory, at some risk to quality. If a LoRA adapter is being merged in, merge first and then quantize, and re-run evaluations on the quantized file because quality can shift.

    In the long read: Shipping it and keeping it working

    Sources docs.vllm.ai / quantization, arXiv 2402.18158

  9. GPTQ Practitioner

    A one-shot post-training method that quantizes weights to 3 or 4 bits using approximate second-order information.

    In the long read: Shipping it and keeping it working

    Sources arXiv 2210.17323, docs.vllm.ai / quantization

  10. AWQ (Activation-aware Weight Quantization) Practitioner

    A weight-only quantization method that protects the most important weight channels, chosen by looking at activations.

    In the long read: Shipping it and keeping it working

    Sources arXiv 2306.00978, docs.vllm.ai / quantization

  11. GGUF and llama.cpp Practitioner

    A single-file model format, with many quantization types, used by llama.cpp and similar local runtimes.

    In the long read: Shipping it and keeping it working

    Sources huggingface.co / gguf, github.com / llama.cpp

  12. Quantization quality trade-off Practitioner

    Fewer bits can save memory, but may change quality and speed. Measure the exact model, format and task rather than treating compression as a guaranteed improvement.

    In the long read: Shipping it and keeping it working

    Sources arXiv 2210.17323, arXiv 2404.14047, arXiv 2402.18158

  13. Latency and throughput Practitioner

    Latency is how long one user waits; throughput is how much total work the server completes; pushing one usually costs the other.

    In the long read: Shipping it and keeping it working

    Sources docs.nvidia.com / metrics, arXiv 2309.06180

  14. Serving cost Beginner

    What it costs to run the model: hardware or hosting time, driven by model size, traffic, and how well requests are batched.

    In the long read: Shipping it and keeping it working

    Sources arXiv 2106.09685, arXiv 2309.06180, docs.aws.amazon.com / model-customization-import-model

  15. Hosted endpoints for fine-tuned models Beginner

    Managed services that run your fine-tuned model behind an API so you do not operate the servers yourself.

    Sources developers.openai.com / supervised-fine-tuning, docs.aws.amazon.com / model-customization-import-model, huggingface.co / index

  16. Model licences and fine-tuned derivatives Beginner

    The base model's licence decides what you may do with a model you fine-tuned from it, including naming, attribution and use limits.

    In the long read: Shipping it and keeping it working

    Sources llama.com / license, github.com / LICENSE, github.com / LICENSE, ai.google.dev / terms, opensource.googleblog.com / gemma-4-expanding-the-gemmaverse-with-apache-20, huggingface.co / gemma-4-31b-it, huggingface.co / Qwen3-8B, huggingface.co / LICENSE, huggingface.co / Mistral-7B-Instruct-v0.3, docs.mistral.ai / models, apache.org / LICENSE-2.0

  17. Monitoring in production and continuous evaluation Beginner

    Keep measuring the deployed model on real traffic, log inputs and outputs, and feed new failures back into the eval set.

    In the long read: Shipping it and keeping it working

    Sources developers.openai.com / evaluation-best-practices, arXiv 1810.11953

  18. Drift (data and distribution shift) Beginner

    Real inputs change over time and stop looking like the data the model was tuned and tested on.

    In the long read: Shipping it and keeping it working

    Sources papers.nips.cc / 86df7dcfd896fcaf2674f757a2463eba-Abstract, arXiv 1810.11953

  19. Versioning models, datasets and evals Practitioner

    Give every model, dataset, prompt template and eval set a version, and record which versions produced each result.

    In the long read: Shipping it and keeping it working

    Sources doc.dvc.org / versioning-data-and-models, huggingface.co / repositories-getting-started, arXiv 1810.03993, arXiv 1803.09010

  20. When to retrain Practitioner

    Retrain when evidence says so: monitored quality drops, inputs drift, requirements change, or a better base model arrives.

    In the long read: Shipping it and keeping it working

    Sources papers.nips.cc / 86df7dcfd896fcaf2674f757a2463eba-Abstract, developers.openai.com / evaluation-best-practices, arXiv 2308.08747, arXiv 2310.03693

Where the evidence disagrees

These questions are open or fast-moving. Each link goes to the concept, and the long read sets out both sides with sources.

Every definition comes from sources fetched on 28 or 29 October 2026. Vendor features, defaults and prices change; check the linked pages before you rely on them. Nothing here is legal advice.

Rather follow one fine-tune from start to finish?

Read the long read