Reference
Fine-tuning, every concept
Every term you are likely to meet when you fine-tune a language model, from what a weight is to when to retrain, each in one or two plain sentences. Beginner concepts are where to start; practitioner concepts are the details you need once you train for real. Where the long read What happens when you fine-tune a model explains a concept, the entry links to that part.
No concept matches. Try a shorter word, or clear the filter.
Foundations 18
What fine-tuning is, what it is not, and when to use it.
Weights (parameters) Beginner
The learned numerical settings that shape a model's predictions. Training updates them; an ordinary prompt leaves them unchanged.
In the long read: Should you fine-tune at all?
Pretraining and the base model Beginner
The first, very large training run on general text that produces a base model, which continues text but does not follow instructions well.
In the long read: Should you fine-tune at all?
Sources huggingface.co / chat_templating, arXiv 2405.05904, arXiv 2305.11206
Fine-tuning Beginner
Training an already pretrained model further on new examples so that its weights change and its behaviour shifts toward those examples.
In the long read: Should you fine-tune at all?
Sources docs.aws.amazon.com / custom-models, huggingface.co / index
Prompting and system prompts Beginner
Writing instructions and context into the text sent to the model; a system prompt is a standing instruction placed at the start of a conversation.
In the long read: Should you fine-tune at all?
Sources huggingface.co / chat_templating, docs.cloud.google.com / tune-models, docs.mistral.ai / finetuning, platform.claude.com / overview, platform.claude.com / claude-prompting-best-practices
Few-shot examples (in-context learning) Beginner
Putting a handful of worked examples in the prompt so the model copies the pattern, with no training.
In the long read: Should you fine-tune at all?
Sources arXiv 2005.14165, developers.openai.com / model-optimization
Retrieval-augmented generation (RAG) Beginner
Looking up relevant documents at question time and adding them to the prompt, so the model answers from current sources without retraining.
In the long read: Should you fine-tune at all?
Sources arXiv 2005.11401, arXiv 2403.10131, arXiv 2403.01432
Tool use (function calling) Beginner
Letting the model request calls to functions or APIs your code runs, so it can fetch live data or take actions instead of relying on memory.
In the long read: Should you fine-tune at all?
Sources platform.claude.com / overview, docs.cloud.google.com / tune-models
Base, instruct and chat models Beginner
A base model continues text; an instruct or chat model is a base model further trained to follow instructions and hold conversations.
In the long read: Should you fine-tune at all?
Sources huggingface.co / Llama-3.1-8B, arXiv 2203.02155, arXiv 2308.08747, huggingface.co / chat_templating
Open-weight vs closed models Beginner
Open-weight models let you download the weights and train them yourself; closed models can only be tuned through whatever service their maker or a cloud partner offers.
In the long read: Shipping it and keeping it working
Sources huggingface.co / Llama-3.1-8B
Continued pretraining (domain adaptation) Practitioner
Continuing the original next-word training on a large pile of raw, unlabelled text from your domain so the model gets used to its language.
In the long read: Should you fine-tune at all?
Sources arXiv 2004.10964, arXiv 2403.08763, docs.aws.amazon.com / custom-models, arXiv 2312.05934
Supervised fine-tuning (SFT) Beginner
Training on input and target-response pairs to encourage the demonstrated behaviour, including patterns of tone, length and format.
In the long read: What a training example looks like
Sources docs.aws.amazon.com / custom-models, developers.openai.com / model-optimization, docs.cloud.google.com / tune-models, arXiv 2305.11206
Instruction tuning Beginner
Supervised fine-tuning on many different tasks written as plain-language instructions, so the model learns to follow instructions in general.
In the long read: Should you fine-tune at all?
Sources arXiv 2109.01652
Where fine-tuning fits Beginner
Consistent style, format or behaviour; narrow repeatable tasks; and replacing a large model with a smaller, cheaper, faster one.
In the long read: Should you fine-tune at all?
Sources developers.openai.com / model-optimization, docs.mistral.ai / finetuning, claude.com / fine-tune-claude-3-haiku, docs.cloud.google.com / tune-models
Fine-tuning and new knowledge (behaviour versus facts) Beginner
Fine-tuning can learn some facts, but retrieval is a useful first choice for a changing knowledge base. In one comparison, retrieval beat further training on documents across the tested knowledge-intensive tasks.
In the long read: Should you fine-tune at all?
Sources arXiv 2312.05934, arXiv 2403.01432, arXiv 2309.14316, arXiv 2404.00213, arXiv 2005.11401, arXiv 2405.05904, arXiv 2410.03717
New facts in training data can raise hallucination Practitioner
One controlled study found that unfamiliar facts in training examples were learned more slowly and could increase hallucination in the tested models. This is a risk, not a prediction for every run.
In the long read: Should you fine-tune at all?
Sources arXiv 2405.05904, arXiv 2405.05904
What fine-tuning cannot fix Beginner
It cannot keep facts current, reach live systems, or replace a clear definition of success. The weights keep no record of where a fact came from, though a model can be trained to cite passages placed in its prompt or retrieved for it. Whether training can take a model beyond its base model's ability is disputed: one view holds that reinforcement learning mostly sharpens what the base model can already do, another reports new solutions after prolonged training.
In the long read: Should you fine-tune at all?
Sources arXiv 2005.11401, arXiv 2305.15717, platform.claude.com / overview, developers.openai.com / model-optimization, arXiv 2310.11511, arXiv 2402.03300, arXiv 2504.13837, arXiv 2505.24864
Costs and effort Beginner
Data collection and cleaning, training compute, evaluation, serving the custom model, and re-doing it all when the base model changes.
In the long read: Shipping it and keeping it working
Sources docs.aws.amazon.com / custom-models, docs.fireworks.ai / fine-tuning-models, docs.together.ai / fine-tuning-overview, together.ai / pricing
Decision ladder: prompt, then retrieve, then fine-tune Beginner
Start with prompting and examples, add retrieval for knowledge and tools for live data, and fine-tune only for behaviour, format or cost problems that remain.
In the long read: Should you fine-tune at all?
Sources docs.cloud.google.com / tune-models, docs.mistral.ai / finetuning, developers.openai.com / model-optimization
Data 40
What one example looks like, where examples come from, and how to clean and split them.
Training example Beginner
One demonstration: an input and the exact output you want the model to learn to produce.
In the long read: What a training example looks like
Sources developers.openai.com / supervised-fine-tuning, huggingface.co / sft_trainer
Prompt and target (completion) Beginner
The part of an example the model reads, and the part it should write back.
In the long read: What a training example looks like
Sources huggingface.co / dataset_formats, huggingface.co / sft_trainer
Roles (system, user, assistant, tool) Beginner
Labels that mark who wrote each message in chat training data.
In the long read: What a training example looks like
Sources huggingface.co / chat_templating, docs.cloud.google.com / gemini-supervised-tuning-prepare, huggingface.co / dataset_formats
Single-turn and multi-turn examples Beginner
One question and answer, versus a whole conversation in which every assistant turn is a target.
In the long read: What a training example looks like
Sources huggingface.co / dataset_formats, huggingface.co / sft_trainer
JSONL messages format Beginner
A file with one JSON training example per line, each holding a list of role-tagged messages.
In the long read: What a training example looks like
Sources developers.openai.com / supervised-fine-tuning, docs.cloud.google.com / gemini-supervised-tuning-prepare, huggingface.co / sft_trainer
Tool-calling examples Practitioner
Examples that show a tool call, its result and the final reply, together with the list of available tools.
Sources huggingface.co / dataset_formats, developers.openai.com / supervised-fine-tuning
Dataset types Practitioner
Language modelling, prompt-completion and preference data suit different training methods.
Sources huggingface.co / dataset_formats
Tokens, tokenizers and special tokens Beginner
A token is the small piece of text a model reads; a tokenizer turns text into token IDs. Special tokens are reserved markers for boundaries such as the end of a turn.
In the long read: What a training example looks like
Sources huggingface.co / chat_templating, huggingface.co / sft_trainer, huggingface.co / 4
Chat template Beginner
The model-specific recipe that turns a list of messages into the exact token string the model reads.
In the long read: What a training example looks like
Sources huggingface.co / chat_templating, huggingface.co / sft_trainer
Template mismatch Practitioner
Training text formatted differently from inference text, which silently lowers quality.
In the long read: What a training example looks like
Sources huggingface.co / chat_templating, huggingface.co / sft_trainer
Quality over quantity Beginner
Correctness and coverage matter as much as count. A small clean set can beat a noisy one, while broader tasks may still require many varied examples.
In the long read: The data decides what it learns
Sources arXiv 2305.11206, arXiv 2305.14314, arXiv 2307.08701, docs.cloud.google.com / gemini-supervised-tuning-prepare
Superficial Alignment Hypothesis Practitioner
The contested idea that fine-tuning mostly teaches style and format while pretraining supplies knowledge and ability.
In the long read: The data decides what it learns
Sources arXiv 2305.11206, arXiv 2312.01552, arXiv 2410.03717
How many examples to start with Beginner
Start with a small, careful pilot and measure the result. If it shows no gain, inspect the task, prompt, examples and evaluation before adding more data; provider minimums are not guarantees.
In the long read: The data decides what it learns
Sources developers.openai.com / supervised-fine-tuning, docs.cloud.google.com / gemini-supervised-tuning-prepare, arXiv 2305.11206
Human-written examples Beginner
Examples written by people who know the task: slow and costly, with the most control over quality.
In the long read: The data decides what it learns
Sources arXiv 2305.11206, developers.openai.com / supervised-fine-tuning
Curated logs Beginner
Records of real past interactions that are selected and cleaned before being used as training data.
In the long read: The data decides what it learns
Sources developers.openai.com / supervised-fine-tuning, edpb.europa.eu / edpb-opinion-ai-models-gdpr-principles-support-responsible-ai_en
Synthetic data Beginner
Training examples generated by a model rather than written by a person.
In the long read: The data decides what it learns
Sources crfm.stanford.edu / alpaca, arXiv 2306.11644, arXiv 2307.08701
Distillation (teacher and student) Beginner
Training a smaller student model to imitate the outputs of a stronger teacher model.
In the long read: The data decides what it learns
Sources arXiv 1503.02531, arXiv 2305.15717, docs.aws.amazon.com / custom-models, openai.com / row-terms-of-use, arXiv 2501.12948
Self-Instruct Practitioner
Using a model to grow a small seed set of instructions into a large dataset, then filtering for quality and diversity.
Synthetic data risks and model collapse Practitioner
Copying the generator's errors, and the loss of rare patterns when models train repeatedly on model-made data.
In the long read: The data decides what it learns
Sources arXiv 2305.17493, arXiv 2404.01413, arXiv 2305.15717
Terms and licences on model outputs Practitioner
Provider terms and model licences that limit using outputs to train other, especially competing, models.
In the long read: The data decides what it learns
Sources openai.com / row-terms-of-use, openai.com / services-agreement, anthropic.com / commercial-terms, ai.google.dev / terms, dev.meta.ai / license, crfm.stanford.edu / alpaca
Deduplication Beginner
Removing exact and near-duplicate examples within the training set and across splits.
In the long read: The data decides what it learns
Sources arXiv 2107.06499, arXiv 2202.07646, developers.google.com / dividing-datasets
Quality filtering Beginner
Removing wrong, off-topic, malformed or unsafe examples using rules, people or a grader model.
In the long read: The data decides what it learns
Sources arXiv 2307.08701, arXiv 2305.11206
Consistency of labels and format Beginner
Treating similar inputs the same way so the model learns one clear behaviour.
In the long read: The data decides what it learns
Sources arXiv 2103.14749
Diversity and coverage Beginner
Varied examples that together cover every kind of request the model will meet in real use.
In the long read: The data decides what it learns
Sources arXiv 2305.11206, arXiv 2212.10560, arXiv 2305.15717
Balancing Practitioner
Adjusting the mix so rare but important cases are not drowned out by common ones.
Edge cases and refusals Beginner
Explicit examples for unusual inputs, for declining harmful requests, and for not refusing safe ones.
In the long read: The data decides what it learns
Sources arXiv 2310.03693, arXiv 2308.01263
Provenance Practitioner
A record kept with each example of where it came from, so problem rows can be traced and removed.
In the long read: The data decides what it learns
Sources arXiv 2404.01413, edpb.europa.eu / edpb-opinion-ai-models-gdpr-principles-support-responsible-ai_en
Training, validation and test sets Beginner
Data the model learns from, data used to compare settings, and data kept back for the final check.
In the long read: The data decides what it learns
Sources scikit-learn.org / cross_validation, developers.google.com / dividing-datasets, docs.cloud.google.com / gemini-supervised-tuning-prepare, developers.openai.com / supervised-fine-tuning
Data leakage Practitioner
Evaluation information reaching training or development decisions, making results look better than reality.
In the long read: The data decides what it learns
Sources arXiv 2207.07048, arXiv 2107.06499, scikit-learn.org / cross_validation
Group-aware splits and stable identifiers Practitioner
Keeping related examples in the same split, with fixed IDs so the split stays the same every time the data is rebuilt.
In the long read: The data decides what it learns
Personal data and legal basis Beginner
Information about identifiable people, which needs a lawful reason and care before it is used for training.
In the long read: The data decides what it learns
Sources edpb.europa.eu / edpb-opinion-ai-models-gdpr-principles-support-responsible-ai_en, edpb.europa.eu / opinion-282024-certain-data-protection-aspects_en
Licences and rights on data Beginner
The terms attached to datasets and documents that decide whether you may train on them.
Sources crfm.stanford.edu / alpaca
Memorisation risk Beginner
A model storing and repeating training text word for word, which can leak private data.
In the long read: The data decides what it learns
Sources arXiv 2012.07805, arXiv 2202.07646, arXiv 2205.12506
Differentially private fine-tuning Practitioner
Training with added noise so no single example can strongly shape the model, giving a provable privacy bound.
Sources arXiv 2110.05679
Data inspection Beginner
Reading raw, rendered and masked examples by eye to catch problems that metrics miss.
In the long read: What a training example looks like
Sources karpathy.github.io / recipe, huggingface.co / chat_templating
Pilot runs Beginner
Small, cheap fine-tunes that check the pipeline and the data's effect before a full run.
In the long read: The data decides what it learns
Sources developers.openai.com / supervised-fine-tuning, karpathy.github.io / recipe, huggingface.co / sft_trainer
Diagnosing data problems Practitioner
Matching symptoms such as never stopping or poor production scores to their usual data causes.
Sources huggingface.co / chat_templating, huggingface.co / sft_trainer, scikit-learn.org / cross_validation, arXiv 2405.05904
Loss masking (completion-only or assistant-only loss) Practitioner
Choosing which tokens count toward the loss, the prediction penalty. Assistant-only training scores the reply while still reading the prompt as context; other setups may score more text.
In the long read: What a training example looks like
Sources huggingface.co / sft_trainer, unsloth.ai / lora-hyperparameters-guide, arXiv 2305.14314, arXiv 2405.14394
Sequence length (max_length) and truncation Beginner
The maximum tokens a training run is configured to allow per example. It can be shorter than the model's own context maximum. The prompt, the answer and the chat-template tokens have to fit inside it together. With truncation on, anything longer is cut; other pipelines reject or filter the example.
In the long read: What a training example looks like
Sources huggingface.co / sft_trainer, huggingface.co / reducing_memory_usage, huggingface.co / model_memory_anatomy, docs.cloud.google.com / gemini-supervised-tuning-prepare
Held-out test set discipline Beginner
The test set is never trained on, never used to tune prompts or pick checkpoints, and is scored the same way before and after.
In the long read: The data decides what it learns
Sources scikit-learn.org / cross_validation, arXiv 2311.04850, developers.google.com / dividing-datasets
Training 59
The training loop, its settings, LoRA and other efficient methods, memory and tools.
Next-token prediction Beginner
The model is trained to give high probability to the token that actually comes next.
In the long read: What happens during training
Sources huggingface.co / sft_trainer
Loss (cross-entropy) Beginner
An average prediction penalty on the example's target tokens. Lower means a better fit under the same data and scoring setup; it is not an error percentage or proof of better task answers.
In the long read: What happens during training
Sources huggingface.co / sft_trainer, developers.google.com / glossary
Teacher forcing Practitioner
During training, every prediction is made from the true previous tokens of the example rather than from the model's own earlier outputs.
In the long read: What happens during training
Exposure bias (training versus generation mismatch) Practitioner
A model trained only on true previous tokens must read its own output when it generates, so early mistakes can carry forward and errors can accumulate. Scheduled sampling, which gradually feeds the model its own predictions during training, is one proposed fix.
In the long read: What happens during training
Sources arXiv 1511.06732, arXiv 1506.03099
Perplexity Practitioner
Perplexity is the exponential of the average cross-entropy loss; lower is better.
In the long read: What happens during training
Sources huggingface.co / perplexity
Gradient and backpropagation Beginner
A gradient says how to change each weight to reduce the loss; backpropagation computes it.
In the long read: What happens during training
Sources docs.pytorch.org / autogradqs_tutorial, huggingface.co / model_memory_anatomy
Optimizer (Adam and AdamW) Beginner
The optimizer turns gradients into weight updates; AdamW is Adam with decoupled weight decay.
In the long read: What happens during training
Sources arXiv 1412.6980, arXiv 1711.05101, huggingface.co / trainer, huggingface.co / model_memory_anatomy
Weight decay Practitioner
Weight decay gently shrinks weights each step as a mild guard against overfitting.
In the long read: What happens during training
Sources huggingface.co / trainer, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673
Learning rate Beginner
How large each weight update is. Too small can make learning slow; too large can make training unstable or damage useful behaviour. Choose it using validation results.
In the long read: What happens during training
Sources developers.google.com / hyperparameters, huggingface.co / trainer, huggingface.co / sft_trainer, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673
Learning-rate schedule (linear, cosine, constant) Practitioner
A schedule changes the learning rate over training, for example linear or cosine decay.
In the long read: What happens during training
Sources huggingface.co / optimizer_schedules, huggingface.co / trainer, unsloth.ai / lora-hyperparameters-guide
Warmup Practitioner
Warmup ramps the learning rate up from near zero over the first steps.
In the long read: What happens during training
Sources arXiv 1706.02677, huggingface.co / trainer, unsloth.ai / lora-hyperparameters-guide
Gradient clipping (max gradient norm) Practitioner
Gradient clipping caps the size of the gradient so one bad batch cannot wreck the model.
In the long read: What happens during training
Sources huggingface.co / trainer, huggingface.co / sft_trainer
Batch size Beginner
How many examples one device runs in one forward and backward pass. Several of these microbatches can be accumulated before one weight update.
In the long read: What happens during training
Sources developers.google.com / hyperparameters, huggingface.co / trainer, learn.microsoft.com / fine-tuning
Step and epoch Beginner
A step updates the weights once after a batch. An epoch is one full pass over the training data. More epochs revisit the same examples, which raises overfitting and forgetting risk, and do not guarantee better answers on new inputs.
In the long read: What happens during training
Sources developers.google.com / hyperparameters, huggingface.co / callback, unsloth.ai / lora-hyperparameters-guide, huggingface.co / trainer
Gradient accumulation and effective batch size Beginner
Combining gradients from several microbatches, with loss normalization, so one weight update covers more examples. Without that normalization, the update would grow with the number of passes. This saves memory for intermediate results, not for the weights or the optimizer state. On one GPU, microbatch 1 with 8 accumulation steps is an effective batch of 8: comparable to a batch of 8 under the same normalization, not guaranteed identical.
In the long read: What happens during training
Sources huggingface.co / grad_accumulation, unsloth.ai / lora-hyperparameters-guide
Full fine-tuning Beginner
Full fine-tuning updates every weight in the model.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2405.09673, huggingface.co / model_memory_anatomy
Parameter-efficient fine-tuning (PEFT) Beginner
Training only a small share of the parameters, either a few added ones (as in LoRA) or a chosen subset of the existing ones (BitFit updates only the bias terms), while the rest stay frozen, instead of updating every weight as full fine-tuning does.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources huggingface.co / index, arXiv 2106.09685, docs.together.ai / fine-tuning-overview, arXiv 2405.09673, arXiv 2106.10199
LoRA (Low-Rank Adaptation) Beginner
LoRA saves training memory by learning a small correction, called an adapter, beside unchanged base weights. It represents the correction using two small matrices, tables of numbers.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2106.09685, arXiv 2405.09673
LoRA rank (r) Beginner
The shared width of a LoRA adapter's two thin matrices: how many independent directions the correction can use. It does not set how large the change is. A rank of 1 can still be large. A higher rank trains more numbers and uses more memory. It does not by itself mean better answers, and it is not a count of layers or of examples.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2106.09685, arXiv 2305.14314, arXiv 2405.09673, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide
LoRA alpha (scaling) Practitioner
LoRA alpha scales the size of the correction by alpha divided by rank. Rank itself is not that size.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2106.09685, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673
rsLoRA (rank-stabilised LoRA) Practitioner
rsLoRA scales by alpha divided by the square root of rank so high ranks still learn.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2312.03732, huggingface.co / lora
Target modules Practitioner
Target modules are the weight matrices that receive LoRA adapters.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2106.09685, arXiv 2305.14314, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide
LoRA dropout Practitioner
LoRA dropout randomly drops adapter inputs during training as light regularisation.
Sources huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673
LoRA initialisation variants (PiSSA, OLoRA, LoftQ) Practitioner
PiSSA, OLoRA and LoftQ are alternative ways to initialise LoRA adapters.
Sources huggingface.co / lora
Quantization (in training) Beginner
Quantization stores numbers in fewer bits to save memory.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2305.14314
QLoRA Beginner
QLoRA trains a LoRA adapter while storing the frozen base weights in 4-bit form to save memory. The adapter and training calculations use higher precision. Training a 16-bit LoRA and then quantizing the merged model is not QLoRA; that later step is inference compression.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2305.14314, unsloth.ai / lora-hyperparameters-guide
NF4 (4-bit NormalFloat) and double quantization Practitioner
NF4 is a 4-bit format for normally distributed weights; double quantization also compresses the scaling constants.
Sources arXiv 2305.14314
Paged optimizers Practitioner
Paged optimizers spill optimizer state to CPU RAM during GPU memory spikes.
Sources arXiv 2305.14314
DoRA (Weight-Decomposed Low-Rank Adaptation) Practitioner
DoRA splits weights into magnitude and direction and uses LoRA for the direction.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2402.09353, huggingface.co / lora
Adapters (bottleneck adapters) Practitioner
Adapters insert small trainable layers inside a frozen model.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 1902.00751, arXiv 2106.09685
Prefix tuning and prompt tuning (soft prompts) Practitioner
Prompt tuning learns soft prompt embeddings added only at the model's input; prefix tuning learns trainable prefix vectors inserted at every layer. Both leave the model's own weights frozen.
Sources arXiv 2101.00190, arXiv 2104.08691, huggingface.co / prompting
Packing and padding-free batching Practitioner
Packing groups several short examples into the same fixed-length training row to avoid wasted padding, which can split an example across rows. Padding-free batching instead flattens a sampled batch into one sequence without padding, keeping every example complete.
Sources huggingface.co / reducing_memory_usage, huggingface.co / sft_trainer
Typical starting hyperparameters Practitioner
Library and vendor docs publish starting hyperparameters to use for a first run.
In the long read: What happens during training
Sources huggingface.co / trainer, huggingface.co / sft_trainer, huggingface.co / lora, unsloth.ai / lora-hyperparameters-guide, arXiv 2405.09673, learn.microsoft.com / fine-tuning
Where training memory goes Beginner
Training memory holds weights, gradients, optimizer states and activations.
In the long read: Full fine-tuning, LoRA and QLoRA
Mixed precision (fp16 and bf16) Beginner
Mixed precision computes in 16-bit (bf16 or fp16) while keeping a 32-bit master copy of weights.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 1710.03740, huggingface.co / mixed_precision_training, huggingface.co / sft_trainer
Gradient checkpointing Beginner
Gradient checkpointing recomputes some activations to save memory.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 1604.06174, huggingface.co / grad_checkpointing, huggingface.co / reducing_memory_usage
8-bit optimizers Practitioner
8-bit optimizers store optimizer states in 8 bits to save memory.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources arXiv 2110.02861, huggingface.co / model_memory_anatomy
CPU offloading Practitioner
Offloading moves optimizer states or activations from GPU to CPU memory.
Sources huggingface.co / reducing_memory_usage, deepspeed.ai / zero
Data parallel training (DDP) Beginner
Data parallel training copies the model to each GPU and averages gradients.
Sources docs.pytorch.org / ddp
Sharded training (FSDP and DeepSpeed ZeRO) Practitioner
FSDP and ZeRO shard weights, gradients and optimizer states across GPUs.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources deepspeed.ai / zero, arXiv 1910.02054, docs.pytorch.org / fsdp, github.com / axolotl
Training loss versus validation loss Beginner
Training loss measures fit to examples used for updates; validation loss measures fit to held-out examples. Falling training loss alone does not establish better answers to new questions.
In the long read: What happens during training
Sources huggingface.co / trainer, learn.microsoft.com / fine-tuning, developers.google.com / overfitting
Overfitting Beginner
Fitting training examples too closely so performance on new inputs suffers. Falling training loss while validation loss rises is a warning; judge task results too.
In the long read: What happens during training
Sources developers.google.com / overfitting, unsloth.ai / lora-hyperparameters-guide, arXiv 2203.02155
Underfitting Beginner
Underfitting is not learning enough to do well even on training data.
In the long read: What happens during training
Sources developers.google.com / overfitting, unsloth.ai / lora-hyperparameters-guide
Early stopping Beginner
Ending a run when a chosen validation measure stops improving. The chosen measure and patience determine when to stop; loss alone need not identify the best task performance.
In the long read: What happens during training
Checkpoints and choosing one Beginner
A saved snapshot of the model during training. Choosing one by its validation score means that score is no longer an unbiased final measure. The test set is.
In the long read: What happens during training
Sources developers.google.com / glossary, huggingface.co / trainer, learn.microsoft.com / fine-tuning, huggingface.co / sft_trainer
Catastrophic forgetting Beginner
Losing skills or knowledge a model had before while training it on something new, so it gets worse at things it used to do well.
In the long read: What happens during training
Sources arXiv 1612.00796, arXiv 2308.08747, arXiv 2405.09673, arXiv 2403.08763, github.com / lm-evaluation-harness, arXiv 2211.09110
Mitigation: mixing in general data (replay) Practitioner
Replay mixes some earlier or general data into training to reduce forgetting.
In the long read: What happens during training
Sources arXiv 2205.12393, arXiv 2403.08763
Mitigation: lower learning rate, fewer epochs, LoRA Practitioner
Lower learning rates, fewer epochs and LoRA tend to reduce forgetting.
In the long read: What happens during training
Sources arXiv 2405.09673
Random seed Beginner
A seed fixes random number generators so runs can be repeated.
In the long read: What happens during training
Sources huggingface.co / trainer, docs.pytorch.org / randomness
Determinism and version pinning Practitioner
Exact results also depend on library versions and hardware, so pin and record them.
In the long read: What happens during training
Sources docs.pytorch.org / randomness, huggingface.co / sft_trainer
Config and experiment logging Beginner
Experiment logging records config, metrics and outputs of every run.
In the long read: What happens during training
Sources huggingface.co / callback, github.com / axolotl, learn.microsoft.com / fine-tuning
Hugging Face Transformers (Trainer) Practitioner
Transformers is the base library for loading and training open models.
In the long read: Full fine-tuning, LoRA and QLoRA
TRL (SFTTrainer) Practitioner
TRL is Hugging Face's post-training library, including SFTTrainer.
In the long read: Full fine-tuning, LoRA and QLoRA
PEFT library Practitioner
PEFT is Hugging Face's library of LoRA and other parameter-efficient methods.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources pypi.org / peft, huggingface.co / index
Axolotl Practitioner
Axolotl is a YAML-config-driven fine-tuning framework.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources github.com / axolotl, pypi.org / axolotl
Unsloth Practitioner
Unsloth is a library and app for fast, low-memory fine-tuning on one machine.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources github.com / unsloth, pypi.org / unsloth
torchtune Practitioner
torchtune is a PyTorch-native fine-tuning library that is no longer maintained.
In the long read: Full fine-tuning, LoRA and QLoRA
LLaMA-Factory Practitioner
LLaMA-Factory is a unified fine-tuning toolkit with a no-code web UI.
In the long read: Full fine-tuning, LoRA and QLoRA
DeepSpeed and PyTorch FSDP Practitioner
DeepSpeed and PyTorch FSDP implement sharded multi-GPU training.
In the long read: Full fine-tuning, LoRA and QLoRA
Sources pypi.org / deepspeed, deepspeed.ai / zero, docs.pytorch.org / fsdp
Evaluation 20
How to tell whether a fine-tune worked, and what it may have broken.
Evaluation first (success criteria and a baseline) Beginner
Write down what a better model must do and how you will measure it, and measure the current model, before you train anything.
In the long read: Did it work?
Sources platform.claude.com / define-success, developers.openai.com / supervised-fine-tuning, developers.openai.com / evaluation-best-practices, developers.openai.com / model-optimization, platform.claude.com / overview
Baselines Beginner
The scores you compare against: the base model with a good prompt, the model before tuning, and a stronger reference model.
In the long read: Did it work?
Sources developers.openai.com / supervised-fine-tuning, platform.claude.com / define-success, arXiv 2404.04475
Before and after on the same items (paired comparison) Practitioner
Compare two models question by question on the same items, so the difference is measured more precisely.
In the long read: Did it work?
Sources arXiv 2411.00640
Exact match Beginner
Score an answer as right only if it equals the expected answer, usually after light normalisation.
In the long read: Did it work?
Deterministic checks (format, schema, rules) Beginner
Code that checks an output against fixed rules, such as valid JSON, required fields, allowed values or length limits.
In the long read: Did it work?
Sources json-schema.org / what-is-jsonschema, platform.claude.com / develop-tests
Generation length limit (max_new_tokens) Practitioner
The cap on how many new tokens a model may write in one answer, set separately from the length of the prompt. An answer that hits the cap is cut off before its end-of-sequence token, so an evaluation should count cut-off answers as failures or report them separately.
In the long read: Did it work?
Sources huggingface.co / llm_tutorial
Reference-based metrics (BLEU, ROUGE, BERTScore) Practitioner
Scores that compare a generated text with one or more human-written reference texts by overlap or similarity.
In the long read: Did it work?
Sources aclanthology.org / P02-1040, aclanthology.org / W04-1013, arXiv 1904.09675, platform.claude.com / develop-tests, developers.openai.com / evaluation-best-practices
Model-graded evaluation (LLM-as-a-judge) Beginner
Use a language model, given a rubric, to grade or compare outputs that are hard to check with code.
In the long read: Did it work?
Sources arXiv 2306.05685, platform.claude.com / develop-tests
Judge biases: position, length, self-preference Practitioner
Model judges can favour the answer shown first, the longer answer, or answers written by themselves.
In the long read: Did it work?
Sources arXiv 2306.05685, arXiv 2305.17926, arXiv 2404.04475, arXiv 2404.13076, arXiv 2404.18796, platform.claude.com / develop-tests
Human evaluation Beginner
People, ideally domain experts with a written rubric, grade or compare model outputs.
In the long read: Did it work?
Sources developers.openai.com / evaluation-best-practices, arXiv 2306.05685, platform.claude.com / develop-tests
Repetition and degeneration Beginner
Freely generated text can loop, repeat itself or turn bland, which a training or validation loss measured on reference text does not reveal. Read free generations and check them for repetition.
In the long read: Did it work?
Sources arXiv 1904.09751
Pairwise comparison Practitioner
Show two outputs for the same input and ask which is better, then turn many such votes into a ranking.
Sources arXiv 2403.04132, arXiv 2306.05685
Rates with their denominators (applicable, completed, timed out) Beginner
Name what each rate measures. In a worked example, 36 successes among 40 eligible requests is 90%, counting a timeout as failure; 36 among 39 completed evaluations is about 92%. Report the timeout separately.
In the long read: Did it work?
Sources arXiv 2411.00640
Uncertainty on small test sets (standard error, confidence intervals) Practitioner
A score from a small test set is an estimate; report a range that shows how much it could move by chance.
In the long read: Did it work?
Sources arXiv 2411.00640, doi.org / 1009213286, asaip.psu.edu, statsmodels.org / statsmodels.stats.proportion.proportion_confint
Variance across runs and seeds Practitioner
The same training recipe with a different random seed, or the same eval with sampling, can give noticeably different scores.
In the long read: Did it work?
Sources arXiv 2002.06305, arXiv 2406.10229, arXiv 2411.00640
Not over-reading small differences Beginner
Treat a small score gap as a tie unless the uncertainty analysis says otherwise.
In the long read: Did it work?
Sources arXiv 2411.00640, arXiv 2403.04132, arXiv 2406.10229
Safety regression after fine-tuning Beginner
Fine-tuning can weaken a chat model's safety behaviour, such as refusing harmful requests, even with harmless-looking training data.
In the long read: Did it work?
Sources arXiv 2310.03693, developers.openai.com / supervised-fine-tuning
Red-teaming Beginner
Deliberately trying to make the model misbehave, to find failures before users do.
In the long read: Did it work?
Sources arXiv 2202.03286, arXiv 2209.07858
Benchmark contamination Beginner
When test items, or close copies, were in the training data, so the score overstates real ability.
In the long read: Did it work?
Sources arXiv 2310.18018, arXiv 2311.01964, arXiv 2311.04850, arXiv 2310.17623, arXiv 2406.04244, arXiv 2308.08493
Goodhart's law in evaluation Beginner
Once you optimise hard for a measurement, it can stop tracking the thing you actually care about.
In the long read: Did it work?
Sources arXiv 1803.04585, arXiv 2210.10760
Preference and RL 33
Training by comparison or score instead of by example, and what goes wrong.
Preference tuning (preference feedback) Beginner
Training signal that says which of two answers is better, or how good an answer is, instead of showing the ideal answer.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2203.02155, arXiv 1706.03741, arXiv 2009.01325, arXiv 2305.18290, developers.openai.com / model-optimization, docs.cloud.google.com / tune-models
Policy and reference model Beginner
The policy is the model being trained. The reference is an unchanged copy used as a comparison point, often the model after supervised fine-tuning.
In the long read: Beyond examples: preferences and rewards
Reward model (Bradley-Terry) Beginner
A separate model trained on human comparisons to output one score per answer, so preferred answers score higher.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2203.02155, arXiv 2009.01325
RLHF Beginner
Reinforcement learning from human feedback. A classic setup trains on demonstrations, learns a reward model from human rankings, then trains the assistant toward higher reward scores; the score is a proxy for quality.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2203.02155, arXiv 2204.05862, arXiv 1706.03741
PPO Practitioner
The clipped policy-gradient RL algorithm used in classic RLHF. Alongside the policy it trains a learned value estimator, whose size and architecture vary: often a model of comparable size to the policy, while InstructGPT used one 6 billion parameter value function for policies of every size up to 175 billion.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 1707.06347, arXiv 2402.03300, arXiv 2203.02155
KL penalty Beginner
A stay-close penalty for changes in answer probabilities relative to an unchanged reference model. It discourages drift but does not guarantee safety or useful answers.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2203.02155, arXiv 2210.10760, arXiv 2503.14476
Alignment tax Practitioner
Loss of other abilities, such as benchmark scores, that can come with preference training.
Sources arXiv 2203.02155
Best-of-n and rejection sampling Practitioner
Generate several answers, keep the highest-scoring one, and optionally fine-tune on the kept answers.
Sources arXiv 2210.10760, arXiv 2501.12948
DPO Beginner
Trains directly on chosen and rejected pairs with a classification-style loss, without a separate reward model or an RL loop.
In the long read: Beyond examples: preferences and rewards
Implicit reward and beta Practitioner
In DPO, an answer's implicit reward is beta times the log of the ratio between its probability under the trained model and under the frozen reference model. Higher beta keeps the trained model closer to the reference.
In the long read: Beyond examples: preferences and rewards
Sources huggingface.co / dpo_trainer, developers.openai.com / direct-preference-optimization, arXiv 2305.18290
IPO Practitioner
A DPO variant that optimizes preferences with an identity transform to avoid overfitting when preferences look deterministic.
Sources arXiv 2310.12036
KTO Practitioner
A preference method that learns from single answers marked good or bad, instead of pairs.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2402.01306
ORPO Practitioner
Combines SFT and preference tuning in one stage using an odds-ratio penalty, with no reference model.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2403.07691
SimPO Practitioner
A reference-free DPO variant that scores answers by average per-token log-probability and adds a target margin.
Sources arXiv 2405.14734
Choosing within the DPO family Practitioner
Pick by the data you have (pairs or single labels) and whether you can afford a reference model; data quality matters more than the loss.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2406.09279, arXiv 2402.01306
Online vs offline preference learning Practitioner
Online methods judge fresh answers from the current model during training; offline methods train on a fixed dataset collected beforehand.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2405.08448, arXiv 2402.04792
RLAIF Beginner
Reinforcement learning where an AI model, not a human, provides the preference labels or scores.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2309.00267, arXiv 2212.08073
Constitutional AI Beginner
Training a harmless assistant from written principles, using self-critique and AI-judged comparisons instead of human harm labels.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2212.08073
RLVR Beginner
Reinforcement learning where a program checks the answer, such as unit tests or an answer key, and that check is the reward.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2411.15124, arXiv 2501.12948
GRPO Beginner
Group relative policy optimization generates several answers to a prompt and compares reward scores within that group. Above-average scores get positive relative signals; below-average scores get negative ones. The scorer can be imperfect.
In the long read: Beyond examples: preferences and rewards
GRPO variants and REINFORCE-style methods Practitioner
Fixes and simpler alternatives (Dr. GRPO, DAPO, RLOO) that change loss averaging, clipping or prompt filtering to reduce length bias and waste.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2503.20783, arXiv 2503.14476, arXiv 2402.14740
DeepSeek-R1 and R1-Zero Beginner
Reasoning models trained largely with reinforcement learning on checkable problems. R1-Zero ran RL straight from the base model with no supervised step. R1 started with a small supervised stage on thousands of cold-start examples, ran RL, then a substantial supervised stage on about 800,000 samples (about 600,000 reasoning and 200,000 non-reasoning, the reasoning ones gathered by rejection sampling), then a final RL stage.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2501.12948
Outcome vs process rewards Practitioner
Outcome rewards score only the final answer; process rewards score each reasoning step.
Sources arXiv 2305.20050, arXiv 2402.03300
pass@k and maj@k Practitioner
pass@k is the chance any of k tries is right; maj@k is the chance the most common answer is right; comparing them shows whether RL added ability or just reliability.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2402.03300, arXiv 2504.13837, arXiv 2505.24864
Reinforcement fine-tuning (RFT) Beginner
Training where the model generates answers, a grader or reward function scores them, and the model is pushed toward higher-scoring answers. Managed services run this for you: you supply prompts and a grader.
In the long read: Beyond examples: preferences and rewards
Sources developers.openai.com / reinforcement-fine-tuning, learn.microsoft.com / overview, docs.aws.amazon.com / reinforcement-fine-tuning, docs.cloud.google.com / reinforcement-tuning, docs.aws.amazon.com / custom-models, developers.openai.com / model-optimization
Reward hacking Beginner
The model scores highly by exploiting flaws in the reward instead of doing the intended task.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2209.13085, arXiv 2503.11926
Reward model over-optimization Practitioner
Pushing hard on a learned reward makes its score rise while true quality rises and then falls.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2210.10760, arXiv 2406.02900
Sycophancy Beginner
Telling users what they want to hear rather than what is true, a behaviour linked to preference training.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2310.13548, arXiv 2212.09251, openai.com / expanding-on-sycophancy
Length bias Beginner
Preference training tends to make answers longer because raters, reward models and judges often favour longer answers.
In the long read: Beyond examples: preferences and rewards
Sources arXiv 2310.03716, arXiv 2403.19159
Evaluating reward models Practitioner
Checking a reward model on pairs where the better answer is known before training against it.
Sources arXiv 2403.13787
SFT first, then preference tuning Beginner
The usual order: teach format and basics with SFT, then refine with preference tuning or RL.
In the long read: Beyond examples: preferences and rewards
Sources developers.openai.com / direct-preference-optimization, docs.cloud.google.com / gemini-preference-tuning, arXiv 2203.02155
How much preference data Practitioner
Published sizes range from thousands to tens of thousands of comparisons, and dozens to hundreds of prompts to start RFT; quality matters most.
Sources arXiv 1909.08593, huggingface.co / ultrafeedback_binarized, arXiv 2406.09279
Tooling Practitioner
Open-source libraries such as TRL, OpenRLHF and verl implement DPO, GRPO, PPO and related trainers.
Sources huggingface.co / index, github.com / OpenRLHF, github.com / verl
Shipping and operating 20
Serving the result, licences, where to fine-tune, and keeping it working.
Self-hosted fine-tuning Practitioner
Training an open-weight model on hardware you control with open libraries, keeping full control of data, method and the resulting weights.
In the long read: Shipping it and keeping it working
Sources huggingface.co / index, huggingface.co / index, docs.together.ai / fine-tuning-overview
Hosted fine-tuning services Practitioner
Vendor services that run fine-tuning for you through an upload and an API. Which models and methods are offered changes often; see the long read for the position as of 28 October 2026.
In the long read: Shipping it and keeping it working
Sources developers.openai.com / deprecations, developers.openai.com / model-optimization, ai.google.dev / model-tuning, docs.cloud.google.com / tune-models, docs.aws.amazon.com / custom-models, docs.aws.amazon.com / custom-model-fine-tuning, claude.com / fine-tune-claude-3-haiku, docs.mistral.ai / finetuning, docs.together.ai / fine-tuning-overview, docs.fireworks.ai / fine-tuning-models, docs.cloud.google.com / tuning, learn.microsoft.com / fine-tuning, docs.together.ai / overview, docs.mistral.ai / finetuning
Maintenance and model lifecycle Practitioner
A fine-tune hosted by a provider usually lives only as long as its base model there: when the provider retires the base, the hosted fine-tune goes too and must be rebuilt. Open-weight fine-tunes you have downloaded keep working for as long as you serve them, though newer base models may overtake them.
In the long read: Shipping it and keeping it working
Sources developers.openai.com / deprecations, docs.together.ai / fine-tuning-overview
Merging LoRA adapters into the base model Beginner
Add the LoRA correction into the base weights, leaving one model without a separate adapter calculation. Test the merged model you will actually serve.
In the long read: Shipping it and keeping it working
Sources arXiv 2106.09685, huggingface.co / lora
Serving adapters separately Practitioner
Keep one copy of the base model in memory and apply different small adapters per request.
In the long read: Shipping it and keeping it working
Sources arXiv 2106.09685, huggingface.co / lora
An adapter is tied to its base model Beginner
A LoRA adapter stores a correction to one base model's frozen weights, so it only works with that exact model; its config records the base model's name and, when set, its revision. Swapping or updating the base model means training the adapter again.
In the long read: Shipping it and keeping it working
Multi-adapter serving (vLLM, S-LoRA) Practitioner
Serving systems that batch requests for many different LoRA adapters on one shared base model.
In the long read: Shipping it and keeping it working
Sources docs.vllm.ai / lora, arXiv 2311.03285
Quantization for serving Beginner
Store model weights with fewer bits so the model needs less memory, at some risk to quality. If a LoRA adapter is being merged in, merge first and then quantize, and re-run evaluations on the quantized file because quality can shift.
In the long read: Shipping it and keeping it working
GPTQ Practitioner
A one-shot post-training method that quantizes weights to 3 or 4 bits using approximate second-order information.
In the long read: Shipping it and keeping it working
AWQ (Activation-aware Weight Quantization) Practitioner
A weight-only quantization method that protects the most important weight channels, chosen by looking at activations.
In the long read: Shipping it and keeping it working
GGUF and llama.cpp Practitioner
A single-file model format, with many quantization types, used by llama.cpp and similar local runtimes.
In the long read: Shipping it and keeping it working
Quantization quality trade-off Practitioner
Fewer bits can save memory, but may change quality and speed. Measure the exact model, format and task rather than treating compression as a guaranteed improvement.
In the long read: Shipping it and keeping it working
Sources arXiv 2210.17323, arXiv 2404.14047, arXiv 2402.18158
Latency and throughput Practitioner
Latency is how long one user waits; throughput is how much total work the server completes; pushing one usually costs the other.
In the long read: Shipping it and keeping it working
Serving cost Beginner
What it costs to run the model: hardware or hosting time, driven by model size, traffic, and how well requests are batched.
In the long read: Shipping it and keeping it working
Sources arXiv 2106.09685, arXiv 2309.06180, docs.aws.amazon.com / model-customization-import-model
Hosted endpoints for fine-tuned models Beginner
Managed services that run your fine-tuned model behind an API so you do not operate the servers yourself.
Sources developers.openai.com / supervised-fine-tuning, docs.aws.amazon.com / model-customization-import-model, huggingface.co / index
Model licences and fine-tuned derivatives Beginner
The base model's licence decides what you may do with a model you fine-tuned from it, including naming, attribution and use limits.
In the long read: Shipping it and keeping it working
Sources llama.com / license, github.com / LICENSE, github.com / LICENSE, ai.google.dev / terms, opensource.googleblog.com / gemma-4-expanding-the-gemmaverse-with-apache-20, huggingface.co / gemma-4-31b-it, huggingface.co / Qwen3-8B, huggingface.co / LICENSE, huggingface.co / Mistral-7B-Instruct-v0.3, docs.mistral.ai / models, apache.org / LICENSE-2.0
Monitoring in production and continuous evaluation Beginner
Keep measuring the deployed model on real traffic, log inputs and outputs, and feed new failures back into the eval set.
In the long read: Shipping it and keeping it working
Sources developers.openai.com / evaluation-best-practices, arXiv 1810.11953
Drift (data and distribution shift) Beginner
Real inputs change over time and stop looking like the data the model was tuned and tested on.
In the long read: Shipping it and keeping it working
Sources papers.nips.cc / 86df7dcfd896fcaf2674f757a2463eba-Abstract, arXiv 1810.11953
Versioning models, datasets and evals Practitioner
Give every model, dataset, prompt template and eval set a version, and record which versions produced each result.
In the long read: Shipping it and keeping it working
Sources doc.dvc.org / versioning-data-and-models, huggingface.co / repositories-getting-started, arXiv 1810.03993, arXiv 1803.09010
When to retrain Practitioner
Retrain when evidence says so: monitored quality drops, inputs drift, requirements change, or a better base model arrives.
In the long read: Shipping it and keeping it working
Sources papers.nips.cc / 86df7dcfd896fcaf2674f757a2463eba-Abstract, developers.openai.com / evaluation-best-practices, arXiv 2308.08747, arXiv 2310.03693
Where the evidence disagrees
These questions are open or fast-moving. Each link goes to the concept, and the long read sets out both sides with sources.
- Can fine-tuning teach new facts?
- How much data is enough?
- Should the prompt be masked from the loss?
- Does LoRA match full fine-tuning?
- DPO or PPO?
- Does reinforcement learning add new ability?
- Is the KL penalty needed?
- How far can a model judge be trusted?
- How much does quantization cost?
- Is hosted fine-tuning of closed models going away?
Every definition comes from sources fetched on 28 or 29 October 2026. Vendor features, defaults and prices change; check the linked pages before you rely on them. Nothing here is legal advice.
Rather follow one fine-tune from start to finish?