Explainer
Loss masking
Choosing which tokens count toward loss, the training prediction penalty.
A token is a piece of text the model reads or predicts. Training scores how well it predicts the example's next tokens using a penalty called loss. Lower loss means a better fit to those tokens when the data and scoring setup are the same. It is not a percentage, and it does not check whether an answer is true or useful. The training section works through the numbers; evaluation checks the task itself (TRL docs).
Loss masking chooses which tokens count toward that penalty. In assistant-only training, the reply and its end-of-turn marker count, while the prompt does not. The model still reads the prompt as context. Scoring the marker teaches the model when the answer should end. Other setups also score the prompt; assistant-only scoring is a choice for the task, not a universal rule (TRL docs).
Going deeper
Contested: masking is a sensible default, not a law. Also scoring the instruction helped across 21 benchmarks in one study, most when instructions were long, outputs short and examples few (Shi et al. 2024, arxiv.org). If you rely on assistant-only loss, check how many tokens are actually scored before trusting a run.
In the long read: What a training example looks like. Every related term is in Fine-tuning, every concept.
Read next: Held-out test set