Explainer
GRPO and verifiable rewards
Reinforcement learning that scores each answer against other answers to the same prompt.
When a program can check an answer, such as code against tests or a maths answer against a key, its score can supply the reward. This is reinforcement learning with verifiable rewards. Group relative policy optimization (GRPO) generates several answers to one prompt and compares their scores with their group's average. Above-average answers get a positive learning signal; below-average answers get a negative one. The comparison follows the chosen scorer, which may be imperfect; reward is not independent proof of truth (Shao et al. 2024).
Illustration: four answers score 1, 0, 0 and 1. Their average is 0.5. Using the population standard deviation as the spread gives 0.5. Subtract the average from each score and divide by that spread to get the relative signals. Implementations can use different scaling conventions; the example shows the direction and relative comparison.
If all four scores are equal, none is above or below the average. The relative reward signal is then zero for every answer; implementations handle the zero spread rather than dividing by zero. There is no reward preference to learn from that group. Other terms, such as an enabled KL penalty, can still contribute to updates (TRL docs). DeepSeek-R1-Zero used GRPO without a supervised fine-tuning stage first and rose from 15.6% to 77.9% on the AIME 2024 maths problems during training (DeepSeek-AI 2025).
Going deeper
DeepSeek-R1-Zero, trained with RL straight from a base model, rose from 15.6% to 77.9% on AIME 2024 during training (arxiv.org). As of 28 October 2026, TRL 1.16.0's GRPO defaults to 8 answers per prompt and no KL penalty (huggingface.co), and Amazon Bedrock's reinforcement fine-tuning trains with GRPO (docs.aws.amazon.com). A checker can still be gamed if its tests are incomplete.
In the long read: Beyond examples: preferences and rewards. Every related term is in Fine-tuning, every concept.
Read next: Reward hacking