AI Dojo Open on the typewriterThe typewriter

Explainer

Synthetic data

1 min read
Assumes: you know what fine-tuning is.
First published 29 October 2026. Last edited 16 November 2026.

Training examples written by a model instead of a person.

Synthetic data is training data a model writes, from questions to whole conversations, which you then filter. It is fast and cheap: Alpaca built 52,000 instruction examples from 175 human-written seeds for under $500 of API cost (crfm.stanford.edu). It also inherits the generating model's blind spots and errors, and filtering matters: one study kept about 9,000 of Alpaca's 52,000 examples and got better results (Chen et al. 2023, arxiv.org). Check the generating model's terms before you train on its output.

Going deeper

Training models over and over on model-made data can make rare patterns fade, called model collapse (Shumailov et al. 2023, arxiv.org); keeping the real data alongside avoided it in one set of experiments (Gerstgrasser et al. 2024, arxiv.org). Keep real data in the mix, spot-check synthetic examples, and record where each one came from.

In the long read: The data decides what it learns. Every related term is in Fine-tuning, every concept.