Explainer
Temperature and sampling
How a model picks one word from many possibilities, and the setting that makes it more or less adventurous.
At each step a language model gives a probability to every possible next token. Sampling means drawing one at random, weighted by those probabilities, so likely tokens usually win and unlikely ones sometimes do. Temperature reshapes the odds before the draw. Low temperature makes the favourite win nearly every time, which suits tasks with one right answer. Higher temperature spreads the chances out, which gives more varied wording and ideas, and more chance of drifting off course. Most chat apps set these for you; developer tools usually let you change them. Sampling is the main reason the same question can produce different replies.
Going deeper
Holtzman et al. (2019) showed that decoding which aims for the most likely text tends to produce bland, repetitive output, and proposed nucleus (top-p) sampling, which draws only from the smallest set of tokens whose probabilities add up to at least p (arxiv.org).