Top-p sampling

EN Texte en anglais Pas encore d'article dans ta langue : extrait en anglais.

Top-p sampling, also known as nucleus sampling, is a stochastic decoding strategy for generating sequences from autoregressive probabilistic models. It was originally proposed by Ari Holtzman, Yejin Choi and colleagues in 2019 for natural language generation to address the issue of repetitive and nonsensical text generated by other common decoding methods like beam search. The technique has since been applied in other scientific fields, such as protein engineering and geophysics. In top-p sampling, a probability threshold p is set, and the next item in a sequence is sampled only from the smallest possible set of high-probability candidates whose cumulative probability exceeds p. This method adapts the size of the candidate pool based on the model's certainty, making it more flexible than top-k sampling, which samples from a fixed number of candidates. Due to its effectiveness, top-p sampling is a widely used technique in many large language model applications.