repetition penalty
In plain wordsA setting that discourages an AI model from repeating itself by lowering the chances of words it has already used, so its text does not get stuck in loops.
A decodingdecoding, n.: The method a language model uses to turn its predictions into actual text — for example, always picking the most likely next word, or choosing somewhat at random among the likely ones. adjustment that lowers the scores of tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. that have already appeared, to counter the tendency of language modelslanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. to fall into repetitive loops, especially under greedy decodinggreedy decoding, n.: The simplest way for a language model to write: always pick the single most likely next word. It is fast and predictable but can produce dull or repetitive text. or low temperaturetemperature, n.: A setting that controls how adventurous an AI model's word choices are. Low temperature makes it pick the safest, most likely words; high temperature makes it more varied and surprising, but more error-prone.. In the multiplicative form introduced with CTRL, the logitslogit, n.: A raw score that a model gives each possible answer before the scores are turned into probabilities. In statistics, the 'log-odds': a probability rewritten on a scale running from minus to plus infinity. of previously generated tokens are divided by a factor θ (about 1.2 was recommended); because dividing a negative logit would raise it, common implementations multiply negative logits by θ instead. API-style variants subtract from logits instead: a presence penalty applies a fixed amount to any token already present, and a frequency penalty grows with the number of occurrences. Other remedies block exact repeats of n-grams or penalize extending a repeated sequence. Penalties are blunt: they also discourage legitimate repetition such as names, code identifiers and list structure, and high settings hurt fluency and accuracy.
A presence penalty of 0.6 stopped the summaries from restating the same point three times.
Origin
The multiplicative repetition penalty was introduced in CTRL by Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong and Richard Socher of Salesforce (September 2019); OpenAI's GPT-3-era API offered additive frequency and presence penalties. Blocking repeated trigrams goes back to abstractive-summarization work of 2017.
Related
- decodingdecoding, n.: The method a language model uses to turn its predictions into actual text — for example, always picking the most likely next word, or choosing somewhat at random among the likely ones.
- temperaturetemperature, n.: A setting that controls how adventurous an AI model's word choices are. Low temperature makes it pick the safest, most likely words; high temperature makes it more varied and surprising, but more error-prone.
- top-p samplingtop-p sampling, n.: A way of making AI-written text varied but sensible: the model picks at random among the smallest group of likely next words whose chances add up to a set share, such as 90%.
- logit biaslogit bias, n.: A setting that lets developers make specific words more or less likely in an AI model's output — even ban a word outright — by adjusting the model's scores before it chooses.
- logitlogit, n.: A raw score that a model gives each possible answer before the scores are turned into probabilities. In statistics, the 'log-odds': a probability rewritten on a scale running from minus to plus infinity.
Sources
- Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C. & Socher, R. (2019). CTRL: A Conditional Transformer Language Model for Controllable Generation. arXiv:1909.05858. arxiv.org/abs/1909.05858
Mentioned in
decodingdecoding, n.: The method a language model uses to turn its predictions into actual text — for example, always picking the most likely next word, or choosing somewhat at random among the likely ones., top-p samplingtop-p sampling, n.: A way of making AI-written text varied but sensible: the model picks at random among the smallest group of likely next words whose chances add up to a set share, such as 90%., logit biaslogit bias, n.: A setting that lets developers make specific words more or less likely in an AI model's output — even ban a word outright — by adjusting the model's scores before it chooses.