Controlling Language Model Generation with NVIDIA’s LogitsProcessorZoo
Generating text with language models often involves selecting the next token based on a distribution of probabilities.
A straightforward approach like greedy search selects the most probable token, but this can result in generic or repetitive outputs.