Sequence Generation

Understand Beam Search for Sequence Generation

A model that generates text or another sequence makes a series of local decisions. At each step, it assigns scores or probabilities to possible next tokens. The simplest decoder chooses the most likely token, appends it, and repeats. That strategy is called greedy decoding. It is cheap and easy to understand, but an early choice that looks best by itself can lead to a worse complete sequence. Once greedy decoding commits to that choice, it cannot reconsider it.