An autoregressive model can receive cleaner context during training than it receives during generation. Under teacher forcing, the next-token prediction is conditioned on a reference prefix from the training sequence. During free-running decoding, the model instead conditions on tokens it generated itself. Once a generated token differs from the intended continuation, later predictions operate on a prefix that training may have represented less often.
This mismatch is commonly called exposure bias. It is not simply a claim that autoregressive models make errors. The specific issue is that the distribution of prefixes presented to the model can change between optimization and generation, and an early deviation can change every subsequent conditional prediction.
Teacher forcing changes the source of the prefix
For a target sequence y_1, ..., y_T, maximum-likelihood training commonly decomposes the conditional log probability as:
log p(y | x) = sum_t log p(y_t | y_<t, x)Each training term uses the reference prefix y_<t. Even if the model assigns its highest probability to a different token at position t - 1, the next training term still receives the reference token at that position.
Generation has a different data path. If the decoder selects ŷ_1, the next distribution is conditioned on ŷ_1. After another selection it is conditioned on ŷ_1, ŷ_2, and so on. The generated prefix becomes part of the input state for later predictions.
The distinction is easy to hide when training loss is reported as a single average. A low next-token loss on reference prefixes does not directly state how the model behaves after its own prefix departs from a reference trajectory.
One deviation changes the conditional problem
Suppose a model is generating a structured expression and the intended prefix is:
SELECT name FROMIf decoding instead produces:
SELECT name WHEREthen the next-token distribution is no longer being queried under the intended prefix. The model must continue from a different syntactic state. A later token that appears incorrect relative to the reference can still be locally consistent with the prefix that the model actually generated.
This makes error attribution less direct than counting mismatched positions independently. The first deviation changes the context for later positions, so downstream differences are not separate trials under the original conditions.
The effect can be mild when nearby prefixes have similar continuation distributions. It can be severe when one token changes syntax, entity identity, numeric state, or another constraint that controls the rest of the sequence.
Sequence length creates more opportunities for prefix drift
Autoregressive generation repeatedly feeds prior outputs back into the model. Longer outputs therefore contain more positions at which the generated trajectory can depart from a reference trajectory. That statement does not imply that quality must decrease monotonically with length; later predictions can recover, and many valid tasks admit multiple continuations.
Still, evaluation based only on one-step predictions under reference prefixes can miss behavior that appears after self-generated context accumulates. The relevant distinction is between evaluating conditional predictions on externally supplied prefixes and evaluating complete trajectories produced by the decoder.
This also separates exposure bias from context-window limits. A model can experience prefix-distribution mismatch well before any context is truncated. Context capacity controls how much prior state can be represented; exposure bias concerns which prior state is presented.
Decoding policy affects which prefixes become inputs
Greedy decoding, sampling, and beam search can expose the same model to different generated prefixes. Greedy decoding commits to the highest-scoring token at each position. Sampling may enter lower-probability branches. Beam search retains several prefixes before selecting among complete or partial hypotheses.
These policies do not change the teacher-forced training distribution by themselves, but they change the inference trajectories that become model inputs. A sampling temperature that spreads probability mass across more tokens can increase the variety of visited prefixes. A restrictive decoder can reduce that variety while still making irreversible early choices.
For this reason, exposure bias cannot be characterized solely from model weights. The decoder determines part of the prefix distribution observed at inference time.
Scheduled sampling alters training prefixes but changes the objective
One proposed response is scheduled sampling, which sometimes replaces reference history with model-generated tokens during training. The intention is to expose the model to prefixes closer to those it may encounter during generation.
That modification is not equivalent to ordinary teacher-forced maximum likelihood. Once sampled model outputs enter the conditioning history, the training procedure operates on a different prefix process. Its behavior depends on how often generated tokens are used, how those tokens are selected, and how the schedule changes over optimization.
The mechanism also introduces a semantic concern: a generated prefix may no longer correspond to the reference continuation used as the next target. If an earlier sampled token changes the valid trajectory, continuing to optimize toward the original next reference token can pair a mismatched prefix with a target derived from another path.
This does not make mixed-prefix training inherently invalid. It means the intervention changes more than input noise and should be evaluated as a distinct training objective rather than treated as a transparent correction.
Sequence-level objectives move evaluation closer to generation
Another family of approaches evaluates or optimizes complete generated sequences using a sequence-level score. This can align optimization with properties that token-level likelihood does not express directly, but it also changes the optimization problem.
A sequence-level objective needs a score that is meaningful for the task. Exact match, constraint satisfaction, edit distance, preference signals, or task-specific utility can emphasize different output properties. None is a generic replacement for likelihood across all applications.
The useful distinction is between two questions. Token-level maximum likelihood asks whether the reference next token receives probability under a supplied prefix. Sequence-level evaluation asks what happens when the model and decoder jointly produce a trajectory. Both can be informative, but they measure different behavior.
Prefix perturbation can isolate recovery behavior
Complete free-running generation combines many effects: initial token uncertainty, decoder choices, accumulated deviations, and stopping behavior. A more targeted diagnostic can deliberately perturb a prefix and inspect subsequent predictions.
For example, an evaluator can compare model distributions after a reference prefix and after a plausible one-token deviation. The comparison can record whether probability mass remains concentrated on continuations compatible with the altered prefix, whether the model returns toward a valid structure, or whether uncertainty expands sharply.
Such perturbations need to respect the task. Replacing a token with an impossible symbol says little about behavior on realistic model errors. Perturbations drawn from plausible alternatives, decoder confusions, or valid neighboring states provide a clearer view of recovery under reachable prefixes.
This diagnostic does not produce a universal exposure-bias score. It separates recovery behavior from ordinary reference-prefix accuracy and makes the changed conditioning state explicit.
Training loss and generation quality remain different measurements
Teacher-forced likelihood is useful because it gives a tractable objective over observed sequences. Its limitation is not that it secretly performs generation incorrectly; it is that the conditioning prefixes are supplied from the data rather than produced by the model.
A deployment evaluation should therefore include the actual decoding path when sequence behavior matters. Token loss can still identify changes in conditional fit, while free-running metrics expose trajectory effects such as invalid structure, premature termination, repeated content, or failure to recover after an early deviation.
Exposure bias is most useful as a description of this train-inference prefix mismatch, not as a catch-all explanation for every generation defect. Decoder scoring, context truncation, data coverage, model capacity, and output constraints can create similar visible failures through different mechanisms. Keeping the prefix source explicit makes those mechanisms easier to separate.