A transformer can expose an intermediate residual state that strongly favors a token under the model’s final vocabulary projection, then produce a different token after later blocks run. The logit lens makes that intermediate preference visible. It does not establish that the preference caused the final output.
That boundary matters when developers use layer-by-layer token rankings to inspect model behavior. The logit lens is a readout: it asks what the model’s output head would report if applied to an intermediate representation. The actual forward pass asks a different question because every remaining block can transform that representation before the final readout.
The readout reuses the final output map
Consider a decoder-only transformer with residual state h_l after layer l. A simplified final prediction can be written as:
logits = W_U norm(h_L)where h_L is the final residual state, norm is the model’s final normalization operation, and W_U is the vocabulary projection or unembedding matrix. Bias terms can be included when the architecture uses them.
A basic logit-lens readout substitutes an intermediate state:
lens_l = W_U norm(h_l)The resulting vector can be ranked exactly like ordinary logits. This makes token-space inspection convenient because every inspected layer is expressed in the same vocabulary coordinates.
The substitution is not part of the model’s normal computation. Intermediate states are inputs to later transformer blocks, not direct inputs to the final output head during the ordinary forward pass. Reusing the final normalization and unembedding therefore imposes a particular interpretation on h_l.
A high intermediate logit is not an additive credit
Suppose token A has a large lens logit at layer 12 and remains the final top token at layer 24. It is tempting to describe layer 12 as having produced the answer. The observation supports a narrower statement: the layer-12 residual state is readable as favoring A under that lens.
Later blocks may reinforce the relevant direction, suppress competing directions, rotate or rescale features, or add information needed for the final decision. The same final token can survive these changes while its internal support changes substantially.
The reverse case is equally informative. A token can rank highly at an intermediate layer and disappear later. That does not make the earlier readout erroneous. It shows that the state at that depth and the final state map differently through the chosen output projection.
Logits also pass through softmax only after the final vocabulary scores are formed. Differences in intermediate logits can be useful for comparing candidates, but they should not be treated as independent probabilities contributed by individual layers.
Normalization is part of the measurement
Applying the final normalization to each intermediate state is more than a formatting detail. Layer normalization or RMS normalization changes scale before the vocabulary projection, and architecture-specific placement determines which tensor the real output head receives.
For a model with a final normalization module, omitting that operation from an intermediate readout can make the lens inconsistent with the final readout path. Applying it, however, still does not make the intermediate state a state that the model was required to decode directly at that depth.
Implementation details also matter. Libraries can expose hidden states before or after particular block operations, and a model family may use pre-normalization, post-normalization, or additional transformations around the output head. A layer index has little analytical value unless the captured tensor is identified precisely.
Layer trajectories are descriptive evidence
The strongest use of a basic logit lens is comparative. For a fixed prompt and token position, developers can inspect how a small set of candidate logits or logit differences changes with depth.
A trajectory can reveal that a candidate becomes readable early, appears only near the end, alternates with a competitor, or loses support after a particular region of the network. These are observations about representation dynamics. They can direct attention toward layers worth examining with stronger methods.
Comparisons should keep the token position, output projection, normalization convention, and candidate definition fixed. Changing several of these at once makes a layer-to-layer curve difficult to interpret.
Top-token displays can also hide useful movement. If two candidates retain the same rank while their logit difference changes sharply, a top-1 trace will miss that change. Tracking selected logit differences often gives a more stable view of a specific hypothesis than recording only the winning token.
Readability and causal use require different tests
A representation can contain information that a probe can decode even when the remaining network does not rely on that information for the behavior being studied. This is a general probe limitation, not a special defect of the logit lens.
Causal claims require an intervention or another analysis tied to the model’s computation. Activation patching, ablation, or controlled replacement can test whether changing an internal state changes the output under defined conditions. Such tests have their own assumptions, but they ask a causal question that a passive vocabulary projection does not.
This distinction also prevents overinterpreting abrupt lens changes. A sharp transition between adjacent layers can identify a region associated with a representational change. It does not, on its own, isolate the component responsible. Attention and feed-forward sublayers, residual additions, normalization, and later interactions can all affect the observed readout.
The logit lens is most useful when its output is described literally: an intermediate residual state has a particular vocabulary-space readout under a specified projection. That phrasing preserves what the measurement actually shows and leaves causal attribution to methods designed to test it.