A decoder-only transformer normally exposes token logits only after its final block and output normalization. The residual stream inside earlier blocks has the same model-width shape, which makes another operation possible: take an intermediate state, apply the model’s output-side normalization when required, and project that state through the output matrix.
The resulting vocabulary scores form the logit lens. They provide a token-space view of an internal representation before the remaining transformer blocks have processed it. That view is useful for inspecting how candidate tokens change across depth, but it is not a record of tokens that the model has secretly selected in advance.
The output projection creates a common coordinate system
Consider a transformer with residual state h_l after layer l. A simplified final prediction path can be written as:
z = W_U * norm(h_L) + b
p = softmax(z)W_U maps the final hidden representation to vocabulary logits. Some architectures omit b, and normalization placement varies by model family.
A basic logit-lens probe reuses the same output mapping at an earlier layer:
z_l = W_U * norm(h_l) + bThis turns every selected layer into a vector over the same vocabulary. Developers can then compare ranks, logit gaps, or selected token scores across depth without fitting a separate classifier for each layer.
The shared projection is the main attraction. A hidden dimension has no direct human-readable token label, while the vocabulary already supplies a familiar coordinate system. Reusing the output head also avoids adding a probe with independently trained parameters.
Intermediate logits are counterfactual readouts
The model does not normally emit z_l. It sends h_l through later attention and feed-forward blocks before producing final logits. Projecting h_l early therefore asks a counterfactual question: what token preferences are visible if the final output map is applied at this point?
That distinction prevents an easy interpretability error. Suppose the top projected token at layer 12 is Paris, while the final output favors London. The layer-12 result does not establish that the model made a Paris decision and later reversed it. Later blocks can rotate, add, suppress, or otherwise transform features in the residual stream. The early projection only reports how the current state aligns with output-token directions.
The same caution applies when a token rank rises smoothly across layers. A monotonic trajectory is an observable pattern in the readout, not proof of a single internal feature accumulating unchanged from block to block.
Normalization placement changes the readout
Applying the output matrix to raw intermediate states can differ materially from applying the model’s final normalization first. In architectures with a final LayerNorm or RMSNorm, the normal prediction path includes that transformation. A probe that omits it no longer matches the output path being approximated.
For a model whose final path is:
h_L -> final_norm -> output_heada closer layerwise readout is:
h_l -> final_norm -> output_headThis still introduces an intervention in the sense that final_norm is being applied earlier than it is during ordinary inference. The operation is useful for comparability, but its scores should not be described as native intermediate logits unless the architecture actually computes them there.
Implementation details also matter around tied embeddings. Some models use the token embedding matrix as the output projection, while others use a separate matrix. A correct probe should use the model’s actual output head rather than assume weight tying.
Token rank is often more stable than raw probability
Applying softmax to intermediate logits produces numbers that sum to one, but that does not make them calibrated next-token probabilities at that layer. The remaining blocks are absent from the counterfactual path, and intermediate logit scales can differ across depth.
For inspection, token rank and logit differences can be easier to compare than softmax probabilities. If two candidate tokens a and b have logits z_a and z_b, their difference:
delta = z_a - z_bshows which candidate the readout favors without depending on the rest of the vocabulary normalization. Tracking delta across layers can expose where a preference becomes visible or changes sign.
Even this measurement has a boundary. A larger gap at an intermediate layer does not guarantee that later computation will preserve the preference.
Position selection changes the question
Transformer hidden states exist at every sequence position. A logit-lens analysis therefore needs an explicit position, not just a layer.
For next-token generation, the final prompt position is often the relevant state because its final representation feeds the next-token logits. Probing earlier positions asks a different question about representations attached to those token locations. In a causal transformer, later positions can attend to earlier positions, while earlier positions cannot incorporate later tokens from the same sequence.
Comparisons should keep position semantics fixed. Mixing the final prompt position from one example with an arbitrary internal position from another can create trajectories that look comparable while representing different prediction contexts.
Tokenization also affects interpretation. A vocabulary entry may represent a whole word, a word fragment, punctuation, whitespace-prefixed text, or bytes depending on the tokenizer. A projected token label should be interpreted as the model’s token unit, not automatically as a linguistic word.
The lens can expose where a token preference becomes readable
A practical use of the logit lens is to locate depth ranges where specific output candidates become linearly readable through the existing output head. For a fixed prompt and position, a compact trace might record the ranks of a few relevant tokens:
layer 4: token A rank 812, token B rank 1050
layer 12: token A rank 47, token B rank 190
layer 20: token A rank 6, token B rank 31
layer 28: token A rank 2, token B rank 9
final: token A rank 1, token B rank 7Such a trace supports a narrow statement: the output projection can read token A increasingly strongly from those intermediate states. It does not identify which attention head or feed-forward component caused the change.
That narrower claim is still useful. If a behavior appears only in late-layer readouts, an investigation can focus on the corresponding depth range. If a target token is already highly ranked early but disappears later, the later transformation becomes the more relevant region to inspect.
Readability and causal influence are different properties
The logit lens is observational. It maps an existing activation to vocabulary space without showing that the visible direction causes the final output.
A feature can be readable from h_l yet have little causal effect if later computation removes or ignores it. Conversely, an internal feature can strongly affect the final prediction without aligning cleanly with a single vocabulary direction at the probed layer.
Causal questions require interventions. Activation patching, ablation, controlled replacement, or other perturbations can test whether changing a component changes downstream behavior. Those methods answer a different question from the logit lens and can be paired with it: the lens identifies a readable pattern, while an intervention tests whether a selected internal state or component matters to the output under specified conditions.
Cross-model comparisons need architectural care
Layer numbers are not directly comparable across models with different depths. Layer 12 in a 24-block model sits at a different relative depth from layer 12 in an 80-block model. Output normalization, vocabulary, tokenization, embedding tying, and residual scaling can also differ.
Even within one model family, comparing raw logit magnitudes across checkpoints can be misleading if parameter scales or normalization behavior changed. Relative depth, token rank, and within-model logit differences often provide clearer comparison targets, provided the token identities themselves are compatible.
A logit lens is most defensible when treated as a projection with explicit assumptions: a chosen residual state, a chosen position, the correct output transformation, and a stated comparison statistic. Its value comes from making internal token preferences inspectable. Its limit is equally concrete: a readable projection is evidence about representation alignment, not a complete account of the computation that produced the final token.