A causal transformer does not have to expose every earlier token to every query position. With sliding-window attention, position i can attend only to a bounded range of preceding keys. Tokens outside that range are absent from that layer’s attention operation, even if they still belong to the model input.
That boundary changes more than the attention matrix shape. It separates direct access from information that can reach a position only after being propagated through intermediate hidden states.
The mask defines direct access
For a causal window of width W, a common conceptual mask permits position i to attend to positions satisfying:
max(0, i - W + 1) <= j <= iExact endpoint conventions can differ across implementations, so W must be interpreted according to the model and serving stack in use. The core property is the bounded span: keys sufficiently far behind i receive no attention weight at that layer because they are masked out before the attention result is formed.
This is different from assigning a very small probability to an old token. A masked key is excluded from the permitted attention set. Changing softmax temperature or sampling parameters does not restore access to it.
Local layers can still propagate distant information
A local window does not imply that the final hidden state depends only on the last W input tokens. Stacked layers can relay information forward.
Suppose each layer permits a token to read a limited region behind it. A hidden state near the right edge can contain information gathered by an earlier layer from positions farther left. A later layer can then read that hidden state. The effective receptive field can therefore grow with depth even though each individual attention operation remains local.
This distinction matters when inspecting model behavior. Direct attention edges describe which states a layer can read immediately. They do not, by themselves, describe the full set of input positions that may have influenced a deep representation.
The amount and fidelity of such propagation are model properties, not guarantees implied by the mask. A distant fact being inside the theoretical receptive field does not guarantee that the network preserves or uses it.
Window size and context length are separate limits
A model can accept a sequence longer than one local attention window. Context length determines which token positions can be supplied under the model’s positional and runtime constraints. The local attention rule determines which of those positions are directly visible to a given query in a given layer.
For example, an input may contain thousands of positions while a local layer reads only a bounded suffix for each query. Both statements can be true at once. Treating the advertised context length as the direct attention span can therefore misrepresent the architecture.
Architectures may also mix local and global patterns. Some layers can use bounded attention while other layers expose a wider range. In that case, receptive-field analysis has to follow the actual layer schedule rather than applying one window value to the entire network.
The KV cache can have a bounded live region
During autoregressive decoding, full causal attention commonly retains keys and values for all prior positions needed by future queries. A strictly local layer has a different requirement. Once a cached position falls permanently outside every future query’s permitted window, that layer no longer needs the position for its own attention computation.
Conceptually, if the next query can read only the most recent W positions, the live KV region for that layer can remain bounded by the window rather than growing with the complete generated sequence.
The exact memory behavior is implementation-specific. A serving engine may allocate cache in blocks, retain extra positions for alignment, combine local and nonlocal layers, or use a cache layout whose physical allocation does not shrink exactly at the logical mask boundary. Architectural eligibility for eviction and actual allocator behavior are separate concerns.
Mixed attention makes this especially visible. A local layer may be able to discard old KV state while a full-attention layer in the same model still requires state from the complete active context. Total cache consumption therefore cannot be inferred from the local window alone.
Truncating input is not equivalent to local attention
Removing old tokens from the input changes the computation from the first layer onward. Keeping those tokens in the sequence while masking direct access in later positions is different: their earlier hidden states can participate in local propagation before they move outside a later query’s window.
This also separates local attention from application-level prompt trimming. Prompt trimming deletes positions before model execution. Sliding-window attention keeps positions subject to the model’s context rules but constrains attention edges according to position and layer.
As a result, replacing a local-attention model with manual truncation is not a semantics-preserving transformation. The two procedures can expose different hidden-state paths even when the final query sees the same number of recent raw tokens.
The useful boundary is architectural, not textual
Sliding-window attention operates on model positions and cached states, not on paragraphs, messages, or semantic units. A window can cut through a sentence or a structured record because its boundary follows token positions.
For systems that depend on distant evidence, the relevant question is therefore whether the architecture provides a computational path from that evidence to the output position. Direct local access, propagation through stacked layers, occasional wider-attention layers, and application-side retrieval are different paths with different constraints. A large accepted context does not make those paths interchangeable.