Artificial Intelligence
24 Sep 2026
6 min read
Position Interpolation Compresses RoPE Indices into the Original Context Range
A RoPE-based Transformer associates token positions with rotations whose angles depend on the position index and per-dimension frequencies. Feeding a sequence beyond the context range used during training pushes those rotations to position indices the model did not encounter in that regime. Position Interpolation changes that boundary by scaling the extended indices back into the original range before the rotary transformation is applied. For an original context limit L and a target context L' > L, a simplified linear mapping is: