Representations

Steer Transformer Activations with Residual Stream Vectors

A transformer can produce different output behavior even when its weights and input tokens stay fixed. One way to cause that change is to alter an intermediate hidden state during the forward pass. Activation steering does this deliberately by adding a vector to a selected residual-stream position or set of positions. The mechanism is simple enough to express as an intervention, but its effect is not a global model setting. The chosen direction, coefficient, layer, token positions, and decoding setup all affect the result. Treating those choices as part of the inference configuration makes the behavior easier to reason about and test.