A transformer can produce different continuations without changing its prompt, weights, or decoding settings if an internal activation is modified during the forward pass. Activation steering uses this property as an inference-time control mechanism. A vector representing a target attribute is added to, or subtracted from, a hidden representation at selected model locations.
The operation is simple, but its effect depends on where the vector came from, where it is injected, and how strongly it is scaled. A direction that separates two sets of prompts in one layer is not automatically a portable semantic control across layers, model revisions, or prompt distributions.
A steering vector is a direction in model state
Consider a transformer residual state h_l at layer l. A basic intervention replaces that state with
h'_l = h_l + alpha * v_lwhere v_l is a steering vector defined in the representation space of that layer and alpha is an intervention coefficient. The model then continues its normal forward computation from h'_l.
This differs from logit manipulation. A logit processor edits scores near the output distribution, after the network has computed its internal representation for the current token. Activation steering changes an intermediate state, so later attention and feed-forward computations can respond to the modified representation.
It also differs from parameter adaptation. No weight update is required for the intervention itself. The base parameters remain fixed while the hidden state is changed for a particular inference path.
The vector dimensions must match the activation location being edited. More subtly, the coordinates have meaning only relative to that model state. Copying a vector into another model with the same hidden width does not establish semantic equivalence between the two spaces.
Contrastive activations can define a direction
One common construction starts with paired inputs associated with opposite sides of an attribute. Run both inputs through the model, collect activations at a fixed layer and token position, then subtract one representation from the other.
For a pair with positive activation h_pos and negative activation h_neg, the difference is
v = h_pos - h_negWith several pairs, a simple estimator can average the differences:
v = mean_i(h_pos_i - h_neg_i)The resulting vector encodes the average contrast present in those samples at that measurement location. It should not be treated as a proof that the model stores the attribute as one isolated linear feature. The difference can contain correlated properties from wording, syntax, topic, token position, or other regularities in the contrast set.
Pair construction therefore changes the intervention. If positive prompts are formal and negative prompts are casual while the intended attribute is unrelated to writing style, the difference vector can mix both signals. Matching nuisance properties across the two sides reduces obvious confounding, but it does not guarantee a pure direction.
Averaging also assumes that a shared direction is useful across examples. If individual differences point in inconsistent directions, their mean can be small or can represent a compromise that fits none of the cases well. Inspecting cosine similarity among per-pair difference vectors can expose this condition before the mean is treated as a stable control.
Layer and token position are part of the intervention
Transformer layers do not expose interchangeable state spaces. Early states are close to token and local-context processing, while later states have passed through more model computation. An attribute that is separable at one depth can be weak, entangled, or represented differently at another.
For that reason, v_l should normally be derived and applied at a consistent layer. Moving a vector from layer l to layer k merely because both residual streams have the same width changes the intervention without evidence that the coordinate systems align.
Token position matters as well. A vector measured at the final prompt token summarizes a different computational context from a vector measured at every token. During autoregressive generation, injecting at each generated position repeatedly perturbs the trajectory, whereas a one-time prompt-side edit changes the initial state from which generation proceeds.
These choices can produce visibly different behavior even with the same numeric vector and coefficient. An activation-steering configuration is therefore more accurately identified by the tuple of model revision, hook location, token policy, vector construction, and scale than by the vector file alone.
Steering strength changes more than target intensity
The coefficient alpha controls the magnitude of the injected displacement. Treating it as a simple intensity knob is convenient, but the downstream network is nonlinear. Doubling alpha does not imply twice the behavioral effect.
Small displacements can leave the generated distribution nearly unchanged. Larger values can make the target attribute more frequent, but they can also move the state into regions that are atypical for ordinary model inference. At that point, unrelated output properties can shift: fluency, topical focus, formatting, repetition, or instruction adherence may change along with the intended behavior.
The sign of the coefficient is also meaningful when the vector came from a directional contrast. Positive and negative scaling move the state in opposite directions, but symmetric numeric displacement does not guarantee symmetric behavioral response. Later transformer blocks can map the two perturbed states through different nonlinear regimes.
A useful evaluation therefore varies alpha across a bounded range instead of reporting one hand-selected value. The relevant curve contains both target behavior and collateral metrics. A coefficient that strongly changes the target but also damages unrelated capabilities represents a different operating point from a weaker intervention with smaller side effects.
Activation control is model-revision specific
A steering vector depends on internal representations rather than a public input-output contract. Fine-tuning, quantization, architecture changes, or even a different checkpoint from the same model family can alter those representations.
Quantization deserves separate care. An inference engine may expose hidden states in a floating-point accumulation format even when weights use lower precision, or it may fuse operations in a way that makes the desired hook unavailable. The mathematical intervention can remain well defined while a particular runtime cannot expose the exact state needed to apply it.
Compiler and serving optimizations can create a similar boundary. A framework hook used during experimentation may prevent graph capture or kernel fusion, adding latency that is not inherent to vector addition itself. Production cost should therefore be measured with the actual interception mechanism rather than inferred from the arithmetic count of the steering operation.
Persisted vectors also need metadata. Model identifier, exact revision, layer, activation location, token selection rule, preprocessing assumptions, and scale convention are part of the artifact. Without them, a vector can have the correct shape and still be applied to the wrong representation.
Evaluation needs an unsteered reference
The most direct comparison uses the same model, prompts, and decoding policy with the intervention disabled. This isolates the effect of the activation edit from changes in sampling temperature, prompt wording, or model revision.
Target measurements should match the intended behavior. Collateral measurements should cover properties that the intervention is not supposed to change. For structured output, that can include schema validity. For question answering, it can include answer accuracy on prompts unrelated to the steering attribute. For free-form generation, task-specific checks are preferable to assuming that one generic score captures every side effect.
Prompt diversity matters because a contrast-derived direction can fit its source examples closely. Evaluation prompts that reuse the same templates can overstate generality. Separating vector-construction prompts from evaluation prompts gives a clearer signal about whether the direction transfers beyond its extraction set.
Activation steering is most useful when treated as a controlled intervention on a specific internal state, not as a universal semantic slider. The vector, injection site, scale, and evaluation distribution form one mechanism. Changing any of them can change the behavior being controlled, even when the base model weights remain untouched.