Model Interpretability

Steer Language Models by Editing Hidden Activations

Prompting changes what a language model reads. Fine-tuning changes its parameters. There is another, more experimental way to influence generation: change the model’s internal activations while it runs. This technique is commonly called activation steering or activation engineering. A simple version measures how hidden representations differ between examples that express opposite properties, turns that difference into a steering vector, and adds a scaled version of the vector during inference. The model weights stay unchanged.