Patching a reasoning model's chain of thought in its first three tokens took fully safe outputs from 29 to 82 percent
AspireAI community · arXiv (Hanwen Zhang, Enyi Jiang, Ruike Zhu et al., Stanford University / UIUC)
The safest place to intervene in a reasoning model is not obvious. Later hidden states may be more predictive of risk, but the authors ask a different question: when is a risk easiest to detect, and does that point also offer the most effective control?
REACT learns a safety direction from paired safe and unsafe trajectories and applies it once, as a conditional activation intervention, to a single early chain-of-thought hidden state. Model weights stay frozen and every subsequent token is generated with no further edits. To study timing, the authors trained directions separately at each of the first five positions and regenerated fully after each ungated edit.
The answer is that detection and control peak at different times. Across three distilled reasoning models and three generation seeds, the strongest gains concentrate in a two-to-three-token early window. On DeepSeek-R1-Distill-Llama-8B the effect falls to approximately zero at the fourth and fifth tokens, even though later states remain more predictive of risk. On that same model, REACT raised the rate of fully safe chain-of-thought-answer outputs on StrongREJECT-273 from 29.06 percent to 82.05 percent, with benign refusal and usefulness near baseline.
Caveat: measured on distilled 8B reasoning models and one 273-item safety benchmark, with an anonymized implementation.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.