A measure of what a unit of weight change buys put SFT five times above random, RL at 400 to 1,500 and DPO at 400 to 23,000, and found pretraining windows no more consequential than random directions
AspireAI community · arXiv (Shikhar Shiromani)
Post-training moves a language model's weights by about a thousandth of their norm and changes behavior sharply, while pretraining moves them by a third and buys a slow drift - but there has been no way to compare those moves on one scale. This work introduces leverage: the Fisher quadratic form of an update, computed with one forward-mode pass per direction, divided by the same form for random directions of the same size at the same checkpoint. Ordering released Olmo-3, Tulu-3, Zephyr and OLMo-2 checkpoints over four orders of magnitude, leverage puts pretraining windows at the random level on prose, SFT five times above it, RL-Zero and RLVR at 400 to 1,500, and DPO at 400 to 23,000. The finding that pretraining accumulates into noise - windows of 30k to 1M steps are no more consequential on prose than random directions of their size - matches a noisy quadratic model of SGD. Training precision changes what an update looks like too: bf16 parameters without an fp32 master copy freeze 36 to 99 percent of coordinates and lower leverage up to sevenfold. Controls support the readings: two re-runs of a released DPO stage from its anchor land within 1 percent of the released displacement and 4 percent of its leverage. The limitation is that leverage is computed on prose, and the instrument is still new enough that only four checkpoint families have been measured.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.