A frozen model with a random bias added to its output logits beat a full weight-perturbation ensemble on MATH-500, 81.3 percent to 74.7 percent, with a five-times smaller search budget
AspireAI community · arXiv (Kyutai)
The Neural Thickets hypothesis says pretrained models sit in dense neighbourhoods of task-specific experts in weight space, and that random weight perturbations can uncover them. The authors reproduce those gains, then remove the geometric explanation. Perturbing a single decoder block already recovers 85 to 89 percent of the gain, and keeping the model entirely frozen while adding a random bias to its output logits matches or exceeds full-weight perturbation on most reasoning tasks. On OLMo-3-7B-Instruct this 'logit PSV' variant lifts MATH-500 from 63.3 percent to 81.3 percent, against 74.7 percent for the full-weight ensemble, and lifts GSM8K from 87.5 to 91.9 percent. The apparent advantage is not only scale: across 0.5B to 7B models logit PSV is more accurate on 13 of 15 reasoning cells and within 0.6 points on the other two, and at a matched budget the two methods are statistically indistinguishable on 22 of 28 model-task pairs. A first-order analysis explains why - weight perturbations act mainly through their induced logit shifts - and the paper frames the whole effect as a familiar trio: output diversity, validation-based selection, and voting, which is self-consistency with a better selection step. The limitation is honest: on USPTO the frozen variant wins by predicting the majority reaction class more often, and its balanced accuracy is actually lower, 26.7 percent against 29.7 percent.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.