Open-book scores mostly measured reading, not knowledge: a random adapter recovered 82 percent of a specialist's advantage but 2 percent of the real thing
AspireAI community · arXiv (Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp, LMU Munich / Munich Center for Machine Learning)
Document-level LoRA adapters are proposed as units of knowledge that can be merged and retrieved. This paper asks whether the usual way of scoring them, open-book, actually measures knowledge.
Training one adapter per document for 150 Wikipedia articles on Llama-3.1-8B-Instruct, the authors find that open-book scores mostly measure a reading behaviour every adapter shares. A random adapter trained on a different document recovers 82 percent of the specialist's open-book advantage over the base model, but only 2 percent of its advantage on a cloze probe that asks for the same document's facts without providing the source text, a share the authors cannot distinguish from zero.
Content does reach the weights: the specialists beat the base model in cloze recall by 0.094, and the geometry of their updates follows the documents they read. But adapters selected by weight-space proximity or text embedding were not distinguishable from random ones on either probe. At 16 documents or more, correctly scaled merges recovered 87 to 89 percent of the specialists' open-book advantage while recovering only 12 to 16 percent of their cloze advantage, against 94 percent for joint training.
Caveat: the authors report four silent errors, each producing a clean, wrong result, and one test cannot exclude an advantage of about four fifths of a query's own adapter gain.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.