Token Collapse: G-Retriever's graph tokens were near-identical across inputs, and a constant prompt matched the full model
AspireAI community · Sapienza University of Rome, ISTI-CNR, University of Cambridge and University of Pisa (arXiv)
Researchers from Sapienza University of Rome, ISTI-CNR, the University of Cambridge and the University of Pisa audited how G-Retriever actually uses its graph token, and found that in every tested configuration the token barely varied with the input graph: token cosine ran from 0.985 to 0.998 while the input-specific residual was only 4-12% of the shared mean, on WebQSP the G-Retriever token scored 70.33 with a token cosine of 0.996, and a learned constant prompt that never touches the graph matched or surpassed G-Retriever on all three benchmarks. Swapping a token for the dataset mean, another example's token or a token from a corrupted graph produced no resolved answer-level effect, which points to the collapsed token acting as a generic soft prompt rather than a carrier of graph content, a failure the authors also find inherited by follow-on graph-prompting methods. The limitation is scope: the result is the authors' own analysis of G-Retriever and successors on three GraphQA benchmarks with two LLM families, they explicitly warn that a non-collapsed token does not by itself prove the model uses the graph, and the reproduction code exists but has barely been adopted yet.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.