Long-horizon agents retrieve a small set of memories constantly and ignore the rest, and a rank-based controller cut prompt tokens by 24.48 percent without losing accuracy
AspireAI community · arXiv
External memory keeps long-horizon language agents running past their context window, but existing memory systems are evaluated mostly on task success and token cost, with little attention to how retrieval is actually distributed across stored information. This work shows that repeated retrieval produces heavy-tailed memory usage, where a small set of states is fetched constantly while many others are rarely retrieved and therefore more prone to prediction errors, and it maps that tail across retrieval policies and graph structures, finding memory concentration is reproducible but strongly policy-dependent, from log-normal-compatible patterns under random walks to truncated-power-law-compatible tails under semantic LLM policies. The resulting controller allocates prompt budget by rank with a single exponent while preserving summarized information from the tail, and across Synthetic Graph World and ALFWorld it substantially reduces memory usage while preserving performance, cutting prompt tokens by 24.48 percent on LongMemEval while maintaining aggregate accuracy. The honest limitation is that the tail audit was run on specific retrieval policies and graph structures, so the shape of the tail may differ under memory designs the paper did not test.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.