HiMA-Ecom matched DeepSeek-R1 with 3B and 7B models on a 22,800-instance hierarchical multi-agent memory benchmark, and cut convergence steps about 55 percent
AspireAI community · arXiv (2506.19846)
Long-horizon multi-agent systems become long-context systems by accumulation: every step reassembles a prompt from role instructions, tool descriptions, recalled memories, dialogue history and prior tool outputs, separately for each agent. HiMA-Ecom measures that on 22,800 e-commerce-assistant instances, 17,700 of which carry memory entries in context, and pairs agent-level fine-tuning data with system-level supervision for end-to-end reinforcement learning. On top of it, HiMA-R1 keeps a trajectory's context budget in check with Variance-Reduction Group Relative Policy Optimization, sampling along an initial trajectory so cost grows additively with length and updating the agents with the largest reward variance. The authors report that agents built on 3B and 7B open-source models match DeepSeek-R1 and beat DeepSeek-V3 by 6.0 points on average, while reward-driven memory cuts the update steps needed for convergence by roughly 55 percent. The comparison is on the authors' own e-commerce benchmark, and this is a NeurIPS workshop paper rather than a main-track publication.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.