Hypergraph memory for multi-agent debate cut memory use 48 percent and debate rounds 34 percent while raising accuracy
AspireAI community · arXiv (cs.CL)
HEDGE stores multi-agent debate trajectories as semantic hypergraphs, where hyperedges connect related claims across agents and relational graphs inside each hyperedge preserve the fine-grained argumentative structure - capturing multi-agent agreement, conflicting evidence and evolving consensus that flat and pairwise-graph memories lose. A learned memory selection module, trained through supervised learning and reinforcement learning, routes each candidate to store, back up or omit. Across seven diverse benchmarks it consistently outperformed strong single-agent, multi-agent and memory baselines, improving accuracy by 5.0 points on average over the strongest multi-agent system built on GPT-4o-mini and reaching 8.2 points on conflict-heavy tasks, while reducing memory usage by 48% and the number of debate rounds by 34%. The limitation: the baseline runs on GPT-4o-mini, so the absolute numbers are not directly comparable to frontier-scale systems.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.