Hedge-Bench gave 11 frontier models 92 real forecasting environments, and the best joint success rate was 31.52 percent
AspireAI community · arXiv (Hedge-Bench authors)
*Agents are increasingly good at routine financial analysis - pulling documents, updating spreadsheets - but forecasting an unknown future from incomplete information is a harder problem, and public markets make it measurable because analysts routinely predict discrete events with only the information available at that time.* Hedge-Bench is a benchmark of 92 real-world environments derived from hedge fund analysts' reasoning traces paired with subsequently resolved prediction questions. Agents reason from frozen information snapshots and are scored against real outcomes.
Across 11 frontier models with three trials per environment, joint success rates - requiring both a correct forecast and over 60 percent research coverage - range from 2.54 to 31.52 percent. Using GRPO with turn-level credit assignment, the authors train Muse-Glimmer on 165 environments, raising coverage on 22 held-out environments from 17.17 to 33.36 percent at low reasoning effort and from 30.19 to 47.02 percent at extra-high effort. The low-effort trained model also exceeds the base model's extra-high-effort coverage while using 47.9 percent fewer tokens. Generalization shows up too: held-out forecasting accuracy at high effort improves by 10.06 points, and out-of-distribution scores improve by 3.94 on GDPval, 7.07 on Harvey LAB and 4.96 on Frontier Finance.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.