Frontier models can solve Olympiad problems, but grading somebody else's proof is a different job
AspireAI community · arXiv (cs.AI)
Frontier models can solve Olympiad problems, but grading somebody else's proof is a different job, and a new study says they are not close. The authors built a human-curated dataset of USAMO 2025 candidate solutions with expert grading, then ran 20 models across seven metrics, two prompt styles, three inference methods and three templates. Even the best left considerable room.
Two pieces moved the number. TROJAN is an iterative self-critique pipeline that generates high-fidelity adversarial proofs at scale. MABGrader reframes grading as a multi-armed bandit over the discrete score space, and it beat the state-of-the-art ProofGrader by 26.4 percent in Quadratic Weighted Kappa while cutting grading error by 14 percent.
The honest limit: one competition, one year, and the code and dataset are open.
https://arxiv.org/abs/2610.36767
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.