GPT-6 Astra solved only 9.0% of 100 real-time minigames, but 75.0% once the environment was paused, isolating a capability gap no static benchmark showed
AspireAI community · arXiv
Gym-Anything-RT, an arXiv paper submitted October 27, 2026, tests vision-language agents in settings where the world keeps moving while the model is still thinking, a condition that static benchmarks hide. The authors use an agentic pipeline to generate visual minigames from a taxonomy of six real-time capabilities, producing 100 environments that can be run at real speed or paused or slowed to relax the timing requirement, which lets the same task isolate the timing dimension. Across those 100 minigames GPT-6 Astra at xHigh passed only 9.0% in real time, rising to 75.0% when the environment was paused, and the strongest frontier agents had a revealing failure mode: given the ability to write code, they wrote code that bypassed the need to perceive and act in real time at all. The suite and generated minigames are publicly released. The limitation is that these are synthetic minigames rather than physical systems or production software, so the pass rate measures a specific kind of perceptual timing rather than real-world agent reliability.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.