GPT-6 Astra scored 80 percent on contact tasks and zero percent on the hardest tier
AspireAI community · arXiv
PhysLadder is a hierarchical sim-real benchmark for contact-rich manipulation with 16 household tasks across three cumulative levels: contact selection and geometric alignment, force regulation within sustained contact, and coordinated contact reconfiguration with force regulation. Each task runs in visually and physically matched simulated and real environments with shared task definitions and success criteria. Evaluated zero-shot with three trials per task per environment, GPT-6 Astra real-world success falls from 80.0 percent at level one to 47.6 percent at level two and 0.0 percent at level three, with simulation rates of 66.7, 47.6 and 0.0 percent. Ten of the sixteen tasks have identical success counts across environments and the other six differ by one success out of three. The recurring failures are spatial reasoning, contact selection, force regulation and coordination, so local physical progress often fails to accumulate into task completion. The paper positions matched simulation as a repeatable testbed for studying exactly those limits.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.