Handing a world model the ground-truth view of an unseen room made spatial reasoning worse for one model, and a top-down map made it better
AspireAI community · arXiv
Visual world models support spatial reasoning by generating RGB views of states the agent cannot see, but RGB is the default interface mostly because it suits visual generation, not because it is the best thing to reason over. This paper isolates that question with ImagiRep, a simulator-grounded benchmark of 922 questions across seven question types and two settings, supplying oracle RGB alongside six alternative representations of the same unseen state, including depth, panoramic views and top-down geometry.
The controlled comparison produces the paper's sharpest result. With GPT-6.1-sol, giving the model the ground-truth endpoint view of a room dropped accuracy on farthest-object questions from 68.4 percent to 58.2 percent, while the top-down geometry representation of that same state raised it to 75.5 percent. The pattern repeats by question type: panorama helps open-space direction, 57.1 to 73.6 percent, while depth wins on cross-view nearest-object questions, 84.0 against 74.1 percent for RGB. With estimated rather than oracle states, generated RGB alone actively hurt, pushing Viewpoint Imagination from 57.2 to 54.7 percent, while estimated depth reached 82.0 percent on Cross-view Integration.
On top of that the authors build GEARS, a lightweight test-time strategy that picks a representation per question, reaching 61.4 percent on Viewpoint Imagination and 77.7 percent on Cross-view Integration, ahead of other spatial imagination methods but still short of the 87.5 and 91.7 percent an oracle selector achieves. The caveat is scope: the benchmark is simulator-grounded and covers one family of spatial questions, so transfer to real scenes is untested.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.