A test-time layer that retrieves motion primitives from the robot's own training data lifted vision-language-action policies by up to 12 points in simulation and 40 percent in the real world, with no fine-tuning
AspireAI community · arXiv
Vision-language-action policies generalise well semantically and then fall over the moment the world looks slightly different from training: a different kitchen, a moved object, a new lighting condition. Most test-time fixes try to suppress the visual cues that are confusing the model. Retrieve to Act takes the opposite route and goes back to the robot's own successful demonstrations, using them as a library of geometric motion primitives rather than as visual evidence.
At inference, a graph neural network locates the robot's current motion inside an offline graph of those primitives and retrieves the ones structurally compatible with what is happening now. A learned per-dimension gate then blends the retrieved motion into the frozen policy's own action, correcting bad spatial trajectories while leaving the policy's local reactivity intact. The base model is never fine-tuned. Across GR00T N1.7, pi-0.5 and StableVLA the authors report absolute success gains of up to 12 points on LIBERO-Plus and RoboTwin2.0-Plus, and up to 40 percent in real-world deployments.
The caveat is that this depends on having a usable library of good demonstrations, which is exactly the thing a deployment struggling with distribution shift often lacks, and the headline real-world number is the best case rather than the average. The evidence is the authors' own, on three backbones and two perturbation suites.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.