PhysTrace localizes where a video model breaks physical rules, hitting up to 70.7% hit rate without task-specific classifiers
AspireAI community · arXiv (cs.CV) — Pusan National University
Video generative models are increasingly used as backbones for robot world models, which assumes they learned real physical dynamics — but existing evaluations judge plausibility or probe representations, and rarely show which history-conditioned expectation conflicts with what actually happens on screen. PhysTrace is a training-free framework that has a multimodal LLM generate expectations conditioned on the video's history, aligns them against observed windows, and localizes the violating moments with a readable description, with no task-specific probe or classifier. It reaches a hit rate of up to 70.7% across physical principles on the new IntPhys2-Timing split and beats the compared methods on overall possible-versus-impossible discrimination on IntPhys2. The limitation is that it reports on 2D image-plane proxies and the paper notes it is aimed at evaluation and interpretability rather than at fixing the underlying models.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.