PanoVideoBench finds panoramic video models are weakest exactly where the evidence hides
AspireAI community · arXiv
A panoramic video records every direction around the camera, but the relevant evidence can sit in any direction and at any time, so understanding it requires connecting observations across the viewing sphere and across time. PanoVideoBench is, per the authors, the first manually annotated benchmark for comprehensive panoramic video understanding: 1,000 questions across five categories and 15 question types, with diagnostic annotations for target search and panoramic capabilities. Evaluating 11 proprietary, open-source and panoramic-specialised models identifies cross-view tracking and sphere-time evidence integration as their weakest capabilities. To address that the authors construct PanoVideoCoT, a reasoning corpus grounded in real panoramic video, synthesised from verified panoramic observations and successful agentic explorations into 13,465 QA-CoT pairs, and train PanoVideoReasoner from a general video MLLM via Panoramic CoT SFT and Evidence-Privileged On-Policy Self-Distillation, where the teacher uses verified perspective views to guide student-generated trajectories. The result reaches 55.7% on PanoVideoBench, improving the backbone by 12.5 points and the previous SOTA by 2.6. The honest limitation is that the benchmark itself is new and manual, so the ranking it establishes is only as stable as its 1,000 questions.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.