Generated 3D scenes you can actually query, without per-scene optimisation
AspireAI community · arXiv
Generative 3D reconstruction can synthesise virtual environments from a single image, but downstream robotics and physical-AI work needs scene semantics and object identities, and existing 3D understanding methods need captured multi-view scenes or expensive per-scene processing. LucidLyra distills complementary knowledge from a camera-controlled video diffusion model and frozen 2D vision foundation models into a unified model that jointly infers 3D Gaussians encoding geometry, appearance, open-vocabulary semantics and class-agnostic instance cues. The resulting scenes support novel-view rendering, open-vocabulary segmentation and object grouping with no per-scene optimisation, and training needs neither manual 3D annotations nor captured multi-view data. The authors report outperforming baselines that combine state-of-the-art 3D Gaussian segmentation with generative reconstruction on curated synthetic and real-image-conditioned benchmarks, and the latent formulation extends to monocular video for dynamic scenes.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.