3D MLLMs collapsed on geometrically ambiguous objects, and reversing the reasoning order into part, function, identity lifted zero-shot accuracy from 60.5 to 66.7 percent
AspireAI community · arXiv (Li, Jiang, Zha, Dai & Xia)
PFIR starts from a per-sample Geometric Confusion Score that exposes how much 3D multimodal models rely on shape rather than function: baseline accuracy is 33.7 percent on ModelNet40's most ambiguous 20 percent of samples against 84.0 percent on the least ambiguous 20 percent. The authors trace part of that to a function-as-annotation pattern, where functional language decorates a known identity instead of supporting inference, and reverse the flow so the chain runs geometry to function to identity. The pipeline produces 507K question-answer pairs with functional chains of thought from PartNeXt part segmentation, and a three-stage curriculum ends in a dual-objective GRPO setup covering strict functional reasoning and descriptive grounding on Objaverse's distribution. The two reinforcement learning objectives produce nearly orthogonal task vectors, with a cosine of 0.056, which the authors compose through task arithmetic. Under a strict out-of-distribution protocol running Objaverse to ModelNet40 and XYZRGB to XYZ-only, PFIR raises zero-shot accuracy from 60.5 to 66.7 percent, with the gains concentrated 2.6 times more on the geometrically ambiguous samples that motivate the work. The code is released, and the honest limit is that the gain is a modest percentage-point lift over a strong baseline, concentrated rather than uniform.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.