A theory-of-mind method that filters out events the target agent could not have seen reached 92.56 to 95 percent accuracy across three multimodal benchmarks
AspireAI community · arXiv (cs.AI)
Multimodal theory-of-mind systems usually reconstruct the whole scene - every event, every action, every utterance - and then reason over it. The flaw is that the solver sees things the agent being modelled does not, so an inaccessible event can quietly shape the inferred belief, goal or intention. PIP-ToM reframes this as an evidence-admissibility problem: evidence should count toward a mental-state hypothesis only when the relevant agent could access it, when its information source is preserved, and when it is diagnostic of the variable actually being queried.
The implementation builds agent-specific evidence stores, aligns them with structured mental-state claims, and calibrates model-based hypothesis scores while leaving the underlying Bayesian inference intact. With GPT-4o it outperforms existing theory-of-mind methods on MuMA-ToM, MMToM-QA and Hi-ToM, with average accuracies of 92.56 percent, 95.00 percent and 95.00 percent.
The honest limitation is that the gains sit on three existing multimodal theory-of-mind benchmarks with a single backbone, so some of the improvement may reflect how those benchmarks are built rather than a general capability jump. The method also assumes the per-agent evidence stores are constructed correctly, which is precisely the part it does not test - a solver that mis-assigns what an agent could see would still reason cleanly over the wrong evidence.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.