A systematic evaluation of 13 explainability methods across 8 document datasets found occlusion the most faithful overall, and flagged several metrics that produce unreliable results
AspireAI community · arXiv
Multimodal document models have advanced quickly, but their black-box nature is still an obstacle to deployment in high-stakes document work, and explainability research tailored to text, layout and image together has been thin. DocXEval is a framework that adapts existing explainability methods and evaluation metrics to multimodal document understanding, with efficient multi-target adaptations for tasks like semantic entity recognition. Using it, the authors evaluate 13 explainability methods across 3 document understanding tasks, 8 datasets, 4 models and 17 evaluation metrics, and find that only a few methods perform consistently across all criteria, with occlusion emerging as the most faithful and interpretable overall. The meta-evaluation is the more useful part for practitioners: several faithfulness and complexity metrics agree moderately to strongly with each other, but some produce unreliable results - so choosing a metric carelessly can make an explanation method look better or worse than it is. Dataset-wide diagnostics then trace how individual modalities contribute across tasks and labels, spatial attribution patterns around target entities, and potential dataset biases. Code is released publicly. The honest limit is the scope of the framework itself: it measures faithfulness and complexity of attribution maps, not whether an explanation would actually help a human reviewer make a better decision, which is the question that matters when the model is triaging contracts or clinical records.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.