MTG-AD gave 36 hours of video a professional inline audio-description dataset with 8,000-plus descriptions, and its evaluation renders the model's script to check it actually fits the silence
AspireAI community · arXiv (MTG-AD and MTG-AD-Eval)
Inline audio descriptions for blind and low-vision viewers have to fit inside the silences that already exist in a video, and multimodal language models have not been evaluated on that constraint end to end. MTG-AD is a professionally annotated dataset of 36 hours of video averaging about five minutes per clip and 8,000-plus descriptions in total, co-created by paired blind and sighted audio-description professionals, and MTG-AD-Eval is a reference-free evaluation framework built on expert guidelines that scores the whole task by rendering generated scripts into audio tracks and measuring how well they fit the original natural silences. Analysis across frontier models shows they describe visual events accurately and substantially cut speech overlap, but still leave gaps in narrative pacing and flow, and the paper validates that its automated metrics correlate significantly with human ratings from sighted raters and seven blind and low-vision participants, who narrowly prefer human-authored descriptions yet find the model-generated ones highly beneficial for comprehension. The limitation is that the human validation is small, with seven blind and low-vision participants, so the reported correlation should be read as promising rather than settled.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.