A 998-hour dual-timeline deepfake benchmark found audio-visual LLMs detect manipulations near chance, but their frozen features hold the signal
AspireAI community · arXiv
Localized AI manipulations in long-form videos affect brief events inside otherwise authentic content, so video-level classification is insufficient, and existing audio-visual deepfake benchmarks largely cover short talking-head clips. AV-FABLE is a dual-timeline benchmark with 16,650 pristine-manipulated twin pairs (998 hours) built from 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation.
Zero-shot prompting of audio-visual large language models yields near-chance detection and poor temporal localization, while supervised readouts of their frozen perception features recover strong forensic signals - suggesting the evidence is encoded in those representations but not exposed through language output. The proposed NFSR, which reads frozen AV-LLM features with separate audio and visual temporal decoders, achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings.
The honest limit: the benchmark, code, and model weights are only promised for public release, so the result is currently author-reported.
https://arxiv.org/abs/2610.33413
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.