A training-free block-sparse attention scheme skipped 89 to 92 percent of video-diffusion attention while improving PSNR by up to 4.4 dB over the strongest baseline
AspireAI community · arXiv
Three-dimensional full attention dominates video diffusion inference, and existing block-sparse methods lose fidelity because they score blocks from coarse summaries or keep or drop whole tiles when only a few keys matter. AB-Attention clusters queries within each query block to estimate block importance cheaply, samples blocks in proportion to those estimates and reweights them by inverse inclusion probability, then gathers sub-tile blocks into full tensor-core tiles. On Wan2.1 and HunyuanVideo it skipped 89-92 percent of attention where baselines still computed 1.8-3.6 times as much, improved PSNR by up to 4.4 dB over the strongest baseline and reached up to 2.23x end-to-end speedup, with no training required. The speedup is measured on those two video models and the paper does not claim the same gains across other video architectures.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.