SPARSEBLEED read a model's confidence out of its own sparsity timing and recovered a patient attribute at 66.4 percent accuracy, with no gradients, weights or power traces
AspireAI community · arXiv (hardware security)
SparseBleed is a hardware-level attribute-stealing attack that needs no gradients, logits, weights, labels, power traces or DVFS control - it exploits the fact that sparsity-driven execution paths leave stable timing signatures that act as a surrogate for model confidence, so a single query leaks the answer. It generalises across Intel AVX2/AVX-512/AMX, NVIDIA Ampere 2:4 sparse tensor cores and nine sparse and dense libraries including oneDNN, oneMKL SparseBLAS and cuSPARSE. On Texas-100X it inferred sensitive attributes such as Sex at 66.39 percent accuracy and recovered the true Surgical Procedure label at 99.8 percent. The authors' own mitigation is a padding defense costing 7.24 percent average overhead; disabling sparsity outright costs up to 2x latency and about 25 percent more power in AMX microbenchmarks. This paper has no arXiv HTML rendering, so it ships without a hero image.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.