A GPU-resident inverted index scored every posting instead of pruning, and searched all 8.8 million MS MARCO passages in 1.27 milliseconds with exact results
AspireAI community · arXiv
Learned sparse models such as SPLADE match dense bi-encoder quality while keeping lexical exact-match behaviour, but they are served on CPUs with inverted-index traversal algorithms such as WAND, whose pruning makes results approximate and whose control flow is sequential and data-dependent. GPUSparse argues a GPU should not run a faster traversal but remove traversal entirely: the inverted index stays in GPU memory and queries are scored with one fused scatter-add kernel, with nothing pruned. It searches all 8.8M MS MARCO passages in 1.27 ms per query, 787 queries per second on one H100, and the results are exact, with Recall@10, @100 and @1000 against an exhaustive FP32 reference all at 1.000 and full-scale quality matching Pyserini's exact SPLADE retrieval to three decimals. The paper's own accounting is worth the attention: under core pinning one H100 is worth about seven CPU cores, so it is 5.6x faster than a single core but 2.8x slower than a 32-core socket, meaning a CPU socket is cheaper for approximate serving and the GPU is the better choice when exactness is required. The limitation is that this is a retrieval engine result, not an end-to-end application benchmark, and the comparison is against exact CPU block-max pruning rather than against a modern approximate GPU retriever.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.