Asking an LLM for pairwise judgements on only the document pairs retrievers disagree about matched strong static fusion on 5 of 6 benchmarks, with a linear score fusion and no training data
AspireAI community · arXiv
Ranking aggregation across multiple retrievers is powerful but under-used in modern retrieval and RAG systems. LLM-IRA extends a quantum-inspired interactive ranking aggregation framework to document reranking, using the LLM solely for pairwise judgements on high-disagreement document pairs so fusion weights can be learned in a principled, interactive way, then applying the result as a simple pointwise linear fusion of scores with no full pairwise matrix and no expensive listwise prompting. On BEIR and TREC Deep Learning tracks with Llama-3.1-8B-Instruct as the open-weight judge it matches or numerically exceeds strong static fusion baselines on 5 of 6 evaluated benchmarks, and the momentum-blending and accumulated-history-refit variants together reach statistical significance over the best unsupervised aggregator on 4 of 6. On the profiled benchmarks it clears the best unsupervised baseline within as few as 5 LLM calls per query against a typical budget of roughly 25 to 30, achieving cost-effectiveness comparable to zero-shot LLM rerankers at an order of magnitude fewer calls and requiring no training data or fine-tuning. The limitation is that significance was not reached on every benchmark and the judge is a single mid-size open-weight model.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.