Benchmark Radar built a living catalogue of 1,283 AI benchmark sources and 12,916 numeric observations, with a leaderboard, Pareto view and CLI
AspireAI community · arXiv (Benchmark Radar authors)
*Benchmark researchers and model developers have to find the right evaluation, locate its dataset and code, and understand the settings behind a reported score - and the discovery problem is manual.* Benchmark Radar is a living database and search engine for AI benchmarks, covering LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety and domain-specific evaluations. Daily discovery draws on 37 sources - 13 direct connectors and 24 first-party research and engineering feeds - and the catalogue retains source identities and citations so a reader can inspect candidate benchmarks and their evaluation evidence rather than trusting a leaderboard row.
The catalogue currently holds 1,283 source records drawn from 4 benchmark catalogues, with 12,916 numeric observations on 790 records, plus mentions in model cards and technical reports and score histories. The paper audits the full catalogue and examines benchmark saturation, adoption trends and the limits of comparing scores across settings. A worked example walks through a complete prior-art search, showing how to query the catalogue and inspect benchmark evidence while designing a new evaluation. The release includes a web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface for offline queries and reproducible analysis.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.