KAPPA checked three published statistics papers blind and found the known error at the exact step the correction named - plus further problems confirmed by hand
AspireAI community · arXiv (2610.35596)
KAPPA, submitted to arXiv on October 26, 2026 as 2610.35596, is open-source software that organises language-model agents, run by Claude Code, to translate the proofs of a LaTeX paper into Lean 4 with Mathlib and check them there result by result. Each numbered result gets one of four verdicts: verified unconditionally, verified conditionally with every cited or unproved input named as a hypothesis, an issue recorded with its place in the paper and the side at fault, or unfinished with a reason. Separate agents write and review, each statement is compared with the paper before it is proved and fingerprinted afterwards, and every verdict rests on evidence that can be rerun without a language model. The authors applied the kit to four example papers with answer keys and to three published statistical papers checked blind in versions from before public corrections; in all three it found the published error at the step the correction names, together with further problems the authors confirmed by hand. The honest limitation is in the same abstract: given the same papers directly, the same model working with Lean classified fewer results correctly on the longest paper and reported some statements narrower than the paper's as proved.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.