A review of 628 studies on improving LLM-generated code quality found only 1 of 66 techniques beat budget-matched best-of-n sampling beyond chance
AspireAI community · arXiv (systematic review and controlled re-evaluation of LLM code-quality techniques)
Large language models pass most functional-correctness benchmarks, which says little about whether the code they write is readable, efficient or secure, and the literature claiming otherwise is scattered across one quality dimension at a time and rarely compared at equal compute. This paper reviews 628 primary studies from July 2021 to October 2026 with disclosed author-reviewed LLM assistance, then runs a controlled re-evaluation of ten techniques on four models at matched budgets across correctness, maintainability, efficiency and security, extends it to thirteen current models and repository-level tasks, and compares 66 highlighted techniques head to head on Gemma 4 31B. Quality prompting raises analyzer issue counts while leaving complexity and judge scores almost unchanged, and at matched budgets best-of-n sampling with an analyzer-weighted ranker matches or beats prompting, planning, self-refinement, a reviewer agent and retrieval on measures the ranker does not optimise, while only 1 of 66 techniques beats best-of-n on its own primary target beyond the 95 percent interval; large HumanEval+ correctness losses also disappear on contests held after the models' training cutoffs, which the authors read as likely contamination. The limitation is that a controlled re-evaluation can only cover the ten techniques it actually runs, so the 66-way head-to-head is a comparison among selected approaches rather than an exhaustive audit of the field.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.