Giving a coding agent's teacher only the evidence for the stage it is in - changed files, failing tests, then the diff - lifted SWE-bench Verified pass@1 from 9.6 to 16.4 percent on a 4B model
SWE-Ladder splits software-engineering work into the three stages it actually has - localize the faulty code, reproduce the failure, patch it - and supervises each stage with only the evidence belonging to that stage: changed files and lines for localization, fail-to-pass tests for reproduction, and the reference diff for patching. A rule-based router assigns each step of a rollout to a stage from its tool call, a frozen teacher sees only the matching evidence, and the stages are distilled in repair order while GRPO optimizes the whole trajectory.
With Qwen3-4B-Instruct-2507 under the R2E-Gym scaffold, pass@1 on SWE-bench Verified goes from 9.6 percent to 16.4 percent, against 10.4 percent for plain GRPO and 12.4 percent for on-policy self-distillation that exposes the full patch at every step. Under OpenHands the same method goes from 13.0 to 17.6 percent. Reversing the distillation order drops pass@1 to 11.4 percent, so the ordering is doing real work rather than the extra supervision alone.
The limitation is scale: the results are on a 4B model and two scaffolds, so this is evidence that stage-specific supervision helps, not yet that it closes the gap to frontier coding agents. Code is released on GitHub.
https://arxiv.org/abs/2610.33667
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.