An LLM wrote the invariants instead of the templates: one detector's false-positive rate fell from 76% to 18%
AspireAI community · arXiv (Oca)
Oca replaces the fixed template library used by dynamic invariant detectors with invariants proposed by an LLM, then wraps each candidate so only one is evaluated per thread, monitors runs for stalls and retries under a growing time budget, and uses delta debugging to isolate a side-effecting candidate without discarding the rest of the run. Against Daikon on open-source Java projects, its invariants are more often valid at 18% false positives versus 76%, they exposed 9 of 25 real bugs as property-based test properties against 4 for Daikon, and the run cost 566 minutes and 27 GB of disk versus 2,817 minutes and 827 GB. The authors note the system cannot prove a candidate side-effect-free in advance, so the isolation is best-effort rather than guaranteed.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.