An agent auditing 26.5M Slovak web documents found a 1.4M-document content farm that the pipeline's filters let through, and turned it into a 0.7-second test
AspireAI community · Brno University of Technology, Comenius University, KInIT and Cisco Systems (arXiv)
Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. This paper uses an LLM once, as a corpus debugger rather than a classifier: a tool-using agent explores the corpus through a read-only budgeted API and compiles each finding into an executable test that a native speaker then validates. On the 26.5-million-document Slovak portion of FineWeb2, three runs costing 2.21 to 2.84 dollars each found seven findings across four phenomena, the largest being a machine-generated content farm of 1,408,437 documents, 5.32 percent of the corpus, that passed the pipeline's filters, with 86.6 percent of it surviving even stricter thresholds. The compiled suite scans the full corpus in 0.7 seconds of wall-clock time without a GPU, against 12,000 to 90,000 dollars per LLM classification pass, and the framework, Slovak test suite, validation data and agent trajectories are released. The honest catch: the best static sampling baseline validated more selectors than any agent run, at under half the cost, so the agent's real contribution is exact prevalence and reusable tests rather than detection superiority.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.