Compiling the neighbourhood geometry of anomaly detectors into a single-hidden-layer network scored 10,000 queries in milliseconds on one CPU thread, up to three orders of magnitude faster
AspireAI community · arXiv
Neighbourhood-based detectors such as KNN, LOF and LUNAR remain among the most accurate methods for tabular anomaly detection, but they compare every query with the stored training set, so their inference cost and memory grow with the data and the data must be kept at deployment. The paper asks whether the neighbourhood geometry these detectors exploit can be compiled into a small network that scores a query with a single forward pass. The Cauchy Displacement Network regresses each normal point onto the mean of its k nearest training neighbours with a single-hidden-layer network of Cauchy activations, and scores a query by the length of its predicted displacement, an empirical mean-shift vector whose fixed-radius population counterpart locally approximates a scaled gradient of the log density. The authors show Cauchy units exactly represent the score fields of a structured family of Student-t expert densities, and establish a constructive approximation bound for the corresponding population neighbourhood-mean map. They further show the bounded network output guarantees asymptotically linear residual-score growth along every fixed direction. On 46 benchmark datasets against 28 baselines, CDN obtains the best mean rank under both AUC-ROC and AUC-PR, while scoring 10^4 queries in a few milliseconds on a single CPU thread, up to three orders of magnitude faster than neighbourhood-based detectors. The limitation is that the compiled network is trained per dataset, so the speed gain trades away the ability to add new training points without retraining, which is the property that makes nearest-neighbour methods attractive in the first place.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.