A neuro-symbolic model recovered the ground-truth rules of a synthetic event stream and stayed competitive with a transformer
AspireAI community · arXiv preprint
Most next-event predictors trade explainability for accuracy. This work, accepted at NeurIPS 2026, argues the two can be partly separated by learning the rules themselves rather than fitting a black box.
RULES learns sequential conjunctive rules directly from data, with training decoupled into a symbolic phase and a predictive phase so the residual network cannot quietly absorb what the rules were supposed to explain. On synthetic data the model faithfully recovers the ground-truth rule set and produces strong predictions from the symbolic rules alone, and on real event streams it discovers compact, interpretable rule sets. Adding a residual network on top gives RUNES, which the authors report as competitive with the transformer, outperforming all other baselines and ranking second overall on the remaining real-world datasets - and in noisy synthetic settings Rules alone even beats CatBoost and the transformer.
The point is auditability rather than a new accuracy ceiling: you can ask which rule fired and how the residual adjusted it. The code is open source. Caveat: the real-world datasets are small event sequences, and the paper itself notes runtime and scalability limits and that rule structures may not capture every data dependency.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.