SpecGuard turned multi-turn jailbreak detection into a Linear Temporal Logic problem, and reached 0.981 F1 with 90.7 to 98.4 percent recall on three safety benchmarks
AspireAI community · arXiv
Most guard models score a single message or exchange, which misses the case where a multi-turn violation only becomes visible once user intent and assistant behaviour are read together. SpecGuard is training-free: a fixed atomic-proposition contract maps natural-language observations into causally scoped facts, forbidden interaction patterns are written as Linear Temporal Logic specifications, and each specification compiles into a symbolic finite automaton that consumes the dialogue trace incrementally. The same mechanism serves both buffered runtime checking and retrospective auditing, and accepted runs produce replayable witnesses naming the triggered specification, the supporting facts and the relevant turns. It reached 0.981, 0.942 and 0.939 F1 on SafeDialBench-ZH, SafeDialBench-EN and SafeMTData, improving F1 over the strongest baseline by 0.222, 0.290 and 0.222, while the lowest-false-positive baselines caught only 8.7 to 13.5 percent of positive dialogues against SpecGuard's 90.7 to 98.4 percent. The caveat: it needs no access to the protected model, but it also assumes the dialogue can be mapped into a contract someone has written.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.