A runtime gate that weighed reversibility and consequence cut harmful agent trajectories from 14.75 to 7.02 percent, but at the cost of 45 percent more human approvals
AspireAI community · arXiv
Tool-using agents increasingly modify external systems, yet most runtime safeguards allocate scrutiny using generic uncertainty scores or coarse mutation heuristics. CASEL, a Consequence-Aware Selective Execution Layer, is a model-agnostic policy that combines calibrated failure probability, state-dependent consequence, reversibility, ambiguity and evidence freshness to pick, for each proposed tool call, between direct execution, verification, sandboxing, clarification, approval and abstention - whichever minimises a per-mode cost. In a controlled simulation over 20 seeds with 120,000 test actions each, with every method given the same intervention menu and the same compute budget, CASEL reduced harmful-side-effect trajectories from 14.75 percent for an uncertainty-ranked gate to 7.02 percent, and mean normalised harm from 0.184 to 0.123. A simpler gate ranked by the product of failure probability and consequence reached comparable harm - 5.82 percent and 0.120 - but sent 45 percent more actions to human approval, while removing reversibility-aware routing pushed the rate back up to 15.53 percent. A trajectory-level risk budget made things worse, raising harm to 11.82 percent at equal cost. The authors are explicit about the limit: intervention effectiveness is assumed rather than measured, so these results rank allocation rules under the simulator's assumptions and do not establish deployment effectiveness.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.