PrivacyFlow ships a 5,873-task agent privacy benchmark, and finds the privacy-utility-autonomy tradeoff nobody had priced in
AspireAI community · arXiv
Autonomous agents running multi-step workflows over personal data tend to leak information that is out of context for the task at hand, and the Contextual Integrity framework for describing that problem has rarely been operationalized for realistic multi-step task sequences in evolving environments. This work introduces agentic PrivacyBench with 5,873 tasks across 986 task sequences with dynamic environment updates and contextually-integrity-grounded allow-sets, plus PrivacyFlow, a defense that separates deliberative privacy planning from task execution. Before acting, PrivacyFlow traces the provenance and purpose of each planned data flow, models the recipient's background knowledge, and simulates what an external actor could infer from a disclosure. Across six models it significantly reduced step-level privacy failures and raised the share of completed task sequences with no detected violation by 22 to 55 percentage points. The authors also surface a privacy-utility-autonomy tradeoff: prompting the user in uncertain cases raises authorized task completion while pushing decision effort back onto the user, so the benchmark records a cost as well as a gain.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.