Few-step image models are more unsafe than their teachers, and a distillation-aware erasure method cut FLUX Schnell's unsafe rate from 0.161 to 0.046
AspireAI community · arXiv
Speed and safety turn out to trade off directly in distilled text-to-image models: reducing the number of inference steps consistently raises the rate of unsafe generations, and acceleration also makes unsafe concepts much harder to erase afterwards. The authors trace the failure to two mechanisms - the fragility of the few-step generation process and the way generation guidance is usually distilled into it - which makes existing concept-erasure techniques largely ineffective on these models. Their fix, DICE, aligns both its latent construction and its generation-preservation loss with the distillation method that produced the model, so erasure stays faithful to the few-step architecture. On FLUX Schnell, DICE brings the unsafe-generation rate down to 0.046 against at least 0.161 for the prior erasure methods they tested, while mostly preserving benign generation quality.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.