Replaying refusals cut attack success by 51.1 points but raised over-refusal by 53.4, while distilling base logits avoided most safety erosion at a fraction of the cost
AspireAI community · arXiv (Ruellan, Benavides-Prado)
Benign fine-tuning erodes safety alignment, and sequential adaptation compounds the risk. Two standard mitigations exist - replaying harmful-prompt refusals and benign-compliance examples, or matching base-model logits via distillation - and this paper factors them apart to see what each actually contributes during sequential LoRA fine-tuning of Mistral-7B and Llama-3-8B across code generation, German summarization and science QA task sequences. Behaviorally, supervised replay exposure yields the largest reduction in HarmBench attack success rate on Mistral, averaging 51.1 points below base, but at a substantial over-refusal cost averaging 53.4 points above base on OR-Bench-hard. Logit distillation, with no supervised replay loss at all, largely prevents safety erosion at a fraction of that cost: an average 5.4-point attack-success reduction against only a 5.1-point rise in over-refusal. Llama shows the same asymmetry. Combining the two moderates exposure's over-refusal without hurting downstream task scores. Geometrically the models diverge: on Mistral distillation preserves the base refusal direction more closely than exposure, while on Llama refusal behavior after exposure largely survives ablation of the extracted refusal direction - meaning preserving the refusal direction is not a reliable proxy for behavioral safety. The limitation is scope - two models, three task sequences and one adapter method.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.