Shrinking a model for edge use can make it comply with a harmful prompt it refused in full precision, with no attack needed
AspireAI community · arXiv
Edge deployment pushes models through quantization and pruning to cut memory, compute and energy, and this paper argues those transformations are also a security operation: they change representations, decision pathways and trust boundaries. The authors measure three things across six instruction-tuned models in two tiers - sub-4B edge models and 7B to 8B baselines - using 100 harmful prompts: how often the refusal/compliance outcome flips relative to the FP16 model, how often a prompt the FP16 model safely refused is bypassed with no adversarial engineering at all, and how logit entropy shifts at the refusal margin. The gap between edge and benchmark scale is stark. Llama-3.2-1B-Instruct at 20 percent L1 sparsity shows a 24.00 percent decision-boundary flip rate and a 14.00 percent safety-refusal erosion rate, rising from 20.00 and 9.00 percent at 10 percent sparsity. Phi-3.5-mini at 3.8B reaches 22.00 percent flip and 10.00 percent erosion under INT4/NF4. The larger models hold much better until thresholds break: Qwen-2.5-7B under INT4 sits at 8.00 percent flip and 4.00 percent erosion, and Meta-Llama-3.1-8B at 6.00 and 2.00 percent. The worked example makes the point concrete - a phishing-email request that FP16 refuses cleanly is answered after INT4 quantization. Limitation: 100 prompts per model and a single judge, so the rates are directional rather than precise population estimates.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.