Fine-tuned mixture-of-experts models leaked the route their training data took, and the signature was recoverable from generation alone with no access to that data
AspireAI community · arXiv
As mixture-of-experts models are deployed in distributed settings, from federated learning to edge-cloud systems, their routing exposes an underexplored security surface. The authors describe structural memorization: a fine-tuned model memorizes not just its training data but the internal pathway used to process it, producing a routing fingerprint unique to the training process. They show this emerges consistently even from a few samples, that its intensity depends on which modules were trainable, and that the memorized expert pathways can be recovered through the model's unconditional generation using only routing and output logits, with no knowledge of the fine-tuning data or procedure. In distributed deployments the exposure is amplified because routing information crosses multiple trust boundaries, including federated aggregation points, edge-cloud interfaces and multi-tenant monitoring systems. Traditional membership inference metrics viewed through expert routing fail to identify the data corresponding to these structural pathways, so the authors devise composite metrics combining reference-model tail likelihood with complexity-aware normalization. The limitation is that the attack is demonstrated on the routing interface itself, so its practical severity depends on how much routing detail a given deployment actually exposes.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.