AMD trained an EAGLE3.1 speculative-decoding draft from scratch on Instinct MI350X, hit 2.81 acceptance length and 1.47x throughput, and published both the draft and the recipe
AspireAI community · AMD
*Speculative decoding only pays off if the draft model is worth serving, and until now AMD's stack could serve one but not train one.* The Quark team trained an EAGLE3.1 draft from scratch for the MiniMax-M3-MXFP4 MoE target entirely on an eight-GPU AMD Instinct MI350X node, using a ROCm-native loop of vLLM hidden-state extraction, TorchSpec's disaggregated streaming trainer and FSDP2, then exported it as a vLLM-loadable LlamaForCausalLMEagle3 checkpoint. On the SPEED-Bench qualitative suite the draft averages 2.81 acceptance length across eleven domains - 3.33 on coding and 3.17 on math, down to 2.02 on roleplay - and stays flat at 2.71 to 2.72 from 1K to 32K tokens of context. Served through vLLM ROCm v0.27.1 with native AITER MXFP4 kernels at TP=8, output throughput per GPU rises from 281.7 to 415.2 tok/s, a 1.47x gain that narrows from 1.77x at concurrency 1 to 1.40x at concurrency 128, and median end-to-end latency falls from 13.15 s to 9.68 s, while time to first token gets 18.5 percent worse at 154 ms versus 130 ms. The draft ships as amd/MiniMax-M3-EAGLE3.1 and AMD Quark 0.13 packages the whole path as YAML recipes with a validated Qwen3-8B quick start, though every measured number here comes from AMD's own benchmark on one target model.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.