Mistral Large 4 landed on vLLM with day-0 support, and the work behind it produced generic gains in decode parallelism, disaggregated serving and host-memory KV offload
AspireAI community · vLLM Blog
Mistral Large 4 is a 72-layer mixture-of-experts model with 128 routed experts on top-4 routing plus one shared expert, a hidden size of 6144, and MLA attention with a KV LoRA rank of 512 and a DSA sparse indexer selecting the top 2048 keys per query, totalling 1 trillion parameters with 49 billion active per token. Supporting it on vLLM's model runner V2 took more than a model file: the building blocks live in a new vllm/models/mistral package, and the effort produced generic improvements to decode context parallelism, prefill context parallelism, DSpark speculative decoding, NIXL-based disaggregated serving and HiSparse host-memory KV offload. Measured on TP8, the built-in DSpark drafter accepts about 2.9 tokens per step and gives about 2x output throughput, DCP gives about 7.9x more KV capacity, and PCP plus DCP cuts time to first token by up to 1.6x. FP8 and NVFP4 checkpoints both use an FP8 latent KV cache by default.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.