The best batch speech model's word error rate rose 77 percent at 2.4-second chunks, and Arm SME2 with INT8 weights is the remedy
AspireAI community · PyTorch Blog
Streaming automatic speech recognition is not batch transcription with a delay: a model that wins on complete recordings can fall apart at the chunk cadence used in deployment. The PyTorch case study makes the gap concrete - in one cross-model study Qwen3-ASR-1.7B had the best batch accuracy of the models tested, but its word error rate rose 77 percent when evaluated with 2.4-second audio chunks, while NVIDIA Nemotron Speech Streaming 0.6B had the lowest average streaming word error rate over 560 ms chunks.
The post takes the multilingual Nemotron 3.5 from PyTorch to ExecuTorch, PyTorch's edge inference framework, and profiles it on a Google Pixel 11. Exported models become a .pte file the device runtime loads; supported operators are delegated to XNNPACK, which on Android can select microkernels from Arm's KleidiAI. Arm Scalable Matrix Extension 2 extends the instruction set for matrix work including general matrix multiplication, and in the C1 CPU cluster implementation the matrix unit is shared across cores, with KleidiAI supplying microkernels that use it. The post shows SME2 cutting total ASR processing time under FP16 and INT8 quantisation with per-channel weights (QC8W), and pairs operator timings with native kernel samples to find the remaining bottlenecks.
The caveat is that the accuracy and latency numbers come from one phone and one model family, and the choice of chunk size is a straight trade - longer chunks improve final-transcript accuracy and cost latency.
https://pytorch.org/blog/optimizing-streaming-speech-recognition-on-arm-edge-devices-with-executorch/
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.