aspireai.
PUBLIC EDITION

VocabX rebuilt a production grader's vocabulary around the cost of each token, and cut measured inference cost by 77 percent without changing the base model or the training recipe

AspireAI community · Hugging Face
Most vocabulary adaptation methods pick tokens by frequency. VocabX instead weights each saved token by the field it appears in, because in the measured system a token saved in the response was worth 7.4 times one saved in the input. It works on two layers under one criterion: rebuilding how domain words are split into sub-word pieces, and mining repeated whole spans from expressions to full JSON envelopes into single macro tokens. In an automated grader for Portuguese essays built on Qwen3-4B, the output layer went from 151,669 entries to about 65,000, the repeated JSON envelope span fell from 12 tokens to 1, and measured inference cost dropped 77 percent with the same base model and the same training recipe. Across five public domains and three model families it compressed more than the strongest reference method in all fifteen combinations, by 1.2 to 9.3 points. The honest caveat: these numbers come from a paper still under review that the authors say may change, and the headline 77 percent comes from a single production system.
Read the story and explore more from the AspireAI community.
Discover more stories
Stay curious.
THE BIG PICTURE
A FRESH PERSPECTIVE
Research
One distance function matched sparse-autoencoder features across layers and compressed feature circuits into supernodes
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.
Swipe into something new
01 / 1729
The conversation continues
Explore Builds — projects by Kiran Adimatyam ↗
A fresh perspective changes everything.
ASPIREAI — STAY CURIOUS.