V-Lumen gave a frozen 2B vision-language model an external visual memory and won 17.8 accuracy points, with lookup latency that does not move as the table grows
AspireAI community · arXiv
*Widening or deepening a vision-language model charges more dense compute on every forward pass, so the interesting question is whether visual capacity can grow somewhere other than in the backbone.* V-Lumen discretizes visual features with a frozen codebook, addresses them with spatially aware hashing and text-conditioned relevance signals, and fuses the retrieved embeddings back through a gated residual path - all non-parametric, so capacity grows in storage rather than in per-token compute. Attached to a frozen Qwen3-VL-2B it adds 17.79 plus or minus 0.47 points on average across five reasoning-intensive benchmarks with gains on all five, and transfers zero-shot to three further benchmarks whose images are disjoint from training. The cost side is the part that matters for deployment: across a 40x range of table size, total lookup latency moves by less than 1 percent, from 11.16 to 11.23 milliseconds, because the lookup is paid once at prefill and never per token, though prefill itself grows 38 percent over vanilla while decode is unchanged. At a matched parameter budget, shifting capacity out of a LoRA or MoE adapter and into V-Lumen preserves accuracy on two backbones, which suggests external visual memory is a separate axis rather than a better fine-tuning method. The limitation is scope: text-conditioned hashing only applies to single-image samples, and 7 percent of MMMU-Pro and 61 percent of FRIEDA examples contain more than one image.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.