Tessera found that web RAG was dropping table and infobox evidence before the model ever saw it, and recovered 5 to 6 points of answer accuracy without touching the reader
AspireAI community · arXiv
Tracing answer evidence from source HTML through to reader input across the full English Wikipedia, the authors found that standard chunking separates structured content such as tables and infoboxes from the headers and context that give it meaning, so evidence present on the page fails to reach the reader intact, and stronger extraction, context expansion and larger reader budgets do not fix it. Tessera is a training-free delivery layer that rebuilds each page into fine-grained evidence units, completes each unit with its page title, section path and table headers, selects the units relevant to the query and packs them into the reader context at an unchanged token budget, leaving the index, the retriever and the reader alone. Across three question-answering benchmarks it improves answer accuracy by 5 to 6 points on text and screenshot pipelines, consistently across readers, retrievers and reader budgets, beating HTML-aware and structure-aware baselines at matched budgets, and the gains carry into multi-step agentic search. Code is published at github.com/sarendis56/Tessera; the measurements are on Wikipedia-derived QA benchmarks rather than production traffic.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.