LangChain's OpenWiki made a retrieval agent roughly 5x more accurate on cross-source questions, and an evidence graph cut cost per correct answer by 58 percent
AspireAI community · LangChain
Agents working across policies, manuals, wikis and tickets have to work out which rules apply, what changed and how one source affects another - and reconstructing that on every question is expensive. LangChain tested OpenWiki, its open-source wiki compiler, as a persistent memory layer over a synthetic library of 111 long-form insurance documents with 972 controlled facts, deliberately seeded with outdated versions, contradictions and indirect reference chains. Holding the model, instructions, documents and questions constant and changing only what memory the agent had, the wiki lifted accuracy on connected questions from 0.10 with keyword search alone to 0.52 and cut cost per correct answer from about 0.55 dollars to 0.19. Simple lookups were the control: search and wiki scored 0.84 against 0.83, so memory bought nothing there while latency rose from 9 to 12 seconds. The remaining weakness was evidence retrieval - citation overlap fell to 0.25 on connected questions - which the evidence graph addressed by materialising wiki citations as typed edges (cite, supersede, refers_to) between claims, pages, paragraphs and topics. With the graph, accuracy rose a further 27 percent, citation consistency doubled, median answer time fell 40 percent and cost per correct answer fell 58 percent. The honest caveat is the one LangChain states itself: compilation and update costs are excluded from those figures, the 111-document wiki cost 4.15 dollars to compile, and whether memory pays off depends on reuse.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.