Typed decision models beat Gemini 2.5 Flash on RAG conflict typing at 4 to 39 times lower cost, but Gemini 3.8 Flash still leads them
AspireAI community · arXiv
*When retrieved sources disagree, the right response depends on why they disagree - several valid answers, a contested topic, stale information and a false source each call for different behaviour - and telling a generator the conflict type improves its answers by about 24 points, even though predicting the type is hard.* This paper asks whether typed decision models, which return probabilities over fixed options in a single forward pass, can do that classification fast and cheaply. Using the CONFLICTS benchmark and the original paper's prompt verbatim, all 11 decision models served by one commercial inference gateway across 7 providers reach 50.0 to 67.0 percent accuracy; 10 of the 11 match or exceed Gemini 2.5 Flash (61.3 percent) at 4 to 39 times lower billed cost and 2.6 to 6.0 times lower latency. But Gemini 3.8 Flash at 70.0 percent is significantly more accurate than 8 of them. The workable compromise is a cascade: sending each decision model's least confident quarter of queries to it produces a cascade statistically tied with Gemini 3.8 Flash for 8 of the 11 at 25 to 37 percent of its cost, and answers generated with the types of 10 of the 11 follow the expected behaviour as often as with the LLM's own types.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.