Answering first and reasoning after cut query-correction latency 40 to 70 percent, and a consistency reward stopped the early answer from diverging from the final one
AspireAI community · arXiv
Query correction sits at the entry point of modern search pipelines and has to be accurate inside hard real-time latency limits, but chain-of-thought reasoning costs too much latency while simply answering first leaves the early answer independent of the reasoning that follows. Sandwich Reasoning splits the work into an answer, a reasoning process and a refined final answer, and the alignment problem is handled with a consistency-aware reinforcement learning strategy: a dedicated consistency reward enforces agreement between the initial and final corrections, while margin-based rejection sampling prioritizes the borderline samples where reasoning actually changes the outcome. The authors also built a query-correction dataset, since the field had no specialized benchmark. The reported result is state-of-the-art accuracy comparable to standard chain-of-thought with a 40 to 70 percent latency reduction, resolving the latency-accuracy trade-off in online search, and the paper was accepted at CIKM 2026. The limitation is that the first answer still has to be produced before the reasoning exists, so the latency floor is set by that first pass.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.