A 0.8 billion parameter model that does document parsing, general OCR, key information extraction, figure captioning and cross-page table reconstruction in one checkpoint, scoring 96.74 on OmniDocBench v1.6
OvisOCR3 is built on Qwen3.5-0.8B and consolidates five OCR-adjacent tasks into a single model, selected by prompt: document parsing to structured JSON, layout JSON or Markdown with LaTeX formulas and HTML tables; general OCR with bounding boxes and chart data; key information extraction to key-value records or JSON templates; document figure captioning with localized captions and Mermaid output; and cross-page table reconstruction, where the model merges table fragments that continue onto the next page.
Training runs in four stages - continued pretraining, domain-balanced mid-training, multi-teacher on-policy distillation from five 4B domain teachers, and mixed-task reinforcement learning with per-domain rewards. The reported results are 96.74 on OmniDocBench v1.6, the highest overall scores on PureDocBench, Real5-OmniDocBench and MDPBench, first place on OCRBench v2 with 65.9 on English and 61.3 on Chinese, and 90.50 on OCRBench KIE. Weights are Apache 2.0 on Hugging Face with Transformers, vLLM and SGLang support.
The limitation is stated by the authors themselves: the model can produce incorrect, incomplete, repeated or malformed output, small or blurred text and dense layouts hurt recognition, and cross-page reconstruction depends on first-stage parsing quality and correct page order. Several benchmarks are in-house.
https://arxiv.org/abs/2610.33066
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.