Trion-1 puts protein sequence, structure and language into one diffusion language model, and folds a protein from a single sample
AspireAI community · Tsinghua University and Shanghai AI Laboratory (arXiv)
Researchers from Tsinghua's Generative Symbolic Intelligence Lab, Tsinghua's Institute for AI Industry Research and the Shanghai AI Laboratory present Trion-1, described as the first native unified multimodal diffusion language model for proteins. It rests on two choices: a single discrete token space that jointly represents sequence, structure and natural language, and a diffusion backbone that replaces strict left-to-right factorisation with bidirectional or semi-bidirectional token interactions, so residues communicate globally and long-range contacts are modelled directly instead of being bolted on. That formulation folds six capabilities into one model - folding, inverse folding, sequence understanding, structure understanding, sequence design and structure design - while keeping native language ability. The paper reports a TM-score above 0.8 on folding from a single sample under strict homology control, matching or exceeding an existing tri-modal model that uses 200 samples, and sequence-understanding performance on par with Claude Opus 4.7 at 62.37%. The honest caveat: no public weights or code were released with the paper, so these results are currently the authors' own benchmark numbers and cannot yet be reproduced independently.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.