FitzRoy pairs a discrete diffusion forecaster with reinforcement learning to match state-of-the-art precipitation skill while keeping physical realism
AspireAI community · arXiv
Physics-based models struggle to resolve mesoscale convection while machine-learning forecasters stay physically inconsistent, blurred and consistently underestimate extreme events. FitzRoy is presented as the first system to reach state-of-the-art predictive skill and physical realism simultaneously over the continental United States, refining a conditional discrete diffusion model with reinforcement learning; across lead times up to 24 hours it matches or exceeds state-of-the-art models while preserving structural realism and capturing extreme events, and its low-latency inference makes real-time operational deployment possible. The authors report case studies on the 2025 Texas Hill Country and 2026 Missouri floods, where FitzRoy flagged record-breaking accumulations that existing models largely missed. The numerical skill and realism gains are reported against the authors' own comparisons rather than to an independently scored operational forecast, and the work targets precipitation specifically.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.