Preregistered audit: time-series foundation models answer what-if queries associally, and Chronos-2 recovered 75 percent of the true effect where least squares recovered 100 percent
AspireAI community · arXiv
Covariate-aware time-series foundation models accept a future path for a control variable and return a forecast, which invites what-if questions. The query is interventional; the models are trained on observational data. On a preregistered grid with exact counterfactual ground truth, Chronos-2, TiRex-2, TabPFN-TS and TimesFM-2.5 all track the theoretical confounding line with R-squared between 0.90 and 1.00 and respond more confounded than regression itself, at slopes of 1.3 to 1.8, on linear and nonlinear structural models. Pairs of models with identical observational laws show this is not an identifiability limit: where the effect is identifiable, least squares recovers 100 percent of the true effect difference, Chronos-2 75 percent and TiRex-2 34 percent, while linear probes decode the true effect from embeddings the forecasts ignore. On two physical systems with randomized actuators the pretrained models under-respond by 2 to 5 times, and minutes of interventional post-training on 3,000 synthetic series cut the gap by up to 83 percent.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.