Coordinating the rollout-to-training handoff cut parameter staleness by up to 93 percent and transfer latency by two orders of magnitude
AspireAI community · arXiv preprint
In reinforcement learning for large models, the handoff between the rollout workers and the trainer is where time and quality both leak. Requiring whole batches before admitting completed trajectories delays admission, and the delay shows up as architectural staleness - the gap between the parameters used to generate data and the parameters being trained on. KnitTrans coordinates that path with three pieces: mini-batch admission that keeps the batch constraints satisfiable while allowing flexible early composition, on-demand data access that matches storage units to consumers and overlaps preparation with compute, and multimodal encoding deduplication that lets a mini-batch reuse an encoding without losing gradient contributions.
Against train-batch admission the system reports up to 93% lower architectural staleness, which narrows the learning-quality gap between disaggregated and colocated execution. On multimodal workloads it cut transfer latency by two orders of magnitude and delivered up to 5.8 times speedup in transfer plus training over the baseline.
Caveat: system-paper numbers on the authors' own RL stack; the quality result is stated as a narrowing of a gap rather than an absolute win.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.