A single geometry factor picked the best stochastic policy for few-step flow reinforcement learning, and its optimizer is shared across every update magnitude and direction
AspireAI community · arXiv
Few-step flow models generate samples through a handful of large denoising transitions, so applying GRPO requires making those transitions stochastic, and when one transition spans a wide range of noise levels the choice of stochastic policy becomes a major optimization factor. Prior constructions constrain the policy to preserve target flow coefficients, but that constraint admits a whole family of policies with no criterion for choosing among them. Geo-FSRL introduces a selection principle: at a fixed conditioning state and timestep pair, choose the admissible policy that minimizes the conditional KL divergence induced by any given nonzero velocity-output update. Within a coefficient-preserving family of isotropic Gaussian transitions this reduces to minimizing one scalar geometry factor that also governs velocity-space Fisher information and conditional transition log-likelihood-ratio variance, giving a unique closed-form optimizer shared across all nonzero update magnitudes and directions. The method trains on the original few-step schedule and keeps the deterministic solver at inference. The limitation is that the result is scoped to coefficient-preserving isotropic Gaussian transitions, so it does not automatically transfer to other stochastic policy families.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.