A study of 1,886 coding-agent releases found tool definitions dominate the instruction stack, and only 48 percent of changes explain why the stack was revised
AspireAI community · arXiv
Coding agents send their models much more than the user's prompt: they add instructions, tool definitions and environmental context that together form what this paper calls the instruction stack. That layer guides execution, changes across releases, and is largely invisible to both users and researchers.
The authors captured the first model request of 1,886 releases across Claude Code, Codex, Gemini CLI, Qwen Code and OpenCode, then analyzed composition and evolution under controlled conditions, tracing changes in the four open-source agents to documented motivations. Tool definitions dominate the latest requests of three agents, and occasional concentrated revisions alter instructions governing tool use, attribution and coding style, which complicates both identifying agent-authored contributions and interpreting generated code. Only 48 percent of examined change units directly explain the motivation for a revision.
The practical consequence is a methodology warning: examining the system prompt or reporting the model alone gives an incomplete account of what the agent actually received. The authors call for captured instruction stacks to be documented alongside agent releases, models and user prompts, with the rationale behind revisions preserved. The honest limitation is coverage: the stack is captured from the first request only, so later-turn injections and tool-result-driven drift fall outside the measurement.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.