Prompt caching cut an agent's input cost about 62 percent on a six-turn workload, but only with a byte-stable prefix
AspireAI community · AWS Machine Learning Blog
Every turn of an agent loop re-sends the tool definitions, the instructions and the whole history, because the model has no memory between requests. AWS's practical guide to prompt caching on Bedrock with OpenAI models walks through the two mechanisms - implicit caching, where Bedrock picks the reusable prefix as a conversation grows, and explicit caching, where the code marks the boundary with prompt_cache_breakpoint - and argues for using both.
The numbers come from a six-turn workload where GPT-6 Astra tied for the best implicit result at about 62 percent lower input cost than running without caching, and explicit rolling breakpoints saved about 61 percent on the same model. The operational details matter as much: the reusable prefix must reach 1,024 tokens to qualify, cache writes cost 1.25 times the normal input rate, cached prefixes stay eligible for reuse for at least 30 minutes after the latest write or successful reuse, and the tools and instructions must be serialised identically every time or the prefix will not match. In one test a roughly 4,000-token tools-and-instructions prefix wrote 4,120 tokens on the first request and read those 4,120 on later ones.
The caveat is that these are workload results on one model family, and the AWS post itself flags that when you cache with agents, tools and system prompts can be leaked or exploited - so caching needs to sit inside a prompt-caching governance practice rather than be added silently.
https://aws.amazon.com/blogs/machine-learning/groundhog-day-for-ai-agents-prompt-caching-patterns-for-using-openai-models-on-amazon-bedrock/
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.