Giving a 4B model effect-typed tools raised task success from 53 to 99 percent, and filtering refusal templates in public tool-use corpora restored BFCL accuracy from 31.4 to 84.8 percent
AspireAI community · arXiv (EffectWorld authors)
*Small open models can call tools but do not reason about what a call does: they delete without asking, repeat writes, re-read state they already hold, and issue one call per turn where a short program would do.* EffectWorld is an executable world of effect-typed tools in which every tool declares its effect, whether it is idempotent, and which resources it reads and writes. Those declarations let a replay verifier decide whether a trace is effect-safe without labels or judge models, let an executor run several dependent calls in one turn, and let a compiler-style optimizer turn wasteful traces into verified training signal.
Low-rank adapters trained on that data produce large gains: on Qwen3-4B, task success rises from 53 to 99 percent in distribution, from 52 to 81 percent on a held-out split with unseen task kinds and distractor tools, and from 74 to 83 percent on an unseen domain, using about a third of the output tokens. On the Berkeley Function-Calling Leaderboard the adapter matches base-model live accuracy and lifts multi-turn accuracy from 21.9 to 26.6. The authors also found a trap that in-domain evaluation alone would miss: refusal templates in widely used public tool-use corpora teach a small model not to call tools, cutting BFCL non-live accuracy from 88.0 to 31.4 - filtering them restores 84.8. Negative examples added nothing over an SFT-only control; targeted positives carried the learning.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.