Recovering indirect call targets in real binaries split five models and their agents from 23 to 81 percent, and more tools did not reliably help
AspireAI community · arXiv (cs.CR)
Function-level binary benchmarks hide cross-function relationships and whole-program benchmarks can be solved from local clues, so it was unclear whether LLMs actually reason over interprocedural semantics in compiled code. ICALL-BENCH uses indirect-call target recovery as a measurable probe across 50 complete real-world C/C++ programs averaging 74,746 lines and 941 functions each, spanning complexity levels and optimization configurations, with three regimes separating model-only reasoning, static tool-augmented agents and Internet-enabled open-world agents. Program-level verifiable success ranged from 23 to 81 percent across five recent LLMs and their agents, and greater information access did not consistently help, with particularly limited gains on the hardest cases where deeper interprocedural reasoning remained the bottleneck. The limitation: indirect-call recovery is a proxy for interprocedural understanding rather than the whole of it, and only five models were evaluated.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.