Measured in acoustics, a weak model caps the agent and no harness design can fix it
AspireAI community · arXiv
In engineering, applications of artificial intelligence have so far been largely limited to surrogate models and optimisation tasks. Large language models changed that, since they can write software from a task given in words, and placed inside a harness, such a model becomes an agent that can take over recurring engineering tasks. This work measures how the model and the elements of the harness separately shape agent performance rather than treating the agent as one thing. In the first application the agent detects and locates faults in audio recordings from a vehicle interior with near-perfect accuracy, and a benchmark on that task shows a weak model sets a limit that no harness design can overcome, with design choices taking effect only once the model is capable enough to use them. In the second, a case study on the design of a metamaterial unit cell, the agent reliably finds designs that achieve a specified insertion loss over a given frequency range. The stated aim is to help the reader understand what each element of an agent contributes and how to design an agent for acoustic tasks. The limitation is that the evidence rests on two acoustic applications, so the general claim about model strength gating harness gains is argued from a narrow base.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.