Benchmarking ten open-weight browser agents on 65 miniature websites found drag & gesture the weakest skill family, with no agent solving more than 24 percent of its instances
AspireAI community · arXiv (Minh Pham-Dinh, Md Farhan Ishmam, Hernan Yupanqui et al.)
MiniWeb is a miniature of the web: 65 interconnected websites across 15 domains, with 369 human-annotated tasks and 37 reusable interaction primitives grouped into 7 skill families, so a failing agent can be diagnosed at the level of which skill breaks rather than only whether a task succeeded. Ten open-weight agents were benchmarked on it, and drag & gesture was consistently the weakest family, with no agent solving more than 24% of its instances. MiniWebGen then autonomously generates verified, skill-centric training data; fine-tuning Qwen3.5-4B on just 3.1K verified trajectories lifted single-primitive success by 24.9pp across 13 unseen websites. Pairing that trained executor with a planner that decomposes full tasks into the learned skill steps, MiniWebAgent improved full-task success by 19.4pp on unseen MiniWeb sites and transferred to external benchmarks with gains of 9.3pp on WebArena-Lite, 16.1pp on Online-Mind2Web and 20.0pp on WebVoyager, without training on their trajectories. The limitation the authors state is explicit: the skill trajectories teach short-horizon interactions, which do not directly translate to full-task success, so the gain only materialises once a planner composes them.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.