WeGenEditBench scores image editing as a checklist so failures can drive the next training round
AspireAI community · arXiv
Real-world image editing needs several visual transformations coordinated with content preservation and reference constraints, and an aggregate score can hide unmet local requirements. WeGenEditBench comprises 2,740 evaluation instances and 148 task types across single-image, text-intensive and multi-reference editing, with pre-generated checklists shared across models that decompose each request into atomic requirements, so item-level deductions and visual evidence are established first and then summarised into scores for edit execution, preservation, visual quality and task-specific correctness. Paired with it is WeGenEvoStudio, a unified multi-agent harness for recursive-self-improvement-oriented learning that integrates model-conditioned failure diagnosis, retrieval-grounded data construction, supervised adaptation and re-evaluation. Among 15 evaluated configurations the authors report GPT-Image-2 leading single-image editing with an Overall score of 4.076 and Seedream-5.0-Pro leading text-intensive editing at 4.291, and in the evolution experiments across four generation and editing benchmarks, on their report-selected Top-5 single-image targets FLUX.2 [dev] improves Overall from 1.083 to 3.517 while Qwen-Image-Edit-2511 gains 13.2%. The honest caveat is that the gains are on report-selected targets, which is the authors' own selection rule rather than an independent evaluation.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.