Meta Superintelligence Labs releases MIMESIS — a small, purpose-built user simulator that exposes a flaw in how most agent benchmarks are scored.
AspireAI community · huggingface.co
*Meta Superintelligence Labs releases MIMESIS — a small, purpose-built user simulator that exposes a flaw in how most agent benchmarks are scored.*
Off-the-shelf assistant LLMs playing the "user" are too cooperative. A fixed GPT-5.5 agent hits 63.6% on τ-bench with real humans but climbs to 82 to 84 percent when a frontier API model plays the user — an inflation the team says misranks agents.
MIMESIS comes in 4B and 9B sizes on Qwen3.5, trained to reproduce real-human patterns like hidden criteria, shifting goalposts, and false premises. Its 9B variant posts a SOUL-Index of 65.7, beating Claude Opus 5 at 64.9, and lifts downstream agent scores by roughly five points across nine unseen simulators. Code is open on GitHub.
Honest caveat: SOUL-Index and RealUserSim are the authors' benchmarks; no independent reproduction yet.
Read the story and explore more from the AspireAI community.
Two critical PHP remote code execution flaws — one found by GPT-5.6 Sol — were patched on October 21, 2026, and OpenAI published a disclosure framework alongside the fix. Fixed versions: PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35. Both allow unauthenticated remote code execution, and PHP's advisory says risk is highest for mod_php, where a demonstrated exploit triggers before any application code runs. The CVEs are distinct failure modes. CVE-2026-109618 (OOM corruption): out-of-memory handling can interrupt PHP while internal request data is only partially initialized, and request cleanup may then treat the invalid data as valid. CVE-2026-109619 (stale global state): request-owned internal state can survive after its backing memory is freed, and a later request served by the same worker may use that stale state. OpenAI states GPT-5.6 Sol identified the first flaw through its Patch the Planet initiative, and that Wiz independently discovered and reported both vulnerabilities. PHP's maintainers led the fixes and release, with Wiz, Cloudflare and OpenAI contributing research and coordination. Wiz's write-up adds that it used GPT-5.6 Sol and Gemini 4 Argon throughout, with Gemini 4 Argon contributing to finding the vulnerability and validating the patch. OpenAI describes the gap between discovery and public exploitation as a "Defender's Window," proposes a five-stage response framework, and committed $1 billion in subsidised Daybreak model access for organizations with limited security resources. Honest caveats: these are OpenAI's and Wiz's own characterizations, and OpenAI also notes the same capabilities help attackers build offensive tooling faster. PHP 8.1 and older no longer receive updates from the PHP project.