An agent says "task done." The database disagrees. Microsoft and Hugging Face shipped a benchmark that measures exactly that gap.
AspireAI community · huggingface.co
An agent says "task done." The database disagrees. Microsoft and Hugging Face shipped a benchmark that measures exactly that gap.
ThinkingBox-Bench runs 507 stateful business workflows — retail, hospitality, auto insurance, neobank IT, consulting — each repeated 20 times per model. It grades the actual database state, not the agent's final reply.
Headline: 67% of failed attempts terminated cleanly with no tool error. The agent sounded correct because nothing crashed, but the record it was supposed to update was still wrong, missing, or had an extra change nobody wanted.
Kimi-K3 solves 93.9% of tasks at least once, the broadest coverage. But it nails all 20 attempts on only 13.4% of tasks. Claude Opus 5 inverts that: 47.5% of tasks on every single attempt.
If you ship agents that touch real records, single-attempt accuracy is the wrong column. Paper on arXiv (2608.19741).
Read the story and explore more from the AspireAI community.
Two critical PHP remote code execution flaws — one found by GPT-5.6 Sol — were patched on October 21, 2026, and OpenAI published a disclosure framework alongside the fix. Fixed versions: PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35. Both allow unauthenticated remote code execution, and PHP's advisory says risk is highest for mod_php, where a demonstrated exploit triggers before any application code runs. The CVEs are distinct failure modes. CVE-2026-109618 (OOM corruption): out-of-memory handling can interrupt PHP while internal request data is only partially initialized, and request cleanup may then treat the invalid data as valid. CVE-2026-109619 (stale global state): request-owned internal state can survive after its backing memory is freed, and a later request served by the same worker may use that stale state. OpenAI states GPT-5.6 Sol identified the first flaw through its Patch the Planet initiative, and that Wiz independently discovered and reported both vulnerabilities. PHP's maintainers led the fixes and release, with Wiz, Cloudflare and OpenAI contributing research and coordination. Wiz's write-up adds that it used GPT-5.6 Sol and Gemini 4 Argon throughout, with Gemini 4 Argon contributing to finding the vulnerability and validating the patch. OpenAI describes the gap between discovery and public exploitation as a "Defender's Window," proposes a five-stage response framework, and committed $1 billion in subsidised Daybreak model access for organizations with limited security resources. Honest caveats: these are OpenAI's and Wiz's own characterizations, and OpenAI also notes the same capabilities help attackers build offensive tooling faster. PHP 8.1 and older no longer receive updates from the PHP project.