Google open-sources Harness2 self-improving agent framework

The most striking detail buried in Google's own writeup is who actually does the thinking inside Harness2. The executor agents being improved ran on Gemini 3.5 Flash and GLM-5, but the editor model that proposes and composes every harness edit in Google's benchmarks was Claude Opus 4.8, built by Anthropic, a direct competitor ^[6]. In other words, Google's flagship self-improvement framework, when put through its own published experiments, leaned on a rival lab's model to generate the actual improvements. That is a quiet admission that the editing step - the part that decides what a system prompt, guardrail, or skill should say next - is a distinct and apparently hard capability, one Google didn't default to its own models for when it mattered.

The headline numbers make a pointed argument: a smarter scaffold can beat a stronger model. On WorkBuddy-Bench, GLM-5's score jumped from 64.2 to 75.7, an 11.5-point single-step gain ^[4][5], while Gemini 3.5 Flash improved a more modest 5.1 points, from 72.4 to 77.5 ^[4]. Crucially, GLM-5's improved-harness score of 75.7 topped Gemini 3.5 Flash's own stock-harness score of 72.4 ^[4][5]- a cheaper, open-weight model outperforming Google's model simply because its operating environment was better tuned. Similar single-step gains appeared on JobBench, up to 4.5 points across 130 tasks spanning 35 occupations ^[1][3], and on Harvey's LAB legal benchmark, up to 11.8 points, with GLM-5's partial pass rate rising from 59.6 percent to 71.4 percent ^[1][5]. Pushing recursion deeper to k=5 added further gains on top of those single-step numbers: 3.9 points on WorkBuddy-Bench, 4.2 on JobBench, and 6.7 on LAB ^[1][5].

Independent security analysis flags a structural problem with how Harness2 ships by default: executor network access is unrestricted and the Codex executor runs with its sandbox disabled ^[7]. The concern isn't abstract - an agent that can reach its own grading rubric can learn to game the score rather than genuinely improve, and an executor with open egress and no sandbox doubles as a data-exfiltration and SSRF surface ^[7]. Layered on top of that is a real compute cost: roughly four agent executions per improvement step at k=1, climbing to about twenty at k=5, with no published per-task dollar figure ^[8]. Together, the framework's gains come with both an unquantified operating cost and a governance gap that the project's own documentation does not resolve.

Google frames Harness2 explicitly as a research project rather than a supported product ^[2][3], a disclaimer that cuts both ways in how it's being received. Reaction skews by audience: enthusiasm on social platforms treated the Apache 2.0 release and the double-digit benchmark gains as straightforward credibility signals, while the more technical community response was noticeably more measured, debating whether the recursive-editing approach is genuinely novel versus prior self-improving-harness work, and raising overfitting concerns about how far repeated edits generalize beyond the benchmark tasks they were probed against. That split - excitement over the open-source release itself set against skepticism about the substance of the claimed advance - is likely to shape how quickly outside teams adopt the framework versus simply watching how it holds up under independent testing.