Keep the judge outside the agent
Thursday a Thoughtworks engineer argued for tests kept as data tables an agent cannot quietly change, and AWS described building a local emulator with agents only after each test had passed against real AWS. Saturday Momentic rated just 9.6% of 1,000 agent-written unit tests useful, and Google released a sidecar that confirms each production agent failure before reporting it.
Sunday Google Research described /teamwork, where many agents explore and only results that pass a falsifier survive: a Lean proof checker, unmodified tests or a simulator. A Rust port of its AV1 decoder matched the C++ bit for bit and ended 20 to 30% faster, by Google's account.
SourcesMartin Fowler: tests as dataStack Overflow: your coding agent should not grade its own homeworkGoogle Research: /teamwork