No. 12 Sunday 25 October Game dev, software and AI

Measure where it counts

Google let an agent work overnight on a 200,000-line codebase, scored it on a benchmark it never saw, and kept two of six changes. Plus guardrails that enforce instead of ask, and Unreal tools that test what players actually load.

The short version

  • Software and AI Google ran Gemini 4 Argon for 11 hours on its agent framework with a rulebook, a pinned goal and a hidden benchmark. Maintainers kept 2 of 6 changes; both were design fixes.
  • Software and AI Sean Goedecke argues slow software is a design problem: if half a request waits on the database, a faster language can at best halve it.
  • Software and AI Google's Data Agent Kit shows two kinds of guardrail: a skill that asks the agent to stop, and an IAM rule that blocks write tools outright.
  • Software and AI Vercel now creates a separate, limited Stripe key for each agent, and Claude's SDKs moved to Apache 2.0 with breaking changes. Pin before you upgrade.
  • Game dev An Unreal tool author says packaged games ignore save-file redirects that work in the editor. Today's Try this checks your own build.
Software and AI Lesson
Six boxes in a row. Session Copy and Lazy Imports are highlighted and marked Shipped; Event Cache, OTel Gating, Contents Rewrite and Replay Index are grey and marked Cut.
Image: Google for Developers blog

Google left an agent alone overnight with 200,000 lines of code

Google gave its coding agent one goal, a written rulebook and a speed test it could not see, then let it work for 11 hours on a big open-source project. It proposed six speedups; the maintainers kept two.

ContextGemini 4 Argon is Google's coding model. ADK, the Agent Development Kit, is Google's open-source Python framework for building agents, with about 800 source files and nearly 17,000 unit tests.

A first try ran about two hours with a one-paragraph prompt and no guidance. The agent changed a public JSON schema, ran only 216 nearby tests and left its benchmark numbers out of the pull request.

For the second try the team added a .agents/skills folder with an index and 12 guideline files, and pinned one goal: an unmerged pull request with before and after numbers. Argon worked from 9:11 PM to 8:19 AM. It read the guidelines first, measured a baseline before touching code, added regression tests and ran all 16,892 unit tests.

Two of six changes shipped. Filtering session events before copying them cut the framework's time before each model request by about 40% in long sessions, and a 158-turn run fell from 65.6 to 56.1 seconds. Lazy imports cut cold import time by 21%. The figures are Google's own.

2 of 6agent changes the maintainers kept

Technical details

Speed was judged on four workloads run 30 times each (package startup, a 200-step conversation, a 20-sub-agent handoff and a 12-node workflow), plus a holdout script, bench_turn.py, that the agent never saw. Round two ran in Antigravity's /goal mode, which keeps the goal pinned when the conversation is compacted.

The four cut changes: a multi-agent event cache that was 17% faster on a synthetic test but could return stale data across sessions; a read of a private OpenTelemetry attribute that could break on a minor upgrade; a history-formatting rewrite that was harder to read with no end-to-end gain; and a replay index that only helped synthetic tests.

Lazy loading left time to first request flat (3.07 to 3.09 s) because the import cost moves into the first turn, so Google suggests a warm-up request. Simply deleting three imports passed all 16,892 open-source tests but broke 25 internal Google test targets that patched those attributes.

Google's post is dated October 13; InfoQ covered it on October 24, which is when it reached us. The shipped commit says the changes were generated by Argon.

Why it mattersThis is a reusable recipe for long, unattended agent runs, with the failure mode shown next to the fix. The benchmark the agent never saw is what made the numbers believable.

For youCopy the shape in Claude Code: a skills folder of repo rules, a benchmark script kept out of the repo, and a goal that requires before and after numbers in the PR. Try it on plugin load time or a slow Supabase query path.

unit tests
Small automatic checks that each confirm one piece of code still does what it should.
pull request
A proposed change to a codebase that people review before it is merged.
regression tests
Tests added to catch a specific problem if it ever comes back.
Lazy imports
Loading a code library only at the moment it is first used, instead of at startup.

Sources Google for Developers blogInfoQThe shipped commit

More like this Less like this

Prove the speedup

Design beats language, and a cheaper model needs its weak spot checked.

Software and AI Lesson

Slow software is usually slow by design, not by language

Sean Goedecke argues most apps are slow because of how they are built, so rewriting them in a faster language buys little. Caching, batching and skipping work buy far more.

His examples: a tokenizer rewritten in assembly won largely by avoiding regular expressions, and a naive Rust port of a C program ran about 30% faster because it used B-trees. Both were design changes riding along with the rewrite.

  • Cache what you compute twice.
  • Move slow work out of the request.
  • Run independent work in parallel.
  • Fix N+1 queries.
  • Rewrite only hot spots that are truly CPU-bound.
Technical details

Commenters pushed back that a typical query takes about 5 ms against a 500 ms response, so the database is not always the culprit; Goedecke says his view comes from profiling at a few companies. His advice for the agent era is to have agents profile and redesign first.

Why it mattersAgents make rewrites cheap, which makes the wrong rewrite tempting. Today's lead agrees: both changes Google kept were about doing less work, not faster work.

For youBefore porting any Node hot path to Rust, profile it. In your SaaS apps the time is most likely Postgres round trips, which a rewrite would not touch.

B-trees
A way of organising sorted data so that finding, adding and removing items stays fast as it grows.
N+1 queries
Asking the database once for a list, then once more for every item in it, instead of fetching everything in one go.
CPU-bound
Limited by how fast the processor computes, rather than by waiting on disks, networks or databases.

Sources Sean Goedecke

More like this Less like this

Software and AI Release

A Gemma fine-tune that thinks 39% less

BottleCap AI released free weights for a version of Google's Gemma 4 model that writes far less hidden reasoning before answering, at almost the same accuracy on its own tests.

Cut in hidden reasoning, per benchmark (vendor-reported). Average across 12: 38.9%, with accuracy down 0.15 points on average.

Highlighted: the biggest accuracy loss, 4 points on AA-LCR. Stuck, looping answers fell from 3.64% to 0.76%.

Technical details

The model is bottlecapai/ThinkingCap-gemma-4-26B-A4B-it on Hugging Face, with GGUF builds, meant as a drop-in on the same vLLM setup and sampling settings. Accuracy changes ranged from -4.00 points on AA-LCR and -0.96 on GPQA Diamond to +1.83 on IFBench and +1.45 on LongBench v2. A paid version claims larger cuts, with no evaluation shown.

Why it mattersFor a self-hosted model, thinking tokens are most of the cost and the wait. Check the tail before switching: long-context work lost the most.

For youWorth a test if you run a local model for grading or classification in a pipeline; run it on your own long inputs first.

thinking tokens
The hidden reasoning text a model writes before its answer; you pay for it and wait for it.
fine-tune
A copy of a model trained a little further to change how it behaves.

Sources BottleCap AI

More like this Less like this

A rule asks, a wall enforces

Guardrails that block, keys scoped to one job, and the week's agent tools.

Software and AI Lesson

A skill asks the agent to stop. A deny rule makes it

ContextData Agent Kit connects coding agents such as Claude Code, Codex and Cursor to more than 15 Google Cloud data services through MCP, with 37 skills, under Apache 2.0.

Google's free Data Agent Kit, now generally available, protects databases two ways: a written skill that tells the agent to ask before deleting, and an access rule that blocks write tools no matter what the agent decides.

The two layers, side by side.

The skill The deny rule
What it is accidental-data-loss-prevention, written guidance the agent reads An IAM deny policy on the mcp.googleapis.com/tools.call permission
What it covers DROP TABLE, TRUNCATE, DELETE without WHERE, production bucket deletes, project deletion, key destruction Every MCP tool not marked read-only
Enforced? No. The agent is asked to stop, explain and wait for consent Yes, at organisation, folder or project level
Blind spot An agent that ignores or never loads the skill Blocked tools still show in the tool list, and it covers Google Cloud MCP servers only

Use the skill for judgment and the deny rule for anything you cannot undo. Neither tells an agent whether a timed-out write actually happened.

Technical details

The documented condition is api.getAttribute('mcp.googleapis.com/tool.isReadOnly', false) == false. The companion attributes resource.service and tool.name work only through gcloud, not the console. The alternative is running the agent as a read-only service account through impersonation.

Ankit Dongre (viaSocket), quoted by InfoQ, argues tool results should carry an idempotency key, commit status and whether the action can be rolled back, which the kit's results lack.

Why it mattersMost agent safety today is written as instructions. This is a clear case of the difference between asking and enforcing, and the same split applies to any database an agent can reach.

For youSame split for Supabase: keep the 'ask before destructive SQL' line in CLAUDE.md, but give the agent's MCP connection a read-only role so the rule cannot be talked around.

MCP
Model Context Protocol: a standard way for AI agents to discover and call tools and data sources.
IAM
Identity and access management: the cloud's rules for who may do what.
skill
A folder of instructions an agent loads when a task calls for it.

Sources InfoQGoogle Cloud docs: block read-write toolsPlugin repository

More like this Less like this

Software and AI Release

One payment key per agent, made for you

Vercel can now create a separate, limited Stripe key for each agent or task, so a reporting agent can read charges while an onboarding agent can only edit customers.

Vercel creates a restricted key in your Stripe account for each agent or task and stores it, so there is no secret to paste. The existing MCP and bring-your-own-key options are unchanged.

A leaked or misused key can only do what its own permissions allow.

Why it mattersScoped, per-task credentials are the enforced half of agent safety: when one key leaks or an agent misbehaves, the damage stops at that key's permissions.

For youIf any of your SaaS agents touch Stripe, give each its own read-only or narrow key this way rather than sharing the secret key.

restricted key
A Stripe key that can only perform the actions you selected, such as reading charges.
Rotate
Replace a secret key with a new one, so an old copy stops working.

Sources Vercel changelog

More like this Less like this

Software and AI Release For practitioners

Shipped for agent builders this week

Smaller releases for people building with Claude and other agents: a new way to buy the Claude API, SDK licence changes, docs written for agents, and a dashboard for many agents at once.

Show the details

From the release notes and changelogs, October 22 to 24.

  • Claude on Google Cloud, run by Anthropic

    The Claude API operated by Anthropic, reached from your Google Cloud project with IAM access and Marketplace billing. Fast mode for Claude Opus 5.5 is available there.

  • Claude SDKs move to Apache 2.0

    The new SDK majors switch from the MIT licence and carry breaking changes of their own, apart from the API version. Pin versions before upgrading.

  • Supabase Knowledge Base

    Troubleshooting and migration guides at supabase.com/kb, in public beta, written for developers and their agents.

  • Corgi

    A terminal multiplexer that shows whether each coding agent is working, waiting on you or done, with a token view and git worktrees. The page does not state a licence.

Why it mattersLicence and breaking changes in an SDK are easy to miss in an automatic dependency bump.

For youPin the Anthropic SDK in your Node SaaS repos before Dependabot proposes the new major; read the migration guide first.

SDK
Software development kit: the official library for calling a service from your code.
multiplexer
A program that runs several terminal sessions in one window and lets you switch between them.

Sources Supabase changelogCorgi

More like this Less like this

Test what players actually run

Saves, AI trees and the recorder you make trailers with.

Game dev Lesson

A renamed variable can survive in the editor and vanish in the shipped game

ContextCoreRedirects are lines in an Unreal config file that tell the engine an old name now means a new one, so data saved under the old name still loads.

An Unreal Engine tool maker says that when you rename a saved variable, the editor carries old values across but a packaged game using the standard loader does not, so players can lose progress with no error.

Savekeeper, a new $14.99 tool for Unreal Engine 5.8, turns that into a test. Its author bases the editor versus packaged claim on reading engine source; the tool's own tests run in the editor.

  1. Keep real saves

    Before release, capture the save files a build writes into a fixtures folder: -run=Savekeeper -Capture=1.0.0.

  2. Test each new build

    Run -run=Savekeeper against the new classes. It reports, per stored value, whether a player keeps it in the editor, in a packaged game with the stock loader, and with its own loader.

  3. Fail the build

    It writes an HTML report and JUnit output, and exits with code 2 when a value is lost, so CI can stop the release.

  4. Write proven fixes

    For a renamed field it proposes a redirect, replays the save through it and checks the value arrives. -Apply writes the proven lines to DefaultEngine.ini.

Technical details

The author points to the condition !RequiresCookedData() || IsSaveGame() in Class.cpp (lines 1621 to 1622 in 5.8) and says LoadGameFromMemory never marks its archive as a save game, so property redirects and Blueprint property GUIDs are skipped in cooked builds. Its own load node applies redirects to the bytes before the stock load. It reads saves from 5.4 on, only 5.8 saves were tested, and custom Serialize overrides and third-party save plugins are not covered. The author says the code was written with AI tools.

Why it mattersLost saves are among the most damaging bugs a game can ship, and this one hides from editor testing. The test pattern (real old saves as fixtures, checked on every build) works with or without the tool.

For youYour inventory and progression plugins persist state that buyers ship in their games. A save-compat fixture test in CI would be a selling point on Fab; see Try this.

packaged game
The cooked, shippable build players run, as opposed to the game running inside the editor.
JUnit output
A standard test-result file format that CI systems can read.
CI
Continuous integration: servers that build and test every change automatically.

Sources Unreal Engine forums

More like this Less like this

Game dev Release

A converter that moves Unreal AI from behavior trees to StateTree

A new paid tool converts Unreal behavior trees into StateTree, Epic's newer AI system, keeps the original, and lists every node it could not translate instead of guessing.

A $19.99 tool for Unreal Engine 5.8 on itch.io. It converts a behavior tree into a StateTree that shares the same blackboard, leaves the original untouched, and names every node it cannot translate.

A CI switch, FailOnUnconverted, stops a build when anything was left untranslated.
Technical details

Blueprint tasks, decorators and services cannot be translated and go on the sheet; an unconverted task fails, and an unconverted decorator blocks its branch. Cooldowns are timed per AI controller. It is one of several small, AI-built Unreal 5.8 command-line tools posted to the forums this week, each made to run in CI.

Why it mattersNaming what it cannot do, and failing safe there, is the right design for any migration tool, human-written or AI-built.

For youIf your combat or interaction plugins ship behavior tree examples, this shows buyers will start asking for StateTree versions; the fail-safe conversion sheet is a pattern to copy in your own tools.

behavior trees
A common way to script game AI as a tree of choices and actions.
StateTree
Unreal's newer system for game AI and logic, built around states and transitions.
blackboard
The shared memory an Unreal AI reads and writes, such as its current target.
EQS
Environment Query System: Unreal's tool for asking questions like 'where is the best cover nearby'.

Sources Unreal Engine forums

More like this Less like this

Game dev Release

OBS Studio 33 moves plugins and its browser

The free recording and streaming app most developers use for devlogs and trailers has a big update that changes where plugins live and upgrades its built-in browser.

150browser engine version, up from 127

Third-party plugins get a new install location; old locations still load until OBS 34. Ubuntu 24.04 and FFmpeg 6 and 7 are no longer supported.

Technical details

The browser engine is CEF (Chromium Embedded Framework). First-party plugins move to a core folder, preview snapping works along a whole axis, and NVENC encoding now runs on Arm64. Qt 6.4 to 6.9 support was also dropped.

Why it mattersBrowser overlays and third-party plugins are what usually break; check them on a spare profile before a recording session.

browser engine
The built-in Chromium that draws web-based overlays and docks inside OBS.

Sources OBS Studio 33.0.0 release

More like this Less like this

Learn one thing

What is Amdahl's law?

Amdahl's law says that speeding up one part of a job can only save the time that part took. If the part you improve is half the job, the whole job can at best get twice as fast.

Think of it like thisA trip is a 20-minute walk to the station and a 20-minute train ride. A train twice as fast saves 10 minutes; even a teleporting train saves only 20, because you still have to walk.

Computer architect Gene Amdahl described it in 1967 for parallel computers: adding processors only speeds up the work that can be split between them. The rest sets a hard floor.

It applies to any software. A web request that spends half its time waiting on the database cannot get more than twice as fast from faster code. That is why profiling, which measures where time actually goes, comes before optimising.

Today it shows up three times: Goedecke's case against language rewrites, Google's lazy imports that left time to first request flat because the cost moved elsewhere, and a benchmark where agents wrote correct code that did not make the system faster.

The part you do not speed up decides how fast the whole thing can get.
profiling
Measuring where a program actually spends its time, function by function.
parallel computers
Machines that split one job across many processors working at once.

Try this today

Game dev 45 minutes For practitioners

Load an old save in a packaged build after renaming a field

In under an hour you will know whether a renamed save-game variable survives in the game players actually run, not just in the editor.

You needAn Unreal Engine 5 project that saves with a SaveGame class and the standard Save/Load Game to Slot nodes.

  1. Make an old save

    Package the current build, play it, and save a game with a distinctive value in one SaveGame property, for example a score of 1234. Copy the .sav file somewhere safe.

  2. Rename the property

    Rename that property in your SaveGame class and add a property redirect from the old name to the new one under [CoreRedirects] in DefaultEngine.ini.

  3. Check in the editor

    Put the old .sav back in the project's SaveGames folder, load it in Play In Editor, and print the renamed property.

  4. Check in a packaged build

    Package again, put the old .sav in the packaged game's SaveGames folder, load it, and print the same property.

[CoreRedirects]
+PropertyRedirects=(OldName="/Script/MyGame.MySaveGame.OldScore",NewName="/Script/MyGame.MySaveGame.Score")

You know it worked whenIf the editor prints 1234 and the packaged build prints 0, you have reproduced the gap the Savekeeper author describes, and you need a migration step or a loader that applies redirects before you ship the rename.

For youRun it on your inventory plugin's save class before the next Fab update; if the gap is real, document it for buyers and ship a migration helper.

Play In Editor
Running the game inside the Unreal editor window instead of as a packaged build.
property redirect
A config line telling Unreal that data saved under an old property name belongs to a new one.

Sparks

What is catching on right now, and the idea worth borrowing from each.

The lab

Research from the front line, in plain words: what it solves, how, and what you could do with it.

Software and AI PreprintarXiv preprint

Pause a coding agent at its first code edit, let a stronger model write one fix plan, and it solves far more bugs for one extra call.

PAUSE: Search-Free Steering of Coding Agents Before Their First Source-Code EditViral Patel, Boyin Tan, Jason Xue, et al. (MBZUAI)

Diagram of a coding run: read, edit a scratch file, run, read, think; the first source edit is blocked by PAUSE, a planner writes one repair plan from the evidence, and the same executor then edits and runs with checks passing.
Image: Patel et al. (arXiv)
The problem
Agents explore and test for a long time and still ship failing patches. Reviewing them on a schedule is expensive, and waiting for an alarm misses runs that look fine but are wrong.
The idea
The first edit to a source file is a natural checkpoint: by then the agent has explored, but not yet committed. PAUSE blocks that edit, gives a planner model the task and the history so far, and appends its single plan (root cause, code to change, intended fix) to the conversation. The agent then carries on with its context and workspace intact.
The result
With a small open coding model as the agent, SWE-bench Verified went from 50.2% to 65.0% solved, for $26 of planner calls across 500 tasks. A planner reviewing on a schedule reached 67.6% but cost $232. In OpenHands the gain was 45.4% to 62.2%.
Use it
Try it as a hook: on a session's first file edit, ask a stronger model for a short root cause and fix plan, and feed it back as context. Measure solved tasks and tokens on your own bug list.

For youA first-edit PreToolUse hook that calls a stronger model once is about 30 lines; worth testing on plugin bug fixes where the agent tends to patch symptoms.

More like this Less like this

  1. Idea
  2. Prototype
  3. Code released
  4. Shipping

ReadPaper (arXiv)Full text (HTML)

Software and AI PreprintarXiv preprint

Coding agents write correct code that does not make the system faster: the best setup undid only 42% of a planted slowdown.

Benchmarking Coding Agents for System-Level Performance OptimizationYanze Zhang, Maggie Yongqi Guan, Shudong Liu, et al. (University of Macau)

Four-stage process figure: 58 anti-patterns narrowed to 9, 28 repositories narrowed to 7 systems, anti-patterns injected and reviewed by experts, and contamination checks.
Image: Zhang et al. (arXiv)
The problem
Agent benchmarks test single functions or repos, but real performance problems in services come from how the services talk to each other.
The idea
MicroPerf-Bench plants 9 kinds of known performance mistakes, such as chatty services and long call chains, into 7 real open-source microservice systems without breaking them: 65 tasks, each with tests and a load check.
The result
Over 1,755 runs and 9 agent and model setups, the best (OpenHands with GLM-5.2) scored 42.0 out of 100. The usual failure: changes that pass the tests and break no rules but do not speed the system up, because the agent picked the wrong bottleneck.
Use it
Before asking an agent to optimise, hand it a profile or trace and make a before and after load-test number the definition of done. A green test run is not evidence of a speedup.

For youSame rule for your SaaS work: give the agent the slow query log or p95 per endpoint, not just 'make it faster'.

More like this Less like this

  1. Idea
  2. Prototype
  3. Code released
  4. Shipping

ReadPaper (arXiv)Code

Game dev PaperACM Transactions on Graphics

A small change to the standard stable solver lets soft bodies keep their bounce at big time steps, instead of slowly going dead.

Energy-Controllable Time Integration for Elastodynamic ContactKevin You, Juntian Zheng, Minchen Li (Carnegie Mellon University)

Five panels of a soft armadillo figure falling and bouncing on a checkered floor over frames 0 to 260, with ghosted trails showing its motion.
Image: You, Zheng and Li (arXiv)
The problem
Solvers that stay stable at large time steps, such as implicit Euler, quietly drain energy, so jelly and cloth lose their life. Solvers that conserve energy tend to blow up on stiff contact.
The idea
A modified implicit Euler step that aims at an energy target you choose, keeping energy in the large, visible motions. It works with barrier-style contact, so it keeps the guarantees of no inverted elements and no interpenetration.
The result
In a soft collision test at 1/30 s steps, the method ended closer to the reference speed (0.819 m/s against 0.864) than BDF2 (0.776). Most other results are visual comparisons; no code was found.
Use it
If you write your own soft-body, cloth or jelly solver, add an energy target to the step so bouncy objects stay bouncy at game frame rates.

For youOnly relevant if a world-tools plugin ever ships its own soft-body solver; Chaos users can skip it.

More like this Less like this

  1. Idea
  2. Prototype
  3. Code released
  4. Shipping

ReadPaper (ACM TOG)Preprint (arXiv)

Also moving

This week on Toa Radar

Read the weekly summary: The week agents got fences and rulebooks
Tune mode is on in this browser. Turn it off