Cloudflare cut its AI code review bill by 68%
Cloudflare runs a team of AI agents that reviews its engineers' code changes. Over a few weeks it cut the average cost per review from $2.43 to $0.77, and checked that the useful findings did not go away.
The reviewer is several agents: a coordinator that hands each code change to specialists for code quality, security, performance, documentation and compliance. Costs had crept up from $1.19 a review in April to about $2.46 by late August, so one engineer went through the bill line by line.
The biggest single fix was caching. Every reviewer's first request now starts with the same block of tool definitions and shared rules, so the provider can reuse it instead of charging full price again. Then came a series of small cuts: removing tools reviewers could not use, skipping reviewers on re-reviews that did not need them, stopping as soon as the review was posted, and killing cancelled jobs that kept running.
Model choices were tested, not assumed. A split test found the newer mid-size model 47% to 62% cheaper per run, so all those reviewers moved to it. Of the $1.67 saved per review, $0.65 came from code changes and $1.06 from model and effort settings.
$0.77average cost per review, down from $2.43
Technical details
Cost history as reported: $1.19 per review in April, about $2.46 by late August, $2.43 averaged over the two most expensive weeks, $0.77 now. Median review time fell from 4m 09s to 1m 32s. The shared cached prefix (September 16) took cost per review from $2.47 to $2.02 on its own; the call-trimming changes of September 21 to 23 saved about $0.20 more, including about $0.07 per job from stopping once the review is posted.
Three sub-reviewers share one first request: the first writes the cache and the other two wait up to 20 seconds to read it, saving about nine cents per reviewer. The coordinator moved to Opus 5.5 at medium effort; Sonnet reviewers moved from Sonnet 5 to Sonnet 5.5 after a split test; code quality runs at high effort, adding about two cents per review. The reviewer runs on OpenCode v2.
Quality checks: a finding counted as acted on if its flagged lines changed in the author's next push (a baseline of nearby lines changed about 15% of the time). Serious code quality findings acted on rose from 34% to 59%. Two models from different providers judged 657 findings blind: important findings per review went from 0.28 to 0.30, and findings judged too severe fell from 0.46 to 0.14 per review. Medium effort did lose real performance findings (about 64% fewer important ones), and the author notes the before and after steps are not a controlled experiment.
Why it mattersAgent pipelines in CI are billed per token, and most of that bill is often re-sent context. Measuring whether the first request hits the cache, then testing cheaper settings against a quality metric, is a playbook any team can copy.
For youThe same levers apply to your routines and subagent-heavy sessions: put the stable rules and tool list first, drop tools a subagent never uses, and check cache reads on the first request in your API usage before touching models.
- caching
- Reusing the start of a request the model provider has already processed, which is billed at a fraction of the normal price.
- split test
- Sending a random half of real jobs to each option and comparing the results, so the difference comes from the option and not the jobs.
- effort
- A setting that controls how much a model thinks before answering; more effort costs more tokens.
