Guide · Updated October 2026 · Last reviewed October 29, 2026

How to review AI-written code: a checklist for agent PRs

Coding agents made writing code cheap. Checking it didn't get cheaper. A large Harvard study found that after firms adopted AI coding agents, pull requests took 49% longer to get through review, and reviewers asked for changes almost twice as often. This guide is a practical, tool-neutral checklist for reviewing a pull request an agent wrote: what to look at first, where agent code tends to go wrong, and how to set up your repo so CI on a clean checkout, not the agent's own terminal, decides what merges. Every study figure and GitHub setting below was checked on the primary source on October 29, 2026.

1. The verification gap, in numbers

The study is "Artificial Intelligence in the Firm: Bottlenecks in Software Production" by Fiona Chen and James Stratton of Harvard University (job market paper; first version January 7, 2026, current version August 4, 2026). It uses about 300 million work events (GitHub activity, Jira issues and Google Calendar events) from 718 firms, provided by the engineering analytics platform Jellyfish, and compares firms before and after they adopted AI coding assistants and agents.

After firms adopted AI coding agents	Change	Baseline
Lines of code	+30%	5,017 lines per worker-month
Commits / pull requests	+20% / +23%	Not restated here
Delivered output (Jira issues and epics resolved)	Small, not statistically significant. Rules out a rise above 12%	n/a
Time from PR submitted to merged	+49% (+3.45 days)	7.03 days
Share of PRs with changes requested	+12 points, "nearly doubling the rate"	13% of PRs
Review comments per PR	+35% (+0.58)	1.66
Share of workers doing code review	+14%	29% of workers

The authors' reading: writing code is only one step, and "written code must be reviewed and tested before it can be deployed to production." When agents speed up the writing, review becomes the bottleneck. They also report that the bottleneck "persists following the adoption of AI code review tools" and that "human reviewers continue to play a central role." AI coding assistants (autocomplete and chat) showed no significant effect on review time. The jump showed up with agents.

Our take: this is a working paper, not a peer-reviewed article, and it measures firm-level averages, not your team. But it matches what solo builders feel: the agent finishes in two minutes and you spend twenty working out whether to trust it. The fix isn't to stop reviewing. It's to review in a set order, and to let machines check what machines can check.

Chen & Stratton, Harvard (PDF, current version Aug 4, 2026) · figures from the introduction and section on code review

2. Rule one: CI on a clean checkout is the gate

A Hotfix Daily reader corrected one of our early tips, and they were right: "proof of done" isn't the agent's terminal output. The agent ran its tests in its own working copy, with your local files, caches, environment variables and whatever it installed along the way. None of that ships.

CI starts from a fresh checkout of the commit you're about to merge. That catches the file the agent created but never added, the test that only passes because your .env is there, the dependency installed globally but missing from the lockfile, and the step it quietly skipped. So:

Treat "All tests pass" as a claim. Look for the green check on the PR, not the sentence in the chat.
Make the CI job required. On GitHub, once required status checks are on, "all required status checks must pass before collaborators can merge changes into the protected branch" (docs).
Run the same commands CI runs. Use the clean-install command for your stack (for example npm ci, not npm install) so the lockfile is what gets tested.
If CI is red, the PR isn't done, however confident the summary sounds.

Our safe-setup guide has a minimal GitHub Actions workflow you can copy.

Agents change every week. Your review habits shouldn't have to.

Hotfix Daily gives you five things worth knowing in AI coding and agents, with sources, plus one practical fix. Free, weekdays, about 3 minutes.

Join free and get 13 ready-to-paste Starter Prompts for coding agents, vibe coding and game dev right away.

Follow by RSS

3. The 5-minute first pass

Before reading a single line of logic, answer these. Most bad agent PRs fail here, and it costs you minutes, not an afternoon.

Does the PR say what it was asked to do? Put your original request next to the description. If the agent "also refactored" things you didn't ask for, send those back as a separate PR.
Which files changed? Run git diff --stat main...HEAD (or read the Files changed tab). Files outside the area you named are the first warning sign.
How big is it? If you can't review it in one sitting, ask the agent to split it. Big diffs are where things slip through.
Did any tests change? Check test files before the code. New tests are good. Edited or deleted tests need a reason.
Did config change? Look for AGENTS.md, CLAUDE.md, agent settings folders, .mcp.json, .github/workflows/, package.json scripts, lockfiles, Dockerfiles and .gitignore. These change how code runs or gets checked, so read every line.
Is CI green on this exact commit? If not, stop here.

4. The full checklist for agent PRs

Once the first pass is clean, go through these in order. They're tool-neutral: they work the same whether the PR came from a terminal agent, an IDE agent or a cloud agent.

Scope and intent

The change solves the problem you described, not a nearby easier one.
No unrelated renames, reformatting or "cleanup" that bloats the diff.
Nothing was deleted to make an error go away (a removed check, a commented-out block, a catch that swallows errors).
Any TODOs or placeholder values the agent left are listed in the PR, not hidden in the code.

Tests

There is at least one test that fails without the change and passes with it. Ask the agent to show it failing first if you're unsure.
No tests were skipped, deleted or marked as expected failures without a written reason.
Assertions weren't loosened to fit the new output (a strict equality turned into "is not null", a number range widened, snapshots regenerated wholesale).
Mocks don't replace the very thing under test. A test that mocks the function it claims to test proves nothing.
Edge cases are covered: empty input, missing fields, very large values, time zones, errors from the network or database.

Correctness and "does this exist?"

Every function, method, flag and config key the code calls actually exists in the version you use. Agents sometimes write plausible calls to APIs that were renamed, deprecated or never existed. Check unfamiliar ones in the library's docs.
Error handling does something useful. Watch for empty catch blocks, errors logged and then ignored, and retries with no limit.
The code doesn't duplicate a helper that already exists elsewhere in your repo.
Run it yourself once: click through the feature or call the endpoint. Reading isn't the same as using.

Dependencies

Every new package is one you meant to add. Search an unfamiliar name before you trust it: check that it's the real, maintained package and not a look-alike.
The lockfile changed only as much as the manifest change explains. A one-line manifest change that rewrites hundreds of lockfile lines deserves a look.
On GitHub, dependency review shows "which dependencies were added, removed, or updated" and "vulnerability data for these dependencies" in the PR's Files changed tab (docs).

Security and data

No secrets, tokens or real customer data in code, tests, fixtures or logs. GitHub's push protection "blocks pushes that contain secrets before they reach your repository" (docs), but don't rely on it alone.
Every new route, endpoint or database query checks who the user is and what they're allowed to see. Missing permission checks are easy to miss because the code still "works".
User input is validated, and queries use parameters, not strings glued together.
Nothing widened permissions: CORS set to allow everything, a database rule opened up, a workflow given write tokens it didn't need.

Before you merge

Database migrations can run on real data and can be rolled back.
New settings or environment variables are documented, with safe defaults.
The PR description says how it was verified: which tests, which manual check, and the CI run.
CI is green on the latest commit, after the last push, not an earlier one.

5. Red flags in agent-written PRs

You see	What to check
"Fixed the failing test"	Did it fix the code, or change the test? Read the test diff first.
"All tests pass" but no CI run	Push it and wait for CI on a clean checkout. The agent's terminal isn't proof.
A new package you don't recognize	Is it real, maintained and the one you meant? Was it needed at all?
Changes to workflow, agent or build config	Read every line. These files decide what gets checked and what runs.
A very large diff for a small ask	Ask for a smaller PR limited to the request.
try/catch added around the failing line	Is the error handled, or just hidden?
A call to a function you've never seen	Check it exists in your library version.
Hard-coded values in place of config	Test-only shortcuts that will break in production.

6. Make the repo enforce it (GitHub)

Checklists slip when you're tired. Branch protection doesn't. GitHub's protected branches page lists the settings that matter for agent PRs. On main:

Require status checks before merging. Pick your CI job. With "Require branches to be up to date before merging" (the strict setting), the branch must include the latest main before it can merge.
Require pull request reviews before merging, with at least one approving review.
Dismiss stale pull request approvals when commits are pushed, so an approval doesn't carry over to code the agent changed afterwards.
Require approval of the most recent reviewable push by "someone other than the person who pushed it". Useful when an agent pushes under your account.
Require review from Code Owners for sensitive paths. Code owners "are automatically requested for review when someone opens a pull request that modifies code that they own" (docs). Put .github/workflows/, auth code and migrations behind it.
Require conversation resolution before merging, so review comments can't be left hanging.

Our take: if you work alone, required status checks are the one to turn on today. Required reviews matter once a second person, or a second agent, can open PRs.

About protected branches · About code owners · checked Oct 29, 2026

7. Make review cheaper before the PR exists

The study's point is that review is now the slow part. You can shrink it at the start:

Ask for a plan first. Agree on the files and approach before the agent writes code. Rejecting a plan is cheaper than rejecting a diff.
One task per PR. Small, single-purpose PRs are faster to review and easier to roll back.
Ask for the test first. Have the agent write a failing test for the bug or feature, then the fix.
Put your rules in the repo. An AGENTS.md (or your agent's equivalent) with "don't edit tests to make them pass", "no new dependencies without asking" and "run the full test command before saying done" saves the same review comment every time.
Ask for a verification note. Have the agent end every PR description with what it changed, what it ran, and what it didn't check.

A prompt you can paste at the end of any agent task:

Before you say you're done:
1. List every file you changed and why.
2. List any test you added, edited, skipped or deleted, and why.
3. List any new dependency and why it's needed.
4. Paste the exact test command you ran and its output.
5. Tell me what you did NOT verify.
Then open a pull request and wait for CI.

The printable version

CI green on the latest commit, from a clean checkout.
Diff matches the request. Nothing extra.
Test diffs read first. No weakened or deleted tests.
Config, workflow and agent files read line by line.
New dependencies checked by name.
Unfamiliar APIs checked against the docs.
Auth, input validation and secrets checked on every new path.
Ran the feature once yourself.
Branch protection: required checks, fresh approvals.

FAQ

Does AI-generated code take longer to review?

In one large study, yes. A Harvard working paper by Fiona Chen and James Stratton (current version Aug 4, 2026), covering 718 firms, found that after firms adopted AI coding agents the time from opening a pull request to merging it rose 49% (from a 7.03-day baseline), the share of PRs with changes requested nearly doubled, and comments per PR rose 35%. It is a working paper, not a peer-reviewed article.

Is it enough that the agent says the tests pass?

No. The agent ran the tests in its own working copy, with your local files, caches and environment variables. CI runs them on a fresh checkout of what was actually committed, which catches files the agent forgot to add and tests that only pass on your machine. Make that CI check required before merging.

What should I look at first in an AI agent pull request?

Start with the list of changed files (git diff --stat) and compare it with what you asked for. Then check whether any test files changed, whether new dependencies appeared, and whether agent or CI config files such as AGENTS.md, .github/workflows/ or package.json scripts were touched. Only then read the code line by line.

Can an AI code reviewer replace a human reviewer?

Not on its own. In the Harvard study, the review bottleneck persisted after firms adopted AI code review tools, and human reviewers stayed central. An automated reviewer is useful as a first pass, but a person should still check intent, tests and anything touching security, money or data.

How do I make GitHub enforce these checks?

Use a branch protection rule or ruleset on main: require a pull request with at least one approving review, require status checks to pass (your CI job), and optionally require branches to be up to date, dismiss stale approvals when new commits are pushed, require approval of the most recent push by someone other than the pusher, and require review from code owners.

Found this useful? Hotfix Daily covers AI coding agents and the habits that keep them honest every weekday. Get the updates free, plus 13 Starter Prompts. Or share it on X.

How we write these: we read the primary source on the date shown and link it for every figure and setting we name. Study figures come from the authors' own PDF. "Our take" lines are our opinion. This guide is tool-neutral and has no affiliate links or sponsors.