Code reviews are one of those things everyone agrees are important & yet they’re often the first casualty of a busy sprint. On a team, a PR sits in the queue waiting for someone who has enough context and enough uninterrupted time to look at it properly. And on a solo project, the reviewer is usually future-you – and future-you is not always available. 🙂
AI powered code review looks like the obvious fix for this. Let a model inspect every PR the moment its opened & point out possible bugs, missing error handling, security issues and code which is more complicated than it needs to be.
But here’s the catch – a bad automated reviewer does not save time, it creates more work. A review with 20 useful observations is valuable. A review with 50 comments where 30 are irrelevant, repetitive or plain wrong is noise dressed up as productivity. People stop reading the comments after a while and then a real issue gets buried in all the nonsense – at that point the tool is worse than useless.
So can an LLM comment on a pull request? Sure, almost any decent model can do that now. The real question is – can it produce a review which a developer will keep taking seriously?
AI review as the first pass
I don’t see AI review as a replacement for a good human reviewer, especially for changes touching authentication, payments, data migrations or architectural decisions. But as a first pass before a human review, it makes a lot of sense. It catches the mechanical stuff – missing error handling, dead code, input validation gaps, tests which don’t cover the changed behaviour, inconsistencies across files which a linter cannot understand. The author fixes the simpler issues & the human reviewer gets a cleaner PR to look at, spending their (expensive) attention on design & intent instead of pointing out a loose == which should’ve been a ===.
And then there are projects where a human review simply does not happen – personal projects, small internal tools, understaffed teams. There the choice is not AI review vs human review, its AI review vs no review. A model is not infallible but a reasonably good review beats merging code which nobody other than the author has examined.
One more case which is increasingly relevant – reviewing AI generated code. If an LLM wrote the code then asking the same model in the same session whether its own work is correct is not much of a review. A separate review pass with fresh context – ideally a different model – has a much better chance of challenging the original assumptions.
What about just using a coding agent for reviews?
Before we go further, let’s talk about the option most of us already have installed – coding agents. Code review is now a first class feature in these tools. Codex CLI ships a /review command – a dedicated reviewer which reads your uncommitted changes, a commit or a diff against a base branch & reports prioritized findings without touching your working tree (you can even pin a separate review_model for it in config). Claude Code has a /code-review command which runs multiple review agents in parallel over a PR branch, filters findings by confidence & can post the result as a PR comment – plus a separate /security-review. OpenCode, being provider agnostic, will happily review a diff with whichever model & provider you’ve configured. Even Cursor has review option built into the editor – you can run a review over your local changes right from source control – along with Bugbot, its separate PR reviewing bot for GitHub & GitLab.
This is a perfectly valid way of getting AI code reviews and I use it myself. The strengths are real. There is zero setup – the tool is already installed, already authenticated & already covered by the subscription or API key you’re paying for anyway, so the marginal cost of one more review feels like nothing. And if you have a subscription like on Claude Code or Codex or Kimi Code then the review costs practically nothing. It can review code before a commit even exists – a CI reviewer can only see a pull request while a local agent can review uncommitted changes and the cheapest place to catch a bug is before it ever leaves your machine. The agent is not limited to the diff either – it can read surrounding files & chase references and the review becomes a conversation; you can push back, ask why and when a finding is legit the fix is one prompt away in the same session. Your project rules apply too – review instructions living in CLAUDE.md / AGENTS.md shape the review the same way they shape code generation.
Where it falls short is exactly where it sounds like it would. It runs only when the author remembers to run it – a local review is a discipline, not a guarantee and under deadline pressure discipline is the first thing to go, which is precisely when review matters the most. Though coding agents like Claude Code, Codex, OpenCode, etc. can nowadays be configured to do code reviews before they commit any changes or after they’ve finished writing code. Creating a custom command/skill for doing code review and having it run automatically is rather straightforward nowadays.
Then there is the self review bias – if the same agent session wrote the code, asking it to review its own work in the same context is not much of a review; it will happily confirm its own assumptions. A fresh session helps, a different model helps more. Fine for personal hygiene, useless as a shared quality bar.
And frankly, you can automate these – both OpenAI & Anthropic ship official GitHub Actions for their agents and Anthropic offers a hosted Code Review product for Claude Code. But then you’re configuring a general purpose agent harness (with shell access, sandboxing decisions & all) for a job which only needs to read a diff & post comments. Anthropic’s hosted reviews are billed on token usage & their docs quote an average of $15–25 per review at the time of writing. That is real money at PR volume.
So my take is simple – coding agent review is the right tool for the inner loop & the wrong tool for the team guarantee. Use it for self review before you push, for a deep interactive pass on one tricky change or as a second opinion from a different model on AI generated code. Don’t rely on it when you need every PR reviewed without exception, a visible record on the PR itself or a merge gate. The two are not competing – an agent review before the push & an automated review on the PR complement each other rather nicely.
For the PR side of that equation, the usual answer has been the hosted review products. Which brings me to my gripe.
The tooling landscape (and my gripe with it)
I’ve tried a bunch of the hosted AI review tools over time & my experience has been underwhelming. Some tools would claim to use top tier models like Claude Opus for their reviews. Having used Opus extensively myself – seeing the quality of those reviews & the nonsensical noise they generate, any sensible person can deduce that whatever is producing them, it isn’t Opus running with any reasonable configuration. And irrespective of what is actually running behind the scenes, the practical result was the same – too many comments which I could not trust.
There is a lesson in that beyond one product. An AI review tool is more than the model name on its pricing page – the prompts, the context gathering, the diff processing & the comment publishing – all of it shapes the final review. Put a good model inside a poor review pipeline & you can still get a poor review. With a hosted tool you control none of it – the vendor picks the model, the reasoning configuration & the pricing – you get whatever falls out.
The open source side was not exactly buzzing either. For the longest time, PR-Agent was pretty much the only practical choice if you wanted a self run, bring your own model review pipeline for your PRs. I gave it a shot sometime back but it just did not click for me.
So where do we go from here? I found something which clicked & I ended up creating tooling around it for automating PR reviews. More on that in Part 2.