AI Code Review System

Orchestration. The orchestrator coordinates the run and does not review code itself. It fetches the diff, routes files to the relevant agents, collects their results, removes duplicates, and posts the final report. When two agents flag the same line, a fixed priority order decides which finding is kept, so the outcome does not depend on chance.

Specialized agents. Nine agents review the same change, each through one lens such as security, React patterns, or performance. I chose narrow scopes because a short, focused prompt is easier to calibrate and test than one prompt covering everything. This is a design choice; I have not benchmarked it against a single-prompt review.

Model assignment. In the version described in the linked article, pattern-matching checks (types, styles, Storybook) run on Haiku, reasoning-heavy checks (security, algorithmic complexity, architecture) on Sonnet, and orchestration on Opus. The aim is cost control: lighter models for simpler checks. I have no measured cost comparison to report.

Pre-review filter. A shell script removes generated files, lockfiles, vendored code, and unrelated changes before any agent sees the diff.

Finding validation. Before a finding is posted, the orchestrator applies these checks:

• Existence: the flagged code must appear in the diff, because agents can report changes nobody made.
• Evidence: each finding must quote 1-3 lines of the changed code.
• Style: purely stylistic suggestions are dropped.
• Wording heuristic: findings phrased with hedging words such as "might" or "consider" are dropped. This is a heuristic for removing uncertain comments. Wording alone does not show whether a finding is correct, so it can discard valid findings and let weak ones through.

Failure handling. If one agent fails, the others continue and the report contains the remaining results. This covers errors in a single agent only; an API outage, a CI failure, or a network problem can still stop the review.

What exists. A working pipeline of Markdown prompt files and a shell pre-filter, designed to run in a GitLab merge request workflow and post one structured comment per merge request. Each report separates merge blockers from suggestions and gives severity labels and exact line references.

What I observed. Calibrating the quality gates took the most time. Filters that were too strict missed relevant findings, and filters that were too loose let noise back in. I tuned them over repeated test runs and manual review of the results. I did not measure false-positive rates, time savings, or API costs, so I make no quantitative claims.

Limits. Applying the same checks to every merge request makes the process repeatable, but I have not measured review quality, and model output can still miss real problems or report wrong ones. A report is input for the reviewer, not approval to merge, and the merge decision stays with a person.

Dependencies. There is no custom Python or Node.js application, but the pipeline relies on GitLab CI, a shell script, and access to Claude models.