Code review
Code review
Codebot reviews every pull request on the repositories you turn on, and posts what it finds where your team already reads reviews.
Turn on a repository in Code review and Codebot reviews its pull requests automatically: when they open, when new commits are pushed, and when a draft is marked ready. Nothing to configure in CI.
What a review looks like
A review is posted on the pull request itself, as Codebot (the TryCodeBot app on GitHub):
- Inline comments on the lines with problems. Each finding has a severity, a short explanation and, where it helps, a suggested fix.
- A summary comment, kept up to date as the pull request changes: the verdict, the commit that was reviewed, how many earlier findings are now resolved or still open, and the reviewer’s overview. See Re-reviews and summaries.
- A verdict: a comment, an approval or a request for changes, depending on your review settings.
- A check on the head commit, which can block the merge while critical issues are open.
You read and discuss the review on the pull request, like any other review. The web app keeps the run details: status, duration and a transcript of what the reviewer looked at.
Severity
Every finding has one of four severities:
| Severity | Meaning |
|---|---|
| Critical | Breaks production, loses or corrupts data, or opens a security hole. Blocking. |
| High | A real bug or security problem that should block merging. Blocking. |
| Medium | A likely bug, missing tests for risky logic, or a maintainability problem worth fixing before merge. |
| Low | A minor suggestion or nit. |
Critical and high findings are blocking issues. They are always posted, whatever your inline comment setting, and they’re what the approval rules and the merge gate act on.
What the reviewer looks at
The reviewer gets the pull request’s title, description and diff, a checkout of the repository at the pull request’s head commit, and your repository guidelines. It reads the surrounding code to check how a change fits, focusing on the riskiest changes first.
Reviews are read-only: the reviewer doesn’t install dependencies, build the project, run tests or scripts, or push anything.
Time limits and large pull requests
A review has about 20 minutes. Five minutes before the limit, Codebot tells the reviewer to wrap up, so it posts what it found and says what it didn’t get to, rather than timing out with nothing.
Very large pull requests don’t fit in one prompt. Codebot fills the reviewer’s view with whole files, source code first, then tests, deleted files, and lockfiles or generated code last. Files that don’t fit are listed with their line counts, and the reviewer opens the risky ones from the checkout. The summary says which files it didn’t review.
Keep lockfiles and generated code out of reviews entirely with ignored files.
Review on demand
Comment @trycodebot review on a pull request to start a review right away. It runs even when a skip rule would skip the pull request, and reviews the whole pull request rather than only the commits since the last review.
On GitHub, owners, members and collaborators of the repository can ask for a review. On Azure DevOps, mention the account you connected Codebot with, for example @Codebot review.
Pull requests from forks
Pull requests from forks are skipped by default. A review runs the fork’s code in a sandbox that holds your model keys, so only turn Skip forks off for repositories where you trust contributors. On GitHub, a member can still review a fork’s pull request with @trycodebot review.