pruefbyte

AI code review for GitLab merge requests, in the spirit of pr-agent. The review itself is done by OpenCodeReview (ocr). pruefbyte runs it in a merge request pipeline and posts the findings as inline discussions from a dedicated bot account.

• Findings are anchored to the right diff line, with GitLab "apply suggestion" blocks where possible.
• Re-running the pipeline never duplicates a comment.
• A push only sends the files whose diff changed to the model; files reviewed cleanly before are not reviewed again.
• The bot resolves its own threads once the flagged code has changed and the finding is gone.
• Findings that can't be placed on the diff, or that go over the comment limit, go into a single summary note, which is updated in place.
• Works with every LLM provider OCR supports (Anthropic, OpenAI, Bedrock, DashScope, OpenRouter, …) and with any OpenAI- or Anthropic-compatible endpoint.

Setup

1. Bot user. Create a GitLab user (e.g. pruefbyte-bot) and add it to your group or projects as Developer. Create a personal access token for it with scope api.

2. CI/CD variables (group level, masked). Only secrets go here:

   Variable	Value
   PRUEFBYTE_GITLAB_TOKEN	the bot's PAT
   PRUEFBYTE_LLM_API_KEY	the LLM API key

   Provider, model and all review settings go into the repository's .pruefbyte.yml (see Configuration), so that pruefbyte local reviews with exactly the same settings. A PRUEFBYTE_LLM_MODEL or similar CI variable would override the file in CI only, and local runs would no longer match.

3. Image. CI publishes ghcr.io/feinarbyte/pruefbyte for linux/amd64 and linux/arm64: latest from main, X.Y.Z and X.Y from vX.Y.Z tags, and sha-<commit> for each of these pushes. If the package is private, give the GitLab runners pull access (a GitHub token with read:packages in DOCKER_AUTH_CONFIG). To build your own:

   docker buildx build --platform linux/amd64,linux/arm64 \
     -t registry.example.com/tools/pruefbyte:latest --push .

4. Pipeline. Include the template in each project (or in a shared CI config):

   include:
     - project: tools/pruefbyte
       file: templates/pruefbyte.gitlab-ci.yml
   variables:
     PRUEFBYTE_IMAGE: ghcr.io/feinarbyte/pruefbyte:latest

   The job runs in merge request pipelines (not in merge train pipelines, and not in pipelines that run in a fork, which lack the CI/CD variables), in the test stage: if the project defines stages:, keep test or override the job's stage:. If the project's other jobs run in branch pipelines, switch to merge request pipelines while a merge request is open, as GitLab recommends. Otherwise every push runs two pipelines, and the merge check only looks at the merge request pipeline, which then holds nothing but this job:

   workflow:
     rules:
       - if: $CI_PIPELINE_SOURCE == "merge_request_event"
       - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
         when: never
       - if: $CI_COMMIT_BRANCH

Configuration

Settings are layered; later layers win:

1. Built-in defaults.
2. The global file, from --config or PRUEFBYTE_CONFIG. See pruefbyte.example.yml for every key.
3. .pruefbyte.yml in the repository, read from the merge request's base commit. A merge request can't change its own review settings: config changes take effect once they are merged. The same holds for OCR's own rule files, .opencodereview/rule.json and ocr.rule_file: pruefbyte reads them at the base commit and passes one rule file that keeps the merge request's copies from applying.
4. Environment variables PRUEFBYTE_<SECTION>_<KEY>, e.g. PRUEFBYTE_REVIEW_MIN_SEVERITY=medium. Lists are comma-separated.

The repository file may set llm.provider (OCR built-in providers only, and only when the global config does not set one: a provider the operator names keeps the shared API key with that vendor), llm.model, ocr.* (except binary and extra_args) and review.*. Anything that decides where credentials are sent, or what gets executed, is rejected there: custom providers with their own llm.url belong in the global file.

llm.model can be any model ID the provider serves. With the OCR that pruefbyte bundles (the Docker image and release binaries, OCR v1.12.12 or newer), the model lists of OCR's built-in providers are only suggestions: a model that is not listed works, and OCR logs [ocr] WARNING: model "…" is not in the suggested models for provider "…"; the provider will validate it. A wrong ID therefore fails at the provider's API, not earlier. OCR v1.12.10 and older rejects unlisted models; that is what pruefbyte 0.1.0 bundles, and what an older ocr on your PATH (with go install builds, or set via ocr.binary) may still do. pruefbyte version shows which OCR is used.

Secrets are only ever read from the env vars named by gitlab.token_env and llm.api_key_env. ocr runs with a private, temporary HOME, so its config file and session logs never touch the runner.

Example .pruefbyte.yml:

llm:
  provider: anthropic
  model: claude-sonnet-5
ocr:
  effort: high
  exclude: ["**/generated/**"]
  rules:
    - path: "**/*.go"
      rule: "Errors must be wrapped with %w; flag bare returns of err from external calls."
      merge_system_rule: true # add to OCR's built-in Go rules instead of replacing them
review:
  min_severity: medium
  max_comments: 15

When it runs

pruefbyte reviews the merge request whenever it is run; it has no draft, label, author or branch filters of its own. When it runs is decided by the CI job's rules:. Override the job in your project to add conditions, for example:

pruefbyte-review:
  rules:
    - if: $CI_MERGE_REQUEST_EVENT_TYPE == "merge_train"
      when: never
    - if: $CI_PROJECT_ID != $CI_MERGE_REQUEST_PROJECT_ID
      when: never
    - if: $CI_MERGE_REQUEST_DRAFT == "true"             # skip drafts
      when: never
    - if: $CI_MERGE_REQUEST_LABELS =~ /(^|,)no-review(,|$)/
      when: never
    - if: $GITLAB_USER_LOGIN == "renovate-bot"
      when: never
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^release\//
      when: never
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

Overriding rules: replaces the template's list, so keep its first two entries (merge trains, fork pipelines). To review on demand only, use when: manual.

The pruefbyte-full label works the other way round: it does not decide whether the job runs, but makes pruefbyte review every file of the merge request again instead of reusing earlier reviews (see Reusing earlier reviews). pruefbyte reads it from the merge request when the job runs, so add it and rerun the job. It applies to every pipeline while it is set; remove it to go back to reuse.

Behaviour

Situation	What happens
MR has new commits since the pipeline started	Skipped; the newer pipeline reviews it.
Finding on a line in the diff	Inline discussion. Suggestion block if OCR proposed replacement code for exactly the lines it quotes.
Finding about removed code	Inline discussion on the removed line, without a suggestion block.
Finding outside the diff, or GitLab rejects the position	Listed in the summary note.
Same finding as an earlier run (open or resolved)	Not posted again. Matched by a hidden fingerprint of file + the code the finding quotes (its text if it quotes none), so rewording doesn't count as new.
Earlier bot thread not reported again and its code changed	Reply and resolve, but only after a complete review that covered the file, and only once: a thread someone reopens stays open.
OCR fails	A failure note with the redacted error; job exits 1.
File's diff unchanged since a review that found nothing in it	Not sent to the model again; the summary note says how many files were reused and from which review. Its threads stay as they are.
review.fail_on_severity reached	Comments are posted; job exits 3.

Reusing earlier reviews

Fix rounds after review findings usually change a few files, yet each pipeline would review the whole diff again. With review.reuse_unchanged: true (the default), pruefbyte only sends OCR the files that need it:

• A file is reused when its diff is the same as in an earlier review that completed it and reported nothing for it: same path, same blob at the merge request's base and at its head (so a rebase that changes the file on either side reviews it again).
• The settings must also be the same: OCR version, provider, model, effort, language, ocr.extra_args, and the rules and excludes from .pruefbyte.yml, ocr.rule_file and .opencodereview/rule.json. Changing any of them reviews every file. Editing the merge request's description does not. A file that ocr.extra_args names counts by its path only: after editing it, run one full review.
• A file OCR commented on, or warned about, is always reviewed again. Its findings, the review.fail_on_severity gate and the summary note only ever come from a review, and once you fix the finding the file's diff changes anyway.
• After an incomplete review (timeout, ocr.max_tokens_budget, failed groups), only the files OCR completed count as reviewed; the next pipeline reviews the rest.
• Reused files are only left out of OCR's selection. OCR still reads them when a changed file needs them as context.
• A thread on a reused file is neither resolved nor posted again; only a review that covers the file again can resolve it.

If nothing but reused files is left, OCR makes no LLM call, and the summary note says so. What each review covered is kept in a hidden, versioned marker in the bot's summary note, so it needs no cache or artifacts. pruefbyte therefore creates that note whenever there is something to keep, not only when it has findings to list.

Trade-off. A change in one file can break another file whose diff did not change, for example a changed function signature that an unchanged caller now uses wrongly. The unchanged file is not reviewed again, and its earlier review saw the old version of the other file. Review everything again with the pruefbyte-full label (or pruefbyte review --full), or set review.reuse_unchanged: false to always review the whole diff. A full review replaces what is kept.

Local review

Run the CI review on your machine before you push, so the bot has nothing left to say:

pruefbyte local

It reviews what your merge request will contain: everything from the merge base with the target branch (default: origin's HEAD; or e.g. --target origin/develop) up to your working tree, including staged, unstaged and untracked files. Your index, branch and files stay untouched. --committed reviews only commits, which is exactly what CI sees after a push. Findings print in the terminal (--format json for tools), and nothing is posted.

The settings are the CI's, read the same way and from the same places: .pruefbyte.yml and OCR rule files at the target branch, with the same rule merging, excludes, effort, provider and model, and the same review.min_severity / review.categories filtering. A finding that reaches review.fail_on_severity exits 3, as in CI, so the command works as a pre-push hook. If you edit .pruefbyte.yml on your branch, the run tells you that CI, and so the local run, uses the target branch's version until your change is merged. One difference remains: CI gives OCR the merge request's title and description as background; locally the branch name and commit messages stand in.

Like CI, a rerun does not review files again whose diff is unchanged since the last run on the same branch and target, and in which that run found nothing (see Reusing earlier reviews). This is kept in the repository's git directory (pruefbyte/ under git rev-parse --git-common-dir), which all worktrees share and which is never committed. A different --target, or a detached HEAD, reviews every file; --full does so on demand.

The API key is the first one found of:

1. the env var named by llm.api_key_env (PRUEFBYTE_LLM_API_KEY),
2. your own OCR setup (~/.opencodereview/config.json: api_key or api_key_cmd),
3. the provider's env var, e.g. ANTHROPIC_API_KEY.

No GitLab token is needed. If your CI uses a global config file (PRUEFBYTE_CONFIG), pass the same file with --config.

Install

Release binaries include OpenCodeReview (ocr), the exact version CI uses, so a single download is all you need. On first use pruefbyte unpacks it into your user cache directory. Pick whichever install suits you:

With mise:

mise use -g github:feinarbyte/pruefbyte

Linux / macOS, latest release into ~/.local/bin:

os=$(uname -s | tr '[:upper:]' '[:lower:]'); arch=$(uname -m | sed 's/x86_64/amd64/; s/aarch64/arm64/')
mkdir -p ~/.local/bin
curl -fsSL "https://github.com/feinarbyte/pruefbyte/releases/latest/download/pruefbyte_${os}_${arch}.tar.gz" \
  | tar -xz -C ~/.local/bin pruefbyte

Windows (PowerShell), latest release into %LOCALAPPDATA%\Programs\pruefbyte:

$arch = if ($env:PROCESSOR_ARCHITECTURE -eq 'ARM64') { 'arm64' } else { 'amd64' }
$dir = "$env:LOCALAPPDATA\Programs\pruefbyte"; $zip = "$env:TEMP\pruefbyte.zip"
Invoke-WebRequest "https://github.com/feinarbyte/pruefbyte/releases/latest/download/pruefbyte_windows_$arch.zip" -OutFile $zip
Expand-Archive $zip $dir -Force; Remove-Item $zip
[Environment]::SetEnvironmentVariable('Path', "$([Environment]::GetEnvironmentVariable('Path', 'User'));$dir", 'User')

Each release also lists the archives with a checksums.txt (releases).

With Go 1.25 or newer. This builds from source without OCR, so ocr must be on your PATH as well:

go install github.com/feinarbyte/pruefbyte/cmd/pruefbyte@latest
npm install -g @alibaba-group/open-code-review   # or: brew install open-code-review

With Docker, with ocr included and nothing to install. Run it as yourself so new git objects in your repository stay yours:

docker run --rm -it --user "$(id -u):$(id -g)" -v "$PWD:/repo" -w /repo \
  -e PRUEFBYTE_LLM_API_KEY ghcr.io/feinarbyte/pruefbyte pruefbyte local

Check the install with pruefbyte version; it also says whether ocr is built in. An ocr.binary setting overrides the built-in copy.

To try the CI path against a real merge request without posting, set PRUEFBYTE_GITLAB_TOKEN and run pruefbyte review --project group/project --mr 42 --dry-run. pruefbyte config print shows the effective configuration.

Development

mise install            # Go toolchain and golangci-lint
gofmt -l .              # must print nothing
golangci-lint run ./...
go test -race ./...

GitHub Actions (.github/workflows/ci.yml) runs these checks plus go mod tidy on pushes to main and v* tags and on every pull request. It also builds all release binaries (Linux, macOS and Windows; amd64 and arm64) with GoReleaser (.goreleaser.yaml). Once these pass, it builds the multi-arch image. On main and v* tags the image is pushed to GHCR; for pull requests it is only built.

To release, push a tag such as v0.1.0. CI then publishes the image tags 0.1.0 and 0.1, plus a GitHub release with the binaries and checksums. Try the release build locally with goreleaser release --snapshot --clean.

The OCR version is pinned in internal/ocrbin/VERSION, for both the Docker image and the release binaries. To update OCR, change that file. Release builds use the embedocr build tag and embed the gzip-compressed ocr that go run ./internal/ocrbin/fetch downloads and checks against OCR's published checksums. Plain go build and go test need neither.

Layout:

Path	Purpose
cmd/pruefbyte	CLI (cobra) and wiring
internal/config	layered config, repo-file allow-list, env overrides
internal/ocr	drives the ocr CLI and parses its JSON
internal/ocrbin	the pinned OCR version; the ocr embedded in release builds (fetch downloads it)
internal/gitlab	client-go wrapper bound to one MR; dry-run decorator
internal/review	orchestration: filter, place, dedupe, publish, resolve
internal/gitutil	reads the repo config at the base commit; fetches missing commits

OCR is used as a subprocess. Its Go packages all live under internal/, so they can't be imported from another module; its JSON output is the stable interface.

License

pruefbyte is released under the MIT License.

The Docker image and the release binaries also bundle the OpenCodeReview (ocr) binary, which is licensed under the Apache License 2.0. Both license texts are in the image under /usr/share/licenses/, and in each release archive as LICENSE and LICENSE.open-code-review.