A second reviewer alongside Greptile: free for open source, and what it caught on #35

[nandal]

Hi OpenHack team 👋

I'm Sandeep from wrok.in. You may know me from #25–#27. We make a team of AI agents that review pull requests, free for public open-source repos. You already run Greptile, and it does good work here: on #35 it caught the unescaped braces that broke every validation, and the hunter-grades-itself loop. So this isn't a pitch to replace it. It's a case for adding a second reviewer next to it.

Why two reviewers

Every model has blind spots, and different models have different ones. A security scanner knows this better than anyone: it's why #35 adds cross-model validation, so the hunter doesn't grade its own findings. Code review works the same way. A second reviewer built on a different model reads the same diff from another angle, and the bugs one misses are often the ones the other catches.

What it caught on #35

I replayed #35 privately at 62a20a3, the commit Greptile scored 5/5. Nothing was posted on the PR. Two findings, both of the kind a scanner can't afford: findings dropped without anyone deciding they were safe.

1. An unfinished validation is recorded as a false positive · openhack/agents/validator_swarm.py:218

ValidatorAgent.run can finish without ever calling validate_finding: a plain-text reply (validator.py:159), hitting max iterations (:231), or a cancelled session. Each returns type: "validation_complete" with empty validated_findings and false_positives. In the swarm that vote is neither confirmed, rejected nor validation_failed, so it skips the new unreviewed branch and lands in else, which synthesizes a false_positive ("Failed strict majority validation (0/1…)"). With the default single vote, a finding nobody judged is discarded as safe.

Fix: count a run with no verdict as unreviewed. For example, in the elif, treat not (r["validated_findings"] or r["false_positives"]) the same as validation_failed.

2. "needs more info" counts as a vote that it's safe · openhack/agents/validator.py:114

_handle_validate_finding puts every status other than confirmed into false_positives, needs_more_info included. The swarm then counts that as a rejection, so uncertain votes can add up to a false-positive majority.

Fix: give needs_more_info its own bucket and route it to unreviewed.

I verified both by hand against 62a20a3. The security-audit agent also reviewed #29, #33 and #35 and found nothing to flag, which says something good about how carefully these changes are written.

What you'd get

• A team, not a single bot: a code reviewer and a security auditor on every PR, each of which can run a different model from Greptile's.
• One comment per PR, and only findings it can back up from the code. It drops its own claims that don't hold before posting.
• It only comments. It never approves, blocks or merges anything, so it sits quietly next to Greptile.

What it costs

Nothing. No card, no API key. Free reviews have fair-use limits: one review every 45 minutes per account, and diffs up to 200 KB. Past those, it says so on the PR instead of guessing.

Your code and data

• It reads only the PR's diff and nearby files, at review time, and we keep no copy.
• Free reviews run on an open-source model lane whose vendor may use requests to improve its models. That's acceptable only because the code is public, and private repos never use this lane. Data handling.
• If you'd rather use your own key and model, that works too, and those reviews stay off the free lane.

Trying it, or removing it

1. Install the GitHub App on this repository only.
2. Comment /w review on feat(agents): cross-model validation and strict majority consensus (P0) #35 to see the review above live.
3. Uninstalling from GitHub settings removes it completely.

---

_{Written with AI assistance, reviewed and sent by me. My open PRs here (#25–#27) are separate from this and shouldn't be weighed by it either way. If a second reviewer isn't a fit for OpenHack, please close this discussion. No reply is needed and I won't follow up. Thanks for building OpenHack in the open.}