Independent LLM PR review¶

The standing assignment for this repo: Claude plans → DeepSeek codes → Claude reviews. The review runs by itself when the test suite passes. Everything else is a comment on an issue or a pull request — no tools to install, same from a laptop or the GitHub mobile app.

The commands¶

Two you'll use constantly, two you'll rarely need.

You don't normally type either review command. Claude reviews every PR automatically once pytest succeeds. /review forces a re-run; /review-deepseek covers the one case that can't be automatic.

Rules that apply to all four:

• It must be the first thing in the comment. "Can you /oc fix this?" does nothing. That's deliberate: it means quoting an earlier comment doesn't set an agent off again.
• Everything after the phrase is the instruction, in plain English.
• Only someone with write access to the repo can trigger them.

Examples worth copying¶

On an issue, once the plan is agreed:

/oc implement the plan in this issue

/oc implement this, but only the model and migration — leave the
template work for a follow-up

On a PR, after a review:

/oc the review flagged an unscoped query at apps/documents/views.py:142 —
fix it and add the cross-household test

/oc this is failing pytest on test_topic_filter — work out why and fix it

/oc merge main into this branch and resolve the conflict in services.py

You can also comment on a specific line in the PR's Files tab, and opencode picks up the file and line automatically:

/oc this needs the require_owner() guard

Forcing a review (rarely needed — Claude's runs automatically):

/review

/review-deepseek

Neither takes an instruction; the brief is already written (.github/prompts/review.md).

From a terminal¶

Identical, just posted a different way:

gh issue comment 239 --repo ollyoxford/maud --body "/oc implement the plan in this issue"
gh pr comment 240 --repo ollyoxford/maud --body "/review"

From the GitHub mobile app¶

1. Open the repo → Issues (or Pull requests)
2. Tap the issue or PR
3. Scroll to the bottom, past any existing comments
4. Type into the comment box and send

It's the ordinary comment box — the same one you'd use to write a note to yourself. There is no special agent UI, no button, no menu. If a suggestion menu pops up when you type /, that's GitHub's own slash commands: ignore it and keep typing.

A worked example, end to end¶

1. Plan. Open a Claude session, agree what you're building, and have it write the issue: the plan plus acceptance criteria. Say it's #251.
2. Code. Comment on #251: /oc implement the plan in this issue
3. Wait a few minutes. A PR appears — say #252 — with the code.
4. Wait for CI. pytest and docs-drift run on it.
5. The review runs itself the moment pytest goes green — nothing to type.
6. Claude posts a findings table and adds review:pass or review:changes-requested.
7. Fix, if needed. Comment on #252: /oc fix the Blocker in the review — the missing require_owner() on the delete view
8. The new commit clears the verdict label, tests re-run, and the review re-runs behind them. /review forces one by hand if you need it.
9. Merge, once CI is green and the label says review:pass. That step is yours and stays yours.

Why it exists¶

Claude plans most PRs here. A model reviewing its own diff shares the blind spot that produced the bug, so the reviewer is deliberately a different model family with no shared context.

pytest and docs-drift already catch regressions and drift. What they can't catch is judgement: a misread plan, a query that skipped the scoped manager, a memory rule quietly broken. That's the gap this fills — and every merge to main auto-deploys within ~2 minutes, so there's no staging between a merged PR and real users' documents.

How it actually works¶

Three ideas, if you haven't worked with GitHub Actions before.

A workflow is a recipe file in .github/workflows/. GitHub reads it, and when something happens in the repo it rents a fresh computer for a few minutes and runs the recipe there. Nothing runs on your machine.

An event is the "something happens" — a comment posted, a commit pushed, a label added. Each recipe declares which events it cares about. Ours listen for comments — and, for the review, for another workflow finishing.

A token is a password for robots. A recipe usually needs to act back on GitHub — post a comment, add a label, push code — so GitHub mints it a temporary one, GITHUB_TOKEN, created fresh per run and destroyed after.

So /oc isn't a command in any special sense. It's an ordinary comment that a recipe happens to be watching for. That's why it works from a phone with nothing installed: you aren't running anything, you're leaving a note, and something on GitHub's servers is watching for notes shaped like that. It's also why the comment stays in the thread forever — a permanent record of exactly what you asked for.

When the review runs¶

Automatically, when the Tests workflow succeeds on a PR. Nothing to type.

Reviewing is the one stage in the loop with no decision in it — you always want the review — so a manual trigger there was ceremony. Worse than ceremony: a step you always say yes to is one you eventually forget, and the failure mode is merging unreviewed while believing you have a review process. A gate that depends on remembering isn't a gate.

Hanging it off Tests-succeeded also gets the ordering right for free. The reviewer only ever sees code the suite has already passed, so its attention goes on judgement rather than on what pytest already proves.

The stages that do hold a decision stay yours: whether the plan is right before code gets written, which findings to accept, and whether to merge. Three touch points instead of four, and the one that went is the one you'd never have said no at.

• Draft PRs are skipped — that's how you say "not yet, still pushing", without needing a command to withhold.
• Every green test run re-reviews. The verdict label is cleared on each push anyway, so anything else would leave a stale pass standing. The concurrency group cancels a review a newer push has superseded.

The two modes¶

The rule underneath everything: the model that wrote it never reviews it.

Claude both plans and reviews by default. That's fine — reviewing someone else's implementation of your own spec is not self-review, and the spec's author is well placed to spot a misreading. The gap it leaves is an ambiguous plan: Claude won't flag its own under-specification, so if the code is odd in a way that traces back to the brief, that's a plan bug.

The reviewer is chosen from the branch name rather than from your memory of who wrote what. A claude/* branch never auto-reviews, and /review refuses on it by hand too, pointing you at /review-deepseek. Those PRs sit at llm-review pending with "comment /review-deepseek" written into the status, so the one case that still needs you can't go unnoticed. The guard is only as honest as the branch name, so any branch Claude commits to is named claude/<slug> — Claude Code on the web does that automatically; from the CLI you name it yourself, which is where it's easy to get wrong.

Both reviewers read the same brief, deliberately. If they read different briefs you couldn't tell a model difference from a prompt difference.

Repeat reviews¶

A PR gets reviewed again on every green test run, so a long-lived PR accumulates reviews. The brief tells each reviewer to read the ones before it and to report only what is new or genuinely still open — a finding an independent reviewer has explicitly discharged is closed, and re-raising it is noise rather than a second opinion.

This was learned the hard way on #247, which collected four reviews carrying about one and a half reviews' worth of distinct content. Each pass re-raised findings the previous one had resolved, which is the same "manufactured findings train the reader to skim" failure the brief warns about, reached from the other direction.

The brief also tells a reviewer that finds its own model family in the commit trailers to say so and stop rather than post a findings table anyway. A thorough self-review is still a self-review, and a page of authoritative-looking analysis buries the one line that matters: this needs the other reviewer.

Trigger phrases must not overlap¶

/review-deepseek also starts with /review, so the Claude workflow carries an explicit negation. Without it, one comment fires both reviewers: double cost, and they race each other for the verdict label. Remember that before adding a third phrase.

How it's wired¶

The reviewer workflows concatenate AGENTS.md and the brief into one prompt. The rulebook is inlined rather than auto-discovered — discovery is convention, not contract, and a reviewer that silently doesn't know the house rules still produces confident-looking output. That's also why the brief doesn't restate the invariants: CLAUDE.md stays the single place they live.

Design decisions worth not undoing¶

• Reviewers are read-only. contents: read, so they comment and label but physically cannot push. If a reviewer disagrees with you, it has to argue.
• Every workflow is gated on author_association. opencode has no built-in permission check.
• Reviewer checkouts land on main, not the PR head. An issue_comment event carries no PR ref, and this is what we want: a PR can't edit the criteria it's judged by.
• startsWith, not contains. Quoting a previous review comment would otherwise re-trigger the job.

The coding agent¶

/oc <instruction> on an issue opens a PR; on a PR it commits to that branch.

It authenticates through the opencode GitHub App rather than GITHUB_TOKEN, and that is not incidental — see the first gotcha below.

The merge gate¶

review-gate.yml publishes an llm-review commit status: success on review:pass, failure on review:changes-requested, pending otherwise. On a new push it clears both verdict labels and resets to pending — that half is the point, because otherwise a stale pass keeps the status green over commits the reviewer never saw.

It is advisory until llm-review is added to branch protection as a required check. Do that only once the reviewer has earned it. A gate you don't trust gets bypassed, and a bypassed gate is worse than none because it looks like coverage.

Rate each review afterwards¶

On merge, add one of review:caught-real-bug or review:noise. It takes a second and it's the only way to find out whether the review is earning its place — a review nobody trusts is worse than no review, because it launders the diff.

Gotchas that will cost you an afternoon¶

Two of these (1, 2) are the same underlying fact; three more (3, 4, 6) are all "don't let strangers spend your money or read your keys"; the rest is housekeeping.

1. Anything done with GITHUB_TOKEN doesn't count as an event.

GitHub's rule, to stop infinite loops — otherwise a workflow that pushes a commit would start a workflow that pushes a commit, forever.

The trap: if the coding agent pushed using that token, GitHub would treat the push as a non-event, so pytest and docs-drift would never run on it. And the PR wouldn't look broken — it would look clean, because no checks are pending at all. Absence of a red X reads as "passed". Hence the opencode GitHub App: a separate identity, so its pushes are real events and CI fires. Don't "simplify" that back to use_github_token: true.

2. That same rule is why triggers are comments, not labels.

Tapping a label is a nicer phone interaction, so the original design was label → a workflow posts /review → the review workflow sees it. But the middle step would post using GITHUB_TOKEN, so the reviewer would never wake up. Comments trigger things; labels record state.

3. author_association is the thing keeping strangers out.

With every comment GitHub reports the commenter's relationship to the repo — OWNER, MEMBER, COLLABORATOR, CONTRIBUTOR or NONE. Our workflows require one of the first three. Without that check, anyone who can comment could run an agent with write access, on our money.

4. Never use pull_request_target.

There are two ways to trigger on a PR. The safe one withholds secrets when the PR comes from a stranger's fork. pull_request_target hands over full secrets — and people reach for it precisely because they want the secrets, then check out the stranger's code and run it with access to their keys. It's the classic way repos get their secrets stolen. The comment trigger gives us secrets safely instead.

5. share: false matters.

opencode can publish a transcript of the agent's session to a public URL, and it defaults to on for public repos. That transcript contains chunks of the codebase.

6. Fork PRs get no secrets.

If someone forks the repo and opens a PR, GitHub deliberately hides our secrets from workflows running on it — so none of this fires on outside contributions. Fine while this is a solo repo; a real design question the day it isn't.

7. Model IDs move.

Providers retire names. The deepseek-chat and deepseek-reasoner aliases retired 2026-07-24. A workflow failing with "unknown model" is almost always this; check models.dev for the current ID.

8. claude-code-action needs id-token: write even though it never pushes.

It authenticates as the Claude GitHub App by exchanging a GitHub OIDC token, and minting that token needs the scope. Read-only-on-code and OIDC are separate axes, so "this job only comments, it needs no write permissions" is the wrong inference — and it fails at the very last step, after checkout and prompt-building have all succeeded, with Could not fetch an OIDC token repeated three times.

9. The spend risk is loops, not tokens.

A review is roughly 4p. What actually costs money is two agents triggering each other, or one hanging for hours. timeout-minutes kills a job after 20–30 minutes, and the per-PR concurrency group means a second run cancels the first rather than stacking.

Configuration¶

• DEEPSEEK_API_KEY and CLAUDE_CODE_OAUTH_TOKEN — repository secrets (Settings → Secrets and variables → Actions). Not the local .env: a workflow runs in a fresh container that never sees it.
• The opencode GitHub App must be installed on the repository.
• Both reviewer models are pinned. Leave them pinned — if the model can drift, a change in findings can't be attributed to the model or to the brief.

If nothing happens¶

In rough order of likelihood:

1. The phrase wasn't the first thing in the comment.
2. The workflow isn't merged to main yet. Comment-triggered workflows always run the version on the default branch — so a new one added in a PR does nothing until that PR merges, and testing it on its own PR is impossible. (pull_request-triggered workflows are the opposite: they run the PR branch's version, so review-gate.yml does take effect immediately. Easy to trip over, because two workflows in the same folder behave differently.)
3. The opencode GitHub App isn't installed.
4. A secret is missing or the DeepSeek balance is zero (a key with no credit looks valid and fails every call).
5. The model ID retired — see gotcha 7.

The Actions tab shows every run and its logs, and is the first place to look.

Background¶

The full design and its reasoning: docs/multi-llm-pr-review-plan.md (maud-specific) and docs/two-model-review-playbook.md (repo-agnostic).