Automate GitHub PR Comments with Hermes Webhooks

A GitHub pull-request webhook can start a Hermes review when a PR opens, synchronizes, or reopens. The webhook payload supplies metadata—not the code diff—so a sound workflow validates the event, then retrieves the authoritative diff with the authenticated gh CLI before reviewing.

Build the route safely

Enable the webhook platform, create a named route with a strong per-route secret, accept only pull_request events, and filter actions before the agent wakes. Set delivery to github_comment with repository and PR-number fields resolved from the signed payload. The review prompt should treat titles, descriptions, branch names, and commit messages as untrusted data.

The gateway must be reachable by GitHub and gh auth status must succeed on the gateway host. Test locally with deliver: log before allowing real comments. Health-check the webhook service separately from the agent run.

Review workflow

1. Validate the HMAC signature and event type.
2. Drop irrelevant actions with route filters.
3. Run gh pr diff <number> --repo <owner/repo>.
4. Review the fetched diff for correctness, security, and maintainability.
5. Post one concise, actionable comment.
6. Use GitHub's delivery identifier for retry deduplication.

Release checklist

• Keep the webhook secret out of source control and screenshots.
• Filter to opened, synchronize, and reopened before agent execution.
• Run the public gateway in a sandboxed environment.
• Confirm the prompt explicitly fetches the diff.
• Start with log-only delivery on a disposable repository.
• Set rate and payload-size limits.
• Review GitHub Recent Deliveries after the test.

Pitfalls

Never set INSECURE_NO_AUTH in production. A PR description is attacker-controlled and can contain prompt injection. Without gh pr diff, the agent may review only prose. GitHub retries can create duplicate comments if delivery IDs are missing. A valid webhook response proves acceptance, not that the later agent run or comment succeeded.

Verification steps

1. Confirm /health returns the webhook service status.
2. Send a correctly signed test event and expect an accepted response.
3. Send a bad signature and require 401.
4. Open a small test PR and verify the comment references actual changed lines.
5. Redeliver the same event and confirm idempotency prevents duplication.

Follow the official GitHub PR webhook guide and webhook reference for current fields and security controls.