PR Test Agent
- Type
- Side project
- Stack
- Python, Claude Agent SDK, GitHub Actions
- Source
- GitHub (opens in a new tab)
01What it is
An AI agent that writes and verifies unit tests for pull requests.
A pull request changes code. This bot decides whether unit tests are needed, writes or updates them with an agent that uses reusable skills, runs them in a sandbox and shows the result on the pull request.
The agent’s output is trusted no further than it can be checked. Tests arrive as a separate stacked pull request into the original branch, and only after they pass repeated shuffled runs and cover the changed lines.
It runs in GitHub Actions rather than as an app, so there is no live preview, only the repository.
02Screenshots
How it works
03Decisions
How it is built, and why.
- 3.1
The model gets no shell
It can read files, write test files only and call one tool that runs tests in a sandbox. There is no shell, no web access and no git, and hooks check every file call against a policy before it runs.
- 3.2
Jobs are split by what they may do
One job reads, one generates, one verifies and one publishes. The model credential exists only in the step that runs the agent, and the write token only in the job that publishes.
- 3.3
Proposals are verified, not trusted
A generated test is accepted only if it builds, passes three shuffled runs and covers the changed lines. A guard rejects patterns that weaken the suite, such as skipped tests.
- 3.4
The agent is measured
Evaluation scenarios with deterministic expectations run several times each, and every bug found by a real run became a failing test first.