CI Integration

Run Caliper automatically on every pull request. Install the CLI, run it against the PR number, and post findings as inline GitHub comments.

Quick start

npx caliper init automatically generates .github/workflows/caliper.yml when a GitHub remote is detected. If you need to add it manually, copy from the bundled template:

cp node_modules/@caliperai/caliper/templates/caliper-workflow.yml .github/workflows/caliper.yml

Or add the workflow manually:

# .github/workflows/caliper.yml
name: Caliper Review
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  pull-requests: write
  contents: read

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      - run: npm install -g @caliperai/caliper
      - run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

This runs Caliper's full AI review on every PR and posts findings as inline comments. The permissions block grants the minimum access needed: read the PR diff, write review comments.

Checks-only mode

Run only the deterministic convention checks compiled from your CLAUDE.md files. These checks are grep patterns, AST analysis, and file scans that execute in sub-second with no API calls. Use this mode to enforce mechanical rules on every PR without using the Anthropic API.

- run: caliper check

Prerequisites: A .caliper/ directory with checks.js already generated by npx caliper init (which does require an API key as a one-time setup step).

Full review mode

Runs the complete multi-phase AI review pipeline: triage, context, review, synthesis, lenses, consolidation, and narrative. Findings are posted as inline PR comments. The --ci flag auto-approves all findings and skips interactive prompts.

- run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Setting up secrets

The Anthropic API key must be stored as a GitHub Actions secret so it is not exposed in your workflow file.

1. Get an API key from console.anthropic.com.

2. Add the secret to your repository:

• Go to your repository's Settings > Secrets and variables > Actions
• Click New repository secret
• Name: ANTHROPIC_API_KEY
• Value: your sk-ant-... key
• Click Add secret

Package access

@caliperai/caliper is a public npm package — no npm token needed for installation.

Using with required status checks

When --fail-on-blocking is passed, Caliper exits with code 1 if any blocking findings are found. This lets you use the Caliper workflow as a required status check that prevents merging PRs with blocking issues.

To set it up as a required check:

1. Run the workflow at least once on a PR so GitHub registers the check name.
2. Go to Settings > Branches > edit the branch protection rule for your default branch.
3. Enable Require status checks to pass before merging.
4. Search for and select the Caliper job name (e.g., review).

Now PRs with blocking findings cannot be merged until the issues are resolved.

WARNING

If you use checks-only mode, the workflow only catches violations of deterministic convention checks. It will not block on AI-detected issues since no AI review runs.

Controlling API usage in CI

--max-cost flag

Set a ceiling in USD. If the estimated usage exceeds this amount, the review is skipped.

--min-severity flag

Controls which findings are posted. Setting it to recommendation filters out nits, reducing noise. Setting it to blocking posts only the most critical findings.

Available severity levels (from highest to lowest): blocking, recommendation, nit.

Combining flags

- run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking --max-cost 1.50 --min-severity recommendation
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Example workflows

Basic full review

name: Caliper Review
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  pull-requests: write
  contents: read

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      - run: npm install -g @caliperai/caliper
      - run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Checks-only (no API key)

name: Caliper Checks
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      - run: npm install -g @caliperai/caliper
      - run: caliper check

Monorepo with path filters

Use paths filters to run Caliper only when relevant directories change:

name: Caliper Review
on:
  pull_request:
    types: [opened, synchronize]
    paths:
      - "src/**"
      - "lib/**"
      - "packages/**"

permissions:
  pull-requests: write
  contents: read

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      - run: npm install -g @caliperai/caliper
      - run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

This prevents Caliper from running when only documentation, CI config, or other non-code files change.

Production hardening

Recommended permissions

Always declare explicit permissions rather than relying on defaults. Caliper needs:

permissions:
  pull-requests: write # Post review comments
  contents: read # Read PR diff and repo files

If your repository has restricted default permissions (Settings > Actions > General > Workflow permissions), the workflow will fail to post comments without these explicit grants.

Fork PRs

GitHub Actions triggered by pull_request from forks run with read-only GITHUB_TOKEN and cannot access repository secrets. This means:

• Fork PRs will not receive AI review (no ANTHROPIC_API_KEY available)
• Deterministic convention checks still run (no API key needed)

If you need AI review on fork PRs, use the pull_request_target trigger, but be aware of the security implications — only do this if you review fork PRs before running workflows.

--max-cost ceilings

For teams with many PRs, set a per-review ceiling to prevent unexpected API usage:

- run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking --max-cost 2.00
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The --max-cost flag skips the review entirely if the estimated usage exceeds the threshold.

Exit codes

Code	Meaning
0	Review completed successfully with no blocking findings (or checks passed)
1	Blocking findings were found (when --fail-on-blocking is used) or an error occurred
2	Convention check violations found (deterministic checks-only mode)

Loop guard: caliper check (stop hook mode) also exits 0 when stop_hook_active=true, even if violations are present. This prevents Claude from entering an infinite fix-and-recheck cycle — the violations are delivered as a system message instead, and Claude handles the fix on the next turn.

Findings are always posted as a COMMENT review — Caliper never uses REQUEST_CHANGES, so it cannot lock your PR. The --fail-on-blocking exit code is how CI enforcement works: it integrates with GitHub's required status checks to prevent merging when blocking findings exist.

Troubleshooting

Workflow fails with "ANTHROPIC_API_KEY is not set"

The secret is not reaching the step. Verify that:

• The secret is named exactly ANTHROPIC_API_KEY in your repository settings.
• The workflow passes it as an environment variable.
• If using environments, the secret is available to the environment the job runs in.

npm install errors

@caliperai/caliper is a public npm package. Check that your Node.js version is 20+ and that npm can reach the registry.

No PR comments are posted

• Confirm the workflow triggers on pull_request events. Push events do not have PR context.
• Check that GH_TOKEN has write permissions. The default GITHUB_TOKEN usually suffices, but repositories with restricted permissions may need explicit permissions: in the workflow.

Review runs but finds nothing

• Verify .caliper/ is checked into your repository with config.yaml and policy.md.
• Run npx caliper init locally to generate or update checks.js and conventions.yaml.
• Check that excludeFiles patterns in config.yaml are not filtering out the changed files.

Workflow is slow

• Use paths filters to skip runs on non-code changes (see monorepo example).
• Add --max-cost to limit per-review API usage.
• Use --min-severity recommendation or --min-severity blocking to reduce the volume of posted findings.
• Small PRs (< 100 changed lines, 3 or fewer files, no high-risk files) automatically use a fast path that skips synthesis, lenses, consolidation, and narrative.

Stale comments from previous runs

In CI mode (--ci), Caliper automatically deletes stale comments from previous runs before posting new findings. No manual cleanup is needed.

Debugging failures

Set CALIPER_DEBUG=1 as an environment variable in your workflow to get full stack traces:

- run: caliper ${{ github.event.pull_request.number }} --ci --fail-on-blocking
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    CALIPER_DEBUG: "1"