Custom Convention Checks

Caliper enforces your team's conventions through deterministic checks that run in milliseconds with zero API calls. Most users never need to write checks by hand — caliper refresh compiles your CLAUDE.md into checks automatically. This page is a reference for teams that want full control over check definitions.

Every check is a plain JavaScript object that declares what to look for, where to look, and what to tell the developer when a violation is found. This guide covers every check type, how to write them by hand, and how to auto-compile them from your CLAUDE.md.

Check types overview

Each check has a type field that determines how it runs:

Type	What it does
grep	Searches file contents for a regex pattern
command	Runs a shell command; fails if exit code is non-zero
file-contains	Asserts that matching files contain a required pattern
file-exists	Asserts that companion files exist for matching files
max-lines	Fails if any matching file exceeds a line count
file-path	Fails if any changed file path matches a forbidden pattern
ast	Structural analysis of functions (lines, params, complexity, depth)
requires	If a file matches a when pattern, it must also contain a requires pattern
forbids	If a file matches a when pattern, it must not contain a forbids pattern
ai-check	Semi-deterministic: a regex pre-filter finds candidate code, then a focused AI question determines if it violates

Every check object shares these common fields:

{
  id: "unique-check-id",           // Unique identifier
  name: "Human-readable name",     // Shown in CLI output
  section: "Category",             // Groups checks in output (e.g., "Security", "Code Quality")
  type: "grep",                    // One of the types above
  fixHint: "How to fix this",      // Shown when the check fails
}

Writing grep checks

Grep checks scan changed files for a regex pattern. A match means a violation.

Basic grep check

{
  id: "no-console-log",
  name: "No console.log in production code",
  section: "Code Quality",
  type: "grep",
  pattern: "console\\.log\\(",
  fileGlob: "src/**/*.{ts,tsx}",
  fixHint: "Use a proper logger or remove debug logging before committing",
}

Pattern syntax

The pattern field is an extended regular expression (ERE). Caliper runs it through grep -E for single-line checks, or through JavaScript's RegExp for multiline and lookaround patterns.

Common patterns:

Goal	Pattern
Literal function call	eval\\s*\\(
Type annotation	:\\s*any\\b
Start of line	^const\\s+
Either/or	\\.log\\(|console\\.warn\\(
Negated lookahead	@ts-ignore(?!\\s+--)

Structural limitation: content vs. path

Grep sees only file content — it never knows the path of the file it is running inside. Rules where validity depends on the containing file's location cannot be expressed as grep. For example, "code in clients/foo/ must not import from clients/bar/" requires knowing both the import target and the file's location — no regex trick can encode that. caliper refresh routes these rules to ai-check automatically.

Scoping with fileGlob

The fileGlob field restricts which files are checked. It supports standard glob syntax:

• src/**/*.ts — all .ts files under src/
• **/*.{ts,tsx} — all TypeScript files anywhere
• scripts/*.mjs — only .mjs files directly in scripts/

When fileGlob is omitted, the check runs against all files in the configured srcDirs.

Multiline matching

Set multiline: true to match patterns that span multiple lines. This switches from grep -E to JavaScript's RegExp with the ms flags, which makes . match newlines and ^/$ match line boundaries.

{
  id: "no-empty-catch",
  name: "No empty catch blocks",
  section: "Error Handling",
  type: "grep",
  pattern: "catch\\s*\\([^)]*\\)\\s*\\{\\s*\\}",
  multiline: true,
  fixHint: "Handle errors explicitly or add a comment explaining why the catch is empty",
}

Multiline mode is also automatically enabled when the pattern contains lookaround assertions ((?=, (?!, (?<).

Filtering results

Two optional fields let you exclude false positives from grep results without complicating the pattern itself.

skipComments — set to true to automatically ignore JS/TS comment lines (//, /* */, and JSDoc /** … */ blocks). Useful when a pattern legitimately appears in comments but should only be enforced in code:

{
  id: "allowed-hosts-no-wildcards",
  name: "No wildcards in the approved host allowlist",
  section: "Egress Control",
  type: "grep",
  pattern: "[*?]",
  fileGlob: "lib/net/allowed-hosts.ts",
  skipComments: true,
  fixHint: "Add exact hostnames only — no wildcards or regex patterns",
}

excludePattern — a regex applied to each matched line's content. Lines whose content matches are excluded from results. Use this when comment stripping isn't enough and you need a custom filter:

{
  id: "no-console-log",
  name: "No console.log statements",
  section: "Code Quality",
  type: "grep",
  pattern: "console\\.log",
  excludePattern: "^\\s*//",
  fixHint: "Remove or replace with structured logging",
}

Both fields apply regardless of whether multiline is set.

More grep examples

Forbid non-null assertions on lookups:

{
  id: "no-non-null-assertion",
  name: "No non-null assertions on lookups",
  section: "Type Safety",
  type: "grep",
  pattern: "\\.(get\\([^)]+\\)|\\[[^\\]]+\\])\\s*!",
  fixHint: "Provide a fallback value instead of using ! on map/object lookups",
}

Forbid hardcoded secrets:

{
  id: "no-hardcoded-secrets",
  name: "No hardcoded API keys or secrets",
  section: "Security",
  type: "grep",
  pattern: "(api_key|apikey|secret|password|token)\\s*[:=]\\s*['\"][A-Za-z0-9+/=]{16,}['\"]",
  fixHint: "Use environment variables for secrets, never hardcode them",
}

Forbid execSync with template strings (shell injection risk):

{
  id: "no-execsync-template",
  name: "No execSync with template strings",
  section: "Security",
  type: "grep",
  pattern: "execSync\\s*\\(\\s*`",
  fixHint: "Use execFileSync with array args to prevent shell injection",
}

Writing AST checks

AST checks parse TypeScript and JavaScript files and analyze function-level structure. They support six rule kinds.

max-lines — Function length limit

{
  id: "fn-max-lines",
  name: "Functions under 30 lines",
  section: "Code Quality",
  type: "ast",
  rule: { kind: "max-lines", maxLines: 30 },
  fixHint: "Extract named helpers to keep functions focused and under 30 lines",
}

Reports every function whose body exceeds the threshold. Output includes the function name, file, line number, and actual line count.

max-params — Parameter count limit

{
  id: "fn-max-params",
  name: "Functions have at most 4 parameters",
  section: "Code Quality",
  type: "ast",
  rule: { kind: "max-params", maxParams: 4 },
  fixHint: "Use an options object instead of many positional parameters",
}

max-complexity — Cyclomatic complexity limit

{
  id: "max-complexity",
  name: "Cyclomatic complexity under 15",
  section: "Code Quality",
  type: "ast",
  rule: { kind: "max-complexity", maxComplexity: 15 },
  fixHint: "Reduce branching by extracting conditions into named functions or using early returns",
}

max-depth — Nesting depth limit

{
  id: "max-depth",
  name: "Maximum nesting depth of 3",
  section: "Code Quality",
  type: "ast",
  rule: { kind: "max-depth", maxDepth: 3 },
  fixHint: "Use early returns or extract nested logic into helper functions",
}

requires — Conditional requirement within functions

If a function body matches the when pattern, it must also contain the requires pattern. This enforces co-occurrence at the function level.

{
  id: "fetch-needs-error-handling",
  name: "fetch() calls must have error handling",
  section: "Error Handling",
  type: "ast",
  rule: {
    kind: "requires",
    when: "\\bfetch\\(",
    requires: "catch|try",
  },
  fixHint: "Wrap fetch() calls in try-catch or chain .catch()",
}

forbids — Conditional prohibition within functions

If a function body matches the when pattern, it must not contain the forbids pattern.

{
  id: "async-no-sync-fs",
  name: "Async functions must not use sync fs methods",
  section: "Performance",
  type: "ast",
  rule: {
    kind: "forbids",
    when: "async\\s+function|async\\s*\\(",
    forbids: "readFileSync|writeFileSync|existsSync",
  },
  fixHint: "Use async fs methods (readFile, writeFile) in async functions",
}

Structural limitation: uniform function scope

AST checks apply to all functions uniformly — they cannot filter by export status, async keyword, decorator, or function name. Rules scoped to "exported functions only" or "async functions only" cannot be expressed as ast. Use ai-check for attribute-scoped function rules.

Scoping AST checks

AST checks only run on TypeScript and JavaScript files (.ts, .tsx, .js, .jsx, .mjs, .cjs). You can further restrict scope with fileGlob:

{
  id: "src-fn-max-lines",
  name: "Source functions under 50 lines",
  section: "Code Quality",
  type: "ast",
  fileGlob: "src/**/*.{ts,tsx}",
  rule: { kind: "max-lines", maxLines: 50 },
  fixHint: "Extract named helpers to keep functions focused",
}

Writing command checks

Command checks run an arbitrary shell command. A non-zero exit code means failure. The command's stderr/stdout (last 20 lines) is shown as the violation detail.

{
  id: "typecheck",
  name: "TypeScript compiles without errors",
  section: "Build",
  type: "command",
  command: "npx tsc --noEmit",
  slow: true,
  fixHint: "Fix TypeScript compilation errors before committing",
}

The slow flag

Set slow: true for commands that take more than a few seconds (e.g., full type checks, test suites). Slow checks are skipped when Caliper runs in quick mode.

Environment variables

Command checks receive these environment variables:

Variable	Value
CALIPER_CHANGED_FILES	Newline-separated list of changed file paths
CALIPER_PR_NUMBER	PR number (empty string if running locally)
CALIPER_PR_TITLE	PR title (empty string if not available)

Use these to scope your command to only the relevant files:

{
  id: "eslint-changed",
  name: "ESLint passes on changed files",
  section: "Lint",
  type: "command",
  command: "echo \"$CALIPER_CHANGED_FILES\" | grep '\\.tsx\\?$' | xargs -r npx eslint",
  slow: true,
  fixHint: "Fix ESLint errors in the changed files",
}

Security note

Commands run via execFileSync("sh", ["-c", command]). They execute with the same permissions as the user running Caliper. Only repo owners define command checks (in .caliper/checks.js or via caliper refresh), so this is equivalent to CI scripts.

Writing max-lines checks

Max-lines checks fail if any changed file matching fileGlob exceeds a line count.

{
  id: "no-large-files",
  name: "Source files under 300 lines",
  section: "Code Quality",
  type: "max-lines",
  fileGlob: "src/**/*.ts",
  maxLines: 300,
  fixHint: "Split this file into smaller focused modules",
}

For per-function line limits, use ast instead. max-lines measures the entire file. To limit individual functions, use type: "ast" with rule: { kind: "max-lines", maxLines: N } — see the AST section above.

Writing file-exists checks

File-exists checks verify that each changed file matching fileGlob has a companion file. The companion path is computed by replacing the file extension with companionSuffix.

{
  id: "test-file-required",
  name: "Every source file must have a test",
  section: "Testing",
  type: "file-exists",
  fileGlob: "src/**/*.ts",
  companionSuffix: ".test.ts",
  fixHint: "Create a corresponding .test.ts file in tests/ for this module",
}

How it works: For a changed file src/utils/parser.ts, the check strips the extension (.ts) and appends the suffix, looking for src/utils/parser.test.ts. If that file does not exist on disk, the check fails.

Structural limitation: companionSuffix only replaces the file extension — it cannot change the directory. For src/foo.ts with suffix .test.ts, the check looks for src/foo.test.ts, not tests/foo.test.ts. For companion files in a different directory tree, use a command check or ai-check.

More examples:

// Every migration needs a rollback
{
  id: "migration-rollback",
  name: "Migrations must have rollback files",
  section: "Database",
  type: "file-exists",
  fileGlob: "db/migrations/*.up.sql",
  companionSuffix: ".down.sql",
  fixHint: "Create a .down.sql rollback file for this migration",
}

Writing file-contains checks

File-contains checks assert that every changed file matching fileGlob contains a required regex pattern. Files that match the glob but do not contain the pattern are violations.

{
  id: "scripts-import-dotenv",
  name: "Scripts must import dotenv",
  section: "Configuration",
  type: "file-contains",
  fileGlob: "scripts/**/*.ts",
  pattern: "import.*dotenv|require.*dotenv",
  fixHint: "Add 'import \"dotenv/config\"' at the top of this script",
}

Structural limitation: file-contains verifies that the pattern exists somewhere in the file — not at a specific position, in a specific order, or a specific number of times. Rules like "import must be on the first line" or "every exported function must have a JSDoc comment" cannot be expressed as file-contains. Use ai-check for position- or count-sensitive requirements.

When to use file-contains vs requires: Use file-contains when the requirement is unconditional (every matching file must have it). Use requires (below) when the requirement is conditional on another pattern being present.

More examples:

// React components must have display names
{
  id: "react-display-name",
  name: "React components must set displayName",
  section: "React",
  type: "file-contains",
  fileGlob: "src/components/**/*.tsx",
  pattern: "displayName\\s*=",
  fixHint: "Add ComponentName.displayName = 'ComponentName' to this component",
}

File path checks

File-path checks validate the paths of changed files against a regex pattern. Any changed file whose path matches the pattern is a violation.

{
  id: "no-uppercase-dirs",
  name: "No uppercase letters in directory names",
  section: "Naming",
  type: "file-path",
  pattern: "/[A-Z][^/]*/",
  fixHint: "Use lowercase directory names (kebab-case preferred)",
}

{
  id: "no-spaces-in-filenames",
  name: "No spaces in file names",
  section: "Naming",
  type: "file-path",
  pattern: " ",
  fixHint: "Replace spaces with hyphens in file names",
}

{
  id: "kebab-case-files",
  name: "Source files must use kebab-case",
  section: "Naming",
  type: "file-path",
  pattern: "src/.*[A-Z].*\\.[jt]sx?$",
  fixHint: "Rename this file to use kebab-case (e.g., my-component.tsx)",
}

Writing requires and forbids checks

The top-level requires and forbids check types work at the file level (not function level, unlike AST requires/forbids). They support multiline mode for cross-line patterns.

requires — File-level conditional requirement

If a file contains the when pattern, it must also contain the requires pattern.

{
  id: "express-error-middleware",
  name: "Express apps must register error middleware",
  section: "Error Handling",
  type: "requires",
  fileGlob: "src/**/*.ts",
  when: "express\\(\\)",
  requires: "app\\.use\\(.*err",
  fixHint: "Register an error-handling middleware in this Express app",
}

Structural limitation: requires checks that the requires pattern exists somewhere in the file — not necessarily near or paired with the when match. Per-call-site rules like "every execSync call must pass a timeout option" cannot be expressed this way (a file with one correct call and one missing timeout would pass). Use ast with rule: { kind: "requires" } for function-scoped rules, or ai-check for per-call-site rules.

forbids — File-level conditional prohibition

If a file contains the when pattern, it must not contain the forbids pattern.

{
  id: "no-sync-in-handlers",
  name: "Route handlers must not use sync I/O",
  section: "Performance",
  type: "forbids",
  fileGlob: "src/routes/**/*.ts",
  when: "router\\.(get|post|put|delete)\\(",
  forbids: "readFileSync|writeFileSync",
  fixHint: "Use async I/O methods in route handlers to avoid blocking the event loop",
}

Structural limitation: forbids checks that the forbids pattern is absent from the entire file — not just absent near the when match. Rules like "never use string interpolation inside .query() calls" cannot be expressed this way (an unrelated template literal elsewhere in the file would trigger a false positive). Use ast with rule: { kind: "forbids" } for function-scoped rules, or ai-check for per-call-site rules.

Multiline mode

Set multiline: true when the when or requires/forbids patterns need to span multiple lines:

{
  id: "class-has-constructor",
  name: "Classes must define a constructor",
  section: "Architecture",
  type: "requires",
  fileGlob: "src/**/*.ts",
  when: "^class\\s+\\w+",
  requires: "constructor\\s*\\(",
  multiline: true,
  fixHint: "Add an explicit constructor to this class",
}

Writing ai-check checks

ai-check is semi-deterministic: a regex pre-filter (contentPattern) finds candidate code sections, then a focused AI question determines whether each match is actually a violation. Files with no pattern matches are skipped at zero cost — only the matched snippets are sent to the AI.

Use ai-check only when no deterministic type can express the rule but the candidates can be narrowed to specific code locations with a regex. Every contentPattern match costs an AI call, so keep the pattern specific.

{
  id: "meaningful-error-handling",
  name: "Error handling must be meaningful",
  section: "Error Handling",
  type: "ai-check",
  description: "Catch blocks must log, re-throw, or meaningfully handle errors",
  contentPattern: "catch\\s*\\(",
  question: "Does this catch block silently swallow the error without logging, re-throwing, or handling it meaningfully?",
  fixHint: "Log the error, re-throw it, or handle it with a user-facing message",
}

Fields

Field	Required	Description
description	✓	Human-readable description of what the check enforces
contentPattern	✓	Regex to find candidate code (files without matches are skipped at no cost)
question	✓	Focused yes/no question for the AI, answerable from a short snippet
fileGlob		Scope to files matching a glob pattern
severity		Override severity: blocking, recommendation, or nit

When to use ai-check vs conventions

ai-check runs automatically on every npx caliper check and costs an AI call per match. Conventions are only evaluated during AI review phases (npx caliper review, npx caliper <pr>).

Use ai-check when:

• The rule can be narrowed to specific code locations with a regex
• The check needs to run on every agent turn (stop hook)

Use a convention when:

• There's no reliable regex pre-filter
• The rule is about overall design or architecture, not a code pattern

The .caliper/checks.js file

All custom checks live in .caliper/checks.js at your repository root. This file is a JavaScript module that exports an array of check objects.

File structure

// .caliper/checks.js

/** @type {import('@caliperai/caliper').CheckConfig[]} */
export const checks = [
  {
    id: "no-console-log",
    name: "No console.log in production code",
    section: "Code Quality",
    type: "grep",
    pattern: "console\\.log\\(",
    fileGlob: "src/**/*.{ts,tsx}",
    fixHint: "Use a proper logger or remove debug logging",
  },
  {
    id: "fn-max-lines",
    name: "Functions under 40 lines",
    section: "Code Quality",
    type: "ast",
    rule: { kind: "max-lines", maxLines: 40 },
    fixHint: "Extract named helpers to keep functions focused",
  },
  // ... more checks
];

The file is loaded via dynamic import(), so it must use ESM syntax (export) rather than CommonJS (module.exports). The runner looks for a named export called checks, falling back to the default export.

When .caliper/checks.js is absent

If no checks.js file exists, Caliper falls back to a built-in convention pack based on your detected framework (e.g., the TypeScript pack includes function length limits, any bans, and security patterns). Run npx caliper refresh to generate a customized checks.js from your CLAUDE.md.

Using npx caliper refresh to auto-generate checks

The npx caliper refresh command reads your CLAUDE.md (and other AI instruction files like .cursor/rules/*.md, .github/copilot-instructions.md) and uses AI to compile mechanically-enforceable rules into check objects. It writes the results to .caliper/checks.js.

# Auto-accept all generated checks
npx caliper refresh

# Review each proposed check interactively
npx caliper refresh --interactive

# Force re-extraction (bypass cache)
npx caliper refresh --force

# Write a trace file for debugging
npx caliper refresh --trace

Interactive mode

With --interactive, Caliper presents each generated check for your approval before writing it. You see the check name, the original CLAUDE.md text it was derived from, and the generated command or pattern. Press Enter to keep a check (default) or n to skip it.

What compiles well

Rules with specific, measurable criteria compile best. See the CLAUDE.md Style Guide for detailed guidance on writing rules that Caliper can compile. Key principles:

• Be quantitative: "under 30 lines" compiles, "keep functions short" does not
• Name exact patterns: "eval()" compiles, "unsafe patterns" does not
• Specify file scope: "in src/ files" narrows the check and reduces false positives
• Use negative framing for bans: "Never use", "No", "Do not"

How caliper refresh routes rules

When compiling rules from your CLAUDE.md, Caliper follows a decision ladder:

1. Deterministic check — if the rule can be fully and correctly expressed as grep, ast, file-exists, etc.
2. ai-check — if there's any doubt, or if the rule requires judgment but has a locatable code anchor. An ai-check is always better than skipping a rule or encoding it incorrectly in a deterministic check.
3. Convention — only if the rule cannot be narrowed to any specific code location at all (no useful regex pre-filter exists).

If caliper refresh generates an ai-check for a rule you expected to become a grep or ast check, it means the rule hit one of the structural limitations described in this guide.

What becomes conventions

Rules requiring judgment with no code anchor (e.g., "prefer composition over inheritance") cannot be compiled into any check. These become conventions that are surfaced during AI review phases instead.

Testing checks locally

Run all convention checks

# Run checks against staged changes
npx caliper check

# Run checks against all source files
npx caliper check --all-files

Verify a specific check

To test a single check, add it to .caliper/checks.js, make a change that should trigger it, stage the change, and run npx caliper check. Look for your check's name in the output.

Iterate on grep patterns

Before adding a grep check, test the pattern directly:

# Test your regex against the codebase
grep -rnE "console\.log\(" src/

# Verify it matches what you expect and nothing else
grep -rnE "your-pattern-here" --include="*.ts" src/

Iterate on AST checks

AST checks analyze every function in changed files. To verify thresholds:

1. Add the check to .caliper/checks.js
2. Touch a file that should trigger it: touch src/some-file.ts && git add src/some-file.ts
3. Run npx caliper check and review the output

Validate the checks.js file loads

# Quick syntax check — node will error if the file has issues
node -e "import('.caliper/checks.js').then(m => console.log(m.checks.length + ' checks loaded'))"

Common pitfalls

• Regex escaping: In JavaScript strings, backslashes must be doubled. \b in a regex becomes "\\b" in the pattern string. If your pattern works in grep but not in Caliper, check the escaping.
• fileGlob syntax: Use **/*.ts (double-star) to match recursively. A single * only matches one directory level.
• Overly broad patterns: A grep check for "any" will match comments, variable names, and string literals. Use word boundaries (\\bany\\b) or context (:\\s*any\\b) to narrow matches.
• Missing slow: true: If your command check takes more than a second or two, mark it as slow so --quick mode skips it. Otherwise it slows down pre-commit hooks.