DocsGetting Started

Getting Started

Get CodeSheriff scanning your PRs in under 2 minutes. No CI changes, no infrastructure to manage.

Overview

CodeSheriff is a GitHub App that automatically scans pull requests for security issues introduced by AI coding assistants. It combines three detection layers:

SemgrepFree

Pattern-based static analysis using OWASP and CWE rule sets. Fast and zero false-positive tuned.

TruffleHogFree

Secrets detection across 700+ credential types — API keys, tokens, passwords, and more.

Claude AI DetectorsTeam+

LLM-based analysis that catches hallucinated APIs, semantic auth bugs, and logic errors traditional scanners miss.

Install the GitHub App

CodeSheriff runs as a GitHub App and only needs permission to read PR diffs, post comments, and create check runs.

Visit the GitHub App page

Go to app.codesheriff.dev/sign-up and click “Install GitHub App”.

Choose your organization or account

Select the GitHub organization or personal account where you want to install CodeSheriff.

Select repositories

Choose "All repositories" for full coverage, or select specific repositories to start. You can change this anytime.

Authorize and install

Review the permissions (read-only on code, write on check runs and PR comments) and click Install.

That's it! Once installed, CodeSheriff will automatically scan every new PR opened against your selected repositories. No further configuration is required.

Your first scan

Open any pull request on a configured repository and CodeSheriff will run automatically. Here's what to expect:

1. A GitHub Check Run appears within seconds:

github checks

CodeSheriff Security Scan
Status: In progress → Complete (23 seconds)

✓ Semgrep (OWASP): 0 findings
✓ TruffleHog secrets: 1 finding — HIGH
✗ Claude AI: 2 findings — 1 CRITICAL, 1 MEDIUM

Risk score: 68/100

2. Inline comments appear on affected lines:

pr comment

🚨 CRITICAL: Hallucinated API — stripe.charges.createInstant()

This method does not exist in stripe-node v12.x (or any version).
AI models frequently hallucinate Stripe SDK methods that sound plausible
but have never existed.

Suggested fix: Use stripe.paymentIntents.create() instead.

Severity: CRITICAL | Detector: Claude AI | Rule: hallucinated-sdk-api

3. The PR check blocks or warns based on your thresholds:

github status

● CodeSheriff — 3 findings (1 critical)

  This PR introduced critical security findings.
  Review the inline comments before merging.

  [View full report →]

codesheriff.yml

Drop a .codesheriff.yml in your repo root to customize CodeSheriff's behavior. All fields are optional.

yaml

# .codesheriff.yml
version: 1

scan:
  # Maximum files to scan per PR (default: 20 on Free, 50 on Team)
  max_files: 50

  # File patterns to exclude from scanning
  exclude:
    - "**/*.test.ts"
    - "**/fixtures/**"
    - "**/__mocks__/**"
    - "generated/**"

detectors:
  semgrep: true
  trufflehog: true
  claude_ai: true   # Team+ only

thresholds:
  # "fail" blocks the PR check run, "warn" posts a comment only
  critical: fail
  high: warn
  medium: warn
  low: off

notifications:
  slack:
    channel: "#security-alerts"
    on:
      - critical
      - high

Ignoring findings

Suppress false positives with inline comments or in .codesheriff.yml:

Inline suppression (single line):

typescript

// codesheriff-ignore: hardcoded-secret (test credentials only)
const TEST_API_KEY = "sk_test_4xKj...";

Global ignore in config:

yaml

# .codesheriff.yml
ignore:
  rules:
    - trufflehog.generic-api-key   # too noisy in this repo
  paths:
    - "src/test/fixtures/**"
    - "**/*.snap"

Ignoring critical findings requires a team-level approval in the CodeSheriff dashboard. This creates an audit trail for compliance purposes.

Semgrep rules

CodeSheriff runs the following Semgrep rule sets on every scan:

Rule set	Coverage	Languages
p/owasp-top-ten	Injection, XSS, insecure deserialization	JS, TS, Python, Go, Java
p/cwe-top-25	Buffer overflows, path traversal, SSRF	C, C++, Python, Go, Java
p/react	dangerouslySetInnerHTML, eval injection	JS, TS (React)
p/nodejs	Command injection, prototype pollution	JS, TS
p/django	SQL injection, CSRF, XSS	Python
p/golang	SQL injection, path traversal, crypto misuse	Go

Secrets detection

TruffleHog scans the full PR diff (including commit history) for 700+ secret types. Common detectors:

AWS access keys & secrets

GitHub personal access tokens

Stripe live & test keys

Google Cloud service accounts

Twilio API credentials

Slack webhook URLs

SendGrid API keys

Database connection strings

JWT secrets

Private keys (RSA, EC)

Cloudflare API tokens

Generic high-entropy strings

TruffleHog scans the diff content sent via GitHub webhooks. CodeSheriff never clones your repository or stores source code.

Claude AI detectorsTeam+

Claude AI detectors run semantic analysis that static rules can't catch. Current detectors:

Hallucinated SDK methods

Verifies API calls against known SDK versions. Catches methods that AI invented but don't exist.

IDOR & missing authz

Identifies routes that fetch or mutate resources without checking ownership or roles.

Auth flow logic errors

Detects incorrect JWT validation, missing token expiry checks, and session fixation patterns.

Unsafe deserialization

Flags eval(), Function() constructors, and pickle.loads() applied to user-controlled input.

Race conditions

Detects check-then-act patterns and TOCTOU vulnerabilities in async code.

SARIF exportTeam+

Every scan produces a SARIF 2.1.0 report downloadable from the dashboard or via API. Compatible with GitHub Advanced Security Code Scanning, Azure DevOps, and any SARIF-aware tool.

bash

# Download SARIF report via API
curl -H "Authorization: Bearer <token>" \
  https://api.codesheriff.dev/v1/scans/{scan_id}/sarif \
  -o results.sarif

# Upload to GitHub Advanced Security
gh api \
  --method POST \
  -H "Accept: application/vnd.github+json" \
  /repos/{owner}/{repo}/code-scanning/sarifs \
  -f sarif=$(cat results.sarif | gzip | base64 -w 0) \
  -f ref="refs/heads/main" \
  -f commit_sha=$(git rev-parse HEAD)

Slack notificationsTeam+

Connect a Slack workspace from the CodeSheriff dashboard under Settings → Integrations → Slack. You can configure per-repository channels and severity thresholds.

Sample Slack alert format:

slack message

🚨 CodeSheriff — CRITICAL finding

Repo:    acme/backend
PR:      #142 — "feat: add payment processing"
Author:  @johndoe

Finding: Hallucinated API detected (stripe.charges.createInstant)
File:    src/utils/payments.ts:47
Risk:    74/100

[View PR comment →]  [Dismiss →]

Next steps

Install the GitHub App Explore the dashboard View pricing plans