Run Shakedown against your own checkout

Quick start

You need Node 22.18 or later and a PayPal sandbox app. Always use the scoped package name: a bare npx shakedown runs an unrelated package.

npx @shakedown-dev/cli preflight --target http://localhost:3000
npx @shakedown-dev/cli run --target http://localhost:3000
npx @shakedown-dev/cli report

preflight checks your setup, the sandbox lock and your store. run sends the customers and writes the reports to .shakedown/. report opens the HTML report, or prints the last run as the terminal receipt, Markdown, JUnit or JSON.

Environment

Secrets come from the environment or .env.local, never from the config file, and Shakedown never prints them.

Exit codes

0 pass · 1 leaks · 2 inconclusive (strict mode) · 3 safety lock · 4 config · 5 the store isn’t answering.

What your store needs to answer

Shakedown talks to your store over HTTP. Leaky Llama, the demo store in this repository, is the reference implementation.

Config

Shakedown loads the first shakedown.config.{ts,mts,js,mjs,json,yaml,yml} it finds. Unknown settings are an error, and so is anything that looks like a secret.

import { defineConfig } from '@shakedown-dev/cli'

export default defineConfig({
  target: { url: 'http://localhost:3000' },
  cast: ['double-clicker', 'cart-shuffler', 'echo', 'bouncer'],
  seed: 2026, // the same seed replays the same customers and amounts
  budget: { minutes: 10 },
})

By default a run sends the four customers that need no AI, so it costs nothing beyond sandbox calls. Add policy-lawyer to test your support assistant, with your refund policy as rules in policy.

CI

The build fails on a leak (exit code 1), and comment keeps one scoreboard comment on the pull request up to date, using the workflow’s own token.

- run: npx @shakedown-dev/cli run --ci
  env:
    PAYPAL_CLIENT_ID: ${{ secrets.PAYPAL_CLIENT_ID }}
    PAYPAL_CLIENT_SECRET: ${{ secrets.PAYPAL_CLIENT_SECRET }}
    SHAKEDOWN_PROBE_SECRET: ${{ secrets.SHAKEDOWN_PROBE_SECRET }}
- if: always() && github.event_name == 'pull_request'
  run: npx @shakedown-dev/cli comment
  env:
    GITHUB_TOKEN: ${{ github.token }}

The console

The console is an AG Studio dashboard over every run: the Scoreboard, the cast, the leak waterfall, each finding with its evidence, and the ledger. Click a leak or a customer and the rest of the page follows. Its Triage agent answers questions from a summary computed in code, and hands everything else to AG Studio’s built-in agents. Its Claude spend has its own cap.

How it decides

The customers are scripts and the AI does the talking; code keeps the score. A check passes or fails from PayPal’s sandbox records: captures, refunds and the webhooks the store accepted. The same seed gives the same verdicts and amounts on every re-run, and a saved run can be judged again by newer graders without repeating a call. The model never decides whether money leaked.

Responsible use, and how each rule is enforced

Sandbox only

Every PayPal call goes through a sandbox lock that refuses any host but the sandbox API (packages/paypal/src/sandbox-lock.ts), and PAYPAL_ENV must be sandbox or nothing runs.

Your integration only

A target must be local or on a private network, or allow-listed and serving your verification token, before the first request (packages/core/src/guards.ts).

Fictional customers

The cast are scripted test characters. They pay with PayPal’s published sandbox test cards and use generated example.com addresses.

Bounded by design

Each campaign has a request and wall-clock budget, and every Claude call passes a spend gate with a hard cap that refuses before anything is sent (packages/ai/src/gate.ts).

Outcomes, not playbooks

Findings report what would have leaked, the evidence and the fix. See RESPONSIBLE_USE.md and SECURITY.md in the repository.