CLAUDE.md · KeygraphHQ/shannon

Sections

1. Commands
2. Architecture
3. Development Notes
4. Code Style Guidelines
5. Key Files
6. Package Installation
7. Troubleshooting

First lines (shown under the repo's AGPL-3.0 license)

# CLAUDE.md

AI-powered penetration testing agent for defensive security analysis. Automates vulnerability assessment by combining reconnaissance tools with AI-powered code analysis.

## Commands

**Prerequisites:** Docker, AI provider credentials (`.env` for local, `npx @keygraph/shannon setup` or env vars for npx)

### Dual CLI

Shannon supports two CLI modes, auto-detected based on the current working directory:

| | **npx** (`npx @keygraph/shannon`) | **Local** (`./shannon`) |
|---|---|---|
| **Install** | Zero-install via npm | Clone the repo |
| **Image** | Pulled from Docker Hub (`keygraph/shannon:latest`) | Built locally (`shannon-worker`) |
| **State** | `~/.shannon/` | Project directory |
| **Credentials** | `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) or env vars | `./.env` |
| **Config** | `~/.shannon/config.toml` (via `npx @keygraph/shannon setup`) | N/A |
| **Prompts** | Bundled in Docker image | Mounted from `./apps/worker/prompts/` (live-editable) |

Mode auto-detection: local mode activates when env var `SHANNON_LOCAL=1` is set by the `./shannon` entry point (`apps/cli/src/mode.ts`). Otherwise npx mode.

### npx Quick Start

```bash
# Configure credentials (interactive wizard)
npx @keygraph/shannon setup

# Or export env vars directly (non-interactive / CI)
export ANTHROPIC_API_KEY=your-key

# Run
npx @keygraph/shannon start -u <url> -r /path/to/repo
```

### Local (Development) Quick Start

```bash
# Setup
echo "ANTHROPIC_API_KEY=your-key" > .env

# Build (auto-runs if image missing)
./shannon build

# Run
./shannon start -u <url> -r ./my-repo
./shannon start -u <url> -r ./my-repo -c ./apps/worker/configs/my-config.yaml
./shannon start -u <url> -r /any/path/to/repo
```

### Common Commands

```bash
# Setup (npx mode only — one-time credential configuration)
npx @keygraph/shannon setup

# Workspaces & Resume
./shannon start -u <url> -r ./my-repo -w my-audit    # New named workspace
./shannon start -u <url> -r ./my-repo -w my-audit    # Resume (same command)

# Monitor
./shannon scans                       # List running and completed scans, with each report's path
./shannon logs [<workspace>]          # Show a scan's live log (default: the single running scan, else the most recent)
./shannon logs [<workspace>] --agent <name>   # Tail one agent's own log (from .shannon/agents/)
./shannon logs [<workspace>] --list-agents    # List the agents that have their own log
./shannon status [<workspace>]        # Live phase/agent progress of one scan, read from Temporal (redraws, then exits; same default target)
# Dashboard: http://localhost:8233

# Stop
./shannon stop [<workspace>]          # Stop one scan (default: the single running scan; confirms first; --yes/-y to skip)
./shannon stop --all                  # Stop all running scans (Temporal stays up; confirms first)
./shannon reset                       # Stop everything and wipe all Temporal data + volumes (type 'confirm' to proceed; cannot be skipped)

# Version
./shannon version                     # npx: package version; local: git SHA

# Image management
./shannon build [--no-cache]          # Local mode: build worker image

# Build TypeScript (development)
pnpm run build                       # Build all packages via Turborepo
pnpm run check                       # Type-check all packages
pnpm biome                           # Biome lint + format + import sorting check
pnpm biome:fix                       # Auto-fix lint, format, and import sorting
```

**Monorepo tooling:** pnpm workspaces, Turborepo for task orchestration, Biome for linting/formatting. TypeScript compiler options shared via `tsconfig.base.json` at the root. All packages extend it, overriding only `rootDir` and `outDir`. Shared devDependencies (`typescript`, `@types/node`, `turbo`, `@biomejs/biome`) are hoisted to the root workspace.

**Options:** `-c <file>` (YAML config), `--models-config <file>` (pi `models.json` defining models pi's catalogue lacks), `-o <path>` (output directory), `-w <name>` (named workspace; auto-resumes if exists), `--validate-auth` (run preflight and auth validation only, then stop; no pentest or report; requires a fresh workspace and an `authentication` block in the config), `--validate-model` (run the preflight model checks only — credential/registry resolution plus the cyber-access verification for cyber-gated providers — then stop; no pentest or report; requires a fresh workspace; needs no config; mutually exclusive with `--validate-auth`), `--pipeline-testing` (minimal prompts, 10s retries), `--keep-container` (preserve worker container after exit for log inspection), `--yes`/`-y` (skip the confirmation prompt on `stop`; required for non-interactive use; `reset` requires a typed `confirm` and cannot be skipped)

## Architecture

### Monorepo Layout

```
apps/cli/        — @keygraph/shannon (published to npm, bundled with tsdown)
apps/worker/     — @shannon/worker (private, Temporal worker + pipeline logic)
```

### CLI Package (`apps/cli/`)
Published as `@keygraph/shannon` on npm. Contains Docker orchestration and a direct `@temporalio/client` integration for read-only status plus bounded workflow lifecycle operations; no worker/pipeline business logic or prompts. Bundled with tsdown for single-file ESM output (deps stay external).

- `apps/cli/src/index.ts` — CLI dispatcher (`setup`, `start`, `stop`, `reset`, `logs`, `status`, `scans`, `build`, `version`)
- `apps/cli/src/temporal-client.ts` — `@temporalio/client` integration: connects to the frontend on `127.0.0.1:7233` (published by compose), provides `describeScan` (status + `pendingActivities` → running agents), `queryProgress` (live `getProgress` query → `PipelineState`), `getTerminalOutcome` (workflow `result()`), and bounded lifecycle RPCs for `stop`. `stop` requests cancellation first, waits up to 10 seconds, then requests termination only when necessary and verifies closure within a bounded window. No worker of its own; scans are visible within Temporal's retention window, which `ensureInfra` (`apps/cli/src/docker.ts`) converges to `168h` (7 days) on every successful `shannon start`; override with `SHANNON_TEMPORAL_RETENTION` (a positive whole-hour value like `72h`)
- `apps/cli/src/scan/` — `status` rendering: `pipeline.ts` (static phase/agent plan + `run*Agent` activity-type→agent map + mirrored `PipelineState`/`AgentMetrics` types; keep in sync with the worker), `derive.ts` (pure phase/agent state derivation shared by the tree and `--json`), `render.ts` (one renderer for both the live query state and the terminal result). The tree shows model work only: every row is an agent, an Agentic SAST stage, or a report step that is currently running or failed. Reconciliation is model work owned by a class, so its wall time renders as a trailing `+ duration` on that class's exploitation row (its analysis row when `exploit: false`) rather than as a row of its own; deterministic bookkeeping stages (`report:*` renumber/assemble/finalize/surface) never appear once they complete. `DerivedPhase.children` (renders sub-rows) and `DerivedPhase.meta` (`duration` vs a `k/N done` tally) are independent — Agentic SAST lists stages under a duration, exploitation lists classes under a tally
- `apps/cli/src/mode.ts` — Auto-detection: local mode if `SHANNON_LOCAL=1` env var is set
- `apps/cli/src/docker.ts` — Compose lifecycle, image pull/build, and ephemeral `docker run` worker spawning. Each worker carries workspace, task-queue, and preselected workflow-ID labels so stop can correlate the local worker with its Temporal execution before `session.json` exists. Before `docker run`, start fsyncs that exact candidate under the workspace's hidden internals and clears it only when `session.json` registers the same ID; stop reconciles any candidate left by an interrupted launch. Start also checks the image's workflow-ID protocol label and refuses a stale worker that would ignore the preselected ID
- `apps/cli/src/home.ts` — State directory management (`~/.shannon/` for npx, `./` for local)
- `apps/cli/src/env.ts` — `.env` loading, TOML fallback (npx only) via `apps/cli/src/config/resolver.ts`, credential validation, provider-scoped env flag building
- `apps/cli/src/model-spec.ts` — `SHANNON_AI_MODEL` (`<provider>:<model-id>`) parsing; mirrors `apps/worker/src/ai/models.ts`
- `apps/cli/src/config/resolver.ts` — Cascading config (npx only): env vars → `~/.shannon/config.toml` (parsed with `smol-toml`)
- `apps/cli/src/config/writer.ts` — TOML serialization and secure file persistence (0o600)
- `apps/cli/src/commands/setup.ts` — Interactive TUI wizard (`@clack/prompts`) for provider credential setup (npx only)
- `apps/cli/src/paths.ts` — Repo/config/models-config path resolution (any absolute or relative path). `MODELS_CONFIG_CONTAINER_PATH` is fixed at `/app/models.json` because the worker names it to pi rather than discovering it
- `apps/cli/src/version.ts` — Version reporting (npx: `package.json` version; local: `git-<sha>`)
- `apps/cli/src/tty.ts` — Terminal capability detection: `requireInteractive` guard (fails fast off-TTY instead of hanging on a prompt), `supportsColor` color gating (`NO_COLOR`/`FORCE_COLOR`), and `stdoutIsTerminal` for spinner/cursor output
- `apps/cli/src/commands/` — Command handlers
- `apps/cli/infra/compose.yml` — Bundled Temporal compose file for npx mode
- `apps/cli/tsdown.config.ts` — tsdown bundler config
- `shannon` — Node.js entry point (`#!/usr/bin/env node`) that delegates to `apps/cli/dist/index.mjs`

### Docker Architecture
Infra (Temporal) runs via `docker-compose.yml`. Workers are ephemeral `docker run --rm` containers, one per scan, each with a unique task queue, preselected workflow ID, matching identity labels, and isolated volume mounts. `shannon stop --all` takes the union of labeled running workers and Temporal-running workflows, so an orphaned workflow is still stopped after its worker has disappeared.

- `docker-compose.yml` — Infra only: `shannon-temporal` (port 7233/8233). Network: `shannon-net`
- `Dockerfile` — 2-stage build (builder + Chainguard Wolfi runtime). Uses pnpm. Entrypoint: `CMD ["node", "apps/worker/dist/temporal/worker.js"]`
- No `docker-compose.docker.yml` — host gateway handled via `--add-host` flag in CLI
- `/etc/hosts` forwarding — at worker spawn, `forwardEtcHostsFlags` in `apps/cli/src/docker.ts` reads the host's `/etc/hosts` and emits one `--add-host` flag per valid user-added entry. Loopback IPs (`127.x`, `::1`) are rewritten to `host-gateway`; IPv6 addresses are bracketed. Disable per-scan via `SHANNON_FORWARD_HOSTS=false`. Native Windows is refused at startup (`blockNativeWindows` in `apps/cli/src/index.ts`, pointing to the WSL2 guide in `docs/platforms.md`); WSL2 reads its own `/etc/hosts` via the Linux path.

### Worker Package (`apps/worker/`)
- `apps/worker/src/paths.ts` — Centralized path constants (`PROMPTS_DIR`, `CONFIGS_DIR`, `WORKSPACES_DIR`)
- `apps/worker/src/session-manager.ts` — Agent definitions (`AGENTS` record). Agent types in `apps/worker/src/types/agents.ts`
- `apps/worker/src/config-parser.ts` — YAML config parsing with JSON Schema validation
- `apps/worker/src/ai/pi/pi-executor.ts` — pi harness integration (agent-level retry disabled so Temporal owns restarts; provider-level retry on, see `apps/worker/src/ai/pi/retry-settings.ts`)
- `apps/worker/src/services/` — Business logic layer (Temporal-agnostic). Activities delegate here. Key: `agent-execution.ts`, `error-handling.ts`, `container.ts`
- `apps/worker/src/types/` — Consolidated types: `Result<T,E>`, `ErrorCode`, `AgentName`, `ActivityLogger`, etc.
- `apps/worker/src/utils/` — Shared utilities (file I/O, formatting, concurrency)

### Temporal Orchestration
Durable workflow orchestration with crash recovery, queryable progress, intelligent retry, and parallel execution (5 concurrent agents in vuln/exploit phases).

- `apps/worker/src/temporal/workflows.ts` — Main workflow (`pentestPipelineWorkflow`)
- `apps/worker/src/temporal/activities.ts` — Thin wrappers — heartbeat loop, error classification, container lifecycle. Business logic delegated to `apps/worker/src/services/`
- `apps/worker/src/temporal/activity-logger.ts` — `TemporalActivityLogger` implementation of `ActivityLogger` interface
- `apps/worker/src/temporal/summary-mapper.ts` — Maps `PipelineSummary` to `WorkflowSummary`
- `apps/worker/src/temporal/worker.ts` — Combined worker + client entry point (per-invocation task queue, submits workflow, waits for result)
- `apps/worker/src/temporal/shared.ts` — Types, interfaces, query definitions
### Five-Phase Pipeline

1. **Pre-Recon** (`pre-recon`) — Source code analysis to build the architectural baseline
2. **Recon** (`recon`) — Attack surface mapping from initial findings
3. **Vulnerability Analysis** (5 parallel agents) — injection, xss, auth, authz, ssrf
4. **Exploitation** (5 parallel agents, conditional) — Exploits confirmed vulnerabilities
5. **Reporting** (`report`) — Executive-level security report

Around those phases:

- Optional agentic static analysis runs before the pentest when `agentic_sast.enabled` is `"true"`, as a child workflow.
- After each class's analysis, reconciliation groups its findings into exploitation tasks.