audit-code install-tools.sh:
- Buffer the opengrep release JSON before grep -m1; curl died with (23)
under pipefail when grep quit early.
- Use ${m}: in the PowerShell block; $m: parsed as a scope-qualified var.
- On Arch, skip paru/yay when pacman -Q shows every package installed,
since --needed still invokes sudo.
- Add --check-only (fast, installs nothing, non-zero naming missing tools)
and --user-only (no system package managers, no sudo).
log-run.py (both skills): put the skill dir on sys.path so running it as
a script from any cwd no longer raises ModuleNotFoundError.
audit-terraform: move deps from requirements.txt into pyproject
dependency groups and add scripts/install-tools.sh (uv sync --group tools,
then check trivy, tflint, tofu, terragrunt, gh).
Both SKILL.md files gain a 0.5 Preflight step and call scripts through
uv run --project ${SKILL_DIR}. tools_unavailable is now a map of tool to
exact install command; audit-terraform skips trivy when absent and stops
with an install hint instead of crashing when tofu/terragrunt is missing.
317 lines
13 KiB
Markdown
317 lines
13 KiB
Markdown
---
|
|
name: audit-terraform
|
|
description: Automated tool-driven audit of terraform or terragrunt changes — runs trivy/tflint/terraform validate and dispatches parallel agents for plan validity, AWS security posture, and repo consistency, in the local working tree or a git ref / PR. For a guided human walkthrough of a PR use `review-pr` instead. Auto-activates on "audit this terraform", "check this terragrunt change", or "/audit-terraform".
|
|
---
|
|
|
|
# audit-terraform
|
|
|
|
You review terraform/terragrunt changes by running a mechanical collection
|
|
script, collecting first-pass Trivy config findings, and fanning out a
|
|
targeted LLM review pass against its manifest. One of the parallel agents
|
|
is a walkthrough agent that produces the reviewer-facing summary of what
|
|
the change does, plan-unit by plan-unit.
|
|
|
|
**Always announce at start:** "Using audit-terraform to walk through the
|
|
change and audit against plan validity, Trivy findings, AWS best practices,
|
|
and consistency."
|
|
|
|
## When to invoke
|
|
|
|
- User says "review terraform" / "review the terragrunt change" / etc.
|
|
- User runs `/audit-terraform` with or without an argument.
|
|
|
|
## Modes
|
|
|
|
| Invocation | Mode | Diff target | Output |
|
|
|-------------------------------------|---------|----------------------------|---------------------------------|
|
|
| `/audit-terraform` | local | local working tree vs base | interactive walkthrough in chat |
|
|
| `/audit-terraform <ref>` | ref | `<ref>` vs base | `audit-terraform-<short>.md` |
|
|
| `/audit-terraform <pr_number>` | ref | PR head ref vs base | same as above |
|
|
|
|
For ref/PR mode you must work in a fresh worktree. For local mode you work
|
|
in the user's current repo.
|
|
|
|
## Procedure
|
|
|
|
### 0. Resolve `SKILL_DIR`
|
|
|
|
`${SKILL_DIR}` below means the absolute directory containing *this* SKILL.md.
|
|
You were given that path when this skill loaded — export it once before any
|
|
other command so the bundled scripts resolve wherever the plugin is installed:
|
|
|
|
```
|
|
export SKILL_DIR=<absolute path to the directory holding this SKILL.md>
|
|
```
|
|
|
|
### 0.5 Preflight — every run
|
|
|
|
```
|
|
bash ${SKILL_DIR}/scripts/install-tools.sh --check-only
|
|
```
|
|
|
|
It installs nothing and returns in milliseconds when everything is present,
|
|
so run it every time. Each plugin version runs from its own directory with a
|
|
fresh, empty `.venv`, so expect it to fail on the first run after an update.
|
|
|
|
- **Exit 0:** continue.
|
|
- **Non-zero:** it lists what is missing. Run
|
|
`bash ${SKILL_DIR}/scripts/install-tools.sh` (only `uv sync` into
|
|
`${SKILL_DIR}/.venv`; never sudo), then re-run `--check-only`.
|
|
- **Still missing** (`trivy`, `tflint`, `tofu`, `terragrunt`, `gh` are native
|
|
binaries the script does not install): show the user the install commands
|
|
the script printed and ask before running any. If they decline, continue:
|
|
the collection script records each missing tool in `tools_unavailable` with
|
|
its install command, and stops with an error if a plan needs a missing
|
|
`tofu` / `terragrunt`.
|
|
|
|
### 1. Resolve mode, repo identity, and worktree
|
|
|
|
First resolve `NWO` (owner/repo) so every `gh` call works regardless of
|
|
cwd — git checkout, jj workspace, or outside any repo:
|
|
|
|
```fish
|
|
set NWO (gh repo view --json nameWithOwner -q .nameWithOwner 2>/dev/null
|
|
or jj git remote list 2>/dev/null \
|
|
| awk '$1=="origin"{print $2}' \
|
|
| sed -E 's#.*github.com[:/]([^/]+/[^/.]+?)(\.git)?$#\1#'
|
|
or git remote get-url origin 2>/dev/null \
|
|
| sed -E 's#.*github.com[:/]([^/]+/[^/.]+?)(\.git)?$#\1#')
|
|
```
|
|
|
|
Always pass `--repo "$NWO"` on `gh` calls — do not let `gh` autodetect from
|
|
the cwd, since that runs `git` internally and fails in jj-only workspaces
|
|
with `fatal: not a git repository`.
|
|
|
|
Then resolve mode:
|
|
|
|
- No argument: mode = `local`. `REPO = <cwd>`.
|
|
- Argument matches `^[0-9]+$`: GitHub PR number. Resolve the head ref:
|
|
|
|
```
|
|
gh pr view <PR> --repo "$NWO" --json headRefName,headRepository \
|
|
-q '.headRefName + "@" + .headRepository.url'
|
|
```
|
|
|
|
If `gh` is missing or not authenticated, stop with: "Install `gh` and run
|
|
`gh auth login`, or pass a git ref instead of a PR number." If `NWO` is
|
|
empty, stop with: "Cannot determine GitHub repo from this directory.
|
|
Run from inside a checkout of the repo, or pass an explicit ref."
|
|
- Any other string: treat as a git ref.
|
|
|
|
For ref/PR mode, isolate the checkout without depending on the cwd being
|
|
a git repo:
|
|
|
|
- If a native worktree tool (e.g. `EnterWorktree`) is available AND the cwd
|
|
is a git checkout of `$NWO`, prefer `superpowers:using-git-worktrees` to
|
|
create `~/.claude/cache/audit-terraform/<short-ref>/`.
|
|
- Otherwise (jj workspaces, or invoked from outside the repo), do a fresh
|
|
clone — this never touches the surrounding workspace:
|
|
|
|
```
|
|
gh repo clone "$NWO" ~/.claude/cache/audit-terraform/<short-ref>/
|
|
git -C ~/.claude/cache/audit-terraform/<short-ref>/ checkout <ref>
|
|
```
|
|
|
|
For PR mode, `<ref>` is the head branch returned by `gh pr view` above.
|
|
|
|
`REPO = ~/.claude/cache/audit-terraform/<short-ref>/`.
|
|
|
|
> **jj users:** Local mode works in jj workspaces that have a colocated
|
|
> `.git` (the script reads the working tree but resolves the diff base via
|
|
> git). For non-colocated additional `jj workspace add` checkouts, run
|
|
> ref/PR mode instead — the fresh-clone path works regardless of cwd.
|
|
|
|
### 2. Resolve output directory
|
|
|
|
- Ref mode: `OUTPUT = <REPO>/.audit-terraform/`
|
|
- Local mode: `OUTPUT = ~/.claude/cache/audit-terraform/local-<UTC-timestamp>/`
|
|
|
|
Create the directory.
|
|
|
|
### 3. Run the collection script
|
|
|
|
```
|
|
uv run --project ${SKILL_DIR} python ${SKILL_DIR}/scripts/collect-changes.py \
|
|
--repo <REPO> --base <base-or-detect> --head <head-ref-or-HEAD> \
|
|
--output-dir <OUTPUT> --mode <local|ref>
|
|
```
|
|
|
|
- If exit code != 0, read `<OUTPUT>/manifest.json` `errors[]` and report them
|
|
to the user verbatim. Do not run subagents. Stop.
|
|
- If the manifest has zero `plan_units` AND zero `catalog` entries, tell the
|
|
user "no terraform changes detected" and stop.
|
|
- The collection step also writes raw Trivy config results to
|
|
`<OUTPUT>/trivy-findings.json` and normalizes compact `trivy_findings` into
|
|
the manifest for downstream security review.
|
|
|
|
### 4. Fan out focused subagents IN PARALLEL
|
|
|
|
Dispatch four Task subagents in a single message (parallel execution). Each
|
|
gets the system prompt from `${SKILL_DIR}/agents/<agent>.md`
|
|
and these variables substituted into its task:
|
|
|
|
- `MANIFEST = <OUTPUT>/manifest-<agent>.json` (per-agent slice; full manifest stays at `<OUTPUT>/manifest.json` for debugging)
|
|
- `REFERENCE_SETS = <OUTPUT>/reference_sets.json` (consistency only)
|
|
- `CONSISTENCY_NORMS = <OUTPUT>/consistency_norms.json` (consistency only)
|
|
- `REPO = <REPO>`
|
|
- `OUTPUT = <OUTPUT>/findings-<agent>.json` (walkthrough uses the same
|
|
filename but its payload is the walkthrough JSON, not findings).
|
|
|
|
Agents: `walkthrough-reviewer`, `aws-bp-reviewer`, `consistency-reviewer`,
|
|
`tf-hygiene-reviewer`.
|
|
|
|
Default path:
|
|
- `walkthrough-reviewer` produces the reviewer-facing PR summary —
|
|
overview plus adaptive per-plan-unit walkthrough. Emits no findings.
|
|
- `aws-bp-reviewer` consumes normalized `trivy_findings` first, suppresses
|
|
overlap/noise, and adds only contextual AWS best-practice findings Trivy is
|
|
likely to miss.
|
|
- `consistency-reviewer` stays repo-internal only.
|
|
- `tf-hygiene-reviewer` consumes `tflint_findings`, suppresses noise, and
|
|
adds module-hygiene findings tflint can't infer (var/output descriptions,
|
|
version pinning, lifecycle, terragrunt patterns). Strictly non-security.
|
|
|
|
`fsbp-reviewer` and `cis-reviewer` are legacy benchmark-specific prompts kept
|
|
for explicit fallback or cross-check work, not the default review path.
|
|
|
|
### 5. Aggregate findings
|
|
|
|
After the active agents return:
|
|
|
|
1. Load `findings-walkthrough-reviewer.json` separately as the
|
|
`walkthrough` payload (`overview` + `plan_units[]`). Discard if
|
|
malformed and note in summary.
|
|
2. Load each remaining `findings-<agent>.json`. Discard any agent file
|
|
that's malformed (note in the summary).
|
|
3. Deduplicate findings sharing `{resource, control}`. First-to-finish
|
|
wins; add a `also_flagged_by` array on the survivor with the loser's
|
|
`agent` and `control`.
|
|
4. Group findings by severity: critical, high, medium, low.
|
|
|
|
### 5.5 Record telemetry — MANDATORY, one CLI call
|
|
|
|
After all subagents return (success or failure), invoke the logger
|
|
**once**. It scans `<OUTPUT>/findings-<agent>.json` for each agent and
|
|
appends one `subagent_run` row to
|
|
`~/.claude/cache/audit-terraform/runs.jsonl` with the finding count read
|
|
from the file, plus the token / duration metadata you pass in.
|
|
|
|
Build a JSON blob from each Task call's `<usage>` block and pipe it in:
|
|
|
|
```fish
|
|
echo '{
|
|
"walkthrough-reviewer": {"model":"sonnet","input_tokens":N,"output_tokens":N,"duration_ms":N},
|
|
"aws-bp-reviewer": {"model":"sonnet","input_tokens":N,"output_tokens":N,"duration_ms":N},
|
|
"consistency-reviewer": {"model":"sonnet","input_tokens":N,"output_tokens":N,"duration_ms":N},
|
|
"tf-hygiene-reviewer": {"model":"haiku","input_tokens":N,"output_tokens":N,"duration_ms":N}
|
|
}' | uv run --project ${SKILL_DIR} python ${SKILL_DIR}/scripts/log-run.py \
|
|
--output-dir <OUTPUT> --run-id $RUN_ID --repo <REPO> \
|
|
--mode <local|ref> --usage-json -
|
|
```
|
|
|
|
`RUN_ID` is a short random hex (8 chars) generated once at the start of
|
|
the run. Record it in the chat headline so the user can correlate later
|
|
verdicts back to the run.
|
|
|
|
Skipping this step means no precision or tokens-per-kept data — do not
|
|
skip it.
|
|
|
|
**Per-agent default models:**
|
|
|
|
| Agent | Default model |
|
|
|---|---|
|
|
| `walkthrough-reviewer`| Sonnet |
|
|
| `aws-bp-reviewer` | Sonnet |
|
|
| `consistency-reviewer`| Sonnet |
|
|
| `tf-hygiene-reviewer` | Haiku 4.5 |
|
|
|
|
`tf-hygiene-reviewer` runs on Haiku because tflint already did the heavy
|
|
mechanical work — the agent's job is triage + a small number of
|
|
contextual additions.
|
|
|
|
### 6. Output
|
|
|
|
#### Local mode (interactive)
|
|
|
|
Send a chat message structured like:
|
|
|
|
```
|
|
Reviewed N plan units, M resources (X plan+diff, Y diff-only, Z plan-only).
|
|
|
|
WALKTHROUGH
|
|
<overview paragraph>
|
|
|
|
Substantive plan units:
|
|
- <plan_dir> — <2-4 sentences: what + why + destroy/replace notes>
|
|
- ...
|
|
Trivial:
|
|
- <plan_dir> — <one-liner>
|
|
- ...
|
|
|
|
CRITICAL (n)
|
|
- <resource> (<dir>): <control> — <issue>
|
|
fix: <fix>
|
|
|
|
HIGH (n)
|
|
- ...
|
|
|
|
MEDIUM (n) — say "expand medium" to see
|
|
LOW (n) — say "expand low" to see
|
|
|
|
CONSISTENCY (n)
|
|
- <dir>: <issue>
|
|
|
|
Full findings: <OUTPUT>/findings-*.json
|
|
```
|
|
|
|
Offer follow-ups: "Ask me to drill into anything, expand a section, or
|
|
generate fixes."
|
|
|
|
#### Ref mode (report)
|
|
|
|
Write `<REPO>/audit-terraform-<short-ref>.md` structured as:
|
|
|
|
1. Summary table (resources by detection source, findings by severity).
|
|
2. **Walkthrough** — overview paragraph, then two subsections:
|
|
- "Substantive plan units" (each substantive plan_dir as a subheading
|
|
with the summary beneath, destroy/replace call-outs bolded)
|
|
- "Trivial plan units" (one-line bullets)
|
|
3. Plan summary per changed dir (add/change/destroy counts).
|
|
4. Findings grouped by severity → category → resource, each with control
|
|
reference, evidence (file:line), suggested fix.
|
|
5. Consistency findings (separate section).
|
|
6. Skipped resources (non-AWS, plan-failed-but-still-reviewed, etc.).
|
|
|
|
Send a 5-line headline to chat plus the file path.
|
|
|
|
### 7. Collect verdicts (signal-quality feedback loop)
|
|
|
|
After the user has reviewed findings, ask for verdicts so the skill can
|
|
measure precision over time. This applies to local mode only — prompt
|
|
interactively (skip on `--no-feedback`).
|
|
|
|
For each finding, capture `{kept | dismissed | false_positive}` plus an
|
|
optional note. Append a `verdict` record per finding to
|
|
`~/.claude/cache/audit-terraform/runs.jsonl` via
|
|
`scripts.telemetry.append_verdict`.
|
|
|
|
> Ref-mode reports do not collect verdicts. `review_stats.py` takes no
|
|
> arguments; it only aggregates what local mode already wrote.
|
|
|
|
### 8. Always surface the worktree path
|
|
|
|
In ref mode, end with: "Worktree left at `<REPO>` for follow-up review."
|
|
|
|
## Failure modes
|
|
|
|
- **Plan failed in some dir:** Manifest's `errors[]` populated, script exited
|
|
non-zero. Show the errors. Do NOT run agents. Tell user to fix and re-run.
|
|
- **No `gh`:** see step 1.
|
|
- **Scanner or plan tool missing:** `tools_unavailable` maps each missing tool
|
|
to its install command. A missing `trivy` / `tflint` only skips that scan;
|
|
say so in the output. A missing `tofu` / `terragrunt` fails collection.
|
|
- **Module with no callsites:** Manifest has a warning in `errors[]`; report
|
|
it but still run the subagents (they handle diff-only entries fine).
|
|
- **One agent fails:** Report what the surviving agents found and note the
|
|
agent that failed.
|