mroberts 5e167d3a0d State the Checks API gap instead of prescribing an impossible tick
Fine-grained tokens have no Checks permission. GitHub's permission reference
lists no Checks section and no check-run endpoint, and checks is absent from
the token form's pre-fill parameters, so the earlier instruction to tick
Checks: Read asked for a box that does not exist. A token created with every
listed permission still could not read check runs, which is what surfaced this.

The prompt now states the consequence rather than offering a remedy: gh pr
checks reports commit statuses only and gh run view returns no annotations.
Both degrade to empty output rather than a permission error, so without the
note they read as a broken CI integration. Job logs are unaffected; they fall
under Actions, which is granted.

secret_scanning_alerts and vulnerability_alerts move into the pre-filled URL,
leaving nothing for the operator to tick beyond repository selection.
2026-07-31 08:48:26 -05:00

ai-sandbox

A shareable mise task that runs an AI coding agent inside a Docker Sandbox, scoped to exactly one GitHub repository and one set of read-only AWS roles.

Description

Giving an agent your everyday credentials gives it your everyday blast radius. A long-lived PAT reaches every repository you can reach; ~/.aws reaches every role your SSO session can assume, admin included. This task narrows both, and puts the boundary somewhere the agent cannot edit.

It provides a single command, ai:sbx, which:

  • Derives the repository from origin. No repository name is typed or configured, so the sandbox identity cannot drift from the checkout you are standing in. The sandbox name is ai-<owner>-<repo>-<digest>, stable across runs.
  • Scopes GitHub access to one repository. Setup opens the GitHub token form in your browser with the owner, expiry, name, and permissions already filled in — you pick the repository and paste the token back. It is stored with sbx secret set and injected by Docker's host-side proxy; it is never placed in GH_TOKEN, never written into the repository, and is not readable by the agent.
  • Keeps your AWS admin profiles out of the sandbox entirely. Your ~/.aws directory and your SSO token cache are never mounted or copied. Instead, the host runs aws configure export-credentials against named read-only profiles you approve once, and only the resulting short-lived role credentials are written into the sandbox. The agent cannot use your SSO session to discover or request other roles.
  • Renames profiles for Terraform. Host profiles are commonly suffixed to mark the grant — api-portal-readonly — while Terraform code references the account name, api-portal. A trailing -readonly is stripped when the profile is written into the sandbox, so unmodified Terraform resolves the read-only credentials.
  • Refreshes AWS credentials on every launch, since exported SSO credentials are short-lived.
  • Requires nothing from the repository. All state lives under ~/.config/ai-sbx/. Repositories that want first-class support can opt in with three lines of mise.toml; repositories that do not are unaffected, and developers who do not use mise never notice.

What the sandbox receives

Receives Does not receive
Temporary read-only role credentials Your SSO access or refresh token
Profile names, minus the -readonly suffix Your host ~/.aws/config or credentials
Proxy-injected GitHub auth for one repository A readable GitHub token
The repository working tree Admin profile names or role assignments

Installation

Dependencies

Install these on the host — none of them are needed inside the sandbox.

Tool Purpose Install
mise Runs the task and distributes it Getting started
Docker Sandboxes (sbx) Sandbox, secret store, credential proxy Get started — Docker Desktop is not required
KVM + membership of the kvm group Sandboxes are microVMs sudo usermod -aG kvm $USER, then re-login
AWS CLI v2, jq aws configure export-credentials Required only when using --aws-profile
git, sha256sum Repository identity Already present on most systems
xdg-open / open / $BROWSER Opens the token form Optional — the link is printed if absent

mise must be recent enough to load remote git:: task includes. Verified working on 2026.7.17; verified broken on 2025.10.6, which drops git:: entries silently — no error, no clone attempted, mise tasks ls simply prints nothing. If that is what you see, run mise self-update (note: mise upgrade updates your tools, not mise itself).

Verify:

mise --version
sbx version
aws --version
jq --version
lsmod | grep kvm        # must show kvm_intel, kvm_amd or kvm
id -nG | grep -w kvm    # you must be in the kvm group

sbx requires 0.37 or later — --clone replaced the older --branch flag, and this task uses --clone.

On Arch derivatives, install the docker-sbx AUR package, not docker-sandbox-bin. The latter ships only the CLI binary, omitting the microVM kernel, rootfs, and containerd-shim-nerdbox-v1. Without those, sbx has no VM to boot and falls back to mounting filesystems on the host, which fails with operation not permitted for any non-root user.

Adding the task to your personal mise config makes ai:sbx available in every Git repository on the machine, without touching a single repository.

Add to ~/.config/mise/config.toml:

[task_config]
includes = [
  "git::https://git.mroberts.dev/mroberts/ai-sandbox.git//tasks?ref=v1.2.0",
]

Pin ref to a tag or, better, a commit SHA. The task executes on your host with your credentials — a moving ref means an unreviewed change runs the next time the cache expires.

The repository is public, so https needs no credentials and works on a fresh machine with no SSH agent. For a private fork, use the SSH form instead, which requires a key registered with the forge:

includes = [
  "git::ssh://[email protected]/mroberts/ai-sandbox.git//tasks?ref=v1.2.0",
]

Optional personal defaults:

[env]
AI_SBX_AGENT = "claude"
AI_SBX_MODE = "clone"
# AI_SBX_TEMPLATE = "git.mroberts.dev/you/claude-sbx:v1"   # unset = stock image

Confirm it loaded:

mise tasks ls   # expect: ai:sbx

mise caches the clone under $MISE_CACHE_DIR/remote-git-tasks-cache. Force a refetch with MISE_TASK_REMOTE_NO_CACHE=true.

Repository-level install (optional)

A repository whose team has adopted the workflow can add the same include to its mise.toml:

[task_config]
includes = [
  "git::https://git.mroberts.dev/mroberts/ai-sandbox.git//tasks?ref=v1.2.0",
]

Developers with mise get ai:sbx; developers without mise are unaffected. Do not put AWS profile names or tokens in a repository config — the approved profile list is per-user state, and no secret belongs in a repository.

Both installs can coexist. Nothing about the workflow requires the repository-level one.

Usage

1. Authenticate your read-only AWS profiles on the host

aws sso login --profile api-portal-readonly
aws sso login --profile prod-readonly

Setup fails fast with the exact aws sso login command if a profile is missing or its session has expired.

2. Set up a repository

cd ~/src/api-portal

mise run ai:sbx -- setup \
  --aws-profile api-portal-readonly \
  --aws-profile prod-readonly

Derives the repository from origin, creates the sandbox, then opens the GitHub token form in your browser with everything pre-filled:

name          ai-sbx api-portal
target_name   CareEvolution
expires_in    30
metadata=read  contents=write  pull_requests=write  issues=write
workflows=write  actions=write  statuses=read  security_events=write
secret_scanning_alerts=read  vulnerability_alerts=read

One thing the form cannot pre-fill: Repository access → Only select repositories → api-portal. GitHub has no query parameter for repository selection.

The Checks API is out of reach

Fine-grained tokens cannot read check runs. This is not a permission you forgot to grant — GitHub's permission reference has no Checks section and lists no check-run endpoint, so there is no box to tick. Inside the sandbox:

Command Behaviour
gh pr checks shows commit statuses only, not check runs
gh run view returns no annotations
gh run view --log works — job logs fall under Actions

Both degrade to empty output rather than a permission error, so they read as a broken CI integration unless you know why. A 403 names what it wanted in the X-Accepted-GitHub-Permissions response header.

Only an installation token from a GitHub App can reach the Checks API. That was evaluated and rejected for this workflow: minting one requires the App private key on every developer's machine, and device-flow user tokens — the alternative that needs no key — were measured and do not honour repository_id, so they reach every repository in the installation.

Generate the token and paste it at the prompt. It is read with the terminal echo off and piped straight into the sbx secret store, so it never reaches your shell history.

There is no API to create a fine-grained token — GitHub only supports pre-filling the form — so this step is inherently a browser round trip. Rotating later is the same round trip:

mise run ai:sbx -- token

3. Run the agent

mise run ai:sbx -- run

Refreshes AWS credentials, then attaches. Pass agent arguments after a second --:

mise run ai:sbx -- run -- "Review the Terraform plan for the staging workspace"

4. Inside the sandbox

gh is already authenticated through the proxy, for that repository only:

gh pr list
gh pr create --fill

AWS named profiles work as Terraform expects:

aws sts get-caller-identity --profile api-portal

AWS_PROFILE=api-portal terraform init
AWS_PROFILE=api-portal terraform plan
provider "aws" {
  profile = "api-portal"
  region  = "us-east-1"
}

Commands

Command Effect
setup [options] Configure the repository, create the sandbox, open the token form, install AWS profiles, Claude configuration and plugins, and mise
token Replace the GitHub token for this repository — expiry, revocation, permission change
run [-- args...] Refresh AWS credentials and the repository's mise tools, then attach to the agent
refresh Refresh AWS credentials, without attaching
config Re-apply your Claude configuration and plugins after the host changes, without recreating the sandbox
status Show repository, sandbox, agent, mode, token expiry setting, profile mapping, stored secrets
remove Remove the sandbox and this repository's local configuration

setup options

Option Default Effect
--aws-profile NAME none Host profile to expose. Repeatable. Trailing -readonly stripped inside the sandbox
--agent NAME claude Sandbox agent. See sbx create --help for the list
--clone on Give the agent a private in-container clone; its commits reach the host via the sandbox-<name> git remote
--template REF AI_SBX_TEMPLATE Custom sandbox image
--stock-template off Ignore AI_SBX_TEMPLATE for this repository
--kit PATH none Mixin kit to apply. Repeatable
--direct off Mount the host working tree read-write
--replace off Destroy and recreate an existing sandbox

Environment defaults

Variable Default Overrides
AI_SBX_AGENT claude --agent
AI_SBX_MODE clone --clone / --direct
AI_SBX_TOKEN_DAYS 30 token expiry pre-filled on the form (1–366, or none)
AI_SBX_TEMPLATE unset --template / --stock-template

Carrying your Claude configuration into the sandbox

Sandboxes deliberately ignore your host ~/.claude. The agent runs as a separate agent user with HOME pointing elsewhere, so even a read-only mount of ~/.claude is not picked up. Three mechanisms exist, covering progressively more:

Skills — supported, no build required:

sbx skills import          # add --dry-run to preview

Copies each skill directory from ~/.claude/skills (and ~/.agents/skills, ~/.copilot/skills, ~/.cursor/skills, ~/.factory/skills) into a shared store mounted into every new sandbox. Symlinks and loose top-level files are skipped. Skills under ~/.claude/plugins/ are not scanned — only the top-level skills directory.

Kits — for tools, env vars, network rules, and startup commands:

mise run ai:sbx -- setup --kit ~/kits/my-kit

setup — configuration and plugins, no image required:

For the claude agent, setup copies CLAUDE.md, AGENTS.md, agents, commands and hooks from ~/.claude into the sandbox, runs sbx skills import, then adds every marketplace in ~/.claude/plugins/known_marketplaces.json and installs every plugin your host has enabled. config re-applies all of it without recreating the sandbox.

The copy is an allowlist. Credentials, conversation transcripts, history.jsonl and shell snapshots are never copied, and anything added to ~/.claude later stays on the host until the allowlist names it.

A marketplace on a host other than github.com gets a matching network policy rule, scoped to that sandbox — the default policy denies it otherwise.

Plugin enablement survives here because claude plugin install writes enabledPlugins itself, after the sandbox has been created. Baking plugins into an image does not survive: sbx recreates ~/.claude/settings.json at creation, which drops enablement while leaving the plugin files in place.

Templates — for toolchains and anything else the base image lacks:

export AI_SBX_TEMPLATE=ghcr.io/you/claude-sbx:v1
mise run ai:sbx -- setup       # every repository now uses it

Set AI_SBX_TEMPLATE once in ~/.config/mise/config.toml and every repository picks it up; override per repository with --template, or opt out with --stock-template.

Two constraints worth knowing before building one:

  • The sandbox's Docker daemon pulls templates from a registry and does not share your host image store, so the image must be pushed somewhere reachable. Docker Hub reuses your sbx login; for other registries use sbx secret set --registry.
  • sbx v0.37.0 and v0.37.1 cannot consume custom templates. Every layer stacked on the base image is silently dropped: the sandbox boots with base content only and sbx create still exits 0. This is upstream #366 — the erofs snapshotter stopped building the merged fsmeta. v0.35.0 is unaffected. Until it is fixed, a template is an expensive no-op and setup is the mechanism that works.

Repository toolchains

setup and run install mise in the sandbox and resolve the repository's pinned tools, so the agent runs the versions the project specifies rather than whatever the base image ships.

The mise binary is copied from the host: mise.jdx.dev is outside the default network policy, so the network installer fails at the tarball step. Tools resolve through shims rather than mise activate — the agent runs non-interactive shells, which never fire the activation hook and would otherwise silently get system versions.

A personal mise config that must stay out of the repository goes in the repository's config directory:

$XDG_CONFIG_HOME/ai-sbx/repos/<digest>/mise.local.toml

It is copied to the workspace root on every run. This matters under --clone, where the agent gets a fresh git clone and an untracked mise.local.toml on the host would not reach it. Repositories with no mise configuration are left alone.

AWS profile naming

Only a trailing -readonly is removed. Everything else passes through:

Host profile Sandbox profile
api-portal-readonly api-portal
prod-readonly prod
dev dev
readonly-first readonly-first
team-readonly-readonly team-readonly

Two host profiles that collapse to the same sandbox name — dev-readonly and dev — are rejected at setup, before any credentials are exported. Silently letting one overwrite the other would hand Terraform the wrong identity under a name that looks right.

Variants such as _readonly, -ro, and -read-only are not stripped.

Where state lives

~/.config/ai-sbx/repos/<digest>/config     mode 600, no secrets

Holds repository identity, sandbox name, agent, mode, and the approved host profile names — no secrets. The GitHub token lives in the sbx secret store; AWS credentials exist only inside the sandbox and only until they expire.

Inspect the current repository's state with mise run ai:sbx -- status.

Security notes

  • The repository is not the boundary. A repository-controlled file such as .envrc or mise.toml could otherwise choose which credentials get loaded. Profile approval lives in your user-owned config; the repository only supplies its own identity, which is cross-checked against origin on every run.
  • One token per repository, per developer, with an expiry. Nothing is shared: no private key, no client secret, no broker. Each developer's token is capped by their own access, and organization owners can require approval and enforce a maximum lifetime.
  • The token acts as you. Its commits and comments carry your identity, so treat the agent's output as your own work. Revoke at Fine-grained tokens and re-run token.
  • Rotation is manual. The token expires on the schedule you picked; token replaces it. There is no automatic renewal, because there is no API to create one.
  • Read-only AWS roles. The sandbox boundary limits reach, not intent. Grant roles that cannot cause damage if the agent misbehaves. Terraform plan needs read access; apply should stay outside the sandbox.
  • Contents: write includes force-push and branch deletion. There is no finer split. Branch protection or rulesets are the actual guard, not token scoping.
  • --direct weakens isolation. The agent writes directly to your working tree. Prefer the default --clone.

Development

bash tests/profile-mapping.test.sh
bash tests/token-url.test.sh
bash tests/invocation-directory.test.sh
shellcheck -x tasks/ai/sbx tests/*.sh

The token test checks URL encoding, that target_name carries the owner rather than the full repository name, that every permission survives into the query at a valid level, and that no parameter GitHub silently ignores is sent — an ignored parameter reads as "granted" when reviewing the link. No test touches the network, GitHub, or sbx.

Task names come from directory nesting, not from colons in filenames: tasks/ai/sbx registers as ai:sbx, whereas a file literally named tasks/ai:sbx registers as ai_sbx. Task files must be executable.

S
Description
Mise tasks for setuping an ai sandbox scoped to a repository
Readme
483 KiB
v1.5.0
Latest
2026-07-31 13:19:11 +00:00
Languages
Shell 100%