crafting-table/README.md

crafting-table

Polyglot dev/build/audit container — the build farm for the Sulkta ecosystem.

What this is

A single Docker container with every toolchain we work with, fronted by a FastAPI HTTP API + async job runner. Used as a reliable place to compile / test / audit any Sulkta repo regardless of where the caller is — agents, Claude sessions, ad-hoc curl, scheduled cron.

Eventual surface (v0.1 full): HTTP API + MCP server + project registry + job runner + structured findings + email digest + autonomous patch loop through clawdforge.

Spec: Sulkta-Coop/openclaw-workspace/memory/spec-crafting-table.md (LAN-only).

Status — v0.1

• Step 1: Dockerfile + per-language smoke
• Step 2: SQLite ledger + project registry
• Step 3: HTTP API skeleton (FastAPI, port 8810)
• Step 4: Job runner core (asyncio worker pool, git worktree, subprocess)
• Step 5: Per-language parsers (Rust / Python / Go / TS first)
• Step 6: Findings extraction + storage
• Step 7: MCP server (stdio JSON-RPC, 8 tools)
• Step 8: Email digest scheduler
• Step 9: Autonomous patch loop (clawdforge integration)
• Step 10: Production recipes — clawdforge, cauldron, tradecraft

Toolchains in v0.1

Lang	Versions / extras
Python	3.11 (Debian default) + uv, pipx, pip-audit, ruff, mypy, pytest, semgrep
Node	22.11.0 LTS + npm, pnpm, tsx, eslint, typescript
Bun	latest (rolling)
Go	1.22.10 + govulncheck, staticcheck
Rust	stable (rustup) + clippy, rustfmt, cargo-audit, cargo-deny
Ruby	3.1 (Debian default) + bundler, bundler-audit, rubocop
PHP	8.2 (Debian default) + composer, phpstan, phpunit
JDK	17 (default) + 21 (Temurin, alongside via JAVA_HOME_21)
Maven	3.x (Debian)
Gradle	8.10
.NET	8.0 SDK
Swift	5.9.2 (Ubuntu 22.04 tarball — works on Debian bookworm)
Kotlin	1.9.25 (compiler)
C/C++	clang + lld + cmake + ninja + valgrind
Bash	bash + shellcheck + bats + shfmt
Generic	git, jq, yq, ripgrep, fd, gh-cli, curl, wget

HTTP API surface

LAN-only. Every request needs Authorization: Bearer <token>. The default LAN allowlist is 10/8, 172.16/12, 192.168/16, 127/8, ::1/128; override via CRAFTING_LAN_CIDRS.

Method	Path	Who	What
GET	/healthz	LAN	Liveness + runner stats
POST	/admin/tokens	admin	Mint a new bearer
GET	/admin/tokens	admin	List tokens
DELETE	/admin/tokens/{name}	admin	Revoke a token
POST	/projects	any	Register (becomes owner)
GET	/projects	any	List own (or all if admin)
GET	/projects/{name}	owner	Detail (404 if not visible)
PUT	/projects/{name}	owner	Update
DELETE	/projects/{name}	owner	Remove (cascades jobs+findings)
POST	/projects/{name}/jobs	owner	Enqueue a recipe run
GET	/jobs?project=&status=&limit=	any	List own (or all if admin)
GET	/jobs/{id}	owner	State + last 200 log lines
GET	/jobs/{id}/log	owner	Full log (file stream)
GET	/jobs/{id}/findings	owner	Structured findings (wave 1: empty)

Cross-token access returns 404, not 403 — same existence-leak guard as clawdforge sessions.

Quickstart

After build + first boot, the admin bearer is written to /data/admin-bearer.txt (chmod 600 inside the container — readable from the bind-mounted appdata path on the host).

ADMIN=$(cat /mnt/user/appdata/crafting-table/data/admin-bearer.txt)

# Mint a project-scoped token
TOKEN=$(curl -s http://192.168.0.5:8810/admin/tokens \
  -H "Authorization: Bearer $ADMIN" \
  -H "content-type: application/json" \
  -d '{"name":"clawdforge","is_admin":false,"ip_cidrs":[]}' | jq -r .bearer)

# Register the project
curl -s http://192.168.0.5:8810/projects \
  -H "Authorization: Bearer $TOKEN" \
  -H "content-type: application/json" \
  -d '{
    "name": "clawdforge",
    "git_url": "http://192.168.0.5:3001/Sulkta-Coop/clawdforge.git",
    "default_branch": "main",
    "languages": ["python", "rust"],
    "subprojects": [
      {"path": "clients/python", "language": "python",
       "test": "pytest tests/", "lint": "ruff check .", "audit": "pip-audit",
       "timeout_secs": 600},
      {"path": "clients/rust", "language": "rust",
       "build": "cargo build --release", "test": "cargo test --all",
       "audit": "cargo audit", "timeout_secs": 1800}
    ]
  }'

# Kick off a test job
JOB=$(curl -s http://192.168.0.5:8810/projects/clawdforge/jobs \
  -H "Authorization: Bearer $TOKEN" \
  -H "content-type: application/json" \
  -d '{"recipe":"test","subproject":"clients/python"}' | jq -r .job_id)

# Poll status
watch -n2 "curl -s http://192.168.0.5:8810/jobs/$JOB \
  -H 'Authorization: Bearer $TOKEN' | jq '.job.status, .log_tail[-5:]'"

# Stream the full log
curl http://192.168.0.5:8810/jobs/$JOB/log \
  -H "Authorization: Bearer $TOKEN"

Build + smoke

docker network inspect sulkta >/dev/null 2>&1 || docker network create sulkta
docker compose build

# Run the per-toolchain hello-world smoke
docker compose run --rm crafting-table /usr/local/bin/smoke.sh
# expect: "=== ALL TOOLCHAINS GREEN ==="

# Bring up the API
docker compose up -d
curl http://192.168.0.5:8810/healthz

Image notes

• Base: debian:bookworm-slim. Swift uses the upstream Ubuntu 22.04 tarball which links against bookworm's libicu/libstdc++ baseline.
• Runs as non-root user crafter (uid 1000) with passwordless sudo. The API server runs as crafter. Recipe commands run as crafter too — never elevated to root.
• Volume mount points (production):
  • /data — SQLite ledger, admin bearer file, per-job logs
  • /workspace — bare clones + per-job worktrees
  • /caches — cargo / maven / gradle / npm / pip / bun caches
• Network: external sulkta bridge (same one clawdforge + cauldron use). Create with docker network create sulkta if missing.
• Image size baseline is large (8-15 GB expected). Per spec: that's fine.

Architecture notes

Recipe security

Recipe commands run via /bin/sh -c so any shell metachar works. This is by design — admins set them. Recipes like cargo build && cargo test work as expected; ; rm -rf / would too if an admin set it. The container sandbox + crafter user are the safety net, not the recipe parser.

Workspace strategy

• /workspace/<project>/.cache/ — bare clone of the upstream
• /workspace/<project>/<job_id>/ — git worktree pointing at the requested branch+sha, removed after the job ends
• Periodic gc (1h timer) prunes worktrees older than 24h and runs git gc --prune=now on bare clones quiet for >7d

Recipe immutability

Every job snapshots the project's recipe at run-time (recipe_snapshot_json). Editing a project's recipe doesn't retcon the view of in-flight or already-finished jobs.

Process group + timeout

Recipe subprocesses spawn in their own process group via start_new_session=True. On timeout we os.killpg(pgid, SIGTERM) and grace for 10s before escalating to SIGKILL. Without process-group kill, multi-process recipes (cargo build spawning rustc, etc.) could leave orphans that hold the stdout pipe open.

Restart resilience

Jobs marked running but stranded by a process crash are NOT auto-resumed. On startup the runner sweeps them to failed with exit_code=-1 and appends a synthetic log line [crafting-table] runner restart, job orphaned. Callers can re-enqueue if they want.

SQLite + WAL

Single host, single process — SQLite is plenty. WAL mode (PRAGMA journal_mode=WAL + synchronous=NORMAL) gives many readers + one writer without lock contention. The runner is the only mutator of jobs/ findings; HTTP workers mostly read.

Layout

.
├── Dockerfile           # monolith image: toolchains + Python app
├── compose.yml          # build + run wiring
├── smoke.sh             # per-language hello-world test
├── crafting_table/
│   ├── server.py        # FastAPI app + endpoints
│   ├── db.py            # SQLite + migrations
│   ├── auth.py          # bearer + IP allowlist
│   ├── runner.py        # async job pool + subprocess exec
│   ├── workspace.py     # bare clone + worktree materialization + gc
│   ├── models.py        # Pydantic schemas
│   └── config.py        # env-driven config
├── tests/               # pytest suite (~60 tests)
├── pyproject.toml
├── requirements.txt
└── .env.example

Tests

python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[test]'
pytest tests/

Digest

Daily 06:00 PT email digest. One message per project per day; aggregates the last 24h of jobs per recipient and sends via SMTP relay (Lucy postfix).

Set the SMTP block in .env to enable — leaving CRAFTING_SMTP_HOST unset keeps the scheduler off and logs digest disabled — CRAFTING_SMTP_HOST not set at startup. The /digests and /admin/digest/run-now endpoints still work in dry-run mode regardless.

CRAFTING_SMTP_HOST=postfix.sulkta.com
CRAFTING_SMTP_PORT=587
CRAFTING_SMTP_USER=crafting-table@sulkta.com
CRAFTING_SMTP_PASS=...
CRAFTING_SMTP_FROM=crafting-table@sulkta.com
CRAFTING_SMTP_TLS=1

Each project's notify.email + notify.on fields control delivery:

notify.on event	When it fires
audit_pass	a passing audit job
audit_fail	a failing audit job
test_fail	a failing test job
lint_warn	a lint job with warning-severity findings
cve_found	any job whose findings include a cve
patch_drafted	(wave 3 / step 9) auto-patch was drafted
nightly_summary	catch-all — show ALL jobs in the project's section

Empty notify.on defaults to ["audit_fail", "cve_found", "patch_drafted"]. Empty notify.email silently excludes the project.

Manual trigger from the LAN admin token:

# Render today's digest without sending
curl -sH "Authorization: Bearer $ADMIN" \
  -X POST http://192.168.0.5:8810/admin/digest/run-now \
  -d '{"dry_run": true}' | jq .

# Render an arbitrary date as JSON
curl -sH "Authorization: Bearer $ADMIN" \
  http://192.168.0.5:8810/digests/2026-04-29 | jq .

Idempotency: digest_runs table holds UNIQUE(date, project_name), so the 06:00 loop is safe to re-fire on the same day — only the first call sends.

License

MIT