Public-flip audit: generalize internal hosts/paths + drop Sulkta-internal refs
URLs, mount paths, and LAN host bindings parameterized via env or relative paths
so the repo stands up from a clean clone anywhere. Drop cross-codebase refs
("mirrors clawdforge's pattern"), Sulkta-Coop client/merchant test fixtures,
and audit-changelog scaffolding from comments. README terser, technical content
preserved.
This commit is contained in:
parent
8b1774130b
commit
b335405c02
23 changed files with 238 additions and 266 deletions
16
.env.example
16
.env.example
|
|
@ -1,4 +1,4 @@
|
|||
# crafting-table runtime config — every key is optional; defaults shown.
|
||||
# crafting-table runtime config. Every key is optional; defaults shown.
|
||||
#
|
||||
# Copy to `.env` and edit, or pass each key explicitly to `docker compose`.
|
||||
|
||||
|
|
@ -26,7 +26,7 @@ CRAFTING_DEFAULT_JOB_TIMEOUT=1800
|
|||
|
||||
# Override the default LAN allowlist if you want stricter scoping.
|
||||
# Default: 10/8, 172.16/12, 192.168/16, 127/8, ::1/128
|
||||
# CRAFTING_LAN_CIDRS=192.168.0.0/16,127.0.0.0/8
|
||||
# CRAFTING_LAN_CIDRS=10.0.0.0/8,127.0.0.0/8
|
||||
|
||||
# Workspace gc — how often to sweep for stale worktrees, and the age cutoff.
|
||||
CRAFTING_GC_INTERVAL=3600
|
||||
|
|
@ -36,20 +36,20 @@ CRAFTING_GC_AGE=86400
|
|||
# If CRAFTING_SMTP_HOST is empty, the digest scheduler stays disabled and
|
||||
# the server logs a "digest disabled" warning at startup. Setting it on
|
||||
# enables the daily 06:00 PT loop.
|
||||
# CRAFTING_SMTP_HOST=postfix.sulkta.com
|
||||
# CRAFTING_SMTP_HOST=smtp.example.com
|
||||
# CRAFTING_SMTP_PORT=587
|
||||
# CRAFTING_SMTP_USER=crafting-table@sulkta.com
|
||||
# CRAFTING_SMTP_USER=crafting-table@example.com
|
||||
# CRAFTING_SMTP_PASS=
|
||||
# CRAFTING_SMTP_FROM=crafting-table@sulkta.com
|
||||
# CRAFTING_SMTP_FROM=crafting-table@example.com
|
||||
# CRAFTING_SMTP_TLS=1
|
||||
|
||||
# --- Autonomous patch loop (wave 3, optional) ------------------------------
|
||||
# --- Autonomous patch loop (optional) --------------------------------------
|
||||
# All four CRAFTING_CLAWDFORGE_* + CRAFTING_GITEA_* must be set for the
|
||||
# patcher to come up. Missing any → patcher disabled, /jobs/{id}/patches
|
||||
# returns 503. Runner hook silently no-ops.
|
||||
# CRAFTING_CLAWDFORGE_URL=http://192.168.0.5:8800
|
||||
# CRAFTING_CLAWDFORGE_URL=http://clawdforge.internal:8800
|
||||
# CRAFTING_CLAWDFORGE_TOKEN=cf_...
|
||||
# CRAFTING_GITEA_URL=http://192.168.0.5:3001
|
||||
# CRAFTING_GITEA_URL=http://gitea.internal:3000
|
||||
# CRAFTING_GITEA_TOKEN=
|
||||
# CRAFTING_PATCHER_MAX_ATTEMPTS=3
|
||||
# CRAFTING_PATCHER_BRANCH_PREFIX=crafting-table/auto/
|
||||
|
|
|
|||
256
README.md
256
README.md
|
|
@ -1,42 +1,27 @@
|
|||
# crafting-table
|
||||
|
||||
Polyglot dev/build/audit container — the build farm for the Sulkta ecosystem.
|
||||
Polyglot dev/build/audit container. HTTP API + async job runner + autonomous
|
||||
patch loop + email digest.
|
||||
|
||||
## What this is
|
||||
A single Docker image carries every toolchain in the matrix below. You hand
|
||||
it a git URL + a per-language recipe (build / test / lint / audit), it
|
||||
materializes a worktree, runs the recipe, parses the tool output into
|
||||
structured findings, and stores everything in a SQLite ledger. Optional
|
||||
patch loop drafts a unified diff via an external Claude-style agent, verifies
|
||||
it by re-running the recipe, and opens a PR.
|
||||
|
||||
A single Docker container with every toolchain we work with, fronted by a
|
||||
FastAPI HTTP API + async job runner. Used as a reliable place to compile /
|
||||
test / audit any Sulkta repo regardless of where the caller is — agents,
|
||||
Claude sessions, ad-hoc curl, scheduled cron.
|
||||
Use it as a replacement for ad-hoc per-repo build environments. One image,
|
||||
one runner, every language.
|
||||
|
||||
Eventual surface (v0.1 full): HTTP API + MCP server + project registry +
|
||||
job runner + structured findings + email digest + autonomous patch loop
|
||||
through clawdforge.
|
||||
|
||||
Spec: `Sulkta-Coop/openclaw-workspace/memory/spec-crafting-table.md` (LAN-only).
|
||||
|
||||
## Status — v0.1 complete (10 of 10)
|
||||
|
||||
- [x] Step 1: Dockerfile + per-language smoke
|
||||
- [x] Step 2: SQLite ledger + project registry
|
||||
- [x] Step 3: HTTP API skeleton (FastAPI, port 8810)
|
||||
- [x] Step 4: Job runner core (asyncio worker pool, git worktree, subprocess)
|
||||
- [x] Step 5: Per-language parsers (Rust / Python / Go / TS first)
|
||||
- [x] Step 6: Findings extraction + storage
|
||||
- [x] Step 7: MCP server (stdio JSON-RPC, 8 tools) — see [mcp/README.md](mcp/README.md)
|
||||
- [x] Step 8: Email digest scheduler
|
||||
- [x] Step 9: Autonomous patch loop (clawdforge integration → unified diff → worktree apply → verify recipe → push branch → Gitea PR)
|
||||
- [x] Step 10: Production recipes — clawdforge, cauldron, tradecraft (see [examples/recipes/](examples/recipes/))
|
||||
|
||||
## Toolchains in v0.1
|
||||
## Toolchains
|
||||
|
||||
| Lang | Versions / extras |
|
||||
|----------|--------------------------------------------------------------------|
|
||||
| Python | 3.11 (Debian default) + uv, pipx, pip-audit, ruff, mypy, pytest, semgrep |
|
||||
| Node | 22.11.0 LTS + npm, pnpm, tsx, eslint, typescript |
|
||||
| Bun | latest (rolling) |
|
||||
| Go | 1.22.10 + govulncheck, staticcheck |
|
||||
| Rust | stable (rustup) + clippy, rustfmt, cargo-audit, cargo-deny |
|
||||
| Go | 1.25.9 + govulncheck, staticcheck |
|
||||
| Rust | stable (rustup) + clippy, rustfmt |
|
||||
| Ruby | 3.1 (Debian default) + bundler, bundler-audit, rubocop |
|
||||
| PHP | 8.2 (Debian default) + composer, phpstan, phpunit |
|
||||
| JDK | 17 (default) + 21 (Temurin, alongside via `JAVA_HOME_21`) |
|
||||
|
|
@ -47,13 +32,18 @@ Spec: `Sulkta-Coop/openclaw-workspace/memory/spec-crafting-table.md` (LAN-only).
|
|||
| Kotlin | 1.9.25 (compiler) |
|
||||
| C/C++ | clang + lld + cmake + ninja + valgrind |
|
||||
| Bash | bash + shellcheck + bats + shfmt |
|
||||
| Nix | single-user install with cache.nixos.org + cache.iog.io substituters |
|
||||
| Generic | git, jq, yq, ripgrep, fd, gh-cli, curl, wget |
|
||||
|
||||
## HTTP API surface
|
||||
`cargo-audit` and `cargo-deny` are NOT baked into the image — both flaked
|
||||
during build (libgit2-sys C bindings + GitHub release download flakiness).
|
||||
Install at runtime with `cargo install cargo-audit cargo-deny` if you need them.
|
||||
|
||||
LAN-only. Every request needs `Authorization: Bearer <token>`. The default
|
||||
LAN allowlist is `10/8`, `172.16/12`, `192.168/16`, `127/8`, `::1/128`;
|
||||
override via `CRAFTING_LAN_CIDRS`.
|
||||
## HTTP API
|
||||
|
||||
LAN-only. Every request needs `Authorization: Bearer <token>`. Default IP
|
||||
allowlist is `10/8`, `172.16/12`, `192.168/16`, `127/8`, `::1/128`; override
|
||||
via `CRAFTING_LAN_CIDRS`.
|
||||
|
||||
| Method | Path | Who | What |
|
||||
|--------|-----------------------------------|--------|--------------------------------|
|
||||
|
|
@ -70,36 +60,36 @@ override via `CRAFTING_LAN_CIDRS`.
|
|||
| GET | `/jobs?project=&status=&limit=` | any | List own (or all if admin) |
|
||||
| GET | `/jobs/{id}` | owner | State + last 200 log lines |
|
||||
| GET | `/jobs/{id}/log` | owner | Full log (file stream) |
|
||||
| GET | `/jobs/{id}/findings` | owner | Structured findings (see Findings) |
|
||||
| POST | `/jobs/{id}/patches` | owner | Trigger an auto-patch attempt (wave 3) |
|
||||
| GET | `/jobs/{id}/findings` | owner | Structured findings |
|
||||
| POST | `/jobs/{id}/patches` | owner | Trigger an auto-patch attempt |
|
||||
| GET | `/patches?project=&status=&limit=`| any | List own patch attempts |
|
||||
| GET | `/patches/{id}` | owner | Patch attempt detail |
|
||||
|
||||
Cross-token access returns **404, not 403** — same existence-leak guard as
|
||||
clawdforge sessions.
|
||||
Cross-token access returns **404, not 403** — existence-leak guard.
|
||||
|
||||
## Quickstart
|
||||
|
||||
After build + first boot, the admin bearer is written to
|
||||
`/data/admin-bearer.txt` (chmod 600 inside the container — readable from
|
||||
the bind-mounted appdata path on the host).
|
||||
the bind-mounted host path).
|
||||
|
||||
```bash
|
||||
ADMIN=$(cat /mnt/user/appdata/crafting-table/data/admin-bearer.txt)
|
||||
HOST=http://localhost:8810
|
||||
ADMIN=$(cat ./data/admin-bearer.txt)
|
||||
|
||||
# Mint a project-scoped token
|
||||
TOKEN=$(curl -s http://192.168.0.5:8810/admin/tokens \
|
||||
TOKEN=$(curl -s "$HOST/admin/tokens" \
|
||||
-H "Authorization: Bearer $ADMIN" \
|
||||
-H "content-type: application/json" \
|
||||
-d '{"name":"clawdforge","is_admin":false,"ip_cidrs":[]}' | jq -r .bearer)
|
||||
-d '{"name":"alpha","is_admin":false,"ip_cidrs":[]}' | jq -r .bearer)
|
||||
|
||||
# Register the project
|
||||
curl -s http://192.168.0.5:8810/projects \
|
||||
# Register a project
|
||||
curl -s "$HOST/projects" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "content-type: application/json" \
|
||||
-d '{
|
||||
"name": "clawdforge",
|
||||
"git_url": "http://192.168.0.5:3001/Sulkta-Coop/clawdforge.git",
|
||||
"name": "alpha",
|
||||
"git_url": "http://git.example.com/org/alpha.git",
|
||||
"default_branch": "main",
|
||||
"languages": ["python", "rust"],
|
||||
"subprojects": [
|
||||
|
|
@ -108,38 +98,35 @@ curl -s http://192.168.0.5:8810/projects \
|
|||
"timeout_secs": 600},
|
||||
{"path": "clients/rust", "language": "rust",
|
||||
"build": "cargo build --release", "test": "cargo test --all",
|
||||
"audit": "cargo audit", "timeout_secs": 1800}
|
||||
"timeout_secs": 1800}
|
||||
]
|
||||
}'
|
||||
|
||||
# Kick off a test job
|
||||
JOB=$(curl -s http://192.168.0.5:8810/projects/clawdforge/jobs \
|
||||
JOB=$(curl -s "$HOST/projects/alpha/jobs" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "content-type: application/json" \
|
||||
-d '{"recipe":"test","subproject":"clients/python"}' | jq -r .job_id)
|
||||
|
||||
# Poll status
|
||||
watch -n2 "curl -s http://192.168.0.5:8810/jobs/$JOB \
|
||||
-H 'Authorization: Bearer $TOKEN' | jq '.job.status, .log_tail[-5:]'"
|
||||
curl -s "$HOST/jobs/$JOB" -H "Authorization: Bearer $TOKEN" | jq '.job.status, .log_tail[-5:]'
|
||||
|
||||
# Stream the full log
|
||||
curl http://192.168.0.5:8810/jobs/$JOB/log \
|
||||
-H "Authorization: Bearer $TOKEN"
|
||||
curl "$HOST/jobs/$JOB/log" -H "Authorization: Bearer $TOKEN"
|
||||
```
|
||||
|
||||
## Build + smoke
|
||||
|
||||
```bash
|
||||
docker network inspect sulkta >/dev/null 2>&1 || docker network create sulkta
|
||||
docker compose build
|
||||
|
||||
# Run the per-toolchain hello-world smoke
|
||||
# Per-toolchain hello-world
|
||||
docker compose run --rm crafting-table /usr/local/bin/smoke.sh
|
||||
# expect: "=== ALL TOOLCHAINS GREEN ==="
|
||||
|
||||
# Bring up the API
|
||||
docker compose up -d
|
||||
curl http://192.168.0.5:8810/healthz
|
||||
curl http://localhost:8810/healthz
|
||||
```
|
||||
|
||||
## Image notes
|
||||
|
|
@ -149,23 +136,22 @@ curl http://192.168.0.5:8810/healthz
|
|||
- Runs as non-root user `crafter` (uid 1000) with passwordless sudo. The
|
||||
API server runs as `crafter`. Recipe commands run as `crafter` too —
|
||||
never elevated to root.
|
||||
- Volume mount points (production):
|
||||
- Volume mount points:
|
||||
- `/data` — SQLite ledger, admin bearer file, per-job logs
|
||||
- `/workspace` — bare clones + per-job worktrees
|
||||
- `/caches` — cargo / maven / gradle / npm / pip / bun caches
|
||||
- Network: external `sulkta` bridge (same one clawdforge + cauldron use).
|
||||
Create with `docker network create sulkta` if missing.
|
||||
- Image size baseline is large (8-15 GB expected). Per spec: that's fine.
|
||||
- `/nix` — pre-seeded single-user Nix store (Plutarch / IOG flakes)
|
||||
- Image size baseline is large (8-15 GB). By design.
|
||||
|
||||
## Architecture notes
|
||||
## Architecture
|
||||
|
||||
### Recipe security
|
||||
Recipe commands run via `/bin/sh -c` so any shell metachar works. This is
|
||||
**by design** — admins set them. Recipes like `cargo build && cargo test`
|
||||
work as expected; `; rm -rf /` would too if an admin set it. The container
|
||||
Recipe commands run via `/bin/sh -c` so any shell metachar works. **By
|
||||
design** — admins set them. Recipes like `cargo build && cargo test` work
|
||||
as expected; `; rm -rf /` would too if an admin set it. The container
|
||||
sandbox + `crafter` user are the safety net, not the recipe parser.
|
||||
|
||||
### Workspace strategy
|
||||
### Workspace
|
||||
- `/workspace/<project>/.cache/` — bare clone of the upstream
|
||||
- `/workspace/<project>/<job_id>/` — git worktree pointing at the requested
|
||||
branch+sha, removed after the job ends
|
||||
|
|
@ -175,26 +161,26 @@ sandbox + `crafter` user are the safety net, not the recipe parser.
|
|||
### Recipe immutability
|
||||
Every job snapshots the project's recipe at run-time
|
||||
(`recipe_snapshot_json`). Editing a project's recipe doesn't retcon the
|
||||
view of in-flight or already-finished jobs.
|
||||
view of in-flight or finished jobs.
|
||||
|
||||
### Process group + timeout
|
||||
Recipe subprocesses spawn in their own process group via
|
||||
`start_new_session=True`. On timeout we `os.killpg(pgid, SIGTERM)` and grace
|
||||
for 10s before escalating to `SIGKILL`. Without process-group kill,
|
||||
multi-process recipes (cargo build spawning rustc, etc.) could leave
|
||||
orphans that hold the stdout pipe open.
|
||||
multi-process recipes (cargo build spawning rustc, etc.) leave orphans that
|
||||
hold the stdout pipe open.
|
||||
|
||||
### Restart resilience
|
||||
Jobs marked `running` but stranded by a process crash are NOT auto-resumed.
|
||||
On startup the runner sweeps them to `failed` with `exit_code=-1` and
|
||||
appends a synthetic log line `[crafting-table] runner restart, job
|
||||
orphaned`. Callers can re-enqueue if they want.
|
||||
On startup the runner sweeps them to `failed` with `exit_code=-1` and a
|
||||
synthetic log line `[crafting-table] runner restart, job orphaned`. Callers
|
||||
can re-enqueue if they want.
|
||||
|
||||
### SQLite + WAL
|
||||
Single host, single process — SQLite is plenty. WAL mode (`PRAGMA
|
||||
journal_mode=WAL` + `synchronous=NORMAL`) gives many readers + one writer
|
||||
without lock contention. The runner is the only mutator of `jobs`/
|
||||
`findings`; HTTP workers mostly read.
|
||||
Single host, single process. WAL mode (`PRAGMA journal_mode=WAL` +
|
||||
`synchronous=NORMAL`) gives many readers + one writer without lock
|
||||
contention. The runner is the only mutator of `jobs`/`findings`; HTTP
|
||||
workers mostly read.
|
||||
|
||||
## Layout
|
||||
|
||||
|
|
@ -211,13 +197,13 @@ without lock contention. The runner is the only mutator of `jobs`/
|
|||
│ ├── workspace.py # bare clone + worktree materialization + gc
|
||||
│ ├── models.py # Pydantic schemas
|
||||
│ ├── digest.py # email digest scheduler
|
||||
│ ├── patcher.py # autonomous patch loop (clawdforge → diff → verify → PR)
|
||||
│ ├── patcher.py # autonomous patch loop
|
||||
│ ├── parsers/ # per-language Finding extractors
|
||||
│ └── config.py # env-driven config
|
||||
├── tests/ # pytest suite (143 tests)
|
||||
├── tests/ # pytest suite
|
||||
├── mcp/ # crafting-table-mcp — MCP stdio bridge (separate pip install)
|
||||
├── examples/
|
||||
│ ├── recipes/ # production recipes — clawdforge, cauldron, tradecraft
|
||||
│ ├── recipes/ # example recipe JSONs
|
||||
│ └── register-all.sh # bulk-register helper
|
||||
├── pyproject.toml
|
||||
├── requirements.txt
|
||||
|
|
@ -234,12 +220,12 @@ pytest tests/
|
|||
|
||||
## Findings
|
||||
|
||||
After every job, the runner reads the captured log and hands it to a
|
||||
After every job the runner reads the captured log and hands it to a
|
||||
per-language parser. The parser turns native tool output (clippy JSON,
|
||||
ruff JSON, govulncheck NDJSON, eslint JSON, tsc human errors, etc.) into
|
||||
structured rows in the `findings` table.
|
||||
|
||||
### Parsers in v0.1
|
||||
### Parsers
|
||||
|
||||
| Language | Recipes parsed | Tool output expected |
|
||||
|-------------|------------------------------|------------------------------------------------------|
|
||||
|
|
@ -250,10 +236,9 @@ structured rows in the `findings` table.
|
|||
| javascript | (alias of typescript) | same |
|
||||
| _other_ | (any) | falls back to `GenericParser` — emits one `recipe_fail` row when exit_code != 0, else nothing |
|
||||
|
||||
Resolution order in the registry: exact `language` match (parsers
|
||||
self-gate on recipe via `Parser.matches`), then `GenericParser`. Adding a
|
||||
new language is a single file in `crafting_table/parsers/` plus an entry
|
||||
in `PARSERS`.
|
||||
Resolution order: exact `language` match (parsers self-gate on recipe via
|
||||
`Parser.matches`), then `GenericParser`. Adding a new language is a single
|
||||
file in `crafting_table/parsers/` plus an entry in `PARSERS`.
|
||||
|
||||
### Finding kinds
|
||||
|
||||
|
|
@ -262,8 +247,7 @@ in `PARSERS`.
|
|||
`code` = advisory id (RUSTSEC-..., GO-..., PYSEC-...) and a
|
||||
`suggested_fix` of the form "bump <pkg> to <version>" when patched
|
||||
versions are known.
|
||||
- `test_fail` — failed test name extracted from `cargo test` /
|
||||
`pytest` output.
|
||||
- `test_fail` — failed test name extracted from `cargo test` / `pytest` output.
|
||||
- `recipe_fail` — fallback when no language-specific parser fired and the
|
||||
recipe exited non-zero. `code` = `exit_<n>`, message names the recipe.
|
||||
|
||||
|
|
@ -271,8 +255,8 @@ in `PARSERS`.
|
|||
|
||||
Every finding row carries a 16-char `fingerprint` hash over
|
||||
`kind|file|line|code` (NOT the message — tool wording drifts). The same
|
||||
lint reappearing across nightly runs produces the same fingerprint, so a
|
||||
later wave can dedup digest output and surface only "new since last run."
|
||||
lint reappearing across nightly runs produces the same fingerprint, so
|
||||
later passes can dedup digest output and surface only "new since last run."
|
||||
|
||||
### Consuming findings
|
||||
|
||||
|
|
@ -287,14 +271,14 @@ GET /jobs/{job_id}/findings
|
|||
]}
|
||||
```
|
||||
|
||||
Authorization is project-token-scoped (same model as `/jobs/{id}`). The
|
||||
matching `job` row's `findings_count` mirrors the array length so
|
||||
callers can decide whether to fetch the detail.
|
||||
Authorization is project-token-scoped (same as `/jobs/{id}`). The matching
|
||||
`job` row's `findings_count` mirrors the array length so callers can decide
|
||||
whether to fetch the detail.
|
||||
|
||||
## Digest
|
||||
|
||||
Daily 06:00 PT email digest. One message per project per day; aggregates the
|
||||
last 24h of jobs per recipient and sends via SMTP relay (Lucy postfix).
|
||||
last 24h of jobs per recipient and sends via SMTP.
|
||||
|
||||
Set the SMTP block in `.env` to enable — leaving `CRAFTING_SMTP_HOST` unset
|
||||
keeps the scheduler off and logs `digest disabled — CRAFTING_SMTP_HOST not set`
|
||||
|
|
@ -302,11 +286,11 @@ at startup. The `/digests` and `/admin/digest/run-now` endpoints still work
|
|||
in dry-run mode regardless.
|
||||
|
||||
```bash
|
||||
CRAFTING_SMTP_HOST=postfix.sulkta.com
|
||||
CRAFTING_SMTP_HOST=smtp.example.com
|
||||
CRAFTING_SMTP_PORT=587
|
||||
CRAFTING_SMTP_USER=crafting-table@sulkta.com
|
||||
CRAFTING_SMTP_USER=crafting-table@example.com
|
||||
CRAFTING_SMTP_PASS=...
|
||||
CRAFTING_SMTP_FROM=crafting-table@sulkta.com
|
||||
CRAFTING_SMTP_FROM=crafting-table@example.com
|
||||
CRAFTING_SMTP_TLS=1
|
||||
```
|
||||
|
||||
|
|
@ -319,7 +303,7 @@ Each project's `notify.email` + `notify.on` fields control delivery:
|
|||
| `test_fail` | a failing test job |
|
||||
| `lint_warn` | a lint job with warning-severity findings |
|
||||
| `cve_found` | any job whose findings include a `cve` |
|
||||
| `patch_drafted` | (wave 3 / step 9) auto-patch was drafted |
|
||||
| `patch_drafted` | an auto-patch was drafted |
|
||||
| `nightly_summary` | catch-all — show ALL jobs in the project's section |
|
||||
|
||||
Empty `notify.on` defaults to `["audit_fail", "cve_found", "patch_drafted"]`.
|
||||
|
|
@ -330,12 +314,12 @@ Manual trigger from the LAN admin token:
|
|||
```bash
|
||||
# Render today's digest without sending
|
||||
curl -sH "Authorization: Bearer $ADMIN" \
|
||||
-X POST http://192.168.0.5:8810/admin/digest/run-now \
|
||||
-X POST http://localhost:8810/admin/digest/run-now \
|
||||
-d '{"dry_run": true}' | jq .
|
||||
|
||||
# Render an arbitrary date as JSON
|
||||
curl -sH "Authorization: Bearer $ADMIN" \
|
||||
http://192.168.0.5:8810/digests/2026-04-29 | jq .
|
||||
http://localhost:8810/digests/2026-04-29 | jq .
|
||||
```
|
||||
|
||||
Idempotency: `digest_runs` table holds `UNIQUE(date, project_name)`, so the
|
||||
|
|
@ -343,37 +327,37 @@ Idempotency: `digest_runs` table holds `UNIQUE(date, project_name)`, so the
|
|||
|
||||
## Autonomous patch loop
|
||||
|
||||
Wave 3 wires crafting-table into clawdforge so a project with
|
||||
`notify.auto_patch=true` gets an automatic patch attempt on every
|
||||
actionable finding (lint with file/line; cve with a known fix). Lifecycle:
|
||||
Wires crafting-table into an external Claude-agent host (default: clawdforge)
|
||||
so a project with `notify.auto_patch=true` gets an automatic patch attempt
|
||||
on every actionable finding (lint with file/line; cve with a known fix).
|
||||
Lifecycle:
|
||||
|
||||
1. Runner finishes a job + parsers populate findings.
|
||||
2. Post-job hook fires: pulls the highest-severity actionable finding,
|
||||
reads ±20 lines of context from the worktree.
|
||||
3. Patcher opens a clawdforge session (`POST /sessions`), sends one
|
||||
turn with the finding + source context + project metadata, expects
|
||||
3. Patcher opens an agent session, sends one turn with the finding + source
|
||||
context + project metadata, expects
|
||||
`{"diff": ..., "explanation": ..., "confidence": ...}` back.
|
||||
4. Diff applied to a fresh worktree on `crafting-table/auto/<job_id>-<finding_id>`.
|
||||
Apply failure → status `apply_failed`.
|
||||
5. Recipe re-runs against the patched worktree (the **verify** step).
|
||||
Fail → `verify_failed`.
|
||||
6. Pass → commit + push + open Gitea PR. Status `pr_opened`.
|
||||
7. clawdforge session always closed.
|
||||
7. Agent session always closed.
|
||||
|
||||
Configuration (env vars):
|
||||
|
||||
```
|
||||
CRAFTING_CLAWDFORGE_URL=http://192.168.0.5:8800
|
||||
CRAFTING_CLAWDFORGE_URL=http://clawdforge.internal:8800
|
||||
CRAFTING_CLAWDFORGE_TOKEN=cf_...
|
||||
CRAFTING_GITEA_URL=http://192.168.0.5:3001
|
||||
CRAFTING_GITEA_URL=http://gitea.internal:3000
|
||||
CRAFTING_GITEA_TOKEN=<gitea PAT>
|
||||
CRAFTING_PATCHER_MAX_ATTEMPTS=3
|
||||
CRAFTING_PATCHER_BRANCH_PREFIX=crafting-table/auto/
|
||||
```
|
||||
|
||||
If any of the four required vars is missing, the patcher stays disabled
|
||||
and `POST /jobs/{id}/patches` returns 503. The runner hook silently no-ops
|
||||
in that case so existing job flow is unaffected.
|
||||
and `POST /jobs/{id}/patches` returns 503. The runner hook silently no-ops.
|
||||
|
||||
**Verification cost matters.** The verify step re-runs the failing recipe
|
||||
on the patched worktree — for projects with multi-minute builds this
|
||||
|
|
@ -389,69 +373,53 @@ Manual trigger:
|
|||
|
||||
```bash
|
||||
curl -sH "Authorization: Bearer $TOKEN" \
|
||||
-X POST http://192.168.0.5:8810/jobs/$JOB/patches \
|
||||
-X POST http://localhost:8810/jobs/$JOB/patches \
|
||||
-d '{"finding_id": 42}' | jq .
|
||||
# → {"ok": true, "attempt": {"status": "pr_opened", "pr_url": "...", ...}}
|
||||
```
|
||||
|
||||
## First production recipes
|
||||
## Example recipes
|
||||
|
||||
Three recipes ship in `examples/recipes/`:
|
||||
Three example recipes ship in `examples/recipes/`:
|
||||
|
||||
| Recipe | Subprojects | Schedule (audit) | auto_patch |
|
||||
|---------------|-------------|------------------|------------|
|
||||
| `clawdforge` | 14 (one per SDK + root) | nightly 02:00 | **true** |
|
||||
| `cauldron` | 1 (Flask app, `.`) | nightly 02:00 | **true** |
|
||||
| `tradecraft` | 1 (`.`) | nightly 02:00 | **false** (manual review) |
|
||||
| Recipe | Subprojects | auto_patch |
|
||||
|---------------|-------------|------------|
|
||||
| `alpha` | 14 (one per SDK + root) | **true** |
|
||||
| `beta` | 1 (Flask app, `.`) | **true** |
|
||||
| `gamma` | 1 (`.`) | **false** (manual review) |
|
||||
|
||||
Each ships with a placeholder `REPLACE_WITH_GITEA_TOKEN` in `git_url`;
|
||||
`examples/register-all.sh` substitutes `$GITEA_TOKEN` at register time so
|
||||
no real token ever lands in the repo.
|
||||
Each ships with a placeholder `REPLACE_WITH_GITEA_TOKEN` in `git_url` and
|
||||
`REPLACE_WITH_GIT_HOST` for the host. `examples/register-all.sh` substitutes
|
||||
`$GITEA_TOKEN` + `$GIT_HOST` at register time so no real token or host ever
|
||||
lands in the repo.
|
||||
|
||||
Smoke procedure (post-deploy):
|
||||
|
||||
```
|
||||
1. docker compose up -d
|
||||
2. TOKEN=$(cat /mnt/user/appdata/crafting-table/data/admin-bearer.txt)
|
||||
3. CRAFTING_TABLE_TOKEN=$TOKEN GITEA_TOKEN=<your-pat> bash examples/register-all.sh
|
||||
4. curl -H "Authorization: Bearer $TOKEN" http://192.168.0.5:8810/projects \
|
||||
→ expect 3 projects (clawdforge, cauldron, tradecraft)
|
||||
2. TOKEN=$(cat ./data/admin-bearer.txt)
|
||||
3. CRAFTING_TABLE_TOKEN=$TOKEN GITEA_TOKEN=<your-pat> \
|
||||
GIT_HOST=git.example.com bash examples/register-all.sh
|
||||
4. curl -H "Authorization: Bearer $TOKEN" http://localhost:8810/projects
|
||||
5. curl -X POST -H "Authorization: Bearer $TOKEN" \
|
||||
http://192.168.0.5:8810/projects/clawdforge/jobs \
|
||||
http://localhost:8810/projects/alpha/jobs \
|
||||
-d '{"recipe":"test","subproject":"clients/python"}'
|
||||
→ expect job_id
|
||||
6. Poll GET /jobs/{job_id} until status terminal → expect succeeded
|
||||
6. Poll GET /jobs/{job_id} until status terminal
|
||||
```
|
||||
|
||||
Per-recipe smoke status (today, pre-deploy):
|
||||
|
||||
- `clawdforge` — 14 subprojects; `clients/python` & `clients/typescript`
|
||||
& `clients/go` & `clients/rust` known clean from existing CI; ruby /
|
||||
php / kotlin / java / csharp / swift compile-cleanly today but
|
||||
toolchain availability inside the crafting-table image is what step 1
|
||||
smoke verified. Bash subproject's `test/run.sh` may not exist (manual
|
||||
check needed post-deploy).
|
||||
- `cauldron` — single Flask subproject; pip-audit & pytest known to run
|
||||
cleanly from the cauldron repo's own CI history.
|
||||
- `tradecraft` — single subproject; auto_patch is **off** by design
|
||||
(production app, manual PR review only).
|
||||
|
||||
## MCP bridge
|
||||
|
||||
The `mcp/` subdirectory ships a self-contained `crafting-table-mcp` Python
|
||||
package that exposes the HTTP API to MCP-aware clients (Claude Desktop, Claude
|
||||
Code, Cursor, Zed, custom agents). See [mcp/README.md](mcp/README.md) for the
|
||||
tool surface, installation, and configuration.
|
||||
|
||||
Quickstart:
|
||||
`mcp/` ships a self-contained `crafting-table-mcp` Python package that
|
||||
exposes the HTTP API to MCP-aware clients (Claude Desktop, Claude Code,
|
||||
Cursor, Zed, etc.). See [mcp/README.md](mcp/README.md).
|
||||
|
||||
```bash
|
||||
pip install -e mcp
|
||||
export CRAFTING_TABLE_BASE_URL=http://192.168.0.5:8810
|
||||
export CRAFTING_TABLE_BASE_URL=http://localhost:8810
|
||||
export CRAFTING_TABLE_TOKEN=ct_...
|
||||
crafting-table-mcp # stdio JSON-RPC server
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
MIT — see [LICENSE](LICENSE).
|
||||
|
|
|
|||
67
compose.yml
67
compose.yml
|
|
@ -1,12 +1,13 @@
|
|||
# crafting-table v0.1 — wave 1 compose (steps 2+3+4 wired in).
|
||||
# crafting-table compose.
|
||||
#
|
||||
# Default `command` is the API server. To run the per-language smoke after
|
||||
# a rebuild, do:
|
||||
# Default command is the API server. To run the per-toolchain smoke after
|
||||
# a rebuild:
|
||||
# docker compose run --rm crafting-table /usr/local/bin/smoke.sh
|
||||
#
|
||||
# Volumes mount real Lucy appdata paths so /data + /workspace + /caches
|
||||
# survive container recreation. Port is bound to LAN only — no Rackham
|
||||
# proxy.
|
||||
# Volumes are relative paths under the compose project dir by default; point
|
||||
# them at host appdata via env vars if you want persistence across container
|
||||
# recreation. Port binds to all interfaces by default — set CRAFTING_BIND_IP
|
||||
# to scope it to a single host IP on a multi-homed box.
|
||||
name: crafting-table
|
||||
|
||||
services:
|
||||
|
|
@ -23,44 +24,42 @@ services:
|
|||
- "8810"
|
||||
user: crafter
|
||||
working_dir: /home/crafter
|
||||
# env_file is optional; copy .env.example to .env to override defaults.
|
||||
# env_file is optional. Copy .env.example to .env to override defaults.
|
||||
# CRAFTING_SECRETS_ENV can point at a host-side secrets file outside the
|
||||
# repo (vault-managed) — leave unset to skip.
|
||||
env_file:
|
||||
- path: .env
|
||||
required: false
|
||||
- path: /mnt/cache/appdata/secrets/crafting-table.env
|
||||
- path: ${CRAFTING_SECRETS_ENV:-./secrets/crafting-table.env}
|
||||
required: false
|
||||
ports:
|
||||
- "192.168.0.5:8810:8810"
|
||||
- "${CRAFTING_BIND_IP:-0.0.0.0}:8810:8810"
|
||||
volumes:
|
||||
- /mnt/user/appdata/crafting-table/data:/data
|
||||
- /mnt/user/appdata/crafting-table/workspace:/workspace
|
||||
- /mnt/user/appdata/crafting-table/caches:/caches
|
||||
# Nix store — bind mount to /mnt/cache (88+ GB free) NOT a bare
|
||||
# docker-managed volume. A docker-managed volume lives at
|
||||
# /var/lib/docker/volumes/, which is INSIDE the docker.img loop
|
||||
# file (200 GB allocated, shared with all images + container layers
|
||||
# + every other volume). The Plutarch + haskell-nix closure is
|
||||
# tens of GB; running nix develop against Liqwid-Labs/agora once
|
||||
# was enough to fill docker.img to 100% and break every container
|
||||
# on Lucy. Caught 2026-05-06 mid-build.
|
||||
- ${CRAFTING_DATA_DIR:-./data}:/data
|
||||
- ${CRAFTING_WORKSPACE_DIR:-./workspace}:/workspace
|
||||
- ${CRAFTING_CACHES_DIR:-./caches}:/caches
|
||||
# Nix store — bind mount to a host path with plenty of free space,
|
||||
# NOT a bare docker-managed volume. Docker-managed volumes live at
|
||||
# /var/lib/docker/volumes/, which is inside the docker.img loop file
|
||||
# on Unraid-style hosts and shared with every other volume + image
|
||||
# layer. The Plutarch + haskell-nix closure is tens of GB and will
|
||||
# fill docker.img.
|
||||
#
|
||||
# Bind to /mnt/cache so Plutarch + haskell-nix closures live on
|
||||
# the cache pool, not docker.img. Pre-seed the host path with the
|
||||
# image's stock /nix install ONCE at container init (or by
|
||||
# `docker create + docker cp` when the path is empty) — bare bind
|
||||
# mount to an empty host dir shadows the image's /nix install.
|
||||
# Override CRAFTING_NIX_DIR to point at the cache pool / NVMe / etc.
|
||||
# Pre-seed the host path with the image's stock /nix install ONCE at
|
||||
# container init (bare bind mount to an empty host dir shadows the
|
||||
# image's /nix install).
|
||||
#
|
||||
# If the bind path is missing or empty when you `docker compose
|
||||
# up -d` for the first time, do this once:
|
||||
# mkdir -p /mnt/cache/appdata/crafting-table/nix
|
||||
# chown 1000:1000 /mnt/cache/appdata/crafting-table/nix
|
||||
# First-time seed when the bind path is empty:
|
||||
# mkdir -p "$CRAFTING_NIX_DIR"
|
||||
# chown 1000:1000 "$CRAFTING_NIX_DIR"
|
||||
# docker create --name ct-seed crafting-table:local
|
||||
# docker cp ct-seed:/nix/. /mnt/cache/appdata/crafting-table/nix/
|
||||
# docker cp ct-seed:/nix/. "$CRAFTING_NIX_DIR/"
|
||||
# docker rm ct-seed
|
||||
- /mnt/cache/appdata/crafting-table/nix:/nix
|
||||
networks: [sulkta]
|
||||
- ${CRAFTING_NIX_DIR:-./nix}:/nix
|
||||
networks: [crafting-table]
|
||||
restart: unless-stopped
|
||||
|
||||
networks:
|
||||
sulkta:
|
||||
external: true
|
||||
crafting-table:
|
||||
driver: bridge
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
"""Bearer + IP allowlist authentication.
|
||||
|
||||
Mirrors clawdforge's pattern:
|
||||
- Bearer tokens hashed at rest (SHA-256). No plaintext stored.
|
||||
- Per-token IP allowlist (CIDR list). NULL means "any RFC1918 + loopback"
|
||||
via the global LAN allowlist.
|
||||
|
|
@ -9,8 +8,7 @@ Mirrors clawdforge's pattern:
|
|||
- Loopback always allowed (test client uses 127.0.0.1; FastAPI's
|
||||
`request.client.host` returns 'testclient' under TestClient and we patch
|
||||
that in tests).
|
||||
- Bearer tokens NEVER appear in error messages or log lines. Same hygiene
|
||||
as clawdforge.
|
||||
- Bearer tokens NEVER appear in error messages or log lines.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
"""SQLite ledger + migrations.
|
||||
|
||||
Why SQLite (not MariaDB like clawdforge): single-process, single-host service,
|
||||
no need for cross-host replication. The runner is the only writer; every
|
||||
HTTP worker reads. SQLite in WAL mode handles single-writer-many-readers
|
||||
cleanly. Trade-off documented in README.
|
||||
Why SQLite: single-process, single-host service, no cross-host replication
|
||||
needed. The runner is the only writer; every HTTP worker reads. SQLite in
|
||||
WAL mode handles single-writer-many-readers cleanly. Trade-off documented
|
||||
in README.
|
||||
|
||||
Why stdlib `sqlite3` + `run_in_executor` (not aiosqlite): one less dependency
|
||||
and the queries are tiny (fetchone / fetchall). The runner does its own log
|
||||
|
|
@ -13,7 +13,7 @@ Migration system:
|
|||
- Each entry in MIGRATIONS is (version_id, sql_text). Versions are date-tagged
|
||||
so they sort lexicographically.
|
||||
- Apply in order, INSERT OR IGNORE into schema_migrations to handle
|
||||
multi-worker boot races (mirrors cauldron's pattern).
|
||||
multi-worker boot races.
|
||||
- Migrations are append-only; never edit a landed migration, add a new one.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
|
|
|||
|
|
@ -10,11 +10,11 @@ Design notes:
|
|||
- Idempotency is enforced by a `digest_runs` table with UNIQUE(date, project_name).
|
||||
Calling run_once twice for the same date will only send one email per project.
|
||||
- SMTP send is done via stdlib smtplib (sync) wrapped in run_in_executor so
|
||||
we don't block the loop while postfix grumbles.
|
||||
we don't block the loop while the relay grumbles.
|
||||
- If SMTP isn't configured, the server lifespan logs a warning and skips
|
||||
scheduler startup. The /digests endpoints still work for dry-run rendering.
|
||||
- Patch-drafted / auto-patches / bugs.sulkta.com numbers are zero-state
|
||||
placeholders for v0.1 wave 2C — wave 3 / step 9 wires them.
|
||||
- Patch-drafted / auto-patches / external-tracker numbers are zero-state
|
||||
placeholders until the patch loop is wired up.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -85,7 +85,7 @@ class SmtpConfig:
|
|||
def _parse_pr_url(pr_url: str) -> tuple[str, str, int] | None:
|
||||
"""Pull (owner, repo, number) out of a Gitea-style PR URL.
|
||||
|
||||
Accepts URLs like ``http://192.168.0.5:3001/Sulkta-Coop/clawdforge/pulls/42``.
|
||||
Accepts URLs like ``http://git.example.com/org/repo/pulls/42``.
|
||||
Returns None if the URL doesn't look right — caller treats that as
|
||||
"can't determine state, assume open".
|
||||
"""
|
||||
|
|
@ -222,7 +222,7 @@ def _render_text(
|
|||
lines.append("")
|
||||
lines.append("Open follow-ups:")
|
||||
lines.append(f" - {open_followups} unmerged auto-patches")
|
||||
lines.append(" - 0 manual review tickets in bugs.sulkta.com")
|
||||
lines.append(" - 0 manual review tickets in external tracker")
|
||||
lines.append("")
|
||||
lines.append(f"Full log: {full_log_url}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
|
@ -283,7 +283,7 @@ tr td:first-child {{ font-size: 1.2em; }}
|
|||
<h3>Open follow-ups</h3>
|
||||
<ul>
|
||||
<li>{open_followups} unmerged auto-patches</li>
|
||||
<li>0 manual review tickets in bugs.sulkta.com</li>
|
||||
<li>0 manual review tickets in external tracker</li>
|
||||
</ul>
|
||||
<p class="foot">Full log: <a href="{full_log_url}">{full_log_url}</a></p>
|
||||
</body></html>
|
||||
|
|
@ -314,7 +314,7 @@ class DigestScheduler:
|
|||
time_zone: str = "America/Los_Angeles",
|
||||
hour: int = 6,
|
||||
minute: int = 0,
|
||||
full_log_base_url: str = "http://192.168.0.5:8810/digests",
|
||||
full_log_base_url: str = "http://localhost:8810/digests",
|
||||
gitea_pr_state_check=None,
|
||||
):
|
||||
self.db = db
|
||||
|
|
|
|||
|
|
@ -1056,16 +1056,21 @@ class Patcher:
|
|||
) -> bool:
|
||||
"""Commit the worktree changes to a new branch and push to origin.
|
||||
|
||||
Author is forced to ``Kayos <kayos@sulkta.com>``. We pass through
|
||||
--no-gpg-sign because crafting-table containers don't have signing
|
||||
keys; commit messages reference the finding id so the PR review
|
||||
can navigate back to the finding row in the API.
|
||||
Author defaults to ``crafting-table <crafting-table@localhost>``,
|
||||
overridable via ``CRAFTING_PATCHER_AUTHOR_NAME`` +
|
||||
``CRAFTING_PATCHER_AUTHOR_EMAIL``. We pass through --no-gpg-sign
|
||||
because crafting-table containers don't have signing keys; commit
|
||||
messages reference the finding id so the PR review can navigate
|
||||
back to the finding row in the API.
|
||||
"""
|
||||
import os
|
||||
author_name = os.environ.get("CRAFTING_PATCHER_AUTHOR_NAME", "crafting-table")
|
||||
author_email = os.environ.get("CRAFTING_PATCHER_AUTHOR_EMAIL", "crafting-table@localhost")
|
||||
env = {
|
||||
"GIT_AUTHOR_NAME": "Kayos",
|
||||
"GIT_AUTHOR_EMAIL": "kayos@sulkta.com",
|
||||
"GIT_COMMITTER_NAME": "Kayos",
|
||||
"GIT_COMMITTER_EMAIL": "kayos@sulkta.com",
|
||||
"GIT_AUTHOR_NAME": author_name,
|
||||
"GIT_AUTHOR_EMAIL": author_email,
|
||||
"GIT_COMMITTER_NAME": author_name,
|
||||
"GIT_COMMITTER_EMAIL": author_email,
|
||||
"PATH": "/usr/local/bin:/usr/bin:/bin",
|
||||
}
|
||||
msg = (
|
||||
|
|
|
|||
|
|
@ -6,8 +6,7 @@ Authentication model:
|
|||
- Tokens are flagged is_admin=1 or 0. Admin can do everything.
|
||||
- Per-app tokens (is_admin=0) can register projects (becoming the owner)
|
||||
and only see/touch projects where owner_token matches their name.
|
||||
- Cross-token project access returns 404 (NOT 403) — same existence-leak
|
||||
guard clawdforge uses for sessions.
|
||||
- Cross-token project access returns 404 (NOT 403) — existence-leak guard.
|
||||
|
||||
Endpoints:
|
||||
- GET /healthz — public-ish (still needs LAN IP)
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "clawdforge",
|
||||
"git_url": "http://kayos:REPLACE_WITH_GITEA_TOKEN@192.168.0.5:3001/Sulkta-Coop/clawdforge.git",
|
||||
"name": "alpha",
|
||||
"git_url": "http://git:REPLACE_WITH_GITEA_TOKEN@REPLACE_WITH_GIT_HOST/org/alpha.git",
|
||||
"default_branch": "main",
|
||||
"languages": ["python", "rust", "go", "ruby", "php", "java", "csharp", "swift", "kotlin", "c", "cpp", "bash", "typescript", "mcp"],
|
||||
"subprojects": [
|
||||
|
|
@ -20,5 +20,5 @@
|
|||
{"path": ".", "language": "python", "build": "pip install -e .", "test": "pytest tests/", "lint": null, "audit": null, "timeout_secs": 600}
|
||||
],
|
||||
"schedule": {"audit": "0 2 * * *", "test": "0 8 * * *"},
|
||||
"notify": {"email": ["cobb@sulkta.com"], "on": ["audit_fail", "test_fail", "cve_found", "patch_drafted"], "auto_patch": true}
|
||||
"notify": {"email": ["alerts@example.com"], "on": ["audit_fail", "test_fail", "cve_found", "patch_drafted"], "auto_patch": true}
|
||||
}
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "cauldron",
|
||||
"git_url": "http://kayos:REPLACE_WITH_GITEA_TOKEN@192.168.0.5:3001/Sulkta-Coop/cauldron.git",
|
||||
"name": "beta",
|
||||
"git_url": "http://git:REPLACE_WITH_GITEA_TOKEN@REPLACE_WITH_GIT_HOST/org/beta.git",
|
||||
"default_branch": "main",
|
||||
"languages": ["python"],
|
||||
"subprojects": [
|
||||
|
|
@ -15,5 +15,5 @@
|
|||
}
|
||||
],
|
||||
"schedule": {"audit": "0 2 * * *", "test": "0 */6 * * *"},
|
||||
"notify": {"email": ["cobb@sulkta.com"], "on": ["audit_fail", "test_fail", "cve_found", "patch_drafted"], "auto_patch": true}
|
||||
"notify": {"email": ["alerts@example.com"], "on": ["audit_fail", "test_fail", "cve_found", "patch_drafted"], "auto_patch": true}
|
||||
}
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "tradecraft",
|
||||
"git_url": "http://kayos:REPLACE_WITH_GITEA_TOKEN@192.168.0.5:3001/TradeCraft/tradecraft.git",
|
||||
"name": "gamma",
|
||||
"git_url": "http://git:REPLACE_WITH_GITEA_TOKEN@REPLACE_WITH_GIT_HOST/org/gamma.git",
|
||||
"default_branch": "main",
|
||||
"languages": ["python"],
|
||||
"subprojects": [
|
||||
|
|
@ -15,5 +15,5 @@
|
|||
}
|
||||
],
|
||||
"schedule": {"audit": "0 2 * * *"},
|
||||
"notify": {"email": ["cobb@sulkta.com"], "on": ["audit_fail", "cve_found"], "auto_patch": false}
|
||||
"notify": {"email": ["alerts@example.com"], "on": ["audit_fail", "cve_found"], "auto_patch": false}
|
||||
}
|
||||
|
|
@ -3,19 +3,19 @@
|
|||
#
|
||||
# Reads the bearer token from $CRAFTING_TABLE_TOKEN, falling back to
|
||||
# /data/admin-bearer.txt (the path inside the container) if unset. The
|
||||
# admin bearer file is also bind-mounted at
|
||||
# /mnt/user/appdata/crafting-table/data/admin-bearer.txt on the Lucy host
|
||||
# — that's the recommended source on the host side.
|
||||
# admin bearer file is also bind-mounted at $CRAFTING_DATA_DIR/admin-bearer.txt
|
||||
# on the host (default ./data).
|
||||
#
|
||||
# IMPORTANT: the recipe JSON files in recipes/ ship with a placeholder
|
||||
# git_url containing "REPLACE_WITH_GITEA_TOKEN". This script substitutes
|
||||
# $GITEA_TOKEN into each recipe before posting; commit-time the real
|
||||
# token never lives on disk.
|
||||
# IMPORTANT: the recipe JSON files in recipes/ ship with placeholders
|
||||
# REPLACE_WITH_GITEA_TOKEN + REPLACE_WITH_GIT_HOST in git_url. This script
|
||||
# substitutes $GITEA_TOKEN + $GIT_HOST before posting; the real token never
|
||||
# lands on disk.
|
||||
set -euo pipefail
|
||||
|
||||
BASE_URL=${CRAFTING_TABLE_URL:-http://192.168.0.5:8810}
|
||||
BASE_URL=${CRAFTING_TABLE_URL:-http://localhost:8810}
|
||||
TOKEN=${CRAFTING_TABLE_TOKEN:-$(cat /data/admin-bearer.txt 2>/dev/null || echo "")}
|
||||
GITEA_TOKEN=${GITEA_TOKEN:-}
|
||||
GIT_HOST=${GIT_HOST:-}
|
||||
|
||||
if [ -z "$TOKEN" ]; then
|
||||
echo "no crafting-table token (set CRAFTING_TABLE_TOKEN or ensure /data/admin-bearer.txt exists)" >&2
|
||||
|
|
@ -25,12 +25,17 @@ if [ -z "$GITEA_TOKEN" ]; then
|
|||
echo "no Gitea token (set GITEA_TOKEN to substitute into recipe git_url)" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ -z "$GIT_HOST" ]; then
|
||||
echo "no git host (set GIT_HOST, e.g. git.example.com:3000)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DIR="$(dirname "$0")/recipes"
|
||||
for recipe in "$DIR"/*.json; do
|
||||
name="$(basename "$recipe" .json)"
|
||||
echo "registering $name from $recipe..."
|
||||
body="$(sed "s|REPLACE_WITH_GITEA_TOKEN|$GITEA_TOKEN|g" "$recipe")"
|
||||
body="$(sed -e "s|REPLACE_WITH_GITEA_TOKEN|$GITEA_TOKEN|g" \
|
||||
-e "s|REPLACE_WITH_GIT_HOST|$GIT_HOST|g" "$recipe")"
|
||||
code=$(printf '%s' "$body" | curl -s -o /tmp/register-resp.json \
|
||||
-w "%{http_code}" \
|
||||
-X POST \
|
||||
|
|
|
|||
|
|
@ -1,8 +1,7 @@
|
|||
# crafting-table-mcp
|
||||
|
||||
Model Context Protocol (MCP) server that bridges to the
|
||||
[crafting-table](http://192.168.0.5:3001/Sulkta-Coop/crafting-table) LAN HTTP
|
||||
service (port 8810).
|
||||
Model Context Protocol (MCP) server that bridges to the crafting-table HTTP
|
||||
service (default port 8810).
|
||||
|
||||
Drops the crafting-table tool surface into any MCP-aware client — Claude
|
||||
Desktop, Claude Code, Cursor, Zed, custom agents — so the model can register
|
||||
|
|
@ -53,7 +52,7 @@ sets these via its `env` block when it spawns the subprocess.
|
|||
|
||||
| Variable | Default | Notes |
|
||||
| ------------------------------ | ------------------------ | ----------------------------------------------------------- |
|
||||
| `CRAFTING_TABLE_BASE_URL` | `http://localhost:8810` | Override to your crafting-table host (e.g. `http://192.168.0.5:8810`). |
|
||||
| `CRAFTING_TABLE_BASE_URL` | `http://localhost:8810` | Override to your crafting-table host (e.g. `http://crafting-table.internal:8810`). |
|
||||
| `CRAFTING_TABLE_TOKEN` | (required) | App bearer token (`ct_...`). Mint with `POST /admin/tokens` against the live service. |
|
||||
| `CRAFTING_TABLE_MCP_LOG` | `WARNING` | Optional. Set `INFO` or `DEBUG` for stderr logs. |
|
||||
|
||||
|
|
@ -61,7 +60,7 @@ sets these via its `env` block when it spawns the subprocess.
|
|||
|
||||
```bash
|
||||
pip install crafting-table-mcp
|
||||
export CRAFTING_TABLE_BASE_URL=http://192.168.0.5:8810
|
||||
export CRAFTING_TABLE_BASE_URL=http://crafting-table.internal:8810
|
||||
export CRAFTING_TABLE_TOKEN=ct_...
|
||||
crafting-table-mcp # stdio JSON-RPC server
|
||||
```
|
||||
|
|
@ -77,7 +76,7 @@ Add to `~/Library/Application Support/Claude/claude_desktop_config.json`
|
|||
"crafting-table": {
|
||||
"command": "crafting-table-mcp",
|
||||
"env": {
|
||||
"CRAFTING_TABLE_BASE_URL": "http://192.168.0.5:8810",
|
||||
"CRAFTING_TABLE_BASE_URL": "http://crafting-table.internal:8810",
|
||||
"CRAFTING_TABLE_TOKEN": "ct_REPLACE_ME"
|
||||
}
|
||||
}
|
||||
|
|
@ -94,7 +93,7 @@ Or if you'd rather not rely on `crafting-table-mcp` being on `$PATH`:
|
|||
"command": "/usr/bin/python3",
|
||||
"args": ["-m", "crafting_table_mcp"],
|
||||
"env": {
|
||||
"CRAFTING_TABLE_BASE_URL": "http://192.168.0.5:8810",
|
||||
"CRAFTING_TABLE_BASE_URL": "http://crafting-table.internal:8810",
|
||||
"CRAFTING_TABLE_TOKEN": "ct_REPLACE_ME"
|
||||
}
|
||||
}
|
||||
|
|
@ -156,7 +155,7 @@ Project shape:
|
|||
```jsonc
|
||||
{
|
||||
"name": "alpha", // slug, required
|
||||
"git_url": "http://192.168.0.5:3001/.../alpha.git", // required
|
||||
"git_url": "http://git.example.com/org/alpha.git", // required
|
||||
"default_branch": "main", // optional, default "main"
|
||||
"languages": ["python", "rust"], // optional
|
||||
"subprojects": [
|
||||
|
|
@ -172,7 +171,7 @@ Project shape:
|
|||
],
|
||||
"schedule": {"audit": "0 2 * * *"}, // optional cron strings
|
||||
"notify": { // optional
|
||||
"email": ["cobb@sulkta.com"],
|
||||
"email": ["alerts@example.com"],
|
||||
"on": ["audit_fail", "cve_found"]
|
||||
}
|
||||
}
|
||||
|
|
@ -252,7 +251,7 @@ patch is drafted right now — the message says so explicitly. Once wave 3
|
|||
lands, the server will:
|
||||
|
||||
1. Pull the finding(s) off the job
|
||||
2. Send them to clawdforge with the project context
|
||||
2. Send them to the configured agent host with the project context
|
||||
3. Get back a unified diff
|
||||
4. Apply in a worktree, re-run the failing recipe
|
||||
5. If it passes, push to a `crafting-table/auto/<finding-id>` branch and
|
||||
|
|
@ -287,8 +286,7 @@ required.
|
|||
## Why this exists
|
||||
|
||||
crafting-table centralizes the polyglot build/audit toolchain on one LAN
|
||||
host so every Sulkta repo doesn't need each agent to ship its own swift /
|
||||
rust / dotnet / php toolchain. MCP is the natural integration layer — any
|
||||
MCP-aware client (Claude Desktop, Claude Code, Cursor, Zed, custom agents)
|
||||
can now treat crafting-table as a native tool surface. Spec lives at
|
||||
`memory/spec-crafting-table.md` in the openclaw-workspace repo (LAN-only).
|
||||
host so each agent doesn't need to ship its own swift / rust / dotnet / php
|
||||
toolchain. MCP is the integration layer — any MCP-aware client (Claude
|
||||
Desktop, Claude Code, Cursor, Zed, custom agents) can treat crafting-table
|
||||
as a native tool surface.
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
"crafting-table": {
|
||||
"command": "crafting-table-mcp",
|
||||
"env": {
|
||||
"CRAFTING_TABLE_BASE_URL": "http://192.168.0.5:8810",
|
||||
"CRAFTING_TABLE_BASE_URL": "http://crafting-table.internal:8810",
|
||||
"CRAFTING_TABLE_TOKEN": "ct_REPLACE_ME"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
"crafting-table": {
|
||||
"command": "crafting-table-mcp",
|
||||
"env": {
|
||||
"CRAFTING_TABLE_BASE_URL": "http://192.168.0.5:8810",
|
||||
"CRAFTING_TABLE_BASE_URL": "http://crafting-table.internal:8810",
|
||||
"CRAFTING_TABLE_TOKEN": "ct_REPLACE_ME"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ description = "Model Context Protocol (MCP) server that bridges to crafting-tabl
|
|||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
authors = [{ name = "Kayos", email = "kayos@sulkta.com" }]
|
||||
keywords = ["crafting-table", "mcp", "model-context-protocol", "claude", "sulkta", "audit", "build"]
|
||||
keywords = ["crafting-table", "mcp", "model-context-protocol", "claude", "audit", "build"]
|
||||
classifiers = [
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
|
|
@ -21,7 +21,7 @@ classifiers = [
|
|||
]
|
||||
dependencies = [
|
||||
"mcp>=1.2.0",
|
||||
"requests>=2.31.0",
|
||||
"requests>=2.32.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
|
@ -35,8 +35,8 @@ test = [
|
|||
crafting-table-mcp = "crafting_table_mcp.__main__:main"
|
||||
|
||||
[project.urls]
|
||||
Homepage = "http://192.168.0.5:3001/Sulkta-Coop/crafting-table"
|
||||
Source = "http://192.168.0.5:3001/Sulkta-Coop/crafting-table"
|
||||
Homepage = "https://git.sulkta.com/Sulkta-Coop/crafting-table"
|
||||
Source = "https://git.sulkta.com/Sulkta-Coop/crafting-table"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/crafting_table_mcp"]
|
||||
|
|
|
|||
|
|
@ -24,10 +24,10 @@ Design notes
|
|||
avoid stalling the event loop.
|
||||
- Most tool results return a single ``mcp.types.TextContent`` block with a
|
||||
JSON body. ``crafting_table_get_job`` and ``crafting_table_get_findings``
|
||||
return TWO blocks following the clawdforge ``session_turn`` pattern:
|
||||
block 0 is a human-readable prose summary, block 1 is the full JSON. The
|
||||
prose block is what the LLM reads first; the JSON block preserves
|
||||
structured detail for tool-calling agents that want to introspect.
|
||||
return TWO blocks: block 0 is a human-readable prose summary, block 1 is
|
||||
the full JSON. The prose block is what the LLM reads first; the JSON
|
||||
block preserves structured detail for tool-calling agents that want to
|
||||
introspect.
|
||||
- 404 responses surface as MCP errors with actionable messages — e.g. the
|
||||
``crafting_table_list_projects`` hint when a registered name can't be
|
||||
found.
|
||||
|
|
@ -281,7 +281,7 @@ def _tool_definitions() -> list[types.Tool]:
|
|||
name=TOOL_DRAFT_PATCH,
|
||||
description=(
|
||||
"Draft a patch (unified diff) addressing one finding on a "
|
||||
"job. The server opens a clawdforge session, asks the model "
|
||||
"job. The server opens an agent session, asks the model "
|
||||
"for a unified diff, applies it to a fresh worktree, "
|
||||
"re-runs the failing recipe to verify, and on success "
|
||||
"pushes a branch and opens a Gitea PR. No auto-merge — "
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ import pytest
|
|||
from crafting_table_mcp.client import CraftingTableClient
|
||||
|
||||
|
||||
BASE_URL = "http://192.168.0.5:8810"
|
||||
BASE_URL = "http://crafting-table.test:8810"
|
||||
TOKEN = "ct_test_token_xxxxxxxx"
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ from crafting_table_mcp.client import (
|
|||
)
|
||||
|
||||
|
||||
BASE_URL = "http://192.168.0.5:8810"
|
||||
BASE_URL = "http://crafting-table.test:8810"
|
||||
TOKEN = "ct_test_token_xxxxxxxx"
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ from crafting_table_mcp.server import (
|
|||
)
|
||||
|
||||
|
||||
BASE_URL = "http://192.168.0.5:8810"
|
||||
BASE_URL = "http://crafting-table.test:8810"
|
||||
TOKEN = "ct_test_token_xxxxxxxx"
|
||||
|
||||
|
||||
|
|
@ -159,7 +159,7 @@ class TestRegisterProject(unittest.TestCase):
|
|||
c = _client()
|
||||
project = {
|
||||
"name": "alpha",
|
||||
"git_url": "http://192.168.0.5:3001/Sulkta-Coop/alpha.git",
|
||||
"git_url": "http://git.example.com/org/alpha.git",
|
||||
"default_branch": "main",
|
||||
"languages": ["python"],
|
||||
"subprojects": [
|
||||
|
|
@ -591,7 +591,7 @@ class TestDraftPatch(unittest.TestCase):
|
|||
"attempt_number": 1,
|
||||
"status": "pr_opened",
|
||||
"branch_name": "crafting-table/auto/j-1-42",
|
||||
"pr_url": "http://192.168.0.5:3001/X/Y/pulls/9",
|
||||
"pr_url": "http://git.example.com/X/Y/pulls/9",
|
||||
"diff_excerpt": "--- a/x\n+++ b/x",
|
||||
"session_id": "s-1",
|
||||
"error": None,
|
||||
|
|
|
|||
|
|
@ -92,9 +92,9 @@ async def test_digest_aggregates_jobs_in_window(db_only):
|
|||
from crafting_table.digest import DigestScheduler, SmtpConfig
|
||||
|
||||
_seed_project(db_only, name="alpha", owner_token="oa",
|
||||
email=["cobb@sulkta.com"], notify_on=["nightly_summary"])
|
||||
email=["alerts@example.com"], notify_on=["nightly_summary"])
|
||||
_seed_project(db_only, name="beta", owner_token="ob",
|
||||
email=["cobb@sulkta.com"], notify_on=["nightly_summary"])
|
||||
email=["alerts@example.com"], notify_on=["nightly_summary"])
|
||||
|
||||
_seed_job(db_only, project_name="alpha", job_id="a-pass",
|
||||
recipe="audit", status="succeeded", hours_ago=2)
|
||||
|
|
@ -291,7 +291,7 @@ def test_admin_digest_run_now_endpoint(client):
|
|||
# Register a project under alpha with email + nightly_summary
|
||||
payload = sample_project_payload(name="ep")
|
||||
payload["notify"] = {
|
||||
"email": ["cobb@sulkta.com"],
|
||||
"email": ["alerts@example.com"],
|
||||
"on": ["nightly_summary"],
|
||||
"auto_patch": False,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ def _patcher_with_mocks(db: DB, workspace: WorkspaceManager, *, runner=None):
|
|||
claw.close_session = AsyncMock()
|
||||
gitea = MagicMock(spec=GiteaClient)
|
||||
gitea.open_pr = AsyncMock(
|
||||
return_value={"html_url": "http://192.168.0.5:3001/X/Y/pulls/1"}
|
||||
return_value={"html_url": "http://git.example.com/X/Y/pulls/1"}
|
||||
)
|
||||
p = Patcher(
|
||||
db=db,
|
||||
|
|
@ -398,7 +398,7 @@ async def test_pushed_and_pr_opened_on_success(db_only, tmp_path):
|
|||
attempt = await p.maybe_draft(job_id, finding_id=finding_id)
|
||||
assert attempt is not None, "expected a PatchAttempt"
|
||||
assert attempt.status == "pr_opened", f"unexpected: {attempt.status} / {attempt.error}"
|
||||
assert attempt.pr_url == "http://192.168.0.5:3001/X/Y/pulls/1"
|
||||
assert attempt.pr_url == "http://git.example.com/X/Y/pulls/1"
|
||||
assert attempt.branch_name and "crafting-table/auto/" in attempt.branch_name
|
||||
assert gitea.open_pr.await_count == 1
|
||||
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@ def test_post_patches_with_finding_id(client):
|
|||
attempt_number=1,
|
||||
status="pr_opened",
|
||||
branch_name="crafting-table/auto/j-1-42",
|
||||
pr_url="http://192.168.0.5:3001/X/Y/pulls/9",
|
||||
pr_url="http://git.example.com/X/Y/pulls/9",
|
||||
diff_excerpt="--- a/x\n+++ b/x",
|
||||
session_id="s-1",
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue