clawdforge/README.md
Kayos 940861f70a v0.2: multi-turn /sessions endpoints backed by ACPX
- Dockerfile: install acpx@latest alongside @anthropic-ai/claude-code
- compose.yml: bind /mnt/user/appdata/clawdforge/acpx-sessions:/root/.acpx/sessions
- DB: additive sessions + session_events tables in store.py SCHEMA
- clawdforge/acpx_runner.py: AcpxManager + AcpxSession, bounded async pool,
  per-invocation subprocess model (acpx CLI itself owns the queue-owner
  lifecycle, so each turn = one fresh `acpx prompt -s <uuid>` call)
- server.py: POST/GET/DELETE /sessions + POST /sessions/{id}/turn + GET /sessions
- Per-app isolation: 404 (not 403) on cross-token session access — no
  existence leak across tokens
- Lifespan-managed TTL sweeper: every 60s soft-closes idle sessions past
  CLAWDFORGE_SESSION_TTL_SECS (1h default), hard-deletes ledger rows past
  CLAWDFORGE_SESSION_HARD_TTL_SECS (24h default)
- session_events audit table parallel to existing runs table
  (events: create, turn, close, sweep_close, hard_delete)
- /healthz now reports acpx_present + acpx_version + open_sessions count
- tests/test_sessions.py: 16 tests covering create/turn/close/list/isolation/
  sweep/pool-full/regression. /run regression test asserts byte-identical
  v0.1 response shape.

ACPX research notes (v0.6.1, openclaw/acpx):
- npm package is `acpx`, not `@openclaw/acpx`
- Sessions are scoped by (agentCommand, cwd, name?). We mint our own UUID
  as `--name` and give every session a unique cwd subdir, so the scope key
  is collision-free across apps.
- session_id source: ours. We pass --name <uuid>, ACPX records it under
  ~/.acpx/sessions/<encoded-id>.json. We never need to parse ACPX's
  acpxRecordId — our UUID is canonical.
- Subprocess lifetime: per-invocation, NOT per-session. The acpx CLI itself
  spawns/maintains a per-session "queue owner" process via local IPC; each
  `acpx prompt` call we make either elects itself owner or enqueues. The
  AcpxSession class is therefore a thin (uuid, cwd, asyncio.Lock) handle,
  not a long-lived stdio pipe. The spec's "owns one stdio pipe pair" model
  was rewritten to match reality — flagged here.
- Close semantics: soft-close via `acpx sessions close <name>`. The
  on-disk record stays (ACPX's `sessions prune` is the hard-delete path,
  not invoked from clawdforge). DELETE /sessions/<id> is documented as
  idempotent (200 with already_closed=true on second call) so SDKs can
  call close() in finally/Drop blocks safely.
- File uploads: ACPX has no file-attach ACP method exposed via the CLI.
  We prepend a [Attached files] header listing absolute paths; the agent
  uses its Read tool to open them. Same behavior as /run --files in v0.1.
- Permissions: --approve-all on the turn invocation since the container is
  unattended and callers are bearer-token-trusted. Future v0.3 may expose
  a per-session permission policy.

/run endpoint unchanged — backwards compat verified by
test_run_endpoint_unchanged + test_run_endpoint_unchanged_error_shape.

Spec: memory/spec-clawdforge-v0.2.md
ACPX CLI ref: https://github.com/openclaw/acpx/blob/main/docs/CLI.md
2026-04-29 06:22:55 -07:00

233 lines
8.1 KiB
Markdown

# clawdforge
LAN-only HTTP service that runs `claude -p` subprocess calls on behalf of Sulkta apps.
One container holds the Claude Code subscription auth; multiple apps consume via bearer
tokens + IP allowlist.
## Why
- **Auth in one place** — only this container needs to be `claude /login`'d, not every app
- **Smaller app images** — apps stay tiny Python/Go containers, no node/npm/claude-cli
- **Audit log** — every prompt + response chars + duration in one SQLite db
- **Reusable bone** — petalparse, cauldron, johnny5 all consume the same surface
## Surface
```
GET /healthz liveness + claude --version smoke
POST /run run a prompt, return parsed result
POST /files upload a file, get a file_token to pass to /run
POST /sessions create a multi-turn session (v0.2)
POST /sessions/<id>/turn send a turn to a session (v0.2)
GET /sessions/<id> read session state (v0.2)
DELETE /sessions/<id> soft-close a session (v0.2)
GET /sessions list this token's sessions (v0.2)
POST /admin/tokens mint a per-app token (admin)
GET /admin/tokens list app tokens (admin)
DELETE /admin/tokens/<name> revoke a token (admin)
```
### `POST /run`
```json
{
"prompt": "Sterilize this ingredient line: 'about 2 cups of cooked white rice'",
"model": "sonnet",
"system": "You are a precise recipe parser. Always reply with valid JSON.",
"files": ["ff_..."],
"timeout_secs": 60
}
```
Returns:
```json
{
"ok": true,
"result": { "qty": 2, "unit": "cup", "food": "rice", "note": "cooked, white", "approx": true },
"duration_ms": 4321,
"stop_reason": "end_turn"
}
```
`result` is the inner `{"type":"result","result":"..."}` from `claude -p --output-format json`,
auto-stripped of code fences and JSON-parsed if possible. If the inner is not valid JSON, it's
returned as a string.
### `POST /files`
multipart/form-data, field `file`, optional `ttl_secs` (60..86400, default 3600).
Returns `{"file_token": "ff_...", "ttl_secs": 3600, "size": 12345}`.
Use that token in subsequent `/run` requests to attach the file via `claude -p --files`.
## Auth
Two layers:
1. **IP allowlist** — global CIDR list in `ALLOW_CIDRS` env. Loopback always allowed.
Per-app allowlist optional on top (mint with `ip_cidrs: [...]`).
2. **Bearer token**`Authorization: Bearer cf_<...>` for `/run` and `/files`,
`Authorization: Bearer <ADMIN_BOOTSTRAP_TOKEN>` for `/admin/*`.
Tokens are SHA-256 hashed in SQLite. The plaintext is shown ONCE at create time.
## Deploy
1. SSH to Lucy: `ssh lucy`
2. `mkdir -p /mnt/user/appdata/clawdforge/{data,claude-config,claude-alt-config}`
3. Drop `.env` at `/mnt/cache/appdata/secrets/clawdforge.env` (chmod 600, root:root) — see `.env.example`
4. Clone the repo to `/opt/stacks/clawdforge` (Lucy uses Gitea reverse-tunnel pattern)
5. `cd /opt/stacks/clawdforge && docker compose up -d --build`
6. **Auth Claude CLI** (one-time, persists on volume):
```
docker exec -it clawdforge claude /login
```
Walk through the device-auth flow. Credentials persist at `/root/.claude/` inside the
container, mapped to `/mnt/user/appdata/clawdforge/claude-config/` on host.
7. Smoke:
```
curl http://192.168.0.5:8800/healthz
```
Should report `claude_present: true` + a version string.
8. Mint a token for the first consumer:
```
curl -sS -X POST http://192.168.0.5:8800/admin/tokens \
-H "Authorization: Bearer $ADMIN_BOOTSTRAP_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"cauldron","ip_cidrs":["172.24.0.0/16"]}'
```
Save the returned `token` into the consumer's env.
## Client snippet (Python)
```python
import os, requests
CF = "http://192.168.0.5:8800"
TOKEN = os.environ["CLAWDFORGE_TOKEN"]
r = requests.post(
f"{CF}/run",
headers={"Authorization": f"Bearer {TOKEN}"},
json={
"prompt": 'Reply with JSON: {"hello": "world"}',
"model": "sonnet",
"timeout_secs": 30,
},
timeout=60,
)
r.raise_for_status()
print(r.json()["result"]) # {'hello': 'world'}
```
## Multi-turn / Sessions (v0.2)
`/run` is one-shot: stateless, fast, returns a single result. When you need
multi-turn context (build something step-by-step, debug across iterations,
long-running structured tool-call work), use `/sessions/*`.
The session surface is backed by [ACPX](https://github.com/openclaw/acpx) —
the OpenClaw team's headless Agent Client Protocol CLI. Clawdforge wraps it
so apps don't need to manage ACPX subprocesses or session metadata directly.
Sessions persist on disk under `/root/.acpx/sessions/` (mounted from the
host) so they survive container rebuilds.
### Quickstart — three turns in one session
```bash
TOKEN=$CLAWDFORGE_TOKEN
CF=http://192.168.0.5:8800
# 1. Create a session
SID=$(curl -sS -X POST $CF/sessions \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"agent":"claude"}' | jq -r .session_id)
# 2. Send a turn
curl -sS -X POST $CF/sessions/$SID/turn \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"prompt":"Help me draft a SQL migration. First describe what you need to know."}'
# 3. Send a follow-up — the session keeps context
curl -sS -X POST $CF/sessions/$SID/turn \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"prompt":"Postgres 16, table users, add column tier text default \"free\""}'
# 4. Close
curl -sS -X DELETE $CF/sessions/$SID -H "Authorization: Bearer $TOKEN"
```
### Turn response shape
```json
{
"ok": true,
"session_id": "abc123...",
"turn_index": 2,
"events": [
{"jsonrpc":"2.0","method":"session/update","params":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"..."}}},
{"jsonrpc":"2.0","id":"req-1","result":{"stopReason":"end_turn"}}
],
"stop_reason": "end_turn",
"duration_ms": 12345
}
```
`events` is the raw ACP NDJSON stream from acpx, parsed into objects. Each
entry is a JSON-RPC message — `session/update` for streamed agent output,
tool calls, plan updates, etc., and a final `result` envelope with the
`stopReason`. No streaming/SSE in v0.2; the full event list is returned
when the turn ends.
### Per-app isolation
Every session is owned by the token that created it. Cross-token access
returns `404` (not `403`) so token A can't even probe whether token B's
session exists.
### TTL + cleanup
A background sweeper runs every `CLAWDFORGE_SWEEP_INTERVAL_SECS` (default 60s):
- Sessions idle longer than `CLAWDFORGE_SESSION_TTL_SECS` (default 1h) are
soft-closed via `acpx sessions close`.
- Sessions whose `closed_at` is older than `CLAWDFORGE_SESSION_HARD_TTL_SECS`
(default 24h) are hard-deleted from clawdforge's ledger.
- Closed sessions stay queryable via `GET /sessions/<id>` until the hard-TTL
fires.
### Container / deploy
The container needs both `claude` and `acpx` on PATH plus a host-mounted
volume for ACPX's session store:
```yaml
# compose.yml (already configured)
volumes:
- /mnt/user/appdata/clawdforge/acpx-sessions:/root/.acpx/sessions
```
```dockerfile
ENV ACPX_BIN=acpx
RUN npm install -g acpx@latest
```
ACPX shares Claude Code auth from the same `/root/.claude/` volume the v0.1
runtime already used, so a single `claude /login` ceremony covers both
`/run` and `/sessions/*`.
## Notes
- The CLI is `@anthropic-ai/claude-code` (not the Python `anthropic` SDK).
- ACPX is the upstream session driver — see https://github.com/openclaw/acpx
and `docs/CLI.md` in that repo for protocol semantics. Clawdforge owns
the per-app ledger and TTL policy; ACPX owns session content.
- Default model is `sonnet`; per-request override via `model` field on `/run`.
For sessions, model is fixed at create time (configurable later).
- Per-run working directory is staged under `RUNS_DIR` and torn down on exit, so
`claude` can't pollute the container's working tree.
- Per-session working directory is staged under `ACPX_SESSIONS_CWD` (default
`/data/acpx-cwds/<session_id>`) and torn down on close.
- File uploads are scoped to the uploading app — token A can't reference token B's files.