Commit graph

7 commits

Author SHA1 Message Date
3578c9321b workspace: explicit fetch right after bare clone (populates remote-tracking refs that --bare doesn't) 2026-04-29 13:58:28 -07:00
129d630391 workspace: per-project lock + fetch into remote-tracking refs
Two bugs caught by the first real concurrent dogfood (15 SDK build jobs
queued at once for clawdforge):

1. Concurrent fetch race — multiple jobs from same project all called
   `git fetch +refs/heads/*:refs/heads/*` simultaneously. Git refuses
   to fetch into a local branch ref that another worktree has checked
   out, so jobs after the first failed fast with:
       fatal: refusing to fetch into branch 'refs/heads/main' checked
       out at '/workspace/clawdforge/<other-job-id>'

2. Worktree-from-local-branch — `worktree add ... main` reserved the
   local branch ref to the worktree, blocking subsequent fetches even
   when the lock above wasn't held.

Fix:
- Per-project asyncio.Lock around the materialize() body (clone +
  fetch + worktree-add). Different projects still parallelize; same
  project serializes through the cache dir.
- Fetch into remote-tracking refs only:
    +refs/heads/*:refs/remotes/origin/*
  Local refs/heads/* are never written, so no worktree can hold them.
- worktree add uses `--detach origin/<branch>` so each worktree is at
  the remote-tracking ref. Multiple detached worktrees share the same
  remote ref without conflict.

The recipe itself runs outside the lock, so concurrency=4 still
parallelizes the actual build/test work — only the brief
materialize step (clone or fetch + worktree create) serializes.

Tests still 6/6 green on test_runner.py (uses _StubWorkspace, not the
real WorkspaceManager, so it's unaffected — the real exercise is the
live re-queue against fixed code).
2026-04-29 13:56:44 -07:00
61a9814b67 patcher: pin model='opus' on clawdforge sessions; CRAFTING_PATCHER_MODEL env override
Code-work prompts (read CVE/lint context, draft a unified diff that
verifies cleanly against the failing recipe) reward Opus's longer
context + careful reasoning. Cauldron-style high-frequency Sonnet calls
are unaffected — this only changes what crafting-table's patcher asks
clawdforge to use for its drafted-patch sessions.

Pairs with clawdforge dbbead2 which adds the optional `model` field on
POST /sessions and propagates ANTHROPIC_MODEL into the acpx subprocess
env on both create + turn.

Override knob: CRAFTING_PATCHER_MODEL env (e.g. "sonnet" if cost > quality).

Tests: 16/16 patcher tests still green.
2026-04-29 12:37:54 -07:00
4eab869df0 v0.1 wave 3 (steps 9+10): autonomous patch loop + production recipes
Step 9 — autonomous patch loop:
- patcher.py: clawdforge session → unified diff → worktree apply → verify recipe → push branch → open Gitea PR
- migration 007: patch_attempts (UNIQUE per finding+attempt, max 3 attempts)
- runner.py: post-parse hook fires patcher.maybe_draft_for_job when notify.auto_patch=true
- server.py: POST /jobs/{id}/patches, GET /patches, GET /patches/{id}
- digest.py: patch-drafted lines + open-follow-up count via Gitea PR state check
- mcp: crafting_table_draft_patch stub replaced with real implementation
- tests/test_patcher.py + tests/test_patches_api.py: 27 new tests

No auto-merge — patches stop at PR-open. Cobb merges.

Step 10 — production recipes:
- examples/recipes/clawdforge.json: 14 subprojects across all SDKs, audit nightly
- examples/recipes/cauldron.json: single Flask subproject, audit nightly
- examples/recipes/tradecraft.json: nightly audit, auto_patch=false (manual review)
- examples/register-all.sh: bulk-register helper with GITEA_TOKEN substitution
- README "Autonomous patch loop" + "First production recipes" sections

Tests: server 116→143, mcp 65→67. All green.

Spec: memory/spec-crafting-table.md
2026-04-29 09:04:48 -07:00
d467b2f5be v0.1 wave 2A (steps 5+6): per-language parsers + findings extraction
- parsers/ package: rust / python / go / typescript / generic
- parser registry with language+recipe -> fallback resolution
- fingerprint hash (kind+file+line+code) for cross-run dedup
- runner.py post-exec hook: parse log, persist findings, count on job row
  (extraction runs before mark_job_finished so callers polling on terminal
  status see findings_count populated atomically)
- db.insert_finding / list_findings / increment_findings_count DAOs already
  shipped in wave 1; wired here
- GET /jobs/{id}/findings now returns real data (server route already
  shipped; was returning empty list because nothing populated the table)
- tests/test_parsers/: 6 modules + 11 fixtures (rust/python/go/typescript)
- tests/test_runner_findings.py: 3 integration tests
- README: tick steps 2-6, add Findings section

Suite: 108 passing (62 wave-1 + 46 new).
Spec: memory/spec-crafting-table.md
2026-04-29 08:36:16 -07:00
98306ca2e0 v0.1 wave 2C (step 8): email digest scheduler
- digest.py: DigestScheduler with daily 06:00 PT loop
- SmtpConfig env-driven (CRAFTING_SMTP_*)
- notify.on event filter respected per project
- GET /digests/{date} + POST /admin/digest/run-now (dry_run flag)
- migration 006: digest_runs (idempotency via UNIQUE(date, project_name))
- text + HTML email bodies; matches spec's worked example
- Server lifespan integration; gracefully disables if SMTP not configured
- tests/test_digest.py: 8 tests (aggregation / filter / smtp mock / idempotency / endpoint)

Patch-drafted line is a placeholder until wave 3 / step 9 ships.

Spec: memory/spec-crafting-table.md
2026-04-29 08:33:37 -07:00
0ec3a04676 v0.1 wave 1 (steps 2+3+4): SQLite ledger + FastAPI skeleton + async job runner
- db.py: migrations + DAOs for tokens / projects / jobs / findings (SQLite WAL)
- auth.py: SHA-256 bearer hashing + LAN-CIDR allowlist + admin/app token tiers
- models.py: Pydantic shapes (Project, Subproject, Schedule, Notify, Job, CreateJobRequest)
- server.py: FastAPI on port 8810; /healthz, /admin/tokens/*, /projects/*, /jobs, /jobs/{id}, /jobs/{id}/log, /jobs/{id}/findings
- runner.py: bounded asyncio pool, per-job timeout with process-group SIGTERM→SIGKILL escalation, orphaned-job recovery on boot
- workspace.py: bare-clone + worktree materialization, gc
- config.py: env-driven
- 62 tests across db / auth / projects / jobs / runner / e2e — all green

Cross-token project access returns 404 (not 403) — existence-leak guard.
Bearer tokens hashed at rest; admin token bootstrapped on first boot.
Recipe subprocess uses start_new_session=True so killpg targets the
whole process tree on timeout — child processes can't escape SIGKILL.
Pump task guarded with wait_for(2s) + cancel fallback against any
orphan that survives the group kill.

Wave 2 (parsers + findings extraction + MCP + email digest) pending.

Spec: memory/spec-crafting-table.md
2026-04-29 08:17:41 -07:00