runner: pipe prompts > 64KB via stdin to avoid OS argv limit

Cobb's seed-cleanup job hit OSError [Errno 7] Argument list too long with
a 577KB prompt. Linux ARG_MAX is typically 128KB-2MB depending on kernel +
env; passing the full prompt as 'claude -p <PROMPT>' fails for big jobs.

Fix: detect prompt size > 64KB threshold, omit the positional prompt
argument from the CLI invocation and pipe via subprocess.run(input=...)
instead. claude -p reads the prompt from stdin when no positional given.

System prompt + flags still pass as CLI args (those stay small).
This commit is contained in:
Kayos 2026-04-28 22:08:47 -07:00
parent 1b4f62950b
commit 8d1da6e20d

View file

@ -36,14 +36,18 @@ class Runner:
files: list[str] | None = None,
timeout_secs: int | None = None,
) -> RunResult:
cmd = [
self.claude_bin,
"-p",
prompt,
"--output-format",
"json",
"--model",
model or self.default_model,
# Linux ARG_MAX caps a CLI argument around 128KB-2MB depending on
# kernel + env. Long prompts (recipe corpora, big curate jobs) blow
# past it. For prompts > 64KB we pipe via stdin instead.
ARG_MAX_THRESHOLD = 64 * 1024 # bytes; leaves headroom for env+other args
use_stdin = len(prompt.encode("utf-8")) > ARG_MAX_THRESHOLD
cmd = [self.claude_bin, "-p"]
if not use_stdin:
cmd.append(prompt)
cmd += [
"--output-format", "json",
"--model", model or self.default_model,
]
if system:
cmd += ["--append-system-prompt", system]
@ -66,6 +70,7 @@ class Runner:
text=True,
timeout=timeout,
cwd=str(cwd),
input=prompt if use_stdin else None,
)
except subprocess.TimeoutExpired as e:
duration_ms = int((time.monotonic() - started) * 1000)