Commit graph

30 commits

Author SHA1 Message Date
98233182fd forge: high effort for prose-craft passes, max only for audit
gen, cleanup, narrate_prep and rewrite drop from max to high
effort. Audit keeps max — it is the one pass doing real reasoning
(canon drift, timeline gaps, retcons) rather than prose-craft, so
it is worth the frontier spend. Prose-craft is "good enough" at
high. This also keeps the all-Opus skald pattern under the
$200/month claude -p cap landing next month.
2026-05-14 22:32:52 -07:00
d2442f0a87 forge: rewrite pass — re-author prose in an author's voice
New Forge::rewrite + PassKind::Rewrite. An author re-authors
existing chapter prose entirely in their voice — sentence rhythm,
word choice, paragraph shape all become theirs — while canon
(names, dates, places, events, order, technical facts) is preserved
exactly. Not editing; re-authoring. SystemMode::Replace, max effort.

skald rewrite --chapter <uuid> [--author slug] overwrites body_md
with the rewritten version. The pre-rewrite prose is stashed in the
new chapters.body_md_original column on first rewrite (migration
0008, idempotent) so the original is never lost. body_md_tts is
cleared — it was annotated against the old prose and must be
regenerated by a fresh prepare-narration.

prepare-narration gains --single-voice: skips the character speaker
roster so no [voice:X] dialogue tags are inserted, only beat
markers. Right for one-voice narration.

Migration 0008 also extends generation_runs.kind to allow 'rewrite'.
2026-05-14 21:35:20 -07:00
303b6c73f4 narrate: route tortoise_* voices to TORTOISE_URL
Engine dispatch was a binary kokoro-vs-everything-else split that
sent tortoise voices to F5. Now three branches:
  kokoro_*    → KOKORO_URL    (Lucy default :7794)
  tortoise_*  → TORTOISE_URL  (Lucy default :7795)
  *           → F5_TTS_URL    (Lucy default :7792)

substitute_speaker_voices also runs for tortoise voices, so
multi-voice [voice:slug] tags survive in the prose. Tortoise voices
without a per-character mapping fall back to the narrator voice
server-side (tortoise_server.py logs the fallback).
2026-05-14 09:46:16 -07:00
d1631ddffe engines: import f5-tts + kokoro + tortoise sidecars into the tree
The python FastAPI sidecars have lived ad-hoc at /mnt/cache/appdata/
<engine>/build/ on Lucy without version control. Bringing them into
the skald repo so the engine code travels with the cross-engine
routing it depends on.

This commit lands the VANILLA version of each engine on main:

  engines/f5-tts/    SWivid F5-TTS (CC-BY-NC weights flagged)
  engines/kokoro/    hexgrad Kokoro-82M (Apache 2.0 top to bottom)
  engines/tortoise/  neonbjb Tortoise-TTS (Apache 2.0 top to bottom)

Engine-specific kludges (question doubling, GPU coordination,
pause-duration tuning) get layered on engine/* branches per the
README. Main stays the safe-to-read baseline.
2026-05-14 09:40:01 -07:00
1c3fc11484 narrate: pronunciation overrides for proper nouns
skald narrate pre-processes body_md_tts with word-boundary regex
substitutions from pronunciation_overrides where phoneme_format =
'respelling'. Story-scoped overrides win over global; longer words
substitute first (so 'Bryukhanov' wins over a hypothetical 'Bry'
override). Case is preserved on the first letter so 'Pripyat' at
sentence start stays capitalised after respelling to 'Prip-yat'.

Seeded Coast-Down with 52 entries: Russian/Ukrainian surnames
(Dyatlov, Akimov, Bryukhanov, Stolyarchuk, Yuvchenko, Khmel, etc.),
first names (Pyotr, Lyudmilla, Anatoly, Vasily, Sasha, Aleksandr,
Leonid), patronymics (Stepanovich, Fyodorovich, Mykolaivna,
Hryhorivna), and places (Pripyat, Chernobyl, Kyiv, Kopachi,
Lubyanka). Plus the operational acronyms NIKIET, RBMK, AZ-5, SIUR,
SIUB, ChNPP, MSCh.

Other phoneme_format values (ipa, arpabet) are no-ops for now —
Kokoro's misaki tokenizer doesn't expose a stable lexicon-injection
API across the HTTP boundary in v0.1. Future: pass IPA forms in a
new server-side request field and inject into the pipeline's g2p
lexicon for more accurate phonetics.
2026-05-14 08:56:17 -07:00
c9bd38034c multi-voice: per-character dialogue rendering
Schema: characters.voice_id + characters.slug (migration 0007).
voice_id is FK to voices(id); slug is the stable lowercase token
the narrate_prep pass uses inside [voice:slug]...[/voice].

Forge::narrate_prep takes &[CharacterSpeaker]. System prompt
expanded to instruct the author to wrap dialogue lines in voice
tags based on a roster supplied in the user prompt (slug + name +
short hint from key_facts). Unattributed dialogue stays unwrapped
and inherits the narrator voice.

skald narrate substitutes [voice:<character-slug>] →
[voice:<kokoro-voice-name>] right before sending to Kokoro, using
characters.voice_id JOIN voices.reference_path as the map. Slugs
with no voice or no character row fall back to the narrator voice
defensively (logged as warn).

kokoro_server.py v0.4: splitter recognises [voice:X]...[/voice]
blocks at the paragraph level. Each text node carries an optional
voice attribution; renderer feeds it to Kokoro per-segment. Outside
voice blocks the request's default voice is used. voices_used is
reported back so callers can verify multi-voice actually ran.

Only kokoro-routed renders pre-process voice tags; F5 paths leave
the tags in place (F5 multi-voice not implemented). Defensive
fallback: orphan/unclosed [/voice] markers are silently absorbed
rather than failing the render.
2026-05-14 08:35:33 -07:00
330bc8bde2 migration 0005: idempotent ADD COLUMN IF NOT EXISTS
Caught when redeploying after the 0006 patch: the live DB had
migration 5 stamped with a stale checksum + the column already
present, so neither re-apply nor checksum-only-fix worked cleanly.
Making 0005 idempotent fixes both paths.
2026-05-13 20:32:41 -07:00
2ed3d3373a migration 0006: extend generation_runs.kind to allow narrate_prep
Migration 0005 added the chapters.body_md_tts column but missed
this check constraint update — caught at runtime when
prepare-narration tried to insert kind='narrate_prep'.

Postgres doesn't ALTER CHECK in place; we drop + re-add.
2026-05-13 20:28:59 -07:00
89c35fd9d3 narrate: body_md_tts column + narrate_prep pass + Kokoro routing
Two new things working together:

1. Migration 0005 adds chapters.body_md_tts (nullable). Narrate path
   prefers it over body_md when present — that's the annotated-for-
   audiobook variant. Falls back to body_md if not set.

2. New Forge::narrate_prep pass: author (or House) annotates prose
   with [breath] / [pause:Xs] / [scene] beat markers AND occasional
   humanizing narrator stumbles (em-dash repetition, self-correction,
   hesitation — sparingly, 1-3 per chapter). Apart from stumbles, the
   prose is verbatim. Author voice threads through.

3. New CLI: 'skald prepare-narration --chapter <uuid> [--author slug]
   [--overwrite]'. Records as generation_runs row kind=narrate_prep.

4. skald narrate now routes by voice.source — kokoro_* voices hit
   KOKORO_URL (Apache 2.0 stack, audiobook-tuned with the v0.2 render-
   and-stitch server), everything else hits F5_TTS_URL (voice-cloning
   path). Voice DB row carries source as the dispatch key.

Why no new tag for narrator stumbles: em-dash repetition and self-
correction are just prose patterns Kokoro reads correctly because of
its punctuation cues. No new server-side machinery.
2026-05-13 20:24:38 -07:00
aece970b50 forge: skald owns chapter numbers, not the LLM
Caught on the 2026-05-13 Coast-Down 10-chapter Orson run: the LLM
labeled two chapters 'Chapter 1' instead of 9 and 10, and
ON CONFLICT (story_id, n) DO UPDATE silently overwrote them.
8 visible chapters from 10 successful gen+cleanup passes; 27k
words of work, ~6k buried. The audit caught the symptom but the
data damage was already done.

Fix:
- continue_story::run computes next_n from MAX(chapters.n) before
  the batch loop; each iteration's authoritative n is next_n,
  incremented after success.
- forge::generate + cleanup take chapter_n: Option<i32>. The gen
  prompt is now 'Write Chapter N. Begin with: ## Chapter N — ...'
  instead of the vague 'Write the next chapter.'
- We still parse_chapter() the LLM output but only to extract the
  title; if the LLM-returned n disagrees with ours, we log a warn
  and use the authoritative N at INSERT time.

The (story_id, n) unique constraint stays — it's now a defensive
catch for skald bugs, not the LLM's free-spirited numbering.
2026-05-13 18:29:22 -07:00
75a609d507 web: chapter audio player + render button
Chapter view now shows a narration card between title and prose
with three states:
  - succeeded → HTML5 <audio> + voice + duration + download link
  - running   → 'rendering…' banner with relative start time
  - none/failed → 'Render audio' POST button (spawns background
                  tokio task calling narrate::run)

ServeDir mounted at /audio serves WAVs from the f5-tts bind-mount
read-only. Range requests work, so 16-min chapters seek cleanly.

Deploy needs: compose mount /mnt/cache/appdata/f5-tts/audio:/audio:ro
on skald (already staged in /mnt/cache/appdata/skald/compose.yml on
Lucy).
2026-05-13 17:08:43 -07:00
c2bb12fdd0 narrate: F5-TTS HTTP client + skald narrate CLI
skald-core::narrate ships a thin reqwest client + voice DB access
(get_by_name, get_default, get_by_id). The boundary is the f5-tts
container's purpose-built FastAPI sidecar (python lives there
because torch + transformers + safetensors do); skald never touches
python.

CLI: skald narrate --chapter <uuid> [--voice slug] [--speed 1.0].
Voice resolution: --voice flag → story.preferred_voice_id → system
default. Persists narration_runs row (engine='f5-tts', engine_version
pinned, status: running → succeeded|failed). Output path stored is
the f5-tts container's view (/audio/<story>-<n>-<run>.wav); web
playback wiring deferred.
2026-05-13 16:45:04 -07:00
3a749b7643 forge: always --effort max + multi-chapter batch (cap 20)
forge.rs threads Effort::Max on gen + cleanup. Audit + summarize stay
default — they're structured-output / tool-shaped tasks where extended
thinking doesn't help. Bumps subprocess timeout from 600s to 1800s so
max-effort prose-craft has the wall clock it needs.

continue_story::run takes a chapter_count param; loops gen+cleanup per
chapter with each iteration's just-written prose appended to context.
Audit fires once at end against the combined batch vs parent canon.
Cap is 20 (~5h wall clock, ~$600 at max effort — beyond that is
operationally absurd).

CLI: 'skald continue --chapters N'. Web: numeric field on both new-
story and continue forms, 1..=20, defaults to 1.

Vendored clawdforge SDK refreshed for the Effort enum.
2026-05-13 14:26:53 -07:00
20e262c85d web: new-story form gets 'fire now' too — same pattern as continue form
Cobb tried creating a story via /stories/new tonight, ticked through
the form, hit Create, then expected gen to fire. It didn't, because
the fire-now checkbox only existed on the continue form. Story sat
in 'seed' for 30 min before he asked.

Fix: same path as continue. NewStoryForm picks up an optional 'fire'
field; new_story_create spawns a tokio::spawn task that calls
continue_story::run() with parent_story_id=None semantics:
- Context is the story's own prompt (not parent's bible)
- Audit pass is skipped (no parent to compare against)
- Status flow: seed → generating → cleaning → complete

The form copy explains the audit-skip for first chapters so the
user isn't surprised to see no findings.

Also fired Cobb's pending story manually via 'skald continue
--story bd73dd19...' so it actually generates this round.
2026-05-13 13:44:41 -07:00
1e19305432 web: author picker on both forms + fire-gen background task
Both /stories/new and /stories/:id/continue now carry an author
<select> populated from the authors table. New-saga + continue
panels pre-select the parent story's author when continuing
(propagates the voice across sequels by default; user can override).

Background fire-generation: continue form has a 'fire generation
now' checkbox. When checked, the POST creates the seed story row
AND spawns a tokio::spawn task that calls continue_story::run()
in the background. The user redirects to the new story's detail
page and can refresh to watch status flow seed → generating →
cleaning → auditing → complete. Failure path logs to the
container's tracing output (and generation_runs rows pick up
'failed' status).

Unchecked behavior: same as before — sequel sits in 'seed' state
until 'skald continue' fires it manually. Useful for queuing
multiple drafts before committing the opus spend.

CSS adds .checkbox-label styling so the checkbox + label flow
horizontally with the rest of the form looking sane.

Compiles clean. Smoke-test: open /stories/.../continue, pick an
author, tick 'fire now,' submit. Should redirect to a 'generating'
status; opening generation log shows the running gen pass.
2026-05-13 12:48:16 -07:00
c899019b35 web v0.3 visual revamp: Norse palette + ornament + mobile + forms
Big CSS overhaul + page-level adjustments toward a Norse 'museum-
quality' aesthetic (not the gaming-rune-bro variant). Restraint +
weight + carved typography.

Palette shift:
- Warmed-black bg (#0a0807) with subtle radial gradient grain
- Bone-cream ink (#dbcfb0) replaces the cleaner cream we had
- Oxblood accent (#a13a3a) replaces the coffee-shop soft gold
- Weathered bronze (#b08443) as secondary accent for headers + meta
- Status colors land warmer: sage-moss ok, rust crit, amber-bronze warn

Typography:
- All caps + 2-3px letter-spacing on display headers (Trajan Pro /
  Cinzel chain via font-family stack — falls back to weighted serif
  on machines without the carved face)
- Serif prose chain unchanged (Iowan Old Style → Hoefler → Georgia)
- Drop cap on the first paragraph of each chapter — small literary
  flourish in oxblood-bronze

Ornament:
- Inline SVG knotwork divider (`ornament()` macro) — two flanking
  circles + an interlace curve between them, used as section breaks.
  Below the welcome h1; can be sprinkled wherever a visual register
  break helps.

Pages adjusted:
- topbar: SKALD wordmark in 4-letterspaced display caps; thin
  oxblood underline accent; '+ new saga' nav button on the right
- sidebar: 'STORIES' → 'SAGAS' (one of cobb's earlier asks);
  story-row hover gets oxblood left-accent (not gold); per-status
  pill colors (complete=ok, generating/cleaning/auditing=warn,
  failed=crit, seed=warn)
- story detail: dedicated .story-actions row with '✦ continue this
  saga' (primary, oxblood-bordered button) and 'generation log →'
  link
- welcome panel: revised copy + ornament + 'begin a new one' link
- forms: bordered surface inputs, all-caps display labels, primary-
  styled submit buttons matching the continue-saga action

Mobile (max-width: 800px):
- Single-column grid; sidebar lays out above the main panel
- Sidebar wrapped in <details open> so it's collapsible (taps the
  'Sagas (N)' header). No JS — native HTML semantics.
- Chapter-list collapses word-count column on narrow
- Char-list goes single-column
- All sizes downscale (28px h1, 16px prose, etc)

Next pass after I screenshot: tune any contrast/spacing that looks
off on the actual render.
2026-05-13 12:43:37 -07:00
b08d6ee8bc v0.3 step 2: Orson Black soul + author-aware forge + skald continue CLI
Three new pieces lock the gen pipeline:

1. seeds/authors/orson-black.md — Orson Black's soul. ~2k words,
   strict-headings format per cobb's decision. Voice / Worldview /
   Specifics / Pet peeves / Sense of humor / Biography (fully
   fictional — synthetic literary persona; coal town Durham 1948,
   father died of pneumoconiosis at 14, two winters as welder's
   apprentice on the Tyne, etc) / Anchor authors (Orwell, McCarthy,
   DeLillo, Judt, Modiano) / Do / Don't.

2. skald-core::forge — author-aware. Forge::generate, ::cleanup,
   and ::audit now take Option<&AuthorWithRevision>. When Some:
   scaffold + soul composed and passed as SystemMode::Replace —
   the model BECOMES the author. When None: house neutral scaffold,
   Append mode, claude defaults stay. audit() always neutral,
   regardless of author. Real prompt templates ship for gen +
   cleanup (the prose-craft IP we were deferring) — scaffold has
   {{display_name}}, {{pass_directive}}, {{soul}} substitutions,
   plus separate gen/cleanup directive blocks.

3. skald continue --story <uuid> [--author SLUG] [--direction
   STR] [--target-words N] [--recent N] [--skip-audit] — the
   pipeline CLI:

   load story → resolve author (--flag wins, else story.author_id,
   else None) → pin author/revision onto story → assemble
   ContinuationContext from parent chain → run gen pass →
   parse heading + insert chapter + passages → run cleanup pass
   on the draft → replace chapter body + passages → run audit
   pass (parent prose vs new chapter vs bible) → parse JSON
   findings into audit_findings table → status flow
   seed→generating→cleaning→auditing→complete.

   Plus skald authors seed --slug --display-name --tagline --file
   --note for loading souls from disk into the DB.

End-to-end testable: seed Orson Black, create a sequel stub via
web or SQL, fire 'skald continue' against it. Coast-Down chapter 8
in Orson's voice is the smoke test.
2026-05-13 12:06:28 -07:00
713ba41977 v0.3 step 1: migration 0004 + authors module + web form panels
Migration 0004 — authors + author_revisions + stories.author_id +
stories.author_revision_id + stories.cross_story_memory +
author_corpus. Soul versioning built in from day one per cobb's
locked decisions:
- authors.id immutable identity (slug + display_name + tagline + model)
- author_revisions tracks each soul revision with n monotonic
- Partial unique index 'idx_author_revisions_current' enforces
  exactly one is_current=true per author
- stories.author_revision_id pins to the exact soul used at gen
  time (so 'this was the Orson Black active when chapter 8 was
  written' is always recoverable)
- author_corpus tracks 'authored' + 'read' relationships for the
  v0.3 cross-story memory toggle

skald-core::authors module — CRUD: get_by_slug,
get_with_current_revision, get_current_revision, get_revision,
create_or_get (idempotent), add_revision (transactional, demotes
prior is_current=true), assign_to_story (also touches
author_corpus).

Web v0.1 forms (the second feedback bucket — 'no way to make new
stories', 'no options for sequels'): handlers + form panels +
POST routes for /stories/new and /stories/:id/continue. Both
create a story stub with status='seed'; actual generation will be
fired by 'skald continue' (next commit) walking seed rows.

Norse visual revamp + mobile collapse deferred — vetting full gen
is the priority per cobb's 'green light for v0.3'. Coming back to
the aesthetic after the pipeline works end-to-end against a real
Orson Black-authored Chapter 8 of Coast-Down.
2026-05-13 12:01:29 -07:00
7187bf5ace web: v0.1 read-only inspector — story list + drill into bible/chapters/runs
Read-only inspector built on axum + maud (per CLAUDE.md locked
stack). No JS, no htmx yet (v0.2). Single inline stylesheet:
dark serif aesthetic — looks like a writer's tool, not a
developer's CRUD app.

Routes:
- GET /                                      — welcome panel
- GET /stories/:id                           — story detail
- GET /stories/:id/chapters/:n               — chapter prose + summary
- GET /stories/:id/runs                      — generation log

Sidebar always shows the story list with chapter count, word
total, summary coverage ratio (e.g. '5/7 summ'), status badge.

Story detail panel:
- metabar (status / chapter count / word count / character count
  / canon fact count / 'updated 3h ago' / generation-log link)
- Chapters list with summary-present indicators (✓ summary / ○ no
  summary)
- Bible — characters (split real / fictional, key facts truncated
  to 220 chars)
- Bible — canon (collapsible <details> per category)

Chapter view:
- Summary aside box (if generated; otherwise CLI hint)
- Full prose body, paragraph-split, serif typography, 68ch column

Generation log view:
- table of every gen/cleanup/audit/summary run for the story,
  oldest to newest, with status colored (succeeded/failed/running)

Wired into 'skald serve' alongside /health.

Smoke test: http://lucy:7780/ when image redeploys.
2026-05-13 11:41:41 -07:00
e1e782177d vendor/clawdforge: refresh SDK — adds SystemMode::Replace for personas
Re-vendored from clawdforge@d4c3a9d. RunRequest now carries
`system_mode: Option<SystemMode>` where SystemMode is Append (the
default, current behavior — append to claude's base prompt) or
Replace (new — replaces claude's base prompt entirely).

Replace mode is the unlock for v0.3 author personas: Orson Black /
Bay / Kayos as authors can't have Claude's default helpful-honest
defaults bleeding through. Replace makes the model BECOME the
persona instead of 'Claude playing the persona.'

Existing skald::summarize call stays on default (Append) — the
summarizer is more 'tool-use over text' than persona, and Claude's
defaults help there. The gen + cleanup passes will switch to
Replace once authors are wired (next step).
2026-05-13 11:30:13 -07:00
4230aa472f docs/authors.md: v0.3 design brainstorm — authors as personas with souls
Captures the late-session design pivot:
- Authors live in DB as named entities with soul.md-style markdown
- Soul replaces opus's default system prompt via clawdforge (append
  today, replace via clawdforge enhancement later)
- Per-story cross_story_memory toggle for cross-corpus pulls
- Audit pass STAYS NEUTRAL — authors write/revise, audit checks
- Schema sketch (migration 0004), soul template, 4 seed-author proposals
- 6 open questions for cobb to decide

Doc is IN PROGRESS — will lock once cobb steers on the open
questions, then migration + code follow.
2026-05-13 11:20:43 -07:00
d6cb0b6df8 context: split coverage into prose_coverage + chapter_coverage
The old parent_coverage was raw-prose / parent-words — a signal of
'how much actual prose opus is reading.' But the more actionable
signal is 'is every chapter represented somehow' which sits at 1.0
for any parent with summaries (or placeholders) for older chapters.

Add chapter_coverage = 1.0 when every chapter has either a summary
or full-recent-prose row in the context. Keep prose_coverage as
the precise raw-words metric for ops that care about token budget.

Deprecate parent_coverage with a one-release shim (renames to
prose_coverage).

show_context CLI prints both percentages.
2026-05-13 11:18:31 -07:00
402b257ed0 serve: graceful-shutdown bug — was exit-looping every 15s
The serve loop wrapped axum::serve in tokio::time::timeout(15s),
which caps the WHOLE serve future, not just the shutdown drain.
Net effect: skald-serve cleanly returned Ok after 15 seconds every
time, docker restart picked it up, container went through the
exit-loop. Made any long-running docker exec (like summarize, with
opus calls that take 60-180s) racy at best, dead at worst — the
embedded postgres got 'database system was not properly shut down'
every 15s on the dot.

Fix: move the 15s deadline INSIDE the shutdown future. axum::serve
runs forever; the shutdown future fires on SIGTERM/SIGINT, then
gives in-flight requests 15s, then forces exit. Container only
goes down on a real signal.

Same bug exists in cwho-panel (copy-pasted from there). Fixing
there in a separate commit.
2026-05-13 10:56:22 -07:00
39e991240a summarize: first real forge call — generate per-chapter summaries
skald summarize --story <uuid> walks every chapter without an
existing summary, calls Forge::summarize() (clawdforge → opus →
~250 words of plot/character/setting/threads), and inserts the
result into chapter_summaries.

Side effects:
- generation_runs row per chapter (kind='summary', status flow
  running → succeeded|failed). Errors update the row + bail; happy
  path closes it with ended_at + tokens.
- ON CONFLICT (chapter_id) means re-running with --force replaces
  the previous summary cleanly.

CLI:
  skald summarize --story <uuid>           # only-missing
  skald summarize --story <uuid> --force   # re-summarize all

Reads from env (loaded by skald.env in the container):
  CLAWDFORGE_URL    — base URL of clawdforge HTTP service
  CLAWDFORGE_TOKEN  — app-level bearer (per-app, not the admin token)
  SKALD_MODEL       — defaults to 'opus'

This is the first subcommand that actually exercises the forge.
Unlocks ContinuationContext::assemble's coverage metric (was stuck
at 24%% on Coast-Down because the 5 placeholder summaries don't
actually carry the prose). After running summarize against
Coast-Down: coverage should jump to ~100%% and the context blob
for any sequel becomes fully canon-faithful without dragging the
full ~21k words of earlier-chapter prose along.

Forge prompt template for summarize ships REAL (not stubbed) — it's
the simplest pass and has a well-defined shape. The gen/cleanup/
audit prompts remain stubs pending the deeper prose-craft session.
2026-05-13 10:42:51 -07:00
b32938ef43 dockerfile: copy vendor/ during cache layer (path-dep needs full crate) 2026-05-13 10:30:58 -07:00
5b418369c0 context: assemble DB→opus blob + skald show-context CLI
skald-core::context is the bridge between 'rows in postgres' and
'prompt-ready markdown blob.' ContinuationContext::assemble(pool,
parent_story_id, recent_n) pulls:

  - parent story meta (title, series, total word count)
  - characters split real / fictional
  - canon_facts grouped by category
  - chapter summaries for everything older than the recent window
  - FULL prose for the last recent_n chapters

render_markdown() formats it with the most-condensed data first
(characters, canon) and the richest detail last (recent chapter
prose). Opus reads it linearly so by the time it's writing the new
chapter, the previous chapter's prose is freshest in its context
window.

The 'continuation reads ≥85% of parent' rule lands here via
parent_coverage() which counts recent prose + summaries-as-proxy
(250 words / summary) against parent word_count. The web UI / CLI
can warn before firing a gen pass if coverage is below threshold.

New CLI subcommand:

  skald show-context --story <uuid> --recent <N>

Assembles + prints the blob to stdout (eprintln'd stats summary
goes to stderr). No LLM call — pre-flight inspection so we see
what would be sent before paying for it. Useful for prompt-eng
work in the next session.

Module structure now:

  skald-core/
    config.rs     ForgeConfig
    context.rs    ContinuationContext (new)
    db.rs         connect_and_migrate
    forge.rs      Forge — three-pass orchestration
    ingest.rs     markdown parser
    models.rs     row types
    lib.rs        MIGRATOR + module exports

  skald/
    main.rs       clap CLI
    serve.rs      axum + /health + migrations
    import.rs     skald import-markdown
    show_context.rs  skald show-context  (new)
2026-05-13 10:30:16 -07:00
f71b533e52 v0.2 scaffold: vendor clawdforge SDK + forge module + Whisper plan
The Rust SDK already existed at Sulkta-Coop/clawdforge clients/rust/ — async,
reqwest-based, bearer-auth, exposes Client::run() + Session for multi-turn.
Vendoring it into vendor/clawdforge so skald is self-contained: no
git-submodule + no needing the clawdforge repo cloned next to skald.
Trade-off accepted: updates require manual re-copy until both sides
stabilize and we publish to a private cargo registry.

What landed:

- vendor/clawdforge/ — full SDK source from Sulkta-Coop/clawdforge HEAD.
  Pinned in skald-core/Cargo.toml as a path dep.
- skald-core/src/forge.rs — three-pass orchestration shell. Forge wraps
  clawdforge::Client; generate() / cleanup() / audit() each build a
  RunRequest with the right system prompt + model alias (always opus),
  call client.run(), return a PassOutput.
  Prompt templates are TODO stubs (SYSTEM_GEN_TODO etc) — filling in the
  actual prose-craft prompts is its own deep session.
- skald-core/src/config.rs — ForgeConfig { base_url, app_token, model }.
  Resolved by the binary from env (CLAWDFORGE_URL + CLAWDFORGE_TOKEN);
  lib stays env-agnostic.
- skald-core::AuditFinding + AuditResponse — parse shape for what the
  third-Opus canon audit returns, ready to map onto audit_findings rows.
- docs/tts-pipeline.md — full plan for v0.2 narration + post-TTS audit
  chain. Whisper-large-v3 STT does text-to-text verification on every
  render; an optional Gemini Flash audio pass catches subjective issues
  (prosody, tone) Whisper can't see. Reroll loop on crit findings.

What's still stubbed:

- Prompt templates in forge.rs (gen / cleanup / audit) — placeholders
  that describe the role but don't constrain output shape yet.
- context.rs (assemble the LLM context blob from DB rows) — entire module
  TBD.
- No CLI subcommand yet for invoking forge — that comes after context.rs.

Naming note: in Rust 2024 'gen' is a reserved keyword (for generators),
so the method is Forge::generate(), not Forge::gen().
2026-05-13 10:18:56 -07:00
4a91e0738d schema: narration_findings — audio-layer audit table
Closes the TTS schema layer. The v0.2 render pipeline auto-runs an
audit chain after each chapter narration:

  F5 render → narration_runs (succeeded)
    → ffmpeg chunk into ~30s windows
    → Whisper-large-v3 STT each chunk
    → word-level diff vs source chapter text
    → mismatches → narration_findings (kind=pronunciation|skip|insert)
    → ffmpeg silence/clip detect → narration_findings (kind=glitch)
    → (optional) Gemini Flash audio review pass
      → narration_findings (kind=prosody|tone)
    → unresolved crits trigger automatic re-roll with new seed

Distinct from audit_findings: that table is canon/continuity at the
text layer, populated by the third-Opus canon-audit pass.
narration_findings is audio-quality only, populated by detectors
that consume the rendered WAV.

The 'detector' field captures which model produced the finding so
we can tune thresholds per detector when one over- or under-flags.

cobb's audio agent intuition was right: STT-and-diff catches the
'name came out wrong' case airtight, and a separate audio-native
LLM call catches the subtler 'this sentence sounded weird' cases
Whisper can't see.
2026-05-13 10:10:04 -07:00
465c94b745 schema: voices + pronunciation_overrides + narration_runs (v0.2 prep)
TTS layer landed as schema-only — synthesis pipeline ships in v0.2.
Putting the tables in v0.1 means imports already carry the right
shape; we won't need a 'migrate every existing story' pass later.

Decisions locked 2026-05-13:
- Engine: F5-TTS (best 8GB FOSS option, mid-2026 SOTA)
- Default voice source: LJ Speech (Linda Johnson, PD released
  specifically for TTS training — airtight for sharing/uploading
  generated audio. The 'AI-consent-released' license posture is
  the difference between 'should be fine' and 'definitely fine.')
- Variety voices: Hi-Fi TTS speaker IDs (Apache 2.0, same consent
  shape). LibriVox is optional but never default.
- Pronunciation overrides DB layer (story-scoped + global) to fix
  proper-noun mispronunciation — the actual TTS-quality gap on
  Cobb's bar of 'must not wake me up.' Pre-pass with Opus extracts
  proper nouns + IPA, operator verifies, table caches forever.

Tables:
- voices — name, license, reference_path/text, sample_rate, default flag
- pronunciation_overrides — story-scoped or global, IPA/arpabet
- narration_runs — TTS audit trail mirroring generation_runs
- stories.preferred_voice_id FK

Unique constraints:
- one default voice (partial index)
- one row per (story, word) override
- one global row per word
2026-05-13 10:07:32 -07:00
f575ad3722 scaffold v0.1: postgres+pgvector inside-container, schema, markdown ingest, CLI
Skald is a generic story-writer. The database is the product; the
binary is the tooling. Everything story-specific lives in rows, not
in code. cwho's monorepo + binary-per-role pattern transplanted to
this domain.

What this commit ships:
- Cargo workspace (resolver=3, edition 2024): skald-core (lib) +
  skald (bin)
- Migration 0001: stories, characters, canon_facts, chapters,
  chapter_summaries, passages (vector(1536)), generation_runs,
  audit_findings, tags. pgvector + pg_trgm extensions. ivfflat
  index deferred until we have data (post-import the first ~1k
  passages and add the index).
- skald-core::ingest — markdown parser for the cwho/coast-down shape:
  '# Title' → '## Chapter N — date' headings → '# Continuity Bible'
  section with character roster (real + fictional sub-sections) +
  setting / mystery / historical / liberty / hook sub-sections.
  Decomposed into structured rows; original bullet body preserved
  in key_facts/body fields for fidelity. 6 unit tests cover the
  shape.
- skald-core::db — Postgres connection pool + migration runner.
- skald-core::models — row types via sqlx::FromRow.
- skald binary — clap CLI: 'serve' (http + migrations) and
  'import-markdown' (one-shot ingest).
- Dockerfile — multi-stage: rust:1.95-bookworm builder, pgvector/
  pgvector:pg17 runtime, tini under PID 1, custom entrypoint.sh
  that boots embedded postgres then execs skald serve.
- compose.yml — singleton container, postgres data in volume,
  story corpus mounted read-only at /seed.

Decisions locked 2026-05-13:
1. DB in same container 'till we have a real working tool' (cobb)
2. postgres+pgvector (NOT sqlite) — keeps semantic-search story
3. Network-not-socket connection (postgresql://localhost:5432) from
   day one so future split is config-only, not code-rewrite

Not yet wired:
- Web UI
- clawdforge calls (gen → cleanup → canon-audit pipeline)
- Embedding pass
- TTS sidecar
2026-05-13 09:04:28 -07:00