From 8a410b0c3c52300cea695db03d3f9cdd16cb9d36 Mon Sep 17 00:00:00 2001 From: Sulkta Date: Sun, 28 Jun 2026 13:25:25 -0700 Subject: [PATCH] docs: public-facing README + generic configuration docs Rewrite the README for external users, neutralize example identities in code comments, generalize the OIDC group note, and simplify the gitleaks workflow header. --- .forgejo/workflows/gitleaks.yml | 16 +---- README.md | 119 +++++++++++++++++++++++++------- cauldron/aggregator.py | 6 +- cauldron/bulk_sterilize.py | 2 +- cauldron/config.py | 2 +- cauldron/consolidate_foods.py | 4 +- cauldron/discover_recipes.py | 2 +- cauldron/forge.py | 4 +- cauldron/oidc.py | 11 ++- cauldron/recipe_index.py | 4 +- cauldron/server.py | 4 +- cauldron/sterilizer.py | 2 +- 12 files changed, 118 insertions(+), 58 deletions(-) diff --git a/.forgejo/workflows/gitleaks.yml b/.forgejo/workflows/gitleaks.yml index 634152b..68b7eeb 100644 --- a/.forgejo/workflows/gitleaks.yml +++ b/.forgejo/workflows/gitleaks.yml @@ -1,16 +1,6 @@ -# .forgejo/workflows/gitleaks.yml -# -# Sulkta canonical gitleaks workflow. Drop a copy into every public repo at -# `.forgejo/workflows/gitleaks.yml` after the Forgejo act_runner is registered -# (task #295). -# -# Pairs with the pre-receive hook installed on every bare repo — that one is -# the strict enforcement layer (rejects the push); this one provides the -# per-PR red ✗ that branch-protection rules can require before merge. -# -# Layer 1 (this workflow): visible per-PR status, can be a required check. -# Layer 2 (pre-receive hook): strict enforcement at the server. -# Layer 3 (scheduled cron sweep): nightly full-history sweep across all repos. +# Scans the repository for committed secrets with gitleaks on every push and +# pull request. Make it a required status check in branch-protection rules to +# block merges that introduce credentials. name: gitleaks diff --git a/README.md b/README.md index 1b8daad..60dcee5 100644 --- a/README.md +++ b/README.md @@ -1,35 +1,90 @@ -# cauldron +# Cauldron -Mealie-backed meal planner + shopping-list aggregator. Wraps a Mealie -instance with: an ingredient sterilizer (free-form quantities → structured -parses), a weekly meal-plan generator, and a household shopping list that -collapses cross-recipe duplicates. +Cauldron is a self-hosted companion app for [Mealie](https://mealie.io) that +adds AI-assisted meal planning and a smart, household-shared shopping list. +Mealie stays the source of truth for your recipes, plans and shopping lists; +Cauldron layers extra automation on top and stores only per-user preferences +and cached metadata. + +## Features + +- **Ingredient sterilizer** — turns free-form quantities ("about 2 cups", + "a couple handfuls") into structured `{qty, unit, food}` parses and writes + them back to Mealie, so downstream tooling has clean data to work with. +- **Weekly meal-plan generator** — builds a per-week plan from your recipe + library, honouring pinned picks, dietary targets (calories / macros) and + allergen exclusions. +- **Household shopping list** — aggregates ingredients across every recipe in + a plan and collapses duplicates into single lines (e.g. "2 cups rice + + 1.25 lb rice" → "~2 lb rice") using a USDA-derived food-density table. +- **Recipe discovery, dedupe and enrichment** — scrape-and-import new recipes, + cluster and merge duplicates, and enrich recipes with structured metadata + for smarter planning. +- **Multi-user, OIDC-secured** — every user connects their own Mealie token + (encrypted at rest); an optional operator tier exposes the destructive admin + tools. ## Stack -- Flask + gunicorn, Python 3.12 -- Authentik (or any OIDC provider) for sessions -- MariaDB for per-user prefs + Fernet-encrypted Mealie tokens -- [clawdforge](https://github.com/Sulkta-Coop/clawdforge) for the AI layer +- Flask + gunicorn (Python 3.12+) +- Any OpenID Connect provider for authentication (tested with Authentik) +- MariaDB for per-user preferences and Fernet-encrypted Mealie tokens +- An LLM runner reachable over HTTP for the AI features. This project targets + [clawdforge](https://github.com/Sulkta-Coop/clawdforge), but any service + exposing the same `POST /run` contract works. -Mealie remains the source of truth for recipes / plans / shopping lists. -Cauldron stores per-user prefs + cached metadata only. +## Quick start -## Run +Cauldron ships as a container and is configured entirely through environment +variables. ```bash -cp .env.example .env # fill in secrets +cp .env.example .env # then fill in the blanks (see Configuration) docker compose up -d --build curl http://localhost:7790/healthz ``` -`CAULDRON_ENV_FILE=/path/to/secrets.env docker compose up -d` if your env -file lives outside the repo. +If you prefer to keep your secrets file outside the repo: + +```bash +CAULDRON_ENV_FILE=/path/to/cauldron.env docker compose up -d +``` + +### Requirements + +- A running Mealie instance and an API token. +- An OIDC provider with a client configured for Cauldron's redirect URI. +- A MariaDB database and user. +- An LLM HTTP runner for the AI features (sterilize / plan / discover / + enrich). Without it the app still runs; AI-backed endpoints return an error. + +## Configuration + +All settings are read from the environment. See `.env.example` for the full, +documented list. The most important variables: + +| Variable | Purpose | +| --- | --- | +| `SECRET_KEY` | Flask session signing key | +| `BIND_HOST` / `BIND_PORT` | Listen address (defaults `0.0.0.0:7790`) | +| `MEALIE_BASE_URL` | Base URL of your Mealie instance | +| `MEALIE_API_TOKEN` | System Mealie token for admin batch operations | +| `CLAWDFORGE_URL` / `CLAWDFORGE_TOKEN` | LLM runner endpoint + auth | +| `OIDC_ISSUER` / `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` / `OIDC_REDIRECT_URI` | OpenID Connect login | +| `DB_HOST` / `DB_PORT` / `DB_NAME` / `DB_USER` / `DB_PASSWORD` | MariaDB connection | +| `CAULDRON_FERNET_KEY` | Key used to encrypt per-user Mealie tokens at rest | +| `ADMIN_BEARER` | Bearer token for the batch (`/api/...`) endpoints | +| `CAULDRON_ADMIN_SUBS` | OIDC subjects granted the operator-tier admin tools | +| `CAULDRON_BASE_URL` / `CAULDRON_BEHIND_TLS` / `CAULDRON_TRUSTED_PROXIES` | Public-deploy hardening (CSRF origin guard, HSTS, secure cookies, trusted-proxy `X-Forwarded-*` handling) | + +When deployed behind a reverse proxy, set `CAULDRON_BASE_URL`, +`CAULDRON_BEHIND_TLS=true` and list the proxy peer in +`CAULDRON_TRUSTED_PROXIES` so forwarded headers are honoured safely. ## Endpoints ``` -GET /healthz liveness + clawdforge upstream +GET /healthz liveness + LLM-runner upstream check GET /login /auth/callback OIDC flow GET /me account + integration status GET /plan /list household plan + shopping list @@ -40,21 +95,37 @@ POST /api/sterilize/apply/ (admin bearer) write parses back Admin-bearer endpoints expect `Authorization: Bearer $ADMIN_BEARER`. -## Layout +## Project layout ``` cauldron/ - config.py env-driven config - forge.py clawdforge HTTP client + config.py env-driven configuration + forge.py LLM-runner HTTP client mealie.py Mealie API client sterilizer.py ingredient parse + apply pipeline - aggregator.py cross-recipe shopping aggregator - server.py Flask app + aggregator.py cross-recipe shopping-list aggregator + foods.py USDA-seeded food catalogue + density lookup + recipe_index.py local fuzzy recipe search index + server.py Flask app + routes + data/ seed data (USDA SR Legacy foods) scripts/ - build_foods_seed.py USDA → foods seed - clean_foods_seed.py clawdforge-curated cleanup pass + build_foods_seed.py build the foods seed from a USDA dataset + clean_foods_seed.py LLM-curated cleanup pass over the seed +tests/ unit tests ``` +## Development + +```bash +pip install -r requirements.txt +python -m pytest # run the test suite +``` + +## Contributing + +Issues and pull requests are welcome. Please keep changes focused, add tests +for new behaviour, and make sure `pytest` passes before opening a PR. + ## License -MIT. +MIT — see [LICENSE](LICENSE). diff --git a/cauldron/aggregator.py b/cauldron/aggregator.py index a8a7e99..2fd02b7 100644 --- a/cauldron/aggregator.py +++ b/cauldron/aggregator.py @@ -1,6 +1,6 @@ """Unit-aware shopping list aggregator. -Alice's killer feature: take ingredients from N recipes, return a single +The killer feature: take ingredients from N recipes, return a single consolidated shopping list with per-food totals. Examples: @@ -290,9 +290,9 @@ def _aggregate_one_food( # qty=None safety net: if every contributor was a no-qty ingredient # (Mealie's parser couldn't extract a number), nothing else above # produced a line. Emit a placeholder so the food APPEARS on the - # shopping list — Bob still needs to know to buy onions even if the + # shopping list — the cook still needs to know to buy onions even if the # recipe just said "1 onion, chopped". UI surfaces this as a - # "qty unspecified" hint, nudging Alice to run sterilize. + # "qty unspecified" hint, nudging the user to run sterilize. if no_qty_items and not lines: lines.append(ShoppingLine( food=food, qty=None, unit="ea", diff --git a/cauldron/bulk_sterilize.py b/cauldron/bulk_sterilize.py index 8e14d23..777171e 100644 --- a/cauldron/bulk_sterilize.py +++ b/cauldron/bulk_sterilize.py @@ -33,7 +33,7 @@ log = logging.getLogger(__name__) def _ingredient_needs_sterilizing(ing: dict) -> bool: """Heuristic: an ingredient row needs work if it has display/note content but no resolved food. Already-parsed rows (food.id present) are skipped - so we don't waste clawdforge calls or risk regressing Alice's manual + so we don't waste clawdforge calls or risk regressing the user's manual cleanup.""" food = ing.get("food") or {} food_name = food.get("name") if isinstance(food, dict) else None diff --git a/cauldron/config.py b/cauldron/config.py index 57933b6..383d530 100644 --- a/cauldron/config.py +++ b/cauldron/config.py @@ -11,7 +11,7 @@ class Config: mealie_api_url: str # internal URL cauldron uses for HTTP calls (LAN-internal) mealie_public_url: str # external URL shown to users for token-mint UI - mealie_api_token: str # system token (Alice's "Cauldron" token, used for admin batch ops) + mealie_api_token: str # system token (the service "Cauldron" token, used for admin batch ops) clawdforge_url: str clawdforge_token: str diff --git a/cauldron/consolidate_foods.py b/cauldron/consolidate_foods.py index 4f5f48b..c578405 100644 --- a/cauldron/consolidate_foods.py +++ b/cauldron/consolidate_foods.py @@ -81,7 +81,7 @@ def _foods_in_household(mealie: Mealie, household_id: str) -> list[dict]: elif not hh: # Some Mealie versions don't tag foods with householdId at the # food level — they're group-scoped instead. In that case, we - # consolidate across the whole group (Alice is admin so writes + # consolidate across the whole group (an admin user can write # land regardless). out.append(f) return out @@ -94,7 +94,7 @@ def _cluster( ) -> list[list[dict]]: """Pair-based: emit one 2-food candidate per (i, j) where token_set_ratio >= threshold. Replaces the original single-link agglomerative which - produced a 50+ food megacluster on Alice's catalog by chaining weak + produced a 50+ food megacluster on a real catalog by chaining weak similarities (`2% milk` → `acai berry` → `acai berry juice` → ...). Each emitted pair is a clean Sonnet-decision unit — easier prompt, diff --git a/cauldron/discover_recipes.py b/cauldron/discover_recipes.py index 69509c7..c53c618 100644 --- a/cauldron/discover_recipes.py +++ b/cauldron/discover_recipes.py @@ -11,7 +11,7 @@ Pipeline per URL: Same daemon-thread + cancel + stuck-recovery pattern as enrich/sterilize. Seed sources are hardcoded URL lists per source_seed (allrecipes-popular, -bbc-popular, smitten-kitchen-recent, ...). Alice supplies a seed name OR +bbc-popular, smitten-kitchen-recent, ...). The user supplies a seed name OR a literal list of URLs via the admin endpoint. Either way, the runner walks the list, scrape→insert→enrich each, and emits progress. """ diff --git a/cauldron/forge.py b/cauldron/forge.py index d9ec08a..8042b55 100644 --- a/cauldron/forge.py +++ b/cauldron/forge.py @@ -261,7 +261,7 @@ class Forge: fp = meta.get("flavor_profile") or [] if fp: extras.append("flavor:" + "/".join(fp[:3])) - # Kid-fit signal for households with kids (Alice has Dana + Evan) + # Kid-fit signal for households with kids (e.g. two children) kf = meta.get("kid_friendly_score") if isinstance(kf, int) and kf > 0: extras.append(f"kid={kf}") @@ -362,7 +362,7 @@ class Forge: "\nPICKER PROFILES — per-member historical picking patterns:\n" + "\n".join(lines) + "\n\n" "Use these to bias AI-chosen slots toward each member's " - "preferences. e.g., if Alice's profile shows cuisines=[asian:6, " + "preferences. e.g., if a user's profile shows cuisines=[asian:6, " "mexican:4] and proteins=[chicken:8], lean toward asian-chicken " "recipes for the AI-filled slots when other constraints permit. " "Picks still take precedence over profile bias.\n" diff --git a/cauldron/oidc.py b/cauldron/oidc.py index 38fd4be..281350b 100644 --- a/cauldron/oidc.py +++ b/cauldron/oidc.py @@ -1,12 +1,11 @@ """Authentik OIDC integration via Authlib. -Cauldron is gated to the Authentik 'Cauldron' application, which is bound -to the 'Example Household' group. Authentik enforces the group membership; we -just trust the userinfo response. +Cauldron is gated to an OIDC application that is bound to whichever group +you choose in your identity provider. The provider enforces the group +membership; we just trust the userinfo response. -We use 'sub_mode=user_email' on the Authentik provider, so the OIDC `sub` -claim is the user's email — stable, human-readable, and matches our -existing Sulkta SSO pattern. +We use a provider configured so the OIDC `sub` claim is the user's email — +stable, human-readable, and a convenient stable key for per-user prefs. """ from authlib.integrations.flask_client import OAuth diff --git a/cauldron/recipe_index.py b/cauldron/recipe_index.py index 8a9389a..8178543 100644 --- a/cauldron/recipe_index.py +++ b/cauldron/recipe_index.py @@ -84,7 +84,7 @@ def _parse_dt(s): def refresh_household_index(*, mealie_client, db, household_id: int) -> int: """Pull every recipe in the user's household, flatten, replace the - local index. Returns row count. Mealie's perPage caps high (Alice has + local index. Returns row count. Mealie's perPage caps high (a large library can have 226 recipes — single page works); we paginate just in case.""" all_rows = [] page = 1 @@ -115,7 +115,7 @@ def _norm(s: str | None) -> str: def search_index(rows: list[dict], q: str, *, limit: int = 80) -> list[dict]: """Score each row against the query across multiple fields and return - top-N. Alice-style: typo-tolerant, phrase-aware, multi-field weighted. + top-N. Fuzzy matching: typo-tolerant, phrase-aware, multi-field weighted. Field weights (max-of-pre-weighted scores wins): name × 1.00 diff --git a/cauldron/server.py b/cauldron/server.py index d5f3693..ef058c6 100644 --- a/cauldron/server.py +++ b/cauldron/server.py @@ -53,7 +53,7 @@ forge = Forge( default_model=cfg.default_model, default_timeout=cfg.default_timeout_secs, ) -# System-tier Mealie client (Alice's "Cauldron" token; admin batch ops only) +# System-tier Mealie client (the service "Cauldron" token; admin batch ops only) system_mealie = Mealie(base_url=cfg.mealie_api_url, api_token=cfg.mealie_api_token) system_sterilizer = Sterilizer(mealie=system_mealie, forge=forge, model=cfg.default_model) @@ -142,7 +142,7 @@ def create_app() -> Flask: # One-time backfill: re-key the legacy cauldron_foods (USDA + curated) # densities into cauldron_food_metadata, keyed by Mealie food.id. Runs # only when the new metadata table is empty. The system MEALIE_API_TOKEN - # may be expired (Alice's "Cauldron" token was minted long ago); fall + # may be expired (the service "Cauldron" token may have been minted long ago); fall # back to any stored per-user token since household admins can list # all foods anyway. def _resolve_backfill_mealie() -> "Mealie": diff --git a/cauldron/sterilizer.py b/cauldron/sterilizer.py index e161106..d97bc1e 100644 --- a/cauldron/sterilizer.py +++ b/cauldron/sterilizer.py @@ -2,7 +2,7 @@ structured (qty, unit, food, note) so shopping-list aggregation works. Why this exists: Mealie has its own CRF parser, but it's mediocre and produces -inconsistent results. Alice's hand-typed recipes have lots of "about 2 cups +inconsistent results. Hand-typed recipes have lots of "about 2 cups cooked white rice" / "1 small handful kale" / "a pinch of salt" etc. that slip past the parser. We send these to Sonnet via clawdforge and get back clean structured form.