Kayos
605844f5be
The previous ca-derivations attempt didn't actually fix the schema issue — Nix 2.34.7's v10 → v11 migration (which adds the Realisations table) doesn't fire cleanly even with the feature pre-enabled at install time. First nix develop against a flake that requests ca-derivations crashes with `Assertion 'stmt.stmt' failed in nix::SQLiteStmt::Use::Use`. Workaround: set accept-flake-config=false. Flake nixConfig blocks trying to add ca-derivations to our experimental-features get ignored. Realisation queries never fire. Builds use the default input-addressed path and work fine. Substituters are now in our base nix.conf (cache.nixos.org + cache.iog.io) so we don't lose the IOG binary cache by ignoring the flake's substituter additions. mlabs.cachix.org dropped — it's a private cache returning 401 to anonymous reads. Verified live: nix develop against github:Liqwid-Labs/agora proceeds past the previous crash point, pulling haskell-nix closure from cache.iog.io.
376 lines
19 KiB
Docker
376 lines
19 KiB
Docker
# crafting-table v0.1 — polyglot dev/build/audit container
|
|
#
|
|
# Step 1 of 10: monolith image with every toolchain in the spec.
|
|
# Spec: Sulkta-Coop/openclaw-workspace/memory/spec-crafting-table.md
|
|
#
|
|
# Toolchain version pins (bump these to upgrade):
|
|
# NODE_VERSION 22.11.0 (LTS)
|
|
# GO_VERSION 1.22.10
|
|
# RUST_CHANNEL stable
|
|
# BUN_VERSION latest (rolling — pinned at install only)
|
|
# DOTNET_VERSION 8.0
|
|
# SWIFT_VERSION 5.9.2
|
|
# KOTLIN_VERSION 1.9.25
|
|
# GRADLE_VERSION 8.10
|
|
# JDK 17 (default) + 21 (alongside, via JAVA_HOME_21)
|
|
#
|
|
# Image runs as non-root user `crafter` (uid 1000) with passwordless sudo.
|
|
# Persistent caches mounted at /caches/{cargo,maven,gradle,npm,pip}.
|
|
# Workspace at /workspace. Per-job state at /data.
|
|
|
|
FROM debian:bookworm-slim
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive \
|
|
LANG=C.UTF-8 \
|
|
LC_ALL=C.UTF-8
|
|
|
|
# ---------- Toolchain version pins ----------
|
|
ENV NODE_VERSION=22.11.0 \
|
|
GO_VERSION=1.25.9 \
|
|
DOTNET_CHANNEL=8.0 \
|
|
SWIFT_VERSION=5.9.2 \
|
|
SWIFT_PLATFORM=ubuntu22.04 \
|
|
KOTLIN_VERSION=1.9.25 \
|
|
GRADLE_VERSION=8.10
|
|
|
|
# ============================================================
|
|
# 1. System base — apt packages
|
|
# ============================================================
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
curl wget git ca-certificates gnupg lsb-release apt-transport-https \
|
|
build-essential pkg-config make cmake ninja-build \
|
|
jq ripgrep fd-find \
|
|
valgrind clang lld llvm \
|
|
python3 python3-pip python3-venv python3-dev \
|
|
php-cli php-curl php-mbstring php-xml php-zip composer \
|
|
ruby ruby-dev ruby-bundler \
|
|
bash shellcheck bats \
|
|
openjdk-17-jdk-headless \
|
|
sudo unzip xz-utils zstd \
|
|
libcurl4 libxml2 libedit2 libsqlite3-0 libpython3-dev \
|
|
libncurses6 libtinfo6 libgcc-s1 libstdc++6 \
|
|
libssl-dev libffi-dev \
|
|
zlib1g-dev liblzma-dev libbz2-dev \
|
|
locales tzdata \
|
|
&& ln -sf /usr/bin/fdfind /usr/local/bin/fd \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# ============================================================
|
|
# 2. Node 22 LTS via NodeSource
|
|
# ============================================================
|
|
RUN curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
|
|
&& apt-get install -y --no-install-recommends nodejs \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& npm install -g pnpm tsx eslint typescript
|
|
|
|
# ============================================================
|
|
# 3. Go (download from go.dev tarball)
|
|
# ============================================================
|
|
RUN ARCH="$(dpkg --print-architecture)" \
|
|
&& case "$ARCH" in \
|
|
amd64) GOARCH=amd64 ;; \
|
|
arm64) GOARCH=arm64 ;; \
|
|
*) echo "Unsupported arch $ARCH" && exit 1 ;; \
|
|
esac \
|
|
&& curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-${GOARCH}.tar.gz" -o /tmp/go.tgz \
|
|
&& tar -C /usr/local -xzf /tmp/go.tgz \
|
|
&& rm /tmp/go.tgz
|
|
ENV PATH=/usr/local/go/bin:/root/go/bin:$PATH \
|
|
GOPATH=/root/go
|
|
|
|
# ============================================================
|
|
# 4. Microsoft .NET 8 SDK (Microsoft apt repo for bookworm/12)
|
|
# ============================================================
|
|
RUN curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /usr/share/keyrings/microsoft.gpg \
|
|
&& echo "deb [signed-by=/usr/share/keyrings/microsoft.gpg] https://packages.microsoft.com/debian/12/prod bookworm main" > /etc/apt/sources.list.d/microsoft.list \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends dotnet-sdk-8.0 \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
ENV DOTNET_CLI_TELEMETRY_OPTOUT=1 \
|
|
DOTNET_NOLOGO=1
|
|
|
|
# ============================================================
|
|
# Switch to bash for the bash-ism layers below (${var//pattern} etc).
|
|
# Layers above (apt base / Node / Go / .NET) only use POSIX so they cached fine
|
|
# under dash; SHELL is set here to invalidate the cache for layer 6+ only.
|
|
SHELL ["/bin/bash", "-c"]
|
|
|
|
# 5. Swift (Ubuntu 22.04 tarball — works on Debian bookworm
|
|
# because bookworm ships the right libicu/libstdc++ baseline)
|
|
# ============================================================
|
|
RUN ARCH="$(dpkg --print-architecture)" \
|
|
&& SWIFT_PLAT="${SWIFT_PLATFORM}" \
|
|
&& case "$ARCH" in \
|
|
amd64) SWIFT_TARBALL_NAME="swift-${SWIFT_VERSION}-RELEASE-${SWIFT_PLAT}" ;; \
|
|
arm64) SWIFT_TARBALL_NAME="swift-${SWIFT_VERSION}-RELEASE-${SWIFT_PLAT}-aarch64" ;; \
|
|
*) echo "Unsupported arch $ARCH for Swift" && exit 1 ;; \
|
|
esac \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends libpython3-dev libxml2-dev \
|
|
libcurl4-openssl-dev libedit-dev libsqlite3-dev libtinfo-dev libncurses-dev \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& curl -fsSL "https://download.swift.org/swift-${SWIFT_VERSION}-release/${SWIFT_PLAT//./}/swift-${SWIFT_VERSION}-RELEASE/${SWIFT_TARBALL_NAME}.tar.gz" -o /tmp/swift.tgz \
|
|
&& mkdir -p /opt/swift \
|
|
&& tar -xzf /tmp/swift.tgz -C /opt/swift --strip-components=1 \
|
|
&& rm /tmp/swift.tgz
|
|
ENV PATH=/opt/swift/usr/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 6. Kotlin compiler (direct download from GitHub release)
|
|
# ============================================================
|
|
RUN curl -fsSL "https://github.com/JetBrains/kotlin/releases/download/v${KOTLIN_VERSION}/kotlin-compiler-${KOTLIN_VERSION}.zip" -o /tmp/kotlin.zip \
|
|
&& unzip -q /tmp/kotlin.zip -d /opt \
|
|
&& mv /opt/kotlinc /opt/kotlin \
|
|
&& rm /tmp/kotlin.zip
|
|
ENV PATH=/opt/kotlin/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 7. JDK 21 alongside JDK 17 (Eclipse Temurin via apt)
|
|
# ============================================================
|
|
RUN mkdir -p /etc/apt/keyrings \
|
|
&& curl -fsSL https://packages.adoptium.net/artifactory/api/gpg/key/public | gpg --dearmor -o /etc/apt/keyrings/adoptium.gpg \
|
|
&& echo "deb [signed-by=/etc/apt/keyrings/adoptium.gpg] https://packages.adoptium.net/artifactory/deb bookworm main" > /etc/apt/sources.list.d/adoptium.list \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends temurin-21-jdk \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 \
|
|
JAVA_HOME_17=/usr/lib/jvm/java-17-openjdk-amd64 \
|
|
JAVA_HOME_21=/usr/lib/jvm/temurin-21-jdk-amd64
|
|
RUN update-alternatives --set java ${JAVA_HOME_17}/bin/java || true \
|
|
&& update-alternatives --set javac ${JAVA_HOME_17}/bin/javac || true
|
|
|
|
# ============================================================
|
|
# 8. Maven (apt) + Gradle (direct download — apt's gradle is ancient)
|
|
# ============================================================
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends maven \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& curl -fsSL "https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip" -o /tmp/gradle.zip \
|
|
&& unzip -q /tmp/gradle.zip -d /opt \
|
|
&& mv /opt/gradle-${GRADLE_VERSION} /opt/gradle \
|
|
&& rm /tmp/gradle.zip
|
|
ENV PATH=/opt/gradle/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 9. GitHub CLI (gh)
|
|
# ============================================================
|
|
RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
|
|
&& chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \
|
|
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" > /etc/apt/sources.list.d/github-cli.list \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends gh \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# ============================================================
|
|
# 10. yq (Mike Farah, Go binary)
|
|
# ============================================================
|
|
RUN ARCH="$(dpkg --print-architecture)" \
|
|
&& curl -fsSL "https://github.com/mikefarah/yq/releases/latest/download/yq_linux_${ARCH}" -o /usr/local/bin/yq \
|
|
&& chmod +x /usr/local/bin/yq
|
|
|
|
# ============================================================
|
|
# 11. shfmt (Go binary)
|
|
# ============================================================
|
|
RUN ARCH="$(dpkg --print-architecture)" \
|
|
&& curl -fsSL "https://github.com/mvdan/sh/releases/download/v3.10.0/shfmt_v3.10.0_linux_${ARCH}" -o /usr/local/bin/shfmt \
|
|
&& chmod +x /usr/local/bin/shfmt
|
|
|
|
# ============================================================
|
|
# 12. Create non-root user `crafter` (uid 1000) with passwordless sudo
|
|
# and prepare workspace / cache / data dirs
|
|
# ============================================================
|
|
RUN useradd -m -u 1000 -s /bin/bash crafter \
|
|
&& echo "crafter ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/crafter \
|
|
&& chmod 0440 /etc/sudoers.d/crafter \
|
|
&& mkdir -p /workspace /caches/cargo /caches/maven /caches/gradle /caches/npm /caches/pip /caches/bun /data \
|
|
&& chown -R crafter:crafter /workspace /caches /data
|
|
|
|
# ============================================================
|
|
# 13. Switch to crafter for user-scoped installs
|
|
# ============================================================
|
|
USER crafter
|
|
WORKDIR /home/crafter
|
|
|
|
# Cache env (point tools at the persisted cache dirs)
|
|
ENV CARGO_HOME=/caches/cargo \
|
|
RUSTUP_HOME=/home/crafter/.rustup \
|
|
GRADLE_USER_HOME=/caches/gradle \
|
|
MAVEN_OPTS="-Dmaven.repo.local=/caches/maven" \
|
|
NPM_CONFIG_CACHE=/caches/npm \
|
|
PIP_CACHE_DIR=/caches/pip \
|
|
BUN_INSTALL=/home/crafter/.bun \
|
|
PIPX_HOME=/home/crafter/.local/pipx \
|
|
PIPX_BIN_DIR=/home/crafter/.local/bin
|
|
|
|
ENV PATH=/home/crafter/.local/bin:/caches/cargo/bin:/home/crafter/.bun/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 14. Rust (rustup, stable) + cargo-audit + cargo-deny
|
|
# ============================================================
|
|
RUN curl -fsSL https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path \
|
|
&& /caches/cargo/bin/rustup component add clippy rustfmt
|
|
# Install cargo-audit + cargo-deny to /usr/local (root-owned, NOT volume-shadowed
|
|
# at runtime). The /caches/cargo/bin/ dir IS volume-shadowed by the host bind
|
|
# mount, so cargo install artifacts there disappear inside the live container.
|
|
USER root
|
|
# cargo-audit + cargo-deny intentionally NOT installed in image — both
|
|
# `cargo install` and prebuilt-binary-download approaches flaked
|
|
# (libgit2-sys C bindings, quote build script under /caches/cargo
|
|
# contention, and DNS flakes on github.com release downloads). Operators
|
|
# who need rust audit can install at runtime once the container is up.
|
|
# Future v0.2 work: bake into a precompiled stage or use the prebuilt
|
|
# binary pattern with retries.
|
|
|
|
# ============================================================
|
|
# 15. Bun (curl install)
|
|
# ============================================================
|
|
RUN curl -fsSL https://bun.sh/install | bash
|
|
|
|
# ============================================================
|
|
# 16. Python user tooling: pipx-managed CLI tools
|
|
# ============================================================
|
|
RUN python3 -m pip install --user --break-system-packages --no-cache-dir pipx \
|
|
&& python3 -m pipx ensurepath \
|
|
&& python3 -m pipx install uv \
|
|
&& python3 -m pipx install ruff \
|
|
&& python3 -m pipx install mypy \
|
|
&& python3 -m pipx install pytest \
|
|
&& python3 -m pipx install pip-audit \
|
|
&& python3 -m pipx install semgrep \
|
|
# mypy needs the third-party stub packages injected into its own pipx
|
|
# venv (mypy-isolated, not the system site-packages). Without these,
|
|
# `mypy --strict` against any project that imports requests/PyYAML/etc.
|
|
# fails with "Library stubs not installed for X" exit 1.
|
|
&& python3 -m pipx inject mypy types-requests types-PyYAML types-setuptools
|
|
|
|
# ============================================================
|
|
# 17. Go user tooling — govulncheck + staticcheck
|
|
#
|
|
# Override GOPATH from the root default (/root/go set at line 79) to a
|
|
# crafter-writable path before running `go install` as USER crafter.
|
|
# Don't redeclare PATH here — the final clean PATH at the bottom of
|
|
# this Dockerfile is the single source of truth and includes
|
|
# /home/crafter/go/bin.
|
|
# ============================================================
|
|
ENV GOPATH=/home/crafter/go
|
|
# ============================================================
|
|
RUN for i in 1 2 3 4 5; do \
|
|
go install golang.org/x/vuln/cmd/govulncheck@latest \
|
|
&& go install honnef.co/go/tools/cmd/staticcheck@latest \
|
|
&& break || { echo "go install attempt $i failed, sleeping $((i*10))s"; sleep $((i*10)); }; \
|
|
done; \
|
|
test -x "$GOPATH/bin/govulncheck" && test -x "$GOPATH/bin/staticcheck" || { echo "go install failed after 5 attempts"; exit 1; }
|
|
|
|
# GOPATH already set above; PATH handled by the final clean ENV at the
|
|
# bottom (which includes /home/crafter/go/bin). No per-layer PATH ENV
|
|
# needed here — the layered approach drifted from accumulator-style PATH
|
|
# into hand-rolled lists earlier and broke (see git blame for the
|
|
# resulting band-aid commits).
|
|
|
|
# ============================================================
|
|
# 18. Ruby user tooling: bundler-audit, rubocop
|
|
# ============================================================
|
|
RUN gem install --user-install --no-document bundler-audit rubocop || \
|
|
sudo gem install --no-document bundler-audit rubocop
|
|
ENV PATH=/home/crafter/.local/share/gem/ruby/3.1.0/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 19. PHP user tooling: phpstan, phpunit (composer global)
|
|
# ============================================================
|
|
ENV COMPOSER_HOME=/home/crafter/.composer
|
|
RUN composer global require --no-interaction phpstan/phpstan phpunit/phpunit
|
|
ENV PATH=/home/crafter/.composer/vendor/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 19.5. Nix (single-user) — for Plutarch / Plutus / IOG flakes
|
|
# ============================================================
|
|
# Why: Cardano smart-contract toolchains (Plutarch, plutus-core, the
|
|
# Liqwid Agora `agora-scripts` exporter) ship as Nix flakes with
|
|
# haskell-nix-pinned GHC. Building them with system cabal alone is
|
|
# a manual-version-pinning fight against the IOG snapshot.
|
|
#
|
|
# Single-user install at /nix, no daemon. Sandbox disabled — nested
|
|
# sandboxes don't work cleanly under Docker. Flakes + nix-command +
|
|
# ca-derivations enabled at install time so the SQLite store is
|
|
# initialized with the Realisations schema. Without ca-derivations
|
|
# pre-enabled, store schema is v10 (no Realisations table); flakes
|
|
# that request ca-derivations via nixConfig (Plutarch / Liqwid Agora
|
|
# / IOG Hydra cache, etc) crash on first realisation query with
|
|
# `Assertion 'stmt.stmt' failed in nix::SQLiteStmt::Use::Use`.
|
|
#
|
|
# Substituters: cache.nixos.org (default) + cache.iog.io (IOG Hydra
|
|
# binary cache for Plutarch + Cardano + Haskell.nix closure) +
|
|
# mlabs.cachix.org (MLabs builds — used by Liqwid stack).
|
|
# trusted-substituters lets the user accept new substituters via
|
|
# accept-flake-config without re-confirmation.
|
|
#
|
|
# Cache: /nix is a Docker-managed named volume in compose.yml so the
|
|
# multi-GB haskell-nix downloads persist across container rebuilds.
|
|
USER root
|
|
RUN mkdir -m 0755 /nix && chown crafter:crafter /nix
|
|
USER crafter
|
|
WORKDIR /home/crafter
|
|
RUN mkdir -p /home/crafter/.config/nix \
|
|
&& printf '%s\n' \
|
|
'experimental-features = nix-command flakes' \
|
|
'sandbox = false' \
|
|
'accept-flake-config = false' \
|
|
'substituters = https://cache.nixos.org https://cache.iog.io' \
|
|
'trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=' \
|
|
> /home/crafter/.config/nix/nix.conf \
|
|
&& curl -fsSL https://nixos.org/nix/install -o /tmp/install-nix.sh \
|
|
&& sh /tmp/install-nix.sh --no-daemon --no-channel-add --no-modify-profile \
|
|
&& rm /tmp/install-nix.sh
|
|
|
|
# Why accept-flake-config=false:
|
|
# Nix 2.34.7's schema-upgrade path from v10 to v11 (which adds the
|
|
# Realisations table that ca-derivations queries) doesn't fire
|
|
# cleanly, even with ca-derivations enabled at install time.
|
|
# Plutarch / Liqwid Agora / IOG flakes set
|
|
# `extra-experimental-features = [... "ca-derivations" ...]` in
|
|
# their nixConfig. With accept-flake-config=true that gets added
|
|
# to our experimental-features, then any flake operation crashes
|
|
# with `Assertion 'stmt.stmt' failed in nix::SQLiteStmt::Use::Use`.
|
|
# Disabling sidesteps the issue — ca-derivations is an optimization
|
|
# for content-addressed builds, not a hard requirement.
|
|
#
|
|
# Substituters set in our base nix.conf so we don't depend on the
|
|
# flake's additions. mlabs.cachix.org dropped — it's a private cache
|
|
# returning 401 to anonymous reads.
|
|
ENV PATH=/home/crafter/.nix-profile/bin:$PATH
|
|
|
|
# ============================================================
|
|
# 20. Smoke script — bake in
|
|
# ============================================================
|
|
COPY --chown=crafter:crafter smoke.sh /usr/local/bin/smoke.sh
|
|
USER root
|
|
RUN chmod +x /usr/local/bin/smoke.sh
|
|
|
|
# ============================================================
|
|
# 21. Application — FastAPI + async runner (wave 1: steps 2+3+4)
|
|
# ============================================================
|
|
COPY pyproject.toml requirements.txt /app/
|
|
# --force-reinstall is the critical flag: without it, pip skips packages
|
|
# already in /root/.local/ (left there by the earlier pipx bootstrap as
|
|
# USER root). uvicorn under /usr/local/bin needs them in
|
|
# /usr/local/lib/python3.11/dist-packages, so force the system install.
|
|
RUN pip install --break-system-packages --no-cache-dir --force-reinstall -r /app/requirements.txt
|
|
COPY --chown=crafter:crafter crafting_table /app/crafting_table
|
|
RUN chown -R crafter:crafter /app
|
|
|
|
# ============================================================
|
|
# 22. Final ENV / WORKDIR / CMD
|
|
# ============================================================
|
|
USER crafter
|
|
WORKDIR /workspace
|
|
|
|
# Final clean PATH — single source of truth that overrides any earlier
|
|
# accumulator drift in the layered ENV PATH= statements above. Lists
|
|
# every toolchain bin so cargo/rustc, swift, kotlinc, gradle, bun, go +
|
|
# govulncheck/staticcheck, ruff/mypy/pytest/uv, phpstan, bundler-audit
|
|
# are all reachable from the crafter user shell with no per-recipe prefix.
|
|
ENV PATH=/home/crafter/.nix-profile/bin:/home/crafter/.local/bin:/home/crafter/.composer/vendor/bin:/home/crafter/.local/share/gem/ruby/3.1.0/bin:/home/crafter/.bun/bin:/home/crafter/go/bin:/home/crafter/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin:/caches/cargo/bin:/opt/swift/usr/bin:/opt/kotlin/bin:/opt/gradle/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
|
|
|
|
ENV PYTHONPATH=/app \
|
|
PYTHONUNBUFFERED=1
|
|
CMD ["uvicorn", "crafting_table.server:app", "--host", "0.0.0.0", "--port", "8810"]
|