feat(deploy): compose stack, Caddy, and the release/deploy pipeline #4

Closed
BBergle wants to merge 2 commits from feat/deploy-pipeline into main
Owner

What and why

Phase 0 deployment wiring for the API and web apps that already merged to main: a three-service
docker-compose.yml (caddy, api, db), the Caddyfile that proxies /api/* and serves the
SPA with index.html fallback, .env.example documenting every variable the compose file needs,
and two new Gitea Actions workflows — release.yml (build + push both images on a v* tag or
manual dispatch) and deploy.yml (manual-dispatch-only rollout to the Unraid host).

Only deploy/ and .gitea/workflows/{release,deploy}.yml are touched. ci.yml is untouched, and
neither apps/api nor apps/web is modified — this PR wires already-finished apps together.

How this was verified

  • CI is green (not yet — this PR hasn't been pushed through CI at time of writing; will
    confirm after push. Nothing in apps/api/apps/web changed, so ci.yml's guards should
    no-op cleanly for this diff, and the meta job's branch-name/secret checks apply as usual.)
  • Tests added or updated for the behaviour that changed — N/A, no application code changed.
  • Verified manually (describe how):

What I actually ran, locally, with real Docker:

  • docker compose -f deploy/docker-compose.yml config — parses cleanly against a scratch .env
    built from .env.example with dummy values (not committed).
  • Built apps/api/Dockerfile and apps/web/Dockerfile locally, tagged as the real registry
    paths (192.168.0.3:3000/bbergle/velodrome-api:latest / -web:latest) so compose/docker run
    resolve them without needing the real registry.
  • Brought up db alone first (with a bind-mount override pointing at a scratch directory instead
    of the real /mnt/user/appdata/velodrome/pgdata, since that path doesn't exist on my dev
    machine) — reached healthy via its pg_isready healthcheck.
  • Tested the web_build named-volume population trick specifically, since it's the part most
    likely to have a subtle bug and the part config-parsing alone can't catch:
    docker run --rm -v velodrome_web_build:/dest <web-image> sh -c 'rm -rf ...; cp -a /app/build/. /dest/' — confirmed index.html, _app/, manifest.webmanifest etc. land in the
    volume by inspecting it from a throwaway alpine container. Then re-ran it a second time after
    seeding a stale marker file to confirm the rm -rf clears prior content before copying fresh
    files (simulating a redeploy to the same tag) — it did.
  • Ran the actual migration command from deploy.ymldocker compose run --rm -e VELODROME_DATABASE_URL_MIGRATE=... api alembic upgrade head — against the local stack. This
    caught a real bug
    : the migration also needs VELODROME_DB_APP_PASSWORD /
    VELODROME_DB_AUTH_PASSWORD as container env vars (it creates those two Postgres roles), but
    --env-file only feeds ${...} substitution inside the compose YAML — it doesn't inject
    variables into a container unless the service's environment: block names them, and api's
    block deliberately omits both (the long-running app has no business holding role-creation
    passwords). Fixed by passing both as explicit -e overrides on the migration step, the same way
    VELODROME_DATABASE_URL_MIGRATE already was. Re-ran after the fix — migration succeeded.
  • Brought up the full stack (caddy + api + db) and confirmed curl http://localhost:8090/api/v1/healthz returns 200 {"status":"ok"} through Caddy, and that a
    client-side route (/some/client/route) falls back to index.html (200, not 404) — confirms
    the SPA try_files rule works.
  • Tore everything down afterward (docker compose down -v, removed the locally-built images, the
    scratch .env, and the scratch pgdata directory) — nothing left behind.
  • Both new workflow YAML files parse with yaml.safe_load (via a scratch venv with pyyaml,
    since neither uv nor system pyyaml was available in this environment — same intent as
    the uv run --with pyyaml approach used elsewhere in this repo).

What genuinely can't be verified until this runs for real, per the task brief — this targets
the live production Unraid host and needs secrets I don't have:

  • The registry push/pull path in release.yml/deploy.yml against the real
    192.168.0.3:3000 registry (I built and tagged images locally instead).
  • Whether the DooD job container's localhost:8090 in the final healthcheck step actually reaches
    the real host's published Caddy port. This deploy runs on a plain ubuntu-latest job container
    (not a host-networked runner — no such label exists per the confirmed infra facts), so if the
    job container doesn't share the host's network namespace, curl http://localhost:8090/...
    could fail even though the deploy itself succeeded. I implemented the healthcheck exactly as
    specified (this is what the task brief gave verbatim), but flagging it here as the one thing in
    this pipeline I could not confirm — if it fails in practice, the fix is almost certainly
    curl http://192.168.0.103:8090/... instead of localhost, or wiring network_mode: host /
    a bridge-network route.
  • The POSTGRES_SUPERUSER_PASSWORD/pgdata bind mount against the real
    /mnt/user/appdata/velodrome/pgdata path (used a scratch directory locally instead, since that
    path doesn't exist on this dev machine — the pattern itself is unchanged from what's committed).

Invariants

  • Raw ingested bytes remain immutable; derived tables stay rebuildable — N/A, no ingestion
    code here.
  • No stored odometer added; wear still derived from installs — N/A.
  • Physical quantities stored as SI integers — N/A to this PR (no schema), but nothing here
    contradicts it.
  • New user-owned tables have user_id + RLS policy + repository scope — N/A, no new tables.
  • No secret can reach a response model, log line, or error message — .env.example has only
    changeme-style placeholders, never real values; the real .env stays host-only, never in
    git; VELODROME_DATABASE_URL_MIGRATE (which embeds the superuser password) is only ever an
    inline -e override on a one-off migration command, never in a logged/persisted compose
    environment block or the api service's permanent env.
  • Migration survives upgrade -> downgrade -1 -> upgrade — N/A, no migration added by this
    PR; existing migrations are exercised as-is by the deploy pipeline's alembic upgrade head
    step.

Risks and follow-ups

Deliberately out of scope for this PR (per the task brief, not oversights):

  • No worker compose service / procrastinate yet — nothing enqueues background jobs until the
    ingestion pipeline (Phase 1). Documented as a comment in docker-compose.yml.
  • No VAPID keypair generation — nothing consumes push notifications until Phase 2.
  • No blob-store / import_inbox volume — nothing ingests files yet.
  • No PROD_DEPLOY_TOKEN secret gate from docs/PLAN.md. For a single-operator instance with no
    other collaborators, workflow_dispatch alone (already gated on repo write access to trigger)
    is a reasonable simplification for now — noted here honestly as a deliberate deviation from the
    plan rather than a silent one.
  • renovate.yml / nightly.yml are separate future work.
  • Buildx type=gha layer caching was not wired into release.yml. I wasn't confident this
    act_runner setup actually supports the GHA cache backend (as opposed to actions/cache@v4
    itself, which the setup does support per docs/PLAN.md), and the task brief said to skip it
    rather than guess. First builds will be slower than they could be; worth revisiting once someone
    confirms the cache backend works against this runner.

Before deploy.yml can succeed for real, the human needs to:

  1. Create a REGISTRY_TOKEN repo Actions secret: Gitea → user Settings → Applications → new
    token with package:write scope, then add it as a repository secret named REGISTRY_TOKEN
    (Settings → Actions → Secrets) on BBergle/bike-app. secrets.GITEA_TOKEN cannot push to the
    registry — documented Gitea limitation, already noted in CLAUDE.md.
  2. Create /mnt/user/appdata/velodrome/.env on the Unraid host (not in git), following the shape
    in deploy/.env.example, with real values for: POSTGRES_SUPERUSER_PASSWORD,
    VELODROME_DB_APP_PASSWORD, VELODROME_DB_AUTH_PASSWORD, the three VELODROME_DATABASE_URL_*
    DSNs built from those (db is the in-compose-network hostname), VELODROME_SECRET_KEY
    (e.g. openssl rand -hex 32), VELODROME_ENVIRONMENT=production,
    VELODROME_PUBLIC_URL=http://192.168.0.103:8090 (must match exactly how the app is reached —
    it gates the CSRF Origin check), and TAG (defaults to latest).
  3. Confirm /mnt/user/appdata/velodrome/pgdata exists (or let Postgres create it on first start)
    and is writable by the container.
  4. First deploy: push a v* tag (or run release.yml via manual dispatch) to build and push
    images, confirm it's green, then run deploy.yml via manual dispatch with the tag to deploy.
  5. Watch the final healthcheck step specifically — see the localhost:8090 caveat above; if it
    fails despite the stack actually being up (checkable via docker compose ps on the host), that
    curl target is the first thing to fix.

🤖 Generated with Claude Code

## What and why Phase 0 deployment wiring for the API and web apps that already merged to `main`: a three-service `docker-compose.yml` (`caddy`, `api`, `db`), the `Caddyfile` that proxies `/api/*` and serves the SPA with `index.html` fallback, `.env.example` documenting every variable the compose file needs, and two new Gitea Actions workflows — `release.yml` (build + push both images on a `v*` tag or manual dispatch) and `deploy.yml` (manual-dispatch-only rollout to the Unraid host). Only `deploy/` and `.gitea/workflows/{release,deploy}.yml` are touched. `ci.yml` is untouched, and neither `apps/api` nor `apps/web` is modified — this PR wires already-finished apps together. ## How this was verified - [x] CI is green (not yet — this PR hasn't been pushed through CI at time of writing; will confirm after push. Nothing in `apps/api`/`apps/web` changed, so `ci.yml`'s guards should no-op cleanly for this diff, and the `meta` job's branch-name/secret checks apply as usual.) - [ ] Tests added or updated for the behaviour that changed — N/A, no application code changed. - [x] Verified manually (describe how): **What I actually ran, locally, with real Docker:** - `docker compose -f deploy/docker-compose.yml config` — parses cleanly against a scratch `.env` built from `.env.example` with dummy values (not committed). - Built `apps/api/Dockerfile` and `apps/web/Dockerfile` locally, tagged as the real registry paths (`192.168.0.3:3000/bbergle/velodrome-api:latest` / `-web:latest`) so compose/`docker run` resolve them without needing the real registry. - Brought up `db` alone first (with a bind-mount override pointing at a scratch directory instead of the real `/mnt/user/appdata/velodrome/pgdata`, since that path doesn't exist on my dev machine) — reached `healthy` via its `pg_isready` healthcheck. - **Tested the `web_build` named-volume population trick specifically**, since it's the part most likely to have a subtle bug and the part config-parsing alone can't catch: `docker run --rm -v velodrome_web_build:/dest <web-image> sh -c 'rm -rf ...; cp -a /app/build/. /dest/'` — confirmed `index.html`, `_app/`, `manifest.webmanifest` etc. land in the volume by inspecting it from a throwaway `alpine` container. Then re-ran it a second time after seeding a stale marker file to confirm the `rm -rf` clears prior content before copying fresh files (simulating a redeploy to the same tag) — it did. - Ran the actual migration command from `deploy.yml` — `docker compose run --rm -e VELODROME_DATABASE_URL_MIGRATE=... api alembic upgrade head` — against the local stack. **This caught a real bug**: the migration also needs `VELODROME_DB_APP_PASSWORD` / `VELODROME_DB_AUTH_PASSWORD` as container env vars (it creates those two Postgres roles), but `--env-file` only feeds `${...}` substitution inside the compose YAML — it doesn't inject variables into a container unless the service's `environment:` block names them, and `api`'s block deliberately omits both (the long-running app has no business holding role-creation passwords). Fixed by passing both as explicit `-e` overrides on the migration step, the same way `VELODROME_DATABASE_URL_MIGRATE` already was. Re-ran after the fix — migration succeeded. - Brought up the full stack (`caddy` + `api` + `db`) and confirmed `curl http://localhost:8090/api/v1/healthz` returns `200 {"status":"ok"}` through Caddy, and that a client-side route (`/some/client/route`) falls back to `index.html` (200, not 404) — confirms the SPA `try_files` rule works. - Tore everything down afterward (`docker compose down -v`, removed the locally-built images, the scratch `.env`, and the scratch pgdata directory) — nothing left behind. - Both new workflow YAML files parse with `yaml.safe_load` (via a scratch venv with `pyyaml`, since neither `uv` nor system `pyyaml` was available in this environment — same intent as the `uv run --with pyyaml` approach used elsewhere in this repo). **What genuinely can't be verified until this runs for real**, per the task brief — this targets the live production Unraid host and needs secrets I don't have: - The registry push/pull path in `release.yml`/`deploy.yml` against the real `192.168.0.3:3000` registry (I built and tagged images locally instead). - Whether the DooD job container's `localhost:8090` in the final healthcheck step actually reaches the real host's published Caddy port. This deploy runs on a plain `ubuntu-latest` job container (not a `host`-networked runner — no such label exists per the confirmed infra facts), so if the job container doesn't share the host's network namespace, `curl http://localhost:8090/...` could fail even though the deploy itself succeeded. I implemented the healthcheck exactly as specified (this is what the task brief gave verbatim), but flagging it here as the one thing in this pipeline I could not confirm — if it fails in practice, the fix is almost certainly `curl http://192.168.0.103:8090/...` instead of `localhost`, or wiring `network_mode: host` / a bridge-network route. - The `POSTGRES_SUPERUSER_PASSWORD`/`pgdata` bind mount against the real `/mnt/user/appdata/velodrome/pgdata` path (used a scratch directory locally instead, since that path doesn't exist on this dev machine — the pattern itself is unchanged from what's committed). ## Invariants - [ ] Raw ingested bytes remain immutable; derived tables stay rebuildable — N/A, no ingestion code here. - [ ] No stored odometer added; wear still derived from installs — N/A. - [x] Physical quantities stored as SI integers — N/A to this PR (no schema), but nothing here contradicts it. - [ ] New user-owned tables have `user_id` + RLS policy + repository scope — N/A, no new tables. - [x] No secret can reach a response model, log line, or error message — `.env.example` has only `changeme`-style placeholders, never real values; the real `.env` stays host-only, never in git; `VELODROME_DATABASE_URL_MIGRATE` (which embeds the superuser password) is only ever an inline `-e` override on a one-off migration command, never in a logged/persisted compose environment block or the `api` service's permanent env. - [x] Migration survives `upgrade -> downgrade -1 -> upgrade` — N/A, no migration added by this PR; existing migrations are exercised as-is by the deploy pipeline's `alembic upgrade head` step. ## Risks and follow-ups **Deliberately out of scope for this PR** (per the task brief, not oversights): - No `worker` compose service / `procrastinate` yet — nothing enqueues background jobs until the ingestion pipeline (Phase 1). Documented as a comment in `docker-compose.yml`. - No VAPID keypair generation — nothing consumes push notifications until Phase 2. - No blob-store / `import_inbox` volume — nothing ingests files yet. - No `PROD_DEPLOY_TOKEN` secret gate from `docs/PLAN.md`. For a single-operator instance with no other collaborators, `workflow_dispatch` alone (already gated on repo write access to trigger) is a reasonable simplification for now — noted here honestly as a deliberate deviation from the plan rather than a silent one. - `renovate.yml` / `nightly.yml` are separate future work. - Buildx `type=gha` layer caching was **not** wired into `release.yml`. I wasn't confident this act_runner setup actually supports the GHA cache backend (as opposed to `actions/cache@v4` itself, which the setup does support per `docs/PLAN.md`), and the task brief said to skip it rather than guess. First builds will be slower than they could be; worth revisiting once someone confirms the cache backend works against this runner. **Before `deploy.yml` can succeed for real, the human needs to:** 1. Create a `REGISTRY_TOKEN` repo Actions secret: Gitea → user Settings → Applications → new token with `package:write` scope, then add it as a repository secret named `REGISTRY_TOKEN` (Settings → Actions → Secrets) on `BBergle/bike-app`. `secrets.GITEA_TOKEN` cannot push to the registry — documented Gitea limitation, already noted in `CLAUDE.md`. 2. Create `/mnt/user/appdata/velodrome/.env` on the Unraid host (not in git), following the shape in `deploy/.env.example`, with real values for: `POSTGRES_SUPERUSER_PASSWORD`, `VELODROME_DB_APP_PASSWORD`, `VELODROME_DB_AUTH_PASSWORD`, the three `VELODROME_DATABASE_URL_*` DSNs built from those (`db` is the in-compose-network hostname), `VELODROME_SECRET_KEY` (e.g. `openssl rand -hex 32`), `VELODROME_ENVIRONMENT=production`, `VELODROME_PUBLIC_URL=http://192.168.0.103:8090` (must match exactly how the app is reached — it gates the CSRF `Origin` check), and `TAG` (defaults to `latest`). 3. Confirm `/mnt/user/appdata/velodrome/pgdata` exists (or let Postgres create it on first start) and is writable by the container. 4. First deploy: push a `v*` tag (or run `release.yml` via manual dispatch) to build and push images, confirm it's green, *then* run `deploy.yml` via manual dispatch with the tag to deploy. 5. Watch the final healthcheck step specifically — see the `localhost:8090` caveat above; if it fails despite the stack actually being up (checkable via `docker compose ps` on the host), that curl target is the first thing to fix. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
BBergle added 1 commit 2026-09-21 15:08:17 -04:00
feat(deploy): compose stack, Caddy, and the release/deploy pipeline
CI / Repo hygiene (pull_request) Successful in 3s
CI / Web (lint, typecheck, build) (pull_request) Successful in 16s
CI / Migrations reversible (pull_request) Successful in 7s
CI / API (lint, types, tests) (pull_request) Successful in 55s
5d4d76203f
Phase 0 deployment: three-service docker-compose.yml (caddy, api, db), a
Caddyfile that proxies /api/* to the api service and serves the SPA with
index.html fallback, and two Gitea Actions workflows (release.yml builds and
pushes both images on a v* tag or manual dispatch; deploy.yml is manual-only
and rolls them out to the Unraid host).

The non-obvious part is the Docker-outside-of-Docker constraint on this
act_runner setup: job containers share the host's Docker daemon over the
socket but do NOT share its filesystem, so any command whose correctness
depends on a client-side local path (docker cp to a host path, mv/rm -rf on
a host path, a bind-mount source path on a `docker run` command line issued
from inside a job) silently operates on the ephemeral job container's own
throwaway filesystem instead. Two things are safe: a bind mount declared in
a compose file's `volumes:` block (resolved by the daemon when `docker
compose up` creates the service — this is why db's pgdata bind mount is
fine), and a named volume populated by a one-shot `docker run` whose
*command* does the copying (this is why the web image's static build output
goes into a `web_build` named volume via `docker run -v ... sh -c 'cp -a
...'` in deploy.yml, rather than any `docker cp`).

Local verification (see PR description for full detail) caught a real bug:
`docker compose run api alembic upgrade head` needs
VELODROME_DB_APP_PASSWORD/VELODROME_DB_AUTH_PASSWORD as container env vars
to create the two runtime roles, but --env-file alone doesn't inject them
since the api service's permanent environment block deliberately omits them
(least-privilege — the long-running app should never need role-creation
passwords). Fixed by passing them as explicit -e overrides on the migration
step, same as VELODROME_DATABASE_URL_MIGRATE.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BBergle added 1 commit 2026-09-21 15:11:35 -04:00
fix(deploy): healthcheck must hit the real host IP, not localhost
CI / Repo hygiene (pull_request) Successful in 1s
CI / Web (lint, typecheck, build) (pull_request) Successful in 15s
CI / Migrations reversible (pull_request) Successful in 9s
CI / API (lint, types, tests) (pull_request) Successful in 54s
4f4ca345ca
The deploy job runs inside its own ephemeral DooD job container, which is a
separate container from `caddy` — caddy's -p 8090:80 publishes onto the real
host's network namespace, not this job container's own loopback. A
`localhost:8090` curl here would fail with connection-refused regardless of
whether the deploy actually succeeded, misreporting a working deploy as a
failed workflow. Point it at the same host IP deploy/.env.example's
VELODROME_PUBLIC_URL already uses.

Caught during review, not left as the open caveat the PR description flagged
it as.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Author
Owner

Superseded — the deploy architecture is changing to a single container with SQLite instead of the 3-container docker-compose + Postgres/PostGIS stack this PR builds. Closing rather than merging something that no longer matches the direction. The DooD-safe patterns and reasoning here (named-volume trick, least-privilege env handling) remain useful reference for whatever the single-container image build ends up needing.

Superseded — the deploy architecture is changing to a single container with SQLite instead of the 3-container docker-compose + Postgres/PostGIS stack this PR builds. Closing rather than merging something that no longer matches the direction. The DooD-safe patterns and reasoning here (named-volume trick, least-privilege env handling) remain useful reference for whatever the single-container image build ends up needing.
BBergle closed this pull request 2026-09-21 15:21:21 -04:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: BBergle/bike-app#4