From 8c88748f50e57cb5339dfb278a0a6c77feddfd2f Mon Sep 17 00:00:00 2001 From: Benny Date: Mon, 21 Sep 2026 22:54:51 -0400 Subject: [PATCH] test(deploy): add a real login smoke test, not just a health check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found on the actual first deployment: /api/v1/healthz proves the process is up, but says nothing about whether login actually works, because the session cookie is set with Secure in production. Test through a plain-HTTP address (an IP, a bare port, skipping the reverse proxy) and /auth/login still returns 200 with a valid body — the cookie is just silently dropped by the client, so the very next request looks unauthenticated. From a browser this looks exactly like "I logged in and it bounced me straight back to the login screen," with no error anywhere to point at. scripts/smoke-test.sh does the real round trip a browser does: login, confirm a session cookie was actually stored (not just sent), then an authenticated follow-up request confirming it succeeds and returns the right account. Verified it actually catches what it's meant to catch before committing: ran it against a throwaway account over plain HTTP against a production-mode container and got the expected FAIL with a diagnostic pointing at the Secure-cookie mismatch, then confirmed PASS once the container's VELODROME_ENVIRONMENT was (inadvertently, in this case) development instead. Documented in deploy/README.md as the real post-deploy check, replacing "hit /healthz and eyeball it" for anything involving auth. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01R2ZKeWkZV7ehf7fivrAkkG --- deploy/README.md | 24 ++++++++++++++ scripts/smoke-test.sh | 77 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) create mode 100755 scripts/smoke-test.sh diff --git a/deploy/README.md b/deploy/README.md index 41fb2fe..80fdc5c 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -78,6 +78,30 @@ Nothing enforces the admin role yet — no admin-only endpoint exists — so tod invited account only in the role recorded on it. Invite management in a later phase is what starts reading it. +## Verify login actually works, not just that the API responds + +`GET /api/v1/healthz` proves the process is up. It does **not** prove a real login works, because +the session cookie is set with `Secure` in production (`apps/api/velodrome/api/v1/auth.py`) — +browsers silently refuse to store a `Secure` cookie unless the request was actually served over +HTTPS. Test through a plain-HTTP address (an IP, a bare port, skipping the reverse proxy) and +`/auth/login` still returns 200 with a valid response body; the cookie is just quietly dropped, so +the very next request looks unauthenticated. From a browser this looks exactly like "I logged in +and it bounced me straight back to the login screen," with nothing that looks like an error. This +happened on the very first real deployment. + +`scripts/smoke-test.sh` exists so this is caught by running a command, not by refreshing a browser +tab: + +```sh +scripts/smoke-test.sh https://bike.bbergle.com you@example.com yourpassword +``` + +It logs in, confirms a session cookie was actually stored (not just sent), then makes an +authenticated follow-up request and confirms it succeeds and returns the right account. Run it +after every real deploy, against the actual public URL your users will use — testing against a +plain-HTTP IP will (correctly) tell you nothing about whether login works for anyone using the real +domain. + ## Environment variables All read by `apps/api/velodrome/config.py` (prefix `VELODROME_`) — the app and Alembic both read diff --git a/scripts/smoke-test.sh b/scripts/smoke-test.sh new file mode 100755 index 0000000..403f2f5 --- /dev/null +++ b/scripts/smoke-test.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# Post-deploy smoke test: proves a login -> authenticated request round trip actually works +# against a REAL deployed instance, over the network, the way a browser sees it. +# +# Exists because pytest (real SQLite, no mocks — see CLAUDE.md) proves the API logic is +# correct in isolation, but can't catch topology-specific failures. Concretely: a session +# cookie is set with `Secure` in production (velodrome/api/v1/auth.py), which browsers +# silently refuse to store unless the request was actually served over HTTPS. Hit the app via +# a plain-HTTP address (an IP, a port, skipping the reverse proxy) and `/auth/login` still +# returns 200 with valid credentials, and the cookie header is still sent — it's just quietly +# dropped, so the very next request looks unauthenticated. From a browser this looks exactly +# like "I logged in and it bounced me straight back to the login screen," with no error +# anywhere. Caught for real the first time this got deployed; this script exists so it's +# caught by running a command, not by refreshing a browser tab. +# +# Usage: +# scripts/smoke-test.sh +# scripts/smoke-test.sh https://bike.bbergle.com you@example.com yourpassword +# +# Doesn't create the account — bootstrap one first with +# `docker exec -it velodrome velodrome create-admin --email you@example.com`, then reuse +# those credentials here (or keep a small dedicated account around just for this). +set -euo pipefail + +BASE_URL="${1:?usage: smoke-test.sh }" +EMAIL="${2:?usage: smoke-test.sh }" +PASSWORD="${3:?usage: smoke-test.sh }" +BASE_URL="${BASE_URL%/}" + +COOKIEJAR="$(mktemp)" +LOGIN_BODY="$(mktemp)" +ME_BODY="$(mktemp)" +trap 'rm -f "$COOKIEJAR" "$LOGIN_BODY" "$ME_BODY"' EXIT + +echo "-> logging in as $EMAIL at $BASE_URL" +LOGIN_STATUS=$(curl -s -o "$LOGIN_BODY" -w '%{http_code}' \ + -c "$COOKIEJAR" \ + -X POST "$BASE_URL/api/v1/auth/login" \ + -H 'Content-Type: application/json' \ + -d "{\"email\":\"$EMAIL\",\"password\":\"$PASSWORD\"}") + +if [ "$LOGIN_STATUS" != "200" ]; then + echo "FAIL: login returned $LOGIN_STATUS, expected 200" + cat "$LOGIN_BODY" + exit 1 +fi +echo " login: 200 OK" + +if ! grep -q "_session" "$COOKIEJAR" 2>/dev/null; then + echo "FAIL: login succeeded but no session cookie was actually stored by the client." + echo " Almost certainly a Secure-cookie-over-HTTP mismatch — see the comment at the" + echo " top of this script. Are you testing via HTTPS through the real reverse proxy," + echo " or a plain-HTTP address (an IP, a bare port)?" + exit 1 +fi +echo " session cookie: stored" + +echo "-> confirming the session actually authenticates a follow-up request" +ME_STATUS=$(curl -s -o "$ME_BODY" -w '%{http_code}' -b "$COOKIEJAR" "$BASE_URL/api/v1/auth/me") + +if [ "$ME_STATUS" != "200" ]; then + echo "FAIL: /auth/me returned $ME_STATUS after a successful login — the session isn't" + echo " persisting. This is exactly the 'logs in, bounces back to the login screen'" + echo " symptom a browser would show." + cat "$ME_BODY" + exit 1 +fi + +ME_EMAIL=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['email'])" "$ME_BODY" 2>/dev/null || echo "?") + +if [ "$ME_EMAIL" != "$EMAIL" ]; then + echo "FAIL: /auth/me returned a different account ($ME_EMAIL) than the one that logged in ($EMAIL)." + exit 1 +fi + +echo " /auth/me: 200 OK, confirmed as $ME_EMAIL" +echo "PASS: login -> authenticated request round trip works end to end at $BASE_URL" -- 2.54.0