test(deploy): add a real login smoke test, not just a health check #11

Merged
BBergle merged 1 commits from test/deploy-login-smoke-test into main 2026-09-21 22:57:35 -04:00
2 changed files with 101 additions and 0 deletions
+24
View File
@@ -78,6 +78,30 @@ Nothing enforces the admin role yet — no admin-only endpoint exists — so tod
invited account only in the role recorded on it. Invite management in a later phase is what starts invited account only in the role recorded on it. Invite management in a later phase is what starts
reading it. reading it.
## Verify login actually works, not just that the API responds
`GET /api/v1/healthz` proves the process is up. It does **not** prove a real login works, because
the session cookie is set with `Secure` in production (`apps/api/velodrome/api/v1/auth.py`) —
browsers silently refuse to store a `Secure` cookie unless the request was actually served over
HTTPS. Test through a plain-HTTP address (an IP, a bare port, skipping the reverse proxy) and
`/auth/login` still returns 200 with a valid response body; the cookie is just quietly dropped, so
the very next request looks unauthenticated. From a browser this looks exactly like "I logged in
and it bounced me straight back to the login screen," with nothing that looks like an error. This
happened on the very first real deployment.
`scripts/smoke-test.sh` exists so this is caught by running a command, not by refreshing a browser
tab:
```sh
scripts/smoke-test.sh https://bike.bbergle.com you@example.com yourpassword
```
It logs in, confirms a session cookie was actually stored (not just sent), then makes an
authenticated follow-up request and confirms it succeeds and returns the right account. Run it
after every real deploy, against the actual public URL your users will use — testing against a
plain-HTTP IP will (correctly) tell you nothing about whether login works for anyone using the real
domain.
## Environment variables ## Environment variables
All read by `apps/api/velodrome/config.py` (prefix `VELODROME_`) — the app and Alembic both read All read by `apps/api/velodrome/config.py` (prefix `VELODROME_`) — the app and Alembic both read
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env bash
# Post-deploy smoke test: proves a login -> authenticated request round trip actually works
# against a REAL deployed instance, over the network, the way a browser sees it.
#
# Exists because pytest (real SQLite, no mocks — see CLAUDE.md) proves the API logic is
# correct in isolation, but can't catch topology-specific failures. Concretely: a session
# cookie is set with `Secure` in production (velodrome/api/v1/auth.py), which browsers
# silently refuse to store unless the request was actually served over HTTPS. Hit the app via
# a plain-HTTP address (an IP, a port, skipping the reverse proxy) and `/auth/login` still
# returns 200 with valid credentials, and the cookie header is still sent — it's just quietly
# dropped, so the very next request looks unauthenticated. From a browser this looks exactly
# like "I logged in and it bounced me straight back to the login screen," with no error
# anywhere. Caught for real the first time this got deployed; this script exists so it's
# caught by running a command, not by refreshing a browser tab.
#
# Usage:
# scripts/smoke-test.sh <base_url> <email> <password>
# scripts/smoke-test.sh https://bike.bbergle.com you@example.com yourpassword
#
# Doesn't create the account — bootstrap one first with
# `docker exec -it velodrome velodrome create-admin --email you@example.com`, then reuse
# those credentials here (or keep a small dedicated account around just for this).
set -euo pipefail
BASE_URL="${1:?usage: smoke-test.sh <base_url> <email> <password>}"
EMAIL="${2:?usage: smoke-test.sh <base_url> <email> <password>}"
PASSWORD="${3:?usage: smoke-test.sh <base_url> <email> <password>}"
BASE_URL="${BASE_URL%/}"
COOKIEJAR="$(mktemp)"
LOGIN_BODY="$(mktemp)"
ME_BODY="$(mktemp)"
trap 'rm -f "$COOKIEJAR" "$LOGIN_BODY" "$ME_BODY"' EXIT
echo "-> logging in as $EMAIL at $BASE_URL"
LOGIN_STATUS=$(curl -s -o "$LOGIN_BODY" -w '%{http_code}' \
-c "$COOKIEJAR" \
-X POST "$BASE_URL/api/v1/auth/login" \
-H 'Content-Type: application/json' \
-d "{\"email\":\"$EMAIL\",\"password\":\"$PASSWORD\"}")
if [ "$LOGIN_STATUS" != "200" ]; then
echo "FAIL: login returned $LOGIN_STATUS, expected 200"
cat "$LOGIN_BODY"
exit 1
fi
echo " login: 200 OK"
if ! grep -q "_session" "$COOKIEJAR" 2>/dev/null; then
echo "FAIL: login succeeded but no session cookie was actually stored by the client."
echo " Almost certainly a Secure-cookie-over-HTTP mismatch — see the comment at the"
echo " top of this script. Are you testing via HTTPS through the real reverse proxy,"
echo " or a plain-HTTP address (an IP, a bare port)?"
exit 1
fi
echo " session cookie: stored"
echo "-> confirming the session actually authenticates a follow-up request"
ME_STATUS=$(curl -s -o "$ME_BODY" -w '%{http_code}' -b "$COOKIEJAR" "$BASE_URL/api/v1/auth/me")
if [ "$ME_STATUS" != "200" ]; then
echo "FAIL: /auth/me returned $ME_STATUS after a successful login — the session isn't"
echo " persisting. This is exactly the 'logs in, bounces back to the login screen'"
echo " symptom a browser would show."
cat "$ME_BODY"
exit 1
fi
ME_EMAIL=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['email'])" "$ME_BODY" 2>/dev/null || echo "?")
if [ "$ME_EMAIL" != "$EMAIL" ]; then
echo "FAIL: /auth/me returned a different account ($ME_EMAIL) than the one that logged in ($EMAIL)."
exit 1
fi
echo " /auth/me: 200 OK, confirmed as $ME_EMAIL"
echo "PASS: login -> authenticated request round trip works end to end at $BASE_URL"