test(deploy): add a real login smoke test, not just a health check
CI / Repo hygiene (pull_request) Successful in 2s
CI / Web (lint, typecheck, build) (pull_request) Successful in 14s
CI / Migrations reversible (pull_request) Successful in 12s
CI / API (lint, types, tests) (pull_request) Successful in 59s

Found on the actual first deployment: /api/v1/healthz proves the process is
up, but says nothing about whether login actually works, because the
session cookie is set with Secure in production. Test through a plain-HTTP
address (an IP, a bare port, skipping the reverse proxy) and /auth/login
still returns 200 with a valid body — the cookie is just silently dropped by
the client, so the very next request looks unauthenticated. From a browser
this looks exactly like "I logged in and it bounced me straight back to the
login screen," with no error anywhere to point at.

scripts/smoke-test.sh does the real round trip a browser does: login,
confirm a session cookie was actually stored (not just sent), then an
authenticated follow-up request confirming it succeeds and returns the
right account. Verified it actually catches what it's meant to catch before
committing: ran it against a throwaway account over plain HTTP against a
production-mode container and got the expected FAIL with a diagnostic
pointing at the Secure-cookie mismatch, then confirmed PASS once the
container's VELODROME_ENVIRONMENT was (inadvertently, in this case)
development instead.

Documented in deploy/README.md as the real post-deploy check, replacing
"hit /healthz and eyeball it" for anything involving auth.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R2ZKeWkZV7ehf7fivrAkkG
This commit is contained in:
2026-09-21 22:54:51 -04:00
co-authored by Claude Sonnet 5
parent 65879e8659
commit 8c88748f50
2 changed files with 101 additions and 0 deletions
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env bash
# Post-deploy smoke test: proves a login -> authenticated request round trip actually works
# against a REAL deployed instance, over the network, the way a browser sees it.
#
# Exists because pytest (real SQLite, no mocks — see CLAUDE.md) proves the API logic is
# correct in isolation, but can't catch topology-specific failures. Concretely: a session
# cookie is set with `Secure` in production (velodrome/api/v1/auth.py), which browsers
# silently refuse to store unless the request was actually served over HTTPS. Hit the app via
# a plain-HTTP address (an IP, a port, skipping the reverse proxy) and `/auth/login` still
# returns 200 with valid credentials, and the cookie header is still sent — it's just quietly
# dropped, so the very next request looks unauthenticated. From a browser this looks exactly
# like "I logged in and it bounced me straight back to the login screen," with no error
# anywhere. Caught for real the first time this got deployed; this script exists so it's
# caught by running a command, not by refreshing a browser tab.
#
# Usage:
# scripts/smoke-test.sh <base_url> <email> <password>
# scripts/smoke-test.sh https://bike.bbergle.com you@example.com yourpassword
#
# Doesn't create the account — bootstrap one first with
# `docker exec -it velodrome velodrome create-admin --email you@example.com`, then reuse
# those credentials here (or keep a small dedicated account around just for this).
set -euo pipefail
BASE_URL="${1:?usage: smoke-test.sh <base_url> <email> <password>}"
EMAIL="${2:?usage: smoke-test.sh <base_url> <email> <password>}"
PASSWORD="${3:?usage: smoke-test.sh <base_url> <email> <password>}"
BASE_URL="${BASE_URL%/}"
COOKIEJAR="$(mktemp)"
LOGIN_BODY="$(mktemp)"
ME_BODY="$(mktemp)"
trap 'rm -f "$COOKIEJAR" "$LOGIN_BODY" "$ME_BODY"' EXIT
echo "-> logging in as $EMAIL at $BASE_URL"
LOGIN_STATUS=$(curl -s -o "$LOGIN_BODY" -w '%{http_code}' \
-c "$COOKIEJAR" \
-X POST "$BASE_URL/api/v1/auth/login" \
-H 'Content-Type: application/json' \
-d "{\"email\":\"$EMAIL\",\"password\":\"$PASSWORD\"}")
if [ "$LOGIN_STATUS" != "200" ]; then
echo "FAIL: login returned $LOGIN_STATUS, expected 200"
cat "$LOGIN_BODY"
exit 1
fi
echo " login: 200 OK"
if ! grep -q "_session" "$COOKIEJAR" 2>/dev/null; then
echo "FAIL: login succeeded but no session cookie was actually stored by the client."
echo " Almost certainly a Secure-cookie-over-HTTP mismatch — see the comment at the"
echo " top of this script. Are you testing via HTTPS through the real reverse proxy,"
echo " or a plain-HTTP address (an IP, a bare port)?"
exit 1
fi
echo " session cookie: stored"
echo "-> confirming the session actually authenticates a follow-up request"
ME_STATUS=$(curl -s -o "$ME_BODY" -w '%{http_code}' -b "$COOKIEJAR" "$BASE_URL/api/v1/auth/me")
if [ "$ME_STATUS" != "200" ]; then
echo "FAIL: /auth/me returned $ME_STATUS after a successful login — the session isn't"
echo " persisting. This is exactly the 'logs in, bounces back to the login screen'"
echo " symptom a browser would show."
cat "$ME_BODY"
exit 1
fi
ME_EMAIL=$(python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['email'])" "$ME_BODY" 2>/dev/null || echo "?")
if [ "$ME_EMAIL" != "$EMAIL" ]; then
echo "FAIL: /auth/me returned a different account ($ME_EMAIL) than the one that logged in ($EMAIL)."
exit 1
fi
echo " /auth/me: 200 OK, confirmed as $ME_EMAIL"
echo "PASS: login -> authenticated request round trip works end to end at $BASE_URL"