refactor(api): move from Postgres+RLS to single-engine SQLite
Reverses a shipped, tested, merged decision (D4/PR #2) rather than building on it — see docs/DECISIONS.md D15 for the full record: what was rejected (Postgres as a second container; Postgres+PostGIS bundled inside the single container via a supervisor), what this costs (no database-level RLS, no PostGIS, procrastinate needs replacing — all stated as a concern before this was decided, and reaffirmed anyway, which is the user's call to make about their own instance). The one invariant-critical consequence: isolation between users now rests entirely on the repository-layer scope (db.py's `Scope.select()`), not two layers. CLAUDE.md's invariant #4 is revised accordingly. This is not a downgrade-and-hope — `Scope` is built so an unfiltered query against a user-owned table is structurally harder to write than a scoped one (there is no method on `Scope` that returns one), and tests/test_auth.py::test_scoped_session_blocks_cross_user_reads replaces the old RLS proof with the same empirical standard: it doesn't trust the query builder filters correctly because the code reads correctly, it registers two real users and checks. test_unscoped_session_can_see_every_user_when_misused is the deliberately alarming companion — it demonstrates exactly what a reviewer must now catch, since nothing else will. Six real, non-obvious SQLite behaviours found and fixed by actually running this against a real file, not assumed from docs: - Foreign keys, ON DELETE CASCADE included, are OFF by default per connection — deleting a user silently left orphaned sessions/api_tokens, no error either way. Fixed with PRAGMA foreign_keys=ON on every connect. - Transactions default to DEFERRED, which only takes a write lock on the first actual write — a real check-then-act race for invite redemption (two concurrent redemptions could both read used_count < max_uses as true before either commits). Fixed by disabling the driver's implicit BEGIN and issuing BEGIN IMMEDIATE ourselves — SQLAlchemy's own documented recipe for this, not improvised. - DateTime(timezone=True) does NOT round-trip tzinfo on SQLite — a tz-aware datetime goes in, a naive one comes back out, and every `expires_at < datetime.now(UTC)` comparison in auth/service.py then raises TypeError. Fixed once at the Base level with a UTCDateTime TypeDecorator rather than per-column. - Uuid(as_uuid=True) stores as 32-char hex with NO hyphens on SQLite, not str(uuid)'s hyphenated form. A test fixture that raw-inserted the hyphenated form left rows the ORM's own later UPDATE (via invite.used_count += 1's autoflush) could never match by primary key, updating zero rows and raising StaleDataError. Fixed by using .hex to match exactly what the ORM itself writes. - BEGIN IMMEDIATE applies to every transaction, reads included — a long-lived test fixture that autobegins a transaction via a bare read and never explicitly closes it holds SQLite's exclusive write lock for the rest of the test, and a later scoped_session() call fails with "database is locked". Not an app-code bug (every real session block closes cleanly on exit), but real enough to document since the next person writing a test against the db_auth fixture will hit it too. - Python's sqlite3 module deprecates its own implicit datetime adapter as of 3.12 — silent today, warns on every raw-SQL datetime bind. Only ever hit test fixture code (the ORM path never uses it, confirmed by running the ORM-only health test with warnings promoted to errors and it stayed clean); fixed there with an explicit .isoformat() rather than left for a future Python version to turn into a real failure. Also, since with_for_update() silently no-ops on SQLite (confirmed — SQLAlchemy emits no SQL for it, no error either) rather than actually locking anything: removed it from register()'s invite-redemption query and corrected the comment to attribute the concurrency guarantee to BEGIN IMMEDIATE, where it now actually lives. One PR, not several, for the same reason PR #2 was: the migration, the models, db.py, and the docs recording why are five views of one decision — splitting them wouldn't make review easier, just disconnected. 552 insertions / 548 deletions across 17 files, most of it necessarily touching what PR #2 shipped rather than net-new code. Deliberately deferred, not solved here: PostGIS's replacement for spatial storage, procrastinate's replacement for background jobs, and the EXCLUDE USING gist constraint's replacement for component_installs — none of those tables exist yet (Phase 1-2), so none of it is broken, and docs/DECISIONS.md D15 records exactly what each future phase needs to decide before it can be built. .gitea/workflows/deploy pipeline (PR #4, built for the old 3-container Postgres compose stack) was closed as superseded rather than merged; the single-container image build is follow-up work, not part of this change. Verified: ruff check, ruff format --check, and mypy --strict all clean. 13/13 pytest passing against a real SQLite file, including with DeprecationWarning promoted to an error (confirms the sqlite3 adapter deprecation fix actually holds, not just that it's quiet by default). Full alembic upgrade -> downgrade -1 -> upgrade cycle run clean. alembic check clean with no include_object filter needed at all now (SQLite starts with nothing but what our own migrations create — no PostGIS/TIGER noise to filter out in the first place). CI's exact migration command sequence reproduced locally end to end before touching the workflow file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -1,21 +1,18 @@
|
||||
"""Baseline: identity tables, two runtime DB roles, RLS policies.
|
||||
"""Baseline: identity tables (users, invites, sessions, api_tokens).
|
||||
|
||||
This migration creates the schema AND the two application-facing Postgres roles it depends on
|
||||
(velodrome_app, velodrome_auth) — see db.py and auth/service.py module docstrings for the full
|
||||
rationale. It runs as a privileged/owner connection (VELODROME_DATABASE_URL_MIGRATE), which is
|
||||
why it's able to CREATE ROLE and GRANT at all; neither of the two roles it creates could do this
|
||||
to itself.
|
||||
No roles, no RLS, no GRANTs — SQLite has none of those concepts. Isolation between users is
|
||||
enforced entirely at the application layer now; see db.py and CLAUDE.md's invariant #4, and
|
||||
docs/DECISIONS.md D15 for the full story of why this migration looks nothing like the Postgres
|
||||
version it replaced.
|
||||
|
||||
Revision ID: 0001
|
||||
Revises:
|
||||
Create Date: 2026-09-21
|
||||
"""
|
||||
|
||||
import os
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.dialects import postgresql as pg
|
||||
|
||||
from alembic import op
|
||||
|
||||
@@ -25,77 +22,10 @@ branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def _require_password(env_var: str) -> str:
|
||||
value = os.environ.get(env_var)
|
||||
if not value:
|
||||
raise RuntimeError(
|
||||
f"{env_var} must be set before running this migration — see deploy/.env.example. "
|
||||
"There is no default: these are the passwords for real runtime database roles."
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def _dollar_quoted(password: str) -> str:
|
||||
"""Safely embed a password literal in DDL.
|
||||
|
||||
CREATE ROLE's PASSWORD clause is DDL, not DML — it does NOT accept bind parameters over the
|
||||
wire (Postgres rejects `PASSWORD $1` with a syntax error; ask how we know). Dollar-quoting
|
||||
sidesteps manual escaping entirely rather than hand-rolling quote-doubling, which is easy to
|
||||
get subtly wrong for a password an operator chose.
|
||||
"""
|
||||
tag = "$velodrome_pw$"
|
||||
if tag in password:
|
||||
raise RuntimeError(
|
||||
f"password must not contain the literal sequence {tag!r} — pick a different one"
|
||||
)
|
||||
return f"{tag}{password}{tag}"
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
app_password = _require_password("VELODROME_DB_APP_PASSWORD")
|
||||
auth_password = _require_password("VELODROME_DB_AUTH_PASSWORD")
|
||||
|
||||
# --- Runtime roles -------------------------------------------------------------------
|
||||
# velodrome_app: NOBYPASSRLS — every request-scoped query after identity is established.
|
||||
# velodrome_auth: BYPASSRLS — ONLY auth/service.py's pre-identity lookups. See db.py.
|
||||
# Roles are cluster-wide in Postgres, not per-database — if this migration ever runs
|
||||
# against a second database sharing the same cluster (exactly what a local dev + test setup
|
||||
# commonly looks like), a plain CREATE ROLE fails with "role already exists" even though
|
||||
# THIS database has never seen the migration before. Postgres has no CREATE ROLE IF NOT
|
||||
# EXISTS, so the standard idiom is catching duplicate_object in a DO block. ALTER ROLE
|
||||
# afterwards keeps the password in sync with the current env var either way, rather than
|
||||
# silently keeping whatever password the role happened to be created with previously.
|
||||
op.execute(
|
||||
f"""
|
||||
DO $$
|
||||
BEGIN
|
||||
CREATE ROLE velodrome_app LOGIN PASSWORD {_dollar_quoted(app_password)}
|
||||
NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS;
|
||||
EXCEPTION WHEN duplicate_object THEN
|
||||
ALTER ROLE velodrome_app WITH LOGIN PASSWORD {_dollar_quoted(app_password)}
|
||||
NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION NOBYPASSRLS;
|
||||
END
|
||||
$$;
|
||||
"""
|
||||
)
|
||||
op.execute(
|
||||
f"""
|
||||
DO $$
|
||||
BEGIN
|
||||
CREATE ROLE velodrome_auth LOGIN PASSWORD {_dollar_quoted(auth_password)}
|
||||
NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION BYPASSRLS;
|
||||
EXCEPTION WHEN duplicate_object THEN
|
||||
ALTER ROLE velodrome_auth WITH LOGIN PASSWORD {_dollar_quoted(auth_password)}
|
||||
NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION BYPASSRLS;
|
||||
END
|
||||
$$;
|
||||
"""
|
||||
)
|
||||
|
||||
# --- Tables ----------------------------------------------------------------------------
|
||||
op.create_table(
|
||||
"users",
|
||||
sa.Column("id", pg.UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("id", sa.Uuid(as_uuid=True), primary_key=True),
|
||||
sa.Column("email", sa.String(320), nullable=False, unique=True),
|
||||
sa.Column("display_name", sa.String(200), nullable=False),
|
||||
sa.Column("password_hash", sa.Text(), nullable=False),
|
||||
@@ -103,18 +33,16 @@ def upgrade() -> None:
|
||||
sa.Column("timezone", sa.String(64), nullable=False, server_default="UTC"),
|
||||
sa.Column("unit_system", sa.String(10), nullable=False, server_default="imperial"),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False, server_default=sa.true()),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
|
||||
),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
)
|
||||
|
||||
op.create_table(
|
||||
"invites",
|
||||
sa.Column("id", pg.UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("id", sa.Uuid(as_uuid=True), primary_key=True),
|
||||
sa.Column("code_hash", sa.LargeBinary(32), nullable=False, unique=True),
|
||||
sa.Column(
|
||||
"created_by",
|
||||
pg.UUID(as_uuid=True),
|
||||
sa.Uuid(as_uuid=True),
|
||||
sa.ForeignKey("users.id"),
|
||||
nullable=False,
|
||||
),
|
||||
@@ -128,23 +56,20 @@ def upgrade() -> None:
|
||||
|
||||
op.create_table(
|
||||
"sessions",
|
||||
sa.Column("id", pg.UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("id", sa.Uuid(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"user_id",
|
||||
pg.UUID(as_uuid=True),
|
||||
sa.Uuid(as_uuid=True),
|
||||
sa.ForeignKey("users.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("token_hash", sa.LargeBinary(32), nullable=False, unique=True),
|
||||
sa.Column("client", sa.String(20), nullable=False, server_default="web"),
|
||||
sa.Column("user_agent", sa.Text(), nullable=True),
|
||||
sa.Column("ip", pg.INET(), nullable=True),
|
||||
sa.Column(
|
||||
"created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
|
||||
),
|
||||
sa.Column(
|
||||
"last_seen_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
|
||||
),
|
||||
# Plain string — SQLite has no INET type, and nothing queries the address's structure.
|
||||
sa.Column("ip", sa.String(45), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
|
||||
)
|
||||
@@ -152,87 +77,29 @@ def upgrade() -> None:
|
||||
|
||||
op.create_table(
|
||||
"api_tokens",
|
||||
sa.Column("id", pg.UUID(as_uuid=True), primary_key=True),
|
||||
sa.Column("id", sa.Uuid(as_uuid=True), primary_key=True),
|
||||
sa.Column(
|
||||
"user_id",
|
||||
pg.UUID(as_uuid=True),
|
||||
sa.Uuid(as_uuid=True),
|
||||
sa.ForeignKey("users.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("name", sa.String(200), nullable=False),
|
||||
sa.Column("token_hash", sa.LargeBinary(32), nullable=False, unique=True),
|
||||
sa.Column(
|
||||
"scopes",
|
||||
pg.ARRAY(sa.String()),
|
||||
nullable=False,
|
||||
server_default="{}",
|
||||
),
|
||||
# JSON-encoded TEXT — SQLite has no array type; see models/identity.py.
|
||||
sa.Column("scopes", sa.JSON(), nullable=False, server_default="[]"),
|
||||
sa.Column("last_used_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
|
||||
)
|
||||
op.create_index("ix_api_tokens_user_id", "api_tokens", ["user_id"])
|
||||
|
||||
# --- Grants ------------------------------------------------------------------------------
|
||||
# velodrome_auth needs full read/write on these four tables — it's the only thing that ever
|
||||
# creates a user, a session, or redeems an invite. velodrome_app needs the same grants
|
||||
# because RLS policies (below) restrict WHICH ROWS it sees, not whether the underlying
|
||||
# privilege exists — GRANT and POLICY are two independent layers, both required.
|
||||
# Deliberately NOT granting TRUNCATE: it's a whole-table operation that RLS cannot filter
|
||||
# (Postgres RLS policies do not apply to TRUNCATE at all), so granting it to velodrome_app
|
||||
# would let any bug in ordinary request-handling code wipe an entire table in one statement,
|
||||
# defeating the isolation these policies exist to provide. Neither runtime role needs it —
|
||||
# tests use DELETE for fixture cleanup instead (see tests/conftest.py).
|
||||
for table in ("users", "invites", "sessions", "api_tokens"):
|
||||
op.execute(f"GRANT SELECT, INSERT, UPDATE, DELETE ON {table} TO velodrome_app")
|
||||
op.execute(f"GRANT SELECT, INSERT, UPDATE, DELETE ON {table} TO velodrome_auth")
|
||||
|
||||
# --- Row-level security ------------------------------------------------------------------
|
||||
# Policies below apply to velodrome_app only — velodrome_auth has BYPASSRLS and ignores them
|
||||
# entirely, by design (see module docstring). current_setting('app.user_id', true) returns
|
||||
# NULL when unset, which makes every policy below deny-by-default for an unscoped connection.
|
||||
|
||||
op.execute("ALTER TABLE users ENABLE ROW LEVEL SECURITY")
|
||||
op.execute(
|
||||
"CREATE POLICY own_row ON users FOR ALL "
|
||||
"USING (id = current_setting('app.user_id', true)::uuid) "
|
||||
"WITH CHECK (id = current_setting('app.user_id', true)::uuid)"
|
||||
)
|
||||
|
||||
op.execute("ALTER TABLE sessions ENABLE ROW LEVEL SECURITY")
|
||||
op.execute(
|
||||
"CREATE POLICY own_rows ON sessions FOR ALL "
|
||||
"USING (user_id = current_setting('app.user_id', true)::uuid) "
|
||||
"WITH CHECK (user_id = current_setting('app.user_id', true)::uuid)"
|
||||
)
|
||||
|
||||
op.execute("ALTER TABLE api_tokens ENABLE ROW LEVEL SECURITY")
|
||||
op.execute(
|
||||
"CREATE POLICY own_rows ON api_tokens FOR ALL "
|
||||
"USING (user_id = current_setting('app.user_id', true)::uuid) "
|
||||
"WITH CHECK (user_id = current_setting('app.user_id', true)::uuid)"
|
||||
)
|
||||
|
||||
op.execute("ALTER TABLE invites ENABLE ROW LEVEL SECURITY")
|
||||
op.execute(
|
||||
"CREATE POLICY own_rows ON invites FOR ALL "
|
||||
"USING (created_by = current_setting('app.user_id', true)::uuid) "
|
||||
"WITH CHECK (created_by = current_setting('app.user_id', true)::uuid)"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
for table in ("invites", "api_tokens", "sessions", "users"):
|
||||
op.execute(f"DROP POLICY IF EXISTS own_rows ON {table}")
|
||||
op.execute(f"DROP POLICY IF EXISTS own_row ON {table}")
|
||||
|
||||
# Children before parents (FK order).
|
||||
# Children before parents (FK order) — matters even with ON DELETE CASCADE enforced (see
|
||||
# db.py's PRAGMA foreign_keys=ON), since dropping a table Postgres-style doesn't rely on that
|
||||
# at all; SQLite's DROP TABLE has no dependency ordering of its own to lean on.
|
||||
op.drop_table("api_tokens")
|
||||
op.drop_table("sessions")
|
||||
op.drop_table("invites")
|
||||
op.drop_table("users")
|
||||
|
||||
# Dropping the tables drops the GRANTs that referenced them; the roles themselves remain
|
||||
# until explicitly dropped here.
|
||||
op.execute("DROP ROLE IF EXISTS velodrome_app")
|
||||
op.execute("DROP ROLE IF EXISTS velodrome_auth")
|
||||
|
||||
Reference in New Issue
Block a user