BBergle/bike-app
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add planning docs for self-hosted cycling app

Browse Source 
Planning output only; no application code yet.

Key findings driving the design:

- The Bryton Rider 650 has on-device Wi-Fi (Main Menu -> Data Sync) and
  uploads to Bryton's cloud with no phone and no Bryton Active app. Paired
  with the reverse-engineered Bryton cloud API — which returns the original
  unmodified FIT bytes — this makes ride sync fully hands-off, and higher
  fidelity than the current Strava route (Strava's API cannot return the
  original file, only smoothed streams).
- Build fresh rather than forking Endurain or FitTrackee; borrow Endurain's
  gear/component structure and strava-gear's retroactive time-ranged wear
  computation.
- PWA rather than a native iOS app: iOS 16.4+ gives home-screen PWAs real
  push notifications, which was the only thing that used to force native.

Docs:
  docs/PLAN.md       stack, schema, ingestion, auth, notifications, roadmap,
                     CI/CD, risks, verification
  docs/RESEARCH.md   Bryton cloud protocol, FIT library comparison,
                     maintenance intervals, geo services, Gitea gotchas
  docs/DECISIONS.md  decisions taken, alternatives rejected, rationale

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
BBergle
2026-09-20 20:55:30 -04:00
co-authored by Claude Opus 5
commit d3f5ed7e7b
5 changed files with 1239 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PLAN.md
+674
View File
@@ -0,0 +1,674 @@
# Self-Hosted Cycling App — Implementation Plan
## Context
You want a self-hosted Strava replacement that syncs rides from your **Bryton Rider 650**, tracks
your cumulative mileage, and adds two things Strava does badly or not at all: a **spare-parts
inventory** and a **maintenance record**. It runs on your own Linux server, with source control in
self-hosted Gitea and an act_runner on the same box for automated builds.
The specific pain driving this: today your Rider 650 syncs over Bluetooth to the Bryton Active
phone app, which forwards to Strava — but Active has no background sync, so **you have to remember
to open the app**. Research turned up a better path that removes the phone from the loop entirely
(see "The sync breakthrough" below).
You also want **mileage-milestone notifications** — "every 200 miles, clean and lube the drivetrain" —
which makes the maintenance side push-based rather than something you have to remember to go look at.
Directory is empty; this is greenfield. Decisions already made:
- **Build fresh** (borrow data models from Endurain and strava-gear, don't fork either)
- **Multi-user** — you plus friends/family, invite-only
- **PWA, not a native iOS app** — installed to the iPhone home screen
- **Imperial units by default** — you think in miles; storage stays SI, display is miles
- **Phased roadmap** with the self-hosting-only features staged in deliberately
---
## The sync breakthrough
Two facts verified during research change the design:
1. **Your Rider 650 has on-device Wi-Fi.** Its main menu has a `Data Sync` entry where the head unit
itself joins a Wi-Fi hotspot and uploads tracks to Bryton's cloud — **no phone, no Active app**.
2. **Bryton's cloud API is fully reverse-engineered** and returns the **original, unmodified FIT
bytes**. Working MIT reference implementation: `github.com/jorge-huxley/intervalssync`
(Python, updated 2026-09-17).
So the pipeline becomes fully hands-off:
```
ride ends → Rider 650 joins home Wi-Fi → uploads to Bryton cloud
→ your server's poller fetches the original FIT → app
```
**First action before writing any code:** on the Rider 650, go to `Main Menu → Data Sync`, join your
home Wi-Fi, and confirm a test ride uploads without the phone. If it only syncs on manual menu
trigger rather than automatically, the fallback is still good (USB, below) — but verify this, because
it determines whether Phase 2 fully solves your complaint.
**Why not Strava as the source:** Strava's API has no `export_original` endpoint — you get decoded,
smoothed streams, never the original file. Its June 2026 tier restructure also caps new apps at 10
users and requires the *developer* to hold a paid Strava subscription. Dead end; skip it.
**Why not Bluetooth direct:** Bryton's BLE sync protocol is not reverse-engineered by anyone —
no Gadgetbridge support, no ANT-FS, no published UUIDs. No third-party app, native or otherwise,
can pull rides off the head unit. Explicitly out of scope.
---
## Stack
| Layer | Choice | Why |
|---|---|---|
| API | Python 3.12, FastAPI, Pydantic v2, SQLAlchemy 2.0 async, Alembic | `fitdecode` is Python-only, so Python is forced; FastAPI emits OpenAPI 3.1 free |
| DB | PostgreSQL 16 + PostGIS 3.4 (`postgis/postgis:16-3.4`) | Needed for heatmaps, bbox queries, self-segment matching |
| Jobs | **procrastinate** (Postgres-backed queue) | Transactional enqueue; no Redis; built-in cron and retries |
| Frontend | SvelteKit `adapter-static` **SPA + PWA** — no Node process in prod | Static files served by Caddy; enforces API-first by construction |
| Maps | MapLibre GL JS from day one | Renders raster tiles now, self-hosted PMTiles vector later — a config change, not a rewrite |
| FIT parsing | `fitdecode` | Thread-safe, preserves header+CRC, correct developer-field handling; `python-fitparse`'s own maintainers point here |
| Auth | Argon2id + **opaque bearer tokens, no JWT** | Instant revocation, device list, no key rotation. Statelessness buys nothing at 15 users |
| Notifications | `apprise` library (ntfy default) | A library, not another container |
**Rejected:** Redis (nothing needs it), TimescaleDB (see streams decision), SSR, Celery, native iOS.
### The PWA decision
A home-screen-installed PWA gets you the icon, full-screen chrome-free display, offline caching, and
— since iOS 16.4 — **real push notifications**, which is the only thing that used to force native.
Going native would cost $99/yr for an Apple Developer account, TestFlight or sideloading to get it
onto family phones, and a second codebase forever. The two real PWA gaps on iOS (Web Bluetooth,
Background Sync) don't matter here: Bryton BLE is a dead end anyway, and the *server* does all syncing.
Design consequences — the iOS-specific traps, all of which have cheap answers if you know them upfront:
| Limitation | Design response |
|---|---|
| **Push requires "Add to Home Screen"** | Silently fails from a plain Safari tab. Treat install as a mandatory onboarding flow with a persistent card (iOS has no `beforeinstallprompt`, so there's no programmatic install). Only offer "Enable reminders" once `display-mode: standalone` is true. |
| **A denied notification permission is sticky** | The user must delete and reinstall the PWA to be asked again. So: request only from a direct user gesture, only in standalone, only after an explanatory screen. Never burn the prompt. |
| **An installed PWA has its own storage partition, separate from Safari** | The user *will* be logged in in Safari and logged out in the installed app and think it's broken. Document it in onboarding; 90-day cookie; "keep me signed in" on by default. |
| No Web NFC | Works anyway with **no app**: an NTAG sticker encoding `https://host/b/<tag_uid>` — iOS background tag reading fires from the lock screen and opens the URL, and the PWA's scope claims it so it opens *in* the app. Server-side that's one route. |
| No `BarcodeDetector` | `getUserMedia` → **`zxing-wasm`**, lazy-loaded so the ~300KB WASM isn't in the shell bundle. |
| No Background Sync | Irrelevant — all syncing is server-side. Offline writes go to a small IndexedDB outbox flushed on `online`/`visibilitychange`, keyed by the **client-generated UUIDv7 that is already the row's PK**, so replay is idempotent with no server-side idempotency table. |
| Storage eviction | Bounded caches with explicit `ExpirationPlugin` limits (unbounded caches are what trigger eviction of the *whole origin*). Never treat client storage as durable; mark unsaved outbox items visibly rather than optimistically pretending they saved. |
| Aggressive service-worker termination | The SW does two things only: caching and displaying pushes. Nothing load-bearing. |
Service worker via `@vite-pwa/sveltekit` in `injectManifest` mode. Caching: precache the hashed shell;
cache-first on `index.html` for navigations (so it opens instantly and offline); **NetworkFirst with a
3s timeout on `/garage` and `/maintenance/due`** so the garage with no signal still shows "chain: 4,100
/ 4,800 mi" — the highest-value offline surface in the app; CacheFirst on tiles; **network-only on all
mutations**, never silently cached.
Keep the backend API-first anyway (versioned `/api/v1`, bearer tokens, committed OpenAPI schema).
It costs almost nothing, and if Apple ever makes the PWA route untenable, a native client becomes a
code-generation exercise against a stable contract rather than a rewrite.
---
## Service topology
**v1 — 4 containers, under 2GB RAM total:**
- `caddy` — serves the static SPA, proxies `/api/*`. Same-origin, so no CORS. Your existing reverse proxy terminates TLS in front.
- `api` — uvicorn/FastAPI. Runs Alembic on entrypoint.
- `worker` — same image, `procrastinate worker`. Owns periodic tasks too, so no separate scheduler.
- `db` — postgis/postgis:16-3.4.
Volumes: `pgdata`, `blobstore` (content-addressed raw FIT + attachments), `import_inbox` (USB watch folder bind-mount).
**Phase 2 adds zero containers** (poller is a periodic task; Open-Meteo is a public API).
**Phase 3 adds two:** `tileserver` (tileserver-gl-light + regional PMTiles, ~200MB–1GB) and `topodata` (Open Topo Data + region-clipped SRTM).
**Phase 4+ behind opt-in compose profiles:** `ollama`, `grafana`, and optionally `valhalla`/`photon`/`overpass`.
**Explicitly NOT on day one:** Valhalla, Photon, Overpass, Nominatim (>1TB/128GB RAM — never), Ollama, Redis, MinIO, Grafana, a separate scheduler.
---
## Schema
Conventions: UUIDv7 PKs (time-ordered, URL-safe, client-generatable). All `timestamptz` UTC. **All
physical quantities as SI integers** — distance in metres, time in seconds, speed in mm/s, altitude
in cm, money in minor units. Units are a presentation concern.
### Two architectural rules that everything else depends on
**Rule 1 — raw bytes are the only truth.** Every ingested file is written to a content-addressed blob
store *before* parsing and is never mutated or deleted (`ON DELETE RESTRICT`). Every table below is a
**rebuildable projection**: delete the projection, re-parse, and you must get the same result. This is
what turns "parser bug corrupted 4,000 elevations" and "retroactively reprocess all history against a
better DEM" from disasters/migrations into routine batch jobs. It is the single most important rule here.
**Rule 2 — there is no odometer column anywhere.** Component wear is *derived* by replaying the
activity stream against time-ranged install records (the strava-gear insight, made relational).
Correcting "I actually swapped that chain a week earlier" is one `UPDATE`, and every downstream number
self-corrects. A stored counter cannot do that, nor handle parts moving between bikes, without a
reconciliation nightmare.
### Tables
**Identity:** `users` (carries `timezone` and `unit_system`, **defaulting to imperial** for you — it
drives display only, never storage), `invites` (only `sha256(code)` stored), `sessions` (opaque token
hashes), `api_tokens` (scoped, for Home Assistant/Grafana).
**Raw storage:** `raw_files` — `content_sha256`, `storage_path` (`blobstore/ab/cd/<sha>.fit`),
`source` (`upload|usb|bryton_cloud|gpx_import`), `source_ref`, `parse_state`
(`pending|parsed|failed|quarantined|not_an_activity`), `parser_version`, `fit_type`.
`UNIQUE (user_id, content_sha256)`.
**Activities:** `activities` (+ `activity_laps`, `activity_stats`). Notable columns:
- `ascent_device_m` **and** `ascent_dem_m` kept separately with an `elevation_source` flag, so Phase 3
DEM reprocessing never destroys the barometric original.
- `track geometry(LineStringZM, 4326)` full-res, plus `track_simplified` (~10m) for list maps, plus a
generated `bbox`. GIST index on the simplified track.
- `wet_fraction` (denormalised from weather, so the wear query needs no join), `is_indoor`,
`counts_for_wear` (user override).
- `fit_time_created` + `device_serial` → partial unique index. This is the natural dedupe key.
**Streams — columnar arrays, one row per channel** (`activity_streams`: `channel`, `n`, `scale`,
`values_i32[]`). Decision and justification:
- **Size.** A 3h ride at 1Hz × ~9 channels: normalized per-sample rows ≈ 1.2MB + 0.3MB index;
scaled-int arrays ≈ 150–250KB after TOAST/LZ4, zero index cost. ~5× multiplier, compounding forever
under a full-retention requirement.
- **Read pattern.** Every real query is "give me the whole stream to draw a chart" — one TOAST fetch,
and it maps 1:1 onto the JSON the client wants with no row-to-column pivot.
- **Lat/lon are free.** FIT stores position as int32 semicircles; `values_i32` holds them losslessly.
- **Not JSONB** (untyped, 3–5× larger, slow to deserialise). **Not TimescaleDB** — it solves
cross-entity scans over a firehose; we do per-entity blob reads. Adopting it means abandoning the
`postgis/postgis` base image and taking on extension-version coupling at every Postgres upgrade.
- Aggregates (HR zone totals, power curve) are **precomputed at ingest** into `activity_stats`, never
scanned from streams.
**Bikes and parts** — the part of the schema most people get wrong:
- `bikes` — includes `initial_distance_m` (km ridden before this system existed) and `nfc_tag_uid`.
- `component_models` — shared catalogue (kind, manufacturer, model, `spec jsonb`, `gtin` barcode).
- `components` — a **tracked individual physical object** with identity and history. Has
`purchase_cost_minor`, `initial_distance_m`, `inventory_item_id` provenance.
- `component_installs` — **time-ranged association**, mounted to a bike XOR a parent component
(cassette → wheelset → bike). A GIST `EXCLUDE` constraint enforces that a component can only be in
one place at a time. Moving a wheelset between bikes is two rows.
- `inventory_items` — **fungible shelf stock** with a `quantity`, `min_quantity` low-stock threshold,
`location`, `gtin`. Plus `inventory_transactions`, an append-only ledger whose running sum *is* the
quantity.
**Why inventory and components are separate tables:** a spare chain on the shelf has no identity worth
tracking — you own "3 × Shimano CN-M8100", not three named chains. A *fitted* chain has identity,
install history, accrued km, and a cost-per-km. "Install from inventory" is the state transition:
decrement quantity, create a `components` row carrying the cost and a provenance link, create an
install row. Modelling stock as components-without-installs forces fake identities onto consumables
(sealant, cables, bar tape) and makes "how many chains do I have left?" a COUNT over a table that also
contains every chain you retired since 2019.
**Maintenance:** `service_rules` (scoped to a component XOR bike XOR kind), `service_events`,
`service_event_components`, `attachments` (receipts, photos, manuals — polymorphic on entity_type/id).
**Notifications:** `push_subscriptions`, `notification_log`, `notification_prefs`,
`odometer_milestones` — defined in the milestones section below.
**Integrations:** `integration_credentials` (AES-GCM encrypted, key from the compose `.env`),
`integration_health` (consecutive failures + error taxonomy, drives alerting).
**Weather:** `weather_observations` keyed on a **0.05° grid cell + UTC hour**, so nearby rides reuse
cached data, plus a per-activity `activity_weather` rollup.
### The wear engine
`service_rules` carries `metric` (`distance | ride_time | calendar`), `threshold`, `basis`
(`since_install | since_last_service`), `wet_multiplier`, and `include_indoor`. Three SQL layers:
1. `v_component_activity` — joins installs to activities on the time range.
2. `component_usage(component, since, wet_mult, include_indoor)` — sums
`distance_m * (1 + (wet_mult - 1) * wet_fraction)`, so a fully-wet ride on rim pads
(`wet_multiplier = 4.0`) counts 4×, a dry ride 1×, a half-wet ride 2.5×. Raw distance is retained
separately so the UI can show *"480 km ridden / 1,150 km effective wear"*.
3. `v_component_due` — percentage used, remaining, and a **projected due date** from your trailing
90-day rate.
One view answers every maintenance question in the product:
- chain @ 4,800km — `distance`, `since_install`, wet 2.0
- chain wax @ 350km — `distance`, `since_last_service`, wet 3.0
- fork lowers @ 50 ride **hours** — `ride_time`, `since_last_service`, `include_indoor=false`
- sealant @ 90 **days** — `calendar`, `since_last_service`
- BB @ 6 months **OR** 4,800km — two rules on one component; whichever hits 100% first wins the badge
`component_usage_cache` is refreshed by a debounced job after ingest, install edits, and service events.
The view stays authoritative; the dashboard reads the cache.
**Cost-per-km falls straight out** of `purchase_cost_minor / raw_distance_m` — a genuinely
differentiating number no cloud service gives you, for zero incremental work once these tables exist.
---
## Mileage milestones and notifications
This is the feature that makes the maintenance side push-based instead of something you have to
remember to go and check. Two distinct kinds of milestone, sharing one delivery system.
### Kind 1 — recurring maintenance intervals
"Every 200 miles, clean and lube the drivetrain" is already expressible: a `service_rule` with
`metric='distance'`, `threshold=321869` (200 mi in metres), `basis='since_last_service'`. The
`since_last_service` basis is what makes it **recurring** — log the service, the basis moves forward,
and the counter re-arms automatically. No separate "repeating rule" concept is needed.
**Seed catalogue, shipped as defaults** so the app is useful the moment you add a bike, with every
rule editable and dismissible. Intervals below are the consensus from mainstream cycling maintenance
guides; stored in metres/seconds/days, displayed in miles.
*Recurring tasks (`basis='since_last_service'`):*
| Task | Interval | Metric | Wet × | Notes |
|---|---|---|---|---|
| **Clean & lube drivetrain** | **200 mi** | distance | 2.5 | your example; the headline default |
| Wipe & re-lube chain (dry lube) | 150 mi | distance | 3.0 | dusty conditions shorten this |
| Wipe & re-lube chain (wet lube) | 250 mi | distance | 3.0 | |
| Re-wax chain (if waxing) | 200 mi | distance | 4.0 | |
| Measure chain wear with a gauge | 500 mi | distance | 1.0 | replace at 0.5% for 11/12-speed |
| Inspect brake pads | 500 mi | distance | 2.0 | discs: replace under 1.5mm |
| Check BB & headset for play | 500 mi | distance | 1.0 | |
| Check tyre pressure | 7 days | calendar | — | |
| Inspect / top up tubeless sealant | 90 days | calendar | 1.0 | it evaporates |
| Inspect cables & housing | 1,000 mi | distance | 1.5 | |
| Bolt torque check | 90 days | calendar | — | |
| Service hub/BB/headset bearings | 2,000 mi **or** 180 days | two rules | 2.0 | whichever first |
| Replace cables & housing | 2,500 mi | distance | 1.5 | |
| Full annual service | 365 days | calendar | — | |
| Fork lowers service | 50 ride hours | ride_time | 1.0 | MTB; `include_indoor=false` |
| Fork/shock full service | 150 ride hours | ride_time | 1.0 | |
*Replacement rules (`basis='since_install'`, `is_replacement=true`, retires the component):*
| Part | Interval | Wet × |
|---|---|---|
| Chain | 2,000 mi | 2.0 |
| Cassette | 6,000 mi | 2.0 |
| Chainrings | 15,000 mi | 2.0 |
| Rear tyre | 2,500 mi | 1.5 |
| Front tyre | 5,000 mi | 1.5 |
| Disc brake pads | 1,200 mi | 4.0 |
| Rim brake pads | 1,500 mi | 4.0 |
| Bar tape | 365 days | — |
Note the rear tyre wearing 2–3× faster than the front is why `tyre_front` and `tyre_rear` are separate
`kind` values rather than one "tyre" kind with a shared interval.
### Kind 2 — odometer achievement milestones
The other reading of "mileage milestones": *"the Ribble just passed 5,000 miles"*, *"you've done 1,000
miles this year."* Cheap to add and satisfying, which is the whole point of a mileage tracker.
```sql
CREATE TABLE odometer_milestones (
id uuid PRIMARY KEY, user_id uuid NOT NULL,
scope text NOT NULL, -- 'user' | 'bike' | 'component'
scope_id uuid,
period text NOT NULL, -- 'lifetime' | 'year' | 'month'
period_key text, -- '2026' for yearly
threshold_m bigint NOT NULL,
reached_at timestamptz NOT NULL,
activity_id uuid REFERENCES activities(id), -- the ride that crossed it
UNIQUE (user_id, scope, scope_id, period, period_key, threshold_m)
);
```
Ladders (all configurable): bikes every 500 mi lifetime; you every 1,000 mi lifetime; round numbers
100/250/500/1,000/2,500/5,000/10,000; annual goal progress at 25/50/75/100%. The `UNIQUE` constraint
means a milestone fires exactly once, ever — and the `activity_id` link lets the notification say
*"your 5,000th mile on the Ribble was on this morning's ride."*
### Delivery
```sql
CREATE TABLE push_subscriptions (
id uuid PRIMARY KEY, user_id uuid NOT NULL REFERENCES users(id) ON DELETE CASCADE,
endpoint text NOT NULL UNIQUE, -- e.g. https://web.push.apple.com/...
p256dh text NOT NULL, auth text NOT NULL,
user_agent text, is_standalone boolean,
created_at timestamptz NOT NULL DEFAULT now(),
last_success_at timestamptz, failure_count int NOT NULL DEFAULT 0
);
CREATE TABLE notification_log ( -- idempotency AND the in-app inbox
id uuid PRIMARY KEY, user_id uuid NOT NULL,
kind text NOT NULL, -- 'service_due'|'milestone'|'low_stock'|'sync_failed'|'weekly_summary'
dedupe_key text NOT NULL,
title text, body text, url text,
created_at timestamptz NOT NULL DEFAULT now(),
pushed_at timestamptz, read_at timestamptz,
UNIQUE (user_id, dedupe_key)
);
CREATE TABLE notification_prefs (
user_id uuid PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
service_warn boolean DEFAULT true, -- fire at warn_at_pct (80%)
service_due boolean DEFAULT true, -- fire at 100%
milestones boolean DEFAULT true,
low_stock boolean DEFAULT true,
sync_health boolean DEFAULT true,
digest_mode text DEFAULT 'immediate', -- 'immediate' | 'daily' | 'weekly'
digest_hour smallint DEFAULT 18,
quiet_hours_start time, quiet_hours_end time -- interpreted in users.timezone
);
```
**The dedupe key is what stops this becoming spam**, and it's the one part that's easy to get wrong.
Key format: `service_due:<rule_id>:<component_id>:<cycle_seq>:<threshold>`, where `cycle_seq` is the
count of service events logged against that (component, rule) so far. Consequences:
- Within one cycle, each threshold fires **exactly once** — one nudge at 80%, one at 100%, then silence.
It does not re-notify nightly about the same chain.
- Logging the service increments `cycle_seq`, so the *next* 200-mile crossing is a new key and fires
again. That's how "every 200 miles" repeats forever without a cron-style recurrence engine.
- Milestones use `milestone:<scope>:<scope_id>:<period_key>:<threshold_m>` and are naturally once-ever.
**Evaluation job** `evaluate_notifications` runs (a) after every ingest, so a milestone or a newly-due
service arrives within minutes of the ride landing, and (b) nightly, to catch calendar-based rules that
no ride triggers. It reads `component_usage_cache` against `v_component_due`, inserts `notification_log`
rows, and enqueues sends on the `notify` queue. Quiet hours defer rather than drop.
**Channels, in order of reliability:**
1. **ntfy via `apprise`** — the primary. `apprise` is already in the stack for poller alerts, works on
iOS through the ntfy app with no PWA-install requirement, and is the most robust option. Roughly an
hour of work, which is why notifications can ship in Phase 1 rather than waiting for the PWA plumbing.
2. **Web Push (VAPID)** — the nicer experience. Apple's `web.push.apple.com` endpoint speaks standard
RFC 8291, so `pywebpush` reaches an iPhone with **no Apple Developer account and no APNs
certificate** — the only requirement is that the PWA is installed to the home screen. `410 Gone`/`404`
prunes the subscription row; `429` backs off. `navigator.setAppBadge(n)` puts a count of outstanding
due items on the home-screen icon for free.
3. **Email** via apprise, for weekly digests.
**Push is never the only path.** Every notification is a `notification_log` row rendered as an in-app
inbox, so a dead push channel degrades the experience without breaking the feature. That matters
because iOS silently drops push subscriptions after OS updates and long idle periods.
---
## Ingestion pipeline
**One canonical path.** Every source funnels through
`ingest_bytes(user_id, data, source, source_ref) -> RawFile` before any parsing. One parser, one
dedupe implementation, one set of side effects.
```
manual upload ─┐
USB watcher ─┼→ ingest_bytes() → raw_files row + blob (same txn) → enqueue parse_raw_file
Bryton poller ─┘ │
▼
fitdecode → discriminate → rebuild projection in ONE txn
→ enqueue compute_stats, enrich_weather, recompute_wear
```
Idempotent by construction: `INSERT ... ON CONFLICT (user_id, content_sha256) DO NOTHING RETURNING id`.
Nothing returned ⇒ already have it ⇒ enqueue nothing. Uploading the same file 100 times costs 100
hashes and zero rows.
**Dedupe, three layers in order:**
1. **Content hash** — catches re-uploads and USB rescans. **Filename is never consulted** (Bryton reuses names).
2. **FIT natural key** (`device_serial` + `file_id.time_created`) — catches the same ride arriving via
USB *and* the cloud where the bytes differ. This is what makes dual-source operation safe.
3. **Temporal overlap heuristic** — same user, start within 90s, duration within 5%, distance within 2%
⇒ flag `duplicate_of_id`, keep both, offer a UI action. **Never auto-delete, only auto-hide.**
**Activity vs. course discrimination — a real bug waiting to happen.** Bryton writes routes as `.fit`
too. Check `file_id.type` (4 = activity, 6 = course, 32 = monitoring). Bryton's encoder is not
Garmin's, so if the type is absent or nonstandard, fall back to message-shape inspection: ≥1 `session`
**and** ≥1 timestamped `record` ⇒ activity; `course`/`course_point` messages or positions without
timestamps ⇒ course. Anything unclassifiable ⇒ `quarantined`, blob retained, one alert, surfaced in an
admin list. **Never silently drop.**
Other Bryton hardening: map nonstandard manufacturer/product IDs via serial prefix and store the raw
values; store laps verbatim but **never derive session totals by summing laps** (use the `session`
message); compute `moving_time` from records with speed > 0.5 m/s if absent.
**Sources** implement a common `ActivitySource` protocol:
- **Upload** — `POST /api/v1/uploads`, multipart, 50MB cap, accepts `.fit`, `.fit.gz`, `.gpx`, `.tcx`.
- **USB watcher** — 60s periodic scan of `/import/inbox/**/*.fit`. A host udev rule on volume label
`Bryton` mounts the device **read-only** and rsyncs into the inbox. **Discover the subfolder at
runtime by recursive glob** — sources disagree on whether it's `Activities/`, `Actives/`, or root, so
don't hardcode it (your 650 is documented as `Bryton/Activities/`, but verify).
- **Bryton cloud (Phase 2)** — Meteor DDP over SockJS to `m3.brytonactive.com`: `login` with the
SHA-256 digest → `subscribe("activityList")` → read `userActivities` (**filter `_deleted`
tombstones**) → diff against `raw_files.source_ref` → `GET /api/activity?id=<id>` with `X-User-Id`,
`X-Auth-Token`, `x-api-key`, `User-Agent: okhttp/4.12.0` → raw original FIT bytes. Poll every 20 min,
jittered. **Vendor** the `intervalssync` protocol logic with the upstream commit SHA in a header
comment rather than taking a runtime dependency on a reverse-engineering project.
**Credential warning:** the Bryton SHA-256 digest **is** the credential — it replays as a password.
Store it AES-GCM encrypted with a key from the compose `.env` (not in the DB), never return it from any
endpoint (the Pydantic response model simply doesn't contain the field), and redact it in logs. Say so
plainly in the setup UI.
**Job runner — procrastinate**, chosen over Celery (needs Redis, poor async, Postgres broker is
second-class), arq (Redis-only — a whole container for ~50 tasks/day), and APScheduler (a scheduler,
not a durable queue — no retries, no dead-lettering). The decisive property is **transactional
enqueue**: the `raw_files` INSERT and the parse-job enqueue commit atomically on one connection. No
orphaned blobs, no jobs referencing rolled-back rows. Impossible with a Redis broker without inventing
an outbox. Queues: `ingest` (2), `enrich` (4), `maintenance` (1).
**Poller failure alerting** — `integration_health` tracks `consecutive_failures` and an error taxonomy
(`auth|protocol|network|ratelimit`). Via apprise/ntfy, max once per 24h: 3 consecutive failures; no
success in 48h while enabled; **`auth` or `protocol` errors alert immediately on the first failure** —
`protocol` is the API-changed-under-us signal. A **nightly canary** fetches the activity list only and
asserts it parses, so breakage surfaces on rest days rather than three weeks later. Persistent UI
banner while unhealthy; `/api/v1/health/integrations` feeds Home Assistant in Phase 4.
---
## Auth
**Opaque bearer tokens against a `sessions` table. No JWT.** At your scale, verification is one indexed
PK lookup (~0.1ms), and you get instant revocation, an "active devices" list, and no key-rotation or
clock-skew bugs. JWT's only advantage is stateless horizontal scale, which will never arrive here.
One token, two transports, one verification path: web gets `Set-Cookie: HttpOnly; Secure; SameSite=Lax`,
any future non-browser client gets `Authorization: Bearer`. A single FastAPI dependency reads bearer
first, then cookie, then sets `app.user_id` for RLS. **The web client gets no capability another client
lacks** — the cookie is transport convenience only.
- **CSRF:** cookie-authenticated mutations require `Origin` to match the configured public URL; bearer
requests skip it (attackers can't set that header). SameSite=Lax as defence in depth.
- **Isolation, enforced twice:** a repository layer where every query starts from a `scoped(User)` base,
**and Postgres RLS enabled from day one** with `SET LOCAL app.user_id` per request transaction.
Migrations run as a `BYPASSRLS` owner; the app connects as a non-owner. RLS is cheap now and means
re-auditing every query later. Friends/family visibility in Phase 2 is an *additive* widened policy,
never removal of the default scope.
- **Invites:** open signup does not exist as a setting. Admin generates a code; only `sha256(code)` is
stored; registration validates and increments `used_count` in the same transaction with
`SELECT ... FOR UPDATE` so a shared link can't be used twice concurrently. First user is bootstrapped
by CLI (`docker compose run api velodrome create-admin`), not a web setup wizard a scanner could race.
- Argon2id `t=3, m=64MiB, p=4`; login rate-limited per-IP and per-account in Postgres.
---
## Unique self-hosting features (staged by value/effort)
These are the payoff for self-hosting — things Strava structurally cannot do.
**Free, because they're schema properties:**
- **No privacy zones, ever.** No third party holds your data, so show real door-to-door routes.
- **Unlimited full-resolution retention**, forever, of the original files.
- **Cost-per-km on every component**, and per-kind averages ("my chains cost £0.019/km").
- **Receipts and photos attached** to parts, service events, and bikes.
- **Wet-weighted wear** — rim pads genuinely wear ~4× faster in the rain, and you have the weather data.
**Cheap and high value:**
- **Mileage-milestone and maintenance pushes** straight to your phone — the reason the garage data is
worth keeping. Strava's gear tracking can't do interval reminders at all.
- **Grafana pointed straight at Postgres** — roughly an afternoon, the cheapest analytics in the plan.
- **Barcode-scan parts** into inventory at purchase/install time (`zxing-wasm`).
- **Low-stock alerts** — "you're down to your last chain and the fitted one is at 87%".
- **Bulk-import your entire GPX archive** regardless of file count — no API quotas.
**Phase 3–4, genuinely differentiated:**
- **Retroactive elevation reprocessing** of every historical ride against a better DEM — the direct
payoff for the immutable-bytes rule.
- **Personal segment matching** against your own ride archive — your own PRs, no third-party segment
database, nothing made public.
- **Overnight batch compute** on idle server time: heatmap tiles, segment PRs, power curves.
- **Home Assistant entities** — "km until chain due", "days since last ride", "spare chains in stock".
- **NFC tag per bike** — tap the frame, its maintenance page opens (iOS Shortcuts, no app needed).
- **Local LLM ride summaries** via Ollama, behind a compose profile. Nothing leaves the house.
---
## Roadmap
Estimates assume one developer working evenings and weekends.
**Phase 0 — Scaffolding (2 weeks).** Monorepo, `uv`/`ruff`/`mypy --strict`, FastAPI skeleton with
`/healthz` and OpenAPI, Alembic baseline (users/invites/sessions **with RLS policies from the first
migration**), SvelteKit static SPA shell with login, manifest + service worker + precache passing
Lighthouse installability, VAPID keypair, Caddy, compose, Gitea Actions green, image in the registry,
deployed.
*Done when:* you log in at the real URL, add it to your iPhone home screen, it launches standalone —
and a push to `main` rebuilds and redeploys it.
**Phase 1 — Zero-touch ride history (6–8 weeks).** Ingestion core (all three dedupe layers, course
discrimination, quarantine); **the Bryton cloud poller as the primary path**, polling every 15 min with
the full `integration_health` alerting stack; USB watcher for historical backfill and as the break-glass
path; manual upload; activity list/detail with MapLibre and stream charts; totals and trends by
week/month/year and per bike, in miles; bikes CRUD; **odometer milestone notifications** (they only need
activities, so they ship here); invites; nightly `pg_dump -Fc` + restic.
*Done when:* you finish a ride, tap Data Sync on the 650, put the bike away, and it's on your phone
within 15 minutes with **zero further interaction** — and you stop opening Strava to look at your own
data. **This phase fixes your original complaint; protect it from scope creep.**
**Phase 2 — The garage (5–7 weeks).** Components and time-ranged installs; inventory with the stock
ledger and install-from-stock; service events with photo/receipt attachments; **the seeded service-rule
catalogue** across all three metrics with due/warn badges and projected-due dates;
`component_usage_cache` and its recompute job; **maintenance notifications end-to-end** (ntfy first,
Web Push second, in-app inbox always) with the cycle-seq dedupe; low-stock alerts; Open-Meteo weather
enrichment; wet-ride weighting switched on.
*Done when:* you get a push saying *"Drivetrain clean & lube due on the Ribble — 205 miles since last
time. You have 2 chains on shelf B,"* and can log the job from the garage floor.
**Phase 3 — Own the map + cheap differentiators (4–6 weeks).** Regional PMTiles + tileserver-gl, Open
Topo Data with retroactive re-elevation of the whole archive, overnight heatmap generation, personal
segment matching, cost-per-km dashboards, barcode scanning, NFC deep-link routes, nested installs
(wheelsets).
*Done when:* no third-party network request is needed in normal use.
**Phase 4 — Home and household (3–5 weeks).** Offline write outbox, friends/family visibility and a
household feed, Home Assistant entities via scoped API tokens, Grafana on the `obs` profile, Ollama ride
summaries on the `ai` profile, weekly summary push.
*Done when:* a display in the hallway shows the next thing that needs doing to a bike.
**Phase 5 — Routing (optional).** Valhalla, Photon, Overpass surface tags, route planning with `.fit`
course export back to the Rider 650. Several GB of RAM for something Komoot already does well — build
it only if Phase 4 leaves appetite.
---
## Repo layout and Gitea CI/CD
**Monorepo** — one maintainer, and API and client change together constantly.
```
apps/api/ pyproject.toml, velodrome/{api,models,ingest,sources,jobs,wear}/, alembic/, tests/
apps/web/ SvelteKit PWA, src/lib/api/ (generated types)
packages/openapi/ openapi.json ← COMMITTED; the contract artefact
deploy/ docker-compose.yml, .prod.yml, Caddyfile, .env.example,
systemd/velodrome-backup.{service,timer}, scripts/restore-drill.sh
.gitea/workflows/ ci.yml release.yml deploy.yml renovate.yml nightly.yml
```
**`ci.yml`** — push/PR with `paths:` filters so a web change doesn't run pytest.
`actions/cache@v4` works (act_runner has a built-in cache server). API job runs lint/mypy/pytest against
a **`postgis/postgis:16-3.4` service container** (service containers work in Docker mode) with real
Alembic migrations and **a corpus of ~20 real Rider 650 `.fit` files as golden parser fixtures**. An
`openapi-drift` job regenerates the schema and `git diff --exit-code`s it. A `migrations` job asserts
`upgrade → downgrade -1 → upgrade` succeeds and `alembic check` finds no model drift.
**`release.yml`** — on `v*` tags, buildx multi-stage, push to the Gitea registry.
**Authenticate with a PAT secret (`package:write` scope) — `secrets.GITEA_TOKEN` *cannot* push to the
Gitea container registry.** This is a documented Gitea limitation and will waste an hour if forgotten.
**`deploy.yml`** — `workflow_dispatch` + on release. Runs on a `[self-hosted, host]`-labelled runner in
host mode so it can reach the Docker socket: `compose pull` → `run --rm api alembic upgrade head` →
`compose up -d` → `curl -f /healthz`. Migrations run as an explicit step **before** `up -d`, never from
the container entrypoint, so failures fail the deploy visibly. Note **`jobs.*.environment` is silently
ignored by Gitea** — there are no environment protection rules, so the gate is manual dispatch plus a
namespaced `PROD_*` secret.
**`renovate.yml`** — self-hosted Renovate (Dependabot is GitHub-only), weekly cron **plus
`workflow_dispatch`**, because Gitea's cron scheduler has shipped flaky releases and you need a manual
trigger. Exclude `fitdecode` and anything Bryton-adjacent from auto-merge.
**`nightly.yml`** — Bryton canary, projection-integrity check, `restic check --read-data-subset=5%`,
heatmap/segment recompute.
**Security note:** the runner holds the Docker socket, which is root-equivalent on the host. Acceptable
for a private single-maintainer instance — but never make this repo public or add untrusted
collaborators without disabling Actions on fork PRs.
**Backups are a systemd timer on the host, not a Gitea Action** — `pg_dump -Fc` + restic to B2/S3 with
**append-only repo credentials** (so a compromised app host can't delete history), plus a restic
snapshot of `blobstore/`. Backups must not depend on CI, because CI is the thing most likely to be
broken when you need a restore. Quarterly `restore-drill.sh` restores into a throwaway stack and
asserts activity counts match.
---
## Top risks
**1. The Bryton private API breaks silently.** Hardcoded API key, undocumented protocol, zero
stability guarantee — and the failure mode is *silence*. Mitigations: the poller is one
`ActivitySource` among several, and the USB path ships first in Phase 1 so the system is never
*dependent* on it; nightly canary; immediate alerting on `auth`/`protocol` errors; vendored protocol
pinned to an upstream SHA so fixes are a diff, not a re-derivation; dual-source dedupe on the FIT
natural key means you can fall back to USB mid-week and lose nothing and duplicate nothing.
**2. Losing or corrupting years of ride history.** The realistic threats are mundane — a parser bug
writes wrong elevation to 4,000 rides, a migration drops a column, a disk dies. The raw-bytes-are-truth
rule is the control: every derived table rebuilds from the blob store, so a parser bug is a
`parser_version` bump and a requeue, not data loss. Plus offsite append-only backups independent of CI,
migration up/down/up testing in CI, golden FIT fixtures, and quarterly restore drills — an untested
backup is a hypothesis.
**3. Never shipping.** This scope is multiple person-years if attacked at once, and the failure mode is
a half-built system where you're *still* syncing manually. Mitigations: every phase has a *behavioural*
done criterion, not a feature list; v1 is capped at four containers; the heavy geo/AI services are
behind opt-in profiles in Phases 3–5; the highest-value differentiators (cost-per-km, wet-weighted
wear) are schema properties that cost nothing; and **Phase 2 is scheduled early and protected**,
because if the project stalls right after it, it has still succeeded.
---
## Verification
**Before coding:** on the Rider 650, `Main Menu → Data Sync` → join home Wi-Fi → ride → confirm the
activity reaches Bryton's cloud with the phone switched off. Then plug it in over USB and `ls -R` the
mounted volume to confirm the actual `.fit` path.
**Phase 0:** `curl https://host/healthz` returns 200; push to `main` produces a new registry image and
a redeployed container; `docker compose logs` shows migrations applied.
**Phase 1 — ingestion:**
- Upload a real Rider 650 `.fit` → activity appears with correct distance, elevation, and map track.
- Upload the **same file twice** → exactly one activity, one `raw_files` row.
- Upload a Bryton **route/course** `.fit` → does *not* become an activity; classified correctly.
- Run the poller against your real Bryton account → new rides ingest with byte-identical content to the
USB copy (compare `content_sha256`).
- Import the same ride via **both** USB and cloud → one activity; dedupe layer 2 catches it.
- Break the credential deliberately → an `auth` alert fires immediately and the UI banner appears.
- Confirm the stored Bryton credential appears in **no** API response and **no** log line.
- `pytest` green against the golden-fixture corpus in CI.
- Log in as a second invited user → sees none of your data. Verify RLS directly: `SET app.user_id` to
user B, `SELECT * FROM activities`, expect zero of user A's rows.
- Cross 1,000 miles on a bike → exactly one milestone notification, naming the ride that crossed it.
**Phase 2 — garage and notifications:**
- Create a bike, install a chain, import 3 rides → chain shows summed distance. Edit the install date
backwards → the number self-corrects with no manual recomputation.
- 90-day sealant rule → due badge at the right date. 50-ride-hour fork rule → tracks hours not miles,
and ignores indoor rides.
- Install a part from inventory → quantity decrements, a `components` row appears carrying the cost,
ledger balances.
- **The recurrence test, which is the one that matters:** set the 200-mile drivetrain rule, ride past
160 mi → one "80%" notification. Ride past 200 mi → one "due" notification. **Run the evaluator ten
more times → no further notifications.** Log the service → ride another 200 mi → it fires again.
- Import a wet ride (`wet_fraction` near 1.0) on rim pads → effective wear advances ~4× raw distance,
and the UI shows both numbers.
- Install the PWA on a second family member's iPhone, enable reminders, trigger a due rule → push
arrives on the lock screen and the app badge shows the count.
- Disable push and re-run → the notification still appears in the in-app inbox and via ntfy.
**End-to-end, the real test:** finish a ride, put the bike away, don't touch your phone. Within 15
minutes the ride is in the app with weather, every fitted component's wear has moved, and if the
drivetrain crossed 200 miles your phone has already told you.