Skip to content

Architecture

HTMX templates              Ninja API endpoints (grow as needed)
       \                       /
        \                     /
         Services layer (thin views call into this)
                   |
                   v
              Django ORM
                   |
                   v
            SQLite (Fly volume) → Postgres when needed

Server-rendered templates with HTMX for partial updates, Alpine.js for small client-side state (dropdowns, modals). Services are plain Python functions in apps/<app>/services.py. Models are Django ORM with custom managers for household scoping.

Apps

App What lives here
apps/common/ Middleware (current-household context), household-scoped managers, encrypted text field, shared context processors, static template checks
apps/households/ Household, HouseholdMembership, Person; signup service; dashboard view; invite tokens
apps/auth_google/ "Sign in with Google" flow (openid email profile); GoogleIdentity keyed by Google sub. Independent from emailscan's Gmail OAuth.
apps/documents/ Document model (with source field flagging upload / email_in / gmail_triage to drive per-source review surfaces); upload + confirm views; topic / category / search / timeline / person / named-item views; bulk operations; auto-assign; activity timeline
apps/classifier/ Claude Sonnet integration. Stub mode keyword-based; live mode sends pdfplumber-extracted page-1 text inline with the email context, with prompt caching. Page-1 text replaces base64 PDF binaries as a measured ~5x cost reduction with no precision/recall hit (validated on the spike's hand-labelled set). Non-PDF and parse-failure cases degrade to email metadata + filename signal alone. Doctype vocabulary is loaded from apps/classifier/taxonomy.yaml via taxonomy.py — single source of truth with three columns per doctype (classifier_examples, heuristic_fingerprint, suggested_tags); the latter two are populated in subsequent slices.
apps/emailscan/ Gmail OAuth + scan pipeline. Heuristic pre-filter, thread + content-hash dedup, recurring-email dedup, Claude classification, lifecycle bucketing. Scans run via the run_gmail_scan django-tasks task with three callers — OAuth callback (first scan), weekly cron, manual "Scan now" button — all funnelling through enqueue_scan which holds the single rate-limit guard (scan_state != "running"). Ongoing scans pull a delta window via scan_window_start (last_scan_at − 3 days overlap). Also the Postmark inbound webhook — persists the raw payload as InboundEmailPayload and enqueues process_inbound_email so the webhook returns <100ms regardless of attachment count (issue #85).
apps/triage/ Triage UI: two-tab nav (Gmail findings + Email-in Review). Gmail tab is the original accept/dismiss queue; Email-in Review lists already-filed source='email_in' docs that the user hasn't confirmed yet — Confirm sets reviewed_at, Edit opens the standard per-doc form. Per-tab count chips + summed nav badge. Adapter delegates to emailscan in prod, stub in tests.
apps/profiles/ PropertyProfile + VehicleProfile sidecars. Schedule definitions for houses (schedules/house.py) and vehicles (schedules/vehicle.py). DVLA + DVSA lookup service.
apps/activity/ ActivityEvent model + per-event undo (auto-assign undo, confirm-review undo).
apps/api/ Django Ninja API surface. One /api/health endpoint today; grows as consumers need them.

Data model highlights

  • Household / Person / NamedItem — the multi-tenant root. Every Document, EmailScanFinding, ActivityEvent has a household FK. NamedItem.kind is house or vehicle. NamedItem editing surfaces a kebab actions menu next to the page title via the shared templates/_partials/entity_actions_menu.html partial: Rename (POST /documents/named-items/<id>/rename/rename_named_item service, pre-checks the unique_together = ("household", "name", "kind") constraint and raises ValueError instead of IntegrityError, records named_item_renamed); Same as another … (POST /documents/named-items/<id>/merge/merge_named_items service, reassigns linked Document.named_item from source to target, optionally renames target to a user-chosen canonical name, deletes source, records named_items_merged); and Delete (POST /documents/named-items/<id>/delete/delete_named_item service, unlinks linked Documents via Document.named_item.on_delete=SET_NULL rather than cascading, records named_item_deleted). The same partial now serves Person pages too (issue #99): Rename (POST /documents/people/<id>/rename/rename_person) and Delete (POST /documents/people/<id>/delete/delete_person, which reassigns the person's Documents to the household's SELF person inside a transaction). The SELF person and any Person with a linked_user can't be deleted — removing a member who can sign in is an access decision (revoke their membership), not a tile deletion (issue #108). Joining a household via invite creates a linked Person (ensure_member_person), so every member has exactly one Person with linked_user set iff they can authenticate. People are first-class on the dashboard (issue #112): a pill strip for solo households graduating to tiles with per-person document counts, plus create_person behind /documents/people/new/. Document.person is user-editable via the "Filed under" person chip — by the file preview on the confirm form, beside the Topic chip on triage review (a filing decision, not document metadata, so it renders as switchable chrome rather than a form field; the chip drives a hidden person_id input) — the classifier's free-text person_subject guess is persisted on the Document and pre-selects the picker via resolve_person_subject, but is never silently honoured; pickerless paths (upload, inline accept) attach to SELF. PersonProfile (issue #116) is the 1:1 facts sidecar on Person — date_of_birth plus a schema-light facts dict (curated labels in FACT_LABELS, custom keys allowed) edited from the person page; activity events record fact labels only, and build_classifier_context never reads profile values. The review screen's unregistered-person hint is actionable (issue #117): an inline mini-form (HTML form= attribute, since the review page is one big form) POSTs to triage:add_person_from_hint, duplicate-guarded via resolve_person_subject; the confirm-form variant links to /documents/people/new/?name=…&next=… with an open-redirect-guarded return. The account page (/account/, issue #114) holds login-level details — display name (written to User.first_name and the SELF Person together), email, password (Django's PasswordChangeView), connected Google — and links the household settings pages (invites, suppression).
  • Document — the unit of filing. Has topic, document_type_id, provider, issue_date, expiry_date, lifecycle_state (internal — not user-visible), filing_status, reviewed_at, named_item, person, an M2M to Tag, and proposed_tags (JSONField — AI suggestions awaiting user review, cleared when the confirm form is saved). Carries the AI's proposal AND the user's eventual confirmation.
  • Tag — household-scoped, free-form, name + slug. Cross-cutting axis the user defines themselves (kids, rental, claim-2025). Slug is derived from name and is the URL/uniqueness key — "Kids" and "kids" collapse to the same Tag. Service helpers in apps/documents/tagging.py. The classifier proposes 2–5 tags per document on upload / email-in / triage-accept; suggestions render as one-click ✨-chips on the confirm form alongside chips for existing household tags the user can reuse.
  • EmailScanFinding — a candidate document found in Gmail awaiting Triage decision. Holds the AI's proposed metadata, plus enough Gmail identifiers (gmail_message_id, attachment_id) to fetch the actual file bytes later. On accept it becomes a Document; in the future the bytes will be downloaded onto Document.file at that point.
  • PropertyProfile / VehicleProfile — 1:1 sidecars on NamedItem. Property is wizard-driven (country + ownership/tenure/type/age/facts — the country step branches the questions and the checklist, issue #94); Vehicle is lookup-driven (DVLA + DVSA from the registration plate).
  • Invite — server-side household invitation (issue #98): UUID-addressed, single-use, revocable, 14-day expiry, optional invited_email binding. Minted from /household/invites/, accepted at /join/<uuid>/ (which also creates the member's linked Person), revoked from the same settings page. Replaced the signed bearer token; the old invite_url context processor is gone so no page render carries a join link.
  • DismissedSignal — learned dismissal suppression (issue #93). Dismissing a finding records/increments a household-scoped fingerprint (sender domain, normalised subject, attachment SHA-1). At dismiss_count >= 2, matching scan candidates are dropped before the classifier runs. Listed + revocable at /triage/suppression/.
  • ActivityEvent — append-only log; is_undoable flag drives the timeline Undo affordance.

The classification pipeline

Upload                      Gmail Triage
   |                              |
   v                              v
Claude (live mode)           Heuristic pre-filter (per-doctype fingerprints + household context tokens)
   |                              |
   v                              v
ClassifierProposal      Thread + content-hash + recurring-email + learned-suppression dedup
   |                              |
   v                              v
Document (filing_status=KEPT,    Claude (live mode, with email context + lazy-fetched bytes)
 reviewed_at=null)                |
                                  v
                       topic=null suppressed; the rest →
                            EmailScanFinding (status=pending) → Triage UI → Document

Both paths share the classifier service. Both produce a Document with the same shape.

The taxonomy file

apps/classifier/taxonomy.yaml is the single place a document type is defined (issue #96) — 104 doctypes across the nine topics, each with three columns: classifier_examples (fed to the prompt vocabulary), heuristic_fingerprint (keywords + sender domains the pre-Claude scoring layer matches), and suggested_tags (tag priors surfaced at confirm time). Add a doctype once and all three behaviours pick it up. python manage.py generate_doctype_fingerprints drafts fingerprints for new doctypes (offline-deterministic by default, --live asks Claude) — review the YAML diff before committing.

The heuristic + the gate

Scan-time scoring (apps/emailscan/heuristic.py) matches each candidate against every doctype fingerprint: sender-domain hit +2, keyword hit +2 in the subject / +1 in the body, capped at 4 per doctype, summed across the catalog; plus +3 per household-context token and +2 for a useful attachment, threshold ≥ 5. Household context tokens are derived at call time from registered data (apps/emailscan/context.py, issue #95): house names + postcodes, vehicle names + plates, family names, employer hints from connected email addresses. The same derived context is injected into the classifier prompt as a HOUSEHOLD CONTEXT block, along with the household's existing tag names (tag priors, slice 2e). Post-classification, refine_suggested_tags tops up suggestions from the matched doctype's curated suggested_tags — skipped when the classifier's doctype confidence is low, since wrong-doctype priors read as irrelevant recommendations (#119) — and drops tags that duplicate structured fields. On the confirm form only the AI's per-document ✨ suggestions render as chips — existing household tags surface through the AI's reuse-verbatim prompt or the input's autocomplete datalist, never as a chips row of everything. The confirm page's category field is a dropdown whose options narrow live to the selected topic (taxonomy-sourced via a json_script map; legacy free-text values stay selectable), expiry/recurrence inputs only render when they carry a value (quiet '+ add' toggles reveal them), and the document header renders on the navy shell: thumbnail (click to open) + "Filed under" chip + a quiet download link — no white file row.

The classifier runs the spike-validated 9-topic prompt (SPIKE_PROMPT_V2, Phase C cutover): it returns topic ∈ the nine topics or null. topic=null candidates are suppressed at scan time — no finding row reaches the triage UI — which is what lets Medical / Finance / Family-IDs findings surface instead of everything being squeezed through the old is_property_relevant boolean. Confidence buckets findings within a topic (a low-confidence "final" lands in Needs your eye, not Signed and final).

The schedule + completeness panel

Lives in apps/profiles/services.py:

  • _schedule_for(named_item) returns the right schedule list for the kind (house or vehicle) — and, for houses, for the property's country: UK houses get the full TA6-shaped schedule, French houses get schedules/france.py (titre de propriété, taxe foncière, DPE, diagnostic immobilier…), anywhere else gets a short generic list (issue #94).
  • _profile_for(named_item) returns the right profile sidecar.
  • evaluate_schedule(named_item) matches the schedule against the household's filed documents, classifies each entry as have / missing / not_applicable, and returns counts + a percentage.

Schedule items are dataclasses with a condition callable that reads the profile's facts dict — that's how the MOT slot gets suppressed for vehicles under 3 years old, or how leasehold-only items get hidden for freehold houses.

The operator backend

The Django admin (/admin/) is restyled to the front-end design system via templates/admin/base_site.html, which overrides the admin's CSS custom properties (navy chrome, teal accent, light high-contrast content) — no admin templates are forked beyond that thin shell, so it survives upgrades. Every colour pair is WCAG-AA-checked (body text ≥ 4.5:1); the load-bearing rule, shared with the front end, is that teal is too pale for white text, so admin buttons use navy text on teal and body links use a darker teal (#0A6E62), never the pale accent. The brand variables are applied to the light/dark theme selectors alike so the theme toggle and OS preference both resolve to the one accessible theme.

Running Maud as a business lives in two apps — see Operator console for the full reference:

  • apps/billingPlan (seeded Free / First Sweep / Ongoing) + Subscription (one per household; absent row = default plan). Limits are display-only (limits_enforced off); Stripe is planned in via stripe_* fields and a 501-until-configured webhook stub at /billing/webhooks/stripe/.
  • apps/ops — the staff-only console at /ops/ (dashboard, tenant directory + detail, plan management, suspend/reactivate/wipe, read-only view-as) and the append-only OpsAuditLog. OpsMiddleware (after CurrentHouseholdMiddleware) implements the view-as household override and the suspension gate.
  • Metering — every live classifier call writes a ClassifierUsage row (household, source, token counts; never content), priced by an in-code table so the console reports Claude cost per tenant per month. Suspended households are refused by run_scan and the inbound-email task.

Multi-tenant isolation

The HouseholdScopedManager reads the current request's household from a thread-local set by apps/common/middleware.py:CurrentHouseholdMiddleware. Every default Document.objects.all() is therefore implicitly scoped — a view that forgets to filter still won't leak.

Document.all_tenants is the explicit cross-household manager; reserved for management commands and audit code.

Contract tests in apps/common/tests/test_isolation.py and per-app test files exercise both correctness and the failure case (cross-household reads must return empty).

Interesting concrete details

  • Document.reviewed_at is stamped by both the per-block "Confirm all" and by bulk-Move-to-property — moving a doc is an assertion that it belongs there, no second-confirm step needed.
  • Topic page is a review queue. Property blocks only render if they have at least one unreviewed doc. Confirmed blocks vanish until new docs arrive. Browse-all-of-this-property's-docs lives on the property detail page.
  • Email scan dedup is four-keyed: by gmail_message_id (re-runs), by (thread_id, attachment_name) (forwarded threads), by attachment_sha1 (forwarded duplicates across threads), by (from_header, normalised_subject) (recurring marketing). On top of that, learned suppression (DismissedSignal) silently drops anything the household has dismissed twice — before the classifier is paid to look.
  • Per-document size cap. settings.MAX_DOCUMENT_BYTES (25 MB) is enforced on every Document-create path: uploads get a 413, email-in attachments are skipped with skipped_count++. Django's FILE_UPLOAD_MAX_MEMORY_SIZE / DATA_UPLOAD_MAX_MEMORY_SIZE are pinned to the same value.
  • Triage-accepted documents include the file bytes. accept_finding calls attachments().get() against the captured (gmail_message_id, attachment_id) and writes the result onto Document.file. Failure (revoked OAuth, deleted message, network) is non-fatal — the Document persists metadata-only and the user can re-run python manage.py backfill_triage_files later, which retries every empty-file Document linked to a finding.
  • Boot-time migrations. entrypoint.sh runs python manage.py migrate against the volume-backed DB on every container boot. Fly's release_command runs on a separate machine without the volume — that path silently no-op'd against an empty DB until we noticed.
  • Background task queue. Async work uses django-tasks with the db-backed django-tasks-db backend. Dev / tests default to ImmediateBackend (synchronous in-process). Prod sets TASK_BACKEND to the DB backend in fly.toml and runs python manage.py db_worker as a co-process alongside gunicorn (entrypoint.sh). The worker shares the SQLite volume — co-located until the Postgres migration lets us split it out onto a separate machine.
  • Token usage logging. The classifier logs every Claude call with input / output / cache-read / cache-creation tokens at INFO level so we can audit cost. Filenames are redacted in these (and inbound-attachment) log lines via apps/common/logging.py:redact_filename — they routinely carry personal data ("Jane_Smith_payslip.pdf") and fly logs retains whatever we emit, so we log a stable short hash plus the bare extension instead of the name.
  • Uploaded files and thumbnails are served through household-scoped views, never MEDIA_URL. MEDIA isn't served in prod (DEBUG=False) and a raw /media/ URL would be world-readable across tenants. So documents:file_view and documents:thumbnail stream the bytes from a @login_required view that filters by request.household and 404s otherwise. Page-1 thumbnails (generated async by generate_thumbnail, blank until the worker runs) render in the shared _partials/document_row.html listing and on the confirm page, falling back to a generic icon when absent.
  • Inline serving is allowlisted to prevent stored XSS. file_view only sends Content-Disposition: inline for a narrow set of safe types (INLINE_SAFE_CONTENT_TYPES: PDF + common raster images). Anything else — including an uploaded or emailed-in .html/.svg, which would otherwise run script in the app's origin against whoever opens it — is forced to download with a neutral application/octet-stream type. Every response also carries X-Content-Type-Options: nosniff so the browser honours the declared type rather than guessing.
  • Thumbnail generation is bomb-guarded. generate_thumbnail caps Image.MAX_IMAGE_PIXELS and treats Pillow's decompression-bomb error and warning as a skip, so a few-KB malicious image can't expand to gigapixels and exhaust the 512 MB worker.
  • Rate limiting is two-layered. Login brute-force goes through django-axes (DB-backed so lockout state is shared across gunicorn workers): 5 failures per (username, IP) → 1-hour 429. Signup and invite-accept use a lighter per-IP fixed-window counter in the default cache (apps/common/throttle.py) — LocMem per-process today, which is fine at these limits on a single machine. Client IP comes from Fly's Fly-Client-IP header (client_ip), shared by both layers.
  • Email changes are verified. /account/ email edits mail a signed confirmation link (django.core.signing, 3-day expiry, salt-bound) to the new address; the change applies only when the same signed-in user opens it, with uniqueness re-checked at apply time. Outbound email is env-driven (EMAIL_BACKEND — console in dev; point it at SMTP + creds in prod, e.g. Postmark outbound).