Echter Server-Cron für recurring forms — wave-send läuft jetzt
unabhängig von Owner-Tab-State. Bisheriger M10c webapp-side scheduler
bleibt als Belt-and-suspenders aktiv (idempotent).
Architektur:
1. **Owner-private internal_meta auf unlisted snapshots**
- Drizzle: neue jsonb-column `internal_meta` (Drizzle migration
0001_internal_meta.sql).
- public-routes.ts strippt sie strukturell — die explicit select()-
projection enthält sie nicht (recipients + sender würden sonst
via share-link leaken).
- publish-route akzeptiert sie im Body, persistiert auf insert +
update.
- ALLOWED_COLLECTIONS um 'lasts' und 'forms' erweitert (war ein
latenter Bug — formsStore.setVisibility('unlisted') hätte ohne
diese Ergänzung 400 zurückbekommen; M4b lief vermutlich nie
end-to-end durch).
2. **shared-privacy publishUnlistedSnapshot**
- PublishUnlistedOptions erweitert um optionales `internalMeta`.
Forwarded an /api/v1/unlisted/:collection/:recordId body.
3. **Webapp formsStore**
- lib/wave-mail.ts: buildFormInternalMeta(form, broadcastSettings)
baut den Owner-Private-Blob: { kind, recurrence: {frequency,
recipientEmails, lastSentAt}, sender: {fromEmail, fromName,
replyTo, legalAddress}, formMeta: {title, description} }.
Returns null wenn Voraussetzungen fehlen (kein recurrence, keine
recipients, fehlende broadcast-settings).
- stores/forms.svelte.ts: setVisibility / regenerateUnlistedToken /
setUnlistedExpiry laden broadcastSettings via Dexie + decrypt,
bauen internalMeta, übergeben an publishUnlistedSnapshot. Form
wird vor dem buildFormInternalMeta-Call dekrypted.
4. **mana-mail internal bulk-send route**
- createInternalRoutes(accountService, broadcastOrchestrator,
maxRecipients) — Signature erweitert.
- Neue POST /api/v1/internal/mail/bulk-send: gleicher Payload-shape
wie user-facing /v1/mail/bulk-send aber userId aus Body statt
JWT. X-Service-Key-gate sitzt bei /api/v1/internal/* prefix.
Audit-trail trägt principalId aus Body. Cap = 5000 (gleicher
Wert wie user-facing).
5. **apps/api forms wave-worker**
- 5-min setInterval, advisory-lock-gated (key 0x464f5257 'FORW').
- Tick: select snapshots WHERE collection='forms' AND
internal_meta IS NOT NULL AND revoked_at IS NULL. Filter auf
kind='forms-recurrence' + isWaveDue (lastSentAt + period <= now,
never-sent fires sofort). Pro fälligem snapshot: build HTML/text
mailbody (mirror webapp wave-mail-render), POST an mana-mail
internal-bulk-send mit X-Service-Key + userId, dann jsonb_set
auf internal_meta.recurrence.lastSentAt. Per-snapshot errors
werden als console.warn geloggt, Tick läuft weiter.
- Disable via FORMS_WAVE_WORKER_DISABLED=true (tests / multi-
replica deployments).
- Wired in apps/api/src/index.ts neben startArticleImportWorker().
Trade-offs:
- internal_meta wird beim setVisibility/regenerate/setExpiry frisch
aus broadcast-settings gebaut — wenn der User später broadcast-
settings ändert (zB neuer fromEmail) muss er das Form re-publishen
damit die snapshot-internal_meta aktualisiert wird. Doc-it: zukünftiger
Patch könnte ein "settings drift"-Warning ins UI surfacen.
- Worker-Update von lastSentAt geht NICHT zurück in den webapp-form
(settings.recurrence.lastSentAt ist verschlüsselt, server kann
nicht schreiben). Owner-UI zeigt ältere lastSentAt von manuellen
Sends; auto-cron-sends sind in den Server-Logs sichtbar. Future
patch: GET /api/v1/forms/:id/recurrence-status (auth) gibt das
snapshot.internal_meta zurück, UI rendert Auto-Cron-State.
- Webapp-side wave-scheduler (M10c) läuft parallel weiter — wenn
Owner-Tab offen ist, kann beides feuern. Idempotent durch
lastSentAt-check (weekly/monthly buckets), aber theoretisch könnte
double-fire passieren wenn die Calls innerhalb 1ms versetzt sind.
Real-world ignorierbar; future patch: scheduler liest jetzt
internal_meta.lastSentAt vom server-side state.
apps/api buildet (1776 modules). mana-mail buildet (523 modules).
svelte-check 0 errors in forms/. Forms-Tests 70/70 unverändert.
DB-Migration 0001_internal_meta.sql muss manuell appliziert werden
(siehe feedback memory: hand-authored SQL migrations sind nicht in
pnpm setup:db).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes surfaced by the end-to-end smoke test.
1. broadcast-track.ts: inner route paths double-prefixed.
Routes were declared as '/track/open/:token' etc, then
mounted at '/api/v1/track', yielding '/api/v1/track/track/open/:token'
— every tracking endpoint returned 404. Dropped the redundant
'/track/' prefix so the full path is now
'/api/v1/track/{open,click,unsubscribe}/:token' as the
orchestrator + client both expect.
Verified with live curl:
- /track/open/BAD → 200 image/gif 42 bytes (graceful no-signal)
- /track/click/?url missing → 400 missing url
- /track/click?url=javascript: → 400 bad url
- /track/click?url=https://ok + bad token → 302 graceful
- /track/unsubscribe/BAD GET → 400 HTML
- /track/unsubscribe/BAD POST → 400 (RFC 8058)
2. shared-branding/tsconfig.json: allowImportingTsExtensions
missing. shared-types/src/index.ts uses explicit .ts
imports (intentional, for Tailwind's module resolver); any
downstream tsconfig without allowImportingTsExtensions emits
8 errors. shared-auth already had this fix — shared-branding
gets the same treatment. noEmit:true is set, so no rewrite
flag needed.
Verified: shared-branding pnpm check → 0 errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The three final pre-dogfood items:
1. drizzle.config: schemaFilter now includes 'broadcast' alongside
'mail'. Without this, `bun run db:push` skipped the broadcast
tables — schema existed in code but not in Postgres. Tested via
db:push + psql \dt (3 tables created: campaigns, events, sends).
2. .env.development: new MANA-MAIL SERVICE section with Stalwart
knobs + broadcast config (tracking secret, rate limits, send
throttle). DEV secret is explicitly labelled non-production —
prod rotates via env.
3. generate-env.mjs: new block writes services/mana-mail/.env on
`pnpm setup:env`. Mirrors the invoices / research / events
pattern. All 16 broadcast/mail vars flow through from SSOT.
Verified end-to-end:
- pnpm setup:env → services/mana-mail/.env contains
BROADCAST_TRACKING_SECRET + rate limits
- bun run src/index.ts → /health returns 200 with the new config
- psql → broadcast.campaigns / events / sends are materialised
Broadcast module is now fully ready to send real mail — nothing
else required before the first dogfood campaign.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last plan milestone. Users can verify their sending-domain
setup without leaving the broadcast settings page.
Server (mana-mail)
- services/dns-check.ts: parseSpf / parseDkim / parseDmarc are pure
functions. SPF accepts include:<mailDomain>, flags weak (+all) and
wrong (include missing) and multi-record (RFC 7208 §3.2). DKIM
needs v=DKIM1 + a p= public-key segment. DMARC requires v=DMARC1,
flags p=none as weak (monitoring only), ok on quarantine/reject.
All three are case-insensitive.
- lookupTxt(): DNS-over-HTTPS against Cloudflare 1.1.1.1 — avoids
the Bun/container udp-resolver flakiness and works everywhere.
Multi-string TXT (`"a" "b"`) get concatenated before parsing.
- checkDomain(): one call, three parallel DoH lookups, returns a
structured result with suggested copy-paste records scoped to the
user's actual mail domain from config.
- Route: GET /v1/mail/dns-check?domain=&selector= (JWT auth). Zod
validates the domain looks sensible before hitting DoH.
- 16 unit tests covering all three parsers + multi-record edge case.
Client
- api.ts: runDnsCheck(domain, selector?) helper with typed result.
- components/DnsCheckBanner.svelte: derives domain from the default
from-email (after @), calls the check on-demand, renders per-record
status chips (ok / weak / wrong / missing) with messages, exposes
copy-pasteable SPF + DMARC records when anything's off. DKIM setup
is provider-specific so we show a hint rather than a canned record.
Last-check timestamp persists to settings.dnsCheck so the banner
survives a reload without re-hitting the API.
- Wired into SettingsForm between Impressum and Standard-Footer —
where the user is already thinking about "what's required to
actually send".
All checks clean:
- webapp pnpm check: 0 broadcast errors (4 pre-existing articles errors
from parallel Spaces work, unrelated)
- mana-mail tests: 36/36 across tracking-token + link-rewriter + dns-check
- mana-mail build: 2.51 MB (+8 KB for juice — dns-check itself is ~3 KB)
Plan: docs/plans/broadcast-module.md §M8. All 10 milestones now done.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the last two dogfood blockers before real-campaign use.
link-rewriter.ts
- rewriteClickLinks(): walks <a href="http…"> in the HTML body and
replaces each URL with /api/v1/track/click/{token}?url={original}
so clicks go through the tracking endpoint. Regex-based because
Tiptap output is well-formed; returns a count for debugging.
- Leaves mailto: / tel: / anchor fragments alone — wrapping those
breaks the recipient's native handler and accomplishes nothing.
- `skipUrls` param carries the unsubscribe + web-view URLs (already
tracking endpoints themselves) so they don't get double-wrapped.
- 11 unit tests covering http/https rewriting, skip list, non-http
schemes, attribute preservation, multi-link count, quoted-attr
variants, idempotency.
Orchestrator wiring
- substituteUrls now calls rewriteClickLinks after the preview-
placeholder swap and before the open-pixel injection. The
unsubscribe + web-view URLs from this same function are passed
in as skip entries so they survive the pass untouched.
- Constructor gains `sendThrottleMs` param (default 150ms).
- Main send loop awaits sleep(throttleMs) between iterations. 150ms
= ~6/sec = ~360/min, safely below most SMTP provider limits.
100-recipient campaign = ~15s extra wall-clock but that's fine
for MVP (and most campaigns are way smaller).
Config
- New env BROADCAST_SEND_THROTTLE_MS (default 150). Wired from
loadConfig to the orchestrator constructor.
The broadcast module is now functionally complete for dogfooding.
Remaining before a real campaign can actually go out: run
`cd services/mana-mail && bun run db:push` to materialise the
broadcast.* schema tables.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the "could actually dogfood" gap: legal address can be set,
sent campaigns have a proper view with live stats, and the send path
respects DSGVO.
Webapp
- components/SettingsForm.svelte: sender defaults + Impressum (required,
highlighted amber until filled) + footer. Matches the invoices
SenderProfileForm pattern — immediate save, dedicated section per
concern.
- /broadcasts/settings/+page.svelte: mounts the form. ComposeView step
3's "Einstellungen öffnen" CTA now lands somewhere.
- views/DetailView.svelte: read-only view for sent/scheduled/cancelled
campaigns. 5-card stats grid (sent, open, click, bounce, unsub) with
rate percentages. Polls mana-mail every 30s for up to 30 min after
mount, persists back to Dexie via applyServerStatus so the list view
+ widget catch up. Includes a preview of the actual rendered campaign
so "what went out" is visible after the fact.
- /broadcasts/[id]/+page.svelte: DetailView for non-drafts; drafts
bounce to /edit via $effect-triggered goto.
- ListView row-click now routes by status (draft → edit, else → detail).
mana-mail compliance
- Orchestrator loadUnsubscribedEmails(): queries broadcast.sends WHERE
status='unsubscribed' scoped to the user, filters the recipient list
BEFORE any send rows get written. Campaign's totalRecipients reflects
the post-skip count so open rates aren't inflated by "virtual sends".
Skipped count surfaces in result.errors for the UI to show.
- jmap-client.submitEmail: new extraHeaders param. Sets custom headers
via JMAP's `header:<Name>:asText` property convention.
- Orchestrator sets RFC 8058 headers per recipient:
List-Unsubscribe: <https://.../track/unsubscribe/{token}>
List-Unsubscribe-Post: List-Unsubscribe=One-Click
This is what makes Gmail / Apple Mail show their native "Abmelden"
button in the message header (not just a body link).
All checks clean: 0 TS errors, 37/37 webapp tests, 9/9 tracking-token
tests, mana-mail bun build = 2.50 MB.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end send path lives: click "Jetzt senden" in step 4 → client
resolves recipients → POST /v1/mail/bulk-send → mana-mail loops through
JMAP with per-recipient signed URLs → status flips draft → sent.
mana-mail (backend)
- New Postgres schema `broadcast.{campaigns,sends,events}` in Drizzle.
Campaigns + sends keyed on the webapp's local ids so joins are free;
events append-only with send_id FK, dedup at query-time not write-time
so tracking pixel hits don't contend on a transaction.
- tracking-token.ts: HMAC-SHA256 over JSON({campaignId, sendId, nonce}),
base64url.base64url encoded. JSON inner payload instead of delimiter
splits so IDs can contain any character. timingSafeEqual for the HMAC
comparison. 9 unit tests covering roundtrip / tamper / malformed.
- broadcast-orchestrator.ts: takes pre-resolved recipient list, inlines
CSS once via juice (webResources.images=false so no external fetches
slow the loop), per-recipient substitutes `{{unsubscribe_url}}` /
`{{web_view_url}}` + injects open pixel, submits each mail through
the user's own JMAP account. Writes sends rows first (status=queued)
so a crash mid-loop leaves truthful DB state. Returns aggregate
stats + per-email errors.
- Routes: POST /v1/mail/bulk-send (JWT, cap at 5000 recipients via
zod + config), GET /v1/mail/campaigns/:id/events (JWT, aggregates
opens + clicks + unsubscribes with COUNT DISTINCT for the "unique"
metric), GET/POST /v1/track/{open,click,unsubscribe}/:token (public,
no auth, signed URL is the only gate).
- Track routes mounted OUTSIDE /api/v1/mail/* because the JWT
middleware guards that subtree — recipients aren't logged in.
- Config: BROADCAST_TRACKING_SECRET (separate from SERVICE_KEY so the
blast radius of a leak stays narrow),
BROADCAST_MAX_RECIPIENTS_PER_CAMPAIGN (default 5000),
BROADCAST_MAX_RECIPIENTS_PER_HOUR (default 500, not yet enforced).
- Added juice@^11 dependency.
Webapp (client)
- api.ts: sendCampaign() resolves the audience from Dexie contacts,
renders the full email HTML + plaintext with placeholders, POSTs to
mana-mail. Contacts NEVER leave the client decrypted — the server
only sees the flat recipient list the user's client produced.
- fetchCampaignStats() for M7 dashboard/detail polling.
- ComposeView step 4 replaced: confirmation modal with "sicher?"
question, sending state with spinner, done state with delivered
count + expandable per-email error list + "Zur Übersicht" button.
- Status transitions to 'sent' with cached stats after successful
send via applyServerStatus.
Known M4 gaps (fill in M5)
- Open/click/unsubscribe track endpoints return valid responses but
event dedup is rough — one insert per hit, dedup at query time
only. M5 adds windowed IP-hash dedup.
- Synchronous send loop. 100 recipients ≈ 15s blocking. M5/M6 moves
this to an async job queue with SSE progress.
- Each recipient generates a "Sent" folder entry in the user's
Stalwart mailbox. Fine for 50-recipient newsletters, silly for
5000. Phase 2 carves out a dedicated broadcast mailbox.
Plan: docs/plans/broadcast-module.md §M4.
Next: M5 open/click tracking with dedup + rate-limits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend: Hono/Bun service on port 3042 with JMAP client for Stalwart,
account provisioning (@mana.how addresses on user registration),
thread/message/send/label API endpoints, and JWT + service-key auth.
Frontend: Mail module with 3-column inbox UI (mailboxes, thread list,
detail/compose), local-first encrypted drafts in Dexie, and API-driven
thread fetching. Scoped CSS with theme tokens.
Integration: Dexie v11 schema, mail pgSchema in mana_platform,
mana-auth fire-and-forget hook for account provisioning,
getManaMailUrl() in API config, app registry + branding update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
