mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-14 19:01:08 +02:00
feat(broadcast): M4 bulk-send via mana-mail + tracking infrastructure
End-to-end send path lives: click "Jetzt senden" in step 4 → client
resolves recipients → POST /v1/mail/bulk-send → mana-mail loops through
JMAP with per-recipient signed URLs → status flips draft → sent.
mana-mail (backend)
- New Postgres schema `broadcast.{campaigns,sends,events}` in Drizzle.
Campaigns + sends keyed on the webapp's local ids so joins are free;
events append-only with send_id FK, dedup at query-time not write-time
so tracking pixel hits don't contend on a transaction.
- tracking-token.ts: HMAC-SHA256 over JSON({campaignId, sendId, nonce}),
base64url.base64url encoded. JSON inner payload instead of delimiter
splits so IDs can contain any character. timingSafeEqual for the HMAC
comparison. 9 unit tests covering roundtrip / tamper / malformed.
- broadcast-orchestrator.ts: takes pre-resolved recipient list, inlines
CSS once via juice (webResources.images=false so no external fetches
slow the loop), per-recipient substitutes `{{unsubscribe_url}}` /
`{{web_view_url}}` + injects open pixel, submits each mail through
the user's own JMAP account. Writes sends rows first (status=queued)
so a crash mid-loop leaves truthful DB state. Returns aggregate
stats + per-email errors.
- Routes: POST /v1/mail/bulk-send (JWT, cap at 5000 recipients via
zod + config), GET /v1/mail/campaigns/:id/events (JWT, aggregates
opens + clicks + unsubscribes with COUNT DISTINCT for the "unique"
metric), GET/POST /v1/track/{open,click,unsubscribe}/:token (public,
no auth, signed URL is the only gate).
- Track routes mounted OUTSIDE /api/v1/mail/* because the JWT
middleware guards that subtree — recipients aren't logged in.
- Config: BROADCAST_TRACKING_SECRET (separate from SERVICE_KEY so the
blast radius of a leak stays narrow),
BROADCAST_MAX_RECIPIENTS_PER_CAMPAIGN (default 5000),
BROADCAST_MAX_RECIPIENTS_PER_HOUR (default 500, not yet enforced).
- Added juice@^11 dependency.
Webapp (client)
- api.ts: sendCampaign() resolves the audience from Dexie contacts,
renders the full email HTML + plaintext with placeholders, POSTs to
mana-mail. Contacts NEVER leave the client decrypted — the server
only sees the flat recipient list the user's client produced.
- fetchCampaignStats() for M7 dashboard/detail polling.
- ComposeView step 4 replaced: confirmation modal with "sicher?"
question, sending state with spinner, done state with delivered
count + expandable per-email error list + "Zur Übersicht" button.
- Status transitions to 'sent' with cached stats after successful
send via applyServerStatus.
Known M4 gaps (fill in M5)
- Open/click/unsubscribe track endpoints return valid responses but
event dedup is rough — one insert per hit, dedup at query time
only. M5 adds windowed IP-hash dedup.
- Synchronous send loop. 100 recipients ≈ 15s blocking. M5/M6 moves
this to an async job queue with SSE progress.
- Each recipient generates a "Sent" folder entry in the user's
Stalwart mailbox. Fine for 50-recipient newsletters, silly for
5000. Phase 2 carves out a dedicated broadcast mailbox.
Plan: docs/plans/broadcast-module.md §M4.
Next: M5 open/click tracking with dedup + rate-limits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS
parent
becba67dad
commit
f17383f9f2
14 changed files with 1410 additions and 28 deletions
90
pnpm-lock.yaml
generated
90
pnpm-lock.yaml
generated
|
|
@ -2792,6 +2792,9 @@ importers:
|
|||
jose:
|
||||
specifier: ^6.1.2
|
||||
version: 6.2.2
|
||||
juice:
|
||||
specifier: ^11.1.1
|
||||
version: 11.1.1
|
||||
postgres:
|
||||
specifier: ^3.4.5
|
||||
version: 3.4.9
|
||||
|
|
@ -9178,6 +9181,10 @@ packages:
|
|||
cheerio-select@2.1.0:
|
||||
resolution: {integrity: sha512-9v9kG0LvzrlcungtnJtpGNxY+fzECQKhK4EGJX2vByejiMX84MFNQw4UxPJl3bFbTMw+Dfs37XaIkCwTZfLh4g==}
|
||||
|
||||
cheerio@1.0.0:
|
||||
resolution: {integrity: sha512-quS9HgjQpdaXOvsZz82Oz7uxtXiy6UIsIQcpBj7HRw2M63Skasm9qlDocAM7jNuaxdhpPU7c4kJN+gA5MCu4ww==}
|
||||
engines: {node: '>=18.17'}
|
||||
|
||||
cheerio@1.2.0:
|
||||
resolution: {integrity: sha512-WDrybc/gKFpTYQutKIK6UvfcuxijIZfMfXaYm8NMsPQxSYvf+13fXUJ4rztGGbJcBQ/GF55gvrZ0Bc0bj/mqvg==}
|
||||
engines: {node: '>=20.18.1'}
|
||||
|
|
@ -10436,6 +10443,10 @@ packages:
|
|||
resolution: {integrity: sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==}
|
||||
engines: {node: '>=6'}
|
||||
|
||||
escape-goat@3.0.0:
|
||||
resolution: {integrity: sha512-w3PwNZJwRxlp47QGzhuEBldEqVHHhh8/tIPcl6ecf2Bou99cdAt0knihBV0Ecc7CGxYduXVBDheH1K2oADRlvw==}
|
||||
engines: {node: '>=10'}
|
||||
|
||||
escape-html@1.0.3:
|
||||
resolution: {integrity: sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==}
|
||||
|
||||
|
|
@ -11699,6 +11710,9 @@ packages:
|
|||
htmlparser2@10.1.0:
|
||||
resolution: {integrity: sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==}
|
||||
|
||||
htmlparser2@9.1.0:
|
||||
resolution: {integrity: sha512-5zfg6mHUoaer/97TxnGpxmbR7zJtPwIYFMZ/H5ucTlPZhKvtum05yiPK3Mgai3a0DyVxv7qYqoweaEd2nrYQzQ==}
|
||||
|
||||
http-cache-semantics@4.2.0:
|
||||
resolution: {integrity: sha512-dTxcvPXqPvXBQpq5dUr6mEMJX4oIEFv6bwom3FDwKRDsuIjjJGANqhBuoAn9c1RQJIdAKav33ED65E2ys+87QQ==}
|
||||
|
||||
|
|
@ -12411,6 +12425,11 @@ packages:
|
|||
resolution: {integrity: sha512-ZZow9HBI5O6EPgSJLUb8n2NKgmVWTwCvHGwFuJlMjvLFqlGG6pjirPhtdsseaLZjSibD8eegzmYpUZwoIlj2cQ==}
|
||||
engines: {node: '>=4.0'}
|
||||
|
||||
juice@11.1.1:
|
||||
resolution: {integrity: sha512-4SBfZqKcc6DrIS+5b/WiGoWaZsdUPBH+e6SbRlNjJpaIRtfoBhYReAtobIEW6mcLeFFDXLBJMuZwkJLkBJjs2w==}
|
||||
engines: {node: '>=18.17'}
|
||||
hasBin: true
|
||||
|
||||
jwa@2.0.1:
|
||||
resolution: {integrity: sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg==}
|
||||
|
||||
|
|
@ -12883,6 +12902,9 @@ packages:
|
|||
resolution: {integrity: sha512-DGqD7Hjpi/1or4F/aYAspXKNm5Yili0QDAFAY4QYvpqpgiY6+1jOfqpmByzjxbWd/T9mChbCArXAbDAsTm5oXA==}
|
||||
engines: {node: '>=0.12'}
|
||||
|
||||
mensch@0.3.4:
|
||||
resolution: {integrity: sha512-IAeFvcOnV9V0Yk+bFhYR07O3yNina9ANIN5MoXBKYJ/RLYPurd2d0yw14MDhpr9/momp0WofT1bPUh3hkzdi/g==}
|
||||
|
||||
merge-descriptors@1.0.3:
|
||||
resolution: {integrity: sha512-gaNvAS7TZ897/rVaZ0nMtAyxNyi/pdbjbAwUpFQpN70GqnVfOiXpeUUMKRBmzXaSQ8DdTX4/0ms62r2K+hE6mQ==}
|
||||
|
||||
|
|
@ -13154,6 +13176,11 @@ packages:
|
|||
engines: {node: '>=4'}
|
||||
hasBin: true
|
||||
|
||||
mime@2.6.0:
|
||||
resolution: {integrity: sha512-USPkMeET31rOMiarsBNIHZKLGgvKc/LrjofAnBlOttf5ajRvqiRA8QsenbcooctK6d6Ts6aqZXBA+XbkKthiQg==}
|
||||
engines: {node: '>=4.0.0'}
|
||||
hasBin: true
|
||||
|
||||
mimic-fn@1.2.0:
|
||||
resolution: {integrity: sha512-jf84uxzwiuiIVKiOLpfYk7N46TSy8ubTonmneY9vrpHNAnp0QBt2BxWV9dO3/j+BoVAb+a5G6YDPW3M5HOdMWQ==}
|
||||
engines: {node: '>=4'}
|
||||
|
|
@ -14942,6 +14969,9 @@ packages:
|
|||
resolution: {integrity: sha512-stxByr12oeeOyY2BlviTNQlYV5xOj47GirPr4yA1hE9JCtxfQN0+tVbkxwCtYDQWhEKWFHsEK48ORg5jrouCAg==}
|
||||
engines: {node: '>=20'}
|
||||
|
||||
slick@1.12.2:
|
||||
resolution: {integrity: sha512-4qdtOGcBjral6YIBCWJ0ljFSKNLz9KkhbWtuGvUyRowl1kxfuE1x/Z/aJcaiilpb3do9bl5K7/1h9XC5wWpY/A==}
|
||||
|
||||
slugify@1.6.9:
|
||||
resolution: {integrity: sha512-vZ7rfeehZui7wQs438JXBckYLkIIdfHOXsaVEUMyS5fHo1483l1bMdo0EDSWYclY0yZKFOipDy4KHuKs6ssvdg==}
|
||||
engines: {node: '>=8.0.0'}
|
||||
|
|
@ -15909,6 +15939,10 @@ packages:
|
|||
resolution: {integrity: sha512-kiGUalWN+rgBJ/1OHZsBtU4rXZOfj/7rKQxULKlIzwzQSvMJUUNgPwJEEh7gU6xEVxC0ahoOBvN2YI8GH6FNgA==}
|
||||
engines: {node: '>=10.12.0'}
|
||||
|
||||
valid-data-url@3.0.1:
|
||||
resolution: {integrity: sha512-jOWVmzVceKlVVdwjNSenT4PbGghU0SBIizAev8ofZVgivk/TVHXSbNL8LP6M3spZvkR9/QolkyJavGSX5Cs0UA==}
|
||||
engines: {node: '>=10'}
|
||||
|
||||
validate-npm-package-name@5.0.1:
|
||||
resolution: {integrity: sha512-OljLrQ9SQdOUqTaQxqL5dEfZWrXExyyWsozYlAWFawPVNuD83igl7uJD2RTkNMbniIYgt8l81eCJGIdQF7avLQ==}
|
||||
engines: {node: ^14.17.0 || ^16.13.0 || >=18.0.0}
|
||||
|
|
@ -16190,6 +16224,10 @@ packages:
|
|||
web-namespaces@2.0.1:
|
||||
resolution: {integrity: sha512-bKr1DkiNa2krS7qxNtdrtHAmzuYGFQLiQ13TsorsdT6ULTkPLKuu5+GsFpDlg6JFjUTwX2DyhMPG2be8uPrqsQ==}
|
||||
|
||||
web-resource-inliner@8.0.0:
|
||||
resolution: {integrity: sha512-Ezr98sqXW/+OCGoUEXuOKVR+oVFlSdn1tIySEEJdiSAw4IjrW8hQkwARSSBJTSB5Us5dnytDgL0ZDliAYBhaNA==}
|
||||
engines: {node: '>=10.0.0'}
|
||||
|
||||
web-streams-polyfill@3.3.3:
|
||||
resolution: {integrity: sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==}
|
||||
engines: {node: '>= 8'}
|
||||
|
|
@ -23528,7 +23566,7 @@ snapshots:
|
|||
obug: 2.1.1
|
||||
std-env: 4.0.0
|
||||
tinyrainbow: 3.1.0
|
||||
vitest: 4.1.3(@opentelemetry/api@1.9.1)(@types/node@22.19.17)(@vitest/coverage-v8@4.1.3)(@vitest/ui@4.1.3)(jsdom@29.0.2(@noble/hashes@2.0.1))(vite@6.4.2(@types/node@22.19.17)(jiti@2.6.1)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))
|
||||
vitest: 4.1.3(@opentelemetry/api@1.9.1)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.3)(@vitest/ui@4.1.3)(jsdom@29.0.2(@noble/hashes@2.0.1))(vite@6.4.2(@types/node@24.12.2)(jiti@2.6.1)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))
|
||||
|
||||
'@vitest/expect@4.1.3':
|
||||
dependencies:
|
||||
|
|
@ -23590,7 +23628,7 @@ snapshots:
|
|||
sirv: 3.0.2
|
||||
tinyglobby: 0.2.15
|
||||
tinyrainbow: 3.1.0
|
||||
vitest: 4.1.3(@opentelemetry/api@1.9.1)(@types/node@22.19.17)(@vitest/coverage-v8@4.1.3)(@vitest/ui@4.1.3)(jsdom@29.0.2(@noble/hashes@2.0.1))(vite@6.4.2(@types/node@22.19.17)(jiti@2.6.1)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))
|
||||
vitest: 4.1.3(@opentelemetry/api@1.9.1)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.3)(@vitest/ui@4.1.3)(jsdom@29.0.2(@noble/hashes@2.0.1))(vite@6.4.2(@types/node@24.12.2)(jiti@2.6.1)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))
|
||||
|
||||
'@vitest/utils@4.1.3':
|
||||
dependencies:
|
||||
|
|
@ -24953,6 +24991,20 @@ snapshots:
|
|||
domhandler: 5.0.3
|
||||
domutils: 3.2.2
|
||||
|
||||
cheerio@1.0.0:
|
||||
dependencies:
|
||||
cheerio-select: 2.1.0
|
||||
dom-serializer: 2.0.0
|
||||
domhandler: 5.0.3
|
||||
domutils: 3.2.2
|
||||
encoding-sniffer: 0.2.1
|
||||
htmlparser2: 9.1.0
|
||||
parse5: 7.3.0
|
||||
parse5-htmlparser2-tree-adapter: 7.1.0
|
||||
parse5-parser-stream: 7.1.2
|
||||
undici: 6.24.1
|
||||
whatwg-mimetype: 4.0.0
|
||||
|
||||
cheerio@1.2.0:
|
||||
dependencies:
|
||||
cheerio-select: 2.1.0
|
||||
|
|
@ -26115,6 +26167,8 @@ snapshots:
|
|||
|
||||
escalade@3.2.0: {}
|
||||
|
||||
escape-goat@3.0.0: {}
|
||||
|
||||
escape-html@1.0.3: {}
|
||||
|
||||
escape-string-regexp@1.0.5: {}
|
||||
|
|
@ -27922,6 +27976,13 @@ snapshots:
|
|||
domutils: 3.2.2
|
||||
entities: 7.0.1
|
||||
|
||||
htmlparser2@9.1.0:
|
||||
dependencies:
|
||||
domelementtype: 2.3.0
|
||||
domhandler: 5.0.3
|
||||
domutils: 3.2.2
|
||||
entities: 4.5.0
|
||||
|
||||
http-cache-semantics@4.2.0: {}
|
||||
|
||||
http-errors@2.0.1:
|
||||
|
|
@ -28898,6 +28959,15 @@ snapshots:
|
|||
object.assign: 4.1.7
|
||||
object.values: 1.2.1
|
||||
|
||||
juice@11.1.1:
|
||||
dependencies:
|
||||
cheerio: 1.0.0
|
||||
commander: 12.1.0
|
||||
entities: 7.0.1
|
||||
mensch: 0.3.4
|
||||
slick: 1.12.2
|
||||
web-resource-inliner: 8.0.0
|
||||
|
||||
jwa@2.0.1:
|
||||
dependencies:
|
||||
buffer-equal-constant-time: 1.0.1
|
||||
|
|
@ -29429,6 +29499,8 @@ snapshots:
|
|||
next-tick: 1.1.0
|
||||
timers-ext: 0.1.8
|
||||
|
||||
mensch@0.3.4: {}
|
||||
|
||||
merge-descriptors@1.0.3: {}
|
||||
|
||||
merge-descriptors@2.0.0: {}
|
||||
|
|
@ -30085,6 +30157,8 @@ snapshots:
|
|||
|
||||
mime@1.6.0: {}
|
||||
|
||||
mime@2.6.0: {}
|
||||
|
||||
mimic-fn@1.2.0: {}
|
||||
|
||||
mimic-fn@2.1.0: {}
|
||||
|
|
@ -32354,6 +32428,8 @@ snapshots:
|
|||
ansi-styles: 6.2.3
|
||||
is-fullwidth-code-point: 5.1.0
|
||||
|
||||
slick@1.12.2: {}
|
||||
|
||||
slugify@1.6.9: {}
|
||||
|
||||
smob@1.6.1: {}
|
||||
|
|
@ -33325,6 +33401,8 @@ snapshots:
|
|||
'@types/istanbul-lib-coverage': 2.0.6
|
||||
convert-source-map: 2.0.0
|
||||
|
||||
valid-data-url@3.0.1: {}
|
||||
|
||||
validate-npm-package-name@5.0.1: {}
|
||||
|
||||
validator@13.15.35: {}
|
||||
|
|
@ -33710,6 +33788,14 @@ snapshots:
|
|||
|
||||
web-namespaces@2.0.1: {}
|
||||
|
||||
web-resource-inliner@8.0.0:
|
||||
dependencies:
|
||||
ansi-colors: 4.1.3
|
||||
escape-goat: 3.0.0
|
||||
htmlparser2: 9.1.0
|
||||
mime: 2.6.0
|
||||
valid-data-url: 3.0.1
|
||||
|
||||
web-streams-polyfill@3.3.3: {}
|
||||
|
||||
web-streams-polyfill@4.0.0-beta.3: {}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue