mana-sync's billing middleware short-circuited every push/pull with
402 for users without a sync subscription. Cards promises free Sync
in its Phase-1 GUIDELINES, so it shouldn't gate its own users on a
mana-credits subscription it never sells.
Implementation:
• billing.NewChecker now takes an exemptApps slice. The middleware
extracts {appId} from the URL path and short-circuits before the
user lookup if the app is in the set.
• Configurable via the BILLING_EXEMPT_APPS env var (comma-separated).
• Set BILLING_EXEMPT_APPS=cards on the mana-sync container so the
cards.mana.how Sync loop stops 402-ing.
• Tests cover the exemption + the empty/whitespace edge cases. All
other apps keep the original behaviour (fail-open if mana-credits
is unreachable, 402 if it explicitly says inactive).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
All 5 milestones landed today in one continuous session: registry,
health cache, fallback router, observability, and consumer migration.
115 service-side tests, validator covers 2538 files.
Replaces the mana-sync event-stream export (GET /backup/export) with a
fully client-driven `.mana` v2 archive: webapp reads Dexie, decrypts
per-field, packages JSONL + manifest, optionally PBKDF2+AES-GCM seals
with a passphrase.
- New: backup/v2/{format,passphrase,export,import}.ts + format.test.ts
(10 tests: round-trip, sealed path, 3 failure modes incl. wrong-
passphrase vs. tamper distinction).
- UI: ExportImportPanel with module multi-select, optional passphrase,
progress + sealed-file detection — replaces the old backup flow in
Settings → MyData.
- Removes services/mana-sync/internal/backup/ and the corresponding
client helpers + v1 tests. No parallel paths, no legacy shim.
- Why client-driven: zero-knowledge users hold their vault key only
client-side, so a server exporter cannot produce plaintext archives;
GDPR Art. 20 portability is better served by plaintext-by-default.
- Cross-account restore works via re-encryption under the target
vault key (no MK transfer needed).
DATA_LAYER_AUDIT.md §8 rewritten to reflect the new architecture.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the gap between "invite flow UI exists" and "two users in the
same space actually see each other's data". Three pieces land together
because they're meaningless without each other.
mana-auth — new internal endpoint:
GET /api/v1/internal/users/:userId/memberships
Returns [{organizationId, role}, ...] for the user. mana-sync uses
this to populate the multi-member RLS session config.
mana-sync — membership lookup:
new internal/memberships package with an HTTP client + 5 min
per-user cache, fail-open (empty list = pre-Spaces behavior).
Config gets MANA_AUTH_URL (default http://localhost:3001).
Handler.NewHandler takes the Lookup. Every Push/Pull/Stream call
now passes spaceIDsFor(userID) to Store methods.
GetChangesSince + GetAllChangesSince extend their WHERE clause:
WHERE (user_id = $1 OR space_id = ANY($memberSpaces))
so co-members see each other's rows, not just the author.
apps/web — encryption skip for shared-space records:
encryptRecord now checks record.spaceId:
- `_personal:<userId>` sentinel OR no active shared space → encrypt
with user master key (E2E as today).
- Active space resolves to non-personal type AND spaceId matches
that space → skip encryption; write lands plaintext.
decryptRecord is unchanged because its per-field isEncrypted() guard
already passes plaintext through.
Phase-1 compromise: shared-space data is protected by server RLS
only, not E2E. Phase 2 adds per-Space shared keys with per-member
wrap — tracked in docs/plans/spaces-foundation.md.
Plus docs/plans/shared-space-smoketest.md: step-by-step Zwei-User-Test
mit erwarteten Ergebnissen und Debugging-Hinweisen bei Problemen.
Build + go test + web check all green.
Plan: docs/plans/spaces-foundation.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the second RLS policy needed for shared spaces. Users can read
rows in any space they're a member of, in addition to their own rows.
Changes:
- New policy sync_changes_space_member_read (SELECT only) uses
app.current_user_space_ids session config: rows with space_id in
that comma-separated list pass RLS.
- WITH CHECK is not extended — writes still require user_id match, so
only the author can write. Members read, owner/author writes.
- withUser() is now a thin wrapper around withUserAndMemberships(),
which accepts the caller's Space membership list and sets the new
session config alongside app.current_user_id.
- The comma-join is empty-filtered so stray blank entries can't match
rows with literal empty space_id (defense in depth).
Forward-compatible: today every space has exactly one member (the
author), so the membership list is always empty and the new policy
is a no-op — user_id isolation remains the only active guard.
When shared spaces start being used (clubs/teams/brand spaces with
invites), the HTTP handlers will fetch the caller's membership from
mana-auth and pass it to withUserAndMemberships. No migration needed
at that point — the policy is already live.
Subscription fan-out (WS/SSE broadcast to all space members) is still
per-user; that's a follow-up tied to the membership lookup infra.
Go build + existing tests pass.
Plan: docs/plans/spaces-foundation.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Server-side:
- sync_changes gains a nullable space_id TEXT column + partial index
on (user_id, space_id, app_id, created_at) WHERE space_id IS NOT NULL.
- RecordChange takes spaceID as a first-class parameter; *string so
empty strings land as real SQL NULL and the partial index skips them.
- ChangeRow + all three SELECTs (GetChangesSince, GetAllChangesSince,
StreamAllUserChanges) propagate space_id through to clients.
- changeFromRow surfaces SpaceID on the wire Change shape.
- New extractSpaceID helper reads the incoming payload — prefers top-
level spaceId, falls back to data.spaceId (inserts) or
fields.spaceId.value (updates). Tolerates pre-v28 clients.
- 6 Go tests cover the helper + round-trip.
Client-side:
- PendingChange gains an optional spaceId.
- Dexie creating hook stamps spaceId from the active record onto the
pending-change row (already set by the v28 scope hook).
- Dexie updating hook reads spaceId from the pre-update record and
stamps it on the pending-change so updates carry space context even
though spaceId itself is immutable and never in `fields`.
- buildChangeset forwards spaceId to the server.
Explicitly NOT in scope this pass:
- RLS remains user_id-scoped; multi-member shared-space reads need a
second policy that joins against auth.members. Follow-up once shared
spaces are actually used — today everything is personal.
- Subscription fan-out is still per-user; fan-out to all members of a
shared space is part of the same follow-up.
Go tests: 6/6 pass. Web type-check clean (0 errors across 7139 files).
Plan: docs/plans/spaces-foundation.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an opaque JSON `actor` column alongside the existing field_timestamps
so cross-device consumers can distinguish user / ai / system writes. The
server never parses the shape — it just stores and re-emits the blob the
webapp stamped in its Dexie hook.
- `sync/types.go` — Change.Actor as json.RawMessage with omitempty; nil
for pre-actor clients so wire remains backward-compatible
- `store/postgres.go`
- Migrate: CREATE TABLE includes `actor JSONB` for fresh DBs;
ALTER TABLE ADD COLUMN IF NOT EXISTS actor JSONB for existing ones
(idempotent, safe to re-run)
- RecordChange signature takes json.RawMessage; pgx writes nil as NULL
- All three SELECT paths (GetChangesSince, GetAllChangesSince,
StreamAllUserChanges) return actor, Scan into ChangeRow.Actor
- ChangeRow.Actor added with doc noting "missing = user" consumer rule
- `sync/handler.go` — Change.Actor threaded through HandleSync →
RecordChange, and populated on both changeFromRow (pull/POST replies)
and convertChanges (SSE stream)
- Tests: roundtrip of an AI-actor payload + omitempty verification for
pre-actor clients. All existing tests still pass.
Webapp types still need `actor?: Actor` on SyncChange + PendingChange to
match the wire, and applyServerChanges needs to stamp __lastActor /
__fieldActors from incoming changes for Workbench attribution on other
devices — both tracked as separate follow-ups.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- DATA_LAYER_AUDIT.md: new section 8 covering the export/import flow
end-to-end — architecture diagram, .mana format, protocol-stability
commitments we locked in pre-launch (eventId + schemaVersion + op
vocab + tombstones-forever), encryption-boundary argument, file
map, and the remaining backup backlog (M4b, M5, signature,
resumable download, dedup table).
- services/mana-sync/CLAUDE.md: /backup/export row in API table with
explicit note that it sits outside the billing gate, new Backup /
Restore section with format sketch + split between writer.go (pure)
and handler.go (shim), test-coverage line mentions the backup cases,
project-structure tree lists backup/*.go, Security section mentions
RLS still applies to the export path.
No code changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Refactor: HTTP handler becomes a thin shim over a pure WriteBackup(w,
userID, createdAt, iter) function. RowIterator abstracts the store, so
tests feed synthetic ChangeRow slices and production feeds
StreamAllUserChanges. Zero behavior change in production — same bytes
on the wire.
Tests (all pass):
- TestWriteBackup_Roundtrip: three rows across two apps, assert zip has
2 entries, events.jsonl has 3 JSON lines in order, insert omits
fieldTimestamps, update surfaces them, manifest apps are sorted,
eventsSha256 equals a recomputed sha of the decompressed body.
- TestWriteBackup_EmptyUser: empty userID refused up-front.
- TestWriteBackup_NoRows: zero-row export still produces a valid zip
with an empty events.jsonl and a manifest with eventCount=0 and a
non-empty sha (sha of empty input).
- TestWriteBackup_DefaultsSchemaVersionZeroRowsToOne: legacy rows with
schema_version=0 clamp to 1 so the manifest never claims a protocol
version that never existed.
Paired with the vitest zip parser suite on the TS side, this closes
the Go-writes / JS-reads round-trip without needing live mana-sync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recovering three files dropped when a parallel terminal reset past the
original M1 commit:
- cmd/server/main.go: register GET /backup/export outside billingMiddleware
- lib/api/services/backup.ts: browser-side downloadBackup() helper
- settings/my-data/+page.svelte: "Backup & Wiederherstellung" section
Pairs with the earlier backup handler + schema_version work already on
main (79996f946). With this commit the endpoint is actually reachable
end-to-end and the download button works.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sync_changes gains schema_version column (default 1, idempotent ADD)
- Change/Changeset carry schemaVersion; server refuses > MaxSupported
- server->client changes now carry eventId + schemaVersion so the
restore path can dedup via eventId and route through a migration
chain keyed on schemaVersion
- backup JSONL gains schemaVersion per line
Pre-M2 clients (omit the field) are treated as v1 for compatibility.
This is the stability contract we commit to before launch: once v1
events are in the wild, all future builds must replay them forward.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Dockerfile only copied services/mana-sync, but go.mod has a replace
directive pointing to ../../packages/shared-go which needs to be in the
build context. Switch context to repo root and copy both packages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add MANA_CREDITS_URL and MANA_SERVICE_KEY to configuration table
- Document billing gate on sync endpoints (402 behavior, 5min cache, fail-open)
- Add billing/check.go to project structure
- Add stream endpoint to API table
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Defense-in-depth on top of the existing application-level WHERE clauses:
- Migrate() now ENABLE + FORCE row level security on sync_changes and
installs a policy that gates rows on current_setting('app.current_user_id').
FORCE makes the policy apply to the table owner too, so the application
role used by mana-sync cannot bypass it regardless of grants.
- New withUser(ctx, userID, fn) helper opens a transaction and calls
set_config('app.current_user_id', userID, true) before running fn.
Empty userIDs are rejected up-front so an unauthenticated request can
never reach the database with an empty RLS scope (which would match
every row).
- RecordChange / GetChangesSince / GetAllChangesSince all run inside
withUser. WITH CHECK on the policy double-validates the user_id column
on insert against the active session, so a future code path that
forgets the WHERE clause cannot leak data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bug 1: NotifyUser() early-returned when no WebSocket clients existed,
skipping SSE subscriber notifications entirely. Fixed by restructuring
to check WS clients and SSE subscribers independently.
Bug 2: SSE stream cursor defaulted to client's `since` parameter when
no initial data existed. If `since` was in the future (or very recent),
live updates had created_at < cursor and were silently filtered out.
Fixed by defaulting cursor to now() when no initial data is returned.
Bug 3: NotifyUser used original sseSubs slice instead of sseSubsCopy
after releasing the read lock (race condition).
Verified E2E: Push from client A → SSE stream on client B receives
live change event with correct data within ~1 second.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New endpoint GET /sync/{appId}/stream sends Server-Sent Events with
change data directly, replacing the WebSocket notification + HTTP pull
round-trip pattern.
Server (Go):
- HandleStream() in handler.go: SSE endpoint with initial sync + live streaming
- Hub.Subscribe()/Unsubscribe() in hub.go: channel-based SSE subscriber system
- Notification type for type-safe SSE events
- convertChanges() helper extracted from duplicated code
- WriteTimeout set to 0 for SSE long-lived connections
Protocol: Client connects to /sync/{appId}/stream?collections=a,b&since=...
Server sends initial changes, then streams live changes as other clients sync.
Heartbeat every 30s keeps connection alive. Push still uses POST /sync/{appId}.
WebSocket remains available as fallback (not removed).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server now returns hasMore: true when there are more than 1000 changes
pending for a collection. Client continues pulling in a loop until
hasMore is false, using the last row's timestamp as cursor.
Prevents data loss after long offline periods where >1000 changes
accumulated for a single collection.
Server changes (Go):
- GetChangesSince() accepts limit parameter
- HandlePull() fetches limit+1, trims, sets hasMore
- SyncedUntil uses last row's timestamp when paginating
Client changes (TypeScript):
- Pull loop: while (hasMore) { fetch → apply → advance cursor }
- Cursor only persisted after all pages fetched
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add unified /ws endpoint that serves all app notifications over a single connection.
The server now includes appId in the sync-available message payload so the client
knows which app to pull. Legacy /ws/{appId} endpoint remains for backward compatibility.
Backend (Go):
- hub.go: Message struct gains AppId field, NotifyUser sends to all user clients
(unified clients receive everything, legacy clients filtered by appId)
- main.go: new GET /ws route (empty appId = unified mode)
Frontend (sync.ts):
- Single connectUnifiedWs() replaces 27 per-app connectWs() calls
- Parses msg.appId from server to pull only the affected app
- Reconnect/offline logic simplified to one WS
This reduces WebSocket connections from 27 per user to 1, cutting server
connection overhead by ~96%.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sync integration:
- Redirect service reads links from mana-sync's sync_changes table
- Analytics service queries clicks from sync_changes
- Click tracking writes to sync_changes (visible to all clients)
- Public profile reads from sync_changes
- Server DB points to mana_sync database (not separate uload DB)
- Removed uload-database dependency from server
Stripe:
- Real Stripe checkout session creation (monthly/yearly)
- Webhook handler with signature verification
- Webhook route bypasses JWT auth
Documentation:
- Root CLAUDE.md: added uload to project table, dev commands, local-first list
- mana-sync CLAUDE.md: added uLoad, Taktik, Calc to connected apps
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bash-based integration test that verifies the full sync cycle:
1. Health check
2. Client A pushes insert
3. Client B pulls and sees the change
4. Client B pushes update (field-level)
5. Client A pulls and sees the update
6. Client A pushes delete
7. Unauthorized request rejected (401)
Requires running mana-sync + mana-auth. Run with:
./services/mana-sync/test/e2e-sync-flow.sh [TOKEN]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement the foundational local-first data layer for ManaCore apps:
- New @manacore/local-store package (Dexie.js IndexedDB, sync engine, Svelte 5 reactive queries)
- New mana-sync Go service (sync protocol, WebSocket push, field-level LWW conflict resolution)
- Todo app migrated as pilot: stores read/write IndexedDB, guest mode with onboarding seed data
- PillNavigation: prominent login pill for unauthenticated users
- SyncIndicator component showing local/syncing/offline status
- GuestWelcomeModal on first visit for Todo app
- Removed demo-mode auth_required checks from Todo components (all writes are now local)
- CSP fix for local development (localhost:3001, localhost:3050)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
