Replaces the mana-sync event-stream export (GET /backup/export) with a
fully client-driven `.mana` v2 archive: webapp reads Dexie, decrypts
per-field, packages JSONL + manifest, optionally PBKDF2+AES-GCM seals
with a passphrase.
- New: backup/v2/{format,passphrase,export,import}.ts + format.test.ts
(10 tests: round-trip, sealed path, 3 failure modes incl. wrong-
passphrase vs. tamper distinction).
- UI: ExportImportPanel with module multi-select, optional passphrase,
progress + sealed-file detection — replaces the old backup flow in
Settings → MyData.
- Removes services/mana-sync/internal/backup/ and the corresponding
client helpers + v1 tests. No parallel paths, no legacy shim.
- Why client-driven: zero-knowledge users hold their vault key only
client-side, so a server exporter cannot produce plaintext archives;
GDPR Art. 20 portability is better served by plaintext-by-default.
- Cross-account restore works via re-encryption under the target
vault key (no MK transfer needed).
DATA_LAYER_AUDIT.md §8 rewritten to reflect the new architecture.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the gap between "invite flow UI exists" and "two users in the
same space actually see each other's data". Three pieces land together
because they're meaningless without each other.
mana-auth — new internal endpoint:
GET /api/v1/internal/users/:userId/memberships
Returns [{organizationId, role}, ...] for the user. mana-sync uses
this to populate the multi-member RLS session config.
mana-sync — membership lookup:
new internal/memberships package with an HTTP client + 5 min
per-user cache, fail-open (empty list = pre-Spaces behavior).
Config gets MANA_AUTH_URL (default http://localhost:3001).
Handler.NewHandler takes the Lookup. Every Push/Pull/Stream call
now passes spaceIDsFor(userID) to Store methods.
GetChangesSince + GetAllChangesSince extend their WHERE clause:
WHERE (user_id = $1 OR space_id = ANY($memberSpaces))
so co-members see each other's rows, not just the author.
apps/web — encryption skip for shared-space records:
encryptRecord now checks record.spaceId:
- `_personal:<userId>` sentinel OR no active shared space → encrypt
with user master key (E2E as today).
- Active space resolves to non-personal type AND spaceId matches
that space → skip encryption; write lands plaintext.
decryptRecord is unchanged because its per-field isEncrypted() guard
already passes plaintext through.
Phase-1 compromise: shared-space data is protected by server RLS
only, not E2E. Phase 2 adds per-Space shared keys with per-member
wrap — tracked in docs/plans/spaces-foundation.md.
Plus docs/plans/shared-space-smoketest.md: step-by-step Zwei-User-Test
mit erwarteten Ergebnissen und Debugging-Hinweisen bei Problemen.
Build + go test + web check all green.
Plan: docs/plans/spaces-foundation.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the second RLS policy needed for shared spaces. Users can read
rows in any space they're a member of, in addition to their own rows.
Changes:
- New policy sync_changes_space_member_read (SELECT only) uses
app.current_user_space_ids session config: rows with space_id in
that comma-separated list pass RLS.
- WITH CHECK is not extended — writes still require user_id match, so
only the author can write. Members read, owner/author writes.
- withUser() is now a thin wrapper around withUserAndMemberships(),
which accepts the caller's Space membership list and sets the new
session config alongside app.current_user_id.
- The comma-join is empty-filtered so stray blank entries can't match
rows with literal empty space_id (defense in depth).
Forward-compatible: today every space has exactly one member (the
author), so the membership list is always empty and the new policy
is a no-op — user_id isolation remains the only active guard.
When shared spaces start being used (clubs/teams/brand spaces with
invites), the HTTP handlers will fetch the caller's membership from
mana-auth and pass it to withUserAndMemberships. No migration needed
at that point — the policy is already live.
Subscription fan-out (WS/SSE broadcast to all space members) is still
per-user; that's a follow-up tied to the membership lookup infra.
Go build + existing tests pass.
Plan: docs/plans/spaces-foundation.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Server-side:
- sync_changes gains a nullable space_id TEXT column + partial index
on (user_id, space_id, app_id, created_at) WHERE space_id IS NOT NULL.
- RecordChange takes spaceID as a first-class parameter; *string so
empty strings land as real SQL NULL and the partial index skips them.
- ChangeRow + all three SELECTs (GetChangesSince, GetAllChangesSince,
StreamAllUserChanges) propagate space_id through to clients.
- changeFromRow surfaces SpaceID on the wire Change shape.
- New extractSpaceID helper reads the incoming payload — prefers top-
level spaceId, falls back to data.spaceId (inserts) or
fields.spaceId.value (updates). Tolerates pre-v28 clients.
- 6 Go tests cover the helper + round-trip.
Client-side:
- PendingChange gains an optional spaceId.
- Dexie creating hook stamps spaceId from the active record onto the
pending-change row (already set by the v28 scope hook).
- Dexie updating hook reads spaceId from the pre-update record and
stamps it on the pending-change so updates carry space context even
though spaceId itself is immutable and never in `fields`.
- buildChangeset forwards spaceId to the server.
Explicitly NOT in scope this pass:
- RLS remains user_id-scoped; multi-member shared-space reads need a
second policy that joins against auth.members. Follow-up once shared
spaces are actually used — today everything is personal.
- Subscription fan-out is still per-user; fan-out to all members of a
shared space is part of the same follow-up.
Go tests: 6/6 pass. Web type-check clean (0 errors across 7139 files).
Plan: docs/plans/spaces-foundation.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an opaque JSON `actor` column alongside the existing field_timestamps
so cross-device consumers can distinguish user / ai / system writes. The
server never parses the shape — it just stores and re-emits the blob the
webapp stamped in its Dexie hook.
- `sync/types.go` — Change.Actor as json.RawMessage with omitempty; nil
for pre-actor clients so wire remains backward-compatible
- `store/postgres.go`
- Migrate: CREATE TABLE includes `actor JSONB` for fresh DBs;
ALTER TABLE ADD COLUMN IF NOT EXISTS actor JSONB for existing ones
(idempotent, safe to re-run)
- RecordChange signature takes json.RawMessage; pgx writes nil as NULL
- All three SELECT paths (GetChangesSince, GetAllChangesSince,
StreamAllUserChanges) return actor, Scan into ChangeRow.Actor
- ChangeRow.Actor added with doc noting "missing = user" consumer rule
- `sync/handler.go` — Change.Actor threaded through HandleSync →
RecordChange, and populated on both changeFromRow (pull/POST replies)
and convertChanges (SSE stream)
- Tests: roundtrip of an AI-actor payload + omitempty verification for
pre-actor clients. All existing tests still pass.
Webapp types still need `actor?: Actor` on SyncChange + PendingChange to
match the wire, and applyServerChanges needs to stamp __lastActor /
__fieldActors from incoming changes for Workbench attribution on other
devices — both tracked as separate follow-ups.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Refactor: HTTP handler becomes a thin shim over a pure WriteBackup(w,
userID, createdAt, iter) function. RowIterator abstracts the store, so
tests feed synthetic ChangeRow slices and production feeds
StreamAllUserChanges. Zero behavior change in production — same bytes
on the wire.
Tests (all pass):
- TestWriteBackup_Roundtrip: three rows across two apps, assert zip has
2 entries, events.jsonl has 3 JSON lines in order, insert omits
fieldTimestamps, update surfaces them, manifest apps are sorted,
eventsSha256 equals a recomputed sha of the decompressed body.
- TestWriteBackup_EmptyUser: empty userID refused up-front.
- TestWriteBackup_NoRows: zero-row export still produces a valid zip
with an empty events.jsonl and a manifest with eventCount=0 and a
non-empty sha (sha of empty input).
- TestWriteBackup_DefaultsSchemaVersionZeroRowsToOne: legacy rows with
schema_version=0 clamp to 1 so the manifest never claims a protocol
version that never existed.
Paired with the vitest zip parser suite on the TS side, this closes
the Go-writes / JS-reads round-trip without needing live mana-sync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sync_changes gains schema_version column (default 1, idempotent ADD)
- Change/Changeset carry schemaVersion; server refuses > MaxSupported
- server->client changes now carry eventId + schemaVersion so the
restore path can dedup via eventId and route through a migration
chain keyed on schemaVersion
- backup JSONL gains schemaVersion per line
Pre-M2 clients (omit the field) are treated as v1 for compatibility.
This is the stability contract we commit to before launch: once v1
events are in the wild, all future builds must replay them forward.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Defense-in-depth on top of the existing application-level WHERE clauses:
- Migrate() now ENABLE + FORCE row level security on sync_changes and
installs a policy that gates rows on current_setting('app.current_user_id').
FORCE makes the policy apply to the table owner too, so the application
role used by mana-sync cannot bypass it regardless of grants.
- New withUser(ctx, userID, fn) helper opens a transaction and calls
set_config('app.current_user_id', userID, true) before running fn.
Empty userIDs are rejected up-front so an unauthenticated request can
never reach the database with an empty RLS scope (which would match
every row).
- RecordChange / GetChangesSince / GetAllChangesSince all run inside
withUser. WITH CHECK on the policy double-validates the user_id column
on insert against the active session, so a future code path that
forgets the WHERE clause cannot leak data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bug 1: NotifyUser() early-returned when no WebSocket clients existed,
skipping SSE subscriber notifications entirely. Fixed by restructuring
to check WS clients and SSE subscribers independently.
Bug 2: SSE stream cursor defaulted to client's `since` parameter when
no initial data existed. If `since` was in the future (or very recent),
live updates had created_at < cursor and were silently filtered out.
Fixed by defaulting cursor to now() when no initial data is returned.
Bug 3: NotifyUser used original sseSubs slice instead of sseSubsCopy
after releasing the read lock (race condition).
Verified E2E: Push from client A → SSE stream on client B receives
live change event with correct data within ~1 second.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New endpoint GET /sync/{appId}/stream sends Server-Sent Events with
change data directly, replacing the WebSocket notification + HTTP pull
round-trip pattern.
Server (Go):
- HandleStream() in handler.go: SSE endpoint with initial sync + live streaming
- Hub.Subscribe()/Unsubscribe() in hub.go: channel-based SSE subscriber system
- Notification type for type-safe SSE events
- convertChanges() helper extracted from duplicated code
- WriteTimeout set to 0 for SSE long-lived connections
Protocol: Client connects to /sync/{appId}/stream?collections=a,b&since=...
Server sends initial changes, then streams live changes as other clients sync.
Heartbeat every 30s keeps connection alive. Push still uses POST /sync/{appId}.
WebSocket remains available as fallback (not removed).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server now returns hasMore: true when there are more than 1000 changes
pending for a collection. Client continues pulling in a loop until
hasMore is false, using the last row's timestamp as cursor.
Prevents data loss after long offline periods where >1000 changes
accumulated for a single collection.
Server changes (Go):
- GetChangesSince() accepts limit parameter
- HandlePull() fetches limit+1, trims, sets hasMore
- SyncedUntil uses last row's timestamp when paginating
Client changes (TypeScript):
- Pull loop: while (hasMore) { fetch → apply → advance cursor }
- Cursor only persisted after all pages fetched
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add unified /ws endpoint that serves all app notifications over a single connection.
The server now includes appId in the sync-available message payload so the client
knows which app to pull. Legacy /ws/{appId} endpoint remains for backward compatibility.
Backend (Go):
- hub.go: Message struct gains AppId field, NotifyUser sends to all user clients
(unified clients receive everything, legacy clients filtered by appId)
- main.go: new GET /ws route (empty appId = unified mode)
Frontend (sync.ts):
- Single connectUnifiedWs() replaces 27 per-app connectWs() calls
- Parses msg.appId from server to pull only the affected app
- Reconnect/offline logic simplified to one WS
This reduces WebSocket connections from 27 per user to 1, cutting server
connection overhead by ~96%.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement the foundational local-first data layer for ManaCore apps:
- New @manacore/local-store package (Dexie.js IndexedDB, sync engine, Svelte 5 reactive queries)
- New mana-sync Go service (sync protocol, WebSocket push, field-level LWW conflict resolution)
- Todo app migrated as pilot: stores read/write IndexedDB, guest mode with onboarding seed data
- PillNavigation: prominent login pill for unauthenticated users
- SyncIndicator component showing local/syncing/offline status
- GuestWelcomeModal on first visit for Todo app
- Removed demo-mode auth_required checks from Todo components (all writes are now local)
- CSP fix for local development (localhost:3001, localhost:3050)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
