till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-22 17:26:41 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

feat(manacore/web): unified IndexedDB sync via Dexie hooks, eliminate cross-app readers

Browse source 
Activate sync for the unified manacore IndexedDB by adding automatic change tracking
via Dexie hooks on all 120+ tables. This replaces the unused manual trackChange() approach
and eliminates the need for 12 separate cross-app IndexedDB reader instances.

Key changes:
- database.ts: Dexie hooks auto-record _pendingChanges for every write, TABLE_TO_SYNC_NAME mapping
- sync.ts: rewritten with correct backend URLs, auth token, table name translation, server change guard
- layout: unified sync engine replaces per-app manacoreStore/tag/link sync + 12 cross-app readers
- cross-app-queries.ts: rewritten to query unified DB directly instead of via cross-app-stores
- legacy-migration.ts: one-time migration from old per-app DBs (manacore-todo etc.) to unified DB
- local-store.ts: refactored to use unified DB with collection wrappers instead of createLocalStore()
- Deleted cross-app-stores.ts (383 lines) and change-tracker.ts (80 lines)
- Updated ActivityFeed, TasksTodayWidget, CalendarEventsWidget, ContactsFavoritesWidget, spiral/collect.ts
- Updated CLAUDE.md with unified IndexedDB architecture documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS 2026-04-02 12:40:31 +02:00
parent 88864fd3a1
commit 05e5e957e8
14 changed files with 782 additions and 853 deletions
Download patch file Download diff file Expand all files Collapse all files

105
CLAUDE.md
View file

@ -570,51 +570,67 @@ $: doubled = count * 2;
All web apps use a **local-first** data layer: reads/writes go to IndexedDB (Dexie.js) first, sync to server in the background. This enables guest mode, offline CRUD, and instant UI. All web apps use a **local-first** data layer: reads/writes go to IndexedDB (Dexie.js) first, sync to server in the background. This enables guest mode, offline CRUD, and instant UI.
### Key Components ### Unified IndexedDB Architecture
| Component | Location | Purpose | The ManaCore unified app uses a **single IndexedDB** (`manacore`) containing all 120+ collections from all apps. Table names that collide across apps are prefixed (e.g., `todoProjects`, `cardDecks`, `presiDecks`).
|-----------|----------|---------|
| `@manacore/local-store` | `packages/local-store/` | Dexie.js collections, sync engine, Svelte 5 reactive queries |
| `mana-sync` | `services/mana-sync/` | Go sync server (WebSocket push, field-level LWW conflict resolution) |
| Todo Hono Server | `apps/todo/apps/server/` | Lightweight compute server (RRULE, reminders, admin) on Bun |
### Data Flow
``` ```
Guest: App → IndexedDB (Dexie.js) → UI (no sync) ┌─────────────────────────────────────────────┐
Logged in: App → IndexedDB → UI → SyncEngine → mana-sync (Go) → PostgreSQL │ Unified IndexedDB: "manacore" │
← WebSocket push ← │ │
│ tasks, todoProjects, labels, ... (todo) │
│ calendars, events (calendar) │
│ contacts (contacts) │
│ conversations, messages (chat) │
│ ... 120+ collections across 27 apps │
│ │
│ _pendingChanges (tagged with appId) │
│ _syncMeta (keyed by [appId+coll]) │
└──────────────────┬──────────────────────────┘
│ Dexie hooks auto-track
│ all writes as pending changes
▼
┌──────────────────────────────────────────────┐
│ Unified Sync Engine (sync.ts) │
│ One sync channel per appId │
│ POST /sync/{appId} (push) │
│ GET /sync/{appId}/pull (pull) │
│ WS /ws/{appId} (real-time notifications) │
└──────────────────┬───────────────────────────┘
▼
mana-sync (Go)
PostgreSQL (sync_changes)
``` ```
### Migrated Apps (21/23) #### Key Files
| App | Collections | Status | | File | Purpose |
|-----|------------|--------| |------|---------|
| Todo | tasks, projects, labels, taskLabels, reminders | Done | | `apps/manacore/apps/web/src/lib/data/database.ts` | Unified Dexie DB, SYNC_APP_MAP, table name mappings, Dexie hooks |
| Zitare | favorites, lists | Done | | `apps/manacore/apps/web/src/lib/data/sync.ts` | Unified sync engine (push/pull/WS per appId) |
| Calendar | calendars, events | Done | | `apps/manacore/apps/web/src/lib/data/legacy-migration.ts` | One-time migration from old per-app DBs |
| Clock | alarms, timers, worldClocks | Done | | `packages/local-store/` | Standalone local-store (used by individual apps, not the unified app) |
| Contacts | contacts | Done | | `services/mana-sync/` | Go sync server (WebSocket push, field-level LWW) |
| Cards | decks, cards | Done |
| Picture | images, boards, boardItems, tags, imageTags | Done |
| Presi | decks, slides | Done |
| Inventar | collections, items, locations, categories | Done |
| NutriPhi | meals, goals, favorites | Done |
| Planta | plants, plantPhotos, wateringSchedules, wateringLogs | Done |
| Storage | files, folders, tags, fileTags | Done |
| Chat | conversations, messages, templates | Done |
| Questions | collections, questions, answers | Done |
| Mukke | songs, playlists, playlistSongs, projects, markers | Done |
| Context | spaces, documents | Done |
| Photos | albums, albumItems, favorites, tags, photoTags | Done |
| SkilltTree | skills, activities, achievements | Done |
| CityCorners | locations, favorites | Done |
| Times | clients, projects, timeEntries, tags, templates, settings | Done |
| uLoad | links, tags, folders, linkTags | Done |
| Calc | calculations, savedFormulas | Done |
| ManaCore | userSettings, dashboardConfigs | Done |
**Not migrated (no CRUD data model):** Matrix (protocol client), Playground (stateless) #### How Sync Works
1. Module stores write directly to Dexie tables (`db.table('tasks').add(...)`)
2. Dexie hooks in `database.ts` automatically record each write to `_pendingChanges` with the correct `appId`
3. The unified sync engine groups pending changes by `appId` and pushes to `POST /sync/{appId}`
4. Table names are mapped between unified names (e.g., `todoProjects`) and backend names (e.g., `projects`) via `TABLE_TO_SYNC_NAME`
5. Server changes are pulled per collection and applied with a guard flag to prevent re-sync loops
#### Adding a New App Module
1. Add table definitions to `database.ts` schema (in `db.version(1).stores({...})`)
2. Add table-to-appId mapping in `SYNC_APP_MAP`
3. Add any renamed tables to `TABLE_TO_SYNC_NAME`
4. Create module in `src/lib/modules/{app}/` with collections, queries, stores
5. Dexie hooks automatically handle change tracking — no manual `trackChange()` needed
### Standalone Apps (Legacy)
Individual apps in `apps/*/apps/web/` still use `@manacore/local-store` with per-app IndexedDB databases (`manacore-{appId}`). When users first open the unified ManaCore app, `legacy-migration.ts` migrates data from these old DBs into the unified DB.
### Dev Commands (Local-First Stack) ### Dev Commands (Local-First Stack)
@ -626,19 +642,6 @@ pnpm dev:todo:local # Web + sync + server (no auth needed)
pnpm dev:todo:full # Everything incl. auth + DB setup pnpm dev:todo:full # Everything incl. auth + DB setup
``` ```
### Adding Local-First to a New App
1. Create `apps/{app}/apps/web/src/lib/data/local-store.ts` — define collections with `createLocalStore()`
2. Create `apps/{app}/apps/web/src/lib/data/guest-seed.ts` — onboarding data
3. Rewrite stores to use `collection.getAll()` / `collection.insert()` instead of API calls
4. In layout: `await store.initialize()`, `store.startSync()` on login, `allowGuest={true}` on AuthGate
5. Set `userEmail = ''` for guests so PillNav shows login button
6. Add `GuestWelcomeModal` for first-visit experience
### Architecture Plan
Full migration plan: `.claude/plans/local-first-architecture-migration.md`
## Shared Packages (`packages/`) ## Shared Packages (`packages/`)
| Package | Purpose | | Package | Purpose |

12
apps/manacore/apps/web/src/lib/components/ActivityFeed.svelte
View file

@ -1,9 +1,5 @@
<script lang="ts"> <script lang="ts">
import { import { db } from '$lib/data/database';
crossTaskCollection,
crossEventCollection,
crossContactCollection,
} from '$lib/data/cross-app-stores';
interface Props { interface Props {
maxItems?: number; maxItems?: number;
@ -38,7 +34,7 @@
today.setHours(0, 0, 0, 0); today.setHours(0, 0, 0, 0);
const todayStr = today.toISOString(); const todayStr = today.toISOString();
const tasks = await crossTaskCollection.getAll(); const tasks = await db.table('tasks').toArray();
recentTasks = tasks recentTasks = tasks
.filter((t) => t.isCompleted && t.completedAt && t.completedAt >= todayStr) .filter((t) => t.isCompleted && t.completedAt && t.completedAt >= todayStr)
.sort((a, b) => (b.completedAt || '').localeCompare(a.completedAt || '')) .sort((a, b) => (b.completedAt || '').localeCompare(a.completedAt || ''))
@ -61,7 +57,7 @@
const now = new Date().toISOString(); const now = new Date().toISOString();
const tomorrow = new Date(Date.now() + 86400000).toISOString(); const tomorrow = new Date(Date.now() + 86400000).toISOString();
const events = await crossEventCollection.getAll(); const events = await db.table('events').toArray();
upcomingEvents = events upcomingEvents = events
.filter((e) => e.startDate >= now && e.startDate <= tomorrow) .filter((e) => e.startDate >= now && e.startDate <= tomorrow)
.sort((a, b) => a.startDate.localeCompare(b.startDate)) .sort((a, b) => a.startDate.localeCompare(b.startDate))
@ -81,7 +77,7 @@
try { try {
// Recently added contacts // Recently added contacts
const contacts = await crossContactCollection.getAll(); const contacts = await db.table('contacts').toArray();
recentContacts = contacts recentContacts = contacts
.filter((c) => !c.isArchived) .filter((c) => !c.isArchived)
.sort((a, b) => (b.createdAt || '').localeCompare(a.createdAt || '')) .sort((a, b) => (b.createdAt || '').localeCompare(a.createdAt || ''))

3
apps/manacore/apps/web/src/lib/components/dashboard/widgets/CalendarEventsWidget.svelte
View file

@ -8,7 +8,6 @@
import { _ } from 'svelte-i18n'; import { _ } from 'svelte-i18n';
import { useUpcomingEvents } from '$lib/data/cross-app-queries'; import { useUpcomingEvents } from '$lib/data/cross-app-queries';
import type { CrossAppEvent } from '$lib/data/cross-app-stores';
import { APP_URLS } from '@manacore/shared-branding'; import { APP_URLS } from '@manacore/shared-branding';
const isDev = typeof window !== 'undefined' && window.location.hostname === 'localhost'; const isDev = typeof window !== 'undefined' && window.location.hostname === 'localhost';
@ -18,7 +17,7 @@
const MAX_DISPLAY = 5; const MAX_DISPLAY = 5;
function formatEventTime(event: CrossAppEvent): string { function formatEventTime(event: any): string {
const start = new Date(event.startDate); const start = new Date(event.startDate);
const today = new Date(); const today = new Date();
const tomorrow = new Date(today); const tomorrow = new Date(today);

5
apps/manacore/apps/web/src/lib/components/dashboard/widgets/ContactsFavoritesWidget.svelte
View file

@ -8,7 +8,6 @@
import { _ } from 'svelte-i18n'; import { _ } from 'svelte-i18n';
import { useFavoriteContacts } from '$lib/data/cross-app-queries'; import { useFavoriteContacts } from '$lib/data/cross-app-queries';
import type { CrossAppContact } from '$lib/data/cross-app-stores';
import { APP_URLS } from '@manacore/shared-branding'; import { APP_URLS } from '@manacore/shared-branding';
const MAX_DISPLAY = 5; const MAX_DISPLAY = 5;
@ -17,12 +16,12 @@
const isDev = typeof window !== 'undefined' && window.location.hostname === 'localhost'; const isDev = typeof window !== 'undefined' && window.location.hostname === 'localhost';
const contactsUrl = isDev ? APP_URLS.contacts.dev : APP_URLS.contacts.prod; const contactsUrl = isDev ? APP_URLS.contacts.dev : APP_URLS.contacts.prod;
function getDisplayName(contact: CrossAppContact): string { function getDisplayName(contact: any): string {
const parts = [contact.firstName, contact.lastName].filter(Boolean); const parts = [contact.firstName, contact.lastName].filter(Boolean);
return parts.length > 0 ? parts.join(' ') : contact.email || 'Unbekannt'; return parts.length > 0 ? parts.join(' ') : contact.email || 'Unbekannt';
} }
function getInitials(contact: CrossAppContact): string { function getInitials(contact: any): string {
const name = getDisplayName(contact); const name = getDisplayName(contact);
const parts = name.split(' '); const parts = name.split(' ');
if (parts.length >= 2) { if (parts.length >= 2) {

15
apps/manacore/apps/web/src/lib/components/dashboard/widgets/TasksTodayWidget.svelte
View file

@ -8,7 +8,7 @@
import { _ } from 'svelte-i18n'; import { _ } from 'svelte-i18n';
import { useOpenTasks } from '$lib/data/cross-app-queries'; import { useOpenTasks } from '$lib/data/cross-app-queries';
import { crossTaskCollection, type CrossAppTask } from '$lib/data/cross-app-stores'; import { db } from '$lib/data/database';
import { APP_URLS } from '@manacore/shared-branding'; import { APP_URLS } from '@manacore/shared-branding';
import { format, isToday, isTomorrow, isPast } from 'date-fns'; import { format, isToday, isTomorrow, isPast } from 'date-fns';
import { de } from 'date-fns/locale'; import { de } from 'date-fns/locale';
@ -48,7 +48,7 @@
// Track tasks being toggled (for optimistic UI) // Track tasks being toggled (for optimistic UI)
let togglingIds: Set<string> = $state(new Set()); let togglingIds: Set<string> = $state(new Set());
async function handleToggleComplete(e: MouseEvent, task: CrossAppTask) { async function handleToggleComplete(e: MouseEvent, task: any) {
e.preventDefault(); e.preventDefault();
e.stopPropagation(); e.stopPropagation();
@ -56,18 +56,19 @@
togglingIds = new Set([...togglingIds, task.id]); togglingIds = new Set([...togglingIds, task.id]);
// Write directly to IndexedDB — sync engine will push to server // Write directly to unified IndexedDB — Dexie hooks track the change for sync
await crossTaskCollection.update(task.id, { await db.table('tasks').update(task.id, {
isCompleted: !task.isCompleted, isCompleted: !task.isCompleted,
completedAt: task.isCompleted ? null : new Date().toISOString(), completedAt: task.isCompleted ? null : new Date().toISOString(),
} as Partial<CrossAppTask>); updatedAt: new Date().toISOString(),
});
togglingIds = new Set([...togglingIds].filter((id) => id !== task.id)); togglingIds = new Set([...togglingIds].filter((id) => id !== task.id));
} }
function getSubtaskProgress(task: CrossAppTask): string | null { function getSubtaskProgress(task: any): string | null {
if (!task.subtasks || task.subtasks.length === 0) return null; if (!task.subtasks || task.subtasks.length === 0) return null;
const done = task.subtasks.filter((s) => s.isCompleted).length; const done = task.subtasks.filter((s: any) => s.isCompleted).length;
return `${done}/${task.subtasks.length}`; return `${done}/${task.subtasks.length}`;
} }
</script> </script>

80
apps/manacore/apps/web/src/lib/data/change-tracker.ts
View file

@ -1,80 +0,0 @@
/**
* Change Tracker — records local writes to _pendingChanges with appId routing.
*
* Usage in mutation stores:
* import { trackChange } from '$lib/data/change-tracker';
* await taskTable.put(task);
* await trackChange('tasks', task.id, 'insert', task);
*/
import { db, TABLE_TO_APP } from './database';
interface PendingChange {
appId: string;
collection: string;
recordId: string;
op: 'insert' | 'update' | 'delete';
fields?: Record<string, { value: unknown; updatedAt: string }>;
data?: Record<string, unknown>;
deletedAt?: string;
createdAt: string;
}
/**
* Record a local change to _pendingChanges for later sync.
*/
export async function trackChange(
collection: string,
recordId: string,
op: 'insert' | 'update' | 'delete',
data?: Record<string, unknown>,
fields?: Record<string, { value: unknown; updatedAt: string }>
): Promise<void> {
const appId = TABLE_TO_APP[collection];
if (!appId) {
console.warn(`[ChangeTracker] No appId mapping for collection "${collection}"`);
return;
}
const now = new Date().toISOString();
const change: PendingChange = {
appId,
collection,
recordId,
op,
createdAt: now,
};
if (fields) change.fields = fields;
if (data) change.data = data;
if (op === 'delete') change.deletedAt = now;
await db.table('_pendingChanges').add(change);
}
/**
* Record a field-level update change (LWW).
* Only the changed fields are tracked, not the entire record.
*/
export async function trackFieldUpdate(
collection: string,
recordId: string,
updatedFields: Record<string, unknown>
): Promise<void> {
const now = new Date().toISOString();
const fields: Record<string, { value: unknown; updatedAt: string }> = {};
for (const [key, value] of Object.entries(updatedFields)) {
fields[key] = { value, updatedAt: now };
}
await trackChange(collection, recordId, 'update', undefined, fields);
}
/**
* Record a soft-delete change.
*/
export async function trackDelete(collection: string, recordId: string): Promise<void> {
await trackChange(collection, recordId, 'delete');
}

222
apps/manacore/apps/web/src/lib/data/cross-app-queries.ts
View file

@ -1,58 +1,21 @@
/** /**
* Cross-App Reactive Queries * Cross-App Reactive Queries
* *
* Live queries that read directly from other apps' IndexedDB databases. * Live queries on the unified IndexedDB. Auto-update when data changes
* Auto-update when data changes (local writes, sync, other tabs). * (local writes, sync, other tabs) via Dexie's liveQuery.
* Replaces REST API polling with instant reactive reads.
*/ */
import { useLiveQueryWithDefault } from '@manacore/local-store/svelte'; import { useLiveQueryWithDefault } from '@manacore/local-store/svelte';
import { import { db } from './database';
crossTaskCollection,
crossEventCollection,
crossContactCollection,
crossConversationCollection,
crossFavoriteCollection,
crossImageCollection,
crossAlarmCollection,
crossTimerCollection,
crossFileCollection,
crossSongCollection,
crossPlaylistCollection,
crossPresiDeckCollection,
crossSpaceCollection,
crossDocumentCollection,
crossCardsDeckCollection,
crossCardsCardCollection,
type CrossAppTask,
type CrossAppEvent,
type CrossAppContact,
type CrossAppConversation,
type CrossAppFavorite,
type CrossAppImage,
type CrossAppAlarm,
type CrossAppTimer,
type CrossAppFile,
type CrossAppSong,
type CrossAppPlaylist,
type CrossAppDeck,
type CrossAppSpace,
type CrossAppDocument,
type CrossAppCardsDeck,
type CrossAppCardsCard,
} from './cross-app-stores';
// ─── Todo Queries ─────────────────────────────────────────── // ─── Todo Queries ───────────────────────────────────────────
/** All open (incomplete) tasks, sorted by order. */ /** All open (incomplete) tasks, sorted by order. */
export function useOpenTasks() { export function useOpenTasks() {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossTaskCollection.getAll(undefined, { const all = await db.table('tasks').orderBy('order').toArray();
sortBy: 'order', return all.filter((t: any) => !t.isCompleted && !t.deletedAt);
sortDirection: 'asc', }, [] as any[]);
});
return all.filter((t) => !t.isCompleted && !t.deletedAt);
}, [] as CrossAppTask[]);
} }
/** Tasks due today or overdue. */ /** Tasks due today or overdue. */
@ -62,18 +25,13 @@ export function useTodayTasks() {
today.setHours(0, 0, 0, 0); today.setHours(0, 0, 0, 0);
const todayStr = today.toISOString().slice(0, 10); const todayStr = today.toISOString().slice(0, 10);
const all = await crossTaskCollection.getAll(undefined, { const all = await db.table('tasks').orderBy('order').toArray();
sortBy: 'order', return all.filter((t: any) => {
sortDirection: 'asc',
});
return all.filter((t) => {
if (t.isCompleted || t.deletedAt) return false; if (t.isCompleted || t.deletedAt) return false;
if (!t.dueDate) return false; if (!t.dueDate) return false;
const due = t.dueDate.slice(0, 10); return t.dueDate.slice(0, 10) <= todayStr;
return due <= todayStr;
}); });
}, [] as CrossAppTask[]); }, [] as any[]);
} }
/** Tasks upcoming in the next N days. */ /** Tasks upcoming in the next N days. */
@ -87,18 +45,14 @@ export function useUpcomingTasks(days = 7) {
future.setDate(future.getDate() + days); future.setDate(future.getDate() + days);
const futureStr = future.toISOString().slice(0, 10); const futureStr = future.toISOString().slice(0, 10);
const all = await crossTaskCollection.getAll(undefined, { const all = await db.table('tasks').orderBy('dueDate').toArray();
sortBy: 'dueDate', return all.filter((t: any) => {
sortDirection: 'asc',
});
return all.filter((t) => {
if (t.isCompleted || t.deletedAt) return false; if (t.isCompleted || t.deletedAt) return false;
if (!t.dueDate) return false; if (!t.dueDate) return false;
const due = t.dueDate.slice(0, 10); const due = t.dueDate.slice(0, 10);
return due > todayStr && due <= futureStr; return due > todayStr && due <= futureStr;
}); });
}, [] as CrossAppTask[]); }, [] as any[]);
} }
// ─── Calendar Queries ─────────────────────────────────────── // ─── Calendar Queries ───────────────────────────────────────
@ -113,16 +67,12 @@ export function useUpcomingEvents(days = 7) {
const nowStr = now.toISOString(); const nowStr = now.toISOString();
const futureStr = future.toISOString(); const futureStr = future.toISOString();
const all = await crossEventCollection.getAll(undefined, { const all = await db.table('events').orderBy('startDate').toArray();
sortBy: 'startDate', return all.filter((e: any) => {
sortDirection: 'asc',
});
return all.filter((e) => {
if (e.deletedAt) return false; if (e.deletedAt) return false;
return e.startDate >= nowStr && e.startDate <= futureStr; return e.startDate >= nowStr && e.startDate <= futureStr;
}); });
}, [] as CrossAppEvent[]); }, [] as any[]);
} }
// ─── Contacts Queries ─────────────────────────────────────── // ─── Contacts Queries ───────────────────────────────────────
@ -130,13 +80,9 @@ export function useUpcomingEvents(days = 7) {
/** Favorite contacts. */ /** Favorite contacts. */
export function useFavoriteContacts(limit = 5) { export function useFavoriteContacts(limit = 5) {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossContactCollection.getAll(undefined, { const all = await db.table('contacts').orderBy('firstName').toArray();
sortBy: 'firstName', return all.filter((c: any) => c.isFavorite && !c.isArchived && !c.deletedAt).slice(0, limit);
sortDirection: 'asc', }, [] as any[]);
});
return all.filter((c) => c.isFavorite && !c.isArchived && !c.deletedAt).slice(0, limit);
}, [] as CrossAppContact[]);
} }
// ─── Chat Queries ─────────────────────────────────────────── // ─── Chat Queries ───────────────────────────────────────────
@ -144,27 +90,24 @@ export function useFavoriteContacts(limit = 5) {
/** Recent conversations, sorted by updatedAt desc. */ /** Recent conversations, sorted by updatedAt desc. */
export function useRecentConversations(limit = 5) { export function useRecentConversations(limit = 5) {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossConversationCollection.getAll(undefined, { const all = await db.table('conversations').toArray();
sortBy: 'updatedAt', return all
sortDirection: 'desc', .filter((c: any) => !c.isArchived && !c.deletedAt)
}); .sort((a: any, b: any) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? ''))
return all.filter((c) => !c.isArchived && !c.deletedAt).slice(0, limit); .slice(0, limit);
}, [] as CrossAppConversation[]); }, [] as any[]);
} }
// ─── Zitare Queries ───────────────────────────────────────── // ─── Zitare Queries ─────────────────────────────────────────
/** A random favorite quote. */ /** A random favorite quote. */
export function useRandomFavorite() { export function useRandomFavorite() {
return useLiveQueryWithDefault( return useLiveQueryWithDefault(async () => {
async () => { const all = await db.table('zitareFavorites').toArray();
const all = await crossFavoriteCollection.getAll(); const active = all.filter((f: any) => !f.deletedAt);
const active = all.filter((f) => !f.deletedAt); if (active.length === 0) return null;
if (active.length === 0) return null; return active[Math.floor(Math.random() * active.length)];
return active[Math.floor(Math.random() * active.length)]; }, null as any);
},
null as CrossAppFavorite | null
);
} }
// ─── Picture Queries ──────────────────────────────────────── // ─── Picture Queries ────────────────────────────────────────
@ -172,12 +115,12 @@ export function useRandomFavorite() {
/** Recent generated images. */ /** Recent generated images. */
export function useRecentImages(limit = 6) { export function useRecentImages(limit = 6) {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossImageCollection.getAll(undefined, { const all = await db.table('images').toArray();
sortBy: 'createdAt', return all
sortDirection: 'desc', .filter((i: any) => !i.archivedAt && !i.deletedAt)
}); .sort((a: any, b: any) => (b.createdAt ?? '').localeCompare(a.createdAt ?? ''))
return all.filter((i) => !i.archivedAt && !i.deletedAt).slice(0, limit); .slice(0, limit);
}, [] as CrossAppImage[]); }, [] as any[]);
} }
// ─── Clock Queries ────────────────────────────────────────── // ─── Clock Queries ──────────────────────────────────────────
@ -185,17 +128,19 @@ export function useRecentImages(limit = 6) {
/** Enabled alarms. */ /** Enabled alarms. */
export function useEnabledAlarms() { export function useEnabledAlarms() {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossAlarmCollection.getAll(); const all = await db.table('alarms').toArray();
return all.filter((a) => a.enabled && !a.deletedAt); return all.filter((a: any) => a.enabled && !a.deletedAt);
}, [] as CrossAppAlarm[]); }, [] as any[]);
} }
/** Active/running timers. */ /** Active/running timers. */
export function useActiveTimers() { export function useActiveTimers() {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossTimerCollection.getAll(); const all = await db.table('timers').toArray();
return all.filter((t) => (t.status === 'running' || t.status === 'paused') && !t.deletedAt); return all.filter(
}, [] as CrossAppTimer[]); (t: any) => (t.status === 'running' || t.status === 'paused') && !t.deletedAt
);
}, [] as any[]);
} }
// ─── Storage Queries ──────────────────────────────────────── // ─── Storage Queries ────────────────────────────────────────
@ -204,15 +149,15 @@ export function useActiveTimers() {
export function useStorageStats() { export function useStorageStats() {
return useLiveQueryWithDefault( return useLiveQueryWithDefault(
async () => { async () => {
const files = await crossFileCollection.getAll(); const files = await db.table('files').toArray();
const active = files.filter((f) => !f.isDeleted && !f.deletedAt); const active = files.filter((f: any) => !f.isDeleted && !f.deletedAt);
const totalSize = active.reduce((sum, f) => sum + (f.size || 0), 0); const totalSize = active.reduce((sum: number, f: any) => sum + (f.size || 0), 0);
const recent = active const recent = active
.sort((a, b) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? '')) .sort((a: any, b: any) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? ''))
.slice(0, 5); .slice(0, 5);
return { totalFiles: active.length, totalSize, recentFiles: recent }; return { totalFiles: active.length, totalSize, recentFiles: recent };
}, },
{ totalFiles: 0, totalSize: 0, recentFiles: [] as CrossAppFile[] } { totalFiles: 0, totalSize: 0, recentFiles: [] as any[] }
); );
} }
@ -222,21 +167,21 @@ export function useStorageStats() {
export function useMukkeStats() { export function useMukkeStats() {
return useLiveQueryWithDefault( return useLiveQueryWithDefault(
async () => { async () => {
const songs = await crossSongCollection.getAll(); const songs = await db.table('songs').toArray();
const playlists = await crossPlaylistCollection.getAll(); const playlists = await db.table('mukkePlaylists').toArray();
const activeSongs = songs.filter((s) => !s.deletedAt); const activeSongs = songs.filter((s: any) => !s.deletedAt);
const activePlaylists = playlists.filter((p) => !p.deletedAt); const activePlaylists = playlists.filter((p: any) => !p.deletedAt);
const recent = activeSongs const recent = activeSongs
.sort((a, b) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? '')) .sort((a: any, b: any) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? ''))
.slice(0, 5); .slice(0, 5);
return { return {
totalSongs: activeSongs.length, totalSongs: activeSongs.length,
totalPlaylists: activePlaylists.length, totalPlaylists: activePlaylists.length,
favoriteCount: activeSongs.filter((s) => s.favorite).length, favoriteCount: activeSongs.filter((s: any) => s.favorite).length,
recentSongs: recent, recentSongs: recent,
}; };
}, },
{ totalSongs: 0, totalPlaylists: 0, favoriteCount: 0, recentSongs: [] as CrossAppSong[] } { totalSongs: 0, totalPlaylists: 0, favoriteCount: 0, recentSongs: [] as any[] }
); );
} }
@ -245,12 +190,12 @@ export function useMukkeStats() {
/** Recent presentation decks. */ /** Recent presentation decks. */
export function useRecentDecks(limit = 5) { export function useRecentDecks(limit = 5) {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossPresiDeckCollection.getAll(undefined, { const all = await db.table('presiDecks').toArray();
sortBy: 'updatedAt', return all
sortDirection: 'desc', .filter((d: any) => !d.deletedAt)
}); .sort((a: any, b: any) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? ''))
return all.filter((d) => !d.deletedAt).slice(0, limit); .slice(0, limit);
}, [] as CrossAppDeck[]); }, [] as any[]);
} }
// ─── Context Queries ──────────────────────────────────────── // ─── Context Queries ────────────────────────────────────────
@ -258,22 +203,25 @@ export function useRecentDecks(limit = 5) {
/** Recent documents + spaces. */ /** Recent documents + spaces. */
export function useRecentDocuments(limit = 5) { export function useRecentDocuments(limit = 5) {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossDocumentCollection.getAll(undefined, { const all = await db.table('documents').toArray();
sortBy: 'updatedAt', return all
sortDirection: 'desc', .filter((d: any) => !d.deletedAt)
}); .sort((a: any, b: any) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? ''))
return all.filter((d) => !d.deletedAt).slice(0, limit); .slice(0, limit);
}, [] as CrossAppDocument[]); }, [] as any[]);
} }
export function useSpaces() { export function useSpaces() {
return useLiveQueryWithDefault(async () => { return useLiveQueryWithDefault(async () => {
const all = await crossSpaceCollection.getAll(undefined, { const all = await db.table('contextSpaces').toArray();
sortBy: 'pinned', return all
sortDirection: 'desc', .filter((s: any) => !s.deletedAt)
}); .sort((a: any, b: any) => {
return all.filter((s) => !s.deletedAt); if (a.pinned && !b.pinned) return -1;
}, [] as CrossAppSpace[]); if (!a.pinned && b.pinned) return 1;
return 0;
});
}, [] as any[]);
} }
// ─── Cards Queries ───────────────────────────────────────── // ─── Cards Queries ─────────────────────────────────────────
@ -282,16 +230,16 @@ export function useSpaces() {
export function useCardsProgress() { export function useCardsProgress() {
return useLiveQueryWithDefault( return useLiveQueryWithDefault(
async () => { async () => {
const decks = await crossCardsDeckCollection.getAll(); const decks = await db.table('cardDecks').toArray();
const cards = await crossCardsCardCollection.getAll(); const cards = await db.table('cards').toArray();
const activeDecks = decks.filter((d) => !d.deletedAt); const activeDecks = decks.filter((d: any) => !d.deletedAt);
const activeCards = cards.filter((c) => !c.deletedAt); const activeCards = cards.filter((c: any) => !c.deletedAt);
const now = new Date().toISOString(); const now = new Date().toISOString();
const dueCards = activeCards.filter((c) => c.nextReview && c.nextReview <= now); const dueCards = activeCards.filter((c: any) => c.nextReview && c.nextReview <= now);
return { return {
totalDecks: activeDecks.length, totalDecks: activeDecks.length,
totalCards: activeCards.length, totalCards: activeCards.length,
cardsLearned: activeCards.filter((c) => (c.reviewCount ?? 0) > 0).length, cardsLearned: activeCards.filter((c: any) => (c.reviewCount ?? 0) > 0).length,
dueForReview: dueCards.length, dueForReview: dueCards.length,
decks: activeDecks, decks: activeDecks,
}; };
@ -301,7 +249,7 @@ export function useCardsProgress() {
totalCards: 0, totalCards: 0,
cardsLearned: 0, cardsLearned: 0,
dueForReview: 0, dueForReview: 0,
decks: [] as CrossAppCardsDeck[], decks: [] as any[],
} }
); );
} }

382
apps/manacore/apps/web/src/lib/data/cross-app-stores.ts
View file

@ -1,382 +0,0 @@
/**
* Cross-App IndexedDB Readers
*
* Opens other apps' IndexedDB databases for direct read access.
* All apps on the same origin share IndexedDB, so ManaCore can
* read from manacore-todo, manacore-calendar, etc. directly.
*
* Data is reactive via Dexie's liveQuery — updates when any app
* writes to the same database (including via sync).
*
* NOTE: These stores are read-only from ManaCore's perspective.
* Writes that need sync should go through the owning app's collections.
*/
import { createLocalStore, type BaseRecord } from '@manacore/local-store';
// ─── Todo Types ─────────────────────────────────────────────
export interface CrossAppTask extends BaseRecord {
title: string;
description?: string;
projectId?: string | null;
priority: 'low' | 'medium' | 'high' | 'urgent';
isCompleted: boolean;
completedAt?: string | null;
dueDate?: string | null;
dueTime?: string | null;
scheduledDate?: string | null;
estimatedDuration?: number | null;
order: number;
subtasks?: { id: string; title: string; isCompleted: boolean; order: number }[] | null;
labels?: { id: string; name: string; color: string }[];
}
export interface CrossAppProject extends BaseRecord {
name: string;
color: string;
icon?: string | null;
order: number;
isArchived: boolean;
isDefault: boolean;
}
// ─── Calendar Types ─────────────────────────────────────────
export interface CrossAppEvent extends BaseRecord {
calendarId: string;
title: string;
description?: string | null;
startDate: string;
endDate: string;
allDay: boolean;
location?: string | null;
recurrenceRule?: string | null;
color?: string | null;
}
export interface CrossAppCalendar extends BaseRecord {
name: string;
color: string;
isDefault: boolean;
isVisible: boolean;
}
// ─── Contacts Types ─────────────────────────────────────────
export interface CrossAppContact extends BaseRecord {
firstName?: string;
lastName?: string;
email?: string;
phone?: string;
company?: string;
jobTitle?: string;
photoUrl?: string;
isFavorite?: boolean;
isArchived?: boolean;
}
// ─── Chat Types ─────────────────────────────────────────────
export interface CrossAppConversation extends BaseRecord {
title?: string;
modelId?: string;
isArchived?: boolean;
isPinned?: boolean;
spaceId?: string;
}
export interface CrossAppMessage extends BaseRecord {
conversationId: string;
sender: 'user' | 'assistant' | 'system';
messageText: string;
}
// ─── Zitare Types ───────────────────────────────────────────
export interface CrossAppFavorite extends BaseRecord {
quoteId: string;
}
// ─── Picture Types ──────────────────────────────────────────
export interface CrossAppImage extends BaseRecord {
prompt?: string;
publicUrl?: string;
storagePath?: string;
filename?: string;
width?: number;
height?: number;
isFavorite?: boolean;
isPublic?: boolean;
archivedAt?: string | null;
}
// ─── Clock Types ────────────────────────────────────────────
export interface CrossAppAlarm extends BaseRecord {
label?: string;
time: string;
enabled: boolean;
repeatDays?: number[];
}
export interface CrossAppTimer extends BaseRecord {
label?: string;
durationSeconds: number;
remainingSeconds: number;
status: 'idle' | 'running' | 'paused' | 'finished';
startedAt?: string;
}
// ─── Storage Types ──────────────────────────────────────────
export interface CrossAppFile extends BaseRecord {
name: string;
originalName?: string;
mimeType?: string;
size?: number;
parentFolderId?: string | null;
isFavorite?: boolean;
isDeleted?: boolean;
}
export interface CrossAppFolder extends BaseRecord {
name: string;
parentFolderId?: string | null;
path?: string;
depth?: number;
isFavorite?: boolean;
isDeleted?: boolean;
}
// ─── Mukke Types ────────────────────────────────────────────
export interface CrossAppSong extends BaseRecord {
title: string;
artist?: string;
album?: string;
duration?: number;
favorite?: boolean;
}
export interface CrossAppPlaylist extends BaseRecord {
name: string;
description?: string;
}
// ─── Presi Types ────────────────────────────────────────────
export interface CrossAppDeck extends BaseRecord {
title: string;
description?: string;
isPublic?: boolean;
}
export interface CrossAppSlide extends BaseRecord {
deckId: string;
order: number;
content?: unknown;
}
// ─── Context Types ──────────────────────────────────────────
export interface CrossAppSpace extends BaseRecord {
name: string;
description?: string;
pinned?: boolean;
}
export interface CrossAppDocument extends BaseRecord {
spaceId: string;
title: string;
type?: 'text' | 'context' | 'prompt';
pinned?: boolean;
}
// ─── Cards Types ───────────────────────────────────────────
export interface CrossAppCardsDeck extends BaseRecord {
name: string;
description?: string;
color?: string;
cardCount?: number;
lastStudied?: string;
isPublic?: boolean;
}
export interface CrossAppCardsCard extends BaseRecord {
deckId: string;
front: string;
back: string;
difficulty?: number;
nextReview?: string;
reviewCount?: number;
}
// ─── Store Instances ────────────────────────────────────────
// These open existing IndexedDB databases created by other apps.
// No sync config — ManaCore only reads, the owning app handles sync.
export const todoReader = createLocalStore({
appId: 'todo',
collections: [
{
name: 'tasks',
indexes: [
'projectId',
'dueDate',
'isCompleted',
'priority',
'order',
'[isCompleted+order]',
'[projectId+order]',
],
},
{
name: 'projects',
indexes: ['order', 'isArchived'],
},
],
});
export const calendarReader = createLocalStore({
appId: 'calendar',
collections: [
{
name: 'events',
indexes: ['calendarId', 'startDate', 'endDate', 'allDay', '[calendarId+startDate]'],
},
{
name: 'calendars',
indexes: ['isDefault', 'isVisible'],
},
],
});
export const contactsReader = createLocalStore({
appId: 'contacts',
collections: [
{
name: 'contacts',
indexes: ['firstName', 'lastName', 'email', 'company', 'isFavorite', 'isArchived'],
},
],
});
export const chatReader = createLocalStore({
appId: 'chat',
collections: [
{ name: 'conversations', indexes: ['isArchived', 'isPinned', 'spaceId'] },
{ name: 'messages', indexes: ['conversationId', 'sender', '[conversationId+sender]'] },
],
});
export const zitareReader = createLocalStore({
appId: 'zitare',
collections: [{ name: 'favorites', indexes: ['quoteId'] }],
});
export const pictureReader = createLocalStore({
appId: 'picture',
collections: [{ name: 'images', indexes: ['isFavorite', 'isPublic', 'archivedAt', 'prompt'] }],
});
export const clockReader = createLocalStore({
appId: 'clock',
collections: [
{ name: 'alarms', indexes: ['enabled', 'time'] },
{ name: 'timers', indexes: ['status'] },
],
});
export const storageReader = createLocalStore({
appId: 'storage',
collections: [
{
name: 'files',
indexes: ['parentFolderId', 'mimeType', 'isFavorite', 'isDeleted', 'name'],
},
{ name: 'folders', indexes: ['parentFolderId', 'path', 'depth', 'isFavorite', 'isDeleted'] },
],
});
export const mukkeReader = createLocalStore({
appId: 'mukke',
collections: [
{ name: 'songs', indexes: ['artist', 'album', 'genre', 'favorite', 'title'] },
{ name: 'playlists', indexes: ['name'] },
],
});
export const presiReader = createLocalStore({
appId: 'presi',
collections: [
{ name: 'decks', indexes: ['isPublic'] },
{ name: 'slides', indexes: ['deckId', 'order', '[deckId+order]'] },
],
});
export const contextReader = createLocalStore({
appId: 'context',
collections: [
{ name: 'spaces', indexes: ['pinned', 'prefix'] },
{ name: 'documents', indexes: ['spaceId', 'type', 'pinned', 'title', '[spaceId+type]'] },
],
});
export const cardsReader = createLocalStore({
appId: 'cards',
collections: [
{ name: 'decks', indexes: ['isPublic'] },
{ name: 'cards', indexes: ['deckId', 'difficulty', 'nextReview', 'order', '[deckId+order]'] },
],
});
// ─── Typed Collection Accessors ─────────────────────────────
// Todo
export const crossTaskCollection = todoReader.collection<CrossAppTask>('tasks');
export const crossProjectCollection = todoReader.collection<CrossAppProject>('projects');
// Calendar
export const crossEventCollection = calendarReader.collection<CrossAppEvent>('events');
export const crossCalendarCollection = calendarReader.collection<CrossAppCalendar>('calendars');
// Contacts
export const crossContactCollection = contactsReader.collection<CrossAppContact>('contacts');
// Chat
export const crossConversationCollection =
chatReader.collection<CrossAppConversation>('conversations');
export const crossMessageCollection = chatReader.collection<CrossAppMessage>('messages');
// Zitare
export const crossFavoriteCollection = zitareReader.collection<CrossAppFavorite>('favorites');
// Picture
export const crossImageCollection = pictureReader.collection<CrossAppImage>('images');
// Clock
export const crossAlarmCollection = clockReader.collection<CrossAppAlarm>('alarms');
export const crossTimerCollection = clockReader.collection<CrossAppTimer>('timers');
// Storage
export const crossFileCollection = storageReader.collection<CrossAppFile>('files');
export const crossFolderCollection = storageReader.collection<CrossAppFolder>('folders');
// Mukke
export const crossSongCollection = mukkeReader.collection<CrossAppSong>('songs');
export const crossPlaylistCollection = mukkeReader.collection<CrossAppPlaylist>('playlists');
// Presi
export const crossPresiDeckCollection = presiReader.collection<CrossAppDeck>('decks');
export const crossSlideCollection = presiReader.collection<CrossAppSlide>('slides');
// Context
export const crossSpaceCollection = contextReader.collection<CrossAppSpace>('spaces');
export const crossDocumentCollection = contextReader.collection<CrossAppDocument>('documents');
// Cards
export const crossCardsDeckCollection = cardsReader.collection<CrossAppCardsDeck>('decks');
export const crossCardsCardCollection = cardsReader.collection<CrossAppCardsCard>('cards');

136
apps/manacore/apps/web/src/lib/data/database.ts
View file

@ -229,3 +229,139 @@ export const SYNC_APP_MAP: Record<string, string[]> = {
export const TABLE_TO_APP: Record<string, string> = Object.fromEntries( export const TABLE_TO_APP: Record<string, string> = Object.fromEntries(
Object.entries(SYNC_APP_MAP).flatMap(([appId, tables]) => tables.map((table) => [table, appId])) Object.entries(SYNC_APP_MAP).flatMap(([appId, tables]) => tables.map((table) => [table, appId]))
); );
// ─── Table Name Mapping (Unified ↔ Backend) ──────────────────
// The unified DB renames tables to avoid collisions (e.g., todoProjects, cardDecks).
// The backend (mana-sync) knows the original names from standalone apps.
/** Unified table name → backend collection name (only renamed tables). */
export const TABLE_TO_SYNC_NAME: Record<string, string> = {
// todo
todoProjects: 'projects',
// chat
chatTemplates: 'templates',
// picture
pictureTags: 'tags',
// cards
cardDecks: 'decks',
// zitare
zitareFavorites: 'favorites',
zitareLists: 'lists',
// mukke
mukkePlaylists: 'playlists',
mukkeProjects: 'projects',
// storage
storageFolders: 'folders',
storageTags: 'tags',
// presi
presiDecks: 'decks',
// inventar
invCollections: 'collections',
invItems: 'items',
invLocations: 'locations',
invCategories: 'categories',
// photos
photoFavorites: 'favorites',
photoTags: 'tags',
photoMediaTags: 'photoTags',
// citycorners
ccLocations: 'locations',
ccFavorites: 'favorites',
// times
timeClients: 'clients',
timeProjects: 'projects',
timeTags: 'tags',
timeTemplates: 'templates',
timeSettings: 'settings',
// context
contextSpaces: 'spaces',
// questions
qCollections: 'collections',
// nutriphi
nutriFavorites: 'favorites',
// memoro
memoroTags: 'tags',
memoroSpaces: 'spaces',
// uload
uloadTags: 'tags',
uloadFolders: 'folders',
// guides
guideCollections: 'collections',
// shared: tags
globalTags: 'tags',
tagGroups: 'tagGroups',
// shared: links
manaLinks: 'links',
};
/** Get the backend collection name for a unified table. */
export function toSyncName(tableName: string): string {
return TABLE_TO_SYNC_NAME[tableName] ?? tableName;
}
/** Build reverse map: for a given appId, maps backend collection name → unified table name. */
export const SYNC_NAME_TO_TABLE: Record<string, Record<string, string>> = {};
for (const [appId, tables] of Object.entries(SYNC_APP_MAP)) {
const map: Record<string, string> = {};
for (const tableName of tables) {
const syncName = toSyncName(tableName);
map[syncName] = tableName;
}
SYNC_NAME_TO_TABLE[appId] = map;
}
/** Get the unified table name for a backend collection + appId. */
export function fromSyncName(appId: string, syncCollection: string): string {
return SYNC_NAME_TO_TABLE[appId]?.[syncCollection] ?? syncCollection;
}
// ─── Change Tracking via Dexie Hooks ─────────────────────────
// Automatically records pending changes for every write to sync-relevant tables.
// This means module stores (taskTable.add(), etc.) don't need manual trackChange() calls.
let _applyingServerChanges = false;
/** Set to true while applying server changes to prevent sync loops. */
export function setApplyingServerChanges(v: boolean): void {
_applyingServerChanges = v;
}
const pendingChangesTable = db.table('_pendingChanges');
for (const [appId, tables] of Object.entries(SYNC_APP_MAP)) {
for (const tableName of tables) {
const table = db.table(tableName);
table.hook('creating', function (_primKey, obj) {
if (_applyingServerChanges) return;
const now = new Date().toISOString();
pendingChangesTable.add({
appId,
collection: tableName,
recordId: obj.id,
op: 'insert',
data: { ...obj },
createdAt: now,
});
});
table.hook('updating', function (modifications, primKey) {
if (_applyingServerChanges) return;
const now = new Date().toISOString();
const fields: Record<string, { value: unknown; updatedAt: string }> = {};
for (const [key, value] of Object.entries(modifications)) {
if (key === 'id') continue;
fields[key] = { value, updatedAt: now };
}
pendingChangesTable.add({
appId,
collection: tableName,
recordId: primKey as string,
op: (modifications as Record<string, unknown>).deletedAt ? 'delete' : 'update',
fields,
deletedAt: (modifications as Record<string, unknown>).deletedAt as string | undefined,
createdAt: now,
});
});
}
}

173
apps/manacore/apps/web/src/lib/data/legacy-migration.ts Normal file
View file

@ -0,0 +1,173 @@
/**
* Legacy Database Migration
*
* Migrates data from old per-app IndexedDB databases (manacore-todo,
* manacore-calendar, etc.) into the unified `manacore` database.
*
* This runs once on app startup. After migration, old DBs are kept
* (not deleted) as a safety net — they can be removed later.
*/
import Dexie from 'dexie';
import { db, SYNC_APP_MAP, TABLE_TO_SYNC_NAME } from './database';
const MIGRATION_KEY = 'manacore-unified-migrated';
const MIGRATION_VERSION = '1';
/**
* Reverse of TABLE_TO_SYNC_NAME: for a given appId, maps
* old DB table name → unified DB table name.
*/
function buildLegacyToUnifiedMap(): Record<string, Record<string, string>> {
const result: Record<string, Record<string, string>> = {};
for (const [appId, tables] of Object.entries(SYNC_APP_MAP)) {
const map: Record<string, string> = {};
for (const unifiedName of tables) {
// The old DB used the backend collection name (before renaming)
const legacyName = TABLE_TO_SYNC_NAME[unifiedName] ?? unifiedName;
map[legacyName] = unifiedName;
}
result[appId] = map;
}
return result;
}
const LEGACY_TO_UNIFIED = buildLegacyToUnifiedMap();
/**
* Migrate all legacy per-app databases into the unified DB.
* Idempotent — checks if each record already exists before inserting.
* Skips if migration was already completed.
*/
export async function migrateFromLegacyDbs(): Promise<void> {
// Skip if already migrated
if (typeof localStorage !== 'undefined') {
const migrated = localStorage.getItem(MIGRATION_KEY);
if (migrated === MIGRATION_VERSION) return;
}
const appIds = Object.keys(SYNC_APP_MAP);
let migratedAny = false;
for (const appId of appIds) {
const legacyDbName = `manacore-${appId}`;
// Check if legacy DB exists
const exists = await Dexie.exists(legacyDbName);
if (!exists) continue;
try {
await migrateSingleApp(appId, legacyDbName);
migratedAny = true;
} catch (err) {
console.warn(`[LegacyMigration] Failed to migrate ${legacyDbName}:`, err);
// Continue with other apps — don't block on one failure
}
}
// Also migrate shared stores
await migrateSharedStore('manacore-tags', {
tags: 'globalTags',
tagGroups: 'tagGroups',
});
await migrateSharedStore('manacore-links', {
links: 'manaLinks',
});
// Mark migration as complete
if (typeof localStorage !== 'undefined') {
localStorage.setItem(MIGRATION_KEY, MIGRATION_VERSION);
}
if (migratedAny) {
console.log('[LegacyMigration] Migration complete');
}
}
/**
* Migrate a single app's legacy DB into the unified DB.
*/
async function migrateSingleApp(appId: string, legacyDbName: string): Promise<void> {
const legacyDb = new Dexie(legacyDbName);
// Open without specifying schema — Dexie will read the existing schema
await legacyDb.open();
const tableMapping = LEGACY_TO_UNIFIED[appId] ?? {};
const legacyTables = legacyDb.tables.map((t) => t.name);
for (const legacyTableName of legacyTables) {
// Skip internal tables
if (legacyTableName.startsWith('_')) continue;
// Find the unified table name
const unifiedTableName = tableMapping[legacyTableName] ?? legacyTableName;
// Check if this table exists in the unified DB
try {
db.table(unifiedTableName);
} catch {
// Table doesn't exist in unified DB — skip
continue;
}
// Read all records from legacy table
const records = await legacyDb.table(legacyTableName).toArray();
if (records.length === 0) continue;
// Batch upsert into unified DB (idempotent via bulkPut)
const unifiedTable = db.table(unifiedTableName);
await unifiedTable.bulkPut(records);
}
// Migrate sync cursors (_syncMeta)
try {
const syncMeta = await legacyDb.table('_syncMeta').toArray();
for (const meta of syncMeta) {
const collection = meta.collection;
const unifiedCollection = tableMapping[collection] ?? collection;
await db.table('_syncMeta').put({
appId,
collection: unifiedCollection,
lastSyncedAt: meta.lastSyncedAt ?? meta.syncedUntil ?? '1970-01-01T00:00:00.000Z',
pendingCount: 0,
});
}
} catch {
// _syncMeta may not exist in legacy DB
}
legacyDb.close();
}
/**
* Migrate a shared store (tags, links) from its own legacy DB.
*/
async function migrateSharedStore(
legacyDbName: string,
tableMapping: Record<string, string>
): Promise<void> {
const exists = await Dexie.exists(legacyDbName);
if (!exists) return;
try {
const legacyDb = new Dexie(legacyDbName);
await legacyDb.open();
for (const [legacyName, unifiedName] of Object.entries(tableMapping)) {
try {
const records = await legacyDb.table(legacyName).toArray();
if (records.length === 0) continue;
await db.table(unifiedName).bulkPut(records);
} catch {
// Table may not exist
}
}
legacyDb.close();
} catch (err) {
console.warn(`[LegacyMigration] Failed to migrate ${legacyDbName}:`, err);
}
}

117
apps/manacore/apps/web/src/lib/data/local-store.ts
View file

@ -1,14 +1,17 @@
/** /**
* ManaCore App — Local-First Data Layer * ManaCore App — Local-First Data Layer
* *
* Defines the IndexedDB database, collections, and guest seed data. * Provides typed collection accessors on the unified DB for core ManaCore data.
* Uses the unified `manacore` Dexie database (not a separate per-app DB).
*
* Collections: userSettings, dashboardConfigs * Collections: userSettings, dashboardConfigs
* Tags use the shared tagLocalStore from @manacore/shared-stores. * Tags use the shared tagLocalStore from @manacore/shared-stores.
*/ */
import { createLocalStore, type BaseRecord } from '@manacore/local-store'; import type { BaseRecord } from '@manacore/local-store';
import type { WidgetConfig } from '$lib/types/dashboard'; import type { WidgetConfig } from '$lib/types/dashboard';
import type { TileNode } from '$lib/types/tiling'; import type { TileNode } from '$lib/types/tiling';
import { db } from './database';
import { guestSettings, guestDashboardConfigs } from './guest-seed.js'; import { guestSettings, guestDashboardConfigs } from './guest-seed.js';
// ─── Types ────────────────────────────────────────────────── // ─── Types ──────────────────────────────────────────────────
@ -33,30 +36,96 @@ export interface LocalDashboardConfig extends BaseRecord {
tiling?: TileNode; tiling?: TileNode;
} }
// ─── Store ────────────────────────────────────────────────── // ─── Collection Wrappers ────────────────────────────────────
// Wraps Dexie tables with a LocalCollection-compatible API so existing
// consumers (queries.ts, dashboard.svelte.ts, tiling.svelte.ts) work unchanged.
const SYNC_SERVER_URL = import.meta.env.PUBLIC_SYNC_SERVER_URL || 'http://localhost:3050'; function createCollectionWrapper<T extends BaseRecord>(tableName: string) {
const table = db.table<T>(tableName);
export const manacoreStore = createLocalStore({ return {
appId: 'manacore', async get(id: string): Promise<T | undefined> {
collections: [ const record = await table.get(id);
{ if (record && (record as any).deletedAt) return undefined;
name: 'userSettings', return record;
indexes: ['key'],
guestSeed: guestSettings,
}, },
{
name: 'dashboardConfigs',
indexes: [],
guestSeed: guestDashboardConfigs,
},
],
sync: {
serverUrl: SYNC_SERVER_URL,
},
});
// Typed collection accessors async getAll(
export const settingsCollection = manacoreStore.collection<LocalUserSettings>('userSettings'); _filter?: unknown,
options?: { sortBy?: string; sortDirection?: 'asc' | 'desc' }
): Promise<T[]> {
let results = await table.toArray();
results = results.filter((r) => !(r as any).deletedAt);
if (options?.sortBy) {
const key = options.sortBy as keyof T;
const dir = options.sortDirection === 'desc' ? -1 : 1;
results.sort((a, b) => {
const aVal = String(a[key] ?? '');
const bVal = String(b[key] ?? '');
return aVal.localeCompare(bVal) * dir;
});
}
return results;
},
async insert(record: T): Promise<void> {
const now = new Date().toISOString();
await table.put({
...record,
createdAt: record.createdAt ?? now,
updatedAt: now,
});
},
async update(id: string, changes: Partial<T>): Promise<void> {
await table.update(id, {
...changes,
updatedAt: new Date().toISOString(),
} as any);
},
async delete(id: string): Promise<void> {
await table.update(id, {
deletedAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
} as any);
},
async count(): Promise<number> {
const all = await table.toArray();
return all.filter((r) => !(r as any).deletedAt).length;
},
};
}
export const settingsCollection = createCollectionWrapper<LocalUserSettings>('userSettings');
export const dashboardCollection = export const dashboardCollection =
manacoreStore.collection<LocalDashboardConfig>('dashboardConfigs'); createCollectionWrapper<LocalDashboardConfig>('dashboardConfigs');
// ─── Store-compatible facade ────────────────────────────────
// Provides initialize() / startSync() / stopSync() so the layout
// can call manacoreStore.initialize() without breaking.
let _initialized = false;
export const manacoreStore = {
async initialize(): Promise<void> {
if (_initialized) return;
_initialized = true;
// Seed guest data if tables are empty
const settingsCount = await db.table('userSettings').count();
if (settingsCount === 0 && guestSettings.length > 0) {
await db.table('userSettings').bulkPut(guestSettings);
}
const dashboardCount = await db.table('dashboardConfigs').count();
if (dashboardCount === 0 && guestDashboardConfigs.length > 0) {
await db.table('dashboardConfigs').bulkPut(guestDashboardConfigs);
}
},
// No-ops — sync is handled by the unified sync engine
startSync(_getToken: () => Promise<string | null>): void {},
stopSync(): void {},
};

259
apps/manacore/apps/web/src/lib/data/sync.ts
View file

@ -7,10 +7,14 @@
* Architecture: * Architecture:
* Unified DB → PendingChange (tagged with appId) → SyncChannel per appId → mana-sync /sync/{appId} * Unified DB → PendingChange (tagged with appId) → SyncChannel per appId → mana-sync /sync/{appId}
* mana-sync /sync/{appId} → WebSocket push → SyncChannel → applies to Unified DB * mana-sync /sync/{appId} → WebSocket push → SyncChannel → applies to Unified DB
*
* Backend protocol (mana-sync Go):
* Push: POST /sync/{appId} — body: { clientId, since, changes: [{ table, id, op, fields, data }] }
* Pull: GET /sync/{appId}/pull?collection={name}&since={cursor}
* WS: GET /ws/{appId} — auth: { type: "auth", token: "..." }
*/ */
import { db, SYNC_APP_MAP, TABLE_TO_APP } from './database'; import { db, SYNC_APP_MAP, toSyncName, fromSyncName, setApplyingServerChanges } from './database';
import type Dexie from 'dexie';
// ─── Types ──────────────────────────────────────────────────── // ─── Types ────────────────────────────────────────────────────
@ -42,21 +46,23 @@ interface SyncChannelState {
lastError: string | null; lastError: string | null;
} }
type SyncStatus = 'idle' | 'syncing' | 'error' | 'offline'; export type SyncStatus = 'idle' | 'syncing' | 'error' | 'offline';
// ─── Config ─────────────────────────────────────────────────── // ─── Config ───────────────────────────────────────────────────
const PUSH_DEBOUNCE = 1000; const PUSH_DEBOUNCE = 1000;
const PULL_INTERVAL = 30_000; const PULL_INTERVAL = 30_000;
const WS_RECONNECT_DELAY = 5000; const WS_RECONNECT_DELAY = 5000;
const WS_AUTH_TIMEOUT = 10_000;
// ─── Unified Sync Manager ───────────────────────────────────── // ─── Unified Sync Manager ─────────────────────────────────────
export function createUnifiedSync(serverUrl: string, getToken: () => Promise<string | null>) { export function createUnifiedSync(serverUrl: string, getToken: () => Promise<string | null>) {
const channels = new Map<string, SyncChannelState>(); const channels = new Map<string, SyncChannelState>();
let clientId = getOrCreateClientId(); const clientId = getOrCreateClientId();
let status: SyncStatus = 'idle'; let status: SyncStatus = 'idle';
let online = typeof navigator !== 'undefined' ? navigator.onLine : true; let online = typeof navigator !== 'undefined' ? navigator.onLine : true;
let _statusListeners: Array<(s: SyncStatus) => void> = [];
// ─── Lifecycle ────────────────────────────────────────── // ─── Lifecycle ──────────────────────────────────────────
@ -80,17 +86,6 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
connectWs(appId); connectWs(appId);
} }
// Watch _pendingChanges for new writes
db.table('_pendingChanges').hook('creating', (primKey, obj) => {
// Auto-tag with appId based on collection
if (!obj.appId && obj.collection) {
obj.appId = TABLE_TO_APP[obj.collection] || 'manacore';
}
// Debounced push
const appId = obj.appId;
if (appId) schedulePush(appId);
});
// Listen for online/offline // Listen for online/offline
if (typeof window !== 'undefined') { if (typeof window !== 'undefined') {
window.addEventListener('online', handleOnline); window.addEventListener('online', handleOnline);
@ -99,7 +94,7 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
} }
function stopAll(): void { function stopAll(): void {
for (const [appId, channel] of channels) { for (const [, channel] of channels) {
if (channel.pushTimer) clearTimeout(channel.pushTimer); if (channel.pushTimer) clearTimeout(channel.pushTimer);
if (channel.pullTimer) clearInterval(channel.pullTimer); if (channel.pullTimer) clearInterval(channel.pullTimer);
if (channel.ws) { if (channel.ws) {
@ -108,6 +103,7 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
} }
} }
channels.clear(); channels.clear();
_statusListeners = [];
if (typeof window !== 'undefined') { if (typeof window !== 'undefined') {
window.removeEventListener('online', handleOnline); window.removeEventListener('online', handleOnline);
@ -125,6 +121,11 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
channel.pushTimer = setTimeout(() => push(appId).catch(() => {}), PUSH_DEBOUNCE); channel.pushTimer = setTimeout(() => push(appId).catch(() => {}), PUSH_DEBOUNCE);
} }
/** Called from Dexie hooks when a pending change is recorded. */
function onPendingChange(appId: string): void {
schedulePush(appId);
}
async function push(appId: string): Promise<void> { async function push(appId: string): Promise<void> {
const channel = channels.get(appId); const channel = channels.get(appId);
if (!channel) return; if (!channel) return;
@ -141,30 +142,50 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
if (pending.length === 0) return; if (pending.length === 0) return;
status = 'syncing'; setStatus('syncing');
try { try {
const changeset = buildChangeset(pending, clientId); // Get oldest sync cursor for the `since` field
const res = await fetch(`${serverUrl}/sync/${appId}/push`, { const oldestCursor = await getOldestSyncCursor(appId);
// Build changeset in backend protocol format
const changeset = buildChangeset(pending, clientId, oldestCursor);
const res = await fetch(`${serverUrl}/sync/${appId}`, {
method: 'POST', method: 'POST',
headers: { headers: {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
Authorization: `Bearer ${token}`, Authorization: `Bearer ${token}`,
'X-Client-Id': clientId,
}, },
body: JSON.stringify(changeset), body: JSON.stringify(changeset),
}); });
if (!res.ok) throw new Error(`Push failed: ${res.status}`); if (!res.ok) throw new Error(`Push failed: ${res.status}`);
const data = await res.json();
// Apply server changes from the response
if (data.serverChanges?.length > 0) {
await applyServerChanges(appId, data.serverChanges);
}
// Update sync cursor
if (data.syncedUntil) {
for (const tableName of channel.tables) {
await setSyncCursor(appId, tableName, data.syncedUntil);
}
}
// Clear synced pending changes // Clear synced pending changes
const ids = pending.map((p) => p.id).filter((id): id is number => id !== undefined); const ids = pending.map((p) => p.id).filter((id): id is number => id !== undefined);
await db.table('_pendingChanges').bulkDelete(ids); await db.table('_pendingChanges').bulkDelete(ids);
channel.lastError = null; channel.lastError = null;
status = 'idle'; setStatus('idle');
} catch (err) { } catch (err) {
channel.lastError = err instanceof Error ? err.message : 'Push failed'; channel.lastError = err instanceof Error ? err.message : 'Push failed';
status = 'error'; setStatus('error');
} }
} }
@ -177,26 +198,30 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
const token = await getToken(); const token = await getToken();
if (!token) return; if (!token) return;
status = 'syncing'; setStatus('syncing');
try { try {
for (const tableName of channel.tables) { for (const tableName of channel.tables) {
const syncName = toSyncName(tableName);
const cursor = await getSyncCursor(appId, tableName); const cursor = await getSyncCursor(appId, tableName);
const res = await fetch( const res = await fetch(
`${serverUrl}/sync/${appId}/pull?collection=${tableName}&since=${encodeURIComponent(cursor)}&clientId=${clientId}`, `${serverUrl}/sync/${appId}/pull?collection=${encodeURIComponent(syncName)}&since=${encodeURIComponent(cursor)}`,
{ {
headers: { Authorization: `Bearer ${token}` }, headers: {
Authorization: `Bearer ${token}`,
'X-Client-Id': clientId,
},
} }
); );
if (!res.ok) continue; if (!res.ok) continue;
const data = await res.json(); const data = await res.json();
if (!data.changes || data.changes.length === 0) continue; if (!data.serverChanges || data.serverChanges.length === 0) continue;
// Apply changes to local DB // Apply changes to local DB
await applyServerChanges(tableName, data.changes); await applyServerChanges(appId, data.serverChanges);
// Update cursor // Update cursor
if (data.syncedUntil) { if (data.syncedUntil) {
@ -205,10 +230,10 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
} }
channel.lastError = null; channel.lastError = null;
status = 'idle'; setStatus('idle');
} catch (err) { } catch (err) {
channel.lastError = err instanceof Error ? err.message : 'Pull failed'; channel.lastError = err instanceof Error ? err.message : 'Pull failed';
status = 'error'; setStatus('error');
} }
} }
@ -218,23 +243,30 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
const channel = channels.get(appId); const channel = channels.get(appId);
if (!channel || !online) return; if (!channel || !online) return;
const wsUrl = serverUrl.replace(/^http/, 'ws') + `/sync/${appId}/ws?clientId=${clientId}`; const wsUrl = serverUrl.replace(/^http/, 'ws') + `/ws/${appId}`;
try { try {
const ws = new WebSocket(wsUrl); const ws = new WebSocket(wsUrl);
ws.onopen = () => { ws.onopen = async () => {
channel.ws = ws; channel.ws = ws;
// Authenticate — backend requires auth within 10 seconds
const token = await getToken();
if (token && ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify({ type: 'auth', token }));
}
}; };
ws.onmessage = (event) => { ws.onmessage = (event) => {
try { try {
const msg = JSON.parse(event.data); const msg = JSON.parse(event.data);
if (msg.type === 'push') { if (msg.type === 'sync-available') {
// Server notifies us of new changes — trigger pull // Server notifies us of new changes — trigger pull
pull(appId).catch(() => {}); pull(appId).catch(() => {});
} }
} catch {} } catch {
// Ignore malformed messages
}
}; };
ws.onclose = () => { ws.onclose = () => {
@ -253,6 +285,91 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
} }
} }
// ─── Apply Server Changes ───────────────────────────────
async function applyServerChanges(appId: string, changes: any[]): Promise<void> {
setApplyingServerChanges(true);
try {
// Group changes by table (server returns backend collection names)
const byTable = new Map<string, any[]>();
for (const change of changes) {
const serverTable = change.table;
// Map backend collection name → unified table name
const unifiedTable = fromSyncName(appId, serverTable);
if (!byTable.has(unifiedTable)) byTable.set(unifiedTable, []);
byTable.get(unifiedTable)!.push(change);
}
for (const [tableName, tableChanges] of byTable) {
const table = db.table(tableName);
await db.transaction('rw', table, async () => {
for (const change of tableChanges) {
const recordId = change.id;
if (change.deletedAt || change.op === 'delete') {
// Soft delete or hard delete
const existing = await table.get(recordId);
if (existing) {
if (change.deletedAt) {
await table.update(recordId, {
deletedAt: change.deletedAt,
updatedAt: change.deletedAt,
});
} else {
await table.delete(recordId);
}
}
} else if (change.op === 'insert') {
// Upsert for inserts
const existing = await table.get(recordId);
if (!existing) {
await table.put(change.data ?? { id: recordId, ...change });
} else {
// Record exists — merge with LWW
const updates: Record<string, unknown> = {};
const changeData = change.data ?? change;
for (const [key, val] of Object.entries(changeData)) {
if (key === 'id') continue;
updates[key] = val;
}
if (Object.keys(updates).length > 0) {
await table.update(recordId, updates);
}
}
} else if (change.op === 'update' && change.fields) {
// Field-level LWW update
const existing = await table.get(recordId);
if (!existing) {
// Record doesn't exist locally — reconstruct from fields
const record: Record<string, unknown> = { id: recordId };
for (const [key, fc] of Object.entries(change.fields as Record<string, any>)) {
record[key] = fc.value;
}
await table.put(record);
} else {
// Merge — only update fields that are newer
const updates: Record<string, unknown> = {};
for (const [key, fc] of Object.entries(change.fields as Record<string, any>)) {
const serverTime = fc.updatedAt ?? '';
const localTime = (existing as any).updatedAt ?? '';
if (serverTime >= localTime) {
updates[key] = fc.value;
}
}
if (Object.keys(updates).length > 0) {
await table.update(recordId, updates);
}
}
}
}
});
}
} finally {
setApplyingServerChanges(false);
}
}
// ─── Helpers ───────────────────────────────────────────── // ─── Helpers ─────────────────────────────────────────────
async function getSyncCursor(appId: string, collection: string): Promise<string> { async function getSyncCursor(appId: string, collection: string): Promise<string> {
@ -273,66 +390,40 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
}); });
} }
async function applyServerChanges(tableName: string, changes: any[]): Promise<void> { async function getOldestSyncCursor(appId: string): Promise<string> {
const table = db.table(tableName); const channel = channels.get(appId);
if (!channel) return '1970-01-01T00:00:00.000Z';
await db.transaction('rw', table, async () => { let oldest = new Date().toISOString();
for (const change of changes) { for (const tableName of channel.tables) {
if (change.deletedAt) { const cursor = await getSyncCursor(appId, tableName);
// Soft delete if (cursor < oldest) oldest = cursor;
const existing = await table.get(change.id); }
if (existing) { return oldest;
await table.update(change.id, {
deletedAt: change.deletedAt,
updatedAt: change.updatedAt,
});
}
} else if (change.op === 'delete') {
await table.delete(change.id);
} else {
// Upsert — field-level LWW
const existing = await table.get(change.id);
if (!existing) {
await table.put(change.data ?? change);
} else {
// Only update fields that are newer
const updates: Record<string, unknown> = {};
const changeData = change.data ?? change;
for (const [key, val] of Object.entries(changeData)) {
if (key === 'id') continue;
const serverTime = change.fields?.[key]?.updatedAt ?? change.updatedAt;
const localTime = (existing as any).updatedAt ?? '';
if (serverTime >= localTime) {
updates[key] = val;
}
}
if (Object.keys(updates).length > 0) {
await table.update(change.id, updates);
}
}
}
}
});
} }
function buildChangeset(pending: PendingChange[], cid: string) { /**
* Build changeset in backend protocol format.
* Maps unified table names to backend collection names.
*/
function buildChangeset(pending: PendingChange[], cid: string, since: string) {
return { return {
clientId: cid, clientId: cid,
since,
changes: pending.map((p) => ({ changes: pending.map((p) => ({
collection: p.collection, table: toSyncName(p.collection),
recordId: p.recordId, id: p.recordId,
op: p.op, op: p.op,
fields: p.fields, fields: p.fields,
data: p.data, data: p.data,
deletedAt: p.deletedAt, deletedAt: p.deletedAt,
createdAt: p.createdAt,
})), })),
}; };
} }
function handleOnline() { function handleOnline() {
online = true; online = true;
status = 'idle'; setStatus('idle');
// Resume sync for all channels // Resume sync for all channels
for (const appId of channels.keys()) { for (const appId of channels.keys()) {
pull(appId).catch(() => {}); pull(appId).catch(() => {});
@ -342,7 +433,7 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
function handleOffline() { function handleOffline() {
online = false; online = false;
status = 'offline'; setStatus('offline');
// Close all WebSockets // Close all WebSockets
for (const channel of channels.values()) { for (const channel of channels.values()) {
if (channel.ws) { if (channel.ws) {
@ -352,15 +443,29 @@ export function createUnifiedSync(serverUrl: string, getToken: () => Promise<str
} }
} }
function setStatus(s: SyncStatus) {
status = s;
for (const listener of _statusListeners) {
listener(s);
}
}
return { return {
startAll, startAll,
stopAll, stopAll,
onPendingChange,
get status() { get status() {
return status; return status;
}, },
get online() { get online() {
return online; return online;
}, },
onStatusChange(listener: (s: SyncStatus) => void) {
_statusListeners.push(listener);
return () => {
_statusListeners = _statusListeners.filter((l) => l !== listener);
};
},
getChannel: (appId: string) => channels.get(appId), getChannel: (appId: string) => channels.get(appId),
pushNow: push, pushNow: push,
pullNow: pull, pullNow: pull,

80
apps/manacore/apps/web/src/lib/modules/spiral/collect.ts
View file

@ -1,34 +1,28 @@
/** /**
* Cross-App Activity Collector * Cross-App Activity Collector
* *
* Reads from all cross-app IndexedDB readers and produces * Reads from the unified IndexedDB and produces
* AppSnapshot objects for the Mana Spiral. * AppSnapshot objects for the Mana Spiral.
*/ */
import { MANA_APP_INDEX } from '@manacore/spiral-db'; import { MANA_APP_INDEX } from '@manacore/spiral-db';
import { import { db } from '$lib/data/database';
crossTaskCollection,
crossEventCollection,
crossContactCollection,
crossConversationCollection,
crossFavoriteCollection,
crossImageCollection,
crossAlarmCollection,
crossFileCollection,
crossSongCollection,
crossPresiDeckCollection,
crossSpaceCollection,
crossCardsDeckCollection,
crossCardsCardCollection,
type CrossAppTask,
type CrossAppContact,
type CrossAppImage,
} from '$lib/data/cross-app-stores';
import type { AppSnapshot } from './stores/mana-spiral.svelte'; import type { AppSnapshot } from './stores/mana-spiral.svelte';
/** /**
* Collect snapshots from all cross-app readers. * Safe wrapper for db.table().toArray() — returns empty array on error.
* Each collection is read once and summarized into an AppSnapshot. */
async function safeGetAll(tableName: string): Promise<any[]> {
try {
return await db.table(tableName).toArray();
} catch {
return [];
}
}
/**
* Collect snapshots from all app tables in the unified DB.
* Each table is read once and summarized into an AppSnapshot.
*/ */
export async function collectAppSnapshots(): Promise<AppSnapshot[]> { export async function collectAppSnapshots(): Promise<AppSnapshot[]> {
const snapshots: AppSnapshot[] = []; const snapshots: AppSnapshot[] = [];
@ -49,24 +43,24 @@ export async function collectAppSnapshots(): Promise<AppSnapshot[]> {
cardDecks, cardDecks,
cards, cards,
] = await Promise.all([ ] = await Promise.all([
safeGetAll(crossTaskCollection), safeGetAll('tasks'),
safeGetAll(crossEventCollection), safeGetAll('events'),
safeGetAll(crossContactCollection), safeGetAll('contacts'),
safeGetAll(crossConversationCollection), safeGetAll('conversations'),
safeGetAll(crossFavoriteCollection), safeGetAll('zitareFavorites'),
safeGetAll(crossImageCollection), safeGetAll('images'),
safeGetAll(crossAlarmCollection), safeGetAll('alarms'),
safeGetAll(crossFileCollection), safeGetAll('files'),
safeGetAll(crossSongCollection), safeGetAll('songs'),
safeGetAll(crossPresiDeckCollection), safeGetAll('presiDecks'),
safeGetAll(crossSpaceCollection), safeGetAll('contextSpaces'),
safeGetAll(crossCardsDeckCollection), safeGetAll('cardDecks'),
safeGetAll(crossCardsCardCollection), safeGetAll('cards'),
]); ]);
// Todo // Todo
if (tasks.length > 0) { if (tasks.length > 0) {
const completed = (tasks as CrossAppTask[]).filter((t) => t.isCompleted).length; const completed = tasks.filter((t: any) => t.isCompleted).length;
snapshots.push({ snapshots.push({
app: 'Todo', app: 'Todo',
appIndex: MANA_APP_INDEX.todo, appIndex: MANA_APP_INDEX.todo,
@ -91,7 +85,7 @@ export async function collectAppSnapshots(): Promise<AppSnapshot[]> {
// Contacts // Contacts
if (contacts.length > 0) { if (contacts.length > 0) {
const favs = (contacts as CrossAppContact[]).filter((c) => c.isFavorite).length; const favs = contacts.filter((c: any) => c.isFavorite).length;
snapshots.push({ snapshots.push({
app: 'Contacts', app: 'Contacts',
appIndex: MANA_APP_INDEX.contacts, appIndex: MANA_APP_INDEX.contacts,
@ -128,7 +122,7 @@ export async function collectAppSnapshots(): Promise<AppSnapshot[]> {
// Picture // Picture
if (images.length > 0) { if (images.length > 0) {
const favs = (images as CrossAppImage[]).filter((i) => i.isFavorite).length; const favs = images.filter((i: any) => i.isFavorite).length;
snapshots.push({ snapshots.push({
app: 'Picture', app: 'Picture',
appIndex: MANA_APP_INDEX.picture, appIndex: MANA_APP_INDEX.picture,
@ -213,15 +207,3 @@ export async function collectAppSnapshots(): Promise<AppSnapshot[]> {
return snapshots; return snapshots;
} }
/**
* Safe wrapper for collection.getAll() — returns empty array on error
* (e.g. if the other app's DB doesn't exist yet)
*/
async function safeGetAll(collection: { getAll: () => Promise<unknown[]> }): Promise<unknown[]> {
try {
return await collection.getAll();
} catch {
return [];
}
}

46
apps/manacore/apps/web/src/routes/(app)/+layout.svelte
View file

@ -16,20 +16,8 @@
import { tagLocalStore, tagMutations, useAllTags } from '$lib/stores/tags.svelte'; import { tagLocalStore, tagMutations, useAllTags } from '$lib/stores/tags.svelte';
import { linkLocalStore, linkMutations } from '@manacore/shared-links'; import { linkLocalStore, linkMutations } from '@manacore/shared-links';
import { manacoreStore } from '$lib/data/local-store'; import { manacoreStore } from '$lib/data/local-store';
import { import { createUnifiedSync } from '$lib/data/sync';
todoReader, import { migrateFromLegacyDbs } from '$lib/data/legacy-migration';
calendarReader,
contactsReader,
chatReader,
zitareReader,
pictureReader,
clockReader,
storageReader,
mukkeReader,
presiReader,
contextReader,
cardsReader,
} from '$lib/data/cross-app-stores';
import { dashboardStore } from '$lib/stores/dashboard.svelte'; import { dashboardStore } from '$lib/stores/dashboard.svelte';
import { import {
THEME_DEFINITIONS, THEME_DEFINITIONS,
@ -203,9 +191,12 @@
AppEvents.themeChanged(mode); AppEvents.themeChanged(mode);
} }
// Unified sync manager — one sync engine for all apps
const SYNC_SERVER_URL = import.meta.env.PUBLIC_SYNC_SERVER_URL || 'http://localhost:3050';
let unifiedSync: ReturnType<typeof createUnifiedSync> | null = null;
async function handleSignOut() { async function handleSignOut() {
manacoreStore.stopSync(); unifiedSync?.stopAll();
tagMutations.stopSync();
await authStore.signOut(); await authStore.signOut();
goto('/login'); goto('/login');
} }
@ -244,29 +235,18 @@
manacoreStore.initialize(), manacoreStore.initialize(),
tagLocalStore.initialize(), tagLocalStore.initialize(),
linkLocalStore.initialize(), linkLocalStore.initialize(),
// Cross-app readers (read-only, no sync — owning apps handle sync)
todoReader.initialize(),
calendarReader.initialize(),
contactsReader.initialize(),
chatReader.initialize(),
zitareReader.initialize(),
pictureReader.initialize(),
clockReader.initialize(),
storageReader.initialize(),
mukkeReader.initialize(),
presiReader.initialize(),
contextReader.initialize(),
cardsReader.initialize(),
]); ]);
// Migrate data from legacy per-app databases (one-time, idempotent)
await migrateFromLegacyDbs();
// Initialize shared-uload (opens uLoad IndexedDB for cross-app link creation) // Initialize shared-uload (opens uLoad IndexedDB for cross-app link creation)
initSharedUload(); initSharedUload();
// Start syncing to server // Start unified sync — one engine for all apps via Dexie hooks
const getToken = () => authStore.getValidToken(); const getToken = () => authStore.getValidToken();
manacoreStore.startSync(getToken); unifiedSync = createUnifiedSync(SYNC_SERVER_URL, getToken);
tagMutations.startSync(getToken); unifiedSync.startAll();
linkMutations.startSync(getToken);
// Initialize dashboard from IndexedDB // Initialize dashboard from IndexedDB
await dashboardStore.initialize(); await dashboardStore.initialize();