till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-14 20:21:09 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

feat(shared-llm): Phase 4 — persistent LLM task queue

Browse source 
Until now, modules wanting to use the orchestrator had to await each
LLM call inline in their store code. That's fine for foreground tasks
("user clicked summarize") but a non-starter for background work
("auto-tag every new note", "generate a title for every voice memo
after STT finishes"). Background tasks need to:

  - Queue up while no LLM tier is ready, then drain when one becomes
    available (e.g. user just enabled the browser tier from settings)
  - Survive page reloads, browser restarts, and the user navigating
    away mid-execution
  - Run one at a time without blocking the foreground UI
  - Allow modules to subscribe to results reactively without polling
  - Retry transient failures (network, model loading) but not
    semantic ones (tier-too-low, content blocked)

Phase 4 ships exactly that.

Architecture:

  packages/shared-llm/src/queue.ts — LlmTaskQueue class
    + QueuedTask interface (the persistent row shape)
    + EnqueueOptions (refType/refId/priority/maxAttempts)
    + TaskRegistry type (name → LlmTask map)
    + LlmTaskQueueOptions (table + orchestrator + registry +
                           retryBackoffMs + idleWakeupMs)

  Public API:
    - enqueue(task, input, opts) → string  (returns the queued id)
    - get(id), list(filter)
    - retry(id), cancel(id), purge(olderThanMs)
    - start(), stop()  (idempotent processor lifecycle)

  apps/mana/apps/web/src/lib/llm-queue.ts — web app singleton
    - Dedicated `mana-llm-queue` Dexie database (separate from the
      main `mana` IDB; see comment for the rationale: ephemeral
      per-device state, no encryption needed, no sync needed, doesn't
      belong in the long-frozen `mana` schema)
    - Wires up the queue with llmOrchestrator + taskRegistry
    - Exposes startLlmQueue() / stopLlmQueue() for the layout hook

  apps/mana/apps/web/src/lib/llm-task-registry.ts
    - Maps task names → task objects so the queue processor can
      look up the implementation when pulling rows off the table.
      Closures can't be persisted, so we round-trip via name.
    - Currently registers extractDateTask + summarizeTextTask;
      module-side tasks land here as we add them.

  apps/mana/apps/web/src/routes/(app)/+layout.svelte
    - startLlmQueue() in handleAuthReady's Phase A (auth-independent)
      so guests + authenticated users both get the queue
    - stopLlmQueue() in onDestroy as a fire-and-forget cleanup

Processor loop semantics (the heart of the implementation):

  1. On start(), reclaim any 'running' rows from a crashed previous
     session — reset them to 'pending'. The orphan recovery is the
     reason a crash mid-task doesn't leave the queue stuck.
  2. findNextRunnable() picks the highest-priority pending task whose
     `notBefore` (retry-backoff timestamp) is in the past. Sort key:
     priority desc, then enqueuedAt asc (FIFO within priority).
  3. Mark the task running, increment attempts, look up the LlmTask
     in the registry, hand it to orchestrator.run().
  4. On success: mark done, store result + source + finishedAt.
  5. On error:
       - TierTooLowError or ProviderBlockedError → fail immediately,
         no retry. These are not transient — the user's settings or
         the content itself need to change.
       - Anything else → if attempts < maxAttempts, reset to pending
         with notBefore = now + retryBackoffMs (default 60s). Else
         mark failed.
  6. When no work is pending, sleep on a Promise that resolves when
     either (a) someone calls enqueue() (which fires notifyWakeup),
     or (b) idleWakeupMs elapses (default 30s, safety net for any
     missed wakeup signal).

Module-side reactive reads use Dexie liveQuery directly on the queue
table — no special subscription API on the queue itself. This is
consistent with how every other Mana module reads its data, so the
mental model stays uniform:

  const tags = useLiveQuery(
    () => llmQueueDb.tasks
      .where({ refType: 'note', refId, taskName: 'common.extractTags' })
      .reverse().first(),
    [refId]
  );

Smoke test: a new "Queue" tab in /llm-test lets you enqueue the
existing extractDate / summarize tasks and watch the live state of
the queue table via liveQuery. The display includes per-row state
badge (pending/running/done/failed), tier source, attempt count,
input/output, and a "Done/failed löschen" button that exercises
purge().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS 2026-04-09 01:51:20 +02:00
parent 6e20c298ac
commit 3b5d58ecbe
8 changed files with 567 additions and 251 deletions
Download patch file Download diff file Expand all files Collapse all files

255
pnpm-lock.yaml generated
View file

@ -1567,6 +1567,9 @@ importers:
apps/memoro/apps/server:
dependencies:
'@mana/notify-client':
specifier: workspace:*
version: link:../../../../packages/notify-client
'@mana/shared-hono':
specifier: workspace:*
version: link:../../../../packages/shared-hono
@ -2565,34 +2568,6 @@ importers:
specifier: ^2.7.0
version: 2.7.0
packages/cards-database:
dependencies:
drizzle-orm:
specifier: ^0.36.0
version: 0.36.4(@opentelemetry/api@1.9.1)(@types/pg@8.6.1)(@types/react@19.2.14)(bun-types@1.3.11)(kysely@0.28.15)(postgres@3.4.9)(react@19.2.0)
postgres:
specifier: ^3.4.5
version: 3.4.9
devDependencies:
'@supabase/supabase-js':
specifier: ^2.81.1
version: 2.102.1
'@types/node':
specifier: ^22.10.0
version: 22.19.17
dotenv-cli:
specifier: ^7.4.0
version: 7.4.4
drizzle-kit:
specifier: ^0.28.0
version: 0.28.1
tsx:
specifier: ^4.19.0
version: 4.21.0
typescript:
specifier: ^5.7.3
version: 5.9.3
packages/credits:
devDependencies:
svelte:
@ -2762,16 +2737,6 @@ importers:
specifier: ^4.1.2
version: 4.1.3(@opentelemetry/api@1.9.1)(@types/node@24.12.2)(@vitest/coverage-v8@4.1.3)(@vitest/ui@4.1.3)(jsdom@29.0.2(@noble/hashes@2.0.1))(vite@6.4.2(@types/node@24.12.2)(jiti@2.6.1)(lightningcss@1.32.0)(terser@5.46.1)(tsx@4.21.0)(yaml@2.8.3))
packages/shared-api-client:
dependencies:
'@mana/shared-utils':
specifier: workspace:*
version: link:../shared-utils
devDependencies:
typescript:
specifier: ^5.9.3
version: 5.9.3
packages/shared-auth:
dependencies:
'@mana/shared-types':
@ -2885,21 +2850,6 @@ importers:
specifier: ^5.0.0
version: 5.9.3
packages/shared-errors:
devDependencies:
'@nestjs/common':
specifier: ^11.0.17
version: 11.1.18(class-transformer@0.5.1)(class-validator@0.14.4)(reflect-metadata@0.2.2)(rxjs@7.8.2)
'@types/express':
specifier: ^5.0.0
version: 5.0.6
'@types/node':
specifier: ^22.0.0
version: 22.19.17
typescript:
specifier: ^5.9.3
version: 5.9.3
packages/shared-hono:
dependencies:
'@mana/shared-logger':
@ -2987,6 +2937,9 @@ importers:
'@mana/local-llm':
specifier: workspace:*
version: link:../local-llm
dexie:
specifier: ^4.0.0
version: 4.4.2
devDependencies:
'@types/node':
specifier: ^24.10.1
@ -3022,18 +2975,6 @@ importers:
specifier: ^7.0.0
version: 7.4.0(@types/babel__core@7.20.5)
packages/shared-splitscreen:
devDependencies:
svelte:
specifier: ^5.0.0
version: 5.55.1
svelte-check:
specifier: ^4.0.0
version: 4.4.6(picomatch@4.0.4)(svelte@5.55.1)(typescript@5.9.3)
typescript:
specifier: ^5.0.0
version: 5.9.3
packages/shared-storage:
dependencies:
'@aws-sdk/client-s3':
@ -6719,19 +6660,6 @@ packages:
class-validator:
optional: true
'@nestjs/common@11.1.18':
resolution: {integrity: sha512-0sLq8Z+TIjLnz1Tqp0C/x9BpLbqpt1qEu0VcH4/fkE0y3F5JxhfK1AdKQ/SPbKhKgwqVDoY4gS8GQr2G6ujaWg==}
peerDependencies:
class-transformer: '>=0.4.1'
class-validator: '>=0.13.2'
reflect-metadata: ^0.1.12 || ^0.2.0
rxjs: ^7.1.0
peerDependenciesMeta:
class-transformer:
optional: true
class-validator:
optional: true
'@nestjs/config@3.3.0':
resolution: {integrity: sha512-pdGTp8m9d0ZCrjTpjkUbZx6gyf2IKf+7zlkrPNMsJzYZ4bFRRTpXrnj+556/5uiI6AfL5mMrJc2u7dB6bvM+VA==}
peerDependencies:
@ -8446,10 +8374,6 @@ packages:
resolution: {integrity: sha512-MADQgmZT1eKjp06jpI2yozxaU9uVs4GzzgSL+uEq7bVcJ9V1ZXQkeGNql1fsSI0gMy1vhvNTNbUqrx+pZfJVmg==}
engines: {node: '>=18'}
'@tokenizer/inflate@0.4.1':
resolution: {integrity: sha512-2mAv+8pkG6GIZiF1kNg1jAjh27IDxEPKwdGul3snfztFerfPGI1LjDezZp3i7BElXompqEtPmoPx6c2wgtWsOA==}
engines: {node: '>=18'}
'@tokenizer/token@0.3.0':
resolution: {integrity: sha512-OvjF+z51L3ov0OyAU0duzsYuvO01PH7x4t6DJx+guahgTnBHkhJdG7soQeTSFLWN3efnHyibZ4Z8l2EuWwJN3A==}
@ -10540,10 +10464,6 @@ packages:
dot-case@3.0.4:
resolution: {integrity: sha512-Kv5nKlh6yRrdrGvxeJ2e5y2eRUpkUosIW4A2AS38zwSz27zu7ufDwQPi5Jhs3XAlGNetl3bmnGhQsMtkKJnj3w==}
dotenv-cli@7.4.4:
resolution: {integrity: sha512-XkBYCG0tPIes+YZr4SpfFv76SQrV/LeCE8CI7JSEMi3VR9MvTihCGTOtbIexD6i2mXF+6px7trb1imVCXSNMDw==}
hasBin: true
dotenv-expand@10.0.0:
resolution: {integrity: sha512-GopVGCpVS1UKH75VKHGuQFqS1Gusej0z4FyQkPdwjil2gNIv+LNsqBlboOzpJFZKVT95GkCyWJbBSdFEFUWI2A==}
engines: {node: '>=12'}
@ -10564,106 +10484,10 @@ packages:
resolution: {integrity: sha512-uBq4egWHTcTt33a72vpSG0z3HnPuIl6NqYcTrKEg2azoEyl2hpW0zqlxysq2pK9HlDIHyHyakeYaYnSAwd8bow==}
engines: {node: '>=12'}
drizzle-kit@0.28.1:
resolution: {integrity: sha512-JimOV+ystXTWMgZkLHYHf2w3oS28hxiH1FR0dkmJLc7GHzdGJoJAQtQS5DRppnabsRZwE2U1F6CuezVBgmsBBQ==}
hasBin: true
drizzle-kit@0.30.6:
resolution: {integrity: sha512-U4wWit0fyZuGuP7iNmRleQyK2V8wCuv57vf5l3MnG4z4fzNTjY/U13M8owyQ5RavqvqxBifWORaR3wIUzlN64g==}
hasBin: true
drizzle-orm@0.36.4:
resolution: {integrity: sha512-1OZY3PXD7BR00Gl61UUOFihslDldfH4NFRH2MbP54Yxi0G/PKn4HfO65JYZ7c16DeP3SpM3Aw+VXVG9j6CRSXA==}
peerDependencies:
'@aws-sdk/client-rds-data': '>=3'
'@cloudflare/workers-types': '>=3'
'@electric-sql/pglite': '>=0.2.0'
'@libsql/client': '>=0.10.0'
'@libsql/client-wasm': '>=0.10.0'
'@neondatabase/serverless': '>=0.10.0'
'@op-engineering/op-sqlite': '>=2'
'@opentelemetry/api': ^1.4.1
'@planetscale/database': '>=1'
'@prisma/client': '*'
'@tidbcloud/serverless': '*'
'@types/better-sqlite3': '*'
'@types/pg': '*'
'@types/react': '>=18'
'@types/sql.js': '*'
'@vercel/postgres': '>=0.8.0'
'@xata.io/client': '*'
better-sqlite3: '>=7'
bun-types: '*'
expo-sqlite: '>=14.0.0'
knex: '*'
kysely: '*'
mysql2: '>=2'
pg: '>=8'
postgres: '>=3'
prisma: '*'
react: '>=18'
sql.js: '>=1'
sqlite3: '>=5'
peerDependenciesMeta:
'@aws-sdk/client-rds-data':
optional: true
'@cloudflare/workers-types':
optional: true
'@electric-sql/pglite':
optional: true
'@libsql/client':
optional: true
'@libsql/client-wasm':
optional: true
'@neondatabase/serverless':
optional: true
'@op-engineering/op-sqlite':
optional: true
'@opentelemetry/api':
optional: true
'@planetscale/database':
optional: true
'@prisma/client':
optional: true
'@tidbcloud/serverless':
optional: true
'@types/better-sqlite3':
optional: true
'@types/pg':
optional: true
'@types/react':
optional: true
'@types/sql.js':
optional: true
'@vercel/postgres':
optional: true
'@xata.io/client':
optional: true
better-sqlite3:
optional: true
bun-types:
optional: true
expo-sqlite:
optional: true
knex:
optional: true
kysely:
optional: true
mysql2:
optional: true
pg:
optional: true
postgres:
optional: true
prisma:
optional: true
react:
optional: true
sql.js:
optional: true
sqlite3:
optional: true
drizzle-orm@0.38.4:
resolution: {integrity: sha512-s7/5BpLKO+WJRHspvpqTydxFob8i1vo2rEx4pY6TGY7QSMuUfWUuzaY0DIpXCkgHOo37BaFC+SJQb99dDUXT3Q==}
peerDependencies:
@ -12242,10 +12066,6 @@ packages:
resolution: {integrity: sha512-hw9gNZXUfZ02Jo0uafWLaFVPter5/k2rfcrjFJJHX/77xtSDOfJuEFb6oKlFV86FLP1SuyHMW1PSk0U9M5tKkQ==}
engines: {node: '>=18'}
file-type@21.3.4:
resolution: {integrity: sha512-Ievi/yy8DS3ygGvT47PjSfdFoX+2isQueoYP1cntFW1JLYAuS4GD7NUPGg4zv2iZfV52uDyk5w5Z0TdpRS6Q1g==}
engines: {node: '>=20'}
filelist@1.0.6:
resolution: {integrity: sha512-5giy2PkLYY1cP39p17Ech+2xlpTRL9HLspOfEgm0L6CwBXBTgsK5ou0JtzYuepxkaQ/tvhCFIJ5uXo0OrM2DxA==}
@ -13661,10 +13481,6 @@ packages:
resolution: {integrity: sha512-ME4Fb83LgEgwNw96RKNvKV4VTLuXfoKudAmm2lP8Kk87KaMK0/Xrx/aAkMWmT8mDb+3MlFDspfbCs7adjRxA2g==}
engines: {node: '>=20.0.0'}
load-esm@1.0.3:
resolution: {integrity: sha512-v5xlu8eHD1+6r8EHTg6hfmO97LN8ugKtiXcy5e6oN72iD2r6u0RPfLl6fxM+7Wnh2ZRq15o0russMst44WauPA==}
engines: {node: '>=13.2.0'}
load-tsconfig@0.2.5:
resolution: {integrity: sha512-IXO6OCs9yg8tMKzfPZ1YmheJbZCiEsnBdcB03l0OcfK9prKnJb96siuHCr5Fl37/yo9DnKU+TLpxzTUspw9shg==}
engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0}
@ -22003,21 +21819,6 @@ snapshots:
transitivePeerDependencies:
- supports-color
'@nestjs/common@11.1.18(class-transformer@0.5.1)(class-validator@0.14.4)(reflect-metadata@0.2.2)(rxjs@7.8.2)':
dependencies:
file-type: 21.3.4
iterare: 1.2.1
load-esm: 1.0.3
reflect-metadata: 0.2.2
rxjs: 7.8.2
tslib: 2.8.1
uid: 2.0.2
optionalDependencies:
class-transformer: 0.5.1
class-validator: 0.14.4
transitivePeerDependencies:
- supports-color
'@nestjs/config@3.3.0(@nestjs/common@10.4.22(class-transformer@0.5.1)(class-validator@0.14.4)(reflect-metadata@0.2.2)(rxjs@7.8.2))(rxjs@7.8.2)':
dependencies:
'@nestjs/common': 10.4.22(class-transformer@0.5.1)(class-validator@0.14.4)(reflect-metadata@0.2.2)(rxjs@7.8.2)
@ -24780,13 +24581,6 @@ snapshots:
transitivePeerDependencies:
- supports-color
'@tokenizer/inflate@0.4.1':
dependencies:
debug: 4.4.3
token-types: 6.1.2
transitivePeerDependencies:
- supports-color
'@tokenizer/token@0.3.0': {}
'@turbo/darwin-64@2.9.4':
@ -27791,13 +27585,6 @@ snapshots:
no-case: 3.0.4
tslib: 2.8.1
dotenv-cli@7.4.4:
dependencies:
cross-spawn: 7.0.6
dotenv: 16.6.1
dotenv-expand: 10.0.0
minimist: 1.2.8
dotenv-expand@10.0.0: {}
dotenv-expand@11.0.7:
@ -27810,15 +27597,6 @@ snapshots:
dotenv@16.6.1: {}
drizzle-kit@0.28.1:
dependencies:
'@drizzle-team/brocli': 0.10.2
'@esbuild-kit/esm-loader': 2.6.5
esbuild: 0.19.12
esbuild-register: 3.6.0(esbuild@0.19.12)
transitivePeerDependencies:
- supports-color
drizzle-kit@0.30.6:
dependencies:
'@drizzle-team/brocli': 0.10.2
@ -27829,16 +27607,6 @@ snapshots:
transitivePeerDependencies:
- supports-color
drizzle-orm@0.36.4(@opentelemetry/api@1.9.1)(@types/pg@8.6.1)(@types/react@19.2.14)(bun-types@1.3.11)(kysely@0.28.15)(postgres@3.4.9)(react@19.2.0):
optionalDependencies:
'@opentelemetry/api': 1.9.1
'@types/pg': 8.6.1
'@types/react': 19.2.14
bun-types: 1.3.11
kysely: 0.28.15
postgres: 3.4.9
react: 19.2.0
drizzle-orm@0.38.4(@opentelemetry/api@1.9.1)(@types/pg@8.6.1)(@types/react@19.2.14)(bun-types@1.3.11)(kysely@0.28.15)(postgres@3.4.9)(react@19.2.0):
optionalDependencies:
'@opentelemetry/api': 1.9.1
@ -30643,15 +30411,6 @@ snapshots:
transitivePeerDependencies:
- supports-color
file-type@21.3.4:
dependencies:
'@tokenizer/inflate': 0.4.1
strtok3: 10.3.5
token-types: 6.1.2
uint8array-extras: 1.5.0
transitivePeerDependencies:
- supports-color
filelist@1.0.6:
dependencies:
minimatch: 5.1.9
@ -32492,8 +32251,6 @@ snapshots:
rfdc: 1.4.1
wrap-ansi: 9.0.2
load-esm@1.0.3: {}
load-tsconfig@0.2.5: {}
loader-runner@4.3.1: {}