mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-15 00:41:09 +02:00
Mirror of github.com/Memo-2023/mana-monorepo
3b5d58ecbe
Until now, modules wanting to use the orchestrator had to await each
LLM call inline in their store code. That's fine for foreground tasks
("user clicked summarize") but a non-starter for background work
("auto-tag every new note", "generate a title for every voice memo
after STT finishes"). Background tasks need to:
- Queue up while no LLM tier is ready, then drain when one becomes
available (e.g. user just enabled the browser tier from settings)
- Survive page reloads, browser restarts, and the user navigating
away mid-execution
- Run one at a time without blocking the foreground UI
- Allow modules to subscribe to results reactively without polling
- Retry transient failures (network, model loading) but not
semantic ones (tier-too-low, content blocked)
Phase 4 ships exactly that.
Architecture:
packages/shared-llm/src/queue.ts — LlmTaskQueue class
+ QueuedTask interface (the persistent row shape)
+ EnqueueOptions (refType/refId/priority/maxAttempts)
+ TaskRegistry type (name → LlmTask map)
+ LlmTaskQueueOptions (table + orchestrator + registry +
retryBackoffMs + idleWakeupMs)
Public API:
- enqueue(task, input, opts) → string (returns the queued id)
- get(id), list(filter)
- retry(id), cancel(id), purge(olderThanMs)
- start(), stop() (idempotent processor lifecycle)
apps/mana/apps/web/src/lib/llm-queue.ts — web app singleton
- Dedicated `mana-llm-queue` Dexie database (separate from the
main `mana` IDB; see comment for the rationale: ephemeral
per-device state, no encryption needed, no sync needed, doesn't
belong in the long-frozen `mana` schema)
- Wires up the queue with llmOrchestrator + taskRegistry
- Exposes startLlmQueue() / stopLlmQueue() for the layout hook
apps/mana/apps/web/src/lib/llm-task-registry.ts
- Maps task names → task objects so the queue processor can
look up the implementation when pulling rows off the table.
Closures can't be persisted, so we round-trip via name.
- Currently registers extractDateTask + summarizeTextTask;
module-side tasks land here as we add them.
apps/mana/apps/web/src/routes/(app)/+layout.svelte
- startLlmQueue() in handleAuthReady's Phase A (auth-independent)
so guests + authenticated users both get the queue
- stopLlmQueue() in onDestroy as a fire-and-forget cleanup
Processor loop semantics (the heart of the implementation):
1. On start(), reclaim any 'running' rows from a crashed previous
session — reset them to 'pending'. The orphan recovery is the
reason a crash mid-task doesn't leave the queue stuck.
2. findNextRunnable() picks the highest-priority pending task whose
`notBefore` (retry-backoff timestamp) is in the past. Sort key:
priority desc, then enqueuedAt asc (FIFO within priority).
3. Mark the task running, increment attempts, look up the LlmTask
in the registry, hand it to orchestrator.run().
4. On success: mark done, store result + source + finishedAt.
5. On error:
- TierTooLowError or ProviderBlockedError → fail immediately,
no retry. These are not transient — the user's settings or
the content itself need to change.
- Anything else → if attempts < maxAttempts, reset to pending
with notBefore = now + retryBackoffMs (default 60s). Else
mark failed.
6. When no work is pending, sleep on a Promise that resolves when
either (a) someone calls enqueue() (which fires notifyWakeup),
or (b) idleWakeupMs elapses (default 30s, safety net for any
missed wakeup signal).
Module-side reactive reads use Dexie liveQuery directly on the queue
table — no special subscription API on the queue itself. This is
consistent with how every other Mana module reads its data, so the
mental model stays uniform:
const tags = useLiveQuery(
() => llmQueueDb.tasks
.where({ refType: 'note', refId, taskName: 'common.extractTags' })
.reverse().first(),
[refId]
);
Smoke test: a new "Queue" tab in /llm-test lets you enqueue the
existing extractDate / summarize tasks and watch the live state of
the queue table via liveQuery. The display includes per-row state
badge (pending/running/done/failed), tier source, attempt count,
input/output, and a "Done/failed löschen" button that exercises
purge().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .changeset | ||
| .claude | ||
| .github | ||
| .husky | ||
| apps | ||
| docker | ||
| docs | ||
| games | ||
| load-tests | ||
| NewAppIdeas/Roblox Reimagined | ||
| packages | ||
| patches | ||
| scripts | ||
| services | ||
| tests | ||
| .dockerignore | ||
| .editorconfig | ||
| .env.development | ||
| .env.macmini.example | ||
| .env.secrets.example | ||
| .gitignore | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc.json | ||
| CLAUDE.md | ||
| cloudflared-config.yml | ||
| docker-compose.dev.yml | ||
| docker-compose.macmini.yml | ||
| docker-compose.test.yml | ||
| eslint.config.mjs | ||
| gift-codes-2026-02-14.txt | ||
| lint-staged.config.js | ||
| package.json | ||
| playwright.config.ts | ||
| pnpm-lock.yaml | ||
| pnpm-workspace.yaml | ||
| README.md | ||
| TROUBLESHOOTING.md | ||
| turbo.json | ||
| vitest.config.ts | ||
Mana Monorepo
Monorepo containing all Mana projects — a self-hosted multi-app ecosystem with shared packages and unified tooling.
Projects
| Project | Description | Apps |
|---|---|---|
| mana | Multi-app ecosystem platform | Expo mobile, SvelteKit web |
| chat | AI chat application | NestJS backend, Expo mobile, SvelteKit web, Astro landing |
| todo | Task management | NestJS backend, SvelteKit web, Astro landing |
| calendar | Calendar & scheduling | NestJS backend, SvelteKit web, Astro landing |
| clock | Pomodoro & time tracking | NestJS backend, SvelteKit web, Astro landing |
| contacts | Contact management | NestJS backend, SvelteKit web |
| picture | AI image generation | NestJS backend, Expo mobile, SvelteKit web, Astro landing |
| cards | Card/deck management | NestJS backend, Expo mobile, SvelteKit web |
| zitare | Daily inspiration quotes | NestJS backend, Expo mobile, SvelteKit web, Astro landing |
| mukke | Music player | NestJS backend, SvelteKit web |
| planta | Plant care tracker | NestJS backend, SvelteKit web |
| storage | Cloud storage | NestJS backend, SvelteKit web |
| questions | Q&A with web search | SvelteKit web |
| skilltree | Skill tree visualization | NestJS backend, SvelteKit web |
| nutriphi | Nutrition tracking | NestJS backend, SvelteKit web |
| citycorners | City guide | NestJS backend, SvelteKit web, Astro landing |
| presi | Presentation tool | NestJS backend, SvelteKit web |
| photos | Photo management | NestJS backend, SvelteKit web |
Getting Started
Prerequisites
- Node.js 20+
- pnpm 9.15.0+
- Docker (for PostgreSQL, Redis, MinIO)
Installation
pnpm install
Development
# Start infrastructure (PostgreSQL, Redis, MinIO)
pnpm docker:up
# Start any app with auto DB setup
pnpm dev:chat:full
pnpm dev:todo:full
pnpm dev:calendar:full
pnpm dev:contacts:full
# Build & quality
pnpm run build
pnpm run type-check
pnpm run format
See CLAUDE.md for comprehensive development documentation.
Architecture
mana-monorepo/
├── apps/ # Product applications
├── services/ # Microservices (auth, search, LLM, bots)
├── packages/ # Shared packages
├── docker/ # Docker configuration
└── scripts/ # Development & deployment scripts
Tooling
- Package Manager: pnpm 9.15.0
- Build System: Turborepo
- Formatting: Prettier (tabs, single quotes, 100 char width)
- Hosting: Mac Mini (self-hosted) via Docker + Cloudflare Tunnel
- Analytics: Umami (stats.mana.how)
License
Private - All rights reserved