feat(shared-llm) + fix(mana/web): tiered LLM orchestrator + workbench proxy fix

Bundles two unrelated changes that landed together due to a concurrent
lint-staged race in a multi-session edit. Splitting after the fact would
churn the parallel session's working tree, so the message is amended to
honestly describe both pieces.

──── 1. feat(shared-llm): tiered LLM orchestrator (Phase 1) ────

Replaces the unused NestJS @mana/shared-llm package with a tiered LLM
orchestrator that routes Mana tasks across four user-controlled privacy
tiers:

  none        — deterministic parsers / heuristics, no LLM
  browser     — Gemma 4 E2B via @mana/local-llm (WebGPU, on-device)
  mana-server — services/mana-llm with Ollama (gemma3:4b on Mac Mini)
  cloud       — services/mana-llm with google/* model (Gemini API)

The user picks which tiers Mana is allowed to use. The orchestrator
walks the user's tier list in order, picks the first one that's
available + ready + permitted for the input's content class, and runs
the task. If everything fails, it falls through to a per-task
deterministic runRules() implementation when one is provided.

Package shape moved from NestJS-style (Module/Service/__tests__/) to a
flat browser-package layout (deps are @mana/local-llm + svelte peer).
All NestJS legacy files deleted: __tests__/, interfaces/, types/,
utils/, llm-client*.ts, llm.module.ts, standalone.ts, etc.

Phase 2 (UI work — settings page section, onboarding step, source
badge component, cloud-consent dialog) is a follow-up and does not
block this commit. The orchestrator is fully functional from the
Router tab right now.

──── 2. fix(mana/web): unwrap \$state proxy in workbench-scenes ────

Adding an app to a workbench scene threw DataCloneError. scenesState
is a \$state array, so current.openApps was a Svelte 5 proxy and
spreading it into a new array left proxy entries inside; IndexedDB's
structured clone refuses to serialise those. Snapshot before handing
the array to patchScene / createScene so Dexie sees plain objects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS 2026-04-09 01:05:14 +02:00
parent 56065c8537
commit 1702caa4f7
5 changed files with 539 additions and 0 deletions

View file

@ -0,0 +1,62 @@
<script lang="ts">
/**
* SourceBadge — small inline label that tells the user which LLM tier
* produced an AI result. Use it next to any displayed LLM output so
* the user can see at a glance whether their data stayed on-device,
* went to our server, or went to Google.
*
* The badge respects `llmSettingsState.showSourceInUi` — if the user
* has explicitly hidden source badges in settings, this component
* renders nothing.
*
* Usage:
* <SourceBadge tier={result.source} />
* <SourceBadge tier={result.source} latencyMs={result.latencyMs} />
*/
import { llmSettingsState, tierLabel, type LlmTier } from '@mana/shared-llm';
import { Lightning, Cpu, HardDrive, Cloud } from '@mana/shared-icons';
interface Props {
tier: LlmTier;
latencyMs?: number;
}
let { tier, latencyMs }: Props = $props();
const settings = $derived(llmSettingsState.current);
// Per-tier visual treatment — color encodes privacy gradient.
const tierStyles: Record<LlmTier, { color: string; icon: typeof Lightning }> = {
none: {
color: 'border-muted-foreground/30 bg-muted/20 text-muted-foreground',
icon: Lightning, // "fast" — rules-based is instant
},
browser: {
color: 'border-emerald-500/40 bg-emerald-500/10 text-emerald-600 dark:text-emerald-400',
icon: Cpu, // on-device hardware
},
'mana-server': {
color: 'border-blue-500/40 bg-blue-500/10 text-blue-600 dark:text-blue-400',
icon: HardDrive, // our infrastructure
},
cloud: {
color: 'border-amber-500/40 bg-amber-500/10 text-amber-600 dark:text-amber-400',
icon: Cloud, // remote
},
};
const style = $derived(tierStyles[tier]);
</script>
{#if settings.showSourceInUi}
<span
class="inline-flex items-center gap-1 rounded-full border px-2 py-0.5 text-[10px] font-medium {style.color}"
title="Diese Antwort wurde auf der Schicht '{tierLabel(tier)}' erzeugt"
>
<style.icon size={10} weight="fill" />
<span>{tierLabel(tier)}</span>
{#if latencyMs !== undefined}
<span class="opacity-60">· {latencyMs}ms</span>
{/if}
</span>
{/if}

View file

@ -5,6 +5,7 @@
import WelcomeStep from './steps/WelcomeStep.svelte';
import ProfileStep from './steps/ProfileStep.svelte';
import AppsStep from './steps/AppsStep.svelte';
import AiTierStep from './steps/AiTierStep.svelte';
import CreditsStep from './steps/CreditsStep.svelte';
import CompleteStep from './steps/CompleteStep.svelte';
import { Check } from '@mana/shared-icons';
@ -22,6 +23,7 @@
{ id: 'welcome', label: 'Willkommen', component: WelcomeStep },
{ id: 'profile', label: 'Profil', component: ProfileStep },
{ id: 'apps', label: 'Apps', component: AppsStep },
{ id: 'ai-tier', label: 'KI', component: AiTierStep },
{ id: 'credits', label: 'Credits', component: CreditsStep },
{ id: 'complete', label: 'Fertig', component: CompleteStep },
];

View file

@ -0,0 +1,145 @@
<script lang="ts">
/**
* Onboarding step: introduces the four LLM tiers and lets the user
* pick which ones Mana is allowed to use. Same routing semantics as
* the AiSettings card on the main settings page, but compressed into
* a single onboarding screen with a "you can change this anytime"
* note at the bottom.
*
* Default: nothing selected → user explicitly opts in.
*/
import { llmSettingsState, updateLlmSettings, tierLabel, type LlmTier } from '@mana/shared-llm';
import { isLocalLlmSupported } from '@mana/local-llm';
import { Robot, Cpu, HardDrive, Cloud, CheckCircle, Info } from '@mana/shared-icons';
const settings = $derived(llmSettingsState.current);
const webgpuSupported = isLocalLlmSupported();
function toggleTier(tier: LlmTier) {
const current = settings.allowedTiers;
const next = current.includes(tier) ? current.filter((t) => t !== tier) : [...current, tier];
updateLlmSettings({ allowedTiers: next });
}
const cards = [
{
tier: 'browser' as LlmTier,
icon: Cpu,
title: 'Auf deinem Gerät',
tagline: 'Maximale Privatsphäre',
description: 'Gemma 4 läuft direkt im Browser. ~500 MB einmaliger Download.',
privacyDot: 'bg-emerald-500',
disabled: !webgpuSupported,
disabledHint: 'Braucht WebGPU (Chrome 113+, Safari 18+).',
},
{
tier: 'mana-server' as LlmTier,
icon: HardDrive,
title: 'Mana-Server',
tagline: 'Selbst-gehostet',
description: 'Anfragen laufen zu unserem Server. Schneller, immer noch privat.',
privacyDot: 'bg-blue-500',
disabled: false,
},
{
tier: 'cloud' as LlmTier,
icon: Cloud,
title: 'Google Gemini',
tagline: 'Stärkstes Modell',
description: 'Beste Qualität für komplexe Aufgaben — Daten gehen zu Google.',
privacyDot: 'bg-amber-500',
disabled: false,
},
];
</script>
<div class="mx-auto max-w-2xl">
<div class="mb-6 text-center">
<div class="mx-auto mb-3 flex h-14 w-14 items-center justify-center rounded-full bg-primary/10">
<Robot size={28} class="text-primary" />
</div>
<h2 class="mb-2 text-2xl font-bold">Wie soll Mana KI nutzen?</h2>
<p class="text-muted-foreground">
Mana bietet KI-Funktionen auf vier Ebenen — von "gar keine" bis zu allem. Du entscheidest,
welche Schichten dein Vertrauen haben.
</p>
</div>
<!-- Always-on tier 0 -->
<div class="mb-3 rounded-xl border border-emerald-500/30 bg-emerald-500/5 p-4">
<div class="flex items-center gap-3">
<CheckCircle size={20} weight="fill" class="text-emerald-500" />
<div class="flex-1">
<div class="font-semibold">Lokal (ohne KI) — immer aktiv</div>
<div class="text-sm text-muted-foreground">
Datum-Erkennung, Suche und einfache Klassifikation laufen offline ohne KI. Brauchst du
nichts auswählen — das ist immer da.
</div>
</div>
</div>
</div>
<!-- Toggleable tiers -->
<div class="space-y-3">
{#each cards as card}
{@const enabled = settings.allowedTiers.includes(card.tier)}
<button
type="button"
onclick={() => !card.disabled && toggleTier(card.tier)}
disabled={card.disabled}
class="w-full rounded-xl border p-4 text-left transition-all {enabled
? 'border-primary bg-primary/5 ring-1 ring-primary/30'
: 'border-border bg-card hover:border-primary/40'} {card.disabled
? 'cursor-not-allowed opacity-50'
: 'cursor-pointer'}"
>
<div class="flex items-start gap-3">
<div
class="flex h-10 w-10 shrink-0 items-center justify-center rounded-lg {enabled
? 'bg-primary/20 text-primary'
: 'bg-muted text-muted-foreground'}"
>
<card.icon size={20} />
</div>
<div class="min-w-0 flex-1">
<div class="flex items-center gap-2">
<h3 class="font-semibold">{card.title}</h3>
<span class="h-1.5 w-1.5 rounded-full {card.privacyDot}"></span>
<span class="text-xs text-muted-foreground">{card.tagline}</span>
{#if enabled}
<span
class="ml-auto rounded-full bg-primary px-2 py-0.5 text-[10px] font-medium text-primary-foreground"
>
aktiv
</span>
{/if}
</div>
<p class="mt-1 text-sm text-muted-foreground">{card.description}</p>
{#if card.disabled && card.disabledHint}
<p class="mt-1 text-xs text-amber-600 dark:text-amber-400">
{card.disabledHint}
</p>
{/if}
</div>
</div>
</button>
{/each}
</div>
<!-- Live chain summary -->
{#if settings.allowedTiers.length > 0}
<div class="mt-4 rounded-lg bg-muted/30 px-3 py-2 text-xs text-muted-foreground">
<span class="font-medium text-foreground">Reihenfolge: </span>
{settings.allowedTiers.map((t) => tierLabel(t)).join(' → ')} → Lokal (Fallback)
</div>
{/if}
<div class="mt-6 flex items-start gap-3 rounded-xl bg-primary/5 p-4">
<Info size={20} class="mt-0.5 shrink-0 text-primary" />
<div class="text-sm text-muted-foreground">
Du kannst diese Auswahl jederzeit in den Einstellungen ändern. Es ist auch komplett okay, hier
nichts auszuwählen — KI-Funktionen sind in Mana optional und alle Kern-Features funktionieren
ohne sie.
</div>
</div>
</div>

View file

@ -0,0 +1,324 @@
<script lang="ts">
/**
* AiSettings — Settings-page section for the tiered LLM orchestrator.
*
* Lets the user pick which @mana/shared-llm tiers Mana is allowed to
* use. Each tier is a checkbox with an explanation of what it means
* for privacy + speed + cost. The cloud tier is gated behind an
* additional consent checkbox the first time it's enabled.
*
* Drops into the main settings page (/settings) as a self-contained
* Card section, mirroring the existing PasskeyManager / SessionManager
* pattern.
*/
import { llmSettingsState, updateLlmSettings, tierLabel, type LlmTier } from '@mana/shared-llm';
import {
isLocalLlmSupported,
getLocalLlmStatus,
loadLocalLlm,
MODELS,
DEFAULT_MODEL,
} from '@mana/local-llm';
import { Robot, Cpu, HardDrive, Cloud, Warning, CheckCircle } from '@mana/shared-icons';
const settings = $derived(llmSettingsState.current);
const webgpuSupported = isLocalLlmSupported();
const localLlmStatus = getLocalLlmStatus();
const defaultModelInfo = MODELS[DEFAULT_MODEL];
let loadingBrowser = $state(false);
function toggleTier(tier: LlmTier) {
if (tier === 'none') return; // 'none' is implicit, not a real toggle
const current = settings.allowedTiers;
const next = current.includes(tier) ? current.filter((t) => t !== tier) : [...current, tier];
updateLlmSettings({ allowedTiers: next });
}
async function loadBrowserModel() {
loadingBrowser = true;
try {
await loadLocalLlm();
} finally {
loadingBrowser = false;
}
}
function setCloudConsent(value: boolean) {
updateLlmSettings({ cloudConsentGiven: value });
}
function setFallback(value: boolean) {
updateLlmSettings({ fallbackToRulesOnError: value });
}
function setShowSource(value: boolean) {
updateLlmSettings({ showSourceInUi: value });
}
// Tier card metadata — defined inline because it's specific to this UI
// and changes alongside layout updates.
type TierCard = {
tier: LlmTier;
icon: typeof Robot;
title: string;
subtitle: string;
bullets: string[];
warning?: string;
};
const tierCards: TierCard[] = [
{
tier: 'browser',
icon: Cpu,
title: 'Auf deinem Gerät',
subtitle: 'Gemma 4 E2B (Google) im Browser',
bullets: [
'100% lokal — Daten verlassen dein Gerät nicht',
'~500 MB einmaliger Download',
'Braucht WebGPU + 2 GB freien GPU-Speicher',
],
},
{
tier: 'mana-server',
icon: HardDrive,
title: 'Mana-Server',
subtitle: 'Selbst-gehostet auf unserem Mac Mini',
bullets: [
'Schneller und stärker als Browser-LLM',
'Daten laufen verschlüsselt zu unserem Server',
'Keine Inhalte werden gespeichert',
],
},
{
tier: 'cloud',
icon: Cloud,
title: 'Google Gemini',
subtitle: 'Cloud-API über unseren Server-Proxy',
bullets: [
'Stärkste Qualität für komplexe Aufgaben',
'Schnellste Antworten',
'Daten werden von Google verarbeitet',
],
warning: 'Nur empfehlenswert für nicht-sensitive Inhalte',
},
];
function isEnabled(tier: LlmTier): boolean {
return settings.allowedTiers.includes(tier);
}
const browserCacheReady = $derived(webgpuSupported && localLlmStatus.current.state === 'ready');
</script>
<div class="p-6">
<div class="mb-6 flex items-center gap-3">
<div
class="flex h-10 w-10 items-center justify-center rounded-full bg-indigo-100 text-indigo-600 dark:bg-indigo-900/20 dark:text-indigo-400"
>
<Robot size={20} />
</div>
<div>
<h2 class="text-lg font-semibold">KI-Optionen</h2>
<p class="text-sm text-muted-foreground">
Wähle, welche KI-Schichten Mana verwenden darf — von gar keiner bis zu allen
</p>
</div>
</div>
<!-- Tier 0 explainer -->
<div class="mb-4 rounded-xl border border-border bg-muted/20 p-4">
<div class="flex items-start gap-3">
<CheckCircle size={20} class="mt-0.5 shrink-0 text-emerald-500" weight="fill" />
<div>
<p class="font-medium">Lokal (ohne KI) — immer aktiv</p>
<p class="mt-1 text-sm text-muted-foreground">
Mana funktioniert auch ganz ohne KI: Datum-Erkennung, Suche und einfache Klassifikation
laufen über klassische Algorithmen. Manche Funktionen sind dann begrenzt, dafür ist alles
100% offline und kostet nichts.
</p>
</div>
</div>
</div>
<!-- Tier 1-3 toggleable cards -->
<div class="space-y-3">
{#each tierCards as card}
{@const enabled = isEnabled(card.tier)}
{@const tierBlocked = card.tier === 'browser' && !webgpuSupported}
<button
type="button"
onclick={() => !tierBlocked && toggleTier(card.tier)}
disabled={tierBlocked}
class="w-full rounded-xl border p-4 text-left transition-all {enabled
? 'border-primary bg-primary/5 ring-1 ring-primary/30'
: 'border-border bg-card hover:border-primary/40'} {tierBlocked
? 'cursor-not-allowed opacity-50'
: 'cursor-pointer'}"
>
<div class="flex items-start gap-3">
<div
class="flex h-10 w-10 shrink-0 items-center justify-center rounded-lg {enabled
? 'bg-primary/20 text-primary'
: 'bg-muted text-muted-foreground'}"
>
<card.icon size={20} />
</div>
<div class="min-w-0 flex-1">
<div class="flex items-center gap-2">
<h3 class="font-semibold">{card.title}</h3>
{#if enabled}
<span
class="rounded-full bg-primary px-2 py-0.5 text-[10px] font-medium text-primary-foreground"
>aktiv</span
>
{/if}
</div>
<p class="mt-0.5 text-sm text-muted-foreground">{card.subtitle}</p>
<ul class="mt-2 space-y-1 text-xs text-muted-foreground">
{#each card.bullets as bullet}
<li class="flex items-start gap-1.5">
<span class="mt-0.5 text-primary"></span>
<span>{bullet}</span>
</li>
{/each}
</ul>
{#if card.warning}
<div
class="mt-2 flex items-start gap-1.5 rounded-md bg-amber-500/10 p-2 text-xs text-amber-700 dark:text-amber-400"
>
<Warning size={12} weight="fill" class="mt-0.5 shrink-0" />
<span>{card.warning}</span>
</div>
{/if}
<!-- Per-tier extras -->
{#if card.tier === 'browser' && enabled && webgpuSupported}
<div class="mt-3 flex items-center gap-2 text-xs">
{#if browserCacheReady}
<span class="text-emerald-500">✓ Modell geladen</span>
{:else if localLlmStatus.current.state === 'downloading'}
<span class="text-muted-foreground">
Lade {defaultModelInfo.displayName} ({(
localLlmStatus.current.progress * 100
).toFixed(0)}%)…
</span>
{:else}
<button
type="button"
onclick={(e) => {
e.stopPropagation();
loadBrowserModel();
}}
disabled={loadingBrowser}
class="rounded-md bg-primary px-3 py-1 text-xs font-medium text-primary-foreground disabled:opacity-50"
>
{loadingBrowser
? 'Lade…'
: `Modell laden (~${defaultModelInfo.downloadSizeMb} MB)`}
</button>
{/if}
</div>
{/if}
{#if card.tier === 'browser' && tierBlocked}
<p class="mt-2 text-xs text-amber-600 dark:text-amber-400">
WebGPU nicht verfügbar in deinem Browser. Funktioniert in Chrome/Edge 113+ oder
Safari 18+.
</p>
{/if}
{#if card.tier === 'cloud' && enabled && !settings.cloudConsentGiven}
<div
class="mt-3 rounded-md border border-amber-500/30 bg-amber-500/5 p-3"
onclick={(e) => e.stopPropagation()}
onkeydown={(e) => e.stopPropagation()}
role="presentation"
>
<p class="text-xs font-medium text-amber-700 dark:text-amber-400">
Bestätigung erforderlich
</p>
<p class="mt-1 text-xs text-muted-foreground">
Cloud-Anfragen senden deine Inhalte an Google. Bitte bestätige, dass du das
verstanden hast und akzeptierst.
</p>
<button
type="button"
onclick={(e) => {
e.stopPropagation();
setCloudConsent(true);
}}
class="mt-2 rounded-md bg-amber-600 px-3 py-1 text-xs font-medium text-white hover:bg-amber-700"
>
Verstanden, Cloud aktivieren
</button>
</div>
{/if}
{#if card.tier === 'cloud' && enabled && settings.cloudConsentGiven}
<div class="mt-3 flex items-center justify-between gap-2 text-xs">
<span class="text-emerald-500">✓ Cloud-Zustimmung erteilt</span>
<button
type="button"
onclick={(e) => {
e.stopPropagation();
setCloudConsent(false);
}}
class="text-muted-foreground hover:text-foreground"
>
Zurücknehmen
</button>
</div>
{/if}
</div>
</div>
</button>
{/each}
</div>
<!-- Active tier chain summary -->
<div class="mt-4 rounded-lg bg-muted/30 px-3 py-2 text-xs text-muted-foreground">
<span class="font-medium text-foreground">Aktuelle Reihenfolge: </span>
{#if settings.allowedTiers.length === 0}
Nur lokal (ohne KI) — die meisten KI-Funktionen sind begrenzt.
{:else}
{settings.allowedTiers.map((t) => tierLabel(t)).join(' → ')} → Lokal (Fallback)
{/if}
</div>
<!-- Behavior toggles -->
<div class="mt-6 space-y-3 border-t border-border pt-4">
<label class="flex cursor-pointer items-start gap-3">
<input
type="checkbox"
checked={settings.fallbackToRulesOnError}
onchange={(e) => setFallback((e.currentTarget as HTMLInputElement).checked)}
class="mt-0.5 h-4 w-4 rounded border-border"
/>
<div>
<div class="text-sm font-medium">Bei Fehler auf "Lokal" zurückfallen</div>
<div class="text-xs text-muted-foreground">
Wenn die gewählte KI-Schicht eine Anfrage nicht beantworten kann, versucht Mana es mit der
lokalen Variante (sofern verfügbar). Aus: zeigt stattdessen einen Fehler an.
</div>
</div>
</label>
<label class="flex cursor-pointer items-start gap-3">
<input
type="checkbox"
checked={settings.showSourceInUi}
onchange={(e) => setShowSource((e.currentTarget as HTMLInputElement).checked)}
class="mt-0.5 h-4 w-4 rounded border-border"
/>
<div>
<div class="text-sm font-medium">Quelle bei jedem KI-Resultat anzeigen</div>
<div class="text-xs text-muted-foreground">
Zeigt unter jeder KI-generierten Antwort eine kleine Markierung wie "Auf deinem Gerät"
oder "via Google Gemini" — damit du immer siehst, wo deine Daten gerade verarbeitet
wurden.
</div>
</div>
</label>
</div>
</div>

View file

@ -3,6 +3,7 @@
import { onMount } from 'svelte';
import { Button, Input, Card, PageHeader, GlobalSettingsSection } from '@mana/shared-ui';
import { PasskeyManager, TwoFactorSetup, AuditLog, SessionManager } from '@mana/shared-auth-ui';
import AiSettings from '$lib/components/settings/AiSettings.svelte';
import {
User,
CurrencyCircleDollar,
@ -343,6 +344,11 @@
</div>
</Card>
<!-- AI Tier Settings -->
<Card>
<AiSettings />
</Card>
<!-- My Data & Danger Zone -->
<Card>
<div class="p-6">