mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-22 16:46:42 +02:00
feat(llm-aliases): M5 — migrate consumers to MANA_LLM aliases
Final milestone of docs/plans/llm-fallback-aliases.md. Every backend
caller now requests models via the `mana/<class>` alias system instead
of hardcoded `ollama/...` strings. mana-llm resolves aliases through
`services/mana-llm/aliases.yaml` with health-aware fallback (M3) and
emits resolved-model + fallback metrics (M4).
SSOT moved to `packages/shared-ai/src/llm-aliases.ts` so apps/api,
apps/mana/apps/web, and services/mana-ai all import the same
`MANA_LLM` constant via the existing `@mana/shared-ai` workspace
dependency. Three additional sites (memoro-server, mana-events,
mana-research) inline the alias string with a SSOT comment because
they don't pull @mana/shared-ai today.
Migrated 14 sites across 10 files:
- apps/api: writing(LONG_FORM), comic(STRUCTURED), context(FAST_TEXT),
food(VISION), plants(VISION), research orchestrator (3 tiers
collapsed to STRUCTURED+FAST_TEXT/LONG_FORM)
- apps/mana/apps/web: voice/parse-task + parse-habit (STRUCTURED)
- services/mana-ai: planner llm-client + tick.ts (REASONING)
- services/mana-events: website-extractor (STRUCTURED, inlined)
- services/mana-research: mana-llm client (FAST_TEXT, inlined)
- apps/memoro/apps/server: ai.ts (FAST_TEXT, inlined)
Legacy env-vars removed: WRITING_MODEL, COMIC_STORYBOARD_MODEL,
VISION_MODEL, MANA_LLM_DEFAULT_MODEL. The chain in aliases.yaml is
now the single tuning surface; SIGHUP reloads it without redeploys.
New `scripts/validate-llm-strings.mjs` regex-scans 2538 files for
hardcoded `<provider>/<model>` strings and fails the build if any
land outside the SSOT or the explicitly-allowed paths (image-gen
modules, model-inspector code, this validator itself, the registry).
Wired into `validate:all` next to the i18n + theme validators.
Verified: `pnpm validate:llm-strings` clean, `pnpm --filter @mana/api
type-check` clean, `pnpm --filter @mana/ai-service type-check`
clean. Web type-check has 2 pre-existing errors in
SettingsSidebar.svelte (i18n MessageFormatter type drift, last
touched in 988c17a67 — unrelated to this work).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
8a49e3ffd5
commit
fea3adf5fe
19 changed files with 299 additions and 50 deletions
|
|
@ -24,18 +24,18 @@
|
|||
|
||||
import { json } from '@sveltejs/kit';
|
||||
import { env } from '$env/dynamic/private';
|
||||
import { MANA_LLM } from '@mana/shared-ai';
|
||||
import type { RequestHandler } from './$types';
|
||||
|
||||
const MAX_TRANSCRIPT_CHARS = 500;
|
||||
const MAX_HABITS = 50;
|
||||
const LLM_TIMEOUT_MS = 8000;
|
||||
// gemma3:12b is more consistent than 4b at the "pick from this list,
|
||||
// don't paraphrase" instruction — 4b sometimes returns "Joggen" when
|
||||
// "Laufen" was in the list, which the verbatim-validation in coerce
|
||||
// then drops, costing an LLM round-trip for nothing. The accuracy
|
||||
// win matters more here than for parse-task because parse-habit only
|
||||
// runs at all when the cheap client-side substring fast path missed.
|
||||
const DEFAULT_MODEL = 'ollama/gemma3:12b';
|
||||
// Voice → JSON intent: STRUCTURED is the right class. mana-llm's
|
||||
// alias chain picks a model that consistently honours the
|
||||
// "verbatim from this list" constraint that parse-habit needs (the
|
||||
// coerce step still drops paraphrases, so accuracy here is direct
|
||||
// round-trip savings).
|
||||
const DEFAULT_MODEL = MANA_LLM.STRUCTURED;
|
||||
|
||||
interface ParseResult {
|
||||
match: string | null;
|
||||
|
|
|
|||
|
|
@ -19,20 +19,16 @@
|
|||
|
||||
import { json } from '@sveltejs/kit';
|
||||
import { env } from '$env/dynamic/private';
|
||||
import { MANA_LLM } from '@mana/shared-ai';
|
||||
import type { RequestHandler } from './$types';
|
||||
import { coerce, extractJson, fallback } from './coerce';
|
||||
|
||||
const MAX_TRANSCRIPT_CHARS = 1000;
|
||||
const LLM_TIMEOUT_MS = 8000;
|
||||
// gemma3:12b consistently nails relative date math ("nächsten Montag"
|
||||
// from a Wednesday → next Monday's date) and respects "null when
|
||||
// absent" for both dueDate and priority. gemma3:4b gets weekday math
|
||||
// off-by-one and stamps today's date on every bare task. The 12b
|
||||
// model is only ~10% slower in practice on the GPU box (~1.1s vs
|
||||
// ~1.0s for these tiny prompts) so the accuracy win is essentially
|
||||
// free. The deterministic guards in coerce() are still kept as a
|
||||
// safety net in case the GPU box swaps in a weaker model.
|
||||
const DEFAULT_MODEL = 'ollama/gemma3:12b';
|
||||
// Voice → JSON intent (relative dates, priority, title cleanup):
|
||||
// STRUCTURED. The deterministic guards in coerce() stay as a backstop
|
||||
// in case the alias chain falls back to a model with weaker date math.
|
||||
const DEFAULT_MODEL = MANA_LLM.STRUCTURED;
|
||||
|
||||
function buildPrompt(transcript: string, language: string): string {
|
||||
const now = new Date();
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue