Commit graph

74 commits

Author SHA1 Message Date
Till JS
546b94d472 feat(personas): move admin + internal endpoints from mana-auth to apps/api
Schließt die platform/product-split-Lücke: HEAD's apps/api/src/index.ts
referenziert seit dem Forms-M10d-Commit personasInternalRoutes /
personasAdminRoutes — die Implementierung lag aber noch nicht im Repo.
Build war strukturell broken bis hierhin.

Was wandert von mana-auth nach apps/api:

  apps/api/src/modules/personas/
    ├── schema.ts          — pgSchema('personas') mit personas /
    │                        persona_actions / persona_feedback;
    │                        userId ist plain text (Cross-DB-FK auf
    │                        mana-auth's auth.users geht nach Split nicht).
    ├── internal-routes.ts — service-key gated GET /due, POST /:id/actions
    │                        und POST /:id/feedback. Append-only +
    │                        idempotent über deterministische row-ids
    │                        (tickId-i-tool / tickId-module).
    └── admin-routes.ts    — admin-JWT gated CRUD; ruft mana-auth via
                             /api/v1/admin/users + /api/v1/auth/register
                             + /api/v1/internal/users/:id/persona-stamp
                             für den User-Lifecycle.

Persona-runner-Client zeigt jetzt auf apps/api:

  - config.ts: neues apiUrl-Feld (default http://localhost:3060,
    Env MANA_API_URL); authUrl bleibt für /api/v1/auth/login + spaces.
  - clients/mana-auth-internal.ts: drei Calls treffen jetzt
    /api/v1/personas/internal/* statt mana-auth's
    /api/v1/internal/personas/* — Datei-Name bleibt um Call-Site-Diff
    klein zu halten.
  - index.ts: ManaAuthInternalClient bekommt config.apiUrl statt authUrl.

Seed/Cleanup-Skripte:

  - --api= als bevorzugter Flag, --auth= als Legacy-Alias (cached
    Shell-History würde sonst hart brechen).
  - default http://localhost:3060, Env MANA_API_URL.
  - Endpoint-Pfade umgeschrieben:
      POST   /api/v1/admin/personas        → /api/v1/personas/admin
      DELETE /api/v1/admin/personas/:id    → /api/v1/personas/admin/:id

drizzle.config.ts: schema-Array + schemaFilter um 'personas' erweitert.
DB-push ist Pflicht-Schritt vor erstem Boot, sonst 42P01 auf /due.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 20:38:29 +02:00
Till JS
795b39e065 feat(forms): M10d headless wave-cron — server-worker + private internal_meta
Echter Server-Cron für recurring forms — wave-send läuft jetzt
unabhängig von Owner-Tab-State. Bisheriger M10c webapp-side scheduler
bleibt als Belt-and-suspenders aktiv (idempotent).

Architektur:
1. **Owner-private internal_meta auf unlisted snapshots**
   - Drizzle: neue jsonb-column `internal_meta` (Drizzle migration
     0001_internal_meta.sql).
   - public-routes.ts strippt sie strukturell — die explicit select()-
     projection enthält sie nicht (recipients + sender würden sonst
     via share-link leaken).
   - publish-route akzeptiert sie im Body, persistiert auf insert +
     update.
   - ALLOWED_COLLECTIONS um 'lasts' und 'forms' erweitert (war ein
     latenter Bug — formsStore.setVisibility('unlisted') hätte ohne
     diese Ergänzung 400 zurückbekommen; M4b lief vermutlich nie
     end-to-end durch).

2. **shared-privacy publishUnlistedSnapshot**
   - PublishUnlistedOptions erweitert um optionales `internalMeta`.
     Forwarded an /api/v1/unlisted/:collection/:recordId body.

3. **Webapp formsStore**
   - lib/wave-mail.ts: buildFormInternalMeta(form, broadcastSettings)
     baut den Owner-Private-Blob: { kind, recurrence: {frequency,
     recipientEmails, lastSentAt}, sender: {fromEmail, fromName,
     replyTo, legalAddress}, formMeta: {title, description} }.
     Returns null wenn Voraussetzungen fehlen (kein recurrence, keine
     recipients, fehlende broadcast-settings).
   - stores/forms.svelte.ts: setVisibility / regenerateUnlistedToken /
     setUnlistedExpiry laden broadcastSettings via Dexie + decrypt,
     bauen internalMeta, übergeben an publishUnlistedSnapshot. Form
     wird vor dem buildFormInternalMeta-Call dekrypted.

4. **mana-mail internal bulk-send route**
   - createInternalRoutes(accountService, broadcastOrchestrator,
     maxRecipients) — Signature erweitert.
   - Neue POST /api/v1/internal/mail/bulk-send: gleicher Payload-shape
     wie user-facing /v1/mail/bulk-send aber userId aus Body statt
     JWT. X-Service-Key-gate sitzt bei /api/v1/internal/* prefix.
     Audit-trail trägt principalId aus Body. Cap = 5000 (gleicher
     Wert wie user-facing).

5. **apps/api forms wave-worker**
   - 5-min setInterval, advisory-lock-gated (key 0x464f5257 'FORW').
   - Tick: select snapshots WHERE collection='forms' AND
     internal_meta IS NOT NULL AND revoked_at IS NULL. Filter auf
     kind='forms-recurrence' + isWaveDue (lastSentAt + period <= now,
     never-sent fires sofort). Pro fälligem snapshot: build HTML/text
     mailbody (mirror webapp wave-mail-render), POST an mana-mail
     internal-bulk-send mit X-Service-Key + userId, dann jsonb_set
     auf internal_meta.recurrence.lastSentAt. Per-snapshot errors
     werden als console.warn geloggt, Tick läuft weiter.
   - Disable via FORMS_WAVE_WORKER_DISABLED=true (tests / multi-
     replica deployments).
   - Wired in apps/api/src/index.ts neben startArticleImportWorker().

Trade-offs:
- internal_meta wird beim setVisibility/regenerate/setExpiry frisch
  aus broadcast-settings gebaut — wenn der User später broadcast-
  settings ändert (zB neuer fromEmail) muss er das Form re-publishen
  damit die snapshot-internal_meta aktualisiert wird. Doc-it: zukünftiger
  Patch könnte ein "settings drift"-Warning ins UI surfacen.
- Worker-Update von lastSentAt geht NICHT zurück in den webapp-form
  (settings.recurrence.lastSentAt ist verschlüsselt, server kann
  nicht schreiben). Owner-UI zeigt ältere lastSentAt von manuellen
  Sends; auto-cron-sends sind in den Server-Logs sichtbar. Future
  patch: GET /api/v1/forms/:id/recurrence-status (auth) gibt das
  snapshot.internal_meta zurück, UI rendert Auto-Cron-State.
- Webapp-side wave-scheduler (M10c) läuft parallel weiter — wenn
  Owner-Tab offen ist, kann beides feuern. Idempotent durch
  lastSentAt-check (weekly/monthly buckets), aber theoretisch könnte
  double-fire passieren wenn die Calls innerhalb 1ms versetzt sind.
  Real-world ignorierbar; future patch: scheduler liest jetzt
  internal_meta.lastSentAt vom server-side state.

apps/api buildet (1776 modules). mana-mail buildet (523 modules).
svelte-check 0 errors in forms/. Forms-Tests 70/70 unverändert.

DB-Migration 0001_internal_meta.sql muss manuell appliziert werden
(siehe feedback memory: hand-authored SQL migrations sind nicht in
pnpm setup:db).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 17:18:05 +02:00
Till JS
6d67db48d5 feat(forms): M9b conversation LLM-extract — free-text → typed Antwort
Killer-Feature für den Conversation-Mode (M9): User kann auf
choice/yes_no/rating-Feldern in eigenen Worten antworten ("ich nehme
den zweiten Vorschlag" / "klar bin ich dabei" / "so 4 von 5"), ein
LLM mappt das auf die strikte Option-ID / boolean / Integer.

- apps/api/modules/forms/public-routes.ts: neuer
  POST /api/v1/forms/public/:token/conversation/extract Endpoint.
  Rate-limited (30/min/token + 60/min/IP — Owner-Side-Costs für haiku
  trotz unauthenticated-Pfad). freeText hard-cap 1000 Zeichen.
  Token-resolve via unlistedSnapshots, fieldId muss im publish-Schema
  existieren. Dispatch:
    - text/email/number/date: passthrough (free-text IST die Antwort)
    - single_choice/multi_choice/yes_no/rating: mana-llm haiku-Call
      mit field-spezifischem System-Prompt + JSON-only-Output, Parser
      validiert Option-IDs gegen das Schema (Hallucination-Schutz).
  Response { extracted, confidence: 'high' | 'low', alternatives? }.
  confidence='low' wenn LLM unsicher → Client zeigt Warnung im
  Preview-Block, User kann manuell auswählen.

- ConversationFormView: collapsible <details>"Lieber in eigenen
  Worten antworten?"-Block unter den quick-reply-Buttons aller
  choice/yes_no/rating-Felder. User tippt Free-Text → "Verstehen"
  ruft endpoint → Preview-Karte mit der erkannten Antwort
  (teal=high-confidence, amber=low-confidence) → "Übernehmen" oder
  "Abbrechen". commitExtract löst setAnswerAndAdvance aus, läuft
  über den selben Pfad wie quick-reply-Klick.

Schema-Validierung im Parser:
  - single_choice: optionId muss in field.options sein, sonst null
  - multi_choice: filtert nur valide IDs raus, Array kann leer sein
  - yes_no: nur true/false/null erlaubt
  - rating: round(value), bounds-check 1..ratingScale

LLM-Call:
  - model claude-haiku-4-5 (cheapest)
  - temperature 0 (deterministisch)
  - maxTokens 200 (JSON-Output ist klein)
  - Markdown-code-fence-Strip für robustes JSON-Parsing

Trade-offs:
  - Public-Endpoint = ungated LLM-Spend für Form-Owner. Rate-Limits
    + freeText-Cap mitigaten Spam, aber 30 Calls/min × 200 tokens =
    moderate Kosten pro Form. Owner sollte das im Hinterkopf haben.
  - Confidence='low' eskaliert zur User-Sichtbarkeit, bricht aber
    nicht den Flow — User kann übernehmen oder abbrechen.

Forms-Tests 61/61 unverändert (extract braucht Live-LLM für E2E,
absichtlich kein vitest-Mock). svelte-check 0 errors. apps/api
buildet (1772 modules).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 15:31:25 +02:00
Till JS
38e0ae2ff8 feat(forms): M10a wiederkehrende Forms — cohort-tagging + UI
Plan: docs/plans/forms-module.md M10. Erste Hälfte (Datenshape +
Submit-Stamping + ResponsesView-Filter); cron-basierter
Broadcast-Versand bleibt M10b.

- types.ts:
  - FormSettings.recurrence?: { frequency: 'weekly'|'monthly',
    startedAt?, sendVia? }. Im Settings-Blob mitverschlüsselt.
  - LocalFormResponse.cohort?: string. Plaintext (sortier-/filter-
    Feld, kein PII). FormResponse.cohort: string|null.
  - queries.toFormResponse maps null durch.
- lib/cohort.ts (pure):
  - computeCohort(iso, frequency) → "YYYY-WNN" (ISO-Woche, padded für
    lex-sort) | "YYYY-MM". Behandelt Year-Boundary korrekt
    (2027-01-01 → 2026-W53).
  - cohortLabel(cohort, frequency, now=new Date()) → "Diese Woche" /
    "Letzte Woche" / "KW NN / YYYY" für weekly; "Dieser Monat" /
    "Letzter Monat" / "Monatsname YYYY" für monthly. Falsche Keys
    fallen auf Roh-Wert zurück (inkl. Bounds-Check 1-12 für month).
  - sortCohortsDesc(): immutable, lex-sort newest-first.
  - 15/15 Vitest-Cases (compute, label-current/prev/older, malformed,
    sort, no-mutation).
- buildFormBlob (data/unlisted/resolvers.ts) zieht recurrence.frequency
  in den Public-Snapshot. Frequenz ist nicht-sensitive Metadaten —
  kann öffentlich diskoverabel sein, der Server braucht es zum
  cohort-stamping.
- apps/api/src/modules/forms/public-routes.ts:
  - Inline computeCohort (Mirror der pure-Funktion — apps/api kann
    apps/mana nicht importieren).
  - Bei jeder Submission: wenn snapshot.recurrence.frequency gesetzt,
    stamp data.cohort = computeCohort(submittedAt, freq). Cohort
    landet im sync_changes-Insert und reist zum Owner-Client.
  - errorResponse-Aufrufe: { code, field } → { code, details: { field } }
    weil der Hono-helper die details-Struktur erzwingt (M3b
    type-safety-fix mitgenommen).
- SettingsPanel: Recurrence-Block mit Dropdown (Einmalig / Wöchentlich
  / Monatlich) + Hint-Erklärung. Setzt startedAt automatisch beim
  Aktivieren.
- ResponsesView: cohort-chip-bar oberhalb der Status-Tabs. Zeigt
  "Alle Wellen" + ein Chip pro distincter cohort (newest-first), je
  mit Anzahl. Klick filtert die responses-Liste — alle bestehenden
  Status-Counts berechnen sich auf den cohort-gefilterten Set, also
  CSV-Export + Detail-Modal-Status-Pills bleiben konsistent.
- 8 neue i18n-Keys × 5 Locales (forms.builder.recurrence.* +
  forms.responses.cohort.all). i18n-parity 6483 keys aligned.

Tests: 41/41 forms-tests grün (5 csv + 11 branching + 10 auto-sync +
15 cohort). svelte-check 0 errors. apps/api buildet.

M10b open: cron-worker in mana-ai oder mana-notify, der
recurrence-Forms scant + Share-Link via broadcasts an die
Recipients-Liste sendet. Schemata stehen, Pipeline fehlt noch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 14:09:55 +02:00
Till JS
cb9d79d2c9 feat(articles): finale polish — Help-Eintrag + intentional-console + 5-Locale i18n (#16,#10,#17)
#16 — Help-content entry for articles
   The articles module had no entry in `app-registry/help-content.ts`
   so the (?) icon in ModuleShell rendered an empty body. Added
   description + 9 features + 4 tips covering Reader-View, Highlights,
   Bookmarklet, Share-Target, and the new bulk-import flow with the
   "Erneut speichern" rescue path for cookie-walled hits.

#10 — Console statements marked intentional
   The 5 console.log/warn/error calls in import-worker.ts (boot,
   tick errors, GC summary, stale-recovery sweep) were ESLint
   warnings. They're intentional operational logs — same pattern as
   services/mana-ai/src/cron/tick.ts. Added file-level
   `/* eslint-disable no-console */` with a comment explaining the
   pattern + that structured signal lives in Prometheus counters.

#17 — Full 5-locale i18n for the bulk-import UI
   New `articles.import` namespace with 50 keys covering the
   BulkImportForm, JobsList, JobDetailView, and AddUrlForm bulk-link.
   All five locales translated by hand:
     - de.json (canonical, mirrors the original hardcoded German)
     - en.json (English)
     - fr.json (French — bookmarklet → "bookmarklet HTML du navigateur")
     - it.json (Italian — bookmarklet → "bookmarklet HTML del browser")
     - es.json (Spanish — bookmarklet → "bookmarklet HTML del navegador")
   Plural-aware `consent_hint_body` uses ICU plural format
   (`{n, plural, one {…} other {…}}`) so single-vs-multiple article
   counts read naturally in each language.
   The consent-hint sentence is split into 3 keys (body/link/after-link)
   so the link text appears mid-sentence rather than tacked on after.
   Components converted to `$_('articles.import.*')` everywhere — no
   remaining hardcoded strings in the bulk-import UI.

i18n parity validator: 76 namespaces × 5 locales — 6477 canonical
keys, all aligned. validate:i18n-hardcoded baseline unchanged for
articles files (broadcasts/notes/timeline failures are user-WIP).

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 13:39:50 +02:00
Till JS
e37c008a7a chore(articles): polish pass — schema cleanup, MAX cap, filters, docs (#8,#9,#13,#15,#18,#20)
Polish-pass on top of the bulk-import rollout. Five contained items.

#8 + #9 — Dexie v60 schema cleanup
   - Drop articleImportJobs.leasedBy + .leasedUntil. They were defined
     on the original v57 schema as a soft-lease handshake, but the
     worker uses pg_try_advisory_xact_lock and never wrote them.
     Local-* type + projection row stripped.
   - Drop the standalone `state` index on articleImportItems.
     [jobId+state] covers the worker's hot query; the state-solo
     index had no call site.
   Both changes lossless — Dexie just removes the column declarations
   from new rows; existing rows still carry the dead nulls (zombies)
   until the next full row-rewrite. Not worth a hard migration for
   two never-written columns.

#15 — MAX_URLS_PER_JOB hard cap (200)
   articleImportsStore.createJob() throws if the URL list exceeds the
   cap. BulkImportForm surfaces the limit in the live counter chip
   and disables the submit when over. The worker can chew through any
   N, but at high counts the UI gets unwieldy (no virtualisation) and
   wall-clock duration climbs into multi-hour. 200 is a pragmatic
   ceiling — Pocket-export dumps average 50–150.

#13 — Filter-Tabs in JobsList
   Pill-style tabs above the list: Alle / Aktiv / Fertig / Mit Fehlern,
   each with the row count. Disabled when the bucket is empty so the
   user only sees actionable filters. The "Mit Fehlern" filter
   (errorCount > 0) is the most valuable for triage.

#18 — apps/mana/CLAUDE.md
   - Articles row added to the Tool Coverage table (5 propose +
     1 auto, including the new auto-policy import_articles_from_urls).
   - New "Articles bulk-import" section after the AI Workbench part:
     pipeline diagram, table list, actor + metrics + cap pointers.

#20 — ARTICLES_IMPORT_WORKER_DISABLED env var documented
   New row under "Mana API — Articles Bulk-Import Worker" in
   docs/ENVIRONMENT_VARIABLES.md.

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 02:42:46 +02:00
Till JS
e8774fc233 test(articles): worker rollup + field-meta + consent-wall + recovery UI (#6,#14)
#6 — Worker test coverage on the deterministic helpers
   Three new bun-test files in apps/api/src/modules/articles/:
     - field-meta.test.ts (6 tests): pins down the legacy-vs-F3 fix
       so it can never regress silently — including the regression
       check from the live-test-found bug (string vs object compare
       across both shapes evaluates correctly).
     - consent-wall.test.ts (8 tests): the heuristic we extracted
       in #4. German + English vocab, wordcount threshold + the
       boundary case, case-insensitivity.
     - import-worker.test.ts (5 tests): countByState rollup. Pins
       down the consent-wall-counts-as-saved semantics so the
       progress bar doesn't off-by-one and allTerminal stays correct.
   Total 19 bun tests, all green.
   countByState + StateCounts exported (test-only access).

#14 — Consent-wall recovery UI in JobDetailView
   Bulk-import items that hit a cookie-wand land as state='consent-wall'
   with the teaser saved. Before this commit there was no UX path to
   "rescue" them other than navigating to the article and re-saving
   manually. Now:
     - Job-level hint banner appears when warningCount > 0,
       explaining the cookie-wand semantics + linking to
       /articles/settings (where the v2 bookmarklet lives).
     - Per-item action group on consent-wall rows: "Teaser ansehen"
       (open existing article) + "Erneut speichern" (deep-link to
       /articles/add?source=bookmarklet&url=… so the bookmarklet's
       postMessage handshake has the URL pre-populated).

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 01:30:04 +02:00
Till JS
59373c0d57 chore(articles): hygiene pass — shared-ai actor + lib/sync-db + metrics (#5,#7,#11)
#5 — SYSTEM_ARTICLES_IMPORT_WORKER hoisted into @mana/shared-ai
   The worker built its actor inline, bypassing the SystemSource union
   that's the blessed list for system-write principals. Now uses
   makeSystemActor(SYSTEM_ARTICLES_IMPORT_WORKER) like every other
   server-side system writer (mission-runner, projection, …).

#7 — sync-db helper hoisted out of mcp/ into lib/
   Implementation moved to apps/api/src/lib/sync-db.ts; mcp/sync-db.ts
   is a re-export shim so existing MCP imports keep working. Articles
   bulk-import + future modules import from lib/ directly — no more
   "articles depending on mcp" layering smell.

#11 — Prometheus metrics for the worker
   New counters + histogram in lib/metrics.ts under
   mana_api_articles_import_*:
     - ticks_total{result=processed|skipped|error}
     - items_total{result=extracted|error|consent_wall|cancelled}
     - extract_duration_seconds (histogram, 0.25–30s buckets)
     - jobs_completed_total{result=done}
     - pickup_gc_rows_total
   Worker tick + extractor instrumented at the right transition points.
   Steady-state pickup_gc_rows_total > 0 over time signals a stuck
   consumer somewhere — useful operator alert.

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 01:06:15 +02:00
Till JS
b297f68ee4 fix(articles, mana-ai): rollout-block hardening for sync_changes projections
Four cross-cutting fixes that make the bulk-import worker safe to run
under real production load. All four were called out as live-rollout
risks in the post-ship review of docs/plans/articles-bulk-import.md.

#1 — Same fieldMetaTime bug fixed in mana-ai
   The articles fix in 054b9e5be hoists the helper to its own file
   `apps/api/src/modules/articles/field-meta.ts`. The same naive
   `rowFM[k] >= localTime` LWW comparison existed in three more
   projections under services/mana-ai (missions-projection,
   snapshot-refresh, agents-projection). Once any F3 stamp lands
   beside a legacy-string stamp, the comparison evaluates
   `'[object Object]' >= 'ISO-…'` (false) and the older value wins.
   New `services/mana-ai/src/db/field-meta.ts` — same helper,
   deliberately duplicated (each service treats sync_changes as a
   read-only event log; sharing infra across services is out of
   scope here). All 61 mana-ai bun tests still pass.

#2 — Stale 'extracting' items recycle
   If the worker dies mid-fetch (OOM, pod restart), items stay in
   state='extracting' forever and the job never completes. New sweep
   at the start of `processOneJob`: items whose lastAttemptAt is
   older than 5 minutes get bounced back to 'pending' so the next
   tick re-claims them. STALE_EXTRACTING_MS tuned for the 15s
   shared-rss fetch + JSDOM-parse worst case.

#3 — Pickup-row GC
   Every 30 ticks (~once per minute) the worker hard-deletes
   articleExtractPickup rows older than 24h. Without this a stuck
   pickup-consumer (all tabs closed, Web-Lock mismatch) would let
   sync_changes accumulate without bound. Logs the row count when
   non-zero so we can spot stuck consumers in the wild.

#4 — DRY consent-wall heuristic
   Identical CONSENT_KEYWORDS + threshold lived in routes.ts AND
   import-extractor.ts. Hoisted to
   `apps/api/src/modules/articles/consent-wall.ts`; both call sites
   now share one heuristic.

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:53:39 +02:00
Till JS
e99fea1938 feat(forms): M3b public-submit endpoint — schließt den Public-Loop
Server-side Public-Submit für unlisted-shared Forms (Plan
docs/plans/forms-module.md M3.b):

- POST /api/v1/forms/public/:token/submit (apps/api):
  - Token-resolve via unlistedSnapshots-Tabelle (eq, limit 1).
  - Hard-blocks: 404 unbekannt, 410 revoked/expired, 400 wrong
    collection, 400 invalid JSON.
  - Schema-validiert serverseitig: filtert eingehende answers auf
    field-IDs aus dem Snapshot (anti-injection), prüft required
    Antwort-Felder + required consent-Felder.
  - Hashed IP (SHA-256, hex) als Anti-Spam-Fingerprint, plus
    User-Agent + Referer truncated, in submitterMeta.
  - Schreibt sync_changes(table='formResponses', op='insert', data,
    field_meta, actor='system:forms-public-submit', origin='system')
    in einer Transaktion mit set_config('app.current_user_id') für
    RLS — mirror vom articles import-extractor.
  - Token-scoped rate-limit (10/min) + IP-scoped (30/min), gleiche
    Architektur wie unlisted/public-routes.
  - Returns { ok: true, responseId, submittedAt }.

- SharedFormView (apps/mana/apps/web): handleSubmit POSTet jetzt an
  ${PUBLIC_MANA_API_URL || origin:3060}/api/v1/forms/public/:token/submit.
  Submitting-State (Disabled-Button + "Sende ..."), Error-Block bei
  Server-Fehlern, Submitter-Block (Name + Email, beide optional). Der
  DEV-Hinweis ist weg.

Encryption: server speichert plaintext im sync_changes-Blob. Der
Client-side Decrypt-Path ist no-op für non-encrypted shapes
(record-helpers.ts:241), also kein Crash beim Pull. Encrypted-at-rest
für public submissions ist M6 ZK-Mode (eigener per-Form-Key der
Form-Owner client-seitig hält).

Mounted pre-auth in apps/api/src/index.ts neben unlisted/public.

apps/api buildet (1769 modules, no TS errors). svelte-check:
0 errors in forms/. Forms-Modul ist End-to-End nutzbar — User legt
Form an, publisht, setzt visibility=unlisted, kopiert Share-Link,
externe Person füllt aus + sendet, Antwort landet im
ResponsesView des Owners.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:44:42 +02:00
Till JS
1815139dc1 chore: drop context module — registry refs, schema, AI route, AppId
The context module's UI + Dexie tables + i18n bundle were already
removed in d3e2e73ca. This follow-up cleans up everything else that
still referenced it:

- API: rename POST /api/v1/context/import-url → /api/v1/kontext/import-url
  (the kontext singleton was the only consumer); drop the unused
  /ai/generate + /ai/estimate endpoints; rename the credit-op label
  AI_CONTEXT_IMPORT_URL → KONTEXT_IMPORT_URL; drop AI_CONTEXT_GENERATION
  from packages/credits.
- Web: drop registerApp + File icon import from app-registry/apps.ts;
  drop contextModuleConfig from data/module-registry.ts (+ snapshot test);
  drop useRecentDocuments + useSpaces from cross-app-queries.ts; drop
  ContextDocsWidget from widget-registry + dashboard.svelte.ts +
  types/dashboard{,.test}.ts; drop dashboard.widgets.context from all 5
  dashboard locales; drop context entries from hooks.server allowlist,
  splitscreen registry, observatory mockData, spiral collect, crypto
  registry + plaintext-allowlist.
- Dexie: remove documents/contextSpaces/documentTags from v1, v31, v53
  stores blocks; add v57 dropping the three tables on local dev DBs
  that already ran an earlier schema.
- Shared-branding: drop 'context' from AppId union, APP_BRANDING,
  MANA_APPS, APP_ICONS (+ contextSvg), ContextLogo.svelte (+ logos
  barrel re-export).
- Spiral-DB: drop context: 10 from MANA_APP_INDEX (slot now free).
- i18n hardcoded-string baseline: drop 5 context routes/files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:20:04 +02:00
Till JS
8fbdc6db77 feat(notes): isSpaceContext flag replaces kontext module (Option B)
Retire the kontext module entirely; the per-Space standing-context
document is now a regular Note flagged with `isSpaceContext: true`.
Daily use ("URL → Notiz") moves to the notes module as a first-class
action; the same primitive is reused by the (planned) Brand/Firma-Space
onboarding wizard to seed a Space-context Note from a URL.

Why: kontext was inconsistent — its UI was a URL-crawler that wrote
to userContext.freeform (profile module), while its kontextDoc table
+ AI-Mission-Runner auto-injection was a write-only shell with no
real editor. One concept (Notes) now carries both ad-hoc noting and
Space-context, with mutex (max 1 flagged Note per Space).

Notes module:
- types: add `isSpaceContext?: boolean` to LocalNote + Note
- queries: add `useSpaceContextNote()` (the active Space's flagged note)
- store: `markAsSpaceContext(id | null)` with mutex sweep across Space
- ListView: "Aus URL importieren" inline form (URL + crawl-mode +
  KI-Zusammenfassung toggle); "Als Space-Kontext markieren" /
  "Space-Kontext lösen" context-menu item; ★-Badge on flagged notes
- new api.ts: `crawlUrl()` client for POST /api/v1/notes/import-url

Notes API (apps/api):
- new modules/notes/routes.ts with /import-url (ported from kontext;
  same crawler + LLM summary pipeline, NOTES_IMPORT_URL credit op)
- mount at /api/v1/notes; add 'notes' to RESOURCE_MODULES (beta+ tier)
- delete modules/context (UI-less /ai/generate + /ai/estimate had no
  consumers; /import-url moved to notes)
- packages/credits: rename AI_CONTEXT_GENERATION → NOTES_IMPORT_URL

AI Mission Runner:
- default-resolvers: drop kontextResolver + kontextIndexer; the
  notesIndexer flags `isSpaceContext` notes with "★ " prefix and
  bubbles them to the top of the picker
- writing reference-resolver: `kind: 'kontext'` now reads the flagged
  Note via scope-scan instead of the kontextDoc table; tests updated
- writing ReferencePicker: useSpaceContextNote replaces useKontextDoc
- AiDebugBlock + MissionGrantDialog + ai-missions ListView: drop
  'kontextDoc' from ENCRYPTED_SERVER_TABLES set
- ai-agents ListView: drop 'kontext' from POLICY_MODULES

Profile module:
- ContextFreeform.svelte: switch import from kontext/api to notes/api
  (the URL-crawl is the same primitive; it still writes to
  userContext.freeform — only the import path changed)

Dexie:
- v58: notes index gains `isSpaceContext`; kontextDoc table dropped

Kontext module deletion:
- delete apps/mana/apps/web/src/lib/modules/kontext/ entirely
- delete (app)/kontext/ route
- drop registerApp + Scroll icon from app-registry/apps.ts
- drop kontext entry from help-content
- drop kontextModuleConfig from data/module-registry.ts
- drop kontextDoc from crypto registry

mana-auth:
- bootstrap-singletons: drop bootstrapSpaceSingletons function entirely
  (kontextDoc was the only per-Space singleton); userContext bootstrap
  unchanged
- better-auth.config: drop kontextDoc bootstrap call from personal-space
  hook + organizationHooks.afterCreateOrganization
- me-bootstrap: drop per-space bootstrap loop; response shape kept
  (always-empty `spaces: {}`) for backwards-compat with older clients

Note: the still-existing legacy `context` module (CMS-style docs/spaces,
unrelated to kontext) is left in place; its cleanup landed on the
articles-bulk-import branch and is out of scope for this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:14:32 +02:00
Till JS
054b9e5beb fix(articles): import-projection accepts F3 + legacy field_meta shapes
Live-test caught it: the worker projects sync_changes via field-level
LWW, comparing `field_meta[k]` directly. But field_meta is two-shaped
on the wire:

  - Legacy plaintext writes:   { state: '2026-04-28T…' }
  - Field-meta-overhaul writes: { state: { at, actor, origin } }

The naive `rowFM[k] >= localTime` worked for the all-legacy case, but
once a client write (legacy string) followed a worker write (F3
object), the comparison evaluated `'2026-04-28T…' >= '[object …]'`
and the projection silently kept the older value. Live symptom: an
item that was correctly flipped to 'saved' on the client was reported
back as 'extracted' by the projection.

Fix: `fieldMetaTime()` helper that pulls the ISO string out of either
shape; both write paths now compare apples-to-apples.

Verified end-to-end:
  - Synthetic job + item written into sync_changes
  - runTickOnce() → claim → extractFromUrl(example.com) → pickup row
    with title='Example Domain', wordCount=16, actor=
    system:articles-import-worker
  - Item transitions pending → extracting → extracted
  - Simulated client write 'saved'
  - Next tick rolls counters: savedCount 0→1, status running→done,
    finishedAt stamped

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 23:43:14 +02:00
Till JS
5535f2da48 feat(articles): server-side bulk-import worker (Phase 2)
apps/api/src/modules/articles/:
  - import-projection.ts: sync_changes → live LWW projection of jobs
    + items. Cross-user scan for claimable jobs, per-job item scan.
  - import-extractor.ts: per-item state-machine. Claim → fetch → write
    pickup + flip extracted, OR retry up to 3x then 'error'. All writes
    attributed to system:articles-import-worker actor (built inline so
    no shared-ai SystemSource extension needed for now).
  - import-worker.ts: 2s tick, pg_try_advisory_xact_lock keyed on 'ARTI'
    so multi-instance apps/api never double-processes. Concurrency 3
    pending items per job per tick. Job-counter rollups + status flips
    derived from current item states.
  - apps/api/src/index.ts: start the worker at boot.

Pipeline (server side):
  Client write articleImportItems(state='pending')
    → sync push → mana_sync.sync_changes
    → server-worker tick projects 'pending' items
    → extractFromUrl (shared-rss / Readability)
    → write articleExtractPickup row + flip item → 'extracted'

Phase 3 (client-side pickup consumer) and Phase 4+ (store + UI) follow.

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 21:33:49 +02:00
Till JS
7766ea5021 docs(plans): mark llm-fallback-aliases SHIPPED, add M-by-M commit table
All 5 milestones landed today in one continuous session: registry,
health cache, fallback router, observability, and consumer migration.
115 service-side tests, validator covers 2538 files.
2026-04-26 21:27:57 +02:00
Till JS
fea3adf5fe feat(llm-aliases): M5 — migrate consumers to MANA_LLM aliases
Final milestone of docs/plans/llm-fallback-aliases.md. Every backend
caller now requests models via the `mana/<class>` alias system instead
of hardcoded `ollama/...` strings. mana-llm resolves aliases through
`services/mana-llm/aliases.yaml` with health-aware fallback (M3) and
emits resolved-model + fallback metrics (M4).

SSOT moved to `packages/shared-ai/src/llm-aliases.ts` so apps/api,
apps/mana/apps/web, and services/mana-ai all import the same
`MANA_LLM` constant via the existing `@mana/shared-ai` workspace
dependency. Three additional sites (memoro-server, mana-events,
mana-research) inline the alias string with a SSOT comment because
they don't pull @mana/shared-ai today.

Migrated 14 sites across 10 files:
- apps/api: writing(LONG_FORM), comic(STRUCTURED), context(FAST_TEXT),
  food(VISION), plants(VISION), research orchestrator (3 tiers
  collapsed to STRUCTURED+FAST_TEXT/LONG_FORM)
- apps/mana/apps/web: voice/parse-task + parse-habit (STRUCTURED)
- services/mana-ai: planner llm-client + tick.ts (REASONING)
- services/mana-events: website-extractor (STRUCTURED, inlined)
- services/mana-research: mana-llm client (FAST_TEXT, inlined)
- apps/memoro/apps/server: ai.ts (FAST_TEXT, inlined)

Legacy env-vars removed: WRITING_MODEL, COMIC_STORYBOARD_MODEL,
VISION_MODEL, MANA_LLM_DEFAULT_MODEL. The chain in aliases.yaml is
now the single tuning surface; SIGHUP reloads it without redeploys.

New `scripts/validate-llm-strings.mjs` regex-scans 2538 files for
hardcoded `<provider>/<model>` strings and fails the build if any
land outside the SSOT or the explicitly-allowed paths (image-gen
modules, model-inspector code, this validator itself, the registry).
Wired into `validate:all` next to the i18n + theme validators.

Verified: `pnpm validate:llm-strings` clean, `pnpm --filter @mana/api
type-check` clean, `pnpm --filter @mana/ai-service type-check`
clean. Web type-check has 2 pre-existing errors in
SettingsSidebar.svelte (i18n MessageFormatter type drift, last
touched in 988c17a67 — unrelated to this work).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 21:26:03 +02:00
Till JS
9e04385930 feat(augur): unlisted-snapshot publish pipeline
augur.setVisibility now coordinates with the server-side unlisted-
snapshots table — same pattern as library/calendar/places. The local
token-allocation placeholder from M6 is replaced with real publish/
revoke calls; deletion revokes any active link before tombstoning.

  - resolvers.ts: buildAugurEntryBlob with strict whitelist
    (source, claim, kind, vibe, encounteredAt, outcome,
    outcomeNote when resolved). NEVER inlines feltMeaning,
    expectedOutcome, probability, tags, livingOracleSnapshot,
    sourceCategory or related FK references — divinatory captures
    stay sensitive even when shared.
  - SharedAugurEntryView: SSR card with vibe-colored border, kind +
    date meta, outcome badge, "Wie es kam" section only when the
    sign was actually resolved.
  - Dispatcher in /share/[token]/+page.svelte gains the
    augurEntries branch.
  - mana-api ALLOWED_COLLECTIONS extended to four items so the
    publish endpoint accepts augurEntries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 15:38:09 +02:00
Till JS
92bee0d71a feat(unlisted): M8.1 — backend foundation for shareable-link snapshots
First milestone of the unlisted-share rollout plan (docs/plans/
unlisted-sharing.md). Adds the server-side infrastructure that backs
`visibility='unlisted'` — previously the flag was stamped locally but
led nowhere. After this commit, a token points at an actual snapshot
the SSR share-page will render (M8.3+).

Scope: backend only. No client-side publish/revoke calls yet, no
share-route, no UI. That lands in M8.2/M8.3. Anyone hitting the
endpoints manually with curl can exercise the full publish-fetch-
revoke cycle.

Changes:
- New pgSchema `unlisted` with table `snapshots`:
    token (pk, 32-char base64url)
    user_id, space_id, collection, record_id, blob (jsonb)
    created_at, updated_at, expires_at (nullable), revoked_at
  Partial unique index on (user_id, collection, record_id) WHERE
  revoked_at IS NULL so one record has at most one active token.
  Partial btree on expires_at for the cron-cleanup path.
- Hand-authored SQL migration `apps/api/drizzle/unlisted/0000_init.sql`
  (manual-apply per the repo's feedback_api_hand_authored_migrations
  memory). Already applied to the local mana_platform.
- Drizzle schema `apps/api/src/modules/unlisted/schema.ts`. All id
  fields are `text` not uuid — Better-Auth nanoids aren't UUIDs, same
  trap we hit with the website module's publish bug.
- mana-api module `apps/api/src/modules/unlisted/`:
    POST   /api/v1/unlisted/:collection/:recordId (auth)
      Body: { spaceId, blob, expiresAt? }. Re-publish reuses the
      existing active token (by (user,collection,record) lookup); a
      revoke-then-republish mints a fresh token row. Response includes
      a fully-qualified share URL built from Origin/Referer/env.
    DELETE /api/v1/unlisted/:collection/:recordId (auth)
      Soft-revoke. Idempotent — already-revoked returns
      { revoked: 0 } cleanly so client stores can call it
      unconditionally on setVisibility-away.
    GET    /api/v1/unlisted/public/:token (public)
      Rate-limited 20/min/token + 60/min/ip so token enumeration is
      impractical. 404 for unknown, 410 Gone for revoked or expired.
      Cache-Control: private, max-age=60 + X-Robots-Tag: noindex for
      SEO isolation. Returns { token, collection, blob, createdAt,
      updatedAt, expiresAt }.
- ALLOWED_COLLECTIONS hardcoded allowlist in POST handler
  (events, libraryEntries, places — the M8.3+M8.4 scope). Unknown
  collection -> 400 COLLECTION_NOT_ALLOWED. Keeps the schema honest
  about what the server accepts.
- drizzle.config extended to include the new schema in managed
  migrations.
- index.ts wires unlistedPublicRoutes pre-auth (before
  authMiddleware) and unlistedRoutes post-auth.

Verified:
- Migration applied to mana_platform — `unlisted.snapshots` exists
  with both partial indexes.
- pnpm run type-check (api): clean
- pnpm run validate:all: theme-tokens, theme-parity, crypto-registry,
  encrypted-tools all green
- URL build uses Origin/Referer before the env fallback so dev
  (http://localhost:5173) and prod (https://mana.how) both work
  without env churn.

Next: M8.2 — shared-privacy client helper + SharedLinkControls
component.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 17:12:13 +02:00
Till JS
6432ef7e6b feat(comic): M4 — AI-Storyboard aus Cross-Modul-Text
User wählt einen bestehenden Text (Tagebuch-Eintrag, Notiz oder
Bibliotheks-Review), das Modell schlägt eine geordnete
Panel-Sequenz vor (prompt + optional caption + dialogue pro Panel),
der User prüft/editiert und feuert Batch-Gen mit sourceInput-
Tagging — damit wird `useStoriesByInput` später cross-referenzieren
können ("Welche Comics sind aus diesem Journal-Eintrag entstanden?").

Backend:
- POST /api/v1/comic/storyboard (Hono route) nimmt style +
  sourceText + panelCount (+ optional storyContext / sourceModule)
  und ruft llmJson() mit einem response_format=json_object-Prompt
  an mana-llm. System-Prompt instruiert das Modell auf eine exakte
  {panels: [{prompt, caption?, dialogue?}]}-Shape, Rules wie
  "keine Style-Instruktionen" (kommen aus dem Story-Prefix
  downstream) und "kein Panel-Nummerieren".
- Defense-in-depth Coerce auf der Response: Panel ohne prompt
  wird gefiltert, Strings werden gecappt (caption/dialogue 200,
  prompt 800), Zahl der Panels auf panelCount geclampt.
- Model via COMIC_STORYBOARD_MODEL env var überschreibbar;
  Default ollama/gemma3:4b wie writing (lokal + billig).
- Beide Erfolgs- und Fehler-Pfade mit logger.info /
  logger.error + userId + sourceModule für Observability.
- Route registriert in apps/api/src/index.ts als /api/v1/comic.

Client:
- api/storyboard.ts: suggestPanels({style, sourceText, panelCount,
  storyContext?, sourceModule?}) — thin fetch-Wrapper + Error-Messaging
  für 402 / 502 / no-panels-Responses.
- ReferenceInputPicker: Tabs über Journal / Notizen / Bibliothek
  (die drei inhalts-dichtesten Quellen), pro Tab Live-Query +
  Suche + Entry-Liste. Click emittiert {module, entryId, label,
  sourceText} — label ist der Display-Name für die
  "Gequellt aus…"-Chip, sourceText ist bereits decrypted (Queries
  liefern plaintext zurück). Bibliotheks-Einträge ohne Review
  sind disabled (kein Text = nichts zu rendern).
- StoryboardSuggester: 4-Schritt-Flow (pick-source →
  generating-plan → review-plan → rendering). Schritt 3 ist der
  eigentliche Editor: jede Claude-Zeile ist editierbar (Prompt,
  Caption, Dialog) mit Trash-Button; Quality + Format-Toggle
  teilen sich M3-Batch-Style. "Generieren" ruft parallel
  runPanelGenerate() via Promise.allSettled mit
  sourceInput={module, entryId} im panelMeta, alle Panels gehen
  durch den identischen M2-HTTP-Pfad.
- DetailView bekommt einen dritten Editor-Modus "ai" neben
  "single" und "batch" — eine Sparkle-Button-CTA öffnet den
  Suggester.

Kein Writing-Draft / Calendar-Event-Input in dieser Runde —
Drafts brauchen Version-Chain-Resolve, Events sind meist zu dünn
an Prosa. Follow-up wenn gewünscht (rein additiv: Tab + Hook).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 16:06:03 +02:00
Till JS
8a882a3760 feat(wardrobe,picture): Google Nano Banana as a Try-On option
Add Google's Gemini image edit family (Nano Banana) as a user-
selectable model for Wardrobe Try-On next to the existing OpenAI
path. Three concrete choices now expose themselves in the Solo and
Outfit Try-On buttons:

  - openai/gpt-image-2          (default, falls back to gpt-image-1
                                 server-side when the org isn't
                                 verified)
  - google/gemini-3-pro-image-preview   (Nano Banana Pro — premium
                                 identity / character consistency)
  - google/gemini-3.1-flash-image-preview (Nano Banana 2 — newest,
                                 fast, cheapest)

All three accept multi-image refs (face + body + garment) through
the same /api/v1/picture/generate-with-reference endpoint; the only
differences are the provider-specific request/response shape and
the model-id routing.

Server (apps/api/src/modules/picture/routes.ts):
- Guard now accepts `openai/*` and `google/*` prefixes and rejects
  everything else as "not supported for edits". Each provider's key
  is validated separately so missing GEMINI_API_KEY doesn't break
  OpenAI calls and vice versa.
- New `callGeminiEdits(modelName)` helper mirrors the shape of
  callOpenAiEdits: encodes the normalized PNG refs as base64
  inline_data parts, POSTs to
  generativelanguage.googleapis.com/v1beta/models/{model}:generateContent
  with responseModalities=["TEXT","IMAGE"] and imageConfig
  (aspectRatio + imageSize), pulls the generated image out of
  candidates[].content.parts[].inlineData.
- Our internal size strings map cleanly: 1024x1024 → 1:1 / 1K,
  1024x1536 → 2:3 / 1K, 1536x1024 → 3:2 / 1K. Gemini 1K is enough
  for the thumbnail sizes Wardrobe renders; going higher bloats
  payload without visible gain.
- creditsFor() gains a google/ branch proportional to upstream
  pricing (pro ≈ 18, 3.1-flash ≈ 6, 2.5-flash ≈ 5).
- Response `model` reports `${provider}/${modelUsed}` so the picture
  row's model metadata is accurate across providers.

Client (apps/mana/apps/web/src/lib/modules/wardrobe):
- api/try-on.ts: export `TryOnModel` union + `DEFAULT_TRY_ON_MODEL`.
  RunGarmentTryOnParams / RunOutfitTryOnParams gain an optional
  `model` field, threaded through `callGenerateWithReference`.
- components/TryOnModelPicker.svelte: new segmented control, three
  options with label + one-line hint. Grid-auto-fits so it reflows
  on the narrow workbench card.
- components/GarmentTryOnButton.svelte + TryOnButton.svelte: both
  mount the picker above the Sparkle CTA. `estimatedCredits` on the
  button label updates live when the user switches model so the
  cost signal matches what the server will actually charge.

Env (scripts/generate-env.mjs): GEMINI_API_KEY and GOOGLE_API_KEY
now propagate from the root `.env.development` into `apps/api/.env`
so mana-api can pick them up at boot. The route reads GEMINI_API_KEY
with GOOGLE_API_KEY as fallback, matching how mana-llm ships today.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 16:04:21 +02:00
Till JS
27c1860f82 feat(comic): M1 — Datenschicht + Modul-Registrierung
Neues Comic-Modul: aus Text-Inputs (Journal / Notes / Writing / Library
/ Calendar) entsteht ein mehrseitiger Comic, generiert mit gpt-image-2
über die bestehende /picture/generate-with-reference-Route. Plan in
docs/plans/comic-module.md (M1–M5 + optional M6–M8).

M1 schafft die Datenschicht ohne UI:
- Dexie v44 `comicStories` (space-scoped, Indices createdAt/style/
  isFavorite/isArchived). Story hält `panelImageIds: string[]` und
  `panelMeta: Record<panelImageId, {caption, dialogue, promptUsed,
  sourceInput?}>` — Panels selbst sind picture.images-Rows mit
  comicStoryId + comicPanelIndex Back-Refs.
- Fünf Stil-Presets (comic / manga / cartoon / graphic-novel / webtoon)
  mit Prompt-Prefix-Templates in styles.ts; composePanelPrompt webt
  Stil + Panel-Prompt + Caption + Dialog zusammen. Sprechblasen
  werden von gpt-image-2 direkt ins Bild gerendert — kein SVG-Overlay.
- Encryption-Registry-Eintrag: title / description / storyContext /
  tags / panelMeta als JSON-Blob. Struktur (id, style, character-
  MediaIds, panelImageIds, Flags, visibility) bleibt plaintext.
- Module-Registry registriert appId='comic', verifyMediaOwnership auf
  der /picture/generate-with-reference-Route akzeptiert jetzt
  ['me', 'wardrobe', 'comic'] — 'comic'-Slot ist reserviert für M6+
  Anchor-/Backdrop-Uploads.
- Space-Allowlist: comic in brand (Marken-Storys), club (Vereins-
  geschichte), family (Kinder-Abenteuer), team (Release-Comics),
  practice (Patienten-Aufklärung). Personal via '*'-Sentinel.
- mana-apps.ts Eintrag mit comic-Icon (Sprechblase + Lightning-Bolt,
  f97316→dc2626 Gradient). Lokal tier='guest' mit LOCAL TIER PATCH-
  Comment wie Wardrobe, canonical ist 'beta'.

Visibility-System von Anfang an adopted (setVisibility-Methode im
Store, unlistedToken-Generierung inklusive). appendPanel() als
Vorarbeit für M2 bereits da, ohne Aufrufer.

5 Encryption-Roundtrip-Tests grün (panelMeta nested JSON, leeres
panelMeta, partielle panelMeta ohne sourceInput, null-description).
pnpm run check + validate:all sauber (207 Dexie-Tabellen klassifiziert,
comicStories unter den 106 encrypted).

Kein UI, keine Panel-Generierung, keine MCP-Tools — alles M2/M3/M5.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:29:51 +02:00
Till JS
d725a8df8b feat(writing): M3 — one-shot prose generation via mana-llm
Server:
- New llmText() helper in apps/api/src/lib/llm.ts for plain-text
  (non-streaming) completions with token-usage reporting.
- POST /api/v1/writing/generations (Hono + requireTier('beta'))
  accepts system+user prompts, forwards to mana-llm (default model
  ollama/gemma3:4b), returns raw output + model + tokenUsage. The
  endpoint is stateless — draft/version bookkeeping is entirely
  client-side so the same route serves refinement calls later.

Client:
- writing/api.ts — Bearer-authed fetch client (follows the food/
  news-research pattern).
- writing/utils/prompt-builder.ts — pure builder turning a briefing
  (+ optional style preset / extracted principles) into a system+user
  pair. Forbids preamble / sign-off / meta commentary so the output is
  ready to paste into a version.
- writing/stores/generations.svelte.ts — orchestrates the full flow:
  queued → running → call → new LocalDraftVersion → pointer flip →
  succeeded. On failure leaves the current version untouched with the
  error on the generation record. Emits WritingDraftGenerationStarted /
  WritingDraftVersionCreated / WritingDraftGenerationFailed events.

UI:
- Generate button in DetailView.svelte (label flips "Generate" / "Neu
  generieren" based on whether the draft already has content).
- GenerationStatus.svelte strip surfaces queued / running / failed with
  model + duration badges; succeeded generations auto-disappear because
  the new version is already live via the currentVersionId pointer.

M3 is synchronous and non-streaming by design. M7 adds mission-based
long-form with streaming + outline stage + reference injection. M6 will
reuse the same /generations endpoint for selection-refinement prompts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 15:11:48 +02:00
Till JS
91fd88e77d fix(picture): normalize Try-On refs to clean RGB PNG before OpenAI call
gpt-image-1 answered the last Try-On attempt with
  invalid_image_file: Invalid image file or mode for image 2
because one of the references (face/body/garment) was in a format or
color mode OpenAI's edits endpoint rejects — typical culprits are
HEIC from iPhones, CMYK JPEG, palette-mode PNG, APNG, or JPEG with an
ICC profile gpt-image-1 doesn't honour. mana-media stores originals
verbatim so whatever the user uploaded is what we were forwarding.

Route the references through mana-media's existing on-the-fly
/transform endpoint (format=png, w/h=1024, fit=inside) which pipes
the buffer through sharp server-side. One call per ref, all run in
parallel, same latency budget as before. Output is guaranteed
- PNG / RGB (or RGBA if the source had alpha, which gpt-image-1 accepts),
- no more than 1024 px on the longest side → well under OpenAI's
  4 MB/image cap,
- aspect-ratio-preserving (fit=inside) so a portrait body photo
  doesn't get squished into a square.

New helper `getMediaBufferAsPng(mediaId, longestSide)` in lib/media.ts
encapsulates the transform-URL build. The Try-On path in the picture
route now uses it instead of `getMediaBuffer`; all Blob filenames
pin to `.png` since the buffer is already normalized.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 01:55:00 +02:00
Till JS
b204958007 feat(picture): fall back to gpt-image-1 when gpt-image-2 org-unverified
OpenAI started gating gpt-image-2 behind per-organization verification
(platform.openai.com/settings/organization/general → Verify Organization,
propagation up to 15 min). Unverified orgs get:

  "Your organization must be verified to use the model gpt-image-2"

Keeps Try-On broken until the user completes that manual step. Since
the edits endpoint is identical across gpt-image-1 and gpt-image-2
(same image[] multi-ref, same size/quality/n params), detect that
specific rejection and retry once with gpt-image-1.

- buildFormData(modelName) + callOpenAiEdits(modelName) extracted so
  the retry is a one-line re-invoke with the fallback model instead
  of a duplicated fetch block.
- needsGptImage1Fallback() matches /verified to use the model/i in
  the error body AND checks the attempted model was actually
  gpt-image-2 — an explicit openai/gpt-image-1 request stays on 1.
- Response now reports `model: openai/${modelUsed}` so the
  picture.images row records whichever model actually produced the
  image (matters for future re-generation / audit).

Credits unchanged: our flat 3/10/25-per-quality tariff applies to all
openai/* paths. Slight over-charge for the gpt-image-1 fallback until
the user verifies, then gpt-image-2 takes over automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 01:50:21 +02:00
Till JS
15beddeda9 fix(picture): use image[] array syntax for multi-ref gpt-image-2 edits
The try-on path POST'd N reference images as repeated `image` fields in
the multipart body. OpenAI's edits endpoint answers that with
`duplicate_parameter: Duplicate parameter: 'image'. You provided
multiple values for this parameter, whereas only one is allowed. If
you are trying to provide a list of values, use the array syntax
instead e.g. 'image[]=<value>'.`

Switch to the array-syntax field name `image[]`, which OpenAI accepts
for cardinality ≥ 1 (no branching needed for the single-ref case).

Also surface the underlying error from the three 502 branches
(ownership-check, media-fetch, OpenAI call) into both the server log
(structured console.error with refIds + openai body) and the response
`detail` field. The client's callGenerateWithReference now prepends
`detail` to the thrown message so the user sees the concrete reason
in-module instead of a generic "Try-On fehlgeschlagen (502)".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 01:43:01 +02:00
Till JS
c404db5b6e fix(website): publish failed with uuid type error on Better-Auth ids
published_by, created_by, and space_id were declared as uuid, but
Mana user + space ids are Better-Auth nanoids stored as text. The
insert into website.published_snapshots raised `invalid input syntax
for type uuid` and Hono swallowed it as a generic 500.

Changes:
- schema.ts: uuid -> text on the three columns
- 0003_fix_id_types.sql: ALTER COLUMN on existing installs
- publish.ts: replace UUID regex on X-Mana-Space with a nanoid-shaped
  check (it was silently nulling valid space ids)
- publish.ts: log + return the actual error message on the 500 path
  so the next unhandled failure is visible instead of opaque

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 21:43:57 +02:00
Till JS
f20ace0358 test(website): broad automated coverage across the builder surface
83 new tests across 5 files — pure-logic, fast, run on every
push. Caught one real bug + motivated one small refactor.

Coverage:

- apps/mana/.../website/constants.test.ts (8): isValidSlug + RESERVED_SLUGS
  + isValidPath. Caught the 1-char-slug bug (regex allowed length 1;
  UI + plan say min 2). Fixed the regex in both the webapp and the
  mirrored server list.
- apps/mana/.../website/publish.test.ts extended (8 total): adds
  self-parent cycle, 3-level nesting, all-orphans, empty-input cases
  on top of the original determinism + orphan-drop tests.
- apps/mana/.../website/templates.test.ts (7): parameterised over each
  of the 4 bundled templates — clone produces fresh UUIDs, page +
  block counts match, navConfig populated. Plus unknown-template and
  duplicate-slug rejection. Container-nesting is punted to the smoke
  test (none of the bundled templates use columns yet).
- packages/website-blocks/src/schemas.test.ts (38): every block
  (11) + sanity-checks (defaults satisfy own schema, enum + length
  bounds, required fields). Pure Zod — no Svelte runtime needed.
- packages/website-blocks/src/themes/themes.test.ts (12): preset
  parity, resolveTheme overrides, themeCssVars output format +
  heading-font fallback.
- apps/api/src/modules/website/reserved-slugs.test.ts (10): mirror of
  the client tests for the server SSOT, plus new hostname validation
  cases (.mana.how reservation, length, malformed edges).

Refactor:

- apps/api/src/modules/website/reserved-slugs.ts now owns
  isValidHostname + RESERVED_HOSTNAMES. domains.ts imports them.
  Pure functions live next to the other pure validators; easier to
  test + share.

All 83 new tests green. Web-app svelte-check + apps/api type-check
both clean. Existing publish.test.ts / website-blocks tests still
pass (the monorepo-wide count is now well above 83 — these are
the new ones from this commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 21:07:40 +02:00
Till JS
d56ad396d8 feat(wardrobe,picture): try-on integration — outfit → OpenAI edit (M4)
M4 of docs/plans/wardrobe-module.md — the loop closes. A user with at
least a face-ref in the active space can click "Anprobieren" on an
outfit detail page; the client composes a reference call against the
existing M3 `/generate-with-reference` endpoint, persists the result
into the Picture gallery with a `wardrobeOutfitId` back-reference,
and pins a `lastTryOn` snapshot on the outfit so its card instantly
shows the AI preview next time.

Server side — picture/routes.ts:
- verifyMediaOwnership now accepts `apps: string | readonly string[]`.
  Under the hood it runs one list() per app-tag and unions the owned
  set before the missing-id check. Preserves the 500-row per-app
  sanity cap. Single-tag callers unchanged — it's an additive widen.
- Picture /generate-with-reference passes `['me', 'wardrobe']` so
  face/body portraits (me-images) and garment photos (wardrobe) can
  ride in the same referenceMediaIds array. Anything outside those
  two tags still 404s — no expansion of the trust surface.

Client side — wardrobe/api/try-on.ts:
- `runOutfitTryOn({ outfit, garments, faceRefMediaId, bodyRefMediaId?, ... })`
  composes the ref list (face → body → up to 6 garments, respecting
  the 8-slot server cap), picks portrait 1024x1536 by default (or
  1024x1024 in accessory-only mode), and POSTs with
  `model='openai/gpt-image-2'`, `quality='medium'`, `n=1`. One render
  per click; multi-variant is a future Generator-style extension.
- Default prompts are composed in DE from the outfit meta (name +
  occasion); callers can override via `prompt`. Accessory-only mode
  uses a tighter studio-portrait phrasing since the fullbody ref is
  dropped there.
- `isAccessoryOnlyOutfit()` helper — iff every garment is in
  FACE_ONLY_CATEGORIES, skip body-ref and render square. Covers the
  Brille-Try-On headline use case.
- On success: inserts a `picture.images` row with generationMode=
  'reference', referenceImageIds, and wardrobeOutfitId set; then
  calls wardrobeOutfitsStore.setLastTryOn() with imageId + imageUrl
  so OutfitCard + DetailOutfitView immediately flip to the AI cover.

TryOnButton — wardrobe/components/TryOnButton.svelte:
- Three states: ready (click to render), missing-references (shows
  UserCircle + link to /profile/me-images, with the right hint for
  accessory-only vs. fullbody), loading (spinner).
- Credit estimate on the button (10c medium quality).
- Hints: accessory-only, too-many-garments (>6, over server cap),
  and non-personal-space disclosure — the family-space case gets its
  own sentence since "Try-On rendert dich, nicht dein Kind" is
  non-obvious.
- Reads face-ref/body-ref via useImageByPrimary (space-scoped after
  the v40 meImages migration — brand/club/family spaces need their
  own references uploaded).

UI wiring:
- DetailOutfitView replaces the M3 stub button with <TryOnButton/>.
  The existing "Try-On Verlauf"-Strip already reads
  `useOutfitTryOns(outfit.id)` which filters `picture.images` by
  wardrobeOutfitId — it lights up automatically on first render.

Not in M4 (punted to follow-ups):
- Solo-garment try-on on DetailGarmentView ("nur diese Brille auf
  mein Gesicht"). Plan called it out as optional; the outfit flow
  already covers it when the outfit contains only that one garment.
- Multi-variant rendering (n=2/4). Usable "show me 3 looks" needs a
  picker UI on top, not just a param bump.
- Quality + prompt override in the button. A power-user panel can
  come later; default medium + auto-prompt keeps M4's click-to-try-on
  one-tap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 18:52:57 +02:00
Till JS
d518169ce9 feat(website): M7 — observability + analytics + GC + M2-polish
Closes the plan. Prometheus metrics across the website endpoints, a
cookieless analytics block users can opt in to, a read-only orphan-
asset scan script, plus two M2 debts (rollback UI + determinism test).

apps/api:
- New /metrics endpoint (unauth; internal-network only via reverse proxy).
  Scrape with the existing Prometheus config that already covers mana-ai.
- lib/metrics.ts with prom-client Registry and default-metrics prefix
  `mana_api_`. Website-specific counters/histograms:
    website_publish_total{result=success|slug_taken|invalid|error}
    website_publish_duration_seconds (Histogram)
    website_submissions_total{result=received|spam|rate_limit|not_found|invalid}
    website_host_resolve_total{result=hit|miss|error}
    website_domain_verify_total{result=verified|failed}
    website_public_reads_total{result=hit|not_found}
    website_public_read_age_seconds (Histogram — age of served snapshot)
- Instrument publish.ts, submit.ts, public-routes.ts, domains.ts with
  .inc() calls on every code path.

packages/website-blocks:
- New `analytics` block: Plausible + Umami support with self-hosted
  script-URL override. Hidden in edit/preview, emits exactly one
  <script> in public mode. No cookies, no PII. Registered in block-
  registry; 11 blocks total now.

apps/api/scripts/gc-website-assets.ts:
- Read-only scan: walks published_snapshots.blob + submissions.payload
  for /api/v1/media/{id}/ references, asks mana-media for items scoped
  to app=website, flags orphans older than 30d. Writes report to
  /tmp/gc-website-assets-<ts>.json. Deletion toggle is a future commit.

apps/mana/apps/web:
- RollbackDialog component + PublishBar integration. Closes the M2
  debt "Rollback funktioniert" (API + store were there; UI was missing).
- publish.test.ts: snapshot determinism + orphan-drop tests. 4/4 pass.

docs:
- observability/website.md: metric reference, PromQL queries, alert
  suggestions, Grafana dashboard pointer.
- plans/website-builder.md: M7 checklist updated (Per-site-stats +
  submission-retention explicitly deferred with reason), shipping log
  table completed with all M1→M7 commits.

Validation:
- apps/mana/apps/web: pnpm check → 0 errors 0 warnings
- apps/api: tsc --noEmit → clean
- website-blocks tsc → clean
- publish.test.ts → 4/4 pass

Note: validate:all's check:crypto fails on unrelated WIP (wardrobe
module's Dexie tables aren't classified yet in encryption-registry).
Pre-existing failure, not introduced by this commit — the pre-commit
lint-staged run does NOT include check:crypto so it doesn't block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 18:30:49 +02:00
Till JS
4fc9d6c59c feat(wardrobe): module foundation — garments + outfits space-scoped data layer (M1)
M1 of docs/plans/wardrobe-module.md — pure data layer + backend plumbing,
zero UI (that's M2). A user can now hold a digital wardrobe per space:
brand merch, club Trikots, family Kleiderschrank, team Kostüme, practice
Dresscode, and personal closet all live as separate pools under the same
Dexie tables, space-scoped like tags/scenes/agents after Phase 2c.

Data model — two tables, no join:

- wardrobeGarments (Dexie v41): single clothing items / accessories.
  Indexed on `category` + `createdAt` + `isArchived`. Encrypted:
  name/brand/color/size/material/tags/notes. Plaintext: category,
  mediaIds, counters, timestamps — all indexed or structural.
  `mediaIds[0]` is the primary photo used for try-on; additional
  ids are alternate views (back, detail) for M7.

- wardrobeOutfits (Dexie v41): named compositions referencing
  garment ids. Encrypted: name/description/tags. Plaintext:
  garmentIds (FK array), occasion (closed enum — useful for
  undecrypted filtering), season, booleans, lastTryOn snapshot.

- picture.images gains `wardrobeOutfitId?: string | null` as a
  plaintext back-reference. Try-on results land in the Picture
  gallery like any other generation; the outfit detail view
  queries them via this id rather than maintaining a third table.

Space scope:

- `wardrobe` added to all five explicit allowlists in shared-types/
  spaces.ts (personal is wildcard, no edit needed). Each space type
  gets a one-line comment explaining the real-world use case.
- App registry: `wardrobe` entry in shared-branding/mana-apps.ts
  with a rose→fuchsia gradient icon (T-shirt on hanger silhouette),
  color #e11d48, tier 'beta', status 'beta'.
- Module registry: wardrobeModuleConfig imported + appended to
  MODULE_CONFIGS so SYNC_APP_MAP picks it up automatically.

Backend:

- MAX_REFERENCE_IMAGES bumped 4 → 8 in picture/generate-with-
  reference (plus the client-side default in ReferenceImagePicker).
  Justified with a comment: face + body + top + bottom + shoes +
  outerwear + 2 accessories = 8. Cost doesn't scale with ref count
  (OpenAI bills per output), so the bump is a pure capability
  expansion with no credit-side risk.
- New POST /api/v1/wardrobe/garments/upload wraps uploadImageToMedia
  with app='wardrobe'. Registered under /api/v1/wardrobe in index.ts.
  Pattern 1:1 with the profile/me-images/upload endpoint; tier-gating
  falls out of wardrobe NOT being in RESOURCE_MODULES (tier='guest'
  works — consistent with picture's plain CRUD).

Stores emit domain events (WardrobeGarmentAdded, WardrobeOutfitCreated,
WardrobeOutfitTryOn, etc.) so later mana-ai missions can observe
activity without polling.

No UI in this commit. M2 (Garments-Grundlayer) wires the route + grid
+ upload-zone; M3 the Outfit composer; M4 the Try-On integration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 18:27:37 +02:00
Till JS
3eca5ac201 feat(website): M6 — subdomain publish + custom-domain foundation
SvelteKit hook + new DB table + founder-gated API + UI section. Ships
the code path for public-site routing on {slug}.mana.how and custom
hostnames. Cloudflare SaaS Hostnames integration is stubbed — see
plan §M6 "Offene Enden".

apps/api/src/modules/website:
- schema.ts: new `customDomains` table. Fields: id, site_id, hostname
  (unique), status (pending | verifying | verified | failed),
  verification_token, dns_target, verified_at.
- drizzle/website/0002_custom_domains.sql: manual migration with
  partial unique index on (hostname) WHERE status='verified'.
- domains.ts (new, authenticated + founder-gated via
  `requireTier('founder')`): POST/GET/DELETE /sites/:id/domains,
  POST /sites/:id/domains/:domainId/verify. Verify runs CNAME + TXT
  checks via node:dns/promises with an apex-domain A-record fallback.
  Reserved-hostname list prevents users from binding mana.how subdomains.
- public-routes.ts: new GET /public/resolve-host?host= — unauthenticated
  resolver used by hooks.server.ts. Returns { slug, siteId } only for
  verified bindings tied to a currently-published site.

apps/mana/apps/web/src/hooks.server.ts:
- After the existing https/app-subdomain guards, a new
  `resolveWebsiteRewrite()` step rewrites `event.url.pathname`:
    {slug}.mana.how/path → /s/{slug}/path     (pure string)
    custom-host.com/path → /s/{resolved}/path (API call, 60s LRU)
- Browser URL stays on the custom host — this is a server-side rewrite,
  not a 302. APP_SUBDOMAINS + RESERVED_WEBSITE_SUBDOMAINS win over
  website routing. Localhost and apex mana.how are skipped.

apps/mana/apps/web/src/lib/modules/website:
- domains.ts (new): typed client for list/add/verify/remove. Handles
  200 + expected 400 (verification-failed) separately.
- components/DomainsSection.svelte: add-input, per-domain status pill,
  DNS-instructions box (CNAME + TXT with copy-to-clipboard), Verify
  button. Mounted inside SiteSettingsDialog as its own section — the
  existing theme/footer controls stay put.

docs/plans/website-builder.md:
- M6 checklist updated with what shipped vs. ops-gap (CF SaaS).
- `mana-landing-builder` consolidation: DECIDED to keep parallel. Four
  reasons in the plan. Revisit-criterion stated.
- Shipping log table seeded with M1→M6 commits.

Validation:
- pnpm run validate:all: 6/6 gates green
- pnpm run check (web): 0 errors, 0 warnings
- apps/api type-check: green

Apply schema with:
  psql "$DATABASE_URL" -f apps/api/drizzle/website/0002_custom_domains.sql

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 15:29:42 +02:00
Till JS
57be0f61b1 feat(website): M4 — forms + moduleEmbed
Adds two new block types and the server-side infrastructure for
untrusted input + cross-module data embedding.

Forms:
- packages/website-blocks/src/form: declarative fields (text, email,
  tel, url, textarea, number) with required / maxLength / placeholder
  per field. Honeypot hidden input in the renderer; public-mode POST
  to a same-origin SvelteKit proxy that forwards to mana-api.
- apps/api: website.submissions table (schema.ts + 0001_submissions.sql)
  + POST /public/submit/:siteSlug/:blockId. Loads the current published
  snapshot, finds the form block, validates payload against its
  declared fields (trim, type check, length cap), rejects honeypot
  submissions silently, rate-limits per IP (10 / 5 min) in-memory.
  Unknown keys are dropped — clients can only submit declared fields.
- Owner-facing: GET/DELETE /sites/:id/submissions + SubmissionsView
  component + /(app)/website/[siteId]/submissions route. Shows
  incoming submissions with status pill + payload preview + delete.
- apps/mana/.../routes/s/[siteSlug]/__submit/[blockId]/+server.ts:
  same-origin proxy so form posts don't trigger CORS and IP / user-
  agent headers are forwarded via SvelteKit's trusted getClientAddress.

M4 first-pass does NOT wire target-module delivery (contacts / notify).
Submissions stay in the inbox until owner-side tool handlers land
(M4.x). `target` enum is intentionally `['inbox']` only for now.

moduleEmbed:
- packages/website-blocks/src/moduleEmbed: source dropdown
  (picture.board | library.entries), max-items, layout (grid | list),
  optional filter object. The `resolved` field on props is populated at
  publish time by the editor-side resolver — public renderer reads it
  directly, no Dexie / API round-trip needed.
- apps/mana/.../website/embeds.ts: per-source resolvers. picture.board
  enforces `isPublic=true`; library.entries respects filter.isFavorite
  / kind / status so owners can expose a subset (e.g. "my favorites").
- buildSnapshot() walks the tree after assembly and fills in
  block.props.resolved for every moduleEmbed. Publish slower, public
  visits fast. No cross-service call at render time.

Validation:
- pnpm run validate:all: 6/6 gates green
- pnpm run check (web): 0 errors, 0 warnings
- apps/api type-check: green

Apply Postgres with:
  psql "$DATABASE_URL" -f apps/api/drizzle/website/0001_submissions.sql

Plan: docs/plans/website-builder.md (M4 shipped)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 14:36:52 +02:00
Till JS
38dc806549 feat(personas): M3.b-d — tick loop + Claude Agent SDK + persistence
Closes the M3 loop from docs/plans/mana-mcp-and-personas.md. The
runner now picks up due personas, drives them through Claude + MCP
for one simulated turn, collects actions + ratings, and persists
them through service-key internal endpoints in mana-auth.

Internal endpoints (mana-auth, service-key-gated)

- GET  /api/v1/internal/personas/due
    Returns personas whose tickCadence + lastActiveAt say they're
    due. Rules: hourly > 1h, daily > 24h, weekdays > 24h mon-fri.
    NULLS FIRST so never-run personas go ahead of stale ones.

- POST /api/v1/internal/personas/:id/actions
    Batch ≤ 500. Row ids are deterministic
    (`${tickId}-${i}-${toolName}`) + ON CONFLICT DO NOTHING so the
    runner can retry a tick without doubling audit rows. Also
    bumps personas.last_active_at so the next /due call sees it.

- POST /api/v1/internal/personas/:id/feedback
    Batch ≤ 100. Row id is `${tickId}-${module}` — natural key is
    one rating per module per tick.

Runner tick pipeline (services/mana-persona-runner/src/runner/)

- claude-session.ts
    Two phases per tick. runMainTurn feeds the persona's system
    prompt + a German "simulate a day" user prompt to Claude Agent
    SDK's query(), with mana-mcp wired in as a streamable-HTTP MCP
    server. We iterate the returned AsyncGenerator and extract
    tool_use blocks into ActionRows; tool_result with is_error=true
    flips the most recent action. runRatingTurn is a fresh query()
    with tools:[] asking Claude in character to rate each used
    module 1-5 as strict JSON, which we parse with tolerance for
    surrounding whitespace / fences. Unparseable output becomes a
    synthetic '__parse' feedback row so operators see the failure.

- tick.ts
    Orchestrator. Skips if config.paused. Fetches /due, processes
    in batches of config.concurrency (Promise.allSettled so one
    failure doesn't kill the batch), returns {due, ranSuccessfully,
    failed[], durationMs}.

- types.ts
    ActionRow and FeedbackRow shapes shared between claude-session
    and the internal client; mirrors the mana-auth schema but in
    narrow plain TS for the wire.

Runner bootstrap (src/index.ts)

- setInterval(config.tickIntervalMs) starts the tick loop on boot.
  tickInFlight guards against overlap when Claude latency > interval.
  If MANA_SERVICE_KEY or ANTHROPIC_API_KEY is missing, loop is
  disabled with a warn line — /health still works, /diag/login
  still works.
- New dev-only POST /diag/tick fires a single tick on demand and
  returns the result, so you can verify without waiting 60 s.
- Graceful SIGTERM/SIGINT shutdown clears the interval.

Client

- clients/mana-auth-internal.ts
    X-Service-Key client for the three endpoints above. Constructor
    throws if serviceKey is empty — fail loud, not silent.

Boot smoke: /health + /diag/tick both return descriptive 500s when
keys are absent, 200/JSON when present. Warning lines show up on
boot for missing keys. Type-check green across mana-auth, tool-
registry, mcp, persona-runner.

End-to-end smoke recipe (docker up → db:push → seed:personas →
diag/tick → psql) documented in
services/mana-persona-runner/CLAUDE.md. That's the M3 exit gate.

M2.d (cross-space family/team memberships) still deferred.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 14:16:36 +02:00
Till JS
54a12ffd5c feat(webapp): wire isParallelSafe in Companion chat + Mission runner
Enables the M1 parallel-reads optimisation on the webapp side. Both
consumers of runPlannerLoop pass an isParallelSafe predicate derived
from the tool catalog:

  isParallelSafe: (name) =>
    AI_TOOL_CATALOG_BY_NAME.get(name)?.defaultPolicy === 'auto'

Auto-policy tools (list_tasks, get_habits, nutrition_summary, …) run
via Promise.all in batches of 10 when the LLM fans them out in one
round. Propose-policy tools — which surface to the user as Proposal
cards — stay sequential so intent ordering in the inbox is preserved
and pre-execute guardrails can reason about prior-step state.

Tests: 31 existing companion + mission tests pass unchanged; the
parallel path is exercised via the new loop.test.ts cases shipped
with the M1 commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 14:11:24 +02:00
Till JS
89258eb451 feat(profile,api): meImages foundation for AI reference generation (M1)
M1 of docs/plans/me-images-and-reference-generation.md — a user-owned
pool of reference images (face, fullbody, hands, …) that will back
image generation where the user appears as themselves (outfit try-on,
glasses, portraits) via OpenAI /v1/images/edits. Data layer only in
this commit; UI lands in M2, the edits endpoint in M3.

- Dexie v38: meImages table with id/kind/primaryFor/createdAt indices.
  Added to USER_LEVEL_TABLES so the hook stamps userId and skips the
  spaceId/authorId/visibility trio (one human = one face across every
  Space, not per-Space).
- Encryption registry: label + tags encrypted; kind/primaryFor/usage
  stay plaintext because they drive the indexed queries and the
  Reference picker's filtering. mediaId/URLs/dimensions are structural.
- Profile module store: createMeImage, updateMeImage,
  setAiReferenceEnabled (per-image KI opt-in — plan decision #5),
  setPrimary (transactional slot swap — only one row per primary slot),
  deleteMeImage. Emits MeImage* domain events.
- Queries: useAllMeImages, useMeImagesByKind, useReferenceImages
  (only the rows the user opted in for KI), useImageByPrimary.
- POST /api/v1/profile/me-images/upload: thin wrapper over mana-media
  with app='me' as the reference tag. No new MinIO bucket — plan
  decision #1 revised after verifying mana-media uses one bucket and
  only tags references by app.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 13:50:53 +02:00
Till JS
3a68a63728 feat(picture,api): GPT-Image-2 image generation
Adds a third provider path to /api/v1/picture/generate that calls OpenAI
gpt-image-2 when model starts with "openai/". Supports n=1..4 batch
generation with character continuity, base64 response decoded server-side
and uploaded to mana-media for dedup + thumbnails. Credit cost scales
by quality (low=3, medium=10, high=25) × n.

Env plumbing:
- scripts/generate-env.mjs: new apps/api/.env stanza propagates
  OPENAI_API_KEY + REPLICATE_API_TOKEN from .env.secrets
- .env.macmini.example: documents OPENAI_API_KEY for prod

Frontend /picture/generate: model + quality + aspect-ratio + batch-count
selectors, real fetch with auth, persists each image via imagesStore.insert
(encrypted + synced). Wrapped in ModuleShell variant=fill with back-arrow
to /picture and a live credit badge in the header actions slot.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 00:37:15 +02:00
Till JS
efe1810b04 feat(articles): browser-HTML bookmarklet + consent-wall detection + auto-save
Three intertwined improvements so the "save an article" flow actually
works on real-world sites, not just bloggy happy-path URLs.

=== Consent-wall detection ===

apps/api/src/modules/articles/routes.ts: the /extract response now
includes `warning: 'probable_consent_wall'` when the extracted text
is both short (<300 words) AND contains cookie-dialog vocabulary
(Cookies zustimmen / cookie consent / Zustimmung / accept all cookies
/ enable javascript / privacy center / Datenschutzeinstellungen). The
server still returns whatever it got so the client can decide; it just
flags it as probably-not-the-article.

Frontend surfaces that warning prominently instead of silently
persisting a "Cookies zustimmen…" blob as the article body.

=== Browser-HTML extract path ===

Server-side: new POST /api/v1/articles/extract/html endpoint accepting
{ url, html }, running @mana/shared-rss's extractFromHtml on the
caller-supplied HTML. 10 MiB payload cap. Same response shape as
/extract, including the consent-wall warning (in case the bookmarklet
fires before the user dismisses the dialog).

Client-side: new extractFromHtml() in api.ts with the same 25s
timeout + typed network-error mapping as extractArticle.

AddUrlForm gains a postMessage handshake: when loaded with
?source=bookmarklet, it posts `mana-ready` to window.opener and
listens one-shot for `mana-html` with { url, html, title } from the
opener's tab. The HTML goes straight to our own /extract/html
endpoint — same-origin, carries the user's auth cookie. No CORS, no
form-submission CSP tango, no cross-origin token smuggling. If
nothing arrives within 30s we surface a clear error instead of
hanging.

Settings page adds a second "browser-HTML" bookmarklet (marked as
"Empfohlen") alongside the legacy URL bookmarklet. New snippet opens
/articles/add?source=bookmarklet in a new tab, waits for mana-ready,
then postMessages the tab's documentElement.outerHTML over. 15s
safety timeout.

This bypasses cookie-consent walls and soft paywalls because the
HTML already comes from the user's own authenticated, consented
browser tab.

=== Auto-save after successful extract ===

Previously every save path had a two-click UX: preview → confirm.
Now on clean extract the preview skips straight to persist + navigate
to the reader. Consent-wall warning is the only fallback that pauses
the flow — the user gets a "Trotzdem speichern" button to opt into
saving a teaser anyway.

Button in the manual input row is renamed "Vorschau abrufen" → "Speichern"
since it's now the commit action, not the inspect action. Loading-block
messaging distinguishes "Server extrahiert…" vs "Speichere in deine
Leseliste… Gleich weiter zum Reader."

Net click count:
  Bookmarklet v1/v2 on working site:  2 clicks → 1 click
  Manual paste:                        2 clicks → 1 click
  Consent-wall fallback:              2 clicks (explicit "Trotzdem")
  Duplicate:                          2 clicks ("Zum gespeicherten
                                        Artikel")

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 15:29:53 +02:00
Till JS
3357e88a1c feat(articles): new read-it-later module — save / read / highlight
Pocket-style module for saving arbitrary web URLs, extracting readable
content server-side via @mana/shared-rss (Readability + JSDOM), and
storing it AES-GCM encrypted in IndexedDB for offline reading.

M1 skeleton: Dexie v33 (articles, articleHighlights, articleTags),
crypto registry entries, module registration, app-registry entry with
orange icon, empty-state ListView. articleTags is a pure junction
into the existing globalTags system (appId 'tags') — same pattern as
noteTags, eventTags, placeTags.

M2 URL save + reader: POST /api/v1/articles/extract (one endpoint,
not two — client caches the preview payload to avoid a double
server fetch). AddUrlForm with scope-aware dedupe, DetailView with
ReaderView typography shell (serif/sans, light/sepia/dark, size
slider), auto-tracked reading progress with scroll restore.

M3 highlights: TreeWalker-based plain-text offset resolution
(lib/offsets.ts), highlights store, floating HighlightMenu with
create + edit modes, HighlightLayer orchestrator that wraps/unwraps
highlight spans whenever highlights or htmlVersion changes. Four
colours (yellow/green/blue/pink), optional notes, click-to-edit,
dark-mode-aware overlay colours.

Drive-by: removed stale 'pendingProposals' entry from the plaintext
allowlist — the table was dropped in Dexie v29 and the allowlist
audit was flagging it as a dead entry.

Plan: docs/plans/articles-module.md. M4 (tags + filter + progress),
M5 (news:type='saved' migration), M6 (AI tools), M7 (share target),
M8 (highlights view + stats) still open.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 16:20:23 +02:00
Till JS
76d11a84ee feat(auth): server-side tier gating via requireTier middleware
The JWT already carried a `tier` claim but nothing on the server read it
— AuthGate enforcement was client-only, so a valid JWT could hit paid
LLM/research endpoints regardless of the user's access tier.

- shared-hono authMiddleware now extracts `tier` into `c.userTier`,
  defaulting unknown/missing claims to `public` (never silently grants
  higher access).
- New `requireTier(minTier)` middleware + `hasTier`/`getTierLevel`
  helpers. Tier hierarchy (guest < public < beta < alpha < founder) is
  mirrored locally to avoid pulling the Svelte-facing shared-branding
  package into Bun services.
- Applied `requireTier('beta')` as defense-in-depth on resource-heavy
  apps/api modules (chat, context, food, guides, news-research, picture,
  plants, research, traces, who) and the MCP endpoint. Pure CRUD modules
  stay auth-only — access there is gated by ownership, not tier.
- DEV_BYPASS_AUTH now injects `userTier` (defaults to founder, override
  via DEV_USER_TIER).
- Authentication guideline documents the pattern + test suite covers
  hierarchy, passes-at-minimum, and rejection paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 17:38:06 +02:00
Till JS
9b8c69123c feat(wetter): add multi-model source comparison view
New "Quellen-Vergleich" tab on the weather page that fetches the same
location from 5 weather models in parallel (DWD ICON-D2, ICON-EU,
ECMWF IFS, NOAA GFS, Open-Meteo Best Match) and displays them stacked
for easy comparison of temperature, precipitation, and daily forecasts.

Adds /api/v1/wetter/compare endpoint and SourceComparison.svelte.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 13:57:05 +02:00
Till JS
24704e28b6 fix(wetter): mount routes before auth middleware
Weather data is public — no user-specific data involved. Move the
wetter route registration above authMiddleware() so requests don't
require a JWT token. Rate limiting still applies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 13:37:17 +02:00
Till JS
62aac6dfdb feat(wetter): add weather module with Open-Meteo, DWD alerts, and rain nowcast
New module providing weather data for the DACH region via three sources:
- Open-Meteo (DWD ICON-D2 model) for current conditions and 7-day forecast
- DWD warnings endpoint for severe weather alerts
- Rainbow.ai / Open-Meteo fallback for minute-level rain nowcast

Includes API proxy with in-memory caching, Svelte 5 UI with location
picker, hourly/daily forecast, alert cards, and precipitation bar chart.
Two AI tools (get_weather, get_rain_forecast) enable the companion to
answer weather questions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 03:46:15 +02:00
Till JS
acd7e0d6b0 docs: update architecture comparison — 5/10 roadmap items done
Update report to reflect all completed work:
- Matrix: streaming , tool registration updated to 29 tools + MCP
- §5.2 Streaming: marked done
- §5.3 Tool System: marked done
- §6 Table: items 1-3 + 5 struck through with commit refs
- §8 Fazit: updated gaps and recommendations

5 of 10 roadmap items complete in one session:
1. SSE Streaming, 2. Dynamic Tool Registry, 3. Budget Enforcement,
5. MCP Server Export (27/29 tools with DB ops), plus Tool Drift Fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:00:09 +02:00
Till JS
04c806fbb2 feat(mcp): implement remaining 19 tool handlers (27/29 total)
Complete tool handler coverage for the MCP server:

Todo: complete_tasks_by_title
Calendar: create_event (with timeBlock)
Notes: update_note, append_to_note, add_tag_to_note
Places: create_place, visit_place, get_places
Drink: log_drink, get_drink_progress, undo_drink
Food: log_meal, nutrition_summary
Journal: create_journal_entry
Habits: create_habit, log_habit (get_habits improved)
News: save_news_article

27 of 29 tools now have real implementations. Remaining 2
(research_news, get_current_location) need external service
calls that aren't available in the API server context.

Also updates architecture comparison report to mark MCP as done.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 14:08:57 +02:00
Till JS
e969324cc8 feat(mcp): Phase 2 — real DB operations for tool execution
Implement actual sync_changes reads and writes for MCP tool calls:

- sync-db.ts: Connection to mana_sync DB, RLS-scoped withUser(),
  readLatestRecords() for replaying sync state, writeRecord() for
  creating sync_changes entries
- executor.ts: 10 tool handlers implemented:
  - Reads: list_tasks, get_task_stats, list_notes, get_todays_events,
    get_contacts, get_habits
  - Writes: create_task, complete_task, create_note, create_contact
  - Remaining tools return helpful "not yet implemented" message
- server.ts: userId from auth context bound into MCP session via closure
- index.ts: typed Hono app with AuthVariables

Write pattern matches mana-ai: INSERT into sync_changes with
actor={kind:'system', source:'mcp-tool'}, client_id='mcp-server'.
Records appear on user devices on next sync cycle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 13:46:06 +02:00
Till JS
db4dd437bd feat(api): MCP server endpoint — expose AI tools to external clients
Mount an MCP (Model Context Protocol) server at /api/v1/mcp in the
unified Hono API. External clients like Claude Desktop, Cursor, and
VS Code Copilot can discover and call all 29 Mana tools via the
standard MCP protocol.

Architecture:
- WebStandardStreamableHTTPServerTransport for Bun/Hono compatibility
- AI_TOOL_CATALOG → MCP tool definitions with JSON Schema (via Zod)
- Stateful sessions with Mcp-Session-Id header
- Auth via existing authMiddleware (JWT or API key)

Phase 1 scope: tools/list returns all 29 tools with schemas,
tools/call acknowledges with descriptive messages. Phase 2 will add
actual DB reads/writes via sync_changes.

Usage:
  Claude Desktop config:
  {"mcpServers": {"mana": {"url": "http://localhost:3060/api/v1/mcp"}}}

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 13:37:52 +02:00
Till JS
fdd643f4b4 feat(news-research): RSS feed discovery, filter, and AI-context export
New sibling module to news/. Discovers topic-matched RSS feeds via
SearXNG (mana-search) or rel="alternate" probing of a site URL,
filters articles by keyword with a recency + title-match boost,
and exports the top hits as a markdown context block for the AI.

- API: /api/v1/news-research/{discover,validate,search,extract}
- Frontend: /news-research route + workbench ListView (compact card)
- Tool: research_news LLM tool (read-only, runs auto)
- Pin feeds → newsPreferences.customFeeds (encrypted) — covers the
  long-missing custom-RSS subscription gap; reading-list saves still
  go through articlesStore.saveFromUrl into the existing newsArticles
- shared-branding: new news-research entry + binoculars icon

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:31:07 +02:00
Till JS
b768a0ffce refactor(shared-rss): extract RSS parsing + Readability into one package
news-ingester and apps/api both shipped their own copy of rss-parser
+ jsdom + Readability glue. Single source now in packages/shared-rss.
Adds discoverFeeds (rel=alternate + common-paths probe) and validateFeed
which News Research will use. JSDOM virtualConsole is silenced once,
in the package, instead of in two parallel call sites.

- packages/shared-rss: parse, extract, discover, validate
- services/news-ingester: drop local parsers, depend on @mana/shared-rss
- apps/api: drop @mozilla/readability + jsdom direct deps, use shared

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:30:44 +02:00
Till JS
121a0c0a6f feat(api): POST /api/v1/context/import-url — crawler + optional LLM summary
New backend endpoint that wraps mana-crawler + mana-llm in a single
call so the Kontext "Aus URL" UI can hit one route:

- Starts a crawl job (single page or up-to-20-page deep crawl) via
  mana-crawler's /api/v1/crawl, polls status up to 90s, then fetches
  paginated results.
- When multiple pages are returned, joins them into one markdown
  document with H1-per-page section headers separated by ---.
- When summarize=true, routes the collected markdown through
  mana-llm/chat/completions with a system prompt that asks for
  "Überblick / Kernaussagen / Details" H2 structure in the source
  language. sanitizeSummary() strips the common local-LLM artefacts
  (```markdown fences, "Hier ist …:" preamble, stray leading H1)
  so the output drops cleanly into the Kontext doc. On summary
  failure the endpoint returns 502 rather than silently falling
  back to the raw crawl.
- Credits are validated + consumed via @mana/shared-hono/credits
  (1 credit crawl-only, 5 crawl+summary) under the new
  AI_CONTEXT_IMPORT_URL action.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:24:19 +02:00
Till JS
53b3746b98 refactor: rename nutriphi module to food (Essen)
Complete rename across the entire monorepo pre-launch:
- Module, routes, API, i18n, standalone landing app directories
- All code identifiers, display names, logo component
- German user-facing label: "Essen" (English brand stays "Food")
- Dexie table nutriFavorites -> foodFavorites
- Infra configs (docker-compose, cloudflared, nginx, wrangler)

Zero residue of nutriphi remains. No data migration needed (pre-launch).

Follow-up: run pnpm install, update Cloudflare DNS
(food.mana.how), rename Cloudflare Pages project.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 15:30:07 +02:00