till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-14 21:41:09 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
3913 commits 34 branches 38 tags 143 MiB
Commit graph

7 commits

Author SHA1 Message Date
Till JS
795b39e065 feat(forms): M10d headless wave-cron — server-worker + private internal_meta 
Echter Server-Cron für recurring forms — wave-send läuft jetzt
unabhängig von Owner-Tab-State. Bisheriger M10c webapp-side scheduler
bleibt als Belt-and-suspenders aktiv (idempotent).

Architektur:
1. **Owner-private internal_meta auf unlisted snapshots**
   - Drizzle: neue jsonb-column `internal_meta` (Drizzle migration
     0001_internal_meta.sql).
   - public-routes.ts strippt sie strukturell — die explicit select()-
     projection enthält sie nicht (recipients + sender würden sonst
     via share-link leaken).
   - publish-route akzeptiert sie im Body, persistiert auf insert +
     update.
   - ALLOWED_COLLECTIONS um 'lasts' und 'forms' erweitert (war ein
     latenter Bug — formsStore.setVisibility('unlisted') hätte ohne
     diese Ergänzung 400 zurückbekommen; M4b lief vermutlich nie
     end-to-end durch).

2. **shared-privacy publishUnlistedSnapshot**
   - PublishUnlistedOptions erweitert um optionales `internalMeta`.
     Forwarded an /api/v1/unlisted/:collection/:recordId body.

3. **Webapp formsStore**
   - lib/wave-mail.ts: buildFormInternalMeta(form, broadcastSettings)
     baut den Owner-Private-Blob: { kind, recurrence: {frequency,
     recipientEmails, lastSentAt}, sender: {fromEmail, fromName,
     replyTo, legalAddress}, formMeta: {title, description} }.
     Returns null wenn Voraussetzungen fehlen (kein recurrence, keine
     recipients, fehlende broadcast-settings).
   - stores/forms.svelte.ts: setVisibility / regenerateUnlistedToken /
     setUnlistedExpiry laden broadcastSettings via Dexie + decrypt,
     bauen internalMeta, übergeben an publishUnlistedSnapshot. Form
     wird vor dem buildFormInternalMeta-Call dekrypted.

4. **mana-mail internal bulk-send route**
   - createInternalRoutes(accountService, broadcastOrchestrator,
     maxRecipients) — Signature erweitert.
   - Neue POST /api/v1/internal/mail/bulk-send: gleicher Payload-shape
     wie user-facing /v1/mail/bulk-send aber userId aus Body statt
     JWT. X-Service-Key-gate sitzt bei /api/v1/internal/* prefix.
     Audit-trail trägt principalId aus Body. Cap = 5000 (gleicher
     Wert wie user-facing).

5. **apps/api forms wave-worker**
   - 5-min setInterval, advisory-lock-gated (key 0x464f5257 'FORW').
   - Tick: select snapshots WHERE collection='forms' AND
     internal_meta IS NOT NULL AND revoked_at IS NULL. Filter auf
     kind='forms-recurrence' + isWaveDue (lastSentAt + period <= now,
     never-sent fires sofort). Pro fälligem snapshot: build HTML/text
     mailbody (mirror webapp wave-mail-render), POST an mana-mail
     internal-bulk-send mit X-Service-Key + userId, dann jsonb_set
     auf internal_meta.recurrence.lastSentAt. Per-snapshot errors
     werden als console.warn geloggt, Tick läuft weiter.
   - Disable via FORMS_WAVE_WORKER_DISABLED=true (tests / multi-
     replica deployments).
   - Wired in apps/api/src/index.ts neben startArticleImportWorker().

Trade-offs:
- internal_meta wird beim setVisibility/regenerate/setExpiry frisch
  aus broadcast-settings gebaut — wenn der User später broadcast-
  settings ändert (zB neuer fromEmail) muss er das Form re-publishen
  damit die snapshot-internal_meta aktualisiert wird. Doc-it: zukünftiger
  Patch könnte ein "settings drift"-Warning ins UI surfacen.
- Worker-Update von lastSentAt geht NICHT zurück in den webapp-form
  (settings.recurrence.lastSentAt ist verschlüsselt, server kann
  nicht schreiben). Owner-UI zeigt ältere lastSentAt von manuellen
  Sends; auto-cron-sends sind in den Server-Logs sichtbar. Future
  patch: GET /api/v1/forms/:id/recurrence-status (auth) gibt das
  snapshot.internal_meta zurück, UI rendert Auto-Cron-State.
- Webapp-side wave-scheduler (M10c) läuft parallel weiter — wenn
  Owner-Tab offen ist, kann beides feuern. Idempotent durch
  lastSentAt-check (weekly/monthly buckets), aber theoretisch könnte
  double-fire passieren wenn die Calls innerhalb 1ms versetzt sind.
  Real-world ignorierbar; future patch: scheduler liest jetzt
  internal_meta.lastSentAt vom server-side state.

apps/api buildet (1776 modules). mana-mail buildet (523 modules).
svelte-check 0 errors in forms/. Forms-Tests 70/70 unverändert.

DB-Migration 0001_internal_meta.sql muss manuell appliziert werden
(siehe feedback memory: hand-authored SQL migrations sind nicht in
pnpm setup:db).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-06 17:18:05 +02:00
Till JS
92bee0d71a feat(unlisted): M8.1 — backend foundation for shareable-link snapshots 
First milestone of the unlisted-share rollout plan (docs/plans/
unlisted-sharing.md). Adds the server-side infrastructure that backs
`visibility='unlisted'` — previously the flag was stamped locally but
led nowhere. After this commit, a token points at an actual snapshot
the SSR share-page will render (M8.3+).

Scope: backend only. No client-side publish/revoke calls yet, no
share-route, no UI. That lands in M8.2/M8.3. Anyone hitting the
endpoints manually with curl can exercise the full publish-fetch-
revoke cycle.

Changes:
- New pgSchema `unlisted` with table `snapshots`:
    token (pk, 32-char base64url)
    user_id, space_id, collection, record_id, blob (jsonb)
    created_at, updated_at, expires_at (nullable), revoked_at
  Partial unique index on (user_id, collection, record_id) WHERE
  revoked_at IS NULL so one record has at most one active token.
  Partial btree on expires_at for the cron-cleanup path.
- Hand-authored SQL migration `apps/api/drizzle/unlisted/0000_init.sql`
  (manual-apply per the repo's feedback_api_hand_authored_migrations
  memory). Already applied to the local mana_platform.
- Drizzle schema `apps/api/src/modules/unlisted/schema.ts`. All id
  fields are `text` not uuid — Better-Auth nanoids aren't UUIDs, same
  trap we hit with the website module's publish bug.
- mana-api module `apps/api/src/modules/unlisted/`:
    POST   /api/v1/unlisted/:collection/:recordId (auth)
      Body: { spaceId, blob, expiresAt? }. Re-publish reuses the
      existing active token (by (user,collection,record) lookup); a
      revoke-then-republish mints a fresh token row. Response includes
      a fully-qualified share URL built from Origin/Referer/env.
    DELETE /api/v1/unlisted/:collection/:recordId (auth)
      Soft-revoke. Idempotent — already-revoked returns
      { revoked: 0 } cleanly so client stores can call it
      unconditionally on setVisibility-away.
    GET    /api/v1/unlisted/public/:token (public)
      Rate-limited 20/min/token + 60/min/ip so token enumeration is
      impractical. 404 for unknown, 410 Gone for revoked or expired.
      Cache-Control: private, max-age=60 + X-Robots-Tag: noindex for
      SEO isolation. Returns { token, collection, blob, createdAt,
      updatedAt, expiresAt }.
- ALLOWED_COLLECTIONS hardcoded allowlist in POST handler
  (events, libraryEntries, places — the M8.3+M8.4 scope). Unknown
  collection -> 400 COLLECTION_NOT_ALLOWED. Keeps the schema honest
  about what the server accepts.
- drizzle.config extended to include the new schema in managed
  migrations.
- index.ts wires unlistedPublicRoutes pre-auth (before
  authMiddleware) and unlistedRoutes post-auth.

Verified:
- Migration applied to mana_platform — `unlisted.snapshots` exists
  with both partial indexes.
- pnpm run type-check (api): clean
- pnpm run validate:all: theme-tokens, theme-parity, crypto-registry,
  encrypted-tools all green
- URL build uses Origin/Referer before the env fallback so dev
  (http://localhost:5173) and prod (https://mana.how) both work
  without env churn.

Next: M8.2 — shared-privacy client helper + SharedLinkControls
component.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 17:12:13 +02:00
Till JS
c404db5b6e fix(website): publish failed with uuid type error on Better-Auth ids 
published_by, created_by, and space_id were declared as uuid, but
Mana user + space ids are Better-Auth nanoids stored as text. The
insert into website.published_snapshots raised `invalid input syntax
for type uuid` and Hono swallowed it as a generic 500.

Changes:
- schema.ts: uuid -> text on the three columns
- 0003_fix_id_types.sql: ALTER COLUMN on existing installs
- publish.ts: replace UUID regex on X-Mana-Space with a nanoid-shaped
  check (it was silently nulling valid space ids)
- publish.ts: log + return the actual error message on the 500 path
  so the next unhandled failure is visible instead of opaque

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 21:43:57 +02:00
Till JS
3eca5ac201 feat(website): M6 — subdomain publish + custom-domain foundation 
SvelteKit hook + new DB table + founder-gated API + UI section. Ships
the code path for public-site routing on {slug}.mana.how and custom
hostnames. Cloudflare SaaS Hostnames integration is stubbed — see
plan §M6 "Offene Enden".

apps/api/src/modules/website:
- schema.ts: new `customDomains` table. Fields: id, site_id, hostname
  (unique), status (pending | verifying | verified | failed),
  verification_token, dns_target, verified_at.
- drizzle/website/0002_custom_domains.sql: manual migration with
  partial unique index on (hostname) WHERE status='verified'.
- domains.ts (new, authenticated + founder-gated via
  `requireTier('founder')`): POST/GET/DELETE /sites/:id/domains,
  POST /sites/:id/domains/:domainId/verify. Verify runs CNAME + TXT
  checks via node:dns/promises with an apex-domain A-record fallback.
  Reserved-hostname list prevents users from binding mana.how subdomains.
- public-routes.ts: new GET /public/resolve-host?host= — unauthenticated
  resolver used by hooks.server.ts. Returns { slug, siteId } only for
  verified bindings tied to a currently-published site.

apps/mana/apps/web/src/hooks.server.ts:
- After the existing https/app-subdomain guards, a new
  `resolveWebsiteRewrite()` step rewrites `event.url.pathname`:
    {slug}.mana.how/path → /s/{slug}/path     (pure string)
    custom-host.com/path → /s/{resolved}/path (API call, 60s LRU)
- Browser URL stays on the custom host — this is a server-side rewrite,
  not a 302. APP_SUBDOMAINS + RESERVED_WEBSITE_SUBDOMAINS win over
  website routing. Localhost and apex mana.how are skipped.

apps/mana/apps/web/src/lib/modules/website:
- domains.ts (new): typed client for list/add/verify/remove. Handles
  200 + expected 400 (verification-failed) separately.
- components/DomainsSection.svelte: add-input, per-domain status pill,
  DNS-instructions box (CNAME + TXT with copy-to-clipboard), Verify
  button. Mounted inside SiteSettingsDialog as its own section — the
  existing theme/footer controls stay put.

docs/plans/website-builder.md:
- M6 checklist updated with what shipped vs. ops-gap (CF SaaS).
- `mana-landing-builder` consolidation: DECIDED to keep parallel. Four
  reasons in the plan. Revisit-criterion stated.
- Shipping log table seeded with M1→M6 commits.

Validation:
- pnpm run validate:all: 6/6 gates green
- pnpm run check (web): 0 errors, 0 warnings
- apps/api type-check: green

Apply schema with:
  psql "$DATABASE_URL" -f apps/api/drizzle/website/0002_custom_domains.sql

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 15:29:42 +02:00
Till JS
57be0f61b1 feat(website): M4 — forms + moduleEmbed 
Adds two new block types and the server-side infrastructure for
untrusted input + cross-module data embedding.

Forms:
- packages/website-blocks/src/form: declarative fields (text, email,
  tel, url, textarea, number) with required / maxLength / placeholder
  per field. Honeypot hidden input in the renderer; public-mode POST
  to a same-origin SvelteKit proxy that forwards to mana-api.
- apps/api: website.submissions table (schema.ts + 0001_submissions.sql)
  + POST /public/submit/:siteSlug/:blockId. Loads the current published
  snapshot, finds the form block, validates payload against its
  declared fields (trim, type check, length cap), rejects honeypot
  submissions silently, rate-limits per IP (10 / 5 min) in-memory.
  Unknown keys are dropped — clients can only submit declared fields.
- Owner-facing: GET/DELETE /sites/:id/submissions + SubmissionsView
  component + /(app)/website/[siteId]/submissions route. Shows
  incoming submissions with status pill + payload preview + delete.
- apps/mana/.../routes/s/[siteSlug]/__submit/[blockId]/+server.ts:
  same-origin proxy so form posts don't trigger CORS and IP / user-
  agent headers are forwarded via SvelteKit's trusted getClientAddress.

M4 first-pass does NOT wire target-module delivery (contacts / notify).
Submissions stay in the inbox until owner-side tool handlers land
(M4.x). `target` enum is intentionally `['inbox']` only for now.

moduleEmbed:
- packages/website-blocks/src/moduleEmbed: source dropdown
  (picture.board | library.entries), max-items, layout (grid | list),
  optional filter object. The `resolved` field on props is populated at
  publish time by the editor-side resolver — public renderer reads it
  directly, no Dexie / API round-trip needed.
- apps/mana/.../website/embeds.ts: per-source resolvers. picture.board
  enforces `isPublic=true`; library.entries respects filter.isFavorite
  / kind / status so owners can expose a subset (e.g. "my favorites").
- buildSnapshot() walks the tree after assembly and fills in
  block.props.resolved for every moduleEmbed. Publish slower, public
  visits fast. No cross-service call at render time.

Validation:
- pnpm run validate:all: 6/6 gates green
- pnpm run check (web): 0 errors, 0 warnings
- apps/api type-check: green

Apply Postgres with:
  psql "$DATABASE_URL" -f apps/api/drizzle/website/0001_submissions.sql

Plan: docs/plans/website-builder.md (M4 shipped)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 14:36:52 +02:00
Till JS
54a12ffd5c feat(webapp): wire isParallelSafe in Companion chat + Mission runner 
Enables the M1 parallel-reads optimisation on the webapp side. Both
consumers of runPlannerLoop pass an isParallelSafe predicate derived
from the tool catalog:

  isParallelSafe: (name) =>
    AI_TOOL_CATALOG_BY_NAME.get(name)?.defaultPolicy === 'auto'

Auto-policy tools (list_tasks, get_habits, nutrition_summary, …) run
via Promise.all in batches of 10 when the LLM fans them out in one
round. Propose-policy tools — which surface to the user as Proposal
cards — stay sequential so intent ordering in the inbox is preserved
and pre-execute guardrails can reason about prior-step state.

Tests: 31 existing companion + mission tests pass unchanged; the
parallel path is exercised via the new loop.test.ts cases shipped
with the M1 commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-23 14:11:24 +02:00
Till JS
e82851985b feat(questions): deep-research module — mana-search + mana-llm pipeline 
End-to-end deep-research feature for the questions module: a fire-and-
forget orchestrator in apps/api that plans sub-queries with mana-llm,
retrieves sources via mana-search (with optional Readability extraction),
and streams a structured synthesis back to the web app over SSE.

Backend (apps/api/src/modules/research):
- schema.ts: pgSchema('research') with research_results + sources
- orchestrator.ts: three-phase pipeline (plan / retrieve / synthesise)
  with depth-aware config (quick=1×, standard=3×, deep=6× sub-queries)
- pubsub.ts: in-process event bus, single-node, swappable for Redis
- routes.ts: POST /start (202, fire-and-forget), GET /:id/stream (SSE),
  POST /start-sync (test only), GET /:id, GET /:id/sources
- Credit gating via @mana/shared-hono/credits — validate up-front,
  consume best-effort on `done`. Failed runs cost nothing.

Helpers (apps/api/src/lib):
- llm.ts: llmJson() + llmStream() over mana-llm OpenAI-compat API
- search.ts: webSearch() + bulkExtract() over mana-search Go service
- responses.ts: shared errorResponse / listResponse / validationError

Schema deployment:
- drizzle.config.ts (research-scoped) + drizzle/research/0000_init.sql
  hand-authored migration, deployable via psql -f or drizzle-kit push.
- drizzle-kit added as devDep with db:generate / db:push scripts.

Web client (apps/mana/apps/web/src/lib/api/research.ts):
- Typed start() / get() / listSources() / streamProgress(). The stream
  uses fetch + ReadableStream (not EventSource) so we can attach the
  JWT via Authorization header. Special-cases 402 for friendly toast.
- New PUBLIC_MANA_API_URL plumbing in hooks.server.ts + config.ts.

Module store (modules/questions/stores/answers.svelte.ts):
- New write-side store with createManual / startResearch / accept /
  softDelete. startResearch creates an optimistic empty answer, opens
  the SSE stream, debounces token deltas in 100ms batches into the
  encrypted local row, and on `done` replaces the streamed text with
  the parsed { summary, keyPoints, followUps } payload + citations
  resolved against research.sources.id.

Citation rendering (modules/questions/components/AnswerCitations.svelte):
- Tokenises [n] markers in the answer body into clickable pills with
  hover popovers showing title / host / snippet / external link.
- Lazy-loaded via a session-scoped source cache (stores/sources.svelte.ts)
  that deduplicates concurrent fetches.

UI (routes/(app)/questions/[id]/+page.svelte):
- Recherche card with three-state button (start / cancel / re-run),
  animated phase indicator, source counter.
- Confirmation dialog warning about web/LLM transmission since the
  question itself is locally encrypted.
- Toasts for success / error / cancel via @mana/shared-ui/toast.
- Re-run flow soft-deletes prior research-driven answers but keeps
  manual ones intact.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-08 22:15:35 +02:00