Commit graph

17 commits

Author SHA1 Message Date
Till JS
b297f68ee4 fix(articles, mana-ai): rollout-block hardening for sync_changes projections
Four cross-cutting fixes that make the bulk-import worker safe to run
under real production load. All four were called out as live-rollout
risks in the post-ship review of docs/plans/articles-bulk-import.md.

#1 — Same fieldMetaTime bug fixed in mana-ai
   The articles fix in 054b9e5be hoists the helper to its own file
   `apps/api/src/modules/articles/field-meta.ts`. The same naive
   `rowFM[k] >= localTime` LWW comparison existed in three more
   projections under services/mana-ai (missions-projection,
   snapshot-refresh, agents-projection). Once any F3 stamp lands
   beside a legacy-string stamp, the comparison evaluates
   `'[object Object]' >= 'ISO-…'` (false) and the older value wins.
   New `services/mana-ai/src/db/field-meta.ts` — same helper,
   deliberately duplicated (each service treats sync_changes as a
   read-only event log; sharing infra across services is out of
   scope here). All 61 mana-ai bun tests still pass.

#2 — Stale 'extracting' items recycle
   If the worker dies mid-fetch (OOM, pod restart), items stay in
   state='extracting' forever and the job never completes. New sweep
   at the start of `processOneJob`: items whose lastAttemptAt is
   older than 5 minutes get bounced back to 'pending' so the next
   tick re-claims them. STALE_EXTRACTING_MS tuned for the 15s
   shared-rss fetch + JSDOM-parse worst case.

#3 — Pickup-row GC
   Every 30 ticks (~once per minute) the worker hard-deletes
   articleExtractPickup rows older than 24h. Without this a stuck
   pickup-consumer (all tabs closed, Web-Lock mismatch) would let
   sync_changes accumulate without bound. Logs the row count when
   non-zero so we can spot stuck consumers in the wild.

#4 — DRY consent-wall heuristic
   Identical CONSENT_KEYWORDS + threshold lived in routes.ts AND
   import-extractor.ts. Hoisted to
   `apps/api/src/modules/articles/consent-wall.ts`; both call sites
   now share one heuristic.

Plan: docs/plans/articles-bulk-import.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 00:53:39 +02:00
Till JS
6bb9d77be9 feat(sync): F3 — drop updatedAt as a synced data field
Removes `updatedAt` from the wire protocol and from every Local-prefixed
record type. Replaced by two orthogonal mechanisms — deriveUpdatedAt()
for read-side public-facing values, _updatedAtIndex shadow for indexed
sorts.

Local-side:
- New `_updatedAtIndex` shadow column. Stamped by the Dexie creating /
  updating hook on every write. Stripped from the pending-change payload
  so it never travels to mana-sync. Indexed in Dexie v53 on the 22 tables
  that previously indexed `updatedAt`.
- `deriveUpdatedAt(record)` in sync.ts returns max(__fieldMeta[*].at) so
  the public-facing Task / Note / etc. shape keeps an `updatedAt: string`
  property without holding it as data.
- Type-converters across ~60 module/queries.ts and types.ts files now
  call `deriveUpdatedAt(local)` instead of reading `local.updatedAt`.

Module-store sweep:
- Regex codemod removed `updatedAt: new Date().toISOString()` /
  `: now` / `: now()` / `: nowIso()` stamping from 121 store files
  (~382 call sites total). Single-property update calls
  (`{ updatedAt: now }`) collapsed to `{}`; touch-only patterns
  (writing/drafts, writing/generations) kept the call as a no-op
  because the hook now stamps `_updatedAtIndex` automatically on
  any Dexie modification.
- Local* interfaces stripped of `updatedAt: string` (43 types.ts files).
  Public-facing types (Task, Note, Mission, Agent, …) keep
  `updatedAt: string` as a computed read-side property.
- Companion's chat conversation now sorts on a real
  `lastMessageAt` data field instead of touching `updatedAt`.
- Session-only stores (times/session-alarms, session-countdown-timers)
  stamp `updatedAt: now` directly because they're not in Dexie and
  have no field-meta layer to derive from.

Sync engine:
- applyServerChanges sets `_updatedAtIndex` itself when applying
  server changes (max of server-field times for updates, recordTime
  for inserts) so server-replays land orderable.
- Dropped the legacy `localUpdatedAt` fallback — every record now has
  `__fieldMeta`, the per-field at is the canonical source.
- Soft-delete tombstone path stops stamping `updatedAt: serverTime`,
  uses `_updatedAtIndex` instead.

Server-side:
- mana-ai iteration-writer no longer emits `updatedAt` in
  sync_changes.data; receivers derive it from the field-meta map.
- mana-sync types: no change (the wire format already uses
  `field_meta` / `at` from F1).

Out of scope: backend Drizzle schemas (mana-credits, mana-events, …)
keep their `updated_at` columns. Those are pure server-internal — not
part of the sync_changes / __fieldMeta mechanism F3 cleans up.

Tests + checks:
- 0 svelte-check errors over 7652 files.
- 29/29 sync.test.ts (vitest).
- 61 mana-ai bun tests.
- mana-sync go test ./... cached green.

Plan: docs/plans/sync-field-meta-overhaul.md F3.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 23:12:22 +02:00
Till JS
7766ea5021 docs(plans): mark llm-fallback-aliases SHIPPED, add M-by-M commit table
All 5 milestones landed today in one continuous session: registry,
health cache, fallback router, observability, and consumer migration.
115 service-side tests, validator covers 2538 files.
2026-04-26 21:27:57 +02:00
Till JS
2a18cb5ee4 feat(mana-ai): v0.7 — cross-tick Deep Research Max pre-planning
Opt-in path for missions that want Gemini Deep Research Max (up to 60 min
per task) instead of the shallow RSS pre-research. Because Max runs well
past a single 60-second tick, the state is carried across ticks:

  tick N:   submit → INSERT mission_research_jobs row → skip planner
  tick N+k: poll → still running → skip planner (metric pending_skips)
  tick N+m: poll → completed → inject as ResolvedInput, DELETE row, plan

- ManaResearchClient talks to mana-research's new internal
  /v1/internal/research/async endpoints with X-Service-Key +
  X-User-Id. Graceful-null on transport errors so a flaky
  mana-research never crashes the tick loop.
- New table mana_ai.mission_research_jobs with PK (user_id, mission_id)
  — presence is the "pending" flag; delete-on-terminal keeps queries
  trivial.
- handleDeepResearch() encapsulates the state machine; planOneMission
  now returns a discriminated union (planned | skipped | failed) so
  "research pending" isn't miscounted as a parse failure.
- Opt-in at TWO gates to keep cost in check ($3–7/task, 1500 credits
  per run):
    1. MANA_AI_DEEP_RESEARCH_ENABLED=true server-side (default off)
    2. DEEP_RESEARCH_TRIGGER regex matches the mission objective
       (strict: "deep research", "tiefe recherche", "umfassende
       recherche", "hintergrundrecherche", "deep dive")
  Falls back to shallow RSS when either gate fails or the submit
  errors upstream.
- Prom metrics: mana_ai_research_jobs_{submitted,completed,failed}_total
  labelled by provider, plus _pending_skips_total.
- docker-compose wires MANA_RESEARCH_URL + the opt-in flag and adds
  mana-research to depends_on.
- Full write-up with real API response shape (outputs plural, not
  OpenAI-style), step-3 MCP-server plan (security-gated, not built),
  ops + kill-switch: docs/reports/gemini-deep-research.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 17:56:06 +02:00
Till JS
acd7e0d6b0 docs: update architecture comparison — 5/10 roadmap items done
Update report to reflect all completed work:
- Matrix: streaming , tool registration updated to 29 tools + MCP
- §5.2 Streaming: marked done
- §5.3 Tool System: marked done
- §6 Table: items 1-3 + 5 struck through with commit refs
- §8 Fazit: updated gaps and recommendations

5 of 10 roadmap items complete in one session:
1. SSE Streaming, 2. Dynamic Tool Registry, 3. Budget Enforcement,
5. MCP Server Export (27/29 tools with DB ops), plus Tool Drift Fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:00:09 +02:00
Till JS
ce57e11950 feat(mana-ai): server-side token budget enforcement per agent
Implement rolling 24h token budget enforcement in the mana-ai tick loop.
Agents with maxTokensPerDay set are now rate-limited server-side.

Changes:
- PlannerClient: extract usage.total_tokens from mana-llm response
- planOneMission: return {plan, tokensUsed} tuple
- tick loop: check getAgentTokenUsage24h() before planning; skip with
  'skipped-budget' decision if over limit
- tick loop: record token usage after successful plan via
  recordTokenUsage() INSERT into mana_ai.token_usage
- migrate.ts: new mana_ai.token_usage table with rolling window index
- metrics.ts: mana_ai_tokens_used_total counter (by agent_id)

Budget flow:
  Agent.maxTokensPerDay = 50000
  → tick checks: SELECT SUM(tokens_used) WHERE ts > now()-24h
  → if sum >= 50000: skip mission, emit skipped-budget metric
  → else: plan mission, INSERT token_usage row

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 14:41:31 +02:00
Till JS
f7426ab40f feat(ai): policy is read from the owning agent (Phase 4)
Until now AiPolicy lived as a user-global setting consulted for every
AI action. With agents as the principal unit of AI behavior, policy
belongs on the agent — different agents can be aggressive about tasks
but conservative about calendar edits, etc.

Webapp (tools/executor.ts):
- When an AI actor invokes a tool, the executor looks up the owning
  agent via getAgent(actor.principalId) and passes agent.policy into
  resolvePolicy. Falls back to DEFAULT_AI_POLICY when the agent record
  is missing (legacy write, deleted agent, race) so no tool call can
  silently bypass the propose/deny path.
- resolvePolicy already accepted an optional policy arg, so the call
  site change is a single line plus the agent load.

Server (mana-ai):
- ServerAgent gains an optional policy field, projected off the same
  plaintext JSONB that the webapp writes.
- Tick loop filters AI_AVAILABLE_TOOLS through filterToolsByAgentPolicy
  before passing them to the planner prompt. Resolution order mirrors
  the webapp: tools[name] → defaultsByModule → defaultForAi; 'deny'
  drops the tool so the LLM never even sees it.

Phase 5 will surface a per-agent policy editor on the agent-detail
UI. Until then all agents inherit DEFAULT_AI_POLICY (baked in during
createAgent), which means no behavior change for existing users —
every tool that was 'propose' before is still 'propose' now, just
reached via agent.policy instead of the user-level singleton.

Tests: mana-ai 41/41, webapp svelte-check clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:43:04 +02:00
Till JS
0af50f0166 feat(mana-ai): agent-aware tick loop + snapshot projection (Phase 3)
Third phase of the Multi-Agent Workbench. The background mission
runner now respects the owning Agent: agent state gates whether
a mission runs, concurrency is capped per-agent, and server-produced
iterations carry the agent's identity as their Actor.

Data layer:
- db/migrate.ts: new mana_ai.agent_snapshots table (mirrors
  mission_snapshots) with indexes on (user_id, last_applied_at) and
  a partial index on active agents.
- db/agents-projection.ts: refreshAgentSnapshots (incremental LWW
  replay over sync_changes appId='ai' table='agents') +
  loadActiveAgents / loadAgent helpers. mergeRaw exported for tests.
- db/missions-projection.ts: ServerMission.agentId + projection
  reads the JSONB field (undefined for legacy missions).

Tick integration (cron/tick.ts):
- Refreshes both snapshot tables on every pass (parallel).
- Per-user in-tick agent cache (Map<userId, Map<agentId, Agent>>)
  so N missions for one user hit the DB once.
- Gate order: agent archived → skip silently; agent paused → skip;
  per-agent maxConcurrentMissions exhausted this tick → defer to next.
  All skip paths bump mana_ai_agent_decisions_total{decision}.
- Prompt injection: withAgentContext prepends an <agent_context>
  block to the system prompt with the agent's name + role, and
  plaintext systemPrompt + memory when available. Ciphertext
  (enc:1:… blobs) are skipped — server has no key by design. Mirrors
  the Mission Grant privacy stance: encrypted context belongs to the
  foreground runner.

Iteration writer (db/iteration-writer.ts):
- New optional `agent` + `iterationId` + `rationale` inputs.
- When agent is present, the sync_changes row is stamped with a
  makeAgentActor actor (principalId=agentId, displayName=agent.name)
  so the webapp timeline groups the write under the right agent.
- Falls back to an AI actor with LEGACY_AI_PRINCIPAL + 'Mana' when
  the mission has no owning agent; ultimate fallback to the
  mission-runner system actor when iterationId is also missing.

Metrics:
- mana_ai_agent_decisions_total{decision=ran|skipped-paused|
  skipped-archived|skipped-concurrency}. Missions without an agent
  don't produce this metric — plansWrittenBackTotal is the universal
  "did we run" counter.

Tests: 41/41 (was 35) including 6 new cases for the agent LWW merge.
mana-ai type-check clean. Webapp svelte-check: 0 errors (4 unrelated
warnings in a different module).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:46:57 +02:00
Till JS
1771063df4 refactor(actor): identity-aware Actor for Multi-Agent Workbench (Phase 1)
Foundation for the Multi-Agent Workbench roadmap
(docs/plans/multi-agent-workbench.md). Every event, record, and
sync_changes row now carries a principal identity + cached display
name in addition to the three-kind discriminator.

Shape change (source of truth in @mana/shared-ai):
  Before: { kind: 'user' | 'ai' | 'system', ...kind-specific fields }
  After:  discriminated union on kind, with
            - common:  principalId, displayName
            - 'user':  principalId = userId
            - 'ai':    principalId = agentId + missionId/iterationId/rationale
            - 'system': principalId = one of SYSTEM_* sentinel strings
                        ('system:projection', 'system:mission-runner', etc.)

Key design calls (from the plan's Q&A):
- System sub-sources get distinct principalIds (not a shared 'system'
  bucket) — lets Workbench filter + revert distinguish projection
  writes from migration writes from server-iteration writes
- displayName cached on the record so renaming an agent doesn't
  rewrite history
- normalizeActor() compat shim fills principalId/displayName on
  legacy rows with 'legacy:*' sentinels so historical events never
  crash the timeline

New exports:
- BaseActor / UserActor / AiActor / SystemActor (narrowed types)
- makeUserActor, makeAgentActor, makeSystemActor (factories with
  typed return)
- SYSTEM_PROJECTION, SYSTEM_RULE, SYSTEM_MIGRATION, SYSTEM_STREAM,
  SYSTEM_MISSION_RUNNER (principalId constants)
- LEGACY_USER_PRINCIPAL, LEGACY_AI_PRINCIPAL, LEGACY_SYSTEM_PRINCIPAL
- isUserActor / isFromMissionRunner predicates

Webapp:
- data/events/actor.ts now re-exports from shared-ai, keeps runtime
  ambient-context (runAs, getCurrentActor) local
- bindDefaultUser(userId, displayName) lets the auth layer replace
  the legacy placeholder with the real logged-in user actor at login
- Mission runner + server-iteration-staging stamp LEGACY_AI_PRINCIPAL
  as the agentId placeholder — Phase 2 will thread the real agent
- Streaks projection uses makeSystemActor(SYSTEM_PROJECTION)
- All test fixtures migrated to factories

Service:
- mana-ai/db/iteration-writer.ts stamps makeSystemActor(
  SYSTEM_MISSION_RUNNER) instead of the old { kind:'system',
  source:'mission-runner' } shape. Phase 3 will switch this to an
  agent actor per mission.

Tests: 26 shared-ai + 21 webapp vitest + 35 mana-ai — all green.
svelte-check: 0 errors, 0 warnings.

No behavior change; purely a type + shape upgrade. Old sync_changes
rows parse via the normalizeActor compat shim at read time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:13:57 +02:00
Till JS
74bbfda212 feat(ai): Mission Grant consent UI + Workbench audit tab
Phase 3 — user-facing side of the Mission Key-Grant rollout. Users
can now opt into server-side execution, revoke it, and inspect every
decrypt the runner has performed.

Webapp:
- MissionGrantDialog explains the scope (record count, tables, TTL,
  audit visibility, revocation) and calls requestMissionGrant. Error
  paths render distinctly for ZK, not-configured, missing vault.
- Mission detail shows a Server-Zugriff box with status pill
  (aktiv/abgelaufen/nicht erteilt), Neu-erteilen + Zurückziehen
  buttons. Only renders for missions with at least one encrypted-
  table input.
- store.ts: setMissionGrant / revokeMissionGrant helpers, Proxy-
  stripped like the rest of the store's writes.
- Workbench adds a Timeline/Datenzugriff tab switch. Audit tab queries
  the new GET /api/v1/me/ai-audit endpoint, renders decrypt events
  with color-coded status pills (ok/failed/scope-violation) and
  stable reason strings.
- getManaAiUrl() added to api/config for the audit fetch.

mana-ai:
- GET /api/v1/me/ai-audit (JWT-gated via shared-hono authMiddleware)
  backed by readDecryptAudit() — withUser + RLS double-gate so a user
  can only read their own rows.
- Limit capped at 1000, newest-first.

Missions without a grant continue to work exactly as before; the
grant UI is purely additive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 13:53:11 +02:00
Till JS
a6d51afbc9 feat(mana-ai): encrypted resolver + tick uses Mission Grant to decrypt scoped inputs
Phase 2 of Mission Key-Grant. The tick loop now honours a mission's
grant by unwrapping the MDK and passing it + the record allowlist into
the resolvers. Encrypted modules (notes, tasks, calendar, journal,
kontext) resolve server-side instead of returning null.

- crypto/decrypt-value.ts: mirror of webapp AES-GCM wire format
  (enc:1:<iv>.<ct>) — read-only, server never wraps
- db/resolvers/encrypted.ts: factory + 5 concrete resolvers. Scope-
  violation bumps a metric + writes a structured audit row, decrypt
  failures same. Zero-decrypt (no grant, or record absent) = silent
  null, no audit noise.
- db/audit.ts: best-effort append to mana_ai.decrypt_audit; write
  failures never cascade into tick failures.
- cron/tick.ts: buildResolverContext unwraps grant per mission; MDK
  reference only lives for the scope of planOneMission.
- ResolverContext plumbed through resolveServerInputs; existing goals
  resolver unchanged semantically.
- Metrics: mana_ai_decrypts_total{table}, mana_ai_grant_skips_total
  {reason}, mana_ai_grant_scope_violations_total{table} (alert > 0).

Missions without a grant still run exactly as before — plaintext
resolvers fire, encrypted ones short-circuit to null. No behaviour
regression for existing users.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 13:42:31 +02:00
Till JS
9a3025fed8 feat(ai,auth): Mission Grant endpoint + unwrap helper + audit table
Phase 1 of the Mission Key-Grant rollout. Webapp can now request a
wrapped per-mission data key; mana-ai can unwrap and (Phase 2) use it.

mana-auth:
- POST /api/v1/me/ai-mission-grant — HKDF-derives MDK from the user
  master key, RSA-OAEP-2048-wraps with the mana-ai public key, returns
  { wrappedKey, derivation, issuedAt, expiresAt }
- MissionGrantService refuses zero-knowledge users (409 ZK_ACTIVE) and
  returns 503 GRANT_NOT_CONFIGURED when MANA_AI_PUBLIC_KEY_PEM is unset
- TTL clamped to [1h, 30d]

mana-ai:
- configureMissionGrantKey + unwrapMissionGrant with structured failure
  reasons (not-configured / expired / malformed / wrap-rejected)
- mana_ai.decrypt_audit table + RLS policy scoped to
  app.current_user_id — append-only row per server-side decrypt attempt
- MANA_AI_PRIVATE_KEY_PEM env slot; absent = grants silently disabled

No existing behaviour changes: missions without a grant run exactly as
before. Grant flow is wired end-to-end but unused until Phase 2 lands
the encrypted resolver.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 13:41:59 +02:00
Till JS
8fd9b7da79 perf(mana-ai): materialize mission snapshots, drop per-tick full replay
Replaces the O(N sync_changes) LWW replay in every tick with an
incremental snapshot table refresh. Each tick now applies only the
delta since the last run, then runs a single indexed SELECT on the
snapshot table to find due missions.

- `db/migrate.ts` — idempotent migration. Creates `mana_ai` schema and
  `mana_ai.mission_snapshots` table on boot. Partial index on
  active+nextRunAt powers the tick's "due" query.
- `db/snapshot-refresh.ts`
  - `refreshSnapshots(sql)` one-pass: joins sync_changes and snapshots
    on (user_id, mission_id), picks out pairs whose source max
    created_at exceeds the snapshot cursor. Per-pair refresh wrapped
    in `withUser` for RLS scoping on the source SELECT.
  - Bootstrap: missing snapshot rows seed from a full replay of their
    mission's history; subsequent ticks apply only the delta.
  - Delete tombstones purge the snapshot row.
- `db/missions-projection.ts` `listDueMissions` — single SELECT against
  `mana_ai.mission_snapshots` with an indexed WHERE. Dropped the legacy
  cross-user scan + per-user two-phase read (unused now). `mergeAndFilter`
  stays for its existing test coverage.
- `cron/tick.ts` calls `refreshSnapshots` before `listDueMissions` and
  logs when the refresh actually applied rows. No behaviour change
  externally.
- `index.ts` awaits `migrate()` on boot (top-level `await` — Bun
  supports it natively).

Closes the last item on the AI-Workbench roadmap's "future work" list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 01:28:24 +02:00
Till JS
ad1659f036 refactor(mana-ai): RLS-scope mission reads via per-user two-phase query
Closes the "cross-user scan" caveat on the mission read path. The
earlier implementation pulled every aiMissions row server-wide and
partitioned by user_id in memory — fine for a pre-launch single-user
deploy, not a cross-user infrastructure.

New flow:

  1. `listMissionUsers(sql)` — one cross-user DISTINCT query. This is
     the ONLY surface that still reads across users; documented as
     requiring BYPASSRLS on the service's DB role (or ownership without
     FORCE).
  2. `listDueMissionsForUser(sql, userId, now)` — RLS-scoped via
     `withUser(sql, userId, tx => ...)` just like the write path in
     `iteration-writer.ts`. Defense-in-depth: even if the SELECT mis-
     filters, RLS drops any row whose user_id doesn't match the session
     setting.
  3. `listDueMissions(sql, now)` — two-phase composition of the above.

The LWW merge + due-filter logic is factored out into a pure
`mergeAndFilter(rows, userId, now)`. Fully unit-tested (6 Bun cases):
active-due happy-path, future nextRunAt, non-active state, delete
tombstone, multi-row LWW merge, userId stamping.

Matches the pattern already in use for writes (`db/connection.ts:withUser`
+ `db/iteration-writer.ts`). Docstring on `listMissionUsers` spells out
the remaining BYPASSRLS dependency so ops knows what role the service
needs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 01:06:17 +02:00
Till JS
a8425941fb feat(mana-ai): server-side input resolvers (goals for now)
Plugs plaintext-safe Mission context into the Planner prompt per tick.
Before this, `resolvedInputs: []` was always passed — the LLM only saw
the mission's concept + objective. Now goals (the only plaintext
category of linked inputs today) resolve and land in the prompt.

Privacy constraint is explicit and documented: tables in the webapp's
encryption registry (notes, kontext, journal, dreams, …) arrive at
`sync_changes.data` as ciphertext — the master key lives in mana-auth
KEK-wrapped and never reaches this service. Resolvers for encrypted
modules therefore don't exist server-side; missions referencing them
should use the foreground runner which decrypts client-side.

- `db/resolvers/types.ts` — ServerInputResolver contract
- `db/resolvers/record-replay.ts` — single-record LWW replay
  (tighter WHERE than `missions-projection.ts`, used by all resolvers)
- `db/resolvers/goals.ts` — reads `companionGoals` via replayRecord,
  mirrors the webapp's default goalsResolver output shape
- `db/resolvers/index.ts` — registry with `registerServerResolver` /
  `unregisterServerResolver` / `resolveServerInputs`. Seeds `goals`.
  Drift-tolerant: missions pointing at unregistered modules silently
  skip those inputs.
- `cron/tick.ts` — wires `resolveServerInputs(sql, m.inputs, m.userId)`
  into the planner input; updates the outdated "stubbed" comment

5 Bun tests over the registry (handled + unhandled + thrown +
mixed cases + seeded default).

Future: expand to plaintext tables if/when more land (habits without
free-text, dashboard configs, tags), or introduce a decrypt-via-auth
sidecar if users opt into server-side access to encrypted content.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 00:42:45 +02:00
Till JS
5e01763caa feat(ai): close the loop — server write-back + webapp staging effect
Completes the off-tab AI pipeline. mana-ai now writes produced plans
back to `sync_changes` as a server-sourced Mission iteration; the webapp
picks it up on next sync and translates each PlanStep into a local
Proposal via the existing createProposal flow. User sees the resulting
ghost cards in the matching module's AiProposalInbox with full mission
attribution.

Server (mana-ai v0.3):
- `db/connection.ts` — `withUser(sql, userId, fn)` RLS-scoped tx helper
  mirroring the Go `withUser` pattern (SET LOCAL app.current_user_id)
- `db/iteration-writer.ts`
  - `planToIteration(plan, id, now)` — shared-ai AiPlanOutput → inline
    MissionIteration with `source: 'server'` + status='awaiting-review'
  - `appendServerIteration(sql, input)` — INSERT sync_changes row with
    op=update, data={iterations: [...]} + field_timestamps + actor
    JSONB={kind:'system', source:'mission-runner'}
- `cron/tick.ts` — after parse success: build iteration, append to
  mission.iterations, persist via appendServerIteration. Stats now
  include `plansWrittenBack`.

Actor union:
- `packages/shared-ai/src/actor.ts` + webapp actor: `system.source` gains
  `'mission-runner'` so the server's own writes are attributed correctly
  and distinguishable from projection/rule writes

Webapp:
- `data/ai/missions/server-iteration-staging.ts`
  - `startServerIterationStaging()` subscribes to aiMissions via Dexie
    liveQuery; on each Mission update, walks iterations looking for
    `source='server'` entries that haven't been staged yet
  - For each such iteration: creates a Proposal per PlanStep under
    `{kind:'ai', missionId, iterationId, rationale}` so policy + hooks
    fire correctly
  - Writes proposalIds back into plan[].proposalId + status='staged' so
    other tabs and app restarts skip re-staging
  - Idempotent: in-memory `processedIterations` Set + durable
    proposalId marker
- Wired into (app)/+layout.svelte alongside startMissionTick
- 3 unit tests: translate server iteration → proposal, skip
  already-staged, ignore browser iterations

Full pipeline now: user creates Mission in /companion/missions →
mana-ai tick picks it up → calls mana-llm → parses plan →
writes iteration → synced to webapp → staging effect creates
proposals → user approves in /todo (or any module) → task lands with
`{actor: ai, missionId, iterationId, rationale}` attribution.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 00:29:30 +02:00
Till JS
b9710e6c11 feat(mana-ai): scaffold server-side Mission Runner (v0.1)
Background Hono/Bun service that scans mana_sync for due Missions and
will plan them via mana-llm without requiring an open browser tab.
Complements the foreground `startMissionTick` in the webapp.

v0.1 scope — scaffold that's deployable, boots cleanly, and reads real
data. Execution write-back is tracked as the next PR so we don't commit
a half-baked proposal-sync design.

Shipped:
- Hono app on :3066 with `/health` + service-key-gated `/internal/tick`
- `src/db/missions-projection.ts` — field-level LWW replay of
  `sync_changes` for appId='ai' / table='aiMissions' → live Mission
  records. Mirrors the webapp's `applyServerChanges` semantics against
  Postgres instead of Dexie.
- `src/db/connection.ts` — bounded `postgres.js` pool (max 4, idle 30s)
- `src/cron/tick.ts` — overlap-guarded scheduler, `runTickOnce()` also
  reachable via HTTP for CI/ops triggering
- `src/planner/client.ts` — mana-llm HTTP client shape
  (OpenAI-compatible `/v1/chat/completions`)
- `src/middleware/service-auth.ts` — X-Service-Key gate, no end-user JWTs
  reach this service
- Dockerfile + graceful SIGTERM shutdown (stops timer + releases pool)

Not yet implemented (documented in CLAUDE.md with design trade-offs):
- Prompt/parser server-side copies — today they live in the webapp.
  Recommended next step: extract `@mana/shared-ai` package.
- Input resolvers for notes / kontext / goals — need projections or a
  mana-sync internal endpoint
- Plan → Mission-iteration write-back + how proposals get back to the
  user's device (leaning option (a): server writes iterations, the
  webapp's sync effect translates them into local Proposals)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 23:48:30 +02:00