Commit graph

685 commits

Author SHA1 Message Date
Till JS
79a6da3e2e feat(spaces): move access tier from user to space
Migration from user-level tier to Space-level tier, following the
Spaces foundation plan. User-visible effect: the tier that gates
module access now belongs to the active Space, not the user account.
Personal Spaces inherit the user's old tier on signup so nothing
downgrades.

shared-types:
- New SpaceTier type ('guest' | 'public' | 'beta' | 'alpha' | 'founder').
- New spaceTierMeets(actual, required) helper.
- SpaceMetadata gains an optional `tier` field.

mana-auth:
- createPersonalSpaceFor reads user.accessTier and stamps it into the
  personal Space's metadata.tier. A founder-tier user setting up their
  first Space keeps founder access in that Space.
- databaseHooks.user.create.after now forwards accessTier into the
  personal-space creator.

apps/web (scope layer):
- ActiveSpace gains a required `tier: SpaceTier`; rawToActiveSpace
  reads it from organization.metadata, defaulting to 'public' if
  missing or invalid.
- New getEffectiveTier(userFallback) helper resolves the tier to use
  for gating: prefers the active Space's tier, falls back to the
  caller-supplied user tier during the boot window.

apps/web ((app) layout):
- `effectiveTier` $derived replaces every authStore.user?.tier reference
  in the layout's access-gating logic (appItems, routeBlocked,
  routeTierLabels). AuthGate deeper in the UI keeps using user.tier as
  its own fallback — the tier move is additive, not destructive.

What this does NOT do yet:
- The user.accessTier column still exists and is still the initial
  source for personal-space tier. Removing it is a later cleanup once
  every code path reads through the Space primitive.
- No admin API for setting tier on a Space (PUT /api/v1/admin/spaces/
  :id/tier). Follow-up when admin tooling needs it — today admins still
  set user.accessTier, which flows to the personal space on next
  signup.

Resolves the MANA_APPS-tier-patch workaround memory: future sessions
can adjust tier per Space instead of per User.

0 errors across 7151 files. 10/10 scope tests pass.

Plan: docs/plans/spaces-foundation.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 20:10:06 +02:00
Till JS
88e3adb9d3 feat(spaces): multi-member RLS policy in mana-sync (forward-compat)
Adds the second RLS policy needed for shared spaces. Users can read
rows in any space they're a member of, in addition to their own rows.

Changes:
- New policy sync_changes_space_member_read (SELECT only) uses
  app.current_user_space_ids session config: rows with space_id in
  that comma-separated list pass RLS.
- WITH CHECK is not extended — writes still require user_id match, so
  only the author can write. Members read, owner/author writes.
- withUser() is now a thin wrapper around withUserAndMemberships(),
  which accepts the caller's Space membership list and sets the new
  session config alongside app.current_user_id.
- The comma-join is empty-filtered so stray blank entries can't match
  rows with literal empty space_id (defense in depth).

Forward-compatible: today every space has exactly one member (the
author), so the membership list is always empty and the new policy
is a no-op — user_id isolation remains the only active guard.

When shared spaces start being used (clubs/teams/brand spaces with
invites), the HTTP handlers will fetch the caller's membership from
mana-auth and pass it to withUserAndMemberships. No migration needed
at that point — the policy is already live.

Subscription fan-out (WS/SSE broadcast to all space members) is still
per-user; that's a follow-up tied to the membership lookup infra.

Go build + existing tests pass.

Plan: docs/plans/spaces-foundation.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 19:55:17 +02:00
Till JS
0d613e1846 feat(ai): thread TokenUsage through runPlannerLoop → mana-ai budget
Carries per-round token counts from the mana-llm response body
(prompt_tokens + completion_tokens) back through LlmCompletionResponse
→ PlannerLoopResult. The loop sums across rounds and exposes a single
aggregate on result.usage.

Lets mana-ai's tick re-activate per-agent daily-token budget tracking
— tokensUsed was stubbed to 0 in the migration commit (6) because the
loop didn't surface usage yet. Now recordTokenUsage + agentTokenUsage24h
get real numbers again, and the mana_ai_tokens_used_total Prometheus
counter is accurate.

Additive only: consumers without usage needs ignore the new field,
and providers that don't return usage produce zeros (not undefined —
the loop still exposes the object so downstream branches stay trivial).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 18:21:34 +02:00
Till JS
e10c2436a6 feat(spaces): thread space_id through mana-sync protocol + storage
Server-side:
- sync_changes gains a nullable space_id TEXT column + partial index
  on (user_id, space_id, app_id, created_at) WHERE space_id IS NOT NULL.
- RecordChange takes spaceID as a first-class parameter; *string so
  empty strings land as real SQL NULL and the partial index skips them.
- ChangeRow + all three SELECTs (GetChangesSince, GetAllChangesSince,
  StreamAllUserChanges) propagate space_id through to clients.
- changeFromRow surfaces SpaceID on the wire Change shape.
- New extractSpaceID helper reads the incoming payload — prefers top-
  level spaceId, falls back to data.spaceId (inserts) or
  fields.spaceId.value (updates). Tolerates pre-v28 clients.
- 6 Go tests cover the helper + round-trip.

Client-side:
- PendingChange gains an optional spaceId.
- Dexie creating hook stamps spaceId from the active record onto the
  pending-change row (already set by the v28 scope hook).
- Dexie updating hook reads spaceId from the pre-update record and
  stamps it on the pending-change so updates carry space context even
  though spaceId itself is immutable and never in `fields`.
- buildChangeset forwards spaceId to the server.

Explicitly NOT in scope this pass:
- RLS remains user_id-scoped; multi-member shared-space reads need a
  second policy that joins against auth.members. Follow-up once shared
  spaces are actually used — today everything is personal.
- Subscription fan-out is still per-user; fan-out to all members of a
  shared space is part of the same follow-up.

Go tests: 6/6 pass. Web type-check clean (0 errors across 7139 files).

Plan: docs/plans/spaces-foundation.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 16:53:14 +02:00
Till JS
1cd559ca34 feat(mana-ai): server runner on runPlannerLoop, drops text-JSON parser
Migrates the background tick from buildPlannerPrompt + PlannerClient +
parsePlannerResponse to the shared runPlannerLoop with native function
calling. Structurally identical to the webapp runner (commit 5a) —
same catalog, same compact system prompt, same multi-turn chat.

Server-specific twist: the ``onToolCall`` callback is a no-op stub
(returns {success:true, message:'recorded — pending client
application'}). The server has no Dexie access, so it can't actually
execute writes; instead it captures the LLM's chosen tool_calls and
writes them as PlanStep entries on the iteration. The user's client
picks up those planned steps on sync — same shape as before, just
sourced from the LLM's native tool_calls instead of a regex-extracted
JSON block.

Scope trimmed by the SERVER_TOOLS filter: only propose-default (write)
tools go to the server planner. Read-only tools (list_*, get_*) are
hidden because stubbing a response would let the LLM hallucinate that
it saw real data. Read-then-act chains stay with the foreground
runner, which has a real executor.

Deleted: planner/client.ts (old PlannerClient; replaced by
planner/llm-client.ts). Drift guard in tools.ts collapses into a
SERVER_TOOLS = AI_TOOL_CATALOG.filter(propose) derivation — no more
hand-maintained duplicate list; the contract test now asserts the
inverse round-trip against AI_PROPOSABLE_TOOL_SET.

TODO (follow-up): token usage tracking is temporarily set to 0 because
runPlannerLoop doesn't expose per-message usage yet. Budget
enforcement on the server is effectively disabled until the loop
returns that data — the webapp runner is unaffected.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 16:39:20 +02:00
Till JS
698ffe797c feat(spaces): add spaces pg schema — credentials + module_permissions
Groundwork for server-side Space extensions that must NOT live in Dexie:
  - spaces.credentials         — per-space OAuth tokens, API keys, SMTP
                                 configs. Access tokens are stored
                                 encrypted at rest with the service KEK.
  - spaces.module_permissions  — role × module read/write/admin overrides
                                 on top of the SPACE_MODULE_ALLOWLIST
                                 defaults.

Both tables FK to auth.organizations with ON DELETE CASCADE so deleting
a space drops its credentials and permission overrides automatically.

RLS is intentionally deferred — enabling it now would lock out services
that don't yet pass space context. A follow-up migration turns it on
after mana-api speaks the Spaces protocol end-to-end.

To apply locally: bun run db:push in services/mana-auth, or psql -f
sql/004_spaces.sql against the mana_platform DB.

No runtime code reads these tables yet — they're the scaffolding that
Task-8 (mana-sync) and the eventual social-relay/clubs modules will
consume.

Plan: docs/plans/spaces-foundation.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 16:13:33 +02:00
Till JS
da1bb2d6e9 feat(spaces): auto-create personal space on signup via Better Auth hook
Wires databaseHooks.user.create.after to call createPersonalSpaceFor,
which provisions a Better Auth organization of type='personal' with the
user as owner. Every signup now produces a usable default space — no
UI code needed to bootstrap it.

Details:
- Slug derived from email local-part, lowercase, alphanumerics + hyphens,
  max 30 chars, random fallback if nothing usable remains.
- Reserved-slug list (me/admin/api/auth/…) blocks system-route clashes.
- Collision resolver appends -2, -3, … up to 999 before falling back to
  a random suffix. Tests cover both the DB-taken and reserved-slug cases
  via an injectable SlugTakenLookup (no DB needed for unit tests).
- Idempotent: if a personal space already exists for the user, returns
  it instead of creating a duplicate. Guards against retry double-signup.
- Failure propagates — an orphan user without a personal space is worse
  than a retry-able signup error.

Existing dev users will need a backfill or a re-provisioning of the dev
DB — new users are unaffected.

12 tests pass (23 total across the spaces module).

Plan: docs/plans/spaces-foundation.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 16:10:26 +02:00
Till JS
166d6c6ffb feat(spaces): validate space metadata on Better Auth organization hooks
Moves the canonical SpaceType + SPACE_MODULE_ALLOWLIST to @mana/shared-types
(framework-free) so the Bun services can consume them without pulling in
Svelte. shared-branding keeps only the UI-facing labels and descriptions
and re-exports the canonical types for frontend convenience.

Wires two Better Auth organization hooks in mana-auth:
- beforeCreateOrganization asserts metadata.type is a valid SpaceType,
  rejecting the create with a BAD_REQUEST otherwise.
- beforeDeleteOrganization rejects deletion of the personal space.

Covered by bun tests (11 assertions) for the helper module.

No migration and no schema change — type lives in the existing
organization.metadata jsonb column.

Plan: docs/plans/spaces-foundation.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 16:05:38 +02:00
Till JS
e757470cb0 feat(mana-llm): add OpenAI-style tools + tool_calls passthrough
Extends the chat-completions surface so callers can ask any provider
to call named functions and get structured tool_calls back. Wired
through all three provider adapters so the planner and companion can
switch off the fragile JSON-parsing pathway.

- Request: tools[], tool_choice, assistant tool_calls, tool-role
  messages with tool_call_id.
- Response: MessageResponse.tool_calls, Choice.finish_reason adds
  "tool_calls", DeltaContent streams tool_calls.
- Google provider: Tool(function_declarations=...) build, result
  normalised (args dict → JSON string), function_response parts on
  a user turn for tool-role messages.
- OpenAI-compat: 1:1 passthrough of the OpenAI spec.
- Ollama: /api/chat passthrough; model-level capability check via a
  TOOL_CAPABLE_OLLAMA_PATTERNS whitelist (llama3.1+, qwen2.5+,
  mistral, command-r, …) — unsupported models rejected rather than
  silently falling back to prose.
- Router: model_supports_tools() check upfront for both streaming
  and non-streaming paths; ProviderCapabilityError bubbles as 400.

No silent downgrade. Missing tool support = explicit error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 15:22:48 +02:00
Till JS
c612a22371 fix(type-check): unblock two more pre-existing failures
After yesterday's type-check cascade repair (c34175afa), the root
\`pnpm run type-check\` progressed through 5 more packages but still
stopped on two pre-existing failures:

- \`services/mana-media\` delivery route: \`c.body(transformedBuffer)\`
  passed a Node \`Buffer<ArrayBufferLike>\`, but Hono 4.7 types the
  body argument as \`Uint8Array<ArrayBuffer>\` (strict — no
  ArrayBufferLike). \`Uint8Array.from(buf)\` gives a clean copy with a
  fresh \`ArrayBuffer\` backing that the strict type accepts. Runtime
  cost for a handful of KB per image transform is negligible next to
  the Sharp pipeline that produced the buffer.
- \`packages/shared-llm\`: same rune issue as local-stt + local-llm —
  \`store.svelte.ts\` uses \`$state\` and transitively pulls in
  \`local-llm/src/svelte.svelte.ts\`. Plain tsc can't resolve Svelte 5
  runes. Same treatment: \`type-check\` script explicitly skips with a
  message pointing at svelte-check.

Root \`pnpm run type-check\` now reaches \`@context/mobile\`, which has
real code-level type errors (adapter shape mismatches, an RN event-
handler typing drift, and a deleted Supabase module still imported by
\`utils/supabaseTest.ts\`). Those need domain changes, not config
tweaks — out of scope for this repair pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 15:20:08 +02:00
Till JS
4b8fede7fc fix(mana-llm): surface Gemini finish_reason errors instead of returning ""
The google provider called response.text after a chat completion and
passed the resulting string downstream unchanged. When Gemini's content
filter, recitation guard, or max_tokens ceiling fired, response.text
quietly returned "" — which the planner then reported as "no JSON block
found", masking the real cause. Empirically this failed in 45 ms on a
simple Quiz mission.

Introduces providers/errors.py with a small ProviderError hierarchy
(Blocked / Truncated / Auth / RateLimit / Capability). google.py now
inspects response.candidates[0].finish_reason and raises the matching
structured error; the non-streaming path maps it to 422/502/429 via a
new except-branch in main.py, and the streaming path surfaces the kind
as the SSE error type. Capability is wired but not yet used — it lands
with the tool-schema passthrough in the next commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 15:15:37 +02:00
Till JS
5ec1dfc747 chore(db): enforce pgSchema isolation with a lint script
The "every Drizzle table uses pgSchema" rule was documented in
.claude/guidelines/database.md (added yesterday as part of Concern 5)
but enforced only by convention. A new service could slip a raw
\`pgTable()\` past review and collide in the default \`public\` schema
of \`mana_platform\`, and nothing would surface the mistake until a
production migration failed.

- \`scripts/validate-pg-schema-isolation.mjs\` scans every tracked
  TypeScript file under services/, apps/api/, packages/ for call sites
  of \`pgTable(\` (not imports — imports can still be useful for types).
  Strips comments before matching so doc-examples like "use \`pgTable()\`"
  don't trigger false positives.
- Wired as \`pnpm run validate:pg-schema\` and a new CI step in the
  validate job (right after the turbo-recursion check). 721 files
  scan clean today.
- Removed an unused \`pgTable\` import in mana-subscriptions that would
  have been the only import of the symbol remaining after this change.
- Updated .claude/guidelines/database.md — the old verification blurb
  said "no automated lint rule yet", now points at the enforcer.

Drift verified: injecting a synthetic \`pgTable('bad', {})\` into
subscriptions.ts failed with a clear file:line violation pointing at
the database guideline.

Closes the "no automated lint rule" gap noted in the database guideline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 14:45:59 +02:00
Till JS
52d008dd34 fix(goals): start GoalTracker on boot + surface AI proposals inline
startGoalTracker was only ever called from tests, so DrinkLogged /
TaskCompleted / MealLogged events never incremented currentValue and
GoalReached never fired — the progress bars were cosmetic. Wire it into
the (app)/+layout idle boot next to startStreakTracker, with matching
teardown in onDestroy.

Also drop <AiProposalInbox module="goals"/> into the module ListView so
create_goal / pause_goal / resume_goal / complete_goal proposals are
reviewable inline (previously only visible in the mission-detail view).

Refresh the tool-coverage tables while we're at it: apps/mana/CLAUDE.md
now reflects the real catalog state (59 tools, 19 modules — was 37/12),
and services/mana-ai/CLAUDE.md shows the correct server-side propose
subset (31 tools, 16 modules). Also fixes a stale 'location_log' →
'get_current_location' typo in the places row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-20 14:24:39 +02:00
Till JS
a7fe828d32 refactor(auth): extract sso-origins SSOT + harden drift test
TRUSTED_ORIGINS was defined inside better-auth.config.ts, which pulls
in the whole Better Auth stack just to read a list of hostnames. Anyone
who wants to consume the list (infra tooling, compose-env generators,
monitoring) had to either duplicate it or pay the import cost.

- New `sso-origins.ts` — zero-dep module exposing
  `PRODUCTION_TRUSTED_ORIGINS` + `LOCAL_TRUSTED_ORIGINS` + the combined
  `TRUSTED_ORIGINS` list. This is now the canonical place to add a new
  top-level SSO origin.
- `better-auth.config.ts` imports + re-exports so existing consumers
  keep working without a touch.
- `sso-config.spec.ts` imports directly from `./sso-origins` (cleaner
  coupling) and now HARD-FAILS when mana-auth CORS_ORIGINS contains a
  production origin that isn't in trustedOrigins. Previously this was
  a `console.warn` only, meaning dead-drift could silently accumulate
  and then surface as a confusing runtime auth rejection.
- Root CLAUDE.md "Adding an app to SSO" updated to point at the SSOT
  and mention the new hard-fail direction.

No current drift — the mana-auth CORS_ORIGINS already match. The
hardened assertion is defensive for future changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 19:45:42 +02:00
Till JS
8dd3dbc9e5 docs(mana-research): step-by-step API_KEYS.md setup guide
Complete walkthrough per provider — signup URL, free-tier details,
pay-per-use pricing, env-var name, key format — plus sections on where
to paste keys (.env.secrets), BYO-keys vs server-keys, verification
curl commands and troubleshooting (including the cross-service
MANA_SERVICE_KEY mismatch encountered during live testing).

Linked from services/mana-research/CLAUDE.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 18:50:06 +02:00
Till JS
85537cb92a fix(research): default Gemini to 2.5-flash (2.0-flash deprecated for new users)
Google deprecated `gemini-2.0-flash` for new API users — existing
accounts still work, but a freshly-billed key returns 404
"models/gemini-2.0-flash is no longer available to new users". The
working replacement is `gemini-2.5-flash` (same price tier, better
quality, groundingMetadata shape unchanged).

Verified live: the fix produced a real answer with 6 grounding
citations in 2.6s.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 17:01:24 +02:00
Till JS
97abd251e3 fix(events): Eventbrite provider — switch from dead API to web scraping
Eventbrite shut down their public Event Search API (/v3/events/search)
in 2023. The provider now uses the website extractor pipeline
(mana-research + LLM) to scrape Eventbrite's public search pages.
No API key needed — same pipeline as any website source.

Also adds mana-events to generate-env.mjs for automatic .env generation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 16:51:58 +02:00
Till JS
536fc89050 fix(research): Claude Opus 4.7 rejects temperature param + log executor errors
- claude-web-search.ts: only send `temperature` when caller explicitly
  sets one. Opus 4.7 deprecated the param and returns
  400 invalid_request_error "`temperature` is deprecated for this
  model." Sonnet/Haiku still accept it, so keep the opt-in path.
- execute-research.ts: log provider errors via console.warn so future
  integration failures are visible in stdout. Previously the executor
  swallowed the underlying error and only returned a generic errorCode,
  which made diagnosing vendor-specific API changes impossible.

Discovered via smoke-testing with a real Anthropic key — the direct
curl worked, but our provider 400'd because Opus 4.7 tightened the
accepted param set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 16:36:22 +02:00
Till JS
ed801cf725 feat(events): Phase 4 — provider adapters for Eventbrite + Meetup
- Add EventProvider interface (base.ts) with fetchEvents(url, name, ctx, config)
- Refactor iCal parser and website extractor as provider adapters
- Add Eventbrite provider: API v3 search by location, category mapping,
  price info extraction. Requires EVENTBRITE_API_KEY env var.
- Add Meetup provider: GraphQL API search by location, topic→category
  mapping, HTML stripping. Requires MEETUP_API_KEY env var.
- Provider registry (getProvider, PROVIDER_TYPES) replaces hardcoded
  switch in crawl-scheduler
- Crawl scheduler now joins sources with regions for ProviderContext
  (lat/lon/radius/label) — platform providers need this for geo-search
- Source creation accepts 'eventbrite' and 'meetup' types (url optional)
- Both providers gracefully return empty when API keys unconfigured

116 tests (all passing), no regressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 15:43:40 +02:00
Till JS
2c0d866287 feat(events): Phase 3 — AI tools, Event-Scout template, feedback loop
- Add discover_events (auto) and suggest_event (propose) to shared-ai
  tool catalog. discover_events reads the discovery feed, suggest_event
  creates a proposal to save a discovered event to the user's calendar.
- Add Event-Scout agent template with daily "Events der Woche" mission.
  Policy: discover_events=auto, suggest_event=propose, all else denied.
- Add frontend tool implementations in events/tools.ts — discover_events
  calls the feed API, suggest_event delegates to discoveryStore.saveEvent.
- Add feedback.ts — computes implicit user profile from save/dismiss
  history (category affinity + source quality as 0–2x weight multipliers).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 15:37:28 +02:00
Till JS
b5d55fdb21 feat(events): add Event Discovery — Phase 1 + 2
Phase 1: Manual iCal feeds + Discovery tab
- 5 new DB tables in event_discovery schema (regions, interests,
  sources, discovered_events, user_actions)
- iCal parser (node-ical) with deduplication (SHA-256 hash)
- Crawl scheduler (15-min interval, auto-deactivate after 5 errors)
- CRUD routes for regions, interests, sources + paginated feed endpoint
- Frontend: "Meine Events" / "Entdecken" tab navigation in ListView
- Discovery setup wizard (regions via mana-geocoding + interests)
- DiscoveredEventCard with save/dismiss, SourceManager for iCal feeds
- "Merken" creates a local socialEvent from discovered event

Phase 2: Auto source discovery + LLM extraction + relevance scoring
- Source discoverer: web search via mana-research to auto-find iCal
  feeds and venue websites for a region
- Website extractor: crawl via mana-research /extract, then LLM-based
  event extraction via mana-llm with structured JSON output
- Flexible date parsing (ISO, DD.MM.YYYY), markdown fence stripping
- Relevance scorer: category match, freetext match, haversine distance,
  time proximity, weekend bonus (0-100 clamped)
- Routes: POST regions/:id/discover-sources, PUT/DELETE sources/:id/activate|reject
- Frontend: "Automatisch finden" button, suggested vs active sources UI

107 tests (all passing), no regressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 15:30:46 +02:00
Till JS
7d120225dc feat(research): Phase 3b openai-deep-research async + BYO-keys CRUD & UI
Two backlog items landed in one commit because an earlier amend in a
parallel terminal dropped the initial Phase 3b commit and the BYO-keys
work was blocked on the same wiring.

openai-deep-research (async):
- New research.async_jobs table persists the OpenAI response.id, query,
  reservation, and cached result/error.
- POST /v1/research/async reserves credits, submits to the Responses API
  with background=true, returns a taskId. Submit failure refunds.
- GET /v1/research/async/:taskId polls upstream, commits the reservation
  on completion, refunds on failure, short-circuits for terminal states.
- GET /v1/research/async lists the user's async tasks.

BYO-keys:
- research.provider_configs CRUD at /v1/provider-configs. Keys are masked
  (••••last4) on read so the raw secret never re-transits to the browser.
  Currently stored plaintext with a TODO for AES-GCM-256 via the shared
  KEK — single call site in storage/configs.ts.decryptKey().
- New frontend route /research-lab/keys lets the user paste a key per
  provider, toggle enabled, and set daily/monthly credit budgets.
- ListView grew a 🔑 link in the header.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 15:43:12 +02:00
Till JS
8f0a74b2e7 feat(research-lab): tier gate (beta+), 1–5 star ratings, run detail route
- Branding: research-lab registered in @mana/shared-branding with requiredTier: 'beta' + a custom flask-on-purple icon, so guest/public users are filtered out of the workbench picker.
- Backend: compare routes now return resultId alongside each CompareEntry so the frontend can wire ratings to the eval_results rows in research.*.
- Frontend: click-to-rate stars in CompareColumn (persists via POST /v1/runs/:runId/results/:resultId/rate), recent-run list rows are now buttons that navigate to /research-lab/runs/[id], and the detail route reconstructs CompareEntry shapes from eval_results + reuses CompareColumn for a full read-only view of any past run.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 15:28:02 +02:00
Till JS
8823cc0bf0 feat(profile): voice interview with pre-rendered TTS audio + Orpheus/Zonos backends
Voice-based interview for the profile module — users choose between text,
voice (question read aloud + mic for answer), or conversation mode (fully
automatic flow with auto-save).

Interview audio:
- 92 pre-rendered MP3 files (23 questions × 4 voices) via Edge TTS
- Voices: Seraphina (DE-f), Florian (DE-m), Leni (CH-f), Jan (CH-m)
- User picks voice via dropdown, persisted in localStorage
- Web Speech API fallback for missing audio files

Profile UI:
- Interview hero block on overview with 3 start modes (text/voice/conversation)
- Voice/conversation toggle + voice picker in interview view
- Mic button on text/textarea/tags inputs for per-question voice input
- Conversation mode: auto-save + auto-advance after STT transcription
- Recording/transcribing/speaking state indicators

mana-tts service:
- New Orpheus TTS backend (German finetune, SNAC codec)
- New Zonos TTS backend (Zyphra, 200k hours, emotion control)
- Endpoints: POST /synthesize/orpheus, POST /synthesize/zonos
- espeak-ng installed on GPU server for Zonos phonemizer
- Compare script for side-by-side voice quality testing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 15:22:52 +02:00
Till JS
49f315f6be feat(research): Phase 3a — 4 sync research agents
Adds Perplexity Sonar, Claude web_search, OpenAI Responses, and Gemini
Grounding as ResearchAgents behind the same comparison interface as the
search and extract providers.

New endpoints:
  POST /v1/research          — single-agent (or auto-routed to the first
                               provider with a configured key)
  POST /v1/research/compare  — fan-out across N agents, persist all
                               answers + citations in research.eval_*

Each agent normalizes its native response into a common AgentAnswer shape
(answer text + citations[] + tokenUsage), storing the provider's raw
response alongside for later inspection. Implementations use direct HTTP
against each vendor's public API — no SDK deps added.

Auto-routing preference: perplexity-sonar → gemini-grounding →
openai-responses → claude-web-search → (openai-deep-research stubbed for
Phase 3b). Credits orchestration reuses the search/extract executor
pattern (reserve → call → commit/refund).

Deferred to Phase 3b: openai-deep-research (async job queue), migration
of mana-ai + mana-api news-research to call this service directly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 15:06:12 +02:00
Till JS
2bdb48bdd1 feat(research): add mana-research service — Phase 1 + 2
New Bun/Hono service on port 3068 that bundles many web-research providers
behind a unified interface for side-by-side comparison. All eval runs
persist in research.* (mana_platform) so quality can be reviewed later.

Providers (Phase 1+2):
  search:  searxng, duckduckgo, brave, tavily, exa, serper
  extract: readability (via mana-search), jina-reader, firecrawl

Endpoints:
  POST /v1/search, /v1/search/compare       — single + fan-out
  POST /v1/extract, /v1/extract/compare     — single + fan-out
  GET  /v1/runs, /v1/runs/:id               — history
  POST /v1/runs/:run/results/:id/rate       — manual eval
  GET  /v1/providers, /v1/providers/health  — catalog + readiness

Auto-routing: when `provider` is omitted, queries are classified via regex
(fast path, 0ms) with optional mana-llm fallback, then routed to the first
available provider for that query type (news → tavily, academic → exa,
semantic → exa, etc.).

Credits: server-key calls go through mana-credits reserve → commit/refund
so failed provider calls don't charge the user. BYO-keys supported via
research.provider_configs (UI arrives in Phase 4).

Cache: Redis with graceful degradation (1h TTL for search, 24h for
extract). Pay-per-use APIs only — no subscription-gated providers.

Docs: docs/plans/mana-research-service.md + docs/reports/web-research-capabilities.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:42:25 +02:00
Till JS
004fc0b2fd feat(credits): add 2-phase debit (reserve/commit/refund)
Introduces credit_reservations table + three internal endpoints so services
that need to charge only after a downstream call succeeds (notably the
upcoming mana-research fan-out across paid provider APIs) can reserve
credits atomically, then commit on success or refund on failure. One-shot
/credits/use remains for synchronous operations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 14:41:41 +02:00
Till JS
76577869e1 feat(mana-ai): OpenTelemetry tracing + Grafana Tempo backend
Add distributed tracing to the mana-ai background runner so mission
execution can be visualized end-to-end in Grafana.

Instrumentation (services/mana-ai/):
- tracing.ts: OTel provider setup with OTLP/HTTP exporter, withSpan() helper
- tick.ts: tick.planMission span with mission/agent/user attributes
- client.ts: planner.complete span with LLM model, tokens, latency

Infrastructure:
- docker/tempo/tempo.yaml: Grafana Tempo config (OTLP HTTP on 4318)
- docker-compose: tempo service + tempo_data volume + mana-ai env var
- docker/grafana/provisioning/datasources/tempo.yml: auto-provisioned

Trace flow:
  tick.planMission (root span)
    └── planner.complete (child span)
        ├── llm.model = "gpt-4o-mini"
        ├── llm.tokens.total = 1234
        └── llm.response.length = 567

Enable: set OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
View: Grafana → Explore → Tempo datasource

Also fixes: removed broken @mana/subscriptions workspace ref from arcade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:21:23 +02:00
Till JS
acd7e0d6b0 docs: update architecture comparison — 5/10 roadmap items done
Update report to reflect all completed work:
- Matrix: streaming , tool registration updated to 29 tools + MCP
- §5.2 Streaming: marked done
- §5.3 Tool System: marked done
- §6 Table: items 1-3 + 5 struck through with commit refs
- §8 Fazit: updated gaps and recommendations

5 of 10 roadmap items complete in one session:
1. SSE Streaming, 2. Dynamic Tool Registry, 3. Budget Enforcement,
5. MCP Server Export (27/29 tools with DB ops), plus Tool Drift Fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:00:09 +02:00
Till JS
ce57e11950 feat(mana-ai): server-side token budget enforcement per agent
Implement rolling 24h token budget enforcement in the mana-ai tick loop.
Agents with maxTokensPerDay set are now rate-limited server-side.

Changes:
- PlannerClient: extract usage.total_tokens from mana-llm response
- planOneMission: return {plan, tokensUsed} tuple
- tick loop: check getAgentTokenUsage24h() before planning; skip with
  'skipped-budget' decision if over limit
- tick loop: record token usage after successful plan via
  recordTokenUsage() INSERT into mana_ai.token_usage
- migrate.ts: new mana_ai.token_usage table with rolling window index
- metrics.ts: mana_ai_tokens_used_total counter (by agent_id)

Budget flow:
  Agent.maxTokensPerDay = 50000
  → tick checks: SELECT SUM(tokens_used) WHERE ts > now()-24h
  → if sum >= 50000: skip mission, emit skipped-budget metric
  → else: plan mission, INSERT token_usage row

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 14:41:31 +02:00
Till JS
d40a61119e refactor(ai): dynamic tool registry — single-source catalog in shared-ai
Introduce AI_TOOL_CATALOG in @mana/shared-ai as the single source of truth
for all 29 tool schemas (17 propose + 12 auto). Both the webapp policy and
the server-side mana-ai planner now derive their tool lists from the catalog
instead of maintaining independent hardcoded copies.

- New: packages/shared-ai/src/tools/schemas.ts — catalog with ToolSchema type
- Rewrite: proposable-tools.ts — derived from catalog instead of hardcoded array
- Rewrite: services/mana-ai/src/planner/tools.ts — 277→30 lines (imports from catalog)
- Simplify: webapp policy.ts — derives AUTO/PROPOSE from catalog defaultPolicy

Adding a new tool now requires 2 files instead of 3-5:
1. Add schema to AI_TOOL_CATALOG (shared-ai)
2. Add execute function in the module's tools.ts (webapp)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 13:06:07 +02:00
Till JS
3be4612f04 fix(mana-llm): google-genai v1.73 keyword-only Part.from_text()
google-genai >=1.70 changed Part.from_text() from positional to
keyword-only argument. The production container installed v1.73.1
and crashed on startup with "Part.from_text() takes 1 positional
argument but 2 were given".

Fix: Part.from_text(msg.content) → Part.from_text(text=msg.content)

Tested live: curl https://llm.mana.how/v1/chat/completions with
model=google/gemini-2.5-flash returns correct response.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:47:23 +02:00
Till JS
659a7d9774 fix(mana-llm): add google-genai to requirements.txt for Docker builds
google-genai was in pyproject.toml but missing from requirements.txt.
The Dockerfile uses pip install -r requirements.txt, so the Google
provider never loaded in production. Now that the key is set and the
cloud tier upgraded to gemini-2.5-flash, the import fires on startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:40:30 +02:00
Till JS
d83fc370a0 docs: update tool coverage table + server-side research + templates
Catches up all docs with the current state of the AI tool system.

services/mana-ai/CLAUDE.md:
- New v0.6 status section documenting NewsResearchClient,
  pre-planning research injection, config.manaApiUrl, and the full
  28-tool / 11-module inventory (17 propose + 11 auto).

apps/mana/CLAUDE.md:
- New "Tool Coverage" table in the AI Workbench section listing all
  tools per module with their policy (propose vs auto).
- New "Templates" subsection documenting the two-section gallery
  (agent vs workbench templates), the seed-handler registry, and
  the current handlers (meditate, habits, goals).
- Architecture cross-reference updated to include §23.

docs/architecture/COMPANION_BRAIN_ARCHITECTURE.md:
- §23.2 gains a "Server-Side Research (mana-ai, ab v0.6)" subsection
  explaining how NewsResearchClient mirrors the client-side research
  pre-step: same endpoints, same trigger regex, but HTTP-direct from
  the Docker network instead of SvelteKit-internal.

docs/plans/README.md:
- workbench-templates.md added to the roadmap table (T1 shipped).
- Multi-agent description updated to mention 28 tools + server-side
  web-research.
- Architecture cross-reference includes §23.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:35:40 +02:00
Till JS
8a0bf93699 chore(cloud-tier): upgrade default model gemini-2.0-flash → gemini-2.5-flash
gemini-2.0-flash is deprecated June 1 2026. gemini-2.5-flash has been
stable since Q1 2026 with similar pricing ($0.15/$0.60 per 1M tokens
vs $0.10/$0.40 — pricing table already had the entry).

Three files touched:
- packages/shared-llm/src/backends/cloud.ts — client default
- services/mana-llm/src/config.py — server default
- services/mana-llm/src/providers/google.py — Ollama→Gemini fallback
  map + constructor default + deduplicated model list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:32:03 +02:00
Till JS
23b8cc13fb feat(ai-tools): server-side web-research + contacts for agents
Two major tool expansions — the Recherche-Agent and Today-Agent can
now research the web autonomously (no browser needed), and a future
Meeting-Prep agent can read + create contacts.

=== research_news (server-side execution) ===

The biggest addition: mana-ai can now call mana-api's news-research
endpoints (POST /discover + /search) directly, without a browser.

Infrastructure:
- services/mana-ai/src/planner/news-research-client.ts — full HTTP
  client with discover→search pipeline. 15s/30s timeouts. Graceful
  null on any failure (network, mana-api down, bad response) so the
  tick never crashes from research errors.
- config.manaApiUrl added (default http://localhost:3060); wired in
  docker-compose.macmini.yml as http://mana-api:3060 + depends_on
  mana-api with service_healthy condition.

Pre-planning research step (cron/tick.ts):
- Before the planner prompt is built, the tick checks if the
  mission's objective or conceptMarkdown matches research keywords
  (same RESEARCH_TRIGGER regex the webapp uses). When it matches:
  * NewsResearchClient.research(objective) runs discovery + search
  * Results are injected as a synthetic ResolvedInput with id
    '__web-research__' and a formatted markdown context block
  * The Planner then sees real article URLs/titles/excerpts and can
    reference them in create_note / save_news_article steps
  * Log line: "pre-research: N feeds, M articles"

Tool registration:
- research_news added to AI_PROPOSABLE_TOOL_NAMES + mana-ai tools.ts
  with params (query, language?, limit?). This lets the planner also
  explicitly propose a research step as a PlanStep (in addition to
  the pre-planning auto-injection).

=== create_contact ===

- Added to AI_PROPOSABLE_TOOL_NAMES + mana-ai tools.ts with params
  (firstName required, lastName/email/phone/company/notes optional).
- Contacts are encrypted at rest; server planner can plan the step
  but execution stays on the webapp (same as all propose tools).
  Full server-side contact resolution via Key-Grant is a future
  enhancement.
- get_contacts added to webapp AUTO_TOOLS so agents can inspect
  existing contacts without nagging (read-only, auto-policy).

Module coverage now:
   todo (5)   calendar (2)   notes (5)   places (4)
   drink (3)  food (2)       news (1)    journal (1)
   habits (3)  news-research (1)  contacts (1)

  11 modules, 28 tools total (17 propose, 11 auto).

Tests: mana-ai 41/41 (drift-guard passes), shared-ai type-check
clean, webapp svelte-check 0 errors, 0 warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:25:45 +02:00
Till JS
1266b583e4 feat(ai-tools): unlock create_note + create_journal_entry + habit tools for agents
Closes the three biggest tool-coverage gaps so the shipped agent
templates can actually do their job end-to-end. Before this, the
Recherche-Agent couldn't create notes (only edit), the Today-Agent
couldn't create journal entries, and no habit-related tool was
server-proposable at all.

shared-ai (proposable-tools.ts):
- create_note (notes) — key unlock: Recherche-Agent now creates
  per-source notes and the summary report.
- create_journal_entry (journal) — key unlock: Today-Agent proposes
  a poem as a journal entry with optional mood.
- create_habit (habits) — agent can suggest new habits.
- log_habit (habits) — agent can log a habit completion for today.

Organized the list with per-module section comments for readability
now that we're at 15 proposable tools.

mana-ai (planner/tools.ts):
- 5 new tool definitions with full parameter schemas:
  * create_note (title, content?)
  * create_journal_entry (content, title?, mood? enum)
  * create_habit (title, icon, color)
  * log_habit (habitId, note?)
- Drift-guard contract test passes (41/41) — confirms the mana-ai
  tool list is in sync with the shared-ai canonical set.

Webapp (policy.ts):
- get_habits added to AUTO_TOOLS (read-only; agent can inspect
  which habits exist without nagging the user for approval).
- list_notes added to AUTO_TOOLS (was already used in the reasoning
  loop but missing from the explicit auto-list; the planner default
  fell through to 'propose' which was wasteful for a read op).

Module coverage after this change:
   todo (5 tools)   calendar (2)   notes (5 incl. create)
   places (4)       drink (3)      food (2)
   news (1)         journal (1)    habits (3)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:00:17 +02:00
Till JS
4d9b16a683 feat(notes): list + update + append + add_tag tools for the AI
Makes the "read all notes and tag them #Natur/#Technologie/…" use case
fully functional. Four new ModuleTool entries in notes/tools.ts:

- list_notes(limit?, query?, includeArchived?) — auto, read-only. Returns
  id + title + excerpt so the planner can reference concrete notes
  without dumping full bodies.
- update_note(noteId, title?, content?) — proposable. Destructive full
  overwrite. Docstring nudges toward append_to_note when applicable.
- append_to_note(noteId, content) — proposable, non-destructive. Handles
  the trailing-newline separator so markdown stays clean.
- add_tag_to_note(noteId, tag) — proposable, idempotent, case-insensitive.
  Strips leading #, replaces spaces with _, skips if already present.
  Exactly the categorization primitive the user asked for.

All three writes are added to AI_PROPOSABLE_TOOL_NAMES so both the
webapp policy and mana-ai's boot-time drift guard agree (now 11 tools).
Mirrored in services/mana-ai/src/planner/tools.ts.

AiProposalInbox mounted on /notes so approvals land inline in the
notes module too (already appears in the mission-detail cross-module
inbox via the earlier commit).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 00:24:48 +02:00
Till JS
b768a0ffce refactor(shared-rss): extract RSS parsing + Readability into one package
news-ingester and apps/api both shipped their own copy of rss-parser
+ jsdom + Readability glue. Single source now in packages/shared-rss.
Adds discoverFeeds (rel=alternate + common-paths probe) and validateFeed
which News Research will use. JSDOM virtualConsole is silenced once,
in the package, instead of in two parallel call sites.

- packages/shared-rss: parse, extract, discover, validate
- services/news-ingester: drop local parsers, depend on @mana/shared-rss
- apps/api: drop @mozilla/readability + jsdom direct deps, use shared

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:30:44 +02:00
Till JS
e8077a44b4 docs: flesh out Multi-Agent plan shipping log + Team Workbench forward plan
The Multi-Agent Workbench shipped end-to-end (commits 1771063df through
7c89eb625). This commit turns the plan doc into a proper history + post-
mortem and captures the deferred Team-Workbench as its own forward plan
so the architectural breadcrumbs don't rot.

docs/plans/multi-agent-workbench.md:
- Status bumped to  Shipped; every phase checkbox flipped.
- Open-questions section rewritten with the decisions that were
  actually made (name-unique via store write-time check, per-source
  system principalIds, policy fully migrated, scene binding default-
  empty with smart suggestion).
- New "Shipping-Historie" table mapping each phase to its commit, the
  number of files touched, and the test outcome.
- New "Lessons Learnt + Follow-Up Ideen" with:
  * What went better than expected (L3 Actor cutover, getOrCreate
    instead of unique index, displayName caching)
  * Thin spots worth revisiting (avatar not on Actor, missing token
    counter for budget, no missions list on agent detail, no
    drag-reassign, scene binding doesn't drive filters yet)
  * Five deferred follow-up projects (team features, agent memory
    self-update, agent-to-agent messaging, meta-planner, per-agent
    encryption domains)

docs/plans/team-workbench.md (NEW):
- Full forward-looking plan for the deferred Team-Workbench.
- Two use-cases (human multi-user vs multi-agent sharing team
  context) with the observation that they share the same infra.
- Decision candidates table (still open — meant as T0 RFC fodder,
  not baked in).
- Architecture sketch with data-model deltas over the current
  single-user shape.
- Encryption subsection dedicated to the hardest problems: team-key
  wrapping per member (reuses Mission-Grant pattern), member-removal
  rotation (lazy vs eager), Zero-Knowledge-mode incompatibility.
- T0..T6 phasing (~7 weeks for a clean first-pass).
- Section "Wie Multi-Agent dafür den Weg geebnet hat" enumerating
  the four invariants the shipped Phase 0-7 deliberately preserved
  to make this plan cheap when it lands.

docs/plans/README.md (NEW):
- Index doc with the AI/Workbench roadmap as an ASCII flow so future
  contributors can locate themselves in the sequence without reading
  three 400-line plans first.

docs/future/AI_AGENTS_IDEAS.md:
- Header marks Point 1 (encrypted tables) as shipped via the Mission
  Grant plan; points 2-8 stay relevant. Cross-link to all three plan
  docs so this stays the go-to backlog.

services/mana-ai/CLAUDE.md:
- Design-context header expanded to link to all four related docs
  (arch §20-22, both shipped plans, forward team plan, ideas backlog).

No code changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:17:04 +02:00
Till JS
7c89eb625e feat(ai): workbench agent filter + proposal agent chip + docs (Phase 6+7)
Phase 6 — Multi-Agent observability:
- AI Workbench timeline gets a per-agent filter (dropdown with avatars)
  alongside module + mission. TimelineBucket gains agentId +
  agentDisplayName, projected off the bucket's first AI actor.
- Bucket header now leads with the agent's avatar + name (lookup via
  the live useAgents query so renamed agents reflect instantly) and
  falls back to Actor.displayName for deleted agents.
- AiProposalInbox card header replaces the generic Sparkle + "KI
  schlägt vor" with an agent chip "🤖 Cashflow Watcher schlägt vor"
  using the cached Actor.displayName. Ghost-agent label preserved
  via the cached displayName even when the agent record is gone.

Phase 7 — Docs:
- docs/architecture/COMPANION_BRAIN_ARCHITECTURE.md §22 added:
  data model, identity flow, tick gate order, Scene-Agent binding
  semantics, non-goals.
- services/mana-ai/CLAUDE.md status bumped to v0.5 (Multi-Agent
  Workbench) with the per-agent runner features + metrics listed.
- apps/mana/CLAUDE.md AI Workbench section rewritten to cover the
  Agent primitive, per-agent policy, scene lens, and the updated
  timeline header.

Multi-Agent rollout is code-complete end-to-end:
  Phase 0 Plan ✓  Phase 4 Policy-per-agent ✓
  Phase 1 Actor identity ✓  Phase 5 Agent UI + Scene lens ✓
  Phase 2 Agent CRUD ✓  Phase 6 Observability ✓
  Phase 3 Tick agent-aware ✓  Phase 7 Docs ✓

Tests: webapp svelte-check 0 errors, 0 warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:08:42 +02:00
Till JS
f7426ab40f feat(ai): policy is read from the owning agent (Phase 4)
Until now AiPolicy lived as a user-global setting consulted for every
AI action. With agents as the principal unit of AI behavior, policy
belongs on the agent — different agents can be aggressive about tasks
but conservative about calendar edits, etc.

Webapp (tools/executor.ts):
- When an AI actor invokes a tool, the executor looks up the owning
  agent via getAgent(actor.principalId) and passes agent.policy into
  resolvePolicy. Falls back to DEFAULT_AI_POLICY when the agent record
  is missing (legacy write, deleted agent, race) so no tool call can
  silently bypass the propose/deny path.
- resolvePolicy already accepted an optional policy arg, so the call
  site change is a single line plus the agent load.

Server (mana-ai):
- ServerAgent gains an optional policy field, projected off the same
  plaintext JSONB that the webapp writes.
- Tick loop filters AI_AVAILABLE_TOOLS through filterToolsByAgentPolicy
  before passing them to the planner prompt. Resolution order mirrors
  the webapp: tools[name] → defaultsByModule → defaultForAi; 'deny'
  drops the tool so the LLM never even sees it.

Phase 5 will surface a per-agent policy editor on the agent-detail
UI. Until then all agents inherit DEFAULT_AI_POLICY (baked in during
createAgent), which means no behavior change for existing users —
every tool that was 'propose' before is still 'propose' now, just
reached via agent.policy instead of the user-level singleton.

Tests: mana-ai 41/41, webapp svelte-check clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:43:04 +02:00
Till JS
0af50f0166 feat(mana-ai): agent-aware tick loop + snapshot projection (Phase 3)
Third phase of the Multi-Agent Workbench. The background mission
runner now respects the owning Agent: agent state gates whether
a mission runs, concurrency is capped per-agent, and server-produced
iterations carry the agent's identity as their Actor.

Data layer:
- db/migrate.ts: new mana_ai.agent_snapshots table (mirrors
  mission_snapshots) with indexes on (user_id, last_applied_at) and
  a partial index on active agents.
- db/agents-projection.ts: refreshAgentSnapshots (incremental LWW
  replay over sync_changes appId='ai' table='agents') +
  loadActiveAgents / loadAgent helpers. mergeRaw exported for tests.
- db/missions-projection.ts: ServerMission.agentId + projection
  reads the JSONB field (undefined for legacy missions).

Tick integration (cron/tick.ts):
- Refreshes both snapshot tables on every pass (parallel).
- Per-user in-tick agent cache (Map<userId, Map<agentId, Agent>>)
  so N missions for one user hit the DB once.
- Gate order: agent archived → skip silently; agent paused → skip;
  per-agent maxConcurrentMissions exhausted this tick → defer to next.
  All skip paths bump mana_ai_agent_decisions_total{decision}.
- Prompt injection: withAgentContext prepends an <agent_context>
  block to the system prompt with the agent's name + role, and
  plaintext systemPrompt + memory when available. Ciphertext
  (enc:1:… blobs) are skipped — server has no key by design. Mirrors
  the Mission Grant privacy stance: encrypted context belongs to the
  foreground runner.

Iteration writer (db/iteration-writer.ts):
- New optional `agent` + `iterationId` + `rationale` inputs.
- When agent is present, the sync_changes row is stamped with a
  makeAgentActor actor (principalId=agentId, displayName=agent.name)
  so the webapp timeline groups the write under the right agent.
- Falls back to an AI actor with LEGACY_AI_PRINCIPAL + 'Mana' when
  the mission has no owning agent; ultimate fallback to the
  mission-runner system actor when iterationId is also missing.

Metrics:
- mana_ai_agent_decisions_total{decision=ran|skipped-paused|
  skipped-archived|skipped-concurrency}. Missions without an agent
  don't produce this metric — plansWrittenBackTotal is the universal
  "did we run" counter.

Tests: 41/41 (was 35) including 6 new cases for the agent LWW merge.
mana-ai type-check clean. Webapp svelte-check: 0 errors (4 unrelated
warnings in a different module).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:46:57 +02:00
Till JS
1771063df4 refactor(actor): identity-aware Actor for Multi-Agent Workbench (Phase 1)
Foundation for the Multi-Agent Workbench roadmap
(docs/plans/multi-agent-workbench.md). Every event, record, and
sync_changes row now carries a principal identity + cached display
name in addition to the three-kind discriminator.

Shape change (source of truth in @mana/shared-ai):
  Before: { kind: 'user' | 'ai' | 'system', ...kind-specific fields }
  After:  discriminated union on kind, with
            - common:  principalId, displayName
            - 'user':  principalId = userId
            - 'ai':    principalId = agentId + missionId/iterationId/rationale
            - 'system': principalId = one of SYSTEM_* sentinel strings
                        ('system:projection', 'system:mission-runner', etc.)

Key design calls (from the plan's Q&A):
- System sub-sources get distinct principalIds (not a shared 'system'
  bucket) — lets Workbench filter + revert distinguish projection
  writes from migration writes from server-iteration writes
- displayName cached on the record so renaming an agent doesn't
  rewrite history
- normalizeActor() compat shim fills principalId/displayName on
  legacy rows with 'legacy:*' sentinels so historical events never
  crash the timeline

New exports:
- BaseActor / UserActor / AiActor / SystemActor (narrowed types)
- makeUserActor, makeAgentActor, makeSystemActor (factories with
  typed return)
- SYSTEM_PROJECTION, SYSTEM_RULE, SYSTEM_MIGRATION, SYSTEM_STREAM,
  SYSTEM_MISSION_RUNNER (principalId constants)
- LEGACY_USER_PRINCIPAL, LEGACY_AI_PRINCIPAL, LEGACY_SYSTEM_PRINCIPAL
- isUserActor / isFromMissionRunner predicates

Webapp:
- data/events/actor.ts now re-exports from shared-ai, keeps runtime
  ambient-context (runAs, getCurrentActor) local
- bindDefaultUser(userId, displayName) lets the auth layer replace
  the legacy placeholder with the real logged-in user actor at login
- Mission runner + server-iteration-staging stamp LEGACY_AI_PRINCIPAL
  as the agentId placeholder — Phase 2 will thread the real agent
- Streaks projection uses makeSystemActor(SYSTEM_PROJECTION)
- All test fixtures migrated to factories

Service:
- mana-ai/db/iteration-writer.ts stamps makeSystemActor(
  SYSTEM_MISSION_RUNNER) instead of the old { kind:'system',
  source:'mission-runner' } shape. Phase 3 will switch this to an
  agent actor per mission.

Tests: 26 shared-ai + 21 webapp vitest + 35 mana-ai — all green.
svelte-check: 0 errors, 0 warnings.

No behavior change; purely a type + shape upgrade. Old sync_changes
rows parse via the normalizeActor compat shim at read time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:13:57 +02:00
Till JS
fdb8e60d07 feat(ai): web-research pre-step + auto-kontext + save_news_article tool
Mission objectives matching /recherch|research|news|finde|suche|aktuelle|neueste/i
trigger a synchronous deep-research call (mana-search + mana-llm via the
existing /api/v1/research/start-sync pipeline) before the planner runs;
the summary plus top-8 source URLs are injected as a synthetic ResolvedInput
so the planner can stage save_news_article proposals against real URLs.

The kontext singleton is auto-attached to every mission's planner input
(decrypted client-side, gated on non-empty content + not already linked).

save_news_article is a new proposable tool routed through articlesStore
.saveFromUrl (Readability via /api/v1/news/extract/save). AiProposalInbox
mounted on /news so the user can approve/reject inline. mana-ai planner
tool list mirrors the new tool to keep the boot-time drift guard happy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 19:10:13 +02:00
Till JS
c5a4c5158f fix(mana-crawler): default DATABASE_URL to mana_platform in dev
The Go binary's config.go hardcoded "postgresql://…/mana" as the
DATABASE_URL fallback, but no database named "mana" exists locally
or in the macmini compose stack — the platform DB is mana_platform.
Anyone running the crawler without an explicit override got a
"database \"mana\" does not exist" crash at startup. The dev:crawler
script in package.json had been papering over this by setting
DATABASE_URL explicitly; drop that override now that the binary
default is correct.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 18:18:19 +02:00
Till JS
70c62e7584 fix(infra): include shared-logger in mana-ai + mana-auth Dockerfile installers
Some checks are pending
CI / Build mana-api-gateway (push) Blocked by required conditions
CI / Build mana-crawler (push) Blocked by required conditions
CI / Build mana-media (push) Blocked by required conditions
CI / Build mana-credits (push) Blocked by required conditions
CI / Build mana-web (push) Blocked by required conditions
CI / Build chat-backend (push) Blocked by required conditions
CI / Build chat-web (push) Blocked by required conditions
CI / Build todo-backend (push) Blocked by required conditions
CI / Build todo-web (push) Blocked by required conditions
CI / Build calendar-backend (push) Blocked by required conditions
CI / Build calendar-web (push) Blocked by required conditions
CI / Build clock-web (push) Blocked by required conditions
CI / Build contacts-backend (push) Blocked by required conditions
CI / Build contacts-web (push) Blocked by required conditions
CI / Build presi-web (push) Blocked by required conditions
CI / Build storage-backend (push) Blocked by required conditions
CI / Build storage-web (push) Blocked by required conditions
CI / Build telegram-stats-bot (push) Blocked by required conditions
CI / Build food-backend (push) Blocked by required conditions
CI / Build food-web (push) Blocked by required conditions
CI / Build skilltree-web (push) Blocked by required conditions
Docker Validate / Validate Dockerfiles (push) Waiting to run
Docker Validate / Build calendar-web (push) Blocked by required conditions
Docker Validate / Build quotes-web (push) Blocked by required conditions
Docker Validate / Build todo-backend (push) Blocked by required conditions
Docker Validate / Build todo-web (push) Blocked by required conditions
Docker Validate / Build mana-auth (push) Blocked by required conditions
Docker Validate / Build mana-sync (push) Blocked by required conditions
Docker Validate / Build mana-media (push) Blocked by required conditions
Mirror to Forgejo / Push to Forgejo (push) Waiting to run
shared-hono declares @mana/shared-logger as a workspace dep. Without
that package in the installer stage, Bun fails at runtime with ENOENT
reading /app/packages/shared-hono/node_modules/@mana/shared-logger.
Caught when mana-ai crash-looped on first boot.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:34:08 +02:00
Till JS
f0f5b7dcf6 fix(infra): relocate mana-ai from 3066 to 3067 — port clash with news-ingester
news-ingester already owns 3066 (see docker-compose.macmini.yml:1620).
Moving mana-ai to 3067 — the next free slot in the 306x services block
(credits 3061, user 3062, subscriptions 3063, analytics 3064,
events 3065, news-ingester 3066, mana-ai 3067).

Updated: Dockerfile EXPOSE + HEALTHCHECK, config.ts default,
compose service/healthcheck/port mapping, webapp getManaAiUrl()
fallback, root CLAUDE.md service list, mana-ai/CLAUDE.md, and
COMPANION_BRAIN_ARCHITECTURE.md §20 file map.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:32:07 +02:00
Till JS
cb384bc7ef feat(infra): deploy mana-ai + wire Mission Grant keys via docker-compose
Wire the Mission Key-Grant feature into the production Mac Mini
compose stack so mana-ai can boot and mana-auth can mint grants.

- New mana-ai service block (port 3066) — 256m mem limit, depends on
  postgres + mana-llm, tick interval configurable via
  MANA_AI_TICK_INTERVAL_MS / MANA_AI_TICK_ENABLED. Pulls
  MANA_AI_PRIVATE_KEY_PEM from env; absent = grants silently disabled.
- mana-auth environment gains MANA_AI_PUBLIC_KEY_PEM (default empty
  so existing deployments without the keypair degrade to 503
  GRANT_NOT_CONFIGURED rather than failing to boot).
- mana-auth Dockerfile rewritten to the two-stage pnpm+bun pattern
  used by mana-credits/mana-events — required now that mana-auth has
  a @mana/shared-ai workspace dep. The previous single-stage
  Dockerfile with service-scoped build context couldn't resolve any
  @mana/* imports; that only worked historically because it fell
  through at runtime via a pre-built layer.
- mana-ai Dockerfile copies packages/shared-ai into the installer
  stage alongside shared-hono.

The build contexts for mana-auth flip from services/mana-auth to the
repo root. Existing CI/CD paths (scripts/mac-mini/build-app.sh) pass
through to docker compose build and pick up the new context
automatically — no script edits needed.

Flip-on procedure: on the Mac Mini, set MANA_AI_PUBLIC_KEY_PEM +
MANA_AI_PRIVATE_KEY_PEM in .env (already done, see
secrets/mana-ai/README.md on the host), then rebuild mana-auth +
build mana-ai.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 14:24:50 +02:00
Till JS
74bbfda212 feat(ai): Mission Grant consent UI + Workbench audit tab
Phase 3 — user-facing side of the Mission Key-Grant rollout. Users
can now opt into server-side execution, revoke it, and inspect every
decrypt the runner has performed.

Webapp:
- MissionGrantDialog explains the scope (record count, tables, TTL,
  audit visibility, revocation) and calls requestMissionGrant. Error
  paths render distinctly for ZK, not-configured, missing vault.
- Mission detail shows a Server-Zugriff box with status pill
  (aktiv/abgelaufen/nicht erteilt), Neu-erteilen + Zurückziehen
  buttons. Only renders for missions with at least one encrypted-
  table input.
- store.ts: setMissionGrant / revokeMissionGrant helpers, Proxy-
  stripped like the rest of the store's writes.
- Workbench adds a Timeline/Datenzugriff tab switch. Audit tab queries
  the new GET /api/v1/me/ai-audit endpoint, renders decrypt events
  with color-coded status pills (ok/failed/scope-violation) and
  stable reason strings.
- getManaAiUrl() added to api/config for the audit fetch.

mana-ai:
- GET /api/v1/me/ai-audit (JWT-gated via shared-hono authMiddleware)
  backed by readDecryptAudit() — withUser + RLS double-gate so a user
  can only read their own rows.
- Limit capped at 1000, newest-first.

Missions without a grant continue to work exactly as before; the
grant UI is purely additive.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 13:53:11 +02:00