mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-14 23:41:08 +02:00
e82851985b
End-to-end deep-research feature for the questions module: a fire-and-
forget orchestrator in apps/api that plans sub-queries with mana-llm,
retrieves sources via mana-search (with optional Readability extraction),
and streams a structured synthesis back to the web app over SSE.
Backend (apps/api/src/modules/research):
- schema.ts: pgSchema('research') with research_results + sources
- orchestrator.ts: three-phase pipeline (plan / retrieve / synthesise)
with depth-aware config (quick=1×, standard=3×, deep=6× sub-queries)
- pubsub.ts: in-process event bus, single-node, swappable for Redis
- routes.ts: POST /start (202, fire-and-forget), GET /:id/stream (SSE),
POST /start-sync (test only), GET /:id, GET /:id/sources
- Credit gating via @mana/shared-hono/credits — validate up-front,
consume best-effort on `done`. Failed runs cost nothing.
Helpers (apps/api/src/lib):
- llm.ts: llmJson() + llmStream() over mana-llm OpenAI-compat API
- search.ts: webSearch() + bulkExtract() over mana-search Go service
- responses.ts: shared errorResponse / listResponse / validationError
Schema deployment:
- drizzle.config.ts (research-scoped) + drizzle/research/0000_init.sql
hand-authored migration, deployable via psql -f or drizzle-kit push.
- drizzle-kit added as devDep with db:generate / db:push scripts.
Web client (apps/mana/apps/web/src/lib/api/research.ts):
- Typed start() / get() / listSources() / streamProgress(). The stream
uses fetch + ReadableStream (not EventSource) so we can attach the
JWT via Authorization header. Special-cases 402 for friendly toast.
- New PUBLIC_MANA_API_URL plumbing in hooks.server.ts + config.ts.
Module store (modules/questions/stores/answers.svelte.ts):
- New write-side store with createManual / startResearch / accept /
softDelete. startResearch creates an optimistic empty answer, opens
the SSE stream, debounces token deltas in 100ms batches into the
encrypted local row, and on `done` replaces the streamed text with
the parsed { summary, keyPoints, followUps } payload + citations
resolved against research.sources.id.
Citation rendering (modules/questions/components/AnswerCitations.svelte):
- Tokenises [n] markers in the answer body into clickable pills with
hover popovers showing title / host / snippet / external link.
- Lazy-loaded via a session-scoped source cache (stores/sources.svelte.ts)
that deduplicates concurrent fetches.
UI (routes/(app)/questions/[id]/+page.svelte):
- Recherche card with three-state button (start / cancel / re-run),
animated phase indicator, source counter.
- Confirmation dialog warning about web/LLM transmission since the
question itself is locally encrypted.
- Toasts for success / error / cancel via @mana/shared-ui/toast.
- Re-run flow soft-deletes prior research-driven answers but keeps
manual ones intact.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
47 lines
1.8 KiB
SQL
47 lines
1.8 KiB
SQL
-- Research module — initial schema (manually authored to match
|
|
-- apps/api/src/modules/research/schema.ts).
|
|
--
|
|
-- Once `drizzle-kit` is installed in apps/api, future migrations should
|
|
-- be generated via `pnpm --filter @mana/api db:generate` and this file
|
|
-- can become the canonical baseline.
|
|
--
|
|
-- Apply with:
|
|
-- psql "$DATABASE_URL" -f apps/api/drizzle/research/0000_init.sql
|
|
|
|
CREATE SCHEMA IF NOT EXISTS "research";
|
|
|
|
CREATE TABLE IF NOT EXISTS "research"."research_results" (
|
|
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
"user_id" text NOT NULL,
|
|
"question_id" text NOT NULL,
|
|
"depth" text NOT NULL,
|
|
"status" text NOT NULL,
|
|
"sub_queries" jsonb,
|
|
"summary" text,
|
|
"key_points" jsonb,
|
|
"follow_up_questions" jsonb,
|
|
"error_message" text,
|
|
"started_at" timestamptz NOT NULL DEFAULT now(),
|
|
"finished_at" timestamptz
|
|
);
|
|
|
|
CREATE INDEX IF NOT EXISTS "research_results_user_id_idx"
|
|
ON "research"."research_results" ("user_id");
|
|
|
|
CREATE INDEX IF NOT EXISTS "research_results_question_id_idx"
|
|
ON "research"."research_results" ("question_id");
|
|
|
|
CREATE TABLE IF NOT EXISTS "research"."sources" (
|
|
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
"research_result_id" uuid NOT NULL REFERENCES "research"."research_results"("id") ON DELETE CASCADE,
|
|
"url" text NOT NULL,
|
|
"title" text,
|
|
"snippet" text,
|
|
"extracted_content" text,
|
|
"category" text,
|
|
"rank" integer NOT NULL,
|
|
"created_at" timestamptz NOT NULL DEFAULT now()
|
|
);
|
|
|
|
CREATE INDEX IF NOT EXISTS "sources_research_result_id_idx"
|
|
ON "research"."sources" ("research_result_id");
|