mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-14 20:21:09 +02:00
feat(mana-ai): v0.7 — cross-tick Deep Research Max pre-planning
Opt-in path for missions that want Gemini Deep Research Max (up to 60 min
per task) instead of the shallow RSS pre-research. Because Max runs well
past a single 60-second tick, the state is carried across ticks:
tick N: submit → INSERT mission_research_jobs row → skip planner
tick N+k: poll → still running → skip planner (metric pending_skips)
tick N+m: poll → completed → inject as ResolvedInput, DELETE row, plan
- ManaResearchClient talks to mana-research's new internal
/v1/internal/research/async endpoints with X-Service-Key +
X-User-Id. Graceful-null on transport errors so a flaky
mana-research never crashes the tick loop.
- New table mana_ai.mission_research_jobs with PK (user_id, mission_id)
— presence is the "pending" flag; delete-on-terminal keeps queries
trivial.
- handleDeepResearch() encapsulates the state machine; planOneMission
now returns a discriminated union (planned | skipped | failed) so
"research pending" isn't miscounted as a parse failure.
- Opt-in at TWO gates to keep cost in check ($3–7/task, 1500 credits
per run):
1. MANA_AI_DEEP_RESEARCH_ENABLED=true server-side (default off)
2. DEEP_RESEARCH_TRIGGER regex matches the mission objective
(strict: "deep research", "tiefe recherche", "umfassende
recherche", "hintergrundrecherche", "deep dive")
Falls back to shallow RSS when either gate fails or the submit
errors upstream.
- Prom metrics: mana_ai_research_jobs_{submitted,completed,failed}_total
labelled by provider, plus _pending_skips_total.
- docker-compose wires MANA_RESEARCH_URL + the opt-in flag and adds
mana-research to depends_on.
- Full write-up with real API response shape (outputs plural, not
OpenAI-style), step-3 MCP-server plan (security-gated, not built),
ops + kill-switch: docs/reports/gemini-deep-research.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
f10a95e842
commit
2a18cb5ee4
12 changed files with 947 additions and 8 deletions
|
|
@ -75,6 +75,28 @@ Der Runner wird agent-bewusst — Missionen gehoeren einem benannten Agent, Poli
|
|||
- [x] `filterToolsByAgentPolicy` schneidet `deny`-Tools raus bevor der Planner sie sieht.
|
||||
- [x] Metrik `mana_ai_agent_decisions_total{decision}`.
|
||||
|
||||
## Status: v0.7 (Cross-Tick Deep Research, 2026-04-22)
|
||||
|
||||
Opt-in asynchroner Deep-Research-Pfad für Missions, die explizit tiefe Recherche wollen. Ruft `mana-research`'s neue Gemini-Deep-Research-Max-Provider (`gemini-deep-research` / `gemini-deep-research-max`) über den internen Service-to-Service-Endpunkt `/api/v1/internal/research/async` auf. Weil Max bis zu 60 min läuft und unser Tick 60 s, läuft das über Ticks hinweg.
|
||||
|
||||
- [x] `ManaResearchClient` (`clients/mana-research.ts`) — HTTP-Client für mana-research's interne async-Endpoints. `X-Service-Key` + `X-User-Id`. Graceful-null bei Fehler.
|
||||
- [x] `mana_ai.mission_research_jobs` Tabelle — ein Row pro pending Job pro Mission, PK `(user_id, mission_id)`. Präsenz = "läuft gerade". Nach `completed`/`failed` wird gelöscht.
|
||||
- [x] Cross-Tick State-Machine in `cron/tick.ts` (`handleDeepResearch`):
|
||||
- Pending Job → poll → `queued`/`running` skip, `completed` inject Result, `failed` fall-through zu Shallow
|
||||
- Kein Job + `DEEP_RESEARCH_TRIGGER` + `config.deepResearchEnabled` → submit + insert → skip
|
||||
- [x] Neuer Trigger `DEEP_RESEARCH_TRIGGER` ist **strenger** als der heutige `RESEARCH_TRIGGER` — matcht nur "deep research", "tiefe recherche", "umfassende recherche", "hintergrundrecherche", "deep dive". Zusätzlich per ENV gegated (`MANA_AI_DEEP_RESEARCH_ENABLED=true`, default off).
|
||||
- [x] `planOneMission` Rückgabetyp ist jetzt eine Discriminated Union `{outcome:'planned'|'skipped'|'failed'}`. `'skipped'` (= research pending) wird **nicht** als parse-failure gezählt.
|
||||
- [x] Metriken: `mana_ai_research_jobs_submitted_total{provider}`, `_completed_total{provider}`, `_failed_total{provider}`, `_pending_skips_total`.
|
||||
- [x] Docker-Compose: `MANA_RESEARCH_URL`, `MANA_AI_DEEP_RESEARCH_ENABLED`, `depends_on: mana-research`.
|
||||
- [x] `@mana/shared-research` als workspace-dep + `type-check` script in `package.json`.
|
||||
|
||||
Bewusst nicht gemacht (offen):
|
||||
- Mission-Config-Flag in der Webapp. Trigger ist heute Regex-basiert, nicht explizit konfigurierbar. Das reicht für den Pilot; wenn wir öffnen, brauchen wir eine UI-Checkbox im Mission-Detail.
|
||||
- Image-Output (`charts`, Nano-Banana). Steckt in `providerRaw`, wird nicht im Answer-Text gerendert.
|
||||
- Streaming-Thought-Summaries. Würde eine eigene SSE-Brücke zum Frontend brauchen.
|
||||
|
||||
Details zum Deep-Research-Flow: [`docs/reports/gemini-deep-research.md`](../../docs/reports/gemini-deep-research.md) §3.2.
|
||||
|
||||
## Status: v0.6 (Server-side Web-Research + erweiterte Tools)
|
||||
|
||||
Der Runner kann jetzt vor dem Planner-Call eigenstaendig Web-Recherche ausfuehren (ohne Browser). Serverseitig werden 31 propose-Tools ueber 16 Module vom Planner vorgeschlagen (auto-Tools laufen ausschliesslich in der Webapp-Reasoning-Loop — der Server sieht nur propose).
|
||||
|
|
@ -130,6 +152,9 @@ curl -X POST -H "X-Service-Key: dev-service-key" http://localhost:3067/internal/
|
|||
PORT=3067
|
||||
SYNC_DATABASE_URL=postgresql://mana:devpassword@localhost:5432/mana_sync
|
||||
MANA_LLM_URL=http://localhost:3020
|
||||
MANA_API_URL=http://localhost:3060 # news-research (RSS, shallow)
|
||||
MANA_RESEARCH_URL=http://localhost:3068 # gemini-deep-research (deep, v0.7+)
|
||||
MANA_AI_DEEP_RESEARCH_ENABLED=false # opt-in gate for Max tasks
|
||||
MANA_SERVICE_KEY=dev-service-key
|
||||
TICK_INTERVAL_MS=60000
|
||||
TICK_ENABLED=true # flip to false to boot HTTP-only (for Docker health-check)
|
||||
|
|
@ -217,11 +242,23 @@ services/mana-ai/
|
|||
├── src/
|
||||
│ ├── index.ts — Hono bootstrap + tick scheduler wiring
|
||||
│ ├── config.ts — Env loading
|
||||
│ ├── cron/tick.ts — Scan loop, overlap-guarded
|
||||
│ ├── cron/tick.ts — Scan loop, overlap-guarded. v0.7: cross-tick
|
||||
│ │ deep-research state machine in
|
||||
│ │ handleDeepResearch()
|
||||
│ ├── clients/
|
||||
│ │ └── mana-research.ts — v0.7: HTTP client for mana-research's
|
||||
│ │ internal /research/async endpoints
|
||||
│ ├── db/
|
||||
│ │ ├── connection.ts — postgres.js pool
|
||||
│ │ └── missions-projection.ts — sync_changes → Mission LWW replay
|
||||
│ ├── planner/client.ts — mana-llm HTTP client (OpenAI-compatible)
|
||||
│ │ ├── migrate.ts — schema bootstrap (mission_snapshots,
|
||||
│ │ │ decrypt_audit, agent_snapshots,
|
||||
│ │ │ token_usage, mission_research_jobs)
|
||||
│ │ ├── missions-projection.ts — sync_changes → Mission LWW replay
|
||||
│ │ └── research-jobs.ts — v0.7: CRUD for mission_research_jobs
|
||||
│ ├── planner/
|
||||
│ │ ├── llm-client.ts — mana-llm HTTP client (OpenAI-compatible)
|
||||
│ │ └── news-research-client.ts — mana-api RSS-based news-research
|
||||
│ │ (shallow pre-planning step)
|
||||
│ └── middleware/service-auth.ts — X-Service-Key gate for /internal/*
|
||||
├── Dockerfile
|
||||
├── package.json
|
||||
|
|
|
|||
|
|
@ -6,11 +6,13 @@
|
|||
"scripts": {
|
||||
"dev": "bun run --watch src/index.ts",
|
||||
"start": "bun run src/index.ts",
|
||||
"test": "bun test"
|
||||
"test": "bun test",
|
||||
"type-check": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@mana/shared-ai": "workspace:*",
|
||||
"@mana/shared-hono": "workspace:*",
|
||||
"@mana/shared-research": "workspace:*",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
"@opentelemetry/exporter-trace-otlp-http": "^0.57.0",
|
||||
"@opentelemetry/resources": "^1.30.0",
|
||||
|
|
|
|||
110
services/mana-ai/src/clients/mana-research.ts
Normal file
110
services/mana-ai/src/clients/mana-research.ts
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
/**
|
||||
* HTTP client for mana-research's internal service-to-service endpoints.
|
||||
*
|
||||
* Used by the deep-research pre-planning step in the tick loop. We
|
||||
* submit the long-running research task on behalf of the mission's
|
||||
* owner, then poll on the next tick until the job is complete. Credits
|
||||
* are reserved / committed against the user by mana-research — this
|
||||
* client is a thin HTTP wrapper.
|
||||
*
|
||||
* Endpoints:
|
||||
* POST /api/v1/internal/research/async — submit { query, provider }
|
||||
* GET /api/v1/internal/research/async/:id — poll
|
||||
*
|
||||
* Auth: X-Service-Key on every call; X-User-Id identifies the owner of
|
||||
* the credit wallet the reservation + commit hit.
|
||||
*
|
||||
* All methods return `null` on transport/parse errors rather than
|
||||
* throwing — a broken mana-research must not crash the tick loop.
|
||||
*/
|
||||
|
||||
import type { AgentAnswer } from '@mana/shared-research';
|
||||
|
||||
export type DeepResearchProvider =
|
||||
| 'openai-deep-research'
|
||||
| 'gemini-deep-research'
|
||||
| 'gemini-deep-research-max';
|
||||
|
||||
export interface SubmitResult {
|
||||
taskId: string;
|
||||
status: 'queued' | 'running';
|
||||
providerId: DeepResearchProvider;
|
||||
costCredits: number;
|
||||
}
|
||||
|
||||
export type PollStatus = 'queued' | 'running' | 'completed' | 'failed' | 'cancelled';
|
||||
|
||||
export interface PollResult {
|
||||
taskId: string;
|
||||
status: PollStatus;
|
||||
providerId: DeepResearchProvider;
|
||||
result?: { answer: AgentAnswer };
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export class ManaResearchClient {
|
||||
constructor(
|
||||
private baseUrl: string,
|
||||
private serviceKey: string
|
||||
) {}
|
||||
|
||||
async submit(
|
||||
userId: string,
|
||||
query: string,
|
||||
provider: DeepResearchProvider
|
||||
): Promise<SubmitResult | null> {
|
||||
try {
|
||||
const res = await fetch(`${this.baseUrl}/api/v1/internal/research/async`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-Service-Key': this.serviceKey,
|
||||
'X-User-Id': userId,
|
||||
'X-App-Id': 'mana-ai',
|
||||
},
|
||||
body: JSON.stringify({ query, provider }),
|
||||
signal: AbortSignal.timeout(30_000),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const body = await res.text().catch(() => '');
|
||||
console.warn(`[mana-research-client] submit ${res.status}: ${body.slice(0, 200)}`);
|
||||
return null;
|
||||
}
|
||||
return (await res.json()) as SubmitResult;
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
'[mana-research-client] submit error:',
|
||||
err instanceof Error ? err.message : String(err)
|
||||
);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async poll(userId: string, taskId: string): Promise<PollResult | null> {
|
||||
try {
|
||||
const res = await fetch(
|
||||
`${this.baseUrl}/api/v1/internal/research/async/${encodeURIComponent(taskId)}`,
|
||||
{
|
||||
headers: {
|
||||
'X-Service-Key': this.serviceKey,
|
||||
'X-User-Id': userId,
|
||||
'X-App-Id': 'mana-ai',
|
||||
},
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
}
|
||||
);
|
||||
if (!res.ok) {
|
||||
const body = await res.text().catch(() => '');
|
||||
console.warn(`[mana-research-client] poll ${res.status}: ${body.slice(0, 200)}`);
|
||||
return null;
|
||||
}
|
||||
return (await res.json()) as PollResult;
|
||||
} catch (err) {
|
||||
console.warn(
|
||||
'[mana-research-client] poll error:',
|
||||
err instanceof Error ? err.message : String(err)
|
||||
);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -17,6 +17,14 @@ export interface Config {
|
|||
* research step to feed web-research context into the planner prompt
|
||||
* before it produces plan steps. */
|
||||
manaApiUrl: string;
|
||||
/** mana-research HTTP endpoint (Hono/Bun, port 3068). Hosts the
|
||||
* async-research submit/poll endpoints that the deep-research pre-
|
||||
* planning path delegates to for multi-minute Gemini tasks. */
|
||||
manaResearchUrl: string;
|
||||
/** Opt-in gate for the deep-research pre-planning path. Default off
|
||||
* — deep runs cost $1–7 per mission, so we only want them triggered
|
||||
* when explicitly enabled on the server. */
|
||||
deepResearchEnabled: boolean;
|
||||
/** Shared key for service-to-service calls. */
|
||||
serviceKey: string;
|
||||
/** How often the background tick scans for due Missions, in ms. */
|
||||
|
|
@ -55,6 +63,8 @@ export function loadConfig(): Config {
|
|||
),
|
||||
manaLlmUrl: requireEnv('MANA_LLM_URL', 'http://localhost:3020'),
|
||||
manaApiUrl: requireEnv('MANA_API_URL', 'http://localhost:3060'),
|
||||
manaResearchUrl: requireEnv('MANA_RESEARCH_URL', 'http://localhost:3068'),
|
||||
deepResearchEnabled: process.env.MANA_AI_DEEP_RESEARCH_ENABLED === 'true',
|
||||
serviceKey: requireEnv('MANA_SERVICE_KEY', 'dev-service-key'),
|
||||
tickIntervalMs: parseInt(process.env.TICK_INTERVAL_MS ?? '60000', 10),
|
||||
tickEnabled: process.env.TICK_ENABLED !== 'false',
|
||||
|
|
|
|||
|
|
@ -49,6 +49,19 @@ import {
|
|||
} from '../metrics';
|
||||
import { unwrapMissionGrant } from '../crypto/unwrap-grant';
|
||||
import { NewsResearchClient } from '../planner/news-research-client';
|
||||
import { ManaResearchClient, type DeepResearchProvider } from '../clients/mana-research';
|
||||
import {
|
||||
deletePendingResearchJob,
|
||||
getPendingResearchJob,
|
||||
insertPendingResearchJob,
|
||||
touchPendingResearchJob,
|
||||
} from '../db/research-jobs';
|
||||
import {
|
||||
researchJobsSubmittedTotal,
|
||||
researchJobsCompletedTotal,
|
||||
researchJobsFailedTotal,
|
||||
researchJobsPendingSkipsTotal,
|
||||
} from '../metrics';
|
||||
import type { ResolverContext } from '../db/resolvers/types';
|
||||
import type { Config } from '../config';
|
||||
import { withSpan } from '../tracing';
|
||||
|
|
@ -61,6 +74,15 @@ const ENC_PREFIX = 'enc:1:';
|
|||
const RESEARCH_TRIGGER =
|
||||
/\b(recherchier|research|news|finde|suche|aktuelle|neueste|today|history|historisch|on this day)/i;
|
||||
|
||||
/** Strict opt-in for the expensive async deep-research path (Gemini
|
||||
* Deep Research Max, ~$3–7 per task). Only matches explicit wording
|
||||
* so users must deliberately ask for it in the mission objective.
|
||||
* Gated further by `config.deepResearchEnabled` at the tick level. */
|
||||
const DEEP_RESEARCH_TRIGGER =
|
||||
/\b(deep research|tiefe recherche|umfassende recherche|hintergrundrecherche|deep dive)\b/i;
|
||||
|
||||
const DEEP_RESEARCH_PROVIDER: DeepResearchProvider = 'gemini-deep-research-max';
|
||||
|
||||
/** True when the value looks like the webapp's AES-GCM wire format. */
|
||||
function isCiphertext(value: string | undefined): value is string {
|
||||
return typeof value === 'string' && value.startsWith(ENC_PREFIX);
|
||||
|
|
@ -198,7 +220,13 @@ export async function runTickOnce(config: Config): Promise<TickStats> {
|
|||
},
|
||||
() => planOneMission(m, llm, sql, agent, config)
|
||||
);
|
||||
if (planResult === null) {
|
||||
if (planResult.outcome === 'skipped') {
|
||||
// Deep-research job still running — pick this mission
|
||||
// back up on the next tick. No plan produced, no
|
||||
// parse-failure accounting.
|
||||
continue;
|
||||
}
|
||||
if (planResult.outcome === 'failed') {
|
||||
parseFailures++;
|
||||
parseFailuresTotal.inc();
|
||||
continue;
|
||||
|
|
@ -270,13 +298,18 @@ export async function runTickOnce(config: Config): Promise<TickStats> {
|
|||
* (see planner/tools.ts) to keep the LLM from fabricating "read
|
||||
* results".
|
||||
*/
|
||||
type PlanMissionOutcome =
|
||||
| { outcome: 'planned'; plan: { summary: string; steps: PlannedStep[] }; tokensUsed: number }
|
||||
| { outcome: 'skipped'; reason: 'research-pending' }
|
||||
| { outcome: 'failed' };
|
||||
|
||||
async function planOneMission(
|
||||
m: ServerMission,
|
||||
llm: ReturnType<typeof createServerLlmClient>,
|
||||
sql: Sql,
|
||||
agent: ServerAgent | null,
|
||||
config: Config
|
||||
): Promise<{ plan: { summary: string; steps: PlannedStep[] }; tokensUsed: number } | null> {
|
||||
): Promise<PlanMissionOutcome> {
|
||||
const mission = serverMissionToSharedMission(m);
|
||||
// Resolve the mission's Key-Grant (if any) once per tick. An absent
|
||||
// grant is NOT an error — plaintext missions (goals-only) run fine
|
||||
|
|
@ -286,8 +319,28 @@ async function planOneMission(
|
|||
const context = await buildResolverContext(m);
|
||||
const resolvedInputs = await resolveServerInputs(sql, m.inputs, m.userId, context);
|
||||
|
||||
// Pre-planning research step (unchanged from pre-migration).
|
||||
if (RESEARCH_TRIGGER.test(m.objective) || RESEARCH_TRIGGER.test(m.conceptMarkdown)) {
|
||||
// ─── Deep research pre-planning (opt-in, cross-tick) ─────────
|
||||
// A pending job means a previous tick submitted an async research
|
||||
// task; we poll here. A completed result is injected as a
|
||||
// ResolvedInput and the plan proceeds normally; queued/running
|
||||
// means we bail this tick and try again next time. No pending job
|
||||
// + the opt-in trigger fires → we submit and bail.
|
||||
const deepInput = await handleDeepResearch(m, sql, config);
|
||||
if (deepInput === 'pending') {
|
||||
return { outcome: 'skipped', reason: 'research-pending' };
|
||||
}
|
||||
if (deepInput) {
|
||||
resolvedInputs.push(deepInput);
|
||||
}
|
||||
|
||||
// Shallow pre-planning research step (RSS-based, synchronous). We
|
||||
// still run this when deep research didn't fire — same behaviour
|
||||
// as before. Skipped when deep research already supplied a
|
||||
// __web-research__ block so we don't double-feed the planner.
|
||||
if (
|
||||
!deepInput &&
|
||||
(RESEARCH_TRIGGER.test(m.objective) || RESEARCH_TRIGGER.test(m.conceptMarkdown))
|
||||
) {
|
||||
const nrc = new NewsResearchClient(config.manaApiUrl);
|
||||
const research = await nrc.research(m.objective, { language: 'de', limit: 8 });
|
||||
if (research) {
|
||||
|
|
@ -352,6 +405,7 @@ async function planOneMission(
|
|||
}
|
||||
|
||||
return {
|
||||
outcome: 'planned',
|
||||
plan: {
|
||||
summary: loopResult.summary ?? '',
|
||||
steps: loopResult.executedCalls.map((ec) => ({
|
||||
|
|
@ -370,8 +424,122 @@ async function planOneMission(
|
|||
providerErrorsTotal.inc({ provider, kind: err.kind });
|
||||
}
|
||||
console.warn(`[mana-ai tick] mission=${m.id} planner loop failed: ${msg}`);
|
||||
return { outcome: 'failed' };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Cross-tick state machine for the deep-research pre-planning path.
|
||||
*
|
||||
* Return value:
|
||||
* - `'pending'`: a job is currently queued/running upstream; caller
|
||||
* must skip this mission for this tick.
|
||||
* - a ResolvedInput: a job just completed, feed it into the planner.
|
||||
* - `null`: no deep-research involvement — fall through to the
|
||||
* existing shallow path.
|
||||
*/
|
||||
async function handleDeepResearch(
|
||||
m: ServerMission,
|
||||
sql: Sql,
|
||||
config: Config
|
||||
): Promise<
|
||||
'pending' | { id: string; module: string; table: string; title: string; content: string } | null
|
||||
> {
|
||||
const client = new ManaResearchClient(config.manaResearchUrl, config.serviceKey);
|
||||
const existing = await getPendingResearchJob(sql, m.userId, m.id);
|
||||
|
||||
if (existing) {
|
||||
const poll = await client.poll(m.userId, existing.taskId);
|
||||
if (!poll) {
|
||||
// Transport failure — keep the job around, try again next tick.
|
||||
await touchPendingResearchJob(sql, m.userId, m.id);
|
||||
researchJobsPendingSkipsTotal.inc();
|
||||
return 'pending';
|
||||
}
|
||||
|
||||
if (poll.status === 'queued' || poll.status === 'running') {
|
||||
await touchPendingResearchJob(sql, m.userId, m.id);
|
||||
researchJobsPendingSkipsTotal.inc();
|
||||
return 'pending';
|
||||
}
|
||||
|
||||
if (poll.status === 'failed' || poll.status === 'cancelled') {
|
||||
await deletePendingResearchJob(sql, m.userId, m.id);
|
||||
researchJobsFailedTotal.inc({ provider: existing.providerId });
|
||||
console.warn(
|
||||
`[mana-ai tick] mission=${m.id} deep-research failed (${existing.providerId}): ${poll.error ?? poll.status}`
|
||||
);
|
||||
// Fall through to shallow pre-planning this tick.
|
||||
return null;
|
||||
}
|
||||
|
||||
// completed
|
||||
await deletePendingResearchJob(sql, m.userId, m.id);
|
||||
researchJobsCompletedTotal.inc({ provider: existing.providerId });
|
||||
const answer = poll.result?.answer;
|
||||
if (!answer || !answer.answer) {
|
||||
console.warn(`[mana-ai tick] mission=${m.id} deep-research completed without body`);
|
||||
return null;
|
||||
}
|
||||
console.log(
|
||||
`[mana-ai tick] mission=${m.id} deep-research done (${existing.providerId}): ` +
|
||||
`${answer.citations.length} citations, ${answer.answer.length} chars`
|
||||
);
|
||||
return {
|
||||
id: '__web-research__',
|
||||
module: 'news-research',
|
||||
table: 'web',
|
||||
title: `Deep Research: "${m.objective.slice(0, 60)}"`,
|
||||
content: formatDeepResearchContext(m.objective, answer),
|
||||
};
|
||||
}
|
||||
|
||||
// No existing job. Do we want to submit one?
|
||||
if (!config.deepResearchEnabled) return null;
|
||||
if (!DEEP_RESEARCH_TRIGGER.test(m.objective) && !DEEP_RESEARCH_TRIGGER.test(m.conceptMarkdown)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const submission = await client.submit(m.userId, m.objective, DEEP_RESEARCH_PROVIDER);
|
||||
if (!submission) {
|
||||
// Submit failed — fall through to shallow so the mission still runs.
|
||||
console.warn(
|
||||
`[mana-ai tick] mission=${m.id} deep-research submit failed, falling back to shallow`
|
||||
);
|
||||
return null;
|
||||
}
|
||||
await insertPendingResearchJob(sql, m.userId, m.id, submission.taskId, submission.providerId);
|
||||
researchJobsSubmittedTotal.inc({ provider: submission.providerId });
|
||||
researchJobsPendingSkipsTotal.inc();
|
||||
console.log(
|
||||
`[mana-ai tick] mission=${m.id} deep-research submitted ` +
|
||||
`(${submission.providerId}, task=${submission.taskId.slice(0, 16)}…, ${submission.costCredits}c)`
|
||||
);
|
||||
return 'pending';
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the deep-research answer into the same markdown-shape the
|
||||
* shallow pre-research step produces, so downstream planner prompts
|
||||
* don't need to distinguish the two sources.
|
||||
*/
|
||||
function formatDeepResearchContext(
|
||||
query: string,
|
||||
answer: import('@mana/shared-research').AgentAnswer
|
||||
): string {
|
||||
const lines: string[] = [`# Deep-Research: "${query}"`, '', answer.answer.trim(), ''];
|
||||
if (answer.citations.length > 0) {
|
||||
lines.push('## Quellen');
|
||||
for (const c of answer.citations) {
|
||||
lines.push(`- [${c.title}](${c.url})${c.snippet ? ` — ${c.snippet}` : ''}`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
lines.push(
|
||||
'---',
|
||||
'Nutze diese Quellen fuer deinen Plan. Verwende nur URLs die oben stehen; erfinde keine.'
|
||||
);
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/** Parse provider name off a `provider/model` string. Used purely for
|
||||
|
|
|
|||
|
|
@ -119,6 +119,30 @@ export async function migrate(sql: Sql): Promise<void> {
|
|||
WHERE record->>'state' = 'active'
|
||||
`;
|
||||
|
||||
// ─── Pending deep-research jobs ──────────────────────────────
|
||||
// When a mission's pre-planning step kicks off a long-running
|
||||
// research task on mana-research (gemini-deep-research[-max] or
|
||||
// openai-deep-research), we record it here so the NEXT tick knows
|
||||
// to poll instead of re-submitting. One row per (user, mission)
|
||||
// while the job is outstanding; the row is DELETED as soon as the
|
||||
// job completes / fails, so presence == "a job is pending".
|
||||
await sql`
|
||||
CREATE TABLE IF NOT EXISTS mana_ai.mission_research_jobs (
|
||||
user_id TEXT NOT NULL,
|
||||
mission_id TEXT NOT NULL,
|
||||
task_id TEXT NOT NULL,
|
||||
provider_id TEXT NOT NULL,
|
||||
submitted_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
last_polled_at TIMESTAMPTZ,
|
||||
PRIMARY KEY (user_id, mission_id)
|
||||
)
|
||||
`;
|
||||
|
||||
await sql`
|
||||
CREATE INDEX IF NOT EXISTS idx_mission_research_jobs_user
|
||||
ON mana_ai.mission_research_jobs (user_id, submitted_at DESC)
|
||||
`;
|
||||
|
||||
// ─── Token usage tracking (Budget Enforcement) ──────────────
|
||||
// Append-only log of token consumption per planner call. The tick
|
||||
// loop queries the rolling 24h window to enforce Agent.maxTokensPerDay.
|
||||
|
|
|
|||
103
services/mana-ai/src/db/research-jobs.ts
Normal file
103
services/mana-ai/src/db/research-jobs.ts
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
/**
|
||||
* Pending deep-research jobs — bookkeeping for the cross-tick state
|
||||
* machine in the mission planner.
|
||||
*
|
||||
* Lifecycle:
|
||||
* tick N: mission triggers deep research → submit via mana-research
|
||||
* → insert row { taskId, providerId, submitted_at }
|
||||
* → skip planner this tick (result not ready)
|
||||
* tick N+k: row present → poll via mana-research
|
||||
* → if still running: touch last_polled_at, skip planner
|
||||
* → if completed: read result, DELETE row, feed planner
|
||||
* → if failed: DELETE row (mission goes through normal
|
||||
* shallow path next tick)
|
||||
*
|
||||
* Storage only tracks the pending phase. Finished results are consumed
|
||||
* immediately by the tick that sees them — no persistence beyond the
|
||||
* resulting iteration written back to sync_changes.
|
||||
*/
|
||||
|
||||
import type { Sql } from './connection';
|
||||
|
||||
export interface PendingResearchJob {
|
||||
userId: string;
|
||||
missionId: string;
|
||||
taskId: string;
|
||||
providerId: string;
|
||||
submittedAt: Date;
|
||||
lastPolledAt: Date | null;
|
||||
}
|
||||
|
||||
export async function getPendingResearchJob(
|
||||
sql: Sql,
|
||||
userId: string,
|
||||
missionId: string
|
||||
): Promise<PendingResearchJob | null> {
|
||||
const rows = await sql<
|
||||
{
|
||||
user_id: string;
|
||||
mission_id: string;
|
||||
task_id: string;
|
||||
provider_id: string;
|
||||
submitted_at: Date;
|
||||
last_polled_at: Date | null;
|
||||
}[]
|
||||
>`
|
||||
SELECT user_id, mission_id, task_id, provider_id, submitted_at, last_polled_at
|
||||
FROM mana_ai.mission_research_jobs
|
||||
WHERE user_id = ${userId} AND mission_id = ${missionId}
|
||||
LIMIT 1
|
||||
`;
|
||||
if (rows.length === 0) return null;
|
||||
const r = rows[0];
|
||||
return {
|
||||
userId: r.user_id,
|
||||
missionId: r.mission_id,
|
||||
taskId: r.task_id,
|
||||
providerId: r.provider_id,
|
||||
submittedAt: r.submitted_at,
|
||||
lastPolledAt: r.last_polled_at,
|
||||
};
|
||||
}
|
||||
|
||||
export async function insertPendingResearchJob(
|
||||
sql: Sql,
|
||||
userId: string,
|
||||
missionId: string,
|
||||
taskId: string,
|
||||
providerId: string
|
||||
): Promise<void> {
|
||||
await sql`
|
||||
INSERT INTO mana_ai.mission_research_jobs
|
||||
(user_id, mission_id, task_id, provider_id)
|
||||
VALUES (${userId}, ${missionId}, ${taskId}, ${providerId})
|
||||
ON CONFLICT (user_id, mission_id) DO UPDATE SET
|
||||
task_id = EXCLUDED.task_id,
|
||||
provider_id = EXCLUDED.provider_id,
|
||||
submitted_at = now(),
|
||||
last_polled_at = NULL
|
||||
`;
|
||||
}
|
||||
|
||||
export async function touchPendingResearchJob(
|
||||
sql: Sql,
|
||||
userId: string,
|
||||
missionId: string
|
||||
): Promise<void> {
|
||||
await sql`
|
||||
UPDATE mana_ai.mission_research_jobs
|
||||
SET last_polled_at = now()
|
||||
WHERE user_id = ${userId} AND mission_id = ${missionId}
|
||||
`;
|
||||
}
|
||||
|
||||
export async function deletePendingResearchJob(
|
||||
sql: Sql,
|
||||
userId: string,
|
||||
missionId: string
|
||||
): Promise<void> {
|
||||
await sql`
|
||||
DELETE FROM mana_ai.mission_research_jobs
|
||||
WHERE user_id = ${userId} AND mission_id = ${missionId}
|
||||
`;
|
||||
}
|
||||
|
|
@ -72,6 +72,35 @@ export const missionErrorsTotal = new Counter({
|
|||
registers: [register],
|
||||
});
|
||||
|
||||
// ── Deep research (async cross-tick pre-planning) ─────────
|
||||
|
||||
export const researchJobsSubmittedTotal = new Counter({
|
||||
name: 'mana_ai_research_jobs_submitted_total',
|
||||
help: 'Deep-research jobs submitted to mana-research (per tick).',
|
||||
labelNames: ['provider'] as const,
|
||||
registers: [register],
|
||||
});
|
||||
|
||||
export const researchJobsCompletedTotal = new Counter({
|
||||
name: 'mana_ai_research_jobs_completed_total',
|
||||
help: 'Deep-research jobs that returned completed results to the planner.',
|
||||
labelNames: ['provider'] as const,
|
||||
registers: [register],
|
||||
});
|
||||
|
||||
export const researchJobsFailedTotal = new Counter({
|
||||
name: 'mana_ai_research_jobs_failed_total',
|
||||
help: 'Deep-research jobs that returned failed/cancelled/timeout.',
|
||||
labelNames: ['provider'] as const,
|
||||
registers: [register],
|
||||
});
|
||||
|
||||
export const researchJobsPendingSkipsTotal = new Counter({
|
||||
name: 'mana_ai_research_jobs_pending_skips_total',
|
||||
help: 'Tick iterations skipped because a deep-research job is still running.',
|
||||
registers: [register],
|
||||
});
|
||||
|
||||
export const plannerLatency = new Histogram({
|
||||
name: 'mana_ai_planner_request_duration_seconds',
|
||||
help: 'Latency of calls to the mana-llm backend.',
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue