till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-22 17:46:43 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

refactor(api): use Vercel AI SDK + Zod for nutriphi/planta vision routes

Browse source 
Replaces hand-rolled fetch + JSON.parse + cast-to-any with generateObject
from the AI SDK. The model is constrained to the shared Zod schemas in
@mana/shared-types, so the response is validated at the boundary instead
of trusting Gemini to emit the right shape.

Routes refactored:
  - nutriphi/analysis/photo  (image_url → multimodal `image:` content)
  - nutriphi/analysis/text   (free-text meal description)
  - planta/analysis/identify (plant photo identification)

Why this is materially better than the old code:

  - Runtime validation: if Gemini drifts, the AI SDK throws before the
    response leaves the route. Frontend never sees malformed payloads.
  - Provider-portable: createOpenAICompatible({ baseURL: MANA_LLM_URL })
    keeps mana-llm as the central routing/auth/observability point. The
    AI SDK speaks the OpenAI dialect to mana-llm. If we ever swap the
    backend (e.g. claude-sonnet-4-6 for plant ID), it's a one-line model
    name change.
  - System prompts moved from a multi-line example-laden string to a
    short instruction. The schema itself (with .describe() field hints)
    now carries the structural contract that the JSON-by-example
    paragraph used to encode. Token cost goes down, accuracy goes up.
  - Drops manual fetch error handling (status checks, JSON.parse, cast)
    in favour of try/catch around generateObject. Errors are typed.

mana-llm itself is unchanged — it's still the OpenAI-compatible proxy
in front of Gemini Vision. The AI SDK just gives us a typed client and
a schema-aware decoder on top of it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS 2026-04-09 16:59:51 +02:00
parent c2a75bb8e1
commit 0c0e31d2f3
2 changed files with 85 additions and 95 deletions
Download patch file Download diff file Expand all files Collapse all files

67
apps/api/src/modules/planta/routes.ts
View file

@ -1,15 +1,29 @@
/**
* Planta module — Photo upload + AI plant analysis
* Ported from apps/planta/apps/server
* Planta module — Photo upload + AI plant identification.
*
* CRUD for plants, photos, watering handled by mana-sync.
* This module handles S3 uploads and Gemini Vision analysis.
* CRUD for plants, photos, watering is handled by mana-sync. This
* module owns the server-only operations: photo upload to mana-media
* and structured plant identification via the Vercel AI SDK
* (`generateObject`) using the shared PlantIdentificationSchema in
* @mana/shared-types. See nutriphi/routes.ts for the rationale behind
* the AI SDK + Zod approach.
*/
import { Hono } from 'hono';
import { generateObject } from 'ai';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { PlantIdentificationSchema } from '@mana/shared-types';
import { logger, type AuthVariables } from '@mana/shared-hono';
const LLM_URL = process.env.MANA_LLM_URL || 'http://localhost:3025';
const VISION_MODEL = process.env.VISION_MODEL || 'gemini-2.0-flash';
const llm = createOpenAICompatible({
name: 'mana-llm',
baseURL: `${LLM_URL}/api/v1`,
});
const IDENTIFICATION_PROMPT = `Du bist ein Pflanzenexperte. Analysiere das Pflanzenfoto und liefere eine strukturierte Identifikation mit lateinischem Namen, deutschen Trivialnamen, Pflegehinweisen und einer Gesundheitseinschätzung. Antworte auf Deutsch.`;
const routes = new Hono<{ Variables: AuthVariables }>();
@ -46,43 +60,28 @@ routes.post('/photos/upload', async (c) => {
}
});
// ─── AI Analysis (server-only: Gemini Vision) ───────────────
// ─── AI Analysis (Gemini Vision on uploaded URL) ─────────────
routes.post('/analysis/identify', async (c) => {
const { photoUrl } = await c.req.json();
if (!photoUrl) return c.json({ error: 'photoUrl required' }, 400);
try {
const res = await fetch(`${LLM_URL}/api/v1/chat/completions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
messages: [
{
role: 'system',
content:
'Du bist ein Pflanzenexperte. Analysiere das Bild und gib JSON zurück: {scientificName, commonNames[], confidence, healthAssessment, wateringAdvice, lightAdvice, generalTips[]}',
},
{
role: 'user',
content: [
{ type: 'text', text: 'Analysiere diese Pflanze.' },
{ type: 'image_url', image_url: { url: photoUrl } },
],
},
],
model: process.env.VISION_MODEL || 'gemini-2.0-flash',
response_format: { type: 'json_object' },
}),
const { object } = await generateObject({
model: llm(VISION_MODEL),
schema: PlantIdentificationSchema,
system: IDENTIFICATION_PROMPT,
messages: [
{
role: 'user',
content: [
{ type: 'text', text: 'Analysiere diese Pflanze.' },
{ type: 'image', image: new URL(photoUrl) },
],
},
],
});
if (!res.ok) return c.json({ error: 'AI analysis failed' }, 502);
const data = await res.json();
const content = data.choices?.[0]?.message?.content;
const analysis = typeof content === 'string' ? JSON.parse(content) : content;
return c.json(analysis);
return c.json(object);
} catch (err) {
logger.error('planta.analysis_failed', {
error: err instanceof Error ? err.message : String(err),