docs: add Memoro backend Hono/Bun migration plan

Full migration plan for both NestJS services (backend + audio-backend) to Hono/Bun, including endpoint inventory, auth pattern change (Supabase RLS → service role), and phased implementation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 19:21:10 +02:00 · 2026-03-31 18:43:44 +02:00 · 2026-03-31 18:43:44 +02:00 · c736dd52f2
commit c736dd52f2
parent af33b1cead
1 changed files with 499 additions and 0 deletions
--- a/.claude/plans/memoro-server-hono-migration.md
+++ b/.claude/plans/memoro-server-hono-migration.md
@ -0,0 +1,499 @@
+# Memoro Backend Migration: NestJS → Hono/Bun
+
+**Status:** Planned (not started)
+**Priority:** High — required before Memoro can be reactivated in production
+**Scope:** Two NestJS services → two Hono/Bun servers (keep Supabase DB + Gemini/Azure as-is)
+
+---
+
+## Overview
+
+Memoro currently has two NestJS microservices:
+
+| Service | Path | Port | Responsibility |
+|---------|------|------|----------------|
+| `backend` | `apps/memoro/apps/backend/` | 3001 | Business logic, AI, credits, spaces |
+| `audio-backend` | `apps/memoro/apps/audio-backend/` | 1337 | Audio format conversion, Azure transcription |
+
+Migration target: two Hono/Bun servers following the standard manacore pattern.
+
+| New Service | Path | Port | Replaces |
+|-------------|------|------|----------|
+| `server` | `apps/memoro/apps/server/` | 3015 | `backend/` |
+| `audio-server` | `apps/memoro/apps/audio-server/` | 3016 | `audio-backend/` |
+
+---
+
+## Key Architectural Decisions
+
+### 1. Auth: mana-auth JWT (not Supabase middleware)
+New servers use `authMiddleware()` from `@manacore/shared-hono`. This validates
+the JWT from mana-auth and injects `userId` into context.
+
+**Consequence:** Supabase DB operations must switch from user-scoped JWTs (RLS) to
+**service-role key + explicit `user_id` filter**. All queries that relied on Supabase
+RLS auto-filtering by JWT `sub` claim need `eq('user_id', userId)` added explicitly.
+
+### 2. Auth Proxy: Remove
+The `/auth/*` proxy endpoints (signin, signup, google, apple, refresh, etc.) are
+removed. The web app already calls mana-auth directly. Mobile migration is a separate
+concern — handle in the mobile migration plan.
+
+### 3. Supabase: Keep as-is
+Database stays on Supabase (no Drizzle migration). Only the auth mechanism changes
+(service role key + manual user_id filtering instead of per-user JWT).
+
+### 4. External Services: Unchanged
+- Google Gemini API → keep
+- Azure Speech Service → keep
+- Azure Blob Storage → keep
+- Supabase Storage → keep (audio files)
+
+### 5. Credits: Call mana-credits directly
+Replace calls to `mana-core-middleware /credits/*` with direct calls to the
+`mana-credits` service. Endpoints map 1:1.
+
+### 6. Cleanup: Replace GCP Cloud Scheduler → mana-notify cron
+The audio cleanup cron (currently triggered by GCP Cloud Scheduler) can be
+triggered by an internal cron endpoint called by our existing infrastructure
+or by a scheduled remote agent.
+
+---
+
+## Complete Endpoint Inventory
+
+### `server/` — All Routes
+
+#### `POST /api/v1/memos/process` (was `/memoro/process-uploaded-audio`)
+Receives file path after direct-upload to Supabase Storage. Creates memo record,
+triggers async transcription via `audio-server`.
+
+#### `POST /api/v1/memos/:id/append` (was `/memoro/append-transcription`)
+Appends additional audio recording to existing memo.
+
+#### `POST /api/v1/memos/:id/retry-transcription` (was `/memoro/retry-transcription`)
+Retry failed transcription.
+
+#### `POST /api/v1/memos/:id/retry-headline` (was `/memoro/retry-headline`)
+Retry failed headline generation.
+
+#### `POST /api/v1/memos/:id/reprocess` (was `/memoro/reprocess-memo`)
+Reprocess memo with different parameters (e.g. different blueprint).
+
+#### `GET /api/v1/memos/find-by-job/:jobId` (was `/memoro/find-memo-by-job/:jobId`)
+Lookup memo by Azure batch job ID (used for webhook recovery).
+
+#### `POST /api/v1/memos/combine` (was `/memoro/combine-memos`)
+AI-powered memo combination.
+
+#### `POST /api/v1/memos/:id/question` (was `/memoro/question-memo`)
+Ask a question about a memo's transcript.
+
+#### `POST /api/v1/batch-metadata` (was `/memoro/update-batch-metadata`)
+Update Azure batch job metadata in memo (called by audio-server callback).
+
+#### `GET /api/v1/spaces` (was `/memoro/spaces`)
+List user's spaces.
+
+#### `POST /api/v1/spaces` (was `/memoro/spaces`)
+Create space.
+
+#### `GET /api/v1/spaces/:id` (was `/memoro/spaces/:id`)
+Get space details.
+
+#### `DELETE /api/v1/spaces/:id` (was `/memoro/spaces/:id`)
+Delete space.
+
+#### `POST /api/v1/spaces/:id/memos/link` (was `/memoro/link-memo`)
+Link memo to space.
+
+#### `POST /api/v1/spaces/:id/memos/unlink` (was `/memoro/unlink-memo`)
+Unlink memo from space.
+
+#### `GET /api/v1/spaces/:id/memos` (was `/memoro/spaces/:id/memos`)
+Get all memos in a space.
+
+#### `POST /api/v1/spaces/:id/leave` (was `/memoro/spaces/:id/leave`)
+Leave a space.
+
+#### `GET /api/v1/spaces/:id/invites` (was `/memoro/spaces/:id/invites`)
+List invitations for a space.
+
+#### `POST /api/v1/spaces/:id/invite` (was `/memoro/spaces/:id/invite`)
+Invite user to space.
+
+#### `POST /api/v1/spaces/invites/:inviteId/resend`
+Resend invitation.
+
+#### `DELETE /api/v1/spaces/invites/:inviteId`
+Cancel invitation.
+
+#### `GET /api/v1/invites/pending` (was `/memoro/invites/pending`)
+Get current user's pending invitations.
+
+#### `POST /api/v1/invites/accept` (was `/memoro/spaces/invites/accept`)
+Accept space invitation.
+
+#### `POST /api/v1/invites/decline` (was `/memoro/spaces/invites/decline`)
+Decline space invitation.
+
+#### `GET /api/v1/credits/pricing` (was `/memoro/credits/pricing`)
+Get credit pricing constants. Public (no auth needed).
+
+#### `POST /api/v1/credits/check` (was `/memoro/credits/check-transcription`)
+Validate if user has enough credits for an operation.
+
+#### `POST /api/v1/credits/consume` (was `/memoro/credits/consume-operation`)
+Consume credits for completed operation.
+
+#### `GET /api/v1/settings` (was `/settings`)
+Get all user settings.
+
+#### `GET /api/v1/settings/memoro` (was `/settings/memoro`)
+Get Memoro-specific settings.
+
+#### `PATCH /api/v1/settings/memoro` (was `/settings/memoro`)
+Update Memoro settings.
+
+#### `PATCH /api/v1/settings/memoro/data-usage`
+Update data usage acceptance flag.
+
+#### `PATCH /api/v1/settings/memoro/newsletter`
+Update newsletter opt-in.
+
+#### `PATCH /api/v1/settings/profile` (was `/settings/profile`)
+Update user profile.
+
+#### `GET /api/v1/meetings/bots`
+List user's meeting bots.
+
+#### `POST /api/v1/meetings/bots`
+Create meeting recording bot.
+
+#### `GET /api/v1/meetings/bots/:id`
+Get meeting bot.
+
+#### `POST /api/v1/meetings/bots/:id/stop`
+Stop meeting bot.
+
+#### `GET /api/v1/meetings/recordings`
+List recordings.
+
+#### `GET /api/v1/meetings/recordings/:id`
+Get recording.
+
+#### `POST /api/v1/meetings/recordings/:id/to-memo`
+Convert meeting recording to memo.
+
+#### `POST /api/v1/webhooks/meetings` (was `/meetings/webhooks/bot-events`)
+Webhook for meeting bot completion (no auth, validated by signature/key).
+
+#### Internal (service-to-service, X-Service-Key header):
+
+#### `POST /api/v1/internal/transcription-completed`
+Callback from audio-server when transcription finishes.
+
+#### `POST /api/v1/internal/append-transcription-completed`
+Callback from audio-server for append transcription.
+
+#### `POST /api/v1/internal/batch-metadata`
+Update batch job metadata by memo ID.
+
+#### Cleanup (X-Internal-API-Key header):
+
+#### `POST /api/v1/cleanup/run` (was `/cleanup/trigger-from-cron`)
+Trigger audio cleanup for users with 30-day auto-delete enabled.
+
+#### `POST /api/v1/cleanup/manual` (was `/cleanup/trigger-manual`)
+Manual cleanup trigger.
+
+#### `GET /health`
+Health check.
+
+---
+
+### `audio-server/` — All Routes
+
+#### `POST /api/v1/transcribe` (was `/transcribe-realtime`)
+Main transcription endpoint. Called by `server/`. Handles 4-tier fallback:
+fast → retry → format-convert + retry → batch.
+
+#### `POST /api/v1/transcribe/append` (was append transcription flow)
+Transcribe additional audio for existing memo.
+
+#### `GET /api/v1/duration` (was audio duration check)
+Get duration of audio file at a Supabase Storage path.
+
+#### `POST /api/v1/convert` (was FFmpeg conversion flow)
+Convert audio format (M4A → WAV PCM 16kHz mono). Used internally and by server.
+
+#### `GET /health`
+Health check.
+
+---
+
+## New Folder Structure
+
+```
+apps/memoro/apps/server/
+├── src/
+│   ├── index.ts              # Hono app entry, routes wiring
+│   ├── lib/
+│   │   ├── supabase.ts       # Supabase client factory (service role)
+│   │   ├── ai.ts             # Gemini + Azure OpenAI client (abstracted)
+│   │   └── credits.ts        # mana-credits HTTP client
+│   ├── routes/
+│   │   ├── memos.ts          # Process, retry, reprocess, combine, Q&A
+│   │   ├── spaces.ts         # Spaces CRUD + membership
+│   │   ├── invites.ts        # Invitation management
+│   │   ├── credits.ts        # Credits check/consume/pricing
+│   │   ├── settings.ts       # User settings
+│   │   ├── meetings.ts       # Meeting bots + recordings
+│   │   ├── internal.ts       # Service-to-service callbacks
+│   │   └── cleanup.ts        # Audio cleanup cron endpoint
+│   ├── services/
+│   │   ├── memo.service.ts   # Core memo + transcription orchestration
+│   │   ├── headline.service.ts
+│   │   ├── memory.service.ts
+│   │   ├── question.service.ts
+│   │   └── space.service.ts
+│   └── types.ts              # Shared TypeScript interfaces
+├── package.json
+└── tsconfig.json
+
+apps/memoro/apps/audio-server/
+├── src/
+│   ├── index.ts
+│   ├── lib/
+│   │   ├── azure-speech.ts   # Azure Speech Service client
+│   │   ├── azure-blob.ts     # Azure Blob Storage client
+│   │   └── supabase.ts       # Supabase storage download client
+│   ├── routes/
+│   │   ├── transcribe.ts     # Main + append transcription
+│   │   ├── convert.ts        # FFmpeg audio conversion
+│   │   └── duration.ts       # Audio duration check
+│   └── services/
+│       ├── transcription.service.ts  # 4-tier fallback logic
+│       ├── batch.service.ts          # Azure batch transcription
+│       └── ffmpeg.service.ts         # Format conversion
+├── package.json
+└── tsconfig.json
+```
+
+---
+
+## Implementation Phases
+
+### Phase 1: `audio-server` (simpler, no business logic)
+
+1. Create `apps/memoro/apps/audio-server/` with Hono/Bun scaffold
+2. Port `azure-speech.ts`, `azure-blob.ts`, Supabase storage download
+3. Port FFmpeg service (format conversion, duration detection)
+4. Port transcription service with 4-tier fallback
+5. Port batch transcription service (Azure long-running jobs)
+6. Wire routes: `/api/v1/transcribe`, `/api/v1/convert`, `/api/v1/duration`, `/health`
+7. Add `X-Service-Key` auth guard for all routes
+8. Test against live Azure services manually
+
+**No auth migration needed here** — audio-server is internal-only (called by server, not frontend).
+
+### Phase 2: `server` scaffold + auth
+
+1. Create `apps/memoro/apps/server/` with Hono/Bun scaffold
+2. Setup `authMiddleware()` from `@manacore/shared-hono`
+3. Create Supabase service-role client factory
+4. Implement `X-Service-Key` guard for internal routes
+5. Implement `X-Internal-API-Key` guard for cleanup routes
+6. Port `pricing.constants.ts` and credit calculation helpers
+7. Wire health check and credits/pricing (no auth) routes first
+8. Deploy empty shell and confirm reachability
+
+### Phase 3: Memo routes (core)
+
+1. Port `memo.service.ts` — `createMemoFromUploadedFile()`:
+   - Create memo in Supabase (service role + user_id filter)
+   - Call audio-server `/api/v1/transcribe` async
+   - Return full memo object immediately
+2. Port `handleTranscriptionCompleted()` — update memo after transcription callback
+3. Port internal callback routes (`/api/v1/internal/transcription-completed` etc.)
+4. Port retry routes
+5. Port `append-transcription` flow
+
+### Phase 4: AI routes
+
+1. Port `ai.ts` — Gemini primary, Azure OpenAI fallback, same logic as `ai.service.ts`
+2. Port `headline.service.ts`
+3. Port `question.service.ts` (Q&A)
+4. Port `memory.service.ts` (prompt-based memories)
+5. Wire `/api/v1/memos/combine`, `/api/v1/memos/:id/question`
+
+### Phase 5: Spaces + Invites
+
+1. Port `space.service.ts` — CRUD + membership management
+2. Port invitation flow (invite, accept, decline, resend, cancel)
+3. Wire all `/api/v1/spaces/*` and `/api/v1/invites/*` routes
+
+### Phase 6: Settings + Credits + Cleanup
+
+1. Port settings routes — proxy to mana-credits / mana-user services (or Supabase direct)
+2. Port credits routes — call mana-credits service instead of mana-core-middleware
+3. Port audio cleanup service
+4. Wire `/api/v1/cleanup/*`
+
+### Phase 7: Meetings
+
+1. Port meetings proxy service (proxy to external meeting bot API)
+2. Port webhook handler (`/api/v1/webhooks/meetings`)
+3. Wire all `/api/v1/meetings/*` routes
+
+### Phase 8: Remove auth proxy
+
+Remove the NestJS `auth-proxy` module entirely.
+Document in mobile migration plan that mobile must call mana-auth directly.
+
+### Phase 9: Cutover
+
+1. Update `docker-compose.macmini.yml` to use new server/audio-server images
+2. Update Memoro web app env vars: `PUBLIC_BACKEND_URL` → new server port
+3. Run both old and new in parallel briefly for smoke testing
+4. Remove old `backend/` and `audio-backend/` NestJS services
+
+---
+
+## Environment Variables (New Services)
+
+### `server/.env`
+
+```env
+PORT=3015
+NODE_ENV=production
+
+# Auth
+MANA_CORE_AUTH_URL=http://localhost:3001
+CORS_ORIGINS=http://localhost:5173
+
+# Internal service auth
+SERVICE_KEY=your-internal-service-key
+INTERNAL_API_KEY=your-internal-api-key
+
+# Supabase (service role — no user JWT needed for DB)
+MEMORO_SUPABASE_URL=https://your-memoro-project.supabase.co
+MEMORO_SUPABASE_SERVICE_KEY=your-service-role-key
+
+# External services
+GEMINI_API_KEY=your-gemini-key
+AZURE_OPENAI_ENDPOINT=https://...
+AZURE_OPENAI_KEY=your-key
+AZURE_OPENAI_DEPLOYMENT=gpt-4
+
+# Internal services
+AUDIO_SERVER_URL=http://localhost:3016
+MANA_CREDITS_URL=http://localhost:3004
+
+# App config
+MEMORO_APP_ID=973da0c1-b479-4dac-a1b0-ed09c72caca8
+```
+
+### `audio-server/.env`
+
+```env
+PORT=3016
+NODE_ENV=production
+
+# Internal auth
+SERVICE_KEY=your-internal-service-key
+
+# Azure Speech
+AZURE_SPEECH_KEY=your-key
+AZURE_SPEECH_REGION=swedencentral
+
+# Azure Blob (for batch transcription uploads)
+AZURE_STORAGE_ACCOUNT_NAME=your-account
+AZURE_STORAGE_ACCOUNT_KEY=your-key
+
+# Supabase (read-only: download audio files)
+MEMORO_SUPABASE_URL=https://your-memoro-project.supabase.co
+MEMORO_SUPABASE_SERVICE_KEY=your-service-role-key
+
+# Callback
+MEMORO_SERVER_URL=http://localhost:3015
+```
+
+---
+
+## Critical Implementation Notes
+
+### Supabase RLS → Service Role Pattern
+
+Old NestJS pattern (user JWT for RLS):
+```typescript
+const supabase = createClient(url, anonKey, {
+  global: { headers: { Authorization: `Bearer ${userToken}` } }
+});
+const { data } = await supabase.from('memos').select('*'); // RLS filters by sub
+```
+
+New Hono pattern (service role + explicit filter):
+```typescript
+const supabase = createClient(url, serviceKey); // service role, bypasses RLS
+const userId = c.get('userId'); // from authMiddleware()
+const { data } = await supabase.from('memos').select('*').eq('user_id', userId);
+```
+
+Every single Supabase query that previously relied on RLS needs `eq('user_id', userId)` added.
+Access control for spaces (where user might not be owner) needs explicit join check.
+
+### Async Transcription (fire-and-forget)
+
+Hono/Bun equivalent of `setImmediate()`:
+```typescript
+// Fire-and-forget in Bun
+queueMicrotask(() => processTranscriptionAsync(memoId, ...));
+// or
+setTimeout(() => processTranscriptionAsync(memoId, ...), 0);
+```
+
+The endpoint returns immediately after memo creation; transcription runs in background
+and calls back via `/api/v1/internal/transcription-completed`.
+
+### FFmpeg in Bun
+
+Bun can run FFmpeg via `Bun.spawn`:
+```typescript
+const proc = Bun.spawn(['ffmpeg', '-i', inputPath, '-ar', '16000', '-ac', '1', outputPath]);
+await proc.exited;
+```
+
+FFmpeg must be installed in the Docker image (add `RUN apt-get install -y ffmpeg`).
+
+### Azure Batch Job Recovery
+
+Batch jobs store `jobId` in `memo.metadata.processing.transcription.jobId`.
+A cron endpoint (or background polling) can check stuck jobs and re-trigger
+the `transcription-completed` callback. Implement in Phase 3 alongside main flow.
+
+---
+
+## Files to Reference During Implementation
+
+| File | Purpose |
+|------|---------|
+| `apps/memoro/apps/backend/src/memoro/memoro.service.ts` | Core transcription + memo logic (~2600 lines) |
+| `apps/memoro/apps/backend/src/ai/ai.service.ts` | Gemini/Azure AI routing |
+| `apps/memoro/apps/backend/src/ai/headline/headline.service.ts` | Headline generation |
+| `apps/memoro/apps/backend/src/credits/pricing.constants.ts` | Credit cost constants |
+| `apps/memoro/apps/backend/src/credits/credit-consumption.service.ts` | Credit validation/consumption |
+| `apps/memoro/apps/audio-backend/src/audio.service.ts` | 4-tier transcription fallback |
+| `apps/memoro/apps/backend/src/cleanup/audio-cleanup.service.ts` | 30-day audio cleanup |
+| `apps/todo/apps/server/src/index.ts` | Reference Hono/Bun server pattern |
+| `packages/shared-hono/src/` | `authMiddleware`, `healthRoute`, `errorHandler` |
+
+---
+
+## Open Questions (resolve when starting)
+
+- Port numbers 3015/3016 — confirm no conflicts with other services
+- mana-credits URL in dev environment — confirm port
+- Meeting bot external API — which provider? Are credentials available?
+- Space sync (`/sync/spaces/*`) — is this still needed or replaced by local-store sync?
+- Should audio-server be publicly reachable or internal-only? (Recommendation: internal only)