diff --git a/docs/PORT_SCHEMA.md b/docs/PORT_SCHEMA.md index 1abf98f88..ac80f2dd7 100644 --- a/docs/PORT_SCHEMA.md +++ b/docs/PORT_SCHEMA.md @@ -1,8 +1,28 @@ # Port Schema -Canonical port assignments for all Mana services. This is the single source of truth. +> ⚠️ **ASPIRATIONAL — does not match running services as of 2026-04-08.** +> +> This document describes a *planned* reorganization of port assignments +> into clean ranges (3000–3009 core, 3010–3019 infra, 3020–3029 AI/ML, …). +> The reorg has not been executed: the actual ports the services bind to +> live in their `app/main.py` / `start.sh` / `config.ts` and currently +> follow a different scheme. Per-service ports are documented in each +> `services/*/CLAUDE.md`. Notable real-world ports today: +> +> - mana-auth `3001`, mana-credits `3061`, mana-user `3062`, +> mana-subscriptions `3063`, mana-analytics `3064`, mana-events `3065` +> - mana-media `3015`, mana-sync `3050`, mana-search `3021`, +> mana-notify `3040`, mana-crawler `3023` +> - mana-llm `3025`, mana-stt `3020`, mana-tts `3022`, +> mana-image-gen `3026`, mana-video-gen `3026` ⚠️ **collision**, +> mana-voice-bot `3050` ⚠️ **collision with mana-sync** +> +> Two real port collisions exist (image-gen ↔ video-gen, voice-bot ↔ sync) +> that are masked by the fact that they don't all run on the same host +> today. Either execute the reorg below, or pick non-colliding ports and +> update this doc to match reality. -**Last updated:** 2026-03-28 +**Originally drafted:** 2026-03-28 ## Principles diff --git a/services/mana-auth/CLAUDE.md b/services/mana-auth/CLAUDE.md index 924474f8c..b7842b2f5 100644 --- a/services/mana-auth/CLAUDE.md +++ b/services/mana-auth/CLAUDE.md @@ -1,6 +1,6 @@ # mana-auth -Central authentication service for the Mana ecosystem. Rewritten from NestJS (mana-auth) to Hono + Bun. +Central authentication service for the Mana ecosystem. Hono + Bun + Better Auth. ## Tech Stack @@ -11,9 +11,9 @@ Central authentication service for the Mana ecosystem. Rewritten from NestJS (ma | **Auth** | Better Auth (native Hono handler) | | **Database** | PostgreSQL + Drizzle ORM | | **JWT** | EdDSA via Better Auth JWT plugin | -| **Email** | Nodemailer + Brevo SMTP | +| **Email** | Nodemailer → self-hosted Stalwart SMTP (`docs/MAIL_SERVER.md`) | -## Port: 3001 (same as mana-auth — drop-in replacement) +## Port: 3001 ## Better Auth Plugins @@ -99,7 +99,7 @@ NODE_ENV=production MANA_SERVICE_KEY=... MANA_CREDITS_URL=http://mana-credits:3061 MANA_SUBSCRIPTIONS_URL=http://mana-subscriptions:3063 -SMTP_HOST=smtp-relay.brevo.com +SMTP_HOST=stalwart # self-hosted on Mac Mini, see docs/MAIL_SERVER.md SMTP_PORT=587 SMTP_USER=... SMTP_PASS=... diff --git a/services/mana-events/CLAUDE.md b/services/mana-events/CLAUDE.md new file mode 100644 index 000000000..fc4d9a5bd --- /dev/null +++ b/services/mana-events/CLAUDE.md @@ -0,0 +1,90 @@ +# mana-events + +Public RSVP and event-sharing service. Hosts publish event snapshots from the Mana calendar/social modules; the public can RSVP via share-link tokens without needing an account. + +## Tech Stack + +| Layer | Technology | +|-------|------------| +| **Runtime** | Bun | +| **Framework** | Hono | +| **Database** | PostgreSQL + Drizzle ORM | +| **Auth** | EdDSA JWT validation via JWKS from mana-auth (`jose`) — host-side only; RSVP endpoints are intentionally unauthenticated | + +## Port: 3065 + +## Quick Start + +```bash +cd services/mana-events +bun run dev # Watch mode +bun run db:push # Push Drizzle schema +bun run db:studio # Drizzle Studio +bun test # Bun unit tests +``` + +## Why a separate service? + +The Mana unified app stores events in the user's local-first store and syncs them via mana-sync — but those events are private and encrypted at rest. Sharing an event with someone *outside* the user's account requires a public, server-rendered snapshot that lives in plaintext on a service the share-link target can hit without login. mana-events is that surface. + +## API Endpoints + +### Host-side (JWT auth) + +| Method | Path | Description | +|--------|------|-------------| +| POST | `/publish` | Publish an event snapshot (mints share token) | +| PUT | `/:eventId/snapshot` | Update an existing published snapshot | +| DELETE | `/:eventId` | Unpublish | +| PUT | `/:eventId/items` | Set the items/tasks list (e.g. potluck dishes) | +| GET | `/:eventId/items` | Read items | +| GET | `/:eventId/rsvps` | List collected RSVPs | + +### Public (no auth — share token in path) + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/:token` | Fetch the public event snapshot | +| POST | `/:token` | Submit an RSVP | +| POST | `/:token/items/:itemId/claim` | Claim a potluck-style item | + +## Rate limiting + +Public RSVP endpoints are rate-limited per share token (config: `rsvpPerTokenPerHour`, `rsvpMaxPerToken`) to prevent abuse of the unauthenticated surface. Stale rate-limit buckets are swept periodically by `lib/cleanup.ts` (`startRateBucketSweeper`) — important for long-published events that would otherwise accumulate buckets indefinitely. + +## Code layout + +``` +src/ +├── index.ts # Bootstrap (port, sweeper, DB connection) +├── app.ts # Hono app factory — separated so tests can import without bootstrap +├── config.ts # Env loading (PORT, DATABASE_URL, MANA_AUTH_URL, rate limits, CORS) +├── db/ +│ ├── connection.ts +│ └── schema/ +│ ├── events.ts # published event snapshots + items +│ └── index.ts +├── routes/ +│ ├── events.ts # host-side (JWT) +│ ├── rsvp.ts # public (token) +│ └── health.ts +├── middleware/ +│ ├── jwt-auth.ts # JWKS-based EdDSA verification +│ └── error-handler.ts +└── lib/ + ├── cleanup.ts # rate-limit bucket sweeper + └── errors.ts +``` + +The Hono app lives in `app.ts` (exporting a `createApp(db, config)` factory) so unit tests in `__tests__/` can wire a test DB without triggering the production sweeper or binding the port. + +## Configuration + +```env +PORT=3065 +DATABASE_URL=postgresql://... +MANA_AUTH_URL=http://localhost:3001 +CORS_ORIGINS=http://localhost:5173,https://mana.how +``` + +JWT verification fetches the JWKS from `${MANA_AUTH_URL}/api/auth/jwks` lazily on first use and caches the keyset in-process. diff --git a/services/mana-notify/CLAUDE.md b/services/mana-notify/CLAUDE.md index 742d962ce..a4d266700 100644 --- a/services/mana-notify/CLAUDE.md +++ b/services/mana-notify/CLAUDE.md @@ -45,7 +45,7 @@ Go replacement for the NestJS mana-notify service. Unified notification microser | Channel | Service | Worker Concurrency | Max Retries | |---------|---------|-------------------|-------------| -| Email | Brevo SMTP | 5 | 3 | +| Email | Stalwart SMTP (self-hosted, see `docs/MAIL_SERVER.md`) | 5 | 3 | | Push | Expo Push API | 10 | 3 | | Matrix | Matrix Homeserver API | 5 | 3 | | Webhook | HTTP callback | 10 | 5 | @@ -66,7 +66,7 @@ go test ./... # Test | `DATABASE_URL` | postgresql://...localhost:5432/mana_notify | PostgreSQL | | `SERVICE_KEY` | dev-service-key | Service-to-service auth | | `MANA_AUTH_URL` | http://localhost:3001 | JWT validation | -| `SMTP_HOST` | smtp-relay.brevo.com | SMTP host | +| `SMTP_HOST` | stalwart | SMTP host (self-hosted Stalwart) | | `SMTP_PORT` | 587 | SMTP port | | `SMTP_USER` | | SMTP username | | `SMTP_PASSWORD` | | SMTP password | diff --git a/services/mana-stt/CLAUDE.md b/services/mana-stt/CLAUDE.md new file mode 100644 index 000000000..6d91d86a1 --- /dev/null +++ b/services/mana-stt/CLAUDE.md @@ -0,0 +1,79 @@ +# mana-stt + +Speech-to-Text service for the Mana ecosystem. Runs on the Mac Mini M4 (Apple Silicon) and exposes a small FastAPI surface that wraps multiple Whisper backends plus Mistral's hosted Voxtral API. + +## Tech Stack + +| Layer | Technology | +|-------|------------| +| **Runtime** | Python 3.11 + uvicorn | +| **Framework** | FastAPI | +| **Local model** | Whisper Large V3 via [`lightning-whisper-mlx`](https://github.com/mustafaaljadery/lightning-whisper-mlx) (Apple MLX) | +| **Local model (rich)** | WhisperX for word-level timestamps + diarization | +| **Cloud model** | Mistral Voxtral Mini API | +| **Optional** | vLLM Voxtral (GPU) — see `vllm_service.py` | +| **Auth** | JWT validation via mana-auth (`external_auth.py`) + API key fallback (`auth.py`) | +| **Process supervision** | launchd via `com.mana.mana-stt.plist` | + +## Port: 3020 + +## Quick Start + +```bash +cd services/mana-stt +./setup.sh # Create venv + install +.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 3020 +``` + +Production runs via launchd on the Mac Mini — `install-service.sh` (single service) or `install-services.sh` (mana-stt + vllm-voxtral together). + +## API Endpoints + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/health` | Liveness + which backends are loaded | +| GET | `/models` | List available STT models | +| POST | `/transcribe` | Whisper MLX (default, fastest local) | +| POST | `/transcribe/whisperx` | WhisperX with word-level timestamps + diarization | +| POST | `/transcribe/voxtral` | Local Voxtral (vLLM) | +| POST | `/transcribe/voxtral/api` | Mistral Voxtral API (cloud) | +| POST | `/transcribe/auto` | Tries WhisperX first, falls back to Whisper MLX | + +All `/transcribe*` endpoints accept multipart `file` upload + optional `language` form field. Auth via `Authorization: Bearer ` or `X-API-Key`. + +## Backends (`app/`) + +| File | What it loads | +|------|---------------| +| `whisper_service.py` | Whisper Large V3 via MLX (local, default) | +| `whisper_service_cuda.py` | CUDA Whisper (only used on Windows GPU server) | +| `whisperx_service.py` | WhisperX with diarization (local, slower, richer output) | +| `voxtral_service.py` | Local Voxtral via vLLM (optional, needs the second launchd job) | +| `voxtral_api_service.py` | Mistral hosted Voxtral API (cloud) | +| `vllm_service.py` | vLLM client primitives shared with Voxtral | +| `auth.py` | API key auth (fallback path) | +| `external_auth.py` | JWT auth via mana-auth public key | + +Backends are loaded lazily during the FastAPI lifespan and reported by `/health`. Missing dependencies (e.g. CUDA on Mac) are tolerated — the service starts without them. + +## Configuration + +Reads from `services/mana-stt/.env` (loaded by the launchd plist's `set -a; source .env; set +a`). Relevant variables: + +```env +PORT=3020 +MANA_AUTH_URL=http://localhost:3001 # JWKS source for JWT verification +MISTRAL_API_KEY=... # only needed for /transcribe/voxtral/api +STT_API_KEY=... # legacy API key fallback +``` + +## Operations + +- **Logs**: launchd writes to `~/Library/Logs/mana-stt.{out,err}.log` (see plist) +- **Metrics**: Prometheus endpoint at `/metrics` if enabled in config; Grafana dashboard JSON checked in at `grafana-dashboard.json` +- **Restart**: `launchctl kickstart -k gui/$(id -u)/com.mana.mana-stt` + +## Reference + +- `services/mana-stt/README.md` — user-facing setup, model download instructions, language coverage +- `docs/LOCAL_STT_MODELS.md` — WER comparisons, model size/quality tradeoffs