docs(services): add CLAUDE.md for stt + events, fix stale entries, flag port collisions

New service docs:
- services/mana-stt/CLAUDE.md — FastAPI surface with Whisper MLX (local),
  WhisperX (rich), and Voxtral (local + Mistral API). Documents the lazy
  backend loading and the launchd plist setup on the Mac Mini.
- services/mana-events/CLAUDE.md — Hono/Bun service for public RSVP and
  event-sharing. Documents the host (JWT) vs public (token) split, the
  rate-limit sweeper, and the createApp factory pattern that lets unit
  tests run without bootstrapping the production sweeper.

Stale entries fixed:
- mana-auth: dropped "rewritten from NestJS / drop-in replacement" — the
  rewrite is the only mana-auth there is now. Email channel updated from
  Brevo SMTP to self-hosted Stalwart (see docs/MAIL_SERVER.md).
- mana-notify: same Brevo → Stalwart fix in the channel table and env
  var defaults.

PORT_SCHEMA.md flagged as aspirational:
- The doc was dated 2026-03-28 and presented as "single source of truth",
  but cross-checking against actual service source files (config.go,
  main.py, start.sh) shows nothing matches. Added a prominent warning at
  the top with the real ports + two confirmed collisions:
  * mana-image-gen and mana-video-gen both default to PORT 3026
  * mana-voice-bot and mana-sync both default to PORT 3050
  Today these are masked because image-gen + voice-bot live on the
  Windows GPU server while video-gen + sync live on the Mac Mini, but
  the moment they share a host they collide. Either execute the planned
  reorg or pick non-colliding ports and rewrite the doc to match
  reality — flagged as a real follow-up.
This commit is contained in:
Till JS 2026-04-08 12:23:48 +02:00
parent a3a47459c6
commit b0a08ce239
5 changed files with 197 additions and 8 deletions

View file

@ -1,8 +1,28 @@
# Port Schema
Canonical port assignments for all Mana services. This is the single source of truth.
> ⚠️ **ASPIRATIONAL — does not match running services as of 2026-04-08.**
>
> This document describes a *planned* reorganization of port assignments
> into clean ranges (30003009 core, 30103019 infra, 30203029 AI/ML, …).
> The reorg has not been executed: the actual ports the services bind to
> live in their `app/main.py` / `start.sh` / `config.ts` and currently
> follow a different scheme. Per-service ports are documented in each
> `services/*/CLAUDE.md`. Notable real-world ports today:
>
> - mana-auth `3001`, mana-credits `3061`, mana-user `3062`,
> mana-subscriptions `3063`, mana-analytics `3064`, mana-events `3065`
> - mana-media `3015`, mana-sync `3050`, mana-search `3021`,
> mana-notify `3040`, mana-crawler `3023`
> - mana-llm `3025`, mana-stt `3020`, mana-tts `3022`,
> mana-image-gen `3026`, mana-video-gen `3026` ⚠️ **collision**,
> mana-voice-bot `3050` ⚠️ **collision with mana-sync**
>
> Two real port collisions exist (image-gen ↔ video-gen, voice-bot ↔ sync)
> that are masked by the fact that they don't all run on the same host
> today. Either execute the reorg below, or pick non-colliding ports and
> update this doc to match reality.
**Last updated:** 2026-03-28
**Originally drafted:** 2026-03-28
## Principles

View file

@ -1,6 +1,6 @@
# mana-auth
Central authentication service for the Mana ecosystem. Rewritten from NestJS (mana-auth) to Hono + Bun.
Central authentication service for the Mana ecosystem. Hono + Bun + Better Auth.
## Tech Stack
@ -11,9 +11,9 @@ Central authentication service for the Mana ecosystem. Rewritten from NestJS (ma
| **Auth** | Better Auth (native Hono handler) |
| **Database** | PostgreSQL + Drizzle ORM |
| **JWT** | EdDSA via Better Auth JWT plugin |
| **Email** | Nodemailer + Brevo SMTP |
| **Email** | Nodemailer → self-hosted Stalwart SMTP (`docs/MAIL_SERVER.md`) |
## Port: 3001 (same as mana-auth — drop-in replacement)
## Port: 3001
## Better Auth Plugins
@ -99,7 +99,7 @@ NODE_ENV=production
MANA_SERVICE_KEY=...
MANA_CREDITS_URL=http://mana-credits:3061
MANA_SUBSCRIPTIONS_URL=http://mana-subscriptions:3063
SMTP_HOST=smtp-relay.brevo.com
SMTP_HOST=stalwart # self-hosted on Mac Mini, see docs/MAIL_SERVER.md
SMTP_PORT=587
SMTP_USER=...
SMTP_PASS=...

View file

@ -0,0 +1,90 @@
# mana-events
Public RSVP and event-sharing service. Hosts publish event snapshots from the Mana calendar/social modules; the public can RSVP via share-link tokens without needing an account.
## Tech Stack
| Layer | Technology |
|-------|------------|
| **Runtime** | Bun |
| **Framework** | Hono |
| **Database** | PostgreSQL + Drizzle ORM |
| **Auth** | EdDSA JWT validation via JWKS from mana-auth (`jose`) — host-side only; RSVP endpoints are intentionally unauthenticated |
## Port: 3065
## Quick Start
```bash
cd services/mana-events
bun run dev # Watch mode
bun run db:push # Push Drizzle schema
bun run db:studio # Drizzle Studio
bun test # Bun unit tests
```
## Why a separate service?
The Mana unified app stores events in the user's local-first store and syncs them via mana-sync — but those events are private and encrypted at rest. Sharing an event with someone *outside* the user's account requires a public, server-rendered snapshot that lives in plaintext on a service the share-link target can hit without login. mana-events is that surface.
## API Endpoints
### Host-side (JWT auth)
| Method | Path | Description |
|--------|------|-------------|
| POST | `/publish` | Publish an event snapshot (mints share token) |
| PUT | `/:eventId/snapshot` | Update an existing published snapshot |
| DELETE | `/:eventId` | Unpublish |
| PUT | `/:eventId/items` | Set the items/tasks list (e.g. potluck dishes) |
| GET | `/:eventId/items` | Read items |
| GET | `/:eventId/rsvps` | List collected RSVPs |
### Public (no auth — share token in path)
| Method | Path | Description |
|--------|------|-------------|
| GET | `/:token` | Fetch the public event snapshot |
| POST | `/:token` | Submit an RSVP |
| POST | `/:token/items/:itemId/claim` | Claim a potluck-style item |
## Rate limiting
Public RSVP endpoints are rate-limited per share token (config: `rsvpPerTokenPerHour`, `rsvpMaxPerToken`) to prevent abuse of the unauthenticated surface. Stale rate-limit buckets are swept periodically by `lib/cleanup.ts` (`startRateBucketSweeper`) — important for long-published events that would otherwise accumulate buckets indefinitely.
## Code layout
```
src/
├── index.ts # Bootstrap (port, sweeper, DB connection)
├── app.ts # Hono app factory — separated so tests can import without bootstrap
├── config.ts # Env loading (PORT, DATABASE_URL, MANA_AUTH_URL, rate limits, CORS)
├── db/
│ ├── connection.ts
│ └── schema/
│ ├── events.ts # published event snapshots + items
│ └── index.ts
├── routes/
│ ├── events.ts # host-side (JWT)
│ ├── rsvp.ts # public (token)
│ └── health.ts
├── middleware/
│ ├── jwt-auth.ts # JWKS-based EdDSA verification
│ └── error-handler.ts
└── lib/
├── cleanup.ts # rate-limit bucket sweeper
└── errors.ts
```
The Hono app lives in `app.ts` (exporting a `createApp(db, config)` factory) so unit tests in `__tests__/` can wire a test DB without triggering the production sweeper or binding the port.
## Configuration
```env
PORT=3065
DATABASE_URL=postgresql://...
MANA_AUTH_URL=http://localhost:3001
CORS_ORIGINS=http://localhost:5173,https://mana.how
```
JWT verification fetches the JWKS from `${MANA_AUTH_URL}/api/auth/jwks` lazily on first use and caches the keyset in-process.

View file

@ -45,7 +45,7 @@ Go replacement for the NestJS mana-notify service. Unified notification microser
| Channel | Service | Worker Concurrency | Max Retries |
|---------|---------|-------------------|-------------|
| Email | Brevo SMTP | 5 | 3 |
| Email | Stalwart SMTP (self-hosted, see `docs/MAIL_SERVER.md`) | 5 | 3 |
| Push | Expo Push API | 10 | 3 |
| Matrix | Matrix Homeserver API | 5 | 3 |
| Webhook | HTTP callback | 10 | 5 |
@ -66,7 +66,7 @@ go test ./... # Test
| `DATABASE_URL` | postgresql://...localhost:5432/mana_notify | PostgreSQL |
| `SERVICE_KEY` | dev-service-key | Service-to-service auth |
| `MANA_AUTH_URL` | http://localhost:3001 | JWT validation |
| `SMTP_HOST` | smtp-relay.brevo.com | SMTP host |
| `SMTP_HOST` | stalwart | SMTP host (self-hosted Stalwart) |
| `SMTP_PORT` | 587 | SMTP port |
| `SMTP_USER` | | SMTP username |
| `SMTP_PASSWORD` | | SMTP password |

View file

@ -0,0 +1,79 @@
# mana-stt
Speech-to-Text service for the Mana ecosystem. Runs on the Mac Mini M4 (Apple Silicon) and exposes a small FastAPI surface that wraps multiple Whisper backends plus Mistral's hosted Voxtral API.
## Tech Stack
| Layer | Technology |
|-------|------------|
| **Runtime** | Python 3.11 + uvicorn |
| **Framework** | FastAPI |
| **Local model** | Whisper Large V3 via [`lightning-whisper-mlx`](https://github.com/mustafaaljadery/lightning-whisper-mlx) (Apple MLX) |
| **Local model (rich)** | WhisperX for word-level timestamps + diarization |
| **Cloud model** | Mistral Voxtral Mini API |
| **Optional** | vLLM Voxtral (GPU) — see `vllm_service.py` |
| **Auth** | JWT validation via mana-auth (`external_auth.py`) + API key fallback (`auth.py`) |
| **Process supervision** | launchd via `com.mana.mana-stt.plist` |
## Port: 3020
## Quick Start
```bash
cd services/mana-stt
./setup.sh # Create venv + install
.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 3020
```
Production runs via launchd on the Mac Mini — `install-service.sh` (single service) or `install-services.sh` (mana-stt + vllm-voxtral together).
## API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| GET | `/health` | Liveness + which backends are loaded |
| GET | `/models` | List available STT models |
| POST | `/transcribe` | Whisper MLX (default, fastest local) |
| POST | `/transcribe/whisperx` | WhisperX with word-level timestamps + diarization |
| POST | `/transcribe/voxtral` | Local Voxtral (vLLM) |
| POST | `/transcribe/voxtral/api` | Mistral Voxtral API (cloud) |
| POST | `/transcribe/auto` | Tries WhisperX first, falls back to Whisper MLX |
All `/transcribe*` endpoints accept multipart `file` upload + optional `language` form field. Auth via `Authorization: Bearer <jwt>` or `X-API-Key`.
## Backends (`app/`)
| File | What it loads |
|------|---------------|
| `whisper_service.py` | Whisper Large V3 via MLX (local, default) |
| `whisper_service_cuda.py` | CUDA Whisper (only used on Windows GPU server) |
| `whisperx_service.py` | WhisperX with diarization (local, slower, richer output) |
| `voxtral_service.py` | Local Voxtral via vLLM (optional, needs the second launchd job) |
| `voxtral_api_service.py` | Mistral hosted Voxtral API (cloud) |
| `vllm_service.py` | vLLM client primitives shared with Voxtral |
| `auth.py` | API key auth (fallback path) |
| `external_auth.py` | JWT auth via mana-auth public key |
Backends are loaded lazily during the FastAPI lifespan and reported by `/health`. Missing dependencies (e.g. CUDA on Mac) are tolerated — the service starts without them.
## Configuration
Reads from `services/mana-stt/.env` (loaded by the launchd plist's `set -a; source .env; set +a`). Relevant variables:
```env
PORT=3020
MANA_AUTH_URL=http://localhost:3001 # JWKS source for JWT verification
MISTRAL_API_KEY=... # only needed for /transcribe/voxtral/api
STT_API_KEY=... # legacy API key fallback
```
## Operations
- **Logs**: launchd writes to `~/Library/Logs/mana-stt.{out,err}.log` (see plist)
- **Metrics**: Prometheus endpoint at `/metrics` if enabled in config; Grafana dashboard JSON checked in at `grafana-dashboard.json`
- **Restart**: `launchctl kickstart -k gui/$(id -u)/com.mana.mana-stt`
## Reference
- `services/mana-stt/README.md` — user-facing setup, model download instructions, language coverage
- `docs/LOCAL_STT_MODELS.md` — WER comparisons, model size/quality tradeoffs