mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-17 22:49:40 +02:00
This commit bundles two unrelated changes that were swept together by an
accidental `git add -A` in another working session. Documented here so the
history reflects what's actually inside.
═══════════════════════════════════════════════════════════════════════
1. fix(mana-auth): /api/v1/auth/login mints JWT via auth.handler instead
of api.signInEmail
═══════════════════════════════════════════════════════════════════════
Previous attempt (commit 55cc75e7d) tried to fix the broken JWT mint in
/api/v1/auth/login by switching the cookie name from `mana.session_token`
to `__Secure-mana.session_token` for production. That was necessary but
not sufficient: Better Auth's session cookie value isn't just the raw
session token, it's `<token>.<HMAC>` where the HMAC is derived from the
better-auth secret. Reconstructing the cookie from auth.api.signInEmail's
JSON response only gave us the raw token, so /api/auth/token's
get-session middleware still couldn't validate it and the JWT mint kept
silently failing.
Real fix: do the sign-in via auth.handler (the HTTP path) rather than
auth.api.signInEmail (the SDK path). The handler returns a real fetch
Response with a Set-Cookie header containing the fully signed cookie
envelope. We capture that header verbatim and forward it as the cookie
on the /api/auth/token request, which now passes validation and mints
the JWT correctly.
Verified end-to-end on auth.mana.how:
$ curl -X POST https://auth.mana.how/api/v1/auth/login \
-d '{"email":"...","password":"..."}'
{
"user": {...},
"token": "<session token>",
"accessToken": "eyJhbGciOiJFZERTQSI...", ← real JWT now
"refreshToken": "<session token>"
}
Side benefits:
- Email-not-verified path is now handled by checking
signInResponse.status === 403 directly, no more catching APIError
with the comment-noted async-stream footgun.
- X-Forwarded-For is forwarded explicitly so Better Auth's rate limiter
and our security log see the real client IP.
- The leftover catch block now only handles unexpected exceptions
(network errors etc); the FORBIDDEN-checking logic in it is dead but
harmless and left in for defense in depth.
═══════════════════════════════════════════════════════════════════════
2. chore: remove the entire self-hosted Matrix stack (Synapse, Element,
Manalink, mana-matrix-bot)
═══════════════════════════════════════════════════════════════════════
The Matrix subsystem ran parallel to the main Mana product without any
load-bearing integration: the unified web app never imported matrix-js-sdk,
the chat module uses mana-sync (local-first), and mana-matrix-bot's
plugins duplicated features the unified app already ships natively.
Keeping it alive cost a Synapse + Element + matrix-web + bot container
quartet, three Cloudflare routes, an OIDC provider plugin in mana-auth,
and a steady drip of devlog/dependency churn.
Removed:
- apps/matrix (Manalink web + mobile, ~150 files)
- services/mana-matrix-bot (Go bot with ~20 plugins)
- docker/matrix configs (Synapse + Element)
- synapse/element-web/matrix-web/mana-matrix-bot services in
docker-compose.macmini.yml
- matrix.mana.how/element.mana.how/link.mana.how Cloudflare tunnel routes
- OIDC provider plugin + matrix-synapse trustedClient + matrixUserLinks
table from mana-auth (oauth_* schema definitions also removed)
- MatrixService import path in mana-media (importFromMatrix endpoint)
- Matrix notification channel in mana-notify (worker, metrics, config,
channel_type enum, MatrixOptions handler)
- Matrix entries from shared-branding (mana-apps + app-icons),
notify-client, the i18n bundle, the observatory map, the credits
app-label list, the landing footer/apps page, the prometheus + alerts
+ promtail tier mappings, and the matrix-related deploy paths in
cd-macmini.yml + ci.yml
Devlog/manascore/blueprint entries that mention Matrix are left intact
as historical record. The oauth_* + matrix_user_links Postgres tables
stay on existing prod databases — code can no longer write to them, drop
them in a follow-up migration if you want them gone for real.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
222 lines
6.4 KiB
Markdown
222 lines
6.4 KiB
Markdown
# mana-media - Unified Media Platform
|
|
|
|
Central media handling service for all Mana applications with content-addressable storage (CAS) and automatic deduplication.
|
|
|
|
**Stack:** Hono + Bun (migrated from NestJS 2026-03-28)
|
|
|
|
## Overview
|
|
|
|
mana-media provides:
|
|
- **Content-Addressable Storage** - SHA-256 based deduplication across all apps
|
|
- **Upload API** - File uploads with automatic deduplication
|
|
- **Processing** - Thumbnails, WebP conversion, resizing (via BullMQ)
|
|
- **Delivery** - Optimized file serving, on-the-fly transforms
|
|
|
|
**Port:** 3015 (production)
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Start dependencies (Redis + MinIO + PostgreSQL)
|
|
pnpm docker:up
|
|
|
|
# Create database
|
|
PGPASSWORD=devpassword psql -h localhost -U mana -d postgres -c "CREATE DATABASE mana_media;"
|
|
|
|
# Install dependencies
|
|
cd services/mana-media/apps/api
|
|
pnpm install
|
|
|
|
# Push schema
|
|
pnpm db:push
|
|
|
|
# Start development server
|
|
pnpm dev
|
|
```
|
|
|
|
Service runs on `http://localhost:3015`
|
|
|
|
## API Endpoints
|
|
|
|
### Upload
|
|
```bash
|
|
# Upload file
|
|
curl -X POST http://localhost:3015/api/v1/media/upload \
|
|
-F "file=@image.jpg" \
|
|
-F "app=chat" \
|
|
-F "userId=user123"
|
|
|
|
# Response
|
|
{
|
|
"id": "abc123",
|
|
"status": "processing",
|
|
"hash": "sha256...",
|
|
"urls": {
|
|
"original": "http://localhost:3015/api/v1/media/abc123/file",
|
|
"thumbnail": "http://localhost:3015/api/v1/media/abc123/file/thumb"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Get Media
|
|
```bash
|
|
# Get metadata
|
|
GET /api/v1/media/:id
|
|
|
|
# Get by content hash (check if file exists)
|
|
GET /api/v1/media/hash/:sha256hash
|
|
|
|
# Get original file
|
|
GET /api/v1/media/:id/file
|
|
|
|
# Get thumbnail
|
|
GET /api/v1/media/:id/file/thumb
|
|
|
|
# Get medium variant
|
|
GET /api/v1/media/:id/file/medium
|
|
|
|
# On-the-fly transform
|
|
GET /api/v1/media/:id/transform?w=400&h=300&fit=cover&format=webp
|
|
```
|
|
|
|
### List & Delete
|
|
```bash
|
|
# List media (filter by app/user)
|
|
GET /api/v1/media?app=chat&userId=user123&limit=50
|
|
|
|
# Delete
|
|
DELETE /api/v1/media/:id
|
|
```
|
|
|
|
## Client Library
|
|
|
|
```typescript
|
|
import { MediaClient } from '@mana/media-client';
|
|
|
|
const media = new MediaClient('http://localhost:3050');
|
|
|
|
// Upload
|
|
const result = await media.upload(file, { app: 'chat' });
|
|
|
|
// Wait for processing
|
|
const ready = await media.waitForReady(result.id);
|
|
|
|
// Get URLs
|
|
const thumbUrl = media.getThumbnailUrl(result.id);
|
|
const customUrl = media.getTransformUrl(result.id, {
|
|
width: 400,
|
|
format: 'webp'
|
|
});
|
|
```
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ mana-media (Port 3015) │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ Upload Module │ File uploads, dedup │
|
|
│ Process Module │ Sharp thumbnail generation (BullMQ) │
|
|
│ Storage Module │ MinIO S3 abstraction │
|
|
│ Delivery Module │ File serving + on-the-fly transforms │
|
|
│ Database Module │ PostgreSQL + Drizzle ORM │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│ │ │
|
|
▼ ▼ ▼
|
|
┌─────────┐ ┌─────────┐ ┌────────────┐
|
|
│ Redis │ │ MinIO │ │ PostgreSQL │
|
|
│ BullMQ │ │ Storage │ │ mana_media │
|
|
└─────────┘ └─────────┘ └────────────┘
|
|
```
|
|
|
|
## Database Schema
|
|
|
|
### media (Content-Addressable Storage)
|
|
| Column | Type | Description |
|
|
|--------|------|-------------|
|
|
| id | UUID | Primary key |
|
|
| content_hash | TEXT | SHA-256 hash (unique) |
|
|
| original_name | TEXT | Original filename |
|
|
| mime_type | TEXT | MIME type |
|
|
| size | BIGINT | File size in bytes |
|
|
| original_key | TEXT | S3 storage key |
|
|
| status | TEXT | uploading/processing/ready/failed |
|
|
| thumbnail_key | TEXT | Thumbnail S3 key |
|
|
| width/height | INT | Image dimensions |
|
|
|
|
### media_references (User ownership)
|
|
| Column | Type | Description |
|
|
|--------|------|-------------|
|
|
| id | UUID | Primary key |
|
|
| media_id | UUID | FK to media |
|
|
| user_id | UUID | Owner user ID |
|
|
| app | TEXT | Source app (nutriphi, contacts, etc.) |
|
|
| source_url | TEXT | Original source URL |
|
|
|
|
## Processing Pipeline
|
|
|
|
| File Type | Generated Variants |
|
|
|-----------|-------------------|
|
|
| Images | thumb (200x200), medium (800x800), large (1920x1920) |
|
|
| Videos | (planned) thumbnail, HLS streaming |
|
|
| Documents | (planned) thumbnail, text extraction |
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| PORT | 3015 | API port |
|
|
| DATABASE_URL | - | PostgreSQL connection string |
|
|
| REDIS_HOST | localhost | Redis host |
|
|
| REDIS_PORT | 6379 | Redis port |
|
|
| REDIS_PASSWORD | - | Redis password (optional) |
|
|
| S3_ENDPOINT | localhost | MinIO/S3 endpoint |
|
|
| S3_PORT | 9000 | MinIO/S3 port |
|
|
| S3_USE_SSL | false | Use HTTPS for S3 |
|
|
| S3_ACCESS_KEY | minioadmin | S3 access key |
|
|
| S3_SECRET_KEY | minioadmin | S3 secret key |
|
|
| S3_BUCKET | mana-media | Storage bucket |
|
|
| S3_PUBLIC_URL | - | Public URL for media |
|
|
| PUBLIC_URL | http://localhost:3015/api/v1 | Public API URL |
|
|
|
|
## Development
|
|
|
|
```bash
|
|
# Run with watch mode (Bun)
|
|
pnpm dev
|
|
|
|
# Type check
|
|
pnpm type-check
|
|
|
|
# Database commands
|
|
cd apps/api
|
|
pnpm db:push # Push schema to database
|
|
pnpm db:studio # Open Drizzle Studio
|
|
```
|
|
|
|
## Storage Layout
|
|
|
|
```
|
|
mana-media bucket/
|
|
├── originals/ # Original uploads
|
|
│ └── 2026/02/01/
|
|
│ └── {id}.{ext}
|
|
├── processed/ # Generated variants
|
|
│ └── {id}/
|
|
│ ├── thumb.webp
|
|
│ ├── medium.webp
|
|
│ └── large.webp
|
|
└── cache/ # Transform cache
|
|
└── {id}_{params}.webp
|
|
```
|
|
|
|
## Roadmap
|
|
|
|
- [x] v0.1: Basic upload + thumbnails
|
|
- [x] v0.2: PostgreSQL persistence with Drizzle ORM
|
|
- [x] v0.3: Content-addressable storage with SHA-256 deduplication
|
|
- [ ] v0.5: Video thumbnails (FFmpeg)
|
|
- [ ] v0.6: Chunked upload for large files
|
|
- [ ] v0.7: OCR for documents
|
|
- [ ] v0.8: Vector search (Qdrant)
|
|
- [ ] v1.0: Full production ready
|