Commit graph

244 commits

Author SHA1 Message Date
Till-JS
22a6a6496c fix(mana-media): fix Dockerfile pnpm symlinks for production
Keep same directory structure in production stage so pnpm symlinks
resolve correctly. Copy the .pnpm store and app's node_modules
separately.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 02:19:19 +01:00
Till-JS
8b6ff0c679 feat(auth): add API key management for STT/TTS services
- Add api_keys schema in mana-core-auth with SHA-256 hashing
- Create NestJS module with CRUD endpoints and validation
- Add external auth module to STT/TTS for sk_live_ key validation
- Create web UI page at /api-keys for key management
- Support rate limiting per key with configurable limits
- Cache validation results for 5 minutes to reduce auth service load

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 02:12:05 +01:00
Till-JS
4452d371fa 🐛 fix(mana-media): correct path to main.js in Dockerfile
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 01:55:48 +01:00
Till-JS
898f5d2112 🔧 chore(stt,tts): update launchd plists to load .env files
Source .env file before starting uvicorn to enable API key auth
and other environment-based configuration.

Removes hardcoded PORT values in favor of .env configuration.
2026-02-12 01:44:46 +01:00
Till-JS
aab304fc95 🔒️ feat(stt,tts): add API key authentication with rate limiting
Add auth.py module to both STT and TTS services with:
- API key validation via X-API-Key header
- Rate limiting with sliding window (requests per minute)
- Internal API key option for unlimited access
- Environment variable configuration

All protected endpoints now require authentication.
Public endpoints (/health, /docs) remain accessible.
2026-02-11 18:04:22 +01:00
Till-JS
90c2f8573e feat(photos): add Photos app with mana-media EXIF integration
- Add Photos NestJS backend (port 3019) with albums, favorites, tags
- Add Photos SvelteKit web app (port 5189) with gallery, upload, filters
- Extend mana-media with EXIF extraction service using exifr
- Add cross-app photo listing endpoint to mana-media
- Add photo stats endpoint to mana-media
- Add photos to setup-databases.sh

Backend features:
- Albums CRUD with cover image and items management
- Favorites toggle with status check
- Tags CRUD with photo-tag associations
- Photo proxy to mana-media with local data enrichment

Web features:
- Photo grid with infinite scroll
- Photo detail modal with EXIF display
- Album grid and detail views
- Upload dropzone with progress tracking
- Filter bar (app, date range, location, sort)
- i18n support (de/en)
- Svelte 5 runes mode

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-11 17:58:44 +01:00
Till-JS
21d50d1e0b 📝 docs(mana-stt): document Whisper + Mistral API architecture
- Disable vLLM by default (has issues on macOS CPU)
- Use Mistral API for Voxtral transcription (cloud-based)
- Keep Whisper-MLX for local transcription
- Update README with architecture diagram

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-11 16:34:03 +01:00
Till-JS
7c9c2645e3 🐛 fix(mana-stt): adjust vLLM config for CPU mode
- Reduce max-model-len to 4096 for CPU compatibility
- Add max-num-batched-tokens matching the context size
- Add enforce-eager for stable CPU inference

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-11 16:14:14 +01:00
Till-JS
60394076e5 feat(mana-stt): add vLLM integration for Voxtral transcription
- Add vllm_service.py as proxy to vLLM server for Voxtral 3B/4B
- Add voxtral_api_service.py for Mistral API fallback
- Update main.py with /transcribe/voxtral endpoint using vLLM
- Add /transcribe/auto endpoint with automatic fallback chain
- Create setup-vllm.sh and start-vllm-voxtral.sh scripts
- Add launchd plist files for Mac Mini deployment
- Add install-services.sh for automated service installation

Architecture:
- vLLM server runs Voxtral models on port 8100
- mana-stt proxies to vLLM with Mistral API fallback
- Fallback chain: vLLM -> Mistral API

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-11 16:10:00 +01:00
Till-JS
a2e2a5b73c feat(admin): add user data dashboard for cross-project data visualization
Add comprehensive admin dashboard to view and manage user data across all projects:

Backend:
- Add admin endpoints to Chat, Todo, Contacts, Calendar, Picture, Zitare, Presi
- Each backend exposes GET/DELETE /api/v1/admin/user-data/:userId
- Service-to-service auth via X-Service-Key header

Aggregation (mana-core-auth):
- GET /api/v1/admin/users - Paginated user list with search
- GET /api/v1/admin/users/:userId/data - Aggregated data from all backends
- DELETE /api/v1/admin/users/:userId/data - GDPR deletion across all projects

Frontend (ManaCore web):
- New User Data tab in admin navigation
- User search page at /admin/user-data
- User detail page with ProjectDataCard components
- GDPR deletion dialog with email confirmation

Presi:
- Migrate user_id from UUID to TEXT for Better Auth compatibility
- Add SQL migration script
2026-02-11 14:59:18 +01:00
Till-JS
c475923864 fix(mana-media): simplify Dockerfile to single build stage
Use simpler approach that handles pnpm install failures gracefully.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 19:03:05 +01:00
Till-JS
b3dd3c51d3 fix(mana-media): fix Dockerfile build and health check paths
- Use --ignore-scripts to skip postinstall builds during install
- Remove unnecessary shared-builder stage
- Fix health check URL to /api/v1/health

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 18:57:50 +01:00
Till-JS
c965f7e440 fix(mana-media): simplify Dockerfile for pnpm hoisted dependencies
pnpm hoists dependencies to root node_modules, so we don't need to copy
service-level node_modules that don't exist.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 18:55:49 +01:00
Till-JS
5c19500748 fix(mana-media): change userId from UUID to TEXT for Matrix user IDs
Matrix user IDs like @user:matrix.org are not UUIDs, so the schema
needs to accept text strings for the userId field in media_references.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 18:45:40 +01:00
Till-JS
cee30ff7ed feat(matrix-todo-bot): improve UX with German commands without ! prefix
- heute command now shows both today's tasks AND inbox tasks
- All commands work without ! prefix (e.g., 'heute' instead of '!heute')
- Updated all help text and messages to show German commands
- Added direct command recognition for: neu, heute, erledigt, löschen, etc.
- Commands still work with ! prefix for backwards compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 18:32:00 +01:00
Till-JS
d4663b5643 feat(mana-media): add centralized media storage with NutriPhi integration
- Implement mana-media service with PostgreSQL/Drizzle ORM persistence
- Add content-addressable storage (SHA-256) for automatic deduplication
- Add Matrix MXC URL import endpoint to copy images from Matrix
- Create @manacore/media-client package for service consumption
- Integrate mana-media into NutriPhi bot for persistent image storage
- Update pnpm-workspace.yaml to include nested service packages
- Add mana-media to docker-compose with port 3015

Images sent to NutriPhi bot are now stored in mana-media after analysis,
providing persistent storage with deduplication across all apps.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 17:30:14 +01:00
Till-JS
85df234ff2 feat(mana-core-auth): auto-link Matrix users on OIDC login
When users log into Matrix via OIDC (Sign in with Mana Core), their
Matrix user ID is now automatically linked to their Mana account.
This enables automatic bot authentication without requiring a
separate !login command.

- Add autoLinkOnOidcLogin() method to MatrixSessionService
- Hook into OIDC userinfo endpoint to create links automatically
- Calculate Matrix user ID from email using Synapse's template

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:50:28 +01:00
Till-JS
75937d6ce9 fix(mana-core-auth): align JWT issuer validation with Better Auth signing config
Better Auth signs JWTs with issuer=BASE_URL (https://auth.mana.how) for OIDC
compatibility, but validation was using jwt.issuer config which defaults to
'manacore'. This caused "unexpected iss claim value" errors.

Fixed in:
- better-auth.service.ts validateToken()
- jwt-auth.guard.ts
- optional-auth.guard.ts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:50:04 +01:00
Till-JS
0538a6ca10 fix(mana-core-auth): use Better Auth signJWT in refresh endpoint
The refresh endpoint was using manual jwt.sign with RSA keys, but the
server doesn't have JWT_PRIVATE_KEY configured. Changed to use Better
Auth's signJWT method which uses the JWKS/EdDSA keys from the database.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:40:37 +01:00
Till-JS
c196144d1c fix(mana-core-auth): return real refreshToken from signIn endpoint
Same issue as sessionToToken - the signIn method was returning the
session cookie token as refreshToken, but the /api/v1/auth/refresh
endpoint expects the actual refreshToken field from the sessions table.

Now signIn:
- Fetches the session from database after Better Auth creates it
- Uses existing refreshToken if available
- Generates and stores a new refreshToken if missing
- Returns the actual refreshToken that works with token refresh

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:37:54 +01:00
Till-JS
d64016d1e5 fix(mana-core-auth): exclude /api/auth/get-session from global prefix
The get-session endpoint needs to be accessible at /api/auth/get-session
(without the /api/v1 prefix) for SSO to work.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:29:17 +01:00
Till-JS
95e9b3764d feat(mana-core-auth): add /api/auth/get-session endpoint for SSO
The SSO flow in client apps calls /api/auth/get-session with cookies
to check if the user has a valid session. This endpoint was missing
from the NestJS passthrough controller.

Now the endpoint forwards the request with cookies to Better Auth's
native handler and returns the session data.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:27:39 +01:00
Till-JS
9c45879363 feat(nutriphi-bot): auto-analyze images when received
Images are now automatically analyzed when sent to the bot, removing
the need for the !analyze command after sending a photo.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:27:03 +01:00
Till-JS
b6925e0b63 feat(matrix-bots): enable cross-bot SSO via Redis sessions
All 19 Matrix bots now use SessionModule.forRoot({ storageMode: 'redis' })
to share user sessions across all bots via Redis. Users only need to
login once with any bot to be authenticated with all bots.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:16:22 +01:00
Till-JS
c2c80efc50 feat(matrix-bots): add i18n system and direct message fallback
- Add I18nService with per-user language preferences (de/en)
- Add !language/!sprache command to all 4 bots (todo, calendar, contacts, clock)
- Add fallback behavior: messages without commands create tasks/events/contacts/timers
- Improve clock bot duration parsing to accept bare numbers as minutes (e.g. "25" = 25min)
- Add support for more duration formats: "25 minuten", "1 stunde", etc.

Language preferences stored in SessionService, default configurable via BOT_DEFAULT_LANGUAGE env var.
2026-02-02 16:07:27 +01:00
Till-JS
5c688d713e fix(mana-core-auth): return real refreshToken in SSO session-to-token exchange
The sessionToToken method was incorrectly returning the session cookie
token instead of the actual refreshToken from the database. This caused
"No refresh token available" errors when users logged in via SSO
(cross-domain cookie) because the /api/v1/auth/refresh endpoint expects
the refreshToken field from the sessions table, not the cookie token.

Now the method:
- Fetches the session from database by cookie token
- Uses existing refreshToken if available
- Generates and stores a new refreshToken if missing
- Returns the actual refreshToken that works with token refresh

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 16:05:44 +01:00
Till-JS
aba79f5c16 fix(mana-llm): fix SSE double data prefix causing message parsing issues
EventSourceResponse from sse-starlette adds its own 'data:' prefix,
so we should yield dicts with a 'data' key instead of pre-formatted
SSE strings. This was causing 'data: data:' double prefixes and
backticks appearing in chat messages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 15:29:11 +01:00
Till-JS
60d2f6422c fix(matrix-bots): update all bots for async SessionService methods
Update all Matrix bots to properly await async SessionService methods
after the migration to Redis-backed session storage. This includes:

- Adding await to getSessionData, getSession, getToken, isLoggedIn calls
- Making requireAuth helper methods async and return Promise<string>
- Making session data helper methods async (getCurrentConversation, etc.)
- Fixing handleLogout and other methods that use session operations

Affected bots:
- matrix-chat-bot (extensive changes for session helpers)
- matrix-clock-bot (getToken helper + session calls)
- matrix-contacts-bot, matrix-manadeck-bot, matrix-nutriphi-bot
- matrix-ollama-bot, matrix-picture-bot, matrix-planta-bot
- matrix-presi-bot, matrix-project-doc-bot, matrix-skilltree-bot
- matrix-storage-bot, matrix-zitare-bot, matrix-tts-bot
- matrix-mana-bot (help handler)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 15:04:30 +01:00
Till-JS
a416dade7b fix(playground): add missing shared packages to Dockerfile
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 14:55:02 +01:00
Till-JS
8525020e8a feat(playground): integrate shared auth UI for consistent login experience
- Add PlaygroundLogo to shared-branding package
- Add playground to APP_BRANDING, APP_ICONS, and APP_URLS
- Replace custom login/register pages with shared-auth-ui components
- Update authStore with resendVerificationEmail and improved signUp
- Add Caddy reverse proxy entry for playground.mana.how

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 14:53:51 +01:00
Till-JS
2777f604fd feat(bots): enable Redis SSO for todo-bot and calendar-bot
- Activate Redis session storage in both bots for cross-bot SSO
- Update SessionHelper to async methods for Redis-backed SessionService
- Fix async/await issues in todo-bot and calendar-bot matrix.service.ts
- Remove unused imports from calendar-api and todo-api services
- Add CALENDAR_BACKEND_URL and MANA_CORE_SERVICE_KEY to .env.development

Note: SessionService methods are now async (Redis-backed). Other bots
need their matrix.service.ts updated to await these async calls.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 14:51:23 +01:00
Till-JS
feaf27dd14 feat(auth): implement cross-subdomain SSO for all web apps
Add Single Sign-On (SSO) support across all mana.how subdomains:

- Add trySSO() method to @manacore/shared-auth that exchanges session
  cookies for JWT tokens
- Add /api/v1/auth/session-to-token endpoint to mana-core-auth service
- Update all 15 web apps to try SSO during auth initialization

SSO Flow:
1. User logs in on any app (e.g., calendar.mana.how)
2. Session cookie is set with Domain=.mana.how
3. When visiting another app (e.g., todo.mana.how), it checks for
   local tokens first
4. If no local tokens, tries SSO via session cookie
5. Session cookie is exchanged for JWT tokens via new endpoint
6. User is automatically authenticated

Apps updated: calendar, chat, clock, contacts, manacore, manadeck,
nutriphi, picture, planta, presi, questions, skilltree, storage,
todo, zitare

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 13:17:04 +01:00
Till-JS
352070fb2f docs(auth): add SSO documentation and test credentials
- Document cross-domain SSO with COOKIE_DOMAIN configuration
- Add production test credentials for automated testing
- Explain cookie-based SSO flow across *.mana.how subdomains

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 04:09:07 +01:00
Till-JS
f03c09ff17 feat(auth): enable cross-domain SSO via shared cookies on .mana.how
- Configure Better Auth with crossSubDomainCookies for .mana.how domain
- Add COOKIE_DOMAIN environment variable (production: .mana.how)
- Sync trustedOrigins with all production subdomains
- Users now login once and are authenticated across all apps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 03:58:29 +01:00
Till-JS
a7c1908f25 fix(project-doc-bot): add include/exclude to tsconfig for correct dist output
The tsconfig was missing include/exclude which caused the build to
output to dist/src/main.js instead of dist/main.js.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 23:01:12 +01:00
Till-JS
ea0198cc18 fix(bots): install wget for Docker health checks
node:20-slim doesn't include wget or curl, which is required
for the HEALTHCHECK commands in the Dockerfiles.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 17:19:08 +01:00
Till-JS
a8521d7acd 🐛 fix(matrix-bots): disable E2EE crypto module via pnpm override
Add root-level pnpm override to replace @matrix-org/matrix-sdk-crypto-nodejs
with empty-npm-package. This disables E2EE support but allows matrix bots
to run without native module compilation issues.

The bots don't need E2EE since Matrix Synapse handles encryption at
the server level for our use case.

Also removes the explicit pnpm add commands from Dockerfiles since the
override handles the module replacement.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 15:08:28 +01:00
Till-JS
dc0d425f61 🐛 fix(matrix-web): handle Matrix SSO loginToken callback
Add loginWithLoginToken function to exchange Matrix SSO loginToken for credentials.
The app layout now detects the loginToken URL parameter and completes the SSO flow.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 15:02:47 +01:00
Till-JS
38101bca00 🐛 fix(matrix-bots): add pnpm rebuild for native crypto module
The @matrix-org/matrix-sdk-crypto-nodejs package needs a rebuild step
after install to properly select the platform-specific prebuilt binary.
Without this, the module fails to find the correct binding at runtime.

Adds `pnpm rebuild @matrix-org/matrix-sdk-crypto-nodejs || true` after
both the dev and prod install steps in all matrix bot Dockerfiles.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 14:58:05 +01:00
Till-JS
6c937b6c80 🐛 fix(matrix-bots): restore --ignore-scripts to skip root postinstall
The root package.json has postinstall hooks (generate-env, build:packages,
husky) that fail in Docker context. --ignore-scripts prevents these from
running while still allowing the build to complete.

Using node:20-slim (glibc) should handle matrix-sdk crypto modules via
prebuilt binaries without needing install scripts.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 14:47:04 +01:00
Till-JS
a384bed181 🐛 fix(matrix-bots): switch to node:20-slim for native module support
- Changed base images from Alpine to Debian slim (glibc vs musl)
- Removed --ignore-scripts to allow native module builds
- Fixed user creation commands for Debian-based images

The matrix-bot-sdk requires @matrix-org/matrix-sdk-crypto-nodejs which
needs platform-specific native binaries. Alpine's musl libc causes
compatibility issues with these modules.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 14:41:34 +01:00
Till-JS
fe33f4b355 fix(mana-core-auth): complete production readiness with test fixes
- Fix LoggerService mock in better-auth.service.spec.ts
- Fix name assertion in auth.controller.spec.ts (empty string fallback)
- Fix createRemoteJWKSet mock in jwt-auth.guard.spec.ts
- Add Grafana dashboard for Auth Service monitoring
- Add 10 auth-specific Prometheus alert rules
- Update production readiness plan to 100% complete

All 199 unit tests passing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 14:18:58 +01:00
Till-JS
ab49be0bee 🐛 fix(matrix-mana-bot): resolve QEMU emulation failure in CI
- Build matrix-mana-bot only for linux/amd64 (arm64 fails due to QEMU)
- Move pnpm overrides for cpu-features and ssh2 to root package.json
- These native deps cause illegal instruction errors under QEMU emulation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 13:59:04 +01:00
Till-JS
8cd5021b50 🐛 fix(mana-core-auth): use BASE_URL as JWT issuer for OIDC compatibility
OIDC providers like Synapse expect the JWT issuer claim to match the
discovery document's issuer URL. Changed JWT plugin config from
JWT_ISSUER to BASE_URL to ensure consistency.

Also adds:
- @manacore/credit-operations package with operation definitions
- @manacore/shared-credit-ui package with React Native and Svelte components
- CreditInterceptor and @UseCredits decorator in nestjs-integration
- Credit system integration in chat backend
2026-02-01 13:55:05 +01:00
Till-JS
075051a1d4 add-dep: add jsonwebtoken to mana-core-auth dependencies 2026-02-01 13:27:23 +01:00
Till-JS
c0117b2699 🐛 fix: add missing jwt import in better-auth.service 2026-02-01 13:26:11 +01:00
Till-JS
efb077b9ea 🐛 fix(mana-core-auth): use EdDSA for OIDC id_token signing
Set useJWTPlugin: true so id_tokens are signed with EdDSA keys
from JWKS instead of HS256. This fixes Synapse OIDC integration
which verifies tokens via JWKS endpoint.
2026-02-01 13:24:55 +01:00
Till-JS
45152ee954 🚀 feat(matrix-bots): add CI/CD pipeline for automated GHCR deployment
- Update 10 Matrix Bot Dockerfiles for monorepo builds with pnpm
- Add shared package support (bot-services, matrix-bot-common)
- Extend CI pipeline with change detection and build jobs
- Update docker-compose.macmini.yml to use GHCR images
- Enable Watchtower auto-updates for Matrix Bots

Bots: mana, ollama, stats, project-doc, todo, calendar, nutriphi, zitare, clock, tts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 13:02:09 +01:00
Till-JS
f0cf1bc804 🐛 fix(mana-core-auth): OIDC token exchange now works with body-parser
- Removed debug logging that exposed sensitive client_secret in production logs
- The body-parser middleware in main.ts correctly handles form-urlencoded token requests
- handleOidcRequest properly converts parsed body to URLSearchParams for Better Auth
2026-02-01 12:28:41 +01:00
Till-JS
4f90f2863d debug: add logging to handleOidcRequest for token exchange debugging 2026-02-01 12:23:05 +01:00