Commit graph

270 commits

Author SHA1 Message Date
Till-JS
2b979d5548 ♻️ refactor: migrate all remaining bots to shared services
Completed migration of all Matrix bots to @manacore/bot-services:

**SessionService (11 bots migrated):**
- matrix-chat-bot (with conversation/model mapping via setSessionData)
- matrix-contacts-bot
- matrix-skilltree-bot
- matrix-presi-bot
- matrix-questions-bot
- matrix-storage-bot
- matrix-planta-bot
- matrix-manadeck-bot
- matrix-nutriphi-bot (with pendingImage via setSessionData)
- matrix-picture-bot (previous commit)
- matrix-zitare-bot (previous commit)

**TranscriptionService (5 bots migrated):**
- matrix-todo-bot (previous commit)
- matrix-clock-bot (previous commit)
- matrix-zitare-bot (previous commit)
- matrix-nutriphi-bot
- matrix-project-doc-bot

**Code Reduction:**
- Deleted 22 local module files (session + transcription)
- ~1100 lines of duplicate code removed total
- All bots now share identical auth and STT logic

**Special handling:**
- matrix-chat-bot: Extended methods converted to setSessionData/getSessionData
- matrix-nutriphi-bot: pendingImage state via setSessionData
- matrix-project-doc-bot: TranscriptionService used by MediaService

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 00:50:48 +01:00
Till-JS
9b61831cb5 ♻️ refactor: consolidate SessionService & TranscriptionService in @manacore/bot-services
Created shared services to eliminate code duplication across Matrix bots:

**New Services in @manacore/bot-services:**
- SessionService: User authentication via mana-core-auth (was duplicated in 11 bots)
- TranscriptionService: Speech-to-text via mana-stt (was duplicated in 6 bots)

**Migrated Bots:**
- matrix-todo-bot: uses TranscriptionService
- matrix-picture-bot: uses SessionService
- matrix-clock-bot: uses TranscriptionService
- matrix-zitare-bot: uses both SessionService & TranscriptionService

**Code Reduction:**
- Removed ~300 lines of duplicate code from migrated bots
- Centralized service configuration via NestJS modules
- Added comprehensive documentation in CLAUDE.md

Remaining bots can be migrated following the same pattern documented
in packages/bot-services/CLAUDE.md.

Note: @storage/backend type-check fails due to pre-existing drizzle-orm issue

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 00:37:54 +01:00
Till-JS
213740411b feat(llm-playground): add model metadata system and SSD documentation
- Add MODEL_METADATA config for Ollama models with descriptions and modality
- Update default model to gemma3:4b
- Show model descriptions in ModelSelector and ComparisonSelector
- Add docs/OLLAMA_MODELS.md with instructions for adding new models
- Document external 4TB SSD setup in MAC_MINI_SERVER.md
- Add gemma3:12b, gemma3:27b, qwen2.5-coder:14b to model registry
2026-02-01 00:24:34 +01:00
Till-JS
a341aa1b13 🔥 remove: Telegram bots - Matrix-only strategy
Remove all 6 Telegram bot services to focus on Matrix as the sole
messaging platform for full UI/UX control and DSGVO compliance.

Removed services:
- telegram-nutriphi-bot
- telegram-ollama-bot
- telegram-project-doc-bot
- telegram-stats-bot
- telegram-todo-bot
- telegram-zitare-bot

Also:
- Remove Telegram bot scripts from package.json
- Remove telegram-stats-bot from docker-compose.macmini.yml
- Disable Watchtower Telegram notifications
- Remove Telegram devlog
- Add comprehensive MATRIX_BOT_ARCHITECTURE.md documentation

The Matrix-only approach provides:
- Full control over user experience
- Complete DSGVO compliance (all data on own servers)
- No dependency on third-party platforms
- Unified command patterns across all bots
2026-02-01 00:17:14 +01:00
Till-JS
d605366460 feat(llm-playground): add model comparison feature
- Add modality detection (text/vision/code) to models store
- Create comparison store for parallel multi-model streaming
- Add ModelModalityFilter and ModelComparisonSelector components
- Add ComparisonResponseCard with metrics (duration, tokens, t/s)
- Add ComparisonMessageBubble for side-by-side response view
- Integrate comparison mode into ChatInput, MessageList, Sidebar
- Add dev:full script to start mana-llm + playground together
- Add start.sh script for mana-llm Python service
2026-01-31 23:30:16 +01:00
Till-JS
744d0c9c61 fix(mana-core-auth): remove non-existent id field from trusted client
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:28:51 +01:00
Till-JS
c949f5d02a fix(mana-core-auth): fix type compatibility for trusted client config
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:28:06 +01:00
Till-JS
01a2c78e39 fix(mana-core-auth): add all required fields to trusted client config
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:27:33 +01:00
Till-JS
bb428d4b38 fix(mana-core-auth): add Matrix Synapse as trusted OIDC client
Configure Matrix Synapse as a trusted client that skips the consent screen.
This enables seamless SSO login without requiring user consent for each login.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:26:30 +01:00
Till-JS
ee05b6c3ca fix(mana-core-auth): use correct property name 'redirectUrls' for Better Auth
Better Auth expects 'redirectUrls' (lowercase 'urls') but schema had
'redirectURLs' (uppercase 'URLs'). This caused the redirect URI validation
to fail because Drizzle returned the wrong property name.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:22:06 +01:00
Till-JS
fdba0e3425 feat(llm-playground): add production deployment with auth
- Add Dockerfile for multi-stage Docker build
- Add mana-core-auth integration with login/register pages
- Add auth store using Svelte 5 runes
- Add protected route layout with auth guard
- Add health endpoint for container health checks
- Add runtime URL injection via hooks.server.ts
- Add logout button to header
- Update docker-compose.macmini.yml with llm-playground service
- Update cloudflared-config.yml with playground.mana.how route
- Update mana-llm CORS config for playground domain
- Update generate-env.mjs with auth URL variable

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:15:02 +01:00
Till-JS
8207d38ca5 fix(mana-core-auth): use comma-separated redirect_urls for Better Auth OIDC
Better Auth's OIDC provider expects redirect_urls to be a comma-separated
string, not a JSON array. Updated seed script and schema documentation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 18:12:16 +01:00
Till-JS
78a5002968 debug: add logging to OIDC authorize endpoint 2026-01-30 18:02:56 +01:00
Till-JS
f59b6596b0 fix(mana-core-auth): add dedicated Better Auth handler for sign-in
The OIDC request handler was not properly forwarding sign-in requests.
Added a dedicated handler that:
- Directly calls Better Auth's handler
- Properly handles Set-Cookie headers for session cookies
- Exposed getHandler() method from BetterAuthService
- Added trustedOrigins configuration to allow cross-origin requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:56:34 +01:00
Till-JS
f880ef2b7f feat(llm-playground): add SvelteKit LLM playground UI
- Chat interface with message history
- Model selector for available LLM models
- Parameter panel (temperature, max tokens, etc.)
- System prompt editor
- Svelte 5 runes-based stores

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:52:10 +01:00
Till-JS
3edbd0cb26 chore: update dependencies and mana-llm improvements
- Update pnpm-lock.yaml with matrix bot dependencies
- Add environment variables to generate-env.mjs
- Improve mana-llm config and ollama provider

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:50:58 +01:00
Till-JS
df47dafeb5 chore(matrix-bots): standardize package.json across all 9 bots
- Unified package scope to @mana-bots/*
- Added private: true to all packages
- Added pnpm overrides for matrix-sdk-crypto-nodejs
- Added prebuild script to clean dist folder
- Updated all dependencies to latest versions:
  - @nestjs/*: ^10.4.15
  - @nestjs/config: ^3.3.0
  - reflect-metadata: ^0.2.2
  - @types/node: ^22.10.2
  - typescript: ^5.7.2

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:49:15 +01:00
Till-JS
004fe85799 fix(matrix-bots): resolve TypeScript strict null check errors
- Fix parseInt undefined errors in configuration files
- Add fallbacks for Matrix client constructor parameters
- Fix possibly undefined data accesses with non-null assertions
- Update setCurrentConversation to accept null

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:32:09 +01:00
Till-JS
68219a01df feat(matrix-chat-bot): add Matrix bot for AI chat conversations
- Quick chat mode for stateless single messages (!chat)
- Full conversation management (create, list, select, delete)
- Message history with context-aware AI responses
- Model selection (Ollama, OpenRouter, OpenAI, Anthropic)
- Conversation actions: archive, restore, pin, unpin, rename
- German/English command aliases
- Number-based reference system for ease of use
- JWT auth via mana-core-auth
- Health check endpoint on port 3327

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:18:21 +01:00
Till-JS
3ed1453ff4 feat(matrix-skilltree-bot): add Matrix bot for skill tree and XP management
- Skill management: create, list, view details, delete
- XP tracking with level-up notifications
- Branch filtering (intellect, body, creativity, social, practical, mindset, custom)
- Activity history per skill or global
- User statistics (total XP, skill count, highest level, streak)
- German/English command aliases
- Number-based reference system for ease of use
- JWT auth via mana-core-auth
- Health check endpoint on port 3326

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 17:00:01 +01:00
Till-JS
edbe7502d3 fix(mana-core-auth): use Better Auth native sign-in for OIDC login
The OIDC login page was using our custom /api/v1/auth/login endpoint
which returns tokens but doesn't set session cookies. Better Auth's
OIDC provider needs session cookies to recognize logged-in users.

Changes:
- Update login page to use /api/auth/sign-in/email (Better Auth native)
- Add sign-in endpoint handler in oidc.controller.ts
- Add route exclusion in main.ts for the sign-in path

This fixes the infinite redirect loop where users would log in but
then be sent back to login because the OAuth2 authorize endpoint
couldn't detect the session.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:57:52 +01:00
Till-JS
e3cfafe594 feat(matrix-presi-bot): add Matrix bot for presentation management
- Full NestJS bot with matrix-bot-sdk integration
- Deck management: create, list, view, delete, rename
- Slide management: add title/text/bullet/image slides, delete
- Multiple slide types: title, content, bullets, image
- Theme support: list themes, apply to presentation
- Sharing: create links with optional expiration
- German/English command aliases
- Number-based reference system for decks and themes
- JWT auth via mana-core-auth
- Runs on port 3325

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:54:07 +01:00
Till-JS
c5476447ec feat(matrix-questions-bot): add Matrix bot for Q&A research management
- Full NestJS bot with matrix-bot-sdk integration
- Question management: create, list, view, delete, archive
- Research: start quick/standard/deep research via mana-search
- Results: view summaries, key points, follow-up questions
- Sources: view ranked sources with relevance scores
- Answers: view, rate (1-5), accept as solution
- Collections: list and create for organization
- Search: full-text search across questions
- Status tracking: open, researching, answered, archived
- Priority levels: low, normal, high, urgent
- German/English command aliases
- Number-based reference system
- JWT auth via mana-core-auth
- Runs on port 3324

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:48:56 +01:00
Till-JS
fc15366efa fix(mana-core-auth): fix TypeScript error in oidc-login controller 2026-01-30 16:42:10 +01:00
Till-JS
369415527f 🔧 chore(shared-vite-config): add build step for Docker compatibility
- Add build script to compile TypeScript to dist/
- Update exports to point to compiled files
- Update calendar-web Dockerfile to build shared-vite-config

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:41:48 +01:00
Till-JS
4a66341e08 fix(mana-core-auth): extract client_id from returnUrl for OIDC login
When redirected from authorization endpoint, the client_id is encoded
in the returnUrl parameter, not directly in query params. This fix
extracts it properly to display the correct application name.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:41:39 +01:00
Till-JS
3f336de1b9 feat(matrix-planta-bot): add Matrix bot for plant care management
- Full NestJS bot with matrix-bot-sdk integration
- Plant CRUD: list, add, view, edit, delete plants
- Watering: mark as watered, upcoming waterings, history
- Care settings: light, humidity, temperature, soil, notes
- Watering interval configuration
- Health status tracking with emoji indicators
- German/English command aliases
- Number-based reference system for plants
- JWT auth via mana-core-auth
- Runs on port 3322

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:36:30 +01:00
Till-JS
3d4402ad9b fix(mana-core-auth): allow inline scripts in CSP for OIDC login page
The login page uses inline JavaScript for the form submission handler.
Helmet's default CSP was blocking this, preventing users from logging in
via OIDC/SSO flows (e.g., Matrix Synapse).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:35:42 +01:00
Till-JS
ad7f875c5f feat(matrix-manadeck-bot): add Matrix bot for card/deck management
- Full NestJS bot with matrix-bot-sdk integration
- Deck CRUD: list, create, view, delete decks
- Card management: view cards and card details
- AI generation: generate decks with AI (30 Mana)
- Study sessions: start learning sessions
- Progress tracking: due cards, statistics
- Public features: featured decks, leaderboard
- Credit system: mana balance display
- German/English command aliases
- Number-based reference system for decks and cards
- JWT auth via mana-core-auth
- Runs on port 3321

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:29:12 +01:00
Till-JS
64535373ac feat(matrix-contacts-bot): add Matrix bot for contact management
- List, search, and view contact details
- Create, edit, and delete contacts
- Toggle favorites and archive status
- Number-based reference system for easy commands
- German and English command aliases
- Login/logout via mana-core-auth

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:20:11 +01:00
Till-JS
8950692cfd feat(matrix-picture-bot): add Matrix bot for AI image generation
- Generate images via Picture backend with `!generate` command
- Support prompt options (--width, --height, --steps, --negative)
- Model selection with `!models` and `!model [id]`
- Image history and deletion
- Login/logout via mana-core-auth
- Credit balance display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 16:15:08 +01:00
Till-JS
b8ecdb8eb7 🔧 chore(mana-notify): disable email notifications by default
Email alerts were causing too many notifications. Changed default
from true to false so new users won't receive email notifications
unless explicitly enabled. Push notifications remain enabled.
2026-01-30 16:06:24 +01:00
Till-JS
971e269fbd 🔧 chore: update pnpm-lock and bot sync token 2026-01-29 22:59:41 +01:00
Till-JS
f4c49fe8f2 fix(mana-notify): resolve BullMQ circular import issue
Move queue name constants to separate file (queue-names.ts) to avoid
circular dependency between queue.module.ts and processor files.

The @Processor decorator evaluates at module load time, and importing
constants from queue.module.ts created a circular dependency that
resulted in undefined queue names.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 22:58:47 +01:00
Till-JS
384244fe50 ♻️ refactor: migrate bots and chat-backend to mana-llm
Migrate all LLM consumers from direct Ollama calls to centralized
mana-llm service with OpenAI-compatible API.

Migrated services:
- matrix-ollama-bot
- telegram-ollama-bot
- chat-backend
- telegram-project-doc-bot

New env vars: MANA_LLM_URL, LLM_MODEL, LLM_TIMEOUT
Replaces: OLLAMA_URL, OLLAMA_MODEL, OLLAMA_TIMEOUT
2026-01-29 22:56:26 +01:00
Till-JS
b5fa0f42b6 feat(mana-notify): add central notification service
NestJS notification microservice for email, push, Matrix, and webhook
notifications across all ManaCore apps.

Features:
- Multi-channel delivery (email, push, Matrix, webhook)
- Handlebars template engine with defaults
- User notification preferences
- BullMQ async job processing
- Delivery tracking and logging
- Prometheus metrics

Includes @manacore/notify-client package for NestJS integration.
2026-01-29 22:07:38 +01:00
Till-JS
1495dbe476 feat(mana-llm): add central LLM abstraction service
Python/FastAPI service providing unified OpenAI-compatible API for
Ollama and cloud LLM providers (OpenRouter, Groq, Together).

Features:
- Chat completions with streaming (SSE)
- Vision/multimodal support
- Embeddings generation
- Multi-provider routing (provider/model format)
- Prometheus metrics
- Optional Redis caching
2026-01-29 22:01:00 +01:00
Till-JS
4a3295d1d0 feat(mana-crawler): add web crawler service
NestJS-based web crawler service for structured content extraction.

Features:
- Depth-controlled crawling with URL pattern filtering
- robots.txt compliance
- HTML/PDF/Markdown content extraction
- BullMQ job queue for async processing
- Redis caching layer
- Prometheus metrics
2026-01-29 22:00:36 +01:00
Till-JS
c3e340f859 🔧 chore: fix missing shared package dependencies
Add missing workspace dependencies:
- telegram-nutriphi-bot: @manacore/shared-drizzle-config
- telegram-todo-bot: @manacore/shared-drizzle-config
- questions/backend: @manacore/shared-tsconfig + fix tsconfig paths

Fixes pre-commit type-check failures.
2026-01-29 20:41:39 +01:00
Till-JS
9ffbf35f25 🔧 fix(docker): update Dockerfiles for monorepo-aware builds
- Update mana-search Dockerfile to use monorepo root context
- Update api-gateway Dockerfile to use monorepo root context
- Fix CI workflow to use context: . for mana-search
- Use pnpm --filter for proper workspace package installation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 18:39:05 +01:00
Till-JS
60b7cad508 🔧 chore: gitignore build artifacts and misc updates
- Add drizzle.config.* compiled files to gitignore
- Add packages/*/src/*.js patterns for compiled TS
- Update picture design tokens
- Update matrix-tts-bot storage state
- Add microservices API overview doc
2026-01-29 18:37:42 +01:00
Till-JS
59d5d20e0a ♻️ refactor: migrate 12 backends to shared-nestjs-health module
- Create @manacore/shared-nestjs-health package with HealthModule.forRoot()
- Migrate 12 backends: calendar, chat, clock, contacts, nutriphi, picture,
  planta, presi, skilltree, storage, todo, zitare
- Delete 24 local health controller/module files (~312 LOC removed)
- Support custom route (storage uses 'api/v1/health')
- Skip questions backend (has extended DB health check)
- Fix telegram-zitare-bot missing shared-drizzle-config dependency
- Update CONSOLIDATION_OPPORTUNITIES.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 18:35:10 +01:00
Till-JS
fc0ed636fc feat(api-gateway): add Swagger, admin endpoints, and scheduler
- Add Swagger/OpenAPI documentation at /docs endpoint
- Add admin module for system-wide API key management
- Add scheduler for monthly credit reset and usage cleanup
- Add Docker Compose entry for Mac Mini deployment
- Document all endpoints with descriptions and examples
2026-01-29 18:03:16 +01:00
Till-JS
6f1b2654f1 feat: add mana-api-gateway for monetizing core services
Implement custom NestJS API Gateway for mana-search, mana-stt, and mana-tts:

- API Key management with CRUD operations and key regeneration
- Redis-based sliding window rate limiting
- Credit-based billing with tier support (free, pro, enterprise)
- Usage tracking with daily aggregates
- Proxy services to backend microservices
- Prometheus metrics endpoint
- JWT auth for management API, API key auth for public API

Database schema uses separate `api_gateway` schema in shared manacore DB.
2026-01-29 17:30:21 +01:00
Till-JS
6807543d60 🔧 chore: create @manacore/shared-drizzle-config and migrate 16 configs
- Create factory function with dbName, schemaPath, outDir, schemaFilter options
- Migrate 12 app backends: calendar, chat, clock, contacts, nutriphi, picture,
  planta, presi, questions, skilltree, storage, todo
- Migrate 4 services: mana-core-auth, telegram-zitare-bot, telegram-todo-bot,
  telegram-nutriphi-bot
- Update consolidation docs with completed Drizzle config task

Savings: ~160 LOC (16 configs × ~10 LOC each)
2026-01-29 16:54:44 +01:00
Till-JS
f0adea04fd fix(tts-bot): use MP3 format and prevent duplicate messages
- Changed audio output from WAV to MP3 for better browser compatibility
- Added event ID tracking to prevent processing same message twice
2026-01-29 16:20:26 +01:00
Till-JS
41354d58ee fix(matrix-tts-bot): fix Dockerfile pnpm install 2026-01-29 16:04:08 +01:00
Till-JS
58a051645b feat(matrix): add TTS bot for text-to-speech conversion
- NestJS bot that converts text messages to speech via mana-tts
- Commands: !voice, !voices, !speed, !status, !help
- User settings stored in-memory (voice, speed per user)
- Docker config for Mac Mini deployment
- Setup script for bot registration

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-29 16:03:26 +01:00
Till-JS
0c150df0f1 feat(auth): add resend verification email to all login pages
Add ability to resend verification email when login fails with
"Email not verified" error. Implemented across all 14 apps using
Mana Core Auth.

Changes:
- Add POST /api/v1/auth/resend-verification endpoint to mana-core-auth
- Add resendVerificationEmail method to shared-auth client
- Update LoginPage component with resend UI and translations
- Add resendVerificationEmail to all app auth stores
- Add translations for de, en, fr, es, it
- Add PlantaLogo to shared-branding
- Migrate planta login to shared LoginPage component
2026-01-29 14:55:49 +01:00
Till-JS
5a0815708c 🌐 feat: add i18n support to 6 web apps
Add internationalization (DE + EN) to previously missing apps:
- todo: task management translations
- skilltree: skill/XP system translations
- nutriphi: nutrition tracking translations
- planta: plant care translations
- questions: research app translations
- matrix: chat client translations (layout integration)

Each app includes:
- svelte-i18n setup with SSR support
- localStorage persistence ({app}_locale pattern)
- i18n loading state in +layout.svelte
- German (default) and English translations

Updated CONSISTENCY_REPORT.md to mark i18n task as complete.

Also includes:
- mana-tts service placeholder files
2026-01-29 14:48:35 +01:00