Commit graph

7 commits

Author SHA1 Message Date
Till JS
16e0d99c5a feat(gpu-server): complete GPU server setup with AI services, monitoring, and public access
- Set up 5 AI services on Windows GPU server (RTX 3090):
  - mana-llm (Port 3025): OpenAI-compatible LLM gateway via Ollama
  - mana-stt (Port 3020): WhisperX with word timestamps + speaker diarization
  - mana-tts (Port 3022): Kokoro (EN) + Edge TTS (DE) + Piper (local DE)
  - mana-image-gen (Port 3023): FLUX.2 klein 4B image generation
  - Ollama (Port 11434): gemma3:4b/12b, qwen2.5-coder:14b, nomic-embed-text

- Add @manacore/shared-gpu TypeScript client package with SttClient, TtsClient, ImageClient
- Add CUDA-compatible whisper_service using faster-whisper for Windows
- Configure public access via Cloudflare Tunnel (gpu-llm/stt/tts/img.mana.how)
- Add Loki log aggregator (Docker on Mac Mini) + log shipper on GPU server
- Add GPU scrape targets to Prometheus/VictoriaMetrics config
- Add Grafana Loki datasource for GPU service logs
- Add health check with auto-restart, log rotation, and log shipping
- Document complete setup: Always-On config, troubleshooting, architecture

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 21:35:30 +01:00
Till JS
42fe39c6a2 fix(infra): fix deploy tracking dashboard datasource UIDs and instant queries
- Add explicit uid: deploy-tracking to datasource provisioning
- Add instant: true to all Prometheus stat/gauge panel queries
- Pushgateway gauges need instant queries, not range queries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 17:35:41 +01:00
Till JS
3f91c4656a feat(infra): add deploy tracking with PostgreSQL, Pushgateway & Grafana dashboard
Instrument the CD pipeline to record per-deploy and per-service metrics
(build time, image size, startup time, health status) into PostgreSQL and
push gauges to Pushgateway. Adds a Grafana dashboard with 13 panels covering
deploy frequency, build performance, service health, and history.

New files:
- scripts/mac-mini/init-deploy-tracking.sql (idempotent DDL)
- scripts/deploy-metrics.sh (bash library for CI)
- docker/grafana/provisioning/datasources/deploy-tracking.yml
- docker/grafana/dashboards/deploy-tracking.json

Modified:
- docker/prometheus/prometheus.yml (pushgateway scrape job)
- .github/workflows/cd-macmini.yml (build/health instrumentation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 17:08:03 +01:00
Till JS
f264e9f2ae feat(grafana): add GlitchTip error tracking dashboard
- Add PostgreSQL datasource pointing to GlitchTip database
- Add Error Tracking dashboard with 7 panels:
  - Total Open Issues (stat)
  - Issues by Project (pie chart)
  - Total Events (stat)
  - Projects Tracked (stat)
  - Resolved vs Unresolved (stat)
  - New Issues Over Time (stacked bar chart, 30 days)
  - Recent Issues (table with 50 latest, color-coded levels)
- Dashboard links to GlitchTip UI for detailed investigation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 21:14:09 +01:00
Till-JS
9b7d8c36b8 🐛 fix(grafana): correct VictoriaMetrics datasource port (8428 → 9090)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 05:13:48 +01:00
Till-JS
9dfad0128a 📈 feat(monitoring): upgrade to VictoriaMetrics + DuckDB analytics
- Replace Prometheus with VictoriaMetrics (2-year retention)
- Add DuckDB analytics module for business KPIs (unlimited retention)
- Add master overview dashboard combining all metrics
- Add business metrics dashboard for user growth tracking
- Add backup script for VictoriaMetrics snapshots and DuckDB
- Add ADR documentation for monitoring stack decision

Analytics API endpoints:
- GET /api/v1/analytics/health - Service health
- GET /api/v1/analytics/latest - Latest metrics snapshot
- GET /api/v1/analytics/growth - User growth over time
- GET /api/v1/analytics/monthly - Monthly aggregates
- POST /api/v1/analytics/snapshot - Manual snapshot trigger
2026-01-28 12:38:04 +01:00
Till-JS
6d86a08d63 feat: add monitoring dashboard (Prometheus + Grafana + Umami + Admin)
Phase 1: Infrastructure
- Add docker/prometheus/prometheus.yml with scrape configs for all services
- Add docker/grafana/provisioning for auto-configured datasources
- Add docker/grafana/dashboards (system-overview, backends-docker)
- Update docker-compose.macmini.yml with monitoring services:
  - prometheus, grafana, node-exporter, cadvisor
  - postgres-exporter, redis-exporter, umami
- Add grafana.mana.how and analytics.mana.how to Caddyfile

Phase 2: Backend Metrics
- Create packages/shared-nestjs-metrics with:
  - MetricsModule (auto /metrics endpoint)
  - MetricsService (Counter, Histogram, Gauge helpers)
  - MetricsMiddleware (auto HTTP request tracking)

Phase 3: Umami Web Analytics
- Add Umami tracking scripts to all landing pages
- Add Umami tracking scripts to all web apps
- Create scripts/mac-mini/setup-umami-db.sh

Phase 4: Admin Dashboard (ManaCore Web)
- Add admin routes: /admin, /admin/users, /admin/system
- Create StatCard, QuickLinks, UserTable components
- Add Admin link to navigation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 15:31:39 +01:00