diff --git a/cloudflared-config.yml b/cloudflared-config.yml index e0340a173..ef5a4d86a 100644 --- a/cloudflared-config.yml +++ b/cloudflared-config.yml @@ -161,6 +161,14 @@ ingress: - hostname: admin.mana.how service: http://localhost:3071 + # Verdaccio @mana/* npm-Registry. Standalone-Compose-Project unter + # ~/projects/verdaccio/ auf dem Mini (storage + htpasswd survive im + # bind-mount). Phase 2f-1 hatte das nach GPU verlagert, aber das + # Storage-Volume kam dort nie an — am 2026-05-07 zurueckgerollt, + # Mini bleibt Single-Source. + - hostname: npm.mana.how + service: http://localhost:4873 + # ============================================ # Memoro (Code/memoro, separate repo) # ~/projects/memoro-deploy/ on the Mac Mini. diff --git a/docs/PLAN_OPTION_C.md b/docs/PLAN_OPTION_C.md index aa26d32e8..337dc297f 100644 --- a/docs/PLAN_OPTION_C.md +++ b/docs/PLAN_OPTION_C.md @@ -17,7 +17,7 @@ Production-Hot-Path bleibt unverändert auf dem Mini. | Phase 2c — VM + Loki + Alerts | ✅ | Komplett auf GPU-Box. 11 Container neu (VM, Loki, Pushgateway, Blackbox, Vmalert, Alertmanager, Alert-notifier, GPU-eigenes Node-Exporter+Cadvisor+Promtail). VM scrapt 76 Targets, **69 UP / 7 DOWN** (DOWN sind alle pre-existing wrong /metrics endpoints auf Mana-Services, nicht durch Migration). Konfig-Pfade: `monitoring/{prometheus,loki,blackbox,alertmanager,alert-notifier}/`. Bekannte Limits siehe unten. | | Phase 2d — Glitchtip mit dediziertem DB-Stack | ✅ | 4 Container neu (mana-mon-glitchtip + worker + dedizierte glitchtip-postgres + glitchtip-redis). Mini-Postgres scheiterte bei `logs.0001_initial`-Partition-Creation mit OS-level "Permission denied" (macOS-Docker-Storage-Quirk auf externer SSD). Auf der GPU-Box mit Linux-ext4 saubere 333-Tabellen-Migration. Worker enqueuet UND finished Tasks → DB-Writes funktional (vorher hingen sie ewig). Public-Hostname `glitchtip.mana.how` → mana-gpu-server-Tunnel (config v23). | | Phase 2e — Status-Page auf GPU-Box | ✅ | 2 Container neu (`mana-mon-status-gen` + `mana-mon-status-nginx`). Sparse `/srv/mana/source` mit `mana-source-pull.timer` (stündlich) hostet das `generate-status-page.sh` und `mana-apps.ts`. status-gen schreibt in das Docker-Volume `status-output`, das status-nginx auf `:8090` ausliefert. Public-Hostname `status.mana.how` → mana-gpu-server-Tunnel (config v25). Bonus: behebt den Inode-Stale-Bind-Mount-Bug, der auf dem Mini bei jedem CD-`git checkout -f` die Status-Page kaputt machte. `vm.mana.how` (Phase-2c-Workaround für Mini→GPU-VM-Routing) wurde wieder aus dem Tunnel entfernt — VM ist nicht mehr public. | -| Phase 2f — drei weitere Hilfsdienste verlagert | ✅ | (1) **verdaccio** (npm.mana.how, was im mana-platform-Repo): Volume tar-stream + Config-bundle in mana-monorepo (`infrastructure/verdaccio/config.yaml`). (2) **news-ingester** (Bun-Background-Tick): Cross-LAN-DB zur Mini-Postgres. Cross-arch-Limit aufgedeckt — `docker save\|load` zwischen Mini (arm64) und GPU-Box (x86_64) wirft `exec format error`, daher nativer Build mit GPU-Box-eigenem Dockerfile in `infrastructure/news-ingester/` der `@mana/shared-rss` als `file:`-ref vendored. (3) **mana-ai** (AI Mission Runner): Cross-LAN für mana-api/mana-llm/mana-research, RSA-Key-Sync (`MANA_AI_PRIVATE_KEY_PEM`), `mana-ai.mana.how` zum GPU-Tunnel (config v28). Bonus: AI Mission Runner sitzt jetzt im selben docker-network wie gpu-llm/gpu-ollama — künftige direct-LLM-Pfade ohne Cloudflare-Round-Trip. Mini Container 44 → 42. | +| Phase 2f — drei weitere Hilfsdienste verlagert | ⚠️ teilweise zurückgerollt | (1) ~~**verdaccio** (npm.mana.how, was im mana-platform-Repo): Volume tar-stream + Config-bundle in mana-monorepo (`infrastructure/verdaccio/config.yaml`)~~ — am 2026-05-07 zurückgerollt: das Storage-Volume kam nie auf der GPU-Box an, der dortige Container war leer. DNS+Tunnel zurück auf Mini, Mini-Standalone-Compose-Project unter `~/projects/verdaccio/` bleibt Single-Source. (2) **news-ingester** (Bun-Background-Tick): Cross-LAN-DB zur Mini-Postgres. Cross-arch-Limit aufgedeckt — `docker save\|load` zwischen Mini (arm64) und GPU-Box (x86_64) wirft `exec format error`, daher nativer Build mit GPU-Box-eigenem Dockerfile in `infrastructure/news-ingester/` der `@mana/shared-rss` als `file:`-ref vendored. (3) **mana-ai** (AI Mission Runner): Cross-LAN für mana-api/mana-llm/mana-research, RSA-Key-Sync (`MANA_AI_PRIVATE_KEY_PEM`), `mana-ai.mana.how` zum GPU-Tunnel (config v28). Bonus: AI Mission Runner sitzt jetzt im selben docker-network wie gpu-llm/gpu-ollama — künftige direct-LLM-Pfade ohne Cloudflare-Round-Trip. Mini Container 44 → 43 (verdaccio bleibt Mini-side). | | Phase 2g — mana-research auslagern | ✅ | Web-Research-Orchestrator mit 16+ Search-/LLM-Providern. Nativer Build via workspace-Dockerfile (sparse-checkout `services/mana-research` + `packages/{shared-research,shared-types,shared-hono,shared-logger}`). Cross-LAN zu mana-auth/mana-credits/mana-llm/mana-search/postgres/redis (alle auf 192.168.178.131); Redis-Auth via `REDIS_PASSWORD` aus Mini's `.env.macmini` übernommen. `research.mana.how` zum GPU-Tunnel umgebogen via Cloudflare-API (config v29). Beide `PUBLIC_MANA_RESEARCH_URL`-Vars in mana-app-web auf https-URL umgestellt — gleicher Cross-LAN-Bridge-Pattern wie mana-ai (Mini-Container können 192.168.178.11 nicht direkt erreichen, daher Tunnel-Roundtrip). Mini Container 42 → 41. | | Phase 3 — Daten-Migration | n/a | Alle migrierten Apps lesen Mini-Postgres direkt — keine separate Datenmigration | | Phase 4 — Cloudflare-Cutover | ✅ | API-Approach via `cert.pem` apiToken: PUT `/accounts/.../cfd_tunnel/.../configurations` für GPU-Tunnel, dann `cloudflared tunnel route dns --overwrite-dns`. Kein Dashboard-Klick nötig. 3 Hostnames live (grafana/git/stats) | @@ -161,8 +161,9 @@ als langlebigen Windows-Prozess offen → WSL-VM idled nicht aus, Container `mana-app-uload-server`, `mana-app-llm-playground`, `mana-admin`, `mana-mail`, `mana-status-gen`, `mana-infra-landings` (nginx), `memoro-server`, `memoro-audio-server`, `memoro-landing`, `chorportal-app` + `chorportal-prod-postgres` -+ `chorportal-prod-minio`, `news-ingester`, `mana-verdaccio` (npm-Registry, -Migration auf GPU-Box ist späterer Schritt). ++ `chorportal-prod-minio`, `mana-verdaccio` (npm-Registry, Standalone-Compose +unter `~/projects/verdaccio/`; Phase-2f-1-Migration zur GPU am 2026-05-07 +zurückgerollt — Storage-Volume kam dort nie an). **Box-lokale Helpers (laufen auf jeder Box separat):** - `node-exporter` (Host-Metriken) diff --git a/infrastructure/README.md b/infrastructure/README.md index bf39da9f2..ee93309ef 100644 --- a/infrastructure/README.md +++ b/infrastructure/README.md @@ -21,7 +21,6 @@ Hilfsdienste vom Mini abgegeben — siehe [`docs/PLAN_OPTION_C.md`](../docs/PLAN | `gpu-node-exporter`, `gpu-cadvisor`, `gpu-promtail` | (intern) | Self-Monitoring (Phase 2c) | | `glitchtip` + worker + dedizierte postgres + redis | `:8020` → `glitchtip.mana.how` | Error-Tracking mit eigenem DB-Stack (Phase 2d) | | `status-page-gen`, `status-nginx` | `:8090` → `status.mana.how` | Status-Seite (Phase 2e) | -| `verdaccio` | `:4873` → `npm.mana.how` | Private @mana/* npm-Registry (Phase 2f-1) | | `news-ingester` | (intern) | RSS-Crawl + News-Ingestion (Phase 2f-2) | | `mana-ai` | `:3067` → `mana-ai.mana.how` | AI Mission Runner (Phase 2f-3) | | `mana-research` | `:3068` → `research.mana.how` | Web-Research-Orchestrator (Phase 2g) | @@ -83,7 +82,6 @@ Aktive Public-Hostnames (Stand 2026-05-07, config v28): | `glitchtip.mana.how` | `:8020` | Glitchtip (Phase 2d) | | `status.mana.how` | `:8090` | Status-Page (Phase 2e) | | `photon.mana.how` | `:2322` | Photon Geocoder (cross-LAN-Workaround für mana-geocoding's Probe + privacy-local Provider) | -| `npm.mana.how` | `:4873` | Verdaccio @mana/* npm-Registry (Phase 2f-1) | | `mana-ai.mana.how` | `:3067` | AI Mission Runner (Phase 2f-3) | | `research.mana.how` | `:3068` | Web-Research-Orchestrator (Phase 2g) | diff --git a/infrastructure/docker-compose.gpu-box.yml b/infrastructure/docker-compose.gpu-box.yml index 35d3d167c..36bfc8243 100644 --- a/infrastructure/docker-compose.gpu-box.yml +++ b/infrastructure/docker-compose.gpu-box.yml @@ -485,14 +485,6 @@ services: retries: 3 start_period: 30s - # ============================================ - # Phase 2f-1 — Verdaccio npm-Registry (2026-05-07) - # @mana/* private packages. Migrated from Mini (was in mana-platform - # repo's infrastructure/docker-compose.macmini.yml). Read-heavy bei - # CI-Builds, latency-unkritisch — perfekt für GPU-Box-Hosting. Config - # lebt jetzt in mana-monorepo's infrastructure/verdaccio/config.yaml, - # sparse-clone trägt das Verzeichnis auf der GPU-Box ein. - # ============================================ # ============================================ # Phase 2f-2 — news-ingester (2026-05-07) # Background article-ingester — Bun-Service mit 15-min-Tick. Schreibt @@ -613,30 +605,9 @@ services: retries: 3 start_period: 30s - verdaccio: - image: verdaccio/verdaccio:6 - container_name: mana-verdaccio - restart: unless-stopped - ports: - - '4873:4873' - environment: - VERDACCIO_PORT: 4873 - volumes: - - /srv/mana/source/infrastructure/verdaccio/config.yaml:/verdaccio/conf/config.yaml:ro - - verdaccio-storage:/verdaccio/storage - - verdaccio-plugins:/verdaccio/plugins - healthcheck: - test: ['CMD', 'wget', '--quiet', '--tries=1', '--spider', 'http://localhost:4873/-/ping'] - interval: 60s - timeout: 5s - retries: 3 - start_period: 30s - volumes: glitchtip-pg-data: status-output: - verdaccio-storage: - verdaccio-plugins: mana-grafana-data: victoriametrics-data: loki-data: diff --git a/infrastructure/verdaccio/config.yaml b/infrastructure/verdaccio/config.yaml deleted file mode 100644 index c1bae9cc6..000000000 --- a/infrastructure/verdaccio/config.yaml +++ /dev/null @@ -1,87 +0,0 @@ -# Verdaccio config — mana e.V. private npm registry. -# Docs: https://verdaccio.org/docs/configuration -# -# Storage layout (writable inside the container): -# /verdaccio/storage — Tar.gz pro Paket-Version + Index-JSONs -# /verdaccio/htpasswd — bcrypt-gehashte User-Credentials - -storage: /verdaccio/storage -plugins: /verdaccio/plugins - -# Network address Verdaccio listens on. Inside the container always 0.0.0.0; -# the host binding (4873) is configured in docker-compose. -listen: 0.0.0.0:4873 - -# --- Web UI ----------------------------------------------------------- -web: - title: mana e.V. — npm registry - gravatar: false - scope: '@mana' - -# --- Authentication --------------------------------------------------- -# htpasswd file lives in the storage volume so it survives container -# restarts. Add users with: -# docker exec mana-verdaccio htpasswd -B /verdaccio/htpasswd -# -# `max_users: -1` disables web-based self-registration. Users come in -# only via htpasswd (admin-controlled). -auth: - htpasswd: - # Lives inside the storage volume so it survives restarts and gets - # backed up with the rest of the registry state. - file: /verdaccio/storage/htpasswd - # No new self-registration. Add users by inserting a row into - # htpasswd manually (`docker exec mana-verdaccio htpasswd -B …`) - # or by flipping this to a positive value briefly. - max_users: -1 - -# --- Public-package proxying ----------------------------------------- -# Verdaccio fetches `hono`, `react`, etc. from npmjs.org on first request -# and caches them in the storage volume. Speeds up subsequent installs -# and gives us continuity if npmjs.org is down. -uplinks: - npmjs: - url: https://registry.npmjs.org/ - cache: true - timeout: 30s - -# --- Access control --------------------------------------------------- -# Pattern matching is first-match. Order matters: @mana/* before ** -# -# `$authenticated` = user logged in via htpasswd -# `$all` = anyone (public anonymous reads) -# -# Our policy: -# @mana/* — only authenticated users can read or publish -# ** — authenticated users can install (proxied from npmjs) -# and the registry never serves these to anonymous reads -# either (we don't run an open mirror for the public) -packages: - '@mana/*': - access: $authenticated - publish: $authenticated - unpublish: $authenticated - proxy: # not proxied — we own the namespace - - '**': - access: $authenticated - publish: $authenticated - proxy: npmjs - -# --- Logging ---------------------------------------------------------- -# Console-only — Docker captures it. For long-term audit (DSGVO Art. 30 -# claim of code-provenance) we'd add file logging in production. -log: { type: stdout, format: pretty, level: http } - -# --- Security ----------------------------------------------------------- -security: - api: - legacy: true - web: - sign: - expiresIn: 7d - verify: - someProp: someValue - -# --- Limits ----------------------------------------------------------- -max_body_size: 50mb