Adds a 13-step integration test that exercises register → email
verification → login → JWT validation → /me/data → encryption-vault
init/key → logout against a real stack of postgres + redis + mailpit +
mana-auth + mana-notify in docker compose.
Verified locally that this catches every regression we hit on
2026-04-08 in well under a second:
- missing nanoid dependency → register endpoint 500
- missing MANA_AUTH_KEK env passthrough → mana-auth never starts
- missing encryption-vault SQL migrations → vault endpoints 500
- wrong cookie name in /api/v1/auth/login → no accessToken in response
- mana-notify SMTP misconfigured → mailpit poll times out
Files:
- docker-compose.test.yml — minimal isolated stack on alt ports
(postgres 5443, redis 6390, mailpit 1026/8026, mana-auth 3091,
mana-notify 3092). Runs alongside the dev stack without collision.
Postgres healthcheck runs a real query rather than just pg_isready
to avoid the race where pg_isready reports healthy while the docker
init scripts are still running on a unix socket.
- tests/integration/auth-flow.test.ts — bun test that drives the full
flow via fetch + mailpit's REST API. Cleans up its test user from
postgres in afterAll. Self-contained, no extra deps.
- tests/integration/README.md — what's covered, why it exists, how
to run locally + extend.
- scripts/run-integration-tests.sh — orchestrator. Brings up the
stack, pushes the @mana/auth Drizzle schema, applies the
encryption-vault SQL migrations (002, 003), restarts mana-auth so
it sees the fresh tables, runs the test, tears down on exit.
KEEP_STACK=1 to leave it up for manual mailpit inspection.
- docker-compose.dev.yml — also adds Mailpit as a regular dev service
(ports 1025/8025) so local development can have a working email
capture without spinning up the test stack.
- .github/workflows/ci.yml — new auth-integration job that runs on
every PR. Calls run-integration-tests.sh; on failure dumps
mana-auth + mana-notify logs and the mailpit message queue. Marked
as a required check via the existing PR validation pipeline.
Reproduced 3 clean runs and 1 negative-control run (removed nanoid
from package.json → mana-auth container exits → script aborts with
non-zero) before committing. Full happy path runs in ~22s on a warm
Docker cache.
This commit bundles two unrelated changes that were swept together by an
accidental `git add -A` in another working session. Documented here so the
history reflects what's actually inside.
═══════════════════════════════════════════════════════════════════════
1. fix(mana-auth): /api/v1/auth/login mints JWT via auth.handler instead
of api.signInEmail
═══════════════════════════════════════════════════════════════════════
Previous attempt (commit 55cc75e7d) tried to fix the broken JWT mint in
/api/v1/auth/login by switching the cookie name from `mana.session_token`
to `__Secure-mana.session_token` for production. That was necessary but
not sufficient: Better Auth's session cookie value isn't just the raw
session token, it's `<token>.<HMAC>` where the HMAC is derived from the
better-auth secret. Reconstructing the cookie from auth.api.signInEmail's
JSON response only gave us the raw token, so /api/auth/token's
get-session middleware still couldn't validate it and the JWT mint kept
silently failing.
Real fix: do the sign-in via auth.handler (the HTTP path) rather than
auth.api.signInEmail (the SDK path). The handler returns a real fetch
Response with a Set-Cookie header containing the fully signed cookie
envelope. We capture that header verbatim and forward it as the cookie
on the /api/auth/token request, which now passes validation and mints
the JWT correctly.
Verified end-to-end on auth.mana.how:
$ curl -X POST https://auth.mana.how/api/v1/auth/login \
-d '{"email":"...","password":"..."}'
{
"user": {...},
"token": "<session token>",
"accessToken": "eyJhbGciOiJFZERTQSI...", ← real JWT now
"refreshToken": "<session token>"
}
Side benefits:
- Email-not-verified path is now handled by checking
signInResponse.status === 403 directly, no more catching APIError
with the comment-noted async-stream footgun.
- X-Forwarded-For is forwarded explicitly so Better Auth's rate limiter
and our security log see the real client IP.
- The leftover catch block now only handles unexpected exceptions
(network errors etc); the FORBIDDEN-checking logic in it is dead but
harmless and left in for defense in depth.
═══════════════════════════════════════════════════════════════════════
2. chore: remove the entire self-hosted Matrix stack (Synapse, Element,
Manalink, mana-matrix-bot)
═══════════════════════════════════════════════════════════════════════
The Matrix subsystem ran parallel to the main Mana product without any
load-bearing integration: the unified web app never imported matrix-js-sdk,
the chat module uses mana-sync (local-first), and mana-matrix-bot's
plugins duplicated features the unified app already ships natively.
Keeping it alive cost a Synapse + Element + matrix-web + bot container
quartet, three Cloudflare routes, an OIDC provider plugin in mana-auth,
and a steady drip of devlog/dependency churn.
Removed:
- apps/matrix (Manalink web + mobile, ~150 files)
- services/mana-matrix-bot (Go bot with ~20 plugins)
- docker/matrix configs (Synapse + Element)
- synapse/element-web/matrix-web/mana-matrix-bot services in
docker-compose.macmini.yml
- matrix.mana.how/element.mana.how/link.mana.how Cloudflare tunnel routes
- OIDC provider plugin + matrix-synapse trustedClient + matrixUserLinks
table from mana-auth (oauth_* schema definitions also removed)
- MatrixService import path in mana-media (importFromMatrix endpoint)
- Matrix notification channel in mana-notify (worker, metrics, config,
channel_type enum, MatrixOptions handler)
- Matrix entries from shared-branding (mana-apps + app-icons),
notify-client, the i18n bundle, the observatory map, the credits
app-label list, the landing footer/apps page, the prometheus + alerts
+ promtail tier mappings, and the matrix-related deploy paths in
cd-macmini.yml + ci.yml
Devlog/manascore/blueprint entries that mention Matrix are left intact
as historical record. The oauth_* + matrix_user_links Postgres tables
stay on existing prod databases — code can no longer write to them, drop
them in a follow-up migration if you want them gone for real.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Presi NestJS backend (40 source files, 50 deps) was a CRUD wrapper
around decks, slides, and themes — all now handled by local-first sync.
Only the share-link feature requires server-side state (public URLs
without auth), so a minimal Hono + Bun server replaces the entire
NestJS backend:
- apps/presi/apps/server/ — Hono server with share routes + GDPR admin
Uses @manacore/shared-hono for auth (JWKS), health, admin, errors
- Web app API client stripped to share-only (was 270 lines → 90 lines)
- Removed from docker-compose, CI/CD, Prometheus, env generation
- NestJS backend deleted (40 TS files, 8 test specs, 3038 lines)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Both apps are fully local-first via Dexie.js + mana-sync. Their NestJS
backends were pure CRUD wrappers (20 + 31 source files) that are no
longer needed.
Changes:
- Add packages/shared-hono: JWT auth via JWKS (jose), Drizzle DB factory,
health route, generic GDPR admin handler, error middleware
- Migrate zitare lists page from fetch() to listsStore (local-first)
- Rewrite clock timers store from API-based to timerCollection (Dexie)
- Update clock +layout.svelte CommandBar search to use local collections
- Remove zitare-backend + clock-backend from docker-compose, CI/CD,
Prometheus, env generation, setup scripts
- Add docs/TECHNOLOGY_AUDIT_2026_03.md with full repo analysis
Net result: -2 Docker containers, -2 ports, -2728 lines of code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the NestJS mana-search service with a Go implementation for
lower resource usage and faster startup. All 7 API endpoints are 1:1
compatible (search, extract, bulk extract, engines, health, metrics,
cache clear). Uses go-readability for content extraction and
html-to-markdown for Markdown conversion. Redis cache with graceful
degradation, Prometheus metrics, and structured JSON logging.
Binary: 22 MB vs ~200+ MB node_modules.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace 21 separate NestJS Matrix bot processes (~2.1 GB RAM, ~4.2 GB Docker images)
with a single Go binary using plugin architecture (8.6 MB binary, ~30 MB RAM).
New services:
- services/mana-matrix-bot/ — Go Matrix bot with 21 plugins (mautrix-go, Redis sessions)
- services/mana-api-gateway-go/ — Go API gateway (rate limiting, API keys, credit billing)
Deleted:
- 21 services/matrix-*-bot/ directories
- packages/bot-services/ and packages/matrix-bot-common/
- Legacy deploy scripts and CI build jobs
Updated:
- docker-compose.macmini.yml: new Go services, legacy bots removed
- CI/CD: change detection + build jobs for Go services
- Root package.json: new dev:matrix, build:matrix, test:matrix scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add format:check step (prettier --check, hard fail)
- Make lint step a hard fail (remove || echo fallback)
- Add test step (soft fail for now — coverage is thin)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add Dockerfile for zitare-backend (multi-stage build, port 3007)
- Add docker-entrypoint.sh for database setup
- Add zitare-backend service to docker-compose.macmini.yml
- Update matrix-zitare-bot to depend on zitare-backend
- Add zitare-backend to CI workflow (change detection + build job)
Storage-backend build was failing on ARM64 due to QEMU emulation
"Illegal instruction" crash when building native dependencies.
Same approach used for matrix-mana-bot and matrix-tts-bot.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All matrix bots use matrix-bot-sdk which has native dependencies
(cpu-features, ssh2) that cause QEMU emulation failures during CI
arm64 builds. Build amd64 only - can run on arm64 via Rosetta.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Build matrix-mana-bot only for linux/amd64 (arm64 fails due to QEMU)
- Move pnpm overrides for cpu-features and ssh2 to root package.json
- These native deps cause illegal instruction errors under QEMU emulation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
OIDC providers like Synapse expect the JWT issuer claim to match the
discovery document's issuer URL. Changed JWT plugin config from
JWT_ISSUER to BASE_URL to ensure consistency.
Also adds:
- @manacore/credit-operations package with operation definitions
- @manacore/shared-credit-ui package with React Native and Svelte components
- CreditInterceptor and @UseCredits decorator in nestjs-integration
- Credit system integration in chat backend
- Update 10 Matrix Bot Dockerfiles for monorepo builds with pnpm
- Add shared package support (bot-services, matrix-bot-common)
- Extend CI pipeline with change detection and build jobs
- Update docker-compose.macmini.yml to use GHCR images
- Enable Watchtower auto-updates for Matrix Bots
Bots: mana, ollama, stats, project-doc, todo, calendar, nutriphi, zitare, clock, tts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update mana-search Dockerfile to use monorepo root context
- Update api-gateway Dockerfile to use monorepo root context
- Fix CI workflow to use context: . for mana-search
- Use pnpm --filter for proper workspace package installation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add mana-search + SearXNG to docker-compose.macmini.yml
- Add api-gateway dependency on mana-search
- Add CI workflow for building mana-search Docker image
- Add CI workflow for building api-gateway Docker image
- Add NestJS backend with Drizzle ORM and PostgreSQL
- Implement skills CRUD API with XP/level progression system
- Add activities tracking endpoint
- Configure Docker containers for backend (port 3024) and web (port 5195)
- Add skilltree services to docker-compose.macmini.yml
- Add CI build jobs for skilltree-backend and skilltree-web
https://claude.ai/code/session_015XCsTDS9aLZ64Zin4HU6ex
- Add backend Dockerfile with multi-stage build and health checks
- Add web Dockerfile with SvelteKit static env vars
- Add docker-entrypoint.sh for automatic DB migration
- Add nutriphi-backend and nutriphi-web to docker-compose.macmini.yml
- Add CI/CD detection and build jobs for nutriphi
- Update CORS origins in mana-core-auth to include nutriphi.mana.how
- Include nutriphi in deploy:landing:all script
Ports: Backend 3023, Web 5189
Domain: nutriphi.mana.how / nutriphi-api.mana.how
Reverting 618c58c5 which broke the CI workflow.
Will re-add notifications after fixing the issue.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add notify-start job with Telegram notification for build start
- Add notify-complete job with build status and duration notification
- Push CI metrics to Prometheus Pushgateway for Grafana visualization
- Create CI/CD Grafana dashboard with build status, duration, and history
- Add Pushgateway scrape config to Prometheus
Requires TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, and PUSHGATEWAY_URL secrets.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds a NestJS service that delivers Umami analytics via Telegram:
- Telegram commands: /start, /stats, /today, /week, /realtime, /users
- Scheduled reports: Daily at 9:00, Weekly on Monday at 9:00
- Umami API integration with token management
- User statistics from auth database
- Docker + CI/CD pipeline integration
Bot: @stats_mana_bot
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Dockerfile for contacts-backend (port 3015)
- Add Dockerfile for contacts-web (port 5184)
- Add docker-entrypoint.sh for database migrations
- Update CI workflow with contacts-backend and contacts-web build jobs
- Add contacts services to docker-compose.macmini.yml
- Update CORS origins to include contacts.mana.how
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add detect-changes job that analyzes changed files
- Only build services that have changes in their paths
- Detects shared package changes and triggers dependent builds
- Add workflow_dispatch option to force rebuild all services
- Add build summary to GitHub Actions UI
This reduces build time by ~70% when only specific apps change.
Trigger patterns:
- Service path changes → only that service
- Shared packages → dependent services
- pnpm-lock.yaml, ci.yml → all services
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- CI: Build Docker images for linux/amd64 + linux/arm64
- CI: Add manacore-web to build matrix
- Add docker-compose.macmini.yml for Mac Mini deployment
- Add cloudflared-config.yml for Cloudflare Tunnel routing
- Add Mac Mini deployment scripts and documentation
- Configure Cloudflared as launchd service for auto-start
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Merge till-dev branch containing:
- Planta plant care tracking application
- Clock backend with alarms, timers, world clocks
- Zitare backend with favorites and lists
- Various app improvements and fixes
- Auth system updates
- Infrastructure improvements
Note: Some type-check issues may need resolution after merge.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add safe-db-push.mjs script for safer database migrations
- Update docker-entrypoint.sh with db:push fallback when migrations fail
- Add validate-migrations.mjs script for CI migration validation
- Update CI workflow to use migration validation
- Update drizzle.config.ts with improved configuration
- Add picture-backend and picture-web to CI Docker build matrix
- Add picture services to staging deployment workflow
- Add picture-backend to production deployment workflow
- Create Dockerfile and docker-entrypoint.sh for picture-web
- Fix picture-backend Dockerfile port (3003→3006) and health endpoint
- Add picture routes to Caddyfile.staging
- Add REPLICATE_API_TOKEN and MANA_CORE_SERVICE_KEY env vars
Add complete Docker deployment infrastructure for 4 new applications:
- Dockerfiles for backend (NestJS) and web (SvelteKit) apps
- docker-entrypoint.sh scripts with PostgreSQL wait and schema push
- Updated docker-compose.staging.yml with 7 new services
- Updated CI/CD workflows with build matrix and health checks
- Rename ci-main.yml to ci.yml for clarity
- Add PR-based validation (type-check, lint) for dev and main branches
- Add path filtering to skip CI on docs-only changes
- Trigger staging deployment only on push to dev branch
- Keep production deployment manual with confirmation
2025-12-08 12:54:25 +01:00
Renamed from .github/workflows/ci-main.yml (Browse further)
