\`ci.yml\` had a \`pnpm run validate:monorepo\` step that referenced a
script defined nowhere in the repo — CI would fail at that step
whenever the validate job ran. Replacing it with a new bundled
\`validate:all\` script closes that gap and gives contributors a single
local command that mirrors what CI enforces.
- New \`validate:all\` chains the three fast repo-invariant checks
(turbo recursion, pgSchema isolation, crypto registry) with fail-fast
semantics. Runtime ~1s — suitable as a pre-push gate.
- \`validate:dockerfiles\` intentionally left out: its current output
is 41 pre-existing "MISSING" warnings on two web Dockerfiles, which
look like a validator-vs-wildcard-COPY mismatch rather than real
issues. Keeping it as a standalone script so those can be
triaged separately without blocking \`validate:all\`.
- ci.yml: four separate validate steps collapsed into one. The step
rename also removes the dead \`validate:monorepo\` call.
Verified: \`pnpm run validate:all\` exits 0 in ~1s — 138 packages
scanned for turbo recursion, 727 TypeScript files for raw pgTable,
190 Dexie tables classified in the crypto registry (85 encrypted,
105 allowlisted plaintext).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The "every Drizzle table uses pgSchema" rule was documented in
.claude/guidelines/database.md (added yesterday as part of Concern 5)
but enforced only by convention. A new service could slip a raw
\`pgTable()\` past review and collide in the default \`public\` schema
of \`mana_platform\`, and nothing would surface the mistake until a
production migration failed.
- \`scripts/validate-pg-schema-isolation.mjs\` scans every tracked
TypeScript file under services/, apps/api/, packages/ for call sites
of \`pgTable(\` (not imports — imports can still be useful for types).
Strips comments before matching so doc-examples like "use \`pgTable()\`"
don't trigger false positives.
- Wired as \`pnpm run validate:pg-schema\` and a new CI step in the
validate job (right after the turbo-recursion check). 721 files
scan clean today.
- Removed an unused \`pgTable\` import in mana-subscriptions that would
have been the only import of the symbol remaining after this change.
- Updated .claude/guidelines/database.md — the old verification blurb
said "no automated lint rule yet", now points at the enforcer.
Drift verified: injecting a synthetic \`pgTable('bad', {})\` into
subscriptions.ts failed with a clear file:line violation pointing at
the database guideline.
Closes the "no automated lint rule" gap noted in the database guideline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CLAUDE.md flagged this as "CRITICAL" — a child package.json defining
e.g. \`"build": "turbo run build"\` causes a 10+ minute CI hang with
thousands of duplicate task spawns. The rule was documented but never
enforced, so it re-emerged every couple of months as someone copied a
parent script pattern.
- \`scripts/validate-no-recursive-turbo.mjs\` walks every tracked
package.json (via \`git ls-files\`, so node_modules is auto-skipped)
and fails if any non-root package has build/type-check/lint/test/
test:coverage/check scripts containing \`turbo run\`. \`dev\` stays
allowed — delegating it from a parent is the intended ergonomic.
- Wired as \`pnpm run validate:turbo\` + a new CI step in the validate
job (before type-check — fails fast).
- CLAUDE.md §Turborepo updated to point at the enforcer and call out
the full task list (test/test:coverage/check were missing from the
original prose).
Verified: 138 non-root package.json files scan clean. Drift simulation
(injecting \`"build": "turbo run build"\` into apps/mana/apps/web) fails
with a clear message pointing at the offending file + script + fix.
This closes audit item #32 from the architecture review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Before: adding a new Dexie table left the encryption decision implicit.
If you forgot to register it, the table silently shipped in plaintext
forever — no error, no warning, no footprint anywhere. The architecture
audit flagged this as the root of Concern 1.
- `scripts/audit-crypto-registry.mjs` parses database.ts's `.stores()`
blocks and registry.ts's entries, then enforces three invariants:
1. Every Dexie table is either in the encryption registry OR in the
new `plaintext-allowlist.ts` — one conscious classification per
table.
2. No dead registry entries (referring to tables that no longer
exist in Dexie).
3. No table appears in both — single authoritative source.
- `plaintext-allowlist.ts` auto-seeded from current state. 105 entries,
each tagged `// TODO: audit` as an invitation to review whether the
table truly holds nothing sensitive. The allowlist is intentionally
a separate file so additions are reviewable on their own (not buried
inside database.ts schema bumps).
- Wired into `pnpm run check:crypto` + CI validate job — a new table
now fails the PR check instead of slipping past review.
- `check:crypto:seed` regenerates the allowlist if ever needed.
Verified: drift simulation (removing aiMissions from the allowlist)
fails the audit with a clear message pointing at the missing
classification. Current state passes: 187 Dexie tables, 82 encrypted,
105 explicit plaintext.
Concern 1 is now fully closed (A: typed registry entries, B: dev-mode
runtime drift check, C: build-time audit enforcing coverage).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI previously ran `pnpm run test || true` — test failures were silently
swallowed with no artifact, so we had no visibility into what was actually
passing across 1,296 test files.
- New `test:coverage` turbo pipeline task + root script; packages that opt
in by declaring their own `test:coverage` get picked up automatically.
- Wired up three high-value Vitest targets: apps/mana/apps/web (main
frontend, ~590 tests), shared-ui (Svelte component library), and
shared-storage (S3 client). Each emits lcov.info + coverage-summary.json
+ browsable HTML.
- apps/mana/apps/web `"test"` was running in watch mode (just `vitest`),
which hangs under turbo orchestration — changed to `vitest run` and
added `test:watch` for the interactive case.
- CI uploads coverage artifacts (14-day retention) regardless of whether
tests passed. `continue-on-error: true` replaces `|| true` so a failed
suite shows up as a warning annotation on the PR rather than being
invisible. Flip to a hard gate once main is green for a full week.
- Testing guideline documents the pattern + the template vitest config
+ the planned 80% threshold.
- ESLint flat-config `vitest.config.ts` ignore only matched at the root;
widened to `**/vitest.config.{ts,js,mjs}` so nested configs don't trip
the project-service parser.
Coverage baseline produced locally:
shared-storage: 91.37% lines (6 files, 123 tests)
shared-ui: 2.87% lines (mostly Svelte components, untested)
apps/mana/web: 9/59 test files fail — pre-existing, not regression
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a 13-step integration test that exercises register → email
verification → login → JWT validation → /me/data → encryption-vault
init/key → logout against a real stack of postgres + redis + mailpit +
mana-auth + mana-notify in docker compose.
Verified locally that this catches every regression we hit on
2026-04-08 in well under a second:
- missing nanoid dependency → register endpoint 500
- missing MANA_AUTH_KEK env passthrough → mana-auth never starts
- missing encryption-vault SQL migrations → vault endpoints 500
- wrong cookie name in /api/v1/auth/login → no accessToken in response
- mana-notify SMTP misconfigured → mailpit poll times out
Files:
- docker-compose.test.yml — minimal isolated stack on alt ports
(postgres 5443, redis 6390, mailpit 1026/8026, mana-auth 3091,
mana-notify 3092). Runs alongside the dev stack without collision.
Postgres healthcheck runs a real query rather than just pg_isready
to avoid the race where pg_isready reports healthy while the docker
init scripts are still running on a unix socket.
- tests/integration/auth-flow.test.ts — bun test that drives the full
flow via fetch + mailpit's REST API. Cleans up its test user from
postgres in afterAll. Self-contained, no extra deps.
- tests/integration/README.md — what's covered, why it exists, how
to run locally + extend.
- scripts/run-integration-tests.sh — orchestrator. Brings up the
stack, pushes the @mana/auth Drizzle schema, applies the
encryption-vault SQL migrations (002, 003), restarts mana-auth so
it sees the fresh tables, runs the test, tears down on exit.
KEEP_STACK=1 to leave it up for manual mailpit inspection.
- docker-compose.dev.yml — also adds Mailpit as a regular dev service
(ports 1025/8025) so local development can have a working email
capture without spinning up the test stack.
- .github/workflows/ci.yml — new auth-integration job that runs on
every PR. Calls run-integration-tests.sh; on failure dumps
mana-auth + mana-notify logs and the mailpit message queue. Marked
as a required check via the existing PR validation pipeline.
Reproduced 3 clean runs and 1 negative-control run (removed nanoid
from package.json → mana-auth container exits → script aborts with
non-zero) before committing. Full happy path runs in ~22s on a warm
Docker cache.
This commit bundles two unrelated changes that were swept together by an
accidental `git add -A` in another working session. Documented here so the
history reflects what's actually inside.
═══════════════════════════════════════════════════════════════════════
1. fix(mana-auth): /api/v1/auth/login mints JWT via auth.handler instead
of api.signInEmail
═══════════════════════════════════════════════════════════════════════
Previous attempt (commit 55cc75e7d) tried to fix the broken JWT mint in
/api/v1/auth/login by switching the cookie name from `mana.session_token`
to `__Secure-mana.session_token` for production. That was necessary but
not sufficient: Better Auth's session cookie value isn't just the raw
session token, it's `<token>.<HMAC>` where the HMAC is derived from the
better-auth secret. Reconstructing the cookie from auth.api.signInEmail's
JSON response only gave us the raw token, so /api/auth/token's
get-session middleware still couldn't validate it and the JWT mint kept
silently failing.
Real fix: do the sign-in via auth.handler (the HTTP path) rather than
auth.api.signInEmail (the SDK path). The handler returns a real fetch
Response with a Set-Cookie header containing the fully signed cookie
envelope. We capture that header verbatim and forward it as the cookie
on the /api/auth/token request, which now passes validation and mints
the JWT correctly.
Verified end-to-end on auth.mana.how:
$ curl -X POST https://auth.mana.how/api/v1/auth/login \
-d '{"email":"...","password":"..."}'
{
"user": {...},
"token": "<session token>",
"accessToken": "eyJhbGciOiJFZERTQSI...", ← real JWT now
"refreshToken": "<session token>"
}
Side benefits:
- Email-not-verified path is now handled by checking
signInResponse.status === 403 directly, no more catching APIError
with the comment-noted async-stream footgun.
- X-Forwarded-For is forwarded explicitly so Better Auth's rate limiter
and our security log see the real client IP.
- The leftover catch block now only handles unexpected exceptions
(network errors etc); the FORBIDDEN-checking logic in it is dead but
harmless and left in for defense in depth.
═══════════════════════════════════════════════════════════════════════
2. chore: remove the entire self-hosted Matrix stack (Synapse, Element,
Manalink, mana-matrix-bot)
═══════════════════════════════════════════════════════════════════════
The Matrix subsystem ran parallel to the main Mana product without any
load-bearing integration: the unified web app never imported matrix-js-sdk,
the chat module uses mana-sync (local-first), and mana-matrix-bot's
plugins duplicated features the unified app already ships natively.
Keeping it alive cost a Synapse + Element + matrix-web + bot container
quartet, three Cloudflare routes, an OIDC provider plugin in mana-auth,
and a steady drip of devlog/dependency churn.
Removed:
- apps/matrix (Manalink web + mobile, ~150 files)
- services/mana-matrix-bot (Go bot with ~20 plugins)
- docker/matrix configs (Synapse + Element)
- synapse/element-web/matrix-web/mana-matrix-bot services in
docker-compose.macmini.yml
- matrix.mana.how/element.mana.how/link.mana.how Cloudflare tunnel routes
- OIDC provider plugin + matrix-synapse trustedClient + matrixUserLinks
table from mana-auth (oauth_* schema definitions also removed)
- MatrixService import path in mana-media (importFromMatrix endpoint)
- Matrix notification channel in mana-notify (worker, metrics, config,
channel_type enum, MatrixOptions handler)
- Matrix entries from shared-branding (mana-apps + app-icons),
notify-client, the i18n bundle, the observatory map, the credits
app-label list, the landing footer/apps page, the prometheus + alerts
+ promtail tier mappings, and the matrix-related deploy paths in
cd-macmini.yml + ci.yml
Devlog/manascore/blueprint entries that mention Matrix are left intact
as historical record. The oauth_* + matrix_user_links Postgres tables
stay on existing prod databases — code can no longer write to them, drop
them in a follow-up migration if you want them gone for real.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Presi NestJS backend (40 source files, 50 deps) was a CRUD wrapper
around decks, slides, and themes — all now handled by local-first sync.
Only the share-link feature requires server-side state (public URLs
without auth), so a minimal Hono + Bun server replaces the entire
NestJS backend:
- apps/presi/apps/server/ — Hono server with share routes + GDPR admin
Uses @manacore/shared-hono for auth (JWKS), health, admin, errors
- Web app API client stripped to share-only (was 270 lines → 90 lines)
- Removed from docker-compose, CI/CD, Prometheus, env generation
- NestJS backend deleted (40 TS files, 8 test specs, 3038 lines)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Both apps are fully local-first via Dexie.js + mana-sync. Their NestJS
backends were pure CRUD wrappers (20 + 31 source files) that are no
longer needed.
Changes:
- Add packages/shared-hono: JWT auth via JWKS (jose), Drizzle DB factory,
health route, generic GDPR admin handler, error middleware
- Migrate zitare lists page from fetch() to listsStore (local-first)
- Rewrite clock timers store from API-based to timerCollection (Dexie)
- Update clock +layout.svelte CommandBar search to use local collections
- Remove zitare-backend + clock-backend from docker-compose, CI/CD,
Prometheus, env generation, setup scripts
- Add docs/TECHNOLOGY_AUDIT_2026_03.md with full repo analysis
Net result: -2 Docker containers, -2 ports, -2728 lines of code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the NestJS mana-search service with a Go implementation for
lower resource usage and faster startup. All 7 API endpoints are 1:1
compatible (search, extract, bulk extract, engines, health, metrics,
cache clear). Uses go-readability for content extraction and
html-to-markdown for Markdown conversion. Redis cache with graceful
degradation, Prometheus metrics, and structured JSON logging.
Binary: 22 MB vs ~200+ MB node_modules.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace 21 separate NestJS Matrix bot processes (~2.1 GB RAM, ~4.2 GB Docker images)
with a single Go binary using plugin architecture (8.6 MB binary, ~30 MB RAM).
New services:
- services/mana-matrix-bot/ — Go Matrix bot with 21 plugins (mautrix-go, Redis sessions)
- services/mana-api-gateway-go/ — Go API gateway (rate limiting, API keys, credit billing)
Deleted:
- 21 services/matrix-*-bot/ directories
- packages/bot-services/ and packages/matrix-bot-common/
- Legacy deploy scripts and CI build jobs
Updated:
- docker-compose.macmini.yml: new Go services, legacy bots removed
- CI/CD: change detection + build jobs for Go services
- Root package.json: new dev:matrix, build:matrix, test:matrix scripts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add format:check step (prettier --check, hard fail)
- Make lint step a hard fail (remove || echo fallback)
- Add test step (soft fail for now — coverage is thin)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add Dockerfile for zitare-backend (multi-stage build, port 3007)
- Add docker-entrypoint.sh for database setup
- Add zitare-backend service to docker-compose.macmini.yml
- Update matrix-zitare-bot to depend on zitare-backend
- Add zitare-backend to CI workflow (change detection + build job)
Storage-backend build was failing on ARM64 due to QEMU emulation
"Illegal instruction" crash when building native dependencies.
Same approach used for matrix-mana-bot and matrix-tts-bot.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All matrix bots use matrix-bot-sdk which has native dependencies
(cpu-features, ssh2) that cause QEMU emulation failures during CI
arm64 builds. Build amd64 only - can run on arm64 via Rosetta.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Build matrix-mana-bot only for linux/amd64 (arm64 fails due to QEMU)
- Move pnpm overrides for cpu-features and ssh2 to root package.json
- These native deps cause illegal instruction errors under QEMU emulation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
OIDC providers like Synapse expect the JWT issuer claim to match the
discovery document's issuer URL. Changed JWT plugin config from
JWT_ISSUER to BASE_URL to ensure consistency.
Also adds:
- @manacore/credit-operations package with operation definitions
- @manacore/shared-credit-ui package with React Native and Svelte components
- CreditInterceptor and @UseCredits decorator in nestjs-integration
- Credit system integration in chat backend
- Update 10 Matrix Bot Dockerfiles for monorepo builds with pnpm
- Add shared package support (bot-services, matrix-bot-common)
- Extend CI pipeline with change detection and build jobs
- Update docker-compose.macmini.yml to use GHCR images
- Enable Watchtower auto-updates for Matrix Bots
Bots: mana, ollama, stats, project-doc, todo, calendar, nutriphi, zitare, clock, tts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update mana-search Dockerfile to use monorepo root context
- Update api-gateway Dockerfile to use monorepo root context
- Fix CI workflow to use context: . for mana-search
- Use pnpm --filter for proper workspace package installation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add mana-search + SearXNG to docker-compose.macmini.yml
- Add api-gateway dependency on mana-search
- Add CI workflow for building mana-search Docker image
- Add CI workflow for building api-gateway Docker image
- Add NestJS backend with Drizzle ORM and PostgreSQL
- Implement skills CRUD API with XP/level progression system
- Add activities tracking endpoint
- Configure Docker containers for backend (port 3024) and web (port 5195)
- Add skilltree services to docker-compose.macmini.yml
- Add CI build jobs for skilltree-backend and skilltree-web
https://claude.ai/code/session_015XCsTDS9aLZ64Zin4HU6ex
- Add backend Dockerfile with multi-stage build and health checks
- Add web Dockerfile with SvelteKit static env vars
- Add docker-entrypoint.sh for automatic DB migration
- Add nutriphi-backend and nutriphi-web to docker-compose.macmini.yml
- Add CI/CD detection and build jobs for nutriphi
- Update CORS origins in mana-core-auth to include nutriphi.mana.how
- Include nutriphi in deploy:landing:all script
Ports: Backend 3023, Web 5189
Domain: nutriphi.mana.how / nutriphi-api.mana.how
Reverting 618c58c5 which broke the CI workflow.
Will re-add notifications after fixing the issue.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add notify-start job with Telegram notification for build start
- Add notify-complete job with build status and duration notification
- Push CI metrics to Prometheus Pushgateway for Grafana visualization
- Create CI/CD Grafana dashboard with build status, duration, and history
- Add Pushgateway scrape config to Prometheus
Requires TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, and PUSHGATEWAY_URL secrets.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds a NestJS service that delivers Umami analytics via Telegram:
- Telegram commands: /start, /stats, /today, /week, /realtime, /users
- Scheduled reports: Daily at 9:00, Weekly on Monday at 9:00
- Umami API integration with token management
- User statistics from auth database
- Docker + CI/CD pipeline integration
Bot: @stats_mana_bot
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Dockerfile for contacts-backend (port 3015)
- Add Dockerfile for contacts-web (port 5184)
- Add docker-entrypoint.sh for database migrations
- Update CI workflow with contacts-backend and contacts-web build jobs
- Add contacts services to docker-compose.macmini.yml
- Update CORS origins to include contacts.mana.how
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add detect-changes job that analyzes changed files
- Only build services that have changes in their paths
- Detects shared package changes and triggers dependent builds
- Add workflow_dispatch option to force rebuild all services
- Add build summary to GitHub Actions UI
This reduces build time by ~70% when only specific apps change.
Trigger patterns:
- Service path changes → only that service
- Shared packages → dependent services
- pnpm-lock.yaml, ci.yml → all services
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- CI: Build Docker images for linux/amd64 + linux/arm64
- CI: Add manacore-web to build matrix
- Add docker-compose.macmini.yml for Mac Mini deployment
- Add cloudflared-config.yml for Cloudflare Tunnel routing
- Add Mac Mini deployment scripts and documentation
- Configure Cloudflared as launchd service for auto-start
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Merge till-dev branch containing:
- Planta plant care tracking application
- Clock backend with alarms, timers, world clocks
- Zitare backend with favorites and lists
- Various app improvements and fixes
- Auth system updates
- Infrastructure improvements
Note: Some type-check issues may need resolution after merge.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add safe-db-push.mjs script for safer database migrations
- Update docker-entrypoint.sh with db:push fallback when migrations fail
- Add validate-migrations.mjs script for CI migration validation
- Update CI workflow to use migration validation
- Update drizzle.config.ts with improved configuration
- Add picture-backend and picture-web to CI Docker build matrix
- Add picture services to staging deployment workflow
- Add picture-backend to production deployment workflow
- Create Dockerfile and docker-entrypoint.sh for picture-web
- Fix picture-backend Dockerfile port (3003→3006) and health endpoint
- Add picture routes to Caddyfile.staging
- Add REPLICATE_API_TOKEN and MANA_CORE_SERVICE_KEY env vars
Add complete Docker deployment infrastructure for 4 new applications:
- Dockerfiles for backend (NestJS) and web (SvelteKit) apps
- docker-entrypoint.sh scripts with PostgreSQL wait and schema push
- Updated docker-compose.staging.yml with 7 new services
- Updated CI/CD workflows with build matrix and health checks
- Rename ci-main.yml to ci.yml for clarity
- Add PR-based validation (type-check, lint) for dev and main branches
- Add path filtering to skip CI on docs-only changes
- Trigger staging deployment only on push to dev branch
- Keep production deployment manual with confirmation
2025-12-08 12:54:25 +01:00
Renamed from .github/workflows/ci-main.yml (Browse further)
