managarten/.github/workflows/ci.yml
Till JS f39e72340c chore(ci): drop 16 dead build-* jobs + per-product detect-changes branches
The CI workflow accumulated 16 build jobs for apps/services that no
longer exist after the 2026-04 unification (chat, todo, calendar, clock,
contacts, presi, storage, food, skilltree, telegram-stats-bot — all of
their /apps/web and /apps/backend dirs are gone) plus a structurally
broken `build-mana-web` (its Dockerfile starts FROM `sveltekit-base:local`,
an image only the Mac Mini self-hosted runner has). Every push has been
producing red CI runs from these dead jobs while the real production
deploys (cd-macmini.yml) succeeded.

Removed:
- detect-changes per-product outputs + force-build-all branches
- 220 lines of dead per-product detect-logic shell
- 19 lines of per-product summary block
- build-mana-web (broken; CD on Mac Mini covers prod)
- build-{chat,todo,calendar,clock,contacts,presi,storage,food,skilltree}-{web,backend}
- build-telegram-stats-bot

Kept (still build cleanly on ubuntu-latest):
- build-mana-{auth,search,sync,notify,api-gateway,crawler,media,credits}
- validate (PRs)
- auth-integration (PRs)

CI workflow shrank 1290 → 583 lines. The header comment now spells out
which services are in/out of CI and why.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 15:32:43 +02:00

593 lines
22 KiB
YAML

# CI Pipeline: validates PRs and sanity-builds standalone service images.
#
# Flow:
# PR → main/dev : validate + auth-integration (required status check).
# Push → main/dev : sanity-build the changed standalone services
# (mana-auth/-search/-sync/-notify/-api-gateway/-crawler/
# -media/-credits) on GitHub-hosted runners.
#
# What's NOT in CI:
# - mana-web: built by the Mac Mini CD pipeline (cd-macmini.yml). Its
# Dockerfile starts FROM `sveltekit-base:local`, an image only the
# self-hosted runner has. Putting it in CI would require a sveltekit-
# base prebuild step here too — out of scope until we add a real
# consumer for the ghcr.io image.
# - apps/api, mana-events, mana-analytics, mana-geocoding, mana-mail,
# mana-llm, mana-research, mana-mcp, mana-ai, mana-persona-runner,
# mana-user, mana-subscriptions, mana-image-gen, mana-video-gen,
# mana-stt, mana-tts, mana-voice-bot, mana-landing-builder: also
# built by the Mac Mini CD pipeline. Add a sanity-build job here
# only when there's a non-CD consumer for the image.
#
# Production deployment is the cd-macmini.yml workflow, which auto-fires
# on push to main and rebuilds only the affected services on the
# self-hosted runner.
name: CI
on:
push:
branches:
- main
- dev
pull_request:
branches:
- main
- dev
workflow_dispatch:
inputs:
force_build_all:
description: 'Force rebuild all services'
required: false
default: 'false'
type: boolean
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
NODE_VERSION: '20'
PNPM_VERSION: '9.15.0'
jobs:
# ===========================================
# Detect Changes - determines which services to build
# ===========================================
detect-changes:
name: Detect Changes
runs-on: ubuntu-latest
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
outputs:
mana-auth: ${{ steps.changes.outputs.mana-auth }}
mana-sync: ${{ steps.changes.outputs.mana-sync }}
mana-media: ${{ steps.changes.outputs.mana-media }}
mana-notify: ${{ steps.changes.outputs.mana-notify }}
mana-api-gateway: ${{ steps.changes.outputs.mana-api-gateway }}
mana-crawler: ${{ steps.changes.outputs.mana-crawler }}
mana-credits: ${{ steps.changes.outputs.mana-credits }}
mana-search: ${{ steps.changes.outputs.mana-search }}
any-changes: ${{ steps.changes.outputs.any-changes }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for changes
id: changes
run: |
# Force build all if workflow_dispatch with force_build_all
if [ "${{ github.event_name }}" == "workflow_dispatch" ] && [ "${{ inputs.force_build_all }}" == "true" ]; then
echo "Force rebuild all services requested"
echo "mana-auth=true" >> $GITHUB_OUTPUT
echo "mana-sync=true" >> $GITHUB_OUTPUT
echo "mana-media=true" >> $GITHUB_OUTPUT
echo "mana-notify=true" >> $GITHUB_OUTPUT
echo "mana-api-gateway=true" >> $GITHUB_OUTPUT
echo "mana-crawler=true" >> $GITHUB_OUTPUT
echo "mana-credits=true" >> $GITHUB_OUTPUT
echo "mana-search=true" >> $GITHUB_OUTPUT
echo "any-changes=true" >> $GITHUB_OUTPUT
exit 0
fi
# Get changed files between current commit and previous
if [ "${{ github.event_name }}" == "push" ]; then
BASE_SHA="${{ github.event.before }}"
HEAD_SHA="${{ github.sha }}"
# Handle initial commit case
if [ "$BASE_SHA" == "0000000000000000000000000000000000000000" ]; then
CHANGED_FILES=$(git ls-files)
else
CHANGED_FILES=$(git diff --name-only $BASE_SHA $HEAD_SHA 2>/dev/null || git ls-files)
fi
else
# workflow_dispatch without force - build all
echo "Workflow dispatch without force_build_all - building all"
echo "mana-auth=true" >> $GITHUB_OUTPUT
echo "mana-sync=true" >> $GITHUB_OUTPUT
echo "mana-media=true" >> $GITHUB_OUTPUT
echo "mana-notify=true" >> $GITHUB_OUTPUT
echo "mana-api-gateway=true" >> $GITHUB_OUTPUT
echo "mana-crawler=true" >> $GITHUB_OUTPUT
echo "mana-credits=true" >> $GITHUB_OUTPUT
echo "mana-search=true" >> $GITHUB_OUTPUT
echo "any-changes=true" >> $GITHUB_OUTPUT
exit 0
fi
echo "Changed files:"
echo "$CHANGED_FILES"
# Common files that trigger all builds
COMMON_PATTERNS="pnpm-lock.yaml|pnpm-workspace.yaml|package.json|.github/workflows/ci.yml"
# Shared packages that affect multiple services
SHARED_AUTH_PATTERN="packages/shared-auth/|packages/shared-types/"
SHARED_UI_PATTERN="packages/shared-ui/|packages/shared-theme/|packages/shared-icons/|packages/shared-tailwind/|packages/shared-branding/"
SHARED_WEB_PATTERN="packages/shared-auth-ui/|packages/shared-theme-ui/|packages/shared-feedback-ui/|packages/shared-subscription-ui/|packages/shared-splitscreen/"
# Function to check if any pattern matches
check_pattern() {
echo "$CHANGED_FILES" | grep -qE "$1" && echo "true" || echo "false"
}
# Check common changes
COMMON_CHANGED=$(check_pattern "$COMMON_PATTERNS")
SHARED_AUTH_CHANGED=$(check_pattern "$SHARED_AUTH_PATTERN")
SHARED_UI_CHANGED=$(check_pattern "$SHARED_UI_PATTERN")
SHARED_WEB_CHANGED=$(check_pattern "$SHARED_WEB_PATTERN")
echo "Common changed: $COMMON_CHANGED"
echo "Shared auth changed: $SHARED_AUTH_CHANGED"
echo "Shared UI changed: $SHARED_UI_CHANGED"
echo "Shared web changed: $SHARED_WEB_CHANGED"
# mana-auth: services/mana-auth
AUTH_CHANGED=$(check_pattern "services/mana-auth/")
if [ "$COMMON_CHANGED" == "true" ] || [ "$AUTH_CHANGED" == "true" ]; then
echo "mana-auth=true" >> $GITHUB_OUTPUT
else
echo "mana-auth=false" >> $GITHUB_OUTPUT
fi
# mana-search: services/mana-search
SEARCH_CHANGED=$(check_pattern "services/mana-search/")
if [ "$COMMON_CHANGED" == "true" ] || [ "$SEARCH_CHANGED" == "true" ]; then
echo "mana-search=true" >> $GITHUB_OUTPUT
else
echo "mana-search=false" >> $GITHUB_OUTPUT
fi
# Go services (standalone — no COMMON trigger, only own path + shared-go)
SHARED_GO_PATTERN="packages/shared-go/"
SHARED_GO_CHANGED=$(check_pattern "$SHARED_GO_PATTERN")
for GO_SVC in mana-sync mana-notify mana-api-gateway mana-crawler; do
SVC_CHANGED=$(check_pattern "services/${GO_SVC}/")
if [ "$SVC_CHANGED" == "true" ] || [ "$SHARED_GO_CHANGED" == "true" ]; then
echo "${GO_SVC}=true" >> $GITHUB_OUTPUT
else
echo "${GO_SVC}=false" >> $GITHUB_OUTPUT
fi
done
# Hono/Bun services (standalone — only own path)
for HONO_SVC in mana-media mana-credits; do
SVC_CHANGED=$(check_pattern "services/${HONO_SVC}/")
if [ "$SVC_CHANGED" == "true" ]; then
echo "${HONO_SVC}=true" >> $GITHUB_OUTPUT
else
echo "${HONO_SVC}=false" >> $GITHUB_OUTPUT
fi
done
# mana-web build job is handled by the Mac Mini CD pipeline
# (cd-macmini.yml) — that runner has the `sveltekit-base:local`
# image required by apps/mana/apps/web/Dockerfile. CI here only
# validates the standalone services that build cleanly on
# GitHub-hosted runners.
# Per-product apps (chat / todo / calendar / clock / contacts /
# presi / storage / food / skilltree) and `telegram-stats-bot`
# were removed when the unified mana app + apps/api consolidated
# everything in 2026-04. Their CI build jobs have been deleted
# along with the source dirs.
# Check if any service needs building
if grep -q "=true" $GITHUB_OUTPUT; then
echo "any-changes=true" >> $GITHUB_OUTPUT
else
echo "any-changes=false" >> $GITHUB_OUTPUT
fi
- name: Summary
run: |
echo "## Build Summary" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "| Service | Will Build |" >> $GITHUB_STEP_SUMMARY
echo "|---------|------------|" >> $GITHUB_STEP_SUMMARY
echo "| mana-auth | ${{ steps.changes.outputs.mana-auth }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-sync | ${{ steps.changes.outputs.mana-sync }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-media | ${{ steps.changes.outputs.mana-media }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-notify | ${{ steps.changes.outputs.mana-notify }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-api-gateway | ${{ steps.changes.outputs.mana-api-gateway }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-crawler | ${{ steps.changes.outputs.mana-crawler }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-credits | ${{ steps.changes.outputs.mana-credits }} |" >> $GITHUB_STEP_SUMMARY
echo "| mana-search | ${{ steps.changes.outputs.mana-search }} |" >> $GITHUB_STEP_SUMMARY
# ===========================================
# Validation job - runs on PRs
# ===========================================
validate:
name: Validate
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup pnpm
uses: pnpm/action-setup@v2
with:
version: ${{ env.PNPM_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'pnpm'
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Validate monorepo invariants (turbo + pgSchema + crypto)
run: pnpm run validate:all
- name: Type check
run: pnpm run type-check
- name: Format check
run: pnpm run format:check
- name: Lint
run: pnpm run lint
# Coverage is produced but not yet gating — we're establishing a baseline
# before flipping to hard-fail. `continue-on-error: true` surfaces a
# proper failure signal in the PR (unlike `|| true`, which silently
# swallowed everything) but still lets the overall job pass.
# Flip to blocking by removing `continue-on-error` once the suite is
# green on main for a full week. Tracked in docs/plans/concern-4-coverage.md.
- name: Test (with coverage)
id: tests
continue-on-error: true
run: pnpm run test:coverage
- name: Annotate test failure (non-blocking)
if: steps.tests.outcome == 'failure'
run: |
echo "::warning title=Test suite failed::pnpm run test:coverage returned a non-zero exit code. Coverage artifacts were still uploaded. See the 'Test (with coverage)' step above for details."
- name: Upload coverage reports
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-${{ github.run_id }}
path: |
**/coverage/lcov.info
**/coverage/coverage-summary.json
if-no-files-found: warn
retention-days: 14
- name: Security Audit
run: |
# Run pnpm audit and capture exit code
# Exit 0 if only moderate/low vulnerabilities
pnpm audit --audit-level=high || {
echo "::warning::Security vulnerabilities found. Run 'pnpm audit' locally for details."
exit 0 # Don't fail build on audit issues (just warn)
}
- name: Check for known vulnerable packages
run: |
# Check for packages with known critical vulnerabilities
# This is a basic check - for production, consider Snyk or similar
if grep -r "lodash@[0-3]\." pnpm-lock.yaml 2>/dev/null; then
echo "::warning::Potentially vulnerable lodash version detected"
fi
if grep -r "axios@0\.[0-9]\." pnpm-lock.yaml 2>/dev/null; then
echo "::warning::Potentially vulnerable axios version detected"
fi
# ===========================================
# Auth flow integration test
# ===========================================
# Spins up postgres + redis + mailpit + mana-auth + mana-notify via
# docker-compose.test.yml and runs tests/integration/auth-flow.test.ts.
# Catches register/verify/login/JWT/encryption-vault regressions before
# they can be merged. Required check — never bypass.
auth-integration:
name: Auth flow integration test
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
timeout-minutes: 15
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup pnpm
uses: pnpm/action-setup@v2
with:
version: ${{ env.PNPM_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'pnpm'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Run auth flow integration test
run: ./scripts/run-integration-tests.sh
- name: Dump test stack logs on failure
if: failure()
run: |
echo "::group::mana-auth logs"
docker logs mana-test-mana-auth 2>&1 | tail -200 || true
echo "::endgroup::"
echo "::group::mana-notify logs"
docker logs mana-test-mana-notify 2>&1 | tail -200 || true
echo "::endgroup::"
echo "::group::mailpit messages"
curl -s http://localhost:8026/api/v1/messages | head -100 || true
echo "::endgroup::"
# ===========================================
# Build Docker images - only changed services
# ===========================================
build-mana-auth:
name: Build mana-auth
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-auth == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-auth
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: .
file: services/mana-auth/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-mana-search:
name: Build mana-search
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-search == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-search
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: .
file: services/mana-search/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
# Go services
build-mana-sync:
name: Build mana-sync
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-sync == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-sync
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: .
file: services/mana-sync/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-mana-notify:
name: Build mana-notify
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-notify == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-notify
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: .
file: services/mana-notify/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-mana-api-gateway:
name: Build mana-api-gateway
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-api-gateway == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-api-gateway
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: .
file: services/mana-api-gateway/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-mana-crawler:
name: Build mana-crawler
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-crawler == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-crawler
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: .
file: services/mana-crawler/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
# Hono/Bun services
build-mana-media:
name: Build mana-media
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-media == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-media
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: services/mana-media/apps/api
file: services/mana-media/apps/api/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-mana-credits:
name: Build mana-credits
runs-on: ubuntu-latest
needs: detect-changes
if: needs.detect-changes.outputs.mana-credits == 'true'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/mana-credits
tags: type=raw,value=latest
- uses: docker/build-push-action@v5
with:
context: services/mana-credits
file: services/mana-credits/Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max