managarten/cicd/SETUP.md

CI/CD Setup Guide

Last Updated: 2025-11-27 Estimated Time: 30 minutes (Quick Start) to 7 days (Full Implementation)

---

📋 Table of Contents

1. Prerequisites
2. Quick Start (30 Minutes)
3. Phase 1: Infrastructure Foundation
4. Phase 2: First Deployment
5. Phase 3: Web Apps
6. Phase 4: Testing
7. Phase 5: Production
8. Verification
9. Troubleshooting

---

Prerequisites

Required Accounts

• GitHub account (you have this)
• Hetzner Cloud account (need to create)
• Supabase account (you have this)
• Azure OpenAI account (you have this)

Required Tools (Local Machine)

• Git
• Docker Desktop
• pnpm (v9.15.0)
• Node.js (v20+)
• SSH client
• Terminal/Command line

Required Knowledge

• Basic Docker understanding
• Basic GitHub Actions understanding
• SSH and server access
• Command line comfort

---

Quick Start (30 Minutes)

Goal: Get your first service deployed to staging

Step 1: Create Hetzner Account (5 minutes)

1. Go to https://console.hetzner.cloud/
2. Click "Sign Up"
3. Complete registration
4. Verify email
5. Add payment method (credit card or PayPal)
6. May require ID verification (be prepared to upload ID)

Step 2: Provision Server (10 minutes)

1. In Hetzner Console, click "New Project"

   • Name: manacore-staging

2. Click "Add Server"

   • Location: Falkenstein, Germany (or nearest to you)

   • Image: Ubuntu 22.04

   • Type: CCX32 (8 vCPU, 32 GB RAM, $50/month)

   • Networking: Public IPv4

   • SSH Key: Add your public SSH key

     # On your machine, generate if you don't have one:
     ssh-keygen -t ed25519 -C "your_email@example.com"
     
     # Copy public key:
     cat ~/.ssh/id_ed25519.pub
     # Paste into Hetzner
     

   • Name: staging-01

   • Click "Create & Buy now"

3. Wait 1-2 minutes for server to be created

4. Note the server IP address: ___________________

5. Test SSH connection:

   ssh root@YOUR_SERVER_IP
   # Type "yes" to accept fingerprint
   # You should be logged in!
   

6. Update system:

   apt update && apt upgrade -y
   

Step 3: Install Coolify (10 minutes)

1. On your server (via SSH), run:

   curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash
   

2. Wait 5-10 minutes for installation to complete

   • The script will install Docker, Coolify, and dependencies
   • You'll see progress messages

3. Once complete, access Coolify UI:

   https://YOUR_SERVER_IP:8000
   

4. Complete initial setup wizard:

   • Create admin account
   • Set email (for SSL certificates)
   • Configure basic settings

5. Save your Coolify credentials securely!

Step 4: Configure GitHub Secrets (5 minutes)

1. Go to your GitHub repo: https://github.com/wuesteon/manacore-monorepo

2. Go to Settings → Secrets and variables → Actions → New repository secret

3. Add these 5 essential secrets:

   Name: STAGING_HOST
   Value: YOUR_SERVER_IP
   

   Name: STAGING_USER
   Value: root
   

   Name: STAGING_SSH_KEY
   Value: (paste your PRIVATE SSH key)
   # Get it with: cat ~/.ssh/id_ed25519
   # Copy the ENTIRE content including -----BEGIN and -----END
   

   Name: STAGING_SUPABASE_URL
   Value: https://your-project.supabase.co
   

   Name: STAGING_SUPABASE_ANON_KEY
   Value: your-anon-key-here
   

Step 5: Test CI/CD Pipeline (5 minutes)

1. Create test branch:

   cd /Users/wuesteon/dev/mana_universe/manacore-monorepo
   git checkout -b test/cicd-setup
   

2. Make small change (add comment to README):

   echo "\n<!-- Testing CI/CD -->" >> README.md
   git add README.md
   git commit -m "test: verify CI/CD pipeline"
   git push origin test/cicd-setup
   

3. Create Pull Request on GitHub

4. Watch GitHub Actions:

   • Go to Actions tab
   • See "CI - Pull Request" workflow running
   • Verify it completes successfully (green checkmark)

5. Merge PR to main

6. Watch "CI - Main Branch" workflow:

   • Should build Docker image
   • Should push to ghcr.io
   • Check https://github.com/wuesteon?tab=packages

🎉 If you see the green checkmarks, your CI/CD pipeline is working!

---

Phase 1: Infrastructure Foundation (Day 1-2)

1.1 Add Remaining GitHub Secrets

Now that the basics work, add the complete set of secrets:

Staging Secrets (add these 5 more):

STAGING_SUPABASE_SERVICE_ROLE_KEY = your-service-role-key
STAGING_JWT_SECRET = (generate with: openssl rand -base64 64)
STAGING_MANA_SERVICE_URL = http://mana-core-auth:3001
STAGING_AZURE_OPENAI_ENDPOINT = your-azure-endpoint
STAGING_AZURE_OPENAI_API_KEY = your-azure-key

1.2 Create First Dockerfile

For mana-core-auth service:

1. Copy template:

   cp docker/templates/Dockerfile.nestjs services/mana-core-auth/Dockerfile
   

2. No changes needed! The template is already configured for NestJS services in the monorepo.

3. Test build locally:

   docker build -t test-auth -f services/mana-core-auth/Dockerfile .
   

   This will take 5-10 minutes the first time.

4. Test run locally:

   docker run -p 3001:3001 \
     -e SUPABASE_URL=your-url \
     -e SUPABASE_ANON_KEY=your-key \
     test-auth
   

5. Test health endpoint:

   curl http://localhost:3001/api/v1/health
   # Should return: {"status":"ok"}
   

6. If it works, commit and push:

   git add services/mana-core-auth/Dockerfile
   git commit -m "feat: add Dockerfile for mana-core-auth"
   git push
   

7. Watch GitHub Actions build the image and push to ghcr.io

1.3 Deploy to Staging

Option A: Manual Deployment (Recommended First Time)

1. SSH into your server:

   ssh root@YOUR_SERVER_IP
   

2. Create deployment directory:

   mkdir -p ~/manacore-staging
   cd ~/manacore-staging
   

3. Create docker-compose.yml:

   cat > docker-compose.yml << 'EOF'
   version: '3.8'
   
   services:
     mana-core-auth:
       image: ghcr.io/wuesteon/mana-core-auth:latest
       container_name: mana-core-auth
       ports:
         - "3001:3001"
       environment:
         - NODE_ENV=staging
         - PORT=3001
         - SUPABASE_URL=${SUPABASE_URL}
         - SUPABASE_ANON_KEY=${SUPABASE_ANON_KEY}
         - SUPABASE_SERVICE_ROLE_KEY=${SUPABASE_SERVICE_ROLE_KEY}
         - JWT_SECRET=${JWT_SECRET}
       restart: unless-stopped
       healthcheck:
         test: ["CMD", "wget", "-q", "--spider", "http://localhost:3001/api/v1/health"]
         interval: 30s
         timeout: 10s
         retries: 3
   EOF
   

4. Create .env file:

   cat > .env << 'EOF'
   SUPABASE_URL=your-supabase-url
   SUPABASE_ANON_KEY=your-anon-key
   SUPABASE_SERVICE_ROLE_KEY=your-service-role-key
   JWT_SECRET=your-jwt-secret
   EOF
   

   Replace the placeholder values with your actual credentials!

5. Login to GitHub Container Registry:

   # Create a Personal Access Token (PAT) on GitHub:
   # GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic)
   # Scope: read:packages
   
   echo YOUR_PAT | docker login ghcr.io -u wuesteon --password-stdin
   

6. Pull and start:

   docker compose pull
   docker compose up -d
   

7. Check status:

   docker compose ps
   docker compose logs mana-core-auth
   

8. Test health endpoint:

   curl http://localhost:3001/api/v1/health
   

9. Test externally (from your local machine):

   curl http://YOUR_SERVER_IP:3001/api/v1/health
   

Option B: Automated Deployment (After Manual Works)

1. Go to GitHub → Actions → "CD - Staging Deployment"
2. Click "Run workflow"
3. Select service: mana-core-auth
4. Click "Run workflow"
5. Watch the deployment progress

🎉 If you see healthy service, your first deployment is complete!

---

Phase 2: First Deployment (Day 1-2)

2.1 Deploy Remaining Backend Services

Repeat the Dockerfile creation for each backend:

# Chat backend
cp docker/templates/Dockerfile.nestjs apps/chat/apps/backend/Dockerfile

# Maerchenzauber backend
cp docker/templates/Dockerfile.nestjs apps/maerchenzauber/apps/backend/Dockerfile

# Manadeck backend
cp docker/templates/Dockerfile.nestjs apps/manadeck/apps/backend/Dockerfile

# Nutriphi backend
cp docker/templates/Dockerfile.nestjs apps/nutriphi/apps/backend/Dockerfile

# Wisekeep backend (if exists)
cp docker/templates/Dockerfile.nestjs apps/wisekeep/apps/backend/Dockerfile

# Quote backend (if exists)
cp docker/templates/Dockerfile.nestjs apps/quote/apps/backend/Dockerfile

Test each build locally before committing:

docker build -t test-service -f apps/PROJECT/apps/backend/Dockerfile .

Commit all at once:

git add apps/*/apps/backend/Dockerfile
git commit -m "feat: add Dockerfiles for all backend services"
git push

2.2 Update docker-compose.yml

On your server, update ~/manacore-staging/docker-compose.yml to include all services.

Example with 3 backends:

version: '3.8'

services:
  mana-core-auth:
    image: ghcr.io/wuesteon/mana-core-auth:latest
    container_name: mana-core-auth
    ports:
      - '3001:3001'
    environment:
      - NODE_ENV=staging
      - PORT=3001
      # ... env vars
    restart: unless-stopped

  chat-backend:
    image: ghcr.io/wuesteon/chat-backend:latest
    container_name: chat-backend
    ports:
      - '3002:3002'
    environment:
      - NODE_ENV=staging
      - PORT=3002
      # ... env vars
    depends_on:
      - mana-core-auth
    restart: unless-stopped

  maerchenzauber-backend:
    image: ghcr.io/wuesteon/maerchenzauber-backend:latest
    container_name: maerchenzauber-backend
    ports:
      - '3003:3003'
    environment:
      - NODE_ENV=staging
      - PORT=3003
      # ... env vars
    depends_on:
      - mana-core-auth
    restart: unless-stopped

Deploy all services:

cd ~/manacore-staging
docker compose pull
docker compose up -d
docker compose ps  # Should show all services running

---

Phase 3: Web Apps (Day 3-4)

3.1 Create SvelteKit Dockerfiles

# Copy template for each web app
cp docker/templates/Dockerfile.sveltekit apps/chat/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/maerchenzauber/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/manadeck/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/memoro/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/picture/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/wisekeep/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/quote/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/uload/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/manacore/apps/web/Dockerfile

Test one build:

docker build -t test-web -f apps/chat/apps/web/Dockerfile .
docker run -p 3000:3000 -e PUBLIC_SUPABASE_URL=your-url test-web
# Visit http://localhost:3000

3.2 Create Astro Dockerfiles

# Copy template for each landing page
cp docker/templates/Dockerfile.astro apps/chat/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/maerchenzauber/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/memoro/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/picture/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/wisekeep/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/quote/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/bauntown/Dockerfile

3.3 Configure Domains and SSL

In Coolify UI:

1. Add a new "Resource" → "Service"
2. For each web app/landing:
   • Set domain (e.g., chat.manacore.app)
   • Enable "Generate SSL"
   • Set Docker image: ghcr.io/wuesteon/chat-web:latest
   • Configure environment variables
   • Deploy

Or configure Nginx reverse proxy manually (see docs/DEPLOYMENT.md for details)

---

Phase 4: Testing (Day 5)

4.1 Set Up Test Configuration

1. Install test dependencies:

   pnpm install
   

2. The test configs in packages/test-config/ are ready to use.

3. Configure each project to use shared configs.

For NestJS backends, add to apps/PROJECT/apps/backend/package.json:

{
	"scripts": {
		"test": "jest",
		"test:cov": "jest --coverage"
	},
	"jest": {
		"preset": "@manacore/test-config/jest.config.backend.js"
	}
}

4.2 Write Critical Path Tests (100% Coverage)

Focus on @manacore/shared-auth package first:

cd packages/shared-auth
mkdir -p src/__tests__

# Write tests for:
# - Token generation
# - Token validation
# - Token refresh
# - JWT utilities
# - AuthService

# Run tests
pnpm test:cov

# Verify 100% coverage

Use test examples from docs/test-examples/ as reference.

4.3 Enable Coverage in CI

The test.yml workflow is already configured. Just ensure your tests are running:

# Test locally first
pnpm test

# Push and create PR
git add .
git commit -m "test: add auth package tests"
git push

GitHub Actions will automatically run tests and enforce coverage.

---

Phase 5: Production (Day 6-7)

5.1 Provision Production Server

Repeat the Hetzner setup, but:

• Project name: manacore-production
• Server type: CCX42 (16 vCPU, 64 GB RAM, $100/month)
  • Or CCX32 if resources sufficient
• Server name: production-01

5.2 Configure Production Secrets

Add these secrets to GitHub (with PRODUCTION_ prefix):

PRODUCTION_HOST
PRODUCTION_USER
PRODUCTION_SSH_KEY
PRODUCTION_SUPABASE_URL
PRODUCTION_SUPABASE_ANON_KEY
PRODUCTION_SUPABASE_SERVICE_ROLE_KEY
PRODUCTION_JWT_SECRET (different from staging!)
PRODUCTION_MANA_SERVICE_URL
PRODUCTION_AZURE_OPENAI_ENDPOINT
PRODUCTION_AZURE_OPENAI_API_KEY
PRODUCTION_REDIS_PASSWORD

5.3 Set Up GitHub Environments

1. Go to Settings → Environments → New environment
2. Create "production-approval" environment:
   • Add yourself as required reviewer
   • Add your colleague as required reviewer
3. Create "production" environment:
   • Deployment branches: main only

5.4 Deploy to Production

1. Go to Actions → "CD - Production Deployment"
2. Click "Run workflow"
3. Service: mana-core-auth
4. Environment: production
5. Confirmation: Type "deploy"
6. Click "Run workflow"
7. Approve when prompted
8. Watch deployment
9. Verify health checks

Repeat for all services!

---

Verification

Quick Health Check

Check all services:

# On server
cd ~/manacore-staging  # or ~/manacore-production
docker compose ps
docker compose logs --tail=50

# From local machine
curl http://YOUR_SERVER_IP:3001/api/v1/health  # mana-core-auth
curl http://YOUR_SERVER_IP:3002/api/health     # chat-backend
# etc...

Comprehensive Verification

1. All containers running:

   docker compose ps
   # All should show "Up" status
   

2. Health checks passing:

   for service in mana-core-auth chat-backend maerchenzauber-backend; do
     echo "Checking $service..."
     docker compose exec $service wget -q -O - http://localhost:3001/api/v1/health || echo "FAILED"
   done
   

3. Resource usage acceptable:

   docker stats --no-stream
   # CPU should be < 50%, Memory < 80%
   

4. Logs clean (no critical errors):

   docker compose logs --tail=100 | grep -i error
   

5. Web apps accessible:

   • Visit each domain in browser
   • Test basic functionality

---

Troubleshooting

Issue: Docker build fails

Symptom: "ERROR: failed to solve"

Solutions:

1. Check Dockerfile syntax
2. Ensure you're running from monorepo root
3. Check for missing dependencies in package.json
4. Try building with no cache: docker build --no-cache

See: docs/DOCKER_GUIDE.md section 6 for more

---

Issue: GitHub Actions fails

Symptom: Red X on PR, workflow fails

Solutions:

1. Check workflow logs in GitHub Actions tab
2. Verify all secrets are configured
3. Check if build works locally first
4. Ensure correct image names (ghcr.io/wuesteon/...)

See: docs/CI_CD_SETUP.md section 6 for more

---

Issue: Deployment fails with "permission denied"

Symptom: Can't connect to server via SSH in workflow

Solutions:

1. Verify STAGING_SSH_KEY secret contains private key
2. Ensure key includes -----BEGIN and -----END lines
3. Verify STAGING_USER is correct (usually root)
4. Test SSH manually: ssh root@SERVER_IP

---

Issue: Service unhealthy after deployment

Symptom: Health check endpoint returns 500 or times out

Solutions:

1. Check logs: docker compose logs service-name --tail=100
2. Verify environment variables are set correctly
3. Check if database connection works
4. Ensure port is correct
5. Try restarting: docker compose restart service-name

See: docs/DEPLOYMENT.md section 4 for more

---

Issue: Can't pull Docker images on server

Symptom: "unauthorized: unauthenticated"

Solutions:

1. Login to ghcr.io on server:

   echo YOUR_PAT | docker login ghcr.io -u wuesteon --password-stdin
   

2. Verify PAT has read:packages scope
3. Check image exists: https://github.com/wuesteon?tab=packages

See: DOCKER_REGISTRY_SETUP.md for details

---

Next Steps

After completing setup:

1. ✅ Review TODO.md and mark completed tasks
2. ✅ Update CHANGELOG.md with your progress
3. ✅ Train your colleague using this guide
4. ✅ Set up monitoring (Phase 6 in TODO.md)
5. ✅ Implement remaining tests (Phase 4 in TODO.md)
6. ✅ Optimize performance (caching, CDN)

---

Support

Stuck? Need help?

1. Check TROUBLESHOOTING.md (when created)
2. Review relevant documentation in docs/
3. Check GitHub Actions logs
4. Check Docker logs on server
5. Review Hive Mind Final Report: /HIVE_MIND_FINAL_REPORT.md

---

Last Updated: 2025-11-27 Status: Ready to use Estimated Time: 30 minutes (quick start) to 7 days (full implementation)