till/managarten

Fork 0

mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-14 21:41:09 +02:00

Wuesteon 2a002bf6be first auth impl

2025-12-01 13:30:58 +01:00

18 KiB

Raw Blame History

CI/CD Setup Guide

Last Updated: 2025-11-27 Estimated Time: 30 minutes (Quick Start) to 7 days (Full Implementation)

Prerequisites

Required Accounts

GitHub account (you have this)
Hetzner Cloud account (need to create)
Supabase account (you have this)
Azure OpenAI account (you have this)

Required Tools (Local Machine)

Git
Docker Desktop
pnpm (v9.15.0)
Node.js (v20+)
SSH client
Terminal/Command line

Required Knowledge

Basic Docker understanding
Basic GitHub Actions understanding
SSH and server access
Command line comfort

Quick Start (30 Minutes)

Goal: Get your first service deployed to staging

Step 1: Create Hetzner Account (5 minutes)

Go to https://console.hetzner.cloud/
Click "Sign Up"
Complete registration
Verify email
Add payment method (credit card or PayPal)
May require ID verification (be prepared to upload ID)

Step 2: Provision Server (10 minutes)

In Hetzner Console, click "New Project"
- Name: manacore-staging
Click "Add Server"
- Location: Falkenstein, Germany (or nearest to you)
- Image: Ubuntu 22.04
- Type: CCX32 (8 vCPU, 32 GB RAM, $50/month)
- Networking: Public IPv4
- SSH Key: Add your public SSH key
```
# On your machine, generate if you don't have one:
ssh-keygen -t ed25519 -C "your_email@example.com"

# Copy public key:
cat ~/.ssh/id_ed25519.pub
# Paste into Hetzner
```
- Name: staging-01
- Click "Create & Buy now"
Wait 1-2 minutes for server to be created
Note the server IP address: ___________________

Test SSH connection:

ssh root@YOUR_SERVER_IP
# Type "yes" to accept fingerprint
# You should be logged in!

Update system:
```
apt update && apt upgrade -y
```

Step 3: Set up Docker Compose (10 minutes)

On your server (via SSH), run:

curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash

Wait 5-10 minutes for installation to complete
- The script will install Docker, Coolify, and dependencies
- You'll see progress messages
Once complete, access Docker Compose configuration:
```
https://YOUR_SERVER_IP:8000
```
Complete initial setup wizard:
- Create admin account
- Set email (for SSL certificates)
- Configure basic settings
Save your Coolify credentials securely!

Step 4: Configure GitHub Secrets (5 minutes)

Go to your GitHub repo: https://github.com/wuesteon/manacore-monorepo
Go to Settings → Secrets and variables → Actions → New repository secret

Add these 5 essential secrets:

Name: STAGING_HOST
Value: YOUR_SERVER_IP

Name: STAGING_USER
Value: root

Name: STAGING_SSH_KEY
Value: (paste your PRIVATE SSH key)
# Get it with: cat ~/.ssh/id_ed25519
# Copy the ENTIRE content including -----BEGIN and -----END

Name: STAGING_SUPABASE_URL
Value: https://your-project.supabase.co

Name: STAGING_SUPABASE_ANON_KEY
Value: your-anon-key-here

Step 5: Test CI/CD Pipeline (5 minutes)

Create test branch:

cd /Users/wuesteon/dev/mana_universe/manacore-monorepo
git checkout -b test/cicd-setup

Make small change (add comment to README):

echo "\n<!-- Testing CI/CD -->" >> README.md
git add README.md
git commit -m "test: verify CI/CD pipeline"
git push origin test/cicd-setup

Create Pull Request on GitHub
Watch GitHub Actions:
- Go to Actions tab
- See "CI - Pull Request" workflow running
- Verify it completes successfully (green checkmark)
Merge PR to main
Watch "CI - Main Branch" workflow:
- Should build Docker image
- Should push to ghcr.io
- Check https://github.com/wuesteon?tab=packages

🎉 If you see the green checkmarks, your CI/CD pipeline is working!

Phase 1: Infrastructure Foundation (Day 1-2)

1.1 Add Remaining GitHub Secrets

Now that the basics work, add the complete set of secrets:

Staging Secrets (add these 5 more):

STAGING_SUPABASE_SERVICE_ROLE_KEY = your-service-role-key
STAGING_JWT_SECRET = (generate with: openssl rand -base64 64)
STAGING_MANA_SERVICE_URL = http://mana-core-auth:3001
STAGING_AZURE_OPENAI_ENDPOINT = your-azure-endpoint
STAGING_AZURE_OPENAI_API_KEY = your-azure-key

1.2 Create First Dockerfile

For mana-core-auth service:

Copy template:

cp docker/templates/Dockerfile.nestjs services/mana-core-auth/Dockerfile

No changes needed! The template is already configured for NestJS services in the monorepo.

Test build locally:

docker build -t test-auth -f services/mana-core-auth/Dockerfile .

This will take 5-10 minutes the first time.

Test run locally:

docker run -p 3001:3001 \
  -e SUPABASE_URL=your-url \
  -e SUPABASE_ANON_KEY=your-key \
  test-auth

Test health endpoint:

curl http://localhost:3001/api/v1/health
# Should return: {"status":"ok"}

If it works, commit and push:

git add services/mana-core-auth/Dockerfile
git commit -m "feat: add Dockerfile for mana-core-auth"
git push

Watch GitHub Actions build the image and push to ghcr.io

1.3 Deploy to Staging

Option A: Manual Deployment (Recommended First Time)

SSH into your server:
```
ssh root@YOUR_SERVER_IP
```

Create deployment directory:

mkdir -p ~/manacore-staging
cd ~/manacore-staging

Create docker-compose.yml:

cat > docker-compose.yml << 'EOF'
version: '3.8'

services:
  mana-core-auth:
    image: ghcr.io/wuesteon/mana-core-auth:latest
    container_name: mana-core-auth
    ports:
      - "3001:3001"
    environment:
      - NODE_ENV=staging
      - PORT=3001
      - SUPABASE_URL=${SUPABASE_URL}
      - SUPABASE_ANON_KEY=${SUPABASE_ANON_KEY}
      - SUPABASE_SERVICE_ROLE_KEY=${SUPABASE_SERVICE_ROLE_KEY}
      - JWT_SECRET=${JWT_SECRET}
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "http://localhost:3001/api/v1/health"]
      interval: 30s
      timeout: 10s
      retries: 3
EOF

Create .env file:

cat > .env << 'EOF'
SUPABASE_URL=your-supabase-url
SUPABASE_ANON_KEY=your-anon-key
SUPABASE_SERVICE_ROLE_KEY=your-service-role-key
JWT_SECRET=your-jwt-secret
EOF

Replace the placeholder values with your actual credentials!

# Create a Personal Access Token (PAT) on GitHub:
# GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic)
# Scope: read:packages

echo YOUR_PAT | docker login ghcr.io -u wuesteon --password-stdin

Pull and start:

docker compose pull
docker compose up -d

Check status:

docker compose ps
docker compose logs mana-core-auth

Test health endpoint:

curl http://localhost:3001/api/v1/health

Test externally (from your local machine):

curl http://YOUR_SERVER_IP:3001/api/v1/health

Option B: Automated Deployment (After Manual Works)

Go to GitHub → Actions → "CD - Staging Deployment"
Click "Run workflow"
Select service: mana-core-auth
Click "Run workflow"
Watch the deployment progress

🎉 If you see healthy service, your first deployment is complete!

Phase 2: First Deployment (Day 1-2)

2.1 Deploy Remaining Backend Services

Repeat the Dockerfile creation for each backend:

# Chat backend
cp docker/templates/Dockerfile.nestjs apps/chat/apps/backend/Dockerfile

# Maerchenzauber backend
cp docker/templates/Dockerfile.nestjs apps/maerchenzauber/apps/backend/Dockerfile

# Manadeck backend
cp docker/templates/Dockerfile.nestjs apps/manadeck/apps/backend/Dockerfile

# Nutriphi backend
cp docker/templates/Dockerfile.nestjs apps/nutriphi/apps/backend/Dockerfile

# Wisekeep backend (if exists)
cp docker/templates/Dockerfile.nestjs apps/wisekeep/apps/backend/Dockerfile

# Quote backend (if exists)
cp docker/templates/Dockerfile.nestjs apps/quote/apps/backend/Dockerfile

Test each build locally before committing:

docker build -t test-service -f apps/PROJECT/apps/backend/Dockerfile .

Commit all at once:

git add apps/*/apps/backend/Dockerfile
git commit -m "feat: add Dockerfiles for all backend services"
git push

2.2 Update docker-compose.yml

On your server, update ~/manacore-staging/docker-compose.yml to include all services.

Example with 3 backends:

version: '3.8'

services:
  mana-core-auth:
    image: ghcr.io/wuesteon/mana-core-auth:latest
    container_name: mana-core-auth
    ports:
      - '3001:3001'
    environment:
      - NODE_ENV=staging
      - PORT=3001
      # ... env vars
    restart: unless-stopped

  chat-backend:
    image: ghcr.io/wuesteon/chat-backend:latest
    container_name: chat-backend
    ports:
      - '3002:3002'
    environment:
      - NODE_ENV=staging
      - PORT=3002
      # ... env vars
    depends_on:
      - mana-core-auth
    restart: unless-stopped

  maerchenzauber-backend:
    image: ghcr.io/wuesteon/maerchenzauber-backend:latest
    container_name: maerchenzauber-backend
    ports:
      - '3003:3003'
    environment:
      - NODE_ENV=staging
      - PORT=3003
      # ... env vars
    depends_on:
      - mana-core-auth
    restart: unless-stopped

Deploy all services:

cd ~/manacore-staging
docker compose pull
docker compose up -d
docker compose ps  # Should show all services running

Phase 3: Web Apps (Day 3-4)

3.1 Create SvelteKit Dockerfiles

# Copy template for each web app
cp docker/templates/Dockerfile.sveltekit apps/chat/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/maerchenzauber/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/manadeck/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/memoro/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/picture/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/wisekeep/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/quote/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/uload/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/manacore/apps/web/Dockerfile

Test one build:

docker build -t test-web -f apps/chat/apps/web/Dockerfile .
docker run -p 3000:3000 -e PUBLIC_SUPABASE_URL=your-url test-web
# Visit http://localhost:3000

3.2 Create Astro Dockerfiles

# Copy template for each landing page
cp docker/templates/Dockerfile.astro apps/chat/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/maerchenzauber/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/memoro/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/picture/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/wisekeep/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/quote/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/bauntown/Dockerfile

3.3 Configure Domains and SSL

In Docker Compose configuration:

Add a new "Resource" → "Service"
For each web app/landing:
- Set domain (e.g., chat.manacore.app)
- Enable "Generate SSL"
- Set Docker image: ghcr.io/wuesteon/chat-web:latest
- Configure environment variables
- Deploy

Or configure Nginx reverse proxy manually (see docs/DEPLOYMENT.md for details)

Phase 4: Testing (Day 5)

4.1 Set Up Test Configuration

Install test dependencies:
```
pnpm install
```
The test configs in packages/test-config/ are ready to use.
Configure each project to use shared configs.

For NestJS backends, add to apps/PROJECT/apps/backend/package.json:

{
	"scripts": {
		"test": "jest",
		"test:cov": "jest --coverage"
	},
	"jest": {
		"preset": "@manacore/test-config/jest.config.backend.js"
	}
}

4.2 Write Critical Path Tests (100% Coverage)

Focus on @manacore/shared-auth package first:

cd packages/shared-auth
mkdir -p src/__tests__

# Write tests for:
# - Token generation
# - Token validation
# - Token refresh
# - JWT utilities
# - AuthService

# Run tests
pnpm test:cov

# Verify 100% coverage

Use test examples from docs/test-examples/ as reference.

4.3 Enable Coverage in CI

The test.yml workflow is already configured. Just ensure your tests are running:

# Test locally first
pnpm test

# Push and create PR
git add .
git commit -m "test: add auth package tests"
git push

GitHub Actions will automatically run tests and enforce coverage.

Phase 5: Production (Day 6-7)

5.1 Provision Production Server

Repeat the Hetzner setup, but:

Project name: manacore-production
Server type: CCX42 (16 vCPU, 64 GB RAM, $100/month)
- Or CCX32 if resources sufficient
Server name: production-01

5.2 Configure Production Secrets

Add these secrets to GitHub (with PRODUCTION_ prefix):

PRODUCTION_HOST
PRODUCTION_USER
PRODUCTION_SSH_KEY
PRODUCTION_SUPABASE_URL
PRODUCTION_SUPABASE_ANON_KEY
PRODUCTION_SUPABASE_SERVICE_ROLE_KEY
PRODUCTION_JWT_SECRET (different from staging!)
PRODUCTION_MANA_SERVICE_URL
PRODUCTION_AZURE_OPENAI_ENDPOINT
PRODUCTION_AZURE_OPENAI_API_KEY
PRODUCTION_REDIS_PASSWORD

5.3 Set Up GitHub Environments

Go to Settings → Environments → New environment
Create "production-approval" environment:
- Add yourself as required reviewer
- Add your colleague as required reviewer
Create "production" environment:
- Deployment branches: main only

5.4 Deploy to Production

Go to Actions → "CD - Production Deployment"
Click "Run workflow"
Service: mana-core-auth
Environment: production
Confirmation: Type "deploy"
Click "Run workflow"
Approve when prompted
Watch deployment
Verify health checks

Repeat for all services!

Verification

Quick Health Check

Check all services:

# On server
cd ~/manacore-staging  # or ~/manacore-production
docker compose ps
docker compose logs --tail=50

# From local machine
curl http://YOUR_SERVER_IP:3001/api/v1/health  # mana-core-auth
curl http://YOUR_SERVER_IP:3002/api/health     # chat-backend
# etc...

Comprehensive Verification

All containers running:

docker compose ps
# All should show "Up" status

Health checks passing:

for service in mana-core-auth chat-backend maerchenzauber-backend; do
  echo "Checking $service..."
  docker compose exec $service wget -q -O - http://localhost:3001/api/v1/health || echo "FAILED"
done

Resource usage acceptable:

docker stats --no-stream
# CPU should be < 50%, Memory < 80%

Logs clean (no critical errors):

docker compose logs --tail=100 | grep -i error

Web apps accessible:
- Visit each domain in browser
- Test basic functionality

Troubleshooting

Issue: Docker build fails

Symptom: "ERROR: failed to solve"

Solutions:

Check Dockerfile syntax
Ensure you're running from monorepo root
Check for missing dependencies in package.json
Try building with no cache: docker build --no-cache

See: docs/DOCKER_GUIDE.md section 6 for more

Issue: GitHub Actions fails

Symptom: Red X on PR, workflow fails

Solutions:

Check workflow logs in GitHub Actions tab
Verify all secrets are configured
Check if build works locally first
Ensure correct image names (ghcr.io/wuesteon/...)

See: docs/CI_CD_SETUP.md section 6 for more

Issue: Deployment fails with "permission denied"

Symptom: Can't connect to server via SSH in workflow

Solutions:

Verify STAGING_SSH_KEY secret contains private key
Ensure key includes -----BEGIN and -----END lines
Verify STAGING_USER is correct (usually root)
Test SSH manually: ssh root@SERVER_IP

Issue: Service unhealthy after deployment

Symptom: Health check endpoint returns 500 or times out

Solutions:

Check logs: docker compose logs service-name --tail=100
Verify environment variables are set correctly
Check if database connection works
Ensure port is correct
Try restarting: docker compose restart service-name

See: docs/DEPLOYMENT.md section 4 for more

Issue: Can't pull Docker images on server

Symptom: "unauthorized: unauthenticated"

Solutions:

echo YOUR_PAT | docker login ghcr.io -u wuesteon --password-stdin

Verify PAT has read:packages scope
Check image exists: https://github.com/wuesteon?tab=packages

See: DOCKER_REGISTRY_SETUP.md for details

Next Steps

After completing setup:

✅ Review TODO.md and mark completed tasks
✅ Update CHANGELOG.md with your progress
✅ Train your colleague using this guide
✅ Set up monitoring (Phase 6 in TODO.md)
✅ Implement remaining tests (Phase 4 in TODO.md)
✅ Optimize performance (caching, CDN)

Support

Stuck? Need help?

Check TROUBLESHOOTING.md (when created)
Review relevant documentation in docs/
Check GitHub Actions logs
Check Docker logs on server
Review Hive Mind Final Report: /HIVE_MIND_FINAL_REPORT.md

Last Updated: 2025-11-27 Status: Ready to use Estimated Time: 30 minutes (quick start) to 7 days (full implementation)

18 KiB Raw Blame History

CI/CD Setup Guide

📋 Table of Contents

Prerequisites

Required Accounts

Required Tools (Local Machine)

Required Knowledge

Quick Start (30 Minutes)

Step 1: Create Hetzner Account (5 minutes)

Step 2: Provision Server (10 minutes)

Step 3: Set up Docker Compose (10 minutes)

Step 4: Configure GitHub Secrets (5 minutes)

Step 5: Test CI/CD Pipeline (5 minutes)

Phase 1: Infrastructure Foundation (Day 1-2)

1.1 Add Remaining GitHub Secrets

1.2 Create First Dockerfile

1.3 Deploy to Staging

Phase 2: First Deployment (Day 1-2)

2.1 Deploy Remaining Backend Services

2.2 Update docker-compose.yml

Phase 3: Web Apps (Day 3-4)

3.1 Create SvelteKit Dockerfiles

3.2 Create Astro Dockerfiles

3.3 Configure Domains and SSL

Phase 4: Testing (Day 5)

4.1 Set Up Test Configuration

4.2 Write Critical Path Tests (100% Coverage)

4.3 Enable Coverage in CI

Phase 5: Production (Day 6-7)

5.1 Provision Production Server

5.2 Configure Production Secrets

5.3 Set Up GitHub Environments

5.4 Deploy to Production

Verification

Quick Health Check

Comprehensive Verification

Troubleshooting

Issue: Docker build fails

Issue: GitHub Actions fails

Issue: Deployment fails with "permission denied"

Issue: Service unhealthy after deployment

Issue: Can't pull Docker images on server

Next Steps

Support

18 KiB

Raw Blame History