till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-14 17:41:09 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

chore: remove Hetzner legacy artifacts and update docs for Mac Mini self-hosting

Browse source 
Deleted files:
- docker/caddy/Caddyfile.production + Caddyfile.staging (Hetzner reverse proxy configs)
- scripts/deploy/ (deploy-hetzner.sh, build-and-push.sh, health-check.sh, migrate-db.sh, rollback.sh)
- scripts/generate-staging-secrets.sh
- cicd/ directory (11 Hetzner CI/CD planning docs)
- CI_CD_IMPLEMENTATION_SUMMARY.md, CI_CD_README.md, FILES_CREATED.md, HIVE_MIND_FINAL_REPORT.md

Updated docs:
- CLAUDE.md: Remove Hetzner Object Storage references, update to MinIO
- docs/ANALYTICS.md: Cloudflare Tunnel instead of Caddy
- docs/URL_SCHEMA.md: Mac Mini + Cloudflare Tunnel instead of Hetzner IP
- .env.development: Remove "Hetzner in production" comments

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS 2026-03-23 10:12:24 +01:00
parent 94fead0873
commit cc5ba3bb90
27 changed files with 19 additions and 8863 deletions
Download patch file Download diff file Expand all files Collapse all files

11
.env.development
View file

@ -206,7 +206,7 @@ PICTURE_DATABASE_URL=postgresql://manacore:devpassword@localhost:5432/picture
# Replicate API Token for AI image generation
PICTURE_REPLICATE_API_TOKEN=r8_QlvkstNhIc6NBX1ktpQ6ibvzOE2d2UQ1Emamd
# Storage Configuration (uses MinIO locally, Hetzner in production)
# Storage Configuration (uses MinIO locally)
# Uses shared S3_* variables from above - no project-specific override needed for local dev
PICTURE_STORAGE_PUBLIC_URL=http://localhost:9000/picture-storage
@ -229,7 +229,7 @@ NUTRIPHI_APP_ID=nutriphi
# Google Gemini API for food image analysis
GEMINI_API_KEY=AIzaSyBR9iP74hlo-mhI-Cl4QEvKprRzPPMb-GA
# S3 Storage (uses MinIO locally via shared S3_* variables, Hetzner in production)
# S3 Storage (uses MinIO locally via shared S3_* variables)
NUTRIPHI_S3_PUBLIC_URL=http://localhost:9000/nutriphi-storage
# ============================================
@ -428,6 +428,13 @@ MUKKE_DATABASE_URL=postgresql://manacore:devpassword@localhost:5432/mukke
WORLDREAM_SUPABASE_URL=https://gbsrekoykkesullxdvbd.supabase.co
WORLDREAM_SUPABASE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZSIsInJlZiI6Imdic3Jla295a2tlc3VsbHhkdmJkIiwicm9sZSI6ImFub24iLCJpYXQiOjE3NTY1MTU3NzksImV4cCI6MjA3MjA5MTc3OX0.qQlZvHiB56oKTRD90fd8IasZeZELjXOA46f-hnOQA1g
# ============================================
# CITYCORNERS PROJECT
# ============================================
CITYCORNERS_BACKEND_PORT=3025
CITYCORNERS_DATABASE_URL=postgresql://manacore:devpassword@localhost:5432/citycorners
CITYCORNERS_WEB_PORT=5196
WORLDREAM_OPENAI_API_KEY=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
WORLDREAM_GEMINI_API_KEY=AIzaSyB74aUj1KmJlcjNyT5uUiyDODQ6iYoAOjQ
WORLDREAM_REPLICATE_API_TOKEN=r8_QlvkstNhIc6NBX1ktpQ6ibvzOE2d2UQ1Emamd

380
CI_CD_IMPLEMENTATION_SUMMARY.md
View file

@ -1,380 +0,0 @@
# CI/CD Implementation Summary
## Mission Complete ✅
I have successfully implemented a complete CI/CD pipeline for the manacore-monorepo.
## What Was Delivered
### 1. GitHub Actions Workflows (6 workflows)
| Workflow | File | Purpose | Trigger |
| --------------------- | ----------------------- | ------------------------------ | ----------------- |
| PR Validation | `ci-pull-request.yml` | Lint, type-check, build, test | Pull requests |
| Main Branch CI | `ci-main.yml` | Build images, push to registry | Push to main |
| Staging Deployment | `cd-staging.yml` | Auto-deploy to staging | After main CI |
| Production Deployment | `cd-production.yml` | Manual production deploy | Manual + approval |
| Test Coverage | `test-coverage.yml` | Track code coverage | PRs, main, weekly |
| Dependency Updates | `dependency-update.yml` | Automated dependency checks | Weekly |
**Total Lines of Code**: ~1,500 lines of production-ready YAML
### 2. Docker Infrastructure
#### Templates (3 files)
- `docker/templates/Dockerfile.nestjs` - NestJS backend template
- `docker/templates/Dockerfile.sveltekit` - SvelteKit web template
- `docker/templates/Dockerfile.astro` - Astro landing page template
#### Orchestration (2 files)
- `docker-compose.staging.yml` - Full staging environment
- `docker-compose.production.yml` - Production configuration
#### Configuration (2 files)
- `docker/nginx/astro.conf` - Nginx configuration
- `.dockerignore` - Build optimization
**Features**:
- Multi-stage builds for minimal image sizes
- Non-root users for security
- Health checks for monitoring
- Resource limits for stability
- Automated backups
### 3. Deployment Scripts (5 scripts)
All scripts in `scripts/deploy/`:
| Script | Purpose | Features |
| ------------------- | ---------------------------- | -------------------------------------------------- |
| `build-and-push.sh` | Build and push Docker images | Error handling, colored output, progress tracking |
| `deploy-hetzner.sh` | Deploy to Hetzner/Coolify | Zero-downtime, health checks, rollback on failure |
| `health-check.sh` | Verify service health | Multiple endpoints, timeout handling |
| `rollback.sh` | Emergency rollback | Automated backup restoration, confirmation prompts |
| `migrate-db.sh` | Run database migrations | Supabase + Drizzle support, safe execution |
**Total Lines of Code**: ~800 lines of production-ready bash
### 4. Testing Infrastructure (3 config files)
- `vitest.config.ts` - Modern unit testing with Vitest
- `jest.config.js` - Multi-project testing (backend, mobile, shared)
- `playwright.config.ts` - E2E testing with Playwright
- `tests/e2e/example.spec.ts` - Example E2E test suite
**Coverage Features**:
- 50% minimum coverage threshold
- HTML, JSON, and LCOV reports
- Codecov integration
- Multi-project support
### 5. Comprehensive Documentation (4 documents)
| Document | Pages | Topics Covered |
| ---------------------- | ----- | ----------------------------------------------------------- |
| `docs/DEPLOYMENT.md` | 25+ | Full deployment guide, troubleshooting, rollback procedures |
| `docs/CI_CD_SETUP.md` | 20+ | Step-by-step setup, secrets configuration, server setup |
| `docs/DOCKER_GUIDE.md` | 18+ | Docker best practices, troubleshooting, advanced topics |
| `CI_CD_README.md` | 8+ | Quick start, architecture overview, project structure |
**Total Documentation**: 70+ pages of detailed guides
### 6. Additional Configuration
- `.github/dependabot.yml` - Automated dependency updates
- `CI_CD_IMPLEMENTATION_SUMMARY.md` - This file
## Key Features
### Smart Build Detection ✨
Only builds changed projects using Turborepo filters:
```yaml
# Detects changes in specific projects
maerchenzauber: 'apps/maerchenzauber/**'
chat: 'apps/chat/**'
# Only builds affected projects - saves time and resources
```
### Zero-Downtime Deployments 🚀
Rolling update strategy:
```bash
docker compose up -d --scale service=2 # Scale up
sleep 15 # Wait for health
docker compose up -d --scale service=1 # Scale down old
```
### Comprehensive Health Checks 💚
Every service monitored:
- Mana Core Auth: `/api/v1/health`
- Backend services: `/health` or `/api/health`
- Web apps: `/` (root)
- Automated checks after every deployment
### Automated Backups 💾
Production deployments create backups:
- PostgreSQL database dumps
- Docker compose configurations
- Environment files
- Current image tags
- Stored with timestamp
### Security Features 🔒
- Dependency scanning (Dependabot)
- Security audits (weekly)
- Non-root Docker users
- SSH key rotation guidance
- Secret management best practices
## Architecture Overview
```
Pull Request → PR Validation → Merge
↓
Main CI
↓
Build & Push Images
↓
Staging Deployment (Auto)
↓
Manual Approval
↓
Production Deployment
↓
Monitoring & Alerts
```
## Services Covered
The pipeline handles deployment for 6 backend services:
1. **mana-core-auth** (Port 3001) - Authentication service
2. **maerchenzauber-backend** (Port 3002) - Story generation
3. **chat-backend** (Port 3002) - AI chat service
4. **manadeck-backend** (Port 3003) - Card management
5. **nutriphi-backend** (Port 3004) - Nutrition tracking
6. **news-api** (Port 3005) - News aggregation
Plus infrastructure services:
- PostgreSQL database
- Redis cache
- Nginx reverse proxy
## File Count Summary
| Category | Files | Lines of Code |
| ------------------------ | ------ | ---------------- |
| GitHub Actions Workflows | 6 | ~1,500 |
| Docker Templates | 3 | ~300 |
| Docker Compose | 2 | ~400 |
| Deployment Scripts | 5 | ~800 |
| Test Configurations | 4 | ~400 |
| Documentation | 4 | 70+ pages |
| Configuration Files | 3 | ~100 |
| **Total** | **27** | **~3,500 lines** |
## Testing Status
### Workflows Tested
- ✅ Syntax validation (all YAML files)
- ✅ Script execution permissions
- ✅ Documentation completeness
- ⏳ Pending: Live GitHub Actions execution (requires secrets)
- ⏳ Pending: Live deployment (requires server setup)
### Ready for Testing
All workflows are production-ready and can be tested immediately after:
1. Configuring GitHub secrets
2. Setting up deployment servers
3. Adding SSH keys
## Next Steps
### For Implementation Team
1. **Review Documentation**
- Start with `CI_CD_README.md`
- Read `docs/CI_CD_SETUP.md` for setup
- Reference `docs/DEPLOYMENT.md` for operations
2. **Configure Secrets**
- Follow checklist in `docs/CI_CD_SETUP.md#github-secrets`
- ~22 secrets required (11 for staging, 11 for production)
- Generate SSH keys and JWT tokens
3. **Set Up Servers**
- Follow `docs/CI_CD_SETUP.md#deployment-servers`
- Install Docker and Docker Compose
- Configure SSH access
- Set up firewall rules
4. **Test Pipeline**
- Create test PR
- Verify PR validation workflow
- Merge to main
- Monitor staging deployment
- Test production deployment
5. **Set Up Monitoring**
- Configure external uptime monitoring
- Set up error tracking (Sentry)
- Configure log aggregation
- Set up alerts
### Recommended Timeline
| Phase | Duration | Tasks |
| ---------------------- | ------------ | --------------------------------- |
| Phase 1: Setup | 1-2 days | Configure secrets, set up servers |
| Phase 2: Testing | 2-3 days | Test workflows, fix any issues |
| Phase 3: Documentation | 1 day | Train team, create runbooks |
| Phase 4: Go-live | 1 day | First production deployment |
| **Total** | **5-7 days** | From zero to production |
## Cost Estimates
### GitHub Actions
- Free tier: 2,000 minutes/month
- Estimated usage: 500-800 minutes/month
- **Cost**: $0/month (within free tier)
### Docker Registry
- Docker Hub free tier: 1 org, unlimited public repos
- Estimated storage: 10-15GB
- **Cost**: $0/month (or $5/month for private repos)
### Servers (Hetzner)
- Staging: CX21 (2 vCPU, 4GB RAM) - €5.83/month
- Production: CX31 (4 vCPU, 8GB RAM) - €11.66/month
- **Total**: ~€17.49/month (~$19/month)
### Optional Services
- Codecov: Free for open source
- Sentry: Free tier (5K events/month)
- UptimeRobot: Free tier (50 monitors)
- **Cost**: $0/month (within free tiers)
**Total Estimated Cost**: $19-24/month
## Quality Metrics
### Code Quality
- ✅ Automated linting
- ✅ Type checking
- ✅ Format validation
- ✅ Security scanning
- ✅ 50% test coverage minimum
### Deployment Quality
- ✅ Zero-downtime deployments
- ✅ Automated health checks
- ✅ Rollback procedures
- ✅ Pre-deployment backups
- ✅ Extended monitoring
### Documentation Quality
- ✅ 70+ pages of guides
- ✅ Step-by-step instructions
- ✅ Troubleshooting sections
- ✅ Best practices
- ✅ Architecture diagrams
## Success Criteria
### ✅ Completed
- [x] PR validation workflow
- [x] Main branch CI workflow
- [x] Staging deployment automation
- [x] Production deployment workflow
- [x] Test coverage tracking
- [x] Dependency update automation
- [x] Docker templates for all service types
- [x] Production-ready docker-compose files
- [x] Deployment automation scripts
- [x] Health check automation
- [x] Rollback procedures
- [x] Database migration scripts
- [x] Test infrastructure
- [x] Comprehensive documentation
### ⏳ Pending (Requires User Action)
- [ ] GitHub secrets configuration
- [ ] Deployment server setup
- [ ] SSH key generation and distribution
- [ ] First staging deployment test
- [ ] First production deployment test
- [ ] External monitoring setup
- [ ] Team training
## Support
For questions or issues during implementation:
1. **Check Documentation First**
- `CI_CD_README.md` - Quick reference
- `docs/CI_CD_SETUP.md` - Setup guide
- `docs/DEPLOYMENT.md` - Operations guide
- `docs/DOCKER_GUIDE.md` - Docker reference
2. **Review Examples**
- Existing Dockerfiles in `apps/*/apps/backend/`
- Test files in `tests/e2e/`
- Deployment scripts in `scripts/deploy/`
3. **Common Issues**
- Check GitHub Actions logs
- Verify secrets are set correctly
- Test SSH access manually
- Review service logs
## Conclusion
The CI/CD pipeline is complete and production-ready. All code has been written with:
- ✅ Error handling
- ✅ Logging and progress tracking
- ✅ Safety checks and confirmations
- ✅ Comprehensive health checks
- ✅ Automated rollback procedures
- ✅ Security best practices
- ✅ Detailed documentation
The implementation follows industry best practices and is ready for immediate use after completing the setup steps outlined in the documentation.
**Total Development Time**: Complete CI/CD infrastructure in one session
**Total Files Created**: 27 production-ready files
**Total Code Written**: ~3,500 lines
**Documentation Pages**: 70+ pages
**Ready for Production**: Yes ✅
---
**Implementation Date**: 2025-01-27
**Implemented By**: Claude (CODER Agent)
**Status**: Complete and Ready for Deployment

476
CI_CD_README.md
View file

@ -1,476 +0,0 @@
# CI/CD Pipeline Implementation
Complete CI/CD pipeline for the manacore-monorepo with automated testing, building, and deployment.
## What's Included
### GitHub Actions Workflows (6 workflows)
1. **PR Validation** (`.github/workflows/ci-pull-request.yml`)
- Detects changed projects
- Runs lint, format, type-check
- Builds affected projects
- Runs tests with coverage
- Docker build validation
- Security scanning
2. **Main Branch CI** (`.github/workflows/ci-main.yml`)
- Full validation on merge to main
- Builds and pushes Docker images
- Triggers staging deployment
3. **Staging Deployment** (`.github/workflows/cd-staging.yml`)
- Automated deployment to staging
- Zero-downtime rolling updates
- Health checks
- Database migrations
4. **Production Deployment** (`.github/workflows/cd-production.yml`)
- Manual trigger with approval gates
- Pre-deployment backups
- Rolling updates
- Extended monitoring
- Smoke tests
5. **Test Coverage** (`.github/workflows/test-coverage.yml`)
- Runs on PRs and schedule
- Uploads to Codecov
- Enforces 50% minimum coverage
- Generates reports
6. **Dependency Updates** (`.github/workflows/dependency-update.yml`)
- Weekly automated checks
- Security audits
- Creates issues for vulnerabilities
- Updates lock files
### Docker Infrastructure
- **Templates**: Ready-to-use Dockerfiles for NestJS, SvelteKit, and Astro
- **Multi-stage builds**: Optimized for production
- **Security**: Non-root users, health checks, resource limits
- **docker-compose.staging.yml**: Full staging environment
- **docker-compose.production.yml**: Production configuration
### Deployment Scripts
Located in `scripts/deploy/`:
1. **build-and-push.sh**: Build and push Docker images
2. **deploy-hetzner.sh**: Deploy to Hetzner/Hetzner VPSs
3. **health-check.sh**: Verify service health
4. **rollback.sh**: Emergency rollback procedures
5. **migrate-db.sh**: Database migration runner
All scripts include error handling, logging, and safety checks.
### Testing Infrastructure
- **vitest.config.ts**: Unit test configuration
- **jest.config.js**: Multi-project test setup (backend, mobile, shared)
- **playwright.config.ts**: E2E test configuration
- **tests/e2e/**: Example E2E tests
### Documentation
- **docs/DEPLOYMENT.md**: Complete deployment guide (20+ pages)
- **docs/CI_CD_SETUP.md**: Step-by-step setup instructions
- **docs/DOCKER_GUIDE.md**: Docker best practices and troubleshooting
### Configuration Files
- **.dockerignore**: Optimized Docker build context
- **.github/dependabot.yml**: Automated dependency updates
- **docker/nginx/**: Nginx configurations
- **docker/templates/**: Dockerfile templates
## Quick Start
### 1. Set Up GitHub Secrets
Follow the checklist in `docs/CI_CD_SETUP.md#github-secrets`:
```bash
# Required secrets (22 minimum for staging + production)
- DOCKER_USERNAME
- DOCKER_PASSWORD
- STAGING_HOST
- STAGING_USER
- STAGING_SSH_KEY
- PRODUCTION_HOST
- PRODUCTION_USER
- PRODUCTION_SSH_KEY
# ... and more
```
### 2. Configure Deployment Server
```bash
# On your server
sudo adduser deploy
sudo usermod -aG docker deploy
curl -fsSL https://get.docker.com | sh
# Add SSH key
mkdir -p ~/.ssh
echo "ssh-ed25519 YOUR_PUBLIC_KEY" >> ~/.ssh/authorized_keys
# Create deployment directory
mkdir -p ~/manacore-staging
```
### 3. Test the Pipeline
```bash
# Create test PR
git checkout -b test/ci-pipeline
git push origin test/ci-pipeline
# Watch GitHub Actions tab
# All checks should pass ✅
```
### 4. Deploy to Staging
```bash
# Merge PR to main
# Staging deployment happens automatically
# Check status:
./scripts/deploy/health-check.sh staging
```
### 5. Deploy to Production
```bash
# Go to GitHub Actions > CD - Production Deployment
# Click "Run workflow"
# Enter:
# - Service: all
# - Environment: production
# - Confirm: deploy
# Wait for approval gate
# Approve deployment
# Monitor progress
```
## Features
### Smart Build Detection
Only builds changed projects using Turborepo filters:
```yaml
# Detects changes in:
- apps/maerchenzauber/**
- apps/chat/**
- packages/**
# Only builds affected projects
```
### Zero-Downtime Deployments
Rolling update strategy:
```bash
# Scale up with new version
docker compose up -d --scale service=2 service
sleep 15
# Scale down to single instance
docker compose up -d --scale service=1 service
```
### Comprehensive Health Checks
Every service has health endpoints:
```bash
# Automated health checks after deployment
- mana-core-auth: /api/v1/health
- backends: /health or /api/health
- web apps: / (root)
```
### Automated Backups
Production deployments automatically create backups:
```bash
# Pre-deployment backup includes:
- PostgreSQL database dump
- Docker compose configuration
- Environment files
- Current image tags
```
### Security Features
- Dependency scanning (Dependabot)
- Security audits (weekly)
- Non-root Docker users
- Secret scanning
- SSH key rotation guidance
## Architecture
```
┌──────────────┐
│ Pull Request │
└──────┬───────┘
│
▼
┌──────────────────────────────────────┐
│ PR Validation Workflow │
│ - Detect changes │
│ - Lint & format check │
│ - Type check │
│ - Build affected projects │
│ - Run tests │
│ - Docker build validation │
│ - Security scan │
└──────┬───────────────────────────────┘
│
▼
┌──────────────┐
│ Merge to Main│
└──────┬───────┘
│
▼
┌──────────────────────────────────────┐
│ Main Branch CI │
│ - Full validation │
│ - Build all projects │
│ - Build Docker images │
│ - Push to registry │
│ - Trigger staging deployment │
└──────┬───────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Staging Deployment (Automatic) │
│ - Pull latest images │
│ - Run migrations │
│ - Rolling update │
│ - Health checks │
└───────────────────────────────────────┘
│
▼ (Manual)
┌──────────────────────────────────────┐
│ Production Deployment (Manual) │
│ - Request approval │
│ - Create backup │
│ - Pull images │
│ - Run migrations │
│ - Zero-downtime deployment │
│ - Extended health checks │
│ - Smoke tests │
│ - Monitor for 5 minutes │
└───────────────────────────────────────┘
```
## Project Structure
```
manacore-monorepo/
├── .github/
│ ├── workflows/
│ │ ├── ci-pull-request.yml # PR validation
│ │ ├── ci-main.yml # Main branch CI
│ │ ├── cd-staging.yml # Staging deployment
│ │ ├── cd-production.yml # Production deployment
│ │ ├── test-coverage.yml # Coverage tracking
│ │ └── dependency-update.yml # Dependency management
│ └── dependabot.yml # Dependabot config
├── docker/
│ ├── templates/
│ │ ├── Dockerfile.nestjs # NestJS template
│ │ ├── Dockerfile.sveltekit # SvelteKit template
│ │ └── Dockerfile.astro # Astro template
│ └── nginx/
│ └── astro.conf # Nginx config for Astro
├── scripts/
│ └── deploy/
│ ├── build-and-push.sh # Build images
│ ├── deploy-hetzner.sh # Deploy to server
│ ├── health-check.sh # Health verification
│ ├── rollback.sh # Rollback procedure
│ └── migrate-db.sh # Database migrations
├── docs/
│ ├── DEPLOYMENT.md # Deployment guide
│ ├── CI_CD_SETUP.md # Setup instructions
│ └── DOCKER_GUIDE.md # Docker guide
├── tests/
│ └── e2e/
│ └── example.spec.ts # Example E2E test
├── docker-compose.staging.yml # Staging orchestration
├── docker-compose.production.yml # Production orchestration
├── vitest.config.ts # Vitest config
├── jest.config.js # Jest config
├── playwright.config.ts # Playwright config
└── .dockerignore # Docker build exclusions
```
## Services Deployed
The pipeline handles deployment for:
1. **mana-core-auth** (Port 3001)
- Central authentication service
- JWT token management
- User authentication
2. **maerchenzauber-backend** (Port 3002)
- Story generation service
- Azure OpenAI integration
- Character management
3. **chat-backend** (Port 3002)
- Chat API service
- AI conversation handling
- Message persistence
4. **manadeck-backend** (Port 3003)
- Card/deck management
- Collection handling
5. **nutriphi-backend** (Port 3004)
- Nutrition tracking service
6. **news-api** (Port 3005)
- News aggregation service
## Monitoring and Alerts
### Built-in Monitoring
- Health check endpoints
- Docker health checks
- Resource usage tracking
- Log aggregation
### Recommended External Tools
- **Uptime Monitoring**: UptimeRobot, Pingdom
- **Error Tracking**: Sentry
- **Performance**: New Relic, Datadog
- **Logs**: Papertrail, Loggly
## Rollback Procedures
### Automatic Rollback
```bash
# Emergency rollback to previous version
./scripts/deploy/rollback.sh production all
```
**What it does**:
1. Confirms with user
2. Checks for backup
3. Stops current services
4. Restores previous configuration
5. Restores database
6. Starts previous version
7. Runs health checks
### Manual Rollback
```bash
# SSH to server
ssh deploy@api.mana.how
cd ~/manacore-production
# Find backup
ls -lt backups/
# Restore
cp backups/20250127_120000/docker-compose.yml .
docker compose up -d
```
## Cost Optimization
### GitHub Actions Minutes
- Free tier: 2,000 minutes/month
- Smart build detection reduces usage
- Workflow caching saves time
- Estimated usage: ~500-800 minutes/month
### Docker Registry
- Docker Hub free tier: 1 organization, unlimited public repos
- Estimated storage: ~10-15GB for all images
- Alternative: GitHub Container Registry (free)
### Server Resources
**Staging**:
- 2 vCPU, 4GB RAM: ~$10-15/month
- Hetzner CX21: €5.83/month
**Production**:
- 4 vCPU, 8GB RAM: ~$25-35/month
- Hetzner CX31: €11.66/month
## Security Considerations
### Secrets Management
- Never commit secrets to repository
- Use GitHub Secrets for CI/CD
- Rotate secrets every 90 days
- Use different secrets per environment
### Image Security
- Regular base image updates
- Dependabot for dependencies
- Security scanning in CI
- Non-root users in containers
### Network Security
- Firewall on servers
- SSL/TLS for all connections
- Reverse proxy for services
- Rate limiting
## Troubleshooting
### Common Issues
1. **Deployment fails**: Check GitHub Actions logs
2. **Health checks fail**: Review service logs
3. **Build fails**: Test build locally
4. **SSH issues**: Verify keys and permissions
**Full troubleshooting guide**: `docs/DEPLOYMENT.md#troubleshooting`
## Next Steps
1. ✅ Review this README
2. ✅ Read `docs/CI_CD_SETUP.md`
3. ✅ Configure GitHub secrets
4. ✅ Set up deployment server
5. ✅ Test PR workflow
6. ✅ Test staging deployment
7. ✅ Test production deployment
8. ✅ Set up monitoring
9. ✅ Configure alerts
10. ✅ Train team on procedures
## Support
For issues or questions:
1. Check documentation in `docs/`
2. Review GitHub Actions logs
3. Check deployment scripts
4. Contact DevOps team
## License
Private - Manacore Team Only

16
CLAUDE.md
View file

@ -39,6 +39,7 @@ For comprehensive guidelines on code patterns and conventions, see the `.claude/
| **chat** | AI chat application | NestJS backend, Expo mobile, SvelteKit web, Astro landing |
| **zitare** | Daily inspiration quotes | NestJS backend, Expo mobile, SvelteKit web, Astro landing |
| **contacts** | Contact management | NestJS backend, SvelteKit web |
| **citycorners** | City guide for Konstanz | Astro landing |
### Archived Projects (`apps-archived/`)
@ -514,7 +515,7 @@ $: doubled = count * 2;
| `@manacore/shared-nestjs-auth` | NestJS JWT validation guards via mana-core-auth |
| `@mana-core/nestjs-integration` | NestJS module with auth guards + credit client |
| `@manacore/shared-auth` | Client-side auth service for web/mobile apps |
| `@manacore/shared-storage` | S3-compatible storage (MinIO local, Hetzner prod) |
| `@manacore/shared-storage` | S3-compatible storage (MinIO) |
| `@manacore/shared-supabase` | Unified Supabase client |
| `@manacore/shared-types` | Common TypeScript types |
| `@manacore/shared-utils` | Utility functions |
@ -536,7 +537,7 @@ import { formatDate, truncate } from '@manacore/shared-utils';
- Each project has its own Supabase project/schema
- Types typically generated via `supabase gen types`
## Object Storage (MinIO / Hetzner)
## Object Storage (MinIO)
S3-compatible object storage for file uploads, generated images, etc.
@ -544,8 +545,7 @@ S3-compatible object storage for file uploads, generated images, etc.
| Environment | Service | Purpose |
|-------------|---------|---------|
| **Local** | MinIO (Docker) | S3-compatible local storage |
| **Production** | Hetzner Object Storage | Cost-effective S3-compatible cloud storage |
| **Local + Production** | MinIO (Docker) | S3-compatible storage |
### Local Development
@ -597,17 +597,11 @@ const uploadUrl = await storage.getUploadUrl(key, { expiresIn: 3600 });
### Environment Variables
```env
# Local (in .env.development)
# MinIO (local + production via Docker)
S3_ENDPOINT=http://localhost:9000
S3_REGION=us-east-1
S3_ACCESS_KEY=minioadmin
S3_SECRET_KEY=minioadmin
# Production (Hetzner)
S3_ENDPOINT=https://fsn1.your-objectstorage.com
S3_REGION=fsn1
S3_ACCESS_KEY=your-access-key
S3_SECRET_KEY=your-secret-key
```
## Landing Pages (Cloudflare Pages)

210
FILES_CREATED.md
View file

@ -1,210 +0,0 @@
# CI/CD Implementation - Files Created
Complete list of all files created for the CI/CD pipeline implementation.
## Summary
- **Total Files**: 28
- **Total Lines of Code**: ~3,500
- **Documentation Pages**: 70+
- **Workflows**: 6
- **Scripts**: 5
- **Templates**: 3
- **Configurations**: 14
## GitHub Actions Workflows (6 files)
Located in `.github/workflows/`:
1. `ci-pull-request.yml` - PR validation (lint, test, build)
2. `ci-main.yml` - Main branch CI with Docker builds
3. `cd-staging.yml` - Automated staging deployment
4. `cd-production.yml` - Manual production deployment with approval
5. `test-coverage.yml` - Code coverage tracking
6. `dependency-update.yml` - Automated dependency management
## Docker Templates (3 files)
Located in `docker/templates/`:
1. `Dockerfile.nestjs` - NestJS backend template
2. `Dockerfile.sveltekit` - SvelteKit web app template
3. `Dockerfile.astro` - Astro landing page template
## Docker Orchestration (2 files)
Located in repository root:
1. `docker-compose.staging.yml` - Full staging environment with PostgreSQL, Redis, and all services
2. `docker-compose.production.yml` - Production configuration with resource limits
## Deployment Scripts (5 files)
Located in `scripts/deploy/`:
1. `build-and-push.sh` - Build and push Docker images to registry
2. `deploy-hetzner.sh` - Deploy to Hetzner/Hetzner VPSs via SSH
3. `health-check.sh` - Verify service health across environments
4. `rollback.sh` - Emergency rollback with backup restoration
5. `migrate-db.sh` - Database migration runner
## Testing Infrastructure (4 files)
Located in repository root and `tests/`:
1. `vitest.config.ts` - Vitest configuration for unit tests
2. `jest.config.js` - Jest multi-project configuration
3. `playwright.config.ts` - Playwright E2E test configuration
4. `tests/e2e/example.spec.ts` - Example E2E test suite
## Documentation (5 files)
Located in repository root and `docs/`:
1. `CI_CD_README.md` - Main CI/CD overview and quick reference
2. `docs/DEPLOYMENT.md` - Complete deployment guide (25+ pages)
3. `docs/CI_CD_SETUP.md` - Step-by-step setup instructions (20+ pages)
4. `docs/DOCKER_GUIDE.md` - Docker best practices and troubleshooting (18+ pages)
5. `QUICK_START_CICD.md` - 30-minute quick start guide
## Configuration Files (3 files)
Located in repository root and `docker/`:
1. `.dockerignore` - Docker build context optimization
2. `.github/dependabot.yml` - Automated dependency updates configuration
3. `docker/nginx/astro.conf` - Nginx configuration for Astro landing pages
## Summary Documents (2 files)
Located in repository root:
1. `CI_CD_IMPLEMENTATION_SUMMARY.md` - Complete implementation summary
2. `FILES_CREATED.md` - This file
## File Tree
```
manacore-monorepo/
├── .github/
│ ├── workflows/
│ │ ├── ci-pull-request.yml
│ │ ├── ci-main.yml
│ │ ├── cd-staging.yml
│ │ ├── cd-production.yml
│ │ ├── test-coverage.yml
│ │ └── dependency-update.yml
│ └── dependabot.yml
├── docker/
│ ├── templates/
│ │ ├── Dockerfile.nestjs
│ │ ├── Dockerfile.sveltekit
│ │ └── Dockerfile.astro
│ └── nginx/
│ └── astro.conf
├── scripts/
│ └── deploy/
│ ├── build-and-push.sh
│ ├── deploy-hetzner.sh
│ ├── health-check.sh
│ ├── rollback.sh
│ └── migrate-db.sh
├── docs/
│ ├── DEPLOYMENT.md
│ ├── CI_CD_SETUP.md
│ └── DOCKER_GUIDE.md
├── tests/
│ └── e2e/
│ └── example.spec.ts
├── docker-compose.staging.yml
├── docker-compose.production.yml
├── vitest.config.ts
├── jest.config.js
├── playwright.config.ts
├── .dockerignore
├── CI_CD_README.md
├── CI_CD_IMPLEMENTATION_SUMMARY.md
├── QUICK_START_CICD.md
└── FILES_CREATED.md
```
## Lines of Code by Category
| Category | Files | Approx. Lines |
| ------------------------- | ------ | ---------------- |
| GitHub Actions YAML | 6 | 1,500 |
| Deployment Scripts (Bash) | 5 | 800 |
| Docker Configurations | 5 | 400 |
| Test Configurations | 4 | 400 |
| Documentation (Markdown) | 5 | 70+ pages |
| Configuration Files | 3 | 100 |
| **Total** | **28** | **~3,500 lines** |
## Key Features Implemented
### GitHub Actions
- Smart build detection (only affected projects)
- Automated PR validation
- Docker image building and pushing
- Staging auto-deployment
- Production manual deployment with approval
- Test coverage tracking
- Dependency scanning and updates
### Docker
- Multi-stage builds for optimization
- Non-root users for security
- Health checks for monitoring
- Resource limits for stability
- Environment-specific configurations
### Deployment
- Zero-downtime rolling updates
- Automated health checks
- Pre-deployment backups
- Database migrations
- Emergency rollback procedures
### Testing
- Unit tests (Vitest/Jest)
- E2E tests (Playwright)
- Coverage reporting (Codecov)
- Multi-project support
- 50% minimum coverage threshold
### Documentation
- Quick start guide (30 minutes)
- Complete setup guide (step-by-step)
- Deployment operations guide
- Docker best practices
- Troubleshooting sections
## All Files Are
- ✅ Production-ready
- ✅ Error-handled
- ✅ Well-documented
- ✅ Tested syntax
- ✅ Security-focused
- ✅ Performance-optimized
## Usage
All files are ready to use immediately after:
1. Configuring GitHub secrets (22 required)
2. Setting up deployment servers
3. Adding SSH keys
4. Testing the pipeline
See `QUICK_START_CICD.md` for the fastest path to deployment.
---
**Created**: 2025-01-27
**Status**: Complete and Production-Ready

887
HIVE_MIND_FINAL_REPORT.md
View file

@ -1,887 +0,0 @@
# 🧠 HIVE MIND COLLECTIVE INTELLIGENCE - FINAL REPORT
**Swarm ID**: swarm-1764212414813-nbrqx50g3
**Swarm Name**: hive-1764212414796
**Queen Type**: Strategic Coordinator
**Mission**: Complete hosting architecture and CI/CD plan for Hetzner/Docker Compose deployment
**Date**: 2025-11-27
**Status**: ✅ MISSION COMPLETE
---
## 🎯 EXECUTIVE SUMMARY
The Hive Mind collective has successfully analyzed, designed, and implemented a **complete production-ready deployment system** for the manacore-monorepo. Through coordinated effort across 4 specialized worker agents, we've delivered:
- **Comprehensive hosting platform analysis** (Hetzner + Coolify recommended)
- **Complete deployment architecture** for 39 services across 10 projects
- **Fully automated CI/CD pipeline** with GitHub Actions
- **Production-ready testing infrastructure** targeting 80% coverage
- **28 implementation files** with ~3,500 lines of code
- **~200,000 words of documentation** across 15+ comprehensive guides
**Total Investment**: 4 concurrent agent workflows, ~2 hours coordination time
**Deliverables**: Production-ready infrastructure deployable within 30 minutes
---
## 🐝 WORKER AGENT REPORTS
### 1️⃣ RESEARCHER AGENT - Infrastructure Analysis
**Mission**: Research and compare Hetzner vs Coolify hosting options
**Key Findings**:
- ✅ **Recommended Platform**: Docker Compose + Hetzner VPS
- ✅ **Cost Efficiency**: 92% cheaper than traditional PaaS ($50/month vs $300/month)
- ✅ **Performance**: Hetzner beats DigitalOcean in CPU benchmarks (5-10% faster)
- ✅ **Real-World Validation**: User report showed $300 → $25/month savings
**Decision Matrix Score**: 8.40/10 (highest among 4 options evaluated)
**Research Scope**:
- 24+ web searches across official docs, benchmarks, case studies
- Detailed cost breakdowns for 6-project deployment
- Security analysis (ISO 27001, GDPR compliance)
- 9-week implementation roadmap
- Complete Docker multi-stage build examples
**Primary Deliverable**:
📄 `.hive-mind/sessions/research-report-hosting-infrastructure.md` (40+ pages)
**Consensus Vote**: **Approve Docker Compose + Hetzner VPS** ✅
---
### 2️⃣ ANALYST AGENT - Architecture Design
**Mission**: Design complete deployment architecture for 39 services
**Key Deliverables**:
- ✅ **Service Inventory**: 10 NestJS backends, 9 SvelteKit web apps, 9 Astro landing pages, 8 Expo mobile apps
- ✅ **Container Strategy**: Multi-stage Docker builds (Alpine Linux, 120-180 MB final images)
- ✅ **Deployment Topology**: Blue-green deployment with zero-downtime updates
- ✅ **Data Architecture**: Separate Supabase projects per product, shared auth database
- ✅ **Network Architecture**: Cloudflare CDN, SSL/TLS automation, network segmentation
- ✅ **Monitoring Stack**: Prometheus + Grafana + Loki + Sentry
**Architecture Highlights**:
- **Environment Stages**: Development (local) → Staging (Coolify) → Production (Coolify/K8s)
- **Domain Strategy**: `{service}.mana.how` (e.g., `api-chat.mana.how`)
- **Disaster Recovery**: RTO < 1 hour, RPO < 24 hours, automated daily backups
- **Resource Requirements**: 15 vCPU, 15 GB RAM, 100 GB SSD (~$150-300/month single-server)
**Primary Deliverables**:
📄 `docs/DEPLOYMENT_ARCHITECTURE.md` (63,000+ characters)
📄 `docs/DEPLOYMENT_DIAGRAMS.md` (16,000+ characters - ASCII diagrams)
📄 `docs/DEPLOYMENT_RUNBOOKS.md` (8,000+ characters - operational procedures)
**Consensus Vote**: **Approve Architecture Design** ✅
---
### 3️⃣ CODER AGENT - CI/CD Implementation
**Mission**: Implement complete CI/CD pipeline and Docker infrastructure
**Key Deliverables**:
- ✅ **6 GitHub Actions Workflows**: PR validation, main CI, staging/production deployment, coverage tracking, dependency updates
- ✅ **3 Dockerfile Templates**: NestJS, SvelteKit, Astro (multi-stage, optimized for pnpm monorepo)
- ✅ **5 Deployment Scripts**: build-and-push, deploy-hetzner, health-check, rollback, migrate-db
- ✅ **2 Docker Compose Files**: staging and production orchestration
- ✅ **Testing Infrastructure**: Vitest, Jest, Playwright configurations
**Pipeline Features**:
- **Smart Build Detection**: Only builds changed projects (Turborepo filters)
- **Zero-Downtime Deployments**: Rolling updates with automated health checks
- **Security**: Weekly audits, non-root Docker users, SSH key rotation
- **Performance**: Layer caching reduces build time 12-15 min → 2-3 min
**Code Statistics**:
- **28 production-ready files created**
- **~3,500 lines of code**
- **70+ pages of documentation**
**Primary Deliverables**:
📄 `docs/DEPLOYMENT.md` (25+ pages)
📄 `docs/CI_CD_SETUP.md` (20+ pages)
📄 `docs/DOCKER_GUIDE.md` (18+ pages)
📄 `CI_CD_README.md` (8+ pages)
📄 `QUICK_START_CICD.md` (5+ pages - 30-minute fast track)
**Consensus Vote**: **Approve CI/CD Implementation** ✅
---
### 4️⃣ TESTER AGENT - Testing Strategy
**Mission**: Design and implement comprehensive automated testing strategy
**Key Deliverables**:
- ✅ **3 Major Documentation Files**: Master strategy, implementation guide, executive summary (50,000+ words)
- ✅ **Shared Test Configuration Package**: Reusable configs for all app types (Jest, Vitest, Playwright)
- ✅ **7 Production-Quality Test Examples**: Backend, mobile, web, shared (3,400+ lines)
- ✅ **CI/CD Test Automation**: 8 parallel job types in GitHub Actions
**Testing Framework Matrix**:
| App Type | Framework | Coverage | E2E |
|----------|-----------|----------|-----|
| NestJS Backend | Jest | 80% | Supertest |
| React Native Mobile | Jest + jest-expo | 80% | Detox/Maestro |
| SvelteKit Web | Vitest | 80% | Playwright |
| Astro Landing | Vitest | 80% | Playwright |
| Shared Packages | Vitest | 90% | N/A |
**Current State Analysis**:
- **Before**: 25 test files, ~5% coverage
- **Target**: 80% coverage for new code, 100% for critical paths (auth, payments)
- **Impact**: 80%+ bug reduction estimated
**Primary Deliverables**:
📄 `docs/TESTING.md` (35,000+ words - master strategy)
📄 `docs/TESTING_IMPLEMENTATION_GUIDE.md` (8,000+ words)
📄 `docs/TESTING_SUMMARY.md` (7,000+ words)
📄 `packages/test-config/` (6 configuration files)
📄 `docs/test-examples/` (7 example test files)
**Consensus Vote**: **Approve Testing Strategy** ✅
---
## 🎯 COLLECTIVE INTELLIGENCE SYNTHESIS
### CONSENSUS DECISIONS (Majority Vote: 4/4 ✅)
1. **Hosting Platform**: Docker Compose + Hetzner VPS
- **Reasoning**: 92% cost savings, excellent performance, open-source flexibility
- **Vote**: Unanimous approval (Researcher, Analyst, Coder, Tester)
2. **Deployment Strategy**: Blue-Green with Zero-Downtime
- **Reasoning**: Instant rollback, minimal risk, production-proven
- **Vote**: Unanimous approval
3. **Container Orchestration**: Start with Coolify, migrate to K8s when scale demands
- **Reasoning**: Simplicity now, scalability later
- **Vote**: Unanimous approval
4. **Testing Coverage**: 80% minimum, 100% for critical paths
- **Reasoning**: Industry standard, achievable, high ROI
- **Vote**: Unanimous approval
5. **CI/CD Automation**: Full automation with manual approval for production
- **Reasoning**: Balance between speed and safety
- **Vote**: Unanimous approval
---
## 📊 DELIVERABLES MATRIX
### Documentation Created (15+ Files)
| Category | Files | Pages | Word Count | Status |
| --------------------------- | ------ | -------- | ------------ | ----------- |
| **Infrastructure Research** | 1 | 40+ | 50,000+ | ✅ Complete |
| **Architecture Design** | 3 | 45+ | 87,000+ | ✅ Complete |
| **CI/CD Implementation** | 5 | 76+ | 80,000+ | ✅ Complete |
| **Testing Strategy** | 3 | 50+ | 50,000+ | ✅ Complete |
| **Test Examples** | 7 | 25+ | 3,400 lines | ✅ Complete |
| **TOTAL** | **19** | **236+** | **~200,000** | ✅ Complete |
### Code & Configuration Files (40+ Files)
| Category | Files | Lines of Code | Status |
| ---------------------------- | ------- | ------------- | ----------- |
| **GitHub Actions Workflows** | 7 | ~800 | ✅ Complete |
| **Dockerfiles & Compose** | 5 | ~500 | ✅ Complete |
| **Deployment Scripts** | 5 | ~1,200 | ✅ Complete |
| **Test Configurations** | 6 | ~400 | ✅ Complete |
| **Test Examples** | 7 | ~3,400 | ✅ Complete |
| **Documentation Support** | 10+ | ~1,000 | ✅ Complete |
| **TOTAL** | **40+** | **~7,300** | ✅ Complete |
---
## 🚀 RECOMMENDED IMPLEMENTATION PLAN
### Phase 1: Quick Start (30 Minutes)
**Goal**: Validate CI/CD pipeline with one project
1. **Read Quick Start Guide**: `QUICK_START_CICD.md`
2. **Configure GitHub Secrets**: 3-5 essential secrets (see CI_CD_SETUP.md)
3. **Set Up One Server**: Hetzner CCX12 ($19/month)
4. **Test PR Workflow**: Create test PR, verify automated checks
**Success Criteria**: Green checkmark on test PR ✅
---
### Phase 2: Foundation Setup (Week 1-2)
**Goal**: Complete infrastructure foundation
**Week 1 Tasks**:
- [ ] Create Hetzner account and provision staging server
- [ ] Set up Docker Compose on staging server
- [ ] Configure all 22 GitHub secrets
- [ ] Set up Docker registry (GitHub Container Registry)
- [ ] Configure custom domains and DNS
**Week 2 Tasks**:
- [ ] Deploy first project (chat) to staging
- [ ] Test complete CI/CD pipeline
- [ ] Verify health checks and monitoring
- [ ] Train team on deployment workflow
- [ ] Document any environment-specific adjustments
**Success Criteria**: One project running in staging with automated deployments ✅
---
### Phase 3: Production Rollout (Week 3-6)
**Goal**: Deploy all projects to production
**Week 3-4**:
- [ ] Provision production server(s)
- [ ] Set up production environment in Coolify
- [ ] Deploy mana-core-auth service
- [ ] Deploy first 2 projects (chat, picture)
- [ ] Configure monitoring (Prometheus + Grafana)
**Week 5-6**:
- [ ] Deploy remaining 7 projects (maerchenzauber, manacore, manadeck, memoro, uload, nutriphi, others)
- [ ] Set up Cloudflare CDN for static assets
- [ ] Configure SSL/TLS for all domains
- [ ] Implement backup automation
- [ ] Load testing and optimization
**Success Criteria**: All 10 projects running in production with <99.9% uptime ✅
---
### Phase 4: Testing Infrastructure (Week 7-14)
**Goal**: Achieve 80% test coverage
**Week 7-8**: Critical path coverage (auth, payments) - 100%
**Week 9-10**: Backend coverage (5 projects) - 80%
**Week 11-12**: Mobile + Web coverage (16 projects) - 80%
**Week 13-14**: E2E testing (Playwright + Detox/Maestro)
**Success Criteria**: 80% coverage enforced in CI/CD, all critical paths at 100% ✅
---
### Phase 5: Optimization & Hardening (Week 15-16)
**Goal**: Production hardening and performance optimization
- [ ] Security audit and penetration testing
- [ ] Performance optimization (caching, CDN, database queries)
- [ ] Disaster recovery drill
- [ ] Team training and documentation review
- [ ] Establish on-call rotation and incident response procedures
**Success Criteria**: Production-grade reliability, security, and team readiness ✅
---
## 💰 COST ANALYSIS
### Infrastructure Costs (Monthly)
**Option A: Single-Server Setup (Recommended for Start)**
- **Hetzner CCX32**: 8 vCPU, 32 GB RAM, 240 GB SSD - **$50/month**
- **Domains**: 6 domains @ $12/year each - **$6/month**
- **Cloudflare**: Free tier (CDN, SSL, DNS) - **$0/month**
- **GitHub Actions**: Within free tier - **$0/month**
- **Docker Registry**: GitHub Container Registry (free tier) - **$0/month**
- **Total**: **~$56/month**
**Option B: Multi-Server Setup (Scaling Phase)**
- **Hetzner CCX22** (staging): 4 vCPU, 16 GB RAM - **$25/month**
- **Hetzner CCX42** (production): 16 vCPU, 64 GB RAM - **$100/month**
- **Hetzner CX32** (monitoring): 4 vCPU, 8 GB RAM - **$15/month**
- **Domains & CDN**: **$6/month**
- **Total**: **~$146/month**
**Option C: High-Availability Setup (Future)**
- **Hetzner Kubernetes Cluster**: 3 nodes (CCX32 each) - **$150/month**
- **Load Balancer**: **$5/month**
- **Object Storage (R2)**: 10 GB - **$0.15/month**
- **Managed PostgreSQL** (if moving from Supabase): **$50/month**
- **Total**: **~$205/month**
**Comparison to Alternatives**:
- **AWS/Azure/GCP**: $500-1,000/month (3-18x more expensive)
- **Heroku/Railway/Render**: $300-500/month (5-9x more expensive)
- **DigitalOcean App Platform**: $150-300/month (2.5-5x more expensive)
**Hive Mind Consensus**: Start with Option A ($56/month), scale to Option B when traffic demands ✅
---
## 📈 SUCCESS METRICS
### Key Performance Indicators (KPIs)
**Deployment Metrics**:
- ✅ Deployment Time: < 10 minutes (current: manual, 2+ hours)
- ✅ Deployment Frequency: Multiple times per day (current: weekly)
- ✅ Rollback Time: < 5 minutes (current: hours)
- ✅ Failed Deployments: < 5% (current: unknown)
**Quality Metrics**:
- ✅ Test Coverage: 80% minimum (current: ~5%)
- ✅ Critical Path Coverage: 100% (current: ~0%)
- ✅ Build Success Rate: > 95% (current: unknown)
- ✅ Code Review Turnaround: < 24 hours
**Reliability Metrics**:
- ✅ Uptime: 99.9% (current: unknown)
- ✅ Mean Time to Recovery (MTTR): < 1 hour
- ✅ Mean Time Between Failures (MTBF): > 30 days
- ✅ Backup Success Rate: 100%
**Cost Metrics**:
- ✅ Infrastructure Cost: < $100/month (target: $56/month)
- ✅ Cost per Service: < $5/month
- ✅ Cost Reduction: 92% vs traditional PaaS
---
## 🔒 SECURITY & COMPLIANCE
### Security Measures Implemented
**Infrastructure Security**:
- ✅ Non-root Docker containers
- ✅ Read-only filesystems where possible
- ✅ Network segmentation (frontend, backend, data layers)
- ✅ Firewall rules (only ports 22, 80, 443 exposed)
- ✅ SSH key-based authentication (no passwords)
- ✅ Automatic security updates (Dependabot)
**Application Security**:
- ✅ Environment variable encryption (GitHub Secrets)
- ✅ SSL/TLS for all services (Let's Encrypt)
- ✅ JWT-based authentication (@manacore/shared-auth)
- ✅ Row-Level Security (Supabase RLS policies)
- ✅ Input validation and sanitization
- ✅ CORS policies enforced
**CI/CD Security**:
- ✅ Weekly dependency audits
- ✅ Docker image scanning (Trivy)
- ✅ No secrets in code (enforced by pre-commit hooks)
- ✅ Branch protection rules
- ✅ Required code reviews
- ✅ Signed commits (optional, recommended)
**Compliance**:
- ✅ GDPR compliance (Hetzner EU data centers)
- ✅ ISO 27001 certified infrastructure (Hetzner)
- ✅ SOC 2 Type II (Supabase)
- ✅ Automated backup retention policies
- ✅ Audit logs (GitHub Actions, Coolify, Supabase)
---
## 📚 DOCUMENTATION INDEX
### Quick Navigation
**Getting Started**:
1. 🚀 [QUICK_START_CICD.md](./QUICK_START_CICD.md) - 30-minute deployment guide
2. 📖 [CI_CD_README.md](./CI_CD_README.md) - Overview and quick reference
3. 🏗️ [docs/CI_CD_SETUP.md](./docs/CI_CD_SETUP.md) - Complete setup instructions
**Architecture & Design**:
1. 🏛️ [docs/DEPLOYMENT_ARCHITECTURE.md](./docs/DEPLOYMENT_ARCHITECTURE.md) - Complete architecture spec
2. 📊 [docs/DEPLOYMENT_DIAGRAMS.md](./docs/DEPLOYMENT_DIAGRAMS.md) - ASCII diagrams
3. 📋 [docs/DEPLOYMENT_RUNBOOKS.md](./docs/DEPLOYMENT_RUNBOOKS.md) - Operational procedures
**CI/CD Implementation**:
1. 🔧 [docs/DEPLOYMENT.md](./docs/DEPLOYMENT.md) - Deployment operations guide
2. 🐳 [docs/DOCKER_GUIDE.md](./docs/DOCKER_GUIDE.md) - Docker best practices
3. ⚙️ [.github/workflows/](../.github/workflows/) - GitHub Actions workflows
**Testing Strategy**:
1. 🧪 [docs/TESTING.md](./docs/TESTING.md) - Master testing strategy (35,000+ words)
2. 🚀 [docs/TESTING_IMPLEMENTATION_GUIDE.md](./docs/TESTING_IMPLEMENTATION_GUIDE.md) - Quick start
3. 📊 [docs/TESTING_SUMMARY.md](./docs/TESTING_SUMMARY.md) - Executive summary
4. 💡 [docs/test-examples/](./docs/test-examples/) - Production-quality examples
**Infrastructure Research**:
1. 🔍 [.hive-mind/sessions/research-report-hosting-infrastructure.md](./.hive-mind/sessions/research-report-hosting-infrastructure.md) - Complete research report (40+ pages)
---
## 🎓 TEAM TRAINING PLAN
### Developer Onboarding (2-4 Hours)
**Session 1: CI/CD Basics (1 hour)**
- Read: QUICK_START_CICD.md
- Hands-on: Create test PR and observe automated checks
- Practice: Fix failing tests, see green checkmarks
**Session 2: Testing Fundamentals (1 hour)**
- Read: TESTING_IMPLEMENTATION_GUIDE.md
- Hands-on: Write tests for one component using examples
- Practice: Run tests locally, verify coverage
**Session 3: Docker & Deployment (1 hour)**
- Read: DOCKER_GUIDE.md sections 1-4
- Hands-on: Build Docker image locally
- Practice: Test container locally with docker-compose
**Session 4: Advanced Topics (1 hour, optional)**
- Read: DEPLOYMENT_ARCHITECTURE.md sections 1-5
- Discuss: Blue-green deployment, rollback procedures
- Review: Monitoring dashboards, alert thresholds
---
### DevOps Onboarding (4-8 Hours)
**Session 1: Architecture Deep Dive (2 hours)**
- Read: DEPLOYMENT_ARCHITECTURE.md (complete)
- Review: DEPLOYMENT_DIAGRAMS.md
- Discuss: Design decisions and trade-offs
**Session 2: Infrastructure Setup (2 hours)**
- Hands-on: Set up Hetzner server
- Hands-on: Install and configure Coolify
- Practice: Deploy test service
**Session 3: CI/CD Operations (2 hours)**
- Read: CI_CD_SETUP.md (complete)
- Hands-on: Configure GitHub secrets
- Practice: Trigger manual deployment
**Session 4: Incident Response (2 hours)**
- Read: DEPLOYMENT_RUNBOOKS.md
- Practice: Execute rollback procedure
- Practice: Restore from backup
- Review: Monitoring and alerting
---
## 🐛 TROUBLESHOOTING & SUPPORT
### Common Issues & Solutions
**Issue 1: Docker Build Fails in CI**
- **Symptom**: GitHub Actions workflow fails at "Build Docker Image" step
- **Solution**: Check .dockerignore, verify all dependencies in package.json
- **Reference**: DOCKER_GUIDE.md section 6.1
**Issue 2: Tests Fail Locally but Pass in CI**
- **Symptom**: Local test failures but CI shows green
- **Solution**: Clear node_modules and pnpm cache, check Node.js version
- **Reference**: TESTING_IMPLEMENTATION_GUIDE.md section 5.1
**Issue 3: Deployment Succeeds but Service Unhealthy**
- **Symptom**: Deployment completes but health check fails
- **Solution**: Check environment variables, verify Supabase connection
- **Reference**: DEPLOYMENT.md section 4.3
**Issue 4: Coverage Below Threshold**
- **Symptom**: CI fails with "Coverage threshold not met"
- **Solution**: Add missing tests or adjust thresholds temporarily
- **Reference**: TESTING.md section 4
**Issue 5: Slow Build Times**
- **Symptom**: GitHub Actions taking 15+ minutes
- **Solution**: Enable Turborepo remote cache, optimize Docker layers
- **Reference**: CI_CD_SETUP.md section 7
---
## 🔮 FUTURE ENHANCEMENTS
### Short-Term (3-6 Months)
1. **Monitoring Enhancements**
- Grafana dashboard templates for all services
- Custom alerting rules per project
- Integration with Slack/PagerDuty
2. **Performance Optimization**
- Redis caching layer
- Database query optimization
- CDN configuration for API responses
3. **Developer Experience**
- Pre-commit hooks (Husky + lint-staged)
- Commitlint for conventional commits
- VSCode task configurations
4. **Testing Expansion**
- Visual regression testing (Percy/Chromatic)
- Load testing (k6/Artillery)
- Mobile E2E testing (Detox/Maestro)
---
### Long-Term (6-12 Months)
1. **Kubernetes Migration**
- Migrate from Coolify to Hetzner Kubernetes
- Implement Helm charts for all services
- Set up Istio service mesh
2. **Advanced CI/CD**
- Canary deployments with traffic shifting
- Feature flags (LaunchDarkly/Unleash)
- Automated performance regression detection
3. **Multi-Region Deployment**
- Deploy to multiple regions (EU, US, Asia)
- Global load balancing
- Database replication
4. **Observability 2.0**
- Distributed tracing (Jaeger/Zipkin)
- Real user monitoring (RUM)
- Business metrics dashboards
---
## ✅ HIVE MIND CONSENSUS APPROVAL
**Final Consensus Vote**: **4/4 UNANIMOUS APPROVAL** ✅
- ✅ Researcher: Approve (platform choice validated by data)
- ✅ Analyst: Approve (architecture is sound and scalable)
- ✅ Coder: Approve (implementation is production-ready)
- ✅ Tester: Approve (testing strategy is comprehensive)
**Queen's Strategic Assessment**: All objectives achieved. The collective has delivered a complete, production-ready deployment system that balances cost efficiency, scalability, security, and developer experience.
---
## 🎉 MISSION ACCOMPLISHMENT
### Objectives Achieved
- ✅ **Hosting Platform**: Docker Compose + Hetzner VPS recommended with 92% cost savings
- ✅ **Architecture Design**: Complete blueprint for 39 services across 10 projects
- ✅ **CI/CD Pipeline**: Fully automated with GitHub Actions, zero-downtime deployments
- ✅ **Automated Testing**: Comprehensive strategy targeting 80% coverage
- ✅ **Documentation**: 236+ pages, 200,000+ words, production-ready
- ✅ **Code Implementation**: 40+ files, 7,300+ lines of production code
- ✅ **Cost Optimization**: $56/month infrastructure (vs $300+ for alternatives)
- ✅ **Security**: ISO 27001, GDPR compliance, automated audits
- ✅ **Scalability**: Design supports growth from 1 to 100+ services
---
## 📞 NEXT STEPS FOR YOU
### Immediate Actions (Today)
1. **Review This Report**: Read executive summary and consensus decisions
2. **Review Quick Start**: Read QUICK_START_CICD.md (5 minutes)
3. **Budget Approval**: Approve $56/month infrastructure budget
4. **Create Hetzner Account**: Sign up at hetzner.com
### This Week
1. **Read Key Documentation**:
- QUICK_START_CICD.md (30 minutes)
- DEPLOYMENT_ARCHITECTURE.md sections 1-3 (1 hour)
- TESTING_IMPLEMENTATION_GUIDE.md (30 minutes)
2. **Set Up Infrastructure**:
- Provision first Hetzner server
- Set up Docker Compose
- Configure GitHub secrets
3. **Deploy First Project**:
- Follow Phase 1 implementation plan
- Deploy chat project to staging
- Verify automated CI/CD
### Next 2 Weeks
1. **Complete Foundation**: Follow Phase 2 implementation plan
2. **Train Team**: Conduct developer onboarding sessions
3. **Production Deployment**: Deploy first 2 projects to production
---
## 🙏 ACKNOWLEDGMENTS
**Hive Mind Worker Agents**:
- 🔍 **Researcher**: Comprehensive infrastructure analysis (24+ searches, 40+ pages)
- 🏗️ **Analyst**: Complete architecture design (87,000+ characters)
- 💻 **Coder**: Production-ready implementation (28 files, 3,500+ lines)
- 🧪 **Tester**: Comprehensive testing strategy (50,000+ words)
**Collective Intelligence**: Greater than the sum of its parts ✨
---
## 📜 LICENSE & USAGE
All code, configurations, and documentation produced by the Hive Mind are:
- ✅ Royalty-free for use in the manacore-monorepo
- ✅ Modifiable without restriction
- ✅ Distributable within your organization
- ✅ Production-ready and battle-tested patterns
**Warranty**: Provided as-is. Test thoroughly before production deployment.
---
**🧠 Hive Mind Swarm - Mission Complete**
**Date**: 2025-11-27
**Status**: ✅ ALL OBJECTIVES ACHIEVED
**Recommendation**: PROCEED WITH IMPLEMENTATION
_"Alone we are smart. Together we are brilliant."_ - Hive Mind Collective
---
## 📎 APPENDIX
### A. File Locations
**Root Directory**: `/Users/wuesteon/dev/mana_universe/manacore-monorepo/`
**Documentation**:
- `docs/DEPLOYMENT_ARCHITECTURE.md`
- `docs/DEPLOYMENT_DIAGRAMS.md`
- `docs/DEPLOYMENT_RUNBOOKS.md`
- `docs/DEPLOYMENT.md`
- `docs/CI_CD_SETUP.md`
- `docs/DOCKER_GUIDE.md`
- `docs/TESTING.md`
- `docs/TESTING_IMPLEMENTATION_GUIDE.md`
- `docs/TESTING_SUMMARY.md`
- `docs/test-examples/` (directory with 7 files)
**CI/CD**:
- `.github/workflows/test.yml`
- `.github/workflows/ci-pull-request.yml`
- `.github/workflows/ci-main.yml`
- `.github/workflows/cd-staging.yml`
- `.github/workflows/cd-production.yml`
- `.github/workflows/test-coverage.yml`
- `.github/workflows/dependency-update.yml`
**Docker**:
- `docker/templates/Dockerfile.nestjs`
- `docker/templates/Dockerfile.sveltekit`
- `docker/templates/Dockerfile.astro`
- `docker/nginx/nginx.conf`
- `docker-compose.staging.yml`
- `docker-compose.production.yml`
- `.dockerignore`
**Scripts**:
- `scripts/deploy/build-and-push.sh`
- `scripts/deploy/deploy-hetzner.sh`
- `scripts/deploy/health-check.sh`
- `scripts/deploy/rollback.sh`
- `scripts/deploy/migrate-db.sh`
**Test Configuration**:
- `packages/test-config/` (6 configuration files)
- `vitest.config.ts`
- `jest.config.js`
- `playwright.config.ts`
**Quick Starts**:
- `CI_CD_README.md`
- `QUICK_START_CICD.md`
- `CI_CD_IMPLEMENTATION_SUMMARY.md`
- `FILES_CREATED.md`
**Hive Mind**:
- `.hive-mind/sessions/research-report-hosting-infrastructure.md`
- `HIVE_MIND_FINAL_REPORT.md` (this file)
### B. Command Reference
**Quick Start Commands**:
```bash
# Install dependencies
pnpm install
# Run all tests
pnpm test
# Run specific project tests
pnpm --filter @chat/backend test
# Run with coverage
pnpm --filter @chat/backend test:cov
# Build Docker image
pnpm run docker:build
# Deploy to staging
pnpm run deploy:staging
# Deploy to production
pnpm run deploy:production
```
**Development Commands**:
```bash
# Start local development
pnpm run dev
# Start specific project
pnpm run chat:dev
# Type check
pnpm type-check
# Lint & format
pnpm lint
pnpm format
# E2E tests
pnpm test:e2e
```
**Deployment Commands** (via scripts):
```bash
# Build and push all services
./scripts/deploy/build-and-push.sh
# Deploy to Hetzner
./scripts/deploy/deploy-hetzner.sh staging
./scripts/deploy/deploy-hetzner.sh production
# Health check
./scripts/deploy/health-check.sh
# Rollback
./scripts/deploy/rollback.sh
# Database migration
./scripts/deploy/migrate-db.sh
```
### C. Resource Links
**Official Documentation**:
- [Hetzner Cloud Docs](https://docs.hetzner.com/)
- [Coolify Documentation](https://coolify.io/docs)
- [Turborepo Docs](https://turbo.build/repo/docs)
- [pnpm Workspaces](https://pnpm.io/workspaces)
- [GitHub Actions](https://docs.github.com/en/actions)
**Testing Frameworks**:
- [Jest](https://jestjs.io/)
- [Vitest](https://vitest.dev/)
- [Playwright](https://playwright.dev/)
- [Testing Library](https://testing-library.com/)
**Container Ecosystem**:
- [Docker Documentation](https://docs.docker.com/)
- [Docker Compose](https://docs.docker.com/compose/)
- [Multi-stage Builds](https://docs.docker.com/build/building/multi-stage/)
**Monitoring & Observability**:
- [Prometheus](https://prometheus.io/docs/)
- [Grafana](https://grafana.com/docs/)
- [Loki](https://grafana.com/docs/loki/)
- [Sentry](https://docs.sentry.io/)
### D. Support & Contribution
**Questions or Issues?**
1. Check the troubleshooting sections in relevant docs
2. Review the FAQ in TESTING.md and DEPLOYMENT.md
3. Consult the Hive Mind collective wisdom in this report
**Found a Bug or Improvement?**
1. Document the issue with steps to reproduce
2. Propose a solution based on the established patterns
3. Test thoroughly before implementing
4. Update relevant documentation
**Want to Extend the System?**
1. Review the "Future Enhancements" section
2. Follow the established architectural patterns
3. Maintain consistency with existing code style
4. Add tests and documentation
---
**END OF HIVE MIND FINAL REPORT**
_Generated by Strategic Queen Coordinator with collective intelligence from 4 specialized worker agents._
_Total coordination time: ~2 hours_
_Total deliverables: 280+ pages of documentation + 40+ production-ready files_
_Status: Mission Complete ✅_

415
cicd/CHANGELOG.md
View file

@ -1,415 +0,0 @@
# CI/CD Implementation Changelog
All notable changes and progress updates for the CI/CD implementation.
**Format**: Based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
---
## [Unreleased]
### To Be Implemented
- Infrastructure provisioning (Hetzner + Coolify)
- GitHub secrets configuration
- First deployment to staging
- Testing implementation
- Production deployment
- Monitoring setup
---
## [0.7.0] - 2025-11-27
### Added - CI/CD Documentation Hub
- ✅ Created `cicd/` folder for centralized documentation
- ✅ Created `cicd/README.md` - Central navigation hub
- ✅ Created `cicd/TODO.md` - Actionable task list (36 core tasks, 8 phases)
- ✅ Created `cicd/COMPLETED.md` - Progress tracking and deliverables
- ✅ Created `cicd/PLAN.md` - Complete implementation plan and timeline
- ✅ Created `cicd/CHANGELOG.md` - This file
- ✅ Organized all CI/CD documentation in one place
- ✅ Added quick navigation and status tracking
### Changed
- Updated project organization for better CI/CD workflow management
- Consolidated scattered documentation into `cicd/` folder
**Impact**: Team now has a clear roadmap and centralized documentation for CI/CD implementation
**Status**: Documentation phase complete (70% overall progress)
---
## [0.6.0] - 2025-11-27
### Added - GitHub Container Registry Setup
- ✅ Configured GitHub Container Registry (ghcr.io) for Docker images
- ✅ Updated `.github/workflows/ci-main.yml` to use ghcr.io
- ✅ Created `DOCKER_REGISTRY_SETUP.md` with setup instructions
- ✅ Documented team access and troubleshooting
### Changed
- Switched from Docker Hub to GitHub Container Registry
- Image naming: `ghcr.io/wuesteon/service-name:tag`
- Authentication now uses `GITHUB_TOKEN` (automatic, no setup needed)
### Why This Change
- ✅ No additional signup required
- ✅ Automatic authentication in GitHub Actions
- ✅ Team access built-in via GitHub repo permissions
- ✅ No rate limits (unlike Docker Hub free tier)
- ✅ Unlimited private images (500 MB storage)
**Impact**: Zero setup required for Docker registry, automatic team access
---
## [0.5.0] - 2025-11-27
### Added - Hive Mind Final Report
- ✅ Created `HIVE_MIND_FINAL_REPORT.md` - Comprehensive summary
- ✅ Consolidated all 4 worker agent reports
- ✅ Documented consensus decisions
- ✅ Added implementation roadmap and timeline
- ✅ Included cost analysis and success metrics
- ✅ Indexed all 60+ deliverables
**Impact**: Executive-level overview of entire CI/CD implementation available
---
## [0.4.0] - 2025-11-27
### Added - Testing Strategy & Infrastructure
**Delivered by**: Tester Agent
#### Documentation
- ✅ `docs/TESTING.md` (35,000+ words, 2,850 lines)
- ✅ `docs/TESTING_IMPLEMENTATION_GUIDE.md` (8,000+ words)
- ✅ `docs/TESTING_SUMMARY.md` (7,000+ words)
#### Test Configuration Package
- ✅ `packages/test-config/jest.config.backend.js`
- ✅ `packages/test-config/jest.config.mobile.js`
- ✅ `packages/test-config/vitest.config.base.ts`
- ✅ `packages/test-config/vitest.config.svelte.ts`
- ✅ `packages/test-config/playwright.config.base.ts`
- ✅ `packages/test-config/package.json`
- ✅ `packages/test-config/README.md`
#### Test Examples (3,400+ lines)
- ✅ `docs/test-examples/backend/example.controller.spec.ts`
- ✅ `docs/test-examples/backend/example.service.spec.ts`
- ✅ `docs/test-examples/mobile/ExampleComponent.test.tsx`
- ✅ `docs/test-examples/mobile/authService.test.ts`
- ✅ `docs/test-examples/web/Button.test.ts`
- ✅ `docs/test-examples/web/page.server.test.ts`
- ✅ `docs/test-examples/shared/format.test.ts`
- ✅ `docs/test-examples/README.md`
#### CI/CD Integration
- ✅ `.github/workflows/test.yml` - 8 parallel test jobs
**Key Metrics**:
- Documentation: 50,000+ words
- Test configurations: 6 files
- Test examples: 7 files, 3,400+ lines
- Coverage target: 80% minimum, 100% critical paths
**Impact**: Complete testing infrastructure ready for implementation
---
## [0.3.0] - 2025-11-27
### Added - CI/CD Implementation & Deployment Scripts
**Delivered by**: Coder Agent
#### GitHub Actions Workflows
- ✅ `.github/workflows/ci-pull-request.yml` - PR validation
- ✅ `.github/workflows/ci-main.yml` - Main branch CI + Docker builds
- ✅ `.github/workflows/cd-staging.yml` - Staging deployment
- ✅ `.github/workflows/cd-production.yml` - Production deployment
- ✅ `.github/workflows/test-coverage.yml` - Coverage tracking
- ✅ `.github/workflows/dependency-update.yml` - Security audits
#### Docker Infrastructure
- ✅ `docker/templates/Dockerfile.nestjs` - NestJS backend template
- ✅ `docker/templates/Dockerfile.sveltekit` - SvelteKit web template
- ✅ `docker/templates/Dockerfile.astro` - Astro landing template
- ✅ `docker/nginx/nginx.conf` - Nginx configuration
- ✅ `docker-compose.staging.yml` - Staging orchestration
- ✅ `docker-compose.production.yml` - Production orchestration
- ✅ `.dockerignore` - Build optimization
#### Deployment Scripts
- ✅ `scripts/deploy/build-and-push.sh` (250 lines)
- ✅ `scripts/deploy/deploy-hetzner.sh` (300 lines)
- ✅ `scripts/deploy/health-check.sh` (150 lines)
- ✅ `scripts/deploy/rollback.sh` (200 lines)
- ✅ `scripts/deploy/migrate-db.sh` (100 lines)
#### Documentation
- ✅ `docs/CI_CD_SETUP.md` (20+ pages)
- ✅ `docs/DEPLOYMENT.md` (25+ pages)
- ✅ `docs/DOCKER_GUIDE.md` (18+ pages)
- ✅ `CI_CD_README.md` (8+ pages)
- ✅ `QUICK_START_CICD.md` (5+ pages)
**Key Metrics**:
- Workflows: 7 files, ~800 lines
- Docker templates: 3 files
- Deployment scripts: 5 files, ~1,200 lines
- Documentation: 76+ pages, 80,000+ words
**Impact**: Complete CI/CD pipeline and deployment automation ready to use
---
## [0.2.0] - 2025-11-27
### Added - Architecture Design
**Delivered by**: Analyst Agent
#### Documentation
- ✅ `docs/DEPLOYMENT_ARCHITECTURE.md` (63,000+ characters)
- ✅ `docs/DEPLOYMENT_DIAGRAMS.md` (16,000+ characters, 7 ASCII diagrams)
- ✅ `docs/DEPLOYMENT_RUNBOOKS.md` (8,000+ characters)
#### Architecture Components
- ✅ Service inventory (39 deployable services identified)
- ✅ Container strategy (multi-stage Docker builds)
- ✅ Deployment topology (blue-green, zero-downtime)
- ✅ Data architecture (separate Supabase per project)
- ✅ Network architecture (Cloudflare CDN, SSL/TLS)
- ✅ Monitoring stack (Prometheus + Grafana + Loki + Sentry)
- ✅ Disaster recovery procedures
**Key Metrics**:
- Total documentation: 87,000+ characters
- Services analyzed: 39
- Diagrams created: 7
**Impact**: Complete infrastructure architecture designed and documented
---
## [0.1.0] - 2025-11-27
### Added - Infrastructure Research
**Delivered by**: Researcher Agent
#### Research Report
- ✅ `.hive-mind/sessions/research-report-hosting-infrastructure.md` (40+ pages)
#### Analysis Completed
- ✅ Hetzner deep dive (server options, pricing, performance)
- ✅ Coolify deep dive (features, capabilities, integration)
- ✅ Comparative analysis (4 hosting options evaluated)
- ✅ Best practices research (monorepo deployment, Docker, CI/CD)
- ✅ Cost analysis (6-project deployment estimate)
- ✅ Security and compliance review (ISO 27001, GDPR)
- ✅ 9-week implementation roadmap
#### Decision Made
- ✅ **Platform**: Docker Compose + Hetzner VPS
- ✅ **Rationale**: 92% cost savings, excellent performance, flexibility
- ✅ **Estimated Cost**: $50-100/month (vs $300+ for alternatives)
- ✅ **Decision Matrix Score**: 8.40/10
**Key Metrics**:
- Research pages: 40+
- Word count: 50,000+
- Web searches: 24
- Options evaluated: 4
**Impact**: Platform decision made with strong data-driven rationale
---
## [0.0.1] - 2025-11-27 (Initial)
### Added - Hive Mind Initialization
- ✅ Initialized Hive Mind collective intelligence system
- ✅ Spawned 4 specialized worker agents:
- Researcher (infrastructure analysis)
- Analyst (architecture design)
- Coder (CI/CD implementation)
- Tester (testing strategy)
- ✅ Established consensus protocols
- ✅ Set up collective memory and coordination
**Objective**: Design complete hosting architecture and CI/CD plan for Hetzner/Docker Compose deployment
**Status**: Hive Mind operational, workers assigned
---
## Version History Summary
| Version | Date | Phase | Status | Key Deliverable |
| ------- | ---------- | ----------------- | ----------- | -------------------------- |
| 0.7.0 | 2025-11-27 | Documentation Hub | ✅ Complete | `cicd/` folder structure |
| 0.6.0 | 2025-11-27 | Registry Setup | ✅ Complete | GitHub Container Registry |
| 0.5.0 | 2025-11-27 | Final Report | ✅ Complete | Hive Mind summary |
| 0.4.0 | 2025-11-27 | Testing | ✅ Complete | Testing strategy + configs |
| 0.3.0 | 2025-11-27 | CI/CD Code | ✅ Complete | Workflows + scripts |
| 0.2.0 | 2025-11-27 | Architecture | ✅ Complete | Architecture design |
| 0.1.0 | 2025-11-27 | Research | ✅ Complete | Platform selection |
| 0.0.1 | 2025-11-27 | Initialization | ✅ Complete | Hive Mind setup |
---
## Progress Tracking
### Completed (70%)
- [x] Research and platform selection
- [x] Architecture design
- [x] CI/CD pipeline implementation
- [x] Testing strategy and infrastructure
- [x] Deployment scripts and automation
- [x] Comprehensive documentation
- [x] GitHub Container Registry setup
- [x] Documentation hub organization
### In Progress (0%)
- [ ] Infrastructure provisioning
- [ ] GitHub secrets configuration
- [ ] First deployment
- [ ] Testing implementation
### Upcoming (30%)
- [ ] Production deployment
- [ ] Monitoring setup
- [ ] Performance optimization
- [ ] Team training
---
## Key Milestones
### Milestone 1: Planning Complete ✅
**Date**: 2025-11-27
**Deliverables**: Research, architecture, planning documents
**Status**: Complete
### Milestone 2: Code Complete ✅
**Date**: 2025-11-27
**Deliverables**: Workflows, Dockerfiles, scripts, tests
**Status**: Complete
### Milestone 3: Documentation Complete ✅
**Date**: 2025-11-27
**Deliverables**: 200,000+ words of documentation
**Status**: Complete
### Milestone 4: First Deployment ⏳
**Target**: TBD
**Deliverables**: mana-core-auth deployed to staging
**Status**: Pending
### Milestone 5: Production Ready ⏳
**Target**: TBD
**Deliverables**: All services in production
**Status**: Pending
---
## Statistics
### Overall Progress
- **Phase**: Design & Planning → Implementation Pending
- **Completion**: 70%
- **Files Created**: 40+
- **Lines of Code**: ~7,300
- **Documentation Pages**: 280+
- **Word Count**: ~200,000
### By Component
| Component | Files | Lines | Status |
| -------------- | ------ | ---------- | ---------------- |
| GitHub Actions | 7 | ~800 | ✅ Complete |
| Docker | 8 | ~500 | ✅ Complete |
| Scripts | 5 | ~1,200 | ✅ Complete |
| Test Config | 6 | ~400 | ✅ Complete |
| Test Examples | 7 | ~3,400 | ✅ Complete |
| Documentation | 19 | N/A | ✅ Complete |
| **Total** | **52** | **~7,300** | **70% Complete** |
---
## Contributors
### Hive Mind Collective
- 🔍 **Researcher Agent**: Infrastructure analysis and platform selection
- 🏗️ **Analyst Agent**: Architecture design and system planning
- 💻 **Coder Agent**: CI/CD implementation and deployment automation
- 🧪 **Tester Agent**: Testing strategy and test infrastructure
- 👑 **Queen Coordinator**: Synthesis, coordination, and delivery
**Total Coordination Time**: ~2 hours
**Total Output**: 280+ pages, 40+ files, 7,300+ lines of code
---
## Notes
### Next Update
- Update when Phase 1 (Infrastructure Foundation) begins
- Track progress of TODO items
- Document any issues or blockers encountered
### Change Log Guidelines
- Update this file after each significant milestone
- Include date, version, and summary of changes
- Link to relevant documentation or code
- Track metrics and statistics
- Document decisions and rationale
---
**Last Updated**: 2025-11-27
**Next Review**: When infrastructure provisioning begins
**Status**: Planning phase complete, ready for implementation

530
cicd/COMPLETED.md
View file

@ -1,530 +0,0 @@
# CI/CD Implementation - Completed Deliverables
**Last Updated**: 2025-11-27
**Overall Progress**: 70% Complete
---
## ✅ What's Been Delivered
The Hive Mind collective intelligence system has completed the **design, planning, and code implementation** phase. All foundational code and documentation is ready for deployment.
---
## 📊 Completion Status by Phase
| Phase | Status | Progress | Notes |
| --------------------- | ----------- | -------- | --------------------------------- |
| Research & Planning | ✅ Complete | 100% | Platform selection, cost analysis |
| Documentation | ✅ Complete | 100% | 200,000+ words |
| Docker Infrastructure | ✅ Complete | 100% | Templates ready |
| GitHub Actions | ✅ Complete | 100% | 7 workflows created |
| Deployment Scripts | ✅ Complete | 100% | 5 scripts ready |
| Testing Strategy | ✅ Complete | 100% | Configurations + examples |
| Infrastructure Setup | ⏳ Pending | 0% | Awaiting server provisioning |
| Production Deployment | ⏳ Pending | 0% | Awaiting infrastructure |
---
## ✅ Research & Analysis (100%)
### Infrastructure Research
**Status**: ✅ Complete
**Delivered by**: Researcher Agent
**Deliverable**: `.hive-mind/sessions/research-report-hosting-infrastructure.md`
**What's Done**:
- [x] Comprehensive Hetzner vs Coolify analysis (24+ web searches)
- [x] Cost comparison (4 hosting options evaluated)
- [x] Performance benchmarks analyzed
- [x] Security and compliance review (ISO 27001, GDPR)
- [x] 9-week implementation roadmap created
- [x] Real-world case studies reviewed
- [x] **Decision**: Docker Compose + Hetzner VPS recommended (92% cost savings)
**Key Metrics**:
- **Pages**: 40+
- **Word Count**: 50,000+
- **Web Searches**: 24
- **Decision Matrix Score**: 8.40/10
---
### Architecture Design
**Status**: ✅ Complete
**Delivered by**: Analyst Agent
**Deliverables**: 3 comprehensive architecture documents
**What's Done**:
- [x] Complete service inventory (39 deployable services identified)
- [x] Container strategy designed (multi-stage Docker builds)
- [x] Deployment topology planned (blue-green, zero-downtime)
- [x] Data architecture designed (separate Supabase per project)
- [x] Network architecture designed (Cloudflare CDN, SSL/TLS)
- [x] Monitoring stack specified (Prometheus + Grafana + Loki + Sentry)
- [x] Disaster recovery procedures documented
**Key Deliverables**:
- [x] `docs/DEPLOYMENT_ARCHITECTURE.md` (63,000+ characters)
- [x] `docs/DEPLOYMENT_DIAGRAMS.md` (16,000+ characters - ASCII diagrams)
- [x] `docs/DEPLOYMENT_RUNBOOKS.md` (8,000+ characters)
**Key Metrics**:
- **Total Characters**: 87,000+
- **Services Analyzed**: 39
- **Diagrams Created**: 7
---
## ✅ CI/CD Implementation (100%)
### GitHub Actions Workflows
**Status**: ✅ Complete
**Delivered by**: Coder Agent
**Location**: `.github/workflows/`
**What's Done**:
- [x] `ci-pull-request.yml` - PR validation (lint, type-check, test, build)
- [x] `ci-main.yml` - Main branch CI + Docker image builds
- [x] `cd-staging.yml` - Automated staging deployment
- [x] `cd-production.yml` - Production deployment with approval gates
- [x] `test-coverage.yml` - Coverage tracking and enforcement
- [x] `dependency-update.yml` - Weekly security audits
- [x] `test.yml` - Comprehensive test automation (8 parallel jobs)
**Features Implemented**:
- [x] Smart build detection (only changed projects)
- [x] Parallel execution for speed
- [x] Coverage thresholds enforced (80% minimum)
- [x] Automated Docker image builds
- [x] GitHub Container Registry integration
- [x] Branch protection integration
- [x] PR status comments
- [x] Deployment approvals for production
**Key Metrics**:
- **Workflows Created**: 7
- **Lines of YAML**: ~800
- **Parallel Jobs**: 8
- **Estimated CI Time**: 5-10 minutes per PR
---
### Docker Infrastructure
**Status**: ✅ Complete
**Delivered by**: Coder Agent
**Location**: `docker/`
**What's Done**:
- [x] `docker/templates/Dockerfile.nestjs` - NestJS backend template
- [x] `docker/templates/Dockerfile.sveltekit` - SvelteKit web app template
- [x] `docker/templates/Dockerfile.astro` - Astro landing page template
- [x] `docker/nginx/nginx.conf` - Nginx configuration
- [x] `docker-compose.staging.yml` - Staging orchestration
- [x] `docker-compose.production.yml` - Production orchestration
- [x] `.dockerignore` - Build optimization
**Features Implemented**:
- [x] Multi-stage builds for all app types
- [x] Alpine Linux base images (minimal footprint)
- [x] Layer caching optimization
- [x] Non-root users (security)
- [x] Health checks configured
- [x] Resource limits set
- [x] Environment variable injection
- [x] pnpm workspace support
**Key Metrics**:
- **Templates Created**: 3
- **Image Size**: 120-180 MB (optimized)
- **Build Time Reduction**: 12-15 min → 2-3 min (with caching)
- **Lines of Dockerfile**: ~500
---
### Deployment Scripts
**Status**: ✅ Complete
**Delivered by**: Coder Agent
**Location**: `scripts/deploy/`
**What's Done**:
- [x] `build-and-push.sh` - Build and push Docker images (250 lines)
- [x] `deploy-hetzner.sh` - Deploy to Hetzner with zero-downtime (300 lines)
- [x] `health-check.sh` - Post-deployment health verification (150 lines)
- [x] `rollback.sh` - Emergency rollback with backup restoration (200 lines)
- [x] `migrate-db.sh` - Database migration runner (100 lines)
**Features Implemented**:
- [x] Error handling and logging
- [x] Progress indicators
- [x] Safety confirmations
- [x] Automated backups before deployment
- [x] Health check verification
- [x] Rollback capabilities
- [x] Service isolation (deploy single service or all)
- [x] Color-coded output
**Key Metrics**:
- **Scripts Created**: 5
- **Lines of Code**: ~1,200
- **Safety Checks**: 15+
- **Estimated Deployment Time**: 5-10 minutes
---
## ✅ Testing Infrastructure (100%)
### Test Configuration Package
**Status**: ✅ Complete
**Delivered by**: Tester Agent
**Location**: `packages/test-config/`
**What's Done**:
- [x] `jest.config.backend.js` - NestJS backend configuration
- [x] `jest.config.mobile.js` - React Native mobile configuration
- [x] `vitest.config.base.ts` - Shared packages configuration
- [x] `vitest.config.svelte.ts` - SvelteKit web configuration
- [x] `playwright.config.base.ts` - E2E testing configuration
- [x] `package.json` - Package manifest
- [x] `tsconfig.json` - TypeScript configuration
- [x] `README.md` - Usage documentation
**Features Implemented**:
- [x] 80% coverage thresholds enforced
- [x] Auto-clear/restore/reset mocks
- [x] Platform-specific transforms
- [x] Coverage reporters configured
- [x] Module path aliases
- [x] TypeScript support
**Key Metrics**:
- **Configurations Created**: 6
- **Lines of Code**: ~400
- **Coverage Target**: 80% (100% for critical paths)
---
### Test Examples
**Status**: ✅ Complete
**Delivered by**: Tester Agent
**Location**: `docs/test-examples/`
**What's Done**:
- [x] `backend/example.controller.spec.ts` - NestJS controller tests (300 lines)
- [x] `backend/example.service.spec.ts` - NestJS service tests (400 lines)
- [x] `mobile/ExampleComponent.test.tsx` - React Native component tests (450 lines)
- [x] `mobile/authService.test.ts` - React Native service tests (400 lines)
- [x] `web/Button.test.ts` - Svelte 5 component tests (350 lines)
- [x] `web/page.server.test.ts` - SvelteKit server tests (500 lines)
- [x] `shared/format.test.ts` - Utility function tests (400 lines)
- [x] `README.md` - Examples guide (600 lines)
**Key Metrics**:
- **Example Files**: 7
- **Lines of Code**: ~3,400
- **Scenarios Covered**: 100+
- **Production-Ready**: Yes ✅
---
### Testing Strategy Documentation
**Status**: ✅ Complete
**Delivered by**: Tester Agent
**Location**: `docs/`
**What's Done**:
- [x] `TESTING.md` - Master testing strategy (35,000+ words, 2,850 lines)
- [x] `TESTING_IMPLEMENTATION_GUIDE.md` - Developer quick start (8,000+ words)
- [x] `TESTING_SUMMARY.md` - Executive summary (7,000+ words)
**Content Includes**:
- [x] Complete testing infrastructure for all app types
- [x] Test organization patterns and conventions
- [x] Coverage strategy (80% minimum, 100% critical paths)
- [x] Detailed testing scenarios with code examples
- [x] CI/CD integration guide
- [x] 14-week implementation roadmap
- [x] Best practices and troubleshooting
**Key Metrics**:
- **Total Words**: 50,000+
- **Total Lines**: 5,166
- **Code Examples**: 100+
---
## ✅ Documentation (100%)
### CI/CD Documentation
**Status**: ✅ Complete
**Delivered by**: Coder Agent
**What's Done**:
- [x] `QUICK_START_CICD.md` - 30-minute fast track (5+ pages)
- [x] `CI_CD_README.md` - High-level overview (8+ pages)
- [x] `docs/CI_CD_SETUP.md` - Complete setup guide (20+ pages)
- [x] `docs/DEPLOYMENT.md` - Deployment operations (25+ pages)
- [x] `docs/DOCKER_GUIDE.md` - Docker deep dive (18+ pages)
- [x] `CI_CD_IMPLEMENTATION_SUMMARY.md` - Implementation summary
- [x] `FILES_CREATED.md` - File inventory
**Key Metrics**:
- **Pages Created**: 76+
- **Word Count**: 80,000+
- **Screenshots/Diagrams**: Embedded ASCII art
---
### GitHub Container Registry Setup
**Status**: ✅ Complete
**Delivered by**: Queen Coordinator
**Deliverable**: `DOCKER_REGISTRY_SETUP.md`
**What's Done**:
- [x] GitHub Container Registry (ghcr.io) configuration
- [x] Workflows updated to use ghcr.io
- [x] Team access documentation
- [x] Troubleshooting guide
- [x] Comparison table (Docker Hub vs ghcr.io)
- [x] Auto-cleanup workflow example
**Why ghcr.io**:
- [x] No additional signup needed
- [x] Automatic authentication with GITHUB_TOKEN
- [x] Unlimited private images (500 MB free tier)
- [x] No rate limits
- [x] Automatic team access
---
### Hive Mind Final Report
**Status**: ✅ Complete
**Delivered by**: Queen Coordinator
**Deliverable**: `HIVE_MIND_FINAL_REPORT.md`
**What's Done**:
- [x] Executive summary of all work
- [x] Worker agent reports consolidated
- [x] Consensus decisions documented
- [x] Implementation roadmap
- [x] Cost analysis and recommendations
- [x] Success metrics defined
- [x] Troubleshooting index
- [x] File location appendix
**Key Metrics**:
- **Pages**: 40+
- **Word Count**: 30,000+
- **Deliverables Indexed**: 60+
---
## ✅ Configuration Files (100%)
### Root Configuration
**Status**: ✅ Complete
**What's Done**:
- [x] `vitest.config.ts` - Root Vitest configuration
- [x] `jest.config.js` - Multi-project Jest configuration
- [x] `playwright.config.ts` - E2E testing configuration
- [x] `.dockerignore` - Build optimization
---
## 📊 Statistics Summary
### Code & Configuration
- **Total Files Created**: 40+
- **Total Lines of Code**: ~7,300
- **GitHub Actions Workflows**: 7
- **Dockerfile Templates**: 3
- **Deployment Scripts**: 5
- **Test Configurations**: 6
- **Test Examples**: 7
### Documentation
- **Total Pages**: 236+
- **Total Word Count**: ~200,000
- **Documentation Files**: 19
- **Diagrams**: 7 ASCII diagrams
### Coverage
- **Projects Analyzed**: 10
- **Services Identified**: 39
- **Apps Covered**: Backend, Mobile, Web, Landing
- **Frameworks Documented**: NestJS, Expo, SvelteKit, Astro
---
## ⏳ What's Not Done (Awaiting Implementation)
### Infrastructure Setup (0%)
- [ ] Hetzner account creation
- [ ] Server provisioning
- [ ] Coolify installation
- [ ] Domain configuration
- [ ] SSL/TLS setup
**Why Not Done**: Requires budget approval and account setup
---
### Secrets Configuration (0%)
- [ ] GitHub secrets configured
- [ ] Supabase credentials added
- [ ] JWT secrets generated
- [ ] SSH keys configured
**Why Not Done**: Requires infrastructure to be provisioned first
---
### Deployment (0%)
- [ ] First Dockerfile created (service-specific)
- [ ] First deployment to staging
- [ ] Production deployment
- [ ] Full service rollout
**Why Not Done**: Requires infrastructure and secrets first
---
### Testing Implementation (0%)
- [ ] Critical path tests written (auth, payments)
- [ ] Backend tests (80% coverage)
- [ ] Frontend tests (80% coverage)
- [ ] E2E tests
**Why Not Done**: Can be done in parallel with deployment
---
### Monitoring Setup (0%)
- [ ] Prometheus installed
- [ ] Grafana configured
- [ ] Loki for logging
- [ ] Sentry for error tracking
- [ ] Alerting configured
**Why Not Done**: Requires production deployment first
---
## 🎯 Ready for Next Phase
**All prerequisites for implementation are complete**:
- ✅ Platform selected (Docker Compose + Hetzner VPS)
- ✅ Architecture designed and documented
- ✅ Code templates ready to use
- ✅ Workflows configured and tested
- ✅ Deployment scripts ready
- ✅ Testing strategy defined
- ✅ Documentation comprehensive
**Next Steps**:
1. Review `cicd/TODO.md` for actionable tasks
2. Follow `cicd/SETUP.md` for step-by-step guide
3. Start with Phase 1: Infrastructure Foundation
4. Estimated time to first deployment: 30 minutes
---
## 🏆 Quality Metrics
### Code Quality
- ✅ Error handling implemented
- ✅ Logging and progress indicators
- ✅ Safety checks and confirmations
- ✅ Production-ready patterns
### Documentation Quality
- ✅ Comprehensive and detailed
- ✅ Step-by-step instructions
- ✅ Troubleshooting sections
- ✅ Code examples included
- ✅ Best practices documented
### Security
- ✅ Non-root Docker users
- ✅ Secrets management via GitHub
- ✅ SSH key-based authentication
- ✅ SSL/TLS for all services
- ✅ Network segmentation designed
- ✅ Firewall rules specified
---
## 📝 Notes
**Delivered by**: Hive Mind Collective Intelligence
- 🔍 Researcher Agent: Infrastructure analysis
- 🏗️ Analyst Agent: Architecture design
- 💻 Coder Agent: CI/CD implementation
- 🧪 Tester Agent: Testing strategy
- 👑 Queen Coordinator: Synthesis and delivery
**Total Coordination Time**: ~2 hours
**Total Deliverable Size**: 280+ pages, 40+ files
**Status**: Ready for implementation ✅
---
**Last Updated**: 2025-11-27
**Phase**: Design & Planning Complete → Ready for Implementation
**Next Milestone**: First deployment to staging

633
cicd/DEPLOYMENT.md
View file

@ -1,633 +0,0 @@
# Deployment Guide
This guide covers deployment procedures for the manacore-monorepo, including tag-based CI/CD deployments and manual procedures.
---
## Table of Contents
1. [CI/CD Architecture Overview](#cicd-architecture-overview)
2. [Tag-Based Deployment (Recommended)](#tag-based-deployment-recommended)
3. [Manual Workflow Dispatch](#manual-workflow-dispatch)
4. [Supported Projects](#supported-projects)
5. [Deployment Verification](#deployment-verification)
6. [Rollback Procedures](#rollback-procedures)
7. [Troubleshooting](#troubleshooting)
---
## CI/CD Architecture Overview
The CI/CD system uses **4 GitHub Actions workflows** that work together:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ CI/CD PIPELINE OVERVIEW │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ [Developer] │
│ │ │
│ ├─── Push to dev/main ──────► ci.yml (validate + build images) │
│ │ │
│ ├─── Push tag ──────────────► cd-staging-tagged.yml (deploy) │
│ │ (chat-staging-v1.0.0) │
│ │ │
│ ├─── Manual trigger ────────► cd-staging.yml (deploy all) │
│ │ │
│ └─── Manual + approval ─────► cd-production.yml (deploy prod) │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
### Workflow Files
| Workflow | File | Trigger | Purpose |
|----------|------|---------|---------|
| **CI** | `ci.yml` | Push/PR to dev/main | Validate code, build Docker images |
| **CD Staging (Tagged)** | `cd-staging-tagged.yml` | Push tag `*-staging-v*` | Deploy specific project to staging |
| **CD Staging** | `cd-staging.yml` | Manual dispatch | Deploy all services to staging |
| **CD Production** | `cd-production.yml` | Manual + approval | Deploy to production |
---
### 1. CI Workflow (`ci.yml`)
**Triggers:**
- Push to `dev` or `main` branches
- Pull requests to `dev` or `main`
**Jobs:**
```
┌──────────────────────────────────────────────────────────────┐
│ ci.yml │
├──────────────────────────────────────────────────────────────┤
│ │
│ [On PR] │
│ └── validate │
│ ├── Checkout code │
│ ├── Install dependencies (pnpm) │
│ ├── Type check (pnpm run type-check) │
│ └── Lint (pnpm run lint) │
│ │
│ [On Push to dev/main] │
│ └── build-docker-images (matrix) │
│ ├── mana-core-auth │
│ ├── chat-backend, chat-web │
│ ├── manacore-web │
│ ├── todo-backend, todo-web │
│ ├── calendar-backend, calendar-web │
│ └── clock-backend, clock-web │
│ │
│ For each service: │
│ ├── Check if Dockerfile exists │
│ ├── Login to GitHub Container Registry (ghcr.io) │
│ ├── Build Docker image │
│ └── Push to ghcr.io/memo-2023/{service}:latest │
│ │
└──────────────────────────────────────────────────────────────┘
```
**Key Points:**
- PRs run validation only (type-check + lint)
- Pushes build and push Docker images to `ghcr.io`
- Images are tagged as `:latest` on push
- **Does NOT auto-deploy** - deployment requires separate trigger
---
### 2. CD Staging Tagged (`cd-staging-tagged.yml`)
**Triggers:**
- Push tag matching `*-staging-v*` or `*-v*-staging`
- Manual workflow dispatch
**Jobs:**
```
┌──────────────────────────────────────────────────────────────┐
│ cd-staging-tagged.yml │
├──────────────────────────────────────────────────────────────┤
│ │
│ [parse-deployment] │
│ ├── Parse tag: chat-staging-v1.0.0 │
│ │ └── Extract: project=chat, version=v1.0.0 │
│ └── Generate build matrix based on project │
│ │
│ [build] (parallel matrix) │
│ ├── Build Docker image for each app │
│ ├── Tag with version (e.g., v1.0.0) │
│ ├── Tag with staging-latest │
│ └── Push to ghcr.io │
│ │
│ [deploy] (parallel matrix) │
│ ├── SSH to staging server (46.224.108.214) │
│ ├── Sync docker-compose.staging.yml │
│ ├── Pull new images │
│ ├── Update .env with version │
│ ├── docker compose up -d --force-recreate │
│ └── Health check │
│ │
│ [notify] │
│ └── Report success/failure │
│ │
└──────────────────────────────────────────────────────────────┘
```
**Image Tags Created:**
- `ghcr.io/memo-2023/{service}:v1.0.0` (version)
- `ghcr.io/memo-2023/{service}:staging-latest`
- `ghcr.io/memo-2023/{service}:staging-{sha}`
---
### 3. CD Staging (`cd-staging.yml`)
**Triggers:**
- Manual workflow dispatch only
**Jobs:**
```
┌──────────────────────────────────────────────────────────────┐
│ cd-staging.yml │
├──────────────────────────────────────────────────────────────┤
│ │
│ [deploy-staging] │
│ ├── SSH to staging server │
│ ├── Copy docker-compose.staging.yml │
│ ├── Copy environment variables from secrets │
│ ├── Login to ghcr.io on server │
│ ├── docker compose pull │
│ ├── docker compose up -d │
│ ├── Create databases (if not exist) │
│ ├── Run database migrations │
│ └── Health checks (with retry polling) │
│ │
│ [notify-deployment] │
│ └── Report success/failure │
│ │
└──────────────────────────────────────────────────────────────┘
```
**Use Case:** Deploy all services at once, useful for:
- Initial infrastructure setup
- Re-deploying after server issues
- Syncing all services to latest
---
### 4. CD Production (`cd-production.yml`)
**Triggers:**
- Manual workflow dispatch with confirmation
- Must type "deploy" to confirm
- Must be on `main` branch
**Jobs:**
```
┌──────────────────────────────────────────────────────────────┐
│ cd-production.yml │
├──────────────────────────────────────────────────────────────┤
│ │
│ [validate-deployment] │
│ ├── Verify confirmation = "deploy" │
│ ├── Verify branch = main │
│ └── Check recent CI passes │
│ │
│ [request-approval] │
│ └── GitHub environment approval gate │
│ │
│ [create-backup] │
│ ├── Backup PostgreSQL (pg_dumpall) │
│ ├── Backup Redis │
│ ├── Backup docker-compose and .env │
│ └── Tag current deployment │
│ │
│ [deploy-production] │
│ ├── Copy deployment files │
│ ├── Update environment variables │
│ ├── Pull latest images │
│ ├── Run database migrations │
│ ├── Zero-downtime rolling update │
│ └── Verify deployment │
│ │
│ [post-deployment-checks] │
│ ├── Smoke tests on endpoints │
│ └── Monitor for 5 minutes │
│ │
│ [notify-deployment] │
│ └── Report success/failure │
│ │
└──────────────────────────────────────────────────────────────┘
```
**Safety Features:**
- Requires typing "deploy" to confirm
- Must be on `main` branch
- Creates backup before deployment
- Environment approval gate
- Zero-downtime rolling updates
- 5-minute post-deployment monitoring
---
### Typical Deployment Flow
```
1. Developer pushes code to dev branch
└── ci.yml: validates code, builds images
2. Code merged to main via PR
└── ci.yml: builds and pushes :latest images
3. Ready to deploy to staging
└── Developer creates tag: git tag chat-staging-v1.2.0 && git push origin chat-staging-v1.2.0
└── cd-staging-tagged.yml: builds versioned images, deploys to staging
4. Testing on staging...
5. Ready for production
└── Manually trigger cd-production.yml
└── Type "deploy" to confirm
└── Approve in GitHub environment
└── Automatic backup + deploy + monitoring
```
---
### Docker Registry
All images are stored in **GitHub Container Registry (ghcr.io)**:
```
ghcr.io/memo-2023/
├── mana-core-auth:latest
├── mana-core-auth:staging-latest
├── mana-core-auth:v1.0.0
├── chat-backend:latest
├── chat-backend:staging-latest
├── chat-backend:v1.2.0
├── chat-web:latest
└── ...
```
**Tag Conventions:**
- `:latest` - Built on push to dev/main (ci.yml)
- `:staging-latest` - Latest staging deployment
- `:v{version}` - Specific version from tag
- `:staging-{sha}` - Git SHA for traceability
---
### Required Secrets
For the workflows to function, these GitHub secrets must be configured:
| Secret | Used By | Purpose |
|--------|---------|---------|
| `STAGING_SSH_KEY` | cd-staging*.yml | SSH private key for staging server |
| `STAGING_POSTGRES_PASSWORD` | cd-staging.yml | PostgreSQL password |
| `STAGING_REDIS_PASSWORD` | cd-staging.yml | Redis password |
| `STAGING_JWT_SECRET` | cd-staging.yml | JWT signing secret |
| `STAGING_JWT_PUBLIC_KEY` | cd-staging.yml | JWT public key (EdDSA) |
| `STAGING_JWT_PRIVATE_KEY` | cd-staging.yml | JWT private key (EdDSA) |
| `STAGING_SUPABASE_*` | cd-staging.yml | Supabase credentials |
| `STAGING_AZURE_OPENAI_*` | cd-staging.yml | Azure OpenAI credentials |
| `PRODUCTION_*` | cd-production.yml | Production equivalents |
---
## Tag-Based Deployment (Recommended)
The primary deployment method uses git tags to trigger the CI/CD pipeline. When you push a tag matching the expected pattern, GitHub Actions automatically builds and deploys to staging.
### Tag Format
```
{project}-staging-v{version} # Deploy backend only
{project}-v{version}-staging # Deploy backend only (alternative)
{project}-all-staging-v{version} # Deploy ALL apps (backend + web + landing)
```
### Quick Reference
| What to deploy | Tag format | Example |
|----------------|------------|---------|
| Backend only | `{project}-staging-v{version}` | `chat-staging-v1.2.0` |
| All apps | `{project}-all-staging-v{version}` | `chat-all-staging-v1.0.0` |
| Auth service | `mana-core-auth-staging-v{version}` | `mana-core-auth-staging-v1.0.0` |
### Examples
#### Deploy a single backend
```bash
# Deploy chat backend v1.2.0 to staging
git tag chat-staging-v1.2.0
git push origin chat-staging-v1.2.0
# Deploy zitare backend v2.1.0 to staging
git tag zitare-staging-v2.1.0
git push origin zitare-staging-v2.1.0
# Deploy mana-core-auth v1.0.0 to staging
git tag mana-core-auth-staging-v1.0.0
git push origin mana-core-auth-staging-v1.0.0
```
#### Deploy all apps for a project
```bash
# Deploy chat backend + web + landing
git tag chat-all-staging-v1.0.0
git push origin chat-all-staging-v1.0.0
# Deploy picture backend + web + landing
git tag picture-all-staging-v2.0.0
git push origin picture-all-staging-v2.0.0
```
#### One-liner (create and push)
```bash
git tag chat-staging-v1.2.0 && git push origin chat-staging-v1.2.0
```
### Version Numbering
Use semantic versioning (SemVer):
- **Major** (`v2.0.0`): Breaking changes
- **Minor** (`v1.2.0`): New features, backward compatible
- **Patch** (`v1.2.1`): Bug fixes
For hotfixes, append a suffix:
```bash
git tag chat-staging-v1.2.1-hotfix
```
---
## Manual Workflow Dispatch
You can also trigger deployments manually from the GitHub Actions UI.
### Steps
1. Go to **Actions** tab in GitHub
2. Select **"CD - Staging (Tagged Releases)"** workflow
3. Click **"Run workflow"**
4. Fill in the parameters:
- **Project**: Select from dropdown (chat, picture, zitare, etc.)
- **Apps**: Comma-separated list or "all" (e.g., `backend,web` or `all`)
- **Version**: Optional version tag (e.g., `v1.2.0` or `latest`)
5. Click **"Run workflow"**
### When to use manual dispatch
- Testing deployment without creating a permanent tag
- Deploying specific app combinations (e.g., only `web,landing`)
- Re-deploying after fixing infrastructure issues
---
## Supported Projects
| Project | Available Apps | Backend Port | Notes |
|---------|---------------|--------------|-------|
| `chat` | backend, web, landing | 3002 | AI chat application |
| `picture` | backend, web, landing | 3006 | AI image generation |
| `manadeck` | backend, web | 3009 | Card/deck management |
| `zitare` | backend, web, landing | 3007 | Daily inspiration quotes |
| `presi` | backend, web | 3008 | Presentation tool |
| `mana-core-auth` | service | 3001 | Central authentication |
| `manacore` | web | 5173 | Main ecosystem web app |
| `todo` | backend, web | 3018 | Todo application |
| `calendar` | backend, web | 3016 | Calendar application |
| `clock` | backend, web | 3017 | Clock/time application |
---
## Deployment Verification
### Check GitHub Actions status
```bash
# List recent workflow runs
gh run list --workflow="CD - Staging (Tagged Releases)" --limit 5
# View specific run details
gh run view <run-id>
# Watch a running deployment
gh run watch <run-id>
```
### Verify deployment on server
```bash
# SSH to staging server
ssh -i ~/.ssh/hetzner_deploy_key deploy@46.224.108.214
# Check running containers
docker ps | grep chat
# Check container logs
docker logs chat-backend-staging --tail 100
# Health check
curl -s http://localhost:3002/api/v1/health | jq
```
### Verify via public URL
```bash
# Health check endpoints
curl -s https://staging-api-chat.mana.how/api/v1/health | jq
curl -s https://staging-api-zitare.mana.how/api/v1/health | jq
```
---
## Rollback Procedures
### Option 1: Deploy previous version tag
```bash
# Create a new tag pointing to the previous version
git tag chat-staging-v1.1.0-rollback <previous-commit-sha>
git push origin chat-staging-v1.1.0-rollback
```
### Option 2: Re-tag previous commit
```bash
# Find the previous working commit
git log --oneline -10
# Create tag at that commit
git tag chat-staging-v1.1.1 abc1234
git push origin chat-staging-v1.1.1
```
### Option 3: Manual rollback on server
```bash
# SSH to staging
ssh deploy@46.224.108.214
# Pull previous image version
cd ~/manacore-staging
docker pull ghcr.io/memo-2023/chat-backend:v1.1.0
# Update .env with previous version
sed -i 's/CHAT_VERSION=.*/CHAT_VERSION=v1.1.0/' .env
# Restart with previous version
docker compose up -d --force-recreate chat-backend
```
---
## Troubleshooting
### Tag already exists
```bash
# Delete local tag
git tag -d chat-staging-v1.2.0
# Delete remote tag
git push origin --delete chat-staging-v1.2.0
# Re-create tag
git tag chat-staging-v1.2.0
git push origin chat-staging-v1.2.0
```
### Workflow not triggered
Check that your tag matches the expected pattern:
- Must contain `-staging-v` or `-v{version}-staging`
- Project name must be valid (see supported projects)
```bash
# Verify tag format
echo "chat-staging-v1.2.0" | grep -E '^.+-staging-v.+$|^.+-v.+-staging$'
```
### Build failed
```bash
# Check workflow logs
gh run view <run-id> --log-failed
# Common issues:
# - Dockerfile not found: Check path in workflow matrix
# - Build context error: Ensure dependencies are in place
# - Type errors: Run `pnpm type-check` locally first
```
### Deployment succeeded but service unhealthy
```bash
# SSH to server and check logs
ssh deploy@46.224.108.214
docker logs chat-backend-staging --tail 200
# Common issues:
# - Missing environment variables
# - Database connection failed
# - Port conflicts
```
### Container name conflict error
If you see: `Error: The container name "/xxx-staging" is already in use`
```bash
# SSH to server and remove the stale container
ssh deploy@46.224.108.214
docker rm -f todo-web-staging # Replace with actual container name
# Then re-run the deployment
```
This can happen if a container was created outside of docker-compose or if a previous deployment failed mid-way.
---
## Managing Tags
### List existing tags
```bash
# All staging tags
git tag -l "*-staging-*"
# Tags for specific project
git tag -l "chat-staging-*"
# Show tag with commit info
git show chat-staging-v1.2.0
```
### Delete tags
```bash
# Local only
git tag -d chat-staging-v1.2.0
# Remote only
git push origin --delete chat-staging-v1.2.0
# Both
git tag -d chat-staging-v1.2.0 && git push origin --delete chat-staging-v1.2.0
```
### Rename a tag
```bash
# Create new tag at same commit
git tag chat-staging-v1.2.1 chat-staging-v1.2.0
# Delete old tag
git tag -d chat-staging-v1.2.0
git push origin --delete chat-staging-v1.2.0
# Push new tag
git push origin chat-staging-v1.2.1
```
---
## Best Practices
1. **Always test locally first**: Run `pnpm build` and `pnpm type-check` before tagging
2. **Use meaningful versions**: Follow SemVer and increment appropriately
3. **Tag from main/dev branch**: Avoid tagging feature branches
4. **Monitor deployments**: Watch the GitHub Actions run until completion
5. **Verify after deploy**: Always check health endpoints after deployment
6. **Document releases**: Update CHANGELOG.md for significant releases
---
## Related Documentation
### CI/CD Files
- [/.github/workflows/ci.yml](/.github/workflows/ci.yml) - CI workflow (validate + build)
- [/.github/workflows/cd-staging-tagged.yml](/.github/workflows/cd-staging-tagged.yml) - Tag-based staging deployment
- [/.github/workflows/cd-staging.yml](/.github/workflows/cd-staging.yml) - Manual staging deployment
- [/.github/workflows/cd-production.yml](/.github/workflows/cd-production.yml) - Production deployment
### Other Documentation
- [SETUP.md](./SETUP.md) - Initial CI/CD setup
- [TODO.md](./TODO.md) - Remaining implementation tasks
- [/docs/DEPLOYMENT_RUNBOOKS.md](/docs/DEPLOYMENT_RUNBOOKS.md) - Detailed operational procedures
- [/docker-compose.staging.yml](/docker-compose.staging.yml) - Staging Docker Compose config
---
**Last Updated**: 2025-12-09

288
cicd/IMMEDIATE_FIXES_NEEDED.md
View file

@ -1,288 +0,0 @@
# CI/CD Immediate Fixes Needed
**Date**: 2025-11-27
**Status**: CI/CD pipeline running but failing
**PR**: cicd/integration branch
---
## ✅ What's Working
- GitHub Actions workflows are triggering correctly
- mana-core-auth Dockerfile builds locally
- Docker Registry (ghcr.io) configured correctly
- All CI/CD documentation created
---
## ❌ Current Failures
### 1. Docker Build Path Issues
**Error**:
```
ERROR: failed to solve: failed to calculate checksum of ref: "/backend": not found
```
**Problem**:
- Dockerfiles in `apps/*/apps/backend/Dockerfile` use wrong path references
- They copy `COPY backend/package.json` but should copy from monorepo root context
- Example: `apps/maerchenzauber/apps/backend/Dockerfile:33`
**Fix**:
Update all backend Dockerfiles to use correct paths:
```dockerfile
# WRONG (current)
COPY backend/package.json ./backend/package.json
# CORRECT
COPY apps/maerchenzauber/apps/backend/package.json ./backend/package.json
```
**Files to update**:
- `apps/maerchenzauber/apps/backend/Dockerfile`
- `apps/manadeck/apps/backend/Dockerfile`
- `apps/chat/apps/backend/Dockerfile`
- `apps/picture/apps/backend/Dockerfile`
- `apps/nutriphi/apps/backend/Dockerfile`
---
### 2. Private Repo Access (mana-core-nestjs-package)
**Error**:
```
git clone failed: exit code 128
Repository: github.com/Memo-2023/mana-core-nestjs-package.git
```
**Problem**:
- Private repo being cloned in Dockerfile needs GitHub token
- GitHub Actions doesn't have access configured
**SOLUTION DOCUMENTATION**: See **[cicd/PRIVATE_REPO_ACCESS_SOLUTION.md](PRIVATE_REPO_ACCESS_SOLUTION.md)** for complete implementation guide
**Quick Fix (3 steps, 10 minutes)**:
1. Create GitHub Personal Access Token with read access to `mana-core-nestjs-package`
2. Add secret to repo as `PRIVATE_REPO_TOKEN`
3. Update `.github/workflows/ci-main.yml` build step to include:
```yaml
secrets: |
github_token=${{ secrets.PRIVATE_REPO_TOKEN }}
```
**Why this works**: Dockerfiles already have secret mounting logic, they just need the token passed from CI.
**Alternative Options** (see full doc for details):
- Make repo public (not recommended for proprietary code)
- Publish as npm package (future enhancement)
---
### 3. Lint & Format Errors
**Error**:
```
Lint & Format Check: Process completed with exit code 1
```
**Fix**:
```bash
# Run locally to see errors
pnpm run format
pnpm run lint
# Auto-fix most issues
pnpm run format:write
pnpm run lint --fix
# Commit fixes
git add .
git commit -m "fix: auto-format and lint fixes"
```
---
### 4. Type Check Errors
**Error**:
```
Type Check: Process completed with exit code 1
```
**Fix**:
```bash
# See all type errors
pnpm run type-check
# Fix one project at a time
pnpm --filter @maerchenzauber/backend run type-check
pnpm --filter @chat/backend run type-check
# Common fixes:
# - Add missing type imports
# - Fix any/unknown types
# - Update outdated type definitions
```
---
### 5. Test Failures
**Error**:
```
Multiple test suites failing
```
**Fix**:
```bash
# Run tests locally
pnpm test
# Fix per project
pnpm --filter @maerchenzauber/backend test
pnpm --filter @memoro/mobile test
# Common issues:
# - Missing test environment setup
# - Outdated snapshots
# - Missing mocks
```
---
### 6. Build Failures
**Error**:
```
Build Projects: Process completed with exit code 1
```
**Likely causes**:
- Type errors (fix those first)
- Missing dependencies
- Circular dependencies
- Build script errors
**Fix**:
```bash
# Build locally to see errors
pnpm run build
# Or build specific projects
pnpm --filter @maerchenzauber/backend build
```
---
## 🎯 Recommended Fix Order
**Priority 1 (Blocker)**: Docker Build Paths
1. Fix path references in all backend Dockerfiles
2. Test builds locally: `docker build -t test -f apps/PROJECT/apps/backend/Dockerfile .`
3. Commit and push
**Priority 2 (Blocker)**: Private Repo Access
1. Either make mana-core-nestjs-package public
2. Or add GitHub token secrets to CI workflow
3. Test Docker build includes the package
**Priority 3**: Lint & Format
1. Run `pnpm run format:write` and `pnpm run lint --fix`
2. Manually fix remaining issues
3. Commit fixes
**Priority 4**: Type Errors
1. Run `pnpm run type-check` to see all errors
2. Fix project by project
3. Common patterns will emerge
**Priority 5**: Tests
1. Can be fixed in parallel with builds
2. Many may pass once builds work
3. Some tests may need updating
---
## 🚀 Quick Commands
```bash
# Fix formatting and lint
pnpm run format:write
pnpm run lint --fix
# Check what's broken
pnpm run type-check
pnpm run build
pnpm test
# Test Docker builds locally
docker build -t test-maerchenzauber -f apps/maerchenzauber/apps/backend/Dockerfile .
# Check CI status
# Go to: https://github.com/Memo-2023/manacore-monorepo/actions
```
---
## 📝 After All Fixes
Once all checks pass:
1. ✅ All Docker images will auto-build and push to ghcr.io
2. ✅ Can proceed with Hetzner setup
3. ✅ Can deploy to staging
4. ✅ Can deploy to production
---
## 🆘 Key Resources
- **CI/CD Docs**: `cicd/README.md`
- **Setup Guide**: `cicd/SETUP.md`
- **Hive Mind Report**: `HIVE_MIND_FINAL_REPORT.md`
- **GitHub Actions**: `.github/workflows/`
- **Docker Templates**: `docker/templates/`
---
## 📊 Current Status
| Check | Status | Priority |
| ------------------- | ---------- | --------- |
| Docker Builds | ❌ Failing | 🔥 HIGH |
| Private Repo Access | ❌ Failing | 🔥 HIGH |
| Lint & Format | ❌ Failing | ⚠️ MEDIUM |
| Type Check | ❌ Failing | ⚠️ MEDIUM |
| Tests | ❌ Failing | ⚠️ MEDIUM |
| Build | ❌ Failing | ⚠️ MEDIUM |
**Target**: All ✅ GREEN
---
**Estimated Time**: 2-4 hours to fix all issues
**Best Approach**: Fix Docker builds first, then everything else in parallel

182
cicd/IMPLEMENTATION_CHECKLIST.md
View file

@ -1,182 +0,0 @@
# Private Repo Access - Implementation Checklist
**Issue**: Docker builds fail due to private repo `mana-core-nestjs-package` access
**Solution**: Pass GitHub token to Docker builds via secrets
**Time Required**: 10-15 minutes
---
## Checklist
### Step 1: Create Personal Access Token (PAT)
- [ ] Go to GitHub Settings → Developer Settings → Personal Access Tokens → Fine-grained tokens
- [ ] Click "Generate new token"
- [ ] Set token name: `CI-Private-Repo-Access`
- [ ] Set expiration: 1 year
- [ ] Repository access: "Only select repositories" → `Memo-2023/mana-core-nestjs-package`
- [ ] Permissions:
- [ ] Repository permissions → Contents → Read
- [ ] Repository permissions → Metadata → Read (auto-selected)
- [ ] Click "Generate token"
- [ ] Copy token (starts with `github_pat_...`)
**Calendar Reminder**: Set reminder 1 week before token expires to rotate
---
### Step 2: Add Secret to GitHub Repository
- [ ] Navigate to `https://github.com/Memo-2023/manacore-monorepo/settings/secrets/actions`
- [ ] Click "New repository secret"
- [ ] Name: `PRIVATE_REPO_TOKEN`
- [ ] Value: Paste the PAT from Step 1
- [ ] Click "Add secret"
**Verification**: Secret should appear in the list (value is hidden)
---
### Step 3: Update CI Workflow
- [ ] Open `.github/workflows/ci-main.yml`
- [ ] Find the "Build and push" step (around line 126)
- [ ] Add these 2 lines after `build-args:` section:
```yaml
secrets: |
github_token=${{ secrets.PRIVATE_REPO_TOKEN }}
```
**Complete example**:
```yaml
- name: Build and push
if: steps.check-dockerfile.outputs.exists == 'true'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.service.path }}/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
NODE_ENV=production
PORT=${{ matrix.service.port }}
secrets: |
github_token=${{ secrets.PRIVATE_REPO_TOKEN }}
```
- [ ] Save file
- [ ] Commit: `git commit -m "ci: add GitHub token secret for private repo access"`
---
### Step 4: Verify Implementation
**Local Testing** (Optional):
```bash
# Export token
export GITHUB_TOKEN="github_pat_YOUR_TOKEN"
# Test maerchenzauber
docker build \
--secret id=github_token,env=GITHUB_TOKEN \
-t test-maerchenzauber \
-f apps/maerchenzauber/apps/backend/Dockerfile \
.
# Test manadeck
docker build \
--secret id=github_token,env=GITHUB_TOKEN \
-t test-manadeck \
-f apps/manadeck/apps/backend/Dockerfile \
.
```
- [ ] Local builds succeed (if testing locally)
**CI Testing**:
- [ ] Push changes to test branch
- [ ] Check GitHub Actions workflow run
- [ ] Verify build logs show: "Using GitHub token for private repo access"
- [ ] Verify no "exit code 128" errors
- [ ] Verify Docker images pushed to ghcr.io
---
## Success Criteria
- ✅ GitHub secret `PRIVATE_REPO_TOKEN` exists
- ✅ CI workflow file updated with secrets section
- ✅ Build logs show "Using GitHub token for private repo access"
- ✅ Git clone succeeds without authentication errors
- ✅ Tarball creation succeeds: "Mana-core packaged as tarball"
- ✅ Docker images build and push successfully
- ✅ Both maerchenzauber-backend and manadeck-backend images available in ghcr.io
---
## Troubleshooting
**Problem**: "No GitHub token provided" in logs
- **Fix**: Check secret exists and workflow syntax is correct
**Problem**: "git clone failed: exit code 128"
- **Fix**: Token expired or has wrong permissions - regenerate PAT
**Problem**: "Permission denied (publickey)"
- **Fix**: Check Dockerfile uses HTTPS URLs, not SSH
---
## What This Fixes
**Affected Services**:
- ✅ apps/maerchenzauber/apps/backend
- ✅ apps/manadeck/apps/backend
**Impact**:
- Docker builds will succeed in CI
- Images will auto-push to ghcr.io
- Unblocks deployment to staging/production
- No changes needed to local development workflow
---
## Maintenance
**Token Rotation** (Annual):
1. Generate new PAT (same permissions)
2. Update GitHub secret value
3. Old token automatically revoked
4. No code changes needed
**Monitoring**:
- GitHub sends email 1 week before PAT expires
- Set calendar reminder as backup
---
## Related Documentation
- **Full Solution Guide**: [PRIVATE_REPO_ACCESS_SOLUTION.md](PRIVATE_REPO_ACCESS_SOLUTION.md)
- **Issue Tracker**: [IMMEDIATE_FIXES_NEEDED.md](IMMEDIATE_FIXES_NEEDED.md)
- **CI/CD Setup**: [SETUP.md](SETUP.md)
---
**Estimated Implementation Time**: 10-15 minutes
**Risk Level**: LOW (proven pattern)
**Rollback**: Remove secrets section from workflow

777
cicd/PLAN.md
View file

@ -1,777 +0,0 @@
# CI/CD Implementation Plan
**Last Updated**: 2025-11-27
**Status**: Design Complete → Implementation Pending
**Estimated Timeline**: 5-7 days (2-person team)
---
## 📋 Plan Overview
This document outlines the complete plan for implementing CI/CD infrastructure for the manacore-monorepo, from initial setup to production deployment.
---
## 🎯 Goals & Success Criteria
### Primary Goals
1. **Automate deployments** - Deploy with a single commit to main
2. **Zero-downtime updates** - Blue-green deployment strategy
3. **Enforce quality** - Automated testing with 80% coverage
4. **Cost efficiency** - 92% savings vs traditional PaaS ($56/month vs $300+)
5. **Team productivity** - Reduce deployment time from 2+ hours to < 10 minutes
### Success Criteria
- ✅ Staging auto-deploys on merge to main
- ✅ Production deploys take < 10 minutes
- ✅ Rollback can be executed in < 5 minutes
- ✅ Test coverage enforced at 80% minimum
- ✅ All 39 services deployed and healthy
- ✅ Monitoring and alerting operational
- ✅ Team can confidently deploy without assistance
---
## 🏗️ Architecture Overview
### Infrastructure Stack
- **Platform**: Docker Compose orchestration
- **Hosting**: Hetzner Cloud VPS (German data centers)
- **Container Runtime**: Docker + Docker Compose
- **CI/CD**: GitHub Actions
- **Monitoring**: Prometheus + Grafana + Loki
- **Error Tracking**: Sentry
- **CDN**: Cloudflare
### Service Inventory (39 Services Total)
**Authentication**:
- mana-core-auth (NestJS) - Central authentication service
**Chat Project** (4 services):
- chat-backend (NestJS)
- chat-web (SvelteKit)
- chat-mobile (Expo - OTA updates)
- chat-landing (Astro)
**Maerchenzauber Project** (4 services):
- maerchenzauber-backend (NestJS)
- maerchenzauber-web (SvelteKit)
- maerchenzauber-mobile (Expo)
- maerchenzauber-landing (Astro)
**Manadeck Project** (4 services):
- manadeck-backend (NestJS)
- manadeck-web (SvelteKit)
- manadeck-mobile (Expo)
- manadeck-landing (Astro)
**Memoro Project** (3 services):
- memoro-web (SvelteKit)
- memoro-mobile (Expo)
- memoro-landing (Astro)
**Picture Project** (3 services):
- picture-web (SvelteKit)
- picture-mobile (Expo)
- picture-landing (Astro)
**Wisekeep Project** (4 services):
- wisekeep-backend (NestJS)
- wisekeep-web (SvelteKit)
- wisekeep-mobile (Expo)
- wisekeep-landing (Astro)
**Quote Project** (4 services):
- quote-backend (NestJS)
- quote-web (SvelteKit)
- quote-mobile (Expo)
- quote-landing (Astro)
**Nutriphi Project** (2 services):
- nutriphi-backend (NestJS)
- nutriphi-web (SvelteKit)
**Uload Project** (1 service):
- uload-web (SvelteKit)
**Bauntown Project** (1 service):
- bauntown-landing (Astro)
**Manacore Project** (2 services):
- manacore-web (SvelteKit)
- manacore-mobile (Expo)
**Shared Infrastructure** (2 services):
- postgres (PostgreSQL 16)
- redis (Redis 7)
---
## 📅 Implementation Timeline
### Week 1: Foundation (Days 1-2)
**Goal**: Infrastructure setup and first deployment
**Day 1 Morning** (2-3 hours):
- Set up Hetzner account
- Provision staging server (CCX32)
- Install Docker & Docker Compose
- Configure GitHub Container Registry
**Day 1 Afternoon** (3-4 hours):
- Configure GitHub secrets (staging)
- Create first Dockerfile (mana-core-auth)
- Test CI/CD pipeline with test PR
- Deploy mana-core-auth to staging
**Day 2** (6-8 hours):
- Create Dockerfiles for remaining backends (6 services)
- Deploy all backends to staging
- Verify health checks
- Test inter-service communication
---
### Week 1: Web Apps (Days 3-4)
**Goal**: Deploy web apps and landing pages
**Day 3** (6-8 hours):
- Create SvelteKit Dockerfiles (9 services)
- Test builds locally
- Deploy to staging
- Configure reverse proxy/domains
**Day 4** (6-8 hours):
- Create Astro Dockerfiles (9 services)
- Deploy landing pages
- Set up SSL/TLS (Let's Encrypt)
- Test all web apps end-to-end
---
### Week 2: Testing & Production (Days 5-7)
**Goal**: Implement testing and deploy to production
**Day 5** (6-8 hours):
- Write critical path tests (auth, payments) - 100% coverage
- Configure test frameworks
- Enable coverage enforcement in CI
- Fix any failing tests
**Day 6** (6-8 hours):
- Provision production server
- Configure production secrets
- Set up GitHub environments (approval gates)
- Deploy mana-core-auth to production
**Day 7** (6-8 hours):
- Deploy all services to production
- Configure DNS for all domains
- Set up monitoring (Prometheus + Grafana)
- Verify everything works in production
---
### Week 2-3: Monitoring & Optimization (Days 8-10+)
**Goal**: Set up monitoring and optimize
**Day 8** (4-6 hours):
- Install Loki for logging
- Configure Grafana dashboards
- Set up alerting (Prometheus Alertmanager)
- Integrate Sentry for error tracking
**Day 9** (4-6 hours):
- Set up automated backups
- Test backup restoration
- Perform disaster recovery drill
- Document procedures
**Day 10+** (ongoing):
- Write remaining tests (80% coverage target)
- Performance optimization (caching, CDN)
- Team training
- Documentation updates
---
## 🔄 Development Workflow
### Developer Workflow
```
1. Create feature branch
↓
2. Write code + tests
↓
3. Push to GitHub
↓
4. GitHub Actions runs:
- Lint
- Type check
- Build
- Tests (with coverage)
↓
5. PR approved + merged to main
↓
6. GitHub Actions builds Docker images
↓
7. Images pushed to ghcr.io
↓
8. Auto-deploy to staging
↓
9. (Optional) Manual deploy to production
```
### Deployment Workflow
```
Staging (Automatic):
Merge to main → Build → Push → Deploy → Health Check → Done
Production (Manual Approval):
Manual trigger → Approval gate → Backup → Deploy → Health Check →
Monitor 5 min → Done (or Rollback)
```
---
## 🐳 Docker Strategy
### Multi-Stage Builds
All Dockerfiles use multi-stage builds for optimization:
**Stage 1: Dependencies**
- Install pnpm and dependencies
- Uses layer caching
**Stage 2: Build**
- Build application
- Generate production artifacts
**Stage 3: Runtime**
- Alpine Linux base (minimal)
- Copy only production artifacts
- Non-root user
- Health checks configured
### Image Naming Convention
```
ghcr.io/wuesteon/mana-core-auth:latest
ghcr.io/wuesteon/mana-core-auth:main
ghcr.io/wuesteon/mana-core-auth:main-abc1234
ghcr.io/wuesteon/chat-backend:latest
ghcr.io/wuesteon/chat-backend:main
ghcr.io/wuesteon/chat-backend:main-abc1234
```
**Tags**:
- `latest` - Most recent build from main
- `main` - Branch-based tag
- `main-abc1234` - Git commit SHA (for rollbacks)
---
## 🧪 Testing Strategy
### Coverage Targets
- **Critical Paths**: 100% coverage required
- Authentication (`@manacore/shared-auth`)
- Payment/credit system
- Data integrity (migrations, RLS)
- **General Code**: 80% coverage minimum
- Backend services
- Frontend apps
- Shared packages
### Test Types
**Unit Tests**:
- All services and components
- Frameworks: Jest (backend/mobile), Vitest (web/shared)
**Integration Tests**:
- API endpoints with test database
- Service interactions
**E2E Tests** (Phase 2):
- Playwright for web apps
- Detox/Maestro for mobile apps
### CI/CD Integration
- Run on every PR
- Enforce coverage thresholds
- Block merge if tests fail or coverage below 80%
- Parallel execution for speed
---
## 🚀 Deployment Strategy
### Blue-Green Deployment
```
Current (Blue): New (Green):
v1.0 → v1.1 (deploying)
↓
Health check
↓
Tests pass
↓
Traffic → Blue → Switch traffic → Green
↓
Monitor 1 hour
↓
Decommission Blue
```
**Benefits**:
- Zero downtime
- Instant rollback (switch back to blue)
- Test new version before full cutover
### Rollback Procedure
1. Detect issue (monitoring alerts or manual detection)
2. Run `scripts/deploy/rollback.sh`
3. Switch traffic back to previous version
4. Restore database from backup (if needed)
5. Total time: < 5 minutes
---
## 📊 Monitoring Strategy
### Metrics Collection (Prometheus)
**Application Metrics**:
- Request rate (requests/second)
- Error rate (% of failed requests)
- Response time (p50, p95, p99)
- Active connections
**Infrastructure Metrics**:
- CPU usage per service
- Memory usage per service
- Disk usage
- Network I/O
### Logging (Loki + Grafana)
**Log Aggregation**:
- All containers → stdout/stderr → Loki → Grafana
- Structured JSON logs
- Correlation IDs for tracing
**Log Retention**:
- 7 days online (searchable)
- 30 days archived (backup)
### Error Tracking (Sentry)
**What's Tracked**:
- Application errors and exceptions
- Source maps for better stack traces
- User context (anonymized)
- Performance metrics
### Alerting (Prometheus Alertmanager)
**Alert Rules**:
- Service down (health check fails for 2 minutes)
- High error rate (> 5% of requests failing)
- High CPU usage (> 80% for 5 minutes)
- High memory usage (> 90% for 5 minutes)
- Disk space low (< 10% free)
**Notification Channels**:
- Slack (all alerts)
- PagerDuty (critical alerts only)
- Email (daily summary)
---
## 💰 Cost Breakdown
### Infrastructure Costs (Monthly)
**Phase 1: Single Server (Recommended Start)**
| Item | Cost | Notes |
|------|------|-------|
| Hetzner CCX32 | $50 | 8 vCPU, 32 GB RAM, 240 GB SSD |
| Domains (6x) | $6 | $12/year each |
| Cloudflare CDN | $0 | Free tier |
| GitHub Actions | $0 | Within free tier |
| GitHub Container Registry | $0 | 500 MB free |
| **Total** | **$56** | |
**Phase 2: Multi-Server (Production Scale)**
| Item | Cost | Notes |
|------|------|-------|
| Staging (CCX22) | $25 | 4 vCPU, 16 GB RAM |
| Production (CCX42) | $100 | 16 vCPU, 64 GB RAM |
| Monitoring (CX32) | $15 | 4 vCPU, 8 GB RAM |
| Domains | $6 | Same as above |
| CDN, GitHub | $0 | Free tiers |
| **Total** | **$146** | |
**Cost Savings**:
- vs AWS/Azure: $500-1,000/month (89-95% savings)
- vs Heroku/Railway: $300-500/month (71-83% savings)
- vs DigitalOcean: $150-300/month (51-71% savings)
### Resource Allocation (Per Service)
| Service Type | CPU | RAM | Instances | Total |
| -------------- | ---- | ------ | --------- | --------------------------- |
| NestJS Backend | 0.5 | 512 MB | 10 | 5 CPU, 5 GB RAM |
| SvelteKit Web | 0.25 | 256 MB | 9 | 2.25 CPU, 2.25 GB RAM |
| Astro Landing | 0.1 | 128 MB | 9 | 0.9 CPU, 1.1 GB RAM |
| PostgreSQL | 1 | 2 GB | 1 | 1 CPU, 2 GB RAM |
| Redis | 0.25 | 256 MB | 1 | 0.25 CPU, 256 MB RAM |
| Monitoring | 1 | 2 GB | 1 | 1 CPU, 2 GB RAM |
| **Total** | | | | **~10.5 CPU, ~12.5 GB RAM** |
**Conclusion**: CCX32 (8 vCPU, 32 GB RAM) is sufficient for all services with headroom for growth.
---
## 🔐 Security Measures
### Infrastructure Security
- [x] Firewall rules (only ports 22, 80, 443 exposed)
- [x] SSH key-based authentication (no passwords)
- [x] Non-root Docker containers
- [x] Read-only filesystems where possible
- [x] Network segmentation (frontend, backend, data layers)
- [x] Automatic security updates
### Application Security
- [x] Environment variable encryption (GitHub Secrets)
- [x] SSL/TLS for all services (Let's Encrypt)
- [x] JWT-based authentication (@manacore/shared-auth)
- [x] Row-Level Security (Supabase RLS policies)
- [x] Input validation and sanitization
- [x] CORS policies enforced
### CI/CD Security
- [x] Weekly dependency audits (Dependabot)
- [x] Docker image scanning (Trivy)
- [x] No secrets in code
- [x] Branch protection rules
- [x] Required code reviews
- [x] Signed commits (recommended)
### Compliance
- [x] GDPR compliance (Hetzner EU data centers)
- [x] ISO 27001 certified infrastructure
- [x] SOC 2 Type II (Supabase)
- [x] Automated backup retention policies
- [x] Audit logs (GitHub Actions, Coolify, Supabase)
---
## 🔄 Backup & Disaster Recovery
### Backup Strategy
**What's Backed Up**:
- PostgreSQL databases (daily)
- Redis data (daily)
- Docker volumes
- Environment configurations
- Deployment manifests
**Backup Schedule**:
- Daily automated backups at 2 AM UTC
- Retention: 30 days for databases, 7 days for Redis
- Storage: Cloudflare R2 or Hetzner Storage Box
**Backup Verification**:
- Weekly automated restoration tests
- Monthly manual restoration drills
### Disaster Recovery
**Recovery Time Objective (RTO)**:
- Service restart: < 1 hour
- Full server restore: < 2 hours
**Recovery Point Objective (RPO)**:
- < 24 hours (daily backups)
- Supabase PITR available for point-in-time recovery
**Recovery Procedures**:
1. **Service Failure**: Restart container (automated)
2. **Data Corruption**: Restore from latest backup
3. **Server Failure**: Provision new server, restore from backup
4. **Region Failure**: Failover to secondary region (future phase)
---
## 📚 Documentation Strategy
### For Developers
- Quick start guide (30 minutes to first deployment)
- Testing guide (how to write and run tests)
- Troubleshooting guide (common issues)
- Contributing guide (standards and patterns)
### For DevOps
- Architecture documentation (complete system design)
- Deployment runbooks (step-by-step procedures)
- Monitoring guide (dashboards and alerts)
- Incident response playbooks
### For Management
- Cost analysis and projections
- Success metrics and KPIs
- Timeline and milestones
- Risk assessment and mitigation
---
## 🎯 Phase Gates
### Phase 1 Complete When:
- [x] Hetzner account created
- [x] Staging server provisioned and Docker installed
- [x] GitHub secrets configured
- [x] First service deployed to staging
- [x] CI/CD pipeline tested end-to-end
### Phase 2 Complete When:
- [x] All backend services deployed
- [x] All web apps deployed
- [x] All landing pages deployed
- [x] SSL/TLS configured for all domains
- [x] Health checks passing for all services
### Phase 3 Complete When:
- [x] Critical path tests at 100% coverage
- [x] General code at 80% coverage
- [x] Coverage enforcement in CI
- [x] All tests passing consistently
### Phase 4 Complete When:
- [x] Production server provisioned
- [x] All services deployed to production
- [x] Monitoring operational (Prometheus + Grafana + Loki)
- [x] Alerting configured and tested
- [x] Backups automated and verified
---
## 🚧 Risk Management
### Identified Risks
**Risk 1: Budget Overruns**
- **Likelihood**: Low
- **Impact**: Medium
- **Mitigation**: Start with single server ($56/month), scale only when needed
- **Contingency**: Downgrade server size, optimize resource usage
**Risk 2: Deployment Failures**
- **Likelihood**: Medium (during initial rollout)
- **Impact**: High
- **Mitigation**: Blue-green deployment, automated rollback, comprehensive testing
- **Contingency**: Rollback procedures documented and tested
**Risk 3: Service Outages**
- **Likelihood**: Low
- **Impact**: High
- **Mitigation**: Health checks, monitoring, automated restarts
- **Contingency**: Incident response playbooks, 24/7 monitoring
**Risk 4: Data Loss**
- **Likelihood**: Very Low
- **Impact**: Critical
- **Mitigation**: Daily backups, Supabase PITR, backup verification
- **Contingency**: Multiple backup locations, disaster recovery drills
**Risk 5: Security Breaches**
- **Likelihood**: Low
- **Impact**: Critical
- **Mitigation**: Security best practices, automated audits, minimal attack surface
- **Contingency**: Incident response plan, security patches, audit logs
**Risk 6: Migration Complexity**
- **Likelihood**: Medium (now addressed - migration complete)
- **Impact**: Medium
- **Mitigation**: Completed migration from Coolify to Docker Compose, removed legacy artifacts
- **Contingency**: Docker Compose provides simpler, more maintainable deployment
---
## 📈 Success Metrics & KPIs
### Deployment Metrics
- **Deployment Frequency**: Target > 5/week (currently < 1/week)
- **Deployment Duration**: Target < 10 minutes (currently 2+ hours manual)
- **Deployment Success Rate**: Target > 95%
- **Rollback Time**: Target < 5 minutes
### Quality Metrics
- **Test Coverage**: Target 80% minimum (currently ~5%)
- **Critical Path Coverage**: Target 100% (currently ~0%)
- **Build Success Rate**: Target > 95%
- **Code Review Turnaround**: Target < 24 hours
### Reliability Metrics
- **Uptime**: Target 99.9% (43 minutes downtime/month)
- **Mean Time to Recovery (MTTR)**: Target < 1 hour
- **Mean Time Between Failures (MTBF)**: Target > 30 days
- **Backup Success Rate**: Target 100%
### Cost Metrics
- **Infrastructure Cost**: Target < $100/month (achieved: $56/month)
- **Cost per Service**: Target < $5/month
- **Cost Reduction**: 92% vs traditional PaaS
---
## 🎓 Training & Knowledge Transfer
### Developer Training (2-3 hours)
- **Session 1**: CI/CD basics and GitHub Actions
- **Session 2**: Writing and running tests
- **Session 3**: Docker and deployment
- **Session 4**: Troubleshooting and debugging
### DevOps Training (4-8 hours)
- **Session 1**: Architecture deep dive
- **Session 2**: Infrastructure setup (hands-on)
- **Session 3**: CI/CD operations
- **Session 4**: Incident response and recovery
### Documentation
- All procedures documented in `cicd/` folder
- Video tutorials (optional, future)
- Regular knowledge sharing sessions
---
## 🔮 Future Enhancements
### Short-Term (3-6 months)
- [ ] Canary deployments (gradual traffic shifting)
- [ ] Feature flags (LaunchDarkly/Unleash)
- [ ] Visual regression testing (Percy/Chromatic)
- [ ] Load testing (k6/Artillery)
- [ ] Mobile E2E testing (Detox/Maestro)
### Long-Term (6-12 months)
- [ ] Kubernetes migration (when scale demands)
- [ ] Multi-region deployment
- [ ] Global load balancing
- [ ] Database replication
- [ ] Advanced observability (distributed tracing)
---
## ✅ Plan Approval
**Created by**: Hive Mind Collective Intelligence
**Reviewed by**: \***\*\_\*\***
**Approved by**: \***\*\_\*\***
**Approval Date**: \***\*\_\*\***
**Next Steps**:
1. Review this plan with the team
2. Get budget approval ($56-146/month)
3. Start implementation following `TODO.md`
4. Track progress in `CHANGELOG.md`
---
**Last Updated**: 2025-11-27
**Version**: 1.0
**Status**: Ready for Implementation ✅

571
cicd/PRIVATE_REPO_ACCESS_SOLUTION.md
View file

@ -1,571 +0,0 @@
# Private Repository Access Solution for CI/CD
**Date**: 2025-11-27
**Status**: READY TO IMPLEMENT
**Priority**: BLOCKER
---
## Executive Summary
The CI/CD pipeline fails when building Docker images because it needs access to the private repository `github.com/Memo-2023/mana-core-nestjs-package.git`. This document analyzes the problem and provides a complete implementation guide for the recommended solution.
---
## Problem Analysis
### Current Situation
**Affected Services:**
- `apps/maerchenzauber/apps/backend` (Storyteller backend)
- `apps/manadeck/apps/backend` (ManaDeck backend)
**What's Happening:**
1. Both Dockerfiles clone the private repo `mana-core-nestjs-package` during build
2. The Dockerfiles already have secret mounting logic: `RUN --mount=type=secret,id=github_token`
3. GitHub Actions workflow does NOT pass the secret to Docker build
4. Build fails with: `git clone failed: exit code 128`
**Evidence from Dockerfiles:**
Both Dockerfiles already include this logic (lines 15-30):
```dockerfile
RUN --mount=type=secret,id=github_token \
if [ -f /run/secrets/github_token ]; then \
export GITHUB_TOKEN=$(cat /run/secrets/github_token) && \
echo "Using GitHub token for private repo access" && \
git clone https://${GITHUB_TOKEN}@github.com/Memo-2023/mana-core-nestjs-package.git /tmp/mana-core; \
else \
echo "No GitHub token provided, attempting public clone" && \
git clone https://github.com/Memo-2023/mana-core-nestjs-package.git /tmp/mana-core; \
fi
```
**The Gap:**
The Dockerfiles are ready, but the CI workflow doesn't provide the `github_token` secret.
---
## Solution Options Comparison
| Option | Pros | Cons | Cost | Recommendation |
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ----------- | --------------------- |
| **A: GitHub Token in CI** | ✅ Quick implementation<br>✅ Dockerfiles already configured<br>✅ Fine-grained access control<br>✅ No architectural changes | ⚠️ Requires PAT management<br>⚠️ Token rotation needed | FREE | ⭐ **RECOMMENDED** |
| B: Make Repo Public | ✅ Simplest solution<br>✅ No auth needed | ❌ Exposes proprietary code<br>❌ Security risk<br>❌ Not acceptable for most orgs | FREE | ❌ Not recommended |
| C: Publish npm Package | ✅ Professional approach<br>✅ Version management<br>✅ Private npm registry | ❌ Complex setup<br>❌ Ongoing maintenance<br>❌ Registry costs | $7-29/month | 🔮 Future enhancement |
---
## Recommended Solution: Option A - GitHub Token in CI
### Why This Solution?
1. **Already 90% implemented** - Dockerfiles have the secret mounting logic
2. **One-line fix** - Just need to pass the secret in CI workflow
3. **Battle-tested** - Same pattern used in manadeck and maerchenzauber docs
4. **Secure** - GitHub PAT with fine-grained permissions
5. **No code changes** - Works with existing Dockerfile architecture
### Implementation Complexity
- **Estimated Time**: 10-15 minutes
- **Required Actions**: 2 steps
- **Risk Level**: LOW (proven pattern)
- **Testing Required**: Build verification only
---
## Detailed Implementation Guide
### Prerequisites
1. GitHub repository admin access (to create secrets)
2. Ability to create GitHub Personal Access Token (PAT)
3. Access to GitHub Actions workflow files
### Step 1: Create GitHub Personal Access Token
**Why**: GitHub Actions' default `GITHUB_TOKEN` doesn't have permission to access other private repos.
**How**:
1. Go to GitHub Settings → Developer Settings → Personal Access Tokens → Fine-grained tokens
2. Click "Generate new token"
3. Configure:
- **Token name**: `CI-Private-Repo-Access`
- **Expiration**: 1 year (set calendar reminder to rotate)
- **Repository access**: Select "Only select repositories"
- Choose: `Memo-2023/mana-core-nestjs-package`
- **Permissions**:
- Repository permissions → Contents → Read (required)
- Repository permissions → Metadata → Read (auto-selected)
4. Click "Generate token"
5. **CRITICAL**: Copy the token immediately (can't view again)
**Token Format**: `github_pat_11AAAAAA...` (starts with `github_pat_`)
### Step 2: Add Token as GitHub Secret
1. Navigate to `https://github.com/Memo-2023/manacore-monorepo/settings/secrets/actions`
2. Click "New repository secret"
3. Configure:
- **Name**: `PRIVATE_REPO_TOKEN`
- **Value**: Paste the PAT from Step 1
4. Click "Add secret"
### Step 3: Update GitHub Actions Workflow
**File**: `.github/workflows/ci-main.yml`
**Current Code** (lines 126-140):
```yaml
- name: Build and push
if: steps.check-dockerfile.outputs.exists == 'true'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.service.path }}/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
NODE_ENV=production
PORT=${{ matrix.service.port }}
```
**Updated Code** (add 2 lines):
```yaml
- name: Build and push
if: steps.check-dockerfile.outputs.exists == 'true'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.service.path }}/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
NODE_ENV=production
PORT=${{ matrix.service.port }}
secrets: |
github_token=${{ secrets.PRIVATE_REPO_TOKEN }}
```
**What Changed**:
- Added `secrets:` section with `github_token=${{ secrets.PRIVATE_REPO_TOKEN }}`
- This passes the token to Docker build as a secret mount
- Dockerfiles already have logic to use this secret
### Step 4: Verification Steps
**Local Testing** (Optional but recommended):
```bash
# 1. Export token (use your PAT)
export GITHUB_TOKEN="github_pat_YOUR_TOKEN_HERE"
# 2. Test maerchenzauber backend
cd /Users/wuesteon/dev/mana_universe/manacore-monorepo
docker build \
--secret id=github_token,env=GITHUB_TOKEN \
-t test-maerchenzauber \
-f apps/maerchenzauber/apps/backend/Dockerfile \
.
# 3. Test manadeck backend
docker build \
--secret id=github_token,env=GITHUB_TOKEN \
-t test-manadeck \
-f apps/manadeck/apps/backend/Dockerfile \
.
# 4. Verify builds succeeded
docker images | grep test-
```
**CI Testing**:
1. Commit the workflow change
2. Push to a test branch
3. Check GitHub Actions logs for:
- "Using GitHub token for private repo access" (success)
- OR "No GitHub token provided" (failure - secret not passed)
4. Verify build completes without git clone errors
**Success Indicators**:
- ✅ Docker build logs show "Using GitHub token for private repo access"
- ✅ Git clone succeeds
- ✅ Tarball creation succeeds: "Mana-core packaged as tarball"
- ✅ Docker image pushed to ghcr.io
- ✅ No "exit code 128" errors
---
## Architecture Details
### How It Works (Sequence Diagram)
```
GitHub Actions Workflow
│
├─→ Passes secret to Docker build
│ secrets: github_token=${{ secrets.PRIVATE_REPO_TOKEN }}
│
└─→ Docker Build Process
│
├─→ Mounts secret as file: /run/secrets/github_token
│
├─→ Reads token: export GITHUB_TOKEN=$(cat /run/secrets/github_token)
│
├─→ Clones private repo:
│ git clone https://${GITHUB_TOKEN}@github.com/Memo-2023/mana-core-nestjs-package.git
│
├─→ Builds and packages:
│ npm install --force
│ npm run build
│ npm pack → mana-core.tgz
│
├─→ Replaces git URL with local tarball in package.json:
│ "mana-core": "file:../mana-core.tgz"
│
└─→ Installs from tarball:
npm install --legacy-peer-deps
(No git access needed - fully self-contained)
```
### Security Model
**Secret Handling**:
- Token stored in GitHub Secrets (encrypted at rest)
- Passed to Docker as build secret (never in logs)
- Docker secret mount (in-memory, not in image layers)
- Token never written to filesystem in final image
- Token only visible during build, not in running containers
**Access Control**:
- PAT has read-only access to single repo
- Scoped to specific repository
- Can be rotated without code changes
- Can be revoked instantly if compromised
### Production Impact
**What Gets Built**:
- Private package is compiled and bundled into Docker image
- Final image contains `mana-core.tgz` (built artifact)
- No git dependencies in production
- No authentication needed at runtime
**Image Size Impact**:
- Minimal (tarball is compressed)
- Same as current local builds
**Runtime Performance**:
- No impact (package already compiled)
- Faster than git clone at runtime
---
## Alternative Options (Future Consideration)
### Option B: Make Repository Public
**When to Consider**:
- If code is not proprietary
- If open-source is part of business model
- If maintaining PAT becomes burdensome
**How to Implement**:
1. Go to repository settings
2. Change visibility to public
3. Remove secret mounting from Dockerfiles
4. Update CI workflow to remove secrets
**Pros**: Simplest solution
**Cons**: Not viable for most commercial projects
### Option C: Publish as Private npm Package
**When to Consider**:
- When you have 10+ services using the package
- When version management becomes critical
- When you want professional npm workflow
**How to Implement**:
**Option C1: GitHub Packages (Free)**
1. Add `.npmrc` to mana-core-nestjs-package:
```
@memo-2023:registry=https://npm.pkg.github.com
```
2. Update package.json:
```json
{
"name": "@memo-2023/mana-core",
"version": "1.0.0",
"publishConfig": {
"registry": "https://npm.pkg.github.com"
}
}
```
3. Publish:
```bash
npm login --registry=https://npm.pkg.github.com
npm publish
```
4. Update consuming apps:
```json
{
"dependencies": {
"@memo-2023/mana-core": "^1.0.0"
}
}
```
5. Add `.npmrc` to consuming projects:
```
@memo-2023:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=${NPM_TOKEN}
```
**Option C2: npm.js Registry (Paid)**
- Private packages: $7/month for 1 user, $29/month for teams
- More features: badges, stats, better CDN
- Industry standard for professional packages
**Migration Effort**: Medium (3-4 hours)
**Ongoing Maintenance**: Low (publish on new versions)
**Cost**: $0 (GitHub) or $7-29/month (npm.js)
---
## Troubleshooting Guide
### Issue: "No GitHub token provided"
**Symptom**: Docker build logs show fallback message
**Cause**: Secret not passed to Docker build
**Fix**:
1. Verify secret exists: GitHub repo settings → Secrets → `PRIVATE_REPO_TOKEN`
2. Check workflow syntax: `secrets: |` with correct indentation
3. Verify secret name matches exactly
### Issue: "git clone failed: exit code 128"
**Symptom**: Authentication failure during git clone
**Cause**: Invalid or expired PAT
**Fix**:
1. Test PAT manually:
```bash
git clone https://YOUR_PAT@github.com/Memo-2023/mana-core-nestjs-package.git
```
2. If fails: regenerate PAT with correct permissions
3. Update GitHub secret with new PAT
### Issue: "Permission denied (publickey)"
**Symptom**: SSH authentication attempted
**Cause**: Git URL using SSH instead of HTTPS
**Fix**:
- Dockerfiles already use HTTPS URLs
- Verify Dockerfile has `https://${GITHUB_TOKEN}@github.com/...`
### Issue: Build succeeds but package not found
**Symptom**: `npm install` fails to find mana-core
**Cause**: Tarball path incorrect or sed replacement failed
**Fix**:
1. Check build logs for "=== Verifying tarball ===" section
2. Verify tarball exists at expected path
3. Check sed replacement worked: `grep -n "mana-core" package.json`
---
## Maintenance Plan
### Token Rotation (Every 12 Months)
**Why**: Security best practice
**When**: Set calendar reminder 1 week before expiration
**How**:
1. Generate new PAT (same permissions)
2. Update GitHub secret: Settings → Secrets → Edit `PRIVATE_REPO_TOKEN`
3. Trigger test build to verify
4. Delete old PAT
**Downtime**: None (atomic secret update)
### Monitoring
**What to Watch**:
- GitHub Actions build failures on main branch
- Authentication errors in Docker build logs
- PAT expiration date (GitHub sends email reminders)
**Alerts**:
- Set GitHub Actions notification to Slack/email
- Monitor build success rate
---
## Migration from Current State
### What Changes
**Modified Files**:
- `.github/workflows/ci-main.yml` (add 2 lines)
**New Secrets**:
- `PRIVATE_REPO_TOKEN` (in GitHub repo settings)
**No Changes Needed**:
- Dockerfiles (already have secret mounting)
- Package.json files
- Application code
- Local development workflow
### Rollback Plan
**If Implementation Fails**:
1. Remove `secrets:` section from workflow
2. Revert to previous workflow version
3. Builds will fail (same as current state)
**Zero Risk**: Can't make it worse than current state
---
## Success Criteria
### Definition of Done
- ✅ GitHub secret `PRIVATE_REPO_TOKEN` created
- ✅ CI workflow updated with secret passing
- ✅ Test build succeeds on test branch
- ✅ Both maerchenzauber-backend and manadeck-backend images build
- ✅ Docker images pushed to ghcr.io successfully
- ✅ No authentication errors in logs
- ✅ Documentation updated (this file)
### Metrics
**Before**:
- Docker build success rate: 0%
- Manual workarounds: Required
- Developer impact: High (can't merge PRs)
**After**:
- Docker build success rate: 100%
- Manual workarounds: None
- Developer impact: Zero
---
## References
### Similar Implementations
- **apps/manadeck/apps/backend/SSH_LOCKFILE_SOLUTION.md** - Describes the two-layer approach
- **apps/maerchenzauber/docs/ci-npm-ssh-fix.md** - Documents npm SSH fix pattern
- Docker BuildKit secrets: https://docs.docker.com/build/building/secrets/
### Related Documentation
- **cicd/IMMEDIATE_FIXES_NEEDED.md** - Parent issue tracking document
- **cicd/SETUP.md** - Overall CI/CD setup guide
- **cicd/README.md** - CI/CD documentation hub
---
## Appendix: Complete Workflow Example
### Full ci-main.yml Build Step (After Changes)
```yaml
- name: Build and push
if: steps.check-dockerfile.outputs.exists == 'true'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.service.path }}/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
NODE_ENV=production
PORT=${{ matrix.service.port }}
secrets: |
github_token=${{ secrets.PRIVATE_REPO_TOKEN }}
```
### Expected Docker Build Log Output
```
#8 [builder 3/12] RUN --mount=type=secret,id=github_token if [ -f /run/secrets/github_token ]; then...
#8 0.124 Using GitHub token for private repo access
#8 0.856 Cloning into '/tmp/mana-core'...
#8 12.34 Mana-core packaged as tarball at /app/mana-core.tgz
#8 DONE 12.5s
#9 [builder 4/12] COPY apps/maerchenzauber/apps/backend/package.json ./backend/package.json
#9 DONE 0.1s
#10 [builder 5/12] RUN sed -i 's|"git+https://github.com/...
#10 0.012 === Verifying tarball and package.json ===
#10 0.013 -rw-r--r-- 1 root root 1234567 Nov 27 12:34 mana-core.tgz
#10 0.013 Tarball exists at /app/mana-core.tgz
#10 0.014 Checking package.json replacement:
#10 0.015 26: "@mana-core/nestjs-integration": "file:../mana-core.tgz",
#10 0.015 === End verification ===
#10 DONE 0.2s
```
---
**Document Version**: 1.0
**Last Updated**: 2025-11-27
**Next Review**: 2026-11-27 (token rotation)

298
cicd/README.md
View file

@ -1,298 +0,0 @@
# CI/CD Documentation Hub
Central documentation for the manacore-monorepo CI/CD pipeline and deployment infrastructure.
---
## 📚 Quick Navigation
### Getting Started
- 🚀 **[TODO.md](./TODO.md)** - Actionable tasks to complete the CI/CD setup
- 📋 **[PLAN.md](./PLAN.md)** - Complete implementation plan and roadmap
- ⚙️ **[SETUP.md](./SETUP.md)** - Step-by-step setup instructions
### Progress Tracking
- ✅ **[COMPLETED.md](./COMPLETED.md)** - What's been built and delivered
- 📝 **[CHANGELOG.md](./CHANGELOG.md)** - Timeline of changes and updates
### Implementation Guides
- 🐳 **[DOCKER.md](./DOCKER.md)** - Docker configuration and best practices
- 🔄 **[GITHUB_ACTIONS.md](./GITHUB_ACTIONS.md)** - GitHub Actions workflows
- 🚢 **[DEPLOYMENT.md](./DEPLOYMENT.md)** - Deployment procedures
- 🧪 **[TESTING.md](./TESTING.md)** - Testing strategy and implementation
### Reference
- 🔐 **[SECRETS.md](./SECRETS.md)** - Required secrets and environment variables
- 🏗️ **[ARCHITECTURE.md](./ARCHITECTURE.md)** - Infrastructure architecture overview
- 🛠️ **[TROUBLESHOOTING.md](./TROUBLESHOOTING.md)** - Common issues and solutions
---
## 🎯 Current Status
**Overall Progress**: 70% Complete
| Phase | Status | Progress |
| ---------------------------- | -------------- | -------- |
| **Planning & Research** | ✅ Complete | 100% |
| **Documentation** | ✅ Complete | 100% |
| **Docker Templates** | ✅ Complete | 100% |
| **GitHub Actions Workflows** | ✅ Complete | 100% |
| **Deployment Scripts** | ✅ Complete | 100% |
| **Testing Infrastructure** | ✅ Complete | 100% |
| **Infrastructure Setup** | ⏳ Not Started | 0% |
| **Secrets Configuration** | ⏳ Not Started | 0% |
| **First Deployment** | ⏳ Not Started | 0% |
| **Full Rollout** | ⏳ Not Started | 0% |
---
## 🚀 Quick Start (30 Minutes)
Follow these steps to get started immediately:
### 1. Review the Plan (5 minutes)
```bash
cat cicd/PLAN.md
```
### 2. Check What's Done (5 minutes)
```bash
cat cicd/COMPLETED.md
```
### 3. Start with TODOs (10 minutes)
```bash
cat cicd/TODO.md
# Pick the first task and start!
```
### 4. Follow Setup Guide (10 minutes)
```bash
cat cicd/SETUP.md
# Begin Phase 1: Quick Start
```
---
## 📊 What We're Building
### Infrastructure
- **Platform**: Docker Compose + Hetzner VPS
- **Cost**: ~$56/month (92% cheaper than alternatives)
- **Services**: 39+ deployable services across 10 projects
- **Container Registry**: GitHub Container Registry (ghcr.io)
### CI/CD Pipeline
- **Tool**: GitHub Actions
- **Features**: Automated testing, building, deployment
- **Strategy**: Blue-green deployment, zero-downtime
- **Environments**: Staging → Production
### Testing
- **Coverage Target**: 80% minimum, 100% critical paths
- **Frameworks**: Jest, Vitest, Playwright
- **Automation**: Run on every PR, enforce coverage thresholds
---
## 🏗️ Project Structure
```
manacore-monorepo/
├── cicd/ # 👈 You are here
│ ├── README.md # This file
│ ├── TODO.md # Actionable tasks
│ ├── PLAN.md # Implementation roadmap
│ ├── COMPLETED.md # What's done
│ ├── SETUP.md # Setup instructions
│ ├── CHANGELOG.md # Change history
│ ├── DOCKER.md # Docker guide
│ ├── GITHUB_ACTIONS.md # Workflows guide
│ ├── DEPLOYMENT.md # Deployment guide
│ ├── TESTING.md # Testing guide
│ ├── SECRETS.md # Required secrets
│ ├── ARCHITECTURE.md # Architecture overview
│ └── TROUBLESHOOTING.md # Common issues
├── .github/workflows/ # GitHub Actions workflows
├── docker/ # Docker templates and configs
├── scripts/deploy/ # Deployment scripts
├── packages/test-config/ # Shared test configurations
└── docs/ # Extended documentation
```
---
## 🎯 Key Deliverables
The Hive Mind has delivered:
### Documentation (200,000+ words)
- ✅ Infrastructure research report (40+ pages)
- ✅ Architecture design (87,000+ characters)
- ✅ CI/CD implementation guides (80,000+ words)
- ✅ Testing strategy (50,000+ words)
- ✅ Hive Mind final report
### Code & Configuration (40+ files, 7,300+ lines)
- ✅ 7 GitHub Actions workflows
- ✅ 3 Dockerfile templates
- ✅ 5 deployment scripts
- ✅ 6 test configurations
- ✅ 7 test example files
- ✅ Docker compose files (staging, production)
---
## 🤝 Team Workflow
### For Developers
1. Read: `TODO.md` (see what needs to be done)
2. Pick a task from Phase 1 or 2
3. Follow: `SETUP.md` for step-by-step instructions
4. Reference: `TROUBLESHOOTING.md` if stuck
### For DevOps/Leads
1. Review: `PLAN.md` (understand the roadmap)
2. Check: `COMPLETED.md` (see what's ready)
3. Prioritize: `TODO.md` (assign tasks)
4. Monitor: `CHANGELOG.md` (track progress)
---
## 📅 Timeline
**Estimated Total**: 5-7 days for full implementation
| Week | Focus | Deliverable |
| ----------- | --------------------- | ------------------------------------- |
| **Week 1** | Infrastructure setup | Hetzner server + Docker Compose setup |
| **Week 1** | Secrets configuration | All GitHub secrets configured |
| **Week 1** | First deployment | Chat project deployed to staging |
| **Week 2** | Testing validation | CI/CD pipeline tested end-to-end |
| **Week 2** | Production deployment | First project in production |
| **Week 3+** | Full rollout | All 10 projects deployed |
---
## 🔗 Related Documentation
### Root Level
- `/HIVE_MIND_FINAL_REPORT.md` - Complete Hive Mind summary
- `/DOCKER_REGISTRY_SETUP.md` - GitHub Container Registry guide
- `/QUICK_START_CICD.md` - 30-minute fast track
- `/CI_CD_README.md` - High-level overview
### Docs Directory
- `/docs/DEPLOYMENT_ARCHITECTURE.md` - Complete architecture
- `/docs/DEPLOYMENT_DIAGRAMS.md` - ASCII diagrams
- `/docs/DEPLOYMENT_RUNBOOKS.md` - Operational procedures
- `/docs/CI_CD_SETUP.md` - Detailed setup guide
- `/docs/DOCKER_GUIDE.md` - Docker deep dive
- `/docs/TESTING.md` - Master testing strategy
### Hive Mind Research
- `/.hive-mind/sessions/research-report-hosting-infrastructure.md` - 40-page research report
---
## 🆘 Need Help?
### Quick Links
- **Stuck on setup?** → `TROUBLESHOOTING.md`
- **Don't know what to do?** → `TODO.md`
- **Need context?** → `PLAN.md`
- **Want to see progress?** → `COMPLETED.md`
### Support Resources
- Hive Mind Final Report: `/HIVE_MIND_FINAL_REPORT.md`
- Quick Start Guide: `/QUICK_START_CICD.md`
- GitHub Discussions: Create an issue if needed
---
## 🎓 Learning Resources
### Docker
- [Docker Documentation](https://docs.docker.com/)
- [Multi-stage Builds](https://docs.docker.com/build/building/multi-stage/)
- Our guide: `DOCKER.md`
### GitHub Actions
- [GitHub Actions Docs](https://docs.github.com/en/actions)
- [Workflow Syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions)
- Our guide: `GITHUB_ACTIONS.md`
### Docker & Docker Compose
- [Docker Documentation](https://docs.docker.com/)
- [Docker Compose Documentation](https://docs.docker.com/compose/)
### Hetzner
- [Hetzner Cloud Docs](https://docs.hetzner.com/)
- [Hetzner Server Options](https://www.hetzner.com/cloud)
---
## 📝 Contributing
When working on CI/CD tasks:
1. **Before starting**:
- Check `TODO.md` for current priorities
- Read relevant sections in `SETUP.md`
- Update `TODO.md` to mark task as in-progress
2. **During work**:
- Follow existing patterns in templates
- Document any deviations or discoveries
- Test thoroughly before marking complete
3. **After completion**:
- Update `TODO.md` (mark as done)
- Add entry to `CHANGELOG.md`
- Update `COMPLETED.md` if it's a major milestone
- Notify team of completion
---
## 🎯 Success Criteria
We'll know the CI/CD system is successful when:
- ✅ Developers can deploy with a single commit to main
- ✅ Staging environment automatically updates on merge
- ✅ Production deployments take < 10 minutes
- ✅ Rollbacks can be executed in < 5 minutes
- ✅ Test coverage is at 80% and enforced
- ✅ Zero-downtime deployments work reliably
- ✅ Team is confident in the deployment process
---
**Last Updated**: 2025-11-27
**Status**: Implementation in progress
**Next Step**: Review `TODO.md` and start Phase 1

801
cicd/SETUP.md
View file

@ -1,801 +0,0 @@
# CI/CD Setup Guide
**Last Updated**: 2025-11-27
**Estimated Time**: 30 minutes (Quick Start) to 7 days (Full Implementation)
---
## 📋 Table of Contents
1. [Prerequisites](#prerequisites)
2. [Quick Start (30 Minutes)](#quick-start-30-minutes)
3. [Phase 1: Infrastructure Foundation](#phase-1-infrastructure-foundation-day-1-2)
4. [Phase 2: First Deployment](#phase-2-first-deployment-day-1-2)
5. [Phase 3: Web Apps](#phase-3-web-apps-day-3-4)
6. [Phase 4: Testing](#phase-4-testing-day-5)
7. [Phase 5: Production](#phase-5-production-day-6-7)
8. [Verification](#verification)
9. [Troubleshooting](#troubleshooting)
---
## Prerequisites
### Required Accounts
- [ ] GitHub account (you have this)
- [ ] Hetzner Cloud account (need to create)
- [ ] Supabase account (you have this)
- [ ] Azure OpenAI account (you have this)
### Required Tools (Local Machine)
- [ ] Git
- [ ] Docker Desktop
- [ ] pnpm (v9.15.0)
- [ ] Node.js (v20+)
- [ ] SSH client
- [ ] Terminal/Command line
### Required Knowledge
- Basic Docker understanding
- Basic GitHub Actions understanding
- SSH and server access
- Command line comfort
---
## Quick Start (30 Minutes)
**Goal**: Get your first service deployed to staging
### Step 1: Create Hetzner Account (5 minutes)
1. Go to [https://console.hetzner.cloud/](https://console.hetzner.cloud/)
2. Click "Sign Up"
3. Complete registration
4. Verify email
5. Add payment method (credit card or PayPal)
6. May require ID verification (be prepared to upload ID)
### Step 2: Provision Server (10 minutes)
1. In Hetzner Console, click "New Project"
- Name: `manacore-staging`
2. Click "Add Server"
- **Location**: Falkenstein, Germany (or nearest to you)
- **Image**: Ubuntu 22.04
- **Type**: CCX32 (8 vCPU, 32 GB RAM, $50/month)
- **Networking**: Public IPv4
- **SSH Key**: Add your public SSH key
```bash
# On your machine, generate if you don't have one:
ssh-keygen -t ed25519 -C "your_email@example.com"
# Copy public key:
cat ~/.ssh/id_ed25519.pub
# Paste into Hetzner
```
- **Name**: `staging-01`
- Click "Create & Buy now"
3. Wait 1-2 minutes for server to be created
4. Note the server IP address: `___________________`
5. Test SSH connection:
```bash
ssh root@YOUR_SERVER_IP
# Type "yes" to accept fingerprint
# You should be logged in!
```
6. Update system:
```bash
apt update && apt upgrade -y
```
### Step 3: Set up Docker Compose (10 minutes)
1. On your server (via SSH), run:
```bash
curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash
```
2. Wait 5-10 minutes for installation to complete
- The script will install Docker, Coolify, and dependencies
- You'll see progress messages
3. Once complete, access Docker Compose configuration:
```
https://YOUR_SERVER_IP:8000
```
4. Complete initial setup wizard:
- Create admin account
- Set email (for SSL certificates)
- Configure basic settings
5. Save your Coolify credentials securely!
### Step 4: Configure GitHub Secrets (5 minutes)
1. Go to your GitHub repo: `https://github.com/wuesteon/manacore-monorepo`
2. Go to Settings → Secrets and variables → Actions → New repository secret
3. Add these 5 essential secrets:
```
Name: STAGING_HOST
Value: YOUR_SERVER_IP
```
```
Name: STAGING_USER
Value: root
```
```
Name: STAGING_SSH_KEY
Value: (paste your PRIVATE SSH key)
# Get it with: cat ~/.ssh/id_ed25519
# Copy the ENTIRE content including -----BEGIN and -----END
```
```
Name: STAGING_SUPABASE_URL
Value: https://your-project.supabase.co
```
```
Name: STAGING_SUPABASE_ANON_KEY
Value: your-anon-key-here
```
### Step 5: Test CI/CD Pipeline (5 minutes)
1. Create test branch:
```bash
cd /Users/wuesteon/dev/mana_universe/manacore-monorepo
git checkout -b test/cicd-setup
```
2. Make small change (add comment to README):
```bash
echo "\n<!-- Testing CI/CD -->" >> README.md
git add README.md
git commit -m "test: verify CI/CD pipeline"
git push origin test/cicd-setup
```
3. Create Pull Request on GitHub
4. Watch GitHub Actions:
- Go to Actions tab
- See "CI - Pull Request" workflow running
- Verify it completes successfully (green checkmark)
5. Merge PR to main
6. Watch "CI - Main Branch" workflow:
- Should build Docker image
- Should push to ghcr.io
- Check https://github.com/wuesteon?tab=packages
**🎉 If you see the green checkmarks, your CI/CD pipeline is working!**
---
## Phase 1: Infrastructure Foundation (Day 1-2)
### 1.1 Add Remaining GitHub Secrets
Now that the basics work, add the complete set of secrets:
**Staging Secrets** (add these 5 more):
```
STAGING_SUPABASE_SERVICE_ROLE_KEY = your-service-role-key
STAGING_JWT_SECRET = (generate with: openssl rand -base64 64)
STAGING_MANA_SERVICE_URL = http://mana-core-auth:3001
STAGING_AZURE_OPENAI_ENDPOINT = your-azure-endpoint
STAGING_AZURE_OPENAI_API_KEY = your-azure-key
```
### 1.2 Create First Dockerfile
**For mana-core-auth service**:
1. Copy template:
```bash
cp docker/templates/Dockerfile.nestjs services/mana-core-auth/Dockerfile
```
2. No changes needed! The template is already configured for NestJS services in the monorepo.
3. Test build locally:
```bash
docker build -t test-auth -f services/mana-core-auth/Dockerfile .
```
This will take 5-10 minutes the first time.
4. Test run locally:
```bash
docker run -p 3001:3001 \
-e SUPABASE_URL=your-url \
-e SUPABASE_ANON_KEY=your-key \
test-auth
```
5. Test health endpoint:
```bash
curl http://localhost:3001/api/v1/health
# Should return: {"status":"ok"}
```
6. If it works, commit and push:
```bash
git add services/mana-core-auth/Dockerfile
git commit -m "feat: add Dockerfile for mana-core-auth"
git push
```
7. Watch GitHub Actions build the image and push to ghcr.io
### 1.3 Deploy to Staging
**Option A: Manual Deployment (Recommended First Time)**
1. SSH into your server:
```bash
ssh root@YOUR_SERVER_IP
```
2. Create deployment directory:
```bash
mkdir -p ~/manacore-staging
cd ~/manacore-staging
```
3. Create `docker-compose.yml`:
```bash
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
mana-core-auth:
image: ghcr.io/wuesteon/mana-core-auth:latest
container_name: mana-core-auth
ports:
- "3001:3001"
environment:
- NODE_ENV=staging
- PORT=3001
- SUPABASE_URL=${SUPABASE_URL}
- SUPABASE_ANON_KEY=${SUPABASE_ANON_KEY}
- SUPABASE_SERVICE_ROLE_KEY=${SUPABASE_SERVICE_ROLE_KEY}
- JWT_SECRET=${JWT_SECRET}
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:3001/api/v1/health"]
interval: 30s
timeout: 10s
retries: 3
EOF
```
4. Create `.env` file:
```bash
cat > .env << 'EOF'
SUPABASE_URL=your-supabase-url
SUPABASE_ANON_KEY=your-anon-key
SUPABASE_SERVICE_ROLE_KEY=your-service-role-key
JWT_SECRET=your-jwt-secret
EOF
```
**Replace the placeholder values with your actual credentials!**
5. Login to GitHub Container Registry:
```bash
# Create a Personal Access Token (PAT) on GitHub:
# GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic)
# Scope: read:packages
echo YOUR_PAT | docker login ghcr.io -u wuesteon --password-stdin
```
6. Pull and start:
```bash
docker compose pull
docker compose up -d
```
7. Check status:
```bash
docker compose ps
docker compose logs mana-core-auth
```
8. Test health endpoint:
```bash
curl http://localhost:3001/api/v1/health
```
9. Test externally (from your local machine):
```bash
curl http://YOUR_SERVER_IP:3001/api/v1/health
```
**Option B: Automated Deployment (After Manual Works)**
1. Go to GitHub → Actions → "CD - Staging Deployment"
2. Click "Run workflow"
3. Select service: `mana-core-auth`
4. Click "Run workflow"
5. Watch the deployment progress
**🎉 If you see healthy service, your first deployment is complete!**
---
## Phase 2: First Deployment (Day 1-2)
### 2.1 Deploy Remaining Backend Services
Repeat the Dockerfile creation for each backend:
```bash
# Chat backend
cp docker/templates/Dockerfile.nestjs apps/chat/apps/backend/Dockerfile
# Maerchenzauber backend
cp docker/templates/Dockerfile.nestjs apps/maerchenzauber/apps/backend/Dockerfile
# Manadeck backend
cp docker/templates/Dockerfile.nestjs apps/manadeck/apps/backend/Dockerfile
# Nutriphi backend
cp docker/templates/Dockerfile.nestjs apps/nutriphi/apps/backend/Dockerfile
# Wisekeep backend (if exists)
cp docker/templates/Dockerfile.nestjs apps/wisekeep/apps/backend/Dockerfile
# Quote backend (if exists)
cp docker/templates/Dockerfile.nestjs apps/quote/apps/backend/Dockerfile
```
**Test each build locally before committing**:
```bash
docker build -t test-service -f apps/PROJECT/apps/backend/Dockerfile .
```
**Commit all at once**:
```bash
git add apps/*/apps/backend/Dockerfile
git commit -m "feat: add Dockerfiles for all backend services"
git push
```
### 2.2 Update docker-compose.yml
On your server, update `~/manacore-staging/docker-compose.yml` to include all services.
**Example with 3 backends**:
```yaml
version: '3.8'
services:
mana-core-auth:
image: ghcr.io/wuesteon/mana-core-auth:latest
container_name: mana-core-auth
ports:
- '3001:3001'
environment:
- NODE_ENV=staging
- PORT=3001
# ... env vars
restart: unless-stopped
chat-backend:
image: ghcr.io/wuesteon/chat-backend:latest
container_name: chat-backend
ports:
- '3002:3002'
environment:
- NODE_ENV=staging
- PORT=3002
# ... env vars
depends_on:
- mana-core-auth
restart: unless-stopped
maerchenzauber-backend:
image: ghcr.io/wuesteon/maerchenzauber-backend:latest
container_name: maerchenzauber-backend
ports:
- '3003:3003'
environment:
- NODE_ENV=staging
- PORT=3003
# ... env vars
depends_on:
- mana-core-auth
restart: unless-stopped
```
**Deploy all services**:
```bash
cd ~/manacore-staging
docker compose pull
docker compose up -d
docker compose ps # Should show all services running
```
---
## Phase 3: Web Apps (Day 3-4)
### 3.1 Create SvelteKit Dockerfiles
```bash
# Copy template for each web app
cp docker/templates/Dockerfile.sveltekit apps/chat/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/maerchenzauber/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/manadeck/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/memoro/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/picture/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/wisekeep/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/quote/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/uload/apps/web/Dockerfile
cp docker/templates/Dockerfile.sveltekit apps/manacore/apps/web/Dockerfile
```
**Test one build**:
```bash
docker build -t test-web -f apps/chat/apps/web/Dockerfile .
docker run -p 3000:3000 -e PUBLIC_SUPABASE_URL=your-url test-web
# Visit http://localhost:3000
```
### 3.2 Create Astro Dockerfiles
```bash
# Copy template for each landing page
cp docker/templates/Dockerfile.astro apps/chat/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/maerchenzauber/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/memoro/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/picture/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/wisekeep/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/quote/apps/landing/Dockerfile
cp docker/templates/Dockerfile.astro apps/bauntown/Dockerfile
```
### 3.3 Configure Domains and SSL
**In Docker Compose configuration**:
1. Add a new "Resource" → "Service"
2. For each web app/landing:
- Set domain (e.g., `chat.mana.how`)
- Enable "Generate SSL"
- Set Docker image: `ghcr.io/wuesteon/chat-web:latest`
- Configure environment variables
- Deploy
**Or configure Nginx reverse proxy manually** (see `docs/DEPLOYMENT.md` for details)
---
## Phase 4: Testing (Day 5)
### 4.1 Set Up Test Configuration
1. Install test dependencies:
```bash
pnpm install
```
2. The test configs in `packages/test-config/` are ready to use.
3. Configure each project to use shared configs.
**For NestJS backends**, add to `apps/PROJECT/apps/backend/package.json`:
```json
{
"scripts": {
"test": "jest",
"test:cov": "jest --coverage"
},
"jest": {
"preset": "@manacore/test-config/jest.config.backend.js"
}
}
```
### 4.2 Write Critical Path Tests (100% Coverage)
**Focus on `@manacore/shared-auth` package first**:
```bash
cd packages/shared-auth
mkdir -p src/__tests__
# Write tests for:
# - Token generation
# - Token validation
# - Token refresh
# - JWT utilities
# - AuthService
# Run tests
pnpm test:cov
# Verify 100% coverage
```
**Use test examples** from `docs/test-examples/` as reference.
### 4.3 Enable Coverage in CI
The `test.yml` workflow is already configured. Just ensure your tests are running:
```bash
# Test locally first
pnpm test
# Push and create PR
git add .
git commit -m "test: add auth package tests"
git push
```
GitHub Actions will automatically run tests and enforce coverage.
---
## Phase 5: Production (Day 6-7)
### 5.1 Provision Production Server
Repeat the Hetzner setup, but:
- Project name: `manacore-production`
- Server type: CCX42 (16 vCPU, 64 GB RAM, $100/month)
- Or CCX32 if resources sufficient
- Server name: `production-01`
### 5.2 Configure Production Secrets
Add these secrets to GitHub (with `PRODUCTION_` prefix):
```
PRODUCTION_HOST
PRODUCTION_USER
PRODUCTION_SSH_KEY
PRODUCTION_SUPABASE_URL
PRODUCTION_SUPABASE_ANON_KEY
PRODUCTION_SUPABASE_SERVICE_ROLE_KEY
PRODUCTION_JWT_SECRET (different from staging!)
PRODUCTION_MANA_SERVICE_URL
PRODUCTION_AZURE_OPENAI_ENDPOINT
PRODUCTION_AZURE_OPENAI_API_KEY
PRODUCTION_REDIS_PASSWORD
```
### 5.3 Set Up GitHub Environments
1. Go to Settings → Environments → New environment
2. Create "production-approval" environment:
- Add yourself as required reviewer
- Add your colleague as required reviewer
3. Create "production" environment:
- Deployment branches: `main` only
### 5.4 Deploy to Production
1. Go to Actions → "CD - Production Deployment"
2. Click "Run workflow"
3. Service: `mana-core-auth`
4. Environment: `production`
5. Confirmation: Type "deploy"
6. Click "Run workflow"
7. Approve when prompted
8. Watch deployment
9. Verify health checks
**Repeat for all services**!
---
## Verification
### Quick Health Check
**Check all services**:
```bash
# On server
cd ~/manacore-staging # or ~/manacore-production
docker compose ps
docker compose logs --tail=50
# From local machine
curl http://YOUR_SERVER_IP:3001/api/v1/health # mana-core-auth
curl http://YOUR_SERVER_IP:3002/api/health # chat-backend
# etc...
```
### Comprehensive Verification
1. **All containers running**:
```bash
docker compose ps
# All should show "Up" status
```
2. **Health checks passing**:
```bash
for service in mana-core-auth chat-backend maerchenzauber-backend; do
echo "Checking $service..."
docker compose exec $service wget -q -O - http://localhost:3001/api/v1/health || echo "FAILED"
done
```
3. **Resource usage acceptable**:
```bash
docker stats --no-stream
# CPU should be < 50%, Memory < 80%
```
4. **Logs clean** (no critical errors):
```bash
docker compose logs --tail=100 | grep -i error
```
5. **Web apps accessible**:
- Visit each domain in browser
- Test basic functionality
---
## Troubleshooting
### Issue: Docker build fails
**Symptom**: "ERROR: failed to solve"
**Solutions**:
1. Check Dockerfile syntax
2. Ensure you're running from monorepo root
3. Check for missing dependencies in package.json
4. Try building with no cache: `docker build --no-cache`
**See**: `docs/DOCKER_GUIDE.md` section 6 for more
---
### Issue: GitHub Actions fails
**Symptom**: Red X on PR, workflow fails
**Solutions**:
1. Check workflow logs in GitHub Actions tab
2. Verify all secrets are configured
3. Check if build works locally first
4. Ensure correct image names (ghcr.io/wuesteon/...)
**See**: `docs/CI_CD_SETUP.md` section 6 for more
---
### Issue: Deployment fails with "permission denied"
**Symptom**: Can't connect to server via SSH in workflow
**Solutions**:
1. Verify `STAGING_SSH_KEY` secret contains **private** key
2. Ensure key includes `-----BEGIN` and `-----END` lines
3. Verify `STAGING_USER` is correct (usually `root`)
4. Test SSH manually: `ssh root@SERVER_IP`
---
### Issue: Service unhealthy after deployment
**Symptom**: Health check endpoint returns 500 or times out
**Solutions**:
1. Check logs: `docker compose logs service-name --tail=100`
2. Verify environment variables are set correctly
3. Check if database connection works
4. Ensure port is correct
5. Try restarting: `docker compose restart service-name`
**See**: `docs/DEPLOYMENT.md` section 4 for more
---
### Issue: Can't pull Docker images on server
**Symptom**: "unauthorized: unauthenticated"
**Solutions**:
1. Login to ghcr.io on server:
```bash
echo YOUR_PAT | docker login ghcr.io -u wuesteon --password-stdin
```
2. Verify PAT has `read:packages` scope
3. Check image exists: `https://github.com/wuesteon?tab=packages`
**See**: `DOCKER_REGISTRY_SETUP.md` for details
---
## Next Steps
After completing setup:
1. ✅ Review `TODO.md` and mark completed tasks
2. ✅ Update `CHANGELOG.md` with your progress
3. ✅ Train your colleague using this guide
4. ✅ Set up monitoring (Phase 6 in TODO.md)
5. ✅ Implement remaining tests (Phase 4 in TODO.md)
6. ✅ Optimize performance (caching, CDN)
---
## Support
**Stuck? Need help?**
1. Check `TROUBLESHOOTING.md` (when created)
2. Review relevant documentation in `docs/`
3. Check GitHub Actions logs
4. Check Docker logs on server
5. Review Hive Mind Final Report: `/HIVE_MIND_FINAL_REPORT.md`
---
**Last Updated**: 2025-11-27
**Status**: Ready to use
**Estimated Time**: 30 minutes (quick start) to 7 days (full implementation)

642
cicd/TODO.md
View file

@ -1,642 +0,0 @@
# CI/CD Implementation TODO
**Last Updated**: 2025-11-27
**Overall Progress**: 70% Complete
---
## 🎯 How to Use This File
- [ ] Tasks not started are unchecked
- [x] Completed tasks are checked
- 🔥 High priority items
- ⚡ Quick wins (< 30 minutes)
- 🧪 Testing required
- 📝 Documentation needed
**Tip**: Start with Phase 1 Quick Wins for immediate progress!
---
## Phase 1: Infrastructure Foundation (Week 1)
**Goal**: Set up basic infrastructure and validate CI/CD pipeline
### 1.1 Hetzner Account Setup ⚡
- [ ] 🔥 Create Hetzner Cloud account
- [ ] Add payment method
- [ ] Verify account (may require ID verification)
- [ ] Choose data center region (EU for GDPR compliance recommended)
- [ ] **Estimated time**: 15 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 1.2 Provision Staging Server 🔥
- [ ] Create Hetzner CCX32 server (8 vCPU, 32 GB RAM, $50/month)
- OS: Ubuntu 22.04 LTS
- Location: Falkenstein, Germany (or nearest to your team)
- SSH key: Add your public key during creation
- [ ] Note down server IP address: `___________________`
- [ ] Test SSH connection: `ssh root@SERVER_IP`
- [ ] Update system: `apt update && apt upgrade -y`
- [ ] **Estimated time**: 20 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 1.3 Install Docker & Docker Compose on Staging 🔥
- [ ] Install Docker: `curl -fsSL https://get.docker.com | bash`
- [ ] Add user to docker group: `usermod -aG docker $USER`
- [ ] Install Docker Compose: `apt-get update && apt-get install docker-compose-plugin`
- [ ] Verify installation: `docker --version && docker compose version`
- [ ] Test Docker: `docker run hello-world`
- [ ] **Estimated time**: 15 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 1.4 GitHub Secrets Configuration 🔥
- [ ] ⚡ Create Personal Access Token (PAT) for GitHub Container Registry
- GitHub → Settings → Developer settings → Personal access tokens
- Scope: `read:packages`, `write:packages`
- Save token securely: `___________________`
- [ ] Add required secrets to GitHub repo (Settings → Secrets → Actions):
**Staging Secrets** (9 required):
- [ ] `STAGING_HOST` = Your server IP
- [ ] `STAGING_USER` = `root` (or created user)
- [ ] `STAGING_SSH_KEY` = Your private SSH key
- [ ] `STAGING_SUPABASE_URL` = Your Supabase project URL
- [ ] `STAGING_SUPABASE_ANON_KEY` = Supabase anon key
- [ ] `STAGING_SUPABASE_SERVICE_ROLE_KEY` = Supabase service role key
- [ ] `STAGING_JWT_SECRET` = Generate: `openssl rand -base64 64`
- [ ] `STAGING_MANA_SERVICE_URL` = `http://mana-core-auth:3001`
- [ ] `STAGING_AZURE_OPENAI_ENDPOINT` = Your Azure endpoint
- [ ] `STAGING_AZURE_OPENAI_API_KEY` = Your Azure API key
**GitHub Container Registry** (already configured):
- [x] `GITHUB_TOKEN` = Automatically available ✅
- [ ] **Estimated time**: 30 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 1.5 Create First Dockerfile 🔥
- [ ] Choose first service to deploy: **mana-core-auth** (recommended)
- [ ] Copy Dockerfile template: `cp docker/templates/Dockerfile.nestjs services/mana-core-auth/Dockerfile`
- [ ] Customize Dockerfile for mana-core-auth:
- [ ] Update `WORKDIR` path
- [ ] Adjust `package.json` copy paths
- [ ] Set correct `PORT` (default: 3001)
- [ ] 🧪 Test build locally: `docker build -t test-auth -f services/mana-core-auth/Dockerfile .`
- [ ] 🧪 Test run locally: `docker run -p 3001:3001 test-auth`
- [ ] Verify health endpoint: `curl http://localhost:3001/api/v1/health`
- [ ] **Estimated time**: 45 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 1.6 Test CI/CD Pipeline ⚡🔥
- [ ] Create test branch: `git checkout -b test/ci-cd-setup`
- [ ] Make small change to trigger CI (e.g., add comment to README)
- [ ] Push to GitHub: `git push origin test/ci-cd-setup`
- [ ] Create Pull Request
- [ ] Watch GitHub Actions run:
- [ ] Verify lint passes
- [ ] Verify type-check passes
- [ ] Verify build passes
- [ ] Verify tests run (may have some failures - OK for now)
- [ ] Merge to main
- [ ] Watch `ci-main.yml` workflow:
- [ ] Verify Docker image builds
- [ ] Verify push to ghcr.io succeeds
- [ ] Check GitHub Packages for new image
- [ ] **Estimated time**: 30 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 2: First Deployment (Week 1-2)
**Goal**: Deploy first service to staging and validate deployment process
### 2.1 Prepare docker-compose for Staging
- [ ] Review `docker-compose.staging.yml`
- [ ] Update image references to use ghcr.io:
```yaml
image: ghcr.io/wuesteon/mana-core-auth:latest
```
- [ ] Configure environment variables (use `.env.development` as reference)
- [ ] Set up networks and volumes
- [ ] **Estimated time**: 30 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 2.2 Deploy mana-core-auth to Staging 🔥
- [ ] 🧪 Trigger staging deployment workflow manually:
- GitHub → Actions → "CD - Staging Deployment" → Run workflow
- Select service: `mana-core-auth`
- [ ] Watch deployment logs
- [ ] Troubleshoot any errors (see `TROUBLESHOOTING.md`)
- [ ] Verify deployment success
- [ ] **Estimated time**: 45 minutes (including troubleshooting)
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 2.3 Verify Deployed Service 🧪
- [ ] SSH into staging server: `ssh root@STAGING_IP`
- [ ] Check running containers: `cd ~/manacore-staging && docker compose ps`
- [ ] Check logs: `docker compose logs mana-core-auth --tail=50`
- [ ] Test health endpoint from server: `curl http://localhost:3001/api/v1/health`
- [ ] Test health endpoint externally: `curl http://STAGING_IP:3001/api/v1/health`
- [ ] Verify database connection (if applicable)
- [ ] **Estimated time**: 20 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 2.4 Set Up Remaining NestJS Backends
- [ ] Create Dockerfiles for remaining backends:
- [ ] `apps/maerchenzauber/apps/backend/Dockerfile`
- [ ] `apps/chat/apps/backend/Dockerfile`
- [ ] `apps/manadeck/apps/backend/Dockerfile`
- [ ] `apps/nutriphi/apps/backend/Dockerfile`
- [ ] `apps/wisekeep/apps/backend/Dockerfile` (if exists)
- [ ] `apps/quote/apps/backend/Dockerfile` (if exists)
- [ ] 🧪 Test each build locally
- [ ] Commit and push to trigger CI builds
- [ ] Verify all images appear in GitHub Packages
- [ ] **Estimated time**: 2-3 hours (can be parallelized)
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 2.5 Deploy All Backend Services to Staging
- [ ] Update `docker-compose.staging.yml` to include all backend services
- [ ] Trigger deployment: Select "all" in workflow
- [ ] Verify all services running: `docker compose ps`
- [ ] Test each health endpoint
- [ ] Check resource usage: `docker stats`
- [ ] **Estimated time**: 1 hour
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 3: Web Apps & Landing Pages (Week 2)
**Goal**: Deploy SvelteKit web apps and Astro landing pages
### 3.1 Create SvelteKit Dockerfiles
- [ ] Create Dockerfiles for web apps:
- [ ] `apps/maerchenzauber/apps/web/Dockerfile`
- [ ] `apps/chat/apps/web/Dockerfile`
- [ ] `apps/manadeck/apps/web/Dockerfile`
- [ ] `apps/memoro/apps/web/Dockerfile`
- [ ] `apps/picture/apps/web/Dockerfile`
- [ ] `apps/wisekeep/apps/web/Dockerfile` (if exists)
- [ ] `apps/quote/apps/web/Dockerfile` (if exists)
- [ ] `apps/uload/apps/web/Dockerfile`
- [ ] Copy from template: `docker/templates/Dockerfile.sveltekit`
- [ ] Customize each for project-specific needs
- [ ] 🧪 Test builds locally
- [ ] **Estimated time**: 2-3 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 3.2 Create Astro Dockerfiles
- [ ] Create Dockerfiles for landing pages:
- [ ] `apps/maerchenzauber/apps/landing/Dockerfile`
- [ ] `apps/chat/apps/landing/Dockerfile`
- [ ] `apps/memoro/apps/landing/Dockerfile`
- [ ] `apps/picture/apps/landing/Dockerfile`
- [ ] `apps/wisekeep/apps/landing/Dockerfile` (if exists)
- [ ] `apps/quote/apps/landing/Dockerfile` (if exists)
- [ ] `apps/bauntown/Dockerfile` (community site)
- [ ] Copy from template: `docker/templates/Dockerfile.astro`
- [ ] 🧪 Test builds locally
- [ ] **Estimated time**: 1-2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 3.3 Configure Reverse Proxy (Traefik/Nginx)
- [ ] Plan domain structure:
- `chat.mana.how` → Chat web app
- `api-chat.mana.how` → Chat backend
- `maerchenzauber.com` → Landing page
- `app.maerchenzauber.com` → Web app
- etc.
- [ ] Set up Traefik in docker-compose (see docker-compose.production.yml)
- [ ] Configure domain routing labels in Docker Compose services
- [ ] Generate SSL certificates (Let's Encrypt via Traefik)
- [ ] Configure CORS for API endpoints
- [ ] **Estimated time**: 1-2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 3.4 Deploy Web Apps to Staging
- [ ] Add web apps to `docker-compose.staging.yml`
- [ ] Configure environment variables for each web app
- [ ] Deploy all web apps
- [ ] 🧪 Test each web app in browser
- [ ] Verify API connections work
- [ ] **Estimated time**: 2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 4: Testing Infrastructure (Week 2-3)
**Goal**: Implement automated testing across all projects
### 4.1 Set Up Test Configurations
- [ ] Review `packages/test-config/` package
- [ ] Install test dependencies:
```bash
pnpm add -D vitest @vitest/ui jest @types/jest --filter @manacore/test-config
```
- [ ] Configure each project to use shared configs:
- [ ] mana-core-auth: Jest (backend)
- [ ] maerchenzauber: Jest + Vitest (backend + mobile + web)
- [ ] chat: Jest + Vitest
- [ ] etc.
- [ ] **Estimated time**: 1 hour
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 4.2 Write Critical Path Tests (100% Coverage Required) 🔥
- [ ] **@manacore/shared-auth package**:
- [ ] Token generation tests
- [ ] Token validation tests
- [ ] Token refresh tests
- [ ] JWT utilities tests
- [ ] AuthService tests
- Target: 100% coverage
- [ ] **Payment/Credit System** (if applicable):
- [ ] Credit consumption tests
- [ ] Stripe integration tests (use mocks)
- [ ] Payment webhook tests
- Target: 100% coverage
- [ ] Run coverage: `pnpm --filter @manacore/shared-auth test:cov`
- [ ] **Estimated time**: 4-6 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 4.3 Backend Tests (80% Coverage Target)
- [ ] mana-core-auth service:
- [ ] Controller tests
- [ ] Service tests
- [ ] Integration tests
- [ ] Other backend services (use test examples as reference):
- [ ] Copy patterns from `docs/test-examples/backend/`
- [ ] Write controller tests
- [ ] Write service tests
- [ ] Aim for 80% coverage across all backends
- [ ] **Estimated time**: 8-12 hours (can be distributed)
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 4.4 Frontend Tests (80% Coverage Target)
- [ ] Mobile apps (React Native):
- [ ] Component tests
- [ ] Service tests
- [ ] Navigation tests
- [ ] Use patterns from `docs/test-examples/mobile/`
- [ ] Web apps (SvelteKit):
- [ ] Component tests (Svelte 5 runes)
- [ ] Page tests
- [ ] Server function tests
- [ ] Use patterns from `docs/test-examples/web/`
- [ ] **Estimated time**: 12-16 hours (can be distributed)
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 4.5 Enable Coverage Enforcement in CI
- [ ] Verify `test.yml` workflow is configured
- [ ] Set coverage thresholds in test configs (80%)
- [ ] Test PR workflow with coverage check
- [ ] Make coverage a required check for PRs
- [ ] Set up Codecov integration (optional but recommended)
- [ ] **Estimated time**: 1 hour
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 5: Production Deployment (Week 3)
**Goal**: Deploy to production environment
### 5.1 Provision Production Server
- [ ] Create Hetzner CCX42 server (16 vCPU, 64 GB RAM, $100/month)
- OR reuse CCX32 if resources sufficient
- [ ] Install Docker & Docker Compose on production server
- [ ] Configure firewall rules (only 22, 80, 443)
- [ ] Set up SSH key access
- [ ] Clone repository and set up deployment directory
- [ ] **Estimated time**: 30 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 5.2 Configure Production Secrets
- [ ] Add production secrets to GitHub:
- [ ] `PRODUCTION_HOST`
- [ ] `PRODUCTION_USER`
- [ ] `PRODUCTION_SSH_KEY`
- [ ] `PRODUCTION_SUPABASE_URL`
- [ ] `PRODUCTION_SUPABASE_ANON_KEY`
- [ ] `PRODUCTION_SUPABASE_SERVICE_ROLE_KEY`
- [ ] `PRODUCTION_JWT_SECRET` (different from staging!)
- [ ] `PRODUCTION_MANA_SERVICE_URL`
- [ ] `PRODUCTION_AZURE_OPENAI_ENDPOINT`
- [ ] `PRODUCTION_AZURE_OPENAI_API_KEY`
- [ ] `PRODUCTION_REDIS_PASSWORD`
- [ ] **Estimated time**: 20 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 5.3 Set Up GitHub Environments
- [ ] Create "production-approval" environment in GitHub:
- Settings → Environments → New environment
- Name: `production-approval`
- Add required reviewers (yourself + colleague)
- [ ] Create "production" environment:
- Add protection rules
- Set deployment branch to `main` only
- [ ] **Estimated time**: 10 minutes
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 5.4 First Production Deployment 🔥
- [ ] Deploy mana-core-auth to production:
- GitHub → Actions → "CD - Production Deployment"
- Service: `mana-core-auth`
- Type "deploy" to confirm
- Approve deployment when prompted
- [ ] Watch deployment progress
- [ ] Verify health checks pass
- [ ] Test endpoints externally
- [ ] Monitor for 1 hour (as per workflow)
- [ ] **Estimated time**: 1.5 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 5.5 Deploy All Services to Production
- [ ] Deploy remaining backend services
- [ ] Deploy web apps
- [ ] Deploy landing pages
- [ ] Configure DNS for all domains
- [ ] Verify SSL certificates
- [ ] **Estimated time**: 3-4 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 6: Monitoring & Optimization (Week 4+)
**Goal**: Set up monitoring and optimize performance
### 6.1 Set Up Monitoring
- [ ] Install Prometheus on monitoring server (or same server)
- [ ] Install Grafana
- [ ] Configure Prometheus to scrape all services
- [ ] Import Grafana dashboards for:
- [ ] Docker containers
- [ ] NestJS applications
- [ ] PostgreSQL
- [ ] Redis
- [ ] System metrics (CPU, RAM, disk)
- [ ] **Estimated time**: 2-3 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 6.2 Set Up Logging
- [ ] Install Loki for log aggregation
- [ ] Configure all services to output structured JSON logs
- [ ] Set up Grafana Loki data source
- [ ] Create log dashboards
- [ ] **Estimated time**: 2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 6.3 Set Up Alerting
- [ ] Configure Prometheus Alertmanager
- [ ] Set up Slack/Discord webhook for alerts
- [ ] Define alert rules:
- [ ] Service down (health check fails)
- [ ] High CPU usage (> 80% for 5 minutes)
- [ ] High memory usage (> 90%)
- [ ] Disk space low (< 10%)
- [ ] High error rate (> 5% of requests)
- [ ] Test alerts
- [ ] **Estimated time**: 2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 6.4 Error Tracking
- [ ] Set up Sentry account (free tier)
- [ ] Install Sentry SDK in backend services
- [ ] Install Sentry SDK in frontend apps
- [ ] Configure source maps for better error tracking
- [ ] Test error reporting
- [ ] **Estimated time**: 2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 6.5 Performance Optimization
- [ ] Set up Redis for caching
- [ ] Implement caching for frequently accessed data
- [ ] Configure CDN (Cloudflare) for static assets
- [ ] Optimize Docker image sizes (already using multi-stage builds)
- [ ] Set up database connection pooling (PgBouncer)
- [ ] **Estimated time**: 4-6 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 7: Backup & Disaster Recovery (Week 4+)
**Goal**: Ensure data safety and quick recovery
### 7.1 Automated Backups
- [ ] Review backup scripts in `scripts/deploy/`
- [ ] Set up automated daily backups:
- [ ] PostgreSQL databases
- [ ] Redis data
- [ ] Docker volumes
- [ ] Environment configurations
- [ ] Configure backup retention (30 days for databases, 7 days for Redis)
- [ ] Set up Cloudflare R2 or Hetzner Storage Box for backup storage
- [ ] **Estimated time**: 2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 7.2 Test Backup Restoration
- [ ] 🧪 Perform test restoration on staging:
- [ ] Restore PostgreSQL backup
- [ ] Restore Redis backup
- [ ] Verify data integrity
- [ ] Document restoration procedure
- [ ] Time the restoration process (should be < 1 hour)
- [ ] **Estimated time**: 1-2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 7.3 Disaster Recovery Drill
- [ ] 🧪 Simulate production outage
- [ ] Practice rollback procedure using `scripts/deploy/rollback.sh`
- [ ] Practice full server restoration from backup
- [ ] Document lessons learned
- [ ] Update runbooks based on findings
- [ ] **Estimated time**: 2-3 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Phase 8: Documentation & Handoff (Ongoing)
**Goal**: Ensure team can maintain and extend the system
### 8.1 Update Documentation
- [ ] 📝 Update `COMPLETED.md` with all finished tasks
- [ ] 📝 Update `CHANGELOG.md` with timeline
- [ ] 📝 Document any deviations from original plan
- [ ] 📝 Create troubleshooting entries for issues encountered
- [ ] **Estimated time**: 1 hour
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 8.2 Team Training
- [ ] Schedule training session for colleague
- [ ] Walk through:
- [ ] GitHub Actions workflows
- [ ] Deployment procedures
- [ ] Rollback procedures
- [ ] Monitoring dashboards
- [ ] Alert response
- [ ] **Estimated time**: 2-3 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
### 8.3 Runbook Creation
- [ ] Create runbooks for common operations:
- [ ] Deploy new service
- [ ] Roll back deployment
- [ ] Restore from backup
- [ ] Scale service
- [ ] Respond to alerts
- [ ] Store in `cicd/runbooks/`
- [ ] **Estimated time**: 2 hours
- [ ] **Assignee**: \***\*\_\*\***
- [ ] **Due date**: \***\*\_\*\***
---
## Optional Enhancements (Future)
### Mobile App Deployment
- [ ] Set up Expo EAS for OTA updates
- [ ] Configure app store deployment (iOS/Android)
- [ ] Set up TestFlight/Google Play beta testing
### Advanced Testing
- [ ] Set up E2E testing with Playwright
- [ ] Set up mobile E2E testing with Detox/Maestro
- [ ] Implement visual regression testing
- [ ] Set up load testing with k6
### Advanced CI/CD
- [ ] Implement canary deployments
- [ ] Set up feature flags (LaunchDarkly/Unleash)
- [ ] Implement automated performance regression detection
- [ ] Set up multi-region deployment
### Developer Experience
- [ ] Set up Husky pre-commit hooks
- [ ] Configure Commitlint
- [ ] Create VSCode tasks for common operations
- [ ] Set up local development with Tilt or Skaffold
---
## Progress Summary
**Phase 1**: ☐ Not Started | 6 tasks
**Phase 2**: ☐ Not Started | 5 tasks
**Phase 3**: ☐ Not Started | 4 tasks
**Phase 4**: ☐ Not Started | 5 tasks
**Phase 5**: ☐ Not Started | 5 tasks
**Phase 6**: ☐ Not Started | 5 tasks
**Phase 7**: ☐ Not Started | 3 tasks
**Phase 8**: ☐ Not Started | 3 tasks
**Total Core Tasks**: 36
**Total Optional Tasks**: 12
**Estimated Total Time**: 40-60 hours (1-2 weeks for 2 people)
---
## Notes & Blockers
**Current Blockers**:
- [ ] Waiting for: \***\*\_\*\***
- [ ] Blocked by: \***\*\_\*\***
**Important Decisions Needed**:
- [ ] Final domain names for all projects
- [ ] Budget approval for Hetzner servers
- [ ] Supabase project setup for each app
**Questions**:
- [ ] ***
- [ ] ***
---
**Last Updated**: 2025-11-27
**Next Review**: \***\*\_\*\***
**Owned By**: \***\*\_\*\***

610
cicd/TYPE_CHECK_ANALYSIS.md
View file

@ -1,610 +0,0 @@
# TypeScript Type Check Analysis Report
**Generated:** 2025-11-27
**Total Errors:** 673 errors across 11 projects
**CI Status:** FAILING (exit code 1)
---
## Executive Summary
The CI/CD pipeline is failing at the "Type Check" step due to 673 TypeScript errors distributed across 11 projects. The errors fall into clear patterns that can be addressed systematically.
### High-Level Issues
1. **Missing Dependencies** - 56 errors (TS2307)
2. **Missing Properties** - 102 errors (TS2339)
3. **Type Mismatches** - 75 errors (TS2304, TS2322)
4. **Configuration Issues** - 2 errors (Vitest config, Turbo.json)
---
## 1. Error Distribution by Project
### Critical (High Error Count)
| Project | Errors | Priority | Status |
|---------|--------|----------|--------|
| `apps/maerchenzauber/apps/mobile` | 278 | P0 | BLOCKING |
| `apps/picture/apps/web` | 115 | P0 | BLOCKING |
| `apps/picture/apps/mobile` | 111 | P0 | BLOCKING |
| `apps/presi/apps/mobile` | 74 | P1 | BLOCKING |
| `apps/maerchenzauber/apps/backend` | 33 | P1 | BLOCKING |
| `apps/nutriphi/apps/mobile` | 29 | P2 | BLOCKING |
| `apps/chat/apps/web` | 26 | P2 | BLOCKING |
### Low Impact (Minor Issues)
| Project | Errors | Priority | Status |
|---------|--------|----------|--------|
| `games/voxel-lava/apps/web` | 3 | P3 | Minor |
| `packages/test-config` | 2 | P3 | Minor |
| `apps/wisekeep/apps/web` | 1 | P4 | Trivial |
| `apps/nutriphi/apps/web` | 1 | P4 | Trivial |
---
## 2. Error Categorization by Type
### Top Error Codes (by frequency)
| Code | Count | Type | Description |
|------|-------|------|-------------|
| TS2339 | 102 | Property | Property does not exist on type |
| TS2304 | 75 | Type | Cannot find name/type |
| TS2322 | 63 | Assignment | Type not assignable to type |
| TS2307 | 56 | Module | Cannot find module |
| TS2769 | 53 | Overload | No overload matches call |
| TS2551 | 49 | Property | Property does not exist (suggestion) |
| TS2345 | 26 | Argument | Argument type not assignable |
| TS7006 | 24 | Implicit Any | Parameter has implicit 'any' |
| TS7031 | 17 | Index | Not all paths return value |
| TS18048 | 9 | Nullable | Possibly undefined |
| TS18046 | 9 | Nullable | Possibly null/undefined |
---
## 3. Pattern Analysis
### 3.1 Missing Module Patterns (TS2307: 56 occurrences)
#### Missing Internal Packages (CRITICAL)
```
10× @mana-core/nestjs-integration
```
**Impact:** Breaks `apps/maerchenzauber/apps/backend`
**Files Affected:**
- `src/app.module.ts`
- `src/character/character.controller.ts`
- `src/creators/creators.controller.ts`
- `src/feedback/feedback.controller.ts`
- `src/story/story.controller.ts`
- Multiple test files
**Root Cause:** Package likely exists but not properly exported or installed
**Fix:** Verify package exists in `packages/` and is properly referenced in package.json
#### Firebase Dependencies (CRITICAL)
```
7× ../firebaseConfig
4× firebase/firestore
2× firebase/auth
2× firebase/storage
1× firebase-admin
```
**Impact:** Breaks mobile apps (presi, maerchenzauber)
**Root Cause:** Firebase imports in code but Firebase SDK not installed or incomplete migration to Supabase
**Fix Strategy:**
- **Option 1:** Complete Firebase → Supabase migration (remove all Firebase code)
- **Option 2:** Add Firebase dependencies to package.json
- **Recommended:** Option 1 (align with architecture)
#### Missing React Native Dependencies (HIGH)
```
5× react-native-blurhash
2× react-native-vector-icons/MaterialIcons
2× react-native-svg
1× @react-native-async-storage/async-storage
```
**Impact:** Breaks mobile app UI components
**Fix:** Add missing dependencies to mobile app package.json files
#### Test Dependencies (MEDIUM)
```
6× @jest/globals
```
**Impact:** Test files fail type checking
**Fix:** Add `@jest/globals` to devDependencies
#### Other Missing Modules (LOW)
```
3× dotenv
1× axios
1× @picture/design-tokens
```
**Fix:** Add to respective package.json files
---
### 3.2 Missing Property Patterns (TS2339: 102 occurrences)
#### Theme System Issues (CRITICAL - 6 occurrences)
```typescript
// apps/presi/apps/mobile
Property 'border' does not exist on type 'ThemeColors'
Property 'background' does not exist on type 'ThemeColors'
```
**Files Affected:**
- `components/common/ContextMenu.tsx`
- `components/forms/CreateDeckForm.tsx`
- `components/slides/SlideEditor.tsx`
- `components/decks/DeckShareSettings.tsx`
**Root Cause:** Theme type definition incomplete or outdated
**Fix:** Update `ThemeProvider.tsx` or theme types to include missing properties
#### Auth/User Property Mismatches (HIGH - 11 occurrences)
```typescript
// Common patterns:
Property 'sub' does not exist on type 'User' (5×)
Property 'uid' does not exist on type 'User' (3×)
Property 'isAuthenticated' does not exist on type (3×)
Property 'deckCount' does not exist on type 'object' (1×)
```
**Root Cause:** Mismatch between Mana Core Auth user type and Supabase/Firebase user types
**Fix:** Create proper type definitions for auth user objects
#### API Response Type Mismatches (HIGH - 15 occurrences)
```typescript
Property 'data' does not exist on type 'Result<any[]>' (15×)
```
**Root Cause:** Result type wrapper not matching API response expectations
**Fix:** Update Result type or unwrap responses properly
#### UI State Properties (MEDIUM - 7 occurrences)
```typescript
Property 'hovered' does not exist on type 'PressableStateCallbackType' (5×)
Property 'disabled' does not exist on type (3×)
```
**Root Cause:** React Native types version mismatch or incorrect usage
**Fix:** Check React Native version compatibility
---
### 3.3 Type Assignment Issues (TS2322: 63 occurrences)
#### Chat App Type Conflicts (26 errors in chat/apps/web)
**Pattern:**
```typescript
Type 'Model[]' is not assignable to type 'AIModel[]'
Type 'Conversation[]' is not assignable to type 'Conversation[]'
Type 'Template' is not assignable to type 'Template'
```
**Root Cause:** Duplicate type definitions (one in API layer, one in types package)
**Files Affected:**
- `src/lib/stores/chat.svelte.ts`
- `src/lib/stores/conversations.svelte.ts`
- `src/lib/stores/templates.svelte.ts`
- `src/lib/components/templates/TemplateForm.svelte`
- `src/routes/(protected)/chat/+page.svelte`
- `src/routes/(protected)/chat/[id]/+page.svelte`
**Fix:** Unify type definitions - use single source of truth
#### Picture App Type Conflicts (115 errors in picture/apps/web)
Similar pattern to chat app - likely duplicate type definitions.
---
### 3.4 Configuration Issues (2 errors)
#### Vitest Config - `packages/test-config`
```typescript
vitest.config.base.ts(75,5): error TS2769:
Object literal may only specify known properties,
and 'watchExclude' does not exist in type 'InlineConfig'.
vitest.config.svelte.ts(70,5): error TS2769:
Same error
```
**Root Cause:** `watchExclude` removed or renamed in Vitest v3.0+
**Fix:** Remove `watchExclude` or replace with current API
#### Turbo.json - `apps/presi/turbo.json`
```
Invalid turbo.json configuration
No "extends" key found.
```
**Root Cause:** Local turbo.json overriding root config without extending
**Fix:** Add `"extends": "../../turbo.json"` to presi/turbo.json
---
## 4. Prioritized Fix Recommendations
### Phase 1: Blockers (P0 - Required for CI to pass)
#### 1.1 Fix Turbo Configuration (10 min)
**File:** `/Users/wuesteon/dev/mana_universe/manacore-monorepo/apps/presi/turbo.json`
**Action:** Add extends key
```json
{
"extends": "../../turbo.json",
"$schema": "https://turbo.build/schema.json",
"tasks": {
// ... existing config
}
}
```
**Alternative:** Delete `apps/presi/turbo.json` entirely if no custom config needed.
#### 1.2 Fix Test Config (15 min)
**Files:**
- `packages/test-config/vitest.config.base.ts`
- `packages/test-config/vitest.config.svelte.ts`
**Action:** Remove `watchExclude` property (deprecated in Vitest 3.0)
#### 1.3 Add Missing NestJS Integration Package (30 min)
**Investigation needed:**
- Check if `@mana-core/nestjs-integration` exists in workspace
- If not, create it or use correct package name
- Update all imports in `apps/maerchenzauber/apps/backend`
### Phase 2: Critical Fixes (P0/P1 - Major functionality)
#### 2.1 Complete Firebase Migration (2-4 hours per app)
**Affected Apps:**
- `apps/presi/apps/mobile`
- `apps/maerchenzauber/apps/mobile`
**Actions:**
1. Remove all Firebase imports
2. Replace with Supabase equivalents
3. Update auth flows to use Mana Core Auth
4. Remove `firebaseConfig` files
5. Update storage to use Supabase Storage
**Files to delete:**
- `firebaseConfig.ts` (all instances)
- `services/auth.ts` (Firebase-based)
- `services/firestore.ts`
- `services/storage.ts` (Firebase-based)
**Files to update:**
- Replace with Supabase/Mana Core equivalents
#### 2.2 Add Missing React Native Dependencies (15 min)
**Packages to install:**
```bash
# For maerchenzauber/apps/mobile
pnpm add react-native-blurhash react-native-vector-icons --filter @maerchenzauber/mobile
# For presi/apps/mobile
pnpm add react-native-vector-icons @react-native-async-storage/async-storage --filter @presi/mobile
# For picture/apps/mobile
pnpm add react-native-blurhash react-native-svg --filter @picture/mobile
```
#### 2.3 Fix Chat App Type Conflicts (1-2 hours)
**Strategy:**
1. Identify canonical type location (choose one):
- `apps/chat/packages/chat-types/` (recommended)
- `apps/chat/apps/web/src/lib/services/api.ts`
2. Delete duplicate definitions
3. Update all imports to use canonical source
**Types to unify:**
- `AIModel` / `Model`
- `Conversation`
- `Template`
- `Message`
- `Document`
- `Space`
#### 2.4 Fix Picture App Type Conflicts (1-2 hours)
Similar approach to Chat app - unify type definitions.
### Phase 3: Medium Priority (P2 - Quality improvements)
#### 3.1 Fix Theme System (30 min)
**File:** `apps/presi/apps/mobile/components/ThemeProvider.tsx`
**Action:** Add missing properties to theme type definition
```typescript
interface ThemeColors {
// ... existing properties
border: string;
background: string;
}
```
#### 3.2 Fix Auth Type Mismatches (1 hour)
**Create unified auth types:**
```typescript
// packages/shared-auth/src/types.ts
export interface ManaUser {
id: string;
sub: string; // JWT subject
uid: string; // User ID (alias for id)
email: string;
// ... other properties
}
```
Update all auth stores to use this type.
#### 3.3 Add Missing Test Dependencies (10 min)
```bash
pnpm add -D @jest/globals --filter @maerchenzauber/backend
```
### Phase 4: Low Priority (P3/P4 - Polish)
#### 4.1 Fix Implicit Any Types (30 min)
24 occurrences of parameters with `any` type - add proper types.
#### 4.2 Fix Voxel-Lava Minor Issues (15 min)
3 errors related to:
- `LevelMetadata.userId` nullable vs non-nullable
- `AuthService.updatePassword` missing method
#### 4.3 Fix Nutriphi/Wisekeep Minor Issues (10 min)
1-2 trivial type errors each.
---
## 5. Systematic Fix Patterns
### Pattern 1: Missing Module Dependencies
**Detection:**
```bash
grep "error TS2307" cicd/type-check-output.txt
```
**Fix Template:**
1. Identify package name
2. Run: `pnpm add <package> --filter <app-name>`
3. Verify import paths are correct
### Pattern 2: Type Import Conflicts
**Detection:**
```bash
grep "is not assignable to type" cicd/type-check-output.txt
```
**Fix Template:**
1. Find both type definitions
2. Choose canonical source
3. Delete duplicate
4. Update imports
### Pattern 3: Missing Properties on Types
**Detection:**
```bash
grep "Property '.*' does not exist" cicd/type-check-output.txt
```
**Fix Template:**
1. Locate type definition
2. Add missing property with correct type
3. Or remove usage if property doesn't belong
---
## 6. Estimated Fix Timeline
| Phase | Work | Time Estimate |
|-------|------|---------------|
| Phase 1 (Blockers) | Config fixes, missing packages | 1-2 hours |
| Phase 2 (Critical) | Firebase migration, type unification | 8-12 hours |
| Phase 3 (Medium) | Theme system, auth types | 2-3 hours |
| Phase 4 (Low) | Polish, minor fixes | 1-2 hours |
| **TOTAL** | | **12-19 hours** |
### Recommended Approach
**Option A: Quick Pass (Get CI Green - 4-6 hours)**
1. Fix turbo.json (Phase 1.1)
2. Fix test-config (Phase 1.2)
3. Add missing dependencies (Phase 2.2)
4. Temporarily exclude problematic apps from type-check in turbo.json
5. File issues for remaining errors
**Option B: Comprehensive Fix (Full resolution - 12-19 hours)**
1. Execute all phases in order
2. Systematic elimination of all errors
3. Update documentation
4. Add type checking to pre-commit hooks
---
## 7. Files Requiring Immediate Attention
### Configuration Files (BLOCKING)
```
/apps/presi/turbo.json [ADD extends key]
/packages/test-config/vitest.config.base.ts [REMOVE watchExclude]
/packages/test-config/vitest.config.svelte.ts [REMOVE watchExclude]
```
### Missing Dependencies (BLOCKING)
```
/apps/maerchenzauber/apps/backend/package.json [ADD @mana-core/nestjs-integration]
/apps/maerchenzauber/apps/mobile/package.json [ADD react-native-blurhash]
/apps/presi/apps/mobile/package.json [ADD react-native-vector-icons]
/apps/picture/apps/mobile/package.json [ADD react-native-blurhash, react-native-svg]
```
### Type Conflicts (HIGH PRIORITY)
```
/apps/chat/apps/web/src/lib/services/api.ts [DEDUPE types]
/apps/chat/packages/chat-types/src/index.ts [CANONICAL source]
/apps/picture/apps/web/ [DEDUPE types]
```
### Firebase Migration (HIGH PRIORITY)
```
/apps/presi/apps/mobile/firebaseConfig.ts [DELETE - migrate to Supabase]
/apps/presi/apps/mobile/services/auth.ts [DELETE - use Mana Core Auth]
/apps/presi/apps/mobile/services/firestore.ts [DELETE - use Supabase]
/apps/presi/apps/mobile/services/storage.ts [DELETE - use Supabase Storage]
/apps/maerchenzauber/apps/mobile/firebaseConfig.ts [DELETE]
```
---
## 8. Next Steps
### Immediate Actions (within 24 hours)
1. ✅ **Fix Turbo Config** - Unblock type-check command
2. ✅ **Fix Test Config** - Remove deprecated Vitest options
3. ⬜ **Investigate @mana-core/nestjs-integration** - Find or create package
4. ⬜ **Add missing dependencies** - Unblock mobile apps
### Short-term (within 1 week)
5. ⬜ **Complete Firebase migration** - Align with architecture
6. ⬜ **Unify Chat app types** - Fix 26 errors
7. ⬜ **Unify Picture app types** - Fix 115 errors
### Medium-term (within 2 weeks)
8. ⬜ **Fix theme system** - Standardize across apps
9. ⬜ **Fix auth types** - Create canonical ManaUser type
10. ⬜ **Add pre-commit type checking** - Prevent regressions
---
## 9. Recommendations for Prevention
1. **Enable type checking in pre-commit hooks**
- Add to Husky configuration
- Run on changed files only for speed
2. **Add type checking to PR template checklist**
- Require `pnpm type-check` to pass
- Add to GitHub Actions PR checks
3. **Establish canonical type locations**
- Document where shared types live
- Prevent duplicate type definitions
- Use `@manacore/shared-types` for cross-app types
4. **Regular dependency audits**
- Check for missing peer dependencies
- Verify all imports resolve correctly
- Update TypeScript regularly
5. **Architectural alignment**
- Complete Firebase → Supabase migration
- Standardize auth implementation
- Consistent theme system across apps
---
## Appendix A: Error Code Reference
| Code | Category | Description | Common Fixes |
|------|----------|-------------|--------------|
| TS2307 | Module | Cannot find module | Add dependency, fix import path |
| TS2339 | Property | Property does not exist | Add property to type, fix property name |
| TS2322 | Assignment | Type not assignable | Fix type mismatch, use type assertion |
| TS2304 | Type | Cannot find name | Import type, fix typo |
| TS2769 | Overload | No matching overload | Fix function arguments |
| TS2551 | Property | Property missing (with suggestion) | Use suggested property |
| TS7006 | Any | Implicit any | Add type annotation |
| TS2345 | Argument | Wrong argument type | Fix argument type |
---
## Appendix B: Full Error Counts
```
102 TS2339 Property does not exist on type
75 TS2304 Cannot find name
63 TS2322 Type is not assignable to type
56 TS2307 Cannot find module
53 TS2769 No overload matches this call
49 TS2551 Property does not exist (suggestion)
26 TS2345 Argument type not assignable
24 TS7006 Parameter implicitly has 'any' type
17 TS7031 Not all code paths return a value
9 TS18048 Possibly 'undefined'
9 TS18046 Possibly 'null' or 'undefined'
6 TS2694 Namespace has no exported member
5 TS2741 Property is missing in type
4 TS7019 Rest parameter implicitly has 'any[]' type
4 TS2683 'this' implicitly has type 'any'
4 TS2353 Object literal may only specify known properties
3 TS2580 Cannot find name (suggestion)
3 TS2459 Module declares locally but is not exported
3 TS2305 Module has no exported member
2 TS2454 Variable is used before being assigned
2 TS2448 Block-scoped variable used before declaration
1 TS7053 Element implicitly has 'any' type
1 TS2801 Expected 1 arguments, but got 2
1 TS2774 Condition will always return true
1 TS2740 Type is missing properties
1 TS2445 Property is protected
1 TS2352 Conversion may be a mistake
1 TS2349 Cannot invoke expression
1 TS1149 Class expression expected
```
---
**Report End**

169
docker/caddy/Caddyfile.production
View file

@ -1,169 +0,0 @@
# ManaCore Production Reverse Proxy
# Domain: mana.how
# Server: 46.224.108.214
#
# Deploy to: ~/Caddyfile on production server
# Reload with: docker exec caddy caddy reload --config /etc/caddy/Caddyfile
# PWA files must never be cached by the browser/CDN so that
# service worker updates are picked up immediately after deploys.
(pwa-no-cache) {
@pwa-files path /sw.js /manifest.webmanifest /registerSW.js
header @pwa-files Cache-Control "no-cache, no-store, must-revalidate"
}
# ============================================
# Auth Service
# ============================================
auth.mana.how {
reverse_proxy localhost:3001
}
# ============================================
# ManaCore Dashboard (Main)
# ============================================
mana.how {
import pwa-no-cache
reverse_proxy localhost:5000
}
www.mana.how {
redir https://mana.how{uri} permanent
}
# ============================================
# Chat App
# ============================================
chat.mana.how {
import pwa-no-cache
reverse_proxy localhost:5010
}
chat-api.mana.how {
reverse_proxy localhost:3030
}
# ============================================
# Todo App
# ============================================
todo.mana.how {
import pwa-no-cache
reverse_proxy localhost:5011
}
todo-api.mana.how {
reverse_proxy localhost:3031
}
# ============================================
# Calendar App
# ============================================
calendar.mana.how {
import pwa-no-cache
reverse_proxy localhost:5012
}
calendar-api.mana.how {
reverse_proxy localhost:3032
}
# ============================================
# Clock App
# ============================================
clock.mana.how {
import pwa-no-cache
reverse_proxy localhost:5013
}
clock-api.mana.how {
reverse_proxy localhost:3033
}
clock-bot.mana.how {
reverse_proxy localhost:4018
}
# ============================================
# Contacts App
# ============================================
contacts.mana.how {
import pwa-no-cache
reverse_proxy localhost:5014
}
contacts-api.mana.how {
reverse_proxy localhost:3034
}
# ============================================
# Storage App
# ============================================
storage.mana.how {
import pwa-no-cache
reverse_proxy localhost:5015
}
storage-api.mana.how {
reverse_proxy localhost:3035
}
# ============================================
# Skilltree App
# ============================================
skilltree.mana.how {
import pwa-no-cache
reverse_proxy localhost:5020
}
skilltree-api.mana.how {
reverse_proxy localhost:3038
}
# ============================================
# Mukke App
# ============================================
mukke.mana.how {
import pwa-no-cache
reverse_proxy localhost:5180
}
mukke-api.mana.how {
reverse_proxy localhost:3010
}
# ============================================
# Picture App
# ============================================
picture.mana.how {
import pwa-no-cache
reverse_proxy localhost:5021
}
picture-api.mana.how {
reverse_proxy localhost:3040
}
# ============================================
# LLM Playground
# ============================================
playground.mana.how {
reverse_proxy localhost:5090
}
# ============================================
# Games
# ============================================
whopxl.mana.how {
reverse_proxy localhost:5100
}
# ============================================
# Monitoring & Analytics
# ============================================
grafana.mana.how {
reverse_proxy localhost:8000
}
stats.mana.how {
reverse_proxy localhost:8010
}

45
docker/caddy/Caddyfile.staging
View file

@ -1,45 +0,0 @@
# ManaCore Staging Reverse Proxy
# Deploy to: ~/Caddyfile on staging server (46.224.108.214)
# Reload with: docker exec caddy caddy reload --config /etc/caddy/Caddyfile
# Auth service
auth.staging.manacore.ai {
reverse_proxy localhost:3001
}
# Chat
chat.staging.manacore.ai {
reverse_proxy localhost:3000
}
chat-api.staging.manacore.ai {
reverse_proxy localhost:3002
}
# ManaCore main
staging.manacore.ai {
reverse_proxy localhost:5173
}
# Calendar
calendar.staging.manacore.ai {
reverse_proxy localhost:5186
}
calendar-api.staging.manacore.ai {
reverse_proxy localhost:3016
}
# Clock
clock.staging.manacore.ai {
reverse_proxy localhost:5187
}
clock-api.staging.manacore.ai {
reverse_proxy localhost:3017
}
# Todo
todo.staging.manacore.ai {
reverse_proxy localhost:5188
}
todo-api.staging.manacore.ai {
reverse_proxy localhost:3018
}

2
docs/ANALYTICS.md
View file

@ -421,4 +421,4 @@ Im Development-Modus ist Umami normalerweise nicht geladen (kein Script-Tag), da
- **Container**: `mana-mon-umami`
- **Image**: `ghcr.io/umami-software/umami:postgresql-latest`
- **Datenbank**: PostgreSQL (`umami` DB, shared Postgres-Instanz)
- **Port**: 3000 (intern) → 8010 (extern), via Caddy erreichbar unter stats.mana.how
- **Port**: 3000 (intern) → 8010 (extern), via Cloudflare Tunnel erreichbar unter stats.mana.how

27
docs/URL_SCHEMA.md
View file

@ -128,36 +128,17 @@ pnpm deploy:landing:all
### Web Apps & APIs
**Platform:** Hetzner Server (46.224.108.214) via Docker + Caddy
See [PRODUCTION_LAUNCH.md](./PRODUCTION_LAUNCH.md) for deployment details.
**Platform:** Mac Mini (self-hosted) via Docker + Cloudflare Tunnel
---
## DNS Configuration
All subdomains point to Hetzner server, except landing pages which use Cloudflare:
Web apps and APIs are routed via Cloudflare Tunnel to the Mac Mini. Landing pages use Cloudflare Pages.
### A Records (Hetzner - Web Apps & APIs)
### Cloudflare Tunnel (Web Apps & APIs)
```
calendar.mana.how A 46.224.108.214
calendar-api.mana.how A 46.224.108.214
clock.mana.how A 46.224.108.214
clock-api.mana.how A 46.224.108.214
todo.mana.how A 46.224.108.214
todo-api.mana.how A 46.224.108.214
contact.mana.how A 46.224.108.214
contact-api.mana.how A 46.224.108.214
chat.mana.how A 46.224.108.214
chat-api.mana.how A 46.224.108.214
picture.mana.how A 46.224.108.214
picture-api.mana.how A 46.224.108.214
zitare.mana.how A 46.224.108.214
zitare-api.mana.how A 46.224.108.214
app.mana.how A 46.224.108.214
auth.mana.how A 46.224.108.214
```
All `*.mana.how` subdomains are routed via Cloudflare Tunnel to the Mac Mini Docker containers. No A records needed — Cloudflare manages the DNS.
### CNAME Records (Cloudflare - Landing Pages)

170
scripts/deploy/build-and-push.sh
View file

@ -1,170 +0,0 @@
#!/bin/bash
# Build and push Docker images for manacore services
# Usage: ./build-and-push.sh [service] [tag]
# Example: ./build-and-push.sh chat-backend v1.0.0
# Example: ./build-and-push.sh all latest
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Configuration
DOCKER_REGISTRY=${DOCKER_REGISTRY:-"wuesteon"}
SERVICE=${1:-"all"}
TAG=${2:-"latest"}
PLATFORM=${PLATFORM:-"linux/amd64"}
# Function to print colored output
log_info() {
echo -e "${GREEN}[INFO]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Function to build and push a service
build_and_push() {
local service=$1
local dockerfile=$2
local context=${3:-.}
local image_name="${DOCKER_REGISTRY}/${service}"
log_info "Building ${service}..."
# Build the image
if docker buildx build \
--platform ${PLATFORM} \
--tag "${image_name}:${TAG}" \
--tag "${image_name}:latest" \
--file "${dockerfile}" \
--progress plain \
${context}; then
log_info "Successfully built ${service}"
# Push the image
log_info "Pushing ${service} to registry..."
if docker push "${image_name}:${TAG}" && docker push "${image_name}:latest"; then
log_info "Successfully pushed ${service}"
return 0
else
log_error "Failed to push ${service}"
return 1
fi
else
log_error "Failed to build ${service}"
return 1
fi
}
# Function to build all services
build_all() {
local services=(
"mana-core-auth:services/mana-core-auth/Dockerfile"
"maerchenzauber-backend:apps/maerchenzauber/apps/backend/Dockerfile"
"chat-backend:apps/chat/apps/backend/Dockerfile"
"manadeck-backend:apps/manadeck/apps/backend/Dockerfile"
"nutriphi-backend:apps/nutriphi/apps/backend/Dockerfile"
"news-api:apps/news/apps/api/Dockerfile"
)
local failed_services=()
for service_config in "${services[@]}"; do
IFS=':' read -r service dockerfile <<< "$service_config"
if [ -f "$dockerfile" ]; then
if ! build_and_push "$service" "$dockerfile" "."; then
failed_services+=("$service")
fi
else
log_warn "Dockerfile not found for ${service}: ${dockerfile}"
fi
echo ""
done
# Report results
echo ""
echo "=========================================="
if [ ${#failed_services[@]} -eq 0 ]; then
log_info "All services built and pushed successfully!"
return 0
else
log_error "Failed to build/push the following services:"
for service in "${failed_services[@]}"; do
echo " - ${service}"
done
return 1
fi
}
# Check if Docker is installed
if ! command -v docker &> /dev/null; then
log_error "Docker is not installed or not in PATH"
exit 1
fi
# Check if buildx is available
if ! docker buildx version &> /dev/null; then
log_error "Docker buildx is not available"
log_info "Install it with: docker buildx install"
exit 1
fi
# Login to Docker registry
if [ -n "${DOCKER_USERNAME}" ] && [ -n "${DOCKER_PASSWORD}" ]; then
log_info "Logging in to Docker registry..."
echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
fi
# Main execution
log_info "Starting build and push process..."
log_info "Registry: ${DOCKER_REGISTRY}"
log_info "Tag: ${TAG}"
log_info "Platform: ${PLATFORM}"
echo ""
if [ "$SERVICE" == "all" ]; then
build_all
else
# Build specific service
case "$SERVICE" in
"mana-core-auth")
build_and_push "mana-core-auth" "services/mana-core-auth/Dockerfile" "."
;;
"maerchenzauber-backend")
build_and_push "maerchenzauber-backend" "apps/maerchenzauber/apps/backend/Dockerfile" "."
;;
"chat-backend")
build_and_push "chat-backend" "apps/chat/apps/backend/Dockerfile" "."
;;
"manadeck-backend")
build_and_push "manadeck-backend" "apps/manadeck/apps/backend/Dockerfile" "."
;;
"nutriphi-backend")
build_and_push "nutriphi-backend" "apps/nutriphi/apps/backend/Dockerfile" "."
;;
"news-api")
build_and_push "news-api" "apps/news/apps/api/Dockerfile" "."
;;
*)
log_error "Unknown service: $SERVICE"
echo "Available services: all, mana-core-auth, maerchenzauber-backend, chat-backend, manadeck-backend, nutriphi-backend, news-api"
exit 1
;;
esac
fi
log_info "Build and push process completed!"

201
scripts/deploy/deploy-hetzner.sh
View file

@ -1,201 +0,0 @@
#!/bin/bash
# Deploy to Hetzner server via SSH
# Usage: ./deploy-hetzner.sh [environment] [service]
# Example: ./deploy-hetzner.sh staging all
# Example: ./deploy-hetzner.sh production chat-backend
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Configuration
ENVIRONMENT=${1:-"staging"}
SERVICE=${2:-"all"}
# Environment-specific variables
if [ "$ENVIRONMENT" == "production" ]; then
SSH_HOST=${PRODUCTION_HOST}
SSH_USER=${PRODUCTION_USER}
SSH_KEY=${PRODUCTION_SSH_KEY}
DEPLOY_DIR="~/manacore-production"
COMPOSE_FILE="docker-compose.production.yml"
else
SSH_HOST=${STAGING_HOST}
SSH_USER=${STAGING_USER}
SSH_KEY=${STAGING_SSH_KEY}
DEPLOY_DIR="~/manacore-staging"
COMPOSE_FILE="docker-compose.staging.yml"
fi
# Function to print colored output
log_info() {
echo -e "${GREEN}[INFO]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Validate required variables
if [ -z "$SSH_HOST" ] || [ -z "$SSH_USER" ]; then
log_error "SSH configuration missing for ${ENVIRONMENT}"
log_error "Please set: ${ENVIRONMENT^^}_HOST and ${ENVIRONMENT^^}_USER"
exit 1
fi
# SSH command helper
ssh_exec() {
if [ -n "$SSH_KEY" ]; then
ssh -i "$SSH_KEY" -o StrictHostKeyChecking=no "${SSH_USER}@${SSH_HOST}" "$@"
else
ssh -o StrictHostKeyChecking=no "${SSH_USER}@${SSH_HOST}" "$@"
fi
}
# SCP command helper
scp_copy() {
if [ -n "$SSH_KEY" ]; then
scp -i "$SSH_KEY" -o StrictHostKeyChecking=no "$@"
else
scp -o StrictHostKeyChecking=no "$@"
fi
}
log_info "Starting deployment to ${ENVIRONMENT}..."
log_info "Target: ${SSH_USER}@${SSH_HOST}"
log_info "Service: ${SERVICE}"
echo ""
# Step 1: Prepare deployment directory
log_info "Preparing deployment directory..."
ssh_exec << EOF
mkdir -p ${DEPLOY_DIR}
mkdir -p ${DEPLOY_DIR}/logs
mkdir -p ${DEPLOY_DIR}/backups
cd ${DEPLOY_DIR}
EOF
# Step 2: Copy docker-compose file
log_info "Copying docker-compose configuration..."
scp_copy "${COMPOSE_FILE}" "${SSH_USER}@${SSH_HOST}:${DEPLOY_DIR}/docker-compose.yml"
# Step 3: Copy environment file if exists
if [ -f ".env.${ENVIRONMENT}" ]; then
log_info "Copying environment configuration..."
scp_copy ".env.${ENVIRONMENT}" "${SSH_USER}@${SSH_HOST}:${DEPLOY_DIR}/.env"
else
log_warn "No .env.${ENVIRONMENT} file found, using existing environment"
fi
# Step 4: Pull latest images
log_info "Pulling latest Docker images..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
docker compose pull ${SERVICE}
EOF
# Step 5: Run migrations if needed
if [ "$SERVICE" == "all" ] || [ "$SERVICE" == "mana-core-auth" ]; then
log_info "Running database migrations..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
docker compose run --rm mana-core-auth pnpm run db:migrate || echo "Migrations completed or skipped"
EOF
fi
# Step 6: Deploy services
log_info "Deploying services..."
if [ "$SERVICE" == "all" ]; then
# Zero-downtime rolling update for all services
ssh_exec << 'EOF'
cd ${DEPLOY_DIR}
SERVICES=$(docker compose config --services)
for service in $SERVICES; do
echo "Deploying $service..."
# Scale up with new version
docker compose up -d --no-deps --scale $service=2 $service
sleep 15
# Scale down to single instance
docker compose up -d --no-deps --scale $service=1 $service
sleep 5
done
# Cleanup old images
docker image prune -f
EOF
else
# Deploy single service
ssh_exec << EOF
cd ${DEPLOY_DIR}
docker compose up -d --no-deps ${SERVICE}
sleep 10
EOF
fi
# Step 7: Health checks
log_info "Running health checks..."
HEALTH_ENDPOINTS=(
"mana-core-auth:3001:/api/v1/health"
"maerchenzauber-backend:3002:/health"
"chat-backend:3002:/api/health"
)
FAILED_CHECKS=0
for endpoint in "${HEALTH_ENDPOINTS[@]}"; do
IFS=':' read -r service port path <<< "$endpoint"
log_info "Checking health of ${service}..."
if ssh_exec << EOF
HEALTH=\$(docker compose -f ${DEPLOY_DIR}/docker-compose.yml exec -T ${service} wget -q -O - http://localhost:${port}${path} 2>/dev/null || echo "FAILED")
if [[ "\$HEALTH" == *"FAILED"* ]]; then
echo "Health check failed for ${service}"
exit 1
else
echo "Health check passed for ${service}"
exit 0
fi
EOF
then
log_info "✅ ${service} is healthy"
else
log_error "❌ ${service} health check failed"
((FAILED_CHECKS++))
fi
done
echo ""
# Step 8: Display service status
log_info "Current service status:"
ssh_exec << EOF
cd ${DEPLOY_DIR}
docker compose ps
EOF
echo ""
# Final result
if [ $FAILED_CHECKS -eq 0 ]; then
log_info "Deployment to ${ENVIRONMENT} completed successfully! ✅"
exit 0
else
log_error "Deployment completed with ${FAILED_CHECKS} failed health checks"
log_warn "Please check service logs with: ssh ${SSH_USER}@${SSH_HOST} 'cd ${DEPLOY_DIR} && docker compose logs'"
exit 1
fi

88
scripts/deploy/health-check.sh
View file

@ -1,88 +0,0 @@
#!/bin/bash
# Health check script for deployed services
# Usage: ./health-check.sh [environment]
# Example: ./health-check.sh staging
# Example: ./health-check.sh production
set -e
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
ENVIRONMENT=${1:-"staging"}
# Environment-specific configuration
if [ "$ENVIRONMENT" == "production" ]; then
BASE_URL=${PRODUCTION_API_URL:-"https://api.mana.how"}
else
BASE_URL=${STAGING_API_URL:-"https://staging.mana.how"}
fi
log_info() {
echo -e "${GREEN}[INFO]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Health check endpoints
declare -A ENDPOINTS=(
["Mana Core Auth"]="/api/v1/health"
["Maerchenzauber Backend"]="/health"
["Chat Backend"]="/api/health"
)
# Counter for failed checks
FAILED=0
TOTAL=0
log_info "Running health checks for ${ENVIRONMENT}..."
log_info "Base URL: ${BASE_URL}"
echo ""
# Check each endpoint
for service in "${!ENDPOINTS[@]}"; do
endpoint="${ENDPOINTS[$service]}"
url="${BASE_URL}${endpoint}"
((TOTAL++))
log_info "Checking ${service}..."
# Make HTTP request
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" "${url}" -m 10 || echo "000")
if [ "$HTTP_CODE" == "200" ]; then
log_info "✅ ${service}: OK (HTTP ${HTTP_CODE})"
else
log_error "❌ ${service}: FAILED (HTTP ${HTTP_CODE})"
((FAILED++))
fi
echo ""
done
# Summary
echo "=========================================="
log_info "Health Check Summary:"
echo " Total checks: ${TOTAL}"
echo " Passed: $((TOTAL - FAILED))"
echo " Failed: ${FAILED}"
echo "=========================================="
if [ $FAILED -eq 0 ]; then
log_info "All health checks passed! ✅"
exit 0
else
log_error "${FAILED} health check(s) failed ❌"
exit 1
fi

116
scripts/deploy/migrate-db.sh
View file

@ -1,116 +0,0 @@
#!/bin/bash
# Database migration script for Supabase projects
# Usage: ./migrate-db.sh [project] [environment]
# Example: ./migrate-db.sh chat staging
# Example: ./migrate-db.sh mana-core-auth production
set -e
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
PROJECT=${1}
ENVIRONMENT=${2:-"staging"}
log_info() {
echo -e "${GREEN}[INFO]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Validate input
if [ -z "$PROJECT" ]; then
log_error "Project name is required"
echo "Usage: ./migrate-db.sh [project] [environment]"
echo "Available projects: chat, maerchenzauber, manadeck, memoro, picture, nutriphi, news, mana-core-auth"
exit 1
fi
log_info "Running database migrations for ${PROJECT} (${ENVIRONMENT})..."
# Set Supabase environment variables based on project and environment
case "$PROJECT" in
"chat")
if [ "$ENVIRONMENT" == "production" ]; then
SUPABASE_URL=${CHAT_SUPABASE_URL}
SUPABASE_SERVICE_KEY=${CHAT_SUPABASE_SERVICE_KEY}
else
SUPABASE_URL=${STAGING_CHAT_SUPABASE_URL}
SUPABASE_SERVICE_KEY=${STAGING_CHAT_SUPABASE_SERVICE_KEY}
fi
MIGRATION_DIR="apps/chat/supabase/migrations"
;;
"maerchenzauber")
if [ "$ENVIRONMENT" == "production" ]; then
SUPABASE_URL=${MAERCHENZAUBER_SUPABASE_URL}
SUPABASE_SERVICE_KEY=${MAERCHENZAUBER_SUPABASE_SERVICE_KEY}
else
SUPABASE_URL=${STAGING_MAERCHENZAUBER_SUPABASE_URL}
SUPABASE_SERVICE_KEY=${STAGING_MAERCHENZAUBER_SUPABASE_SERVICE_KEY}
fi
MIGRATION_DIR="apps/maerchenzauber/supabase/migrations"
;;
"mana-core-auth")
if [ "$ENVIRONMENT" == "production" ]; then
DATABASE_URL=${PRODUCTION_AUTH_DATABASE_URL}
else
DATABASE_URL=${STAGING_AUTH_DATABASE_URL}
fi
MIGRATION_DIR="services/mana-core-auth/src/db/migrations"
# Use Drizzle for mana-core-auth
log_info "Running Drizzle migrations for mana-core-auth..."
cd services/mana-core-auth
pnpm run db:migrate
exit 0
;;
*)
log_error "Unknown project: $PROJECT"
exit 1
;;
esac
# Check if migration directory exists
if [ ! -d "$MIGRATION_DIR" ]; then
log_warn "No migrations found for ${PROJECT}"
exit 0
fi
# Check for Supabase CLI
if ! command -v supabase &> /dev/null; then
log_error "Supabase CLI is not installed"
log_info "Install it with: npm install -g supabase"
exit 1
fi
# Link to remote project
log_info "Linking to Supabase project..."
supabase link --project-ref $(echo $SUPABASE_URL | sed 's|https://||' | sed 's|.supabase.co||')
# Run migrations
log_info "Applying migrations from ${MIGRATION_DIR}..."
cd $MIGRATION_DIR
# List pending migrations
log_info "Pending migrations:"
ls -1 *.sql 2>/dev/null || log_info "No SQL migrations found"
# Apply migrations using Supabase CLI
for migration in *.sql; do
if [ -f "$migration" ]; then
log_info "Applying migration: $migration"
supabase db push
fi
done
log_info "Database migrations completed successfully! ✅"

213
scripts/deploy/rollback.sh
View file

@ -1,213 +0,0 @@
#!/bin/bash
# Rollback script for emergency deployment rollback
# Usage: ./rollback.sh [environment] [service]
# Example: ./rollback.sh production all
# Example: ./rollback.sh staging chat-backend
set -e
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
ENVIRONMENT=${1:-"staging"}
SERVICE=${2:-"all"}
# Environment-specific variables
if [ "$ENVIRONMENT" == "production" ]; then
SSH_HOST=${PRODUCTION_HOST}
SSH_USER=${PRODUCTION_USER}
SSH_KEY=${PRODUCTION_SSH_KEY}
DEPLOY_DIR="~/manacore-production"
else
SSH_HOST=${STAGING_HOST}
SSH_USER=${STAGING_USER}
SSH_KEY=${STAGING_SSH_KEY}
DEPLOY_DIR="~/manacore-staging"
fi
log_info() {
echo -e "${GREEN}[INFO]${NC} $1"
}
log_warn() {
echo -e "${YELLOW}[WARN]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Validate required variables
if [ -z "$SSH_HOST" ] || [ -z "$SSH_USER" ]; then
log_error "SSH configuration missing for ${ENVIRONMENT}"
exit 1
fi
# SSH command helper
ssh_exec() {
if [ -n "$SSH_KEY" ]; then
ssh -i "$SSH_KEY" -o StrictHostKeyChecking=no "${SSH_USER}@${SSH_HOST}" "$@"
else
ssh -o StrictHostKeyChecking=no "${SSH_USER}@${SSH_HOST}" "$@"
fi
}
log_warn "⚠️ ROLLBACK INITIATED ⚠️"
log_info "Environment: ${ENVIRONMENT}"
log_info "Service: ${SERVICE}"
echo ""
# Confirm rollback
read -p "Are you sure you want to rollback? (yes/no): " confirm
if [ "$confirm" != "yes" ]; then
log_info "Rollback cancelled"
exit 0
fi
echo ""
log_info "Starting rollback process..."
# Step 1: Check for previous deployment backup
log_info "Checking for previous deployment backup..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
if [ ! -d "backups" ] || [ -z "\$(ls -A backups)" ]; then
echo "ERROR: No backups found!"
exit 1
fi
# Get latest backup
LATEST_BACKUP=\$(ls -t backups | head -1)
echo "Latest backup found: \$LATEST_BACKUP"
cd backups/\$LATEST_BACKUP
# Verify backup contents
if [ ! -f "docker-compose.yml" ]; then
echo "ERROR: Backup incomplete - missing docker-compose.yml"
exit 1
fi
echo "Backup validated"
EOF
# Step 2: Stop current services
log_info "Stopping current services..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
docker compose stop ${SERVICE}
EOF
# Step 3: Restore from backup
log_info "Restoring from backup..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
LATEST_BACKUP=\$(ls -t backups | head -1)
# Restore docker-compose file
cp backups/\$LATEST_BACKUP/docker-compose.yml ./docker-compose.yml
# Restore environment file if exists
if [ -f "backups/\$LATEST_BACKUP/.env.backup" ]; then
cp backups/\$LATEST_BACKUP/.env.backup ./.env
fi
echo "Files restored from backup: \$LATEST_BACKUP"
EOF
# Step 4: Restore database if service is auth
if [ "$SERVICE" == "all" ] || [ "$SERVICE" == "mana-core-auth" ]; then
log_info "Restoring database..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
LATEST_BACKUP=\$(ls -t backups | head -1)
if [ -f "backups/\$LATEST_BACKUP/postgres_backup.sql" ]; then
# Restore PostgreSQL backup
docker compose exec -T postgres psql -U \$POSTGRES_USER < backups/\$LATEST_BACKUP/postgres_backup.sql
echo "Database restored"
else
echo "WARNING: No database backup found"
fi
EOF
fi
# Step 5: Start services with previous images
log_info "Starting services with previous configuration..."
ssh_exec << EOF
cd ${DEPLOY_DIR}
# Get image tags from backup
LATEST_BACKUP=\$(ls -t backups | head -1)
if [ -f "backups/\$LATEST_BACKUP/deployment_images.txt" ]; then
echo "Previous deployment images:"
cat backups/\$LATEST_BACKUP/deployment_images.txt
fi
# Start services
docker compose up -d ${SERVICE}
# Wait for services to start
sleep 20
EOF
# Step 6: Health checks
log_info "Running health checks after rollback..."
HEALTH_ENDPOINTS=(
"mana-core-auth:3001:/api/v1/health"
"maerchenzauber-backend:3002:/health"
"chat-backend:3002:/api/health"
)
FAILED_CHECKS=0
for endpoint in "${HEALTH_ENDPOINTS[@]}"; do
IFS=':' read -r service port path <<< "$endpoint"
if ssh_exec << EOF
HEALTH=\$(docker compose -f ${DEPLOY_DIR}/docker-compose.yml exec -T ${service} wget -q -O - http://localhost:${port}${path} 2>/dev/null || echo "FAILED")
if [[ "\$HEALTH" == *"FAILED"* ]]; then
exit 1
else
exit 0
fi
EOF
then
log_info "✅ ${service} is healthy"
else
log_warn "⚠️ ${service} health check failed"
((FAILED_CHECKS++))
fi
done
echo ""
# Step 7: Display service status
log_info "Current service status:"
ssh_exec << EOF
cd ${DEPLOY_DIR}
docker compose ps
EOF
echo ""
# Final result
if [ $FAILED_CHECKS -eq 0 ]; then
log_info "Rollback completed successfully! ✅"
log_info "Services have been restored to previous version"
exit 0
else
log_error "Rollback completed with ${FAILED_CHECKS} failed health checks"
log_warn "Manual intervention may be required"
exit 1
fi

124
scripts/generate-staging-secrets.sh
View file

@ -1,124 +0,0 @@
#!/bin/bash
# Generate Staging Secrets for GitHub
# Run this script and copy the output to GitHub Secrets
set -e
echo "================================================"
echo " STAGING SECRETS GENERATOR"
echo "================================================"
echo ""
echo "Copy each value below to GitHub Settings → Secrets and variables → Actions"
echo ""
echo "Note: Configuration values (host, ports, etc.) are now hardcoded in the workflow"
echo "Only sensitive values (passwords, keys) need to be added as secrets"
echo ""
echo "================================================"
echo ""
# Generate secure random passwords
POSTGRES_PASSWORD=$(openssl rand -base64 32 | tr -d "=+/" | cut -c1-32)
REDIS_PASSWORD=$(openssl rand -base64 32 | tr -d "=+/" | cut -c1-32)
JWT_SECRET=$(openssl rand -base64 64 | tr -d "=+/" | cut -c1-64)
# Generate Ed25519 key pair for JWT
TEMP_KEY_DIR=$(mktemp -d)
ssh-keygen -t ed25519 -f "$TEMP_KEY_DIR/jwt_key" -N "" -C "manacore-staging-jwt" > /dev/null 2>&1
# Convert SSH keys to raw format for JWT
PRIVATE_KEY=$(cat "$TEMP_KEY_DIR/jwt_key" | grep -v "BEGIN" | grep -v "END" | tr -d '\n')
PUBLIC_KEY=$(ssh-keygen -e -m PKCS8 -f "$TEMP_KEY_DIR/jwt_key.pub" 2>/dev/null | grep -v "BEGIN" | grep -v "END" | tr -d '\n' || cat "$TEMP_KEY_DIR/jwt_key.pub" | awk '{print $2}')
# Clean up temp files
rm -rf "$TEMP_KEY_DIR"
# Output all secrets in GitHub format
echo "# ============================================"
echo "# DATABASE SECRETS (2 secrets)"
echo "# ============================================"
echo ""
echo "STAGING_POSTGRES_PASSWORD"
echo "$POSTGRES_PASSWORD"
echo ""
echo "# ============================================"
echo "# REDIS SECRETS (1 secret)"
echo "# ============================================"
echo ""
echo "STAGING_REDIS_PASSWORD"
echo "$REDIS_PASSWORD"
echo ""
echo "# ============================================"
echo "# MANA CORE AUTH SECRETS (3 secrets)"
echo "# ============================================"
echo ""
echo "STAGING_JWT_SECRET"
echo "$JWT_SECRET"
echo ""
echo "STAGING_JWT_PUBLIC_KEY"
echo "$PUBLIC_KEY"
echo ""
echo "STAGING_JWT_PRIVATE_KEY"
echo "$PRIVATE_KEY"
echo ""
echo "# ============================================"
echo "# SUPABASE SECRETS (Fill these manually - 3 secrets)"
echo "# ============================================"
echo ""
echo "STAGING_SUPABASE_URL"
echo "https://YOUR_PROJECT.supabase.co"
echo ""
echo "STAGING_SUPABASE_ANON_KEY"
echo "YOUR_SUPABASE_ANON_KEY_HERE"
echo ""
echo "STAGING_SUPABASE_SERVICE_ROLE_KEY"
echo "YOUR_SUPABASE_SERVICE_ROLE_KEY_HERE"
echo ""
echo "# ============================================"
echo "# AZURE OPENAI SECRETS (Fill these manually - 2 secrets)"
echo "# ============================================"
echo ""
echo "STAGING_AZURE_OPENAI_ENDPOINT"
echo "https://YOUR_RESOURCE.openai.azure.com/"
echo ""
echo "STAGING_AZURE_OPENAI_API_KEY"
echo "YOUR_AZURE_OPENAI_API_KEY_HERE"
echo ""
echo "# ============================================"
echo "# SSH DEPLOYMENT SECRETS (Fill these manually - 1 secret)"
echo "# ============================================"
echo ""
echo "STAGING_SSH_KEY"
echo "Run: cat ~/.ssh/hetzner_deploy_key"
echo "(Copy the ENTIRE output including -----BEGIN and -----END lines)"
echo ""
echo "================================================"
echo " SUMMARY"
echo "================================================"
echo ""
echo "Total secrets to add: 12"
echo " - Auto-generated: 6 (passwords, JWT keys)"
echo " - Manual: 6 (Supabase, Azure, SSH key)"
echo ""
echo "The following are now HARDCODED in the workflow:"
echo " - POSTGRES_HOST, POSTGRES_PORT, POSTGRES_DB, POSTGRES_USER"
echo " - REDIS_HOST, REDIS_PORT"
echo " - MANA_SERVICE_URL"
echo " - STAGING_HOST (46.224.108.214)"
echo " - STAGING_USER (deploy)"
echo ""
echo "================================================"
echo ""
echo "Next steps:"
echo "1. Go to: https://github.com/YOUR_ORG/manacore-monorepo/settings/secrets/actions"
echo "2. Click 'New repository secret' for each value above"
echo "3. Copy the secret name (e.g., STAGING_POSTGRES_PASSWORD)"
echo "4. Copy the secret value (the line below the name)"
echo "5. Fill in Supabase, Azure, and SSH key values manually"
echo ""