🧑‍💻 dx: add automatic database setup and dev:*:full commands

- Add scripts/setup-databases.sh for automatic DB creation and schema push
- Add dev:*:full commands (chat, zitare, contacts, calendar, clock, todo, picture)
- Update docker/init-db to create all databases on first startup
- Add docs/LOCAL_DEVELOPMENT.md with comprehensive local dev guide
- Update CLAUDE.md with new quick start commands

Now developers can run `pnpm dev:chat:full` to automatically:
1. Create the database if missing
2. Push the latest schema
3. Start auth, backend, and web with colored output
This commit is contained in:
Wuesteon 2025-12-08 14:38:18 +01:00
parent e423785a20
commit 67a15cc9ea
5 changed files with 517 additions and 20 deletions

View file

@ -60,19 +60,46 @@ These projects are temporarily archived and excluded from the workspace. To re-a
## Development Commands
```bash
# Install dependencies
pnpm install
For detailed local development setup, see **[docs/LOCAL_DEVELOPMENT.md](docs/LOCAL_DEVELOPMENT.md)**.
### Quick Start (Recommended)
Use `dev:*:full` commands to start any app with automatic database setup:
```bash
pnpm docker:up # Start PostgreSQL, Redis, MinIO
pnpm dev:chat:full # Start chat with auth + auto DB setup
pnpm dev:zitare:full # Start zitare with auth + auto DB setup
pnpm dev:contacts:full # Start contacts with auth + auto DB setup
pnpm dev:calendar:full # Start calendar with auth + auto DB setup
pnpm dev:clock:full # Start clock with auth + auto DB setup
pnpm dev:todo:full # Start todo with auth + auto DB setup
pnpm dev:picture:full # Start picture with auth + auto DB setup
```
These commands automatically:
1. Create the database if missing
2. Push the latest schema
3. Start auth, backend, and web with colored output
### Database Setup
```bash
pnpm setup:db # Setup ALL databases and schemas
pnpm setup:db:chat # Setup just chat
pnpm setup:db:auth # Setup just auth
```
### Individual App Commands
```bash
# Start specific project (runs all apps in project)
pnpm run manacore:dev
pnpm run manadeck:dev
pnpm run picture:dev
pnpm run chat:dev
pnpm run zitare:dev
pnpm run presi:dev
pnpm run contacts:dev
pnpm run mail:dev
# Start specific app within project
pnpm run dev:chat:mobile # Just mobile app
@ -607,6 +634,7 @@ PORT=...
## Project-Specific Documentation
- **[docs/LOCAL_DEVELOPMENT.md](docs/LOCAL_DEVELOPMENT.md)** - Database setup and `dev:*:full` commands
- **[docs/ENVIRONMENT_VARIABLES.md](docs/ENVIRONMENT_VARIABLES.md)** - Complete environment setup guide
Each project has its own `CLAUDE.md` with detailed information:

View file

@ -1,21 +1,37 @@
-- Create databases for all services
-- This script runs on first container initialization
-- Create chat database
CREATE DATABASE chat;
-- Create voxel_lava database
CREATE DATABASE voxel_lava;
-- Create storage database (cloud drive)
CREATE DATABASE storage;
-- Create todo database
CREATE DATABASE todo;
-- Core databases
CREATE DATABASE IF NOT EXISTS chat;
CREATE DATABASE IF NOT EXISTS zitare;
CREATE DATABASE IF NOT EXISTS contacts;
CREATE DATABASE IF NOT EXISTS calendar;
CREATE DATABASE IF NOT EXISTS clock;
CREATE DATABASE IF NOT EXISTS todo;
CREATE DATABASE IF NOT EXISTS manadeck;
CREATE DATABASE IF NOT EXISTS storage;
CREATE DATABASE IF NOT EXISTS mail;
CREATE DATABASE IF NOT EXISTS moodlit;
CREATE DATABASE IF NOT EXISTS finance;
CREATE DATABASE IF NOT EXISTS inventory;
CREATE DATABASE IF NOT EXISTS techbase;
CREATE DATABASE IF NOT EXISTS voxel_lava;
CREATE DATABASE IF NOT EXISTS figgos;
-- Grant all privileges to the default user
GRANT ALL PRIVILEGES ON DATABASE chat TO manacore;
GRANT ALL PRIVILEGES ON DATABASE voxel_lava TO manacore;
GRANT ALL PRIVILEGES ON DATABASE manacore TO manacore;
GRANT ALL PRIVILEGES ON DATABASE storage TO manacore;
GRANT ALL PRIVILEGES ON DATABASE zitare TO manacore;
GRANT ALL PRIVILEGES ON DATABASE contacts TO manacore;
GRANT ALL PRIVILEGES ON DATABASE calendar TO manacore;
GRANT ALL PRIVILEGES ON DATABASE clock TO manacore;
GRANT ALL PRIVILEGES ON DATABASE todo TO manacore;
GRANT ALL PRIVILEGES ON DATABASE manadeck TO manacore;
GRANT ALL PRIVILEGES ON DATABASE storage TO manacore;
GRANT ALL PRIVILEGES ON DATABASE mail TO manacore;
GRANT ALL PRIVILEGES ON DATABASE moodlit TO manacore;
GRANT ALL PRIVILEGES ON DATABASE finance TO manacore;
GRANT ALL PRIVILEGES ON DATABASE inventory TO manacore;
GRANT ALL PRIVILEGES ON DATABASE techbase TO manacore;
GRANT ALL PRIVILEGES ON DATABASE voxel_lava TO manacore;
GRANT ALL PRIVILEGES ON DATABASE figgos TO manacore;
GRANT ALL PRIVILEGES ON DATABASE manacore TO manacore;

275
docs/LOCAL_DEVELOPMENT.md Normal file
View file

@ -0,0 +1,275 @@
# Local Development Guide
This guide explains how to set up and run applications locally with automatic database setup.
## Quick Start
For any project with a backend, use the `dev:*:full` command:
```bash
pnpm dev:chat:full # Start chat with auth + database setup
pnpm dev:zitare:full # Start zitare with auth + database setup
pnpm dev:contacts:full # Start contacts with auth + database setup
# ... etc
```
These commands automatically:
1. Create the database if it doesn't exist
2. Push the latest schema (Drizzle `db:push`)
3. Start the auth service (mana-core-auth)
4. Start the backend and web app with colored output
## Available Full Dev Commands
| Command | Database | Backend Port | Web Port |
|---------|----------|--------------|----------|
| `pnpm dev:chat:full` | chat | 3002 | 5173 |
| `pnpm dev:zitare:full` | zitare | 3007 | 5177 |
| `pnpm dev:contacts:full` | contacts | 3015 | 5184 |
| `pnpm dev:calendar:full` | calendar | 3014 | 5179 |
| `pnpm dev:clock:full` | clock | 3017 | 5187 |
| `pnpm dev:todo:full` | todo | 3018 | 5188 |
| `pnpm dev:picture:full` | picture | 3006 | 5175 |
## Prerequisites
Before running any `dev:*:full` command:
```bash
# 1. Start Docker infrastructure (PostgreSQL, Redis, MinIO)
pnpm docker:up
# 2. Generate environment files (runs automatically on pnpm install)
pnpm setup:env
```
## Database Setup Commands
### Individual Service Setup
```bash
pnpm setup:db:auth # Setup mana-core-auth database + schema
pnpm setup:db:chat # Setup chat database + schema
pnpm setup:db:zitare # Setup zitare database + schema
pnpm setup:db:contacts # Setup contacts database + schema
pnpm setup:db:calendar # Setup calendar database + schema
pnpm setup:db:clock # Setup clock database + schema
pnpm setup:db:todo # Setup todo database + schema
pnpm setup:db:picture # Setup picture database + schema
```
### Setup All Databases
```bash
pnpm setup:db # Creates ALL databases and pushes ALL schemas
```
This is useful when setting up a fresh environment or after pulling new schema changes.
## How It Works
### Docker Init Script
On first `pnpm docker:up`, the PostgreSQL container runs `docker/init-db/01-create-databases.sql` which creates all databases:
- manacore, chat, zitare, contacts, calendar, clock, todo, manadeck
- storage, mail, moodlit, finance, inventory, techbase, voxel_lava, figgos
### Setup Script
The `scripts/setup-databases.sh` script:
1. **Creates database** if it doesn't exist (using `psql`)
2. **Pushes schema** using `drizzle-kit push --force`
The `--force` flag auto-approves schema changes without interactive prompts.
## Troubleshooting
### Database doesn't exist
If you see `database "xxx" does not exist`:
```bash
# Option 1: Run the setup script
pnpm setup:db:chat # or whichever service
# Option 2: Create manually
PGPASSWORD=devpassword psql -h localhost -U manacore -d postgres -c "CREATE DATABASE chat;"
```
### Schema out of date
If you see errors about missing tables/columns:
```bash
# Push the latest schema
pnpm --filter @chat/backend db:push --force
```
### Port already in use
If auth (port 3001) is already running:
```bash
# Check what's using the port
lsof -i :3001
# Kill the process if needed
kill <PID>
```
### Fresh Start (Nuclear Option)
To completely reset all databases:
```bash
# Stop and remove all containers + volumes
pnpm docker:clean
# Start fresh
pnpm docker:up
# Setup all databases
pnpm setup:db
```
## Apps Without Full Commands
Some apps don't have backends or don't use Drizzle:
| App | Reason |
|-----|--------|
| manacore | No backend (uses other services) |
| manadeck | Backend exists but no db:push |
| bauntown, context, maerchenzauber, memoro, news, nutriphi, presi, quote, reader, storage, wisekeep | No backends |
For these, use the regular dev commands:
```bash
pnpm dev:manacore:web
pnpm dev:manadeck:app
```
## Adding a New Application
### Step 1: Create Project Structure
Create the standard project structure under `apps/`:
```
apps/newproject/
├── apps/
│ ├── backend/ # NestJS API (if needed)
│ ├── mobile/ # Expo React Native app
│ ├── web/ # SvelteKit web app
│ └── landing/ # Astro marketing page
├── packages/ # Project-specific shared code
├── package.json # Workspace root
├── pnpm-workspace.yaml
└── CLAUDE.md # Project documentation
```
### Step 2: Configure Backend Database (if applicable)
If your backend uses Drizzle ORM:
1. **Add database to Docker init** (`docker/init-db/01-create-databases.sql`):
```sql
CREATE DATABASE IF NOT EXISTS newproject;
GRANT ALL PRIVILEGES ON DATABASE newproject TO manacore;
```
2. **Add to setup script** (`scripts/setup-databases.sh`):
In the `setup_service()` function, add a new case:
```bash
newproject)
create_db_if_not_exists "newproject"
push_schema "@newproject/backend" "newproject"
;;
```
Also add to the `ALL_DATABASES` array:
```bash
ALL_DATABASES=(
...
"newproject"
)
```
And to the services loop at the bottom:
```bash
for service in auth chat ... newproject; do
```
3. **Add DATABASE_URL to `.env.development`**:
```env
NEWPROJECT_DATABASE_URL=postgresql://manacore:devpassword@localhost:5432/newproject
```
4. **Update `scripts/generate-env.mjs`** to generate the backend `.env` file.
### Step 3: Add Package.json Scripts
Add to root `package.json`:
```json
{
"scripts": {
// Project-level dev (all apps)
"newproject:dev": "turbo run dev --filter=newproject...",
// Individual app commands
"dev:newproject:web": "pnpm --filter @newproject/web dev",
"dev:newproject:mobile": "pnpm --filter @newproject/mobile dev",
"dev:newproject:backend": "pnpm --filter @newproject/backend dev",
"dev:newproject:landing": "pnpm --filter @newproject/landing dev",
"dev:newproject:app": "turbo run dev --filter=@newproject/web --filter=@newproject/backend",
// Full dev with auto database setup
"dev:newproject:full": "./scripts/setup-databases.sh newproject && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:newproject:backend\" \"pnpm dev:newproject:web\"",
// Database shortcuts
"newproject:db:push": "pnpm --filter @newproject/backend db:push",
"newproject:db:studio": "pnpm --filter @newproject/backend db:studio",
// Setup shortcut
"setup:db:newproject": "./scripts/setup-databases.sh newproject"
}
}
```
### Step 4: Create Project CLAUDE.md
Create `apps/newproject/CLAUDE.md` with:
- Project overview
- Structure diagram
- Available commands
- API endpoints (if backend)
- Environment variables
- Tech stack details
See existing projects like `apps/chat/CLAUDE.md` for reference.
### Step 5: Test the Setup
```bash
# Create database and push schema
pnpm setup:db:newproject
# Start with full dev command
pnpm dev:newproject:full
```
## Checklist for New Projects
- [ ] Create project structure under `apps/newproject/`
- [ ] Add `pnpm-workspace.yaml` in project root
- [ ] Add database to `docker/init-db/01-create-databases.sql`
- [ ] Add service to `scripts/setup-databases.sh`
- [ ] Add DATABASE_URL to `.env.development`
- [ ] Update `scripts/generate-env.mjs` for env generation
- [ ] Add scripts to root `package.json`
- [ ] Create `CLAUDE.md` with project documentation
- [ ] Test with `pnpm dev:newproject:full`

View file

@ -17,6 +17,9 @@
"format": "prettier --config .prettierrc.json --write \"**/*.{ts,tsx,js,jsx,json,md,svelte,astro}\"",
"format:check": "prettier --config .prettierrc.json --check \"**/*.{ts,tsx,js,jsx,json,md,svelte,astro}\"",
"setup:env": "node scripts/generate-env.mjs",
"setup:db": "./scripts/setup-databases.sh",
"setup:db:chat": "./scripts/setup-databases.sh chat",
"setup:db:auth": "./scripts/setup-databases.sh auth",
"build:packages": "pnpm --filter '@manacore/*' build",
"postinstall": "node scripts/generate-env.mjs || true && pnpm run build:packages || true",
"manacore:dev": "turbo run dev --filter=manacore...",
@ -39,25 +42,28 @@
"dev:picture:mobile": "pnpm --filter @picture/mobile dev",
"dev:picture:backend": "pnpm --filter @picture/backend dev",
"dev:picture:app": "turbo run dev --filter=@picture/web --filter=@picture/backend",
"dev:picture:full": "./scripts/setup-databases.sh picture && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:picture:backend\" \"pnpm dev:picture:web\"",
"dev:chat:mobile": "pnpm --filter @chat/mobile dev",
"dev:chat:web": "pnpm --filter @chat/web dev",
"dev:chat:landing": "pnpm --filter @chat/landing dev",
"dev:chat:backend": "pnpm --filter @chat/backend start:dev",
"dev:chat:app": "turbo run dev --filter=@chat/web --filter=@chat/backend",
"dev:auth": "pnpm --filter mana-core-auth start:dev",
"dev:chat:full": "concurrently \"pnpm dev:auth\" \"pnpm dev:chat:backend\"",
"dev:chat:full": "./scripts/setup-databases.sh chat && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:chat:backend\" \"pnpm dev:chat:web\"",
"zitare:dev": "turbo run dev --filter=zitare...",
"dev:zitare:mobile": "pnpm --filter @zitare/mobile dev",
"dev:zitare:web": "pnpm --filter @zitare/web dev",
"dev:zitare:landing": "pnpm --filter @zitare/landing dev",
"dev:zitare:backend": "pnpm --filter @zitare/backend dev",
"dev:zitare:app": "turbo run dev --filter=@zitare/web --filter=@zitare/backend",
"dev:zitare:full": "./scripts/setup-databases.sh zitare && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:zitare:backend\" \"pnpm dev:zitare:web\"",
"contacts:dev": "turbo run dev --filter=contacts...",
"dev:contacts:mobile": "pnpm --filter @contacts/mobile dev",
"dev:contacts:web": "pnpm --filter @contacts/web dev",
"dev:contacts:landing": "pnpm --filter @contacts/landing dev",
"dev:contacts:backend": "pnpm --filter @contacts/backend dev",
"dev:contacts:app": "turbo run dev --filter=@contacts/web --filter=@contacts/backend",
"dev:contacts:full": "./scripts/setup-databases.sh contacts && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:contacts:backend\" \"pnpm dev:contacts:web\"",
"contacts:db:push": "pnpm --filter @contacts/backend db:push",
"contacts:db:studio": "pnpm --filter @contacts/backend db:studio",
"contacts:db:seed": "pnpm --filter @contacts/backend db:seed",
@ -67,6 +73,7 @@
"dev:calendar:landing": "pnpm --filter @calendar/landing dev",
"dev:calendar:backend": "pnpm --filter @calendar/backend dev",
"dev:calendar:app": "turbo run dev --filter=@calendar/web --filter=@calendar/backend",
"dev:calendar:full": "./scripts/setup-databases.sh calendar && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:calendar:backend\" \"pnpm dev:calendar:web\"",
"calendar:db:push": "pnpm --filter @calendar/backend db:push",
"calendar:db:studio": "pnpm --filter @calendar/backend db:studio",
"calendar:db:seed": "pnpm --filter @calendar/backend db:seed",
@ -75,6 +82,7 @@
"dev:clock:landing": "pnpm --filter @clock/landing dev",
"dev:clock:backend": "pnpm --filter @clock/backend dev",
"dev:clock:app": "turbo run dev --filter=@clock/web --filter=@clock/backend",
"dev:clock:full": "./scripts/setup-databases.sh clock && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:clock:backend\" \"pnpm dev:clock:web\"",
"clock:db:push": "pnpm --filter @clock/backend db:push",
"clock:db:studio": "pnpm --filter @clock/backend db:studio",
"clock:db:seed": "pnpm --filter @clock/backend db:seed",
@ -92,6 +100,7 @@
"dev:todo:landing": "pnpm --filter @todo/landing dev",
"dev:todo:backend": "pnpm --filter @todo/backend dev",
"dev:todo:app": "turbo run dev --filter=@todo/web --filter=@todo/backend",
"dev:todo:full": "./scripts/setup-databases.sh todo && ./scripts/setup-databases.sh auth && concurrently -n auth,backend,web -c blue,green,cyan \"pnpm dev:auth\" \"pnpm dev:todo:backend\" \"pnpm dev:todo:web\"",
"todo:db:push": "pnpm --filter @todo/backend db:push",
"todo:db:studio": "pnpm --filter @todo/backend db:studio",
"todo:db:seed": "pnpm --filter @todo/backend db:seed",

169
scripts/setup-databases.sh Executable file
View file

@ -0,0 +1,169 @@
#!/bin/bash
# Setup script for creating databases and pushing schemas
# Usage: ./scripts/setup-databases.sh [service]
# Examples:
# ./scripts/setup-databases.sh # Setup all
# ./scripts/setup-databases.sh chat # Setup only chat
# ./scripts/setup-databases.sh auth # Setup only auth
set -e
# Database connection details (from .env.development)
DB_HOST="${DB_HOST:-localhost}"
DB_PORT="${DB_PORT:-5432}"
DB_USER="${POSTGRES_USER:-manacore}"
DB_PASSWORD="${POSTGRES_PASSWORD:-devpassword}"
# Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
echo -e "${GREEN}🗄️ Database Setup Script${NC}"
echo "======================================"
# Function to create database if it doesn't exist
create_db_if_not_exists() {
local db_name=$1
echo -e "${YELLOW}Checking database: ${db_name}${NC}"
if PGPASSWORD=$DB_PASSWORD psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d postgres -tc \
"SELECT 1 FROM pg_database WHERE datname = '$db_name'" | grep -q 1; then
echo -e " ${GREEN}✓ Exists${NC}"
else
echo -e " Creating database ${db_name}..."
PGPASSWORD=$DB_PASSWORD psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d postgres -c "CREATE DATABASE $db_name;" > /dev/null
PGPASSWORD=$DB_PASSWORD psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d postgres -c "GRANT ALL PRIVILEGES ON DATABASE $db_name TO $DB_USER;" > /dev/null
echo -e " ${GREEN}✓ Created${NC}"
fi
}
# Function to push schema for a service
push_schema() {
local filter=$1
local name=$2
echo -e "${YELLOW}Pushing schema for ${name}...${NC}"
# Use --force to auto-approve in development (skips interactive prompts)
if pnpm --filter "$filter" db:push --force 2>/dev/null; then
echo -e " ${GREEN}✓ Schema pushed${NC}"
else
echo -e " ${RED}✗ Failed (may not have db:push script)${NC}"
fi
}
# All databases that should exist
ALL_DATABASES=(
"manacore"
"chat"
"zitare"
"contacts"
"calendar"
"clock"
"todo"
"manadeck"
"storage"
"mail"
"moodlit"
"finance"
"inventory"
"techbase"
"voxel_lava"
"figgos"
)
# Check if specific service requested
SERVICE_FILTER=${1:-""}
setup_service() {
local service=$1
case $service in
auth|mana-core-auth)
create_db_if_not_exists "manacore"
push_schema "mana-core-auth" "mana-core-auth"
;;
chat)
create_db_if_not_exists "chat"
push_schema "@chat/backend" "chat"
;;
zitare)
create_db_if_not_exists "zitare"
push_schema "@zitare/backend" "zitare"
;;
contacts)
create_db_if_not_exists "contacts"
push_schema "@contacts/backend" "contacts"
;;
calendar)
create_db_if_not_exists "calendar"
push_schema "@calendar/backend" "calendar"
;;
clock)
create_db_if_not_exists "clock"
push_schema "@clock/backend" "clock"
;;
todo)
create_db_if_not_exists "todo"
push_schema "@todo/backend" "todo"
;;
manadeck)
create_db_if_not_exists "manadeck"
push_schema "@manadeck/backend" "manadeck"
;;
mail)
create_db_if_not_exists "mail"
push_schema "@mail/backend" "mail"
;;
moodlit)
create_db_if_not_exists "moodlit"
push_schema "@moodlit/backend" "moodlit"
;;
picture)
create_db_if_not_exists "picture"
push_schema "@picture/backend" "picture"
;;
finance)
create_db_if_not_exists "finance"
push_schema "@finance/backend" "finance"
;;
voxel-lava)
create_db_if_not_exists "voxel_lava"
push_schema "@voxel-lava/backend" "voxel-lava"
;;
figgos)
create_db_if_not_exists "figgos"
push_schema "@figgos/backend" "figgos"
;;
*)
echo -e "${RED}Unknown service: $service${NC}"
echo "Available services: auth, chat, zitare, contacts, calendar, clock, todo, manadeck, mail, moodlit, finance, voxel-lava, figgos"
exit 1
;;
esac
}
if [ -n "$SERVICE_FILTER" ]; then
echo -e "Setting up for service: ${SERVICE_FILTER}"
setup_service "$SERVICE_FILTER"
echo -e "\n${GREEN}✓ Setup complete!${NC}"
exit 0
fi
# Setup all databases
echo -e "\n${GREEN}Step 1: Creating databases${NC}"
echo "--------------------------------------"
for db in "${ALL_DATABASES[@]}"; do
create_db_if_not_exists "$db"
done
echo -e "\n${GREEN}Step 2: Pushing schemas${NC}"
echo "--------------------------------------"
# Push schemas for all known services
for service in auth chat zitare contacts calendar clock todo manadeck picture mail moodlit finance voxel-lava figgos; do
setup_service "$service" 2>/dev/null || true
done
echo -e "\n${GREEN}✓ Database setup complete!${NC}"