mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-14 21:41:09 +02:00
- Create mana-media service for centralized media handling - Add upload, processing, and delivery modules - Configure BullMQ for async transcoding jobs - Add S3-compatible storage integration - Create TypeScript client package Features: - Multi-format image/video upload - Async transcoding via ffmpeg - Adaptive streaming (HLS) support - Signed URL delivery Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
6.5 KiB
6.5 KiB
External 4TB SSD - Opportunities & Usage Guide
This document outlines the opportunities enabled by the 4TB external SSD connected to the Mac Mini production server.
Current Setup
| Component | Details |
|---|---|
| Device | Mac Mini M4 (16GB RAM) |
| Internal SSD | 228 GB (30 GB free) |
| External SSD | 4 TB (3.6 TB free) |
| Mount Point | /Volumes/TillJakob-S04 |
| Data Directory | /Volumes/TillJakob-S04/ManaData/ |
Currently Migrated to SSD
| Item | Size | Path |
|---|---|---|
| Ollama Models | ~26 GB | ManaData/ollama/ (symlink: ~/.ollama) |
| STT Models | ~19 GB | ManaData/stt-models/ (symlink: ~/STT-models) |
| FLUX.2 | ~15 GB | ManaData/flux2/ (symlink: ~/FLUX.2) |
| Matrix Media | Variable | ManaData/matrix-media/ |
| PostgreSQL Backups | Variable | ManaData/backups/postgres/ |
Opportunities
1. Larger LLM Models
With 4TB available, we can host significantly larger and more capable models:
| Model | Size | Use Case |
|---|---|---|
llama3:70b-q4 |
~40 GB | Highest quality general-purpose |
mixtral:8x7b |
~26 GB | Fast Mixture of Experts |
codestral:22b |
~13 GB | Best code assistant |
qwen2.5:32b |
~20 GB | Excellent multilingual (German) |
deepseek-coder-v2:16b |
~10 GB | Top coding performance |
llava:34b |
~20 GB | Best vision model |
Potential: Host 10-20 specialized models for different tasks, switch dynamically based on use case.
2. RAG / Knowledge Databases
Enable semantic search and retrieval-augmented generation:
- Vector Database (Qdrant, ChromaDB, Milvus)
- Index documents, PDFs, codebases
- Build knowledge bases for Chat app
- Make company documentation searchable
- Estimated storage: 50-200 GB depending on corpus size
3. Local AI Services Expansion
| Service | Storage | Benefit |
|---|---|---|
| Whisper Large-v3 | ~3 GB | Best-in-class speech recognition |
| ComfyUI + Models | 50-100 GB | Local image generation (SDXL, Flux) |
| MusicGen | ~10 GB | AI music generation |
| Video Models | 20-50 GB | Local video AI |
| TTS Models | ~5 GB | High-quality text-to-speech |
4. Extended Backups & Disaster Recovery
Comprehensive backup strategy enabled by large storage:
Database Backups
- Daily snapshots with 90+ day retention
- Point-in-time recovery capability
- Cross-database consistency backups
- Estimated: 50-100 GB for full history
Docker Infrastructure
- Local image registry for faster deployments
- Build cache persistence across restarts
- Container snapshots before major updates
- Estimated: 100-200 GB
Code & Configuration
- Git repository mirrors (full clone backups)
- Configuration backups (Docker, Nginx, etc.)
- Secrets backup (encrypted)
- Estimated: 10-50 GB
System Recovery
- Full system snapshots via Time Machine or rsync
- Bootable recovery partition
- Quick restore capability
- Estimated: 250-500 GB
5. Media & Content Storage
Centralized media handling for all applications:
MinIO Expansion
| Bucket | Purpose | Est. Size |
|---|---|---|
picture-storage |
AI-generated images | 100+ GB |
storage-storage |
User cloud storage | 500+ GB |
nutriphi-storage |
Meal photos | 50+ GB |
chat-attachments |
Chat file uploads | 100+ GB |
presi-assets |
Presentation media | 50+ GB |
Media Processing
- Video transcoding pipeline with temp storage
- Image optimization cache
- Thumbnail generation storage
- Audio processing workspace
Content Delivery
- Static asset hosting for all apps
- Game assets for games/ projects
- Landing page media (images, videos)
- Documentation assets
6. New Application Possibilities
The storage enables entirely new application categories:
Media Applications
| App Idea | Storage Need | Description |
|---|---|---|
| Video Library | 500+ GB | Local video storage with transcripts, searchable via AI |
| Music Streaming | 200+ GB | Personal music collection, Spotify-like interface |
| Photo Library | 500+ GB | iCloud/Google Photos alternative with AI tagging |
| Podcast Archive | 100+ GB | Download, transcribe, search podcasts |
Document & Knowledge
| App Idea | Storage Need | Description |
|---|---|---|
| Document Vault | 100+ GB | Encrypted document storage with OCR |
| Research Archive | 200+ GB | Papers, articles, bookmarks with AI summaries |
| Code Archive | 50+ GB | Searchable repository of code snippets |
| Learning Library | 100+ GB | Courses, tutorials, educational content |
AI-Powered Services
| App Idea | Storage Need | Description |
|---|---|---|
| Local AI Studio | 200+ GB | ComfyUI, training data, generated outputs |
| Voice Clone Lab | 20+ GB | Custom TTS voices |
| Dataset Hub | 100+ GB | ML training datasets |
7. Development & Testing
Enhanced development workflow:
- Large test datasets for ML experiments
- Build cache for faster CI/CD
- Staging databases with production-like data
- Log aggregation (Loki/ELK) with extended retention
- Performance profiling data storage
Implementation Priority
Phase 1 (Immediate)
- Migrate Ollama models
- Migrate STT/FLUX models
- Setup PostgreSQL backups
- Download additional LLM models
Phase 2 (Short-term)
- Setup local Docker registry
- Expand MinIO to SSD
- Implement extended backup retention
- Add vector database for RAG
Phase 3 (Medium-term)
- Setup ComfyUI for local image generation
- Implement media processing pipeline
- Add video/audio transcription service
Phase 4 (Long-term)
- Build new media-focused applications
- Implement full disaster recovery
- Create AI training infrastructure
Technical Notes
Adding Docker File Sharing for SSD
To enable Docker containers to use SSD storage directly:
- Open Docker Desktop → Settings → Resources → File Sharing
- Add
/Volumes/TillJakob-S04/ManaData/ - Restart Docker Desktop
Symlink Pattern
For applications that don't support custom paths:
# Move data to SSD
mv ~/original-path /Volumes/TillJakob-S04/ManaData/new-location
# Create symlink
ln -s /Volumes/TillJakob-S04/ManaData/new-location ~/original-path
Monitoring SSD Usage
# Check SSD usage
df -h /Volumes/TillJakob-S04
# Check ManaData breakdown
du -sh /Volumes/TillJakob-S04/ManaData/*
Last updated: 2026-02-01