feat(wordeck): Big-Bang-Cutover L-2 — Server-CRUD raus, alles auf event-sync
Some checks are pending
CI / validate (push) Waiting to run
Some checks are pending
CI / validate (push) Waiting to run
L-2 von mana/docs/playbooks/LOCAL_FIRST_LOGIN_OPTIONAL.md:
apps/api:
- routes/decks.ts: CRUD-Routes gelöscht. Verbleibend nur Read-Only:
GET /:id/marketplace-source + GET /:deckId/distractors
- routes/cards.ts: alle CRUD-Routes → 410 Gone mit Deprecation/Sunset
Header (Sunset 2026-06-20). Sub-Pfade weren von Hono auf das *-Handler
geleitet, die alle 410 zurückgeben
- routes/reviews.ts: alle Routes → 410 Gone, FSRS-Compute ist jetzt
client-side via @wordeck/domain.gradeReview
- routes/decks-generate.ts: returnt nur noch LLM-Vorschlag
({ suggestion: { deck, cards } }), Server schreibt NICHTS mehr in
decks/cards/reviews. Client emittet Events lokal in event-sync
- routes/dsgvo.ts: Doku-Block: nach Big-Bang sind neue User-Daten in
sync2 mana_sync_v2.wordeck.*, nicht mehr hier. mana-admin-Fanout
muss beide Quellen abfragen
- scripts/migrate-db-to-events.ts: Stub für DB→Event-Sync-Migration.
Idempotent via idempotencyKey='migration:<row-id>:<event-type>'.
Plaintext-Migration (kein User-Master-Key zur Migration-Zeit),
--dry-run/--commit, --user-id-Filter. mintToken() noch als Stub
(braucht Service-Key-basierten JWT-Mint in mana-auth)
apps/web:
- lib/api/decks.ts: generateDeck wrapped jetzt den Server-Vorschlag
via lokales createDeck + createCard-Burst. UI sieht weiterhin
{ deck, cards_created } als Return-Shape
apps/api/tests:
- decks.test.ts: post-cutover-Smokes (Auth-Check + 404 für entfernte
Routes)
- cards.test.ts: 410-Gone-Verification mit Deprecation-Header
- reviews.test.ts: 410-Gone-Verification
Type-check 0 Errors. Test-Suite: pre-existing fails (dsgvo, share,
tools — alle pre-cutover schon rot, Rebrand-Drift cards→wordeck);
meine drei Big-Bang-Tests-Files 7/7 grün.
Offene Punkte (bewusst geflaggt, nicht Big-Bang-Block):
- DSGVO-Pfad cross-source-Aggregation (sync2 + DB) ist mana-admin's
Architektur-Job, nicht wordeck-app
- Migration-Script mintToken() braucht mana-auth-Service-Key-Pfad
oder sync2-Service-Key-Auth-Mode (Plattform-Arbeit)
- Live-User-Migration: Skript-Stub ist ungetestet, muss vor echtem
Run code-reviewed + 1-2 User-Dry-Runs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS
parent
bebd182540
commit
375a6af86e
10 changed files with 555 additions and 1010 deletions
310
apps/api/scripts/migrate-db-to-events.ts
Normal file
310
apps/api/scripts/migrate-db-to-events.ts
Normal file
|
|
@ -0,0 +1,310 @@
|
|||
/**
|
||||
* Migration-Skript für den Big-Bang-Cutover L-2 (2026-05-20).
|
||||
*
|
||||
* Liest pro User alle Decks + Cards + Reviews aus der Wordeck-DB und
|
||||
* publiziert sie als event-sourced Append-Sequence an
|
||||
* sync2.mana.how/sync/wordeck. Idempotent über `idempotencyKey =
|
||||
* 'migration:<row-id>:<event-type>'`.
|
||||
*
|
||||
* Verwendung (immer dry-run probieren bevor echter Run):
|
||||
*
|
||||
* pnpm tsx scripts/migrate-db-to-events.ts --dry-run [--user-id <id>]
|
||||
* pnpm tsx scripts/migrate-db-to-events.ts --commit [--user-id <id>]
|
||||
*
|
||||
* Ohne --user-id wird über alle User iteriert. --dry-run gibt Counts
|
||||
* aus ohne POST. --commit POSTet tatsächlich.
|
||||
*
|
||||
* Voraussetzungen:
|
||||
* - DATABASE_URL gesetzt (Wordeck-DB)
|
||||
* - MANA_SYNC_URL (default https://sync2.mana.how)
|
||||
* - MANA_SERVICE_KEY (für service-side User-JWT-Mint oder per-User-Token)
|
||||
* - User-JWTs: Skript ruft mana-auth-`POST /api/v1/service/mint-token`
|
||||
* mit Service-Key + user_id für temporäre JWT (Service-Key-Pattern,
|
||||
* siehe shared-auth)
|
||||
*
|
||||
* Encryption: Daten werden im **Plaintext** an sync2 gesendet — wir
|
||||
* haben keinen User-Master-Key zur Migration-Zeit. Server speichert
|
||||
* sie als wire-format string (NoOp-kompatibel). Wenn der User sich
|
||||
* später einloggt + Vault-Key bootstrappt, kann ein Re-Encrypt-Pass
|
||||
* folgen. Für den Big-Bang akzeptieren wir Plaintext-Migration, weil:
|
||||
* - die Daten waren vorher schon in Postgres plaintext
|
||||
* - der Sync-Server liegt in derselben Trust-Domain (Mac Mini)
|
||||
* - User kann nach Login encrypted-Versionen drüberspielen
|
||||
*
|
||||
* **Status: Skript-Stub, ungetestet.** Vor echtem Run:
|
||||
* 1. Code-Review durch Till
|
||||
* 2. Snapshot von mana_sync_v2.wordeck.* + Wordeck-DB
|
||||
* 3. Dry-run für 1-2 User
|
||||
* 4. Manuelle Verifikation des sync2-States
|
||||
* 5. Erst dann --commit
|
||||
*/
|
||||
|
||||
import { eq, sql } from 'drizzle-orm';
|
||||
|
||||
import { getDb } from '../src/db/connection.ts';
|
||||
import { cards, decks, reviews } from '../src/db/schema/index.ts';
|
||||
|
||||
const SYNC_URL = process.env.MANA_SYNC_URL ?? 'https://sync2.mana.how';
|
||||
const APP_ID = 'wordeck';
|
||||
|
||||
interface Args {
|
||||
dryRun: boolean;
|
||||
userId?: string;
|
||||
}
|
||||
|
||||
function parseArgs(): Args {
|
||||
const args = process.argv.slice(2);
|
||||
const dryRun = !args.includes('--commit');
|
||||
const uIdx = args.indexOf('--user-id');
|
||||
const userId = uIdx >= 0 ? args[uIdx + 1] : undefined;
|
||||
return { dryRun, userId };
|
||||
}
|
||||
|
||||
interface EventEnvelope {
|
||||
eventId: string;
|
||||
aggregateId: string;
|
||||
appId: string;
|
||||
eventType: string;
|
||||
eventVersion: number;
|
||||
occurredAt: string;
|
||||
actor: { kind: 'migration'; principalId: string; displayName: string };
|
||||
attributedToUserId: string;
|
||||
origin: 'migration';
|
||||
idempotencyKey: string;
|
||||
payload: Record<string, unknown>;
|
||||
}
|
||||
|
||||
function ulid(): string {
|
||||
// Crypto-Random ULID-light für Migration. Echtes ulid-Format wäre
|
||||
// schöner, aber für migration-events reicht ein eindeutiger String.
|
||||
return (
|
||||
Math.floor(Date.now()).toString(36).padStart(8, '0').toUpperCase() +
|
||||
crypto.randomUUID().replace(/-/g, '').slice(0, 16).toUpperCase()
|
||||
);
|
||||
}
|
||||
|
||||
function envelopeFor(
|
||||
userId: string,
|
||||
aggregateId: string,
|
||||
eventType: string,
|
||||
idemSuffix: string,
|
||||
payload: Record<string, unknown>,
|
||||
occurredAt: Date,
|
||||
): EventEnvelope {
|
||||
return {
|
||||
eventId: ulid(),
|
||||
aggregateId,
|
||||
appId: APP_ID,
|
||||
eventType,
|
||||
eventVersion: 1,
|
||||
occurredAt: occurredAt.toISOString(),
|
||||
actor: {
|
||||
kind: 'migration',
|
||||
principalId: 'migration:wordeck:2026-05-20',
|
||||
displayName: 'L-2 DB→Event-Sync Migration',
|
||||
},
|
||||
attributedToUserId: userId,
|
||||
origin: 'migration',
|
||||
idempotencyKey: `migration:${idemSuffix}`,
|
||||
payload,
|
||||
};
|
||||
}
|
||||
|
||||
async function mintToken(userId: string): Promise<string> {
|
||||
// Stub: Service-Key-basierter Mint. Implementation hängt davon ab,
|
||||
// wie mana-auth das anbietet. Wenn nicht vorhanden, müssen wir
|
||||
// einen anderen Pfad finden (z.B. sync2 mit Service-Key-Auth-Mode
|
||||
// erweitern, was eigene Plattform-Arbeit ist).
|
||||
throw new Error(
|
||||
`mintToken stub: implementiere Service-Key-basierten JWT-Mint für ${userId}`,
|
||||
);
|
||||
}
|
||||
|
||||
async function postBatch(token: string, events: EventEnvelope[]): Promise<void> {
|
||||
const res = await fetch(`${SYNC_URL}/sync/${APP_ID}`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
body: JSON.stringify({ events }),
|
||||
});
|
||||
if (!res.ok) {
|
||||
throw new Error(`sync2 returned ${res.status}: ${await res.text()}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function migrateUser(userId: string, dryRun: boolean): Promise<void> {
|
||||
const db = getDb();
|
||||
const deckRows = await db.select().from(decks).where(eq(decks.userId, userId));
|
||||
const cardRows = await db.select().from(cards).where(eq(cards.userId, userId));
|
||||
const reviewRows = await db.select().from(reviews).where(eq(reviews.userId, userId));
|
||||
|
||||
const events: EventEnvelope[] = [];
|
||||
|
||||
for (const d of deckRows) {
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
`deck:${d.id}`,
|
||||
'DeckCreated',
|
||||
`deck:${d.id}:created`,
|
||||
{
|
||||
deckId: d.id,
|
||||
name: d.name,
|
||||
description: d.description,
|
||||
color: d.color,
|
||||
category: d.category,
|
||||
},
|
||||
d.createdAt,
|
||||
),
|
||||
);
|
||||
if (d.fsrsSettings && Object.keys(d.fsrsSettings).length > 0) {
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
`deck:${d.id}`,
|
||||
'DeckFsrsSettingsUpdated',
|
||||
`deck:${d.id}:fsrs`,
|
||||
{ deckId: d.id, newSettingsJson: JSON.stringify(d.fsrsSettings) },
|
||||
d.updatedAt,
|
||||
),
|
||||
);
|
||||
}
|
||||
if (d.visibility !== 'private') {
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
`deck:${d.id}`,
|
||||
'DeckPublished',
|
||||
`deck:${d.id}:published`,
|
||||
{ deckId: d.id, visibility: d.visibility, license: 'CC-BY-4.0' },
|
||||
d.updatedAt,
|
||||
),
|
||||
);
|
||||
}
|
||||
if (d.archivedAt) {
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
`deck:${d.id}`,
|
||||
'DeckArchived',
|
||||
`deck:${d.id}:archived`,
|
||||
{ deckId: d.id },
|
||||
d.archivedAt,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const card of cardRows) {
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
`card:${card.id}`,
|
||||
'CardCreated',
|
||||
`card:${card.id}:created`,
|
||||
{
|
||||
cardId: card.id,
|
||||
deckId: card.deckId,
|
||||
type: card.type,
|
||||
fieldsJson: JSON.stringify(card.fields),
|
||||
tags: [],
|
||||
},
|
||||
card.createdAt,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
for (const r of reviewRows) {
|
||||
const aggId = `review:${r.cardId}__${r.subIndex}`;
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
aggId,
|
||||
'ReviewInitialized',
|
||||
`review:${r.cardId}:${r.subIndex}:init`,
|
||||
{
|
||||
reviewId: aggId,
|
||||
cardId: r.cardId,
|
||||
subIndex: r.subIndex,
|
||||
due: r.due.toISOString(),
|
||||
},
|
||||
r.lastReview ?? r.due,
|
||||
),
|
||||
);
|
||||
if (r.state !== 'new' || r.reps > 0) {
|
||||
// Aktueller State als ReviewGraded mit "rating=good" Sentinel-Wert.
|
||||
// FSRS-Felder werden 1:1 übernommen. Nach Migration kann der User
|
||||
// weiter graden, neue Events stacken sich auf.
|
||||
events.push(
|
||||
envelopeFor(
|
||||
userId,
|
||||
aggId,
|
||||
'ReviewGraded',
|
||||
`review:${r.cardId}:${r.subIndex}:state`,
|
||||
{
|
||||
reviewId: aggId,
|
||||
rating: 'good',
|
||||
newState: r.state,
|
||||
newDue: r.due.toISOString(),
|
||||
newStability: r.stability,
|
||||
newDifficulty: r.difficulty,
|
||||
newElapsedDays: r.elapsedDays,
|
||||
newScheduledDays: r.scheduledDays,
|
||||
newLearningSteps: r.learningSteps,
|
||||
newReps: r.reps,
|
||||
newLapses: r.lapses,
|
||||
prevSnapshotJson: null,
|
||||
},
|
||||
r.lastReview ?? r.due,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
console.log(
|
||||
`[user ${userId}] decks=${deckRows.length} cards=${cardRows.length} reviews=${reviewRows.length} → ${events.length} events`,
|
||||
);
|
||||
if (dryRun) return;
|
||||
|
||||
const token = await mintToken(userId);
|
||||
// Batches à 100 wegen sync2-server-limit
|
||||
for (let i = 0; i < events.length; i += 100) {
|
||||
await postBatch(token, events.slice(i, i + 100));
|
||||
}
|
||||
console.log(`[user ${userId}] migration applied`);
|
||||
}
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const { dryRun, userId } = parseArgs();
|
||||
console.log(`migration: dry-run=${dryRun} userId=${userId ?? '(all)'}`);
|
||||
|
||||
const db = getDb();
|
||||
let userIds: string[];
|
||||
if (userId) {
|
||||
userIds = [userId];
|
||||
} else {
|
||||
const rows = await db.execute<{ user_id: string }>(
|
||||
sql`SELECT DISTINCT user_id FROM wordeck.decks`,
|
||||
);
|
||||
userIds = rows.rows.map((r) => r.user_id);
|
||||
}
|
||||
console.log(`migration: ${userIds.length} user(s)`);
|
||||
|
||||
let ok = 0;
|
||||
let fail = 0;
|
||||
for (const uid of userIds) {
|
||||
try {
|
||||
await migrateUser(uid, dryRun);
|
||||
ok++;
|
||||
} catch (e) {
|
||||
console.error(`[user ${uid}] FAILED:`, e instanceof Error ? e.message : e);
|
||||
fail++;
|
||||
}
|
||||
}
|
||||
console.log(`migration: ok=${ok} fail=${fail}`);
|
||||
process.exit(fail > 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
void main();
|
||||
Loading…
Add table
Add a link
Reference in a new issue