mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-15 00:01:10 +02:00
feat(profile,api): meImages foundation for AI reference generation (M1)
M1 of docs/plans/me-images-and-reference-generation.md — a user-owned pool of reference images (face, fullbody, hands, …) that will back image generation where the user appears as themselves (outfit try-on, glasses, portraits) via OpenAI /v1/images/edits. Data layer only in this commit; UI lands in M2, the edits endpoint in M3. - Dexie v38: meImages table with id/kind/primaryFor/createdAt indices. Added to USER_LEVEL_TABLES so the hook stamps userId and skips the spaceId/authorId/visibility trio (one human = one face across every Space, not per-Space). - Encryption registry: label + tags encrypted; kind/primaryFor/usage stay plaintext because they drive the indexed queries and the Reference picker's filtering. mediaId/URLs/dimensions are structural. - Profile module store: createMeImage, updateMeImage, setAiReferenceEnabled (per-image KI opt-in — plan decision #5), setPrimary (transactional slot swap — only one row per primary slot), deleteMeImage. Emits MeImage* domain events. - Queries: useAllMeImages, useMeImagesByKind, useReferenceImages (only the rows the user opted in for KI), useImageByPrimary. - POST /api/v1/profile/me-images/upload: thin wrapper over mana-media with app='me' as the reference tag. No new MinIO bucket — plan decision #1 revised after verifying mana-media uses one bucket and only tags references by app. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS
parent
32c95a3780
commit
89258eb451
10 changed files with 790 additions and 4 deletions
|
|
@ -27,6 +27,7 @@ import { musicRoutes } from './modules/music/routes';
|
|||
import { chatRoutes } from './modules/chat/routes';
|
||||
import { contextRoutes } from './modules/context/routes';
|
||||
import { pictureRoutes } from './modules/picture/routes';
|
||||
import { profileRoutes } from './modules/profile/routes';
|
||||
import { storageRoutes } from './modules/storage/routes';
|
||||
import { todoRoutes } from './modules/todo/routes';
|
||||
import { plantsRoutes } from './modules/plants/routes';
|
||||
|
|
@ -40,6 +41,7 @@ import { tracesRoutes } from './modules/traces/routes';
|
|||
import { presiRoutes } from './modules/presi/routes';
|
||||
import { researchRoutes } from './modules/research/routes';
|
||||
import { whoRoutes } from './modules/who/routes';
|
||||
import { websiteRoutes } from './modules/website/routes';
|
||||
import { wetterRoutes } from './modules/wetter/routes';
|
||||
|
||||
const PORT = parseInt(process.env.PORT || '3060', 10);
|
||||
|
|
@ -97,6 +99,7 @@ app.route('/api/v1/music', musicRoutes);
|
|||
app.route('/api/v1/chat', chatRoutes);
|
||||
app.route('/api/v1/context', contextRoutes);
|
||||
app.route('/api/v1/picture', pictureRoutes);
|
||||
app.route('/api/v1/profile', profileRoutes);
|
||||
app.route('/api/v1/storage', storageRoutes);
|
||||
app.route('/api/v1/todo', todoRoutes);
|
||||
app.route('/api/v1/plants', plantsRoutes);
|
||||
|
|
@ -109,6 +112,7 @@ app.route('/api/v1/articles', articlesRoutes);
|
|||
app.route('/api/v1/traces', tracesRoutes);
|
||||
app.route('/api/v1/presi', presiRoutes);
|
||||
app.route('/api/v1/research', researchRoutes);
|
||||
app.route('/api/v1/website', websiteRoutes);
|
||||
app.route('/api/v1/who', whoRoutes);
|
||||
|
||||
// ─── Server Info ────────────────────────────────────────────
|
||||
|
|
|
|||
54
apps/api/src/modules/profile/routes.ts
Normal file
54
apps/api/src/modules/profile/routes.ts
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
/**
|
||||
* Profile module — server endpoints.
|
||||
*
|
||||
* Upload route for me-images (docs/plans/me-images-and-reference-generation.md M1).
|
||||
* Thin wrapper over mana-media — the stored row lands in Dexie on the
|
||||
* client after this returns. We keep server-side storage of the image
|
||||
* in mana-media (CAS + thumbnails) so the Picture generator can pull
|
||||
* the original bytes by `mediaId` for the eventual /v1/images/edits
|
||||
* call (M3) without the client needing to re-upload each time.
|
||||
*/
|
||||
|
||||
import { Hono } from 'hono';
|
||||
import type { AuthVariables } from '@mana/shared-hono';
|
||||
|
||||
const routes = new Hono<{ Variables: AuthVariables }>();
|
||||
|
||||
// Max upload size for me-images. 10MB matches /picture/upload — same
|
||||
// real-world phone-camera PNG range, same mana-media pipeline downstream.
|
||||
const MAX_UPLOAD_BYTES = 10 * 1024 * 1024;
|
||||
|
||||
routes.post('/me-images/upload', async (c) => {
|
||||
const userId = c.get('userId');
|
||||
const formData = await c.req.formData();
|
||||
const file = formData.get('file') as File | null;
|
||||
|
||||
if (!file) return c.json({ error: 'No file' }, 400);
|
||||
if (file.size > MAX_UPLOAD_BYTES) return c.json({ error: 'Max 10MB' }, 400);
|
||||
|
||||
try {
|
||||
const { uploadImageToMedia } = await import('../../lib/media');
|
||||
const buffer = await file.arrayBuffer();
|
||||
// `app='me'` tags the media_references row so a later
|
||||
// GET /api/v1/media?app=me&userId=X can list all me-images,
|
||||
// and the /v1/images/edits path can verify ownership in O(1).
|
||||
const result = await uploadImageToMedia(buffer, file.name, {
|
||||
app: 'me',
|
||||
userId,
|
||||
});
|
||||
|
||||
return c.json(
|
||||
{
|
||||
mediaId: result.id,
|
||||
storagePath: result.id,
|
||||
publicUrl: result.urls.original,
|
||||
thumbnailUrl: result.urls.thumbnail,
|
||||
},
|
||||
201
|
||||
);
|
||||
} catch (_err) {
|
||||
return c.json({ error: 'Upload failed' }, 500);
|
||||
}
|
||||
});
|
||||
|
||||
export { routes as profileRoutes };
|
||||
|
|
@ -89,6 +89,7 @@ import type {
|
|||
LocalBroadcastSettings,
|
||||
} from '../../modules/broadcast/types';
|
||||
import type { LocalArticle, LocalHighlight } from '../../modules/articles/types';
|
||||
import type { LocalMeImage } from '../../modules/profile/types';
|
||||
|
||||
export const ENCRYPTION_REGISTRY: Record<string, EncryptionConfig> = {
|
||||
// ─── Chat ────────────────────────────────────────────────
|
||||
|
|
@ -541,6 +542,16 @@ export const ENCRYPTION_REGISTRY: Record<string, EncryptionConfig> = {
|
|||
],
|
||||
},
|
||||
|
||||
// ─── Me-Images (AI reference pool) ───────────────────────
|
||||
// docs/plans/me-images-and-reference-generation.md M1.
|
||||
// Encrypted: `label` (user-typed — "Portrait Juni", "Outfit Studio")
|
||||
// and `tags` (string[] — free-form tags like "ohne-brille", "studio").
|
||||
// Plaintext (intentional): `kind`, `primaryFor`, `usage`, mediaId,
|
||||
// storagePath, publicUrl, thumbnailUrl, width, height — all indexed
|
||||
// or structural metadata the query layer needs. The image blob itself
|
||||
// lives in MinIO behind owner-RLS, not in Dexie.
|
||||
meImages: entry<LocalMeImage>(['label', 'tags']),
|
||||
|
||||
// Per-agent kontext documents — same schema as kontextDoc but keyed
|
||||
// per agent. Content is free-form markdown.
|
||||
agentKontextDocs: { enabled: true, fields: ['content'] },
|
||||
|
|
@ -748,6 +759,19 @@ export const ENCRYPTION_REGISTRY: Record<string, EncryptionConfig> = {
|
|||
'unsubscribeLandingCopy',
|
||||
]),
|
||||
|
||||
// ─── Website Builder ─────────────────────────────────────
|
||||
// docs/plans/website-builder.md §D4 — content is PUBLIC by design.
|
||||
// Site name, page titles, block props, theme config: the whole point
|
||||
// is that published sites are served to anonymous visitors over SSR.
|
||||
// Encrypting the draft would be security theater — the user publishes
|
||||
// the same content seconds later as plaintext into published_snapshots.
|
||||
// Form submissions (M4) land in target modules (contacts, todo, …)
|
||||
// which carry their own encryption; the submissions-audit row holds
|
||||
// the payload only briefly and gets scrubbed after delivery (M7).
|
||||
websites: { enabled: false, fields: [] },
|
||||
websitePages: { enabled: false, fields: [] },
|
||||
websiteBlocks: { enabled: false, fields: [] },
|
||||
|
||||
// Singleton sender profile. The user's legal address + IBAN live here
|
||||
// and are the most sensitive fields in the module (appear on every PDF
|
||||
// the user issues). logoMediaId / accentColor / number sequence state
|
||||
|
|
|
|||
|
|
@ -895,6 +895,43 @@ db.version(36).upgrade(async (tx) => {
|
|||
}
|
||||
});
|
||||
|
||||
// v37 — Website builder module (docs/plans/website-builder.md).
|
||||
// Three tables for the block-tree CMS. All space-scoped; all plaintext
|
||||
// (public content by design — see plan decision D4).
|
||||
// - websites: root per space. `slug` indexed for the eventual public
|
||||
// resolver + dedupe-within-space. `publishedVersion` indexed so the
|
||||
// editor can fast-filter unpublished drafts.
|
||||
// - websitePages: `[siteId+order]` for the ordered page list in the
|
||||
// editor. `[siteId+path]` for the public path resolver (page by URL).
|
||||
// - websiteBlocks: `[pageId+parentBlockId+order]` is the canonical tree
|
||||
// scan — ordered children of a parent within a page. `[pageId+order]`
|
||||
// is kept separately for the flat render path.
|
||||
db.version(37).stores({
|
||||
websites: 'id, slug, publishedVersion, updatedAt, deletedAt',
|
||||
websitePages: 'id, siteId, [siteId+order], [siteId+path], updatedAt, deletedAt',
|
||||
websiteBlocks:
|
||||
'id, pageId, parentBlockId, [pageId+order], [pageId+parentBlockId+order], type, updatedAt, deletedAt',
|
||||
});
|
||||
|
||||
// v38 — Me-Images: user-owned reference images for AI generation
|
||||
// (docs/plans/me-images-and-reference-generation.md M1).
|
||||
//
|
||||
// User-level table, not space-scoped — see USER_LEVEL_TABLES below.
|
||||
// The same human uses the same face/body across every Space, so the
|
||||
// images live once per user and are reused from every Space's Picture
|
||||
// generator.
|
||||
//
|
||||
// Indices:
|
||||
// - `kind` for the Settings UI's "all face images" / "all fullbody"
|
||||
// filter and for the query hook `useReferenceImages(kind)`.
|
||||
// - `primaryFor` for the hot lookup "give me the current avatar /
|
||||
// face-ref / body-ref" without a full scan. Null values are
|
||||
// dropped by Dexie so the index stays dense.
|
||||
// - `createdAt` for stable ordering (newest uploads first).
|
||||
db.version(38).stores({
|
||||
meImages: 'id, kind, primaryFor, createdAt',
|
||||
});
|
||||
|
||||
// ─── Sync Routing ──────────────────────────────────────────
|
||||
// SYNC_APP_MAP, TABLE_TO_SYNC_NAME, TABLE_TO_APP, SYNC_NAME_TO_TABLE,
|
||||
// toSyncName() and fromSyncName() are now derived from per-module
|
||||
|
|
@ -1077,6 +1114,7 @@ const USER_LEVEL_TABLES: ReadonlySet<string> = new Set([
|
|||
'broadcastSettings',
|
||||
'wetterSettings',
|
||||
'userTagPresets',
|
||||
'meImages',
|
||||
]);
|
||||
|
||||
for (const [appId, tables] of Object.entries(SYNC_APP_MAP)) {
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@
|
|||
*/
|
||||
|
||||
import { db } from '$lib/data/database';
|
||||
import type { LocalUserContext } from './types';
|
||||
import type { LocalUserContext, LocalMeImage } from './types';
|
||||
|
||||
export const userContextTable = db.table<LocalUserContext>('userContext');
|
||||
export const meImagesTable = db.table<LocalMeImage>('meImages');
|
||||
|
|
|
|||
|
|
@ -2,5 +2,5 @@ import type { ModuleConfig } from '$lib/data/module-registry';
|
|||
|
||||
export const profileModuleConfig: ModuleConfig = {
|
||||
appId: 'profile',
|
||||
tables: [{ name: 'userContext' }],
|
||||
tables: [{ name: 'userContext' }, { name: 'meImages' }],
|
||||
};
|
||||
|
|
|
|||
|
|
@ -4,8 +4,16 @@
|
|||
|
||||
import { useLiveQueryWithDefault } from '@mana/local-store/svelte';
|
||||
import { decryptRecords } from '$lib/data/crypto';
|
||||
import { userContextTable } from './collections';
|
||||
import { USER_CONTEXT_SINGLETON_ID, toUserContext, type UserContext } from './types';
|
||||
import { meImagesTable, userContextTable } from './collections';
|
||||
import {
|
||||
USER_CONTEXT_SINGLETON_ID,
|
||||
toUserContext,
|
||||
toMeImage,
|
||||
type UserContext,
|
||||
type MeImage,
|
||||
type MeImageKind,
|
||||
type MeImagePrimarySlot,
|
||||
} from './types';
|
||||
|
||||
/** Reactive live-query for the user context singleton. */
|
||||
export function useUserContext() {
|
||||
|
|
@ -16,3 +24,66 @@ export function useUserContext() {
|
|||
return toUserContext(decrypted);
|
||||
}, null);
|
||||
}
|
||||
|
||||
/**
|
||||
* All non-deleted me-images, newest first. Decrypted on the client —
|
||||
* filters and sorting happen before decrypt where possible (`kind`,
|
||||
* `primaryFor`, `createdAt` are plaintext indices).
|
||||
*/
|
||||
export function useAllMeImages() {
|
||||
return useLiveQueryWithDefault<MeImage[]>(async () => {
|
||||
const locals = await meImagesTable.orderBy('createdAt').reverse().toArray();
|
||||
const visible = locals.filter((row) => !row.deletedAt);
|
||||
const decrypted = await decryptRecords('meImages', visible);
|
||||
return decrypted.map(toMeImage);
|
||||
}, [] as MeImage[]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Me-images filtered by `kind`. Uses the `kind` Dexie index so large
|
||||
* pools still filter in one B-tree lookup.
|
||||
*/
|
||||
export function useMeImagesByKind(kind: MeImageKind) {
|
||||
return useLiveQueryWithDefault<MeImage[]>(async () => {
|
||||
const locals = await meImagesTable.where('kind').equals(kind).toArray();
|
||||
const visible = locals
|
||||
.filter((row) => !row.deletedAt)
|
||||
.sort((a, b) => (b.createdAt ?? '').localeCompare(a.createdAt ?? ''));
|
||||
const decrypted = await decryptRecords('meImages', visible);
|
||||
return decrypted.map(toMeImage);
|
||||
}, [] as MeImage[]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Only images the user explicitly opted in for AI reference use.
|
||||
* This is the authoritative list the Picture generator's Reference
|
||||
* picker reads from — if an image isn't here, it must not be sent
|
||||
* to OpenAI.
|
||||
*/
|
||||
export function useReferenceImages() {
|
||||
return useLiveQueryWithDefault<MeImage[]>(async () => {
|
||||
const locals = await meImagesTable.orderBy('createdAt').reverse().toArray();
|
||||
const visible = locals.filter((row) => !row.deletedAt && row.usage?.aiReference === true);
|
||||
const decrypted = await decryptRecords('meImages', visible);
|
||||
return decrypted.map(toMeImage);
|
||||
}, [] as MeImage[]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Current holder of a primary slot (avatar / face-ref / body-ref),
|
||||
* or null if nobody claimed it yet. Powers the avatar fallback and
|
||||
* the Reference picker's default selection.
|
||||
*/
|
||||
export function useImageByPrimary(slot: MeImagePrimarySlot) {
|
||||
return useLiveQueryWithDefault<MeImage | null>(async () => {
|
||||
const locals = await meImagesTable.where('primaryFor').equals(slot).toArray();
|
||||
const visible = locals.filter((row) => !row.deletedAt);
|
||||
if (visible.length === 0) return null;
|
||||
// The setPrimary store method keeps this to exactly one row. If
|
||||
// somehow more than one slipped through (manual DB edit, race on
|
||||
// a broken migration), prefer the most recent write.
|
||||
visible.sort((a, b) => (b.updatedAt ?? '').localeCompare(a.updatedAt ?? ''));
|
||||
const [decrypted] = await decryptRecords('meImages', [visible[0]]);
|
||||
return toMeImage(decrypted);
|
||||
}, null);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,152 @@
|
|||
/**
|
||||
* Me-Images store — mutation-only service.
|
||||
*
|
||||
* Reads happen via liveQuery helpers in queries.ts. Writes go through
|
||||
* this store so encryption (`label`, `tags`) and primary-slot swapping
|
||||
* stay in one place.
|
||||
*
|
||||
* Plan: docs/plans/me-images-and-reference-generation.md M1.
|
||||
*/
|
||||
|
||||
import { encryptRecord } from '$lib/data/crypto';
|
||||
import { emitDomainEvent } from '$lib/data/events';
|
||||
import { meImagesTable } from '../collections';
|
||||
import { toMeImage } from '../types';
|
||||
import type {
|
||||
LocalMeImage,
|
||||
MeImage,
|
||||
MeImageKind,
|
||||
MeImagePrimarySlot,
|
||||
MeImageUsage,
|
||||
} from '../types';
|
||||
|
||||
export interface CreateMeImageInput {
|
||||
kind: MeImageKind;
|
||||
mediaId: string;
|
||||
storagePath: string;
|
||||
publicUrl: string;
|
||||
thumbnailUrl?: string | null;
|
||||
width: number;
|
||||
height: number;
|
||||
label?: string;
|
||||
tags?: string[];
|
||||
usage?: Partial<MeImageUsage>;
|
||||
primaryFor?: MeImagePrimarySlot | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Usage default on upload: `aiReference=false` (Opt-in per image is
|
||||
* the eigentliche Zustimmungsebene — plan decision #5) and
|
||||
* `showInProfile=true` so the image can back the avatar fallback even
|
||||
* before the user explicitly picks a primary.
|
||||
*/
|
||||
function defaultUsage(override?: Partial<MeImageUsage>): MeImageUsage {
|
||||
return {
|
||||
aiReference: override?.aiReference ?? false,
|
||||
showInProfile: override?.showInProfile ?? true,
|
||||
};
|
||||
}
|
||||
|
||||
export const meImagesStore = {
|
||||
async createMeImage(input: CreateMeImageInput): Promise<MeImage> {
|
||||
const newLocal: LocalMeImage = {
|
||||
id: crypto.randomUUID(),
|
||||
kind: input.kind,
|
||||
label: input.label,
|
||||
mediaId: input.mediaId,
|
||||
storagePath: input.storagePath,
|
||||
publicUrl: input.publicUrl,
|
||||
thumbnailUrl: input.thumbnailUrl ?? null,
|
||||
width: input.width,
|
||||
height: input.height,
|
||||
tags: input.tags ?? [],
|
||||
usage: defaultUsage(input.usage),
|
||||
primaryFor: input.primaryFor ?? null,
|
||||
};
|
||||
const snapshot = toMeImage({ ...newLocal });
|
||||
await encryptRecord('meImages', newLocal);
|
||||
await meImagesTable.add(newLocal);
|
||||
emitDomainEvent('MeImageAdded', 'profile', 'meImages', newLocal.id, {
|
||||
meImageId: newLocal.id,
|
||||
kind: input.kind,
|
||||
primaryFor: newLocal.primaryFor,
|
||||
});
|
||||
return snapshot;
|
||||
},
|
||||
|
||||
async updateMeImage(
|
||||
id: string,
|
||||
patch: Partial<Pick<LocalMeImage, 'label' | 'tags' | 'kind' | 'usage'>>
|
||||
): Promise<void> {
|
||||
const wrapped = { ...patch } as Record<string, unknown>;
|
||||
await encryptRecord('meImages', wrapped);
|
||||
await meImagesTable.update(id, {
|
||||
...wrapped,
|
||||
updatedAt: new Date().toISOString(),
|
||||
});
|
||||
},
|
||||
|
||||
/**
|
||||
* Flip the per-image AI opt-in. Kept as its own method because
|
||||
* it's the hottest privacy-relevant toggle in the Settings UI and
|
||||
* warrants a dedicated event for audit.
|
||||
*/
|
||||
async setAiReferenceEnabled(id: string, enabled: boolean): Promise<void> {
|
||||
const existing = await meImagesTable.get(id);
|
||||
if (!existing) return;
|
||||
const nextUsage: MeImageUsage = {
|
||||
...defaultUsage(existing.usage),
|
||||
aiReference: enabled,
|
||||
};
|
||||
await meImagesTable.update(id, {
|
||||
usage: nextUsage,
|
||||
updatedAt: new Date().toISOString(),
|
||||
});
|
||||
emitDomainEvent('MeImageAiReferenceToggled', 'profile', 'meImages', id, {
|
||||
meImageId: id,
|
||||
enabled,
|
||||
});
|
||||
},
|
||||
|
||||
/**
|
||||
* Claim a primary slot for `id`, clearing any previous holder of
|
||||
* the same slot in the same transaction. At most one image per
|
||||
* slot is ever active — the query layer relies on this invariant.
|
||||
*
|
||||
* Pass `null` as the second argument to unset the slot on `id`
|
||||
* without claiming it for anyone else.
|
||||
*/
|
||||
async setPrimary(id: string, slot: MeImagePrimarySlot | null): Promise<void> {
|
||||
const nowIso = new Date().toISOString();
|
||||
await meImagesTable.db.transaction('rw', meImagesTable, async () => {
|
||||
if (slot === null) {
|
||||
await meImagesTable.update(id, { primaryFor: null, updatedAt: nowIso });
|
||||
return;
|
||||
}
|
||||
// Clear any current holder of this slot (usually zero or one).
|
||||
const current = await meImagesTable.where('primaryFor').equals(slot).toArray();
|
||||
for (const row of current) {
|
||||
if (row.id === id) continue;
|
||||
await meImagesTable.update(row.id, { primaryFor: null, updatedAt: nowIso });
|
||||
}
|
||||
await meImagesTable.update(id, { primaryFor: slot, updatedAt: nowIso });
|
||||
});
|
||||
emitDomainEvent('MeImagePrimaryChanged', 'profile', 'meImages', id, {
|
||||
meImageId: id,
|
||||
slot,
|
||||
});
|
||||
},
|
||||
|
||||
async deleteMeImage(id: string): Promise<void> {
|
||||
const nowIso = new Date().toISOString();
|
||||
await meImagesTable.update(id, {
|
||||
deletedAt: nowIso,
|
||||
updatedAt: nowIso,
|
||||
// Dropping a primary-holder silently leaves the slot empty;
|
||||
// the UI's primary-picker will prompt the user to pick a new
|
||||
// one next time it renders.
|
||||
primaryFor: null,
|
||||
});
|
||||
emitDomainEvent('MeImageDeleted', 'profile', 'meImages', id, { meImageId: id });
|
||||
},
|
||||
};
|
||||
|
|
@ -119,3 +119,91 @@ export function emptyUserContext(): LocalUserContext {
|
|||
interview: { answeredIds: [], skippedIds: [] },
|
||||
} as LocalUserContext;
|
||||
}
|
||||
|
||||
// ── Me-Images: user-owned reference images for AI generation ───────
|
||||
// Plan: docs/plans/me-images-and-reference-generation.md
|
||||
//
|
||||
// Small, curated pool (typically 2–10 images) the user uploads once —
|
||||
// a face portrait, a fullbody shot, maybe hands for ring try-ons.
|
||||
// Per-bild opt-in (`usage.aiReference`) gates whether a given image
|
||||
// may be sent to OpenAI `/v1/images/edits` when the Picture generator
|
||||
// runs in reference mode.
|
||||
//
|
||||
// User-level table (like userContext): no spaceId, no authorId. The
|
||||
// same human uses the same face across every Space.
|
||||
|
||||
/**
|
||||
* Reference kind. `face` and `fullbody` have dedicated primary slots
|
||||
* in the UI (M2). `halfbody`, `hands`, and generic `reference` exist
|
||||
* so the user can hold additional context (hands for rings, half-body
|
||||
* for chest-up generations) without overloading the two main slots.
|
||||
*/
|
||||
export type MeImageKind = 'face' | 'fullbody' | 'halfbody' | 'hands' | 'reference';
|
||||
|
||||
/**
|
||||
* Primary slot a given image fills. At most one image per slot is
|
||||
* active at a time — setPrimary(id, slot) clears the previous holder.
|
||||
* - `avatar`: drives the derived auth.users.image (M2 sync hook).
|
||||
* - `face-ref`: default face fed to the reference generator.
|
||||
* - `body-ref`: default fullbody reference.
|
||||
*/
|
||||
export type MeImagePrimarySlot = 'avatar' | 'face-ref' | 'body-ref';
|
||||
|
||||
export interface MeImageUsage {
|
||||
/** Explicit opt-in per image: may KI verwenden? Default false on upload. */
|
||||
aiReference: boolean;
|
||||
/** Counts towards avatar fallback if primary=avatar is not set. */
|
||||
showInProfile: boolean;
|
||||
}
|
||||
|
||||
export interface LocalMeImage extends BaseRecord {
|
||||
id: string;
|
||||
kind: MeImageKind;
|
||||
label?: string;
|
||||
mediaId: string;
|
||||
storagePath: string;
|
||||
publicUrl: string;
|
||||
thumbnailUrl?: string | null;
|
||||
width: number;
|
||||
height: number;
|
||||
tags: string[];
|
||||
usage: MeImageUsage;
|
||||
primaryFor?: MeImagePrimarySlot | null;
|
||||
}
|
||||
|
||||
export interface MeImage {
|
||||
id: string;
|
||||
kind: MeImageKind;
|
||||
label?: string;
|
||||
mediaId: string;
|
||||
storagePath: string;
|
||||
publicUrl: string;
|
||||
thumbnailUrl?: string | null;
|
||||
width: number;
|
||||
height: number;
|
||||
tags: string[];
|
||||
usage: MeImageUsage;
|
||||
primaryFor?: MeImagePrimarySlot | null;
|
||||
createdAt: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
|
||||
/** Convert a LocalMeImage (Dexie row) to the public MeImage type. */
|
||||
export function toMeImage(local: LocalMeImage): MeImage {
|
||||
return {
|
||||
id: local.id,
|
||||
kind: local.kind,
|
||||
label: local.label,
|
||||
mediaId: local.mediaId,
|
||||
storagePath: local.storagePath,
|
||||
publicUrl: local.publicUrl,
|
||||
thumbnailUrl: local.thumbnailUrl ?? null,
|
||||
width: local.width,
|
||||
height: local.height,
|
||||
tags: local.tags ?? [],
|
||||
usage: local.usage ?? { aiReference: false, showInProfile: true },
|
||||
primaryFor: local.primaryFor ?? null,
|
||||
createdAt: local.createdAt ?? '',
|
||||
updatedAt: local.updatedAt ?? '',
|
||||
};
|
||||
}
|
||||
|
|
|
|||
354
docs/plans/me-images-and-reference-generation.md
Normal file
354
docs/plans/me-images-and-reference-generation.md
Normal file
|
|
@ -0,0 +1,354 @@
|
|||
# Me-Images + Reference-basierte Bildgenerierung — Plan
|
||||
|
||||
## Status (2026-04-23)
|
||||
|
||||
Greenfield. Keine Zeile Code, kein Schema, kein Endpunkt. Vorarbeit: Picture-Modul hat bereits ungenutzte `sourceImageId` + `generationId` Felder (Platzhalter), OpenAI `gpt-image-2` ist für Text-zu-Bild produktiv über `apps/api/src/modules/picture/routes.ts:65-96`.
|
||||
|
||||
## Ziel
|
||||
|
||||
Der Nutzer hinterlegt **mehrere eigene Referenzbilder** (Gesicht, Ganzkörper, weitere Posen/Outfits) in einem zentralen Pool. Diese Bilder werden **explizit opt-in** von KI-Bildgenerierung als Referenz verwendet, primär über **OpenAI `gpt-image-2`** (der `/v1/images/edits`-Endpoint akzeptiert bis zu 16 Reference-Images pro Call) mit Replicate-Fallback und optional lokalem `mana-image-gen` (FLUX + IP-Adapter, später).
|
||||
|
||||
Kernfragen, die dieser Plan beantwortet:
|
||||
1. Wo leben die Referenzbilder? (Datenmodell, Scope, Verschlüsselung)
|
||||
2. Wie kommen sie in den Generator-Payload? (UI + API)
|
||||
3. Wie ruft der Server OpenAI mit Reference-Images? (Backend)
|
||||
4. Welche Use-Cases ergeben sich? (Konsumenten-Module)
|
||||
|
||||
Nicht im Scope:
|
||||
- **Wardrobe/Outfit-Modul** — bekommt einen eigenen Plan (`wardrobe-module.md`), konsumiert nur das hier entstehende Fundament.
|
||||
- **Face-Swap in Video/Live-Streams** — nur Still-Images.
|
||||
- **Per-Space-Avatare** — ein Nutzer hat eine Identität; falls später Bedarf, reicht ein `spaceId`-Zusatzfeld.
|
||||
- **Gesichtsvalidierung / Liveness-Check** — Vertrauensmodell: der Nutzer lädt nur Bilder seiner selbst hoch, wir erzwingen das nicht.
|
||||
|
||||
## Abgrenzung
|
||||
|
||||
- **Kein `photos`**: `photos` ist Album/Tag-orientiert für beliebige Fotos. `meImages` ist ein kuratierter, winziger Pool (typ. 2–10 Bilder) mit klarer KI-Opt-in-Semantik.
|
||||
- **Kein `body`**: `body` trackt Messungen/Workout. Progress-Fotos (Before/After) gehören dort hin, nicht in `meImages` — das hier ist für KI-Referenz, nicht für Fitness-Logging.
|
||||
- **Kein `picture.images`**: `images` sind KI-generierte oder importierte Assets für Boards. `meImages` ist der *Input* für Generierung, nicht das Ergebnis.
|
||||
- **Cross-Link**: `picture.images.sourceImageId` und `picture.images.referenceImageIds[]` zeigen auf `meImages.mediaId` (oder andere media-IDs). Das Picture-Modul bleibt der zentrale Ort, an dem das Ergebnis landet.
|
||||
|
||||
## Entscheidungen
|
||||
|
||||
### 1. Eigene Dexie-Tabelle, **nicht** `auth.users.image` erweitern
|
||||
|
||||
Gründe:
|
||||
- `auth.users.image` ist eine einzelne Text-URL in Better Auth. Mehrere Bilder + Metadaten + KI-Flags passen nicht rein ohne das Auth-Schema zu verunstalten.
|
||||
- Dexie + mana-sync + Encryption-Registry sind das etablierte Pattern für per-User-Daten.
|
||||
- `auth.users.image` bleibt als **abgeleitete Anzeige** erhalten (Primary-Face → Avatar-URL), wird aber über einen Sync-Hook gepflegt, nicht direkt beschrieben.
|
||||
|
||||
### 2. Pro User, **nicht** pro Space
|
||||
|
||||
Ein Mensch hat eine Identität. Space-spezifische Avatare (Brand-Space vs. Personal-Space) sind ein 10%-Fall und können später über ein optionales `spaceOverride: { [spaceId]: meImageId }` Feld im `profile`-Singleton gelöst werden, ohne `meImages` selbst zu ändern.
|
||||
|
||||
### 3. Primär `gpt-image-2` via `/v1/images/edits`, nicht Text-zu-Bild
|
||||
|
||||
Der Text-zu-Bild-Endpoint (`/v1/images/generations`) wird produktiv für freie Generierung genutzt und bleibt wie er ist. Für Reference-Workflows nutzen wir **`/v1/images/edits`** — derselbe Endpoint akzeptiert:
|
||||
- `image` (multipart) — eine oder mehrere Reference-Bilder (gpt-image-2: bis zu 16)
|
||||
- `prompt` — der Transformations-Wunsch
|
||||
- `mask` (optional) — für Inpainting
|
||||
- `size`, `quality`, `n` wie gehabt
|
||||
|
||||
Das ist der native OpenAI-Weg und erspart uns IP-Adapter-Engineering auf dem eigenen GPU-Server. Lokaler Fallback (FLUX + PuLID/InstantID auf RTX 3090) wird als **M5 / später** geplant, nicht in M1-M3.
|
||||
|
||||
### 4. Opt-in pro Bild, nicht global
|
||||
|
||||
Jedes `meImage` hat ein `usage.aiReference: boolean` Flag. Default beim Upload: **false**. Der Nutzer aktiviert gezielt, welche Bilder die KI verwenden darf. Global-Kill-Switch kommt aus dem Profile-Singleton (`profile.aiUsesReferenceImages: boolean`), Default **true**, damit einzelne Opt-ins direkt wirken.
|
||||
|
||||
## Architektur-Überblick
|
||||
|
||||
```
|
||||
┌─ Client (SvelteKit) ────────────────────────────────────┐
|
||||
│ /settings/me-images (Upload + Toggles) │
|
||||
│ picture/GeneratorForm (Reference-Picker) │
|
||||
│ Dexie: meImages (encrypted label/tags/kind) │
|
||||
└──────┬──────────────────────────────────────────────────┘
|
||||
│ mana-sync (encrypted rows)
|
||||
▼
|
||||
┌─ mana-sync → PostgreSQL (mana_sync.meImages) ───────────┐
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
|
||||
┌─ Generate-Flow (NEU) ───────────────────────────────────┐
|
||||
│ POST /api/v1/picture/generate-with-reference │
|
||||
│ { prompt, referenceMediaIds: [...], mode, mask? } │
|
||||
│ │
|
||||
│ Backend: │
|
||||
│ 1. Credits validieren (edits kostet wie generate) │
|
||||
│ 2. Fetch reference buffers aus mana-media (via mediaId) │
|
||||
│ 3. multipart → OpenAI /v1/images/edits │
|
||||
│ oder (Fallback) mana-image-gen /edit │
|
||||
│ 4. Response → uploadImageToMedia → return {images[]} │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
|
||||
┌─ Tool-Registry / MCP ───────────────────────────────────┐
|
||||
│ me.listReferenceImages (read-only, für Personas) │
|
||||
│ me.generateWithReference (triggert obigen Endpoint) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Datenmodell
|
||||
|
||||
### Neue Dexie-Tabelle: `meImages`
|
||||
|
||||
```typescript
|
||||
// apps/mana/apps/web/src/lib/modules/profile/types.ts
|
||||
export type MeImageKind =
|
||||
| 'face' // Kopf/Schulter, neutral
|
||||
| 'fullbody' // Ganzkörper, stehend
|
||||
| 'halfbody' // Hüfte aufwärts
|
||||
| 'hands' // für Schmuck/Ring-Anproben
|
||||
| 'reference'; // sonstige (andere Pose, anderer Lichtkontext)
|
||||
|
||||
export interface LocalMeImage {
|
||||
id: string;
|
||||
kind: MeImageKind;
|
||||
label?: string; // "Portrait neutral Studio", "Outfit Juni"
|
||||
mediaId: string; // → mana-media CAS (quelle-of-truth fürs Bild)
|
||||
storagePath: string; // cached vom mana-media-Response
|
||||
publicUrl: string;
|
||||
thumbnailUrl?: string;
|
||||
width: number;
|
||||
height: number;
|
||||
tags: string[]; // 'smiling', 'glasses-off', 'studio-light'
|
||||
usage: {
|
||||
aiReference: boolean; // Opt-in: darf KI das nutzen?
|
||||
showInProfile: boolean; // für Avatar-Fallback-Logik
|
||||
};
|
||||
primaryFor?: 'avatar' | 'face-ref' | 'body-ref' | null;
|
||||
createdAt: number;
|
||||
updatedAt: number;
|
||||
_pendingSync?: number;
|
||||
}
|
||||
```
|
||||
|
||||
**Primary-Logik**: Pro `primaryFor`-Wert existiert maximal ein `meImage` mit diesem Flag. Setzen eines neuen Primary räumt das alte auf (Store-Methode `setPrimary(id, slot)`).
|
||||
|
||||
### Encryption-Registry-Eintrag
|
||||
|
||||
```typescript
|
||||
// apps/mana/apps/web/src/lib/data/crypto/registry.ts
|
||||
meImages: {
|
||||
enabled: true,
|
||||
fields: ['label', 'tags', 'kind']
|
||||
}
|
||||
```
|
||||
|
||||
`mediaId`, `storagePath`, `publicUrl`, `width`, `height`, `primaryFor`, Timestamps → plaintext (konsistent mit `images` im Picture-Modul). Das Bild selbst liegt hinter mana-media-Auth — nicht verschlüsselt auf Dateiebene, aber nur für den Owner abrufbar. Für Zero-Knowledge-Modus-Nutzer: im M4 kommt optionale client-seitige Blob-Verschlüsselung dazu (out-of-scope für M1).
|
||||
|
||||
### Kein neuer Sync-Endpoint nötig
|
||||
|
||||
mana-sync behandelt `meImages` wie jede andere per-User-Tabelle (userScoped, nicht spaceScoped). Nur Registrierung in der Sync-Schema-Liste.
|
||||
|
||||
### Picture-Modul: bestehende Felder aktivieren + eins ergänzen
|
||||
|
||||
```typescript
|
||||
// apps/mana/apps/web/src/lib/modules/picture/types.ts
|
||||
export interface LocalImage {
|
||||
// ... bestehend
|
||||
sourceImageId?: string | null; // bereits vorhanden — jetzt genutzt
|
||||
referenceImageIds?: string[] | null; // NEU: für multi-reference gpt-image-2
|
||||
generationMode?: 'text' | 'edit' | 'inpaint'; // NEU
|
||||
generationId?: string | null; // bereits vorhanden
|
||||
}
|
||||
```
|
||||
|
||||
Encryption-Registry: `referenceImageIds`, `generationMode` → plaintext (IDs sind random, keine Leak-Gefahr).
|
||||
|
||||
## Backend-Erweiterungen
|
||||
|
||||
### Neuer Endpoint: `POST /api/v1/picture/generate-with-reference`
|
||||
|
||||
Datei: `apps/api/src/modules/picture/routes.ts` (erweitern, nicht neue Datei)
|
||||
|
||||
```typescript
|
||||
routes.post('/generate-with-reference', async (c) => {
|
||||
const userId = c.get('userId');
|
||||
const {
|
||||
prompt,
|
||||
model, // 'openai/gpt-image-2' | 'local/flux-pulid' | …
|
||||
referenceMediaIds, // string[] (mana-media IDs; aus meImages oder picture.images)
|
||||
mode, // 'edit' | 'inpaint'
|
||||
maskMediaId, // optional, nur für inpaint
|
||||
quality,
|
||||
width,
|
||||
height,
|
||||
n,
|
||||
} = await c.req.json();
|
||||
|
||||
// 1. Credits — gleicher Tarif wie /generate (3/10/25 je quality × n)
|
||||
// 2. Reference-Buffers holen (parallel): for each id → fetchMediaBuffer(id, userId)
|
||||
// — mana-media verifiziert, dass userId der Owner ist (keine fremden IDs)
|
||||
// 3. multipart/form-data bauen:
|
||||
// model, prompt, size, quality, n
|
||||
// image[] (als File-Parts; bei n>1 refs: image[]=ref1, image[]=ref2, …)
|
||||
// mask (optional)
|
||||
// 4. POST https://api.openai.com/v1/images/edits
|
||||
// 5. b64_json → uploadImageToMedia → return { images: [...] }
|
||||
});
|
||||
```
|
||||
|
||||
**Lib-Helper neu** in `apps/api/src/lib/media.ts`: `fetchMediaBuffer(mediaId, userId): Promise<ArrayBuffer>` — lädt + verifiziert Ownership in einem Call.
|
||||
|
||||
**Modell-Routing** analog zum bestehenden `/generate`:
|
||||
- `openai/gpt-image-2` (default) → OpenAI `/v1/images/edits`
|
||||
- `local/*` → mana-image-gen `/edit` (siehe M5)
|
||||
- Replicate hat keinen äquivalenten Multi-Reference-Endpoint → wir überspringen Replicate hier; fällt auf OpenAI zurück.
|
||||
|
||||
**Fehler-Matrix**:
|
||||
- 402 Insufficient credits
|
||||
- 404 Reference media not found or not owned
|
||||
- 413 Reference zu groß (OpenAI-Limit: 4MB pro PNG)
|
||||
- 502 OpenAI-Fehler (mit `detail.slice(0,500)` wie bisher)
|
||||
|
||||
### `mana-image-gen` erweitern (M5, nicht M1)
|
||||
|
||||
Python/FastAPI-Seite bekommt einen `POST /edit` Endpoint, der IP-Adapter oder PuLID auf FLUX lädt und `reference_images: list[bytes]` + `prompt` annimmt. Weil Replicate/lokal nicht parallel zu OpenAI im selben Call laufen müssen, ist das ein reiner Fallback für Offline-/Zero-Knowledge-Szenarien und kann später dazukommen.
|
||||
|
||||
## UI: zwei Touchpoints
|
||||
|
||||
### 1. `/settings/me-images` (neu)
|
||||
|
||||
- 2 prominente Slots oben: **Gesicht** (quadratisch, 512×512 empfohlen) und **Ganzkörper** (portrait, min 1024 hoch)
|
||||
- Darunter Grid für zusätzliche Referenzen (Drag-and-Drop, Multi-Select-Upload — Pattern aus `picture/ListView.svelte:165-217` klauen)
|
||||
- Pro Bild-Kachel:
|
||||
- Kind-Badge (Gesicht / Ganzkörper / Hände / …)
|
||||
- Toggle `usage.aiReference` (prominent, mit Tooltip "Wird an OpenAI gesendet wenn du ein Bild mit Referenz generierst")
|
||||
- Primary-Stern (nur einer pro Slot aktiv)
|
||||
- Tag-Editor
|
||||
- Löschen
|
||||
- Oben Globaler Kill-Switch: "KI darf meine Referenzbilder verwenden" (aus `profile`-Singleton)
|
||||
- Hinweis-Card zu Datenschutz: wo landen die Bilder, wer sieht sie, wie löschen
|
||||
|
||||
Zugriff: ⚙ im `profile`-Modul → "Meine Bilder" + direkte Route.
|
||||
|
||||
### 2. Picture-Generator: Reference-Picker
|
||||
|
||||
In `apps/mana/apps/web/src/lib/modules/picture/components/GeneratorForm.svelte` (oder Äquivalent):
|
||||
|
||||
- Neuer "Referenz hinzufügen"-Button öffnet ein Popover
|
||||
- Popover listet:
|
||||
- *Mich*: alle `meImages` mit `usage.aiReference === true` (primary zuerst)
|
||||
- *Aus diesem Modul*: letzte N `images` (für Generation-Chaining)
|
||||
- Multi-Select bis zu 4 Referenzen (Client-Limit, OpenAI erlaubt 16)
|
||||
- Wenn mindestens eine Referenz gewählt: Endpoint switched auf `/generate-with-reference`, UI zeigt "gpt-image-2 Edit" statt "Generate"
|
||||
- Optional: Mask-Drawing für Inpainting (out-of-scope für M2, kommt in M3)
|
||||
|
||||
## Tool-Registry + MCP
|
||||
|
||||
Nach M1+M2 bekommt `packages/mana-tool-registry` (siehe Memory, MCP M1+M1.5 shipped) zwei neue Tools:
|
||||
|
||||
- `me.listReferenceImages()` — read-only, gibt `{ id, kind, label, primaryFor, thumbnailUrl }[]` zurück, nur `aiReference=true` Einträge. Plaintext-Tier (label wird ent-verschlüsselt auf Server-Seite wie andere encrypted Tools).
|
||||
- `me.generateWithReference({ prompt, referenceImageIds, mode })` — wrappt den neuen Endpoint, gibt `{ imageIds, mediaIds }` zurück.
|
||||
|
||||
Damit können Personas (AI Workbench, Chat, ai-missions) und externe MCP-Clients (Claude Desktop) den Nutzer "visualisieren". Beispiel: Persona "Stylistin" bekommt `me.listReferenceImages` + `me.generateWithReference` als Tool-Subset und kann in Chat sagen *"Probieren wir drei Brillen-Looks?"*.
|
||||
|
||||
## Verschlüsselung + Datenschutz
|
||||
|
||||
- **Metadaten** (label, tags, kind): client-seitig AES-GCM-256 vor Dexie-Write, wie im Standard-Pattern.
|
||||
- **Bilddaten**: bleiben in MinIO (mana-media Bucket) mit Owner-RLS. Für Zero-Knowledge-Mode-Nutzer kommt in M4 optionale Client-Blob-Verschlüsselung (Upload verschlüsselt → Server sieht Ciphertext → OpenAI bekommt nur Bilder, wenn der Nutzer den Key entsperrt und den Edit-Call triggert). Das ist ein eigener Workstream und kein Blocker für M1-M3.
|
||||
- **OpenAI-Call**: jeder `/generate-with-reference`-Call geht als HTTPS-Multipart raus. Bilder landen kurzzeitig auf OpenAI-Servern (Policy: 30 Tage). Das muss die Settings-UI explizit erwähnen.
|
||||
- **Audit**: jeder Edit-Call loggt `{userId, referenceMediaIds, prompt, model, timestamp}` in eine neue `picture.generation_log`-Tabelle (nicht encrypted, für Rechnungs-/Abuse-Prüfung — Memoro-seitig, nicht in Dexie).
|
||||
|
||||
## Use-Cases + Modul-Zuordnung
|
||||
|
||||
### M1–M3 decken diese Use-Cases direkt ab:
|
||||
|
||||
| Use Case | Wo im UI | Modul |
|
||||
|---|---|---|
|
||||
| "Zeig mir wie ich mit einer schwarzen Brille aussehe" | Picture Generator → Reference: face → Prompt | `picture` |
|
||||
| "Generiere ein Profilbild im Studio-Look aus meinem Selfie" | Picture Generator → Reference: face → Prompt | `picture` |
|
||||
| "Mach ein Titelbild für meine Präsentation mit meinem Portrait" | Presi → Cover-Generator → Reference-Picker | `presi` (M4 Konsument) |
|
||||
| "Ich in mittelalterlicher Rüstung" / kreative Spielereien | Picture Generator | `picture` |
|
||||
| Avatar automatisch aus primary face ableiten | Profile-Settings | `profile` |
|
||||
|
||||
### Eigener Folge-Plan `wardrobe-module.md` (nicht in diesem Plan):
|
||||
|
||||
| Use Case | Wo im UI | Modul |
|
||||
|---|---|---|
|
||||
| Outfit-Katalog pflegen (T-Shirts, Hosen, Schuhe als einzelne Items) | Wardrobe Gridview | `wardrobe` (neu) |
|
||||
| "Kombiniere diese Jacke mit meinem Outfit aus Foto X" | Wardrobe → Outfit-Composer | `wardrobe` |
|
||||
| Virtual Try-On mit Ganzkörper-Referenz + Garment-Referenz | Wardrobe → Try-On | `wardrobe` |
|
||||
| Jahreszeit-Vorschläge ("Was ziehe ich heute an") | Wardrobe Daily-Card | `wardrobe` |
|
||||
|
||||
### Weitere sinnvolle Konsumenten (eigene Tickets, nicht Teil dieses Plans):
|
||||
|
||||
- **`website`** (Block-Tree CMS, in Planung): Portrait-Block kann `primaryFor='avatar'` automatisch ziehen.
|
||||
- **`presi`**: Cover-Slide-Template mit Nutzer-Portrait.
|
||||
- **`broadcast`** / **`social-relay`**: Avatar-Generierung für Posts.
|
||||
- **`dreams`**: "Ich im Traum" — Nutzer als Protagonist in KI-generierten Traum-Szenen.
|
||||
- **`wishes`**: "Wie würde mir das stehen" — Wishlist-Preview vor dem Kauf.
|
||||
|
||||
## Migrationsplan
|
||||
|
||||
Soft-first/Hard-follow-up-Regel (siehe Memory):
|
||||
|
||||
1. **Soft**: Dexie v27 führt `meImages` ein, Encryption-Registry um den Eintrag erweitern, sync-Schema registrieren. `auth.users.image` bleibt als-is. Neue Primary-Face-Uploads schreiben *zusätzlich* zur `meImages`-Tabelle.
|
||||
2. **Hard (Folge-Commit, einige Tage später)**: One-shot-Migration im Client: existierendes `auth.users.image` → `meImages` mit `kind='face'`, `primaryFor='avatar'`, `usage.aiReference=false` (Opt-in bleibt explizit). `auth.users.image` wird danach zum abgeleiteten Feld, das über einen Sync-Hook aus `meImages(primaryFor='avatar').publicUrl` gefüllt wird.
|
||||
|
||||
## Milestones
|
||||
|
||||
- **M1 — `meImages` Foundation** (~1 Tag)
|
||||
- [ ] Dexie v27: `meImages`-Tabelle
|
||||
- [ ] `apps/mana/apps/web/src/lib/modules/profile/types.ts`: Typen
|
||||
- [ ] Encryption-Registry-Eintrag
|
||||
- [ ] Store (`stores/meImages.svelte.ts`): CRUD + `setPrimary`
|
||||
- [ ] Queries (`useMyImages`, `useReferenceImages`, `useImageByPrimary`)
|
||||
- [ ] Sync-Schema registrieren
|
||||
- [ ] Upload-Wrapper nutzt bestehenden `picture/upload`-Endpoint mit `app=me` (neuer Bucket `me-storage` in MinIO)
|
||||
|
||||
- **M2 — UI Route `/settings/me-images`** (~1 Tag)
|
||||
- [ ] Route + ModuleShell-Wrapping (wie andere Settings-Routen)
|
||||
- [ ] Slot-Komponenten für Face/Fullbody, Grid für Reste
|
||||
- [ ] Drag-and-Drop-Upload + Multi-File
|
||||
- [ ] Opt-in-Toggles pro Bild + global
|
||||
- [ ] Primary-Stern
|
||||
- [ ] Profile-Modul ⚙ → neuer Eintrag "Meine Bilder"
|
||||
- [ ] Hard-Migration `auth.users.image` → `meImages`
|
||||
|
||||
- **M3 — Backend `generate-with-reference`** (~1-2 Tage)
|
||||
- [ ] `fetchMediaBuffer`-Helper in `apps/api/src/lib/media.ts`
|
||||
- [ ] Neue Route `POST /picture/generate-with-reference` mit OpenAI `/v1/images/edits`
|
||||
- [ ] Credit-Validierung identisch zu `/generate`
|
||||
- [ ] Generation-Log-Tabelle
|
||||
- [ ] Fehler-Matrix
|
||||
|
||||
- **M4 — Picture-Generator UI** (~1 Tag)
|
||||
- [ ] Reference-Picker-Popover in GeneratorForm
|
||||
- [ ] Payload-Switch `/generate` vs. `/generate-with-reference`
|
||||
- [ ] `picture.images.referenceImageIds` + `generationMode` persistieren
|
||||
- [ ] Detailansicht eines Bilds zeigt genutzte Referenzen
|
||||
|
||||
- **M5 — Tool-Registry + MCP-Exposure** (~0.5 Tag)
|
||||
- [ ] `me.listReferenceImages` + `me.generateWithReference` in `packages/mana-tool-registry`
|
||||
- [ ] MCP-Server (Port 3069) exponiert die Tools
|
||||
- [ ] Persona-Runner (sobald M2 von Personas-Plan live) kann sie konsumieren
|
||||
|
||||
- **M6 — (optional, später) Lokaler Fallback via mana-image-gen** (mehrere Tage)
|
||||
- [ ] FLUX + PuLID/InstantID auf GPU-Server
|
||||
- [ ] `POST /edit` in mana-image-gen
|
||||
- [ ] Routing über `local/flux-pulid`
|
||||
|
||||
- **M7 — (optional, später) Inpainting-Mask-Drawing** (~2 Tage)
|
||||
- [ ] Canvas-Mask-Editor im Picture-Generator
|
||||
- [ ] Mask als zweites Medium hochladen + an `/edit` übergeben
|
||||
|
||||
- **M8 — (optional, später) Zero-Knowledge-Bilder**
|
||||
- [ ] Client-seitige Verschlüsselung der Bild-Blobs in MinIO
|
||||
- [ ] Bei Generate-Call entschlüsselt der Client und sendet temp an Server → OpenAI → Result wieder verschlüsseln
|
||||
|
||||
## Entschieden (2026-04-23)
|
||||
|
||||
1. **Bucket-Namensgebung**: ~~eigener `me-storage`-Bucket~~ *revidiert* — mana-media nutzt heute einen einzelnen Bucket (`mana-media`); der `app`-String landet als Tag in `media_references.app`. Upload geht mit `app='me'`, kein neuer Bucket nötig. Falls später Lifecycle-Rules pro App-Tag nötig werden, reicht eine mc-Regel mit `--prefix 'me/'` auf dem mana-media-Bucket.
|
||||
2. **`primaryFor='avatar'` → `auth.users.image`**: Client-Dexie-Hook ruft `PUT /api/v1/auth/profile`. mana-sync bleibt außen vor.
|
||||
3. **OpenAI Ref-Image-Format**: Original-Format durchreichen (PNG/JPG/WEBP — OpenAI akzeptiert alle). Keine Server-Konvertierung.
|
||||
4. **Credit-Kosten für Multi-Ref-Edits**: identisch zu `/generate`, pro Output-Bild, unabhängig von Reference-Anzahl.
|
||||
5. **`profile.aiUsesReferenceImages`-Default**: `true` (globaler Panic-Kill-Switch; Pro-Bild-Opt-in ist die eigentliche Hürde).
|
||||
6. **Alter Avatar-Upload-Pfad**: bleibt in M1 unangetastet; M2 biegt `EditProfileModal` auf `/settings/me-images` um und räumt den toten Endpoint-Call weg.
|
||||
|
||||
## Verweise
|
||||
|
||||
- Bestehender Picture-Generate-Endpoint: `apps/api/src/modules/picture/routes.ts:43-227`
|
||||
- Picture Upload-Pattern (für UI-Klau): `apps/mana/apps/web/src/lib/modules/picture/ListView.svelte:165-217`
|
||||
- Encryption-Registry-Pattern: `apps/mana/apps/web/src/lib/data/crypto/registry.ts`
|
||||
- mana-media CAS: `services/mana-media/CLAUDE.md`
|
||||
- MCP-Gateway + Tool-Registry: `services/mana-mcp/CLAUDE.md`, `packages/mana-tool-registry/`
|
||||
- Spaces-Modul-Allowlist (falls neues `wardrobe` kommt): `packages/shared-types/src/spaces.ts:63-184`
|
||||
Loading…
Add table
Add a link
Reference in a new issue