feat(wardrobe,picture): try-on integration — outfit → OpenAI edit (M4)

M4 of docs/plans/wardrobe-module.md — the loop closes. A user with at
least a face-ref in the active space can click "Anprobieren" on an
outfit detail page; the client composes a reference call against the
existing M3 `/generate-with-reference` endpoint, persists the result
into the Picture gallery with a `wardrobeOutfitId` back-reference,
and pins a `lastTryOn` snapshot on the outfit so its card instantly
shows the AI preview next time.

Server side — picture/routes.ts:
- verifyMediaOwnership now accepts `apps: string | readonly string[]`.
  Under the hood it runs one list() per app-tag and unions the owned
  set before the missing-id check. Preserves the 500-row per-app
  sanity cap. Single-tag callers unchanged — it's an additive widen.
- Picture /generate-with-reference passes `['me', 'wardrobe']` so
  face/body portraits (me-images) and garment photos (wardrobe) can
  ride in the same referenceMediaIds array. Anything outside those
  two tags still 404s — no expansion of the trust surface.

Client side — wardrobe/api/try-on.ts:
- `runOutfitTryOn({ outfit, garments, faceRefMediaId, bodyRefMediaId?, ... })`
  composes the ref list (face → body → up to 6 garments, respecting
  the 8-slot server cap), picks portrait 1024x1536 by default (or
  1024x1024 in accessory-only mode), and POSTs with
  `model='openai/gpt-image-2'`, `quality='medium'`, `n=1`. One render
  per click; multi-variant is a future Generator-style extension.
- Default prompts are composed in DE from the outfit meta (name +
  occasion); callers can override via `prompt`. Accessory-only mode
  uses a tighter studio-portrait phrasing since the fullbody ref is
  dropped there.
- `isAccessoryOnlyOutfit()` helper — iff every garment is in
  FACE_ONLY_CATEGORIES, skip body-ref and render square. Covers the
  Brille-Try-On headline use case.
- On success: inserts a `picture.images` row with generationMode=
  'reference', referenceImageIds, and wardrobeOutfitId set; then
  calls wardrobeOutfitsStore.setLastTryOn() with imageId + imageUrl
  so OutfitCard + DetailOutfitView immediately flip to the AI cover.

TryOnButton — wardrobe/components/TryOnButton.svelte:
- Three states: ready (click to render), missing-references (shows
  UserCircle + link to /profile/me-images, with the right hint for
  accessory-only vs. fullbody), loading (spinner).
- Credit estimate on the button (10c medium quality).
- Hints: accessory-only, too-many-garments (>6, over server cap),
  and non-personal-space disclosure — the family-space case gets its
  own sentence since "Try-On rendert dich, nicht dein Kind" is
  non-obvious.
- Reads face-ref/body-ref via useImageByPrimary (space-scoped after
  the v40 meImages migration — brand/club/family spaces need their
  own references uploaded).

UI wiring:
- DetailOutfitView replaces the M3 stub button with <TryOnButton/>.
  The existing "Try-On Verlauf"-Strip already reads
  `useOutfitTryOns(outfit.id)` which filters `picture.images` by
  wardrobeOutfitId — it lights up automatically on first render.

Not in M4 (punted to follow-ups):
- Solo-garment try-on on DetailGarmentView ("nur diese Brille auf
  mein Gesicht"). Plan called it out as optional; the outfit flow
  already covers it when the outfit contains only that one garment.
- Multi-variant rendering (n=2/4). Usable "show me 3 looks" needs a
  picker UI on top, not just a param bump.
- Quality + prompt override in the button. A power-user panel can
  come later; default medium + auto-prompt keeps M4's click-to-try-on
  one-tap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Till JS 2026-04-23 18:52:57 +02:00
parent 2b89bf7955
commit d56ad396d8
5 changed files with 352 additions and 27 deletions

View file

@ -58,25 +58,33 @@ export async function getMediaBuffer(
} }
/** /**
* Verify that every id in `mediaIds` is owned by `userId` under the given * Verify that every id in `mediaIds` is owned by `userId` under one of
* `app` scope. Throws { status: 404 } when one or more ids are not in the * the given app scopes. Throws `{ status: 404, missing }` when any id
* user's reference set the caller turns that into an HTTP response. * doesn't land in the owned set the caller turns that into an HTTP
* response.
* *
* One `list()` round-trip is all we need: the response is the full set of * Accepts a single app string or an array. The Wardrobe try-on flow
* the user's uploads under that app tag, so set-membership check is O(N) * (plan docs/plans/wardrobe-module.md M4) passes `['me', 'wardrobe']`
* in memory. The `limit: 500` cap is the sanity fence a single user with * in one call face-ref and body-ref live under `me`, garments live
* more than 500 reference images under one app is already far beyond the * under `wardrobe`, both legitimate inputs for the same `/v1/images/
* product's intended shape; we'd catch that as a design regression long * edits` POST.
* before it breaks this check. *
* One `list()` round-trip per app. For N apps this is N calls, each
* capped at 500 rows far beyond the product's intended per-app shape
* but the cap is the sanity fence.
*/ */
export async function verifyMediaOwnership( export async function verifyMediaOwnership(
userId: string, userId: string,
mediaIds: readonly string[], mediaIds: readonly string[],
app: string apps: string | readonly string[]
): Promise<void> { ): Promise<void> {
if (mediaIds.length === 0) return; if (mediaIds.length === 0) return;
const owned = await getMediaClient().list({ userId, app, limit: 500 }); const appList = typeof apps === 'string' ? [apps] : apps;
const ownedSet = new Set(owned.map((m) => m.id)); const ownedSet = new Set<string>();
for (const app of appList) {
const list = await getMediaClient().list({ userId, app, limit: 500 });
for (const m of list) ownedSet.add(m.id);
}
const missing = mediaIds.filter((id) => !ownedSet.has(id)); const missing = mediaIds.filter((id) => !ownedSet.has(id));
if (missing.length > 0) { if (missing.length > 0) {
const err = new Error(`Reference media not owned: ${missing.join(', ')}`) as Error & { const err = new Error(`Reference media not owned: ${missing.join(', ')}`) as Error & {

View file

@ -297,12 +297,13 @@ routes.post('/generate-with-reference', async (c) => {
} }
// Ownership check before we spend credits or burn OpenAI quota. // Ownership check before we spend credits or burn OpenAI quota.
// meImages are tagged `app='me'` at upload time by the profile // References span two upload tags: `me` for face/body portraits
// module; a mediaId that isn't in the caller's set is either stale // (profile module) and `wardrobe` for garment photos (wardrobe
// or malicious, treat both as 404. // module, M4 try-on flow). Anything outside those two apps is
// treated as not-owned regardless of mana-media's own view.
try { try {
const { verifyMediaOwnership } = await import('../../lib/media'); const { verifyMediaOwnership } = await import('../../lib/media');
await verifyMediaOwnership(userId, refIds, 'me'); await verifyMediaOwnership(userId, refIds, ['me', 'wardrobe']);
} catch (err) { } catch (err) {
const e = err as Error & { status?: number; missing?: string[] }; const e = err as Error & { status?: number; missing?: string[] };
if (e.status === 404) { if (e.status === 404) {

View file

@ -0,0 +1,183 @@
/**
* Try-On client. Composes a reference-based image-edit call against
* the M3 endpoint `/api/v1/picture/generate-with-reference` using the
* active space's face-ref + body-ref meImages plus every garment in
* the outfit, then persists the result into `picture.images` with the
* outfit's `wardrobeOutfitId` back-reference and updates the outfit's
* `lastTryOn` snapshot.
*
* The caller resolves the primaries reactively (via useImageByPrimary)
* and hands us the raw mediaIds keeps this pure and testable.
*
* Plan: docs/plans/wardrobe-module.md M4.
*/
import { getManaApiUrl } from '$lib/api/config';
import { authStore } from '$lib/stores/auth.svelte';
import { imagesStore } from '$lib/modules/picture/stores/images.svelte';
import { wardrobeOutfitsStore } from '../stores/outfits.svelte';
import { FACE_ONLY_CATEGORIES } from '../types';
import type { Garment, GarmentCategory, Outfit } from '../types';
export type TryOnSize = '1024x1024' | '1536x1024' | '1024x1536';
export interface RunOutfitTryOnParams {
outfit: Outfit;
garments: Garment[]; // resolved LocalWardrobeGarment rows, primary photos must exist
faceRefMediaId: string;
/** Optional — omit for accessory-only mode (glasses/jewelry/hat/accessory). */
bodyRefMediaId?: string | null;
/** Optional override; default is composed from the outfit meta. */
prompt?: string;
/** `medium` balances FLUX-ish detail against credit cost (10c). */
quality?: 'low' | 'medium' | 'high';
size?: TryOnSize;
}
export interface RunTryOnResult {
imageId: string; // picture.images.id (local UUID)
imageUrl: string; // mana-media URL
prompt: string;
model: string;
}
/**
* True iff every garment in the outfit is in a face-only category
* then the try-on renders just the face without a fullbody reference
* (better for brille/schmuck/hut zoom).
*/
export function isAccessoryOnlyOutfit(garments: Garment[]): boolean {
if (garments.length === 0) return false;
return garments.every((g) => FACE_ONLY_CATEGORIES.has(g.category as GarmentCategory));
}
function composeDefaultPrompt(outfit: Outfit, accessoryOnly: boolean): string {
if (accessoryOnly) {
return `Fotorealistisches Portrait von mir mit ${outfit.name}, frontal, studio-Licht, neutraler Hintergrund, Fokus auf dem Accessoire`;
}
const occasionHint = outfit.occasion ? ` (Anlass: ${outfit.occasion})` : '';
return `Fotorealistisches Portrait von mir im Outfit ${outfit.name}${occasionHint}, natürliches Licht, neutraler Hintergrund`;
}
export async function runOutfitTryOn(params: RunOutfitTryOnParams): Promise<RunTryOnResult> {
const { outfit, garments, faceRefMediaId, bodyRefMediaId, prompt, quality, size } = params;
const garmentMediaIds = garments
.map((g) => g.mediaIds[0])
.filter((id): id is string => Boolean(id));
if (garmentMediaIds.length === 0) {
throw new Error('Outfit hat keine Kleidungsstücke mit Foto.');
}
const accessoryOnly = isAccessoryOnlyOutfit(garments);
const effectiveSize: TryOnSize = size ?? (accessoryOnly ? '1024x1024' : '1024x1536');
const effectivePrompt = prompt?.trim() || composeDefaultPrompt(outfit, accessoryOnly);
// Reference order: face first, then body (if present), then garments.
// gpt-image-2 weights early refs slightly more for identity — keeping
// face at [0] makes the person recognizable before the garments
// negotiate for attention.
const referenceMediaIds: string[] = [faceRefMediaId];
if (!accessoryOnly && bodyRefMediaId) {
referenceMediaIds.push(bodyRefMediaId);
}
for (const id of garmentMediaIds) {
if (referenceMediaIds.length >= 8) break; // server caps at 8
referenceMediaIds.push(id);
}
const token = await authStore.getValidToken();
const res = await fetch(`${getManaApiUrl()}/api/v1/picture/generate-with-reference`, {
method: 'POST',
headers: {
'content-type': 'application/json',
...(token ? { Authorization: `Bearer ${token}` } : {}),
},
body: JSON.stringify({
prompt: effectivePrompt,
referenceMediaIds,
model: 'openai/gpt-image-2',
quality: quality ?? 'medium',
size: effectiveSize,
n: 1,
}),
});
if (!res.ok) {
const body = (await res.json().catch(() => ({}))) as {
error?: string;
required?: number;
missing?: string[];
};
if (res.status === 402) {
throw new Error(`Nicht genug Credits (${body.required ?? '?'} erforderlich)`);
}
if (res.status === 404) {
throw new Error(
'Ein oder mehrere Referenzbilder sind im Server-Ownership-Check durchgefallen — vermutlich sind Face/Body noch nicht in diesem Space hochgeladen.'
);
}
throw new Error(body.error ?? `Try-On fehlgeschlagen (${res.status})`);
}
const data = (await res.json()) as {
images?: Array<{ imageUrl: string; mediaId?: string; thumbnailUrl?: string }>;
imageUrl?: string;
mediaId?: string;
thumbnailUrl?: string;
prompt: string;
model: string;
referenceMediaIds?: string[];
};
const first =
(data.images && data.images[0]) ??
(data.imageUrl
? { imageUrl: data.imageUrl, mediaId: data.mediaId, thumbnailUrl: data.thumbnailUrl }
: null);
if (!first) throw new Error('Keine Bilder zurückgegeben');
const now = new Date().toISOString();
const localImageId = crypto.randomUUID();
// Persist the generated image to the Picture gallery + tag it with
// the outfit's wardrobeOutfitId so the outfit detail's Try-On strip
// picks it up via the useOutfitTryOns liveQuery.
await imagesStore.insert({
id: localImageId,
prompt: data.prompt,
negativePrompt: null,
model: data.model,
publicUrl: first.imageUrl,
storagePath: first.mediaId ?? first.imageUrl,
filename: `wardrobe-tryon-${Date.now()}.png`,
format: 'png',
width: effectiveSize === '1024x1536' ? 1024 : effectiveSize === '1536x1024' ? 1536 : 1024,
height: effectiveSize === '1024x1536' ? 1536 : effectiveSize === '1536x1024' ? 1024 : 1024,
isPublic: false,
isFavorite: false,
downloadCount: 0,
generationMode: 'reference',
referenceImageIds: referenceMediaIds,
wardrobeOutfitId: outfit.id,
createdAt: now,
updatedAt: now,
});
// Pin the snapshot on the outfit so OutfitCard + DetailOutfitView
// render the cover instantly without waiting for a full picture.images
// live-query round-trip.
await wardrobeOutfitsStore.setLastTryOn(outfit.id, {
imageId: localImageId,
imageUrl: first.imageUrl,
createdAt: now,
prompt: data.prompt,
model: data.model,
});
return {
imageId: localImageId,
imageUrl: first.imageUrl,
prompt: data.prompt,
model: data.model,
};
}

View file

@ -0,0 +1,140 @@
<!--
Try-On action for the outfit detail page. Handles three states:
ready (face + body present, click to render), waiting (user hasn't
uploaded references yet, show link to /profile/me-images), and
loading (request in flight).
-->
<script lang="ts">
import { Sparkle, UserCircle, Info } from '@mana/shared-icons';
import { getActiveSpace } from '$lib/data/scope';
import { useImageByPrimary } from '$lib/modules/profile/queries';
import { isAccessoryOnlyOutfit, runOutfitTryOn } from '../api/try-on';
import { CATEGORY_LABELS_SINGULAR } from '../constants';
import type { Garment, Outfit } from '../types';
interface Props {
outfit: Outfit;
garments: Garment[]; // the resolved LocalWardrobeGarment rows for this outfit
}
let { outfit, garments }: Props = $props();
const face$ = useImageByPrimary('face-ref');
const body$ = useImageByPrimary('body-ref');
const activeSpace = $derived(getActiveSpace());
const face = $derived(face$.value);
const body = $derived(body$.value);
const accessoryOnly = $derived(isAccessoryOnlyOutfit(garments));
// Face is always required. Body only matters for non-accessory outfits.
const missingFace = $derived(!face);
const missingBody = $derived(!accessoryOnly && !body);
const canTryOn = $derived(!missingFace && !missingBody && garments.length > 0);
let running = $state(false);
let error = $state<string | null>(null);
// Rough credit estimate — mirrors the server tariff from the M3 plan
// (3 low / 10 medium / 25 high; we default to medium). Shown on the
// button so the user knows the hit before clicking.
const estimatedCredits = 10;
async function handleClick() {
if (!face || (!accessoryOnly && !body)) return;
running = true;
error = null;
try {
await runOutfitTryOn({
outfit,
garments,
faceRefMediaId: face.mediaId,
bodyRefMediaId: accessoryOnly ? null : (body?.mediaId ?? null),
});
} catch (err) {
error = err instanceof Error ? err.message : 'Try-On fehlgeschlagen';
} finally {
running = false;
}
}
</script>
{#if missingFace || missingBody}
<div
class="flex items-start gap-3 rounded-xl border border-dashed border-border bg-background/50 p-4 text-sm text-muted-foreground"
>
<UserCircle size={18} weight="regular" class="mt-0.5 flex-shrink-0 text-primary" />
<div class="space-y-1">
<p class="text-foreground">Lade erst Referenzbilder hoch, um dich im Outfit zu sehen.</p>
<p class="text-xs">
Try-On braucht mindestens ein {accessoryOnly
? 'Gesichtsbild'
: 'Gesichts- und ein Ganzkörperbild'}
in diesem Space. Öffne dafür
<a href="/profile/me-images" class="font-medium text-primary hover:underline">
Meine Bilder
</a>.
</p>
</div>
</div>
{:else}
<div class="space-y-2">
<button
type="button"
onclick={handleClick}
disabled={running || !canTryOn}
class="flex w-full items-center justify-center gap-2 rounded-md bg-primary px-4 py-2.5 text-sm font-medium text-primary-foreground transition-colors hover:bg-primary/90 disabled:cursor-not-allowed disabled:opacity-50"
>
{#if running}
<div
class="h-4 w-4 animate-spin rounded-full border-2 border-current border-r-transparent"
></div>
Rendere…
{:else}
<Sparkle size={16} weight="fill" />
Anprobieren · {estimatedCredits} Credits
{/if}
</button>
{#if accessoryOnly}
<p class="flex items-center gap-1.5 text-xs text-muted-foreground">
<Info size={12} weight="regular" class="flex-shrink-0" />
Accessoire-Modus — nur das Gesicht wird gerendert (spart Credits).
</p>
{:else if garments.length > 6}
<p class="flex items-center gap-1.5 text-xs text-muted-foreground">
<Info size={12} weight="regular" class="flex-shrink-0" />
Mit {garments.length} Kleidungsstücken ist der Referenz-Slot knapp — ältere Items werden evtl.
nicht mitgezogen.
</p>
{/if}
{#if activeSpace && activeSpace.type !== 'personal'}
<p class="flex items-start gap-1.5 text-xs text-muted-foreground">
<Info size={12} weight="regular" class="mt-0.5 flex-shrink-0" />
<span>
Try-On nutzt deine Referenzbilder aus diesem Space
<strong class="text-foreground">({activeSpace.name})</strong>, nicht aus Persönlich.
{#if activeSpace.type === 'family'}
Kinder-Outfits werden trotzdem auf dein Gesicht gerendert.
{/if}
</span>
</p>
{/if}
{#if error}
<div
class="rounded-md border border-error/30 bg-error/10 px-3 py-2 text-sm text-error"
role="alert"
>
{error}
</div>
{/if}
</div>
{/if}
{#if !missingFace && !missingBody && garments.length === 0}
<p class="text-xs text-muted-foreground">
Füge mindestens ein {CATEGORY_LABELS_SINGULAR.top ?? 'Kleidungsstück'} hinzu, um Try-On zu aktivieren.
</p>
{/if}

View file

@ -12,11 +12,12 @@
--> -->
<script lang="ts"> <script lang="ts">
import { goto } from '$app/navigation'; import { goto } from '$app/navigation';
import { ArrowLeft, Archive, Heart, PencilSimple, Sparkle, Trash } from '@mana/shared-icons'; import { ArrowLeft, Archive, Heart, PencilSimple, Trash } from '@mana/shared-icons';
import { useAllGarments, useOutfit, useOutfitTryOns } from '../queries'; import { useAllGarments, useOutfit, useOutfitTryOns } from '../queries';
import { wardrobeOutfitsStore } from '../stores/outfits.svelte'; import { wardrobeOutfitsStore } from '../stores/outfits.svelte';
import { garmentPhotoUrl } from '../api/media-url'; import { garmentPhotoUrl } from '../api/media-url';
import { CATEGORY_LABELS_SINGULAR, OCCASION_LABELS, SEASON_LABELS } from '../constants'; import { CATEGORY_LABELS_SINGULAR, OCCASION_LABELS, SEASON_LABELS } from '../constants';
import TryOnButton from '../components/TryOnButton.svelte';
import type { Garment } from '../types'; import type { Garment } from '../types';
interface Props { interface Props {
@ -126,16 +127,8 @@
{/if} {/if}
</div> </div>
<!-- Try-On action (M4 stub) --> <!-- Try-On action (M4) -->
<button <TryOnButton {outfit} garments={resolvedGarments} />
type="button"
disabled
title="Try-On kommt in M4 — bis dahin ist das hier noch stumm."
class="flex w-full items-center justify-center gap-2 rounded-md bg-primary/50 px-4 py-2.5 text-sm font-medium text-primary-foreground opacity-60"
>
<Sparkle size={16} weight="fill" />
Anprobieren (kommt bald)
</button>
{#if tryOns.length > 0} {#if tryOns.length > 0}
<div> <div>