mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-15 00:01:10 +02:00
Diagnosis from the user's last test pinpointed the bug: mana-llm
returns totalFrames=0 (no SSE frames at all) when called from the
browser, but works perfectly when called via curl from the same host
with the same payload. Two compounding causes:
1. credentials: 'include' in our fetch combined with mana-llm's
CORS headers silently breaks the response body. This is the
classic "Access-Control-Allow-Origin: * + Allow-Credentials: true"
mismatch — browsers reject the response per spec but report it
as a 0-byte success rather than an error.
2. Streaming over CORS adds a second layer of fragility. Even if
credentials weren't an issue, the browser fetch API's response
body for SSE under CORS depends on a specific combination of
server headers we evidently don't have.
Fix: drop both the streaming AND the credentials.
- stream: false in the request body. Single JSON response per call,
much friendlier to the browser fetch API.
- No `credentials` field at all (default 'same-origin' for cross-
origin requests = don't send cookies). mana-llm's API key
middleware accepts anonymous requests, so we don't need to send
any auth context.
- Parse the response as `await res.json()` instead of streaming
SSE chunks. Pull `choice.message.content` (or fall back to
`choice.text` for legacy completions API responses).
- Backwards-compatibility shim for `req.onToken`: if a caller
registered a token callback (legacy chat-style streaming UX),
fire it ONCE with the full content at the end. The current
orchestrator + queue model never consumes per-token streams for
remote tiers, so this is a degraded-but-equivalent path. The
playground module uses its own client and isn't affected.
Verified manually with curl:
$ curl -X POST https://llm.mana.how/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"gemma3:4b","messages":[{"role":"user","content":"Hi"}],"max_tokens":50,"stream":false}'
→ returns clean JSON with `choices[0].message.content` populated.
Same call with `stream: true` from the same host also works (full
SSE frames come back). The bug really is browser+credentials
specific, not a service bug.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| src | ||
| package.json | ||
| tsconfig.json | ||