managarten/packages
Till JS e8423e7551 fix(local-llm): use two-step tokenization to fix Gemma 4 generate crash
The previous attempt to fix the "Cannot read properties of null
(reading 'dims')" chat error was incomplete: I only stopped passing
the bogus return_tensor:'pt' option to apply_chat_template. The
underlying issue was that apply_chat_template's all-in-one mode
(return_dict:true) does not produce a proper Tensor-backed
{ input_ids, attention_mask } pair for multimodal-capable processors
like Gemma4Processor — it returns a shape that has no .dims on
input_ids, so model.generate() crashes deep inside the forward pass
the moment it tries to read the sequence length.

Switch to the documented two-step pattern from the Gemma 4 model
card: call apply_chat_template with tokenize:false to get the
formatted prompt as a plain string, then run that string through
processor.tokenizer with return_tensors:'pt' to get a proper Tensor
pair. The tokenizer's return_tensors option is the *Python*
convention and IS supported by transformers.js's Tokenizer class
(the API name collision between apply_chat_template's return_tensor
boolean and Tokenizer's return_tensors string is one of those nasty
spots where the JS port intentionally diverges from Python).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 23:19:24 +02:00
..
cards-database fix(cards-database): add .js extensions to relative imports for NodeNext 2026-04-07 14:01:44 +02:00
credits chore(matrix): scrub final matrix references after subsystem removal 2026-04-08 16:39:42 +02:00
eslint-config fix(timeblocks): type errors from recurrence migration 2026-04-07 13:22:59 +02:00
feedback feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
help feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
local-llm fix(local-llm): use two-step tokenization to fix Gemma 4 generate crash 2026-04-08 23:19:24 +02:00
local-store fix(mana/web): sprint 2 — auth-aware data layer + guest migration 2026-04-07 13:07:12 +02:00
notify-client chore(matrix): final scrub of stale matrix references 2026-04-08 16:47:54 +02:00
qr-export chore(workspace): unify vitest to ^4.1.2 across all packages 2026-04-07 13:58:29 +02:00
shared-api-client feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-auth feat(auth): structured error codes + conditional passkey UI 2026-04-08 12:40:51 +02:00
shared-auth-ui refactor(auth-ui): tighten LoginPage UX, a11y, and dead code 2026-04-08 12:41:19 +02:00
shared-branding fix(mana-auth) + chore: rewrite /api/v1/auth/login JWT mint, remove Matrix stack 2026-04-08 16:32:13 +02:00
shared-config chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-drizzle-config feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-error-tracking feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-errors chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-go chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-hono feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-i18n chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-icons chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-landing-ui chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-links feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-llm chore(workspace): unify vitest to ^4.1.2 across all packages 2026-04-07 13:58:29 +02:00
shared-logger feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-pwa feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-python/manacore_auth feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-splitscreen feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-storage chore(workspace): unify vitest to ^4.1.2 across all packages 2026-04-07 13:58:29 +02:00
shared-stores fix: type errors from ManaCore→Mana rename and stale templates 2026-04-07 13:42:17 +02:00
shared-tags chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-tailwind chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-theme chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-theme-ui chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-tsconfig feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-types chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
shared-ui refactor(mana/web): consolidate ListView scaffolding into BaseListView 2026-04-08 18:40:47 +02:00
shared-uload feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
shared-utils fix(csp): move jsdelivr allowlist to mana-web hooks (Vite SSR cache workaround) 2026-04-08 23:03:38 +02:00
shared-vite-config chore: complete ManaCore → Mana rename (docs, go modules, plists, images) 2026-04-07 12:26:10 +02:00
spiral-db chore(workspace): unify vitest to ^4.1.2 across all packages 2026-04-07 13:58:29 +02:00
subscriptions feat: rename ManaCore to Mana across entire codebase 2026-04-05 20:00:13 +02:00
test-config chore(workspace): unify vitest to ^4.1.2 across all packages 2026-04-07 13:58:29 +02:00
wallpaper-generator chore(workspace): unify vitest to ^4.1.2 across all packages 2026-04-07 13:58:29 +02:00