mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-14 17:41:09 +02:00
chore(test + audit): add test-coverage audit + wire audit:all
#6 test coverage (pivot to reporting): 34/653 tests currently fail (in-flight spaces-foundation migrations). Hard coverage thresholds aren't enforceable until the suite is green, so this session ships a file-presence audit instead of line-coverage gates. - scripts/audit-test-coverage.mjs — counts .svelte + .ts source files vs .test.ts + .spec.ts per module. Reports total ratio, lists modules with 0 tests + ≥3 source files (prioritised by size). - pnpm run audit:test-coverage wires it into audit:*. - docs/optimizable/test-health.md — state + prevention path + top untested modules ranked by impact. Current baseline: 2.6% file-level coverage. 66/78 modules have zero tests. Biggest untested: times (32 src), articles (29), events (27), inventory + skilltree (20 each). #8 audit:all: single entry point for the reporting audits. Runs port-drift + i18n-coverage + test-coverage in --summary mode. Distinct from validate:all (which is gates, not reports). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
8a82f3c543
commit
68c0eb2892
3 changed files with 182 additions and 0 deletions
49
docs/optimizable/test-health.md
Normal file
49
docs/optimizable/test-health.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
# Test Health
|
||||
|
||||
**Status 2026-04-22.** Run `pnpm run audit:test-coverage` for live numbers.
|
||||
|
||||
## Current state
|
||||
|
||||
- **34/653 tests currently fail.** Related to in-flight
|
||||
spaces-foundation migration work (app-registry, api-keys, base-client,
|
||||
crypto record-helpers). Hard coverage thresholds aren't enforceable
|
||||
until the suite is green.
|
||||
- **File-level "coverage" is 2.6%.** 22 test files vs 857 source files
|
||||
(.svelte + .ts). 66 of 78 modules have zero tests.
|
||||
- **12 modules have ≥1 test file.** Hotspots (todo, times, body, a
|
||||
handful of critical infra) are partially covered; long tail is not.
|
||||
|
||||
## Why file presence, not line coverage
|
||||
|
||||
Running `vitest --coverage` requires the suite to pass. Until the 34
|
||||
failing tests are fixed, coverage numbers are undefined. In the
|
||||
meantime, "does this module have *any* automated regression
|
||||
protection?" is a useful signal for prioritising the next session.
|
||||
|
||||
## Top untested modules (by source file count)
|
||||
|
||||
| Module | Source files | Priority |
|
||||
|---|---|---|
|
||||
| times | 32 | HIGH — time-tracking logic, billing-adjacent |
|
||||
| articles | 29 | HIGH — reader pipeline, bookmarklet, parsing |
|
||||
| events | 27 | MED — RSVP / calendar interactions |
|
||||
| inventory | 20 | MED — user-owned item schema |
|
||||
| skilltree | 20 | MED — XP / level math |
|
||||
| library | 19 | LOW — simple CRUD |
|
||||
| photos | 17 | MED — upload / storage paths |
|
||||
| wetter | 17 | LOW |
|
||||
| calc | 16 | MED — arithmetic correctness |
|
||||
| meditate, news, quotes, cards, core, moodlit | 13–16 each | mixed |
|
||||
|
||||
## Prevention path
|
||||
|
||||
1. **Fix the 34 failing tests first** — they're noise blocking any real
|
||||
coverage work.
|
||||
2. **Run `vitest --coverage`** — capture a baseline per module.
|
||||
3. **Add thresholds to `vite.config.ts` `test.coverage.thresholds`**
|
||||
— start lax (50% for modules with tests, ignore untested), tighten
|
||||
over time.
|
||||
4. **Make `pnpm run test:coverage` part of `validate:all`** — only
|
||||
after (1) is done.
|
||||
|
||||
For now: `audit:test-coverage` is the contract.
|
||||
|
|
@ -34,6 +34,8 @@
|
|||
"audit:map": "node scripts/build-complexity-map.mjs",
|
||||
"audit:i18n-coverage": "node scripts/audit-i18n-coverage.mjs",
|
||||
"audit:port-drift": "node scripts/audit-port-drift.mjs",
|
||||
"audit:test-coverage": "node scripts/audit-test-coverage.mjs",
|
||||
"audit:all": "pnpm run audit:port-drift && pnpm run audit:i18n-coverage --summary && pnpm run audit:test-coverage --summary",
|
||||
"generate:dockerfiles": "node scripts/generate-dockerfiles.mjs",
|
||||
"setup:env": "node scripts/generate-env.mjs",
|
||||
"setup:secrets": "node scripts/setup-secrets.mjs",
|
||||
|
|
|
|||
131
scripts/audit-test-coverage.mjs
Normal file
131
scripts/audit-test-coverage.mjs
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
#!/usr/bin/env node
|
||||
/**
|
||||
* Audit test-file presence across modules. Which modules have zero
|
||||
* test files? Which have comprehensive coverage?
|
||||
*
|
||||
* This is a file-level heuristic, not a line-coverage metric — running
|
||||
* `vitest --coverage` is the real thing, but:
|
||||
* 1. 34/653 tests currently fail (2026-04-22; related to in-flight
|
||||
* spaces-foundation work), so coverage thresholds aren't
|
||||
* enforceable yet.
|
||||
* 2. File-presence is a faster, more stable signal for "which module
|
||||
* has no automated regression protection at all?" — enough to
|
||||
* prioritise the next session's test-writing effort.
|
||||
*
|
||||
* Scope: `apps/mana/apps/web/src/lib/modules/*` subdirectories. For each
|
||||
* module, count:
|
||||
* - .svelte files (UI surface)
|
||||
* - .ts files (types, queries, stores, etc.)
|
||||
* - .test.ts / .spec.ts files (tests)
|
||||
* - Percentage of source files that have a sibling test
|
||||
*
|
||||
* Also include top-level packages/shared-* for completeness.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/audit-test-coverage.mjs
|
||||
* node scripts/audit-test-coverage.mjs --summary
|
||||
*/
|
||||
|
||||
import { readdirSync } from 'node:fs';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, join } from 'node:path';
|
||||
|
||||
const __dirname = dirname(fileURLToPath(import.meta.url));
|
||||
const REPO_ROOT = join(__dirname, '..');
|
||||
const MODULES_DIR = join(REPO_ROOT, 'apps/mana/apps/web/src/lib/modules');
|
||||
|
||||
const SUMMARY = process.argv.includes('--summary');
|
||||
|
||||
const SKIP_DIRS = new Set(['node_modules', 'dist', 'build', '.svelte-kit']);
|
||||
|
||||
function walk(dir, collect) {
|
||||
for (const ent of readdirSync(dir, { withFileTypes: true })) {
|
||||
if (SKIP_DIRS.has(ent.name)) continue;
|
||||
const p = join(dir, ent.name);
|
||||
if (ent.isDirectory()) walk(p, collect);
|
||||
else if (ent.isFile()) collect(p);
|
||||
}
|
||||
}
|
||||
|
||||
function classify(files) {
|
||||
let svelte = 0;
|
||||
let ts = 0;
|
||||
let tests = 0;
|
||||
for (const f of files) {
|
||||
if (f.endsWith('.test.ts') || f.endsWith('.spec.ts') || f.endsWith('.test.svelte')) {
|
||||
tests++;
|
||||
} else if (f.endsWith('.svelte')) {
|
||||
svelte++;
|
||||
} else if (f.endsWith('.ts') && !f.endsWith('.d.ts')) {
|
||||
ts++;
|
||||
}
|
||||
}
|
||||
return { svelte, ts, tests };
|
||||
}
|
||||
|
||||
function audit() {
|
||||
const moduleNames = readdirSync(MODULES_DIR, { withFileTypes: true })
|
||||
.filter((e) => e.isDirectory())
|
||||
.map((e) => e.name)
|
||||
.sort();
|
||||
|
||||
const rows = [];
|
||||
for (const mod of moduleNames) {
|
||||
const files = [];
|
||||
try {
|
||||
walk(join(MODULES_DIR, mod), (p) => files.push(p));
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const c = classify(files);
|
||||
const source = c.svelte + c.ts;
|
||||
const coverage = source === 0 ? 0 : c.tests / source;
|
||||
rows.push({ module: mod, ...c, source, coverage });
|
||||
}
|
||||
|
||||
const withTests = rows.filter((r) => r.tests > 0).length;
|
||||
const withoutTests = rows.filter((r) => r.tests === 0).length;
|
||||
const totalTests = rows.reduce((s, r) => s + r.tests, 0);
|
||||
const totalSource = rows.reduce((s, r) => s + r.source, 0);
|
||||
|
||||
console.log(`\n── Test coverage audit (file-presence) ─────────────────\n`);
|
||||
console.log(`Modules: ${rows.length}`);
|
||||
console.log(` with ≥1 test: ${withTests}`);
|
||||
console.log(` without any test: ${withoutTests}`);
|
||||
console.log(`Total source files: ${totalSource} (.svelte + .ts)`);
|
||||
console.log(`Total test files: ${totalTests}`);
|
||||
console.log(
|
||||
`Overall file ratio: ${((totalTests / totalSource) * 100).toFixed(1)}% (target: ≥25% for hot modules)`
|
||||
);
|
||||
console.log('');
|
||||
|
||||
const zeroTests = rows
|
||||
.filter((r) => r.tests === 0 && r.source >= 3)
|
||||
.sort((a, b) => b.source - a.source);
|
||||
if (zeroTests.length > 0) {
|
||||
console.log(`Modules with 0 test files and ≥3 source files (top 15):\n`);
|
||||
for (const r of zeroTests.slice(0, 15)) {
|
||||
console.log(` ${String(r.source).padStart(3)} src ${r.module}`);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
if (SUMMARY) return;
|
||||
|
||||
const withSome = rows.filter((r) => r.tests > 0).sort((a, b) => b.coverage - a.coverage);
|
||||
if (withSome.length > 0) {
|
||||
console.log(`Modules with ≥1 test (by file ratio):\n`);
|
||||
for (const r of withSome) {
|
||||
const bar = '█'.repeat(Math.round(r.coverage * 20)).padEnd(20, '·');
|
||||
console.log(
|
||||
` ${bar} ${(r.coverage * 100).toFixed(0).padStart(3)}% ` +
|
||||
`${r.tests}/${r.source} files ${r.module}`
|
||||
);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Report-only; never exit non-zero.
|
||||
}
|
||||
|
||||
audit();
|
||||
Loading…
Add table
Add a link
Reference in a new issue