Commit graph

5 commits

Author SHA1 Message Date
Till JS
f1e4a39644 feat(geocoding): provider chain with Photon + Nominatim fallbacks
mana-geocoding now tries Pelias first, falls back to public Photon
(komoot.io) and finally to public Nominatim (OSM) when Pelias is
unhealthy or unreachable. The Places module's address lookup keeps
working even when the Pelias container is stopped — which it currently
is on the Mac mini, freeing 3 GB of RAM until Pelias gets migrated to
the GPU server.

Architecture:

  ProviderChain ─ tries providers in priority order, stops on first
                  success. A clean empty-results answer is definitive
                  (don't burn through public-API budget on a query that
                  legitimately has no match). Only network errors / 5xx
                  / 429 trigger fallthrough.

  HealthCache  ─ per-provider, 30s TTL. A failed health probe or a
                  failed search marks the provider unhealthy and skips
                  it for the rest of the cache window. Lazy refresh —
                  no background pinger.

  RateLimiter  ─ single-token FIFO queue, 1100ms gap by default.
                  Used to enforce Nominatim's 1 req/sec policy. Handles
                  abort during inter-task wait by releasing the busy
                  flag so later tasks aren't blocked.

Provider details:

  pelias    — primary, self-hosted DACH index, full OSM taxonomy in
              `peliasCategories`, no rate limit
  photon    — public komoot endpoint, GeoJSON shape, raw `osm_key:
              osm_value` mapped via lib/osm-category-map.ts. Faster
              than Nominatim, no advertised rate limit but be polite.
  nominatim — public OSM endpoint, strict 1 req/sec via the limiter,
              custom User-Agent required (otherwise 403). Last
              resort — fallback for when Photon is also down.

Response shape changes (additive only — existing callers keep
working):

  - results[].provider: 'pelias' | 'photon' | 'nominatim'
  - results[].peliasCategories: only present when Pelias served the
    request (was already absent on Pelias-API patch failures)
  - top-level provider: <name> + tried: <name[]> on success/error
  - new endpoint: GET /health/providers — per-provider snapshot

Configuration via env (defaults shipped):

  GEOCODING_PROVIDERS=pelias,photon,nominatim   # order matters
  PROVIDER_TIMEOUT_MS=5000
  PROVIDER_HEALTH_CACHE_MS=30000
  PHOTON_API_URL=https://photon.komoot.io
  NOMINATIM_API_URL=https://nominatim.openstreetmap.org
  NOMINATIM_USER_AGENT=mana-geocoding/1.0 (+https://mana.how; ...)
  NOMINATIM_INTERVAL_MS=1100

Testing: 115 tests green (was 42). New coverage:
  - osm-category-map.test.ts (47 cases over food/transit/shopping/
    leisure/work/other priority resolution)
  - rate-limiter.test.ts (FIFO, abort-during-wait, abort-during-sleep)
  - chain.test.ts (failover, empty-results-stops, health-cache,
    snapshot)
  - photon-normalizer.test.ts and nominatim-normalizer.test.ts (lock
    the wire-format mapping for both fallback providers)

Live smoke against public Photon verified — both /search and /reverse
return correctly normalized results with provider="photon" when Pelias
is unreachable.
2026-04-28 15:21:11 +02:00
Till JS
286e273b18 test(geocoding): add unit tests + end-to-end smoke test script
**Unit tests (`bun test`, 42 checks, 0 deps)**

- `src/lib/__tests__/category-map.test.ts` locks in the Pelias→
  PlaceCategory priority resolution. Covers the ambiguous multi-category
  case (food beats retail for restaurants, transit beats professional
  for car rentals, transport:rail still maps to transit, …), the simple
  single-category paths, the layer-hint fallback, and regression cases
  from real Konstanz/Stuttgart/Köln venues observed during deploy
  verification.
- `src/lib/__tests__/cache.test.ts` covers LRU eviction order, TTL
  expiry, move-to-end on get (so frequently-read entries survive
  eviction), size tracking, and typed-value storage.

**Smoke test (`./scripts/smoke-test.sh` or `bun run test:smoke`)**

End-to-end curls against a running service, aimed at post-deploy
verification. Health endpoints, forward (venue + street fallback),
focus biasing, reverse geocoding, cache hit. 9 checks total.

Wired up as `test:smoke` in package.json so it runs alongside the
unit tests. Verified working: 42/42 unit tests green locally, 9/9
smoke checks green against the live Mac Mini deployment.

CLAUDE.md Testing section rewritten to reflect the new test layers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 20:21:18 +02:00
Till JS
32d9f25e7f docs(geocoding): update CLAUDE.md with deploy lessons learned
After the 2026-04-11 production deploy, several non-obvious gotchas
surfaced that needed documenting:

- Forward search: autocomplete→search fallback explained, so future-me
  knows why the handler hits two Pelias endpoints for address-style
  queries.
- Pelias infra: corrected object counts (13.4M actual, not 22M), noted
  the libpostal RAM surprise (~1.9 GB, much larger than Pelias docs
  suggest), and added real per-container RAM numbers from production.
- pelias.json: document that we dropped placeholder/pip/interpolation
  (not just how to run them) and why the cleaner degradation matters.
- Wrapper gotchas section: Bun idleTimeout, Colima bind-mount cache
  staleness, and the host.docker.internal-from-blackbox workaround.
- /health/pelias endpoint is now listed in the API table since it's
  the integration point with blackbox monitoring.
- Testing section added — explicitly "no automated tests yet", with a
  curl-based manual smoke test set a human can run after changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 17:59:34 +02:00
Till JS
f7de9fdf2d docs(geocoding): document the Pelias category patch + import gotchas
Expand services/mana-geocoding/CLAUDE.md with:
- The Pelias API patch (geojsonify_place_details.js) that forces the
  category field to always be returned, with regeneration instructions
- The priority-ordered Pelias→PlaceCategory mapping and verified
  example mappings from the DACH index
- A full initial-import walkthrough covering the non-obvious gotchas
  (analysis-icu plugin, dach-latest → planet-latest rename, adminLookup
  disabled, leveldbpath, libpostal config object form, boundary.country
  single-value constraint)

Also register mana-geocoding in the root services list.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 15:50:40 +02:00
Till JS
a47a7bfdba feat(places): add self-hosted geocoding with Pelias (DACH)
New mana-geocoding service (port 3018) wraps a self-hosted Pelias
instance with LRU caching and OSM→PlaceCategory auto-mapping.
All geocoding queries stay within our infrastructure — no user
location data leaves the network.

Places module integration:
- Address autocomplete search in ListView (creates place with
  name, coords, address, category in one step)
- Address search + reverse geocoding button in DetailView
- Auto-fill address via reverse geocoding during tracking
- OSM category mapping (amenity:restaurant→food, shop:*→shopping, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 23:02:25 +02:00