mirror of
https://github.com/Memo-2023/mana-monorepo.git
synced 2026-05-14 23:01:09 +02:00
feat(geocoding): support dual-Photon (self-hosted + public) for GPU
migration The chain now distinguishes two Photon instances: photon-self privacy: 'local' (self-hosted on mana-gpu) photon privacy: 'public' (komoot.io, last-resort fallback) Both wrap the same `PhotonProvider` class with different config — only the URL, name, and privacy stance differ. The new ProviderName variant 'photon-self' lets the chain track per-provider health for them independently (a single 'photon' slot would collide in the health Map). Opt-in registration: `photon-self` is only built when PHOTON_SELF_API_URL is set in the env. When unset (current state), the chain has the same shape as before — full backward compat. After the GPU migration, flipping the env-var on is the only deploy step needed: PHOTON_SELF_API_URL=http://192.168.178.11:2322 Default chain order updated to: photon-self,pelias,photon,nominatim ^^^^^^^^^^^ silently skipped if not registered (env unset) The privacy guarantee is structural: photon-self carries privacy: 'local', so the existing sensitive-query block from the previous hardening commit now has a real local backend post-migration — medical/crisis-service queries get real results instead of the "sensitive_local_unavailable" notice. Tests: 148 (was 141). New coverage: - src/__tests__/app.test.ts: createChain registration logic — verifies photon-self appears iff PHOTON_SELF_API_URL is set, ordering honored, GEOCODING_PROVIDERS env-var filter respected - providers/__tests__/photon-normalizer.test.ts: provider field carries 'photon' or 'photon-self' based on the call argument Recon of mana-gpu (2026-04-28): Windows 11 Pro Build 26200, 64 GB RAM (56 GB free), 739 GB disk free, no WSL2/Docker yet, no native GPU services running. Setup plan documented in docs/runbooks/photon-on-mana-gpu.md (3–4 h, ~1 h of which is download/unpack waiting).
This commit is contained in:
Till JS
parent
104a5a46a0
commit
153ad8049c
8 changed files with 537 additions and 16 deletions
300
docs/runbooks/photon-on-mana-gpu.md
Normal file
300
docs/runbooks/photon-on-mana-gpu.md
Normal file
|
|
@ -0,0 +1,300 @@
|
||||||
|
# Runbook — Self-host Photon on mana-gpu
|
||||||
|
|
||||||
|
**Goal:** install WSL2 + Docker Desktop on the Windows GPU server, run a self-hosted Photon (Europe-wide), and cut the geocoding wrapper over to it as the primary `privacy: 'local'` provider.
|
||||||
|
|
||||||
|
**Estimated time:** 3–4 h, ~1 h of which is unattended download/unpack.
|
||||||
|
|
||||||
|
**Prerequisites:**
|
||||||
|
- Physical access to mana-gpu (Hyper-V install requires reboot)
|
||||||
|
- Admin account
|
||||||
|
- 100 GB free on a fast SSD (current state: **739 GB free on C:**, ✓)
|
||||||
|
- Decent download speed (Europe tarball is ~30 GB)
|
||||||
|
|
||||||
|
**Pre-checked state (2026-04-28):**
|
||||||
|
- Windows 11 Pro Build 26200 — fully WSL2-ready, no upgrade needed
|
||||||
|
- 64 GB RAM, 56 GB free
|
||||||
|
- WSL2: not installed
|
||||||
|
- Docker Desktop: not installed
|
||||||
|
- No native CUDA / Ollama / STT / TTS Windows services detected (so Hyper-V activation should not disrupt anything currently running)
|
||||||
|
- LAN-only — Cloudflare tunnel terminates on the Mac mini, the GPU server is only reachable from the home LAN
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1 — WSL2 + Docker Desktop (~1.5 h, requires physical access)
|
||||||
|
|
||||||
|
### 1.1 Enable Windows features (5 min, requires reboot)
|
||||||
|
|
||||||
|
Open **PowerShell as Administrator** and run:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
|
||||||
|
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
|
||||||
|
```
|
||||||
|
|
||||||
|
Then **reboot**:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
Restart-Computer
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 Install WSL2 + Ubuntu (10 min)
|
||||||
|
|
||||||
|
After reboot, in elevated PowerShell:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
wsl --install --no-distribution
|
||||||
|
wsl --set-default-version 2
|
||||||
|
wsl --install -d Ubuntu-24.04
|
||||||
|
```
|
||||||
|
|
||||||
|
When the Ubuntu setup prompt appears, create a user (e.g. `mana`). Verify:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
wsl --list --verbose
|
||||||
|
# Should show Ubuntu-24.04, VERSION 2, STATE Running
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.3 Install Docker Desktop (15 min)
|
||||||
|
|
||||||
|
Download installer:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
$dl = "$env:TEMP\DockerDesktopInstaller.exe"
|
||||||
|
Invoke-WebRequest -Uri "https://desktop.docker.com/win/main/amd64/Docker%20Desktop%20Installer.exe" -OutFile $dl
|
||||||
|
Start-Process -FilePath $dl -ArgumentList "install","--quiet","--accept-license","--backend=wsl-2" -Wait
|
||||||
|
```
|
||||||
|
|
||||||
|
Then **log out and back in** (or reboot). Launch Docker Desktop from the Start Menu, accept the license, **uncheck** "Send usage statistics" if you care.
|
||||||
|
|
||||||
|
In Settings → Resources → WSL Integration: ensure Ubuntu-24.04 is enabled.
|
||||||
|
|
||||||
|
Verify from PowerShell:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker --version
|
||||||
|
docker run --rm hello-world
|
||||||
|
```
|
||||||
|
|
||||||
|
If `docker run` works, Phase 1 is done.
|
||||||
|
|
||||||
|
### 1.4 GPU services sanity check (5 min)
|
||||||
|
|
||||||
|
If you have any natively-running ML services (Ollama, image-gen, …), verify they still work after the Hyper-V activation. If something broke, the most common fix is to make sure Docker Desktop is running with **WSL2 backend** (not Hyper-V backend) — they coexist with CUDA pass-through more smoothly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2 — Photon container (~45 min, ~30 min of which is download)
|
||||||
|
|
||||||
|
### 2.1 Create data directory + download index (30 min)
|
||||||
|
|
||||||
|
In **PowerShell** (does not need to be admin):
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
mkdir D:\photon-data -Force
|
||||||
|
cd D:\photon-data
|
||||||
|
|
||||||
|
# ~30 GB compressed; takes 5–30 min depending on your link
|
||||||
|
Invoke-WebRequest `
|
||||||
|
-Uri "https://download1.graphhopper.com/public/europe/photon-db-europe-1.0-latest.tar.bz2" `
|
||||||
|
-OutFile "photon-db-europe-1.0-latest.tar.bz2"
|
||||||
|
```
|
||||||
|
|
||||||
|
If you'd rather start with Germany-only (5.8 GB, faster), use:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
Invoke-WebRequest `
|
||||||
|
-Uri "https://download1.graphhopper.com/public/europe/germany/photon-db-germany-1.0-latest.tar.bz2" `
|
||||||
|
-OutFile "photon-db-germany-1.0-latest.tar.bz2"
|
||||||
|
```
|
||||||
|
|
||||||
|
Unpack:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Unpacks into D:\photon-data\photon_data\
|
||||||
|
tar -xjf photon-db-europe-1.0-latest.tar.bz2
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify the structure — you should see `D:\photon-data\photon_data\` with subdirectories.
|
||||||
|
|
||||||
|
### 2.2 Run Photon container (5 min)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker run -d `
|
||||||
|
--name photon `
|
||||||
|
--restart unless-stopped `
|
||||||
|
-p 2322:2322 `
|
||||||
|
-v D:\photon-data\photon_data:/photon/photon_data `
|
||||||
|
-e JAVA_OPTS="-Xmx6g -Xms2g" `
|
||||||
|
komoot/photon
|
||||||
|
```
|
||||||
|
|
||||||
|
**Heap notes:**
|
||||||
|
- `-Xmx6g` (6 GB max heap) is comfortable for Europe; bump to 8g if you see GC churn
|
||||||
|
- `-Xms2g` (2 GB initial heap) avoids cold-start expansion latency
|
||||||
|
|
||||||
|
Check it boots cleanly:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker logs -f photon
|
||||||
|
# Wait until you see lines like: "started Photon" / "OpenSearch ready"
|
||||||
|
# Press Ctrl+C to stop following
|
||||||
|
```
|
||||||
|
|
||||||
|
Smoke from the GPU server itself:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
curl "http://localhost:2322/api?q=Konstanz&limit=2"
|
||||||
|
# Expect a GeoJSON FeatureCollection with at least one feature
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 Open the firewall (2 min)
|
||||||
|
|
||||||
|
The Mac mini needs to reach Photon on port 2322 from the LAN:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
New-NetFirewallRule `
|
||||||
|
-DisplayName "Photon (mana-geocoding)" `
|
||||||
|
-Direction Inbound `
|
||||||
|
-Protocol TCP `
|
||||||
|
-LocalPort 2322 `
|
||||||
|
-Action Allow `
|
||||||
|
-Profile Private `
|
||||||
|
-RemoteAddress 192.168.178.0/24
|
||||||
|
```
|
||||||
|
|
||||||
|
(`-Profile Private` because the network is the home LAN, not Public. Adjust if your network categorisation is different — `Get-NetConnectionProfile` shows it.)
|
||||||
|
|
||||||
|
Verify from the **Mac mini**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh mana-server 'curl -fsS http://192.168.178.11:2322/api?q=Konstanz | head -c 200'
|
||||||
|
```
|
||||||
|
|
||||||
|
Should return a Photon GeoJSON response.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3 — Wrapper cutover (already prepared in code, ~15 min to deploy)
|
||||||
|
|
||||||
|
The wrapper code already supports a `photon-self` provider — we just need to set the env-var and recreate the container.
|
||||||
|
|
||||||
|
### 3.1 Add to `.env.macmini` on mana-server
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh mana-server
|
||||||
|
nano ~/projects/mana-monorepo/.env.macmini
|
||||||
|
```
|
||||||
|
|
||||||
|
Add at the bottom:
|
||||||
|
|
||||||
|
```env
|
||||||
|
PHOTON_SELF_API_URL=http://192.168.178.11:2322
|
||||||
|
```
|
||||||
|
|
||||||
|
Optionally tighten the chain to drop public Nominatim (saving one redundant fallback):
|
||||||
|
|
||||||
|
```env
|
||||||
|
GEOCODING_PROVIDERS=photon-self,photon
|
||||||
|
```
|
||||||
|
|
||||||
|
(Default order without this env-var is `photon-self,pelias,photon,nominatim`. `pelias` is silently skipped because the Pelias container is stopped. Setting `GEOCODING_PROVIDERS=photon-self,photon` is more explicit but not required.)
|
||||||
|
|
||||||
|
### 3.2 Recreate mana-geocoding container
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/projects/mana-monorepo
|
||||||
|
docker compose -f docker-compose.macmini.yml --env-file .env.macmini up -d mana-geocoding
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.3 Verify
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Should now list photon-self as healthy
|
||||||
|
docker exec mana-geocoding bun -e 'fetch("http://127.0.0.1:3018/health/providers").then(r=>r.text()).then(console.log)'
|
||||||
|
|
||||||
|
# Sensitive query should now return REAL results (was empty before cutover)
|
||||||
|
docker exec mana-geocoding bun -e 'fetch("http://127.0.0.1:3018/api/v1/geocode/search?q=Hausarzt+Konstanz&limit=2").then(r=>r.text()).then(t=>console.log(t.substring(0,400)))'
|
||||||
|
|
||||||
|
# Normal query should be served by photon-self (provider:"photon-self", no notice field)
|
||||||
|
docker exec mana-geocoding bun -e 'fetch("http://127.0.0.1:3018/api/v1/geocode/search?q=Konstanz&limit=1").then(r=>r.text()).then(t=>console.log(t.substring(0,400)))'
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for:
|
||||||
|
- `"healthy":true` for `photon-self`
|
||||||
|
- `"provider":"photon-self"` on the result objects
|
||||||
|
- No `"notice":"fallback_used"` (the only notice should appear if photon-self is somehow down)
|
||||||
|
|
||||||
|
### 3.4 Tail the logs for 5 minutes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f --since 1m mana-geocoding
|
||||||
|
```
|
||||||
|
|
||||||
|
Watch for:
|
||||||
|
- `[geocoding-chain] photon-self failed` warnings (means cross-LAN issue — debug from there)
|
||||||
|
- Steady stream of normal `200`s from real user traffic (no warnings)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4 — Cleanup (10 min)
|
||||||
|
|
||||||
|
### 4.1 Remove the Pelias stack from the Mac mini
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/projects/mana-monorepo/services/mana-geocoding/pelias
|
||||||
|
docker compose down -v
|
||||||
|
# `-v` deletes the volumes too — frees ~5 GB disk
|
||||||
|
```
|
||||||
|
|
||||||
|
If you want to keep Pelias around as a "break glass" option, omit the `-v` and just leave the stack stopped (which is the current state).
|
||||||
|
|
||||||
|
### 4.2 Update CLAUDE.md
|
||||||
|
|
||||||
|
`services/mana-geocoding/CLAUDE.md` should reflect the new topology:
|
||||||
|
- Self-hosted Photon on `mana-gpu` is the primary, `privacy: 'local'`
|
||||||
|
- Public Photon stays as last-resort `privacy: 'public'` fallback
|
||||||
|
- Pelias is retired
|
||||||
|
|
||||||
|
### 4.3 Update memory if you keep one
|
||||||
|
|
||||||
|
The `memory/project_*.md` notes should reflect: Pelias retired 2026-04-28, Photon migrated to mana-gpu on $DATE.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5 — Weekly maintenance (10 min/week, optional automation)
|
||||||
|
|
||||||
|
GraphHopper publishes a fresh tarball every week. Manual refresh:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd D:\photon-data
|
||||||
|
docker stop photon
|
||||||
|
Invoke-WebRequest -Uri "https://download1.graphhopper.com/public/europe/photon-db-europe-1.0-latest.tar.bz2" -OutFile "photon-db-new.tar.bz2"
|
||||||
|
mv photon_data photon_data.old
|
||||||
|
tar -xjf photon-db-new.tar.bz2
|
||||||
|
docker start photon
|
||||||
|
# After a smoke-test:
|
||||||
|
rm -r photon_data.old
|
||||||
|
rm photon-db-new.tar.bz2
|
||||||
|
```
|
||||||
|
|
||||||
|
For full automation, schedule the above as a Windows Task Scheduler weekly job. Not urgent — the OSM data being a week stale is fine for our use case.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
If something goes wrong at any step, the rollback is unceremonious:
|
||||||
|
|
||||||
|
1. Remove the env-var: `unset PHOTON_SELF_API_URL` in `.env.macmini`
|
||||||
|
2. Recreate the container: `docker compose ... up -d mana-geocoding`
|
||||||
|
3. The chain falls back to public Photon + Nominatim — **same state as right now**
|
||||||
|
|
||||||
|
The wrapper code is fully backward-compatible: `photon-self` registration is opt-in via the env-var alone, no schema migrations, no data dependencies.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open questions for you
|
||||||
|
|
||||||
|
1. **Europe vs Germany only:** Europe is 30 GB compressed → 80 GB unpacked, 6–8 GB heap. Germany is 5.8 GB compressed → 15 GB unpacked, 2–4 GB heap. We recommended Europe; both work. Pick based on whether you ever want to geocode Spanish or Italian addresses.
|
||||||
|
2. **Cleanup of public providers from chain:** do you want public Photon + Nominatim to stay as last-resort fallbacks (current behavior), or drop them entirely once self-hosted is up? Keeping them costs nothing if self-hosted is healthy.
|
||||||
|
3. **Nightly Photon restart:** OpenSearch occasionally fragments its memory. A weekly restart (or a `docker restart photon` after the data refresh) keeps things tidy. Not urgent.
|
||||||
|
|
@ -153,14 +153,22 @@ docker compose up -d --force-recreate api
|
||||||
PORT=3018
|
PORT=3018
|
||||||
|
|
||||||
# --- Provider chain (tried in order) ----------------------------------
|
# --- Provider chain (tried in order) ----------------------------------
|
||||||
GEOCODING_PROVIDERS=pelias,photon,nominatim
|
# Default order: photon-self,pelias,photon,nominatim
|
||||||
|
# `photon-self` is silently dropped if PHOTON_SELF_API_URL is unset.
|
||||||
|
GEOCODING_PROVIDERS=photon-self,pelias,photon,nominatim
|
||||||
PROVIDER_TIMEOUT_MS=8000 # per-provider request timeout (cold-start safe)
|
PROVIDER_TIMEOUT_MS=8000 # per-provider request timeout (cold-start safe)
|
||||||
PROVIDER_HEALTH_CACHE_MS=30000 # health-cache TTL — skip dead providers
|
PROVIDER_HEALTH_CACHE_MS=30000 # health-cache TTL — skip dead providers
|
||||||
|
|
||||||
# --- Pelias (primary) -------------------------------------------------
|
# --- Self-hosted Photon (privacy: 'local', primary post-migration) ----
|
||||||
|
# Set this to point at the GPU-server-hosted Photon. When unset, the
|
||||||
|
# `photon-self` slot is not registered and the chain falls back to
|
||||||
|
# public providers as before.
|
||||||
|
PHOTON_SELF_API_URL=http://192.168.178.11:2322
|
||||||
|
|
||||||
|
# --- Pelias (legacy, currently stopped — privacy: 'local') ------------
|
||||||
PELIAS_API_URL=http://pelias-api:4000/v1
|
PELIAS_API_URL=http://pelias-api:4000/v1
|
||||||
|
|
||||||
# --- Photon (fallback 1) ----------------------------------------------
|
# --- Public Photon (privacy: 'public', last-resort fallback) ----------
|
||||||
PHOTON_API_URL=https://photon.komoot.io
|
PHOTON_API_URL=https://photon.komoot.io
|
||||||
|
|
||||||
# --- Nominatim (fallback 2) -------------------------------------------
|
# --- Nominatim (fallback 2) -------------------------------------------
|
||||||
|
|
@ -176,9 +184,20 @@ CACHE_PUBLIC_TTL_MS=604800000 # 7d — extended TTL for public-API answe
|
||||||
```
|
```
|
||||||
|
|
||||||
To **disable a provider**, drop it from `GEOCODING_PROVIDERS`. To run with
|
To **disable a provider**, drop it from `GEOCODING_PROVIDERS`. To run with
|
||||||
no Pelias at all (e.g. while it's being migrated), set
|
no local backend at all, set `GEOCODING_PROVIDERS=photon,nominatim` —
|
||||||
`GEOCODING_PROVIDERS=photon,nominatim`. The chain ordering is honored
|
the wrapper will block sensitive queries (see Privacy hardening below)
|
||||||
exactly — the first listed provider is tried first.
|
since no `privacy: 'local'` provider is reachable.
|
||||||
|
|
||||||
|
The dual-Photon split:
|
||||||
|
- `photon-self` — self-hosted Photon (mana-gpu), `privacy: 'local'`, eligible
|
||||||
|
for sensitive queries. Registered iff `PHOTON_SELF_API_URL` is set.
|
||||||
|
- `photon` — public komoot.io endpoint, `privacy: 'public'`, last-resort
|
||||||
|
fallback for non-sensitive queries when self-hosted is down.
|
||||||
|
|
||||||
|
Both share the same `PhotonProvider` class — only the URL, name, and
|
||||||
|
privacy stance differ. See the [migration runbook](../../docs/runbooks/photon-on-mana-gpu.md)
|
||||||
|
and [decision report](../../docs/reports/geocoding-self-hosting-2026-04-28.md)
|
||||||
|
for the operational story.
|
||||||
|
|
||||||
## Provider-chain semantics
|
## Provider-chain semantics
|
||||||
|
|
||||||
|
|
|
||||||
97
services/mana-geocoding/src/__tests__/app.test.ts
Normal file
97
services/mana-geocoding/src/__tests__/app.test.ts
Normal file
|
|
@ -0,0 +1,97 @@
|
||||||
|
/**
|
||||||
|
* Tests for the chain wiring in `createChain()`. The behavioral assertions
|
||||||
|
* here are the migration-critical ones — make sure that:
|
||||||
|
* - `photon-self` is registered iff `PHOTON_SELF_API_URL` is set
|
||||||
|
* - `photon-self` carries `privacy: 'local'` (eligible for sensitive queries)
|
||||||
|
* - the public `photon` slot stays `privacy: 'public'`
|
||||||
|
* - chain order is honored (self before public)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { describe, expect, it } from 'bun:test';
|
||||||
|
import { createChain } from '../app';
|
||||||
|
import type { Config } from '../config';
|
||||||
|
|
||||||
|
function baseConfig(overrides: Partial<Config> = {}): Config {
|
||||||
|
return {
|
||||||
|
port: 3018,
|
||||||
|
pelias: { apiUrl: 'http://127.0.0.1:1' },
|
||||||
|
photon: { apiUrl: 'https://photon.komoot.io' },
|
||||||
|
photonSelf: { apiUrl: undefined },
|
||||||
|
nominatim: {
|
||||||
|
apiUrl: 'https://nominatim.openstreetmap.org',
|
||||||
|
userAgent: 'test',
|
||||||
|
intervalMs: 1100,
|
||||||
|
},
|
||||||
|
cors: { origins: [] },
|
||||||
|
cache: { maxEntries: 100, ttlMs: 1000, publicTtlMs: 7000 },
|
||||||
|
providers: {
|
||||||
|
enabled: ['photon-self', 'pelias', 'photon', 'nominatim'],
|
||||||
|
healthCacheMs: 30_000,
|
||||||
|
timeoutMs: 8000,
|
||||||
|
},
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('createChain — photon-self registration', () => {
|
||||||
|
it('does NOT register photon-self when PHOTON_SELF_API_URL is unset', () => {
|
||||||
|
const chain = createChain(baseConfig());
|
||||||
|
const snapshot = chain.getHealthSnapshot();
|
||||||
|
const names = snapshot.map((p) => p.name);
|
||||||
|
expect(names).not.toContain('photon-self');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('registers photon-self when PHOTON_SELF_API_URL is set', () => {
|
||||||
|
const chain = createChain(
|
||||||
|
baseConfig({
|
||||||
|
photonSelf: { apiUrl: 'http://192.168.178.11:2322' },
|
||||||
|
})
|
||||||
|
);
|
||||||
|
const snapshot = chain.getHealthSnapshot();
|
||||||
|
const names = snapshot.map((p) => p.name);
|
||||||
|
expect(names).toContain('photon-self');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('honors order: photon-self before public photon when both are enabled', () => {
|
||||||
|
const chain = createChain(
|
||||||
|
baseConfig({
|
||||||
|
photonSelf: { apiUrl: 'http://192.168.178.11:2322' },
|
||||||
|
providers: {
|
||||||
|
enabled: ['photon-self', 'photon', 'nominatim'],
|
||||||
|
healthCacheMs: 30_000,
|
||||||
|
timeoutMs: 8000,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
);
|
||||||
|
const snapshot = chain.getHealthSnapshot();
|
||||||
|
// First entry is photon-self, then photon (public), then nominatim.
|
||||||
|
const names = snapshot.map((p) => p.name);
|
||||||
|
expect(names[0]).toBe('photon-self');
|
||||||
|
expect(names).toContain('photon');
|
||||||
|
expect(names).toContain('nominatim');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('a stray empty PHOTON_SELF_API_URL does not register a useless provider', () => {
|
||||||
|
// The config loader trims and treats '' as undefined, but defend in
|
||||||
|
// depth — pass an explicit empty string here too.
|
||||||
|
const chain = createChain(baseConfig({ photonSelf: { apiUrl: undefined } }));
|
||||||
|
const names = chain.getHealthSnapshot().map((p) => p.name);
|
||||||
|
expect(names).not.toContain('photon-self');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('photon-self is filtered to enabled list (drop if not in GEOCODING_PROVIDERS)', () => {
|
||||||
|
const chain = createChain(
|
||||||
|
baseConfig({
|
||||||
|
photonSelf: { apiUrl: 'http://192.168.178.11:2322' },
|
||||||
|
providers: {
|
||||||
|
// User explicitly excludes photon-self via env-var
|
||||||
|
enabled: ['photon', 'nominatim'],
|
||||||
|
healthCacheMs: 30_000,
|
||||||
|
timeoutMs: 8000,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
);
|
||||||
|
const names = chain.getHealthSnapshot().map((p) => p.name);
|
||||||
|
expect(names).not.toContain('photon-self');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
@ -55,11 +55,29 @@ export function createChain(config: Config): ProviderChain {
|
||||||
})
|
})
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// Self-hosted Photon (mana-gpu). Only registered when the env-var is set
|
||||||
|
// — pre-migration this stays absent and the chain falls through to
|
||||||
|
// public providers as before. Once the GPU server is running Photon,
|
||||||
|
// flip PHOTON_SELF_API_URL on and this becomes the primary provider.
|
||||||
|
if (config.photonSelf.apiUrl) {
|
||||||
|
built.set(
|
||||||
|
'photon-self',
|
||||||
|
new PhotonProvider({
|
||||||
|
apiUrl: config.photonSelf.apiUrl,
|
||||||
|
timeoutMs: config.providers.timeoutMs,
|
||||||
|
name: 'photon-self',
|
||||||
|
privacy: 'local',
|
||||||
|
})
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
built.set(
|
built.set(
|
||||||
'photon',
|
'photon',
|
||||||
new PhotonProvider({
|
new PhotonProvider({
|
||||||
apiUrl: config.photon.apiUrl,
|
apiUrl: config.photon.apiUrl,
|
||||||
timeoutMs: config.providers.timeoutMs,
|
timeoutMs: config.providers.timeoutMs,
|
||||||
|
// name + privacy default to 'photon' / 'public' — public komoot
|
||||||
|
// is the always-on safety net behind self-hosted.
|
||||||
})
|
})
|
||||||
);
|
);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,9 +11,18 @@ export interface Config {
|
||||||
apiUrl: string;
|
apiUrl: string;
|
||||||
};
|
};
|
||||||
photon: {
|
photon: {
|
||||||
/** Photon base URL (defaults to public komoot endpoint) */
|
/** Photon base URL — public komoot endpoint by default. Used by
|
||||||
|
* the `'photon'` provider slot which always has `privacy: 'public'`. */
|
||||||
apiUrl: string;
|
apiUrl: string;
|
||||||
};
|
};
|
||||||
|
photonSelf: {
|
||||||
|
/** Self-hosted Photon URL (e.g. `http://192.168.178.11:2322` for the
|
||||||
|
* GPU server). When set, the wrapper registers a separate
|
||||||
|
* `'photon-self'` provider with `privacy: 'local'` — eligible for
|
||||||
|
* sensitive queries. When undefined, the slot is disabled and the
|
||||||
|
* chain only has the public providers (current pre-migration state). */
|
||||||
|
apiUrl: string | undefined;
|
||||||
|
};
|
||||||
nominatim: {
|
nominatim: {
|
||||||
apiUrl: string;
|
apiUrl: string;
|
||||||
userAgent: string;
|
userAgent: string;
|
||||||
|
|
@ -57,6 +66,13 @@ export function loadConfig(): Config {
|
||||||
photon: {
|
photon: {
|
||||||
apiUrl: process.env.PHOTON_API_URL || 'https://photon.komoot.io',
|
apiUrl: process.env.PHOTON_API_URL || 'https://photon.komoot.io',
|
||||||
},
|
},
|
||||||
|
photonSelf: {
|
||||||
|
// Opt-in: only registered when this env-var is explicitly set
|
||||||
|
// (e.g. http://192.168.178.11:2322 once the GPU server is up).
|
||||||
|
// Empty string → treated as unset so a stray "" in .env doesn't
|
||||||
|
// register a useless provider.
|
||||||
|
apiUrl: process.env.PHOTON_SELF_API_URL?.trim() || undefined,
|
||||||
|
},
|
||||||
nominatim: {
|
nominatim: {
|
||||||
apiUrl: process.env.NOMINATIM_API_URL || 'https://nominatim.openstreetmap.org',
|
apiUrl: process.env.NOMINATIM_API_URL || 'https://nominatim.openstreetmap.org',
|
||||||
userAgent:
|
userAgent:
|
||||||
|
|
@ -73,7 +89,13 @@ export function loadConfig(): Config {
|
||||||
publicTtlMs: parseInt(process.env.CACHE_PUBLIC_TTL_MS || String(7 * 24 * 60 * 60 * 1000), 10),
|
publicTtlMs: parseInt(process.env.CACHE_PUBLIC_TTL_MS || String(7 * 24 * 60 * 60 * 1000), 10),
|
||||||
},
|
},
|
||||||
providers: {
|
providers: {
|
||||||
|
// Default order (when GEOCODING_PROVIDERS is unset): try the
|
||||||
|
// self-hosted Photon first if it's been configured, then public
|
||||||
|
// providers as fallback. `photon-self` is silently dropped at
|
||||||
|
// chain-build time if `photonSelf.apiUrl` is undefined, so the
|
||||||
|
// list is the same shape regardless of migration status.
|
||||||
enabled: parseProviderList(process.env.GEOCODING_PROVIDERS, [
|
enabled: parseProviderList(process.env.GEOCODING_PROVIDERS, [
|
||||||
|
'photon-self',
|
||||||
'pelias',
|
'pelias',
|
||||||
'photon',
|
'photon',
|
||||||
'nominatim',
|
'nominatim',
|
||||||
|
|
@ -90,7 +112,7 @@ export function loadConfig(): Config {
|
||||||
|
|
||||||
function parseProviderList(raw: string | undefined, fallback: ProviderName[]): ProviderName[] {
|
function parseProviderList(raw: string | undefined, fallback: ProviderName[]): ProviderName[] {
|
||||||
if (!raw) return fallback;
|
if (!raw) return fallback;
|
||||||
const valid: ProviderName[] = ['pelias', 'photon', 'nominatim'];
|
const valid: ProviderName[] = ['pelias', 'photon-self', 'photon', 'nominatim'];
|
||||||
const parsed = raw
|
const parsed = raw
|
||||||
.split(',')
|
.split(',')
|
||||||
.map((s) => s.trim().toLowerCase())
|
.map((s) => s.trim().toLowerCase())
|
||||||
|
|
|
||||||
|
|
@ -124,4 +124,30 @@ describe('normalizePhotonFeature', () => {
|
||||||
expect(result.latitude).toBeGreaterThan(47);
|
expect(result.latitude).toBeGreaterThan(47);
|
||||||
expect(result.latitude).toBeLessThan(48);
|
expect(result.latitude).toBeLessThan(48);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('stamps provider:"photon" by default (back-compat)', () => {
|
||||||
|
const result = normalizePhotonFeature({
|
||||||
|
type: 'Feature',
|
||||||
|
geometry: { type: 'Point', coordinates: [9.17, 47.66] },
|
||||||
|
properties: { osm_key: 'place', osm_value: 'city', name: 'X' },
|
||||||
|
});
|
||||||
|
expect(result.provider).toBe('photon');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('stamps provider:"photon-self" when called with that name (self-hosted path)', () => {
|
||||||
|
// The dual-Photon migration relies on this: a result from the
|
||||||
|
// self-hosted instance must NOT look like it came from public
|
||||||
|
// komoot. UI uses the provider field to decide whether to show
|
||||||
|
// the "approximate match" badge — fallback_used notice fires only
|
||||||
|
// for `privacy: 'public'` providers.
|
||||||
|
const result = normalizePhotonFeature(
|
||||||
|
{
|
||||||
|
type: 'Feature',
|
||||||
|
geometry: { type: 'Point', coordinates: [9.17, 47.66] },
|
||||||
|
properties: { osm_key: 'place', osm_value: 'city', name: 'X' },
|
||||||
|
},
|
||||||
|
'photon-self'
|
||||||
|
);
|
||||||
|
expect(result.provider).toBe('photon-self');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
|
||||||
|
|
@ -29,13 +29,23 @@ import type {
|
||||||
export interface PhotonConfig {
|
export interface PhotonConfig {
|
||||||
apiUrl: string;
|
apiUrl: string;
|
||||||
timeoutMs: number;
|
timeoutMs: number;
|
||||||
|
/** Override the default provider name. Used when registering a second
|
||||||
|
* Photon instance pointing at a self-hosted backend (`'photon-self'`)
|
||||||
|
* alongside the public komoot endpoint (`'photon'`). */
|
||||||
|
name?: 'photon' | 'photon-self';
|
||||||
|
/** Override the default privacy stance. Self-hosted Photon on our
|
||||||
|
* infrastructure is `'local'`; public komoot is `'public'`. */
|
||||||
|
privacy?: 'local' | 'public';
|
||||||
}
|
}
|
||||||
|
|
||||||
export class PhotonProvider implements GeocodingProvider {
|
export class PhotonProvider implements GeocodingProvider {
|
||||||
readonly name = 'photon' as const;
|
readonly name: 'photon' | 'photon-self';
|
||||||
readonly privacy = 'public' as const;
|
readonly privacy: 'local' | 'public';
|
||||||
|
|
||||||
constructor(private readonly config: PhotonConfig) {}
|
constructor(private readonly config: PhotonConfig) {
|
||||||
|
this.name = config.name ?? 'photon';
|
||||||
|
this.privacy = config.privacy ?? 'public';
|
||||||
|
}
|
||||||
|
|
||||||
async search(req: SearchRequest, signal?: AbortSignal): Promise<ProviderResponse> {
|
async search(req: SearchRequest, signal?: AbortSignal): Promise<ProviderResponse> {
|
||||||
const params = new URLSearchParams({
|
const params = new URLSearchParams({
|
||||||
|
|
@ -64,7 +74,10 @@ export class PhotonProvider implements GeocodingProvider {
|
||||||
status: res.status,
|
status: res.status,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
return { ok: true, results: res.features.map(normalizePhotonFeature) };
|
return {
|
||||||
|
ok: true,
|
||||||
|
results: res.features.map((f) => normalizePhotonFeature(f, this.name)),
|
||||||
|
};
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
return { ok: false, kind: 'unreachable', error: errorMessage(e) };
|
return { ok: false, kind: 'unreachable', error: errorMessage(e) };
|
||||||
}
|
}
|
||||||
|
|
@ -93,7 +106,10 @@ export class PhotonProvider implements GeocodingProvider {
|
||||||
status: res.status,
|
status: res.status,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
return { ok: true, results: res.features.map(normalizePhotonFeature) };
|
return {
|
||||||
|
ok: true,
|
||||||
|
results: res.features.map((f) => normalizePhotonFeature(f, this.name)),
|
||||||
|
};
|
||||||
} catch (e) {
|
} catch (e) {
|
||||||
return { ok: false, kind: 'unreachable', error: errorMessage(e) };
|
return { ok: false, kind: 'unreachable', error: errorMessage(e) };
|
||||||
}
|
}
|
||||||
|
|
@ -161,7 +177,16 @@ interface PhotonFeature {
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export function normalizePhotonFeature(f: PhotonFeature): GeocodingResult {
|
/**
|
||||||
|
* @param providerName Which provider tag to stamp on the result. Defaults
|
||||||
|
* to `'photon'` (public komoot) for backward compat. Pass `'photon-self'`
|
||||||
|
* to mark results as coming from our self-hosted instance — useful for
|
||||||
|
* the UI to know "this came from local infra, no privacy compromise".
|
||||||
|
*/
|
||||||
|
export function normalizePhotonFeature(
|
||||||
|
f: PhotonFeature,
|
||||||
|
providerName: 'photon' | 'photon-self' = 'photon'
|
||||||
|
): GeocodingResult {
|
||||||
const props = f.properties;
|
const props = f.properties;
|
||||||
const [lon, lat] = f.geometry.coordinates;
|
const [lon, lat] = f.geometry.coordinates;
|
||||||
|
|
||||||
|
|
@ -186,7 +211,7 @@ export function normalizePhotonFeature(f: PhotonFeature): GeocodingResult {
|
||||||
// but the consumer side keys off the absence of this field as a
|
// but the consumer side keys off the absence of this field as a
|
||||||
// "result came from a fallback" signal.
|
// "result came from a fallback" signal.
|
||||||
confidence: typeof props.importance === 'number' ? props.importance : 0.5,
|
confidence: typeof props.importance === 'number' ? props.importance : 0.5,
|
||||||
provider: 'photon',
|
provider: providerName,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -41,7 +41,21 @@ export interface GeocodingResult {
|
||||||
provider: ProviderName;
|
provider: ProviderName;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type ProviderName = 'pelias' | 'photon' | 'nominatim';
|
/**
|
||||||
|
* Provider identifiers. Two of these wrap the same `PhotonProvider`
|
||||||
|
* class with different configs:
|
||||||
|
*
|
||||||
|
* - `photon-self`: self-hosted Photon (typically on mana-gpu),
|
||||||
|
* `privacy: 'local'`. Eligible for sensitive queries.
|
||||||
|
* - `photon`: public photon.komoot.io, `privacy: 'public'`. Last-resort
|
||||||
|
* fallback for non-sensitive queries when the self-hosted instance
|
||||||
|
* is down.
|
||||||
|
*
|
||||||
|
* The split exists because the chain identifies providers by name and
|
||||||
|
* tracks per-provider health. A single `photon` slot can't simultaneously
|
||||||
|
* mean two different backends.
|
||||||
|
*/
|
||||||
|
export type ProviderName = 'pelias' | 'photon-self' | 'photon' | 'nominatim';
|
||||||
|
|
||||||
export interface SearchRequest {
|
export interface SearchRequest {
|
||||||
q: string;
|
q: string;
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue