till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-16 19:39:40 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
managarten/services/mana-crawler/src/db
History
Till-JS 4a3295d1d0 ✨ feat(mana-crawler): add web crawler service 
NestJS-based web crawler service for structured content extraction.

Features:
- Depth-controlled crawling with URL pattern filtering
- robots.txt compliance
- HTML/PDF/Markdown content extraction
- BullMQ job queue for async processing
- Redis caching layer
- Prometheus metrics
2026-01-29 22:00:36 +01:00
..
schema ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
connection.ts ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
database.module.ts ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00