till/managarten
1
0
Fork 0
mirror of https://github.com/Memo-2023/mana-monorepo.git synced 2026-05-14 23:01:09 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
managarten/services/mana-crawler
History
Till-JS 4a3295d1d0 ✨ feat(mana-crawler): add web crawler service 
NestJS-based web crawler service for structured content extraction.

Features:
- Depth-controlled crawling with URL pattern filtering
- robots.txt compliance
- HTML/PDF/Markdown content extraction
- BullMQ job queue for async processing
- Redis caching layer
- Prometheus metrics
2026-01-29 22:00:36 +01:00
..
src ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
.env.example ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
.gitignore ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
CLAUDE.md ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
Dockerfile ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
drizzle.config.ts ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
nest-cli.json ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
package.json ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00
tsconfig.json ✨ feat(mana-crawler): add web crawler service 2026-01-29 22:00:36 +01:00