feat(questions): implement questions app NestJS backend

Complete backend implementation for the AI-powered research assistant app:

Database Schema (Drizzle ORM):
- collections: Organize questions into folders with colors and icons
- questions: User questions with status, priority, tags, and research depth
- research_results: Results from mana-search service with summaries and key points
- sources: Extracted content from web search results
- answers: AI-generated answers with ratings and citations

NestJS Modules:
- QuestionModule: CRUD operations with filtering, pagination, and status management
- CollectionModule: Collection management with reordering and question counts
- ResearchModule: Integration with mana-search microservice for web search
- AnswerModule: Answer management with ratings and acceptance tracking
- SourceModule: Source content retrieval and management
- HealthModule: Health checks for database and search service

Features:
- Full JWT authentication via @manacore/shared-nestjs-auth
- Research depths: quick (5 sources), standard (15), deep (30)
- Automatic content extraction and summarization
- Follow-up question generation

Also updated:
- Root package.json: Added questions:* development scripts
- setup-databases.sh: Added questions database setup

https://claude.ai/code/session_01Rk3YVJCU3nM8uvVPghRz6r
This commit is contained in:
Claude 2026-01-28 23:52:22 +00:00
parent c0802af67f
commit ec96d4e952
No known key found for this signature in database
49 changed files with 2346 additions and 2 deletions

176
apps/questions/CLAUDE.md Normal file
View file

@ -0,0 +1,176 @@
# Questions App
AI-powered research assistant that collects user questions and performs comprehensive research using the mana-search microservice.
## Overview
- **Backend Port**: 3011
- **Technology**: NestJS + Drizzle ORM + PostgreSQL
- **Search**: mana-search microservice (SearXNG)
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Questions App │
│ Collections │ Questions │ Research │ Answers │ Sources │
└─────────────────────────┬───────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ mana-search (Port 3021) │
│ Search API │ Extract API │ Redis Cache │
└─────────────────────────┬───────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ SearXNG (Port 8080) │
│ Google │ Bing │ arXiv │ Wikipedia │ GitHub │ ... │
└─────────────────────────────────────────────────────────────┘
```
## Quick Start
```bash
# 1. Start infrastructure (PostgreSQL, Redis, mana-search dependencies)
pnpm docker:up
# 2. Start mana-search service
pnpm dev:search:full
# 3. Start questions backend
pnpm dev:questions:backend
# Or use the combined command:
pnpm dev:questions:full
```
## API Endpoints
### Collections
```bash
POST /api/v1/collections # Create collection
GET /api/v1/collections # List collections
GET /api/v1/collections/:id # Get collection
PUT /api/v1/collections/:id # Update collection
DELETE /api/v1/collections/:id # Delete collection
POST /api/v1/collections/reorder # Reorder collections
```
### Questions
```bash
POST /api/v1/questions # Create question
GET /api/v1/questions # List questions (with filters)
GET /api/v1/questions/:id # Get question
PUT /api/v1/questions/:id # Update question
DELETE /api/v1/questions/:id # Delete question
PUT /api/v1/questions/:id/status # Update status
```
### Research
```bash
POST /api/v1/research/start # Start research
GET /api/v1/research/question/:id # Get results for question
GET /api/v1/research/:id # Get research result
GET /api/v1/research/health/search # Check search service
```
### Answers
```bash
POST /api/v1/answers # Create answer
GET /api/v1/answers/question/:id # List answers for question
GET /api/v1/answers/question/:id/accepted # Get accepted answer
GET /api/v1/answers/:id # Get answer
PUT /api/v1/answers/:id # Update answer
POST /api/v1/answers/:id/rate # Rate answer
POST /api/v1/answers/:id/accept # Accept answer
DELETE /api/v1/answers/:id # Delete answer
```
### Sources
```bash
GET /api/v1/sources/research/:id # Sources by research result
GET /api/v1/sources/question/:id # All sources for question
GET /api/v1/sources/:id # Get source
GET /api/v1/sources/:id/content # Get source content
```
## Research Depths
| Depth | Sources | Extraction | Categories |
|-------|---------|------------|------------|
| `quick` | 5 | No | general |
| `standard` | 15 | Yes | general, news |
| `deep` | 30 | Yes | general, news, science, it |
## Database Schema
```sql
-- Collections for organizing questions
collections (id, user_id, name, description, color, icon, sort_order, ...)
-- User questions
questions (id, user_id, collection_id, title, description, status, priority, tags, ...)
-- Research results from mana-search
research_results (id, question_id, summary, key_points, follow_up_questions, ...)
-- Extracted sources from search
sources (id, research_result_id, url, title, snippet, extracted_content, ...)
-- AI-generated answers
answers (id, question_id, research_result_id, content, rating, is_accepted, ...)
```
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | 3011 | Backend port |
| `DATABASE_URL` | - | PostgreSQL connection |
| `MANA_CORE_AUTH_URL` | http://localhost:3001 | Auth service URL |
| `MANA_SEARCH_URL` | http://localhost:3021 | Search service URL |
| `MANA_SEARCH_TIMEOUT` | 30000 | Search timeout (ms) |
| `DEV_BYPASS_AUTH` | false | Skip auth in dev |
| `DEV_USER_ID` | - | User ID when auth bypassed |
## Development Commands
```bash
# Backend only
pnpm dev:questions:backend
# Type checking
cd apps/questions/apps/backend && pnpm type-check
# Database
cd apps/questions/apps/backend
pnpm drizzle-kit generate # Generate migrations
pnpm drizzle-kit push # Push schema to DB
pnpm drizzle-kit studio # Open Drizzle Studio
```
## Testing the API
```bash
# Create a collection
curl -X POST http://localhost:3011/api/v1/collections \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{"name": "Tech Research", "color": "#6366f1"}'
# Create a question
curl -X POST http://localhost:3011/api/v1/questions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{"title": "What are the best practices for TypeScript?", "researchDepth": "standard"}'
# Start research
curl -X POST http://localhost:3011/api/v1/research/start \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{"questionId": "uuid-here", "depth": "standard"}'
```

View file

@ -0,0 +1,18 @@
# Server
PORT=3011
NODE_ENV=development
# Database
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/questions
# Auth
MANA_CORE_AUTH_URL=http://localhost:3001
DEV_BYPASS_AUTH=true
DEV_USER_ID=dev-user-id
# Mana Search Service
MANA_SEARCH_URL=http://localhost:3021
MANA_SEARCH_TIMEOUT=30000
# CORS
CORS_ORIGINS=http://localhost:3000,http://localhost:5173,http://localhost:8081

View file

@ -0,0 +1,11 @@
import 'dotenv/config';
import { defineConfig } from 'drizzle-kit';
export default defineConfig({
out: './drizzle',
schema: './src/db/schema/*.ts',
dialect: 'postgresql',
dbCredentials: {
url: process.env.DATABASE_URL!,
},
});

View file

@ -0,0 +1,8 @@
{
"$schema": "https://json.schemastore.org/nest-cli",
"collection": "@nestjs/schematics",
"sourceRoot": "src",
"compilerOptions": {
"deleteOutDir": true
}
}

View file

@ -0,0 +1,53 @@
{
"name": "@questions/backend",
"version": "1.0.0",
"private": true,
"scripts": {
"build": "nest build",
"start": "nest start",
"dev": "nest start --watch",
"start:dev": "nest start --watch",
"start:debug": "nest start --debug --watch",
"start:prod": "node dist/main",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"type-check": "tsc --noEmit",
"migration:generate": "drizzle-kit generate",
"migration:run": "tsx src/db/migrate.ts",
"db:push": "drizzle-kit push",
"db:studio": "drizzle-kit studio",
"db:seed": "tsx src/db/seed.ts"
},
"dependencies": {
"@manacore/shared-nestjs-auth": "workspace:*",
"@nestjs/common": "^10.4.15",
"@nestjs/config": "^3.3.0",
"@nestjs/core": "^10.4.15",
"@nestjs/platform-express": "^10.4.15",
"class-transformer": "^0.5.1",
"class-validator": "^0.14.1",
"dotenv": "^16.4.7",
"drizzle-kit": "^0.30.2",
"drizzle-orm": "^0.38.3",
"postgres": "^3.4.5",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.1"
},
"devDependencies": {
"@nestjs/cli": "^10.4.9",
"@nestjs/schematics": "^10.2.3",
"@types/express": "^5.0.0",
"@types/node": "^22.10.2",
"@typescript-eslint/eslint-plugin": "^8.18.1",
"@typescript-eslint/parser": "^8.18.1",
"eslint": "^9.17.0",
"eslint-config-prettier": "^9.1.0",
"eslint-plugin-prettier": "^5.2.1",
"prettier": "^3.4.2",
"source-map-support": "^0.5.21",
"ts-loader": "^9.5.1",
"ts-node": "^10.9.2",
"tsconfig-paths": "^4.2.0",
"tsx": "^4.19.2",
"typescript": "^5.7.2"
}
}

View file

@ -0,0 +1,79 @@
import {
Controller,
Get,
Post,
Put,
Delete,
Body,
Param,
UseGuards,
ParseUUIDPipe,
} from '@nestjs/common';
import { AnswerService } from './answer.service';
import { CreateAnswerDto, UpdateAnswerDto, RateAnswerDto, AcceptAnswerDto } from './dto';
import { JwtAuthGuard, CurrentUser, CurrentUserData } from '@manacore/shared-nestjs-auth';
@Controller('answers')
@UseGuards(JwtAuthGuard)
export class AnswerController {
constructor(private readonly answerService: AnswerService) {}
@Post()
async create(@CurrentUser() user: CurrentUserData, @Body() dto: CreateAnswerDto) {
return this.answerService.create(user.userId, dto);
}
@Get('question/:questionId')
async findByQuestion(
@CurrentUser() user: CurrentUserData,
@Param('questionId', ParseUUIDPipe) questionId: string,
) {
return this.answerService.findByQuestion(user.userId, questionId);
}
@Get('question/:questionId/accepted')
async getAccepted(
@CurrentUser() user: CurrentUserData,
@Param('questionId', ParseUUIDPipe) questionId: string,
) {
return this.answerService.getAccepted(user.userId, questionId);
}
@Get(':id')
async findOne(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
return this.answerService.findOne(user.userId, id);
}
@Put(':id')
async update(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
@Body() dto: UpdateAnswerDto,
) {
return this.answerService.update(user.userId, id, dto);
}
@Post(':id/rate')
async rate(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
@Body() dto: RateAnswerDto,
) {
return this.answerService.rate(user.userId, id, dto);
}
@Post(':id/accept')
async setAccepted(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
@Body() dto: AcceptAnswerDto,
) {
return this.answerService.setAccepted(user.userId, id, dto);
}
@Delete(':id')
async delete(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
await this.answerService.delete(user.userId, id);
return { success: true };
}
}

View file

@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { AnswerController } from './answer.controller';
import { AnswerService } from './answer.service';
import { DatabaseModule } from '../db/database.module';
@Module({
imports: [DatabaseModule],
controllers: [AnswerController],
providers: [AnswerService],
exports: [AnswerService],
})
export class AnswerModule {}

View file

@ -0,0 +1,159 @@
import { Injectable, Inject, NotFoundException } from '@nestjs/common';
import { NodePgDatabase } from 'drizzle-orm/node-postgres';
import { eq, and, desc } from 'drizzle-orm';
import { questions, answers, Answer, NewAnswer } from '../db/schema';
import { CreateAnswerDto, UpdateAnswerDto, RateAnswerDto, AcceptAnswerDto } from './dto';
@Injectable()
export class AnswerService {
constructor(
@Inject('DATABASE_CONNECTION')
private readonly db: NodePgDatabase,
) {}
async create(userId: string, dto: CreateAnswerDto): Promise<Answer> {
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, dto.questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException(`Question with id ${dto.questionId} not found`);
}
const newAnswer: NewAnswer = {
questionId: dto.questionId,
researchResultId: dto.researchResultId,
content: dto.content,
contentMarkdown: dto.contentMarkdown,
summary: dto.summary,
modelId: dto.modelId,
provider: dto.provider,
promptTokens: dto.promptTokens,
completionTokens: dto.completionTokens,
estimatedCost: dto.estimatedCost,
confidence: dto.confidence,
sourceCount: dto.sourceCount,
citations: dto.citations || [],
durationMs: dto.durationMs,
};
const [created] = await this.db.insert(answers).values(newAnswer).returning();
return created;
}
async findByQuestion(userId: string, questionId: string): Promise<Answer[]> {
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException(`Question with id ${questionId} not found`);
}
return this.db
.select()
.from(answers)
.where(eq(answers.questionId, questionId))
.orderBy(desc(answers.createdAt));
}
async findOne(userId: string, id: string): Promise<Answer> {
const [answer] = await this.db.select().from(answers).where(eq(answers.id, id));
if (!answer) {
throw new NotFoundException(`Answer with id ${id} not found`);
}
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, answer.questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException('Answer not found');
}
return answer;
}
async update(userId: string, id: string, dto: UpdateAnswerDto): Promise<Answer> {
await this.findOne(userId, id);
const updateData: Partial<NewAnswer> = {
...dto,
updatedAt: new Date(),
};
const [updated] = await this.db.update(answers).set(updateData).where(eq(answers.id, id)).returning();
return updated;
}
async rate(userId: string, id: string, dto: RateAnswerDto): Promise<Answer> {
await this.findOne(userId, id);
const [updated] = await this.db
.update(answers)
.set({
rating: dto.rating,
feedback: dto.feedback,
updatedAt: new Date(),
})
.where(eq(answers.id, id))
.returning();
return updated;
}
async setAccepted(userId: string, id: string, dto: AcceptAnswerDto): Promise<Answer> {
const answer = await this.findOne(userId, id);
// If accepting, unset other accepted answers for this question
if (dto.isAccepted) {
await this.db
.update(answers)
.set({ isAccepted: false, updatedAt: new Date() })
.where(and(eq(answers.questionId, answer.questionId), eq(answers.isAccepted, true)));
}
const [updated] = await this.db
.update(answers)
.set({
isAccepted: dto.isAccepted,
updatedAt: new Date(),
})
.where(eq(answers.id, id))
.returning();
return updated;
}
async delete(userId: string, id: string): Promise<void> {
await this.findOne(userId, id);
await this.db.delete(answers).where(eq(answers.id, id));
}
async getAccepted(userId: string, questionId: string): Promise<Answer | null> {
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException(`Question with id ${questionId} not found`);
}
const [accepted] = await this.db
.select()
.from(answers)
.where(and(eq(answers.questionId, questionId), eq(answers.isAccepted, true)));
return accepted || null;
}
}

View file

@ -0,0 +1,55 @@
import { IsUUID, IsString, IsOptional, IsNumber, IsArray } from 'class-validator';
export class CreateAnswerDto {
@IsUUID()
questionId: string;
@IsOptional()
@IsUUID()
researchResultId?: string;
@IsString()
content: string;
@IsOptional()
@IsString()
contentMarkdown?: string;
@IsOptional()
@IsString()
summary?: string;
@IsString()
modelId: string;
@IsString()
provider: string;
@IsOptional()
@IsNumber()
promptTokens?: number;
@IsOptional()
@IsNumber()
completionTokens?: number;
@IsOptional()
@IsNumber()
estimatedCost?: number;
@IsOptional()
@IsNumber()
confidence?: number;
@IsOptional()
@IsNumber()
sourceCount?: number;
@IsOptional()
@IsArray()
citations?: any[];
@IsOptional()
@IsNumber()
durationMs?: number;
}

View file

@ -0,0 +1,2 @@
export * from './create-answer.dto';
export * from './update-answer.dto';

View file

@ -0,0 +1,31 @@
import { IsOptional, IsString, IsNumber, IsBoolean, Min, Max } from 'class-validator';
export class UpdateAnswerDto {
@IsOptional()
@IsString()
content?: string;
@IsOptional()
@IsString()
contentMarkdown?: string;
@IsOptional()
@IsString()
summary?: string;
}
export class RateAnswerDto {
@IsNumber()
@Min(1)
@Max(5)
rating: number;
@IsOptional()
@IsString()
feedback?: string;
}
export class AcceptAnswerDto {
@IsBoolean()
isAccepted: boolean;
}

View file

@ -0,0 +1,27 @@
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import configuration from './config/configuration';
import { DatabaseModule } from './db/database.module';
import { HealthModule } from './health/health.module';
import { QuestionModule } from './question/question.module';
import { CollectionModule } from './collection/collection.module';
import { ResearchModule } from './research/research.module';
import { AnswerModule } from './answer/answer.module';
import { SourceModule } from './source/source.module';
@Module({
imports: [
ConfigModule.forRoot({
isGlobal: true,
load: [configuration],
}),
DatabaseModule,
HealthModule,
QuestionModule,
CollectionModule,
ResearchModule,
AnswerModule,
SourceModule,
],
})
export class AppModule {}

View file

@ -0,0 +1,61 @@
import {
Controller,
Get,
Post,
Put,
Delete,
Body,
Param,
UseGuards,
ParseUUIDPipe,
} from '@nestjs/common';
import { CollectionService } from './collection.service';
import { CreateCollectionDto, UpdateCollectionDto } from './dto';
import { JwtAuthGuard, CurrentUser, CurrentUserData } from '@manacore/shared-nestjs-auth';
@Controller('collections')
@UseGuards(JwtAuthGuard)
export class CollectionController {
constructor(private readonly collectionService: CollectionService) {}
@Post()
async create(@CurrentUser() user: CurrentUserData, @Body() dto: CreateCollectionDto) {
return this.collectionService.create(user.userId, dto);
}
@Get()
async findAll(@CurrentUser() user: CurrentUserData) {
return this.collectionService.findAll(user.userId);
}
@Get('default')
async getDefault(@CurrentUser() user: CurrentUserData) {
return this.collectionService.getDefault(user.userId);
}
@Get(':id')
async findOne(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
return this.collectionService.findOne(user.userId, id);
}
@Put(':id')
async update(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
@Body() dto: UpdateCollectionDto,
) {
return this.collectionService.update(user.userId, id, dto);
}
@Delete(':id')
async delete(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
await this.collectionService.delete(user.userId, id);
return { success: true };
}
@Post('reorder')
async reorder(@CurrentUser() user: CurrentUserData, @Body('orderedIds') orderedIds: string[]) {
await this.collectionService.reorder(user.userId, orderedIds);
return { success: true };
}
}

View file

@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { CollectionController } from './collection.controller';
import { CollectionService } from './collection.service';
import { DatabaseModule } from '../db/database.module';
@Module({
imports: [DatabaseModule],
controllers: [CollectionController],
providers: [CollectionService],
exports: [CollectionService],
})
export class CollectionModule {}

View file

@ -0,0 +1,171 @@
import { Injectable, Inject, NotFoundException, ConflictException } from '@nestjs/common';
import { NodePgDatabase } from 'drizzle-orm/node-postgres';
import { eq, and, desc, isNull, count } from 'drizzle-orm';
import { collections, Collection, NewCollection, questions } from '../db/schema';
import { CreateCollectionDto, UpdateCollectionDto } from './dto';
@Injectable()
export class CollectionService {
constructor(
@Inject('DATABASE_CONNECTION')
private readonly db: NodePgDatabase,
) {}
async create(userId: string, dto: CreateCollectionDto): Promise<Collection> {
// If this is set as default, unset other defaults
if (dto.isDefault) {
await this.db
.update(collections)
.set({ isDefault: false })
.where(and(eq(collections.userId, userId), eq(collections.isDefault, true)));
}
// Get max sort order
const existing = await this.db
.select({ sortOrder: collections.sortOrder })
.from(collections)
.where(and(eq(collections.userId, userId), isNull(collections.deletedAt)))
.orderBy(desc(collections.sortOrder))
.limit(1);
const maxSortOrder = existing.length > 0 ? existing[0].sortOrder ?? 0 : 0;
const newCollection: NewCollection = {
userId,
name: dto.name,
description: dto.description,
color: dto.color || '#6366f1',
icon: dto.icon || 'folder',
isDefault: dto.isDefault || false,
sortOrder: maxSortOrder + 1,
};
const [created] = await this.db.insert(collections).values(newCollection).returning();
return created;
}
async findAll(userId: string): Promise<(Collection & { questionCount: number })[]> {
const userCollections = await this.db
.select()
.from(collections)
.where(and(eq(collections.userId, userId), isNull(collections.deletedAt)))
.orderBy(collections.sortOrder);
// Get question counts for each collection
const result = await Promise.all(
userCollections.map(async (collection) => {
const [countResult] = await this.db
.select({ count: count() })
.from(questions)
.where(
and(
eq(questions.collectionId, collection.id),
eq(questions.userId, userId),
isNull(questions.deletedAt),
),
);
return {
...collection,
questionCount: countResult?.count ?? 0,
};
}),
);
return result;
}
async findOne(userId: string, id: string): Promise<Collection> {
const [collection] = await this.db
.select()
.from(collections)
.where(
and(eq(collections.id, id), eq(collections.userId, userId), isNull(collections.deletedAt)),
);
if (!collection) {
throw new NotFoundException(`Collection with id ${id} not found`);
}
return collection;
}
async update(userId: string, id: string, dto: UpdateCollectionDto): Promise<Collection> {
await this.findOne(userId, id);
// If setting as default, unset other defaults
if (dto.isDefault) {
await this.db
.update(collections)
.set({ isDefault: false })
.where(and(eq(collections.userId, userId), eq(collections.isDefault, true)));
}
const updateData: Partial<NewCollection> = {
...dto,
updatedAt: new Date(),
};
const [updated] = await this.db
.update(collections)
.set(updateData)
.where(and(eq(collections.id, id), eq(collections.userId, userId)))
.returning();
return updated;
}
async delete(userId: string, id: string): Promise<void> {
const collection = await this.findOne(userId, id);
// Check if collection has questions
const [questionsCount] = await this.db
.select({ count: count() })
.from(questions)
.where(
and(
eq(questions.collectionId, id),
eq(questions.userId, userId),
isNull(questions.deletedAt),
),
);
if (questionsCount.count > 0) {
throw new ConflictException(
'Cannot delete collection with questions. Move or delete questions first.',
);
}
// Soft delete
await this.db
.update(collections)
.set({ deletedAt: new Date() })
.where(and(eq(collections.id, id), eq(collections.userId, userId)));
}
async getDefault(userId: string): Promise<Collection | null> {
const [collection] = await this.db
.select()
.from(collections)
.where(
and(
eq(collections.userId, userId),
eq(collections.isDefault, true),
isNull(collections.deletedAt),
),
);
return collection || null;
}
async reorder(userId: string, orderedIds: string[]): Promise<void> {
await Promise.all(
orderedIds.map((id, index) =>
this.db
.update(collections)
.set({ sortOrder: index, updatedAt: new Date() })
.where(and(eq(collections.id, id), eq(collections.userId, userId))),
),
);
}
}

View file

@ -0,0 +1,22 @@
import { IsString, IsOptional, IsBoolean } from 'class-validator';
export class CreateCollectionDto {
@IsString()
name: string;
@IsOptional()
@IsString()
description?: string;
@IsOptional()
@IsString()
color?: string;
@IsOptional()
@IsString()
icon?: string;
@IsOptional()
@IsBoolean()
isDefault?: boolean;
}

View file

@ -0,0 +1,2 @@
export * from './create-collection.dto';
export * from './update-collection.dto';

View file

@ -0,0 +1,27 @@
import { IsString, IsOptional, IsBoolean, IsNumber } from 'class-validator';
export class UpdateCollectionDto {
@IsOptional()
@IsString()
name?: string;
@IsOptional()
@IsString()
description?: string;
@IsOptional()
@IsString()
color?: string;
@IsOptional()
@IsString()
icon?: string;
@IsOptional()
@IsBoolean()
isDefault?: boolean;
@IsOptional()
@IsNumber()
sortOrder?: number;
}

View file

@ -0,0 +1,27 @@
export default () => ({
port: parseInt(process.env.PORT || '3011', 10),
nodeEnv: process.env.NODE_ENV || 'development',
database: {
url: process.env.DATABASE_URL,
},
cors: {
origins: process.env.CORS_ORIGINS?.split(',') || [
'http://localhost:3000',
'http://localhost:5173',
'http://localhost:8081',
],
},
auth: {
manaAuthUrl: process.env.MANA_CORE_AUTH_URL || 'http://localhost:3001',
devBypass: process.env.DEV_BYPASS_AUTH === 'true',
devUserId: process.env.DEV_USER_ID,
},
manaSearch: {
url: process.env.MANA_SEARCH_URL || 'http://localhost:3021',
timeout: parseInt(process.env.MANA_SEARCH_TIMEOUT || '30000', 10),
},
});

View file

@ -0,0 +1,15 @@
import { drizzle } from 'drizzle-orm/postgres-js';
import postgres from 'postgres';
import * as schema from './schema';
const connectionString = process.env.DATABASE_URL!;
const client = postgres(connectionString, {
max: 10,
idle_timeout: 20,
connect_timeout: 10,
});
export const db = drizzle(client, { schema });
export type Database = typeof db;

View file

@ -0,0 +1,16 @@
import { Global, Module } from '@nestjs/common';
import { db } from './connection';
export const DATABASE_CONNECTION = 'DATABASE_CONNECTION';
@Global()
@Module({
providers: [
{
provide: DATABASE_CONNECTION,
useValue: db,
},
],
exports: [DATABASE_CONNECTION],
})
export class DatabaseModule {}

View file

@ -0,0 +1,49 @@
import { pgTable, uuid, text, integer, real, timestamp, jsonb, boolean } from 'drizzle-orm/pg-core';
import { questions } from './questions.schema';
import { researchResults } from './research.schema';
export const answers = pgTable('answers', {
id: uuid('id').primaryKey().defaultRandom(),
questionId: uuid('question_id')
.notNull()
.references(() => questions.id, { onDelete: 'cascade' }),
researchResultId: uuid('research_result_id').references(() => researchResults.id, {
onDelete: 'set null',
}),
// Answer content
content: text('content').notNull(),
contentMarkdown: text('content_markdown'),
summary: text('summary'), // Short summary of the answer
// Generation metadata
modelId: text('model_id').notNull(),
provider: text('provider').notNull(), // 'ollama', 'openrouter'
// Token tracking
promptTokens: integer('prompt_tokens'),
completionTokens: integer('completion_tokens'),
estimatedCost: real('estimated_cost'),
// Quality indicators
confidence: real('confidence'), // 0-1 confidence score
sourceCount: integer('source_count'), // Number of sources used
citations: jsonb('citations').default([]), // Array of citation references
// User feedback
rating: integer('rating'), // 1-5 user rating
feedback: text('feedback'), // User feedback text
isAccepted: boolean('is_accepted').default(false), // User marked as accepted answer
// Versioning
version: integer('version').default(1),
previousVersionId: uuid('previous_version_id'),
// Timestamps
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow(),
durationMs: integer('duration_ms'),
});
export type Answer = typeof answers.$inferSelect;
export type NewAnswer = typeof answers.$inferInsert;

View file

@ -0,0 +1,20 @@
import { pgTable, uuid, text, boolean, timestamp } from 'drizzle-orm/pg-core';
export const collections = pgTable('collections', {
id: uuid('id').primaryKey().defaultRandom(),
userId: text('user_id').notNull(),
name: text('name').notNull(),
description: text('description'),
color: text('color').default('#6366f1'),
icon: text('icon').default('folder'),
isShared: boolean('is_shared').default(false),
shareToken: text('share_token').unique(),
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow(),
});
export type Collection = typeof collections.$inferSelect;
export type NewCollection = typeof collections.$inferInsert;

View file

@ -0,0 +1,5 @@
export * from './questions.schema';
export * from './collections.schema';
export * from './research.schema';
export * from './sources.schema';
export * from './answers.schema';

View file

@ -0,0 +1,38 @@
import { pgTable, uuid, text, boolean, timestamp } from 'drizzle-orm/pg-core';
import { collections } from './collections.schema';
export const questions = pgTable('questions', {
id: uuid('id').primaryKey().defaultRandom(),
userId: text('user_id').notNull(),
collectionId: uuid('collection_id').references(() => collections.id, {
onDelete: 'set null',
}),
// Content
title: text('title').notNull(),
description: text('description'),
// Status & Priority
status: text('status').notNull().default('open'), // 'open', 'researching', 'answered', 'archived'
priority: text('priority').default('normal'), // 'low', 'normal', 'high', 'urgent'
// Categorization
tags: text('tags').array().default([]),
category: text('category'),
// Research config
researchDepth: text('research_depth').default('quick'), // 'quick', 'standard', 'deep'
autoResearch: boolean('auto_research').default(false),
// Timestamps
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow(),
answeredAt: timestamp('answered_at', { withTimezone: true }),
// Soft delete
isArchived: boolean('is_archived').default(false),
archivedAt: timestamp('archived_at', { withTimezone: true }),
});
export type Question = typeof questions.$inferSelect;
export type NewQuestion = typeof questions.$inferInsert;

View file

@ -0,0 +1,31 @@
import { pgTable, uuid, text, integer, real, timestamp, jsonb } from 'drizzle-orm/pg-core';
import { questions } from './questions.schema';
export const researchResults = pgTable('research_results', {
id: uuid('id').primaryKey().defaultRandom(),
questionId: uuid('question_id')
.notNull()
.references(() => questions.id, { onDelete: 'cascade' }),
// Research metadata
modelId: text('model_id').notNull(),
provider: text('provider').notNull(), // 'searxng', 'ollama', 'openrouter'
researchDepth: text('research_depth').notNull(), // 'quick', 'standard', 'deep'
// Results
summary: text('summary').notNull(),
keyPoints: jsonb('key_points').default([]),
followUpQuestions: text('follow_up_questions').array().default([]),
// Token tracking
promptTokens: integer('prompt_tokens'),
completionTokens: integer('completion_tokens'),
estimatedCost: real('estimated_cost'),
// Timestamps
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
durationMs: integer('duration_ms'),
});
export type ResearchResult = typeof researchResults.$inferSelect;
export type NewResearchResult = typeof researchResults.$inferInsert;

View file

@ -0,0 +1,41 @@
import { pgTable, uuid, text, integer, timestamp, jsonb, real } from 'drizzle-orm/pg-core';
import { researchResults } from './research.schema';
export const sources = pgTable('sources', {
id: uuid('id').primaryKey().defaultRandom(),
researchResultId: uuid('research_result_id')
.notNull()
.references(() => researchResults.id, { onDelete: 'cascade' }),
// Source metadata
url: text('url').notNull(),
title: text('title').notNull(),
snippet: text('snippet'),
domain: text('domain'),
// Content extraction
extractedContent: text('extracted_content'),
contentMarkdown: text('content_markdown'),
wordCount: integer('word_count'),
readingTime: integer('reading_time'), // in minutes
// Quality indicators
relevanceScore: real('relevance_score'), // 0-1 score from search
position: integer('position'), // Position in search results
engine: text('engine'), // Which search engine found this
// Publication info
author: text('author'),
publishedDate: timestamp('published_date', { withTimezone: true }),
siteName: text('site_name'),
// Additional metadata
metadata: jsonb('metadata').default({}),
// Timestamps
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
extractedAt: timestamp('extracted_at', { withTimezone: true }),
});
export type Source = typeof sources.$inferSelect;
export type NewSource = typeof sources.$inferInsert;

View file

@ -0,0 +1,49 @@
import { Controller, Get, Inject } from '@nestjs/common';
import { NodePgDatabase } from 'drizzle-orm/node-postgres';
import { sql } from 'drizzle-orm';
@Controller('health')
export class HealthController {
constructor(
@Inject('DATABASE_CONNECTION')
private readonly db: NodePgDatabase,
) {}
@Get()
async check() {
const checks = {
database: await this.checkDatabase(),
timestamp: new Date().toISOString(),
};
const healthy = Object.values(checks).every((v) => v === true || typeof v === 'string');
return {
status: healthy ? 'healthy' : 'unhealthy',
checks,
};
}
@Get('live')
liveness() {
return { status: 'ok' };
}
@Get('ready')
async readiness() {
const dbOk = await this.checkDatabase();
return {
status: dbOk ? 'ready' : 'not_ready',
database: dbOk,
};
}
private async checkDatabase(): Promise<boolean> {
try {
await this.db.execute(sql`SELECT 1`);
return true;
} catch {
return false;
}
}
}

View file

@ -0,0 +1,7 @@
import { Module } from '@nestjs/common';
import { HealthController } from './health.controller';
@Module({
controllers: [HealthController],
})
export class HealthModule {}

View file

@ -0,0 +1,41 @@
import { NestFactory } from '@nestjs/core';
import { ValidationPipe, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { AppModule } from './app.module';
async function bootstrap() {
const logger = new Logger('Bootstrap');
const app = await NestFactory.create(AppModule);
const configService = app.get(ConfigService);
const port = configService.get<number>('port', 3011);
// Global prefix
app.setGlobalPrefix('api/v1');
// CORS
app.enableCors({
origin: configService.get<string[]>('cors.origins', [
'http://localhost:3000',
'http://localhost:5173',
'http://localhost:8081',
]),
credentials: true,
});
// Global validation pipe
app.useGlobalPipes(
new ValidationPipe({
whitelist: true,
transform: true,
forbidNonWhitelisted: true,
}),
);
await app.listen(port);
logger.log(`Questions Backend running on port ${port}`);
logger.log(`Health check: http://localhost:${port}/api/v1/health`);
}
bootstrap();

View file

@ -0,0 +1,40 @@
import { IsString, IsOptional, IsArray, IsUUID, IsEnum } from 'class-validator';
export enum QuestionPriority {
LOW = 'low',
NORMAL = 'normal',
HIGH = 'high',
URGENT = 'urgent',
}
export enum ResearchDepth {
QUICK = 'quick',
STANDARD = 'standard',
DEEP = 'deep',
}
export class CreateQuestionDto {
@IsString()
title: string;
@IsOptional()
@IsString()
description?: string;
@IsOptional()
@IsUUID()
collectionId?: string;
@IsOptional()
@IsArray()
@IsString({ each: true })
tags?: string[];
@IsOptional()
@IsEnum(QuestionPriority)
priority?: QuestionPriority;
@IsOptional()
@IsEnum(ResearchDepth)
researchDepth?: ResearchDepth;
}

View file

@ -0,0 +1,2 @@
export * from './create-question.dto';
export * from './update-question.dto';

View file

@ -0,0 +1,40 @@
import { IsString, IsOptional, IsArray, IsUUID, IsEnum } from 'class-validator';
import { QuestionPriority, ResearchDepth } from './create-question.dto';
export enum QuestionStatus {
OPEN = 'open',
RESEARCHING = 'researching',
ANSWERED = 'answered',
ARCHIVED = 'archived',
}
export class UpdateQuestionDto {
@IsOptional()
@IsString()
title?: string;
@IsOptional()
@IsString()
description?: string;
@IsOptional()
@IsUUID()
collectionId?: string;
@IsOptional()
@IsArray()
@IsString({ each: true })
tags?: string[];
@IsOptional()
@IsEnum(QuestionPriority)
priority?: QuestionPriority;
@IsOptional()
@IsEnum(QuestionStatus)
status?: QuestionStatus;
@IsOptional()
@IsEnum(ResearchDepth)
researchDepth?: ResearchDepth;
}

View file

@ -0,0 +1,75 @@
import {
Controller,
Get,
Post,
Put,
Delete,
Body,
Param,
Query,
UseGuards,
ParseUUIDPipe,
} from '@nestjs/common';
import { QuestionService } from './question.service';
import { CreateQuestionDto, UpdateQuestionDto } from './dto';
import { JwtAuthGuard, CurrentUser, CurrentUserData } from '@manacore/shared-nestjs-auth';
@Controller('questions')
@UseGuards(JwtAuthGuard)
export class QuestionController {
constructor(private readonly questionService: QuestionService) {}
@Post()
async create(@CurrentUser() user: CurrentUserData, @Body() dto: CreateQuestionDto) {
return this.questionService.create(user.userId, dto);
}
@Get()
async findAll(
@CurrentUser() user: CurrentUserData,
@Query('collectionId') collectionId?: string,
@Query('status') status?: string,
@Query('search') search?: string,
@Query('tags') tags?: string,
@Query('limit') limit?: string,
@Query('offset') offset?: string,
) {
return this.questionService.findAll(user.userId, {
collectionId,
status,
search,
tags: tags ? tags.split(',') : undefined,
limit: limit ? parseInt(limit, 10) : undefined,
offset: offset ? parseInt(offset, 10) : undefined,
});
}
@Get(':id')
async findOne(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
return this.questionService.findOne(user.userId, id);
}
@Put(':id')
async update(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
@Body() dto: UpdateQuestionDto,
) {
return this.questionService.update(user.userId, id, dto);
}
@Delete(':id')
async delete(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
await this.questionService.delete(user.userId, id);
return { success: true };
}
@Put(':id/status')
async updateStatus(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
@Body('status') status: string,
) {
return this.questionService.updateStatus(user.userId, id, status);
}
}

View file

@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { QuestionController } from './question.controller';
import { QuestionService } from './question.service';
import { DatabaseModule } from '../db/database.module';
@Module({
imports: [DatabaseModule],
controllers: [QuestionController],
providers: [QuestionService],
exports: [QuestionService],
})
export class QuestionModule {}

View file

@ -0,0 +1,157 @@
import { Injectable, Inject, NotFoundException } from '@nestjs/common';
import { NodePgDatabase } from 'drizzle-orm/node-postgres';
import { eq, and, desc, isNull, ilike, or, inArray } from 'drizzle-orm';
import { questions, Question, NewQuestion } from '../db/schema';
import { CreateQuestionDto, UpdateQuestionDto } from './dto';
@Injectable()
export class QuestionService {
constructor(
@Inject('DATABASE_CONNECTION')
private readonly db: NodePgDatabase,
) {}
async create(userId: string, dto: CreateQuestionDto): Promise<Question> {
const newQuestion: NewQuestion = {
userId,
title: dto.title,
description: dto.description,
collectionId: dto.collectionId,
tags: dto.tags || [],
priority: dto.priority || 'normal',
researchDepth: dto.researchDepth || 'quick',
};
const [created] = await this.db.insert(questions).values(newQuestion).returning();
return created;
}
async findAll(
userId: string,
options?: {
collectionId?: string;
status?: string;
search?: string;
tags?: string[];
limit?: number;
offset?: number;
},
): Promise<{ data: Question[]; total: number }> {
const conditions = [eq(questions.userId, userId), isNull(questions.deletedAt)];
if (options?.collectionId) {
conditions.push(eq(questions.collectionId, options.collectionId));
}
if (options?.status) {
conditions.push(eq(questions.status, options.status));
}
if (options?.search) {
conditions.push(
or(
ilike(questions.title, `%${options.search}%`),
ilike(questions.description, `%${options.search}%`),
),
);
}
const limit = options?.limit || 20;
const offset = options?.offset || 0;
const data = await this.db
.select()
.from(questions)
.where(and(...conditions))
.orderBy(desc(questions.createdAt))
.limit(limit)
.offset(offset);
// For total count, we need a separate query
const allMatching = await this.db
.select({ id: questions.id })
.from(questions)
.where(and(...conditions));
return { data, total: allMatching.length };
}
async findOne(userId: string, id: string): Promise<Question> {
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, id), eq(questions.userId, userId), isNull(questions.deletedAt)));
if (!question) {
throw new NotFoundException(`Question with id ${id} not found`);
}
return question;
}
async update(userId: string, id: string, dto: UpdateQuestionDto): Promise<Question> {
// First check if the question exists and belongs to user
await this.findOne(userId, id);
const updateData: Partial<NewQuestion> = {
...dto,
updatedAt: new Date(),
};
const [updated] = await this.db
.update(questions)
.set(updateData)
.where(and(eq(questions.id, id), eq(questions.userId, userId)))
.returning();
return updated;
}
async delete(userId: string, id: string): Promise<void> {
// Soft delete
await this.findOne(userId, id);
await this.db
.update(questions)
.set({ deletedAt: new Date() })
.where(and(eq(questions.id, id), eq(questions.userId, userId)));
}
async updateStatus(userId: string, id: string, status: string): Promise<Question> {
await this.findOne(userId, id);
const [updated] = await this.db
.update(questions)
.set({ status, updatedAt: new Date() })
.where(and(eq(questions.id, id), eq(questions.userId, userId)))
.returning();
return updated;
}
async getByCollection(userId: string, collectionId: string): Promise<Question[]> {
return this.db
.select()
.from(questions)
.where(
and(
eq(questions.userId, userId),
eq(questions.collectionId, collectionId),
isNull(questions.deletedAt),
),
)
.orderBy(desc(questions.createdAt));
}
async getByTags(userId: string, tags: string[]): Promise<Question[]> {
// PostgreSQL array overlap query - find questions that have any of the specified tags
const allQuestions = await this.db
.select()
.from(questions)
.where(and(eq(questions.userId, userId), isNull(questions.deletedAt)))
.orderBy(desc(questions.createdAt));
// Filter in memory for array overlap (Drizzle doesn't have native array overlap)
return allQuestions.filter((q) => q.tags?.some((t) => tags.includes(t)));
}
}

View file

@ -0,0 +1 @@
export * from './start-research.dto';

View file

@ -0,0 +1,43 @@
import { IsUUID, IsOptional, IsEnum, IsArray, IsString, IsNumber } from 'class-validator';
export enum ResearchDepth {
QUICK = 'quick', // 5-10 sources, fast
STANDARD = 'standard', // 15-20 sources, balanced
DEEP = 'deep', // 30+ sources, comprehensive
}
export enum SearchCategory {
GENERAL = 'general',
NEWS = 'news',
SCIENCE = 'science',
IT = 'it',
IMAGES = 'images',
VIDEOS = 'videos',
}
export class StartResearchDto {
@IsUUID()
questionId: string;
@IsOptional()
@IsEnum(ResearchDepth)
depth?: ResearchDepth;
@IsOptional()
@IsArray()
@IsEnum(SearchCategory, { each: true })
categories?: SearchCategory[];
@IsOptional()
@IsArray()
@IsString({ each: true })
engines?: string[];
@IsOptional()
@IsString()
language?: string;
@IsOptional()
@IsNumber()
maxSources?: number;
}

View file

@ -0,0 +1,203 @@
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
export interface SearchOptions {
categories?: string[];
engines?: string[];
language?: string;
limit?: number;
}
export interface SearchResult {
url: string;
title: string;
snippet?: string;
engine: string;
score?: number;
publishedDate?: string;
thumbnail?: string;
}
export interface SearchResponse {
results: SearchResult[];
meta: {
query: string;
duration: number;
total: number;
cached: boolean;
};
}
export interface ExtractOptions {
includeMarkdown?: boolean;
includeHtml?: boolean;
maxLength?: number;
timeout?: number;
}
export interface ExtractedContent {
title: string;
description?: string;
author?: string;
publishedDate?: string;
siteName?: string;
text: string;
markdown?: string;
html?: string;
wordCount: number;
readingTime: number;
ogImage?: string;
}
export interface ExtractResponse {
success: boolean;
content?: ExtractedContent;
error?: string;
meta: {
url: string;
duration: number;
cached: boolean;
};
}
export interface BulkExtractResponse {
results: Array<{
url: string;
success: boolean;
content?: ExtractedContent;
error?: string;
}>;
meta: {
total: number;
successful: number;
failed: number;
duration: number;
};
}
@Injectable()
export class ManaSearchClient {
private readonly logger = new Logger(ManaSearchClient.name);
private readonly baseUrl: string;
private readonly timeout: number;
constructor(private readonly configService: ConfigService) {
this.baseUrl = this.configService.get<string>('manaSearch.url', 'http://localhost:3021');
this.timeout = this.configService.get<number>('manaSearch.timeout', 30000);
}
async search(query: string, options?: SearchOptions): Promise<SearchResponse> {
const url = `${this.baseUrl}/api/v1/search`;
try {
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
query,
options: {
categories: options?.categories || ['general'],
engines: options?.engines,
language: options?.language || 'de-DE',
limit: options?.limit || 20,
},
}),
signal: AbortSignal.timeout(this.timeout),
});
if (!response.ok) {
throw new Error(`Search failed: ${response.status} ${response.statusText}`);
}
return await response.json();
} catch (error) {
this.logger.error(`Search error for "${query}": ${error}`);
throw error;
}
}
async extract(url: string, options?: ExtractOptions): Promise<ExtractResponse> {
const apiUrl = `${this.baseUrl}/api/v1/extract`;
try {
const response = await fetch(apiUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
url,
options: {
includeMarkdown: options?.includeMarkdown ?? true,
includeHtml: options?.includeHtml ?? false,
maxLength: options?.maxLength || 50000,
timeout: options?.timeout || 10000,
},
}),
signal: AbortSignal.timeout(this.timeout),
});
if (!response.ok) {
throw new Error(`Extract failed: ${response.status} ${response.statusText}`);
}
return await response.json();
} catch (error) {
this.logger.error(`Extract error for "${url}": ${error}`);
return {
success: false,
error: error instanceof Error ? error.message : 'Extraction failed',
meta: {
url,
duration: 0,
cached: false,
},
};
}
}
async bulkExtract(urls: string[], options?: ExtractOptions): Promise<BulkExtractResponse> {
const apiUrl = `${this.baseUrl}/api/v1/extract/bulk`;
try {
const response = await fetch(apiUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
urls,
options: {
includeMarkdown: options?.includeMarkdown ?? true,
includeHtml: options?.includeHtml ?? false,
maxLength: options?.maxLength || 50000,
},
concurrency: 5,
}),
signal: AbortSignal.timeout(this.timeout * urls.length),
});
if (!response.ok) {
throw new Error(`Bulk extract failed: ${response.status} ${response.statusText}`);
}
return await response.json();
} catch (error) {
this.logger.error(`Bulk extract error: ${error}`);
throw error;
}
}
async healthCheck(): Promise<boolean> {
try {
const response = await fetch(`${this.baseUrl}/health`, {
signal: AbortSignal.timeout(5000),
});
return response.ok;
} catch {
return false;
}
}
}

View file

@ -0,0 +1,44 @@
import { Controller, Get, Post, Body, Param, UseGuards, ParseUUIDPipe } from '@nestjs/common';
import { ResearchService } from './research.service';
import { ManaSearchClient } from './mana-search.client';
import { StartResearchDto } from './dto';
import { JwtAuthGuard, CurrentUser, CurrentUserData } from '@manacore/shared-nestjs-auth';
@Controller('research')
@UseGuards(JwtAuthGuard)
export class ResearchController {
constructor(
private readonly researchService: ResearchService,
private readonly manaSearchClient: ManaSearchClient,
) {}
@Post('start')
async startResearch(@CurrentUser() user: CurrentUserData, @Body() dto: StartResearchDto) {
return this.researchService.startResearch(user.userId, dto);
}
@Get('question/:questionId')
async getResearchResults(
@CurrentUser() user: CurrentUserData,
@Param('questionId', ParseUUIDPipe) questionId: string,
) {
return this.researchService.getResearchResults(user.userId, questionId);
}
@Get(':id')
async getResearchResult(
@CurrentUser() user: CurrentUserData,
@Param('id', ParseUUIDPipe) id: string,
) {
return this.researchService.getResearchResult(user.userId, id);
}
@Get('health/search')
async checkSearchHealth() {
const healthy = await this.manaSearchClient.healthCheck();
return {
service: 'mana-search',
status: healthy ? 'healthy' : 'unhealthy',
};
}
}

View file

@ -0,0 +1,13 @@
import { Module } from '@nestjs/common';
import { ResearchController } from './research.controller';
import { ResearchService } from './research.service';
import { ManaSearchClient } from './mana-search.client';
import { DatabaseModule } from '../db/database.module';
@Module({
imports: [DatabaseModule],
controllers: [ResearchController],
providers: [ResearchService, ManaSearchClient],
exports: [ResearchService, ManaSearchClient],
})
export class ResearchModule {}

View file

@ -0,0 +1,257 @@
import { Injectable, Inject, Logger, NotFoundException, BadRequestException } from '@nestjs/common';
import { NodePgDatabase } from 'drizzle-orm/node-postgres';
import { eq, and, desc } from 'drizzle-orm';
import {
questions,
researchResults,
sources,
ResearchResult,
NewResearchResult,
NewSource,
} from '../db/schema';
import { ManaSearchClient, SearchResult } from './mana-search.client';
import { StartResearchDto, ResearchDepth } from './dto';
interface DepthConfig {
maxSources: number;
extractContent: boolean;
searchCategories: string[];
}
const DEPTH_CONFIG: Record<ResearchDepth, DepthConfig> = {
[ResearchDepth.QUICK]: {
maxSources: 5,
extractContent: false,
searchCategories: ['general'],
},
[ResearchDepth.STANDARD]: {
maxSources: 15,
extractContent: true,
searchCategories: ['general', 'news'],
},
[ResearchDepth.DEEP]: {
maxSources: 30,
extractContent: true,
searchCategories: ['general', 'news', 'science', 'it'],
},
};
@Injectable()
export class ResearchService {
private readonly logger = new Logger(ResearchService.name);
constructor(
@Inject('DATABASE_CONNECTION')
private readonly db: NodePgDatabase,
private readonly manaSearchClient: ManaSearchClient,
) {}
async startResearch(userId: string, dto: StartResearchDto): Promise<ResearchResult> {
const startTime = Date.now();
// Get the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, dto.questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException(`Question with id ${dto.questionId} not found`);
}
// Check if question is already being researched
if (question.status === 'researching') {
throw new BadRequestException('Research is already in progress for this question');
}
// Update question status
await this.db
.update(questions)
.set({ status: 'researching', updatedAt: new Date() })
.where(eq(questions.id, dto.questionId));
try {
const depth = dto.depth || (question.researchDepth as ResearchDepth) || ResearchDepth.QUICK;
const config = DEPTH_CONFIG[depth];
// Perform search
const searchResponse = await this.manaSearchClient.search(question.title, {
categories: dto.categories || config.searchCategories,
engines: dto.engines,
language: dto.language || 'de-DE',
limit: dto.maxSources || config.maxSources,
});
// Extract content for sources if depth allows
let extractedSources: Array<SearchResult & { extractedContent?: string; markdown?: string }> =
searchResponse.results;
if (config.extractContent && searchResponse.results.length > 0) {
const urls = searchResponse.results.slice(0, config.maxSources).map((r) => r.url);
const bulkExtract = await this.manaSearchClient.bulkExtract(urls);
extractedSources = searchResponse.results.map((result) => {
const extracted = bulkExtract.results.find((e) => e.url === result.url);
return {
...result,
extractedContent: extracted?.success ? extracted.content?.text : undefined,
markdown: extracted?.success ? extracted.content?.markdown : undefined,
};
});
}
// Generate summary from results
const summary = this.generateSummary(question.title, extractedSources);
const keyPoints = this.extractKeyPoints(extractedSources);
const followUpQuestions = this.generateFollowUpQuestions(question.title, extractedSources);
// Save research result
const newResearchResult: NewResearchResult = {
questionId: dto.questionId,
modelId: 'mana-search',
provider: 'searxng',
researchDepth: depth,
summary,
keyPoints,
followUpQuestions,
durationMs: Date.now() - startTime,
};
const [researchResult] = await this.db
.insert(researchResults)
.values(newResearchResult)
.returning();
// Save sources
if (extractedSources.length > 0) {
const sourcesToInsert: NewSource[] = extractedSources.map((source, index) => ({
researchResultId: researchResult.id,
url: source.url,
title: source.title,
snippet: source.snippet,
domain: new URL(source.url).hostname,
extractedContent: source.extractedContent,
contentMarkdown: source.markdown,
relevanceScore: source.score,
position: index + 1,
engine: source.engine,
}));
await this.db.insert(sources).values(sourcesToInsert);
}
// Update question status
await this.db
.update(questions)
.set({ status: 'answered', updatedAt: new Date() })
.where(eq(questions.id, dto.questionId));
return researchResult;
} catch (error) {
// Reset question status on error
await this.db
.update(questions)
.set({ status: 'open', updatedAt: new Date() })
.where(eq(questions.id, dto.questionId));
this.logger.error(`Research failed for question ${dto.questionId}: ${error}`);
throw error;
}
}
async getResearchResults(userId: string, questionId: string): Promise<ResearchResult[]> {
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException(`Question with id ${questionId} not found`);
}
return this.db
.select()
.from(researchResults)
.where(eq(researchResults.questionId, questionId))
.orderBy(desc(researchResults.createdAt));
}
async getResearchResult(
userId: string,
resultId: string,
): Promise<ResearchResult & { sources: any[] }> {
const [result] = await this.db
.select()
.from(researchResults)
.where(eq(researchResults.id, resultId));
if (!result) {
throw new NotFoundException(`Research result with id ${resultId} not found`);
}
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, result.questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException('Research result not found');
}
// Get sources
const resultSources = await this.db
.select()
.from(sources)
.where(eq(sources.researchResultId, resultId))
.orderBy(sources.position);
return {
...result,
sources: resultSources,
};
}
private generateSummary(
question: string,
sources: Array<SearchResult & { extractedContent?: string }>,
): string {
if (sources.length === 0) {
return 'No relevant sources found for this question.';
}
// Simple summary from snippets (in production, this would use an LLM)
const snippets = sources
.filter((s) => s.snippet || s.extractedContent)
.slice(0, 5)
.map((s) => s.snippet || s.extractedContent?.substring(0, 500))
.join('\n\n');
return `Research found ${sources.length} relevant sources for: "${question}"\n\nKey findings:\n${snippets}`;
}
private extractKeyPoints(
sources: Array<SearchResult & { extractedContent?: string }>,
): string[] {
// Extract key points from titles and snippets
return sources
.filter((s) => s.title)
.slice(0, 5)
.map((s) => s.title);
}
private generateFollowUpQuestions(
question: string,
sources: Array<SearchResult>,
): string[] {
// Generate related questions (in production, this would use an LLM)
const baseQuestions = [
`What are the main challenges related to ${question}?`,
`What are the latest developments in ${question}?`,
`How does ${question} compare to alternatives?`,
];
return baseQuestions;
}
}

View file

@ -0,0 +1,35 @@
import { Controller, Get, Param, UseGuards, ParseUUIDPipe } from '@nestjs/common';
import { SourceService } from './source.service';
import { JwtAuthGuard, CurrentUser, CurrentUserData } from '@manacore/shared-nestjs-auth';
@Controller('sources')
@UseGuards(JwtAuthGuard)
export class SourceController {
constructor(private readonly sourceService: SourceService) {}
@Get('research/:researchResultId')
async findByResearchResult(
@CurrentUser() user: CurrentUserData,
@Param('researchResultId', ParseUUIDPipe) researchResultId: string,
) {
return this.sourceService.findByResearchResult(user.userId, researchResultId);
}
@Get('question/:questionId')
async findByQuestion(
@CurrentUser() user: CurrentUserData,
@Param('questionId', ParseUUIDPipe) questionId: string,
) {
return this.sourceService.findByQuestion(user.userId, questionId);
}
@Get(':id')
async findOne(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
return this.sourceService.findOne(user.userId, id);
}
@Get(':id/content')
async getContent(@CurrentUser() user: CurrentUserData, @Param('id', ParseUUIDPipe) id: string) {
return this.sourceService.getContent(user.userId, id);
}
}

View file

@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { SourceController } from './source.controller';
import { SourceService } from './source.service';
import { DatabaseModule } from '../db/database.module';
@Module({
imports: [DatabaseModule],
controllers: [SourceController],
providers: [SourceService],
exports: [SourceService],
})
export class SourceModule {}

View file

@ -0,0 +1,99 @@
import { Injectable, Inject, NotFoundException } from '@nestjs/common';
import { NodePgDatabase } from 'drizzle-orm/node-postgres';
import { eq, and } from 'drizzle-orm';
import { questions, researchResults, sources, Source } from '../db/schema';
@Injectable()
export class SourceService {
constructor(
@Inject('DATABASE_CONNECTION')
private readonly db: NodePgDatabase,
) {}
async findByResearchResult(userId: string, researchResultId: string): Promise<Source[]> {
// Verify user owns the research result
await this.verifyOwnership(userId, researchResultId);
return this.db
.select()
.from(sources)
.where(eq(sources.researchResultId, researchResultId))
.orderBy(sources.position);
}
async findOne(userId: string, id: string): Promise<Source> {
const [source] = await this.db.select().from(sources).where(eq(sources.id, id));
if (!source) {
throw new NotFoundException(`Source with id ${id} not found`);
}
// Verify user owns the source via research result
await this.verifyOwnership(userId, source.researchResultId);
return source;
}
async findByQuestion(userId: string, questionId: string): Promise<Source[]> {
// Verify user owns the question
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException(`Question with id ${questionId} not found`);
}
// Get all sources from all research results for this question
const results = await this.db
.select()
.from(researchResults)
.where(eq(researchResults.questionId, questionId));
if (results.length === 0) {
return [];
}
const allSources: Source[] = [];
for (const result of results) {
const resultSources = await this.db
.select()
.from(sources)
.where(eq(sources.researchResultId, result.id))
.orderBy(sources.position);
allSources.push(...resultSources);
}
return allSources;
}
async getContent(userId: string, id: string): Promise<{ text: string; markdown?: string }> {
const source = await this.findOne(userId, id);
return {
text: source.extractedContent || source.snippet || '',
markdown: source.contentMarkdown || undefined,
};
}
private async verifyOwnership(userId: string, researchResultId: string): Promise<void> {
const [result] = await this.db
.select()
.from(researchResults)
.where(eq(researchResults.id, researchResultId));
if (!result) {
throw new NotFoundException('Research result not found');
}
const [question] = await this.db
.select()
.from(questions)
.where(and(eq(questions.id, result.questionId), eq(questions.userId, userId)));
if (!question) {
throw new NotFoundException('Source not found');
}
}
}

View file

@ -0,0 +1,25 @@
{
"compilerOptions": {
"module": "commonjs",
"declaration": true,
"removeComments": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"allowSyntheticDefaultImports": true,
"target": "ES2022",
"sourceMap": true,
"outDir": "./dist",
"baseUrl": "./",
"incremental": true,
"skipLibCheck": true,
"strictNullChecks": true,
"noImplicitAny": true,
"strictBindCallApply": true,
"forceConsistentCasingInFileNames": true,
"noFallthroughCasesInSwitch": true,
"esModuleInterop": true,
"resolveJsonModule": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}

View file

@ -0,0 +1,9 @@
{
"name": "questions",
"version": "1.0.0",
"private": true,
"description": "Questions app - Collect questions and research answers",
"scripts": {
"dev": "turbo run dev"
}
}

View file

@ -218,6 +218,13 @@
"search:docker:up": "docker-compose -f services/mana-search/docker-compose.yml up -d",
"search:docker:down": "docker-compose -f services/mana-search/docker-compose.yml down",
"search:docker:logs": "docker-compose -f services/mana-search/docker-compose.yml logs -f",
"questions:dev": "turbo run dev --filter=questions...",
"dev:questions:backend": "pnpm --filter @questions/backend dev",
"dev:questions:web": "pnpm --filter @questions/web dev",
"dev:questions:app": "turbo run dev --filter=@questions/web --filter=@questions/backend",
"dev:questions:full": "./scripts/setup-databases.sh questions && ./scripts/setup-databases.sh auth && pnpm dev:search:docker && concurrently -n auth,search,backend -c blue,yellow,green \"pnpm dev:auth\" \"pnpm dev:search\" \"pnpm dev:questions:backend\"",
"questions:db:push": "pnpm --filter @questions/backend db:push",
"questions:db:studio": "pnpm --filter @questions/backend db:studio",
"dev:projectdoc": "pnpm --filter @manacore/telegram-project-doc-bot start:dev",
"dev:projectdoc:full": "./scripts/setup-databases.sh projectdoc && pnpm dev:projectdoc",
"projectdoc:db:push": "pnpm --filter @manacore/telegram-project-doc-bot db:push",

View file

@ -78,6 +78,7 @@ ALL_DATABASES=(
"zitare_bot"
"todo_bot"
"nutriphi_bot"
"questions"
)
# Check if specific service requested
@ -175,9 +176,13 @@ setup_service() {
create_db_if_not_exists "nutriphi_bot"
push_schema "@manacore/telegram-nutriphi-bot" "nutriphi-bot"
;;
questions)
create_db_if_not_exists "questions"
push_schema "@questions/backend" "questions"
;;
*)
echo -e "${RED}Unknown service: $service${NC}"
echo "Available services: auth, chat, zitare, contacts, calendar, clock, todo, manadeck, mail, moodlit, finance, voxel-lava, figgos, planta, nutriphi, presi, storage, projectdoc, zitare_bot, todo_bot, nutriphi_bot"
echo "Available services: auth, chat, zitare, contacts, calendar, clock, todo, manadeck, mail, moodlit, finance, voxel-lava, figgos, planta, nutriphi, presi, storage, projectdoc, zitare_bot, todo_bot, nutriphi_bot, questions"
exit 1
;;
esac
@ -201,7 +206,7 @@ echo -e "\n${GREEN}Step 2: Pushing schemas${NC}"
echo "--------------------------------------"
# Push schemas for all known services
for service in auth chat zitare contacts calendar clock todo manadeck picture mail moodlit finance voxel-lava figgos planta nutriphi presi storage; do
for service in auth chat zitare contacts calendar clock todo manadeck picture mail moodlit finance voxel-lava figgos planta nutriphi presi storage questions; do
setup_service "$service" 2>/dev/null || true
done