Interactive investigation simulation for comparing Sparse, Dense, and Hybrid retrieval on the same Indonesian investigation questions.
The active ID v3 workflow, migration instructions, offline checks, and experiment commands are documented in Research realignment handoff. Earlier benchmark reports are historical and must not be combined with v3 results.
.
├── app/ # Next.js app directory
│ ├── api/ # API routes
│ │ ├── cases/ # Case management endpoints
│ │ ├── sessions/ # Session management
│ │ └── ai/ # AI interaction endpoints
│ ├── page.tsx # Main entry point
│ └── layout.tsx # Root layout
├── components/ # React components
│ ├── CaseSelection.tsx # Case selection UI
│ └── InvestigationDashboard.tsx # Main investigation interface
├── lib/ # Utility functions
│ ├── db.ts # Prisma client
│ ├── embedding.ts # OpenAI embedding generation
│ ├── rag.ts # RAG retrieval logic
│ └── case-data.ts # Case data format specification
├── prisma/ # Database schema and migrations
│ ├── schema.prisma # Database schema
│ └── seed.ts # Database seed script
└── README.md
Create .env.local with required variables:
DATABASE_URL="postgresql://admin:root@localhost:5432/thesis?schema=public"
OPENAI_API_KEY="your-openai-api-key"
NEXT_PUBLIC_API_URL="http://localhost:3000"npm installnpm run prisma:generatenpm run prisma:migratenpm run seedBy default this loads the Indonesian ID v3 case (150 evidence records and 30 scenarios) into an isolated database. Embeddings require the explicit --embed flag; seeding never clears existing research data.
npm run devVisit http://localhost:3000 in your browser.
-
cases: Investigation cases
- id, title, description, createdAt, updatedAt
-
suspects: Case suspects
- id, caseId, name, profile
-
evidence: Investigation evidence with vector embeddings
- id, caseId, type, content, embedding
-
investigation_sessions: Player sessions with RAG toggle
- id, caseId, ragEnabled, createdAt, updatedAt
-
investigation_logs: Player actions during investigation
- id, sessionId, action, target, createdAt
-
ai_interaction_logs: AI responses with metrics
- All prompt/response content
- Timing metrics (retrieval_time_ms, llm_response_time_ms, total_response_time_ms)
- Token usage (prompt_tokens, completion_tokens, total_tokens)
- Hallucination detection flags
- Historical action-score fields retained for prior runs; active runs use retrieval and RAGAS metrics
Use the standardized case data format in lib/case-data.ts.
Example JSON structure for AI case generation:
{
"case": {
"title": "Case Title",
"description": "2-3 sentence description"
},
"suspects": [
{
"name": "Suspect Name",
"profile": "Background and motive"
}
],
"evidence": [
{
"type": "witness_statement|forensic_report|cctv_log|financial_record|email_message|location_report",
"content": "Specific evidence details"
}
]
}To add custom cases:
- Generate case JSON using
lib/case-data.tsformat - Add to
sampleCasesarray inprisma/seed.ts - Run
npm run seed
GET /api/cases- List all casesGET /api/cases/[id]- Get case details with suspects and evidence
POST /api/sessions- Create investigation session- Body:
{ caseId: string, ragEnabled: boolean }
- Body:
POST /api/ai/interact- Get AI response- Body:
{ sessionId, caseId, prompt, ragEnabled } - Response includes: response text, timings, token usage, log ID
- Body:
- Case selection with RAG mode toggle
- Evidence inspection sidebar
- Suspect profile viewing
- Predefined investigation prompts
- Custom free-text prompts
- Real-time AI responses
- Vector embedding of all evidence using OpenAI embeddings
- Semantic similarity search via pgvector
- Injection of relevant evidence into LLM context
- Investigation history retrieval
- Response Timings: retrieval_time_ms, llm_response_time_ms, total_response_time_ms
- Token Usage: prompt_tokens, completion_tokens, total_tokens
- Hallucination Detection: boolean flag + reason field
- Correctness Scoring: optional 0-1 score field
- Auto Logging: all metrics captured per interaction
npm run prisma:studioAccess database GUI at http://localhost:5555
npm run build
npm start- Framework: Next.js 15
- Language: TypeScript
- Styling: Tailwind CSS
- Database: PostgreSQL with pgvector
- ORM: Prisma
- LLM: OpenAI API (GPT-4 Turbo)
- Embeddings: OpenAI text-embedding-3-small
This system enables evaluation of RAG's impact on:
- Decision accuracy in LLM recommendations
- Hallucination reduction with grounded context
- Contextual consistency through evidence retrieval
- Investigation workflow efficiency
The RAG toggle allows A/B comparison between standard LLM and retrieval-augmented approaches using identical case data and investigation scenarios.