🤖 RAG Chatbot

A production-quality, fully open-source Retrieval-Augmented Generation (RAG) chatbot.

✨ Features

• 100% Open Source: No proprietary dependencies or SaaS tools
• Model Agnostic: Works with Claude, Gemini, GPT, or local Ollama models
• Grounded Answers: Strict prompts prevent hallucination
• Citation Support: Every answer includes source references
• Local Embeddings: BAAI/bge-base-en-v1.5 runs on your machine
• Multiple Formats: Ingest PDF, Markdown, and plain text
• Production Ready: FastAPI, type hints, logging, and clean architecture

🏗️ Architecture

┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│  Document       │────▶│  Embedding       │────▶│  ChromaDB       │
│  Ingestion      │     │  Service         │     │  Vector Store   │
└─────────────────┘     └──────────────────┘     └────────┬────────┘
                                                          │
┌─────────────────┐     ┌──────────────────┐     ┌────────▼────────┐
│  FastAPI        │────▶│  RAG Pipeline    │────▶│  Retriever      │
│  /chat          │     │                  │     │                 │
└─────────────────┘     └────────┬─────────┘     └─────────────────┘
                                 │
                        ┌────────▼─────────┐
                        │  LLM Provider    │
                        │  (Pluggable)     │
                        └──────────────────┘

🚀 Quick Start

1. Installation

# Clone the repository
cd C:\Users\dhev0\.gemini\antigravity\scratch\rag-chatbot

# Create virtual environment
python -m venv venv
venv\Scripts\activate  # Windows
# source venv/bin/activate  # Linux/Mac

# Install dependencies
pip install -e .

2. Configuration

# Copy example config
copy .env.example .env

# Edit .env with your settings
# For local operation (recommended), use Ollama:
# LLM_PROVIDER=ollama
# OLLAMA_MODEL=llama3.2

3. Start Ollama (for local LLM)

# Install Ollama from https://ollama.ai/download
# Then pull a model:
ollama pull llama3.2

# Ollama runs automatically after installation

4. Ingest Documents

# Run demo ingestion
python examples/ingest_documents.py --demo

# Or ingest your own files
python examples/ingest_documents.py ./docs/manual.pdf ./docs/guide.md

5. Start the API

python -m uvicorn src.rag.main:app --reload --port 8000

6. Query the System

# Via curl
curl -X POST http://localhost:8000/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{"query": "What is RAG?"}'

# Via example script
python examples/chat_example.py "What is RAG?"

# Interactive mode
python examples/chat_example.py

📡 API Endpoints

Endpoint	Method	Description
/api/v1/health	GET	Health check
/api/v1/stats	GET	Vector store statistics
/api/v1/ingest	POST	Ingest documents
/api/v1/chat	POST	RAG query
/api/v1/clear	DELETE	Clear vector store
/docs	GET	Interactive API docs

Ingest Request

{
  "file_path": "./documents/manual.pdf"
}
// OR
{
  "content": "Your document text here...",
  "source": "document_name"
}

Chat Request

{
  "query": "What is RAG?",
  "top_k": 5,
  "include_context": true
}

Chat Response

{
  "answer": "RAG (Retrieval-Augmented Generation) is a technique that... [Source: DOC-1]",
  "citations": [
    {"source_id": "DOC-1", "source_name": "rag_guide.md", "page": null}
  ],
  "context_chunks": [
    {
      "chunk_id": "rag_guide_abc123",
      "source": "rag_guide.md",
      "score": 0.89,
      "content_preview": "RAG combines retrieval with generation..."
    }
  ],
  "query": "What is RAG?",
  "llm_provider": "Ollama"
}

⚙️ Configuration

All settings via environment variables (.env file):

Variable	Default	Description
LLM_PROVIDER	ollama	ollama, claude, gemini, openai
OLLAMA_MODEL	llama3.2	Model for Ollama
EMBEDDING_MODEL	BAAI/bge-base-en-v1.5	HuggingFace embedding model
CHROMA_PERSIST_PATH	./data/chromadb	Vector store location
CHUNK_SIZE	1500	Chunk size in characters
CHUNK_OVERLAP	200	Overlap between chunks
TOP_K_RESULTS	5	Number of chunks to retrieve

🔧 LLM Provider Setup

Ollama (Recommended - Fully Local)

# Install from https://ollama.ai/download
ollama pull llama3.2
# That's it! Server runs automatically

Claude

# Set in .env
LLM_PROVIDER=claude
ANTHROPIC_API_KEY=your_key_here
CLAUDE_MODEL=claude-3-haiku-20240307

Gemini

# Set in .env
LLM_PROVIDER=gemini
GOOGLE_API_KEY=your_key_here
GEMINI_MODEL=gemini-1.5-flash

OpenAI

# Set in .env
LLM_PROVIDER=openai
OPENAI_API_KEY=your_key_here
OPENAI_MODEL=gpt-4o-mini

🔒 Hallucination Prevention

This system uses strict grounding to prevent hallucinations:

1. System Prompt: Explicitly forbids using external knowledge
2. Citation Requirement: Every claim must reference a source
3. "I Don't Know": Model instructed to admit when context is insufficient
4. Metadata Tracking: Full source traceability for verification

The grounding prompt is defined in src/rag/rag/prompts.py and can be customized.

🧩 Extending the System

Add a New Document Format

# src/rag/ingestion/loaders.py
class DocxLoader(BaseLoader):
    def load(self, source: str | Path) -> list[Document]:
        # Your implementation
        pass

Add a New LLM Provider

# src/rag/llm/my_provider.py
class MyLLM(BaseLLM):
    @property
    def provider_name(self) -> str:
        return "MyProvider"
    
    async def generate(self, system_prompt: str, user_message: str) -> str:
        # Your implementation
        pass

Switch Vector Store

The system uses ChromaDB, but you can implement a new vector store:

# src/rag/vectorstore/qdrant.py
class QdrantVectorStore:
    def add_chunks(self, chunks: list[Chunk]) -> int: ...
    def search(self, query: str, top_k: int) -> list[SearchResult]: ...

📁 Project Structure

rag-chatbot/
├── src/rag/
│   ├── api/            # FastAPI routes and schemas
│   ├── embeddings/     # Local embedding service
│   ├── ingestion/      # Document loaders, cleaners, chunkers
│   ├── llm/            # LLM provider abstraction
│   ├── rag/            # Core RAG pipeline
│   ├── vectorstore/    # ChromaDB integration
│   ├── config.py       # Configuration management
│   └── main.py         # FastAPI application
├── examples/           # Usage examples
├── tests/              # Test suite
├── pyproject.toml      # Python dependencies
└── .env.example        # Configuration template

🧪 Testing

# Run tests
pip install -e ".[dev]"
python -m pytest tests/ -v

# Type checking
python -m mypy src/

📄 License

MIT License - see LICENSE for details.

🙏 Acknowledgments

• ChromaDB - Open-source vector database
• sentence-transformers - Local embeddings
• Ollama - Local LLM inference
• FastAPI - Modern API framework