🦊 GitLab AI — Agentic RAG Documentation Expert

A production-grade, multi-agent RAG chatbot that answers any question about GitLab by retrieving answers directly from the official GitLab documentation — grounded, cited, and hallucination-free. Built with CrewAI, ChromaDB, and local sentence embeddings. Total running cost: $0.00.

---

✨ Live Demo

streamlit run app.py

Open http://localhost:8501

The Streamlit UI showcases the full agentic pipeline with a recruiter-ready interface:

Section	What you see
Hero banner	Title, tagline, and tech badges
Feature cards	Six cards explaining the architecture before you start chatting
Architecture strip	Full pipeline flow as a single visual line
Sidebar	Pipeline diagram · Tech stack · System stats · Clickable sample questions
Chat	Orange user bubbles · Fox-avatar bot cards · Source chips · Pipeline route pill · Response time

---

🏗 Architecture

User Query
    │
    ▼
┌─────────────────────┐
│   Smart Router      │  < 4 words or ambiguous → Full pipeline
│                     │  Clear question (5+ words) → Simple pipeline
└─────────┬───────────┘
          │
    ┌─────┴──────────────────────────────────┐
    │  FULL PIPELINE (vague queries)         │
    │  Intent Classify ──┐  (async parallel) │
    │  Query Rewrite  ───┘  asyncio.gather() │
    └─────────────────────────────────────────┘
          │
          ▼
┌─────────────────────┐
│  Retriever Agent    │  semantic_search / filtered_search / multi_query_search
│  (CrewAI)           │  ChromaDB cosine similarity · RRF fusion
└─────────┬───────────┘
          │  heuristic validation (relevance ≥ 0.40)
          ▼
┌─────────────────────┐
│  Synthesiser Agent  │  Writes grounded answer with citations
│  (CrewAI)           │  Flags deprecated features · Never extrapolates
└─────────┬───────────┘
          │
          ▼
    Cited Answer + Source URLs

Key design decisions

• Async parallel pre-processing — intent classification and query rewriting run with asyncio.gather(), cutting pre-processing from ~8s to ~4s.
• Heuristic validation replaces a full LLM-based validator agent — chunks below 0.40 cosine relevance are dropped before the synthesiser sees them, saving 5–8s per query with no quality loss.
• Smart routing skips the full pipeline for clear, well-formed questions, reducing average latency by 30–50%.
• BGE asymmetric embeddings — query prefix "Represent this sentence for searching relevant passages: " is prepended to queries but not documents, improving retrieval accuracy over symmetric models.

---

🛠 Tech Stack

Layer	Technology	Notes
UI	Streamlit 1.35+	Custom CSS · Chat interface · Feature showcase
Agents	CrewAI	Sequential crew · @tool decorator
LLM	OpenRouter (free tier)	LiteLLM routing · openrouter/ prefix · 3× retry
Embeddings	BAAI/bge-small-en-v1.5	384-dim · local CPU · disk-cached
Vector DB	ChromaDB	Persistent local client · cosine distance
Doc processing	LangChain · tiktoken	Header-aware chunking · token counting
Config	Pydantic BaseSettings	.env → typed singleton
Logging	loguru	Rotating file logs

---

📁 Project Structure

git-guide/
├── app.py                          # Streamlit UI ← start here
├── config/
│   ├── settings.py                 # Pydantic settings singleton
│   └── llm_client.py               # OpenRouter client + factory functions
├── phase1_ingestion/               # One-time data pipeline
│   ├── run_ingestion.py            # Orchestrator
│   ├── scraper.py                  # Git sparse-checkout downloader
│   ├── chunker.py                  # Markdown → DocChunk
│   ├── embedder.py                 # Local BAAI/bge embeddings + cache
│   └── vector_store.py             # ChromaDB wrapper
├── phase2_agents/                  # Runtime agent system
│   ├── run_agents.py               # CLI entry point + routing logic
│   ├── crew.py                     # CrewAI crew definitions
│   ├── parallel_pipeline.py        # Async intent + rewrite
│   ├── agents/
│   │   ├── retriever.py            # Search specialist agent
│   │   └── synthesiser.py          # Answer writer agent
│   └── tools/
│       └── retrieval_tools.py      # 4 search tools (semantic/filtered/multi/id)
├── data/
│   ├── raw/gitlab-docs/            # Downloaded GitLab .md files
│   └── processed/                  # chunks.json · embedding_cache.json
├── vectorstore/chroma_db/          # ChromaDB local database
├── requirements.txt
└── .env                            # API keys (copy from .env.example)

---

🚀 Setup

1. Clone and install

git clone https://github.com/raghavgupta/git-guide.git
cd git-guide
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt

2. Configure environment

cp .env.example .env

Edit .env:

OPENROUTER_API_KEY=sk-or-v1-...   # Free key from https://openrouter.ai/keys
OPENROUTER_MODEL=meta-llama/llama-3.1-8b-instruct:free
EMBEDDING_MODEL=BAAI/bge-small-en-v1.5

3. Build the knowledge base (Phase 1)

This downloads the GitLab docs, chunks them, embeds them locally, and stores them in ChromaDB. Run once; subsequent runs use the embedding cache.

python -m phase1_ingestion.run_ingestion

Flag	Effect
(no flag)	Full pipeline
--reset	Wipe + re-index from scratch
--skip-scrape	Skip git clone, use existing raw files
--verify-only	Run 6 test queries without re-indexing

What happens: git sparse-checkout pulls only /doc (~80 MB, not the full 4 GB repo) → ~8,000 Markdown files → ~30,000 chunks → embedded locally → stored in ChromaDB.

4. Launch the Streamlit UI

streamlit run app.py

Open http://localhost:8501. Click a sample question in the sidebar or type your own.

4b. CLI alternative

# Interactive REPL
python -m phase2_agents.run_agents

# Single query
python -m phase2_agents.run_agents --test-query "how do I cache npm in CI?"

# Verbose (shows all agent reasoning)
python -m phase2_agents.run_agents --verbose

---

💬 Using the UI

Before you start chatting, the main area displays:

• A dark hero banner with tech badges
• Six feature cards explaining the system architecture
• A pipeline flow diagram

Once you send a message:

• The status widget expands and shows which pipeline route was chosen
• The answer appears in a bot card with:
  • 🦊 avatar
  • Full markdown answer
  • ⚡ / 🔄 pipeline pill (simple vs. full)
  • 📎 Clickable source chips linking directly to docs.gitlab.com
  • ⏱ Response time in seconds

Sidebar shortcuts — click any of the six sample questions to submit instantly.

---

💡 Example questions

How do I cache npm packages in GitLab CI?
What is the difference between stages and jobs?
How do I set up a Docker-in-Docker pipeline?
How do I protect a branch in GitLab?
What are GitLab CI/CD variables and how do I use them?
How do I run SAST security scanning in my pipeline?

---

💰 Cost breakdown

Component	Cost
LLM calls (OpenRouter free tier)	$0.00
Embeddings (local CPU, BAAI/bge-small)	$0.00
Vector database (ChromaDB local files)	$0.00
Scraping (public GitLab repo)	$0.00
Total	$0.00

---

🔍 How retrieval works

1. semantic_search — default tool; embeds the query and runs cosine similarity against all ~30,000 chunks.
2. filtered_search — same, plus metadata filters (section=ci, has_code=True, etc.) for precision retrieval.
3. multi_query_search — for ambiguous queries, the LLM generates 2–3 alternative phrasings, embeds all of them, and fuses results with Reciprocal Rank Fusion (RRF, k=60).
4. get_chunk_by_id — fetches a specific chunk by its ID for follow-up lookups.

Chunks below 0.40 cosine relevance are dropped before the synthesiser sees them.

---

🗺 Roadmap

• Multi-turn conversation memory
• Streaming token output in the UI
• Automatic re-indexing when GitLab docs update
• REST API wrapper (FastAPI)
• Docker Compose for one-command deployment

---

👤 Author

Raghav Gupta — built as a portfolio demonstration of production agentic RAG systems.

Stack: Python · CrewAI · ChromaDB · sentence-transformers · OpenRouter · Streamlit