An advanced, locally-hosted ecosystem for transforming unstructured data into interactive, semantically-deduplicated knowledge graphs. KGB leverages the power of local LLMs (via Ollama) and Graph Databases (Neo4j) to provide a private, high-precision knowledge discovery platform.
KGB is built on a modern, asynchronous stack designed for high throughput and local privacy.
- Core Engine (FastAPI): Orchestrates the extraction pipeline, managing concurrency and state.
- Graph Intelligence (Neo4j): Stores complex network data with idempotent Cypher operations.
- Metadata Layer (SQLite + SQLAlchemy): Tracks project settings, extraction history, and audit trails.
- LLM Integration (Ollama): Executes downloaded local models for entity extraction and embedding generation. Cloud model tags are rejected.
- Visualization (vis-network): A physics-based frontend for real-time graph exploration.
Document (PDF/TXT) → Asynchronous Chunking → Two-Pass LLM Extraction → Semantic Deduplication → Idempotent Neo4j Writes → Real-time UI Update
Unlike naive extraction, KGB uses a multi-stage process to minimize hallucinations and maximize link density:
- Pass 1 (Entity Identification): The LLM identifies unique entities based on a project-specific schema (e.g., Person, Org, Technology).
- Pass 2 (Relationship Mapping): A second targeted prompt maps relationships only between the entities discovered in Pass 1, ensuring high-fidelity edges.
KGB solves the "entity explosion" problem by merging similar concepts:
- Vector Embeddings: Generates
nomic-embed-textvectors for every entity name. - Hybrid Similarity: Combines Cosine Similarity (semantic) with Jaccard Similarity (token overlap) to intelligently cluster and merge nodes (e.g., "Apple" and "Apple Inc.").
- Cumulative Knowledge: Descriptions from merged entities are appended, preserving contextual richness across different source documents.
Utilizing Server-Sent Events (SSE) and asynchronous generators, the UI updates in real-time as each text chunk is processed. You can watch the graph materialize as the AI reads the document.
The fastest and most secure way to run KGB. A single command brings up the whole stack — API, Neo4j, and Ollama — with a hardened configuration.
cp .env.docker.example .env.docker
# Set a strong Neo4j password, e.g.: openssl rand -base64 24
$EDITOR .env.docker.env.docker is git-ignored — secrets never enter the image, the repo, or your shell history.
docker-compose --env-file .env.docker up -d --buildThe stack is ready when all services report healthy:
docker-compose --env-file .env.docker psThe first launch also pulls your configured LLM and embedding models into the Ollama container. Track progress with:
docker-compose --env-file .env.docker logs -f ollama-initThen open http://localhost:8000.
# Stop the stack (data persists in named volumes)
docker-compose --env-file .env.docker down
# Stop and wipe all data (⚠️ destroys graphs, projects, and downloaded models)
docker-compose --env-file .env.docker down -vThe deployment is hardened end-to-end:
| Layer | Hardening |
|---|---|
| Network isolation | Neo4j and Ollama run on an internal network with no published ports; only the API is exposed, bound to 127.0.0.1 by default. Neo4j has no internet egress at all. |
| Container privileges | Every container runs non-root, drops all Linux capabilities, and sets no-new-privileges. |
| Filesystem | The API container has a read-only root filesystem with a writable /tmp tmpfs (noexec) and a dedicated volume for SQLite. |
| Supply chain | Base images are version/digest-pinned; the app image is a minimal multi-stage build with no package manager, shell utilities, or debug tools. |
| Secrets | Credentials are injected at runtime from the git-ignored .env.docker; ${NEO4J_PASSWORD} is mandatory — the stack refuses to start without it. |
| Application | CORS is restricted to explicit origins (CORS_ORIGINS), responses carry a restrictive Content Security Policy and other security headers, Cypher identifiers (labels/relationship types) are sanitized against injection, uploads and extraction text are size-capped, and CSV exports are escaped against formula injection. |
| Availability | Healthchecks gate startup ordering; CPU/memory limits cap every container. |
Exposing KGB beyond your machine: set APP_BIND=0.0.0.0 in .env.docker only behind a TLS-terminating reverse proxy with access control (e.g., Caddy/nginx + auth), and add your public origin to CORS_ORIGINS.
Browsing Neo4j from your workstation: don't publish its ports — tunnel instead: ssh -L 7474:localhost:7474 -L 7687:localhost:7687 <host>, then open http://localhost:7474 (bolt://localhost:7687).
- Python 3.10+
- Neo4j Desktop or Neo4j Community Server.
- Ollama running locally.
git clone https://github.com/your-repo/KGB.git
cd KGB
python -m venv venv
source venv/bin/activate # venv\Scripts\activate on Windows
pip install -r requirements.txt# Neo4j
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=your_secure_password
# Ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2:3b
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
OLLAMA_NO_CLOUD=1
# Security (optional) — comma-separated origins allowed to call the API.
# Defaults to http://localhost:8000,http://127.0.0.1:8000
# CORS_ORIGINS=http://localhost:8000ollama pull llama3.2:3b
ollama pull nomic-embed-textOLLAMA_NO_CLOUD must be applied to the Ollama server process, not only
the KGB process. For a shell-managed server, start or restart it with:
OLLAMA_NO_CLOUD=1 ollama serveFor the macOS app, Linux systemd service, or Windows app, configure the server
environment as described in Ollama's documentation and restart Ollama. The
Docker deployment sets this variable on the Ollama server automatically. KGB
also refuses model tags ending in -cloud as a second guardrail.
- Initialize Databases:
python migrate_db.py
- Start the Server:
python -m app.main
- Explore: Navigate to
http://localhost:8000.
├── app/
│ ├── routers/ # API Gateway (Project, Extraction, Graph endpoints)
│ ├── services/ # Business Logic (Neo4j, Ollama, Entity Resolver)
│ ├── utils/ # Utilities (PDF Parsing, Text Chunking)
│ ├── models.py # SQLAlchemy ORM Models
│ └── schemas.py # Pydantic Data Validation
├── frontend/ # Single-Page Application (Vanilla JS)
├── data/ # SQLite Persistence
├── Dockerfile # Hardened multi-stage container image
├── docker-compose.yml # Secured full-stack deployment (API + Neo4j + Ollama)
├── .env.docker.example # Template for Docker deployment secrets/config
├── LICENSE # MIT License
└── repair_project.py # System Recovery Utility
KGB is designed for local execution when it is paired with a downloaded local
Ollama model and an Ollama server started with OLLAMA_NO_CLOUD=1.
- KGB rejects Ollama model tags ending in
-cloud. - The Docker deployment disables Ollama cloud features on the server.
- Manual installations must configure
OLLAMA_NO_CLOUD=1on the Ollama server itself and restart it; setting it only for KGB does not reconfigure an already-running Ollama process. - Keep the API bound to
127.0.0.1. There is currently no user authentication, so do not expose it directly to a LAN or the internet. - Do not treat the current prototype as approved for confidential or regulated documents without an independent review of your complete deployment.
- Treat all AI-generated entities and relationships as unverified leads. Check them against the source documents before relying on them. KGB is not investment advice or a production-grade research database.
- The Docker deployment adds defense-in-depth: internal-only networks for Neo4j and Ollama, non-root containers with dropped capabilities, a read-only API filesystem, and loopback-only port binding. See the Docker security model.
Distributed under the MIT License. You are free to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of this software — attribution required, no warranty provided.