Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,6 @@ dist-ssr
# other secret files
.env.local
.env.langfuse

# local models config
local-models.json
44 changes: 44 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,50 @@ npm run dev
NGROK_URL="<NGROK URL>" # Optional local Supabase Storage tunnel for provider-readable signed URLs
```

### 3. Optional Local LLM Models

To add your own local models (LM Studio, Ollama, LiteLLM, etc.) alongside the cloud models:

1. Start one or more local OpenAI-compatible servers (for example LM Studio on port `1234`).
2. Copy the example catalog and edit entries to match your servers:

```bash
cp local-models.json.example local-models.json
```

Each entry needs an `id`, `name`, `description`, and `baseUrl`. The `id` must match what your server exposes.

Example:

```json
{
"id": "qwen2.5-vl-7b-instruct",
"name": "Qwen2.5 VL 7B",
"description": "Local VLM",
"baseUrl": "http://localhost:1234",
"supportsTools": true,
"supportsVision": true,
"useForAux": true
},
{
"id": "deepseek-v4-flash",
"name": "DeepSeek v4 Flash",
"description": "Third Party LLM",
"baseUrl": "https://api.deepseek.com",
"apiKey": "<Test DeepSeek API Key>",
"supportsTools": true,
"supportsThinking": true,
"supportsVision": false,
"useForAux": false
}
```

3. Optionally set `OPENROUTER_API_KEY="local"` in `.env.local` if your local server expects an API key header (many accept any value). **Note:** this value is also used for OpenRouter cloud model calls, so only use a dummy value if you are not using any OpenRouter-hosted models.

4. Restart `npm run dev`. Models with a `baseUrl` appear in the parametric model picker.

Cloud models (Gemini, Claude, OpenAI, Z-AI) keep using their existing cloud routes. Only ids listed in `local-models.json` with a `baseUrl` are sent to that model's server.

## 🌐 Setting Up ngrok for Local Development

CADAM uses public URLs for provider callbacks and local signed storage URLs:
Expand Down
12 changes: 12 additions & 0 deletions local-models.json.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"id": "qwen2.5-vl-7b-instruct",
"name": "Qwen2.5 VL 7B",
"description": "Local VLM",
"baseUrl": "http://localhost:1234",
"supportsTools": true,
"supportsThinking": false,
"supportsVision": true,
"useForAux": true
}
]
Comment thread
nathan-t4 marked this conversation as resolved.
Loading