|
| 1 | +# Your first request |
| 2 | + |
| 3 | +**Problem** — You have a model name and a prompt, and three providers |
| 4 | +with three SDKs, three env vars, and three wire formats between you and |
| 5 | +a response. lm15 gives you one `Request` type and a router that turns |
| 6 | +the model string into the right provider — and can tell you exactly how |
| 7 | +it decided. |
| 8 | + |
| 9 | +## Recipe |
| 10 | + |
| 11 | +Keys first. lm15 reads nothing implicitly; you put keys in the |
| 12 | +environment, the router finds them there. This loader searches the |
| 13 | +current directory and its parents for a `.env` file: |
| 14 | + |
| 15 | +```python |
| 16 | +import asyncio |
| 17 | +import os |
| 18 | +from pathlib import Path |
| 19 | + |
| 20 | +from lm15 import ( |
| 21 | + AnthropicLM, AsyncLMRouter, GeminiLM, LMRouter, Message, OpenAILM, Request, |
| 22 | +) |
| 23 | + |
| 24 | +def load_env(filename=".env"): |
| 25 | + for directory in (Path.cwd(), *Path.cwd().parents): |
| 26 | + path = directory / filename |
| 27 | + if path.exists(): |
| 28 | + for line in path.read_text().splitlines(): |
| 29 | + line = line.strip().removeprefix("export ") |
| 30 | + if line and not line.startswith("#") and "=" in line: |
| 31 | + key, _, value = line.partition("=") |
| 32 | + os.environ.setdefault(key.strip(), value.strip().strip("\"'")) |
| 33 | + return path |
| 34 | + return None |
| 35 | + |
| 36 | +print(load_env()) |
| 37 | +``` |
| 38 | +```output |
| 39 | +/home/maxime/Projects/lm15-dev/.env |
| 40 | +``` |
| 41 | + |
| 42 | +(From a shell, the one-liner is `export $(grep -v "^#" ../.env | xargs)`.) |
| 43 | + |
| 44 | +Now the request. `LMRouter` is the front door: it maps |
| 45 | +`"gpt-4.1-mini"` to `OpenAILM`, builds the LM with the key from |
| 46 | +`$OPENAI_API_KEY`, and forwards the call. |
| 47 | + |
| 48 | +```python |
| 49 | +router = LMRouter() |
| 50 | +response = router.complete( |
| 51 | + Request(model="gpt-4.1-mini", messages=(Message.user("Say hello in five words."),)) |
| 52 | +) |
| 53 | +print(response.text) |
| 54 | +``` |
| 55 | +```output |
| 56 | +Hello! Hope you're doing well. |
| 57 | +``` |
| 58 | + |
| 59 | +`response` is a frozen dataclass, not a provider blob. Print it and you |
| 60 | +see everything lm15 normalized: |
| 61 | + |
| 62 | +```python |
| 63 | +print(response) |
| 64 | +``` |
| 65 | +```output |
| 66 | +Response( |
| 67 | + text="Hello! Hope you're doing well.", |
| 68 | + model='gpt-4.1-mini-2025-04-14', |
| 69 | + finish_reason='stop', |
| 70 | + usage=Usage(input_tokens=13, output_tokens=8, …), |
| 71 | + id='resp_09b0…', |
| 72 | + provider_data=<dict: 35 keys>, |
| 73 | +) |
| 74 | +``` |
| 75 | + |
| 76 | +How did the router pick OpenAI? Ask it. `resolve()` is pure — no |
| 77 | +network, no key values read — and its return value is the explanation: |
| 78 | + |
| 79 | +```python |
| 80 | +res = router.resolve("gpt-4.1-mini") |
| 81 | +print(res) |
| 82 | +print(res.provider, res.adapter, res.source, res.env_key) |
| 83 | +``` |
| 84 | +```output |
| 85 | +'gpt-4.1-mini' -> provider 'openai' (OpenAILM); via built-in rule prefix='gpt-' — OpenAI GPT family (Responses API; use openai_chat: for Chat Completions); wire model 'gpt-4.1-mini'; key from $OPENAI_API_KEY. |
| 86 | +openai OpenAILM rule OPENAI_API_KEY |
| 87 | +``` |
| 88 | + |
| 89 | +A `provider:` prefix bypasses the rule table and is always unambiguous: |
| 90 | + |
| 91 | +```python |
| 92 | +print(router.resolve("anthropic:claude-sonnet-4-5")) |
| 93 | +``` |
| 94 | +```output |
| 95 | +'anthropic:claude-sonnet-4-5' -> provider 'anthropic' (AnthropicLM); via explicit provider prefix; wire model 'claude-sonnet-4-5'; key from $ANTHROPIC_API_KEY. |
| 96 | +``` |
| 97 | + |
| 98 | +The router never guesses. A string it cannot place raises |
| 99 | +`UnknownModelError` with the fixes spelled out: |
| 100 | + |
| 101 | +```python |
| 102 | +router.resolve("qwen3.5:0.8b") |
| 103 | +``` |
| 104 | +```output |
| 105 | +Traceback (most recent call last): |
| 106 | + … |
| 107 | +lm15.router.UnknownModelError: could not route model 'qwen3.5:0.8b': no provider prefix, no catalog supplied, and none of the 6 built-in rules matched. Use an explicit provider prefix, e.g. "anthropic:qwen3.5:0.8b" (known providers: anthropic, claude-code, gemini, openai, openai-codex, openai_chat). Or pass a model catalog: … |
| 108 | +``` |
| 109 | + |
| 110 | +The router is sugar, not a layer. The direct LM classes are first-class |
| 111 | +and take the same `Request`. The same prompt, three providers: |
| 112 | + |
| 113 | +```python |
| 114 | +prompt = (Message.user("Say hello in five words."),) |
| 115 | + |
| 116 | +openai_lm = OpenAILM(api_key=os.environ["OPENAI_API_KEY"]) |
| 117 | +print(openai_lm.complete(Request(model="gpt-4.1-mini", messages=prompt)).text) |
| 118 | + |
| 119 | +anthropic_lm = AnthropicLM(api_key=os.environ["ANTHROPIC_API_KEY"]) |
| 120 | +print(anthropic_lm.complete(Request(model="claude-sonnet-4-5", messages=prompt)).text) |
| 121 | + |
| 122 | +gemini_lm = GeminiLM(api_key=os.environ["GEMINI_API_KEY"]) |
| 123 | +print(gemini_lm.complete(Request(model="gemini-3-flash-preview", messages=prompt)).text) |
| 124 | +``` |
| 125 | +```output |
| 126 | +Hello there! Hope you're well! |
| 127 | +Hello, how are you today? |
| 128 | +Hello, how are you doing? |
| 129 | +``` |
| 130 | + |
| 131 | +`AsyncLMRouter` is the async mirror — `complete` is awaitable, |
| 132 | +`resolve()` stays sync because it is pure: |
| 133 | + |
| 134 | +```python |
| 135 | +async def main(): |
| 136 | + arouter = AsyncLMRouter() |
| 137 | + response = await arouter.complete( |
| 138 | + Request(model="gemini-3-flash-preview", messages=prompt) |
| 139 | + ) |
| 140 | + print(response.text) |
| 141 | + |
| 142 | +asyncio.run(main()) |
| 143 | +``` |
| 144 | +```output |
| 145 | +Hello, how are you today? |
| 146 | +``` |
| 147 | + |
| 148 | +## How it works |
| 149 | + |
| 150 | +A `Request` is a frozen dataclass: a model string and a tuple of |
| 151 | +`Message` objects (plus optional `Config`, tools, and more in later |
| 152 | +recipes). `router.complete(request)` does three things: resolve the |
| 153 | +model string, build (and cache) the provider LM, and call its |
| 154 | +`complete()`. Resolution walks three fixed rungs — explicit |
| 155 | +`provider:` prefix, optional catalog, built-in prefix rules — first |
| 156 | +match wins, no fallback chains. The full grammar and the catalog rung |
| 157 | +are in [Using the router](../using-the-router.md). |
| 158 | + |
| 159 | +Credentials follow the provider's manifest: `OpenAILM` declares |
| 160 | +`OPENAI_API_KEY`, and that is the only place the router looks unless |
| 161 | +you pass `RouterConfig(api_keys=...)`. A missing key raises |
| 162 | +`MissingCredentialError` at `lm()` time, not a 401 at request time. |
| 163 | +`resolve()` records *which* env var would be read, never its value. |
| 164 | + |
| 165 | +Both paths — router and direct LM — produce the identical `Request` |
| 166 | +and identical wire bytes. lm15 deliberately does not retry, does not |
| 167 | +pool connections behind your back, and does not pick a model for you: |
| 168 | +one call in, one typed `Response` out. |
| 169 | + |
| 170 | +## Variations |
| 171 | + |
| 172 | +- **Keep the LM, drop the router.** `router.lm("gpt-4.1-mini")` |
| 173 | + returns a plain `OpenAILM`; configure it once and never resolve |
| 174 | + again. |
| 175 | +- **Direct LMs are for configuration, not preference.** Reach for them |
| 176 | + when you need a custom `base_url` or compat preset (ollama, vLLM — |
| 177 | + recipe [14](14-local-and-compatible-servers.md)) or when you are a |
| 178 | + library taking an LM object from your caller. |
| 179 | +- **Async direct LMs exist too**: `AsyncOpenAILM`, `AsyncAnthropicLM`, |
| 180 | + `AsyncGeminiLM`, same constructors, awaitable `complete`. |
| 181 | +- **OpenAI has two providers.** Bare `gpt-` routes to the Responses |
| 182 | + API (`OpenAILM`); `"openai_chat:gpt-4.1-mini"` selects Chat |
| 183 | + Completions (`OpenAIChatLM`). The resolve output above says so. |
| 184 | + |
| 185 | +## See also |
| 186 | + |
| 187 | +- [02 — Multi-turn conversations](02-conversations.md) |
| 188 | +- [04 — Controlling generation](04-generation-config.md) |
| 189 | +- [05 — Streaming](05-streaming.md) |
| 190 | +- [Using the router](../using-the-router.md) — grammar, catalogs, credentials |
| 191 | +- [15 — Errors, retries & testing](15-errors-and-testing.md) |
0 commit comments