Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ node_modules
# Testing
coverage

# Generated UI evidence (scripts/orca-ui-evidence.py)
/orca-evidence/

# Turbo
.turbo

Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,6 +809,7 @@ pnpm dev
- http://remotion-demo.json-render.localhost:1355 - Remotion Video Example
- Chat Example: run `pnpm dev` in `examples/chat`
- [Experimental Jev composition](https://json-render.dev/docs/jev): use `experimental_composeSpec` and `experimental_createEvaluator` from core with your own catalog, or select **Jev (Experimental)** in `/playground`. Unreleased; source-build instructions are in the guide.
- [OrcaRouter](https://json-render.dev/docs/orcarouter): select **OrcaRouter** in `/playground` to generate through [OrcaRouter](https://www.orcarouter.ai), an OpenAI-compatible AI gateway that routes many providers behind one endpoint. Connect with an API key or with **Connect with OrcaRouter** (OAuth 2.0 + PKCE); model choices come from the live OrcaRouter catalog.
- Svelte Example: run `pnpm dev` in `examples/svelte` or `examples/svelte-chat`
- Vue Example: run `pnpm dev` in `examples/vue`
- Vite Renderers (React + Vue + Svelte + Solid): run `pnpm dev` in `examples/vite-renderers`
Expand Down
16 changes: 16 additions & 0 deletions apps/web/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,22 @@
# For local development, get your key from https://vercel.com/ai-gateway
AI_GATEWAY_API_KEY=

# OrcaRouter
# OpenAI-compatible AI gateway. Either paste an API key here, or open the
# playground and use "Connect with OrcaRouter" (OAuth 2.0 + PKCE), which
# writes the issued key into this file for you.
# Keys look like sk-orca-...; never commit a real key.
ORCAROUTER_API_KEY=

# OrcaRouter endpoint overrides (all optional)
# Inference and model catalog base. Default: https://api.orcarouter.ai/v1
# ORCA_BASE_URL=https://api.orcarouter.ai/v1
# ORCA_API_BASE_URL=https://api.orcarouter.ai/v1
# OAuth 2.0 authorization origin. Default: https://www.orcarouter.ai
# ORCA_AUTH_BASE_URL=https://www.orcarouter.ai
# Point the credential store at a different env file (defaults to apps/web/.env.local)
# ORCAROUTER_ENV_FILE=

# Dedicated AI Gateway key for the experimental Jev playground option
# Required locally and on Vercel; no fallback to AI_GATEWAY_API_KEY
JEV_AI_GATEWAY_API_KEY=
Expand Down
258 changes: 258 additions & 0 deletions apps/web/app/(main)/docs/orcarouter/page.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,258 @@
import { pageMetadata } from "@/lib/page-metadata";
export const metadata = pageMetadata("docs/orcarouter");

# OrcaRouter

[OrcaRouter](https://www.orcarouter.ai) is an OpenAI-compatible AI gateway that routes many providers behind one endpoint. The json-render web app can route playground and docs-chat generation through it, and `@json-render/core` exports the pieces you need to do the same in your own app.

## Authentication

Two explicit choices are offered, and neither replaces the other:

<table>
<thead>
<tr>
<th>Choice</th>
<th>Label</th>
<th>How the credential is obtained</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>orcarouter</code>
</td>
<td>OrcaRouter - API</td>
<td>
Paste an existing <code>sk-orca-…</code> key from your OrcaRouter
console.
</td>
</tr>
<tr>
<td>
<code>orcarouter-oauth</code>
</td>
<td>OrcaRouter - Auth</td>
<td>
Sign in with an OrcaRouter account over OAuth 2.0 + PKCE. No client
secret and no pre-registered redirect URI.
</td>
</tr>
</tbody>
</table>

Both paths produce the same thing — a normal OrcaRouter API key that belongs to the user, is billed to their account, and can be revoked at any time — so inference and model discovery never need to know which one was used.

### Where the key is stored

The web app writes it to the same place it already keeps provider secrets: `apps/web/.env.local`, which is git-ignored. Set `ORCAROUTER_ENV_FILE` to point at a writable path in a deployment where the app directory is read-only. If the write fails, the key is still used for the lifetime of the process and the panel says so.

## Configuration

<table>
<thead>
<tr>
<th>Variable</th>
<th>Purpose</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>ORCAROUTER_API_KEY</code>
</td>
<td>
An existing OrcaRouter key. Equivalent to pasting one in the panel.
</td>
<td>—</td>
</tr>
<tr>
<td>
<code>ORCA_AUTH_BASE_URL</code>
</td>
<td>
Origin for <code>/auth</code> and the code exchange.
</td>
<td>
<code>https://www.orcarouter.ai</code>
</td>
</tr>
<tr>
<td>
<code>ORCA_API_BASE_URL</code>
</td>
<td>
Origin for <code>/v1/models</code> and inference.
</td>
<td>
<code>https://api.orcarouter.ai</code>
</td>
</tr>
<tr>
<td>
<code>ORCA_BASE_URL</code>
</td>
<td>One origin serving both roles, for self-hosted deployments.</td>
<td>—</td>
</tr>
</tbody>
</table>

Authentication and inference are different origins. Do not derive one from the other by swapping a hostname or appending `/v1`: `https://api.orcarouter.ai/v1/auth/keys` is a 404. Explicit per-role overrides win over `ORCA_BASE_URL`. Remote origins must use HTTPS; plain HTTP is accepted only for loopback.

## Model discovery

The model control is a dropdown built from the live catalog at `GET https://api.orcarouter.ai/v1/models`, requested with the user's key so it reflects the models their workspace can actually call. The key stays on the server; the browser receives only model ids and minimal metadata.

Options are filtered per capability, and a model is only offered when the catalog metadata proves it fits:

<table>
<thead>
<tr>
<th>Entry point</th>
<th>Filter</th>
</tr>
</thead>
<tbody>
<tr>
<td>Text chat</td>
<td>
<code>?capability=chat</code>, and <code>supported_endpoint_types</code>{" "}
must include <code>openai</code>, <code>anthropic</code>,{" "}
<code>gemini</code> or <code>openai-response</code>, excluding
image-generation, video and rerank families.
</td>
</tr>
<tr>
<td>Image / audio / video input</td>
<td>
The chat filter, plus an explicit{" "}
<code>architecture.input_modalities</code> entry for each modality the
request uploads. A model with no <code>architecture</code> block is
never treated as multimodal.
</td>
</tr>
<tr>
<td>Embedding</td>
<td>
<code>?capability=embedding</code>, or an <code>embeddings</code>{" "}
endpoint type.
</td>
</tr>
<tr>
<td>Image generation</td>
<td>
<code>?capability=image</code>, or an <code>image-generation</code>{" "}
endpoint type.
</td>
</tr>
<tr>
<td>Video / rerank</td>
<td>
An <code>openai-video</code> or <code>jina-rerank</code> endpoint type.
</td>
</tr>
</tbody>
</table>

Changing the provider, the capability, or the attachments recomputes the options. A selection that is no longer compatible is cleared instead of being silently kept.

If the catalog cannot be reached, the control falls back to a small, verified seed list and is labelled degraded. It never degrades into a free-text model field.

## Using the provider in your own app

```typescript
import {
createOrcaApiKeySource,
createOrcaPkceSource,
OrcaCredentialStore,
fetchOrcaCatalog,
resolveOrcarouterOrigins,
selectOrcaModels,
} from "@json-render/core";

const origins = resolveOrcarouterOrigins();

// Either entry point produces the same credential shape.
const source = createOrcaApiKeySource(process.env.ORCAROUTER_API_KEY!);
const credential = await source.acquire();

const store = new OrcaCredentialStore();
store.setCredential(credential);

// Discovery and inference consume only the key.
const catalog = await fetchOrcaCatalog({
origins,
apiKey: store.getUsableCredential()!.apiKey,
capability: "chat",
});
const chatModels = selectOrcaModels(catalog, { capability: "chat" });
```

### The connect flow

```typescript
const source = createOrcaPkceSource({
origins,
appName: "My Tool",
flow: "loopback", // or "oob" when you cannot listen on 127.0.0.1
createCodeReceiver: async () => startLoopbackListener(),
openBrowser: (url) => open(url),
});
const credential = await source.acquire();
```

The verifier is generated fresh from a cryptographic RNG for every attempt, never leaves the process, and never appears in a URL, log or error message. The authorize URL always carries `code_challenge_method=S256`.

## Credential lifecycle

A PKCE-issued key is a durable API key, not a refresh token. There is no refresh grant to call, and the key is reused until OrcaRouter revokes it — re-authorizing on every launch would hit the limit of ten PKCE-issued keys per user per day.

A `401` from the relay is terminal. The exact account and credential generation that made the rejected request is marked `needsReauth` and the user is asked to connect again. The stored secret is kept until a replacement succeeds, so a transient failure cannot lose the account. A late failure from an old request never marks a newly reconnected credential as broken.

Users can revoke access at any time from [the OrcaRouter console](https://www.orcarouter.ai/console/authorized-apps).

## Provider evidence

<table>
<thead>
<tr>
<th>Item</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>OpenAI-compatible inference</td>
<td>
<code>https://api.orcarouter.ai/v1/chat/completions</code>
</td>
</tr>
<tr>
<td>Model catalog</td>
<td>
<code>https://api.orcarouter.ai/v1/models</code> (Bearer required)
</td>
</tr>
<tr>
<td>Authorization and exchange</td>
<td>
<code>https://www.orcarouter.ai/auth</code>,{" "}
<code>https://www.orcarouter.ai/api/v1/auth/keys</code>
</td>
</tr>
<tr>
<td>Discovery document</td>
<td>
<code>https://www.orcarouter.ai/.well-known/openid-configuration</code>
</td>
</tr>
<tr>
<td>Revocation and account management</td>
<td>
<code>https://www.orcarouter.ai/console/authorized-apps</code>
</td>
</tr>
</tbody>
</table>
54 changes: 52 additions & 2 deletions apps/web/app/api/docs-chat/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,17 @@ import { convertToModelMessages, stepCountIs, streamText } from "ai";
import type { ModelMessage, UIMessage } from "ai";
import { createBashTool } from "bash-tool";
import { headers } from "next/headers";
import { selectOrcaModels } from "@json-render/core";
import { allDocsPages } from "@/lib/docs-navigation";
import { mdxToCleanMarkdown } from "@/lib/mdx-to-markdown";
import { minuteRateLimit, dailyRateLimit } from "@/lib/rate-limit";
import { discoverOrcaModels } from "@/lib/orcarouter/connect-session";
import {
ORCAROUTER_DEFAULT_MODEL,
OrcaRouterNotConnectedError,
createOrcarouterTransport,
isOrcarouterProvider,
} from "@/lib/orcarouter/provider";

export const maxDuration = 60;

Expand All @@ -21,6 +29,8 @@ Skills: json-render ships AI agent skills that teach coding agents how to use ea

Experimental Jev composition: core exports experimental_composeSpec and experimental_createEvaluator for app-owned catalogs/candidates through Vercel AI Gateway. See /docs/jev for availability, source-build setup, and limits; do not assume the currently published npm version includes it.

OrcaRouter: the playground and this docs assistant can run through OrcaRouter, an OpenAI-compatible AI gateway. Users connect either with an API key or with Connect with OrcaRouter (OAuth 2.0 + PKCE), and model choices come from the live OrcaRouter model catalog. See /docs/orcarouter for setup, credential handling, and capability filtering.

You have access to the full json-render documentation via the bash and readFile tools. The docs are available as markdown files in the /workspace/docs/ directory.

When answering questions:
Expand Down Expand Up @@ -108,15 +118,55 @@ export async function POST(req: Request) {
);
}

const { messages }: { messages: UIMessage[] } = await req.json();
const {
messages,
model,
provider,
}: { messages: UIMessage[]; model?: string; provider?: string } =
await req.json();

const docsFiles = await loadDocsFiles();
const {
tools: { bash, readFile },
} = await createBashTool({ files: docsFiles });

// The docs assistant is a text-only entry point: it never uploads an
// attachment, so it offers chat models without a modality requirement.
let modelRef: Parameters<typeof streamText>[0]["model"] = DEFAULT_MODEL;
if (typeof provider === "string" && isOrcarouterProvider(provider)) {
const modelId =
typeof model === "string" && model.trim()
? model.trim()
: ORCAROUTER_DEFAULT_MODEL;
try {
const { catalog } = await discoverOrcaModels();
const compatible = selectOrcaModels(catalog, { capability: "chat" });
if (!compatible.some((entry) => entry.id === modelId)) {
return Response.json(
{
error: "Incompatible model",
message: `The OrcaRouter catalog does not list "${modelId}" as a chat model. Pick another model.`,
},
{ status: 400 },
);
}
modelRef = createOrcarouterTransport(modelId).model;
} catch (error) {
return Response.json(
{
error: "OrcaRouter unavailable",
message:
error instanceof OrcaRouterNotConnectedError
? error.message
: "The OrcaRouter model catalog is unavailable. Try again shortly.",
},
{ status: 503 },
);
}
}

const result = streamText({
model: DEFAULT_MODEL,
model: modelRef,
system: SYSTEM_PROMPT,
messages: await convertToModelMessages(messages),
stopWhen: stepCountIs(5),
Expand Down
Loading