feat(core): add Cloudflare AI Gateway to the model router#19284
Open
brennanmceachran wants to merge 2 commits into
Open
feat(core): add Cloudflare AI Gateway to the model router#19284brennanmceachran wants to merge 2 commits into
brennanmceachran wants to merge 2 commits into
Conversation
Cloudflare AI Gateway is listed on models.dev but its entry has no api URL (the endpoint is account-scoped), so the registry generator skipped it. Add a PROVIDER_OVERRIDES entry that routes it through Cloudflare's OpenAI-compatible endpoint with URL-template env vars, list it as a gateway in the docs generator, and regenerate the provider registry and model docs. Model strings like cloudflare-ai-gateway/anthropic/claude-3-5-haiku now work with CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_GATEWAY_ID, and CLOUDFLARE_API_TOKEN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
brennanmceachran
requested review from
CalebBarnes,
LekoArts,
PaulieScanlon,
TylerBarnes,
abhiaiyer91,
bookercodes and
wardpeet
as code owners
🦋 Changeset detectedLatest commit: ebd7d27 The changes in this PR will be included in the next version bump. This PR includes changesets to release 23 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@brennanmceachran is attempting to deploy a commit to the Mastra Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
PR triageLinked issue check passed (#19283). Mastra uses CodeRabbit for automated code reviews. Please address all feedback from CodeRabbit by either making changes to your PR or leaving a comment explaining why you disagree with the feedback. Since CodeRabbit is an AI, it may occasionally provide incorrect feedback. PR complexity score
Applied label: Changed test gateChanged Test Gate is pending. The |
github-actions
Bot
added
the
Documentation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (3)
WalkthroughAdds Cloudflare AI Gateway routing with account-scoped configuration, registers Pioneer, UnoRouter, and EmpirioLabs models, updates capability/type metadata, adds routing tests, and refreshes generated provider and gateway documentation. ChangesModel router and provider registry
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/src/content/en/models/gateways/openrouter.mdx (1)
89-89: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd a token for
arcee-ai/trinity-large-thinkingand use it here instead of the literal ID.
docs/src/plugins/remark-model-tokens/models.tsdoesn’t define this model yet, so this row should follow the same placeholder-token pattern as the rest of the docs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/src/content/en/models/gateways/openrouter.mdx` at line 89, Add an appropriate token constant for arcee-ai/trinity-large-thinking in the model token definitions, then update the OpenRouter documentation row to reference that token instead of the literal model ID, following the existing placeholder-token pattern and naming conventions.Sources: Coding guidelines, Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.changeset/cloudflare-ai-gateway-provider.md:
- Line 5: Add a concise Agent configuration example to the changeset,
demonstrating use of the Cloudflare model string
cloudflare-ai-gateway/anthropic/claude-3-5-haiku and the relevant
environment-based configuration, while preserving the existing feature
description.
In `@docs/src/content/en/models/gateways/cloudflare-ai-gateway.mdx`:
- Line 25: The generated Cloudflare AI Gateway example hard-codes a model
identifier instead of using the registered model token. Update the generator
configuration in the models registry to reference the corresponding token for
`cloudflare-ai-gateway/anthropic/claude-3-5-haiku`, then regenerate the MDX
documentation so the example uses the placeholder.
In `@docs/src/content/en/models/index.mdx`:
- Line 152: The provider count is inconsistent between the card and the page
headline. Update the “+ 139 more” text in the provider card to “+ 145 more” if
the intended total is 148; otherwise adjust the headline count so both totals
match.
In `@docs/src/content/en/models/providers/pioneer.mdx`:
- Line 25: The generated Pioneer MDX examples hard-code model IDs instead of
registered placeholders. Update the generator responsible for these examples to
use the model tokens defined in `models.ts`, including the occurrences
corresponding to the referenced lines, then regenerate the page rather than
editing the generated file manually.
In `@docs/src/content/en/models/providers/unorouter.mdx`:
- Line 25: The generated UnoRouter examples hard-code provider model IDs instead
of registered placeholders. Update the documentation generator responsible for
the UnoRouter examples to use the model tokens defined in models.ts, including
all referenced occurrences, then regenerate the MDX page rather than editing the
generated file manually.
In `@packages/core/scripts/generate-model-docs.ts`:
- Around line 589-591: Sanitize the gateway name in the empty-provider fallback
within the environment-variable generation logic: replace all non-alphanumeric
characters in gatewayName with underscores before uppercasing and appending
_API_KEY, so names such as CLOUDFLARE-AI-GATEWAY produce valid shell variable
names.
In `@packages/core/src/llm/model/capabilities/cloudflare-ai-gateway.json`:
- Around line 59-101: Remove non-generative model identifiers from the
temperature capability list in the Cloudflare AI Gateway capabilities
configuration, including embeddings, TTS, transcription, classification,
reranker, and translation models. Retain only models that support temperature
sampling so modelSupportsTemperature reports accurate capabilities.
In `@packages/core/src/llm/model/capabilities/openai.json`:
- Line 45: The model capabilities entry for gpt-realtime-2.1 incorrectly uses
the generic attachment capability to represent image support. Update the OpenAI
capability configuration so this model is listed under the appropriate
image-input capability and removed from the generic attachment list, preserving
separate Files API support semantics.
In `@packages/core/src/llm/model/capabilities/pioneer.json`:
- Around line 66-75: Remove the non-generative guardrail, NER, and diffusion
model identifiers from the temperature capability array in pioneer.json,
including the fastino/gliguard-LLMGuardrails-300M, fastino/gliner2-* entries,
and google/diffusiongemma-26B-A4B-it, so modelSupportsTemperature only
recognizes models that accept temperature.
In `@packages/core/src/llm/model/provider-registry.json`:
- Around line 3155-3170: Update the modelOverrides entry for
anthropic/claude-opus-4-7 to use the OpenAI-compatible ai-gateway-provider, or
remove the override so the configured Cloudflare /compat gateway handles it
instead of `@ai-sdk/anthropic`.
---
Outside diff comments:
In `@docs/src/content/en/models/gateways/openrouter.mdx`:
- Line 89: Add an appropriate token constant for arcee-ai/trinity-large-thinking
in the model token definitions, then update the OpenRouter documentation row to
reference that token instead of the literal model ID, following the existing
placeholder-token pattern and naming conventions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 095b1a5a-34c0-4d29-9353-1f378e417d83
📒 Files selected for processing (26)
.changeset/cloudflare-ai-gateway-provider.mddocs/src/content/en/models/environment-variables.mdxdocs/src/content/en/models/gateways/cloudflare-ai-gateway.mdxdocs/src/content/en/models/gateways/index.mdxdocs/src/content/en/models/gateways/openrouter.mdxdocs/src/content/en/models/gateways/vercel.mdxdocs/src/content/en/models/index.mdxdocs/src/content/en/models/providers/cloudflare-ai-gateway.mdxdocs/src/content/en/models/providers/empiriolabs.mdxdocs/src/content/en/models/providers/index.mdxdocs/src/content/en/models/providers/openai.mdxdocs/src/content/en/models/providers/pioneer.mdxdocs/src/content/en/models/providers/unorouter.mdxdocs/src/content/en/models/sidebars.jspackages/core/scripts/generate-model-docs.tspackages/core/src/llm/model/capabilities/cloudflare-ai-gateway.jsonpackages/core/src/llm/model/capabilities/empiriolabs.jsonpackages/core/src/llm/model/capabilities/openai.jsonpackages/core/src/llm/model/capabilities/openrouter.jsonpackages/core/src/llm/model/capabilities/pioneer.jsonpackages/core/src/llm/model/capabilities/unorouter.jsonpackages/core/src/llm/model/cloudflare-ai-gateway-url.test.tspackages/core/src/llm/model/gateways/models-dev.test.tspackages/core/src/llm/model/gateways/models-dev.tspackages/core/src/llm/model/provider-registry.jsonpackages/core/src/llm/model/provider-types.generated.d.ts
💤 Files with no reviewable changes (2)
- docs/src/content/en/models/providers/cloudflare-ai-gateway.mdx
- packages/core/src/llm/model/capabilities/openrouter.json
| id: "my-agent", | ||
| name: "My Agent", | ||
| instructions: "You are a helpful assistant", | ||
| model: "unorouter/claude-haiku-4-5-20251001" |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use registered model placeholders in illustrative examples.
These generated MDX examples hard-code UnoRouter model IDs. Update the generator to use tokens from docs/src/plugins/remark-model-tokens/models.ts, then regenerate the page; do not manually edit this auto-generated file.
[potential_issue]
Also applies to: 337-338, 355-356
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/src/content/en/models/providers/unorouter.mdx` at line 25, The generated
UnoRouter examples hard-code provider model IDs instead of registered
placeholders. Update the documentation generator responsible for the UnoRouter
examples to use the model tokens defined in models.ts, including all referenced
occurrences, then regenerate the MDX page rather than editing the generated file
manually.
Sources: Coding guidelines, Learnings
Contributor
Author
There was a problem hiding this comment.
Skipping for the same reason as pioneer.mdx: auto-generated models.dev drift from the canonical regeneration, not part of this PR's change surface.
Contributor
There was a problem hiding this comment.
@brennanmceachran, understood — thanks for clarifying. I'll treat this as out of scope for this PR, consistent with the pioneer.mdx case.
✏️ Learnings added
Learnt from: brennanmceachran
Repo: mastra-ai/mastra PR: 19284
File: docs/src/content/en/models/providers/unorouter.mdx:25-25
Timestamp: 2026-07-10T22:22:57.754Z
Learning: In the mastra-ai/mastra repo, generated model documentation pages under `docs/src/content/en/models/providers/*.mdx` and `docs/src/content/en/models/gateways/*.mdx` are auto-generated by `packages/core/scripts/generate-model-docs.ts` from models.dev data. Hard-coded model IDs in illustrative examples (rather than placeholder tokens from `docs/src/plugins/remark-model-tokens/models.ts`) reflect drift in the canonical regeneration process itself, not something to fix within an individual PR that merely adds/updates provider docs. Per brennanmceachran, such findings should be skipped as out of scope for the PR's change surface.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: mikhael28
Repo: mastra-ai/mastra PR: 17896
File: examples/voice-agent/src/mastra/agents/call-center-agent.ts:36-36
Timestamp: 2026-06-23T16:45:35.194Z
Learning: In this repo, the "placeholder-token" rule (replacing model name/provider/model strings with the tokens defined in `docs/src/plugins/remark-model-tokens/models.ts`) applies only to documentation markdown/MDX content and changeset entries. For any documentation file matched by `**/*.{md,mdx}`, ensure literal model name strings are replaced with the corresponding placeholder tokens.
Do not treat literal provider/model ID strings in executable code as a placeholder-token violation: in `.ts`/`.tsx` files (including `examples/**`, tests, and integration code), it’s expected to use real provider/model IDs such as `"openai/gpt-5-mini"`, because the Remark plugin runs only during docs build and not at runtime.
Learnt from: mikhael28
Repo: mastra-ai/mastra PR: 17896
File: integrations/livekit/src/worker.ts:45-48
Timestamp: 2026-06-23T16:45:35.250Z
Learning: In the mastra-ai/mastra repo, the remark plugin model-token placeholder policy should be enforced only for MDX documentation files under docs/src that are processed by the remark-model-tokens plugin. In those MDX files, use the placeholder tokens defined in docs/src/plugins/remark-model-tokens/models.ts instead of writing literal model IDs. Do not apply this restriction to TypeScript/TSX source files, JSDoc comments, or executable example code; in those files, literal model IDs are acceptable.
- Drop models.dev per-model npm hints for cloudflare-ai-gateway via PROVIDER_OVERRIDES so every model stays on the OpenAI-compatible /compat endpoint (claude-opus-4-7 carried an @ai-sdk/anthropic hint that would have routed it through the wrong SDK protocol); regression test added - Use the evergreen __GATEWAY_ANTHROPIC_MODEL_HAIKU__ token in the generated gateway usage example - Sanitize the empty-provider env-var fallback in the docs generator - Add a usage example to the changeset Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds Cloudflare AI Gateway to the model router with the same string DX as the other built-in gateways:
configured via
CLOUDFLARE_ACCOUNT_ID,CLOUDFLARE_GATEWAY_ID, andCLOUDFLARE_API_TOKEN.cloudflare-ai-gatewayis already listed on models.dev (78 models, already named{provider}/{model}, env vars declared) but its entry has noapiURL because Cloudflare's endpoint is account-scoped — so the registry generator'shasApiAndEnvfilter dropped it. This PR supplies the URL through the existingPROVIDER_OVERRIDESmechanism (same asmistral/groq/moonshotai), routing through Cloudflare's OpenAI-compatible endpoint with env-var URL templating — the same mechanismcloudflare-workers-aiuses today:Mastra v0 shipped this provider natively; it fell out in the v1 registry pipeline because of the missing
apifield. This restores it and promotes it to the Gateways docs section.Changes
packages/core/src/llm/model/gateways/models-dev.ts—PROVIDER_OVERRIDESentry (URL template +CLOUDFLARE_API_TOKENas auth; the other two env vars are consumed by the URL template, mirroring the guard from [BUG] Cloudflare API Key #14510).packages/core/scripts/generate-model-docs.ts— listcloudflare-ai-gatewayinGATEWAY_PROVIDERS, display-name special case, gateway description, and a Configuration block that now lists URL-template env vars (so account/gateway IDs show up) with a Cloudflare-specific note that provider keys live in the Cloudflare dashboard (BYOK) rather than in env.provider-registry.json,provider-types.generated.d.ts,capabilities/, and model docs via the canonicalgenerate:providers+generate-model-docs.tsscripts (this also picked up a few hours of routine models.dev drift, same as the 6-hourly automated refresh).docs/src/content/en/models/providers/cloudflare-ai-gateway.mdx(the generator no longer emits it once the provider is in the registry; it's replaced bydocs/src/content/en/models/gateways/cloudflare-ai-gateway.mdx).cloudflare-ai-gateway-url.test.ts— mirrorscloudflare-workers-ai-url.test.ts: a realagent.stream()asserts the interpolatedbaseURL, theCLOUDFLARE_API_TOKENapi key, and that the model id keeps its{provider}/{model}shape (the compat endpoint contract).models-dev.test.ts— registry-inclusion case proving the override admits the provider despite the missingapiURL and selects the token (not the account id) for auth.Design notes
/compatand not the newer REST API (api.cloudflare.com/.../ai/v1): the REST API selects the gateway via header (falling back to the account default) and is documented only with Unified Billing — no BYOK provider keys./compatkeeps explicit per-env gateway selection and BYOK stored keys, matches how Cloudflare's own maintained AI SDK integration (ai-gateway-providerin cloudflare/ai) targets the gateway, and switching later is a one-line URL change in the override. Users who prefer the REST API today can pointCLOUDFLARE_AI_GATEWAY_BASE_URLat it.Verification
pnpm --filter @mastra/core generate:providersandgenerate-model-docs.tsrun clean (CI-workflow sequence, prettier applied).pnpm --filter @mastra/core typecheckgreen with the regenerated types.POST https://gateway.ai.cloudflare.com/v1/{account}/{gateway}/compat/chat/completionswith a synthetic account returns a structuredAiGatewayError401 (route recognized by AI Gateway, auth enforced) — confirming the endpoint shape the override targets.Related issue(s)
Fixes #19283
Type of change
Checklist
🤖 Generated with Claude Code
ELI5
This update lets Mastra use Cloudflare AI Gateway just by writing a model name like
cloudflare-ai-gateway/anthropic/claude-3-5-haiku. Mastra automatically talks to Cloudflare’s OpenAI-compatible gateway endpoint, and Cloudflare handles the key management, routing, and usage behind the scenes.What changed
cloudflare-ai-gateway/...model strings./compatendpoint:https://gateway.ai.cloudflare.com/v1/${CLOUDFLARE_ACCOUNT_ID}/${CLOUDFLARE_GATEWAY_ID}/compatCLOUDFLARE_API_TOKENmodels.dev/router output even though it has no static provider API URL (endpoint is account-scoped).models.devmock + assertions ensuring the override-driven provider works without per-model model overrides.@mastra/coredocumenting the new router support.