Skip to content

Media over QUIC on Pipecat Cloud - #1555

Open
mattshep wants to merge 5 commits into
mainfrom
ms/moq-on-pipecat-cloud
Open

mattshep wants to merge 5 commits into
mainfrom
ms/moq-on-pipecat-cloud

Conversation

@mattshep

@mattshep mattshep commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Documents Media over QUIC (MoQ) on Pipecat Cloud: a guide, the /start reference, and the SDK's MOQSessionArguments. Merging publishes it, so it merges when MoQ is available on Pipecat Cloud.

  • Guide (pipecat-cloud/guides/moq):
    • requirements: pipecat-ai[moq] 1.13.0, pipecat-base 0.2.7 named outright (a cloud build on latest can get the previous base image for up to a day after a release), pipecatcloud 1.3.0 for images that install their own, and @pipecat-ai/client-js 1.14 with @pipecat-ai/moq-transport 0.3.0;
    • the bot side through create_transport;
    • /start with transport: "moq" and its response;
    • the browser through startBotAndConnect();
    • reconnects: a relay update drains the relay and doesn't interrupt a session, and a failed relay costs the session around 15 seconds of audio while both sides reconnect, with no RTVI message lost or repeated;
    • the 409 refusals (retry) and the 400 ones (change the agent or the request).
  • REST reference: moq joins StartRequest.transport's enum, and StartResponse gains the moq object; moq takes the 1 MB body limit.
  • SDK reference: MOQSessionArguments in Session arguments, and a MoQ entry point in Examples. This takes over docs: update for pipecat-cloud PR #210 #1393 (closed): its import guard (except ImportError: pass before a function annotated with the type) fails with NameError on Python 3.11 to 3.13, and it named pipecat-ai 1.12.0 where Pipecat Cloud needs 1.13.0.

The browser versions it names, client-js 1.14 with moq-transport 0.3.0, have been through a relay loss and a relay update in a full session.

Rebased on main. mint broken-links --check-anchors --check-redirects and docs-meta-lint pass, and llms.txt / llms-full.txt are regenerated.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Mintlify preview for this branch: https://daily-ms-moq-on-pipecat-cloud.mintlify.site

A guide to MoQ sessions on Pipecat Cloud (requirements, the bot side, /start
with transport "moq", the browser, reconnects and refusals), "moq" in the
/start reference's request and response, and MOQSessionArguments in the SDK
reference.
Relay updates drain the relay first, so they don't interrupt a session. A
failed relay costs the session its audio until both sides have reconnected,
around 15 seconds, rather than the short gap the guide promised.
@mattshep
mattshep force-pushed the ms/moq-on-pipecat-cloud branch from beb61bc to 0374be0 Compare October 22, 2026 18:16
@mattshep
mattshep marked this pull request as ready for review October 22, 2026 18:16
@mattshep mattshep self-assigned this Oct 22, 2026

@cbrianhill cbrianhill left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code contributed to this review.

Recommended comment; Brian approved. Nothing in the new text is wrong where I could check it, but two things this PR set out to document are missing from the /start reference: the 409 retry contract, and moq in the body-size list the reference links to. Neither blocks.

Four findings, three suggestions and a question, none blocking:

  1. The 1 MB body cap for moq is not in the page the reference links to (openapi.json:4294).
  2. /start's reference does not document the 409 the guide says to retry (openapi.json:4288).
  3. "MoQ isn't offered in the agent's region" has no documented remedy (moq.mdx:133).
  4. Does an explicit createDailyRoom: false get refused? (moq.mdx:132).

Verified:

  • The guide's bot snippet matches create_transport, which dispatches on MOQRunnerArguments and reads the "moq" key (pipecat src/pipecat/runner/utils.py:855); every property in the new MOQSessionArguments section exists on that dataclass with the documented type.
  • The 1.12.0 ImportError sentence is right: pipecatcloud defines MOQSessionArguments only when relay_url is in MOQRunnerArguments.__dataclass_fields__, which arrived in pipecat-ai 1.12.0 (src/pipecatcloud/agent.py:107).
  • The no-duplicate-RTVI-messages claim holds: seq counts across reconnects and epoch identifies the transport instance rather than the connection, per the server MoQ page.

Naming pipecat-base 0.2.7 outright, with the reason a latest build can lag a release by a day, prevents a class of support ticket that is hard to diagnose from the symptom.

"body": {
"type": "object",
"description": "Arbitrary user data / configuration object to pass to the service instance. Accessible as the first parameter of the `bot` method or custom entry point. Maximum size: 1 MB for `transport: \"daily\"` or `\"webrtc\"`, 4 KB for `\"websocket\"`. For `transport: \"websocket\"`, the body is returned in the response's `body` field base64-encoded; the service must be deployed with `websocket_auth = \"token\"` to use this encoding convenience. See the [`Starting Sessions` docs](/pipecat-cloud/fundamentals/active-sessions#passing-data) for details.",
"description": "Arbitrary user data / configuration object to pass to the service instance. Accessible as the first parameter of the `bot` method or custom entry point. Maximum size: 1 MB for `transport: \"daily\"`, `\"webrtc\"` or `\"moq\"`, 4 KB for `\"websocket\"`. For `transport: \"websocket\"`, the body is returned in the response's `body` field base64-encoded; the service must be deployed with `websocket_auth = \"token\"` to use this encoding convenience. See the [`Starting Sessions` docs](/pipecat-cloud/fundamentals/active-sessions#passing-data) for details.",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. The 1 MB body cap for moq is not in the page this line links to (suggestion)
api-reference/pipecat-cloud/rest-reference/openapi.json:4294

Problem. This description now says the 1 MB cap covers moq and sends the reader to Starting Sessions for details. That page's own size-limit list still has two entries, neither mentioning moq, so a reader who follows the link cannot tell which cap applies.

Why.

  1. The new text extends the 1 MB group to moq and points at /pipecat-cloud/fundamentals/active-sessions#passing-data.
  2. That page's Size limits section says the maximum "depends on the transport" and then lists only daily or webrtc at 1 MB and websocket at 4 KB.
  3. active-sessions.mdx is not in this PR, so the reference and the page it cites now disagree about whether moq has a documented cap.

Fix. Add moq to the 1 MB line in pipecat-cloud/fundamentals/active-sessions.mdx. pipecat-cloud/guides/generic-websocket.mdx has the same omission in passing ("smaller than the 1 MB cap for daily and webrtc transports"), worth the same edit while you are in there.

// pipecat-cloud/fundamentals/active-sessions.mdx:231-236
The maximum body size depends on the transport:

- `transport: "daily"` or `"webrtc"` — **1 MB of JSON**   <-- no `moq`
- `transport: "websocket"` — **4 KB of JSON**

Requests exceeding the per-transport limit are rejected with `413 Payload Too Large` before any bot is started.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added moq to the 1 MB line in Starting Sessions. The same omission was in three more places, fixed as well: the WebSocket guide's size comparison and its note on how body reaches the bot, and the transport values session.started lists in the webhook events guide.

"type": "string",
"description": "Explicitly specify which transport the bot should use.",
"enum": ["daily", "webrtc", "websocket"],
"description": "Explicitly specify which transport the bot should use. `moq` starts a Media over QUIC session: the response's `moq` field carries what the browser transport needs, and `createDailyRoom` and `enableDefaultIceServers` cannot be combined with it. See the [Media over QUIC guide](/pipecat-cloud/guides/moq).",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. /start's reference does not document the 409 the guide says to retry (suggestion)
api-reference/pipecat-cloud/rest-reference/openapi.json:4288

Problem. The guide tells clients that /start with transport: "moq" can return 409 Conflict and that the fix is to retry with backoff. The /start operation in this file lists only 200, 400 and 401, so a client built from the rendered REST reference will not implement that retry and will see MoQ sessions fail for minutes after every deploy.

Why.

  1. The guide makes 409 a normal outcome while a deployment rolls out or finishes reporting what it can serve (pipecat-cloud/guides/moq.mdx:124-127).
  2. This description is the only mention of MoQ in the /start operation, and it covers the response's moq field, not the refusals.
  3. The operation's responses block carries 200, 400 and 401 only, so the rendered reference shows no 409 at all.
  4. The troubleshooting page's only 409 entry is a deploy rejected for an unready secret set, so there is no second place a reader finds this.

Fix. Add a 409 response to the /{agentName}/start operation, referencing ErrorResponse as the 400 does, with a description that says to retry with backoff. Naming the retry in this transport description too would put it where someone reading just the enum sees it.

// api-reference/pipecat-cloud/rest-reference/openapi.json:2987-3062 (elided)
"/{agentName}/start": {
  "post": {
    // ...
    "responses": {
      "200": { /* StartResponse */ },
      "400": { "description": "Invalid request", /* ErrorResponse */ },
      "401": { "$ref": "#/components/responses/Unauthorized" }   // <-- no 409
    }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a 409 to /start, with the body the API sends while a deployment is still reporting what it can serve. Its description covers every case /start returns 409 for, not only MoQ, and says that after a failed deployment a retry succeeds only once a later deployment completes. The transport description now points at it.

Comment thread pipecat-cloud/guides/moq.mdx Outdated

- The agent's current deployment can't serve MoQ: for example, its image is older than `pipecat-base` 0.2.7, its `pipecat-ai` is older than 1.13.0, or the `moq` extra isn't installed. Rebuild on the [requirements](#requirements) above and deploy again.
- The request also sets `createDailyRoom` or `enableDefaultIceServers`.
- MoQ isn't offered in the agent's region.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3. A reader who hits "MoQ isn't offered in the agent's region" has nowhere to go (suggestion)
pipecat-cloud/guides/moq.mdx:133

Problem. This is the one entry in either error list with no action attached. Nothing in the docs says which regions offer MoQ, so a customer who gets this 400 cannot tell whether to move the agent, wait for a rollout, or open a ticket.

Why.

  1. Every other entry in the 409 and 400 lists names the change to make: rebuild on the requirements, drop a field, retry once the deployment completes.
  2. The Regions guide lists the four Daily-hosted regions and their architectures and says nothing about MoQ, and a self-hosted region declares its own capabilities.
  3. So there is no list to check and no documented way to find out, here or anywhere this page links.

Fix. Say which regions offer MoQ, or point at the regions listing if pipecat cloud regions list reports it. A sentence in Requirements is enough, linking /pipecat-cloud/guides/regions.

// pipecat-cloud/guides/moq.mdx:129-133
`400 Bad Request` means change the agent or the request:

- The agent's current deployment can't serve MoQ: ... Rebuild on the [requirements](#requirements) above and deploy again.
- The request also sets `createDailyRoom` or `enableDefaultIceServers`.
- MoQ isn't offered in the agent's region.   <-- nothing to act on

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 400 bullet now says to deploy the agent to a Daily-hosted region, linking the Regions guide.

Comment thread pipecat-cloud/guides/moq.mdx Outdated
`400 Bad Request` means change the agent or the request:

- The agent's current deployment can't serve MoQ: for example, its image is older than `pipecat-base` 0.2.7, its `pipecat-ai` is older than 1.13.0, or the `moq` extra isn't installed. Rebuild on the [requirements](#requirements) above and deploy again.
- The request also sets `createDailyRoom` or `enableDefaultIceServers`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4. Does an explicit createDailyRoom: false get refused? (question)
pipecat-cloud/guides/moq.mdx:132

Problem. "The request also sets" reads as being about the field being present, not about its value. Both fields default to false, so a client that always sends them explicitly would be refused for asking for the default.

Why.

  1. createDailyRoom and enableDefaultIceServers are both "default": false in the REST reference, and both carry "example": true.
  2. A client that templates its /start body from those examples plausibly sends createDailyRoom: false on every request, MoQ or not.
  3. If the check is on presence, that request 400s with transport: "moq"; if it is on the value, it succeeds. This line and the transport description ("cannot be combined with it") both read the first way.

Fix. Which is it? If only a truthy value is refused, "sets createDailyRoom or enableDefaultIceServers to true" would say so, and the transport description in openapi.json could match.

// api-reference/pipecat-cloud/rest-reference/openapi.json:4259-4263
"createDailyRoom": {
  "type": "boolean",
  "default": false,     // <-- a client sending `false` is asking for the default
  "description": "Whether to create a Daily room for the service instance",
  "example": true
},

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only true is refused: the API checks the value, not whether the field is present, so a client that always sends false is fine. The guide and the transport description now say "set to true", and the guide says false is accepted.

This branch was successfully deployed

1 active deployment
staging — 655f1419 Deployed Oct 23, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants