New file upload example by mattieruth · Pull Request #237 · pipecat-ai/pipecat-examples

mattieruth · 2026-07-16T20:10:33Z

No description provided.

kompfner · 2026-07-28T13:27:03Z

+conversation context. The LLM can then reason about the uploaded content and
+respond with audio as it would for any other user turn.
+
+Concepts this example is meant to demonstrate:


Nice overview

kompfner · 2026-07-28T13:28:05Z

+4. **Run the bot**:
+
+   ```bash
+   uv run bot.py -u /tmp/uploads


Worth having a default temp location if one is not specified? Simplifies usage.

you don't think that's a privacy/security issue? the runner will use this location to store and delete files. It felt a little presumptive to me to make a default.

I meant just for this example, which exists only to show file uploading. Not proposing baking anything into the runner itself.

kompfner · 2026-07-28T13:29:05Z

+   # Edit .env.local if needed (defaults to localhost:7860)
+   ```
+
+   > **Note:** Environment variables in Vite are bundled into the client and exposed in the browser. For production applications that require secret protection, consider implementing a backend proxy server to handle API requests and manage sensitive credentials securely.


kompfner · 2026-07-28T13:33:09Z

+        user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
+    )
+
+    rtvi = RTVIProcessor(uploads_folder=runner_args.cli_args.uploads_folder)


This is the only configuration needed on the bot server side? That's pretty sweet.

Amazing 🎉

correct. and even then it's only needed if you are supporting an endpoint for the client to upload files to. As @vipyne points out in her PR, PipecatCloud does not have an endpoint for this as of yet, so in that case, uploads_folder should be None and the client will only be able to upload either small files that can be sent wholesale over the transport message channel or urls.

Since we have this (hopefully temporary) limitation today, we ought to mention it in the README, which presents Pipecat Cloud as a deployment option.

i believe it is already 🤔

kompfner · 2026-07-28T13:34:02Z

+
+This bot uses a cascade pipeline: Speech-to-Text → LLM → Text-to-Speech
+
+Generated by Pipecat CLI


So there's also some new CLI affordances coming for configuring a bot to accept file uploads?

i don't have any cli changes planned but maybe i should? When I started this example, I generated it using the CLI which adds this line (and 98% of the lines in this file). Maybe I should remove this comment?

but maybe i should?

I'd have to refresh myself on the existing CLI options to see if something like "accept file uploads" would fit in...without going through them, I have no preference.

this line

I wonder if, if the intent of the CLI is to provide a starting point or scaffold that the user will then build on top of, this line should instead read "Bootstrapped by Pipecat CLI" or something. "Generated by" might lead the reader to believe this file is "managed" by a tool and shouldn't be edited. This is obviously an issue outside the scope of this PR. WDYT, @markbackman?

filipi87 · 2026-07-28T14:18:56Z

+ * This file is auto-generated by pipecat-cli
+ */
+
+export const AVAILABLE_TRANSPORTS = [


Should we add the websocket here ?

filipi87 · 2026-07-28T14:21:00Z

+    "@pipecat-ai/client-js": "file:../../../pipecat-client-web/client-js",
+    "@pipecat-ai/daily-transport": "file:../../../pipecat-client-web-transports/transports/daily",
+    "@pipecat-ai/small-webrtc-transport": "file:../../../pipecat-client-web-transports/transports/small-webrtc-transport"


TODO: Remember to fix this before merging. 🙂

filipi87 · 2026-07-28T14:23:01Z

+        option.textContent = 'SmallWebRTC';
+      } else if (transport === 'daily') {
+        option.textContent = 'Daily';
+      } else if (transport === 'websocket') {


Here we're mentioning websocket, but we don't have it in either the dependencies or AVAILABLE_TRANSPORTS. I think we just need to decide whether we want to support it or remove it.

oh, i don't think i meant to support it. though, i should test that...

actually. i went ahead and added support :)

filipi87 · 2026-07-28T14:25:26Z

+[tool.uv.sources]
+pipecat-ai = { path = "/Users/mattieruth/src/pipecat/pipecat", editable=true }


TODO: Remember to fix this before merging. 🙂

filipi87 · 2026-07-28T14:26:57Z

I don't think any of the other examples have this. Are we planning to keep it?

i can remove it. like i said, I generated this example using the CLI and so some of the stuff was just basic scaffolding. though, in chatting with @markbackman, it seemed like maybe i should leave it? i have no preference/opinion.

filipi87 · 2026-07-28T14:27:07Z

Same here. I don't think any of the other examples have this. Are we planning to keep it?

same reply :)

kompfner · 2026-07-28T18:00:19Z

+    )
+
+    # cli_args is only populated when running via the development runner (not on
+    # Pipecat Cloud, where there is no uploads endpoint and large-file upload is


Wait, so small-file upload is available even when uploads_folder is None?

yes because those can fit in a basic message.

vercel · 2026-09-16T19:55:23Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
pipecat-examples	Ready	Preview	Oct 8, 2026 1:34pm UTC

filipi87 · 2026-10-07T12:21:19Z

+def create_file_storage() -> FileStorage:
+    """Storage backend for client uploads, discovered by the host alongside ``bot()``.
+
+    The host calls this at startup, serves its upload endpoint from the
+    returned backend, and injects it into the bot as
+    ``runner_args.file_storage``.
+    """
+    return S3FileStorage(bucket=os.environ["S3_UPLOADS_BUCKET"], region=os.getenv("AWS_REGION"))


New example showing how clients can upload local files or reference URLs as visual context during a voice conversation. Uses RTVIProcessor with uploads_folder on the server and client.sendFile via the RTVI protocol.

…/../../pipecat with a "remove before merge" TODO comment. - bot.py: guards cli_args/uploads_folder access so it won't crash on Pipecat Cloud. - app.js: URL format parsing uses new URL().pathname (handles query strings, rejects invalid URLs), the .catch().then() chains are reordered so success messages don't render after failures, and bot text is HTML-escaped before the innerHTML insertions. - README.md: added a note about Pipecat Cloud caveat that large-file upload needs a production /files endpoint.

…ists

…oogle cloud

The GCS/S3 variants now define create_file_storage() alongside bot() instead of calling set_runner_file_storage() (removed from pipecat): the host discovers the factory, backs its upload endpoint with the returned storage, and injects it into the bot as runner_args.file_storage, which bot.py hands to its FileResolver.

filipi87 · 2026-10-09T13:26:56Z

+    "@pipecat-ai/client-js": "file:../../../clients/web/client/client-js",
+    "@pipecat-ai/daily-transport": "file:../../../clients/web/transports/transports/daily",
+    "@pipecat-ai/livekit-transport": "file:../../../clients/web/transports/transports/livekit-transport",
+    "@pipecat-ai/small-webrtc-transport": "file:../../../clients/web/transports/transports/small-webrtc-transport/pipecat-ai-small-webrtc-transport-1.10.7.tgz",
+    "@pipecat-ai/websocket-transport": "file:../../../clients/web/transports/transports/websocket-transport"


We need to fix the dependencies here. 🙂

Just saw that we still need to release all of them.

filipi87 · 2026-10-09T13:31:25Z

+description = "Voice AI bot built with Pipecat"
+requires-python = ">=3.11"
+dependencies = [
+    "pipecat-ai[anthropic,aws,cartesia,daily,deepgram,google,livekit,openai,runner,silero,websocket,webrtc]>=1.4.0",


We also need to update to the latest version of Pipecat, which includes file upload support.

filipi87 · 2026-10-09T13:31:50Z

+# TODO: remove before merge — local override for testing against the sibling
+# pipecat checkout (pipecat-ai/pipecat PR #3415) until the feature is released.
+[tool.uv.sources]
+pipecat-ai = { path = "../../../pipecat", editable=true }


And implement this TODO. 😅

yes. all of these I expect to do once the releases are actually available to link against.

filipi87 · 2026-10-09T13:49:18Z

Hi @mattieruth, I tried uploading a .txt file, and the interface shows the following message, as if the upload had succeeded:

You
Uploaded file: about-gpu-2026-10-09T11-50-28-864Z.txt

However, I can see the following errors in Pipecat:

2026-10-09 10:45:44.929 | ERROR    | pipecat.processors.frame_processor:push_error_frame:1002 - AnthropicLLMService#0 exception (/Users/filipi/workspaces/workspace_daily/pipecat/src/pipecat/adapters/services/anthropic_adapter.py:211): Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain
2026-10-09 10:45:44.929 | WARNING  | pipecat.processors.aggregators.llm_context:remove_invalid_file_message:629 - Removing message with file or image content from context due to invalid response.
2026-10-09 10:45:44.930 | DEBUG    | pipecat.processors.metrics.frame_processor_metrics:stop_processing_metrics:308 - AnthropicLLMService#0 processing time: 0.004s
2026-10-09 10:45:44.931 | WARNING  | pipecat.pipeline.worker:_source_push_frame:1521 - PipelineWorker#0: Something went wrong: ErrorFrame#0(error: Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain, fatal: False)

I believe this is the missing piece we discussed yesterday. Just sharing it here in case it’s worth addressing in the example.

filipi87

Works great! I left just a couple of comments about remembering to update the libraries once we release, and whether we can improve error handling in case we push a failed or disallowed frame.

Other than that, it looks great. I've already approved it. 🚀

mattieruth · 2026-10-12T17:24:30Z

Hi @mattieruth, I tried uploading a .txt file, and the interface shows the following message, as if the upload had succeeded:

You
Uploaded file: about-gpu-2026-10-09T11-50-28-864Z.txt

However, I can see the following errors in Pipecat:

2026-10-09 10:45:44.929 | ERROR    | pipecat.processors.frame_processor:push_error_frame:1002 - AnthropicLLMService#0 exception (/Users/filipi/workspaces/workspace_daily/pipecat/src/pipecat/adapters/services/anthropic_adapter.py:211): Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain
2026-10-09 10:45:44.929 | WARNING  | pipecat.processors.aggregators.llm_context:remove_invalid_file_message:629 - Removing message with file or image content from context due to invalid response.
2026-10-09 10:45:44.930 | DEBUG    | pipecat.processors.metrics.frame_processor_metrics:stop_processing_metrics:308 - AnthropicLLMService#0 processing time: 0.004s
2026-10-09 10:45:44.931 | WARNING  | pipecat.pipeline.worker:_source_push_frame:1521 - PipelineWorker#0: Something went wrong: ErrorFrame#0(error: Error mapping context messages to provider format: Unsupported 'file' MIME type: text/plain, fatal: False)

I believe this is the missing piece we discussed yesterday. Just sharing it here in case it’s worth addressing in the example.

hmm. this looks expected to me. Anthropic is not accepting the file type you provided. What were you thinking should happen that didn't?

filipi87 · 2026-10-13T12:43:10Z

What were you thinking should happen that didn't?

I was thinking that it would be nice to reflect in the interface when an error occurs and we are not actually using the file.

For example, we might not have added the file to the context in the first place, or we might have removed it from the context because of an error. In either case, it would be nice to send a specific message to RTVI so that the interface can indicate that the file we thought had been uploaded successfully was actually not accepted.

But, this will probably require some changes to Pipecat as well. So, don't block this PR because of this. 🙂

mattieruth requested review from filipi87, kompfner and markbackman July 16, 2026 20:10

mattieruth mentioned this pull request Jul 16, 2026

Rtvi send file support pipecat-ai/pipecat#3415

Open

mattieruth force-pushed the new-file-upload-example branch from 517490a to 362e8f3 Compare July 23, 2026 21:17

kompfner reviewed Jul 28, 2026

View reviewed changes

Comment thread file-upload/client/src/app.js

kompfner reviewed Jul 28, 2026

View reviewed changes

Comment thread file-upload/client/src/app.js

kompfner approved these changes Jul 28, 2026

View reviewed changes

filipi87 reviewed Jul 28, 2026

View reviewed changes

kompfner reviewed Jul 28, 2026

View reviewed changes

mattieruth force-pushed the new-file-upload-example branch from 96435ea to 437f734 Compare September 16, 2026 19:55

vercel Bot deployed to Preview September 16, 2026 19:55

View deployment

vercel Bot deployed to Preview September 17, 2026 20:00

View deployment

vercel Bot deployed to Preview October 6, 2026 22:19

View deployment

filipi87 reviewed Oct 7, 2026

View reviewed changes

mattieruth and others added 8 commits October 8, 2026 09:19

add file-upload example demonstrating client file/image uploads

2739eda

New example showing how clients can upload local files or reference URLs as visual context during a voice conversation. Uses RTVIProcessor with uploads_folder on the server and client.sendFile via the RTVI protocol.

env file name tweak

8d2eeb8

tmp: adding websocket package. update to non-local version once it ex…

4ef6321

…ists

Add default upload file folder and support for WebSocketTransports

90f9949

Update bot to take an --llm arg to specify which llm to use

25f63ec

Add bot file versions for exemplifying custom file storage on s3 or g…

1b98630

…oogle cloud

Add Google Vertex to llm options

c712289

mattieruth force-pushed the new-file-upload-example branch from 7958504 to 284785a Compare October 8, 2026 13:19

vercel Bot deployed to Preview October 8, 2026 13:19

View deployment

mattieruth added 2 commits October 8, 2026 09:33

Updated README and fixed file upload button to only work when connected

6a7d477

mattieruth force-pushed the new-file-upload-example branch from 284785a to 6a7d477 Compare October 8, 2026 13:33

vercel Bot deployed to Preview October 8, 2026 13:34

View deployment

filipi87 self-requested a review October 8, 2026 21:26

filipi87 reviewed Oct 9, 2026

View reviewed changes

filipi87 approved these changes Oct 9, 2026

View reviewed changes


		This bot uses a cascade pipeline: Speech-to-Text → LLM → Text-to-Speech

		Generated by Pipecat CLI

		[tool.uv.sources]
		pipecat-ai = { path = "/Users/mattieruth/src/pipecat/pipecat", editable=true } No newline at end of file

Conversation

mattieruth commented Jul 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kompfner Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vercel Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

filipi87 commented Oct 9, 2026

Uh oh!

kompfner Jul 28, 2026 •

edited

Loading

vercel Bot commented Sep 16, 2026 •

edited

Loading