Repository navigation
Commit f38055a
Data streams v2 (#1166)
Initial port of data streams v2 to the flutter sdk. This is built on top
of #1160, and that PR must be merged first.
## Summary
<details><summary>Background context on data streams v2</summary>
<p>
Previously, the flutter sdk had its own data streams implementation.
With [data streams
v2](livekit/client-sdk-js#1985), the rust
implementation will both be quite a bit more stable and gain some new
features that make it significantly more performant (single packet data
streams and DEFLATE compression when it makes the payload smaller). This
roughly _doubles_ data stream throughput in local testing.
</p>
</details>
In the corresponding
[swift](livekit/client-sdk-swift#1075) and
[android](livekit/client-sdk-android#997) pull
requests, I was able to fully cut over to data streams v2. However, the
flutter sdk also has to support web, and today uniffi-dart does not
support WASM - see
[here](https://redirect.github.com/Uniffi-Dart/uniffi-dart/issues/106)
for an early issue on the bindgen side tracking this work.
So in lieu of fully cutting over:
- All platforms except for web use the rust provided data streams v2
implementation
- Web keeps the existing pure dart data streams v1 implementation.
Note that it could be possible to make flutter web use the web sdk's
data stream v2 implementation, but it looks like it would introduce a
lot of infrastructure complexity. Given this and that it seems like
flutter web support is somewhat experimental anyway, I'm tempted to keep
it on the existing implementation.
### New behaviors worth being aware of
All of these are either data streams v2 related changes, or bug fixes.
#### 1. `compress` option
When sending a data stream, there is a new `compress` option. Just like
how this works on web / rust, this defaults to `true`. If set to
`false`, then compression will be disabled (useful if you know the data
you are sending isn't compressible / you are doing your own compression,
which is not uncommon in robotics use cases). The vast majority of users
should leave this set to `true`.
#### 2. Max data stream size
In a [rust data streams v2 pull request review
comment](livekit/rust-sdks#1192 (comment)),
we decided that for security reasons it made sense to introduce a
maximum data stream size as a DOS protection. This limit is by default
`5gb` - any data stream that is larger will now read up until that
point, and if the stream keeps going, a "payload too large" error will
be raised on the stream and exposed to a user on the subsequent
`.read()` call.
If a user is sending a large file, they can override this by setting a
new `maxPayloadByteLength` option on the `room.connect` call:
```kotlin
await room.connect(
wsUrl,
token,
connectOptions: const ConnectOptions(
autoSubscribe: true,
dataStream: const DataStreamOptions(maxPayloadByteLength = 1000),
),
);
```
#### 3. Throwing new data stream errors types
The old flutter specific data streams implementation wasn't quite as
strict and didn't surface as many error cases to the caller which were
encountered while reading the stream as the rust implementation now
does. This needs some testing in some of these edge cases to make sure I
didn't inadvertently make this backwards incompatible in a non-aceptable
way.
-----------------
> [!WARNING]
> This pull request was LLM generated and has only been reviewed by a
human who isn't a domain expert in android development. I have tested
this and confirms it works in the happy path, but no other validation
has been done.
>
> A more thorough review of this needs to occur before it could be
merged.
## Todo
- [ ] Get #1160 merged and released.
- [ ] Drop c14d0e7 once
#1169 is merged, this
commit duplicates that fix in here
- [ ] Test this thoroughly. I've run a happy path test and that's it.
I'm holding off on doing more until #1160 is in a more stable place.
- [x] Add changeset
---------
Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>1 parent 81a79ad commit f38055a
30 files changed
Lines changed: 2697 additions & 538 deletions
File tree
- .changes
- lib
- src
- core
- data_stream
- internal
- participant
- types
- test
- core
- ffi
- mock
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
40 | 49 | | |
41 | 50 | | |
42 | 51 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
25 | 26 | | |
26 | 27 | | |
27 | 28 | | |
| |||
69 | 70 | | |
70 | 71 | | |
71 | 72 | | |
| 73 | + | |
72 | 74 | | |
73 | 75 | | |
74 | 76 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1166 | 1166 | | |
1167 | 1167 | | |
1168 | 1168 | | |
1169 | | - | |
1170 | | - | |
| 1169 | + | |
| 1170 | + | |
| 1171 | + | |
| 1172 | + | |
1171 | 1173 | | |
1172 | | - | |
1173 | | - | |
1174 | | - | |
1175 | | - | |
1176 | | - | |
1177 | | - | |
1178 | | - | |
1179 | | - | |
1180 | | - | |
1181 | | - | |
1182 | | - | |
1183 | | - | |
1184 | | - | |
1185 | | - | |
1186 | | - | |
1187 | | - | |
1188 | | - | |
1189 | | - | |
1190 | | - | |
1191 | | - | |
| 1174 | + | |
| 1175 | + | |
1192 | 1176 | | |
1193 | 1177 | | |
1194 | 1178 | | |
| |||
0 commit comments