Skip to content

Commit edd4555

Browse files
authored
Announce engine closure on disposal and terminal reconnect failure (#1268)
Disposing a Room or giving up on reconnection tore down the engine without marking it closed or emitting `EngineClosingEvent`. Subscribers waiting for closure never received it, and an unrecoverable reconnect left `isClosed` false. This change centralizes terminal closure in an idempotent internal `Engine.markClosed()`. It marks the engine closed synchronously and queues `EngineClosingEvent` while the event emitter is still alive. ### Disposal sequence `Room.dispose()` calls `engine.markClosed()` before `super.dispose()` and before any disposal callbacks. This placement matters because Room's cleanup itself calls `engine.cleanUp()`, before it eventually calls `engine.dispose()`. Announcing closure only in Engine's constructor-registered disposal callback is too late for that first transport teardown. `Engine.dispose()` also calls `markClosed()` before `super.dispose()`. Disposal callbacks run in reverse registration order, so callbacks registered later can safely await engine work that is already being cancelled. Engine's constructor-registered callback retains its cleanup sequence: clean up transports, dispose the emitter, dispose signal listeners, and release reliable receive state. Explicit disconnect, retry exhaustion, and unrecoverable reconnect failure use the same closure helper before cleanup. Recoverable retries and full-reconnect transport replacement do not close the engine. Buffer waits started after closure reject immediately, even if the retained channel reports a low buffer. Closing-event delivery remains asynchronous. Disposal does not await completion of arbitrary event handlers. ### What it fixes for a user A room disposed while a full reconnect was tearing down the old connection used to join again with the old token about a second later, as a new participant sid. `restartConnection` now checks closure after the teardown and stops. A dispose or disconnect that lands later, while the reconnect is opening its new signal connection, still rejoins. That is pre-existing, the same on `main`, and left for the session follow-up. ### Buffer wait error type `waitForBufferStatusLow()` used to throw, or fail its wait with, the plain string `'Engine disconnected'`. Data stream writes surface that error, so an app catching `LiveKitException` missed it. Both paths now use `UnexpectedConnectionState`, which client-sdk-js also throws when its engine is closed. ### Relationship to other fixes - #1264, now merged, keeps an explicit disconnect from being undone by a full reconnect. This branch is rebased on `main` and includes it. - #1267 is stacked on this PR, with #1277 on top of that. Merge #1267 right after this one, before a release. Here, a data stream write that is already blocked on a full send buffer still waits on the old completer. When the engine closes, that completer fails with nothing listening, so `disconnect()`, and now also `dispose()`, raise an uncaught `UnexpectedConnectionState` while the write carries on. `main` already does this on `disconnect()`, with the string error. With #1267 on top, the blocked write fails with `UnexpectedConnectionState` and nothing is left uncaught. ### Validation - Nine lifecycle regression tests in `engine_closing_test.dart` cover closure events, recoverable versus terminal reconnect failures, waits started after closure, duplicate closure, and disposal ordering. One verifies that the closing event is delivered with Engine marked disposed but its separate emitter still alive. - `disconnect_during_reconnect_test.dart` disposes the room while a full reconnect is still tearing down, the case where this PR meets #1264. It fails on `main`, where the reconnect opens a new signal socket, and passes here. - Against `main`'s `lib`, 10 of the 13 tests in those two files fail. Removing `engine.markClosed()` from `Room.dispose()` fails 3 of them, removing the post-teardown `_isClosed` check in `restartConnection` fails 2. - Full unit suite at 8d4c725: 521 passed, 1 skipped. `flutter analyze`, `dart analyze web/`, repository format, import order, version consistency, and `git diff --check` pass. - e2e-flutter on macOS against a local `livekit-server --dev` passes Connection basics, PingPong, duplicate identity, Delete Room, both data channel scenarios, full reconnect, disconnect during a full reconnect, migration and node failure. - livekit/e2e-flutter#23 adds the matching scenario, "Dispose during a full reconnect stays gone". It fails in 3 of 3 runs on `main` (88c72c6), where the identity joins again with a new participant sid about 1 s after `dispose()`, and passes in 3 of 3 here. On this head, the scenarios tagged `reconnect`, `data` and `signaling` pass, apart from expected failures and "Reliable data across a resume", the known gap until #1277. - A blocked data stream write with `disconnect()` or `dispose()` was checked with a throwaway test on both branches, with the results described above.
1 parent 88c72c6 commit edd4555

6 files changed

Lines changed: 247 additions & 8 deletions

File tree

‎.changes/buffer-wait-typed-error‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
patch type="fixed" "Data stream writes on a closed engine fail with UnexpectedConnectionState instead of a plain string error"

‎.changes/engine-terminal-closure‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
patch type="fixed" "Fix a room that rejoined after being disposed while a full reconnect was tearing down, and fail data stream writes started after the room was disposed"

‎lib/src/core/engine.dart‎

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -242,14 +242,20 @@ class Engine extends Disposable with EventsEmittable<EngineEvent> {
242242
_setUpSignalListeners();
243243

244244
onDispose(() async {
245-
_isClosed = true;
246245
await cleanUp();
247246
await events.dispose();
248247
await _signalListener.dispose();
249248
_reliableReceivedState.dispose();
250249
});
251250
}
252251

252+
@override
253+
Future<bool> dispose() {
254+
// Disposal callbacks run in reverse registration order and may await engine work.
255+
markClosed();
256+
return super.dispose();
257+
}
258+
253259
Future<void> connect(
254260
String url,
255261
String token, {
@@ -417,14 +423,19 @@ class Engine extends Disposable with EventsEmittable<EngineEvent> {
417423
}
418424

419425
Future<void> waitForBufferStatusLow(Reliability kind) async {
426+
// A wait started after closure cannot observe the closing event.
427+
if (_isClosed || isDisposed) {
428+
throw UnexpectedConnectionState('Engine disconnected');
429+
}
430+
420431
final Completer<void> completer = Completer();
421432

422433
if (isBufferStatusLow(kind) == true) {
423434
completer.complete();
424435
} else {
425436
onClosing() {
426437
if (!completer.isCompleted) {
427-
completer.completeError('Engine disconnected');
438+
completer.completeError(UnexpectedConnectionState('Engine disconnected'));
428439
}
429440
}
430441

@@ -1151,7 +1162,7 @@ class Engine extends Disposable with EventsEmittable<EngineEvent> {
11511162

11521163
if (_reconnectAttempts >= _reconnectCount) {
11531164
logger.fine('reconnectAttempts exceeded, disconnecting...');
1154-
_isClosed = true;
1165+
markClosed();
11551166
await cleanUp();
11561167

11571168
events.emit(
@@ -1263,6 +1274,7 @@ class Engine extends Disposable with EventsEmittable<EngineEvent> {
12631274
unawaited(handleReconnect(ClientDisconnectReason.reconnectRetry));
12641275
} else {
12651276
logger.fine('attemptReconnect: disconnecting...');
1277+
markClosed();
12661278
// clean up before emitting, room's EngineDisconnectedEvent handler
12671279
// drops the event while fullReconnectOnNext is still true and
12681280
// cleanUp() is what resets it
@@ -1677,11 +1689,19 @@ class Engine extends Disposable with EventsEmittable<EngineEvent> {
16771689
events.emit(EngineRoomMovedEvent(response: event.response));
16781690
});
16791691

1692+
// Announce terminal closure before tearing down transports or event listeners.
1693+
// A full reconnect also disposes transports, but must keep the engine open.
1694+
@internal
1695+
void markClosed() {
1696+
if (_isClosed) return;
1697+
_isClosed = true;
1698+
events.emit(EngineClosingEvent());
1699+
}
1700+
16801701
Future<void> disconnect({
16811702
DisconnectReason reason = DisconnectReason.clientInitiated,
16821703
}) async {
1683-
_isClosed = true;
1684-
events.emit(EngineClosingEvent());
1704+
markClosed();
16851705
if (connectionState == ConnectionState.connected) {
16861706
await signalClient.sendLeave();
16871707
} else {

‎lib/src/core/room.dart‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,13 @@ class Room extends DisposableChangeNotifier with EventsEmittable<RoomEvent> {
241241
});
242242
}
243243

244+
@override
245+
Future<bool> dispose() {
246+
// Room cleanup tears down transports before Engine.dispose() is reached.
247+
engine.markClosed();
248+
return super.dispose();
249+
}
250+
244251
/// prepareConnection should be called as soon as the page is loaded, in order
245252
/// to speed up the connection attempt. This function will
246253
/// - perform DNS resolution and pre-warm the DNS cache

‎test/core/disconnect_during_reconnect_test.dart‎

Lines changed: 34 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@
1212
// See the License for the specific language governing permissions and
1313
// limitations under the License.
1414

15-
// disconnect() closes the engine for good, even when a full reconnect is
16-
// already tearing down the old connection and is about to connect again, and
17-
// the room can connect again afterwards.
15+
// disconnect() and Room.dispose() close the engine for good, even when a full
16+
// reconnect is already tearing down the old connection and is about to connect
17+
// again, and the room can connect again after disconnect().
1818

1919
@Timeout(Duration(seconds: 10))
2020
library;
@@ -93,4 +93,35 @@ void main() {
9393
await container.connectRoom();
9494
expect(room.connectionState, ConnectionState.connected);
9595
});
96+
97+
test('disposing the room during a full reconnect does not connect again', () async {
98+
final restartPaused = Completer<void>();
99+
final resumeRestart = Completer<void>();
100+
container.engine.publisher!.onDispose(() async {
101+
restartPaused.complete();
102+
await resumeRestart.future;
103+
});
104+
final disposePaused = Completer<void>();
105+
final resumeDispose = Completer<void>();
106+
container.engine.subscriber!.onDispose(() async {
107+
disposePaused.complete();
108+
await resumeDispose.future;
109+
});
110+
111+
container.engine.fullReconnectOnNext = true;
112+
final droppedHandlers = ws.handlers;
113+
ws.onDispose();
114+
await restartPaused.future;
115+
116+
// The room's teardown reaches the subscriber while the full reconnect still holds the publisher.
117+
final disposing = room.dispose();
118+
await disposePaused.future;
119+
resumeRestart.complete();
120+
// Give a reconnect that was not stopped time to open its socket.
121+
await Future<void>.delayed(const Duration(milliseconds: 500));
122+
resumeDispose.complete();
123+
await disposing;
124+
125+
expect(identical(ws.handlers, droppedHandlers), isTrue, reason: 'the full reconnect opened a new signal socket');
126+
});
96127
}

‎test/core/engine_closing_test.dart‎

Lines changed: 179 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
// Copyright 2026 LiveKit, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
@Timeout(Duration(seconds: 5))
16+
library;
17+
18+
import 'dart:async';
19+
20+
import 'package:flutter_test/flutter_test.dart';
21+
22+
import 'package:livekit_client/livekit_client.dart';
23+
import 'package:livekit_client/src/core/engine.dart';
24+
import 'package:livekit_client/src/internal/events.dart';
25+
import 'package:livekit_client/src/types/internal.dart';
26+
import '../mock/e2e_container.dart';
27+
import '../mock/peerconnection_mock.dart';
28+
29+
void main() {
30+
TestWidgetsFlutterBinding.ensureInitialized();
31+
32+
late E2EContainer container;
33+
late List<EngineEvent> events;
34+
35+
setUp(() async {
36+
resetMockDataChannels();
37+
container = E2EContainer();
38+
await container.connectRoom();
39+
events = [];
40+
container.engine.events.listen(events.add);
41+
});
42+
43+
tearDown(() async {
44+
await container.dispose();
45+
});
46+
47+
test('disposing the room announces engine closure', () async {
48+
await container.room.dispose();
49+
50+
expect(events.whereType<EngineClosingEvent>(), hasLength(1));
51+
expect(container.engine.isClosed, isTrue);
52+
});
53+
54+
test('room closure is announced while the first transport teardown is blocked', () async {
55+
final teardownStarted = Completer<void>();
56+
final releaseTeardown = Completer<void>();
57+
container.engine.publisher!.onDispose(() async {
58+
teardownStarted.complete();
59+
await releaseTeardown.future;
60+
});
61+
62+
final disposing = container.room.dispose();
63+
await teardownStarted.future;
64+
final closedDuringTeardown = container.engine.isClosed;
65+
final closingEventsDuringTeardown = events.whereType<EngineClosingEvent>().length;
66+
final emitterDisposedDuringTeardown = container.engine.events.isDisposed;
67+
releaseTeardown.complete();
68+
await disposing;
69+
70+
expect(closedDuringTeardown, isTrue);
71+
expect(closingEventsDuringTeardown, 1);
72+
expect(emitterDisposedDuringTeardown, isFalse);
73+
});
74+
75+
for (final owner in ['room', 'engine']) {
76+
test('$owner is marked closed before later disposal callbacks run', () async {
77+
final disposable = owner == 'room' ? container.room : container.engine;
78+
bool? closedInCallback;
79+
disposable.onDispose(() async {
80+
closedInCallback = container.engine.isClosed;
81+
});
82+
83+
if (owner == 'room') {
84+
await container.room.dispose();
85+
} else {
86+
await container.engine.dispose();
87+
}
88+
89+
expect(closedInCallback, isTrue);
90+
expect(events.whereType<EngineClosingEvent>(), hasLength(1));
91+
});
92+
}
93+
94+
test('engine closure is delivered while disposing with its emitter still alive', () async {
95+
bool? engineDisposedWhenDelivered;
96+
bool? emitterDisposedWhenDelivered;
97+
container.engine.events.once<EngineClosingEvent>((_) {
98+
engineDisposedWhenDelivered = container.engine.isDisposed;
99+
emitterDisposedWhenDelivered = container.engine.events.isDisposed;
100+
});
101+
102+
await container.engine.dispose();
103+
104+
expect(engineDisposedWhenDelivered, isTrue);
105+
expect(emitterDisposedWhenDelivered, isFalse);
106+
expect(container.engine.events.isDisposed, isTrue);
107+
});
108+
109+
test('an unrecoverable reconnect announces closure before disconnection', () async {
110+
container.wsConnector.connectError = CertificatePinningException(
111+
'Certificate pin mismatch',
112+
host: 'www.example.com',
113+
);
114+
final disconnected = container.engine.events.waitFor<EngineDisconnectedEvent>(
115+
duration: const Duration(seconds: 1),
116+
);
117+
118+
await container.engine.attemptReconnect(ClientDisconnectReason.reconnectRetry);
119+
expect((await disconnected).reason, DisconnectReason.signalingConnectionFailure);
120+
121+
expect(container.engine.isClosed, isTrue);
122+
expect(events.whereType<EngineClosingEvent>(), hasLength(1));
123+
expect(
124+
events.indexWhere((event) => event is EngineClosingEvent),
125+
lessThan(events.indexWhere((event) => event is EngineDisconnectedEvent)),
126+
);
127+
});
128+
129+
test('exhausting reconnect attempts announces engine closure', () async {
130+
// A generic resume failure stays recoverable without escalating to a full reconnect.
131+
container.wsConnector.connectError = Exception('temporary connection failure');
132+
final disconnected = container.engine.events.waitFor<EngineDisconnectedEvent>(
133+
duration: const Duration(seconds: 1),
134+
);
135+
136+
// Drive the attempts directly rather than waiting through the retry delays.
137+
for (var i = 0; i < defaultRetryDelaysInMs.length; i++) {
138+
await container.engine.attemptReconnect(ClientDisconnectReason.reconnectRetry);
139+
}
140+
expect((await disconnected).reason, DisconnectReason.reconnectAttemptsExceeded);
141+
142+
expect(container.engine.isClosed, isTrue);
143+
expect(events.whereType<EngineClosingEvent>(), hasLength(1));
144+
expect(
145+
events.indexWhere((event) => event is EngineClosingEvent),
146+
lessThan(events.indexWhere((event) => event is EngineDisconnectedEvent)),
147+
);
148+
});
149+
150+
test('a recoverable reconnect does not announce engine closure', () async {
151+
container.wsConnector.connectError = Exception('temporary connection failure');
152+
153+
await container.engine.attemptReconnect(ClientDisconnectReason.reconnectRetry);
154+
155+
expect(container.engine.isClosed, isFalse);
156+
expect(events.whereType<EngineClosingEvent>(), isEmpty);
157+
expect(events.whereType<EngineDisconnectedEvent>(), isEmpty);
158+
});
159+
160+
test('a buffer wait started after disposal rejects even when the channel reports low', () async {
161+
await container.room.dispose();
162+
163+
await expectLater(
164+
container.engine.waitForBufferStatusLow(Reliability.reliable),
165+
throwsA(isA<UnexpectedConnectionState>()),
166+
);
167+
});
168+
169+
test('disconnect followed by disposal announces closure only once', () async {
170+
await container.engine.disconnect();
171+
await expectLater(
172+
container.engine.waitForBufferStatusLow(Reliability.reliable),
173+
throwsA(isA<UnexpectedConnectionState>()),
174+
);
175+
await container.room.dispose();
176+
177+
expect(events.whereType<EngineClosingEvent>(), hasLength(1));
178+
});
179+
}

0 commit comments

Comments
 (0)