Skip to content

feat(widget): Rtc transports discovery for widgets #34393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
robintown merged 8 commits into from
Aug 10, 2026
Merged

feat(widget): Rtc transports discovery for widgets #34393

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
78d7c31
feat(widget): Rtc transports discovery for widgets
BillCarsonFr Jul 22, 2026
afe7754
fix linter
BillCarsonFr Jul 30, 2026
78fe92e
update matrix-widget-api and js-sdk
BillCarsonFr Jul 30, 2026
6a9b02b
fix eslint-plugin-element-call path
BillCarsonFr Jul 30, 2026
edec315
update widget-api: Parent-id delegation is deprectated
BillCarsonFr Jul 30, 2026
eb3ff61
Merge branch 'develop' into valere/rtc/msc4515_widget_rtc_transport_d…
BillCarsonFr Jul 30, 2026
ea67168
Merge branch 'develop' into valere/rtc/msc4515_widget_rtc_transport_d…
BillCarsonFr Jul 31, 2026
1b25ade
Merge branch 'develop' into valere/rtc/msc4515_widget_rtc_transport_d…
robintown Aug 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/web/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
"maplibre-gl": "^5.0.0",
"matrix-encrypt-attachment": "^1.0.3",
"matrix-js-sdk": "github:matrix-org/matrix-js-sdk#develop",
"matrix-widget-api": "^1.16.1",
"matrix-widget-api": "^1.18.0",
"memoize-one": "^6.0.0",
"mime": "^4.0.4",
"opus-recorder": "^8.0.3",
Expand Down
45 changes: 18 additions & 27 deletions apps/web/src/stores/widgets/ElementWidgetDriver.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import {
type IWidgetApiErrorResponseDataDetails,
type ISearchUserDirectoryResult,
type IGetMediaConfigResult,
type IRtcTransportsResult,
} from "matrix-widget-api";
import {
ClientEvent,
Expand All @@ -39,7 +40,6 @@ import {
type StateEvents,
type TimelineEvents,
type Room,
type SendDelayedEventRequestOpts,
type MatrixClient,
} from "matrix-js-sdk/src/matrix";
import { logger } from "matrix-js-sdk/src/logger";
Expand Down Expand Up @@ -123,6 +123,7 @@ export class ElementWidgetDriver extends WidgetDriver {
// This is a trusted Element Call widget that we control
this.allowedCapabilities.add(MatrixCapabilities.AlwaysOnScreen);
this.allowedCapabilities.add(MatrixCapabilities.MSC3846TurnServers);
this.allowedCapabilities.add(MatrixCapabilities.MSC4515RtcTransports);
this.allowedCapabilities.add(`org.matrix.msc2762.timeline:${inRoomId}`);
this.allowedCapabilities.add(MatrixCapabilities.MSC4157SendDelayedEvent);
this.allowedCapabilities.add(MatrixCapabilities.MSC4157UpdateDelayedEvent);
Expand Down Expand Up @@ -405,27 +406,12 @@ export class ElementWidgetDriver extends WidgetDriver {
return stickyEvents;
}

private getSendDelayedEventOpts(delay: number | null, parentDelayId: string | null): SendDelayedEventRequestOpts {

@Half-Shot Half-Shot Aug 3, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the delayed event changes because of the matrix-widget-sdk update? They do seem misplaced in this PR?

@BillCarsonFr BillCarsonFr Aug 3, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the delayed event changes because of the matrix-widget-sdk update? They do seem misplaced in this PR?

Yes they are due to the widget-api-changes (remove parent delayed id), but the js-sdk hasn't yet been updated to the MSC changes, so the parentId related types are still around.

There is a js-sdk draft PR matrix-org/matrix-js-sdk#5199
It would be nicer if this landed first :/, but I think they are a bit entangled

@AndrewFerr WDYT?

@AndrewFerr AndrewFerr Aug 5, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's doubtful that I'll be able to finish that js-sdk PR this week, so as long as it's not a strict requirement of this PR, maybe just charge ahead with this one for now.

@BillCarsonFr BillCarsonFr Aug 7, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Half-Shot is it fine to go ahead for you? there will be a follow up PR to finish the clean up of deprecated parent_id thing for delayed event?

@robintown robintown Aug 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is fairly high-priority and it feels acceptable to me to adapt to the parent ID changes in the same breath as upgrading matrix-widget-api, so I will proceed with getting this merged.

if (delay !== null) {
return {
delay,
...(parentDelayId !== null && { parent_delay_id: parentDelayId }),
};
} else if (parentDelayId !== null) {
return {
parent_delay_id: parentDelayId,
};
}
throw new Error("Must provide at least one of delay or parentDelayId");
}

/**
* @experimental Part of MSC4140 & MSC4157
* @see {@link WidgetDriver#sendDelayedEvent}
*/
public async sendDelayedEvent<K extends keyof StateEvents>(
delay: number | null,
parentDelayId: string | null,
delay: number,
eventType: K,
content: StateEvents[K],
stateKey: string | null,
Expand All @@ -435,23 +421,21 @@ export class ElementWidgetDriver extends WidgetDriver {
* @experimental Part of MSC4140 & MSC4157
*/
public async sendDelayedEvent<K extends keyof TimelineEvents>(
delay: number | null,
parentDelayId: string | null,
delay: number,
eventType: K,
content: TimelineEvents[K],
stateKey: null,
targetRoomId: string | null,
): Promise<ISendDelayedEventDetails>;
public async sendDelayedEvent(
delay: number | null,
parentDelayId: string | null,
delay: number,
eventType: string,
content: IContent,
stateKey: string | null = null,
targetRoomId: string | null = null,
): Promise<ISendDelayedEventDetails> {
const { client, roomId } = this.getSendEventTarget(targetRoomId);
const delayOpts = this.getSendDelayedEventOpts(delay, parentDelayId);
const delayOpts = { delay };

let r: SendDelayedEventResponse | null;
if (stateKey !== null) {
Expand Down Expand Up @@ -485,20 +469,17 @@ export class ElementWidgetDriver extends WidgetDriver {
* @see {@link WidgetDriver#sendStickyEvent}
*/
public async sendDelayedStickyEvent(
delay: number | null,
parentDelayId: string | null,
delay: number,
stickyDurationMs: number,
eventType: string,
content: unknown,
targetRoomId?: string | null,
): Promise<ISendDelayedEventDetails> {
const { client, roomId } = this.getSendEventTarget(targetRoomId);
const delayOpts = this.getSendDelayedEventOpts(delay, parentDelayId);

const r = await client._unstable_sendStickyDelayedEvent(
roomId,
stickyDurationMs,
delayOpts,
{ delay },
null,
eventType as keyof TimelineEvents,
content as TimelineEvents[keyof TimelineEvents] & { msc4354_sticky_key: string },
Expand Down Expand Up @@ -765,6 +746,16 @@ export class ElementWidgetDriver extends WidgetDriver {
}
}

public async getRtcTransports(): Promise<IRtcTransportsResult> {
const client = MatrixClientPeg.safeGet();
// Delegate to the authenticated CS endpoint (MSC4143). Any error (e.g. the
// homeserver not supporting it) propagates and is turned into a widget error
// response by ClientWidgetApi. The js-sdk Transport and widget-api IRtcTransport
// types are structurally identical.
const transports = await client._unstable_getRTCTransports();
return { rtc_transports: transports };
}

public async readEventRelations(
eventId: string,
roomId?: string,
Expand Down
98 changes: 30 additions & 68 deletions apps/web/test/unit-tests/stores/widgets/ElementWidgetDriver-test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import {
type ITurnServer as IClientTurnServer,
Direction,
EventType,
MatrixError,
MatrixEvent,
MsgType,
RelationType,
Expand Down Expand Up @@ -89,6 +90,7 @@ describe("ElementWidgetDriver", () => {
const requestedCapabilities = new Set([
"m.always_on_screen",
"town.robin.msc3846.turn_servers",
"org.matrix.msc4515.rtc_transports",
"org.matrix.msc2762.timeline:!1:example.org",
"org.matrix.msc2762.send.event:org.matrix.msc4075.call.notify",
"org.matrix.msc2762.send.event:org.matrix.msc4075.rtc.notification",
Expand Down Expand Up @@ -420,6 +422,31 @@ describe("ElementWidgetDriver", () => {
});
});

describe("getRtcTransports", () => {
let driver: WidgetDriver;

beforeEach(() => {
driver = mkDefaultDriver();
});

it("gets the RTC transports from the homeserver", async () => {
const transports = [{ type: "livekit", livekit_service_url: "https://livekit-jwt.example.com" }];
client._unstable_getRTCTransports.mockResolvedValue(transports);

await expect(driver.getRtcTransports()).resolves.toEqual({ rtc_transports: transports });

expect(client._unstable_getRTCTransports).toHaveBeenCalledWith();
});

it("propagates errors from the homeserver", async () => {
const error = new MatrixError({ errcode: "M_NOT_FOUND", error: "Not found" }, 404);

client._unstable_getRTCTransports.mockRejectedValue(error);

await expect(driver.getRtcTransports()).rejects.toBe(error);
});
});

describe("readEventRelations", () => {
let driver: WidgetDriver;

Expand Down Expand Up @@ -540,18 +567,12 @@ describe("ElementWidgetDriver", () => {
driver = mkDefaultDriver();
});

it("cannot send delayed events with missing arguments", async () => {
await expect(driver.sendDelayedEvent(null, null, EventType.RoomMessage, {})).rejects.toThrow(
"Must provide at least one of",
);
});

it("sends delayed message events", async () => {
client._unstable_sendDelayedEvent.mockResolvedValue({
delay_id: "id",
});

await expect(driver.sendDelayedEvent(2000, null, EventType.RoomMessage, {})).resolves.toEqual({
await expect(driver.sendDelayedEvent(2000, EventType.RoomMessage, {})).resolves.toEqual({
roomId,
delayId: "id",
});
Expand All @@ -565,31 +586,12 @@ describe("ElementWidgetDriver", () => {
);
});

it("sends child action delayed message events", async () => {
client._unstable_sendDelayedEvent.mockResolvedValue({
delay_id: "id-child",
});

await expect(driver.sendDelayedEvent(null, "id-parent", EventType.RoomMessage, {})).resolves.toEqual({
roomId,
delayId: "id-child",
});

expect(client._unstable_sendDelayedEvent).toHaveBeenCalledWith(
roomId,
{ parent_delay_id: "id-parent" },
null,
EventType.RoomMessage,
{},
);
});

it("sends delayed state events", async () => {
client._unstable_sendDelayedStateEvent.mockResolvedValue({
delay_id: "id",
});

await expect(driver.sendDelayedEvent(2000, null, EventType.RoomTopic, {}, "")).resolves.toEqual({
await expect(driver.sendDelayedEvent(2000, EventType.RoomTopic, {}, "")).resolves.toEqual({
roomId,
delayId: "id",
});
Expand All @@ -602,25 +604,6 @@ describe("ElementWidgetDriver", () => {
"",
);
});

it("sends child action delayed state events", async () => {
client._unstable_sendDelayedStateEvent.mockResolvedValue({
delay_id: "id-child",
});

await expect(driver.sendDelayedEvent(null, "id-parent", EventType.RoomTopic, {}, "")).resolves.toEqual({
roomId,
delayId: "id-child",
});

expect(client._unstable_sendDelayedStateEvent).toHaveBeenCalledWith(
roomId,
{ parent_delay_id: "id-parent" },
EventType.RoomTopic,
{},
"",
);
});
});

describe("updateDelayedEvent", () => {
Expand Down Expand Up @@ -743,7 +726,7 @@ describe("ElementWidgetDriver", () => {
delay_id: "id",
});

await expect(driver.sendDelayedStickyEvent(1000, null, 2000, EventType.RoomMessage, {})).resolves.toEqual({
await expect(driver.sendDelayedStickyEvent(1000, 2000, EventType.RoomMessage, {})).resolves.toEqual({
roomId,
delayId: "id",
});
Expand All @@ -757,27 +740,6 @@ describe("ElementWidgetDriver", () => {
{},
);
});
it("sends child action delayed sticky message events", async () => {
client._unstable_sendStickyDelayedEvent.mockResolvedValue({
delay_id: "id-child",
});

await expect(
driver.sendDelayedStickyEvent(null, "id-parent", 2000, EventType.RoomMessage, {}),
).resolves.toEqual({
roomId,
delayId: "id-child",
});

expect(client._unstable_sendStickyDelayedEvent).toHaveBeenCalledWith(
roomId,
2000,
{ parent_delay_id: "id-parent" },
null,
EventType.RoomMessage,
{},
);
});
});

describe("If the feature_dynamic_room_predecessors feature is not enabled", () => {
Expand Down
34 changes: 17 additions & 17 deletions pnpm-lock.yaml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading