Skip to content

Commit cf834b6

Browse files
authored
Merge pull request #1583 from pipecat-ai/docs/pr-6351
docs: update for pipecat PR #6351
2 parents 34d4842 + ef03b50 commit cf834b6

2 files changed

Lines changed: 24 additions & 16 deletions

File tree

‎client/rtvi-standard.mdx‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -391,27 +391,35 @@ What the user said or typed. The observer sends one for each transcription (befo
391391

392392
#### bot-output 🤖
393393

394-
The best-effort representation of the bot's output text, including both spoken and unspoken text. In addition to transcriptions of spoken text, this message type may also include text that the bot outputs but does not speak (e.g., text sent to the client for display purposes only). Along with the text, this event includes a `text_type` field to indicate what the text represents (e.g. "sentence", "word", "code", "url") and fields describing whether and how far the bot has spoken it.
394+
The best-effort representation of the bot's output text, including both spoken and unspoken text. In addition to transcriptions of spoken text, this message type may also include text that the bot outputs but does not speak (e.g., text sent to the client for display purposes only). Along with the text, this event includes an `output_type` field to indicate what the text represents (e.g. "sentence", "word", "code", "url") and fields describing whether and how far the bot has spoken it.
395395

396396
The bot's output is split into segments, such as sentences, each with its own `segment_id`. For clients of protocol 2.0.0 or later, a spoken segment is reported as it is spoken:
397397

398398
- If the TTS service reports word timestamps, the server sends the segment once with `spoken_status: "new"` before it is spoken, then again as each word is spoken, with `spoken_status: "in-progress"` and the last with `"completed"`. Every message carries the segment's full `text` and the same `segment_id`.
399399
- Otherwise, the server sends the segment once, with `spoken_status: "completed"`, after it is spoken.
400400

401-
Text the bot won't speak is sent once, with `will_be_spoken: false` and `spoken_status: "new"`. Clients of protocol 2.0.0 or later don't receive separate `bot-output` messages for individual words (`text_type` `"word"`); word-level progress is in `spoken_progress` instead. Clients on 1.x receive each spoken word as its own message and should read `spoken`.
401+
Text the bot won't speak is sent once, with `will_be_spoken: false` and `spoken_status: "new"`. Clients of protocol 2.0.0 or later don't receive separate `bot-output` messages for individual words (`output_type` `"word"`); word-level progress is in `spoken_progress` instead. Clients on 1.x receive each spoken word as its own message and should read `spoken`.
402402

403403
- **type**: `'bot-output'`
404404
- **data**:
405405
- **text**: `string`
406406

407407
The output text from the bot.
408408

409-
- **text_type**: `string`
409+
- **output_type**: `string`
410410

411-
Indicates what kind of text this is (e.g., "sentence", "word", "code", "url"). "sentence", "word", and "backchannel" are reserved text types defined by the RTVI standard. Other text types may be defined by custom text aggregators used by the server.
411+
Indicates what kind of text this is (e.g., "sentence", "word", "code", "url"). "sentence", "word", and "backchannel" are reserved output types defined by the RTVI standard. Other output types may be defined by custom text aggregators used by the server.
412412

413413
A short acknowledgment the bot says, such as "Mm-hmm.", has type `backchannel`. It is only sent to clients of protocol 2.2.0 or later.
414414

415+
- **text_type**: `string` (deprecated)
416+
417+
Deprecated. Use `output_type` instead, which carries the same value. Will be removed in 2.0.0.
418+
419+
- **aggregated_by**: `string` (deprecated)
420+
421+
Deprecated. Use `output_type` instead, which carries the same value. Will be removed in 2.0.0.
422+
415423
- **segment_id**: `number` (optional)
416424

417425
Identifies the segment this text belongs to, so a client can match a segment's `"new"`, `"in-progress"`, and `"completed"` messages.
@@ -439,10 +447,6 @@ Text the bot won't speak is sent once, with `will_be_spoken: false` and `spoken_
439447

440448
Indicates if this text was spoken by the bot. Clients of protocol 2.0.0 or later should read `will_be_spoken` and `spoken_status` instead.
441449

442-
- **aggregated_by**: `string`
443-
444-
Deprecated. Use `text_type` instead. The server sends both fields with the same value.
445-
446450
#### bot-transcription 🤖
447451

448452
<Warning>

‎llms-full.txt‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22835,27 +22835,35 @@ What the user said or typed. The observer sends one for each transcription (befo
2283522835

2283622836
#### bot-output 🤖
2283722837

22838-
The best-effort representation of the bot's output text, including both spoken and unspoken text. In addition to transcriptions of spoken text, this message type may also include text that the bot outputs but does not speak (e.g., text sent to the client for display purposes only). Along with the text, this event includes a `text_type` field to indicate what the text represents (e.g. "sentence", "word", "code", "url") and fields describing whether and how far the bot has spoken it.
22838+
The best-effort representation of the bot's output text, including both spoken and unspoken text. In addition to transcriptions of spoken text, this message type may also include text that the bot outputs but does not speak (e.g., text sent to the client for display purposes only). Along with the text, this event includes an `output_type` field to indicate what the text represents (e.g. "sentence", "word", "code", "url") and fields describing whether and how far the bot has spoken it.
2283922839

2284022840
The bot's output is split into segments, such as sentences, each with its own `segment_id`. For clients of protocol 2.0.0 or later, a spoken segment is reported as it is spoken:
2284122841

2284222842
- If the TTS service reports word timestamps, the server sends the segment once with `spoken_status: "new"` before it is spoken, then again as each word is spoken, with `spoken_status: "in-progress"` and the last with `"completed"`. Every message carries the segment's full `text` and the same `segment_id`.
2284322843
- Otherwise, the server sends the segment once, with `spoken_status: "completed"`, after it is spoken.
2284422844

22845-
Text the bot won't speak is sent once, with `will_be_spoken: false` and `spoken_status: "new"`. Clients of protocol 2.0.0 or later don't receive separate `bot-output` messages for individual words (`text_type` `"word"`); word-level progress is in `spoken_progress` instead. Clients on 1.x receive each spoken word as its own message and should read `spoken`.
22845+
Text the bot won't speak is sent once, with `will_be_spoken: false` and `spoken_status: "new"`. Clients of protocol 2.0.0 or later don't receive separate `bot-output` messages for individual words (`output_type` `"word"`); word-level progress is in `spoken_progress` instead. Clients on 1.x receive each spoken word as its own message and should read `spoken`.
2284622846

2284722847
- **type**: `'bot-output'`
2284822848
- **data**:
2284922849
- **text**: `string`
2285022850

2285122851
The output text from the bot.
2285222852

22853-
- **text_type**: `string`
22853+
- **output_type**: `string`
2285422854

22855-
Indicates what kind of text this is (e.g., "sentence", "word", "code", "url"). "sentence", "word", and "backchannel" are reserved text types defined by the RTVI standard. Other text types may be defined by custom text aggregators used by the server.
22855+
Indicates what kind of text this is (e.g., "sentence", "word", "code", "url"). "sentence", "word", and "backchannel" are reserved output types defined by the RTVI standard. Other output types may be defined by custom text aggregators used by the server.
2285622856

2285722857
A short acknowledgment the bot says, such as "Mm-hmm.", has type `backchannel`. It is only sent to clients of protocol 2.2.0 or later.
2285822858

22859+
- **text_type**: `string` (deprecated)
22860+
22861+
Deprecated. Use `output_type` instead, which carries the same value. Will be removed in 2.0.0.
22862+
22863+
- **aggregated_by**: `string` (deprecated)
22864+
22865+
Deprecated. Use `output_type` instead, which carries the same value. Will be removed in 2.0.0.
22866+
2285922867
- **segment_id**: `number` (optional)
2286022868

2286122869
Identifies the segment this text belongs to, so a client can match a segment's `"new"`, `"in-progress"`, and `"completed"` messages.
@@ -22883,10 +22891,6 @@ Text the bot won't speak is sent once, with `will_be_spoken: false` and `spoken_
2288322891

2288422892
Indicates if this text was spoken by the bot. Clients of protocol 2.0.0 or later should read `will_be_spoken` and `spoken_status` instead.
2288522893

22886-
- **aggregated_by**: `string`
22887-
22888-
Deprecated. Use `text_type` instead. The server sends both fields with the same value.
22889-
2289022894
#### bot-transcription 🤖
2289122895

2289222896
<Warning>

0 commit comments

Comments
 (0)