Streaming Architecture

This guide explains the common streaming architecture used by Suki modalities that support WebSocket streaming: Ambient, Form filling, and Dictation. The basic flow is the same across all three modalities:

Create a session using a REST API.
Open a WebSocket connection.
Stream live audio over the WebSocket.
End the session using a REST API.

Ambient, Form filling, and Dictation all use the same JSON-over-WebSocket pattern. Send each audio message as a UTF-8 JSON text frame containing exactly one JSON object. The overall streaming pattern is shared, but each modality has its own endpoints, message format, and workflow for retrieving results.

Use the same base host for REST and WebSocket connections within an environment. For example:REST: https://sdp.suki-stage.com WebSocket: wss://sdp.suki-stage.comYour Suki partnership team will provide the correct host and credentials for your environment.

Read the following related guides:

Click Each Button to Learn More

Streaming Comparison Wire Format Stream Ambient Audio Stream Dictation Audio

This guide applies to: Direct HTTP and WebSocket integrations with Suki for Partner Ambient, Form filling, and Dictation APIs. Form filling uses the same ambient socket (GET /ws/stream) with an ambient session ID.

Do not send raw audio as binary WebSocket frames on /ws/stream or /ws/transcribe. Outbound audio is Base64 inside JSON text frames.

What this guide covers

Use this page to understand the end-to-end WebSocket streaming flow before implementing a client. This guide explains:

What Ambient, Form filling, and Dictation have in common.
How their WebSocket endpoints, messages, and result workflows differ.
The network and connection rules that apply across the streaming workflows.
The step-by-step flow for Ambient, Form filling, and Dictation.

For the exact JSON message shapes, audio encoding, and chunking rules, see WebSocket streaming wire format. For a shorter endpoint and workflow comparison, see Ambient vs Dictation streaming.

What’s common for WebSocket streaming

Ambient, Form filling, and Dictation use different WebSocket contracts, but they follow the same high-level integration pattern:

Common steps for WebSocket streaming

Authenticate with Suki. Use the Partner APIs to authenticate and obtain an sdp_suki_token. See Partner authentication.
Create a session with REST. Create the appropriate session using the REST API before opening the WebSocket connection.
Open a secure WebSocket connection. Connect using wss:// and use the same base host as your REST API connection.
Send JSON text frames. Each WebSocket message must be a UTF-8 text frame containing exactly one JSON object. See WebSocket streaming wire format.
Send audio as Base64 inside JSON. Encode the audio as Base64 and place it in the audio field required by the workflow. Do not send audio as a binary WebSocket frame.
End the WebSocket stream according to the workflow. After the final audio chunk, send the required end-of-stream message for that workflow. The exact message differs between Ambient/Form filling and Dictation.
Complete the session and retrieve final data using REST when required. WebSocket streaming and REST session APIs work together. Depending on the workflow, use REST to complete the session and retrieve final results. See Complete the session after streaming.

These steps describe the common integration pattern only. Ambient, Form filling, and Dictation have different WebSocket endpoints, session states, message formats, stream-start and stream-end requirements, and result-handling workflows.Follow the workflow-specific instructions in the comparison below and in How streaming works.

Ambient/Form filling vs Dictation WebSocket streaming

Ambient, Form filling, and Dictation all use WebSockets to stream audio, but they support different use cases and have different streaming contracts.

Ambient streams clinical visit audio to Suki for note generation and related processing. See Stream Ambient audio.
Form filling can use the Ambient WebSocket when it runs on the same socket. See Ambient vs Dictation streaming.
Dictation streams speech to Suki and returns transcript text in real time. See Stream Dictation audio.

Topic	Ambient / Form filling	Dictation
Purpose	Stream encounter audio to Suki for note generation and related processing.	Stream speech to Suki and receive transcript text in real time.
WebSocket endpoint	`GET /ws/stream`	`GET /ws/transcribe`
Session ID	`ambient_session_id`. Ambient and Form filling both use this field name, but each has a different session ID.	`transcription_session_id`
When to open the WebSocket	Open while the Ambient job is still in `CREATED`. See Open WebSocket `GET /ws/stream` and Call duration and reconnection.	Open when the session is `READY` or `IDLE`. See When can you open the Dictation socket.
Audio field	Send Base64-encoded PCM audio in `data`. See Send `AUDIO` chunks and Send audio chunks.	Send Base64-encoded PCM_S16LE audio in `audioData`. See Send JSON text frames and Audio chunks.
How to start streaming	Send the required `START_TIME` message before sending audio. See Send `START_TIME` and Start the stream segment.	No `START_TIME` message is required. See Dictation streaming wire format.
How to end the audio stream	After the last audio chunk, send a final `AUDIO` message with `data` set to `RU9G`. Do not send `EOF` as the value of `data`. See End the stream with `RU9G` and End the stream segment.	After the last audio chunk, send an `EVENT` message with `event` set to `AUDIO_END`. See Send `EVENT` with `AUDIO_END` and End-of-audio message.
What you receive	The WebSocket is primarily used to send audio. After the session ends, retrieve the note and transcript through REST. See Close the socket, then finish with REST and Complete the session after streaming audio.	Receive partial and final transcript frames over the WebSocket. Use REST as needed for the rest of the workflow. See Read transcript frames on the socket and Partial and final transcripts.
Use this when	You want Suki to process a clinical visit and generate a note.	You want your application to receive and use transcript text while the user is speaking.

Ambient end-of-stream marker: For /ws/stream, the final AUDIO.data value must be RU9G. This is the Base64 encoding of the EOF marker used by the Ambient wire format.Send RU9G exactly as specified. Do not send EOF as plain text in the data field.Refer to End the stream segment for more information.

Ambient and Form filling session IDs: Both workflows use the field name ambient_session_id, but the IDs are not interchangeable.

For Ambient, use the ID returned by Create ambient session.
For Form filling, use the ID returned by Create Form filling session when using Form filling with /ws/stream.

High-level architecture

Below is the high-level architecture for Ambient, Form filling, and Dictation flows. Where they differ is the:

Endpoint.
Outbound message contract.
How results come back.

Network and connection management

Plan your integration around the following network requirements and connection behaviors.

Network requirements

For reliable real-time streaming, Suki recommends these minimums:

Requirement	Recommended
Upload speed	1 Mbps
Bitrate	768 kbps
Ping time	150 ms
Unloaded latency	Under 50 ms
Loaded latency	Under 150 ms

Refer to Audio capture and streaming FAQs for the source list.

How do you keep the connection alive

While a stream is paused, send an EVENT message with event: KEEP_ALIVE at least once every five seconds. Ambient streaming allows a maximum pause of 30 minutes. For related idle-timeout guidance (including the 25 second audio inactivity window), see the Ambient API quickstart and Audio capture and streaming FAQs.

Call duration and reconnection

Ambient

There is no upper limit on call duration while the stream is active.
You can reopen GET /ws/stream with the same ambient_session_id only while the ambient job is still in the CREATED state. If the job has moved to another status (for example RUNNING, COMPLETED, or CANCELLED), the handshake fails with FailedPrecondition.
Suki needs a minimum amount of audio to generate content. For calls shorter than 1 minute, Suki may not have enough data, returns empty content, and marks the session as skipped.

Dictation

One Dictation session can support multiple speech sessions over time (for example push-to-talk).
Open /ws/transcribe only when the session is READY or IDLE. See Stream Dictation audio for the session-state rules.

How streaming works

Choose the tab for your product. Ambient, Form filling, and Dictation share the REST → WebSocket → REST pattern. Ambient and Form filling use the same GET /ws/stream wire format. Dictation uses GET /ws/transcribe with a different message contract.

Ambient
Form Filling
Dictation

Use this path when you have an ambient clinical notes session and stream to GET /ws/stream.The WebSocket carries live audio and control messages only. After streaming, use Ambient REST APIs to end the session, poll status, and retrieve transcripts, notes, and structured data.

Form filling also uses GET /ws/stream with the same wire format, but you create a Form filling session and retrieve structured form data through Form filling REST APIs. Open the Form filling tab for that path.

Create an ambient session and seed session context with REST.
Open a WebSocket connection to GET /ws/stream.
Send one START_TIME message for the stream segment.
Send one JSON message per audio chunk (Base64 PCM in data).
Optionally send EVENT messages, such as PAUSE, RESUME, or KEEP_ALIVE, when control is needed.
Send the ambient end marker as the final AUDIO message (data: RU9G).
Close the socket, then end the ambient session with REST and retrieve results.

Create the Ambient session and seed context

Create the session with REST, then seed session context before you stream. Do not open the WebSocket until the ambient session exists and context is seeded.

Open WebSocket `GET /ws/stream`

Authenticate with the documented handshake:

Browser Sec-WebSocket-Protocol: SukiAmbientAuth,<sdp_suki_token>,<ambient_session_id> (token before session ID).
Non-browser headers: sdp_suki_token and ambient_session_id.

Open the socket only when the ambient job is still in the CREATED state. Other statuses return FailedPrecondition.

Send `START_TIME`

Send one START_TIME message for the stream segment before audio chunks. This is required for Ambient in the documented contract.

Send `AUDIO` chunks

Send one JSON text frame per chunk. Audio bytes are Base64 PCM in the data field.Do not:

Send binary WebSocket frames.
Send multiple JSON objects in one frame.
Stream raw audio over HTTP.

Optional `EVENT` messages

When control is needed, send EVENT messages such as PAUSE, RESUME, KEEP_ALIVE, CANCEL, or ABORT. The gateway acts on PAUSE, RESUME, and CANCEL. While paused, send KEEP_ALIVE at least once every five seconds per the documented idle-timeout guidance. Refer to the Audio capture and streaming FAQs for the full event enum.

End the stream with `RU9G`

End the stream segment with a final AUDIO message whose data is RU9G (Base64 of the bytes EOF).

Close the socket, then finish with REST

Close the WebSocket, then end the ambient session with REST and retrieve results. Final transcripts and notes are not guaranteed on the WebSocket. Treat REST as the source of truth after streaming.For field-level message shapes, refer to Ambient streaming wire format. For end-of-session steps, refer to Complete the session after streaming.

Use this path when you have a Form filling session and stream to GET /ws/stream.Form filling uses the same Partner WebSocket and the same ambient message protocol as clinical notes (START_TIME, Base64 PCM in data, optional EVENT messages, and the RU9G end marker). You create the session and retrieve results with Form filling REST APIs, not Ambient clinical notes REST.

Both Form filling and Ambient clinical notes use the response field name ambient_session_id. Those IDs refer to different sessions. Use only the ID returned from Create Form filling session for Form filling REST and for /ws/stream. Do not use an ID from Create ambient session.

Create a Form filling session with REST.
Seed form template context with REST (at least one form_template_id before you end the session).
Open a WebSocket connection to GET /ws/stream using the Form filling session ID.
Send one START_TIME message for the stream segment.
Send one JSON message per audio chunk (Base64 PCM in data).
Optionally send EVENT messages, such as PAUSE, RESUME, or KEEP_ALIVE, when control is needed.
Send the end marker as the final AUDIO message (data: RU9G).
Close the socket, then end the Form filling session with REST and retrieve structured form data.

Create the Form filling session and seed template context

Create the session with Create Form filling session. The response field ambient_session_id is your Form filling session ID.Before you stream (and before you end the session), seed at least one form_template_id with the Form filling context API. See Form filling basic usage and Stream ambient audio.

Open WebSocket `GET /ws/stream`

Authenticate with the same ambient handshake pattern, using the Form filling session ID as ambient_session_id:

Browser Sec-WebSocket-Protocol: SukiAmbientAuth,<sdp_suki_token>,<ambient_session_id> (token before session ID).
Non-browser headers: sdp_suki_token and ambient_session_id.

Open the socket only while the session job is still in the CREATED state. Other statuses return FailedPrecondition. The wire format matches Ambient clinical notes. See Call duration and reconnection and Form filling audio stream.

Send `START_TIME`, `AUDIO`, and optional `EVENT` messages

Use the Ambient /ws/stream contract:

One START_TIME before audio chunks.
Base64 PCM in data for each AUDIO chunk.
Optional EVENT messages such as PAUSE, RESUME, or KEEP_ALIVE.

Do not send binary WebSocket frames or multiple JSON objects in one frame. See Start the stream segment and Send audio chunks.

End the stream with `RU9G`

End the stream segment with a final AUDIO message whose data is RU9G. Do not send EOF as plain text in data. See End the stream segment.

Close the socket, then finish with Form filling REST

Close the WebSocket, then:

End the session with End Form filling session.
Poll status with Get Form filling session status.
Retrieve structured form data with Get structured data, including generated_values and non_generated_values.

Treat Form filling REST as the source of truth after streaming. For a full walkthrough, see Build a Form filling session client.

Use this path when you have a Dictation session and stream to GET /ws/transcribe for real-time transcription.The WebSocket carries outbound audio and inbound partial and final transcript frames. Use the Dictation REST APIs to create the session before streaming, end the session after streaming, and retrieve final or cumulative results as needed.

Create or reuse a Dictation session with REST.
Confirm the session is READY or IDLE.
Open a WebSocket connection to GET /ws/transcribe.
Send one JSON message per audio chunk (Base64 PCM_S16LE in audioData).
Send an explicit end-of-audio message when the user stops speaking (EVENT with AUDIO_END).
Read partial and final transcript frames from the socket, then the terminal EOF frame.
Close the socket, then end the Dictation session with REST and retrieve results as needed.

Create or reuse a Dictation session

Create or reuse a transcription session with REST before you open the socket.

When can you open the Dictation socket

Open GET /ws/transcribe only when the session is READY or IDLE.If the session is RUNNING (another stream is active), COMPLETED, or otherwise cannot accept speech, the handshake fails with FailedPrecondition (for example, transcript session is not accepting new speech sessions).

One Dictation session can support multiple speech sessions over time, such as push-to-talk. After AUDIO_END and server processing, wait until the session returns to READY or IDLE before opening another WebSocket for the next utterance.

Open WebSocket `GET /ws/transcribe`

Authenticate with the documented handshake:

Browser Sec-WebSocket-Protocol: SukiAmbientAuth,<sdp_suki_token>,<transcription_session_id> (token before session ID).
Non-browser headers: sdp_suki_token and transcription_session_id.

The session ID argument differs from Ambient (transcription_session_id instead of ambient_session_id). The protocol argument order is the same pattern: token, then session ID.

Send `AUDIO` chunks

Send one JSON text frame per chunk. Audio bytes are Base64 PCM_S16LE in the audioData field.Do not:

Send binary WebSocket frames.
Send multiple JSON objects in one frame.

Send `EVENT` with `AUDIO_END`

When the user stops speaking, send an explicit end-of-audio message: EVENT with event: AUDIO_END. Do not use Ambient’s RU9G end marker on this endpoint.

Read transcript frames on the socket

The gateway sends partial and final transcript JSON frames (is_final false or true). After the upstream stream ends, it sends a terminal frame where transcript.transcript is EOF.Refer to Dictation transcript frames.

Close the socket, then finish with REST as needed

Close the WebSocket, then end the Dictation session with REST and retrieve final or cumulative results as needed.For outbound message shapes, refer to Dictation streaming wire format.

Available cookbooks

AmbientAPI

Authenticate Browser WebSocket Handshake

Auth browser WebSocket with protocols.

5 min

AmbientAPI

End Ambient After Streaming

Send RU9G, then end session.

5 min

DictationAPI

End Dictation with AUDIO_END

Use AUDIO_END, not ambient RU9G.

5 min

Available tutorials

Ambient

Build an Ambient Streaming Client

Authenticate, create a session, stream PCM audio over WebSocket, and retrieve clinical note results.

20 minIntermediate

Dictation

Build a Dictation Streaming Client

Create a transcription session, stream audio, read partial frames, and print the final transcript.

15 minIntermediate

Form filling

Build a Form filling Session Client

Create a Form filling session, send template context, stream audio, and retrieve structured form data.

20 minIntermediate

Next steps

Follow Ambient vs Dictation streaming to compare endpoints, auth, and message contracts. Read Audio capture best practices before you open the WebSocket. Follow Stream ambient audio when you already have an ambient session and need to push live audio on /ws/stream. Follow Stream Dictation audio when you have a Dictation session and need real-time transcript frames on /ws/transcribe. Open WebSocket streaming wire format for Ambient and Dictation wire format pages, send order, and chunking.

​What this guide covers

​What’s common for WebSocket streaming

​Ambient/Form filling vs Dictation WebSocket streaming

​High-level architecture

​Network and connection management

​Network requirements

​How do you keep the connection alive

​Call duration and reconnection

​How streaming works

​Create the Ambient session and seed context

​Open WebSocket GET /ws/stream

​Send START_TIME

​Send AUDIO chunks

​Optional EVENT messages

​End the stream with RU9G

​Close the socket, then finish with REST

​Create the Form filling session and seed template context

​Open WebSocket GET /ws/stream

​Send START_TIME, AUDIO, and optional EVENT messages

​End the stream with RU9G

​Close the socket, then finish with Form filling REST

​Create or reuse a Dictation session

​When can you open the Dictation socket

​Open WebSocket GET /ws/transcribe

​Send AUDIO chunks

​Send EVENT with AUDIO_END

​Read transcript frames on the socket

​Close the socket, then finish with REST as needed

​Available cookbooks

​Authenticate Browser WebSocket Handshake

​End Ambient After Streaming

​End Dictation with AUDIO_END

​Available tutorials

​Build an Ambient Streaming Client

​Build a Dictation Streaming Client

​Build a Form filling Session Client

​Next steps

What this guide covers

What’s common for WebSocket streaming

Ambient/Form filling vs Dictation WebSocket streaming

High-level architecture

Network and connection management

Network requirements

How do you keep the connection alive

Call duration and reconnection

How streaming works

Create the Ambient session and seed context

Open WebSocket `GET /ws/stream`

Send `START_TIME`

Send `AUDIO` chunks

Optional `EVENT` messages

End the stream with `RU9G`

Close the socket, then finish with REST

Create the Form filling session and seed template context

Open WebSocket `GET /ws/stream`

Send `START_TIME`, `AUDIO`, and optional `EVENT` messages

End the stream with `RU9G`

Close the socket, then finish with Form filling REST

Create or reuse a Dictation session

When can you open the Dictation socket

Open WebSocket `GET /ws/transcribe`

Send `AUDIO` chunks

Send `EVENT` with `AUDIO_END`

Read transcript frames on the socket

Close the socket, then finish with REST as needed

Available cookbooks

Authenticate Browser WebSocket Handshake

End Ambient After Streaming

End Dictation with AUDIO_END

Available tutorials

Build an Ambient Streaming Client

Build a Dictation Streaming Client

Build a Form filling Session Client

Next steps