> ## Documentation Index
> Fetch the complete documentation index at: https://developer.suki.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming Architecture

> Learn how Ambient /ws/stream and Dictation /ws/transcribe fit together: REST sessions, WebSocket JSON frames, connection rules, and where results come from

This guide explains the common streaming architecture used by all Suki modalities. Create a session using a REST API, stream live audio over a WebSocket connection, then end the session using a REST API.

Ambient and Dictation both stream audio as JSON text frames, with each frame containing a single JSON object. Although the streaming pattern is the same, each API uses different endpoints, message formats, and result retrieval workflows.

<Note>
  Use the **same base host** for REST and WebSocket in a given environment (for example staging **`https://sdp.suki-stage.com`** and **`wss://sdp.suki-stage.com`**). Your partnership team confirms which host and credentials apply.
</Note>

Read the following related guides:

<Accordion title="Click Each Button to Learn More" defaultOpen={true}>
  <div className="doc-guide-btn-row">
    <a href="/documentation/how-to/audio-streaming/audio-stream" className="doc-guide-btn">
      Streaming Comparison
    </a>

    <a href="/documentation/how-to/audio-streaming/websocket-streaming-wire-format" className="doc-guide-btn">
      Wire Format
    </a>

    <a href="/documentation/how-to/audio-streaming/ambient-audio-streaming" className="doc-guide-btn">
      Stream Ambient Audio
    </a>

    <a href="/documentation/how-to/audio-streaming/dictation-streaming" className="doc-guide-btn">
      Stream Dictation Audio
    </a>
  </div>
</Accordion>

<Info>
  **This guide applies to:** Direct HTTP and WebSocket integrations with Suki for Partner Ambient, Form filling, and Dictation APIs. Form filling uses the same ambient socket (`GET /ws/stream`) with an ambient session ID.
</Info>

<Warning>
  Do not send raw audio as **binary** WebSocket frames on **`/ws/stream`** or **`/ws/transcribe`**. Outbound audio is Base64 inside JSON text frames.
</Warning>

## What this guide covers

Use this page to understand the end-to-end flow before you implement a client:

* What Ambient (and Form filling) and Dictation share
* Where the two products differ (endpoint, messages, results)
* Network and connection rules that affect both
* Stage-by-stage Ambient and Dictation flows

For field-level JSON shapes and chunking, open [WebSocket streaming wire format](/documentation/how-to/audio-streaming/websocket-streaming-wire-format). For a compact endpoint comparison, open [Ambient vs Dictation streaming](/documentation/how-to/audio-streaming/audio-stream).

## What stays the same

Across Ambient, Form filling, and Dictation:

1. Authenticate with Partner APIs and obtain an `sdp_suki_token`.
2. Create a session with REST.
3. Open a **WebSocket Secure (`wss://`)** connection on the same base host as REST.
4. Send **one UTF-8 JSON text frame** per message (exactly one JSON object per frame).
5. Put audio bytes in a Base64 string field inside that JSON, not in a binary WebSocket frame.
6. Close the socket, then use REST to end the session and retrieve final results as needed.

## What differs

| Topic                        | Ambient (and Form filling on the same socket)                             | Dictation                                                                     |
| :--------------------------- | :------------------------------------------------------------------------ | :---------------------------------------------------------------------------- |
| Goal                         | Stream visit audio for note generation and related processing             | Stream speech for **real-time transcript text** in your app                   |
| Endpoint                     | `GET /ws/stream`                                                          | `GET /ws/transcribe`                                                          |
| Session ID                   | `ambient_session_id`                                                      | `transcription_session_id`                                                    |
| When you may open the socket | Ambient job still in **`CREATED`**                                        | Session is **`READY`** or **`IDLE`**                                          |
| Audio field                  | **`data`** (Base64 PCM)                                                   | **`audioData`** (Base64 **PCM\_S16LE**)                                       |
| Start of stream              | **`START_TIME`** required first                                           | No **`START_TIME`** in the Dictation contract                                 |
| End of stream                | Final **`AUDIO`** with **`data`**: **`RU9G`** (Base64 of bytes **`EOF`**) | **`EVENT`** with **`event`**: **`AUDIO_END`**                                 |
| Results                      | Final note and transcript mainly via **REST** after you end the session   | **Partial and final** transcript frames on the WebSocket, then REST as needed |

## High-level architecture

Below is the high-level architecture for Ambient, Form filling, and Dictation flows. Where they differ is the:

* Endpoint
* Outbound message contract
* How results come back

```mermaid actions={false} theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
flowchart TB
    subgraph Shared["Shared pattern"]
        Auth[Authenticate with Partner APIs<br/>sdp_suki_token]
        REST[Create session with REST]
        Auth --> REST
    end

    REST --> Choice{Product}

    Choice -->|Ambient or Form filling| AmbWS[Open WebSocket<br/>GET /ws/stream]
    Choice -->|Dictation| DictWS[Open WebSocket<br/>GET /ws/transcribe]

    AmbWS --> AmbStream[Send JSON text frames<br/>START_TIME, AUDIO, EVENT, RU9G]
    AmbStream --> AmbClose[Close WebSocket]
    AmbClose --> AmbREST[End session and fetch results with REST]

    DictWS --> DictStream[Send JSON text frames<br/>AUDIO, AUDIO_END]
    DictStream --> DictIn[Receive partial and final<br/>transcript frames on the socket]
    DictIn --> DictClose[Close WebSocket]
    DictClose --> DictREST[End session and fetch results with REST as needed]

    classDef sharedStyle fill:#FFE148,stroke:#D4A017,stroke-width:2px,color:#000000
    classDef ambStyle fill:#FFF394,stroke:#FFE148,stroke-width:2px,color:#000000
    classDef dictStyle fill:#FFD700,stroke:#D4A017,stroke-width:2px,color:#000000

    class Auth,REST sharedStyle
    class AmbWS,AmbStream,AmbClose,AmbREST ambStyle
    class DictWS,DictStream,DictIn,DictClose,DictREST dictStyle
```

## Network and connection management

Plan your integration around the following network requirements and connection behaviors.

### Network requirements

For reliable real-time streaming, Suki recommends these minimums:

| Requirement      | Recommended  |
| :--------------- | :----------- |
| Upload speed     | 1 Mbps       |
| Bitrate          | 768 kbps     |
| Ping time        | 150 ms       |
| Unloaded latency | Under 50 ms  |
| Loaded latency   | Under 150 ms |

Refer to [Audio capture and streaming FAQs](/api-reference/faqs/audio-capture-streaming) for the source list.

### How do you keep the connection alive

While a stream is **paused**, send an **`EVENT`** message with **`event`**: **`KEEP_ALIVE`** at least once every **five seconds**. Ambient streaming allows a maximum pause of **30 minutes**.

For related idle-timeout guidance (including the **25 second** audio inactivity window), see the [Ambient API quickstart](/api-reference/quickstart) and [Audio capture and streaming FAQs](/api-reference/faqs/audio-capture-streaming).

### Call duration and reconnection

**Ambient**

* There is **no upper limit on call duration** while the stream is active.
* You can reopen **`GET /ws/stream`** with the **same `ambient_session_id`** only while the ambient job is still in the **`CREATED`** state. If the job has moved to another status (for example **`RUNNING`**, **`COMPLETED`**, or **`CANCELLED`**), the handshake fails with **`FailedPrecondition`**.
* Suki needs a minimum amount of audio to generate content. For calls shorter than **1 minute**, Suki may not have enough data, returns empty content, and marks the session as **`skipped`**.

**Dictation**

* One Dictation session can support multiple speech sessions over time (for example push-to-talk).
* Open **`/ws/transcribe`** only when the session is **`READY`** or **`IDLE`**. See [Stream Dictation audio](/documentation/how-to/audio-streaming/dictation-streaming) for the session-state rules.

## How streaming works

Choose the tab for your product. Ambient (and Form filling on the same socket) and Dictation share the REST → WebSocket → REST pattern, but use different endpoints and message contracts.

<Tabs>
  <Tab title="Ambient">
    Use this path when you already have an **ambient session** (or a Form filling session on the same socket) and stream to <Badge color="blue" size="sm">GET /ws/stream</Badge>.

    The WebSocket carries live audio and control messages only. After streaming, use ambient REST APIs to end the session, poll status, and retrieve transcripts, notes, and structured data.

    1. Create an ambient session and seed session context with REST.
    2. Open a WebSocket connection to **`GET /ws/stream`**.
    3. Send one **`START_TIME`** message for the stream segment.
    4. Send one JSON message per audio chunk (Base64 PCM in **`data`**).
    5. Optionally send **`EVENT`** messages, such as **`PAUSE`**, **`RESUME`**, or **`KEEP_ALIVE`**, when control is needed.
    6. Send the ambient end marker as the final **`AUDIO`** message (**`data`**: **`RU9G`**).
    7. Close the socket, then end the ambient session with REST and retrieve results.

    ```mermaid actions={false} theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
    flowchart TD
        A([Create ambient session]) --> B[Seed session context]
        B --> C[Open WebSocket<br/>GET /ws/stream]
        C --> D[Send START_TIME]
        D --> E[Send AUDIO chunks<br/>Base64 PCM in data]
        E --> F{Need control?}
        F -->|Yes| G[Send EVENT<br/>PAUSE, RESUME, KEEP_ALIVE, ...]
        G --> E
        F -->|No| H[Send final AUDIO<br/>data RU9G end marker]
        H --> I[Close WebSocket]
        I --> J[End ambient session with REST]
        J --> K[Poll status and fetch<br/>transcript, notes, structured data]
        K --> L([Complete])

        classDef restStyle fill:#FFE148,stroke:#D4A017,stroke-width:2px,color:#000000
        classDef wsStyle fill:#FFF394,stroke:#FFE148,stroke-width:2px,color:#000000
        classDef doneStyle fill:#FFD700,stroke:#D4A017,stroke-width:3px,color:#000000

        class A,B,J,K restStyle
        class C,D,E,G,H,I wsStyle
        class L doneStyle
    ```

    ### Create the Ambient session and seed context

    Create the session with REST, then seed session context before you stream. Do not open the WebSocket until the ambient session exists and context is seeded.

    ### Open WebSocket `GET /ws/stream`

    Authenticate with the documented handshake:

    * Browser **`Sec-WebSocket-Protocol`**: `SukiAmbientAuth,<sdp_suki_token>,<ambient_session_id>` (token before session ID)
    * Non-browser headers: `sdp_suki_token` and `ambient_session_id`

    Open the socket only when the ambient job is still in the **`CREATED`** state. Other statuses return **`FailedPrecondition`**.

    ### Send `START_TIME`

    Send one **`START_TIME`** message for the stream segment before audio chunks. This is required for Ambient in the documented contract.

    ### Send `AUDIO` chunks

    Send one JSON text frame per chunk. Audio bytes are Base64 PCM in the **`data`** field.

    Do not:

    * Send binary WebSocket frames
    * Send multiple JSON objects in one frame
    * Stream raw audio over HTTP

    ### Optional `EVENT` messages

    When control is needed, send **`EVENT`** messages such as **`PAUSE`**, **`RESUME`**, **`KEEP_ALIVE`**, **`CANCEL`**, or **`ABORT`**. The gateway acts on **`PAUSE`**, **`RESUME`**, and **`CANCEL`**. While paused, send **`KEEP_ALIVE`** at least once every **five seconds** per the documented idle-timeout guidance. Refer to the [Audio capture and streaming FAQs](/api-reference/faqs/audio-capture-streaming) for the full event enum.

    ### End the stream with `RU9G`

    End the stream segment with a final **`AUDIO`** message whose **`data`** is **`RU9G`** (Base64 of the bytes **`EOF`**).

    ### Close the socket, then finish with REST

    Close the WebSocket, then end the ambient session with REST and retrieve results. Final transcripts and notes are not guaranteed on the WebSocket. Treat REST as the source of truth after streaming.

    For field-level message shapes, refer to the Ambient tab in [WebSocket streaming wire format](/documentation/how-to/audio-streaming/websocket-streaming-wire-format). For end-of-session steps, refer to [Complete the session after streaming](/documentation/how-to/audio-streaming/websocket-streaming-complete-session).
  </Tab>

  <Tab title="Dictation">
    Use this path when you have a **Dictation session** and stream to <Badge color="blue" size="sm">GET /ws/transcribe</Badge> for real-time transcription.

    The WebSocket carries outbound audio and inbound partial and final transcript frames. Use the Dictation REST APIs to create the session before streaming, end the session after streaming, and retrieve final or cumulative results as needed.

    1. Create or reuse a Dictation session with REST.
    2. Confirm the session is **`READY`** or **`IDLE`**.
    3. Open a WebSocket connection to **`GET /ws/transcribe`**.
    4. Send one JSON message per audio chunk (Base64 **PCM\_S16LE** in **`audioData`**).
    5. Send an explicit end-of-audio message when the user stops speaking (**`EVENT`** with **`AUDIO_END`**).
    6. Read partial and final transcript frames from the socket, then the terminal **`EOF`** frame.
    7. Close the socket, then end the Dictation session with REST and retrieve results as needed.

    ```mermaid actions={false} theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
    flowchart TD
        A([Create or reuse Dictation session]) --> B{Session READY or IDLE?}
        B -->|No| Fail[Handshake fails<br/>FailedPrecondition]
        B -->|Yes| C[Open WebSocket<br/>GET /ws/transcribe]
        C --> D[Send AUDIO chunks<br/>Base64 PCM_S16LE in audioData]
        D --> E[Send EVENT AUDIO_END]
        E --> F[Read partial and final<br/>transcript frames]
        F --> G[Receive EOF transcript frame]
        G --> H[Close WebSocket]
        H --> I[End Dictation session with REST<br/>and fetch results as needed]
        I --> J([Complete])

        classDef restStyle fill:#FFE148,stroke:#D4A017,stroke-width:2px,color:#000000
        classDef wsStyle fill:#FFF394,stroke:#FFE148,stroke-width:2px,color:#000000
        classDef warnStyle fill:#FFD700,stroke:#D4A017,stroke-width:2px,color:#000000

        class A,I restStyle
        class C,D,E,F,G,H wsStyle
        class Fail,B warnStyle
    ```

    ### Create or reuse a Dictation session

    Create or reuse a transcription session with REST before you open the socket.

    ### When can you open the Dictation socket

    Open **`GET /ws/transcribe`** only when the session is **`READY`** or **`IDLE`**.

    If the session is **`RUNNING`** (another stream is active), **`COMPLETED`**, or otherwise cannot accept speech, the handshake fails with **`FailedPrecondition`** (for example, transcript session is not accepting new speech sessions).

    <Note>
      One Dictation session can support multiple speech sessions over time, such as push-to-talk. After **`AUDIO_END`** and server processing, wait until the session returns to **`READY`** or **`IDLE`** before opening another WebSocket for the next utterance.
    </Note>

    ### Open WebSocket `GET /ws/transcribe`

    Authenticate with the documented handshake:

    * Browser **`Sec-WebSocket-Protocol`**: `SukiAmbientAuth,<sdp_suki_token>,<transcription_session_id>` (token before session ID)
    * Non-browser headers: `sdp_suki_token` and `transcription_session_id`

    The session ID argument differs from Ambient (`transcription_session_id` instead of `ambient_session_id`). The protocol argument order is the same pattern: token, then session ID.

    ### Send `AUDIO` chunks

    Send one JSON text frame per chunk. Audio bytes are Base64 **PCM\_S16LE** in the **`audioData`** field.

    Do not:

    * Send binary WebSocket frames
    * Send multiple JSON objects in one frame

    ### Send `EVENT` with `AUDIO_END`

    When the user stops speaking, send an explicit end-of-audio message: **`EVENT`** with **`event`**: **`AUDIO_END`**. Do not use Ambient's **`RU9G`** end marker on this endpoint.

    ### Read transcript frames on the socket

    The gateway sends partial and final transcript JSON frames (`is_final` false or true). After the upstream stream ends, it sends a terminal frame where **`transcript.transcript`** is **`EOF`**.

    Refer to [Dictation transcript frames](/documentation/how-to/audio-streaming/dictation-streaming-transcripts).

    ### Close the socket, then finish with REST as needed

    Close the WebSocket, then end the Dictation session with REST and retrieve final or cumulative results as needed.

    For outbound message shapes, refer to the Dictation tab in [WebSocket streaming wire format](/documentation/how-to/audio-streaming/websocket-streaming-wire-format).
  </Tab>
</Tabs>

## Available tutorials

<div className="hp-io-method-grid tut-hub-card-grid">
  <a className="hp-io-method-card tut-hub-method-card" href="/documentation/tutorials/ambient-websocket-code-example">
    <div className="tut-hub-card-media" aria-hidden="true" />

    <div className="hp-io-method-card-body">
      <span className="hp-wn-badge hp-wn-badge-new">Ambient</span>
      <h3 className="hp-io-method-card-title">Build an Ambient Streaming Client</h3>

      <p className="hp-io-method-card-desc">
        Authenticate, create a session, stream PCM audio over WebSocket, and retrieve clinical note results.
      </p>

      <div className="hp-io-method-card-meta tut-hub-card-foot" aria-label="20 min, Intermediate">
        <div className="tut-hub-card-foot-meta">
          <span className="hp-io-method-card-meta-time">20 min</span>
          <span className="tut-hub-level">Intermediate</span>
        </div>
      </div>
    </div>
  </a>

  <a className="hp-io-method-card tut-hub-method-card" href="/documentation/tutorials/dictation-websocket-code-example">
    <div className="tut-hub-card-media tut-hub-card-media--blue" aria-hidden="true" />

    <div className="hp-io-method-card-body">
      <span className="hp-wn-badge hp-wn-badge-new">Dictation</span>
      <h3 className="hp-io-method-card-title">Build a Dictation Streaming Client</h3>

      <p className="hp-io-method-card-desc">
        Create a transcription session, stream audio, read partial frames, and print the final transcript.
      </p>

      <div className="hp-io-method-card-meta tut-hub-card-foot" aria-label="15 min, Intermediate">
        <div className="tut-hub-card-foot-meta">
          <span className="hp-io-method-card-meta-time">15 min</span>
          <span className="tut-hub-level">Intermediate</span>
        </div>
      </div>
    </div>
  </a>

  <a className="hp-io-method-card tut-hub-method-card" href="/documentation/tutorials/form-filling-websocket-code-example">
    <div className="tut-hub-card-media" aria-hidden="true" />

    <div className="hp-io-method-card-body">
      <span className="hp-wn-badge hp-wn-badge-new">Form Filling</span>
      <h3 className="hp-io-method-card-title">Build a Form Filling Session Client</h3>

      <p className="hp-io-method-card-desc">
        Create a Form filling session, send template context, stream audio, and retrieve structured form data.
      </p>

      <div className="hp-io-method-card-meta tut-hub-card-foot" aria-label="20 min, Intermediate">
        <div className="tut-hub-card-foot-meta">
          <span className="hp-io-method-card-meta-time">20 min</span>
          <span className="tut-hub-level">Intermediate</span>
        </div>
      </div>
    </div>
  </a>
</div>

## Next steps

<Icon icon="file-lines" iconType="solid" /> Follow [Ambient vs Dictation streaming](/documentation/how-to/audio-streaming/audio-stream) to compare endpoints, auth, and message contracts.

<Icon icon="file-lines" iconType="solid" /> Read [Audio capture best practices](/documentation/how-to/audio-streaming/audio-capture-best-practices) before you open the WebSocket.

<Icon icon="file-lines" iconType="solid" /> Follow [Stream ambient audio](/documentation/how-to/audio-streaming/ambient-audio-streaming) when you already have an ambient session and need to push live audio on **`/ws/stream`**.

<Icon icon="file-lines" iconType="solid" /> Follow [Stream Dictation audio](/documentation/how-to/audio-streaming/dictation-streaming) when you have a Dictation session and need real-time transcript frames on **`/ws/transcribe`**.

<Icon icon="file-lines" iconType="solid" /> Open [WebSocket streaming wire format](/documentation/how-to/audio-streaming/websocket-streaming-wire-format) for message types, send order, and chunking.