--- name: suki-audio-streaming description: >- Use when implementing or debugging Suki Partner WebSocket audio streaming for Ambient, Form filling, or Dictation, including WebSocket authentication, JSON message formats, audio encoding, stream lifecycle, and end-of-stream handling. license: SUKI_Proprietary metadata: author: suki version: "2.4" --- ## Audio streaming Use **Audio streaming** when the Suki product session already exists and you need to implement or debug its WebSocket audio layer. Documentation layout (August 2026): the Streaming Wire Format hub is an **overview only**. Message shapes live on product pages: * Ambient / Form filling → `.../websocket-streaming-wire-format-ambient` * Dictation → `.../websocket-streaming-wire-format-dictation` Streaming architecture explains what is shared vs different across products, then walks each product stream sequence. Use the product-specific skill for the complete workflow: - Ambient clinical note generation → `suki-ambient` - Medical Form filling → `suki-form-filling` - Real-time speech-to-text → `suki-dictation` Do not use this skill as a replacement for the product-specific workflow. ## Accuracy rules - Do not invent WebSocket endpoints, message types, field names, or end-of-stream markers. - Do not mix the Ambient / Form filling `/ws/stream` protocol with the Dictation `/ws/transcribe` protocol. - Use the session identifier returned by the API that created the session. - Do not assume that `ambient_session_id` has the same meaning across Ambient and Form filling. - Send audio as the documented Base64-encoded audio field inside a JSON text message. Do not send raw PCM as a binary WebSocket frame. - Use the exact audio format documented for the selected product and streaming API. - Do not assume that a WebSocket contains the final clinical artifact. Retrieve the product output through its REST API or webhook workflow. - If a required wire-format detail is not defined here, read the linked WebSocket documentation before generating code. ## Choose the socket **Environment rule:** For direct API integrations, use staging unless the user explicitly requests production. | Product | WebSocket | Session identifier | | --- | --- | --- | | Ambient | `wss://{host}/ws/stream` | `ambient_session_id` from Ambient session creation | | Form filling | `wss://{host}/ws/stream` | `ambient_session_id` from Form filling session creation | | Dictation | `wss://{host}/ws/transcribe` | `transcription_session_id` | Hosts: | Environment | Host | | --- | --- | | Staging | `sdp.suki-stage.com` | | Production | `sdp.suki.ai` | The REST and WebSocket endpoints use the same environment host. ### API endpoints | Operation | Method | Path | | --- | --- | --- | | Ambient / Form filling stream | `WebSocket` | `/ws/stream` | | Dictation stream | `WebSocket` | `/ws/transcribe` | ### Important session-ID distinction Ambient and Form filling can both return a field named: ```text ambient_session_id ```` These are different session types. * Ambient `ambient_session_id` identifies an Ambient session. * Form filling `ambient_session_id` identifies a Form filling session. Never use an Ambient session ID with the Form filling WebSocket or a Form filling session ID with the Ambient WebSocket. Dictation uses a different identifier: ```text transcription_session_id ``` ## Authentication For Partner WebSocket integrations, authenticate with the Suki Token obtained through the documented partner authentication flow. Use: ```text sdp_suki_token ``` and provide: ```text sdp_provider_id ``` when required by the partner configuration. Follow the Partner authentication documentation for how to obtain and refresh the Suki Token. ### Non-browser WebSocket clients For non-browser clients, send the authentication and session identifiers as the headers documented for the selected WebSocket endpoint. For `/ws/stream`, this includes: ```text sdp_suki_token ambient_session_id ``` For `/ws/transcribe`, this includes: ```text sdp_suki_token transcription_session_id ``` ### Browser WebSocket clients Browser clients cannot set arbitrary WebSocket headers. Use the documented `Sec-WebSocket-Protocol` format. For Ambient and Form filling: ```text SukiAmbientAuth,, ``` For Dictation: ```text SukiAmbientAuth,, ``` The token appears before the session identifier. Do not place the session identifier before the token. ## `/ws/stream`: Ambient and Form filling Ambient and Form filling use the same WebSocket streaming wire format. The product session must already exist before opening the socket. ### Stream lifecycle ```text create product session → obtain product session ID → seed required context → open /ws/stream → authenticate → START_TIME → AUDIO frames → optional EVENT frames → AUDIO + RU9G → close WebSocket → call product REST end → retrieve product output ``` Form filling uses the same streaming protocol but its own Form filling session ID and its own REST APIs. ### Session state For Ambient, open `/ws/stream` while the session is in the documented state for starting the stream. Do not assume a Form filling session can be substituted for an Ambient session. Use the Form filling session lifecycle documented for the Form filling API. ### `START_TIME` The first streaming message is: ```json { "type": "START_TIME", "data": "" } ``` `data` contains the Base64-encoded RFC3339 timestamp. Do not send the timestamp as an unencoded JSON timestamp when implementing the documented wire format. ### `AUDIO` Audio messages use: ```json { "type": "AUDIO", "data": "" } ``` The audio is carried in the `data` field. Do not use `audioData` on `/ws/stream`. ### Event messages The `/ws/stream` protocol supports the documented event messages: ```text PAUSE RESUME KEEP_ALIVE CANCEL ``` Use the documented JSON event structure for these messages. `ABORT` is deprecated. Do not introduce it in new implementations. `CANCEL` and `ABORT` are not interchangeable: * `CANCEL` terminates the ambient session. * `ABORT` is the deprecated stream interruption event and should not be used in new implementations. ### Keep-alive When the `/ws/stream` stream is active, continue sending audio at the documented interval even if individual chunks are small. When the stream is paused, use the documented `KEEP_ALIVE` event to keep the connection alive. The documented pause behavior uses: ```text KEEP_ALIVE every 5 seconds ``` and supports a maximum pause duration of: ```text 30 minutes ``` Do not substitute application-level WebSocket ping behavior for the documented Suki `KEEP_ALIVE` event when implementing the Suki pause protocol. ### End of audio End an Ambient or Form filling audio stream by sending an `AUDIO` message whose `data` value is: ```text RU9G ``` `RU9G` is the Base64 encoding of the Ambient end-of-stream `EOF` marker bytes used by the documented wire format. Send **`RU9G` exactly** as `AUDIO.data`. Do **not** send the plain string `EOF` in `data`. The sequence is: ```text AUDIO frames → AUDIO with data "RU9G" → close WebSocket → product REST end ``` Do not send: ```json { "type": "EVENT", "event": "AUDIO_END" } ``` on `/ws/stream`. That message belongs to Dictation. ## `/ws/transcribe`: Dictation Dictation uses a separate WebSocket endpoint and message protocol: ```text /ws/transcribe ``` Do not use Ambient or Form filling wire-format messages on this socket. ### Stream lifecycle ```text create transcription session → wait for READY or IDLE → open /ws/transcribe → authenticate → AUDIO frames → AUDIO_END → read transcript frames → wait for EOF → close WebSocket → REST end ``` ### Session state Open `/ws/transcribe` only when the transcription session is in a state that allows streaming: ```text READY IDLE ``` Do not open the socket when the session is already in a running or completed state. ### Audio message Dictation audio uses the `audioData` field: ```json { "type": "AUDIO", "audioData": "" } ``` Do not use: ```text data ``` for the Dictation audio field. ### End of audio End the Dictation audio stream with: ```json { "type": "EVENT", "event": "AUDIO_END" } ``` Do not send the Ambient/Form filling `RU9G` marker on `/ws/transcribe`. ### Transcript completion Read the server's transcript messages after sending `AUDIO_END`. The documented completion marker is: ```text transcript.transcript == "EOF" ``` After receiving `EOF`: ```text close WebSocket → call transcription session end API ``` ## Comparison | Item | Ambient | Form filling | Dictation | | -------------- | ------------------------------------ | ------------------------------------------ | ------------------------------------- | | WebSocket | `/ws/stream` | `/ws/stream` | `/ws/transcribe` | | Session ID | Ambient `ambient_session_id` | Form filling `ambient_session_id` | `transcription_session_id` | | Audio field | `data` | `data` | `audioData` | | Start marker | `START_TIME` | `START_TIME` | None | | Audio encoding | Base64 audio | Base64 audio | Base64 PCM_S16LE | | End marker | `AUDIO` with `data: "RU9G"` | `AUDIO` with `data: "RU9G"` | `EVENT` with `event: "AUDIO_END"` | | Transcript EOF | Not used as the stream end mechanism | Not used as the stream end mechanism | `transcript.transcript == "EOF"` | | Final output | Ambient REST APIs / webhook | Form filling structured-data API / webhook | Final transcript / Dictation REST API | ## Audio requirements Use the audio format required by the Suki streaming API. The documented streaming format is: ```text 16 kHz mono signed 16-bit little-endian PCM ``` For PCM audio: ```text sample rate: 16000 Hz channels: 1 sample format: signed 16-bit little-endian ``` Do not stream a WAV file as though its 44-byte WAV header were PCM audio. If your source is WAV, remove the container/header and stream the PCM audio data in the format required by the API. ### Audio chunking Suki recommends approximately 100 ms audio packets. For 16 kHz mono 16-bit PCM: ```text 16,000 samples/second × 0.1 seconds × 2 bytes/sample = 3,200 bytes ``` So approximately: ```text 3,200 bytes ≈ 100 ms ``` is a useful chunk size for PCM streaming. Keep audio pacing close to real time rather than sending a complete recording as fast as possible. ## WebSocket message rules Suki WebSocket audio messages are JSON text messages. Do not send raw PCM audio as a binary WebSocket frame. For `/ws/stream`, the general message structure is: ```json { "type": "AUDIO", "data": "" } ``` For `/ws/transcribe`, it is: ```json { "type": "AUDIO", "audioData": "" } ``` Use standard Base64 encoding. Do not use: * Hex encoding * URL-safe Base64 * Raw binary PCM in place of the JSON message * A different field name because another Suki product uses it ## End-to-end ordering ### Ambient and Form filling Use this ordering: ```text 1. Create the product session. 2. Save the returned product session ID. 3. Seed any required session context. 4. Open the WebSocket. 5. Authenticate the WebSocket. 6. Send START_TIME. 7. Send AUDIO messages. 8. Send supported control events when required. 9. End audio with AUDIO + RU9G. 10. Close the WebSocket. 11. Call the product REST end endpoint. 12. Retrieve the final product output. ``` Do not call the REST end endpoint before closing the WebSocket unless the product documentation explicitly says otherwise. ### Dictation Use this ordering: ```text 1. Create the transcription session. 2. Save transcription_session_id. 3. Wait until the session is READY or IDLE. 4. Open /ws/transcribe. 5. Authenticate the WebSocket. 6. Send AUDIO messages. 7. Send AUDIO_END. 8. Read transcript messages. 9. Wait for transcript.transcript == "EOF". 10. Close the WebSocket. 11. Call the transcription REST end endpoint. ``` ## Push-to-talk Dictation Dictation can be used for repeated push-to-talk interactions. After a Dictation stream reaches `EOF`: 1. Wait until the transcription session is again in a state that allows a new stream. 2. Open a new WebSocket. 3. Reuse the same `transcription_session_id` when the documented session lifecycle allows it. 4. Stream the next audio segment. 5. Send `AUDIO_END`. 6. Wait for `EOF`. 7. Repeat as needed. 8. End the REST session when the overall Dictation interaction is finished. Do not keep the completed WebSocket open for the next push-to-talk segment. ## What the WebSocket does not provide Do not assume that the WebSocket itself contains the final product artifact. ### Ambient The WebSocket streams audio for Ambient processing. Retrieve the generated clinical note or structured output through the documented Ambient REST APIs or partner webhook workflow. ### Form filling The WebSocket streams audio for Form filling. Retrieve structured form output through the Form filling `structured-data` endpoint or the documented webhook workflow. Do not retrieve Form filling output from Ambient `/content`. ### Dictation The WebSocket provides real-time transcript messages. Use the Dictation REST session-end response and transcript messages according to the Dictation API workflow. ## Common mistakes ### Using `/ws/stream` for Dictation Wrong: ```text Dictation → /ws/stream ``` Correct: ```text Dictation → /ws/transcribe ``` ### Using `/ws/transcribe` for Ambient Wrong: ```text Ambient → /ws/transcribe ``` Correct: ```text Ambient → /ws/stream ``` ### Using the wrong audio field For Ambient and Form filling: ```json { "type": "AUDIO", "data": "" } ``` For Dictation: ```json { "type": "AUDIO", "audioData": "" } ``` Do not swap these fields. ### Using `RU9G` for Dictation Wrong: ```json { "type": "AUDIO", "data": "RU9G" } ``` Correct: ```json { "type": "EVENT", "event": "AUDIO_END" } ``` for Dictation. ### Using `AUDIO_END` for Ambient Wrong: ```json { "type": "EVENT", "event": "AUDIO_END" } ``` on `/ws/stream`. Use the documented Ambient/Form filling end marker: ```json { "type": "AUDIO", "data": "RU9G" } ``` ### Sending binary PCM Wrong: ```text WebSocket binary frame → raw PCM ``` Correct: ```text WebSocket text frame → JSON → Base64 audio ``` ### Mixing session IDs Do not use: ```text Ambient ambient_session_id ``` to connect to: ```text Form filling /ws/stream ``` Use the session ID returned by the product's own session-create endpoint. ### Treating the WebSocket as the artifact API Do not assume that receiving audio acknowledgements or stream messages means the final note or form output is available. Follow the product-specific REST or webhook workflow. ## Non-negotiable rules Before generating or reviewing Suki WebSocket streaming code, verify: * [ ] The correct WebSocket endpoint is used. * [ ] The session was created by the corresponding Suki product API. * [ ] The correct session identifier is used. * [ ] `sdp_suki_token` is supplied according to the documented authentication method. * [ ] `sdp_provider_id` is supplied when required. * [ ] Browser authentication uses the documented `Sec-WebSocket-Protocol` format. * [ ] WebSocket messages are JSON text frames. * [ ] Audio is Base64 encoded. * [ ] Audio uses the documented PCM format. * [ ] Ambient/Form filling use `data`. * [ ] Dictation uses `audioData`. * [ ] Ambient/Form filling use `START_TIME`. * [ ] Dictation does not use `START_TIME`. * [ ] Ambient/Form filling end with `AUDIO` + `RU9G`. * [ ] Dictation ends with `AUDIO_END`. * [ ] Dictation waits for transcript `EOF`. * [ ] Deprecated `ABORT` is not introduced in new implementations. * [ ] The WebSocket is closed at the correct point in the product lifecycle. * [ ] The product REST end endpoint is called after streaming ends. * [ ] Final Ambient/Form filling output is retrieved through the appropriate REST API or webhook. * [ ] Ambient and Form filling session IDs are not mixed. * [ ] Long-running or reconnect behavior follows the product-specific documentation. ## When to read more - [Streaming Architecture](https://developer.suki.ai/documentation/how-to/audio-streaming/streaming-architecture): Learn how Ambient /ws/stream, Form filling /ws/stream, and Dictation /ws/transcribe fit together: REST sessions, WebSocket JSON frames, connection rules, and where results come from - [Streaming Wire Format Overview](https://developer.suki.ai/documentation/how-to/audio-streaming/websocket-streaming-wire-format): Choose the correct WebSocket wire format for Ambient, Form filling (`GET /ws/stream`), or Dictation (`GET /ws/transcribe`) JSON text frames - [Ambient Streaming Wire Format](https://developer.suki.ai/documentation/how-to/audio-streaming/websocket-streaming-wire-format-ambient): Format and send Base64 audio chunks as JSON text frames on GET /ws/stream for Ambient and Form filling sessions, including message types, order, and PCM chunking - [Dictation Streaming Wire Format](https://developer.suki.ai/documentation/how-to/audio-streaming/websocket-streaming-wire-format-dictation): Format and send Base64 audio chunks as JSON text frames on GET /ws/transcribe for Dictation sessions, including message types, order, and PCM chunking - [Ambient vs Dictation Streaming Comparison](https://developer.suki.ai/documentation/how-to/audio-streaming/audio-stream): Compare Ambient /ws/stream and Dictation /ws/transcribe: purpose, auth, messages, and when to use each - [Stream Dictation Audio](https://developer.suki.ai/documentation/how-to/audio-streaming/dictation-streaming): Overview: Stream Dictation audio over WebSocket on GET /ws/transcribe for real-time transcript text - [Read Dictation Transcript Frames](https://developer.suki.ai/documentation/how-to/audio-streaming/dictation-streaming-transcripts): Reference for Dictation /ws/transcribe inbound transcript frames: partial, final, EOF, words, and UI handling - [Build Ambient Session Streaming Client](https://developer.suki.ai/documentation/tutorials/ambient-websocket-code-example): Tutorial: Build an end-to-end Ambient session streaming client from login to session results - [Build Form Filling Session Streaming Client](https://developer.suki.ai/documentation/tutorials/form-filling-websocket-code-example): Tutorial: Build an end-to-end Form filling session client from login to structured form data - [Build Dictation Session Streaming Client](https://developer.suki.ai/documentation/tutorials/dictation-websocket-code-example): Tutorial: Build an end-to-end Dictation session streaming client from login to final transcript - [llms-full.txt](https://developer.suki.ai/llms-full.txt): Full-site dump for broad cross-product questions ## Sister skills * Ambient: [https://developer.suki.ai/.well-known/agent-skills/suki-ambient/SKILL.md](https://developer.suki.ai/.well-known/agent-skills/suki-ambient/SKILL.md) * Form filling: [https://developer.suki.ai/.well-known/agent-skills/suki-form-filling/SKILL.md](https://developer.suki.ai/.well-known/agent-skills/suki-form-filling/SKILL.md) * Dictation: [https://developer.suki.ai/.well-known/agent-skills/suki-dictation/SKILL.md](https://developer.suki.ai/.well-known/agent-skills/suki-dictation/SKILL.md) * Webhooks: [https://developer.suki.ai/.well-known/agent-skills/suki-webhooks/SKILL.md](https://developer.suki.ai/.well-known/agent-skills/suki-webhooks/SKILL.md)