Skip to main content

WebSocket Streaming (TTS)

The WebSocket TTS API lets you send complete text and stream the generated audio back over a single connection, so playback can start before synthesis finishes.

wss://api.shisa.ai/ws/tts/realtime
Complete text in, streamed audio out

Despite the path name, this is not a streamed-input API like realtime ASR. You send a complete tts.speak text request; the service streams the generated audio back as binary frames or base64 JSON chunks.

For a single request/response that returns the whole audio file at once, use the POST /tts endpoint instead.

Requirements​

  • An API key allowed to use the voice you select (WebSocket TTS quotas and rate limits are reported under the shisa/tts service).
  • A voice UUID from GET /tts/voices — see the voice catalogue.
  • A server-side WebSocket client that can set an Authorization header. Keep your API key server-side.

Authenticate during the handshake with a standard bearer token:

Authorization: Bearer YOUR_API_KEY

How it works​

  1. Open the WebSocket with the Authorization header.
  2. Send one session.update with the required voice_id and format (plus optional sample_rate, temperature, and audio_transport).
  3. Wait for session.created.
  4. Send one tts.speak request with the complete text.
  5. Read tts.audio.start, the audio frames, tts.audio.done, and tts.usage.
  6. Send more tts.speak requests sequentially, or close the socket.
One synthesis at a time

Only one synthesis may be active per session. Sending a second tts.speak while one is still in progress returns a synthesis_in_progress error. Wait for tts.usage before sending the next request.

Client messages​

session.update​

Send one session.update before synthesizing:

{
  "type": "session.update",
  "id": "cfg_0001",
  "session": {
    "voice_id": "61ba1141-60aa-4bc3-a3b3-be1ec20700b3",
    "format": "mp3",
    "temperature": 0.7,
    "audio_transport": "binary"
  }
}
FieldRequiredNotes
idOptionalClient correlation ID, echoed on config errors.
voice_idRequiredPublic voice UUID from GET /tts/voices.
formatRequiredOutput audio format — mp3, wav, ogg, pcm, or flac. Allowed values depend on the selected voice.
sample_rateOptionalOmit or set 0 for the backend default (24000 Hz).
temperatureOptionalProvider-specific control for speech variation, accepted by some voices. Omit for the default; explicit 0.0 is sent as a real value.
audio_transportOptionalbinary (default) or base64_json. See audio transport.

tts.speak​

Send the complete text to synthesize:

{
  "type": "tts.speak",
  "id": "utt_0001",
  "text": "こんにちは。WebSocket TTS のテストです。"
}
FieldRequiredNotes
idOptionalClient correlation ID, echoed on the TTS events for this request.
textRequiredComplete text to synthesize. Default maximum is 5000 characters.

Service events​

session.created​

The session is configured and ready for tts.speak:

{
  "type": "session.created",
  "session": {
    "id": "router-session-request-id",
    "service": "shisa/tts",
    "voice_id": "61ba1141-60aa-4bc3-a3b3-be1ec20700b3",
    "format": "mp3",
    "sample_rate": 24000,
    "temperature": 0.7,
    "model": "kokoro"
  }
}

tts.audio.start​

Marks an accepted synthesis request and describes the audio you'll receive:

{
  "type": "tts.audio.start",
  "id": "utt_0001",
  "request_id": "synthesis-request-id",
  "format": "mp3",
  "content_type": "audio/mpeg",
  "sample_rate": 24000,
  "voice_id": "61ba1141-60aa-4bc3-a3b3-be1ec20700b3",
  "model": "kokoro",
  "audio_transport": "binary"
}

Audio transport​

How the audio bytes arrive depends on the audio_transport you set in session.update:

  • binary (default) — generated audio arrives as raw binary WebSocket frames. Concatenate them in order.
  • base64_json — audio arrives as tts.audio.delta JSON messages with base64-encoded bytes in audio and an incrementing seq:
{
  "type": "tts.audio.delta",
  "id": "utt_0001",
  "request_id": "synthesis-request-id",
  "seq": 1,
  "audio": "<base64-audio-bytes>"
}

tts.audio.done​

Synthesis for this request is complete:

{
  "type": "tts.audio.done",
  "id": "utt_0001",
  "request_id": "synthesis-request-id",
  "audio_bytes": 12345,
  "chunks": 4
}

tts.usage​

Final usage for the request. One usage record is created per accepted tts.speak:

{
  "type": "tts.usage",
  "request_id": "synthesis-request-id",
  "session_id": "router-session-request-id",
  "client_id": "utt_0001",
  "model": "kokoro",
  "usage": {
    "input_chars": 24,
    "input_bytes": 72,
    "audio_bytes": 12345,
    "status": "completed",
    "final": true
  }
}

Rejected requests (for example a bad voice_id or format) return a tts.error and do not create a usage record.

Minimal Python client​

Install the dependency:

python -m pip install websockets

Set your key and run:

export SHISA_API_KEY="shsk:..."
export TTS_WS_URL="wss://api.shisa.ai/ws/tts/realtime"
export TTS_VOICE_ID="61ba1141-60aa-4bc3-a3b3-be1ec20700b3"
python websocket_tts.py > output.mp3

websocket_tts.py:

#!/usr/bin/env python3
import asyncio
import json
import os
import sys

import websockets


async def main() -> None:
    url = os.environ.get("TTS_WS_URL", "wss://api.shisa.ai/ws/tts/realtime")
    api_key = os.environ["SHISA_API_KEY"]
    voice_id = os.environ["TTS_VOICE_ID"]
    text = os.environ.get("TTS_TEXT", "こんにちは。WebSocket TTS のテストです。")
    audio_format = os.environ.get("TTS_FORMAT", "mp3")

    headers = [("Authorization", f"Bearer {api_key}")]
    audio = bytearray()

    async with websockets.connect(url, additional_headers=headers, max_size=None) as ws:
        await ws.send(
            json.dumps(
                {
                    "type": "session.update",
                    "id": "cfg_0001",
                    "session": {
                        "voice_id": voice_id,
                        "format": audio_format,
                        "audio_transport": "binary",
                    },
                }
            )
        )

        # Wait for the session to be created before speaking.
        while True:
            event = json.loads(await ws.recv())
            if event.get("type") == "session.created":
                break
            if event.get("type") in {"tts.error", "router.error", "error"}:
                raise RuntimeError(event)

        await ws.send(json.dumps({"type": "tts.speak", "id": "utt_0001", "text": text}))

        # Binary frames are audio; JSON frames are events. Stop at tts.usage.
        while True:
            msg = await ws.recv()
            if isinstance(msg, bytes):
                audio.extend(msg)
                continue
            event = json.loads(msg)
            event_type = event.get("type")
            if event_type == "tts.usage":
                print(json.dumps(event, ensure_ascii=False), file=sys.stderr)
                break
            if event_type in {"tts.error", "router.error", "error"}:
                raise RuntimeError(event)

    sys.stdout.buffer.write(audio)


if __name__ == "__main__":
    asyncio.run(main())

Next steps​