WebSocket Streaming (TTS)

The WebSocket TTS API lets you send complete text and stream the generated audio back over a single connection, so playback can start before synthesis finishes.

wss://api.shisa.ai/ws/tts/realtime

Complete text in, streamed audio out

Despite the path name, this is not a streamed-input API like realtime ASR. You send a complete tts.speak text request; the service streams the generated audio back as binary frames or base64 JSON chunks.

For a single request/response that returns the whole audio file at once, use the POST /tts endpoint instead.

Requirements

• An API key allowed to use the voice you select (WebSocket TTS quotas and rate limits are reported under the shisa/tts service).
• A voice UUID from GET /tts/voices — see the voice catalogue.
• A server-side WebSocket client that can set an Authorization header. Keep your API key server-side.

Authenticate during the handshake with a standard bearer token:

Authorization: Bearer YOUR_API_KEY

How it works

1. Open the WebSocket with the Authorization header.
2. Send one session.update with the required voice_id and format (plus optional sample_rate, temperature, and audio_transport).
3. Wait for session.created.
4. Send one tts.speak request with the complete text.
5. Read tts.audio.start, the audio frames, tts.audio.done, and tts.usage.
6. Send more tts.speak requests sequentially, or close the socket.

One synthesis at a time

Only one synthesis may be active per session. Sending a second tts.speak while one is still in progress returns a synthesis_in_progress error. Before sending the next request, wait for tts.usage when an attempt started at the backend, or for the terminal tts.error when a request was rejected before reaching a backend. For backend_request_failed, also read the tts.usage that immediately follows.

Client messages

session.update

Send one session.update before synthesizing:

{
  "type": "session.update",
  "id": "cfg_0001",
  "session": {
    "voice_id": "61ba1141-60aa-4bc3-a3b3-be1ec20700b3",
    "format": "mp3",
    "audio_transport": "binary"
  }
}

Field	Required	Notes
id	Optional	Client correlation ID, echoed on config errors.
voice_id	Required	Public voice UUID from GET /tts/voices.
format	Required	Output audio format — mp3, wav, ogg, pcm, or flac. It must be supported by the selected voice and streamable on that provider.
sample_rate	Optional	Omit or set 0 for the backend default (24000 Hz). Use only a configurable value supported by the selected voice. Qwen-backed voices do not accept an override with ogg.
temperature	Optional	Provider-specific speech-variation control, currently accepted by Qwen-backed voices. Omit for other voices and for the default; explicit 0.0 is sent as a real value.
audio_transport	Optional	binary (default) or base64_json. See audio transport.

Format-specific streaming

The voice catalogue's streaming: true is necessary, but it does not mean every value in formats can stream. Because this endpoint always streams its output, an unsupported provider/format pair returns tts.error with code: "unsupported_streaming_format".

tts.speak

Send the complete text to synthesize:

{
  "type": "tts.speak",
  "id": "utt_0001",
  "text": "こんにちは。WebSocket TTS のテストです。"
}

Field	Required	Notes
id	Optional	Client correlation ID, echoed on the TTS events for this request.
text	Required	Complete text to synthesize. Default maximum is 5000 characters.

Service events

session.created

The session is configured and ready for tts.speak:

{
  "type": "session.created",
  "session": {
    "id": "router-session-request-id",
    "service": "shisa/tts",
    "voice_id": "61ba1141-60aa-4bc3-a3b3-be1ec20700b3",
    "format": "mp3",
    "sample_rate": 24000,
    "model": "speech-2.8-hd"
  }
}

tts.audio.start

Marks an accepted synthesis request and describes the audio you'll receive:

{
  "type": "tts.audio.start",
  "id": "utt_0001",
  "request_id": "synthesis-request-id",
  "format": "mp3",
  "content_type": "audio/mpeg",
  "sample_rate": 24000,
  "voice_id": "61ba1141-60aa-4bc3-a3b3-be1ec20700b3",
  "model": "speech-2.8-hd",
  "audio_transport": "binary"
}

Audio transport

How the audio bytes arrive depends on the audio_transport you set in session.update:

• binary (default) — generated audio arrives as raw binary WebSocket frames. Concatenate them in order.
• base64_json — audio arrives as tts.audio.delta JSON messages with base64-encoded bytes in audio and an incrementing seq:

{
  "type": "tts.audio.delta",
  "id": "utt_0001",
  "request_id": "synthesis-request-id",
  "seq": 1,
  "audio": "<base64-audio-bytes>"
}

tts.audio.done

Synthesis for this request is complete:

{
  "type": "tts.audio.done",
  "id": "utt_0001",
  "request_id": "synthesis-request-id",
  "audio_bytes": 12345,
  "chunks": 4
}

tts.usage

Final usage for a synthesis attempt that started at the backend. On success, it is sent after tts.audio.done:

{
  "type": "tts.usage",
  "request_id": "synthesis-request-id",
  "session_id": "router-session-request-id",
  "client_id": "utt_0001",
  "model": "speech-2.8-hd",
  "usage": {
    "input_chars": 24,
    "input_bytes": 72,
    "audio_bytes": 12345,
    "status": "completed",
    "final": true
  }
}

If the backend request fails, the router sends a tts.error with code: "backend_request_failed" followed by tts.usage. Its usage.status is backend_error when no audio was returned, or partial_backend_error when some audio had already been returned.

The router creates a usage record for attempts that reach the backend, including failed attempts (and sends tts.usage while the connection remains writable). Requests rejected during validation, access control, rate limiting, or backend selection return only tts.error and do not create usage. Do not assume that every tts.error will be followed by tts.usage.

tts.error and router.error

Failures before the WebSocket upgrade return a structured HTTP error rather than a WebSocket event. Common codes are invalid_websocket_upgrade, auth_token_required, invalid_token_format, invalid_token, legacy_api_key_rejected, ws_origin_denied, unknown_service, ws_connection_limit_exceeded, ws_connection_quota_unavailable, and websocket_draining.

TTS configuration, speak, and backend errors have this shape. id echoes the client correlation ID when one is available:

{
  "type": "tts.error",
  "id": "utt_0001",
  "code": "backend_request_failed",
  "message": "Backend TTS request failed"
}

Code	Meaning
invalid_message, invalid_id, id_too_long, invalid_session	A recognized message, correlation ID, or router session state is invalid.
session_already_configured	A session.update has already succeeded on this connection.
missing_voice_id, unknown_voice_id, tts_service_unavailable, service_access_denied	The voice selection, service availability, or access permission is invalid.
unsupported_audio_format, unsupported_sample_rate, unsupported_audio_transport, unsupported_streaming_format	The requested output setting is unavailable for the selected voice or backend.
session_not_configured, empty_text, tts_text_too_long, synthesis_in_progress	The tts.speak state or text is invalid.
rate_limited, service_rate_limited	The API-key or TTS-service rate limit was exceeded.
rate_limiter_unavailable, service_rate_limiter_unavailable	The rate-limiting service is temporarily unavailable.
backend_unavailable	No TTS backend is currently available. No backend request started, so no usage is created.
backend_request_failed	Synthesis started at the backend but failed. Final tts.usage follows this error.
tts_control_frame_too_large	A JSON control message exceeds the configured byte limit. If the read limit detects it first, the connection closes with code 1009.

Control errors detected before a message type can be handled use type: "router.error" with codes such as invalid_json, missing_message_type, and unknown_message_type. Sending a binary client frame closes the connection with WebSocket code 1003; an oversized control frame closes it with code 1009.

Minimal Python client

Install the dependency:

python -m pip install websockets

Set your key and run:

export SHISA_API_KEY="shsk:..."
export TTS_WS_URL="wss://api.shisa.ai/ws/tts/realtime"
export TTS_VOICE_ID="61ba1141-60aa-4bc3-a3b3-be1ec20700b3"
python websocket_tts.py > output.mp3

websocket_tts.py:

#!/usr/bin/env python3
import asyncio
import json
import os
import sys

import websockets


async def main() -> None:
    url = os.environ.get("TTS_WS_URL", "wss://api.shisa.ai/ws/tts/realtime")
    api_key = os.environ["SHISA_API_KEY"]
    voice_id = os.environ["TTS_VOICE_ID"]
    text = os.environ.get("TTS_TEXT", "こんにちは。WebSocket TTS のテストです。")
    audio_format = os.environ.get("TTS_FORMAT", "mp3")

    headers = [("Authorization", f"Bearer {api_key}")]
    audio = bytearray()

    async with websockets.connect(url, additional_headers=headers, max_size=None) as ws:
        await ws.send(
            json.dumps(
                {
                    "type": "session.update",
                    "id": "cfg_0001",
                    "session": {
                        "voice_id": voice_id,
                        "format": audio_format,
                        "audio_transport": "binary",
                    },
                }
            )
        )

        # Wait for the session to be created before speaking.
        while True:
            event = json.loads(await ws.recv())
            if event.get("type") == "session.created":
                break
            if event.get("type") in {"tts.error", "router.error", "error"}:
                raise RuntimeError(event)

        await ws.send(json.dumps({"type": "tts.speak", "id": "utt_0001", "text": text}))

        # Binary frames are audio; JSON frames are events.
        backend_error = None
        while True:
            msg = await ws.recv()
            if isinstance(msg, bytes):
                audio.extend(msg)
                continue
            event = json.loads(msg)
            event_type = event.get("type")
            if event_type == "tts.error":
                if event.get("code") == "backend_request_failed":
                    # Backend failures are followed by final usage.
                    backend_error = event
                    continue
                raise RuntimeError(event)
            if event_type == "tts.usage":
                print(json.dumps(event, ensure_ascii=False), file=sys.stderr)
                if backend_error is not None:
                    raise RuntimeError(backend_error)
                break
            if event_type in {"router.error", "error"}:
                raise RuntimeError(event)

    sys.stdout.buffer.write(audio)


if __name__ == "__main__":
    asyncio.run(main())

Next steps

• Return a whole audio file in one call with the POST /tts endpoint.
• Browse available voices in the voice catalogue.
• See how usage is billed on Pricing.