# zai-org/GLM-TTS — vLLM serving recipe

> Z-AI's two-stage (AR + DiT flow-matching) zero-shot voice-cloning TTS for Chinese and English, served via vLLM-Omni through the OpenAI /v1/audio/speech API. Every request is conditioned on reference audio + its transcript.

Configuration: hardware `h200` (8× H200) · variant `default` (BF16) · task `t2a`

- Interactive page: https://recipes.vllm.ai/zai-org/GLM-TTS
- This configuration as JSON: https://recipes.vllm.ai/zai-org/GLM-TTS.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `task`: `t2a`

## Install

Requires vLLM >= 0.22.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Extra dependencies:

- vLLM-Omni provides the TTS serving backend and the `--omni` serve path. See the installation guide for pre-built wheels or building from source.

```bash
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

## Serve

vLLM-Omni recipe — the generation endpoints and request examples are in the guide below.

```bash
vllm serve zai-org/GLM-TTS \
  --trust-remote-code \
  --omni
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[GLM-TTS](https://huggingface.co/zai-org/GLM-TTS) is Z-AI's two-stage
text-to-speech system — an LLM that generates speech tokens (AR) followed by a
Flow-Matching DiT that produces mel-spectrograms, vocoded by HiFT — served
through **vLLM-Omni** with the OpenAI-compatible `/v1/audio/speech` API. It
targets **Chinese and English**, and **every request is conditioned on
reference audio + its transcript** (zero-shot voice cloning).

### Prerequisites

- **Hardware**: a single CUDA GPU. Reference run: 1x A40 48 GB — ~18-20 GB
  total (AR ~10 GB + DiT ~8 GB), ~16.6 GiB peak; fits 24 GB cards. Both stages
  share GPU 0 by default; split them across GPUs for higher concurrency.
- **vLLM-Omni** targeting vLLM >= 0.22.
- `--trust-remote-code` is required for the GLM-TTS phoneme tokenizer.

### Installation

```bash
uv venv && source .venv/bin/activate
uv pip install git+https://github.com/vllm-project/vllm-omni.git
```

### Launch the server

The deploy config (`vllm_omni/deploy/glm_tts.yaml`) is auto-discovered. Async
chunking (streaming) is on by default:

```bash
vllm serve zai-org/GLM-TTS --omni --trust-remote-code --port 8000
```

For the synchronous (non-streaming) path:

```bash
vllm serve zai-org/GLM-TTS --omni --trust-remote-code --port 8000 --no-async-chunk
```

### Client usage

#### Zero-shot voice clone (curl)

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-TTS",
    "input": "你好，这是一个语音合成测试。",
    "response_format": "wav",
    "ref_audio": "file:///path/to/ref.wav",
    "ref_text": "这是参考音频的文本内容。"
  }' --output test.wav
```

#### Python client

```bash
python examples/online_serving/text_to_speech/glm_tts/openai_speech_client.py \
  --text "你好，这是语音克隆测试。" \
  --ref-audio file:///path/to/ref.wav \
  --ref-text "这是参考音频的文本内容。"
```

### Known limitations

- Output is **24 kHz mono** WAV via the HiFT vocoder (Vocos2D 32 kHz fallback
  with resampling).
- Voice cloning requires `ref_audio` **and** `ref_text` together; the
  reference clip should be **3-10 seconds**. Feature extraction (WhisperVQ
  tokenizer, CampPlus ONNX, mel) runs model-side.
- The first request may be slow due to lazy model loading (WhisperVQ, CampPlus
  ONNX). Warm-cache RTF is ~0.6-0.7x on an A40.

### References

- [Model card](https://huggingface.co/zai-org/GLM-TTS)
- [GLM-TTS GitHub](https://github.com/zai-org/GLM-TTS)
- [vLLM-Omni GLM-TTS example](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_speech/glm_tts)