{
  "hf_id": "zai-org/GLM-TTS",
  "meta": {
    "title": "GLM-TTS",
    "slug": "glm-tts",
    "provider": "GLM (Z-AI)",
    "description": "Z-AI's two-stage (AR + DiT flow-matching) zero-shot voice-cloning TTS for Chinese and English, served via vLLM-Omni through the OpenAI /v1/audio/speech API. Every request is conditioned on reference audio + its transcript.",
    "date_added": "2026-06-17",
    "date_updated": "2026-06-15",
    "tasks": [
      "omni"
    ],
    "related_recipes": [
      "Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice",
      "OpenMOSS-Team/MOSS-TTS",
      "bosonai/higgs-audio-v3-tts-4b"
    ]
  },
  "model": {
    "model_id": "zai-org/GLM-TTS",
    "min_vllm_version": "0.22.0",
    "architecture": "dense",
    "parameter_count": "3B",
    "active_parameters": "3B",
    "context_length": 0,
    "base_args": [
      "--omni",
      "--trust-remote-code"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      },
      "extras": [
        {
          "note": "vLLM-Omni provides the TTS serving backend and the `--omni` serve path. See the installation guide for pre-built wheels or building from source.",
          "command": "uv pip install git+https://github.com/vllm-project/vllm-omni.git"
        }
      ]
    }
  },
  "omni": {
    "tasks": [
      {
        "id": "t2a",
        "label": "Text → Speech",
        "endpoint": "/v1/audio/speech",
        "description": "GLM-TTS zero-shot voice clone via /v1/audio/speech (requires ref_audio + ref_text)",
        "curl": "curl -X POST http://localhost:8000/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"zai-org/GLM-TTS\",\n    \"input\": \"你好，这是一个语音合成测试。\",\n    \"response_format\": \"wav\",\n    \"ref_audio\": \"file:///path/to/ref.wav\",\n    \"ref_text\": \"这是参考音频的文本内容。\"\n  }' --output test.wav\n"
      }
    ]
  },
  "features": {},
  "opt_in_features": [],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 24,
      "description": "BF16 weights — single GPU; ~18-20 GB total (AR ~10 GB + DiT ~8 GB), ~16.6 GiB peak on a 1x A40 48 GB reference run; fits 24 GB cards"
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\n[GLM-TTS](https://huggingface.co/zai-org/GLM-TTS) is Z-AI's two-stage\ntext-to-speech system — an LLM that generates speech tokens (AR) followed by a\nFlow-Matching DiT that produces mel-spectrograms, vocoded by HiFT — served\nthrough **vLLM-Omni** with the OpenAI-compatible `/v1/audio/speech` API. It\ntargets **Chinese and English**, and **every request is conditioned on\nreference audio + its transcript** (zero-shot voice cloning).\n\n## Prerequisites\n\n- **Hardware**: a single CUDA GPU. Reference run: 1x A40 48 GB — ~18-20 GB\n  total (AR ~10 GB + DiT ~8 GB), ~16.6 GiB peak; fits 24 GB cards. Both stages\n  share GPU 0 by default; split them across GPUs for higher concurrency.\n- **vLLM-Omni** targeting vLLM >= 0.22.\n- `--trust-remote-code` is required for the GLM-TTS phoneme tokenizer.\n\n## Installation\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install git+https://github.com/vllm-project/vllm-omni.git\n```\n\n## Launch the server\n\nThe deploy config (`vllm_omni/deploy/glm_tts.yaml`) is auto-discovered. Async\nchunking (streaming) is on by default:\n\n```bash\nvllm serve zai-org/GLM-TTS --omni --trust-remote-code --port 8000\n```\n\nFor the synchronous (non-streaming) path:\n\n```bash\nvllm serve zai-org/GLM-TTS --omni --trust-remote-code --port 8000 --no-async-chunk\n```\n\n## Client usage\n\n### Zero-shot voice clone (curl)\n\n```bash\ncurl -X POST http://localhost:8000/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"zai-org/GLM-TTS\",\n    \"input\": \"你好，这是一个语音合成测试。\",\n    \"response_format\": \"wav\",\n    \"ref_audio\": \"file:///path/to/ref.wav\",\n    \"ref_text\": \"这是参考音频的文本内容。\"\n  }' --output test.wav\n```\n\n### Python client\n\n```bash\npython examples/online_serving/text_to_speech/glm_tts/openai_speech_client.py \\\n  --text \"你好，这是语音克隆测试。\" \\\n  --ref-audio file:///path/to/ref.wav \\\n  --ref-text \"这是参考音频的文本内容。\"\n```\n\n## Known limitations\n\n- Output is **24 kHz mono** WAV via the HiFT vocoder (Vocos2D 32 kHz fallback\n  with resampling).\n- Voice cloning requires `ref_audio` **and** `ref_text` together; the\n  reference clip should be **3-10 seconds**. Feature extraction (WhisperVQ\n  tokenizer, CampPlus ONNX, mel) runs model-side.\n- The first request may be slow due to lazy model loading (WhisperVQ, CampPlus\n  ONNX). Warm-cache RTF is ~0.6-0.7x on an A40.\n\n## References\n\n- [Model card](https://huggingface.co/zai-org/GLM-TTS)\n- [GLM-TTS GitHub](https://github.com/zai-org/GLM-TTS)\n- [vLLM-Omni GLM-TTS example](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_speech/glm_tts)\n",
  "source": {
    "source": "vllm-omni",
    "repo": "vllm-project/vllm-omni",
    "ref": "main",
    "sha": "beeedd99df33b432b7cf74caa8b28e1893124827",
    "path": "recipes/models/zai-org/GLM-TTS.yaml",
    "edit_url": "https://github.com/vllm-project/vllm-omni/edit/main/recipes/models/zai-org/GLM-TTS.yaml",
    "overlays": []
  }
}