vLLM/Recipes
Search
Hunyuan (Tencent)

tencent/Hy4-flash

Tencent HY4 Flash multimodal MoE for everyday productivity and agent workflows — 220B total / 12B active, 1M context, DFly speculative decoding

moe220B / 12B1,048,576 ctxMultimodal
Default configuration: 8× H200 · Tensor Parallel · BF16
export VLLM_USE_RUST_FRONTEND=1
vllm serve tencent/Hy4-flash \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 8192 \
  --limit-mm-per-prompt '{"image":1,"video":0}' \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --reasoning-parser hy_v4 \
  --tool-call-parser hy_v4 \
  --speculative-config '{"num_speculative_tokens":5,"method":"dfly"}'

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /tencent/Hy4-flash.md.

Guide

Overview

HY4 Flash is Tencent's multimodal Mixture-of-Experts model for everyday productivity and agent workflows: 220B total parameters (including 64B of Engram embeddings), 12B active per token, and up to 1M tokens of context.

The model has 42 decoder layers with hybrid sliding-window and compressed sparse attention, Engram embeddings, and a five-layer DFly draft head. Both checkpoints run with BF16 compute:

DFly speculative decoding is on by default; turn off speculative decoding to serve the target model alone. MTP is not supported by this implementation.

Prerequisites

  • Hardware: Hopper or Blackwell data-center GPUs (H100 / H200 / B200 / GB200 / B300 / GB300). AMD Instinct is not supported yet: no ROCm image ships HY4 Flash.
  • vLLM: the dedicated vllm/vllm-openai:hy4-flash Docker image. No released vLLM wheel registers HYV4FlashVLForConditionalGeneration or the DFly drafter yet, so pip installs are not supported.
  • Host memory: Engram tables are offloaded to host RAM by default, so provision host memory in addition to GPU memory.
  • Access: the checkpoints may require Hugging Face authentication with access to the selected repository.

Supported features

  • Tensor parallelism, with optional expert parallelism (TP / TEP strategies).
  • Data parallelism, pipeline parallelism, context parallelism, EPLB, DBO and LoRA are not supported.
  • HY4 uses a paired tool-call / reasoning parser (hy_v4): turning on either Tool calling or Reasoning configures both parsers; Tool calling additionally adds --enable-auto-tool-choice.

Context and memory

The checkpoint advertises 1,048,576 positions. The recipe starts with an 8,192-token context, four concurrent sequences and one image (no video) per prompt; raise --max-model-len, --max-num-seqs and --limit-mm-per-prompt to match your memory budget and workload.

The VRAM hints (529 GB BF16, 345 GB FP8) are the checkpoint download sizes (440.5 GB and 287.1 GB) × 1.2. They are conservative aggregate estimates, not measured GPU minima: Engram storage offloaded to the CPU lowers the actual GPU footprint.

Client usage

Use the selected checkpoint ID as the model name (tencent/Hy4-flash for the default variant, tencent/Hy4-flash-FP8 for FP8):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="tencent/Hy4-flash",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
            {"type": "text", "text": "Summarize this chart."},
        ],
    }],
)
print(resp.choices[0].message.content)

References