tencent/Hy4-flash
Tencent HY4 Flash multimodal MoE for everyday productivity and agent workflows — 220B total / 12B active, 1M context, DFly speculative decoding
export VLLM_USE_RUST_FRONTEND=1
vllm serve tencent/Hy4-flash \
--dtype bfloat16 \
--max-model-len 8192 \
--max-num-seqs 4 \
--max-num-batched-tokens 8192 \
--limit-mm-per-prompt '{"image":1,"video":0}' \
--tensor-parallel-size 8 \
--enable-auto-tool-choice \
--reasoning-parser hy_v4 \
--tool-call-parser hy_v4 \
--speculative-config '{"num_speculative_tokens":5,"method":"dfly"}'The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /tencent/Hy4-flash.md.
Overview
HY4 Flash is Tencent's multimodal Mixture-of-Experts model for everyday productivity and agent workflows: 220B total parameters (including 64B of Engram embeddings), 12B active per token, and up to 1M tokens of context.
The model has 42 decoder layers with hybrid sliding-window and compressed sparse attention, Engram embeddings, and a five-layer DFly draft head. Both checkpoints run with BF16 compute:
- HY4 Flash BF16 — the default variant
- HY4 Flash FP8 — the FP8 variant
DFly speculative decoding is on by default; turn off speculative decoding to serve the target model alone. MTP is not supported by this implementation.
Prerequisites
- Hardware: Hopper or Blackwell data-center GPUs (H100 / H200 / B200 / GB200 / B300 / GB300). AMD Instinct is not supported yet: no ROCm image ships HY4 Flash.
- vLLM: the dedicated
vllm/vllm-openai:hy4-flashDocker image. No released vLLM wheel registersHYV4FlashVLForConditionalGenerationor the DFly drafter yet, so pip installs are not supported. - Host memory: Engram tables are offloaded to host RAM by default, so provision host memory in addition to GPU memory.
- Access: the checkpoints may require Hugging Face authentication with access to the selected repository.
Supported features
- Tensor parallelism, with optional expert parallelism (TP / TEP strategies).
- Data parallelism, pipeline parallelism, context parallelism, EPLB, DBO and LoRA are not supported.
- HY4 uses a paired tool-call / reasoning parser (
hy_v4): turning on either Tool calling or Reasoning configures both parsers; Tool calling additionally adds--enable-auto-tool-choice.
Context and memory
The checkpoint advertises 1,048,576 positions. The recipe starts with an
8,192-token context, four concurrent sequences and one image (no video) per
prompt; raise --max-model-len, --max-num-seqs and --limit-mm-per-prompt
to match your memory budget and workload.
The VRAM hints (529 GB BF16, 345 GB FP8) are the checkpoint download sizes (440.5 GB and 287.1 GB) × 1.2. They are conservative aggregate estimates, not measured GPU minima: Engram storage offloaded to the CPU lowers the actual GPU footprint.
Client usage
Use the selected checkpoint ID as the model name (tencent/Hy4-flash for the
default variant, tencent/Hy4-flash-FP8 for FP8):
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
model="tencent/Hy4-flash",
messages=[{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
{"type": "text", "text": "Summarize this chart."},
],
}],
)
print(resp.choices[0].message.content)