{
  "hf_id": "tencent/Hy4-flash",
  "meta": {
    "title": "HY4-Flash",
    "slug": "hy4-flash",
    "provider": "Hunyuan (Tencent)",
    "description": "Tencent HY4 Flash multimodal MoE for everyday productivity and agent workflows — 220B total / 12B active, 1M context, DFly speculative decoding",
    "date_added": "2026-11-17",
    "date_updated": "2026-11-18",
    "tasks": [
      "text",
      "multimodal"
    ],
    "related_recipes": [
      "tencent/Hy4-preview"
    ],
    "hardware": {
      "mi300x": "unsupported",
      "mi325x": "unsupported",
      "mi350x": "unsupported",
      "mi355x": "unsupported"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {
      "VLLM_USE_RUST_FRONTEND": "1"
    },
    "docker_image": "vllm/vllm-openai:hy4-flash",
    "command": "vllm serve tencent/Hy4-flash \\\n  --dtype bfloat16 \\\n  --max-model-len 8192 \\\n  --max-num-seqs 4 \\\n  --max-num-batched-tokens 8192 \\\n  --limit-mm-per-prompt '{\"image\":1,\"video\":0}' \\\n  --tensor-parallel-size 8 \\\n  --enable-auto-tool-choice \\\n  --reasoning-parser hy_v4 \\\n  --tool-call-parser hy_v4 \\\n  --speculative-config '{\"num_speculative_tokens\":5,\"method\":\"dfly\"}'",
    "argv": [
      "vllm",
      "serve",
      "tencent/Hy4-flash",
      "--dtype",
      "bfloat16",
      "--max-model-len",
      "8192",
      "--max-num-seqs",
      "4",
      "--max-num-batched-tokens",
      "8192",
      "--limit-mm-per-prompt",
      "{\"image\":1,\"video\":0}",
      "--tensor-parallel-size",
      "8",
      "--enable-auto-tool-choice",
      "--reasoning-parser",
      "hy_v4",
      "--tool-call-parser",
      "hy_v4",
      "--speculative-config",
      "{\"num_speculative_tokens\":5,\"method\":\"dfly\"}"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  -e VLLM_USE_RUST_FRONTEND=1 \\\n  vllm/vllm-openai:hy4-flash tencent/Hy4-flash \\\n  --dtype bfloat16 \\\n  --max-model-len 8192 \\\n  --max-num-seqs 4 \\\n  --max-num-batched-tokens 8192 \\\n  --limit-mm-per-prompt '{\"image\":1,\"video\":0}' \\\n  --tensor-parallel-size 8 \\\n  --enable-auto-tool-choice \\\n  --reasoning-parser hy_v4 \\\n  --tool-call-parser hy_v4 \\\n  --speculative-config '{\"num_speculative_tokens\":5,\"method\":\"dfly\"}'",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "-e",
      "VLLM_USE_RUST_FRONTEND=1",
      "vllm/vllm-openai:hy4-flash",
      "tencent/Hy4-flash",
      "--dtype",
      "bfloat16",
      "--max-model-len",
      "8192",
      "--max-num-seqs",
      "4",
      "--max-num-batched-tokens",
      "8192",
      "--limit-mm-per-prompt",
      "{\"image\":1,\"video\":0}",
      "--tensor-parallel-size",
      "8",
      "--enable-auto-tool-choice",
      "--reasoning-parser",
      "hy_v4",
      "--tool-call-parser",
      "hy_v4",
      "--speculative-config",
      "{\"num_speculative_tokens\":5,\"method\":\"dfly\"}"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "single_node_tep": "/tencent/Hy4-flash/strategies/single_node_tep.json",
      "kv_store_centralized_mooncake": "/tencent/Hy4-flash/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/tencent/Hy4-flash/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/tencent/Hy4-flash/hw/h200.json",
      "h100": "/tencent/Hy4-flash/hw/h100.json",
      "b200": "/tencent/Hy4-flash/hw/b200.json",
      "gb200": "/tencent/Hy4-flash/hw/gb200.json",
      "b300": "/tencent/Hy4-flash/hw/b300.json",
      "gb300": "/tencent/Hy4-flash/hw/gb300.json"
    }
  },
  "model": {
    "model_id": "tencent/Hy4-flash",
    "docker_image": "vllm/vllm-openai:hy4-flash",
    "architecture": "moe",
    "parameter_count": "220B",
    "active_parameters": "12B",
    "context_length": 1048576,
    "base_args": [
      "--dtype",
      "bfloat16",
      "--max-model-len",
      "8192",
      "--max-num-seqs",
      "4",
      "--max-num-batched-tokens",
      "8192",
      "--limit-mm-per-prompt",
      "{\"image\":1,\"video\":0}"
    ],
    "default_frontend": "rust",
    "install": {
      "docker": {
        "command": "docker pull vllm/vllm-openai:hy4-flash",
        "note": "Dedicated HY4 Flash image — no released vLLM wheel supports this model yet"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "HY4 tool call parser with automatic tool choice (also enables the paired hy_v4 reasoning parser)",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "hy_v4",
        "--reasoning-parser",
        "hy_v4"
      ]
    },
    "reasoning": {
      "description": "HY4 reasoning parser for thinking-content extraction (also enables the paired hy_v4 tool call parser)",
      "args": [
        "--reasoning-parser",
        "hy_v4",
        "--tool-call-parser",
        "hy_v4"
      ]
    },
    "spec_decoding": {
      "description": "DFly speculative decoding with the checkpoint's built-in five-layer draft (5 speculative tokens)",
      "args": [
        "--speculative-config",
        "{\"num_speculative_tokens\":5,\"method\":\"dfly\"}"
      ]
    }
  },
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 529,
      "description": "BF16 checkpoint (tencent/Hy4-flash) — 441 GB of weights; Engram tables live in host memory"
    },
    "fp8": {
      "model_id": "tencent/Hy4-flash-FP8",
      "precision": "fp8",
      "vram_minimum_gb": 345,
      "description": "FP8 checkpoint (tencent/Hy4-flash-FP8), BF16 compute — 287 GB of weights; Engram tables live in host memory",
      "json": "/tencent/Hy4-flash-FP8.json"
    }
  },
  "guide": "## Overview\n\nHY4 Flash is Tencent's multimodal Mixture-of-Experts model for everyday\nproductivity and agent workflows: 220B total parameters (including 64B of\nEngram embeddings), 12B active per token, and up to 1M tokens of context.\n\nThe model has 42 decoder layers with hybrid sliding-window and compressed\nsparse attention, Engram embeddings, and a five-layer DFly draft head. Both\ncheckpoints run with BF16 compute:\n\n- [HY4 Flash BF16](https://huggingface.co/tencent/Hy4-flash) — the default variant\n- [HY4 Flash FP8](https://huggingface.co/tencent/Hy4-flash-FP8) — the FP8 variant\n\nDFly speculative decoding is on by default; turn off speculative decoding to\nserve the target model alone. MTP is not supported by this implementation.\n\n## Prerequisites\n\n- **Hardware**: Hopper or Blackwell data-center GPUs (H100 / H200 / B200 /\n  GB200 / B300 / GB300). AMD Instinct is not supported yet: no ROCm image\n  ships HY4 Flash.\n- **vLLM**: the dedicated `vllm/vllm-openai:hy4-flash` Docker image. No\n  released vLLM wheel registers `HYV4FlashVLForConditionalGeneration` or the\n  DFly drafter yet, so pip installs are not supported.\n- **Host memory**: Engram tables are offloaded to host RAM by default, so\n  provision host memory in addition to GPU memory.\n- **Access**: the checkpoints may require Hugging Face authentication with\n  access to the selected repository.\n\n## Supported features\n\n- Tensor parallelism, with optional expert parallelism (TP / TEP strategies).\n- Data parallelism, pipeline parallelism, context parallelism, EPLB, DBO\n  and LoRA are not supported.\n- HY4 uses a paired tool-call / reasoning parser (`hy_v4`): turning on either\n  Tool calling or Reasoning configures both parsers; Tool calling additionally\n  adds `--enable-auto-tool-choice`.\n\n## Context and memory\n\nThe checkpoint advertises 1,048,576 positions. The recipe starts with an\n8,192-token context, four concurrent sequences and one image (no video) per\nprompt; raise `--max-model-len`, `--max-num-seqs` and `--limit-mm-per-prompt`\nto match your memory budget and workload.\n\nThe VRAM hints (529 GB BF16, 345 GB FP8) are the checkpoint download sizes\n(440.5 GB and 287.1 GB) × 1.2. They are conservative aggregate estimates, not\nmeasured GPU minima: Engram storage offloaded to the CPU lowers the actual GPU\nfootprint.\n\n## Client usage\n\nUse the selected checkpoint ID as the model name (`tencent/Hy4-flash` for the\ndefault variant, `tencent/Hy4-flash-FP8` for FP8):\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\nresp = client.chat.completions.create(\n    model=\"tencent/Hy4-flash\",\n    messages=[{\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/chart.png\"}},\n            {\"type\": \"text\", \"text\": \"Summarize this chart.\"},\n        ],\n    }],\n)\nprint(resp.choices[0].message.content)\n```\n\n## References\n\n- [HY4 Flash research page](https://hy.tencent.ai/research/hy4-flash?langVersion=en)\n- [BF16 checkpoint](https://huggingface.co/tencent/Hy4-flash)\n- [FP8 checkpoint](https://huggingface.co/tencent/Hy4-flash-FP8)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "51d9b9ab07f18797c5db0092671b20171a621728",
    "path": "recipes/models/tencent/Hy4-flash.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/tencent/Hy4-flash.yaml",
    "overlays": []
  }
}