{
  "hf_id": "XiaomiMiMo/MiMo-V2.6-Pro-RL",
  "meta": {
    "title": "MiMo-V2.6-Pro-RL",
    "slug": "mimo-v2-6-pro-rl",
    "provider": "MiMo (Xiaomi)",
    "description": "Xiaomi's flagship omnimodal MoE (1.02T total / 42B active) with hybrid attention, FP8-compute/mxfp4-stored weights, 1M context, and a DFlash speculative decoder — the default MOPD checkpoint fixes the RL release's tool-call repetition",
    "date_added": "2026-09-22",
    "date_updated": "2026-10-07",
    "tasks": [
      "text",
      "multimodal"
    ],
    "related_recipes": [
      "XiaomiMiMo/MiMo-V2.6-Flash-RL",
      "XiaomiMiMo/MiMo-V2.5-Pro"
    ],
    "hardware": {
      "h200": "verified",
      "gb300": "verified",
      "mi355x": "verified"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:v0.31.0",
    "command": "vllm serve XiaomiMiMo/MiMo-V2.6-Pro-MOPD \\\n  --trust-remote-code \\\n  --max-model-len auto \\\n  --generation-config vllm \\\n  --gpu-memory-utilization 0.95 \\\n  --tensor-parallel-size 8 \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --reasoning-parser mimo",
    "argv": [
      "vllm",
      "serve",
      "XiaomiMiMo/MiMo-V2.6-Pro-MOPD",
      "--trust-remote-code",
      "--max-model-len",
      "auto",
      "--generation-config",
      "vllm",
      "--gpu-memory-utilization",
      "0.95",
      "--tensor-parallel-size",
      "8",
      "--tool-call-parser",
      "mimo",
      "--enable-auto-tool-choice",
      "--reasoning-parser",
      "mimo"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:v0.31.0 XiaomiMiMo/MiMo-V2.6-Pro-MOPD \\\n  --trust-remote-code \\\n  --max-model-len auto \\\n  --generation-config vllm \\\n  --gpu-memory-utilization 0.95 \\\n  --tensor-parallel-size 8 \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --reasoning-parser mimo",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:v0.31.0",
      "XiaomiMiMo/MiMo-V2.6-Pro-MOPD",
      "--trust-remote-code",
      "--max-model-len",
      "auto",
      "--generation-config",
      "vllm",
      "--gpu-memory-utilization",
      "0.95",
      "--tensor-parallel-size",
      "8",
      "--tool-call-parser",
      "mimo",
      "--enable-auto-tool-choice",
      "--reasoning-parser",
      "mimo"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "single_node_tep": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/strategies/single_node_tep.json",
      "pd_cluster": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/strategies/pd_cluster.json",
      "kv_store_centralized_mooncake": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/h200.json",
      "h100": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/h100.json",
      "b200": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/b200.json",
      "gb200": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/gb200.json",
      "b300": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/b300.json",
      "gb300": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/gb300.json",
      "mi300x": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/mi300x.json",
      "mi325x": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/mi325x.json",
      "mi350x": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/mi350x.json",
      "mi355x": "/XiaomiMiMo/MiMo-V2.6-Pro-RL/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "XiaomiMiMo/MiMo-V2.6-Pro-RL",
    "min_vllm_version": "0.31.0",
    "docker_image": {
      "nvidia": "vllm/vllm-openai:v0.31.0",
      "amd": "vllm/vllm-openai-rocm:nightly"
    },
    "architecture": "moe",
    "parameter_count": "1T",
    "active_parameters": "42B",
    "context_length": 1048576,
    "base_args": [
      "--trust-remote-code",
      "--max-model-len",
      "auto",
      "--generation-config",
      "vllm"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:v0.31.0"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "MiMo tool-call parser",
      "args": [
        "--tool-call-parser",
        "mimo",
        "--enable-auto-tool-choice"
      ]
    },
    "reasoning": {
      "description": "MiMo reasoning parser",
      "args": [
        "--reasoning-parser",
        "mimo"
      ]
    },
    "spec_decoding": {
      "description": "DFlash 5-layer MTP drafter (predicts 7 tokens); ships in the dflash/ subfolder of every checkpoint. Set DFLASH_DIR to that directory first (see the guide)",
      "args": [
        "--speculative-config",
        "{\"method\":\"dflash\",\"model\":\"$DFLASH_DIR\",\"num_speculative_tokens\":7}"
      ]
    }
  },
  "opt_in_features": [
    "spec_decoding"
  ],
  "variants": {
    "default": {
      "model_id": "XiaomiMiMo/MiMo-V2.6-Pro-MOPD",
      "label": "MOPD",
      "precision": "fp8",
      "vram_minimum_gb": 680,
      "description": "FP8 (e4m3) compute with mxfp4-stored MoE weights; 566 GB checkpoint — 8x H200 with TP8, or 4x MI355X with TP4. MOPD2-distilled upgrade that fixes tool-call repetition; recommended.",
      "tp": {
        "mi355x": 4
      },
      "extra_args": [
        "--tensor-parallel-size",
        "8",
        "--gpu-memory-utilization",
        "0.95"
      ],
      "json": "/XiaomiMiMo/MiMo-V2.6-Pro-MOPD.json"
    },
    "rl": {
      "label": "RL (original)",
      "precision": "fp8",
      "vram_minimum_gb": 680,
      "description": "Same shape as the MOPD variant (566 GB, FP8-compute/mxfp4-stored) — the original RL release, kept for reproducing earlier results. Suffers from tool-call repetition; prefer MOPD for agentic workloads.",
      "tp": {
        "mi355x": 4
      },
      "extra_args": [
        "--tensor-parallel-size",
        "8",
        "--gpu-memory-utilization",
        "0.95"
      ]
    }
  },
  "hardware_overrides": {
    "amd": {
      "extra_args": [
        "--gpu-memory-utilization",
        "0.93",
        "--kv-cache-dtype",
        "fp8"
      ],
      "extra_env": {
        "VLLM_ROCM_USE_AITER": "1",
        "VLLM_ROCM_QUICK_REDUCE_QUANTIZATION": "INT4"
      }
    }
  },
  "guide": "## Overview\n\nThis recipe serves the MiMo-V2.6-Pro family from two checkpoints on the\nVariant row:\n\n- **MOPD** (default) — [XiaomiMiMo/MiMo-V2.6-Pro-MOPD](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-MOPD),\n  the MOPD2-distilled upgrade and the recommended starting point for new\n  deployments. It fixes the RL release's most visible agentic failure mode —\n  tool-call repetition, where the model keeps re-issuing the same or highly\n  similar tool calls while making no progress — with benchmark quality\n  unchanged, folded into a regular MOPD2 pass via a short specialized-teacher\n  run ([diagnosis blog](https://mimo.xiaomi.com/blog/mimo-v2-6-tool-call-repetition)).\n- **RL (original)** — [XiaomiMiMo/MiMo-V2.6-Pro-RL](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL),\n  the 2026-09-22 RL release, kept for reproducing earlier results. It can get\n  stuck re-issuing the same tool calls in agentic workloads; prefer **MOPD**\n  unless you are reproducing a specific earlier run.\n\nMOPD2 (Technical Report §5.6) is multi-teacher on-policy distillation:\ndomain-specialized mixRL and SFT teachers supervise the student on three\non-policy streams (standard MOPD, teacher-prefix OPD, and SFT-prefix OPD).\n\nThe architecture is shared by both checkpoints: a sparse MoE model with\n1.02T total parameters and 42B active per token — 70 layers (1 dense + 69 MoE)\nwith 384 routed experts (top-8), hybrid attention (sliding-window 128 and\nglobal attention at a 6:1 ratio), and a 5-layer DFlash-style MTP drafter that\npredicts 7 tokens per pass. The model is natively omnimodal (text, image,\nvideo, audio via a 681M-param MiMo ViT and audio encoders) and supports up to\n1M tokens of context.\n\nWeights are stored as mxfp4 and computed as FP8 (block-wise e4m3, 128x128),\nso the checkpoint is 566 GB on disk. Loading this mixed storage format\nrequires vLLM with the MiMo V2 mxfp4/bf16-router support\n([vllm-project/vllm#57784](https://github.com/vllm-project/vllm/pull/57784)),\nfirst shipped in the v0.31.0 stable release.\n\n## Prerequisites\n\n- Hardware: 8x H200 (TP8), 4x MI355X (TP4), or equivalent aggregate VRAM (>= 680 GB)\n\n### Pull the vLLM docker image\n\nvLLM 0.31.0 is the first stable release that can load the mxfp4-stored\nweights ([vllm-project/vllm#57784](https://github.com/vllm-project/vllm/pull/57784));\nearlier stable releases cannot:\n\n```bash\ndocker pull vllm/vllm-openai:v0.31.0\n```\n\nA vLLM nightly wheel built after 2026-09-20 also works (`uv pip install -U\nvllm --extra-index-url https://wheels.vllm.ai/nightly/cu130`).\n\nThe Hugging Face model cards point at `vllm/vllm-openai:mimov25-cu129`, the\nV2.5-era image; that tag predates the mxfp4-stored-weight support linked above,\nso use the v0.31.0 image recommended above.\n\nThis release also registers the `mimo` tool-call and reasoning parsers with\nthe Rust frontend\n([vllm-project/vllm#57933](https://github.com/vllm-project/vllm/pull/57933)).\n\n## Launch command\n\nSingle-node TP8 (H200). Selecting the **RL (original)** variant in the builder\nswaps only the checkpoint id (`XiaomiMiMo/MiMo-V2.6-Pro-RL`); everything else\nis identical:\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2.6-Pro-MOPD \\\n  --tensor-parallel-size 8 \\\n  --trust-remote-code \\\n  --gpu-memory-utilization 0.95 \\\n  --max-model-len auto \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --generation-config vllm\n```\n\n### AMD (MI355X, TP4)\n\n```bash\nVLLM_ROCM_USE_AITER=1 \\\nVLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \\\nvllm serve XiaomiMiMo/MiMo-V2.6-Pro-MOPD \\\n  --tensor-parallel-size 4 \\\n  --trust-remote-code \\\n  --gpu-memory-utilization 0.93 \\\n  --kv-cache-dtype fp8 \\\n  --max-model-len auto \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --generation-config vllm\n```\n\nMXFP4 MoE weights run natively on CDNA 4, so no load-time requantization is\nneeded. `VLLM_ROCM_USE_AITER=1` is off by default and is required to reach\nthe AITER MXFP4 MoE and FP8 block-scaled GEMM kernels. Attention resolves to\n`TRITON_ATTN_DIFFKV`, the only backend covering this model's asymmetric head\ndims (QK 192 / V 128) together with attention sinks and sliding-window\nattention.\n\nOn MI355X, **Tensor Parallel** defaults to **TP4**: 4x288 GiB clears the 680 GiB\nrequirement, so TP4 halves the GPUs per server and doubles the servers per node.\nFor TP8, pass `--tensor-parallel-size 8`.\n\nThe checkpoint pre-shards its fused QKV into `num_key_value_heads` (8) chunks,\neach carrying its own FP8 scales. TP8 consumes them directly; TP4 merges two\nchunks per rank, which vLLM handles by dequantizing, reordering and\nre-quantizing at load time. TP4 costs no accuracy.\n\nWith DFlash speculative decoding (drafter ships inside the checkpoint).\nvLLM does not resolve a `<repo>/dflash` Hub subfolder as the draft model —\n`model` must be a concrete local path to the `dflash/` directory inside the\ndownloaded snapshot. Resolve it per checkpoint (run `hf download\n<checkpoint>` first if it is not cached yet):\n\n```bash\n# MOPD variant\nDFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Pro-MOPD/snapshots/*/dflash)\n# RL (original) variant\n# DFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Pro-RL/snapshots/*/dflash)\necho \"$DFLASH_DIR\"\n```\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2.6-Pro-MOPD \\\n  --tensor-parallel-size 8 \\\n  --trust-remote-code \\\n  --gpu-memory-utilization 0.95 \\\n  --max-model-len auto \\\n  --speculative-config \"{\\\"method\\\":\\\"dflash\\\",\\\"model\\\":\\\"$DFLASH_DIR\\\",\\\"num_speculative_tokens\\\":7}\" \\\n  --compilation-config '{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}' \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --generation-config vllm\n```\n\nThe drafter inherits the target's tensor-parallel size by default. To pin it\nexplicitly, add `\"draft_tensor_parallel_size\": 8` to the speculative config\n(useful if you want the draft on fewer GPUs than the target, or to silence\nmis-detection on asymmetric topologies).\n\n## Client Usage\n\n```bash\ncurl -X POST http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Pro-MOPD\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello MiMo!\"}],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"chat_template_kwargs\": {\"enable_thinking\": true}\n  }'\n```\n\nRecommended sampling: `temperature=1.0`, `top_p=0.95`.\nSet `\"enable_thinking\": false` (or omit the kwargs) to disable thinking mode.\n\n## References\n\n- [MiMo-V2.6-Pro-MOPD on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-MOPD)\n- [MiMo-V2.6-Pro-RL on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL)\n- [MiMo-V2.6 technical report](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/blob/main/MiMo_V2_6_technical_report.pdf) (§5.6 covers MOPD2)\n- [Diagnosing and Mitigating Tool-Call Repetition in MiMo-V2.6](https://mimo.xiaomi.com/blog/mimo-v2-6-tool-call-repetition)\n- [MiMo-V2.6 blog post](https://mimo.xiaomi.com/mimo-v2-6)\n- [vLLM PR #57784 — mxfp4 MoE / bf16 router / DFlash for MiMo V2](https://github.com/vllm-project/vllm/pull/57784)\n- [vLLM PR #57933 — MiMo tool-call/reasoning parsers for the Rust frontend](https://github.com/vllm-project/vllm/pull/57933)\n- [SGLang MiMo cookbook](https://docs.sglang.io/cookbook/autoregressive/Xiaomi/MiMo-V2.5)\n- [vLLM ROCm GPU install docs](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/?device=rocm)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb",
    "path": "recipes/models/XiaomiMiMo/MiMo-V2.6-Pro-RL.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/XiaomiMiMo/MiMo-V2.6-Pro-RL.yaml",
    "overlays": []
  }
}