{
  "hf_id": "XiaomiMiMo/MiMo-V2.6-Flash-RL",
  "meta": {
    "title": "MiMo-V2.6-Flash-RL",
    "slug": "mimo-v2-6-flash-rl",
    "provider": "MiMo (Xiaomi)",
    "description": "Xiaomi's efficiency-balanced omnimodal MoE (309B total / 15B active) with hybrid attention, FP8-compute/mxfp4-stored weights, 1M context, and a DFlash speculative decoder — the default MOPD checkpoint fixes the RL release's tool-call repetition",
    "date_added": "2026-09-22",
    "date_updated": "2026-10-07",
    "tasks": [
      "text",
      "multimodal"
    ],
    "related_recipes": [
      "XiaomiMiMo/MiMo-V2.6-Pro-RL",
      "XiaomiMiMo/MiMo-V2-Flash"
    ],
    "hardware": {
      "h200": "verified",
      "gb300": "verified",
      "mi300x": "verified",
      "mi325x": "verified",
      "mi355x": "verified"
    }
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:v0.31.0",
    "command": "vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \\\n  --trust-remote-code \\\n  --max-model-len auto \\\n  --generation-config vllm \\\n  --gpu-memory-utilization 0.95 \\\n  --tensor-parallel-size 8 \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --reasoning-parser mimo",
    "argv": [
      "vllm",
      "serve",
      "XiaomiMiMo/MiMo-V2.6-Flash-MOPD",
      "--trust-remote-code",
      "--max-model-len",
      "auto",
      "--generation-config",
      "vllm",
      "--gpu-memory-utilization",
      "0.95",
      "--tensor-parallel-size",
      "8",
      "--tool-call-parser",
      "mimo",
      "--enable-auto-tool-choice",
      "--reasoning-parser",
      "mimo"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:v0.31.0 XiaomiMiMo/MiMo-V2.6-Flash-MOPD \\\n  --trust-remote-code \\\n  --max-model-len auto \\\n  --generation-config vllm \\\n  --gpu-memory-utilization 0.95 \\\n  --tensor-parallel-size 8 \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --reasoning-parser mimo",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:v0.31.0",
      "XiaomiMiMo/MiMo-V2.6-Flash-MOPD",
      "--trust-remote-code",
      "--max-model-len",
      "auto",
      "--generation-config",
      "vllm",
      "--gpu-memory-utilization",
      "0.95",
      "--tensor-parallel-size",
      "8",
      "--tool-call-parser",
      "mimo",
      "--enable-auto-tool-choice",
      "--reasoning-parser",
      "mimo"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "single_node_tep": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/single_node_tep.json",
      "multi_node_tp": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/multi_node_tp.json",
      "multi_node_tep": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/multi_node_tep.json",
      "multi_node_dep": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/multi_node_dep.json",
      "pd_cluster": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/pd_cluster.json",
      "kv_store_centralized_mooncake": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/h200.json",
      "h100": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/h100.json",
      "b200": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/b200.json",
      "gb200": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/gb200.json",
      "b300": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/b300.json",
      "gb300": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/gb300.json",
      "mi300x": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/mi300x.json",
      "mi325x": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/mi325x.json",
      "mi350x": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/mi350x.json",
      "mi355x": "/XiaomiMiMo/MiMo-V2.6-Flash-RL/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "XiaomiMiMo/MiMo-V2.6-Flash-RL",
    "min_vllm_version": "0.31.0",
    "docker_image": {
      "nvidia": "vllm/vllm-openai:v0.31.0",
      "amd": "vllm/vllm-openai-rocm:nightly"
    },
    "architecture": "moe",
    "parameter_count": "309B",
    "active_parameters": "15B",
    "context_length": 1048576,
    "base_args": [
      "--trust-remote-code",
      "--max-model-len",
      "auto",
      "--generation-config",
      "vllm"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:v0.31.0"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "MiMo tool-call parser",
      "args": [
        "--tool-call-parser",
        "mimo",
        "--enable-auto-tool-choice"
      ]
    },
    "reasoning": {
      "description": "MiMo reasoning parser",
      "args": [
        "--reasoning-parser",
        "mimo"
      ]
    },
    "spec_decoding": {
      "description": "DFlash 5-layer MTP drafter (predicts 7 tokens); ships in the dflash/ subfolder of every checkpoint. Set DFLASH_DIR to that directory first (see the guide)",
      "args": [
        "--speculative-config",
        "{\"method\":\"dflash\",\"model\":\"$DFLASH_DIR\",\"num_speculative_tokens\":7}"
      ]
    }
  },
  "opt_in_features": [
    "spec_decoding"
  ],
  "variants": {
    "default": {
      "model_id": "XiaomiMiMo/MiMo-V2.6-Flash-MOPD",
      "label": "MOPD",
      "precision": "fp8",
      "vram_minimum_gb": 208,
      "description": "FP8 (e4m3) compute with mxfp4-stored MoE weights; 173 GB checkpoint — 4x H200 with TP4. MOPD2-distilled upgrade that fixes tool-call repetition; recommended.",
      "tp": {
        "mi300x": 2
      },
      "extra_args": [
        "--tensor-parallel-size",
        "4",
        "--gpu-memory-utilization",
        "0.95"
      ],
      "hardware_overrides": {
        "mi300x": {
          "extra_args": [
            "--moe-backend",
            "aiter_triton_mxfp4_bf16"
          ]
        },
        "mi325x": {
          "extra_args": [
            "--moe-backend",
            "aiter_triton_mxfp4_bf16"
          ]
        }
      },
      "json": "/XiaomiMiMo/MiMo-V2.6-Flash-MOPD.json"
    },
    "rl": {
      "label": "RL (original)",
      "precision": "fp8",
      "vram_minimum_gb": 208,
      "description": "Same shape as the MOPD variant (173 GB, FP8-compute/mxfp4-stored) — the original RL release, kept for reproducing earlier results. Suffers from tool-call repetition (the more affected of the two sizes); prefer MOPD for agentic workloads.",
      "tp": {
        "mi300x": 2
      },
      "extra_args": [
        "--tensor-parallel-size",
        "4",
        "--gpu-memory-utilization",
        "0.95"
      ],
      "hardware_overrides": {
        "mi300x": {
          "extra_args": [
            "--moe-backend",
            "aiter_triton_mxfp4_bf16"
          ]
        },
        "mi325x": {
          "extra_args": [
            "--moe-backend",
            "aiter_triton_mxfp4_bf16"
          ]
        }
      }
    }
  },
  "hardware_overrides": {
    "amd": {
      "extra_args": [
        "--kv-cache-dtype",
        "fp8"
      ],
      "extra_env": {
        "VLLM_ROCM_USE_AITER": "1",
        "VLLM_ROCM_QUICK_REDUCE_QUANTIZATION": "INT4"
      }
    }
  },
  "guide": "## Overview\n\nThis recipe serves the MiMo-V2.6-Flash family from two checkpoints on the\nVariant row:\n\n- **MOPD** (default) — [XiaomiMiMo/MiMo-V2.6-Flash-MOPD](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-MOPD),\n  the MOPD2-distilled upgrade and the recommended starting point for new\n  deployments. It fixes the RL release's most visible agentic failure mode —\n  tool-call repetition, where the model keeps re-issuing the same or highly\n  similar tool calls while making no progress (the blog measures the Flash\n  line as the more affected of the two) — with benchmark quality unchanged,\n  folded into a regular MOPD2 pass via a short specialized-teacher run\n  ([diagnosis blog](https://mimo.xiaomi.com/blog/mimo-v2-6-tool-call-repetition)).\n- **RL (original)** — [XiaomiMiMo/MiMo-V2.6-Flash-RL](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL),\n  the 2026-09-22 RL release, kept for reproducing earlier results. It can get\n  stuck re-issuing the same tool calls in agentic workloads; prefer **MOPD**\n  unless you are reproducing a specific earlier run.\n\nMOPD2 (Technical Report §5.6) is multi-teacher on-policy distillation:\ndomain-specialized mixRL and SFT teachers supervise the student on three\non-policy streams (standard MOPD, teacher-prefix OPD, and SFT-prefix OPD).\n\nThe architecture is shared by both checkpoints: a sparse MoE model with\n309B total parameters and 15B active per token — 48 layers (1 dense + 47 MoE)\nwith 256 routed experts (top-8), hybrid attention (sliding-window 128 and\nglobal attention), and a 5-layer DFlash-style MTP drafter that predicts 7\ntokens per pass. The model is natively omnimodal (text, image, video, audio\nvia a 681M-param MiMo ViT and audio encoders) and supports up to 1M tokens of\ncontext.\n\nWeights are stored as mxfp4 and computed as FP8 (block-wise e4m3, 128x128),\nso the checkpoint is only 173 GB on disk. Loading this mixed storage format\nrequires vLLM with the MiMo V2 mxfp4/bf16-router support\n([vllm-project/vllm#57784](https://github.com/vllm-project/vllm/pull/57784)),\nfirst shipped in the v0.31.0 stable release.\n\n## Prerequisites\n\n- Hardware: 4x H200 (TP4), 4x MI355X/MI325X/MI300X (TP4), or equivalent aggregate VRAM (>= 208 GB)\n\n### Pull the vLLM docker image\n\nvLLM 0.31.0 is the first stable release that can load the mxfp4-stored\nweights ([vllm-project/vllm#57784](https://github.com/vllm-project/vllm/pull/57784));\nearlier stable releases cannot:\n\n```bash\ndocker pull vllm/vllm-openai:v0.31.0\n```\n\nMI300X / MI325X (gfx942) additionally need the MiMo MXFP4 routing gate\n([vllm-project/vllm#58262](https://github.com/vllm-project/vllm/pull/58262)),\nwhich merged after the v0.31.0 branch cut — on those GPUs use a nightly wheel\nbuilt after 2026-09-30 (`uv pip install -U vllm --extra-index-url\nhttps://wheels.vllm.ai/nightly/cu130`) or the ROCm nightly image.\n\nThis release also registers the `mimo` tool-call and reasoning parsers with\nthe Rust frontend\n([vllm-project/vllm#57933](https://github.com/vllm-project/vllm/pull/57933)).\n\nThe Hugging Face model cards point at `vllm/vllm-openai:mimov25-cu129`, the\nV2.5-era image; that tag predates the mxfp4-stored-weight support linked above,\nso use the v0.31.0 image recommended above.\n\n## Launch command\n\nSingle-node TP4 (H200). Selecting the **RL (original)** variant in the builder\nswaps only the checkpoint id (`XiaomiMiMo/MiMo-V2.6-Flash-RL`); everything\nelse is identical:\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \\\n  --tensor-parallel-size 4 \\\n  --trust-remote-code \\\n  --gpu-memory-utilization 0.95 \\\n  --max-model-len auto \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --generation-config vllm\n```\n\n### AMD (MI355X, gfx950)\n\n```bash\nVLLM_ROCM_USE_AITER=1 \\\nVLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \\\nvllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \\\n  --tensor-parallel-size 4 \\\n  --trust-remote-code \\\n  --kv-cache-dtype fp8 \\\n  --max-model-len auto \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --generation-config vllm\n```\n\nMI355X also runs TP2 just fine.\n\n### AMD (MI300X / MI325X, gfx942)\n\ngfx942 has no native MXFP4 MoE path: without an explicit backend vLLM's\noracle falls back to OCP emulation — correct, but about 6.5x lower output\nthroughput. Select AITER's Triton W4A16 kernel instead with\n`--moe-backend aiter_triton_mxfp4_bf16`; supported since\n[vllm-project/vllm#58262](https://github.com/vllm-project/vllm/pull/58262)\n(merged 2026-09-30, after the v0.31.0 branch cut — not in any stable release\nyet, so this flag needs a nightly wheel; it also needs aiter >= 0.1.20, which\nthe ROCm nightly image ships). Verified end-to-end on MI300X in that PR — see\nit for the gsm8k and `vllm bench serve` numbers against the emulation\nbaseline:\n\n```bash\nVLLM_ROCM_USE_AITER=1 \\\nVLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \\\nvllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \\\n  --tensor-parallel-size 2 \\\n  --trust-remote-code \\\n  --max-model-len auto \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --moe-backend aiter_triton_mxfp4_bf16 \\\n  --generation-config vllm\n```\nWith DFlash speculative decoding (drafter ships inside the checkpoint).\nvLLM does not resolve a `<repo>/dflash` Hub subfolder as the draft model —\n`model` must be a concrete local path to the `dflash/` directory inside the\ndownloaded snapshot. Resolve it per checkpoint (run `hf download\n<checkpoint>` first if it is not cached yet):\n\n```bash\n# MOPD variant\nDFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Flash-MOPD/snapshots/*/dflash)\n# RL (original) variant\n# DFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Flash-RL/snapshots/*/dflash)\necho \"$DFLASH_DIR\"\n```\n\n```bash\nvllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \\\n  --tensor-parallel-size 4 \\\n  --trust-remote-code \\\n  --gpu-memory-utilization 0.95 \\\n  --max-model-len auto \\\n  --speculative-config \"{\\\"method\\\":\\\"dflash\\\",\\\"model\\\":\\\"$DFLASH_DIR\\\",\\\"num_speculative_tokens\\\":7}\" \\\n  --compilation-config '{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}' \\\n  --reasoning-parser mimo \\\n  --tool-call-parser mimo \\\n  --enable-auto-tool-choice \\\n  --generation-config vllm\n```\n\nThe drafter inherits the target's tensor-parallel size by default. To pin it\nexplicitly, add `\"draft_tensor_parallel_size\": 4` to the speculative config\n(useful if you want the draft on fewer GPUs than the target, or to silence\nmis-detection on asymmetric topologies).\n\n## Client Usage\n\n```bash\ncurl -X POST http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Flash-MOPD\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello MiMo!\"}],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"chat_template_kwargs\": {\"enable_thinking\": true}\n  }'\n```\n\nRecommended sampling: `temperature=1.0`, `top_p=0.95`.\nSet `\"enable_thinking\": false` (or omit the kwargs) to disable thinking mode.\n\n## References\n\n- [MiMo-V2.6-Flash-MOPD on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-MOPD)\n- [MiMo-V2.6-Flash-RL on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL)\n- [MiMo-V2.6 technical report](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/blob/main/MiMo_V2_6_technical_report.pdf) (§5.6 covers MOPD2)\n- [Diagnosing and Mitigating Tool-Call Repetition in MiMo-V2.6](https://mimo.xiaomi.com/blog/mimo-v2-6-tool-call-repetition)\n- [MiMo-V2.6 blog post](https://mimo.xiaomi.com/mimo-v2-6)\n- [vLLM PR #57784 — mxfp4 MoE / bf16 router / DFlash for MiMo V2](https://github.com/vllm-project/vllm/pull/57784)\n- [vLLM PR #57933 — MiMo tool-call/reasoning parsers for the Rust frontend](https://github.com/vllm-project/vllm/pull/57933)\n- [vLLM PR #58262 — MiMo-V2.6 MXFP4 on gfx942 (`--moe-backend aiter_triton_mxfp4_bf16`)](https://github.com/vllm-project/vllm/pull/58262)\n- [SGLang MiMo cookbook](https://docs.sglang.io/cookbook/autoregressive/Xiaomi/MiMo-V2.5)\n- [vLLM ROCm GPU install docs](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/?device=rocm)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb",
    "path": "recipes/models/XiaomiMiMo/MiMo-V2.6-Flash-RL.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/XiaomiMiMo/MiMo-V2.6-Flash-RL.yaml",
    "overlays": []
  }
}