# XiaomiMiMo/MiMo-V2.6-Flash-RL — vLLM serving recipe > Xiaomi's efficiency-balanced omnimodal MoE (309B total / 15B active) with hybrid attention, FP8-compute/mxfp4-stored weights, 1M context, and a DFlash speculative decoder — the default MOPD checkpoint fixes the RL release's tool-call repetition Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (MOPD) · features `tool_calling`, `reasoning` - Interactive page: https://recipes.vllm.ai/XiaomiMiMo/MiMo-V2.6-Flash-RL - This configuration as JSON: https://recipes.vllm.ai/XiaomiMiMo/MiMo-V2.6-Flash-RL.json?hardware=h200 The commands below were rendered for exactly this configuration and take precedence over any example in the guide. ## Other configurations Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line. - `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x` - `variant`: `default`, `rl` - `strategy`: `single_node_tp`, `single_node_tep`, `multi_node_tp`, `multi_node_tep`, `multi_node_dep`, `pd_cluster` (multi-node strategies also need `nodes=`) - `features`: comma-separated list of `tool_calling`, `reasoning`, `spec_decoding` (default: `tool_calling`, `reasoning`) ## Install Requires vLLM >= 0.31.0. ```bash uv venv source .venv/bin/activate uv pip install -U vllm --torch-backend auto ``` Docker image: `vllm/vllm-openai:v0.31.0` ## Serve ```bash vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \ --trust-remote-code \ --max-model-len auto \ --generation-config vllm \ --gpu-memory-utilization 0.95 \ --tensor-parallel-size 8 \ --tool-call-parser mimo \ --enable-auto-tool-choice \ --reasoning-parser mimo ``` Docker equivalent: ```bash docker run --gpus all \ --privileged --ipc=host -p 8000:8000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ vllm/vllm-openai:v0.31.0 XiaomiMiMo/MiMo-V2.6-Flash-MOPD \ --trust-remote-code \ --max-model-len auto \ --generation-config vllm \ --gpu-memory-utilization 0.95 \ --tensor-parallel-size 8 \ --tool-call-parser mimo \ --enable-auto-tool-choice \ --reasoning-parser mimo ``` ## Guide > Background reference. Commands in it are examples and may target another configuration — use the Serve section above. ### Overview This recipe serves the MiMo-V2.6-Flash family from two checkpoints on the Variant row: - **MOPD** (default) — [XiaomiMiMo/MiMo-V2.6-Flash-MOPD](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-MOPD), the MOPD2-distilled upgrade and the recommended starting point for new deployments. It fixes the RL release's most visible agentic failure mode — tool-call repetition, where the model keeps re-issuing the same or highly similar tool calls while making no progress (the blog measures the Flash line as the more affected of the two) — with benchmark quality unchanged, folded into a regular MOPD2 pass via a short specialized-teacher run ([diagnosis blog](https://mimo.xiaomi.com/blog/mimo-v2-6-tool-call-repetition)). - **RL (original)** — [XiaomiMiMo/MiMo-V2.6-Flash-RL](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL), the 2026-09-22 RL release, kept for reproducing earlier results. It can get stuck re-issuing the same tool calls in agentic workloads; prefer **MOPD** unless you are reproducing a specific earlier run. MOPD2 (Technical Report §5.6) is multi-teacher on-policy distillation: domain-specialized mixRL and SFT teachers supervise the student on three on-policy streams (standard MOPD, teacher-prefix OPD, and SFT-prefix OPD). The architecture is shared by both checkpoints: a sparse MoE model with 309B total parameters and 15B active per token — 48 layers (1 dense + 47 MoE) with 256 routed experts (top-8), hybrid attention (sliding-window 128 and global attention), and a 5-layer DFlash-style MTP drafter that predicts 7 tokens per pass. The model is natively omnimodal (text, image, video, audio via a 681M-param MiMo ViT and audio encoders) and supports up to 1M tokens of context. Weights are stored as mxfp4 and computed as FP8 (block-wise e4m3, 128x128), so the checkpoint is only 173 GB on disk. Loading this mixed storage format requires vLLM with the MiMo V2 mxfp4/bf16-router support ([vllm-project/vllm#57784](https://github.com/vllm-project/vllm/pull/57784)), first shipped in the v0.31.0 stable release. ### Prerequisites - Hardware: 4x H200 (TP4), 4x MI355X/MI325X/MI300X (TP4), or equivalent aggregate VRAM (>= 208 GB) #### Pull the vLLM docker image vLLM 0.31.0 is the first stable release that can load the mxfp4-stored weights ([vllm-project/vllm#57784](https://github.com/vllm-project/vllm/pull/57784)); earlier stable releases cannot: ```bash docker pull vllm/vllm-openai:v0.31.0 ``` MI300X / MI325X (gfx942) additionally need the MiMo MXFP4 routing gate ([vllm-project/vllm#58262](https://github.com/vllm-project/vllm/pull/58262)), which merged after the v0.31.0 branch cut — on those GPUs use a nightly wheel built after 2026-09-30 (`uv pip install -U vllm --extra-index-url https://wheels.vllm.ai/nightly/cu130`) or the ROCm nightly image. This release also registers the `mimo` tool-call and reasoning parsers with the Rust frontend ([vllm-project/vllm#57933](https://github.com/vllm-project/vllm/pull/57933)). The Hugging Face model cards point at `vllm/vllm-openai:mimov25-cu129`, the V2.5-era image; that tag predates the mxfp4-stored-weight support linked above, so use the v0.31.0 image recommended above. ### Launch command Single-node TP4 (H200). Selecting the **RL (original)** variant in the builder swaps only the checkpoint id (`XiaomiMiMo/MiMo-V2.6-Flash-RL`); everything else is identical: ```bash vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \ --tensor-parallel-size 4 \ --trust-remote-code \ --gpu-memory-utilization 0.95 \ --max-model-len auto \ --reasoning-parser mimo \ --tool-call-parser mimo \ --enable-auto-tool-choice \ --generation-config vllm ``` #### AMD (MI355X, gfx950) ```bash VLLM_ROCM_USE_AITER=1 \ VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \ vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \ --tensor-parallel-size 4 \ --trust-remote-code \ --kv-cache-dtype fp8 \ --max-model-len auto \ --reasoning-parser mimo \ --tool-call-parser mimo \ --enable-auto-tool-choice \ --generation-config vllm ``` MI355X also runs TP2 just fine. #### AMD (MI300X / MI325X, gfx942) gfx942 has no native MXFP4 MoE path: without an explicit backend vLLM's oracle falls back to OCP emulation — correct, but about 6.5x lower output throughput. Select AITER's Triton W4A16 kernel instead with `--moe-backend aiter_triton_mxfp4_bf16`; supported since [vllm-project/vllm#58262](https://github.com/vllm-project/vllm/pull/58262) (merged 2026-09-30, after the v0.31.0 branch cut — not in any stable release yet, so this flag needs a nightly wheel; it also needs aiter >= 0.1.20, which the ROCm nightly image ships). Verified end-to-end on MI300X in that PR — see it for the gsm8k and `vllm bench serve` numbers against the emulation baseline: ```bash VLLM_ROCM_USE_AITER=1 \ VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 \ vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \ --tensor-parallel-size 2 \ --trust-remote-code \ --max-model-len auto \ --reasoning-parser mimo \ --tool-call-parser mimo \ --enable-auto-tool-choice \ --moe-backend aiter_triton_mxfp4_bf16 \ --generation-config vllm ``` With DFlash speculative decoding (drafter ships inside the checkpoint). vLLM does not resolve a `/dflash` Hub subfolder as the draft model — `model` must be a concrete local path to the `dflash/` directory inside the downloaded snapshot. Resolve it per checkpoint (run `hf download ` first if it is not cached yet): ```bash # MOPD variant DFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Flash-MOPD/snapshots/*/dflash) # RL (original) variant # DFLASH_DIR=$(ls -d ~/.cache/huggingface/hub/models--XiaomiMiMo--MiMo-V2.6-Flash-RL/snapshots/*/dflash) echo "$DFLASH_DIR" ``` ```bash vllm serve XiaomiMiMo/MiMo-V2.6-Flash-MOPD \ --tensor-parallel-size 4 \ --trust-remote-code \ --gpu-memory-utilization 0.95 \ --max-model-len auto \ --speculative-config "{\"method\":\"dflash\",\"model\":\"$DFLASH_DIR\",\"num_speculative_tokens\":7}" \ --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \ --reasoning-parser mimo \ --tool-call-parser mimo \ --enable-auto-tool-choice \ --generation-config vllm ``` The drafter inherits the target's tensor-parallel size by default. To pin it explicitly, add `"draft_tensor_parallel_size": 4` to the speculative config (useful if you want the draft on fewer GPUs than the target, or to silence mis-detection on asymmetric topologies). ### Client Usage ```bash curl -X POST http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "XiaomiMiMo/MiMo-V2.6-Flash-MOPD", "messages": [{"role": "user", "content": "Hello MiMo!"}], "temperature": 1.0, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": true} }' ``` Recommended sampling: `temperature=1.0`, `top_p=0.95`. Set `"enable_thinking": false` (or omit the kwargs) to disable thinking mode. ### References - [MiMo-V2.6-Flash-MOPD on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-MOPD) - [MiMo-V2.6-Flash-RL on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL) - [MiMo-V2.6 technical report](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/blob/main/MiMo_V2_6_technical_report.pdf) (§5.6 covers MOPD2) - [Diagnosing and Mitigating Tool-Call Repetition in MiMo-V2.6](https://mimo.xiaomi.com/blog/mimo-v2-6-tool-call-repetition) - [MiMo-V2.6 blog post](https://mimo.xiaomi.com/mimo-v2-6) - [vLLM PR #57784 — mxfp4 MoE / bf16 router / DFlash for MiMo V2](https://github.com/vllm-project/vllm/pull/57784) - [vLLM PR #57933 — MiMo tool-call/reasoning parsers for the Rust frontend](https://github.com/vllm-project/vllm/pull/57933) - [vLLM PR #58262 — MiMo-V2.6 MXFP4 on gfx942 (`--moe-backend aiter_triton_mxfp4_bf16`)](https://github.com/vllm-project/vllm/pull/58262) - [SGLang MiMo cookbook](https://docs.sglang.io/cookbook/autoregressive/Xiaomi/MiMo-V2.5) - [vLLM ROCm GPU install docs](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/?device=rocm)