# XiaomiMiMo/MiMo-V2.5-Pro — vLLM serving recipe

> Xiaomi's flagship MoE reasoning model (1.02T total / 42B active) with hybrid attention, native FP8 weights, and Multi-Token Prediction

Configuration: hardware `h200` (8× H200) · strategy `pd_cluster` (Prefill/Decode Disaggregation) · variant `default` (FP8) · features `tool_calling`, `reasoning`

- Interactive page: https://recipes.vllm.ai/XiaomiMiMo/MiMo-V2.5-Pro
- This configuration as JSON: https://recipes.vllm.ai/XiaomiMiMo/MiMo-V2.5-Pro.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `single_node_tep`, `pd_cluster` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `spec_decoding` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.21.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

Disaggregated prefill/decode (KV transfer: `nixl`). Start both pools, then the router.

### Prefill — pool spans 2 nodes; this is node 0's command (add `prefill_rank=<n>` to the query for another)

```bash
export VLLM_NIXL_SIDE_CHANNEL_PORT=5557
export VLLM_NIXL_SIDE_CHANNEL_HOST=$PREFILL_NODE_1
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
export GLOO_SOCKET_IFNAME=$IFACE_NAME
export NCCL_SOCKET_IFNAME=$IFACE_NAME
vllm serve XiaomiMiMo/MiMo-V2.5-Pro \
  --trust-remote-code \
  --max-model-len auto \
  --generation-config vllm \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --port 8001 \
  --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_producer","kv_load_failure_policy":"fail"}' \
  --tensor-parallel-size 16 \
  --nnodes 2 \
  --node-rank 0 \
  --master-addr $PREFILL_NODE_1 \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --reasoning-parser mimo
```

### Decode — pool spans 2 nodes; this is node 0's command (add `decode_rank=<n>` to the query for another)

```bash
export VLLM_NIXL_SIDE_CHANNEL_PORT=5558
export VLLM_NIXL_SIDE_CHANNEL_HOST=$DECODE_NODE_1
export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
export GLOO_SOCKET_IFNAME=$IFACE_NAME
export NCCL_SOCKET_IFNAME=$IFACE_NAME
vllm serve XiaomiMiMo/MiMo-V2.5-Pro \
  --trust-remote-code \
  --max-model-len auto \
  --generation-config vllm \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --port 8002 \
  --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \
  --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_consumer","kv_load_failure_policy":"fail"}' \
  --tensor-parallel-size 16 \
  --nnodes 2 \
  --node-rank 0 \
  --master-addr $DECODE_NODE_1 \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --reasoning-parser mimo
```

### Router

```bash
uv pip install vllm-router
```

```bash
vllm-router --policy round_robin \
    --vllm-pd-disaggregation \
    --prefill http://$PREFILL_NODE_1:8001 \
    --decode http://$DECODE_NODE_1:8002 \
    --host $ROUTER_HOST \
    --port $ROUTER_PORT \
    --intra-node-data-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

MiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters
and 42B active per token. It uses 384 routed experts (top-8) with hybrid
attention (full-attention + sliding-window 128 at 6:1 ratio) over 70 layers
(1 dense + 69 MoE) and ships with native FP8 (block-wise e4m3) weights.
A 3-layer Multi-Token Prediction (MTP) head enables speculative decoding for
~3x output speed.

### Prerequisites

- Hardware: 8x H200 (TP8)

#### Pull the vLLM docker image

Stable vLLM does not yet support MiMo V2.5. Use the pre-built image:

```bash
docker pull vllm/vllm-openai:mimov25-cu129
```

### Launch commands

Single-node TP8 (H200):

```bash
vllm serve XiaomiMiMo/MiMo-V2.5-Pro \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --generation-config vllm
```

With tool calling + reasoning:

```bash
vllm serve XiaomiMiMo/MiMo-V2.5-Pro \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --generation-config vllm
```

### Client Usage

```bash
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "XiaomiMiMo/MiMo-V2.5-Pro",
    "messages": [{"role": "user", "content": "Hello MiMo!"}],
    "chat_template_kwargs": {"enable_thinking": true}
  }'
```

Set `"enable_thinking": false` (or omit the kwargs) to disable thinking mode.


### References

- [MiMo-V2.5-Pro on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro)