# openbmb/MiniCPM5-2B — vLLM serving recipe

> MiniCPM5-2B — dense 2B LLM with hybrid Think/No-Think reasoning, native 128K context, and native tool calling support, built on the standard Llama architecture

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`

- Interactive page: https://recipes.vllm.ai/openbmb/MiniCPM5-2B
- This configuration as JSON: https://recipes.vllm.ai/openbmb/MiniCPM5-2B.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `reasoning`, `tool_calling` (default: `tool_calling`)

## Install

Requires vLLM >= 0.21.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve openbmb/MiniCPM5-2B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser minicpm5
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest openbmb/MiniCPM5-2B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser minicpm5
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

**MiniCPM5-2B** is the 2B checkpoint in OpenBMB's MiniCPM5 series — a dense
model built for on-device and resource-constrained deployment, with strong
performance on agentic tool use, code generation, and reasoning tasks.
It uses the **standard `LlamaForCausalLM` architecture**, so vLLM loads it
natively with no custom kernels or model-code fork.

Its headline feature is **hybrid reasoning**: a single checkpoint serves as
both a fast assistant (No-Think) and a deliberate reasoner (Think), toggled by
the chat template's `enable_thinking` flag.

### Prerequisites

- **vLLM ≥ 0.21.0** — MiniCPM5-2B is supported natively as of the v0.21.0
  release.

### Launch command

Use the command builder above. The baseline is simply:

```bash
vllm serve openbmb/MiniCPM5-2B --port 8000
```

At ~2B params the model fits on a single GPU (TP=1). It supports the full
native 128K context; drop `--max-model-len` to `8192` / `32768` to free KV
cache on small GPUs.

### Reasoning modes

Toggle the **Reasoning** feature to serve with deep-thinking on by default
(`--default-chat-template-kwargs '{"enable_thinking": true}'`). You can also
flip it per request via `chat_template_kwargs`:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openbmb/MiniCPM5-2B",
    "messages": [{"role": "user", "content": "Explain GQA in one sentence."}],
    "temperature": 0.9, "top_p": 0.95, "max_tokens": 1024,
    "chat_template_kwargs": {"enable_thinking": true}
  }'
```

| Mode | `enable_thinking` | `temperature` | `top_p` |
| --- | --- | --- | --- |
| Think | `true` | 0.9 | 0.95 |
| No-Think | `false` | 1.0 | 0.95 |

### Tool calling

MiniCPM5-2B emits **XML-style** tool calls. The `minicpm5` parser
([PR #43175](https://github.com/vllm-project/vllm/pull/43175)) is natively
supported in vLLM. Enable it with:

```bash
vllm serve openbmb/MiniCPM5-2B --port 8000 \
  --enable-auto-tool-choice \
  --tool-call-parser minicpm5
```

### References

- [Model card](https://huggingface.co/openbmb/MiniCPM5-2B)
- [vLLM deployment cookbook](https://github.com/OpenBMB/MiniCPM/blob/main/docs/deployment/vllm.md)
- [MiniCPM Tech Report](https://arxiv.org/pdf/2506.07900)