# openbmb/MiniCPM-OCR-1.0 — vLLM serving recipe

> MiniCPM-OCR 1.0 (1.3B) — OCR-specific document-parsing VLM that turns page images and region crops into Markdown text, tables, formulas, charts, and seals

Configuration: hardware `rtx_4090` (1× RTX 4090) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `spec_decoding`

- Interactive page: https://recipes.vllm.ai/openbmb/MiniCPM-OCR-1.0
- This configuration as JSON: https://recipes.vllm.ai/openbmb/MiniCPM-OCR-1.0.json?hardware=rtx_4090

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `rtx_4090`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `awq`, `gptq`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `spec_decoding` (default: `spec_decoding`). Use `features=none` to turn them all off.

## Install

Requires vLLM >= 0.34.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:v0.34.0`

Extra dependencies:

- The minicpm_ocr processors ship in transformers >= 5.21; vLLM 0.34.0 accepts < 5.23

```bash
uv pip install -U "transformers>=5.21.0,<5.23.0"
```

## Serve

```bash
vllm serve openbmb/MiniCPM-OCR-1.0 \
  --trust-request-mm-kwargs \
  --limit-mm-per-prompt '{"image":36,"video":0}' \
  --max-model-len 40960 \
  --max-num-batched-tokens 40960 \
  --tensor-parallel-size 1 \
  --speculative-config '{"method":"dspark","model":"openbmb/MiniCPM-OCR-1.0-DSpark","num_speculative_tokens":15,"enable_adaptive_verification":false}'
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:v0.34.0 openbmb/MiniCPM-OCR-1.0 \
  --trust-request-mm-kwargs \
  --limit-mm-per-prompt '{"image":36,"video":0}' \
  --max-model-len 40960 \
  --max-num-batched-tokens 40960 \
  --tensor-parallel-size 1 \
  --speculative-config '{"method":"dspark","model":"openbmb/MiniCPM-OCR-1.0-DSpark","num_speculative_tokens":15,"enable_adaptive_verification":false}'
```

## Known issues

### ValueError: The checkpoint you are trying to load has model type `minicpm_ocr` but Transformers does not recognize this architecture

transformers < 5.21 predates the `minicpm_ocr` config and processors.
Install transformers 5.21–5.22 (`uv pip install -U "transformers>=5.21.0,<5.23.0"`);
the recipe's dependency step does this.

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

**MiniCPM-OCR 1.0** is a 1.3B OCR-specific model in the MiniCPM-V family. It
pairs a SigLIP-style vision encoder with the **Qwen3.5-0.8B** hybrid
linear/full-attention backbone from MiniCPM-V 4.6, and is trained for
text-centric document parsing: text blocks, tables, formulas, charts, seals,
and key-information extraction. It scores **93.68 on Real5-OmniDocBench** and
**54.7 on OCRBench v2**.

The checkpoint is the recognition stage. Send a page image or a cropped
region together with a region prompt; for PDFs, the companion `minicpm-ocr`
SDK detects layout with PP-DocLayoutV3, crops regions, calls this service,
and assembles the results into Markdown/JSON.

This recipe was run end to end on **one RTX 4090 (24 GB)** with vLLM 0.34.0
and transformers 5.22, on the BF16 checkpoint and both official 4-bit
checkpoints, with and without the DSpark draft model. With DSpark on the
4090, test requests accepted ~60% of drafted tokens (348/570), and
multi-token requests completed 20–45% faster than plain decoding.

### Prerequisites

- **vLLM ≥ 0.34.0** — v0.33.0 registered `MiniCPMOcrForConditionalGeneration`
  but pins transformers < 5.21, before the `minicpm_ocr` processors shipped.
  0.34.0 includes the follow-up downsample fix ([PR #64105](https://github.com/vllm-project/vllm/pull/64105)).
- **transformers ≥ 5.21, < 5.23** — the range the model's processors need
  within vLLM 0.34.0's pin.
- One GPU is enough: BF16 weights are ~2.6 GB, and the DSpark draft model
  adds ~0.6 GB. Use `temperature=0` for OCR.
- **Context** — the default command keeps the model guide's 40,960-token
  window, sized for page crops; the checkpoint supports up to 262,144 tokens.

### Client usage

The prompt selects what to read in the image: `Text Recognition:` for text
blocks and pages, `Table Recognition:` for tables, `Formula Recognition:` for
formulas, `Seal Recognition:` for seals, and `Chart to Table:` for charts.
Key-information extraction takes a field list instead. Tables and charts come
back as OTSL; the SDK converts them to HTML.

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=600)

response = client.chat.completions.create(
    model="openbmb/MiniCPM-OCR-1.0",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://.../page.png"}},
            {"type": "text", "text": "Text Recognition:"},
        ],
    }],
    temperature=0.0,
    max_tokens=2048,
)
print(response.choices[0].message.content)
```

#### Full documents with the SDK

The official SDK handles layout detection, region cropping, and result
assembly around this service. Install it in a separate environment
(`pip install "minicpm-ocr[all]"`) and point it at the server. Pass the
served model name (`--model`) to match the server; the SDK defaults to
`minicpm-ocr`.

The server keeps `--trust-request-mm-kwargs` so the SDK can set
per-request image-processing options such as `downsample_mode`. Keep it only
for trusted clients: without it, vLLM rejects per-request image options.

### References

- [Model card](https://huggingface.co/openbmb/MiniCPM-OCR-1.0)
- [DSpark draft model](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark)
- [GitHub: OpenBMB/MiniCPM-OCR](https://github.com/OpenBMB/MiniCPM-OCR)
- [Deployment guide](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/minicpm_ocr_tutorial_en.md)
- [vLLM support PR #63095](https://github.com/vllm-project/vllm/pull/63095)
