# deepseek-ai/DeepSeek-V3.2 — vLLM serving recipe

> DeepSeek V3.2 MoE model with MLA attention, sparse attention, and scalable RL for strong reasoning and agent capabilities.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (FP8) · features `tool_calling`, `reasoning` · frontend `rust`

- Interactive page: https://recipes.vllm.ai/deepseek-ai/DeepSeek-V3.2
- This configuration as JSON: https://recipes.vllm.ai/deepseek-ai/DeepSeek-V3.2.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `nvfp4`, `redhat_nvfp4`
- `strategy`: `single_node_tp`, `single_node_tep`, `single_node_dep`, `multi_node_tp`, `multi_node_tp_pp`, `multi_node_dep`, `multi_node_tep`, `pd_cluster` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `spec_decoding` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.18.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

Extra dependencies:

- DeepGEMM pinned build for MQA logits (FP8 MoE kernels)

```bash
uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation
```

- Set VLLM_USE_DEEP_GEMM=0 to skip DeepGEMM for the MoE path (recommended on H20) (optional)

```bash
export VLLM_USE_DEEP_GEMM=0
```

## Serve

```bash
export VLLM_USE_RUST_FRONTEND=1
vllm serve deepseek-ai/DeepSeek-V3.2 \
  --trust-remote-code \
  --disable-flashinfer-autotune \
  --tensor-parallel-size 8 \
  --tokenizer-mode deepseek_v32 \
  --tool-call-parser deepseek_v32 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v3
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_USE_RUST_FRONTEND=1 \
  vllm/vllm-openai:latest deepseek-ai/DeepSeek-V3.2 \
  --trust-remote-code \
  --disable-flashinfer-autotune \
  --tensor-parallel-size 8 \
  --tokenizer-mode deepseek_v32 \
  --tool-call-parser deepseek_v32 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v3
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

DeepSeek-V3.2 is a Mixture-of-Experts model that balances computational efficiency with
strong reasoning and agent capabilities through three technical innovations: DeepSeek Sparse
Attention (DSA) for efficient long-context processing, a scalable reinforcement learning
framework achieving GPT-5-level performance, and a large-scale agentic task synthesis pipeline
for robust tool-use generalization.

### Prerequisites

- **Hardware**: Minimum 8x H100/H200 80GB GPUs (BF16) or 3x H200 (NVFP4 variant).
- **vLLM**: Version 0.18.0 or later (nightly recommended).
- **Python**: 3.10+
- **CUDA**: 12.x or later (CUDA 13.x may require extra env vars; see Troubleshooting).
- **Disk**: ~1.3 TB for BF16 weights; ~350 GB for NVFP4 variant.
- **DeepGEMM** (recommended):
  ```bash
  uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation
  ```
  Note: Set `VLLM_USE_DEEP_GEMM=0` to disable MoE DeepGEMM if you experience issues
  (e.g., on H20 GPUs) or want to skip the long warmup.

### Client Usage

Launch the server:
```bash
vllm serve deepseek-ai/DeepSeek-V3.2 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --disable-flashinfer-autotune \
  --tokenizer-mode deepseek_v32 \
  --tool-call-parser deepseek_v32 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v3
```

Use the OpenAI Python SDK to interact with the server:
```python
from openai import OpenAI

client = OpenAI(
    api_key="your-api-key",
    base_url="http://localhost:8000/v1",
)

# Standard chat
response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3.2",
    messages=[{"role": "user", "content": "Hello!"}],
)

# Thinking / reasoning mode
response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3.2",
    messages=[{"role": "user", "content": "Solve this step by step..."}],
    extra_body={"chat_template_kwargs": {"thinking": True}},
)
```

### Troubleshooting

**`ptxas fatal: Value 'sm_110a' is not defined for option 'gpu-name'`**
This can occur on CUDA 13.x. Fix by exporting:
```bash
export TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}
```

**TP=8 performance on Hopper/Blackwell**
Avoid `-tp 8` with FlashMLA-Sparse. Due to kernel restrictions, TP=8 yields only 16 heads
per rank but is padded to 64, causing overhead. Prefer TP=2 (Hopper) or TP=1 (Blackwell)
with DP/EP mode: `vllm serve deepseek-ai/DeepSeek-V3.2 -dp 8 --enable-expert-parallel`.

**DeepGEMM warmup too slow**
Set `VLLM_USE_DEEP_GEMM=0` to disable MoE DeepGEMM and skip the long warmup.

### References

- [DeepSeek-V3.2 on Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-V3.2)
- [DeepSeek API Documentation](https://api-docs.deepseek.com/)
- [vLLM Data Parallel Deployment Guide](https://docs.vllm.ai/en/latest/serving/data_parallel_deployment.html)