# Qwen/Qwen3-VL-4B-Instruct — vLLM serving recipe

> Qwen3-VL dense 4B vision-language model supporting image, video, and text workloads.

Configuration: hardware `xeon6` (4× Xeon 6) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `encoder_parallel`

- Interactive page: https://recipes.vllm.ai/Qwen/Qwen3-VL-4B-Instruct
- This configuration as JSON: https://recipes.vllm.ai/Qwen/Qwen3-VL-4B-Instruct.json?hardware=xeon6

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `xeon6`
- `variant`: `default`
- `strategy`: `single_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `text_only`, `encoder_parallel` (default: `encoder_parallel`). Use `features=none` to turn them all off.

## Install

Requires vLLM >= 0.11.0.

Docker image: `vllm/vllm-openai-cpu:latest-x86_64`

Extra dependencies:

- Recommended for offline multimodal inference (image/video pre-processing helpers) (optional)

```bash
uv pip install qwen-vl-utils==0.0.14
```

## Serve

```bash
vllm serve Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data
```

Docker equivalent:

```bash
docker run --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Qwen3-VL-4B-Instruct is a dense multimodal Qwen3-VL model for image, video,
and text workloads. This recipe targets single-node Intel Xeon 6 BF16 serving.

### Prerequisites

- Hardware: Intel Xeon 6 CPUs
- A vLLM CPU build with Qwen3-VL support (including the model-support changes
  from the `new_qwen3-vl-model` branch when not yet available in a release)

For x86 CPU installation, follow the
[CPU installation guide](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/).

### Intel Xeon 6

```bash
vllm serve Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data
```

### Docker

Use a CPU image containing the required Qwen3-VL implementation.

```bash
docker run \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data
```

The Docker command assumes that the specified CPU image includes the
required model support. If not, build the image from the support branch.
Start with TP=1; tune TP and NUMA placement only after testing on Xeon 6.

### Runtime Tuning

Scheduler, concurrency, memory, and CPU affinity settings depend on the
Xeon 6 topology and workload and are intentionally not hard-coded.

### References

- [Model card](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct)
- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)