vLLM/Recipes
Search
Qwen

Qwen/Qwen3-VL-4B-Instruct

Qwen3-VL dense 4B vision-language model supporting image, video, and text workloads.

Intel Xeon 6 CPU BF16 serving

View on HuggingFace
dense4B262,144 ctxvLLM 0.11.0+multimodaltext
Default configuration: 4× Xeon 6 · Tensor Parallel · BF16
vllm serve Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3-VL-4B-Instruct.md.

Guide

Overview

Qwen3-VL-4B-Instruct is a dense multimodal Qwen3-VL model for image, video, and text workloads. This recipe targets single-node Intel Xeon 6 BF16 serving.

Prerequisites

  • Hardware: Intel Xeon 6 CPUs
  • A vLLM CPU build with Qwen3-VL support (including the model-support changes from the new_qwen3-vl-model branch when not yet available in a release)

For x86 CPU installation, follow the CPU installation guide.

Intel Xeon 6

vllm serve Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data

Docker

Use a CPU image containing the required Qwen3-VL implementation.

docker run \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-4B-Instruct \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data

The Docker command assumes that the specified CPU image includes the required model support. If not, build the image from the support branch. Start with TP=1; tune TP and NUMA placement only after testing on Xeon 6.

Runtime Tuning

Scheduler, concurrency, memory, and CPU affinity settings depend on the Xeon 6 topology and workload and are intentionally not hard-coded.

References