Qwen/Qwen3-VL-30B-A3B-Instruct

Qwen3-VL MoE vision-language model with 30B total / 3B active parameters, supporting image, video, and text workloads.

Validated on Intel Xeon 6 CPU with BF16 serving

View on HuggingFace

moe30B / 3B262,144 ctxvLLM 0.11.0+multimodaltext

Guide

Overview

Qwen3-VL-30B-A3B-Instruct is a multimodal Qwen3-VL MoE model. Intel Xeon 6 CPU serving was validated with BF16.

Prerequisites

Hardware: Intel Xeon 6 CPUs
vLLM >= 0.11.0

pip (Intel Xeon 6 CPUs)

For Intel and AMD x86 CPUs, follow the CPU pre-built wheels installation instructions, or use:

docker pull vllm/vllm-openai-cpu:latest-x86_64

Intel Xeon 6

vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct

Choose tensor parallelism based on system topology; TP/DP and NUMA binding are intentionally not hard-coded in the recipe.

Configuration Notes

TP/DP values and topology-specific CPU binding remain deployment choices.

Runtime and Platform Tuning

The following settings are intentionally not prescribed as portable CPU model defaults because they depend on the workload, hardware, or runtime environment:

--max-num-batched-tokens: scheduler/throughput tuning.
--max-num-seqs: concurrency and scheduler-capacity tuning.
--gpu-memory-utilization: platform memory-budget tuning.
--no-enable-prefix-caching: workload/benchmark cache-policy tuning.
VLLM_ENGINE_ITERATION_TIMEOUT_S: operational runtime timeout.

These settings may still appear in validated hardware-specific overrides. Tune them at deployment time based on platform resources, workload shape, and latency/throughput goals.