Qwen/Qwen3-14B

Overview

Qwen3-14B is a Qwen3 dense model with hybrid thinking and non-thinking modes. Intel Xeon 6 CPU serving was validated with BF16.

Prerequisites

• Hardware: Intel Xeon 6 CPUs
• vLLM >= 0.8.5

pip (Intel Xeon 6 CPUs)

Follow the CPU pre-built wheels installation instructions.

Docker (Intel Xeon 6 CPUs)

docker pull vllm/vllm-openai-cpu:latest-x86_64

Intel Xeon 6

Choose tensor parallelism based on the system topology; the recipe does not prescribe a fixed TP/DP layout or hard-code NUMA node IDs.

vllm serve Qwen/Qwen3-14B

Docker:

docker run -itd --name qwen3-14b-cpu \
  --network host \
  --shm-size 16g \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 \
    --model Qwen/Qwen3-14B \
    --host 0.0.0.0 \
    --port 8000

Configuration Notes

• TP/DP values and explicit CPU bindings are intentionally not prescribed.
• The qwen3 reasoning parser is available in vLLM 0.9.0 and later.

Runtime and Platform Tuning

The following settings are intentionally not prescribed as portable CPU model defaults because they depend on the workload, hardware, or runtime environment:

• --max-num-batched-tokens: scheduler/throughput tuning.
• --max-num-seqs: concurrency and scheduler-capacity tuning.
• --gpu-memory-utilization: platform memory-budget tuning.
• --no-enable-prefix-caching: workload/benchmark cache-policy tuning.
• VLLM_ENGINE_ITERATION_TIMEOUT_S: operational runtime timeout.

These settings may still appear in validated hardware-specific overrides. Tune them at deployment time based on platform resources, workload shape, and latency/throughput goals.

References

• Model card
• vLLM CPU installation