vLLM/Recipes
Qwen

Qwen/QwQ-32B

Qwen's 32B dense reasoning model, validated for Intel Xeon 6 CPU execution.

Validated for Intel Xeon 6 CPU execution

View on HuggingFace
dense32B40,960 ctxvLLM 0.7.3+text
Guide

Overview

QwQ-32B is Qwen's 32B dense reasoning model. The CPU model validation table marks it supported on Intel Xeon 6.

Prerequisites

  • Hardware: Intel Xeon 6 CPUs
  • vLLM CPU >= 0.8.5 for this validated Xeon 6 configuration

pip (Intel Xeon 6 CPUs)

For Intel and AMD x86 CPUs, follow the CPU pre-built wheels installation instructions, or use:

docker pull vllm/vllm-openai-cpu:latest-x86_64

Intel Xeon 6

vllm serve Qwen/QwQ-32B

W8A8 INT8:

vllm serve RedHatAI/QwQ-32B-quantized.w8a8

Choose tensor parallelism based on system topology; the recipe does not prescribe a fixed TP/DP layout or topology-specific CPU binding.

For reasoning-content extraction and automatic tool choice:

vllm serve Qwen/QwQ-32B \
  --reasoning-parser deepseek_r1 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Runtime and Platform Tuning

The following settings are intentionally not prescribed as portable CPU model defaults because they depend on the workload, hardware, or runtime environment:

  • --max-num-batched-tokens: scheduler/throughput tuning.
  • --max-num-seqs: concurrency and scheduler-capacity tuning.
  • --gpu-memory-utilization: platform memory-budget tuning.
  • --no-enable-prefix-caching: workload/benchmark cache-policy tuning.
  • VLLM_ENGINE_ITERATION_TIMEOUT_S: operational runtime timeout.

These settings may still appear in validated hardware-specific overrides. Tune them at deployment time based on platform resources, workload shape, and latency/throughput goals.

References