{
  "hf_id": "Qwen/Qwen3-VL-4B-Instruct",
  "meta": {
    "title": "Qwen3-VL-4B-Instruct",
    "slug": "qwen3-vl-4b-instruct",
    "provider": "Qwen",
    "description": "Qwen3-VL dense 4B vision-language model supporting image, video, and text workloads.",
    "date_added": "2026-11-06",
    "date_updated": "2026-11-06",
    "tasks": [
      "multimodal",
      "text"
    ],
    "performance_headline": "Intel Xeon 6 CPU BF16 serving",
    "related_recipes": [
      "Qwen/Qwen3-VL-30B-A3B-Instruct"
    ],
    "hardware": {
      "xeon6": "verified"
    }
  },
  "recommended_command": {
    "hardware": "xeon6",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai-cpu:latest-x86_64",
    "command": "vllm serve Qwen/Qwen3-VL-4B-Instruct \\\n  --tensor-parallel-size 1 \\\n  --mm-encoder-tp-mode data",
    "argv": [
      "vllm",
      "serve",
      "Qwen/Qwen3-VL-4B-Instruct",
      "--tensor-parallel-size",
      "1",
      "--mm-encoder-tp-mode",
      "data"
    ],
    "docker_command": "docker run --shm-size=16g \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-4B-Instruct \\\n  --tensor-parallel-size 1 \\\n  --mm-encoder-tp-mode data",
    "docker_argv": [
      "docker",
      "run",
      "--shm-size",
      "16g",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai-cpu:latest-x86_64",
      "Qwen/Qwen3-VL-4B-Instruct",
      "--tensor-parallel-size",
      "1",
      "--mm-encoder-tp-mode",
      "data"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "Intel",
      "generation": "cpu",
      "display_name": "Xeon 6",
      "description": "Intel Xeon 6 CPU NUMA node · x86 CPU backend",
      "gpu_count": 4,
      "vram_gb": 1024,
      "multi_node": false,
      "restricted": true
    },
    "by_hardware": {
      "xeon6": "/Qwen/Qwen3-VL-4B-Instruct/hw/xeon6.json"
    }
  },
  "model": {
    "model_id": "Qwen/Qwen3-VL-4B-Instruct",
    "min_vllm_version": "0.11.0",
    "architecture": "dense",
    "parameter_count": "4B",
    "active_parameters": "4B",
    "context_length": 262144,
    "base_args": [],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      },
      "extras": [
        {
          "note": "Recommended for offline multimodal inference (image/video pre-processing helpers)",
          "command": "uv pip install qwen-vl-utils==0.0.14",
          "optional": true
        }
      ]
    }
  },
  "features": {
    "text_only": {
      "description": "Skip loading the vision encoder for text-only workloads to free KV-cache memory.",
      "args": [
        "--language-model-only"
      ]
    },
    "encoder_parallel": {
      "description": "Run the vision encoder in data-parallel mode.",
      "args": [
        "--mm-encoder-tp-mode",
        "data"
      ]
    }
  },
  "opt_in_features": [
    "text_only"
  ],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 12,
      "description": "BF16 inference on Intel Xeon 6",
      "supported_hardware": [
        "xeon6"
      ]
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\nQwen3-VL-4B-Instruct is a dense multimodal Qwen3-VL model for image, video,\nand text workloads. This recipe targets single-node Intel Xeon 6 BF16 serving.\n\n## Prerequisites\n\n- Hardware: Intel Xeon 6 CPUs\n- A vLLM CPU build with Qwen3-VL support (including the model-support changes\n  from the `new_qwen3-vl-model` branch when not yet available in a release)\n\nFor x86 CPU installation, follow the\n[CPU installation guide](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/).\n\n## Intel Xeon 6\n\n```bash\nvllm serve Qwen/Qwen3-VL-4B-Instruct \\\n  --tensor-parallel-size 1 \\\n  --mm-encoder-tp-mode data\n```\n\n## Docker\n\nUse a CPU image containing the required Qwen3-VL implementation.\n\n```bash\ndocker run \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-VL-4B-Instruct \\\n  --tensor-parallel-size 1 \\\n  --mm-encoder-tp-mode data\n```\n\nThe Docker command assumes that the specified CPU image includes the\nrequired model support. If not, build the image from the support branch.\nStart with TP=1; tune TP and NUMA placement only after testing on Xeon 6.\n\n## Runtime Tuning\n\nScheduler, concurrency, memory, and CPU affinity settings depend on the\nXeon 6 topology and workload and are intentionally not hard-coded.\n\n## References\n\n- [Model card](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct)\n- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "b936e1427f8c36572cc74a0a168eb30f4df0079c",
    "path": "recipes/models/Qwen/Qwen3-VL-4B-Instruct.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/Qwen/Qwen3-VL-4B-Instruct.yaml",
    "overlays": []
  }
}