{
  "hf_id": "microsoft/Phi-4-multimodal-instruct",
  "meta": {
    "title": "Phi-4-multimodal-instruct",
    "slug": "phi-4-multimodal-instruct",
    "provider": "Microsoft",
    "description": "Microsoft Phi-4 multimodal model supporting text, image, and audio inputs with text outputs.",
    "date_added": "2026-08-19",
    "date_updated": "2026-09-12",
    "tasks": [
      "multimodal"
    ],
    "performance_headline": "Validated on Intel Xeon 6",
    "hardware": {
      "xeon6": "verified"
    }
  },
  "recommended_command": {
    "hardware": "xeon6",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai-cpu:latest-x86_64",
    "command": "vllm serve microsoft/Phi-4-multimodal-instruct \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1",
    "argv": [
      "vllm",
      "serve",
      "microsoft/Phi-4-multimodal-instruct",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1"
    ],
    "docker_command": "docker run --shm-size=16g \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai-cpu:latest-x86_64 microsoft/Phi-4-multimodal-instruct \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1",
    "docker_argv": [
      "docker",
      "run",
      "--shm-size",
      "16g",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai-cpu:latest-x86_64",
      "microsoft/Phi-4-multimodal-instruct",
      "--trust-remote-code",
      "--tensor-parallel-size",
      "1"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "Intel",
      "generation": "cpu",
      "display_name": "Xeon 6",
      "description": "Intel Xeon 6 CPU NUMA node · x86 CPU backend",
      "gpu_count": 4,
      "vram_gb": 1024,
      "multi_node": false,
      "restricted": true
    },
    "alternatives": {
      "multi_node_tp": "/microsoft/Phi-4-multimodal-instruct/strategies/multi_node_tp.json"
    },
    "by_hardware": {
      "xeon6": "/microsoft/Phi-4-multimodal-instruct/hw/xeon6.json"
    }
  },
  "model": {
    "model_id": "microsoft/Phi-4-multimodal-instruct",
    "min_vllm_version": "0.8.0",
    "architecture": "dense",
    "parameter_count": "5.6B",
    "active_parameters": "5.6B",
    "context_length": 131072,
    "base_args": [
      "--trust-remote-code"
    ],
    "base_env": {},
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto"
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {},
  "opt_in_features": [],
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 14,
      "description": "Full precision BF16",
      "supported_hardware": [
        "xeon6"
      ]
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\nPhi-4-multimodal-instruct processes text, image, and audio inputs and\ngenerates text. `--trust-remote-code` is retained because the model's\nmultimodal LoRA components require it.\n\n## Prerequisites\n\n- Hardware: Intel Xeon 6/Xeon 5 CPUs\n- vLLM >= 0.8.0\n\n### Docker (Intel Xeon 6 CPUs)\n\n```bash\ndocker pull vllm/vllm-openai-cpu:latest-x86_64\n```\n\n## Intel Xeon 6\n\n```bash\nvllm serve microsoft/Phi-4-multimodal-instruct \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1\n```\n\nDocker (the image entrypoint is `vllm serve`):\n\n```bash\ndocker run \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai-cpu:latest-x86_64 microsoft/Phi-4-multimodal-instruct \\\n  --trust-remote-code \\\n  --tensor-parallel-size 1\n```\n\nThe example uses TP=1. Adjust TP/DP for the deployment topology;\nhost-specific CPU placement is intentionally not hard-coded.\n\n## Runtime and Platform Tuning\n\nThe following settings are intentionally not prescribed as portable CPU model\ndefaults because they depend on the workload, hardware, or runtime environment:\n\n- `--max-num-batched-tokens`: scheduler/throughput tuning.\n- `--max-num-seqs`: concurrency and scheduler-capacity tuning.\n- `--gpu-memory-utilization`: platform memory-budget tuning.\n- `--no-enable-prefix-caching`: workload/benchmark cache-policy tuning.\n- `VLLM_ENGINE_ITERATION_TIMEOUT_S`: operational runtime timeout.\n\nThese settings may still appear in validated hardware-specific overrides. Tune\nthem at deployment time based on platform resources, workload shape, and\nlatency/throughput goals.\n\n## References\n\n- [Model card](https://huggingface.co/microsoft/Phi-4-multimodal-instruct)\n- [vLLM Phi-4 usage guide](https://docs.vllm.ai/projects/recipes/en/stable/Microsoft/Phi-4.html)\n- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb",
    "path": "recipes/models/microsoft/Phi-4-multimodal-instruct.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/microsoft/Phi-4-multimodal-instruct.yaml",
    "overlays": []
  }
}