{ "hf_id": "microsoft/Phi-4-mini-instruct", "meta": { "title": "Phi-4", "slug": "phi-4", "provider": "Microsoft", "description": "Microsoft's Phi-4 family of lightweight dense models (mini-instruct, reasoning, multimodal) with 128K context", "date_added": "2026-04-02", "date_updated": "2026-04-17", "tasks": [ "text", "multimodal" ], "related_recipes": [], "hardware": { "h100": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve microsoft/Phi-4-mini-instruct \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "microsoft/Phi-4-mini-instruct", "--tensor-parallel-size", "1" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest microsoft/Phi-4-mini-instruct \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "microsoft/Phi-4-mini-instruct", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/microsoft/Phi-4-mini-instruct/strategies/multi_node_tp.json", "kv_store_centralized_mooncake": "/microsoft/Phi-4-mini-instruct/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/microsoft/Phi-4-mini-instruct/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/microsoft/Phi-4-mini-instruct/hw/h200.json", "h100": "/microsoft/Phi-4-mini-instruct/hw/h100.json", "b200": "/microsoft/Phi-4-mini-instruct/hw/b200.json", "gb200": "/microsoft/Phi-4-mini-instruct/hw/gb200.json", "b300": "/microsoft/Phi-4-mini-instruct/hw/b300.json", "gb300": "/microsoft/Phi-4-mini-instruct/hw/gb300.json", "mi300x": "/microsoft/Phi-4-mini-instruct/hw/mi300x.json", "mi325x": "/microsoft/Phi-4-mini-instruct/hw/mi325x.json", "mi350x": "/microsoft/Phi-4-mini-instruct/hw/mi350x.json", "mi355x": "/microsoft/Phi-4-mini-instruct/hw/mi355x.json" } }, "model": { "model_id": "microsoft/Phi-4-mini-instruct", "min_vllm_version": "0.7.0", "architecture": "dense", "parameter_count": "4B", "active_parameters": "4B", "context_length": 131072, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "text_only": { "description": "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel.", "args": [ "--language-model-only" ] }, "encoder_parallel": { "description": "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only.", "args": [ "--mm-encoder-tp-mode", "data" ] } }, "opt_in_features": [ "text_only", "encoder_parallel" ], "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 10, "description": "Phi-4-mini-instruct, conversational instruction-tuned" }, "mini_reasoning": { "model_id": "microsoft/Phi-4-mini-reasoning", "precision": "bf16", "vram_minimum_gb": 10, "description": "Optimized for reasoning tasks", "json": "/microsoft/Phi-4-mini-reasoning.json" }, "reasoning": { "model_id": "microsoft/Phi-4-reasoning", "precision": "bf16", "vram_minimum_gb": 30, "description": "Advanced reasoning capabilities (14B)" }, "multimodal": { "model_id": "microsoft/Phi-4-multimodal-instruct", "min_vllm_version": "0.8.0", "precision": "bf16", "vram_minimum_gb": 16, "description": "Multimodal instruction-following (text + image)", "extra_args": [ "--trust-remote-code" ] }, "fp8": { "model_id": "RedHatAI/Phi-4-mini-instruct-FP8-dynamic", "precision": "fp8", "vram_minimum_gb": 5, "description": "RedHatAI FP8 (E4M3) with dynamic per-token activation quantization", "extra_args": [ "--kv-cache-dtype", "fp8" ], "json": "/RedHatAI/Phi-4-mini-instruct-FP8-dynamic.json" } }, "hardware_overrides": {}, "guide": "## Overview\n\nThe Phi-4 family includes several lightweight, open models from Microsoft. These models\ncan process text and, in some variants, multimodal inputs like images, to generate\ntext outputs. They come with a 128K token context length.\n\n## Prerequisites\n\n- Hardware: 1x GPU with >=16 GB VRAM (A100, L40S, H100, etc.)\n- vLLM >= 0.7.0 (the `multimodal` variant serves `Phi-4-multimodal-instruct`, which needs vLLM >= 0.8.0)\n\n### Install vLLM\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto\n```\n\n## Launch commands\n\nPhi-4-mini-instruct on a single GPU:\n\n```bash\nvllm serve microsoft/Phi-4-mini-instruct \\\n --host 0.0.0.0 \\\n --max-model-len 4000\n```\n\nPhi-4-multimodal-instruct (requires --trust-remote-code for LoRA modules):\n\n```bash\nvllm serve microsoft/Phi-4-multimodal-instruct \\\n --host 0.0.0.0 \\\n --max-model-len 4000 \\\n --trust-remote-code\n```\n\n## Client Usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\", timeout=3600)\n\nresponse = client.chat.completions.create(\n model=\"microsoft/Phi-4-mini-instruct\",\n messages=[{\"role\": \"user\", \"content\": \"Write a short story\"}],\n temperature=0.0,\n)\nprint(response.choices[0].message.content)\n```\n\nMultimodal (requires Phi-4-multimodal-instruct):\n\n```python\nresponse = client.chat.completions.create(\n model=\"microsoft/Phi-4-multimodal-instruct\",\n messages=[{\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"Describe this image.\"},\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/image.jpg\"}},\n ],\n }],\n)\n```\n\n## Available Phi-4 Variants\n\n- `microsoft/Phi-4-mini-instruct` — conversational instruction-tuned\n- `microsoft/Phi-4-mini-reasoning` — optimized for reasoning\n- `microsoft/Phi-4-reasoning` — advanced reasoning\n- `microsoft/Phi-4-multimodal-instruct` — multimodal (text + image)\n\n## Benchmarking\n\n```bash\nvllm bench serve \\\n --model microsoft/Phi-4-mini-instruct \\\n --dataset-name random \\\n --random-input-len 2000 \\\n --random-output-len 512 \\\n --num-prompts 100\n```\n\n## Troubleshooting\n\n- Multimodal variant fails to load: add `--trust-remote-code`.\n\n## References\n\n- [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct)\n- [Phi-4-multimodal-instruct](https://huggingface.co/microsoft/Phi-4-multimodal-instruct)\n- [Phi-4-reasoning](https://huggingface.co/microsoft/Phi-4-reasoning)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb", "path": "recipes/models/microsoft/Phi-4-mini-instruct.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/microsoft/Phi-4-mini-instruct.yaml", "overlays": [] } }