# microsoft/Phi-4-mini-instruct — vLLM serving recipe

> Microsoft's Phi-4 family of lightweight dense models (mini-instruct, reasoning, multimodal) with 128K context

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://recipes.vllm.ai/microsoft/Phi-4-mini-instruct
- This configuration as JSON: https://recipes.vllm.ai/microsoft/Phi-4-mini-instruct.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `mini_reasoning`, `reasoning`, `multimodal`, `fp8`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `text_only`, `encoder_parallel` (default: none)

## Install

Requires vLLM >= 0.7.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve microsoft/Phi-4-mini-instruct \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest microsoft/Phi-4-mini-instruct \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

The Phi-4 family includes several lightweight, open models from Microsoft. These models
can process text and, in some variants, multimodal inputs like images, to generate
text outputs. They come with a 128K token context length.

### Prerequisites

- Hardware: 1x GPU with >=16 GB VRAM (A100, L40S, H100, etc.)
- vLLM >= 0.7.0 (the `multimodal` variant serves `Phi-4-multimodal-instruct`, which needs vLLM >= 0.8.0)

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

### Launch commands

Phi-4-mini-instruct on a single GPU:

```bash
vllm serve microsoft/Phi-4-mini-instruct \
  --host 0.0.0.0 \
  --max-model-len 4000
```

Phi-4-multimodal-instruct (requires --trust-remote-code for LoRA modules):

```bash
vllm serve microsoft/Phi-4-multimodal-instruct \
  --host 0.0.0.0 \
  --max-model-len 4000 \
  --trust-remote-code
```

### Client Usage

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=3600)

response = client.chat.completions.create(
    model="microsoft/Phi-4-mini-instruct",
    messages=[{"role": "user", "content": "Write a short story"}],
    temperature=0.0,
)
print(response.choices[0].message.content)
```

Multimodal (requires Phi-4-multimodal-instruct):

```python
response = client.chat.completions.create(
    model="microsoft/Phi-4-multimodal-instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}},
        ],
    }],
)
```

### Available Phi-4 Variants

- `microsoft/Phi-4-mini-instruct` — conversational instruction-tuned
- `microsoft/Phi-4-mini-reasoning` — optimized for reasoning
- `microsoft/Phi-4-reasoning` — advanced reasoning
- `microsoft/Phi-4-multimodal-instruct` — multimodal (text + image)

### Benchmarking

```bash
vllm bench serve \
  --model microsoft/Phi-4-mini-instruct \
  --dataset-name random \
  --random-input-len 2000 \
  --random-output-len 512 \
  --num-prompts 100
```

### Troubleshooting

- Multimodal variant fails to load: add `--trust-remote-code`.

### References

- [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct)
- [Phi-4-multimodal-instruct](https://huggingface.co/microsoft/Phi-4-multimodal-instruct)
- [Phi-4-reasoning](https://huggingface.co/microsoft/Phi-4-reasoning)