{ "hf_id": "openai/gpt-oss-20b", "meta": { "title": "GPT-OSS 20B", "slug": "gpt-oss-20b", "provider": "OpenAI", "description": "OpenAI's gpt-oss-20b — 21B-total / 3.6B-active MoE reasoning model with native MXFP4 quant; supports GPU, XPU, and Xeon 6 CPU serving", "date_added": "2025-08-05", "date_updated": "2026-09-12", "tasks": [ "text" ], "performance_headline": "21B/3.6B-A MoE reasoning model with native MXFP4 — runs on 16GB", "related_recipes": [ "openai/gpt-oss-120b" ], "hardware": { "h100": "verified", "h200": "verified", "b200": "verified", "mi300x": "verified", "mi325x": "verified", "mi355x": "verified", "arc_pro_b60": "verified", "arc_pro_b70": "verified", "xeon6": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve openai/gpt-oss-20b \\\n --tensor-parallel-size 1 \\\n --tool-call-parser openai \\\n --enable-auto-tool-choice", "argv": [ "vllm", "serve", "openai/gpt-oss-20b", "--tensor-parallel-size", "1", "--tool-call-parser", "openai", "--enable-auto-tool-choice" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest openai/gpt-oss-20b \\\n --tensor-parallel-size 1 \\\n --tool-call-parser openai \\\n --enable-auto-tool-choice", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "openai/gpt-oss-20b", "--tensor-parallel-size", "1", "--tool-call-parser", "openai", "--enable-auto-tool-choice" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "kv_store_centralized_mooncake": "/openai/gpt-oss-20b/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/openai/gpt-oss-20b/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/openai/gpt-oss-20b/hw/h200.json", "h100": "/openai/gpt-oss-20b/hw/h100.json", "b200": "/openai/gpt-oss-20b/hw/b200.json", "mi300x": "/openai/gpt-oss-20b/hw/mi300x.json", "mi325x": "/openai/gpt-oss-20b/hw/mi325x.json", "mi355x": "/openai/gpt-oss-20b/hw/mi355x.json", "xeon6": "/openai/gpt-oss-20b/hw/xeon6.json", "arc_pro_b60": "/openai/gpt-oss-20b/hw/arc_pro_b60.json", "arc_pro_b70": "/openai/gpt-oss-20b/hw/arc_pro_b70.json" } }, "model": { "model_id": "openai/gpt-oss-20b", "min_vllm_version": "0.10.1", "architecture": "moe", "parameter_count": "21B", "active_parameters": "3.6B", "context_length": 131072, "base_args": [], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "tool_calling": { "description": "OpenAI harmony tool-call parser with automatic tool choice", "args": [ "--tool-call-parser", "openai", "--enable-auto-tool-choice" ] }, "spec_decoding": { "description": "Eagle3 speculative decoding with externally trained draft model; the served checkpoint is unchanged.", "args": [ "--speculative-config", "{\"model\": \"RedHatAI/gpt-oss-20b-speculator.eagle3\", \"num_speculative_tokens\": 3, \"method\": \"eagle3\"}" ] } }, "opt_in_features": [ "spec_decoding" ], "variants": { "default": { "precision": "mxfp4", "vram_minimum_gb": 16, "description": "MXFP4 MoE weights — fits in 16GB VRAM on a single consumer or datacenter GPU", "supported_hardware": [ "h100", "h200", "b200", "mi300x", "mi325x", "mi355x", "arc_pro_b60", "arc_pro_b70", "xeon6" ] } }, "hardware_overrides": { "cpu": { "extra_args": [ "--max-num-batched-tokens", "16384", "--gpu-memory-utilization", "0.8" ] }, "amd": { "extra_args": [ "--attention-backend", "ROCM_AITER_UNIFIED_ATTN", "-cc.pass_config.fuse_rope_kvcache=True", "-cc.use_inductor_graph_partition=True", "--gpu-memory-utilization", "0.95", "--block-size=64" ], "extra_env": { "VLLM_ROCM_USE_AITER": "1", "VLLM_ROCM_QUICK_REDUCE_QUANTIZATION": "INT4", "HSA_NO_SCRATCH_RECLAIM": "1", "AMDGCN_USE_BUFFER_OPS": "0" } } }, "guide": "## Overview\n\n[`gpt-oss-20b`](https://huggingface.co/openai/gpt-oss-20b) is OpenAI's smaller open-weight reasoning model: 21B total parameters with 3.6B activated per token across 32 experts (top-4 routing), shipped with native MXFP4 quantization on the MoE weights. It targets lower-latency and on-device use cases — the model loads in ~16GB of VRAM, runs on a single H100/H200/B200 or AMD MI300X/MI325X/MI355X, and supports the same harmony chat format, configurable reasoning effort (low / medium / high), and built-in tools (browser, python, function calling) as its larger sibling [`gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b).\n\nArchitectural notes:\n- 24 layers alternating sliding-window (window=128) and full attention.\n- YaRN rope scaling (factor=32) extending 4K → 131K context.\n- MXFP4 quant on `model.layers.*.mlp` experts; attention, router, embeddings stay in BF16.\n\n## Prerequisites\n\n- Hardware: NVIDIA H100/H200/B200, AMD MI300X/MI325X/MI355X, Intel XPU B60/B70, or 4x Xeon 6 CPUs.\n- vLLM >= 0.10.1.\n- CUDA >= 12.8 if building from source for NVIDIA GPUs (must match between install and serving).\n\n### Install vLLM\n\n```bash\nuv venv\nsource .venv/bin/activate\nuv pip install vllm --torch-backend=auto\n```\n\n### pip (Intel Xeon 6 CPUs)\n\nFor Intel and AMD x86 CPUs, follow the\n[CPU pre-built wheels](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#pre-built-wheels)\ninstallation instructions.\n\n### Docker (Intel Xeon 6 CPUs)\n\n```bash\ndocker pull vllm/vllm-openai-cpu:latest-x86_64\n```\n\nDocker quickstart:\n\n```bash\ndocker run --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai --model openai/gpt-oss-20b\n```\n\nAMD ROCm wheels:\n\n```bash\nuv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm\n```\n\nIntel XPU (B60 / B70):\n```bash\ndocker pull vllm/vllm-openai-xpu:latest\n```\n\n## Launch commands\n\nSingle GPU (default — works on any 16GB+ card):\n\n```bash\nvllm serve openai/gpt-oss-20b\n```\n\n### Intel Xeon 6\n\nChoose the tensor parallel size based on the system topology and deployment\nrequirements. Add `--tensor-parallel-size ` when needed; the recipe does\nnot prescribe a Xeon 6 TP value or hard-code NUMA node IDs.\n\n```bash\n\nvllm serve openai/gpt-oss-20b \\\n --max-num-batched-tokens 16384 \\\n --gpu-memory-utilization 0.8\n```\n\nDocker:\n\n```bash\ndocker run \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai-cpu:latest-x86_64 openai/gpt-oss-20b \\\n --tensor-parallel-size 1 \\\n --max-num-batched-tokens 16384 \\\n --gpu-memory-utilization 0.8 \\\n --tool-call-parser openai \\\n --enable-auto-tool-choice\n```\n\nBlackwell (B200) with FlashInfer MXFP4+MXFP8 MoE:\n\n```bash\nvllm serve openai/gpt-oss-20b\n```\n\nOn vLLM v0.28.0+ the FlashInfer TRTLLM MXFP4 MoE path is selected automatically on Blackwell; it can be pinned explicitly with `--moe-backend flashinfer_trtllm --quantization-config.moe.activation mxfp8`.\n\nHopper (H100/H200): same as Blackwell minus `--kv-cache-dtype fp8` and the FlashInfer MoE flags.\n\nAMD MI300X/MI325X/MI355X:\n\n```bash\nexport HSA_NO_SCRATCH_RECLAIM=1\nexport AMDGCN_USE_BUFFER_OPS=0\nexport VLLM_ROCM_USE_AITER=1\nexport VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4\n\nvllm serve openai/gpt-oss-20b \\\n --attention-backend ROCM_AITER_UNIFIED_ATTN \\\n -cc.pass_config.fuse_rope_kvcache=True \\\n -cc.use_inductor_graph_partition=True \\\n --gpu-memory-utilization 0.95 \\\n --block-size 64\n```\n\n### Intel XPU(B60, B70)\n\n\n```bash\ndocker run --device /dev/dri \\\n -v /dev/dri/by-path:/dev/dri/by-path --shm-size=16g \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai-xpu:latest \\\n openai/gpt-oss-20b \\\n --max-model-len 8192\n```\n\n## Tool use\n\nThe `/v1/responses` endpoint supports built-in tools (browsing, python, MCP). Setup requires `uv pip install gpt-oss` and either Docker (for the Python sandbox) or `PYTHON_EXECUTION_BACKEND=dangerously_use_uv`. For demo tools:\n\n```bash\nvllm serve openai/gpt-oss-20b --tool-server demo\n```\n\nFor user-defined function calling (toggle the **Tool Calling** feature above, or pass manually):\n\n```bash\nvllm serve openai/gpt-oss-20b --tool-call-parser openai --enable-auto-tool-choice\n```\n\n## Reasoning effort\n\ngpt-oss exposes three reasoning levels — low, medium, high — selected via the system prompt:\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\")\nresponse = client.chat.completions.create(\n model=\"openai/gpt-oss-20b\",\n messages=[\n {\"role\": \"system\", \"content\": \"Reasoning: high\"},\n {\"role\": \"user\", \"content\": \"Explain why eigenvalues matter.\"},\n ],\n)\nprint(response.choices[0].message.content)\n```\n\n## Troubleshooting\n\n- **Attention sinks dtype error on Blackwell:** pass `--quantization-config.moe.activation mxfp8` and `--kv-cache-dtype fp8`.\n- **`tl.language not defined`:** make sure no extra Triton (e.g., `pytorch-triton`) is installed alongside vLLM's bundled Triton.\n- **Harmony vocab download failure:** pre-download tiktoken files and set `TIKTOKEN_ENCODINGS_BASE`.\n\n## References\n\n- [Model card — gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)\n- [Sibling — gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)\n- [OpenAI announcement](https://openai.com/index/introducing-gpt-oss/)\n- [gpt-oss model card paper (arXiv:2508.10925)](https://arxiv.org/abs/2508.10925)\n- [vLLM gpt-oss cookbook](https://cookbook.openai.com/articles/gpt-oss/run-vllm)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb", "path": "recipes/models/openai/gpt-oss-20b.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/openai/gpt-oss-20b.yaml", "overlays": [] } }