{ "hf_id": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", "meta": { "title": "DeepSeek-V4-Flash-Vision-Exp", "slug": "deepseek-v4-flash-vision-exp", "provider": "DeepSeek", "description": "DeepSeek's first experimental multimodal V4 model — the V4-Flash MoE backbone plus a 32-layer vision tower, 1M context, and a fused DSpark draft module.", "date_added": "2026-09-01", "date_updated": "2026-09-14", "tasks": [ "text", "multimodal" ], "performance_headline": "First multimodal V4 — 285B/13B MoE scoring 83.5% on OCRBench from a single GB200 NVL4 tray", "related_recipes": [ "deepseek-ai/DeepSeek-V4-Flash", "deepseek-ai/DeepSeek-V4-Pro" ], "default_hardware": "gb200", "hardware": { "gb200": "verified", "mi350x": "verified", "ascend_950dt_8x": "verified", "ascend_950pr_8x": "verified" } }, "recommended_command": { "hardware": "gb200", "strategy": "single_node_tep", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:deepseekv4-flash-vision", "command": "vllm serve deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \\\n --kv-cache-dtype fp8 \\\n --block-size 256 \\\n --enable-expert-parallel \\\n --tensor-parallel-size 4 \\\n --tool-call-parser deepseek_v4 \\\n --enable-auto-tool-choice \\\n --reasoning-parser deepseek_v4 \\\n --reasoning-config '{\"reasoning_parser\":\"deepseek_v4\",\"reasoning_start_str\":\"\",\"reasoning_end_str\":\"\"}' \\\n --speculative-config '{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":true}'", "argv": [ "vllm", "serve", "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", "--kv-cache-dtype", "fp8", "--block-size", "256", "--enable-expert-parallel", "--tensor-parallel-size", "4", "--tool-call-parser", "deepseek_v4", "--enable-auto-tool-choice", "--reasoning-parser", "deepseek_v4", "--reasoning-config", "{\"reasoning_parser\":\"deepseek_v4\",\"reasoning_start_str\":\"\",\"reasoning_end_str\":\"\"}", "--speculative-config", "{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":true}" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:deepseekv4-flash-vision deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \\\n --kv-cache-dtype fp8 \\\n --block-size 256 \\\n --enable-expert-parallel \\\n --tensor-parallel-size 4 \\\n --tool-call-parser deepseek_v4 \\\n --enable-auto-tool-choice \\\n --reasoning-parser deepseek_v4 \\\n --reasoning-config '{\"reasoning_parser\":\"deepseek_v4\",\"reasoning_start_str\":\"\",\"reasoning_end_str\":\"\"}' \\\n --speculative-config '{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":true}'", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:deepseekv4-flash-vision", "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", "--kv-cache-dtype", "fp8", "--block-size", "256", "--enable-expert-parallel", "--tensor-parallel-size", "4", "--tool-call-parser", "deepseek_v4", "--enable-auto-tool-choice", "--reasoning-parser", "deepseek_v4", "--reasoning-config", "{\"reasoning_parser\":\"deepseek_v4\",\"reasoning_start_str\":\"\",\"reasoning_end_str\":\"\"}", "--speculative-config", "{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":true}" ], "strategy_spec": { "name": "single_node_tep", "deploy_type": "single_node", "display_name": "Tensor + Expert Parallel", "short_label": "TEP", "orientation": "balanced", "description": "Single-node TEP. TP splits dense layers and EP splits expert layers across local GPUs. TP must be set to GPU count to avoid OOM from replicated dense layers. For MoE models only.\n", "hardware_match": { "min_gpus": 2, "max_gpus": 8, "multi_node": false, "architecture": "moe" }, "vllm_args": [ "--enable-expert-parallel" ], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "blackwell", "compute_arch": "sm100", "display_name": "GB200 NVL4", "description": "NVIDIA GB200 Grace-Blackwell compute tray · 192 GB HBM3e/GPU · 4 Blackwell GPUs (NVL72 tray unit)", "gpu_count": 4, "vram_gb": 768, "multi_node": false }, "alternatives": { "single_node_tp": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/strategies/single_node_tp.json", "single_node_dep": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/strategies/single_node_dep.json", "multi_node_tep": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/strategies/multi_node_tep.json", "multi_node_dep": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/strategies/multi_node_dep.json", "kv_store_centralized_mooncake": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "gb200": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/gb200.json", "h100": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/h100.json", "h200": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/h200.json", "b200": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/b200.json", "b300": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/b300.json", "gb300": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/gb300.json", "mi300x": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/mi300x.json", "mi325x": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/mi325x.json", "mi350x": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/mi350x.json", "mi355x": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/mi355x.json", "ascend_950dt_8x": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/ascend_950dt_8x.json", "ascend_950pr_8x": "/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp/hw/ascend_950pr_8x.json" } }, "model": { "model_id": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp", "min_vllm_version": "0.30.0", "nightly_required": true, "docker_image": { "nvidia": "vllm/vllm-openai:deepseekv4-flash-vision", "amd": "vllm/vllm-openai-rocm:nightly" }, "architecture": "moe", "parameter_count": "285B", "active_parameters": "13B", "context_length": 1048576, "flashinfer_autotune": true, "base_args": [ "--kv-cache-dtype", "fp8", "--block-size", "256" ], "install": { "docker": { "command": "docker pull vllm/vllm-openai:deepseekv4-flash-vision", "note": "NVIDIA: use the dedicated DeepSeek-V4-Flash-Vision image until support lands in the standard vLLM image. AMD: vllm/vllm-openai-rocm:v0.30.0 (ROCm 7.2.3) or a nightly >= 2026-09-11 (includes #55107); on v0.30.0 also pass --compilation-config '{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}' (see the AMD section). Ascend 950PR/950DT: pin quay.io/ascend/vllm-ascend:v0.23.0-a5 and overlay vllm-project/vllm-ascend#16259." } } }, "features": { "tool_calling": { "description": "DeepSeek V4 tool-call parsing with automatic tool choice.", "args": [ "--tool-call-parser", "deepseek_v4", "--enable-auto-tool-choice" ] }, "reasoning": { "description": "Enable reasoning/thinking mode with the DeepSeek V4 reasoning parser.", "args": [ "--reasoning-parser", "deepseek_v4", "--reasoning-config", "{\"reasoning_parser\":\"deepseek_v4\",\"reasoning_start_str\":\"\",\"reasoning_end_str\":\"\"}" ], "hardware_overrides": { "npu": { "args": [ "--reasoning-parser", "deepseek_v4" ] } } }, "spec_decoding": { "description": "DSpark self-drafting from the checkpoint's fused draft module — 3 draft tokens, probabilistic sampling, adaptive verification. Mean acceptance length 2.99 tokens/forward on mixed text+image traffic.", "args": [ "--speculative-config", "{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":true}" ], "hardware_overrides": { "amd": { "args": [ "--speculative-config", "{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":false}" ] }, "npu": { "args": [] } } } }, "hardware_opt_in_features": { "ascend_950pr_8x": [ "spec_decoding" ], "ascend_950dt_8x": [ "spec_decoding" ] }, "variants": { "default": { "precision": "fp8", "vram_minimum_gb": 202, "description": "Native FP4+FP8 mixed checkpoint (MoE experts FP4, attention/norm/router FP8) with a BF16 vision tower, aligner, and the fused DSpark draft module.", "hardware_overrides": { "ascend_950pr_8x": { "docker_image": "quay.io/ascend/vllm-ascend:v0.23.0-a5", "install": { "pip": false } }, "ascend_950dt_8x": { "docker_image": "quay.io/ascend/vllm-ascend:v0.23.0-a5", "install": { "pip": false } } } } }, "strategy_hardware": { "single_node_tp": { "npu": "unsupported" }, "single_node_tep": { "npu": "unsupported" }, "multi_node_tep": { "npu": "unsupported" }, "multi_node_dep": { "npu": "unsupported" } }, "hardware_overrides": { "amd": { "extra_args": [ "--tensor-parallel-size", "2", "--gpu-memory-utilization", "0.9", "--max-model-len", "32768" ], "extra_env": { "VLLM_ROCM_USE_AITER": "1", "VLLM_USE_BREAKABLE_CUDAGRAPH": "1", "VLLM_USE_V2_MODEL_RUNNER": "1" } }, "npu": { "extra_args": [ "--data-parallel-size", "4", "--block-size", "128", "--kv-cache-dtype", "auto", "--tokenizer-mode", "deepseek_v4", "--safetensors-load-strategy", "prefetch", "--async-scheduling", "--max-model-len", "8192", "--max-num-seqs", "16", "--max-num-batched-tokens", "2048", "--gpu-memory-utilization", "0.9", "--api-server-count", "1", "--compilation-config", "{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}", "--additional-config", "{\"enable_cpu_binding\": true, \"multistream_overlap_shared_expert\": true, \"enable_shared_expert_dp\": true}" ], "extra_env": { "PYTORCH_NPU_ALLOC_CONF": "expandable_segments:True", "OMP_PROC_BIND": "false", "OMP_NUM_THREADS": "10", "HCCL_BUFFSIZE": "1024", "HCCL_OP_EXPANSION_MODE": "AIV", "TASK_QUEUE_ENABLE": "1", "ASCEND_RT_VISIBLE_DEVICES": "0,1,2,3" } } }, "hardware_notes": { "ascend_950dt_8x": { "title": "vLLM Ascend A5 image required", "body": "Mixed serving on one 8-card 950DT node uses 4 of 8 NPUs (DP4+EP, TP=1). Same command as 950PR. Pin quay.io/ascend/vllm-ascend:v0.23.0-a5 — the NVIDIA pip wheel cannot run here. Set HCCL_BUFFSIZE=1024 before launch. Native Flash-Vision checkpoints need the hash-router bias skip in vllm-project/vllm-ascend#16259 — without it load_weights aborts on model.layers.0.mlp.gate.e_score_correction_bias." }, "ascend_950pr_8x": { "title": "vLLM Ascend A5 image required", "body": "Mixed serving on one 8-card 950PR node uses 4 of 8 NPUs (DP4+EP, TP=1). Same command as 950DT. Pin quay.io/ascend/vllm-ascend:v0.23.0-a5 and set HCCL_BUFFSIZE=1024 before launch. Native Flash-Vision checkpoints need the hash-router bias skip in vllm-project/vllm-ascend#16259 — without it load_weights aborts on model.layers.0.mlp.gate.e_score_correction_bias." } }, "guide": "## Overview\n\nDeepSeek-V4-Flash-Vision-Exp is DeepSeek's first multimodal model in the V4 family.\nIt keeps the DeepSeek-V4-Flash language backbone — 43 layers, 256 routed experts with\n6 active per token, Compressed Sparse Attention plus manifold-constrained\nhyper-connections, a 1,048,576-token context window — and adds a 32-layer / 1024-dim\nViT with a two-layer aligner (~0.5B parameters on top of the 284B backbone). Weights\nare **FP4+FP8 mixed**: MoE experts in FP4, the remaining attention / norm / router\nparams in FP8, vision tower in BF16. The checkpoint is 48 shards / ~168 GB and carries\nthe fused **DSpark** draft module.\n\nDeepSeek reports it beats DeepSeek-V4-Flash-0731 on multimodal agent benchmarks\n(ApexBench 36.5 vs 26.2, Agents' Last Exam 27.3 vs 25.2) while holding text-agent\nparity (Terminal Bench 2.1 at 83.9, DeepSWE at 59.3).\n\n## Prerequisites\n\nVision support is not in a stable vLLM release yet — it lives in\n[vllm-project/vllm#54566](https://github.com/vllm-project/vllm/pull/54566). Until that\nlands, use the pinned `vllm/vllm-openai:deepseekv4-flash-vision` image; the official\nwheel routes this checkpoint to the text-only class and fails on the vision tensors.\n\n- **Weights:** ~168 GB on disk (48 shards). Budget ~202 GB of VRAM before KV cache.\n- **Hardware:** NVIDIA (verified on GB200) and AMD MI350X (gfx950);\n XPU still has no vision implementation.\n- **Tokenizer / chat template:** nothing to pass. The repo ships no Jinja template\n (prompt encoding lives in `encoding/`); vLLM resolves `--tokenizer-mode` to\n `deepseek_v4` automatically, which is what turns OpenAI content blocks into\n `<｜deepseek_image｜>` placeholders.\n\n## Launching the server\n\n### Verified configuration — one GB200 NVL4 tray (TP4 + EP)\n\n```bash\nexport VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS=\"trtllm_fp4_block_scale_moe,flashinfer::trtllm_fp4_block_scale_moe\"\n\nvllm serve deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \\\n --tensor-parallel-size 4 \\\n --enable-expert-parallel \\\n --kv-cache-dtype fp8 \\\n --block-size 256 \\\n --max-model-len 32768 \\\n --speculative-config '{\"method\":\"dspark\",\"model\":\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\"num_speculative_tokens\":3,\"draft_sample_method\":\"probabilistic\",\"enable_adaptive_verification\":true}' \\\n --allowed-local-media-path /mnt/lustre\n```\n\nThree things there are environment-specific, which is why the builder above leaves\nthem out:\n\n- `VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS` excludes the FP4 block-scale MoE op from\n FlashInfer's startup autotuning. Export it if autotuning that op fails or costs more\n startup time than it earns back on your FlashInfer build; it is a tuning knob, not a\n requirement. The Advanced row's `--disable-flashinfer-autotune` turns autotuning\n off wholesale instead.\n- `--max-model-len 32768` bounds KV cache for the benchmark. The checkpoint advertises\n 1M tokens; that is **not** what was measured, and 1M-token KV needs far more memory\n than the weights do. Raise it deliberately, or use the Advanced row's\n `--max-model-len auto`.\n- `--allowed-local-media-path` is only needed for `file://` image URLs. Drop it if\n clients send `http(s)://` or base64 data URLs.\n\n### 8-GPU nodes (H200 / B200 / B300)\n\nSame **Tensor + Expert Parallel** strategy; the builder emits\n`--tensor-parallel-size 8 --enable-expert-parallel`. The language backbone is the one\nthe [DeepSeek-V4-Flash](/deepseek-ai/DeepSeek-V4-Flash) recipe already runs there, but\nthe vision path has no published run on those GPUs — expect to tune\n`--gpu-memory-utilization` and `--max-num-seqs`. The Blackwell\n`--attention_config.indexer_kv_dtype mxfp4` / `--moe-backend deep_gemm_mega_moe` tweaks\nthe text recipe applies are deliberately not set here: the verified vision run did not\nuse them.\n\n### AMD MI350X (gfx950)\n\nROCm vision support is merged (#55107 on the #54566 base); a\n`vllm/vllm-openai-rocm:nightly` built on/after 2026-09-11 runs the checkpoint\nnatively. Three env vars are required:\n\n- `VLLM_ROCM_USE_AITER=1` — the sparse-attention indexer's ROCm path runs only on\n AITER; without it the engine aborts at KV-cache profiling.\n- `VLLM_USE_BREAKABLE_CUDAGRAPH=1` — the multimodal wrapper is not torch-compiled, so\n the default `FULL_AND_PIECEWISE` capture cannot attach without breakable graphs.\n- `VLLM_USE_V2_MODEL_RUNNER=1` — required for DSpark (the V1 ROCm runner refuses it);\n harmless for plain serving.\n\nVerified single-node launch:\n\n```bash\ndocker run -d --name dsv4-vision \\\n --ipc host --network host --privileged --shm-size 128g \\\n --device /dev/kfd --device /dev/dri --group-add video \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n -e VLLM_ROCM_USE_AITER=1 \\\n -e VLLM_USE_BREAKABLE_CUDAGRAPH=1 \\\n -e VLLM_USE_V2_MODEL_RUNNER=1 \\\n vllm/vllm-openai-rocm:nightly \\\n deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \\\n --tensor-parallel-size 2 \\\n --kv-cache-dtype fp8 --block-size 256 \\\n --gpu-memory-utilization 0.9 \\\n --max-model-len 32768\n```\n\nObserved backends: AITER Flash-Attention for the ViT, DeepSeek `fp8_ds_mla` KV cache,\nFP8 Lightning-Indexer cache, `DEEPSEEK_SPARSE_SWA` (block 256), and the\n`ROCM_FLASHMLA_SPARSE_DSV4` MLA backend. TP2 loads 75.2 GiB/card (1.76M-token KV pool\nat 131K context); TP4 loads 38.8 GiB/card.\n\n**Stable v0.30.0 and single-card TP1** (verified 2026-10-02 on one MI350X). The stable\n`vllm/vllm-openai-rocm:v0.30.0` image (ROCm 7.2.3) already contains #54566 and #55107 and\nruns the checkpoint natively — text, vision and DSpark — with the same three env vars. On\nv0.30.0 only, also pass `--compilation-config '{\"cudagraph_mode\":\"FULL_DECODE_ONLY\"}'`:\nv0.30.0's ROCm platform still defaults DeepSeek V4 on gfx950 + ModelRunner V2 to eager\n(the #52644 workaround, removed by #57229 after v0.30.0 was branched), so without it decode\nruns without CUDA graphs. Batch 1 at TP1: 12 tok/s eager vs 94 tok/s with\n`FULL_DECODE_ONLY`, and 137–164 tok/s with DSpark (k=3, adaptive verification off per the\nAMD override). Against eager in the same run, a 25-item known-answer check scored 25/25 and\ndecode-vs-prefill logprobs agreed within eager's range. Nightlies after 2026-09-17 and\nv0.31+ don't need the override. TP1 fits one 288 GB card (147.9 GiB of weights).\n\n\n## Reasoning modes\n\nSame three-tier control as the rest of the V4 family, driven from\n`chat_template_kwargs` rather than a flag — `encoding/encoding_dsv4.py` names the\neffort levels `low` / `high` / `max`. DeepSeek evaluated this model at\n`reasoning_effort: \"max\"` with `temperature = 1.0, top_p = 0.95`.\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\nmodel = \"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\"\n\nresp = client.chat.completions.create(\n model=model,\n messages=[{\"role\": \"user\", \"content\": \"What is 17*19? Return only the final integer.\"}],\n temperature=1.0,\n top_p=0.95,\n extra_body={\n \"chat_template_kwargs\": {\"thinking\": True, \"reasoning_effort\": \"max\"},\n },\n)\nprint(resp.choices[0].message.reasoning)\nprint(resp.choices[0].message.content)\n```\n\n## Client usage — image input\n\nStandard OpenAI multi-part content. Multiple images per request are allowed with no\ncap, so the practical limit is `--max-model-len`.\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\n\nresp = client.chat.completions.create(\n model=\"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\n messages=[{\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/chart.png\"}},\n {\"type\": \"text\", \"text\": \"Read the y-axis label and the highest bar's value.\"},\n ],\n }],\n max_tokens=512,\n)\nprint(resp.choices[0].message.content)\n```\n\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp\",\n \"messages\": [{\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/chart.png\"}},\n {\"type\": \"text\", \"text\": \"Describe the image.\"}\n ]\n }],\n \"max_tokens\": 512\n }'\n```\n\nImage preprocessing is fixed by the checkpoint config (the processor takes no kwargs):\neach image costs at most 387 prompt tokens (`vision_max_n_token: 384` plus the\ncompressor-alignment pad), images below `vision_min_pixels: 147456` are upscaled, and\naspect ratios beyond 8:1 are cropped. An image's exact token count depends on where it\nsits in the prompt, since the compressor pads each image to a 4-token block boundary.\n\n## Speculative decoding\n\nDSpark drafts from the module fused into the checkpoint, so `--speculative-config`\nnames the target repo as its own `model` — there is no separate draft repo to download.\nMeasured over the full OCRBench run on GB200 (mixed text+image traffic): **2.99**\ntokens/forward mean acceptance length, **66.3%** overall acceptance\n(83.9% / 66.5% / 50.7% by draft position). Image tokens in the prompt do not degrade\nacceptance — the drafter reads image content through the target's hidden states.\n\nStay at `num_speculative_tokens: 3` unless you verify acceptance yourself; the\ncheckpoint's trained DSpark block width is 5, but 3 is the depth with published\nnumbers.\n\n## Benchmarking\n\nOCRBench, full 1000 samples, on the verified GB200 configuration: **835/1000\n(83.5%)**, 0 request errors, 312.6 s (~3.2 samples/s). Strongest categories are Key\nInformation Extraction (92.5%), Doc-oriented VQA (90.0%) and Scene Text VQA (89.5%);\nweakest are handwritten math (49.0%) and handwriting recognition (58.0%).\n\nFor throughput:\n\n```bash\nvllm bench serve \\\n --model deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \\\n --dataset-name random \\\n --random-input-len 4096 \\\n --random-output-len 1024 \\\n --num-prompts 200\n```\n\n## Limitations\n\n- **Pre-release.** Flags and defaults can change before #54566 lands; re-check before\n deploying, and keep the image pinned.\n- **XPU has no vision path** — it raises `NotImplementedError` at model construction.\n- **GB200 (NVIDIA) and MI350X (AMD) are the hardware with a verified vision run.**\n Every other pill is the repo's fail-open default, not a tested claim.\n- **PD disaggregation is not offered**, and the CPU/filesystem KV-offload pills stay\n off — neither has a verified run to record. The Mooncake KV-store pills follow the\n repo's fail-open default and are offered, but untested here.\n- **1M context is advertised, not measured.** The reference run used 32K.\n\n## References\n\n- [Model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp)\n- [vLLM support PR #54566](https://github.com/vllm-project/vllm/pull/54566)\n- [vLLM ROCm vision PR #55107](https://github.com/vllm-project/vllm/pull/55107)\n- [DeepSeek-V4-Flash recipe](/deepseek-ai/DeepSeek-V4-Flash) — the text-only sibling\n- [DeepSpec (DSpark)](https://github.com/deepseek-ai/DeepSpec)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "d90e35cffa21729ac55bb0477a0d64f5127dbcda", "path": "recipes/models/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp.yaml", "overlays": [ { "source": "vllm-ascend", "repo": "vllm-project/vllm-ascend", "ref": "main", "sha": "02ac27a19cb7676304a9ca6985ba56ce29664536", "path": "recipes/overlays/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp.yaml", "edit_url": "https://github.com/vllm-project/vllm-ascend/edit/main/recipes/overlays/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp.yaml" } ] } }