deepseek-ai/DeepSeek-V4-Flash-Vision-Exp

Overview

DeepSeek-V4-Flash-Vision-Exp is DeepSeek's first multimodal model in the V4 family. It keeps the DeepSeek-V4-Flash language backbone — 43 layers, 256 routed experts with 6 active per token, Compressed Sparse Attention plus manifold-constrained hyper-connections, a 1,048,576-token context window — and adds a 32-layer / 1024-dim ViT with a two-layer aligner (~0.5B parameters on top of the 284B backbone). Weights are FP4+FP8 mixed: MoE experts in FP4, the remaining attention / norm / router params in FP8, vision tower in BF16. The checkpoint is 48 shards / ~168 GB and carries the fused DSpark draft module.

DeepSeek reports it beats DeepSeek-V4-Flash-0731 on multimodal agent benchmarks (ApexBench 36.5 vs 26.2, Agents' Last Exam 27.3 vs 25.2) while holding text-agent parity (Terminal Bench 2.1 at 83.9, DeepSWE at 59.3).

Prerequisites

Vision support is not in a stable vLLM release yet — it lives in vllm-project/vllm#54566. Until that lands, use the pinned vllm/vllm-openai:deepseekv4-flash-vision image; the official wheel routes this checkpoint to the text-only class and fails on the vision tensors.

• Weights: ~168 GB on disk (48 shards). Budget ~202 GB of VRAM before KV cache.
• Hardware: NVIDIA only — the ROCm and XPU builds have no vision implementation.
• Tokenizer / chat template: nothing to pass. The repo ships no Jinja template (prompt encoding lives in encoding/); vLLM resolves --tokenizer-mode to deepseek_v4 automatically, which is what turns OpenAI content blocks into <｜deepseek_image｜> placeholders.

Launching the server

Verified configuration — one GB200 NVL4 tray (TP4 + EP)

export VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS="trtllm_fp4_block_scale_moe,flashinfer::trtllm_fp4_block_scale_moe"

vllm serve deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --max-model-len 32768 \
  --speculative-config '{"method":"dspark","model":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","num_speculative_tokens":3,"draft_sample_method":"probabilistic","enable_adaptive_verification":true}' \
  --allowed-local-media-path /mnt/lustre

Three things there are environment-specific, which is why the builder above leaves them out:

• VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS excludes the FP4 block-scale MoE op from FlashInfer's startup autotuning. Export it if autotuning that op fails or costs more startup time than it earns back on your FlashInfer build; it is a tuning knob, not a requirement. The Advanced row's --no-enable-flashinfer-autotune turns autotuning off wholesale instead.
• --max-model-len 32768 bounds KV cache for the benchmark. The checkpoint advertises 1M tokens; that is not what was measured, and 1M-token KV needs far more memory than the weights do. Raise it deliberately, or use the Advanced row's --max-model-len auto.
• --allowed-local-media-path is only needed for file:// image URLs. Drop it if clients send http(s):// or base64 data URLs.

8-GPU nodes (H200 / B200 / B300)

Same Tensor + Expert Parallel strategy; the builder emits --tensor-parallel-size 8 --enable-expert-parallel. The language backbone is the one the DeepSeek-V4-Flash recipe already runs there, but the vision path has no published run on those GPUs — expect to tune --gpu-memory-utilization and --max-num-seqs. The Blackwell --attention_config.use_fp4_indexer_cache / --moe-backend deep_gemm_mega_moe tweaks the text recipe applies are deliberately not set here: the verified vision run did not use them.

Reasoning modes

Same three-tier control as the rest of the V4 family, driven from chat_template_kwargs rather than a flag — encoding/encoding_dsv4.py names the effort levels low / high / max. DeepSeek evaluated this model at reasoning_effort: "max" with temperature = 1.0, top_p = 0.95.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
model = "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp"

resp = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "What is 17*19? Return only the final integer."}],
    temperature=1.0,
    top_p=0.95,
    extra_body={
        "chat_template_kwargs": {"thinking": True, "reasoning_effort": "max"},
    },
)
print(resp.choices[0].message.reasoning)
print(resp.choices[0].message.content)

Client usage — image input

Standard OpenAI multi-part content. Multiple images per request are allowed with no cap, so the practical limit is --max-model-len.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
            {"type": "text", "text": "Read the y-axis label and the highest bar's value."},
        ],
    }],
    max_tokens=512,
)
print(resp.choices[0].message.content)

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V4-Flash-Vision-Exp",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
        {"type": "text", "text": "Describe the image."}
      ]
    }],
    "max_tokens": 512
  }'

Image preprocessing is fixed by the checkpoint config (the processor takes no kwargs): each image costs at most 387 prompt tokens (vision_max_n_token: 384 plus the compressor-alignment pad), images below vision_min_pixels: 147456 are upscaled, and aspect ratios beyond 8:1 are cropped. An image's exact token count depends on where it sits in the prompt, since the compressor pads each image to a 4-token block boundary.

Speculative decoding

DSpark drafts from the module fused into the checkpoint, so --speculative-config names the target repo as its own model — there is no separate draft repo to download. Measured over the full OCRBench run on GB200 (mixed text+image traffic): 2.99 tokens/forward mean acceptance length, 66.3% overall acceptance (83.9% / 66.5% / 50.7% by draft position). Image tokens in the prompt do not degrade acceptance — the drafter reads image content through the target's hidden states.

Stay at num_speculative_tokens: 3 unless you verify acceptance yourself; the checkpoint's trained DSpark block width is 5, but 3 is the depth with published numbers.

Benchmarking

OCRBench, full 1000 samples, on the verified GB200 configuration: 835/1000 (83.5%), 0 request errors, 312.6 s (~3.2 samples/s). Strongest categories are Key Information Extraction (92.5%), Doc-oriented VQA (90.0%) and Scene Text VQA (89.5%); weakest are handwritten math (49.0%) and handwriting recognition (58.0%).

For throughput:

vllm bench serve \
  --model deepseek-ai/DeepSeek-V4-Flash-Vision-Exp \
  --dataset-name random \
  --random-input-len 4096 \
  --random-output-len 1024 \
  --num-prompts 200

Limitations

• Pre-release. Flags and defaults can change before #54566 lands; re-check before deploying, and keep the image pinned.
• NVIDIA only. ROCm and XPU raise NotImplementedError at model construction.
• GB200 is the only hardware with a published vision run. Every other pill is the repo's fail-open default, not a tested claim.
• PD disaggregation is not offered, and the CPU/filesystem KV-offload pills stay off — neither has a verified run to record. The Mooncake KV-store pills follow the repo's fail-open default and are offered, but untested here.
• 1M context is advertised, not measured. The reference run used 32K.

References

• Model card
• vLLM support PR #54566
• DeepSeek-V4-Flash recipe — the text-only sibling
• DeepSpec (DSpark)