{
  "hf_id": "inclusionAI/Ling-3.0-flash-VL",
  "meta": {
    "title": "Ling-3.0-flash-VL",
    "slug": "ling-3-0-flash-vl",
    "provider": "inclusionAI",
    "description": "Ling-3.0-flash vision-language MoE model with BF16, FP8, FP4, and INT4 checkpoints",
    "date_added": "2026-09-08",
    "date_updated": "2026-09-08",
    "tasks": [
      "text",
      "multimodal"
    ],
    "related_recipes": [
      "inclusionAI/Ling-3.0-flash"
    ]
  },
  "recommended_command": {
    "hardware": "h200",
    "strategy": "single_node_tp",
    "variant": "default",
    "node_count": 1,
    "deploy_type": "single_node",
    "env": {},
    "docker_image": "vllm/vllm-openai:nightly",
    "command": "vllm serve inclusionAI/Ling-3.0-flash-VL \\\n  --trust-remote-code \\\n  --dtype bfloat16 \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-chunked-prefill \\\n  --compilation-config '{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\"}' \\\n  --enable-prefix-caching \\\n  --tensor-parallel-size 4 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser ling3 \\\n  --reasoning-parser ling3",
    "argv": [
      "vllm",
      "serve",
      "inclusionAI/Ling-3.0-flash-VL",
      "--trust-remote-code",
      "--dtype",
      "bfloat16",
      "--gpu-memory-utilization",
      "0.9",
      "--enable-chunked-prefill",
      "--compilation-config",
      "{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\"}",
      "--enable-prefix-caching",
      "--tensor-parallel-size",
      "4",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "ling3",
      "--reasoning-parser",
      "ling3"
    ],
    "docker_command": "docker run --gpus all \\\n  --privileged --ipc=host -p 8000:8000 \\\n  -v ~/.cache/huggingface:/root/.cache/huggingface \\\n  vllm/vllm-openai:nightly inclusionAI/Ling-3.0-flash-VL \\\n  --trust-remote-code \\\n  --dtype bfloat16 \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-chunked-prefill \\\n  --compilation-config '{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\"}' \\\n  --enable-prefix-caching \\\n  --tensor-parallel-size 4 \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser ling3 \\\n  --reasoning-parser ling3",
    "docker_argv": [
      "docker",
      "run",
      "--gpus",
      "all",
      "--privileged",
      "--ipc=host",
      "-p",
      "8000:8000",
      "-v",
      "~/.cache/huggingface:/root/.cache/huggingface",
      "vllm/vllm-openai:nightly",
      "inclusionAI/Ling-3.0-flash-VL",
      "--trust-remote-code",
      "--dtype",
      "bfloat16",
      "--gpu-memory-utilization",
      "0.9",
      "--enable-chunked-prefill",
      "--compilation-config",
      "{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\"}",
      "--enable-prefix-caching",
      "--tensor-parallel-size",
      "4",
      "--enable-auto-tool-choice",
      "--tool-call-parser",
      "ling3",
      "--reasoning-parser",
      "ling3"
    ],
    "strategy_spec": {
      "name": "single_node_tp",
      "deploy_type": "single_node",
      "display_name": "Tensor Parallel",
      "short_label": "TP",
      "orientation": "latency",
      "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n",
      "hardware_match": {
        "min_gpus": 1,
        "max_gpus": 8,
        "multi_node": false
      },
      "vllm_args": [],
      "parallel_flag": "--tensor-parallel-size"
    },
    "hardware_profile": {
      "brand": "NVIDIA",
      "generation": "hopper",
      "compute_arch": "sm90",
      "display_name": "H200",
      "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node",
      "gpu_count": 8,
      "vram_gb": 1128,
      "multi_node": false
    },
    "alternatives": {
      "kv_store_centralized_mooncake": "/inclusionAI/Ling-3.0-flash-VL/strategies/kv_store_centralized_mooncake.json",
      "kv_store_distributed_mooncake": "/inclusionAI/Ling-3.0-flash-VL/strategies/kv_store_distributed_mooncake.json"
    },
    "by_hardware": {
      "h200": "/inclusionAI/Ling-3.0-flash-VL/hw/h200.json",
      "h100": "/inclusionAI/Ling-3.0-flash-VL/hw/h100.json",
      "b200": "/inclusionAI/Ling-3.0-flash-VL/hw/b200.json",
      "gb200": "/inclusionAI/Ling-3.0-flash-VL/hw/gb200.json",
      "b300": "/inclusionAI/Ling-3.0-flash-VL/hw/b300.json",
      "gb300": "/inclusionAI/Ling-3.0-flash-VL/hw/gb300.json",
      "mi300x": "/inclusionAI/Ling-3.0-flash-VL/hw/mi300x.json",
      "mi325x": "/inclusionAI/Ling-3.0-flash-VL/hw/mi325x.json",
      "mi350x": "/inclusionAI/Ling-3.0-flash-VL/hw/mi350x.json",
      "mi355x": "/inclusionAI/Ling-3.0-flash-VL/hw/mi355x.json"
    }
  },
  "model": {
    "model_id": "inclusionAI/Ling-3.0-flash-VL",
    "min_vllm_version": "nightly",
    "nightly_required": true,
    "architecture": "moe",
    "parameter_count": "124.85B",
    "active_parameters": "5.51B",
    "context_length": 131072,
    "base_args": [
      "--trust-remote-code",
      "--dtype",
      "bfloat16",
      "--gpu-memory-utilization",
      "0.9",
      "--enable-chunked-prefill",
      "--compilation-config",
      "{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\"}",
      "--enable-prefix-caching"
    ],
    "install": {
      "pip": {
        "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --pre \\\n  --extra-index-url https://wheels.vllm.ai/nightly/cu130 \\\n  --extra-index-url https://download.pytorch.org/whl/cu130 \\\n  --index-strategy unsafe-best-match",
        "note": "vLLM nightly isn't released yet — nightly required. Nightly wheels are CUDA 13 (cu130) only; the host needs an r580+ NVIDIA driver."
      },
      "docker": {
        "command": "docker pull vllm/vllm-openai:latest"
      }
    }
  },
  "features": {
    "tool_calling": {
      "description": "Enable Ling 3 automatic tool calling",
      "args": [
        "--enable-auto-tool-choice",
        "--tool-call-parser",
        "ling3"
      ]
    },
    "reasoning": {
      "description": "Split Ling 3 thinking traces into reasoning",
      "args": [
        "--reasoning-parser",
        "ling3"
      ]
    }
  },
  "variants": {
    "default": {
      "precision": "bf16",
      "vram_minimum_gb": 300,
      "description": "BF16 checkpoint; defaults to TP4"
    },
    "fp8": {
      "model_id": "inclusionAI/Ling-3.0-flash-VL-fp8",
      "precision": "fp8",
      "vram_minimum_gb": 150,
      "tp": 2,
      "description": "Serialized block-FP8 checkpoint; defaults to TP2",
      "json": "/inclusionAI/Ling-3.0-flash-VL-fp8.json"
    },
    "fp4": {
      "model_id": "inclusionAI/Ling-3.0-flash-VL-fp4",
      "precision": "fp4",
      "precision_hardware": {
        "brand": "NVIDIA"
      },
      "vram_minimum_gb": 85,
      "tp": 1,
      "description": "Mixed block-FP8 and MXFP4 checkpoint; defaults to TP1",
      "json": "/inclusionAI/Ling-3.0-flash-VL-fp4.json"
    },
    "int4": {
      "model_id": "inclusionAI/Ling-3.0-flash-VL-int4",
      "precision": "int4",
      "vram_minimum_gb": 95,
      "tp": 1,
      "description": "INT4 checkpoint; defaults to TP1",
      "json": "/inclusionAI/Ling-3.0-flash-VL-int4.json"
    }
  },
  "hardware_overrides": {},
  "guide": "## Overview\n\n`inclusionAI/Ling-3.0-flash-VL` uses the\n`BailingMoeV3VLForConditionalGeneration` architecture. It combines the\nLing-3.0-flash hybrid MLA/KDA language backbone with a vision encoder and\nprojector for image understanding. The language backbone has 512 routed\nexperts (8 active per token) and one shared expert.\n\nBF16, FP8, mixed block-FP8/MXFP4, and INT4 checkpoints are available. The\nvision tower and projector remain BF16 in the mixed FP4 checkpoint.\n\n## Prerequisites\n\n- **vLLM:** an official nightly build containing Ling-3.0-flash-VL support\n- **Precision:** BF16, FP8, FP4, or INT4 weights with BF16 compute\n- **Context length:** 131,072 tokens\n\n## Launching the Server\n\n```bash\nNCCL_DEBUG=WARN vllm serve inclusionAI/Ling-3.0-flash-VL \\\n  --trust-remote-code \\\n  --dtype bfloat16 \\\n  --tensor-parallel-size 4 \\\n  --gpu-memory-utilization 0.9 \\\n  --enable-chunked-prefill \\\n  --compilation-config '{\"cudagraph_mode\":\"FULL_AND_PIECEWISE\"}' \\\n  --enable-prefix-caching \\\n  --enable-auto-tool-choice \\\n  --tool-call-parser ling3 \\\n  --reasoning-parser ling3\n```\n\nFor a quantized variant, use the same flags with the corresponding model\nID and tensor parallel size:\n\n| Variant | Model ID | TP |\n|---|---|---:|\n| FP8 | `inclusionAI/Ling-3.0-flash-VL-fp8` | 2 |\n| FP4 | `inclusionAI/Ling-3.0-flash-VL-fp4` | 1 |\n| INT4 | `inclusionAI/Ling-3.0-flash-VL-int4` | 1 |\n\n## Image Requests\n\n```python\nimport base64\nfrom pathlib import Path\nfrom openai import OpenAI\n\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"EMPTY\")\nimage = base64.b64encode(Path(\"image.png\").read_bytes()).decode(\"ascii\")\nresponse = client.chat.completions.create(\n    model=\"inclusionAI/Ling-3.0-flash-VL\",\n    messages=[{\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"image_url\", \"image_url\": {\"url\": f\"data:image/png;base64,{image}\"}},\n            {\"type\": \"text\", \"text\": \"Describe the image.\"},\n        ],\n    }],\n    temperature=0.0,\n    max_tokens=1024,\n    extra_body={\"chat_template_kwargs\": {\"enable_thinking\": False}},\n)\nprint(response.choices[0].message.content)\n```\n\nFor multiple images, add more `image_url` entries to `content`. For\ntext-only requests, use ordinary string content. When serving a quantized\nvariant, use its checkpoint ID in the client request.\n\n## Thinking Mode\n\nAs with Ling-3.0-flash, thinking is selected per request through\n`chat_template_kwargs`. Set `enable_thinking` to `True` to enable it;\nwith `--reasoning-parser ling3`, the parsed trace is in `message.reasoning`\nand the final answer is in `message.content`.\n\n## References\n\n- [Ling-3.0-flash-VL](https://huggingface.co/inclusionAI/Ling-3.0-flash-VL)\n- [Ling-3.0-flash-VL-fp8](https://huggingface.co/inclusionAI/Ling-3.0-flash-VL-fp8)\n- [Ling-3.0-flash-VL-fp4](https://huggingface.co/inclusionAI/Ling-3.0-flash-VL-fp4)\n- [Ling-3.0-flash-VL-int4](https://huggingface.co/inclusionAI/Ling-3.0-flash-VL-int4)\n",
  "source": {
    "source": "vllm",
    "repo": "vllm-project/vllm",
    "ref": "main",
    "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb",
    "path": "recipes/models/inclusionAI/Ling-3.0-flash-VL.yaml",
    "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/inclusionAI/Ling-3.0-flash-VL.yaml",
    "overlays": []
  }
}