{ "hf_id": "Qwen/Qwen3-VL-Reranker-2B", "meta": { "title": "Qwen3-VL-Reranker-2B", "slug": "Qwen3-VL-Reranker-2B", "provider": "Qwen", "description": "Qwen3-VL-Reranker-2B — a 2B vision-language reranker producing relevance scores for query-document pairs across text and multimodal inputs; single-card pooling serving on Ascend 910C with vLLM Ascend", "date_added": "2026-10-14", "date_updated": "2026-11-11", "tasks": [ "embedding", "multimodal" ], "performance_headline": "Single-card pooling serving on Ascend 910C with vLLM Ascend", "default_hardware": "ascend_910c", "hardware": { "ascend_910c": "verified" } }, "recommended_command": { "hardware": "ascend_910c", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3", "command": "vllm serve Qwen/Qwen3-VL-Reranker-2B \\\n --runner pooling \\\n --hf_overrides '{\"architectures\": [\"Qwen3VLForSequenceClassification\"],\"classifier_from_token\": [\"no\", \"yes\"],\"is_original_qwen3_reranker\": true}' \\\n --chat-template /vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja \\\n --max-model-len 1024 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "Qwen/Qwen3-VL-Reranker-2B", "--runner", "pooling", "--hf_overrides", "{\"architectures\": [\"Qwen3VLForSequenceClassification\"],\"classifier_from_token\": [\"no\", \"yes\"],\"is_original_qwen3_reranker\": true}", "--chat-template", "/vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja", "--max-model-len", "1024", "--tensor-parallel-size", "1" ], "docker_command": "docker run --device /dev/davinci0 --device /dev/davinci1 --device /dev/davinci2 --device /dev/davinci3 \\\n --device /dev/davinci4 --device /dev/davinci5 --device /dev/davinci6 --device /dev/davinci7 \\\n --device /dev/davinci8 --device /dev/davinci9 --device /dev/davinci10 --device /dev/davinci11 \\\n --device /dev/davinci12 --device /dev/davinci13 --device /dev/davinci14 --device /dev/davinci15 \\\n --device /dev/davinci_manager --device /dev/hisi_hdc \\\n -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \\\n -v /etc/ascend_install.info:/etc/ascend_install.info \\\n -v /usr/local/dcmi:/usr/local/dcmi \\\n -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \\\n --privileged --net=host --shm-size=16g \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n quay.io/ascend/vllm-ascend:v0.23.0.post1-a3 \\\n vllm serve Qwen/Qwen3-VL-Reranker-2B \\\n --runner pooling \\\n --hf_overrides '{\"architectures\": [\"Qwen3VLForSequenceClassification\"],\"classifier_from_token\": [\"no\", \"yes\"],\"is_original_qwen3_reranker\": true}' \\\n --chat-template /vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja \\\n --max-model-len 1024 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--device", "/dev/davinci0", "--device", "/dev/davinci1", "--device", "/dev/davinci2", "--device", "/dev/davinci3", "--device", "/dev/davinci4", "--device", "/dev/davinci5", "--device", "/dev/davinci6", "--device", "/dev/davinci7", "--device", "/dev/davinci8", "--device", "/dev/davinci9", "--device", "/dev/davinci10", "--device", "/dev/davinci11", "--device", "/dev/davinci12", "--device", "/dev/davinci13", "--device", "/dev/davinci14", "--device", "/dev/davinci15", "--device", "/dev/davinci_manager", "--device", "/dev/hisi_hdc", "-v", "/usr/local/Ascend/driver:/usr/local/Ascend/driver", "-v", "/etc/ascend_install.info:/etc/ascend_install.info", "-v", "/usr/local/dcmi:/usr/local/dcmi", "-v", "/usr/local/bin/npu-smi:/usr/local/bin/npu-smi", "--privileged", "--net=host", "--shm-size", "16g", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3", "vllm", "serve", "Qwen/Qwen3-VL-Reranker-2B", "--runner", "pooling", "--hf_overrides", "{\"architectures\": [\"Qwen3VLForSequenceClassification\"],\"classifier_from_token\": [\"no\", \"yes\"],\"is_original_qwen3_reranker\": true}", "--chat-template", "/vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja", "--max-model-len", "1024", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "Huawei", "generation": "npu", "display_name": "Ascend 910C", "description": "Huawei Ascend 910C · 16× NPU · 64 GB HBM each · Atlas 800 A3 · vLLM Ascend backend", "gpu_count": 16, "vram_gb": 1024, "multi_node": false, "restricted": true }, "by_hardware": { "ascend_910c": "/Qwen/Qwen3-VL-Reranker-2B/hw/ascend_910c.json" } }, "model": { "model_id": "Qwen/Qwen3-VL-Reranker-2B", "min_vllm_version": "0.23.0", "architecture": "dense", "parameter_count": "2B", "active_parameters": "2B", "context_length": 32768, "base_args": [ "--runner", "pooling", "--hf_overrides", "{\"architectures\": [\"Qwen3VLForSequenceClassification\"],\"classifier_from_token\": [\"no\", \"yes\"],\"is_original_qwen3_reranker\": true}", "--chat-template", "/vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja" ], "base_env": {}, "docker_image": { "ascend_910c": "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3" }, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 5, "supported_hardware": [ "ascend_910c" ], "hardware_overrides": { "ascend_910c": { "docker_image": "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3", "docker_serve_command": "vllm serve", "extra_args": [ "--max-model-len", "1024" ] } }, "description": "BF16 checkpoint for single-node Ascend 910C pooling with the official chat template; rerank behaviour recorded on a pre-release build, not yet re-run on the pinned v0.23.0.post1-a3 image" } }, "hardware_overrides": {}, "guide": "## Overview\n\nQwen3-VL-Reranker-2B produces relevance scores for text and multimodal query-document\npairs. This recipe uses the required sequence-classification overrides and the official\nQwen3 VL reranker chat template for pooling serving on Ascend 910C.\n\n## Validation scope and support status\n\nThe main tutorial marks this model discontinued since vLLM Ascend v0.30.0 and points at the\nv0.23.0 documentation, so this recipe is pinned to the `v0.23.0.post1-a3` image and only covers the\nsingle-node pooling scenario that document describes. Atlas 800 A3 is `verified` on that image:\none Atlas 800T A3 node served this model with the rendered command and the required chat\ntemplate, the `/rerank` endpoint scored the capital-of-China document 0.9987 against 0.9248\nfor the gravity document, and candidate reranking on BeIR/SciFact (100 queries x 10\ncandidates) reached **MRR 0.9825 / nDCG@10 0.9869**. The checkpoint config exposes 262144\npositions while the model card lists a 32K context; the runs used `--max-model-len 1024`.\nFull MTEB accuracy, multimodal, long-context, and other hardware scenarios remain unverified.\n\nThe `--chat-template` path above points at the official vLLM Ascend image layout. If vLLM\nis installed differently, copy `examples/pooling/score/template/qwen3_vl_reranker.jinja`\nfrom the vLLM source tree and pass its absolute path instead.\n\n## References\n\n- [Historical vLLM Ascend tutorial (v0.23.0)](https://docs.vllm.ai/projects/ascend/en/v0.23.0/tutorials/models/Qwen3-VL-Reranker.html)\n- [Main tutorial support status](https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/tutorials/models/Qwen3-VL-Reranker.md)\n- [Qwen3-VL-Reranker-2B model card](https://huggingface.co/Qwen/Qwen3-VL-Reranker-2B)\n- [Qwen3-VL-Reranker-2B on ModelScope](https://www.modelscope.cn/models/Qwen/Qwen3-VL-Reranker-2B)\n", "source": { "source": "vllm-ascend", "repo": "vllm-project/vllm-ascend", "ref": "main", "sha": "931bfd948e04a89b735a2f78520093aa9dbc49f0", "path": "recipes/models/Qwen/Qwen3-VL-Reranker-2B.yaml", "edit_url": "https://github.com/vllm-project/vllm-ascend/edit/main/recipes/models/Qwen/Qwen3-VL-Reranker-2B.yaml", "overlays": [] } }