vLLM/Recipes
Search
Qwen

Qwen/Qwen3-VL-Reranker-2B

Qwen3-VL-Reranker-2B — a 2B vision-language reranker producing relevance scores for query-document pairs across text and multimodal inputs; single-card pooling serving on Ascend 910C with vLLM Ascend

Single-card pooling serving on Ascend 910C with vLLM Ascend

View on HuggingFace
dense2B32,768 ctxvLLM 0.23.0+embeddingmultimodal
Default configuration: 16× Ascend 910C · Tensor Parallel · BF16
vllm serve Qwen/Qwen3-VL-Reranker-2B \
  --runner pooling \
  --hf_overrides '{"architectures": ["Qwen3VLForSequenceClassification"],"classifier_from_token": ["no", "yes"],"is_original_qwen3_reranker": true}' \
  --chat-template /vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja \
  --max-model-len 1024 \
  --tensor-parallel-size 1

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3-VL-Reranker-2B.md.

Guide

Overview

Qwen3-VL-Reranker-2B produces relevance scores for text and multimodal query-document pairs. This recipe uses the required sequence-classification overrides and the official Qwen3 VL reranker chat template for pooling serving on Ascend 910C.

Validation scope and support status

The main tutorial marks this model discontinued since vLLM Ascend v0.30.0 and points at the v0.23.0 documentation, so this recipe is pinned to the v0.23.0.post1-a3 image and only covers the single-node pooling scenario that document describes. Atlas 800 A3 is verified on that image: one Atlas 800T A3 node served this model with the rendered command and the required chat template, the /rerank endpoint scored the capital-of-China document 0.9987 against 0.9248 for the gravity document, and candidate reranking on BeIR/SciFact (100 queries x 10 candidates) reached MRR 0.9825 / nDCG@10 0.9869. The checkpoint config exposes 262144 positions while the model card lists a 32K context; the runs used --max-model-len 1024. Full MTEB accuracy, multimodal, long-context, and other hardware scenarios remain unverified.

The --chat-template path above points at the official vLLM Ascend image layout. If vLLM is installed differently, copy examples/pooling/score/template/qwen3_vl_reranker.jinja from the vLLM source tree and pass its absolute path instead.

References