# Qwen/Qwen3-VL-Reranker-2B — vLLM serving recipe

> Qwen3-VL-Reranker-2B — a 2B vision-language reranker producing relevance scores for query-document pairs across text and multimodal inputs; single-card pooling serving on Ascend 910C with vLLM Ascend

Configuration: hardware `ascend_910c` (16× Ascend 910C) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://recipes.vllm.ai/Qwen/Qwen3-VL-Reranker-2B
- This configuration as JSON: https://recipes.vllm.ai/Qwen/Qwen3-VL-Reranker-2B.json?hardware=ascend_910c

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `ascend_910c`
- `variant`: `default`
- `strategy`: `single_node_tp` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.23.0.

Docker image: `quay.io/ascend/vllm-ascend:v0.23.0.post1-a3`

## Serve

```bash
vllm serve Qwen/Qwen3-VL-Reranker-2B \
  --runner pooling \
  --hf_overrides '{"architectures": ["Qwen3VLForSequenceClassification"],"classifier_from_token": ["no", "yes"],"is_original_qwen3_reranker": true}' \
  --chat-template /vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja \
  --max-model-len 1024 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --device /dev/davinci0 --device /dev/davinci1 --device /dev/davinci2 --device /dev/davinci3 \
  --device /dev/davinci4 --device /dev/davinci5 --device /dev/davinci6 --device /dev/davinci7 \
  --device /dev/davinci8 --device /dev/davinci9 --device /dev/davinci10 --device /dev/davinci11 \
  --device /dev/davinci12 --device /dev/davinci13 --device /dev/davinci14 --device /dev/davinci15 \
  --device /dev/davinci_manager --device /dev/hisi_hdc \
  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
  -v /etc/ascend_install.info:/etc/ascend_install.info \
  -v /usr/local/dcmi:/usr/local/dcmi \
  -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
  --privileged --net=host --shm-size=16g \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  quay.io/ascend/vllm-ascend:v0.23.0.post1-a3 \
  vllm serve Qwen/Qwen3-VL-Reranker-2B \
  --runner pooling \
  --hf_overrides '{"architectures": ["Qwen3VLForSequenceClassification"],"classifier_from_token": ["no", "yes"],"is_original_qwen3_reranker": true}' \
  --chat-template /vllm-workspace/vllm/examples/pooling/score/template/qwen3_vl_reranker.jinja \
  --max-model-len 1024 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Qwen3-VL-Reranker-2B produces relevance scores for text and multimodal query-document
pairs. This recipe uses the required sequence-classification overrides and the official
Qwen3 VL reranker chat template for pooling serving on Ascend 910C.

### Validation scope and support status

The main tutorial marks this model discontinued since vLLM Ascend v0.30.0 and points at the
v0.23.0 documentation, so this recipe is pinned to the `v0.23.0.post1-a3` image and only covers the
single-node pooling scenario that document describes. Atlas 800 A3 is `verified` on that image:
one Atlas 800T A3 node served this model with the rendered command and the required chat
template, the `/rerank` endpoint scored the capital-of-China document 0.9987 against 0.9248
for the gravity document, and candidate reranking on BeIR/SciFact (100 queries x 10
candidates) reached **MRR 0.9825 / nDCG@10 0.9869**. The checkpoint config exposes 262144
positions while the model card lists a 32K context; the runs used `--max-model-len 1024`.
Full MTEB accuracy, multimodal, long-context, and other hardware scenarios remain unverified.

The `--chat-template` path above points at the official vLLM Ascend image layout. If vLLM
is installed differently, copy `examples/pooling/score/template/qwen3_vl_reranker.jinja`
from the vLLM source tree and pass its absolute path instead.

### References

- [Historical vLLM Ascend tutorial (v0.23.0)](https://docs.vllm.ai/projects/ascend/en/v0.23.0/tutorials/models/Qwen3-VL-Reranker.html)
- [Main tutorial support status](https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/tutorials/models/Qwen3-VL-Reranker.md)
- [Qwen3-VL-Reranker-2B model card](https://huggingface.co/Qwen/Qwen3-VL-Reranker-2B)
- [Qwen3-VL-Reranker-2B on ModelScope](https://www.modelscope.cn/models/Qwen/Qwen3-VL-Reranker-2B)