Qwen/Qwen3-Reranker-0.6B

Overview

Qwen/Qwen3-Reranker-0.6B scores the relevance of documents to a query. The original checkpoint uses the Qwen3 causal-LM architecture; the required HF overrides select Qwen3ForSequenceClassification and the no/yes classification vectors.

This is a historical Atlas 800 A3 recipe. The current Ascend tutorial states that the model was discontinued in vLLM Ascend v0.30.0 and directs users to the v0.23.0 documentation. The pinned image is retained from the migrated tutorial; no claim about current-main support is made. The hardware entry is not marked verified by this proposal.

The checkpoint config advertises 40,960 positions (the model card describes a 32K context). The serving baseline here is 1,024 tokens. This recipe makes no long-context, 4B, 8B, A2, or Atlas 300I DUO verification claim.

Checkpoint provenance

The model revision is pinned to e61197ed45024b0ed8a2d74b80b4d909f1255473. Its single model.safetensors file is 1,191,588,280 bytes with SHA-256 27cd75a405b9c1b46b59abfd88aaa209e6fed2a1972cde9b70e7659537c5e65b. The config.json SHA-256 is d479c427a9ca5295218063d4f9aca4f297ab4ac27487cca7af42c84643d51ef0.

Both hashes match the checkpoint staged for the accompanying task. Staging integrity does not prove that inference or an accuracy benchmark passed.

If the runtime cannot access Hugging Face, stage this exact revision in a shared cache before starting the container:

hf download Qwen/Qwen3-Reranker-0.6B --revision e61197ed45024b0ed8a2d74b80b4d909f1255473

Client usage

After the pooling server is ready, install requests in the client environment and run the Python example below. It sends a JSON POST to /v1/rerank with the instruction, query, document and assistant suffix required by this model. Use this example for rerank verification; the command builder's generic Verify example targets chat completions.

The examples use port 8000 on the same host as the server. If the server runs on another host or port, update the Python request URL and the benchmark --host / --port arguments accordingly. Changes to the builder's endpoint controls do not update these Guide examples.

import math

import requests

prefix = '<|im_start|>system\nJudge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be "yes" or "no".<|im_end|>\n<|im_start|>user\n'
suffix = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
instruction = "Given a web search query, retrieve relevant passages that answer the query"
query = "What is the capital of China?"
documents = [
    "The capital of China is Beijing.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun.",
]

response = requests.post(
    "http://127.0.0.1:8000/v1/rerank",
    json={
        "model": "Qwen/Qwen3-Reranker-0.6B",
        "query": f"{prefix}<Instruct>: {instruction}\n<Query>: {query}\n",
        "documents": [f"<Document>: {doc}{suffix}" for doc in documents],
    },
    timeout=60,
)
response.raise_for_status()
payload = response.json()
results = payload.get("results")
assert isinstance(results, list) and len(results) == len(documents), payload
assert {item["index"] for item in results} == set(range(len(documents))), payload
scores = {item["index"]: item["relevance_score"] for item in results}
assert all(
    isinstance(score, (int, float))
    and math.isfinite(score)
    and 0.0 <= score <= 1.0
    for score in scores.values()
), payload
assert scores[0] > scores[1], payload
print(payload)

A successful result has HTTP 200, one finite probability score per document, and a higher score for the Beijing passage. This is a functional smoke check, not an MTEB quality measurement.

Evaluation and performance

Run this serving benchmark in the same historical vLLM environment as the pooling server. It targets /v1/rerank with the vllm-rerank backend and synthetic random-rerank input. The command builder's generic Bench example targets text generation; use this rerank command instead.

vllm bench serve \
  --model Qwen/Qwen3-Reranker-0.6B \
  --host localhost \
  --port 8000 \
  --backend vllm-rerank \
  --dataset-name random-rerank \
  --endpoint /v1/rerank \
  --random-input 200 --save-result --result-dir ./

This is a synthetic serving benchmark, not a dataset accuracy test. For MTEB MultiLongDocReranking, follow the historical instructions with a matching vLLM environment. This recipe does not claim to reproduce model-card quality results.

References

• Official model card
• Pinned checkpoint
• Historical Ascend tutorial
• Current support notice