# Qwen/Qwen3-Embedding-0.6B — vLLM serving recipe

> Qwen3-Embedding-0.6B — the smallest Qwen3 Embedding model (0.6B dense), for text embedding and retrieval; single-card pooling deployment on Ascend 910C with vLLM Ascend

Configuration: hardware `ascend_910c` (16× Ascend 910C) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://recipes.vllm.ai/Qwen/Qwen3-Embedding-0.6B
- This configuration as JSON: https://recipes.vllm.ai/Qwen/Qwen3-Embedding-0.6B.json?hardware=ascend_910c

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `ascend_910c`
- `variant`: `default`
- `strategy`: `single_node_tp` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.9.2rc1.

Docker image: `quay.io/ascend/vllm-ascend:v0.23.0.post1-a3`

## Serve

```bash
vllm serve Qwen/Qwen3-Embedding-0.6B \
  --runner pooling \
  --max-model-len 1024 \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --device /dev/davinci0 --device /dev/davinci1 --device /dev/davinci2 --device /dev/davinci3 \
  --device /dev/davinci4 --device /dev/davinci5 --device /dev/davinci6 --device /dev/davinci7 \
  --device /dev/davinci8 --device /dev/davinci9 --device /dev/davinci10 --device /dev/davinci11 \
  --device /dev/davinci12 --device /dev/davinci13 --device /dev/davinci14 --device /dev/davinci15 \
  --device /dev/davinci_manager --device /dev/hisi_hdc \
  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
  -v /etc/ascend_install.info:/etc/ascend_install.info \
  -v /usr/local/dcmi:/usr/local/dcmi \
  -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
  --privileged --net=host --shm-size=16g \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  quay.io/ascend/vllm-ascend:v0.23.0.post1-a3 \
  vllm serve Qwen/Qwen3-Embedding-0.6B \
  --runner pooling \
  --max-model-len 1024 \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Qwen3-Embedding-0.6B is the smallest dense model in the Qwen3 Embedding family, intended
for low-cost text embedding and retrieval. This recipe serves the BF16 checkpoint in
pooling mode on Ascend 910C.

### Validation scope and support status

The main tutorial marks this model discontinued since vLLM Ascend v0.30.0 and points at the
v0.23.0 documentation, so this recipe is pinned to the `v0.23.0.post1-a3` image and only covers the
single-node pooling scenario that document describes. Atlas 800 A3 is `verified` on that image:
one Atlas 800T A3 node served this model with the rendered command, `/v1/models` and
`/v1/embeddings` returned 1024-dimensional vectors, and MTEB BeIR/SciFact retrieval scored
**nDCG@10 0.6661** over 5,183 documents and 300 scored queries. Full MTEB coverage,
long-context, and other hardware scenarios remain unverified.

### Client Usage

```bash
curl -X POST http://localhost:8000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-Embedding-0.6B",
    "input": [
      "The capital of China is Beijing.",
      "Gravity is a force that attracts two bodies towards each other."
    ]
  }'
```

A successful response contains one 1024-dimensional embedding for each input.

### Performance Evaluation

The generic Bench tab targets text generation. Use the embedding backend and endpoint
for this pooling model:

```bash
vllm bench serve \
  --model Qwen/Qwen3-Embedding-0.6B \
  --backend openai-embeddings \
  --host 127.0.0.1 \
  --port 8000 \
  --dataset-name random \
  --endpoint /v1/embeddings \
  --random-input-len 200 \
  --num-prompts 100 \
  --max-concurrency 16 \
  --seed 1234 \
  --save-result \
  --result-dir ./
```

The recorded A3 compatibility run on the nightly build completed 100/100 requests at
219.13 requests/s, with mean
end-to-end latency of 68.59 ms and P99 latency of 97.00 ms. These are reproducibility
measurements, not official performance baselines.

### References

- [Historical vLLM Ascend tutorial (v0.23.0)](https://docs.vllm.ai/projects/ascend/en/v0.23.0/tutorials/models/Qwen3-Embedding.html)
- [Main tutorial support status](https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/tutorials/models/Qwen3-Embedding.md)
- [Qwen3-Embedding-0.6B model card](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B)
- [Qwen3-Embedding-0.6B on ModelScope](https://www.modelscope.cn/models/Qwen/Qwen3-Embedding-0.6B)