vLLM/Recipes
Search
Qwen

Qwen/Qwen3-Embedding-0.6B

Qwen3-Embedding-0.6B — the smallest Qwen3 Embedding model (0.6B dense), for text embedding and retrieval; single-card pooling deployment on Ascend 910C with vLLM Ascend

Single-card pooling serving on Ascend 910C with vLLM Ascend

View on HuggingFace
dense0.6B32,768 ctxvLLM 0.9.2rc1+embedding
Default configuration: 16× Ascend 910C · Tensor Parallel · BF16
vllm serve Qwen/Qwen3-Embedding-0.6B \
  --runner pooling \
  --max-model-len 1024 \
  --tensor-parallel-size 1

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3-Embedding-0.6B.md.

Guide

Overview

Qwen3-Embedding-0.6B is the smallest dense model in the Qwen3 Embedding family, intended for low-cost text embedding and retrieval. This recipe serves the BF16 checkpoint in pooling mode on Ascend 910C.

Validation scope and support status

The main tutorial marks this model discontinued since vLLM Ascend v0.30.0 and points at the v0.23.0 documentation, so this recipe is pinned to the v0.23.0.post1-a3 image and only covers the single-node pooling scenario that document describes. Atlas 800 A3 is verified on that image: one Atlas 800T A3 node served this model with the rendered command, /v1/models and /v1/embeddings returned 1024-dimensional vectors, and MTEB BeIR/SciFact retrieval scored nDCG@10 0.6661 over 5,183 documents and 300 scored queries. Full MTEB coverage, long-context, and other hardware scenarios remain unverified.

Client Usage

curl -X POST http://localhost:8000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-Embedding-0.6B",
    "input": [
      "The capital of China is Beijing.",
      "Gravity is a force that attracts two bodies towards each other."
    ]
  }'

A successful response contains one 1024-dimensional embedding for each input.

Performance Evaluation

The generic Bench tab targets text generation. Use the embedding backend and endpoint for this pooling model:

vllm bench serve \
  --model Qwen/Qwen3-Embedding-0.6B \
  --backend openai-embeddings \
  --host 127.0.0.1 \
  --port 8000 \
  --dataset-name random \
  --endpoint /v1/embeddings \
  --random-input-len 200 \
  --num-prompts 100 \
  --max-concurrency 16 \
  --seed 1234 \
  --save-result \
  --result-dir ./

The recorded A3 compatibility run on the nightly build completed 100/100 requests at 219.13 requests/s, with mean end-to-end latency of 68.59 ms and P99 latency of 97.00 ms. These are reproducibility measurements, not official performance baselines.

References