{ "hf_id": "Qwen/Qwen3-Embedding-0.6B", "meta": { "title": "Qwen3-Embedding-0.6B", "slug": "Qwen3-Embedding-0.6B", "provider": "Qwen", "description": "Qwen3-Embedding-0.6B — the smallest Qwen3 Embedding model (0.6B dense), for text embedding and retrieval; single-card pooling deployment on Ascend 910C with vLLM Ascend", "date_added": "2026-10-14", "date_updated": "2026-11-11", "tasks": [ "embedding" ], "performance_headline": "Single-card pooling serving on Ascend 910C with vLLM Ascend", "default_hardware": "ascend_910c", "hardware": { "ascend_910c": "verified" } }, "recommended_command": { "hardware": "ascend_910c", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3", "command": "vllm serve Qwen/Qwen3-Embedding-0.6B \\\n --runner pooling \\\n --max-model-len 1024 \\\n --tensor-parallel-size 1", "argv": [ "vllm", "serve", "Qwen/Qwen3-Embedding-0.6B", "--runner", "pooling", "--max-model-len", "1024", "--tensor-parallel-size", "1" ], "docker_command": "docker run --device /dev/davinci0 --device /dev/davinci1 --device /dev/davinci2 --device /dev/davinci3 \\\n --device /dev/davinci4 --device /dev/davinci5 --device /dev/davinci6 --device /dev/davinci7 \\\n --device /dev/davinci8 --device /dev/davinci9 --device /dev/davinci10 --device /dev/davinci11 \\\n --device /dev/davinci12 --device /dev/davinci13 --device /dev/davinci14 --device /dev/davinci15 \\\n --device /dev/davinci_manager --device /dev/hisi_hdc \\\n -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \\\n -v /etc/ascend_install.info:/etc/ascend_install.info \\\n -v /usr/local/dcmi:/usr/local/dcmi \\\n -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \\\n --privileged --net=host --shm-size=16g \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n quay.io/ascend/vllm-ascend:v0.23.0.post1-a3 \\\n vllm serve Qwen/Qwen3-Embedding-0.6B \\\n --runner pooling \\\n --max-model-len 1024 \\\n --tensor-parallel-size 1", "docker_argv": [ "docker", "run", "--device", "/dev/davinci0", "--device", "/dev/davinci1", "--device", "/dev/davinci2", "--device", "/dev/davinci3", "--device", "/dev/davinci4", "--device", "/dev/davinci5", "--device", "/dev/davinci6", "--device", "/dev/davinci7", "--device", "/dev/davinci8", "--device", "/dev/davinci9", "--device", "/dev/davinci10", "--device", "/dev/davinci11", "--device", "/dev/davinci12", "--device", "/dev/davinci13", "--device", "/dev/davinci14", "--device", "/dev/davinci15", "--device", "/dev/davinci_manager", "--device", "/dev/hisi_hdc", "-v", "/usr/local/Ascend/driver:/usr/local/Ascend/driver", "-v", "/etc/ascend_install.info:/etc/ascend_install.info", "-v", "/usr/local/dcmi:/usr/local/dcmi", "-v", "/usr/local/bin/npu-smi:/usr/local/bin/npu-smi", "--privileged", "--net=host", "--shm-size", "16g", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3", "vllm", "serve", "Qwen/Qwen3-Embedding-0.6B", "--runner", "pooling", "--max-model-len", "1024", "--tensor-parallel-size", "1" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "Huawei", "generation": "npu", "display_name": "Ascend 910C", "description": "Huawei Ascend 910C · 16× NPU · 64 GB HBM each · Atlas 800 A3 · vLLM Ascend backend", "gpu_count": 16, "vram_gb": 1024, "multi_node": false, "restricted": true }, "by_hardware": { "ascend_910c": "/Qwen/Qwen3-Embedding-0.6B/hw/ascend_910c.json" } }, "model": { "model_id": "Qwen/Qwen3-Embedding-0.6B", "min_vllm_version": "0.9.2rc1", "architecture": "dense", "parameter_count": "0.6B", "active_parameters": "0.6B", "context_length": 32768, "base_args": [ "--runner", "pooling" ], "base_env": {}, "docker_image": { "ascend_910c": "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3" }, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 2, "supported_hardware": [ "ascend_910c" ], "hardware_overrides": { "ascend_910c": { "docker_image": "quay.io/ascend/vllm-ascend:v0.23.0.post1-a3", "docker_serve_command": "vllm serve", "extra_args": [ "--max-model-len", "1024" ] } }, "description": "BF16 checkpoint for single-node Ascend 910C pooling; main compatibility smoke test only" } }, "hardware_overrides": {}, "guide": "## Overview\n\nQwen3-Embedding-0.6B is the smallest dense model in the Qwen3 Embedding family, intended\nfor low-cost text embedding and retrieval. This recipe serves the BF16 checkpoint in\npooling mode on Ascend 910C.\n\n## Validation scope and support status\n\nThe main tutorial marks this model discontinued since vLLM Ascend v0.30.0 and points at the\nv0.23.0 documentation, so this recipe is pinned to the `v0.23.0.post1-a3` image and only covers the\nsingle-node pooling scenario that document describes. Atlas 800 A3 is `verified` on that image:\none Atlas 800T A3 node served this model with the rendered command, `/v1/models` and\n`/v1/embeddings` returned 1024-dimensional vectors, and MTEB BeIR/SciFact retrieval scored\n**nDCG@10 0.6661** over 5,183 documents and 300 scored queries. Full MTEB coverage,\nlong-context, and other hardware scenarios remain unverified.\n\n## Client Usage\n\n```bash\ncurl -X POST http://localhost:8000/v1/embeddings \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"Qwen/Qwen3-Embedding-0.6B\",\n \"input\": [\n \"The capital of China is Beijing.\",\n \"Gravity is a force that attracts two bodies towards each other.\"\n ]\n }'\n```\n\nA successful response contains one 1024-dimensional embedding for each input.\n\n## Performance Evaluation\n\nThe generic Bench tab targets text generation. Use the embedding backend and endpoint\nfor this pooling model:\n\n```bash\nvllm bench serve \\\n --model Qwen/Qwen3-Embedding-0.6B \\\n --backend openai-embeddings \\\n --host 127.0.0.1 \\\n --port 8000 \\\n --dataset-name random \\\n --endpoint /v1/embeddings \\\n --random-input-len 200 \\\n --num-prompts 100 \\\n --max-concurrency 16 \\\n --seed 1234 \\\n --save-result \\\n --result-dir ./\n```\n\nThe recorded A3 compatibility run on the nightly build completed 100/100 requests at\n219.13 requests/s, with mean\nend-to-end latency of 68.59 ms and P99 latency of 97.00 ms. These are reproducibility\nmeasurements, not official performance baselines.\n\n## References\n\n- [Historical vLLM Ascend tutorial (v0.23.0)](https://docs.vllm.ai/projects/ascend/en/v0.23.0/tutorials/models/Qwen3-Embedding.html)\n- [Main tutorial support status](https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/tutorials/models/Qwen3-Embedding.md)\n- [Qwen3-Embedding-0.6B model card](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B)\n- [Qwen3-Embedding-0.6B on ModelScope](https://www.modelscope.cn/models/Qwen/Qwen3-Embedding-0.6B)\n", "source": { "source": "vllm-ascend", "repo": "vllm-project/vllm-ascend", "ref": "main", "sha": "931bfd948e04a89b735a2f78520093aa9dbc49f0", "path": "recipes/models/Qwen/Qwen3-Embedding-0.6B.yaml", "edit_url": "https://github.com/vllm-project/vllm-ascend/edit/main/recipes/models/Qwen/Qwen3-Embedding-0.6B.yaml", "overlays": [] } }