# inclusionAI/Ring-2.6-1T — vLLM serving recipe

> Ring-2.6-1T (BailingMoeV2_5) FP8 thinking model with 1T total / 50B active params, hybrid linear + MLA attention, 128K context

Configuration: hardware `h200` (8× H200) · strategy `multi_node_tp` (Multi-Node Tensor Parallel) · 2 nodes · variant `default` (FP8)

- Interactive page: https://recipes.vllm.ai/inclusionAI/Ring-2.6-1T
- This configuration as JSON: https://recipes.vllm.ai/inclusionAI/Ring-2.6-1T.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp`, `multi_node_tp_pp`, `multi_node_tep`, `multi_node_dep`, `pd_cluster` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.20.2.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

Run one command per node: the head (node rank 0) plus 1 worker. Each node's command differs — don't replay one on every node.

### Head (node 0)

```bash
export GLOO_SOCKET_IFNAME=$IFACE_NAME
export NCCL_SOCKET_IFNAME=$IFACE_NAME
vllm serve inclusionAI/Ring-2.6-1T \
  --trust-remote-code \
  --tensor-parallel-size 16 \
  --nnodes 2 \
  --node-rank 0 \
  --master-addr $HEAD_IP
```

### Node 1

```bash
export GLOO_SOCKET_IFNAME=$IFACE_NAME
export NCCL_SOCKET_IFNAME=$IFACE_NAME
vllm serve inclusionAI/Ring-2.6-1T \
  --trust-remote-code \
  --tensor-parallel-size 16 \
  --nnodes 2 \
  --node-rank 1 \
  --master-addr $HEAD_IP \
  --headless
```

Docker equivalent:

```bash
# Head
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e GLOO_SOCKET_IFNAME=$IFACE_NAME \
  -e NCCL_SOCKET_IFNAME=$IFACE_NAME \
  vllm/vllm-openai:latest inclusionAI/Ring-2.6-1T \
  --trust-remote-code \
  --tensor-parallel-size 16 \
  --nnodes 2 \
  --node-rank 0 \
  --master-addr $HEAD_IP
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Ring-2.6-1T](https://huggingface.co/inclusionAI/Ring-2.6-1T) is inclusionAI's
BailingMoeV2_5 FP8 trillion-scale **thinking** model (1T total / 50B active
parameters) with hybrid linear + MLA attention and a 128K context window.
It is the reasoning-focused counterpart to [Ling-2.6-1T](https://huggingface.co/inclusionAI/Ling-2.6-1T)
in the Ring 2.6 series and emits explicit `<think>...</think>` traces; the
chat template accepts a `reasoning_effort` field (default `high`).

vLLM 0.20.1 shipped the BailingMoeV2.5 MLA RoPE fix (vllm-project/vllm#41185)
that is load-bearing for this architecture — pin v0.20.2 or newer.

### Deployment Configurations

#### Docker (AMD MI300X / MI325X / MI355X, TP=8)

TP=8 fits the model-derived 128K context on an MI300X-class node; MI325X
and MI355X have larger per-GPU HBM and more headroom.

```bash
docker run --rm -it \
  --cap-add=SYS_PTRACE \
  --ipc=host \
  --privileged=true \
  --shm-size=128GB \
  --network=host \
  --device=/dev/kfd \
  --device=/dev/dri \
  --group-add video \
  -e VLLM_ROCM_USE_AITER=1 \
  vllm/vllm-openai-rocm:v0.28.0 \
    inclusionAI/Ring-2.6-1T \
    --tensor-parallel-size 8 \
    --trust-remote-code
```

#### Pip (NVIDIA B300, TP=8)

```bash
uv venv && source .venv/bin/activate
uv pip install -U vllm --torch-backend auto

vllm serve inclusionAI/Ring-2.6-1T \
  --trust-remote-code \
  --tensor-parallel-size 8
```

### Client Usage

#### Text Generation

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="inclusionAI/Ring-2.6-1T",
    messages=[{"role": "user", "content": "Prove there are infinitely many primes."}],
    max_tokens=4096,
    temperature=0.6,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
print(response.choices[0].message.content)
```

Set `reasoning_effort` to `low` / `medium` / `high` to control the depth of
thinking. The model wraps its reasoning trace in `<think>...</think>` before
emitting the final answer.

### References

- [Ring-2.6-1T on Hugging Face](https://huggingface.co/inclusionAI/Ring-2.6-1T)
- [vLLM PR #41185 — BailingMoeV2.5 MLA RoPE fix](https://github.com/vllm-project/vllm/pull/41185)
- [vLLM ROCm GPU install docs](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/?device=rocm)