reflection/Beam-1.0-501B-A23B

Overview

Beam is Reflection's sparse mixture-of-experts language model, with 501B total parameters and 23B active parameters per token. Its 52 layers combine 512-token sliding-window attention with NoPE full-attention layers, route each token through 8 of 256 experts (layers 0-2 use dense MLPs), and support a context length of 1,048,576 tokens.

Beam uses the Harmony message format, with an analysis channel for reasoning and a final channel for the answer. Generation stops on <|return|> at the end of an answer and on <|call|> at the end of a tool call.

Variants

Variant	Checkpoint	Weights on disk	Minimum GPU memory	Recommended hardware
NVFP4 (default)	reflection/Beam-1.0-501B-A23B-NVFP4	294 GB	353 GB	4x B200/B300 or one GB200/GB300 tray (TP4); 8x H100 (TP8, Marlin)
FP8	reflection/Beam-1.0-501B-A23B-FP8	503 GB	604 GB	8x H200 (TP8)
BF16	reflection/Beam-1.0-501B-A23B	1,003 GB	1,204 GB	8x B200/B300 (TP8)

Minimum GPU memory is the weights plus 20% headroom for activations, CUDA graphs and a working KV cache. It is the total across all GPUs, and it is the figure the variant selector above shows.

The NVFP4 checkpoint quantizes only the routed experts; attention, dense MLPs, the router, embeddings and the LM head stay in BF16. On Blackwell it runs W4A4; on Hopper, vLLM serves the FP4 experts through Marlin with BF16 activations.

Prerequisites

• vLLM: the vllm/vllm-openai:reflection-beam Docker image. Beam support has not shipped in a vLLM release yet.
• Hardware: NVIDIA GPUs. Beam's kernels, including a custom all-reduce used with expert parallelism, are CUDA-only.
• Transformers: the tokenizer uses the Transformers 5 format (TokenizersBackend).

Install vLLM

docker pull vllm/vllm-openai:reflection-beam

The vllm serve commands below run inside this image; the command builder above renders the full docker run line.

Launching the server

FP8 with TP8 on H200 or Blackwell

vllm serve reflection/Beam-1.0-501B-A23B-FP8 \
  --tensor-parallel-size 8 \
  --moe-backend triton \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam

This is the best-performing FP8 configuration measured on both H200 and B300. Set --moe-backend triton explicitly; vLLM otherwise selects CUTLASS for the FP8 MoE. vLLM's default attention backends, FlashAttention 3 on Hopper and FlashInfer with TRT-LLM kernels on Blackwell, are already the fastest measured.

NVFP4 with TP4 on Blackwell

vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam

At TP4 the weights take about 73 GB per GPU, leaving room for the full context window many times over; an 8-GPU node can host two replicas. The first start takes about 10 minutes because FlashInfer autotunes the MoE kernel. That is close to vLLM's default 600 s engine-ready timeout, so the generated command sets VLLM_ENGINE_READY_TIMEOUT_S=3600.

NVFP4 with TP8 on H100

vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --tensor-parallel-size 8 \
  --moe-backend marlin \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam

Set --moe-backend marlin explicitly; it is faster than the backend vLLM selects by default on Hopper. The FP8 checkpoint also fits on 8x H100 but leaves much less memory for the KV cache.

BF16 with TP8 on Blackwell

vllm serve reflection/Beam-1.0-501B-A23B \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam

Tuning

• KV cache dtype: the NVFP4 and FP8 commands use --kv-cache-dtype fp8, which halves KV memory per token (on 8x H200 with the FP8 checkpoint, about 14M to 28M tokens). The BF16 command keeps the KV cache in BF16.
• Parallelism: tensor parallelism is the recommended single-node layout. TEP (--enable-expert-parallel) and DEP (--data-parallel-size 8 --enable-expert-parallel) are also supported; DEP is better suited to very large batch sizes.

MegaMoE with attention DP (NVFP4, Blackwell)

For prefill-heavy, high-concurrency traffic, run attention data-parallel with expert parallelism on FlashInfer's CuTe DSL NVFP4 MegaMoE kernel, which overlaps the expert all-to-all with the expert GEMMs. Start from the NVFP4 command and replace the parallel and MoE settings:

vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend flashinfer_moe_ep_cutedsl \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam

• Only for DEP (--data-parallel-size with --enable-expert-parallel). It does not support TEP or pure TP.
• Use --data-parallel-size 4 on a GB200/GB300 tray or four B200/B300 GPUs, and 8 across a full 8-GPU node.
• For low-latency serving, stay on TP4 with flashinfer_trtllm.

DFlash2 speculative decoding

The DFlash2 drafter (reflection/Beam-1.0-501B-A23B-DFlash2) shares the target model's embeddings and LM head:

--speculative-config '{"method":"dflash","model":"reflection/Beam-1.0-501B-A23B-DFlash2","num_speculative_tokens":7,"attention_backend":"FLASH_ATTN","enable_adaptive_verification":true}'

With enable_adaptive_verification, vLLM predicts how likely each draft token is to be accepted and verifies only the ones worth the compute, rather than all 7 every step. It makes little difference at low concurrency and raises throughput under load. Remove it to verify the full draft every step.

The drafter supports up to 15 speculative tokens; 7 is recommended. It speeds up decoding most at low concurrency and adds about 1.7 GB per GPU, which reduces KV-cache capacity. Acceptance depends on natural-language input, so benchmark it with a real-text dataset rather than --dataset-name random.

Output constraints

By default, the beam output grammar constrains every assistant turn from its first token (reasoning, then an answer or a tool call). Set VLLM_BEAM_ALWAYS_CONSTRAIN_OUTPUT=0 to disable it.

Client usage

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/Beam-1.0-501B-A23B-NVFP4",
    "messages": [{"role": "user", "content": "Hello."}]
  }'

Set model to the checkpoint being served: reflection/Beam-1.0-501B-A23B-NVFP4, reflection/Beam-1.0-501B-A23B-FP8, or reflection/Beam-1.0-501B-A23B for BF16. /v1/models lists the served name.

Reasoning effort

Beam reasons at the maximum effort, max (512), unless the request sets reasoning_effort:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/Beam-1.0-501B-A23B-NVFP4",
    "messages": [{"role": "user", "content": "What is 17 * 19?"}],
    "reasoning_effort": "low"
  }'

The Rust frontend (VLLM_USE_RUST_FRONTEND=1) serves /v1/chat/completions and /v1/completions, but not yet /v1/responses or the Anthropic-compatible /v1/messages (both return 404). Use the default Python frontend for those APIs.

References

• Beam model card
• Beam FP8
• Beam NVFP4
• DFlash2 drafter