internlm/Intern-S2-397B

Overview

Intern-S2-397B is Shanghai AI Laboratory's flagship multimodal foundation model for scientific intelligence and long-horizon agents. It packs 397B total / 17B active parameters across 512 experts on the Qwen3.5 MoE architecture (Qwen3_5MoeForConditionalGeneration) with hybrid linear/full attention, a 262K context window, and a built-in shared-weight MTP head for speculative decoding. An official FP8 checkpoint (internlm/Intern-S2-397B-FP8) halves the memory footprint and is the configuration the vendor validates.

Beyond chat and reasoning, it handles image and video inputs and improves agent capabilities for scientific workflows. Time-series inference exists but is currently LMDeploy-only (see Troubleshooting).

Prerequisites

• vLLM version: 0.22.1 or newer (per the official deployment guide)
• Hardware (FP8, recommended): 8x H100 or 8x H200
• Hardware (BF16): 8x H200 (141 GB each)
• Trust remote code: required — the repo ships a custom tokenizer (InternS1Tokenizer)
• MoE kernel env: the deployment guide disables DeepGEMM (VLLM_USE_DEEP_GEMM=0, already set by this recipe) and selects the FlashInfer TRTLLM MoE backend on NVIDIA (applied automatically when you pick an NVIDIA GPU above)

Install vLLM

uv venv
source .venv/bin/activate
uv pip install -U "vllm>=0.22.1" --torch-backend=auto

Launch commands

Recommended — official FP8 checkpoint, TP=8

vllm serve internlm/Intern-S2-397B-FP8 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --mm-encoder-tp-mode data

BF16 (8x H200)

vllm serve internlm/Intern-S2-397B \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --mm-encoder-tp-mode data

With MTP speculative decoding

vllm serve internlm/Intern-S2-397B-FP8 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --mm-encoder-tp-mode data \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Long-context (YaRN, up to ~1M)

The base config sets max_position_embeddings = 262144. For longer contexts, turn on the Long Context feature above — it applies the deployment guide's YaRN RoPE override (factor 4.0) and raises --max-model-len to 1,010,000. Static YaRN holds the scaling factor constant regardless of input length, so leave it off unless you actually serve long prompts; for a ~524K workload, halve the factor to 2.0.

Client Usage

Recommended sampling parameters from the model card:

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="internlm/Intern-S2-397B-FP8",
    messages=[{"role": "user", "content": "Design a synthesis route for paracetamol."}],
    temperature=0.8,
    top_p=0.95,
    max_tokens=32768,
    extra_body={
        "top_k": 50,
        "min_p": 0.0,
        "spaces_between_special_tokens": False,
    },
)
print(resp.choices[0].message.content)

Toggle thinking mode

Thinking is enabled by default. Disable it per request:

resp = client.chat.completions.create(
    model="internlm/Intern-S2-397B-FP8",
    messages=[{"role": "user", "content": "What is AGI?"}],
    temperature=0.8,
    top_p=0.95,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

The model card notes: do not disable thinking mode for agentic tasks.

Troubleshooting

• OOM at full 262K context: drop --max-model-len to 65536 or 131072, or lower --gpu-memory-utilization headroom.
• DeepGEMM warmup failures / slow startup: this recipe already exports VLLM_DEEP_GEMM_WARMUP=skip and VLLM_USE_DEEP_GEMM=0 per the deployment guide — if you launch outside the builder, set them yourself.
• Time-series inputs (time_series_url): currently supported only on LMDeploy, not vLLM — see the model card.
• Wrong parser name: LMDeploy examples use --tool-call-parser interns2-preview; on vLLM the correct parser is qwen3_coder (already wired into the Tool Calling feature above).

References

• Model card
• FP8 checkpoint
• Deployment guide
• Intern-S1 GitHub
• vLLM documentation