# IFM/K2-Horizon-0.9B — vLLM serving recipe

> Compact dense model for mathematics, code, instruction following, and STEM tasks

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `reasoning`, `tool_calling`

- Interactive page: https://recipes.vllm.ai/IFM/K2-Horizon-0.9B
- This configuration as JSON: https://recipes.vllm.ai/IFM/K2-Horizon-0.9B.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp`, `multi_node_tp`, `multi_node_tp_pp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `reasoning`, `tool_calling` (default: `reasoning`, `tool_calling`)

## Install

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve IFM/K2-Horizon-0.9B \
  --trust-remote-code \
  --chat-template-content-format string \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.85 \
  --tensor-parallel-size 1 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest IFM/K2-Horizon-0.9B \
  --trust-remote-code \
  --chat-template-content-format string \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.85 \
  --tensor-parallel-size 1 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

K2-Horizon-0.9B is a compact dense model in the IFM K2-Horizon
family, with 1.08B stored parameters including embeddings and a 131,072-token
context window.

K2-Horizon is an open-weight IFM model family built for transparent
foundation-model research, staged checkpoint analysis, and practical
deployment. The series spans compact dense models such as K2-Horizon-0.9B
for local experimentation, dense mid-size and large models for high-quality
research workloads, and mixture-of-expert (MoE) models for higher-capacity
serving.

### Launch Commands

K2-Horizon-0.9B's `config.json` already enables YaRN, so the full 131,072-token context needs no RoPE override.

```bash
vllm serve IFM/K2-Horizon-0.9B \
  --trust-remote-code \
  --chat-template-content-format string \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.85 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon
```

Lower `--max-model-len` (e.g. 8192) to reduce KV-cache memory.

### Client usage

```python
from openai import OpenAI

client = OpenAI(
    api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=3600
)

resp = client.chat.completions.create(
    model="IFM/K2-Horizon-0.9B",
    messages=[{"role": "user", "content": "Give me three primes above 100."}],
    temperature=1.0, top_p=0.95, max_tokens=2048,
)
print(resp.choices[0].message.content)
```

#### Thinking modes

The model supports selectable thinking through `chat_template_kwargs`, per
request or server-wide via `--default-chat-template-kwargs`:

- `{"reasoning_effort": "high"}` — full thinking (default).
- `{"reasoning_effort": "medium"}` — faster thinking.
- `{"reasoning_effort": "low"}` — fastest thinking.

### References

- [IFM/K2-Horizon-0.9B on Hugging Face](https://huggingface.co/IFM/K2-Horizon-0.9B)
- [K2 Horizon Collection](https://huggingface.co/collections/IFM/k2-horizon)