{ "hf_id": "IFM/K2-Horizon-3.7B", "meta": { "title": "K2-Horizon-3.7B", "slug": "k2-horizon-3.7b", "provider": "IFM", "description": "Dense model for long-context research, serving, and downstream adaptation", "date_added": "2026-08-31", "date_updated": "2026-09-02", "tasks": [ "text" ], "performance_headline": "5.06B stored parameters including embeddings, with up to 512K context", "related_recipes": [ "IFM/K2-Horizon-0.9B", "IFM/K2-Horizon-7B" ], "hardware": { "h200": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tp", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve IFM/K2-Horizon-3.7B \\\n --trust-remote-code \\\n --chat-template-content-format string \\\n --dtype bfloat16 \\\n --tensor-parallel-size 1 \\\n --reasoning-parser k2_horizon \\\n --enable-auto-tool-choice \\\n --tool-call-parser k2_horizon", "argv": [ "vllm", "serve", "IFM/K2-Horizon-3.7B", "--trust-remote-code", "--chat-template-content-format", "string", "--dtype", "bfloat16", "--tensor-parallel-size", "1", "--reasoning-parser", "k2_horizon", "--enable-auto-tool-choice", "--tool-call-parser", "k2_horizon" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest IFM/K2-Horizon-3.7B \\\n --trust-remote-code \\\n --chat-template-content-format string \\\n --dtype bfloat16 \\\n --tensor-parallel-size 1 \\\n --reasoning-parser k2_horizon \\\n --enable-auto-tool-choice \\\n --tool-call-parser k2_horizon", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "IFM/K2-Horizon-3.7B", "--trust-remote-code", "--chat-template-content-format", "string", "--dtype", "bfloat16", "--tensor-parallel-size", "1", "--reasoning-parser", "k2_horizon", "--enable-auto-tool-choice", "--tool-call-parser", "k2_horizon" ], "strategy_spec": { "name": "single_node_tp", "deploy_type": "single_node", "display_name": "Tensor Parallel", "short_label": "TP", "orientation": "latency", "description": "Single-node tensor parallel. Splits the model across all local GPUs. TP size is set to the GPU count at deploy time. The simplest multi-GPU strategy — works for all model architectures.\n", "hardware_match": { "min_gpus": 1, "max_gpus": 8, "multi_node": false }, "vllm_args": [], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e · 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tp": "/IFM/K2-Horizon-3.7B/strategies/multi_node_tp.json", "multi_node_tp_pp": "/IFM/K2-Horizon-3.7B/strategies/multi_node_tp_pp.json", "kv_store_centralized_mooncake": "/IFM/K2-Horizon-3.7B/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/IFM/K2-Horizon-3.7B/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/IFM/K2-Horizon-3.7B/hw/h200.json", "h100": "/IFM/K2-Horizon-3.7B/hw/h100.json", "b200": "/IFM/K2-Horizon-3.7B/hw/b200.json", "gb200": "/IFM/K2-Horizon-3.7B/hw/gb200.json", "b300": "/IFM/K2-Horizon-3.7B/hw/b300.json", "gb300": "/IFM/K2-Horizon-3.7B/hw/gb300.json", "mi300x": "/IFM/K2-Horizon-3.7B/hw/mi300x.json", "mi325x": "/IFM/K2-Horizon-3.7B/hw/mi325x.json", "mi350x": "/IFM/K2-Horizon-3.7B/hw/mi350x.json", "mi355x": "/IFM/K2-Horizon-3.7B/hw/mi355x.json" } }, "model": { "model_id": "IFM/K2-Horizon-3.7B", "architecture": "dense", "parameter_count": "5.06B", "active_parameters": "5.06B", "context_length": 524288, "base_args": [ "--trust-remote-code", "--chat-template-content-format", "string", "--dtype", "bfloat16" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "reasoning": { "description": "Extract K2-Horizon reasoning into message.reasoning", "args": [ "--reasoning-parser", "k2_horizon" ] }, "tool_calling": { "description": "Enable automatic tool choice with the K2-Horizon parser", "args": [ "--enable-auto-tool-choice", "--tool-call-parser", "k2_horizon" ] } }, "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 13, "description": "BF16 serving runtime with a 131K context baseline" } }, "hardware_overrides": {}, "guide": "## Overview\n\nK2-Horizon-3.7B is a small dense model in the IFM K2-Horizon family,\nwith 5.06B stored parameters including embeddings and support for up to\n524,288 tokens of context.\n\nK2-Horizon is an open-weight IFM model family built for transparent\nfoundation-model research, staged checkpoint analysis, and practical\ndeployment. The series spans compact dense models for local experimentation,\ndense mid-size and large models for high-quality research workloads, and\nmixture-of-expert (MoE) models for higher-capacity serving.\n\n## Launch Commands\n\n```bash\nvllm serve IFM/K2-Horizon-3.7B \\\n --trust-remote-code \\\n --chat-template-content-format string \\\n --dtype bfloat16 \\\n --reasoning-parser k2_horizon \\\n --enable-auto-tool-choice \\\n --tool-call-parser k2_horizon\n```\n\n## Client usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\", timeout=3600\n)\n\nresp = client.chat.completions.create(\n model=\"IFM/K2-Horizon-3.7B\",\n messages=[{\"role\": \"user\", \"content\": \"Give me three primes above 100.\"}],\n temperature=1.0, top_p=0.95, max_tokens=2048,\n)\nprint(resp.choices[0].message.content)\n```\n\n### Thinking modes\n\nThe model supports selectable thinking through `chat_template_kwargs`, per\nrequest or server-wide via `--default-chat-template-kwargs`:\n\n- `{\"reasoning_effort\": \"high\"}` — full thinking (default).\n- `{\"reasoning_effort\": \"medium\"}` — faster thinking.\n- `{\"reasoning_effort\": \"low\"}` — fastest thinking.\n\n## References\n\n- [IFM/K2-Horizon-3.7B on Hugging Face](https://huggingface.co/IFM/K2-Horizon-3.7B)\n- [K2 Horizon Collection](https://huggingface.co/collections/IFM/k2-horizon)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb", "path": "recipes/models/IFM/K2-Horizon-3.7B.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/IFM/K2-Horizon-3.7B.yaml", "overlays": [] } }