{ "hf_id": "IFM/K2-Horizon-MoVA-36B-A4B", "meta": { "title": "K2-Horizon-MoVA-36B-A4B", "slug": "k2-horizon-mova-36b-a4b", "provider": "IFM", "description": "Sparse MoVA and MoE long-context model for research and production-style serving experiments", "date_added": "2026-08-31", "date_updated": "2026-09-03", "tasks": [ "text" ], "performance_headline": "37.44B stored parameters including embeddings / 5.95B active parameters per token", "related_recipes": [ "IFM/K2-Horizon-32B", "IFM/K2-Horizon-375B-A23B" ], "hardware": { "h200": "verified" } }, "recommended_command": { "hardware": "h200", "strategy": "single_node_tep", "variant": "default", "node_count": 1, "deploy_type": "single_node", "env": {}, "docker_image": "vllm/vllm-openai:latest", "command": "vllm serve IFM/K2-Horizon-MoVA-36B-A4B \\\n --trust-remote-code \\\n --chat-template-content-format string \\\n --dtype bfloat16 \\\n --enable-expert-parallel \\\n --tensor-parallel-size 8 \\\n --reasoning-parser k2_horizon \\\n --enable-auto-tool-choice \\\n --tool-call-parser k2_horizon", "argv": [ "vllm", "serve", "IFM/K2-Horizon-MoVA-36B-A4B", "--trust-remote-code", "--chat-template-content-format", "string", "--dtype", "bfloat16", "--enable-expert-parallel", "--tensor-parallel-size", "8", "--reasoning-parser", "k2_horizon", "--enable-auto-tool-choice", "--tool-call-parser", "k2_horizon" ], "docker_command": "docker run --gpus all \\\n --privileged --ipc=host -p 8000:8000 \\\n -v ~/.cache/huggingface:/root/.cache/huggingface \\\n vllm/vllm-openai:latest IFM/K2-Horizon-MoVA-36B-A4B \\\n --trust-remote-code \\\n --chat-template-content-format string \\\n --dtype bfloat16 \\\n --enable-expert-parallel \\\n --tensor-parallel-size 8 \\\n --reasoning-parser k2_horizon \\\n --enable-auto-tool-choice \\\n --tool-call-parser k2_horizon", "docker_argv": [ "docker", "run", "--gpus", "all", "--privileged", "--ipc=host", "-p", "8000:8000", "-v", "~/.cache/huggingface:/root/.cache/huggingface", "vllm/vllm-openai:latest", "IFM/K2-Horizon-MoVA-36B-A4B", "--trust-remote-code", "--chat-template-content-format", "string", "--dtype", "bfloat16", "--enable-expert-parallel", "--tensor-parallel-size", "8", "--reasoning-parser", "k2_horizon", "--enable-auto-tool-choice", "--tool-call-parser", "k2_horizon" ], "strategy_spec": { "name": "single_node_tep", "deploy_type": "single_node", "display_name": "Tensor + Expert Parallel", "short_label": "TEP", "orientation": "balanced", "description": "Single-node TEP. TP splits dense layers and EP splits expert layers across local GPUs. TP must be set to GPU count to avoid OOM from replicated dense layers. For MoE models only.\n", "hardware_match": { "min_gpus": 2, "max_gpus": 8, "multi_node": false, "architecture": "moe" }, "vllm_args": [ "--enable-expert-parallel" ], "parallel_flag": "--tensor-parallel-size" }, "hardware_profile": { "brand": "NVIDIA", "generation": "hopper", "compute_arch": "sm90", "display_name": "H200", "description": "NVIDIA H200 SXM 141 GB HBM3e ยท 8-GPU HGX node", "gpu_count": 8, "vram_gb": 1128, "multi_node": false }, "alternatives": { "multi_node_tep": "/IFM/K2-Horizon-MoVA-36B-A4B/strategies/multi_node_tep.json", "multi_node_tp_pp": "/IFM/K2-Horizon-MoVA-36B-A4B/strategies/multi_node_tp_pp.json", "kv_store_centralized_mooncake": "/IFM/K2-Horizon-MoVA-36B-A4B/strategies/kv_store_centralized_mooncake.json", "kv_store_distributed_mooncake": "/IFM/K2-Horizon-MoVA-36B-A4B/strategies/kv_store_distributed_mooncake.json" }, "by_hardware": { "h200": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/h200.json", "h100": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/h100.json", "b200": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/b200.json", "gb200": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/gb200.json", "b300": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/b300.json", "gb300": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/gb300.json", "mi300x": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/mi300x.json", "mi325x": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/mi325x.json", "mi350x": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/mi350x.json", "mi355x": "/IFM/K2-Horizon-MoVA-36B-A4B/hw/mi355x.json" } }, "model": { "model_id": "IFM/K2-Horizon-MoVA-36B-A4B", "architecture": "moe", "parameter_count": "37.44B", "active_parameters": "5.95B", "context_length": 524288, "base_args": [ "--trust-remote-code", "--chat-template-content-format", "string", "--dtype", "bfloat16", "--enable-expert-parallel" ], "base_env": {}, "install": { "pip": { "command": "uv venv\nsource .venv/bin/activate\nuv pip install -U vllm --torch-backend auto" }, "docker": { "command": "docker pull vllm/vllm-openai:latest" } } }, "features": { "reasoning": { "description": "Extract K2-Horizon reasoning into message.reasoning", "args": [ "--reasoning-parser", "k2_horizon" ] }, "tool_calling": { "description": "Enable automatic tool choice with the K2-Horizon parser", "args": [ "--enable-auto-tool-choice", "--tool-call-parser", "k2_horizon" ] } }, "variants": { "default": { "precision": "bf16", "vram_minimum_gb": 90, "description": "BF16 serving runtime with native vLLM execution" } }, "hardware_overrides": {}, "guide": "## Overview\n\nK2-Horizon-MoVA-36B-A4B is a mixture-of-expert (MoE) model in the IFM K2-Horizon\nfamily. It combines MoVA attention and MoE layers, with 37.44B stored parameters\nincluding embeddings, 5.95B active parameters per token, and support for up\nto 524,288 tokens of context.\n\nK2-Horizon is an open-weight IFM model family built for transparent\nfoundation-model research, staged checkpoint analysis, and practical\ndeployment. The series spans compact dense models for local experimentation,\ndense mid-size and large models for high-quality research workloads, and\nmixture-of-expert (MoE) models such as K2-Horizon-MoVA-36B-A4B for higher-capacity\nserving.\n\n## Launch Commands\n\n```bash\nvllm serve IFM/K2-Horizon-MoVA-36B-A4B \\\n --tensor-parallel-size 2 \\\n --trust-remote-code \\\n --chat-template-content-format string \\\n --dtype bfloat16 \\\n --enable-expert-parallel \\\n --reasoning-parser k2_horizon \\\n --enable-auto-tool-choice \\\n --tool-call-parser k2_horizon\n```\n\n## Client usage\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n api_key=\"EMPTY\", base_url=\"http://localhost:8000/v1\", timeout=3600\n)\n\nresp = client.chat.completions.create(\n model=\"IFM/K2-Horizon-MoVA-36B-A4B\",\n messages=[{\"role\": \"user\", \"content\": \"Give me three primes above 100.\"}],\n temperature=1.0, top_p=0.95, max_tokens=2048,\n)\nprint(resp.choices[0].message.content)\n```\n\n### Thinking modes\n\nThe model supports selectable thinking through `chat_template_kwargs`, per\nrequest or server-wide via `--default-chat-template-kwargs`:\n\n- `{\"reasoning_effort\": \"high\"}` โ€” full thinking (default).\n- `{\"reasoning_effort\": \"medium\"}` โ€” faster thinking.\n- `{\"reasoning_effort\": \"low\"}` โ€” fastest thinking.\n\n## References\n\n- [IFM/K2-Horizon-MoVA-36B-A4B on Hugging Face](https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B)\n- [K2 Horizon Collection](https://huggingface.co/collections/IFM/k2-horizon)\n", "source": { "source": "vllm", "repo": "vllm-project/vllm", "ref": "main", "sha": "564ac0de1b228568cafcbfbe8ccaccc82d6023eb", "path": "recipes/models/IFM/K2-Horizon-MoVA-36B-A4B.yaml", "edit_url": "https://github.com/vllm-project/vllm/edit/main/recipes/models/IFM/K2-Horizon-MoVA-36B-A4B.yaml", "overlays": [] } }