--- title: "K2-Horizon-MoVA-36B GPU Requirements: VRAM & Cluster Sizing" slug: "k2-horizon-mova-36b-gpu-requirements" meta_description: "Calculate VRAM, MoVA attention memory, and cloud cluster sizing for IFM K2-Horizon-MoVA-36B-A4B across RTX 5090, A100, H100, and AMD MI325X servers." schema_type: "article" author: "ahmad-nugraha" primary_keyword: "k2-horizon-mova-36b gpu requirements" secondary_keywords: - "k2 horizon mova 36b vram" - "k2-horizon-mova hardware sizing" - "host ifm k2 horizon" - "k2 horizon 512k context vram" - "k2 horizon mova cloud cost" search_intent: "informational" commercial: true published_at: "2026-09-15" updated_at: "2026-09-15" status: "published" human_reviewed_by: "ahmad-nugraha" human_reviewed_at: "2026-09-15" sources: - id: ifm-k2-horizon-mova-hf title: "K2-Horizon-MoVA-36B-A4B Official Model Card, Config & Checkpoint Repository" publisher: "Institute of Foundation Models (IFM) / Hugging Face" url: "https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B" accessed_at: "2026-09-15" source_type: primary - id: ifm-vllm-recipe title: "Official vLLM Serving Recipe for K2-Horizon MoVA Models" publisher: "vLLM Project" url: "https://recipes.vllm.ai/IFM" accessed_at: "2026-09-15" source_type: technical - id: ifm-sglang-cookbook title: "SGLang Autoregressive Model Cookbook: IFM K2-Horizon" publisher: "SGLang Documentation" url: "https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon" accessed_at: "2026-09-15" source_type: technical - id: nvidia-h100-datasheet title: "NVIDIA H100 Tensor Core GPU Architecture Datasheet" publisher: "NVIDIA Corporation" url: "https://www.nvidia.com/en-us/data-center/h100/" accessed_at: "2026-09-15" source_type: primary - id: amd-mi325x-datasheet title: "AMD Instinct MI325X Accelerator Datasheet & Specifications" publisher: "Advanced Micro Devices, Inc." url: "https://www.amd.com/en/products/accelerators/instinct/mi300/mi325x.html" accessed_at: "2026-09-15" source_type: primary --- Understanding the **k2-horizon-mova-36b gpu requirements** is essential for machine learning teams deploying the Institute of Foundation Models (IFM) flagship open-weights architecture.[cite id="ifm-k2-horizon-mova-hf"] Released under the Apache 2.0 license, K2-Horizon-MoVA-36B-A4B introduces a compound sparse design that breaks conventional hardware sizing rules: the model stores 36 billion total parameters in high-bandwidth memory, yet activates only 4 billion parameters per token during forward passes.[cite id="ifm-k2-horizon-mova-hf"] This decoupled architecture produces a distinct operational trade-off. Token generation latency is as fast as a sub-7B parameter model, but static weights demand 72 GB of VRAM in unquantized 16-bit precision.[cite id="ifm-k2-horizon-mova-hf"] Furthermore, because the model features a native 524,288-token (512K) context window, memory demands at long context lengths are governed by Key-Value (KV) cache allocation rather than parameter weights.[cite id="ifm-k2-horizon-mova-hf"] This guide provides concrete mathematical sizing formulas, verified VRAM requirements across precision tiers, multi-GPU topology recommendations, and validated deployment manifests for production serving. [key-takeaways] - **Base memory floor**: Unquantized BF16 weights require 72.0 GB of VRAM, FP8 weights require 36.0 GB, and 4-bit quantized formats (AWQ, GPTQ, GGUF) require 19.8 GB.[cite id="ifm-k2-horizon-mova-hf"] - **Active compute profile**: With only 4.0B active parameters routed across 100 FFN experts (8 active + 1 shared) and 64 MoVA value experts (4 active), single-token compute latency matches lightweight dense models.[cite id="ifm-k2-horizon-mova-hf"] - **512K context KV cache scaling**: The native 512K context window consumes 103.1 GB in FP16 KV cache or 51.5 GB in FP8 KV cache, requiring multi-GPU partitioning or large-memory accelerators (such as AMD MI325X or NVIDIA H200) for long documents.[cite id="amd-mi325x-datasheet"] - **Single-GPU deployment targets**: A single 32GB RTX 5090 runs 4-bit quantized K2-Horizon up to 64K context. A single 80GB enterprise GPU (A100-80GB or H100) hosts the FP8 checkpoint up to 256K context.[cite id="nvidia-h100-datasheet"] - **Interconnect sensitivity**: Because MoVA sparsifies both self-attention and feed-forward projections, multi-GPU clusters require high-speed NVLink or unified memory fabrics; standard PCIe Gen4 buses suffer communication bottlenecks during all-to-all expert routing.[cite id="ifm-vllm-recipe"] [/key-takeaways] --- ## The engineering dilemma: compound sparsity vs memory residency Most sparse Mixture-of-Experts (MoE) architectures, including Mixtral and DeepSeek, confine sparsity strictly to the Feed-Forward Network (FFN) layers, keeping the self-attention projections dense. K2-Horizon-MoVA-36B changes this paradigm by introducing **Mixture-of-Values Attention (MoVA)** directly into the multi-head attention blocks.[cite id="ifm-k2-horizon-mova-hf"] The architectural layout comprises: 1. **Feed-forward sparsity**: 100 fine-grained FFN experts, routing 8 experts per token with 1 shared expert always active (`num_experts: 100`, `num_experts_per_tok: 8`, `num_shared_experts: 1`).[cite id="ifm-k2-horizon-mova-hf"] 2. **Attention value sparsity (MoVA)**: 64 value projection experts, routing 4 experts per token via a softplus gating mechanism (`mova_num_experts: 64`, `mova_num_experts_per_tok: 4`).[cite id="ifm-k2-horizon-mova-hf"] 3. **Layer depth and geometry**: 48 transformer layers (with layers 0, 1, and 2 designated as dense MLP-only), a hidden dimension of 2,560, head dimension of 128, 32 query attention heads, and 8 key-value heads under Grouped Query Attention (GQA 4:1).[cite id="ifm-k2-horizon-mova-hf"] ``` ┌────────────────────────────────────────────────────────────────────────┐ │ K2-Horizon-MoVA-36B Forward Pass Topology │ ├────────────────────────────────────────────────────────────────────────┤ │ │ │ Input Token Vector (Dim: 2,560) │ │ │ │ │ ├──> Attention Layer (Layers 3-47) │ │ │ ├── Query & Key Projections (Dense GQA, 32/8 heads) │ │ │ └── MoVA Value Projection: 64 Experts ──> Top-4 Routed │ │ │ │ │ └──> FFN Layer (Layers 0-47) │ │ ├── 1 Shared Expert (Always Active) │ │ └── 100 Routed Experts ─────────────────> Top-8 Routed │ │ │ │ Total Resident Parameters: 36.0B │ Active Parameters / Token: 4.0B │ └────────────────────────────────────────────────────────────────────────┘ ``` Because all 36 billion parameters must remain resident in VRAM to service arbitrary token routings, hosting the model requires high memory capacity.[cite id="ifm-k2-horizon-mova-hf"] However, because each token only executes matrix multiplications across 4 billion parameters, inference engines do not need the massive compute capability required by standard 70B dense models.[cite id="ifm-k2-horizon-mova-hf"] --- ## Calculating K2-Horizon-MoVA-36B GPU requirements: memory model To calculate the exact GPU memory required for running K2-Horizon-MoVA-36B, engineers must separate static model weights, dynamic KV cache allocations, activation scratchpads, and CUDA runtime overhead: $$VRAM_{\text{total}} = M_{\text{weights}} + M_{\text{kv}}(C, B_{\text{kv}}) + M_{\text{act}} + M_{\text{cuda}}$$ ### 1. Static model weight calculation The memory required to hold the static weights in VRAM depends on the storage precision: $$M_{\text{weights}} = \frac{P_{\text{total}} \times B_{\text{weight}}}{10^9}$$ Where $P_{\text{total}} = 36.0 \times 10^9$ and $B_{\text{weight}}$ represents bytes per parameter: - **16-bit (BF16 / FP16)**: $B_{\text{weight}} = 2.0 \implies M_{\text{weights}} = 72.0 \text{ GB}$[cite id="ifm-k2-horizon-mova-hf"] - **8-bit (FP8 / INT8)**: $B_{\text{weight}} = 1.0 \implies M_{\text{weights}} = 36.0 \text{ GB}$[cite id="ifm-k2-horizon-mova-hf"] - **4-bit (AWQ / GPTQ / GGUF Q4_K_M)**: $B_{\text{weight}} \approx 0.55 \implies M_{\text{weights}} = 19.8 \text{ GB}$ (including scale and zero-point metadata)[cite id="ifm-k2-horizon-mova-hf"] ### 2. GQA KV cache calculation across context lengths K2-Horizon uses Grouped Query Attention with 48 layers (45 attention layers) and 8 key-value heads with a head dimension of 128.[cite id="ifm-k2-horizon-mova-hf"] The formula for KV cache memory as a function of sequence length $C$ and KV precision bytes $B_{\text{kv}}$ is: $$M_{\text{kv}} = 2 \times L \times N_{\text{kv}} \times d_{\text{head}} \times C \times B_{\text{kv}}$$ Evaluating the constants across the 48 transformer layers yielding 98,304 elements per sequence position:[cite id="ifm-k2-horizon-mova-hf"] - In **FP16 KV precision** ($B_{\text{kv}} = 2$), the cache consumes approximately 196.6 KB per token ($0.1966 \text{ GB}$ per 1,000 tokens).[cite id="ifm-k2-horizon-mova-hf"] - In **FP8 KV precision** ($B_{\text{kv}} = 1$), the cache consumes approximately 98.3 KB per token ($0.0983 \text{ GB}$ per 1,000 tokens).[cite id="ifm-k2-horizon-mova-hf"] | Context Length ($C$) | FP16 KV Cache Memory | FP8 KV Cache Memory | Primary Use Case | |---|---|---|---| | **8,192 tokens (8K)** | 1.61 GB | 0.81 GB | Interactive coding & chat | | **32,768 tokens (32K)** | 6.44 GB | 3.22 GB | Multi-turn agent dialogs | | **65,536 tokens (64K)** | 12.88 GB | 6.44 GB | Codebase ingestion | | **131,072 tokens (128K)** | 25.77 GB | 12.88 GB | Long technical manuals | | **262,144 tokens (256K)** | 51.54 GB | 25.77 GB | Multi-file architectural audits | | **524,288 tokens (512K)** | 103.08 GB | 51.54 GB | Full enterprise document corpus | Notice that at 512K context, the FP16 KV cache alone consumes 103.1 GB, which is substantially larger than the base model weights of 72.0 GB.[cite id="ifm-k2-horizon-mova-hf"] For long-context workloads, memory capacity planning is dominated by KV cache management rather than model size. --- ## Hardware sizing matrix by GPU class The following matrix compares common accelerator platforms for running K2-Horizon-MoVA-36B across different precision levels and context limits: | GPU Hardware Platform | VRAM Capacity | Memory Bandwidth | Supported Precision | Max Safe Context | Cluster Topology | |---|---|---|---|---|---| | **NVIDIA RTX 4090** | 24 GB GDDR6X | 1,008 GB/s | INT4 (AWQ/GGUF) | 16,384 tokens | Single GPU (Desktop/Dev) | | **NVIDIA RTX 5090** | 32 GB GDDR7 | 1,792 GB/s | INT4 (AWQ/GGUF) | 65,536 tokens | Single GPU (Workstation) | | **NVIDIA A100-80GB SXM** | 80 GB HBM2e | 2,039 GB/s | FP8 / BF16 | 262,144 tokens (FP8) | Single GPU or 2x Pod | | **NVIDIA H100-80GB SXM** | 80 GB HBM3 | 3,350 GB/s | Native FP8 / BF16 | 262,144 tokens (FP8) | Single GPU (High Throughput) | | **NVIDIA H200 SXM** | 141 GB HBM3e | 4,800 GB/s | Native BF16 | 262,144 tokens (BF16) | Single GPU (Long Context) | | **AMD Instinct MI325X** | 256 GB HBM3e | 6,000 GB/s | Native BF16 | 524,288 tokens (BF16) | Single GPU (Full 512K Native) | | **2x NVIDIA A100-80GB** | 160 GB Aggregate | 4,078 GB/s | Native BF16 | 524,288 tokens (FP8 KV) | Multi-GPU (TP=2, EP=2) | To check current hourly pricing across these cloud GPU classes on verified providers, consult our interactive [GPU cloud price lookup](/lookup/) tool. --- ## Deep dive: choosing between single-GPU and multi-GPU setups Selecting the optimal hardware configuration depends on whether your workload prioritizes local desktop experimentation, high-throughput microservices, or long-context reasoning. ### 1. The desktop developer tier: RTX 4090 (24GB) and RTX 5090 (32GB) Developers looking to run K2-Horizon-MoVA-36B locally can leverage 4-bit quantization (AWQ or GGUF Q4_K_M).[cite id="ifm-k2-horizon-mova-hf"] - On a **24GB RTX 4090**, 4-bit weights consume roughly 19.8 GB. After allocating 1.5 GB for CUDA context and activation memory, approximately 2.7 GB remains for the KV cache. This accommodates sequence lengths up to 16K tokens with FP8 KV cache compression. - On a **32GB RTX 5090**, the additional 8 GB of VRAM provides substantial breathing room. INT4 weights leave roughly 10.7 GB for KV caching, allowing the RTX 5090 to sustain 65K tokens in FP8 KV cache without offloading to host system RAM. Combined with 1,792 GB/s of GDDR7 bandwidth, the RTX 5090 delivers responsive interactive token speeds for local coding and agentic tasks. For teams comparing consumer hardware against low-cost instances, our evaluation of the [best GPU cloud for inference](/best-gpu-cloud-for-inference/) breaks down preemption and storage trade-offs. ### 2. The enterprise single-GPU tier: NVIDIA H100 (80GB) and H200 (141GB) For production API endpoints serving short to medium context lengths (up to 128K tokens), a single 80GB enterprise accelerator provides an efficient deployment footprint.[cite id="nvidia-h100-datasheet"] Running the official FP8 checkpoint on an **H100 80GB SXM** consumes 36.0 GB for weights. An FP8 KV cache of 128K tokens requires 12.9 GB, leaving over 25 GB of unallocated VRAM for batch concurrency and CUDA graphs.[cite id="nvidia-h100-datasheet"] Because the H100 provides native 8-bit floating-point Tensor Cores and 3,350 GB/s of HBM3 bandwidth, decoding 4B active parameters achieves exceptional tokens-per-second throughput.[cite id="nvidia-h100-datasheet"] When workloads demand unquantized BF16 weights or longer sequences, the **NVIDIA H200 141GB** offers enough headroom to hold 72.0 GB of BF16 weights alongside 50 GB of KV cache on a single device, eliminating the need for inter-GPU communication across sequence lengths up to 256K tokens. ### 3. The hyperscale single-node alternative: AMD Instinct MI325X (256GB) The primary bottleneck of serving K2-Horizon at its full 512K context window is that dual-GPU 80GB clusters run out of headroom when using unquantized FP16 KV cache. The **AMD Instinct MI325X** features 256 GB of HBM3e memory and 6.0 TB/s of peak bandwidth.[cite id="amd-mi325x-datasheet"] This allows a single MI325X accelerator to host: - 72.0 GB of base BF16 weights - 103.1 GB of unquantized FP16 KV cache (full 512K tokens) - 15.0 GB of activation and runtime buffers Total VRAM consumption is approximately 190.1 GB, which fits within the 256 GB boundary of a single accelerator with room to spare.[cite id="amd-mi325x-datasheet"] Deploying on a single high-capacity accelerator avoids the cross-GPU communication overhead that typically plagues sparse MoE models. For a detailed total cost analysis of large memory nodes, see our guide comparing [AMD MI325X vs NVIDIA H200 cloud pricing](/amd-mi325x-vs-nvidia-h200-cloud-pricing/). --- ## Interconnect requirements and expert parallelism Because K2-Horizon features sparsity in both attention (MoVA) and feed-forward (MoE) layers, multi-GPU partitioning requires careful orchestration.[cite id="ifm-vllm-recipe"] When splitting K2-Horizon across multiple GPUs, inference engines such as vLLM and SGLang apply a combination of Tensor Parallelism (TP) and Expert Parallelism (EP):[cite id="ifm-vllm-recipe", cite id="ifm-sglang-cookbook"] - **Tensor Parallelism (TP=2)**: Splits the dense query and key projections across 2 GPUs. - **Expert Parallelism (EP=2)**: Distributes the 100 FFN experts and 64 MoVA value experts across the devices, routing token vectors dynamically during inference.[cite id="ifm-vllm-recipe"] ``` ┌────────────────────────────────────────────────────────────────────────┐ │ Interconnect Bandwidth vs MoVA Pipeline Latency │ ├────────────────────────────────────────────────────────────────────────┤ │ │ │ PCIe Gen4 x16 (31.5 GB/s bidirectional): │ │ [Severe Bottleneck] High all-to-all expert dispatch bubbles. │ │ Token generation latency degrades by up to 3.2x. │ │ │ │ PCIe Gen5 x16 (63.0 GB/s bidirectional): │ │ [Marginal] Acceptable for small batch sizes (BS <= 4). │ │ Exhibits queue contention under concurrent requests. │ │ │ │ NVLink 4 / NVLink 5 (900-1,800 GB/s bidirectional): │ │ [Optimal] Near-zero all-to-all dispatch overhead. │ │ Allows overlapping communication with active 4B FFN compute. │ │ │ └────────────────────────────────────────────────────────────────────────┘ ``` Deploying multi-GPU K2-Horizon instances on cloud servers without NVLink (such as commodity PCIe-only multi-GPU instances) introduces severe pipeline stalls during expert routing. For multi-GPU topologies, always prioritize SXM5 or HGX platforms featuring NVLink interconnects. A deeper comparison of engine communication strategies is available in our [vLLM vs SGLang vs TensorRT-LLM](/vllm-vs-sglang-vs-tensorrt-llm/) analysis. --- ## Production deployment manifests Below are verified launch configurations for production deployment using official recipes from the vLLM and SGLang projects. ### 1. vLLM multi-GPU production recipe The official vLLM recipe for K2-Horizon configures Tensor Parallelism, Expert Parallelism, and native tool-calling parsers:[cite id="ifm-vllm-recipe"] ```bash vllm serve IFM/K2-Horizon-MoVA-36B-A4B \ --revision main \ --tensor-parallel-size 2 \ --enable-expert-parallel \ --trust-remote-code \ --dtype bfloat16 \ --max-model-len 131072 \ --gpu-memory-utilization 0.92 \ --reasoning-parser k2_horizon \ --tool-call-parser k2_horizon \ --enable-auto-tool-choice ``` Key configuration flags: - `--tensor-parallel-size 2`: Splits attention across 2 GPUs (such as 2x A100-80GB or 2x H100-80GB).[cite id="ifm-vllm-recipe"] - `--enable-expert-parallel`: Activates distributed routing for MoVA value experts and FFN experts.[cite id="ifm-vllm-recipe"] - `--max-model-len 131072`: Caps context window at 128K tokens to prevent OOM when serving high-concurrency requests.[cite id="ifm-vllm-recipe"] - `--reasoning-parser k2_horizon`: Correctly parses internal reasoning thoughts into the OpenAI-compatible `reasoning_content` field.[cite id="ifm-vllm-recipe"] ### 2. SGLang recipe for high-throughput serving For deployments prioritizing low prefill latency on 2x NVIDIA H100/H200 nodes, SGLang supports FlashAttention-3 and router GEMM partitioning:[cite id="ifm-sglang-cookbook"] ```bash python3 -m sglang.launch_server \ --model-path IFM/K2-Horizon-MoVA-36B-A4B \ --revision main \ --tp 2 \ --ep 2 \ --dtype bfloat16 \ --attention-backend fa3 \ --json-model-override-args '{"xllm_source_router_gemm_partitions":2}' \ --reasoning-parser k2_horizon \ --tool-call-parser k2_horizon \ --host 0.0.0.0 --port 30000 ``` For broader context on sizing and baseline memory rules across model architectures, review our [LLM GPU VRAM requirements guide](/llm-gpu-vram-requirements-guide/). --- ## Cloud hosting cost scenarios and deployment recommendations When deploying K2-Horizon-MoVA-36B in the cloud, infrastructure costs vary based on uptime requirements, target sequence lengths, and precision tiers. ``` ┌────────────────────────────────────────────────────────────────────────┐ │ K2-Horizon-MoVA-36B Workload Architecture Matrix │ ├────────────────────────────────────────────────────────────────────────┤ │ │ │ Workload 1: Local Developer / Small Team Testing │ │ • Hardware: 1x RTX 5090 32GB (INT4 AWQ) │ │ • Target Context: <= 32K tokens │ │ • Strategy: Community Cloud or on-demand workstation instance. │ │ │ │ Workload 2: Production Microservice API (High Concurrency) │ │ • Hardware: 1x H100-80GB SXM (FP8 Weights, FP8 KV Cache) │ │ • Target Context: <= 128K tokens │ │ • Strategy: Dedicated secure cloud pod on CoreWeave, Lambda, RunPod. │ │ │ │ Workload 3: Full 512K Enterprise Context & Reasoning │ │ • Hardware: 1x AMD MI325X 256GB OR 2x NVIDIA A100-80GB SXM │ │ • Target Context: 128K to 512K tokens │ │ • Strategy: Reserved cluster with high-speed unified memory. │ │ │ └────────────────────────────────────────────────────────────────────────┘ ``` ### Recommendation summary: 1. **For prototyping**: Deploy the INT4 quantized model on a single 32GB RTX 5090 or 24GB RTX 4090 on a community cloud provider. This minimizes cost while providing sufficient memory for up to 32K context windows. 2. **For production web applications**: Host the FP8 checkpoint on a single NVIDIA H100 80GB SXM instance. This setup offers high throughput (4B active parameter compute) and handles up to 128K context lengths with low latency. 3. **For long-document analysis (512K context)**: Rent a single 256GB AMD Instinct MI325X or a 2x A100-80GB SXM pod connected via NVLink. Avoid PCIe-only clusters due to all-to-all expert routing overhead. To inspect verified provider rates and cluster availability across all mentioned GPU architectures, use our real-time [GPU cloud price lookup](/lookup/) index. All research and benchmarking methodology adheres to our published [editorial methodology](/methodology/). --- ## Frequently asked questions [faq] ## How much VRAM is required to run K2-Horizon-MoVA-36B? The minimum VRAM required is 24 GB when running a 4-bit quantized version (AWQ, GPTQ, or GGUF) for short context windows up to 16,384 tokens on an RTX 4090.[cite id="ifm-k2-horizon-mova-hf"] For an unquantized BF16 deployment, you need at least 80 GB of VRAM for short context, or 160 GB across two 80GB GPUs to utilize the full 512K context window with FP8 KV caching.[cite id="ifm-k2-horizon-mova-hf"][cite id="ifm-vllm-recipe"] ## Can K2-Horizon-MoVA-36B run on an RTX 4090 or RTX 5090? Yes, using 4-bit quantization. A single 24GB RTX 4090 can host the model with 19.8 GB of weight footprint, leaving roughly 2.7 GB for a short 16K KV cache.[cite id="ifm-k2-horizon-mova-hf"] A 32GB RTX 5090 provides substantial headroom, allowing 4-bit weights and up to a 64K context window in FP8 KV cache without offloading to CPU memory.[cite id="ifm-k2-horizon-mova-hf"] ## What is Mixture-of-Values Attention and how does it affect hardware? MoVA introduces sparsity directly into the self-attention projection.[cite id="ifm-k2-horizon-mova-hf"] In K2-Horizon, the value projections are divided among 64 value experts, with only 4 experts activated per token.[cite id="ifm-k2-horizon-mova-hf"] Combined with 100 FFN experts, this limits active compute to 4B parameters per token while retaining 36B resident parameters in memory, requiring high VRAM capacity but minimal active compute.[cite id="ifm-k2-horizon-mova-hf"] ## Why does K2-Horizon require so much memory at 512K context? Under Grouped Query Attention across 48 layers and 8 key-value heads, the KV cache requires approximately 196.6 KB per token in FP16.[cite id="ifm-k2-horizon-mova-hf"] Across the full 524,288-token context window, the KV cache alone demands 103.1 GB in FP16 or 51.5 GB in FP8, which exceeds the static model weight footprint.[cite id="ifm-k2-horizon-mova-hf"] ## Can I serve K2-Horizon on a dual-GPU server connected via PCIe? Serving K2-Horizon over PCIe multi-GPU connections is not recommended. Because the model routes tokens across both 64 MoVA value experts and 100 FFN experts on every layer, high-frequency all-to-all communication is required.[cite id="ifm-vllm-recipe"] Without high-speed NVLink (900 to 1,800 GB/s), PCIe Gen4 or Gen5 buses introduce severe communication bubbles that negate the performance gains of sparse MoE activation.[cite id="ifm-vllm-recipe"] [/faq]