NVIDIA Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt and 35x lower cost per token on agentic AI wor

NVIDIA announced performance benchmarks for its Vera Rubin NVL72 systems on agentic AI workloads, measured using the SemiAnalysis AgentX workload of real-world agentic coding sessions. Agentic AI differs fundamentally from chat because agents accumulate context across multiple steps that can reach hundreds of thousands of input tokens as they query databases, search information sources, invoke sub-agents, and synthesize results. This pattern occurs across use cases from software development to customer service to deep research.

The measured results show Vera Rubin NVL72 delivers up to 30x higher throughput per megawatt than GB300 NVL72 on agentic workloads, and up to 35x lower cost per million tokens. For power-constrained AI factories, this means 30x more agentic work for the same energy footprint and profit margins determined by cost per million tokens. The company also noted that GB300 NVL72 itself delivers up to 15x better throughput per megawatt than the older Hopper architecture on models like DeepSeek V4 Pro.

NVIDIA achieved these gains through what it calls extreme codesign across every platform layer. This includes disaggregated serving that separates context processing from response generation, rate matching that synchronizes prefill and decode GPU speeds, large-scale expert parallelism for mixture-of-experts models, distributed KV-caching across the GPU domain, KV-aware routing that directs requests to GPUs holding relevant cached context, and fused CUDA kernels like MegaMoE that combine multiple operations into single execution passes. The Vera Rubin GPUs feature enhanced fifth-generation Tensor Cores, third-generation Transformer Engine, and NVFP4 quantization that compresses model weights to 4-bit precision. The NVL72 scale-up domain is connected by sixth-generation NVLink technology delivering 10x higher packet rates and 3x lower latency than Ethernet alternatives.

Vera Rubin is a seven-chip architecture that also includes the Vera CPU, Groq 3 LPU, NVLink 6 Switch, BlueField-4 DPU, Spectrum-6 SPX and ConnectX-9 SuperNIC. The company stated that Vera Rubin is in full production and scaling across the ecosystem.