AgentPerf, the industry's first benchmark designed specifically for agentic AI workloads, shows NVIDIA's Blackwell platf
Agentic AI represents a fundamentally different workload from traditional conversational AI. Where a chat completion is a single sprint consisting of one large language model call and one response, an agent functions like a relay that breaks goals into many steps and continues until completion. This results in dozens to hundreds of LLM calls chained together, with each call passing growing context to the next. Tool calls such as code compilation, execution, database searches and web browsing occur at every handoff. The complexity becomes multiplicative rather than additive.
This distinction matters enormously for performance measurement. Existing AI inference benchmarks measure single LLM calls—how quickly a model responds and how many simultaneous requests a system handles. These benchmarks were not designed for agentic workloads, where chained LLM calls, tool delays and expanding context stress computing systems in fundamentally different ways. For companies building and deploying agents at scale, it is critical to understand agent responsiveness, simultaneous deployment capacity and the productive work that AI infrastructure delivers per dollar and per watt invested.
The first round of AgentPerf measures agentic performance using DeepSeek V4 Pro, a large mixture-of-experts model representing the class of frontier models powering today's most capable agents. On this workload, the NVIDIA GB300 NVL72 delivers the highest performance in the benchmark, running up to 20 times more agents per megawatt than the NVIDIA HGX H200 system. The performance advantage comes from extreme codesign across the full stack. GB300 NVL72 connects 72 GPUs into a single rack-scale system, enabling large MoE models like DeepSeek V4 Pro to distribute model execution efficiently at scale. CUDA kernels accelerate this further by overlapping communication and compute, absorbing the cost of coordinating across experts rather than adding to latency. NVIDIA TensorRT LLM sustains efficiency as concurrent agent sessions scale. For example, it separates input processing from output generation so each can be optimized independently.
These results are grounded in a benchmark methodology built from the ground up to reflect how agentic AI works in production. AgentPerf is built from real coding agent trajectories in which an agent receives a task, reads files, writes and edits code, executes commands and iterates based on results, all drawn from real public code repositories across 12 or more programming languages. The long sequence lengths, tool call patterns and delays are all representative of real-world coding workflows. AgentPerf then measures how many agentic tasks a platform can support simultaneously while meeting defined performance thresholds for responsiveness and output token rate. Tool calls are not executed but simulated using representative CPU processing time, so differences in results reflect accelerated computing performance only.
The results translate directly into infrastructure decisions: how many concurrent agentic tasks can be run per accelerator and per megawatt of power. For enterprises deploying AI agents at scale, those numbers determine how much productive work a given infrastructure investment can actually deliver. Leading inference providers including Baseten, DeepInfra and Together AI are already serving agentic workloads on frontier models such as DeepSeek V4 Pro on NVIDIA Blackwell and powering production agentic applications today. Together AI powers real-time inference for Cursor, an AI-powered agentic coding platform, on NVIDIA Blackwell, where Cursor's agents debug issues, generate features and execute refactors while developers continue working. DeepInfra powers Pam.ai, an AI workforce platform for car dealerships, which deploys agents to book service appointments, handle calls and run outbound sales campaigns, entirely on NVIDIA Blackwell.
As NVIDIA and the open source ecosystem continue to optimize inference software, performance and efficiency on agentic workloads will only improve. The NVIDIA Vera Rubin architecture is now in full production, bringing the next generation of infrastructure capacity to meet the growing demands of agentic AI at scale.