NVIDIA says its upcoming Vera Rubin NVL72 platform can deliver up to 30 times the agentic-AI throughput per megawatt of its GB300 NVL72 system, based on early measurements using a workload designed to mimic production coding agents. The claim positions power efficiency—not just peak tokens per second—as a central battleground for the next generation of AI infrastructure.

The company published the results across two NVIDIA blog posts, describing them as preview data measured with the SemiAnalysis AgentX benchmark. The measurements have not yet been independently reviewed by SemiAnalysis, making the results important but still vendor-reported. For AI builders and infrastructure operators, the comparison matters because multi-step agents consume substantially more context and model calls than conventional chat interactions, increasing both compute demand and power costs.

Why agent workloads change the infrastructure equation

Traditional inference benchmarks often use fixed prompt and output lengths. NVIDIA argues that those tests poorly represent an agent that repeatedly reasons, calls tools, delegates work to sub-agents and carries an expanding context from one step to the next.

The company cites OpenRouter data indicating that agentic requests use about 15 times as many tokens as ordinary chat requests. In an example provided by NVIDIA, an investment-research agent might query financial databases, search filings and news, ask another model to compare peers, run valuation analysis and then assemble a recommendation. Each step can add information that becomes input to later steps.

That pattern creates several performance pressures at once. Systems must process long prompts, preserve and reuse key-value caches, handle variable output lengths and tolerate pauses while tools execute. They also need to maintain acceptable response speed for multiple users, rather than simply maximize aggregate throughput under a fixed synthetic workload.

The NVIDIA results focus on DeepSeek V4 Pro and other large models, including Kimi K3, MiniMax M3, GLM5.3 and Qwen3.5. The headline comparison is between Vera Rubin NVL72 and GB300 NVL72, two rack-scale systems built around a 72-GPU scale-up domain.

What NVIDIA measured—and what remains unverified

NVIDIA says Vera Rubin NVL72 achieved up to 30 times higher AI-factory throughput per megawatt than GB300 NVL72 on SemiAnalysis AgentX. The developer-focused post specifies that the comparison reached this result at 160 tokens per second per user on the AgentX DeepSeek V4 Pro workload. NVIDIA also says the result represents throughput within the same power budget while maintaining the stated interactive serving target.

The AgentX benchmark is described as an open-source component of SemiAnalysis’s InferenceX suite. It replays recorded Claude Code sessions turn by turn, preserving context growth, input and output lengths, reasoning intervals and tool-call delays. The benchmark also varies concurrency and reports throughput per megawatt alongside latency and interactivity measures such as time to first token.

Those details make AgentX more relevant to production agent serving than a single fixed-length request test, but they do not make the 30x figure an independently validated industry standard. NVIDIA says the Vera Rubin measurements are pending SemiAnalysis review. The company also notes that the early data does not include Vera CPU performance for tool calling, which could affect full-system results.

NVIDIA reports other generational comparisons from the same workload. It says GB300 NVL72 delivers up to 15 times the throughput per megawatt of H200 NVL8 for DeepSeek V4 Pro, and up to 80 times the throughput per megawatt for the much larger Kimi K3 2.8T model. It further claims that Vera Rubin can reduce cost per million tokens by as much as 35 times compared with GB300 NVL72. These figures are NVIDIA measurements and should be treated as performance claims until independently reproduced.

The system design behind the efficiency claims

NVIDIA attributes the gains to coordinated changes across hardware, networking and serving software rather than to the GPU alone. The NVL72 design links 72 GPUs through NVIDIA’s NVLink fabric, allowing the platform to distribute mixture-of-experts computation and share cached context across the rack-scale domain.

The company says sixth-generation NVLink and its switches provide higher packet rates and lower latency than off-the-shelf Ethernet alternatives. That connectivity is intended to support large-scale expert parallelism, distributed key-value caching and routing requests toward GPUs that already contain relevant context.

On the software side, NVIDIA points to disaggregated serving, in which prefill and decode run in separately scaled pools. Its NVIDIA Dynamo serving stack is designed to coordinate those stages, use session information and route requests based partly on cache overlap. Such features are particularly relevant to agents because the same task can generate many related model calls over an extended session.

NVIDIA also cites TensorRT-LLM, vLLM and SGLang as parts of the broader serving ecosystem, along with DeepGEMM-based kernels, fused mixture-of-experts execution and lower-precision formats. The source material references both NVFP4 and MXFP4/MXFP8 techniques in different descriptions of the platform. These methods can reduce memory use and increase arithmetic throughput, although the practical quality and compatibility trade-offs depend on the model and deployment configuration.

The company separately says its DSX MaxLPS power-management technology can provision up to 40% more GPUs within the same megawatt budget. That is another NVIDIA-reported capability, not a result established by the AgentX comparison itself.

What the result means for AI operators

If the headline numbers survive independent testing, the commercial effect could be significant. AI factories increasingly face limits on electricity availability, cooling capacity and the cost of serving long-running sessions. More agentic work per megawatt could let an operator serve more customers from an existing power allocation, or reduce the infrastructure required for a fixed level of usage.

The impact will vary by workload. A short conversational assistant may not benefit as much as a coding, research or customer-service agent that repeatedly invokes tools and accumulates context. Model architecture also matters: NVIDIA’s strongest comparisons involve large mixture-of-experts systems, where inter-GPU communication and cache management can dominate serving performance.

For product teams, the figures reinforce the need to measure complete agent workflows rather than isolated model calls. Evaluation should include tool latency, cache reuse, concurrency, user-facing interactivity and cost per completed task. A platform that produces tokens cheaply but stalls between tool calls may not deliver a better application experience.

The claims also increase competitive pressure on accelerator vendors and cloud providers to publish workload-specific efficiency data. “Tokens per second” alone says little about the economics of a stateful agent. Builders will need comparable measurements across models, serving stacks and power envelopes before committing to a hardware generation.

What to watch next

The first signal will be SemiAnalysis’s review of the AgentX measurements. Independent confirmation, including the exact system configurations, software versions, model checkpoints and power-accounting methodology, will determine how broadly the 30x comparison can be interpreted.

Buyers should also watch for results that include CPU-side tool execution, since NVIDIA says the current Vera Rubin data does not. Additional tests across non-coding agents, different context lengths and alternative serving frameworks would show whether the advantage extends beyond the recorded Claude Code sessions.

Availability, pricing and cloud access will be equally important. The efficiency claim can improve theoretical cost per million tokens, but real customer economics will also depend on system acquisition costs, utilization, networking, cooling and software licensing. Evidence from early deployments will reveal whether the benchmark advantage translates into lower prices or higher margins for operators.

Creati.ai perspective

NVIDIA’s announcement is best understood as a shift in how AI infrastructure performance is framed. As agents generate longer, more stateful workloads, power-normalized throughput and cost per completed task become more useful metrics than conventional single-request speed.

The 30x figure is attention-grabbing, but it remains an early NVIDIA measurement pending independent review. The durable lesson for builders is narrower and more actionable: benchmark the entire agent loop, including context reuse, tools, concurrency and latency, before treating any accelerator comparison as a production conclusion.