NVIDIA Vera Rubin Shatters MLPerf Training and Agentic Inference Records
NVIDIA’s new Vera Rubin platform topped every MLPerf Training v6.1 benchmark and delivered over 10× throughput‑per‑watt gains in agentic inference.
Source: NVIDIA Press RoomNVIDIA announced that its Vera Rubin platform has set new records in both the MLPerf Training v6.1 suite and the inaugural MLPerf Endpoints agentic inference benchmark, delivering up to double the performance per GPU on training and more than ten‑fold gains in throughput per watt for agentic workloads. The results signal a substantial leap in efficiency for developers building large language models, image generators, recommender systems and AI agents.
What is the NVIDIA Vera Rubin platform and why does it matter?
Vera Rubin is NVIDIA’s latest AI infrastructure offering, built around the NVL72 GPU accelerator and a tightly integrated software stack that includes CUDA, NVLink, Quantum‑X800 InfiniBand and Spectrum‑X Ethernet. According to the NVIDIA Press Room, the platform is designed to accelerate the entire AI lifecycle—from pre‑training large models to post‑training inference and real‑time agentic reasoning. Its significance lies in delivering more compute power per watt, which reduces operational costs and expands the scale at which developers can train and serve AI models.
How did Vera Rubin perform in the MLPerf Training v6.1 benchmarks?
The MLPerf Training benchmark measures the time required to train a set of standard models to a target accuracy. Vera Rubin was the only system to submit results for every benchmark in the v6.1 round, covering large language models, image‑generation models, recommender systems and reinforcement‑learning agents. In its debut, the NVL72‑based Vera Rubin delivered up to 2× higher performance per GPU compared with NVIDIA’s prior GB300 NVL72 configuration.
A notable highlight was the DeepSeek‑V3 large‑language‑model benchmark. Vera Rubin scaled from 256 to 1,024 GPUs with roughly 97 % efficiency, cutting the training time from 16 minutes to about four minutes. At the extreme scale of 10,240 GPUs, NVIDIA ran the benchmark on GB300 systems, matching Microsoft Azure’s deployment of the same size. These figures demonstrate that the platform can keep thousands of GPUs working in concert without the typical slowdown that occurs when scaling out.
What were the results on the MLPerf Endpoints agentic inference benchmark?
MLPerf Endpoints evaluates how well a system can serve AI agents that perform multi‑step tasks such as travel planning or IT troubleshooting. The benchmark tracks token generation speed per watt while ensuring agents complete their tasks correctly.
In the provisional Kimi‑K3 agentic workload, Vera Rubin’s NVL72 configuration (32 accelerators) achieved 41,414.2 tokens / second at an operating power of 100.89 kW, yielding a throughput‑per‑kilowatt that is more than 10× higher than the GB300 NVL72 reference (48 accelerators, 2,210.4 tokens / second at 60 kW). Both systems met a shared end‑to‑end interactivity target of 50 tokens per second per user, but Vera Rubin served many more concurrent queries (1,024 vs. 48) while maintaining comparable latency (TTFT P90 ≈ 4.7 s vs. 2.3 s). The result shows that Vera Rubin can support a far larger user base without sacrificing response speed, a key advantage for production AI services.
Which hardware and software innovations enabled these gains?
NVIDIA attributes the performance uplift to several layers of optimization:
- CUDA‑powered software stack – Continuous updates to libraries, kernels and compiler optimizations allow each GPU to extract more work per clock cycle.
- NVLink intra‑node networking – Provides high‑bandwidth, low‑latency connections between GPUs within a server, enabling them to act as a single large accelerator.
- Quantum‑X800 InfiniBand and Spectrum‑X Ethernet – These scale‑out networking technologies link multiple nodes across the data center, preserving efficiency at thousands of GPUs.
- Open‑source libraries and frameworks – NVIDIA’s programmable platform lets partners fine‑tune kernels for specific workloads, contributing to the broad ecosystem participation seen in the MLPerf submissions (20 organizations, including major cloud providers and OEMs).
The combination of hardware bandwidth, power‑efficient GPU design and software co‑design creates a virtuous cycle: as the stack improves, the same hardware delivers more value, and the hardware roadmap can prioritize features that the software stack can immediately exploit.
How can creators, developers and researchers explore these results with Grafy?
While Grafy does not provide large‑scale GPU clusters, its suite of multimodal AI tools lets users investigate and prototype around the concepts demonstrated by Vera Rubin.
- Grafy Chat (/chat) – Use the chat interface to ask the AI to summarize the MLPerf results, compare token‑throughput numbers, or explain how NVLink differs from traditional PCIe connections. The side‑by‑side model capability lets you see explanations from several LLMs at once.
- Grafy Reader (/reader) – Upload the publicly available MLPerf benchmark PDFs and let the AI extract key tables, such as the DeepSeek‑V3 scaling efficiency or the Kimi‑K3 throughput‑per‑watt figures. This can save time when you need precise numbers for a presentation or a research note.
By leveraging these apps, creators can quickly generate visual summaries, developers can draft code snippets that simulate token‑throughput calculations, and researchers can pull quantitative data from the benchmark reports without manual scrolling.
What does this performance leap mean for the broader AI ecosystem?
The Vera Rubin results illustrate that the next generation of AI infrastructure can dramatically reduce the cost and time required to train and serve sophisticated models. For startups, this could lower the barrier to entry for building large‑scale language models or real‑time agents. For enterprises, the 10× throughput‑per‑watt improvement translates into measurable energy savings and the ability to serve more users on existing hardware footprints. Moreover, the open‑source nature of NVIDIA’s software stack encourages community contributions, meaning that performance gains are likely to continue accelerating as more partners optimize their workloads.
Source: NVIDIA Press Room, “NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance,” November 16 2026.


