NVIDIA Vera Rubin NVL72 Architecture Deep Dive: Specs, Cooling, and Who Actually Needs One

TL;DR. Vera Rubin NVL72 is a liquid-cooled, rack-scale AI machine engineered for trillion-parameter training and Agentic AI inference. It packs 36 Vera CPUs, 72 Rubin GPUs, 20.7 TB of unified HBM4 memory, and a non-blocking NVLink fabric — all behaving as a single logical accelerator. It also draws over 100 kW per rack and rewrites what a "data center site" needs to look like. For most enterprises, the answer is to consume this capacity, not own it.

The race for AI dominance did not stop with Blackwell. As models shifted from chat to Agentic AI — systems that reason, plan, and act in long tool-using loops — the compute, memory, and bandwidth demands broke previous records. NVIDIA's response is the Vera Rubin NVL72: a single rack engineered to behave as one supercomputer.

The headline numbers from NVIDIA: trillion-parameter training with roughly a quarter of the GPUs Blackwell required, and inference cost dropping to about a tenth of the previous generation. But the headline isn't the architecture. The architecture is what makes the headline possible. Let's open the rack.

The Compute Core: Meet Vera and Rubin

Most "next-gen" CPUs and GPUs are evolutions. Vera and Rubin are co-designed — the CPU exists to feed the GPU, and the GPU exists to be fed without stalls. That co-design is the story.

• The Rubin GPU. Rubin introduces HBM4 memory at 288 GB per GPU with roughly 22 TB/s of memory bandwidth. It also introduces a Transformer Engine optimized for NVFP4 — a 4-bit floating-point format that doubles math density without collapsing model accuracy. The 288 GB number is what matters in practice: it lets a single GPU hold a much larger Mixture-of-Experts (MoE) shard, a longer KV cache for extended-context reasoning, or an entire mid-sized model without partitioning. Less partitioning means less inter-GPU traffic, which means less time spent waiting on the fabric. Memory capacity isn't a spec — it's a software-complexity reduction.

• The Vera CPU. Named after the astronomer, Vera is built around 88 NVIDIA Olympus cores — custom Arm-compatible cores tuned for deterministic latency and large-block data movement. It pairs with LPDDR5X memory for high bandwidth and aggressive power efficiency. The strategic move here is leaving x86 behind for the AI-host role. x86 carries decades of legacy interrupt and instruction baggage that an AI feeder CPU does not need. Olympus is purpose-built to do three things well: marshal data from storage and network, schedule kernels, and stay out of the GPU's way. That is the entire job.

• The Superchip configuration. The base unit is one Vera CPU paired with two Rubin GPUs. This 1:2 ratio is deliberate — modern AI pipelines are GPU-bound, not CPU-bound, so over-provisioning CPU is waste. The fully populated NVL72 rack contains 36 Vera CPUs and 72 Rubin GPUs, exposing 20.7 TB of unified HBM4 as a single addressable pool.

The "unified pool" phrase deserves its own paragraph. In a traditional cluster, every GPU sees its own local memory plus a complicated routing layer to reach others'. With NVL72's NVLink fabric, all 72 GPUs operate against shared, coherent memory. That collapses entire categories of distributed-training engineering — sharding strategies, gradient synchronization tricks, expert-routing logic — into hardware. Engineers stop fighting the topology and start writing models.

The Nervous System: NVLink 6 and the All-to-All Topology

You cannot turn 72 GPUs into one logical machine without a fabric that has zero compromises. NVLink is what makes the rack-as-a-server idea real.

• NVLink 6 switches deliver roughly 3.6 TB/s per GPU — double the previous generation. That bandwidth is symmetric and persistent, not a peak burst.
• All-to-all topology. Every GPU can speak to every other GPU directly, simultaneously, without a hop. There is no oversubscription and no contention pattern that degrades when traffic ramps up.

This matters most for MoE training. In an MoE model, each token is routed to a small subset of "experts" scattered across the cluster. Routing decisions are unpredictable, traffic is bursty, and any fabric that drops to half bandwidth under load wastes the entire investment. NVLink 6 is engineered to never become the bottleneck.

The architectural implication: scale-up bandwidth (inside the rack) is now so far ahead of scale-out bandwidth (between racks) that the rack itself becomes the new unit of design. You don't plan around servers anymore. You plan around racks.

The Network Backbone: ConnectX-9, BlueField-4, and Spectrum-X

When one rack is not enough — frontier model training routinely uses tens of thousands of GPUs — the data-center fabric becomes the next bottleneck. NVIDIA has overhauled every layer.

• ConnectX-9 SuperNIC. The next-generation NIC delivers 1.6 Tb/s of per-GPU bandwidth with programmable RDMA. RDMA matters because it lets GPUs talk to remote GPUs in other racks without the host CPU mediating each packet — what NVIDIA calls GPU-direct. The result is single-digit-microsecond latency for collective operations across thousands of GPUs.

• BlueField-4 DPU. A Data Processing Unit is, in plain English, a CPU dedicated to infrastructure work — networking, storage, security, telemetry, multi-tenant isolation. Without a DPU, those jobs steal cycles from the host CPU and the host CPU steals attention from the GPU. With BlueField-4 doing the dirty work, every CPU and GPU cycle in the rack is available for actual AI compute. That is the whole point of offload — and at NVL72 scale, the math gets serious quickly.

• Spectrum-X with Co-Packaged Optics (CPO). This is one of the quietest revolutions in the rack. Traditional Ethernet switches use pluggable optical transceivers — small modules that convert electrical signals to light at every port. At 1.6 Tbit/s speeds, pluggables become disproportionately power-hungry and failure-prone. CPO embeds the optics directly into the switch ASIC package, eliminating most of those modules. The reported gains are roughly 5x better power efficiency and 10x higher network resiliency for lossless AI Ethernet.

Power, Cooling, and the Physical Reality

The architecture is the headline. The facility is the unspoken footnote.

A populated NVL72 rack draws well over 100 kW — and Rubin pushes that envelope further than its Blackwell predecessor. By comparison, the average enterprise data center is engineered for 8 to 15 kW per rack. That is not a small mismatch. It is an order of magnitude.

Two consequences flow from that fact:

1. Liquid cooling is mandatory, not optional. Air cooling cannot remove that much heat, regardless of fan strategy. Direct-to-chip liquid loops, rear-door heat exchangers, and in some designs full immersion become baseline requirements.
2. Most data centers cannot host this without major retrofit. Power feeds, floor structural ratings, plumbing for coolant, and the chiller plant outside the building all need to be reconsidered. In EMEA specifically — where grid capacity, planning permission, and water access vary sharply by country — the choice of where to host one of these racks is now a strategic decision, not just a procurement one.

This is why the conversation in 2026 has shifted from "buy GPUs" to "where will the GPUs live". For most organizations, the most rational answer is renting capacity from a hyperscaler or a regional sovereign-AI provider that has already solved the facility problem.

The Software Story

Silicon without software is a heater. The Vera Rubin platform comes wrapped in a stack that's just as opinionated as the hardware:

• CUDA, cuDNN, TensorRT-LLM — the low-level math and inference libraries.
• NeMo Framework — training, fine-tuning, and alignment toolkits.
• NIM microservices — pre-packaged inference endpoints, ready to deploy.
• NVIDIA AI Enterprise — the supported, contractually-backed distribution that most enterprises actually buy.
• Kubernetes with the NVIDIA GPU Operator — the orchestration layer.

The honest read: this stack is excellent and it is also a strategic commitment. Once your training pipelines are wired into NeMo and your inference is deployed via NIM, the cost of switching to a different accelerator vendor is non-trivial. That is a feature for NVIDIA and a clear-eyed planning input for the rest of us.

Who Actually Needs This?

This is the question most coverage skips. Vera Rubin NVL72 is purpose-built — and very expensive. The buyer profile is narrow.

• Hyperscalers (AWS, Azure, Google Cloud, Oracle Cloud) — they buy NVL72 racks by the thousands to rent slices of them as managed services. This is their core product.
• Frontier AI labs — the small handful of organizations training trillion-parameter foundation models. They need every TB/s of NVLink bandwidth they can get.
• Sovereign AI initiatives — government-backed programs in the UK, France, Germany, the UAE, Saudi Arabia, and elsewhere that want a domestic frontier-model capability. NVL72 is the reference platform.
• Regional cloud providers and "neoclouds" — companies like CoreWeave, Lambda, Nebius, and several EMEA equivalents that resell GPU capacity as a service.

And just as importantly — who does not need one:

• Most enterprises running inference against pre-trained models for RAG, summarization, search, or copilots.
• Mid-size organizations doing fine-tuning at modest scale.
• Anyone whose AI initiative is still validating its use case.

For that majority, NVL72 is the wrong answer to the wrong question. The right path is to start with a CPU-first inference architecture, prove value, and consume frontier compute as a service when — and only when — the workload genuinely demands it. For organizations whose problem is a hardware refresh on the private-cloud side, the conversation looks completely different — and far more grounded.

The Strategic Takeaway

Vera Rubin NVL72 is not a server. It is a rack-scale product, a facility decision, a software commitment, and a category boundary all at once.

For the small group of organizations that genuinely need to train frontier models or operate truly massive inference fleets, it is an extraordinary instrument. For everyone else — which is almost everyone — the strategic move is to understand the architecture clearly enough to consume it intelligently, without confusing access to the technology with the business value it can create.

The rack is the answer to a specific question. Make sure that's the question you are actually asking.

Vera Rubin (1928–2016) was an American Jewish astronomer, born in Philadelphia to a family of Lithuanian Jewish descent, widely regarded as one of the most consequential figures in 20th-century astrophysics. She studied the rotation speeds of spiral galaxies and discovered that they were spinning far too fast to be explained by visible matter (stars and gas) alone — a finding that provided the decisive empirical evidence for the existence of dark matter. Throughout her career she also fought to open doors for women in science: she was the first woman officially permitted to observe at the Palomar Observatory, and despite her enormous contributions to the field she never received a Nobel Prize, a fact widely discussed as a historical injustice. NASA named its major sky-survey telescope in Chile the "Vera C. Rubin Observatory" in her honor, which is also where NVIDIA's GPU lineup gets its name — continuing the company's tradition of naming chip generations after prominent scientists.

Sources

• NVIDIA Vera Rubin NVL72 — official product page↗
• Related: You Probably Don't Need GPUs to Start AI On-Prem
• Related: The Ultimate Hardware Refresh: Intel Xeon 6 & VMware VCF