NVIDIA's CPU Invasion: What Hundreds of Thousands of Grace Servers Mean for the Datacenter Stack

NVIDIA has quietly shipped hundreds of thousands of Grace standalone CPU servers. Not Grace Hopper Superchips paired with GPUs. Standalone CPUs, competing directly with Intel Xeon and AMD EPYC for general-purpose datacenter workloads. If you missed that, you are not alone. But the implications for infrastructure architects are significant.

The Numbers Behind NVIDIA's CPU Push

Ian Buck, NVIDIA's VP of Hyperscale and HPC (and the inventor of CUDA), disclosed in September 2026 that NVIDIA has shipped "hundreds of thousands of Grace standalone servers" [1]. Earlier in the year, the company reported over 2.5 million total Grace CPUs shipped, but the standalone server figure had not been broken out before. Meta was the first major customer to deploy Grace standalone CPUs at scale, announcing the partnership in February 2026 [2][3].

This is a fundamentally different business than selling GPU-attached host processors. NVIDIA is competing for the CPU socket itself, targeting workloads that do not need a GPU at all: backend data processing, agentic AI orchestration, and data-rich inference pipelines where the CPU is the primary compute engine.

Grace Architecture: What Makes It Different

Grace is built on 72 Arm Neoverse V2 cores connected by NVIDIA's Scalable Coherency Fabric (SCF) [4]. The design prioritizes memory bandwidth and energy efficiency over raw core count. With LPDDR5X memory, Grace achieves high bandwidth at significantly lower power than DDR5-based alternatives. For memory-bound workloads, which describes most agentic AI inference and large-scale data processing, that tradeoff matters more than peak integer throughput.

The key differentiator is not the core design itself, which uses standard Arm IP. It is the system-level integration: NVLink-C2C provides coherent interconnect between Grace and NVIDIA GPUs at 900 GB/s, while the memory subsystem is optimized for continuous data movement rather than burst compute. For workloads that stream through large datasets or manage the state of thousands of concurrent agent sessions, Grace's architecture is a better fit than a chip designed for peak SPEC scores.

Vera CPU: The Custom-Core Successor

Grace proved the market exists. Vera is NVIDIA's bet that it can dominate it. Unlike Grace, which uses licensed Arm cores, Vera features 88 custom-designed Olympus cores, NVIDIA's first in-house server CPU architecture [5][6].

The specifications mark a significant step up:

According to NVIDIA, Vera achieves up to 1.9x IPC improvement over AMD Turin, with up to 2.3x more branch predictions and 2.4x higher instruction fetch operations per cycle [7]. On SPEC CPU 2026 benchmarks, a dual-socket Vera system scored an estimated 925 base, compared to AMD EPYC 9755's 898 [7]. These are early, unofficial results, so they should be treated as directional rather than definitive.

Jensen Huang framed the positioning directly: "Vera is arriving at a turning point for AI. As intelligence becomes agentic, capable of reasoning and acting, the importance of the systems orchestrating that work is elevated" [6].

Why Agentic AI Changes the CPU Equation

The traditional AI datacenter ratio has been roughly eight GPUs per CPU. The GPU does the heavy inference work. The CPU manages orchestration, scheduling, and I/O. In that model, the CPU is an afterthought.

Agentic AI inverts this. An AI agent does not just run inference on a model. It reasons about a task, calls tools, queries databases, manages state across multi-step workflows, and coordinates with other agents. Much of that work is CPU-bound: parsing, routing, state management, API orchestration. The hardware ratio shifts toward one-to-one or even CPU-heavy configurations [1].

NVIDIA estimates the total addressable market for datacenter CPUs at approximately $200 billion, though industry consensus runs lower at roughly $120-170 billion by 2030. Morgan Stanley projects that agentic workloads alone could add $60 billion to the CPU market [1]. Whether those numbers are precise, the directional signal is clear: CPU demand in AI datacenters is growing, not shrinking.

A Vera rack of 256 liquid-cooled chips can sustain over 22,500 concurrent CPU environments [6]. For a platform running thousands of AI agents simultaneously, that density matters.

What This Means for Intel and AMD

The competitive threat is real but nuanced.

Intel's Xeon 6 (Sierra Forest and Granite Rapids) and AMD's EPYC Turin are both strong products for general-purpose datacenter workloads. Intel holds the largest installed base and the broadest software ecosystem. AMD has captured significant share through competitive performance and pricing. Both compete well on legacy workloads, traditional enterprise applications, and cloud computing.

Where NVIDIA has an asymmetric advantage is in the AI-adjacent tier: workloads that sit between pure GPU inference and traditional enterprise computing. An agentic orchestration layer running on Vera gets NVLink-C2C coherent access to Rubin GPUs at 1.8 TB/s. Running the same workload on Xeon or EPYC means communicating with GPUs over PCIe, which is an order of magnitude slower.

For workloads that do not touch GPUs at all, the advantage narrows. Vera's monolithic die design optimizes for specific workload profiles and may sacrifice performance on legacy applications that Intel and AMD chiplet designs handle efficiently [1]. The software ecosystem around x86 is decades deep. Arm adoption in the datacenter is growing but still a fraction of the installed base.

The competitive picture looks like this:

What VCF Architects Should Plan For

If you are running VMware Cloud Foundation, the immediate question is hardware compatibility. VCF 9.1 runs on vSphere, which supports Arm through the Arm SystemReady ES certification program. Grace CPUs from qualified OEM partners (Dell, HPE, Lenovo, Supermicro) can run ESXi, which means VCF workloads can run on Grace today.

Vera compatibility will depend on OEM qualification timing. Given that Dell, HPE, Lenovo, and Supermicro are all listed as Vera infrastructure partners [6], VCF support is a matter of when, not if.

The practical planning question is whether your next hardware refresh should include an NVIDIA CPU tier alongside your existing Intel or AMD infrastructure. For organizations deploying VCF Private AI Services, a mixed architecture, with Vera nodes handling agent orchestration and GPU-adjacent workloads while Xeon or EPYC nodes run traditional VMs, could optimize both cost and performance.

The Intel Xeon 6 + VCF 9 hardware refresh guide we published earlier this year [8] remains valid for general-purpose refresh cycles. But if agentic AI workloads are on your roadmap for 2027, start qualifying NVIDIA CPU platforms now. The ecosystem is moving faster than most enterprise procurement cycles.

The Full-Stack Play

What makes NVIDIA's CPU push different from previous Arm datacenter attempts (Ampere, Graviton outside AWS) is the full-stack integration. NVIDIA is not selling a CPU. It is selling a datacenter architecture where GPU, CPU, DPU, NIC, switch, and software all share a coherent fabric.

That is a familiar strategy. It is exactly what Broadcom did with VMware Cloud Foundation: consolidate compute, storage, networking, and security into a single integrated platform. The question for enterprise architects is whether one integrated stack (NVIDIA for AI infrastructure, VCF for private cloud) makes more sense than assembling best-of-breed components. The answer, increasingly, is that integration wins at scale.

References

[1] Nvidia Has Shipped 'Hundreds of Thousands of Grace Standalone Servers'↗ - Tom's Hardware [2] Meta Expands Nvidia Deal to Use Millions of AI Chips↗ - CNBC [3] Meta Builds AI Infrastructure With NVIDIA↗ - NVIDIA Newsroom [4] NVIDIA Grace CPU and Arm Architecture↗ - NVIDIA [5] NVIDIA Deep Dives Vera CPU: SPEC CPU 2026 Benchmarks Revealed↗ - Tom's Hardware [6] NVIDIA Launches Vera CPU, Purpose-Built for Agentic AI↗ - NVIDIA Newsroom [7] NVIDIA Vera CPU Benchmarked: Beating Intel Xeon and AMD EPYC in Select Workloads↗ - TechPowerUp [8] Intel Xeon 6 + VMware VCF 9 with vSAN ESA: The Complete Hardware Refresh Guide↗ - barhum.ai [9] NVIDIA's AI CPU Push Threatens Intel and AMD↗ - 24/7 Wall St. [10] CPUs Are Back: The Datacenter CPU Landscape in 2026↗ - SemiAnalysis [11] Nvidia's Big CPU Push↗ - ChipPub [12] An Nvidia Move Just Put AMD and Intel on the Defensive↗ - Motley Fool

---

Disclosure: The author is an employee of Broadcom. This article represents the author's technical analysis and does not represent Broadcom's official position.