Nvidia’s Vera Rubin system represents a major advancement in AI infrastructure, designed to tackle the growing energy demands of large-scale artificial intelligence. This next-generation rack-scale system promises a tenfold improvement in performance per watt compared to its predecessor, Blackwell. The goal is to optimize AI compute power while managing the escalating operational costs associated with powerful AI models.
The Drive for Efficiency and Scale
The rapid expansion of AI abilities has created a major bottleneck: energy consumption. AI buildouts require immense processing power, which translates directly into higher electricity usage and heat generation. Nvidia’s Vera Rubin system directly addresses this challenge. It aims to deliver greatly more compute for the energy it consumes.
While a Vera Rubin rack uses about double the energy of a Blackwell rack, consuming roughly 220kW, it delivers exponentially more compute. This efficiency gain is critical for what Nvidia calls “AI factories,” which consist of thousands of racks working in concert. The system is engineered so that all its components behave as a single, cohesive GPU. This design approach helps reduce the cost per token by ten times compared to Blackwell.
Inside the Vera Rubin Architecture
Vera Rubin features a highly integrated and modular design. Each rack contains 18 compute trays. Each tray houses two Vera Rubin superchips, which are the core computing units. A single superchip is a complex assembly, comprising one Vera CPU, two Rubin GPUs, and 17,000 other components.
The Vera CPU offers twice the performance per watt compared to the previous generation Grace CPU. The Rubin GPU itself delivers 50 petaflops of AI performance, representing a 2.5 times increase in performance. Memory is a key aspect, with the Rubin GPU featuring eight stacks of High Bandwidth Memory (HBM4). Unlike the soldered memory in Blackwell, Vera Rubin uses SoCAMM memory units that can be applied and removed. This modularity simplifies maintenance.
Connectivity within the rack is managed by the NVLink Switch chip. This chip connects all 72 GPUs and CPUs, allowing them to function as one unified system. The line rate has doubled from 1.8 terabytes per second to 3.6 terabytes per second. Nine NVLink Switch trays help this, pushing data at an incredible 260 terabytes per second. A vertical NVLink spine, made of 5000 copper cables spanning two miles, connects everything at the back of each rack. The system also includes BlueField DPUs for storage and security, and ConnectX-9 networking controllers for external connections.
Advanced Cooling and Power Delivery
Managing the heat generated by such powerful systems running 24/7 is essential. Vera Rubin is Nvidia’s first system to be 100% liquid cooled. This design requires data centers to have a liquid-cooled base architecture ready for deployment. Cold plates cover the Vera CPU and Rubin GPUs, circulating water to maintain best temperatures.
Liquid cooling offers a counterintuitive benefit: it uses less water overall. This is because it reduces reliance on evaporative cooling technologies, which consume major amounts of water. The system’s power delivery has been redesigned to handle the increased energy demands of 220kW per rack.
Early deployments of the Blackwell system saw some overheating issues. These were often attributed to user error, such as improper seating of liquid cooling valves. The shift to a fully liquid-cooled system in Vera Rubin aims to standardize and improve thermal management. The new compute tray design allows for disassembly in five minutes, a major improvement over the two hours required for Blackwell, which had soldered components.
The Global Supply Chain and Manufacturing Complexities
Building a system like Vera Rubin involves an intricate global supply chain. Each rack contains 1.3 million components, sourced from over 80 different suppliers across more than 20 countries. Core silicon and chips are developed by TSMC. Foxconn handles rack assembly components. Delta Electronics provides liquid cooling elements. Other key suppliers include Amphenol for connectors and copper, Vertiv for cooling distribution systems, and various companies like MegMeet, LiteOn, and Flex for power shelves.
Nvidia created a standard reference design to open up its ecosystem to many providers. This includes companies like Infineon, Analog Devices, and ST-Microelectronics for monolithic power systems, Interplex for chassis, Bizlink for busbars, Pinda for rack manifolds, and Auras, AVC, Boyd, and Coolermaster for cold plates.
This complex supply chain faces challenges. Memory, particularly HBM4, is in short supply. Nvidia works closely with its supply chain partners, providing detailed forecasts to ensure component availability. Tariffs also impact pricing across various components, creating a “whack-a-mole” effect on costs. In response, Nvidia has committed to manufacturing up to $500 billion of AI infrastructure in the U.S. through 2029, including producing Blackwell at TSMC’s new Arizona fabs. Assembly also takes place in the U.S., Taiwan, and at a new Foxconn plant in Mexico.
Competition and Future Outlook
Vera Rubin is expected to ship in the second half of 2026. It will enter a competitive market. AMD plans to ship Helios, its first rack-scale system, later this year. Major customers like Microsoft, Google, Amazon, and Meta are also developing their own AI chips. Despite this, these companies continue to work with Nvidia for their AI infrastructure needs.
Nvidia encourages customers to adopt its annual architecture cadence, recognizing that each generation offers major leaps in performance. Vera Rubin systems can operate alongside existing Blackwell systems to power different workloads. While Vera Rubin will cost more upfront, with analysts estimating a 25% price increase from Blackwell (from around $3-3.2 million to $3.5-4 million per rack), the reduced cost per token makes it a more efficient investment over time.
Looking ahead, Nvidia has already previewed its next architecture after Rubin, codenamed Kyber. This design aims for even greater compute density, with 288 GPUs per rack, a fourfold increase from Blackwell, while only increasing weight by about 50%. This is achieved by removing much of the cabling. The Vera Rubin Ultra, incorporating the Kyber rack design, is expected to ship in 2027. The long-term goal is to reduce connection points, minimize potential failure points, and increase integration. This approach not only makes systems faster but also reduces the total cost of ownership. Greater compute density and lower latency in the end enhance the AI user experience, driving further demand for these powerful systems.
