From A100 to Rubin and Beyond: NVIDIA’s GPU Roadmap Is Rewriting AI Power and Cooling Infrastructure

The GPU Roadmap Is Becoming an Infrastructure Roadmap

For years, NVIDIA GPU development was discussed mainly in terms of FLOPS, memory and training performance.

That is no longer enough.

Every watt consumed by an AI server eventually becomes heat that must be removed from the data center.

As GPU density rises, three infrastructure requirements rise with it:

• Electrical power delivery
• Cooling capacity
• Rack-level power and fluid distribution

This means NVIDIA’s GPU roadmap now influences not only semiconductor suppliers but also:

• Power supply module manufacturers
• Busbar and power shelf suppliers
• CDU manufacturers
• Cold-plate suppliers
• Rack manifold manufacturers
• Hose and quick-disconnect suppliers
• Chiller and heat-rejection companies
• Data-center electrical infrastructure companies

The AI server is becoming an integrated electrical and thermal machine.

From Ampere to Hopper, Blackwell and Rubin

The NVIDIA A100 based on the Ampere architecture became one of the foundations of the modern AI data-center market.

The A100 80GB SXM operates at up to 400W TDP in its standard configuration.

Hopper changed the thermal equation.

The H100 SXM increased maximum TDP to as much as 700W.

H200 retained the same 700W maximum TDP class while increasing GPU memory to 141GB and memory bandwidth to 4.8TB/s.

Blackwell then pushed infrastructure design beyond the individual GPU and increasingly toward the complete server and rack.

NVIDIA AI GPU and Platform Evolution

Power figures should not be interpreted as direct comparisons between individual GPUs and complete systems. They are shown together to illustrate how the infrastructure design boundary is moving from chip to server to rack.

A100 to H100: A 75% Increase in GPU Heat Load

Consider a conventional 8-GPU AI server.

Eight A100 SXM GPUs at 400W represent approximately:

3.2 kW of GPU thermal load

Eight H100 SXM GPUs at 700W represent approximately:

5.6 kW of GPU thermal load

That is around a 75% increase in GPU-only thermal load.

And these numbers exclude:

• CPUs
• Memory
• SSDs
• Network adapters
• Fans and pumps
• Power-conversion losses

This is one reason why the cooling discussion changed so quickly during the Hopper generation.

H200 Did Not Reduce the Thermal Problem

H200 significantly increased memory capacity and bandwidth.

NVIDIA specifies:

• 141GB HBM3e memory
• 4.8TB/s memory bandwidth
• Up to 700W TDP for H200 SXM

From an infrastructure perspective, this was important.

Performance per GPU increased, but the top-end thermal envelope remained around 700W.

For data-center engineers, this meant that cooling capacity remained a first-class design constraint.

Blackwell Changes the Unit of Measurement

With Blackwell, it becomes increasingly misleading to discuss infrastructure only by looking at one GPU.

A better unit is the complete server.

NVIDIA’s DGX B200 contains eight Blackwell GPUs and has a published maximum system power usage of approximately:

14.3 kW

That single 10RU system can therefore release roughly the same order of heat as several conventional server racks from an earlier generation.

This changes how the facility must be designed.

Power and Cooling Load Comparison

The following table translates representative NVIDIA power figures into approximate thermal loads.

Because almost all electrical power entering compute equipment ultimately becomes heat, electrical power can be used as a practical first-order estimate of required heat rejection.

The coolant-flow numbers below are DATAAD engineering estimates assuming water-like coolant and a 10°C temperature rise. They are not NVIDIA cooling specifications.

Actual liquid-cooling systems require additional engineering margin and must account for coolant properties, pressure drop, CDU efficiency, approach temperatures, flow balancing and facility-side heat rejection.

The table nevertheless shows the direction very clearly.

We are moving from kilowatts per server toward more than one hundred kilowatts per rack.

GB200 NVL72: The 120 kW Rack

The GB200 NVL72 represents one of the clearest milestones in this transition.

NVIDIA documentation describes a rack containing:

• 72 Blackwell GPUs
• 36 Grace CPUs
• 18 compute trays
• 9 NVLink switch trays
• Power shelves
• Busbar distribution
• Liquid cooling manifolds

The complete rack consumes approximately 120 kW.

At this level, liquid cooling is no longer an optional accessory.

It is part of the rack architecture.

Power Supply Modules Become Strategic Components

The accompanying field photograph of a U.S. power supply module company illustrates another side of the AI infrastructure market that receives less attention than GPUs.

A 120 kW AI rack cannot simply be connected like a conventional IT rack.

High-density AI systems require sophisticated conversion and distribution between facility AC power and the DC power used by compute equipment.

NVIDIA’s GB200 rack architecture uses multiple power shelves.

Each power shelf can use six approximately 5.5 kW power supply units, delivering up to roughly 33 kW per shelf, with multiple shelves integrated into the rack for power capacity and redundancy.

The power supply is therefore no longer a small supporting component hidden inside a server.

It is becoming rack infrastructure.

The AI Power Chain Is Getting Longer

A modern AI factory increasingly looks like this:

Grid → Transformer → UPS / Energy Storage → PDU → Power Shelf → PSU → Busbar → Compute Tray → GPU

At the same time, the thermal chain may look like:

GPU → Cold Plate → Hose / QD → Rack Manifold → CDU → Facility Water → Chiller / Cooling Tower

These two infrastructure chains must develop together.

There is little value in supplying 120 kW to a rack if the cooling infrastructure can remove only 70 kW.

Likewise, there is little value in installing a 150 kW cooling loop if the electrical system cannot reliably feed the rack.

Power and Cooling Are Becoming One Engineering Problem

The traditional data center treated electrical and mechanical infrastructure as related but largely separate disciplines.

AI is changing that.

GPU workload changes can produce rapid changes in power consumption.

Those changes also become rapid changes in heat generation.

Therefore power management and cooling management increasingly have to operate as one coordinated system.

This also increases the importance of:

• Power monitoring
• Coolant flow monitoring
• Temperature sensors
• Leak detection
• Intelligent CDU control
• Rack-level telemetry
• AI-based load management

The AI factory may eventually use AI to manage the infrastructure that operates AI.

Blackwell Ultra Pushes Rack Density Higher

Blackwell Ultra continues the trend.

NVIDIA’s current infrastructure documentation lists a designed rack power around 135 kW for GB300-class NVL72 infrastructure.

That is approximately 15 kW higher than the 120 kW design point associated with GB200.

The change may appear modest compared with the GPU performance increase, but at data-center scale it becomes significant.

Consider 1,000 racks.

An additional 15 kW per rack represents:

15 MW of additional electrical capacity

and approximately the same additional heat-removal requirement.

This is why relatively small rack-level changes become major utility and infrastructure issues when multiplied across an AI factory.

Vera Rubin Moves the Industry Further Toward Rack-Scale Computing

NVIDIA’s Vera Rubin platform continues the rack-scale strategy.

Vera Rubin NVL72 integrates:

• 72 Rubin GPUs
• 36 Vera CPUs
• Sixth-generation NVLink
• ConnectX-9 networking
• BlueField-4 DPUs

NVIDIA says the new system preserves the general rack-scale approach used by Blackwell while introducing new mechanical, power and cooling enhancements.

The important point is not simply that Rubin is faster.

It is that NVIDIA continues to treat the entire rack as the computer.

Rubin Brings HBM4 and Much Higher Data Movement

Each Rubin GPU is specified with:

• 288GB HBM4
• Up to 22TB/s memory bandwidth
• Sixth-generation NVLink
• Up to 3.6TB/s NVLink bandwidth per GPU

A Vera Rubin NVL72 rack therefore brings together an enormous amount of compute and memory movement inside one rack-scale architecture.

NVIDIA has not published a comparable public per-GPU TDP number on the current Rubin product page.

DATAAD therefore does not estimate one here.

But the architecture itself strongly reinforces the industry's move toward high-capacity liquid cooling and high-density power distribution.

Rubin CPX Is Expected at the End of 2026

Another upcoming development is Rubin CPX.

NVIDIA announced Rubin CPX for workloads involving massive context, including advanced coding and long-form generative video.

The planned Vera Rubin NVL144 CPX platform is expected to provide:

• 8 exaflops of AI compute
• 100TB of fast memory
• 1.7PB/s of memory bandwidth

in a rack-scale system.

NVIDIA has said Rubin CPX is expected to become available at the end of 2026.

Its public announcement does not yet provide a comparable rack-power figure.

That absence itself is a reason not to speculate.

The relevant point is that compute density continues to rise.

Beyond Rubin: Feynman

NVIDIA has also publicly identified Feynman as the generation that follows Rubin.

Detailed GPU TDP, rack power and cooling specifications are not yet publicly available.

However, NVIDIA’s direction is already clear:

the company is developing AI infrastructure on an increasingly rapid architecture cadence, while networking, CPUs, power systems and rack-scale design evolve together with the GPU.

What the Roadmap Means for Infrastructure Suppliers

The next NVIDIA generation does not create opportunity only for semiconductor companies.

Every increase in rack power creates demand elsewhere.

This is why the attached power supply module photograph is important.

The visible hardware may appear less exciting than a GPU.

But without power conversion, distribution and cooling infrastructure, the GPU cannot operate.

The Next Bottleneck May Be the Rack

For much of the AI boom, the industry worried about GPU availability.

The next constraint may increasingly be:

How many high-density GPU racks can a facility actually power and cool?

A data center may have enough floor space for another 100 racks.

But if each rack requires approximately 120 to 135 kW, those 100 racks could require roughly:

12 to 13.5 MW of IT power

before considering the rest of the facility overhead.

That changes site selection.

It changes transformers.

It changes substations.

It changes cooling plants.

And it changes the economics of the entire data center.

DATAAD View

NVIDIA’s development from A100 to Hopper, Blackwell and Rubin tells a larger story.

A100 established the high-density GPU server.

H100 pushed individual GPU TDP toward 700W.

Blackwell shifted the industry toward 14 kW-class servers and approximately 120 kW liquid-cooled racks.

Blackwell Ultra pushes the rack envelope further.

Rubin integrates even more compute, memory and networking into the rack-scale architecture.

Rubin CPX and Feynman indicate that the cycle is not ending.

The result is a fundamental change in the AI supply chain.

The companies that power the GPU, cool the GPU, connect the coolant, distribute electricity and remove the resulting heat are becoming part of the strategic AI infrastructure ecosystem.

The GPU may remain the engine of the AI revolution.

But power and cooling will determine how many of those engines can actually run.