Data Centers Boost Voltage For Higher Efficiency

The power architecture used in HPC and AI data centers today is about to undergo a significant change in an effort to boost power efficiency. While voltages at the chip level will remain the same, the voltages leading to those chips will be kept higher for longer distances.

This change has broad implications for DC-DC converters. The existing architecture brings AC to each rack, converts it to DC, and then drops the voltage in two stages to the necessary chip voltage. The new approach, modeled on agreements established for the electric-vehicle (EV) market, moves AC conversion to the edge of a building, or the end of a row of racks, and supplies all the racks in that row with a higher DC voltage than is currently employed. The result is lower current, reduced losses, and less copper.

This change comes as data centers wrestle with ever-increasing energy demands, with no end in sight. “The power driven by artificial intelligence and GPUs and data centers is massively changing the way people need to approach things,” said Rich Goldman, director, electronics and semiconductors business unit at Ansys, now part of Synopsys. “You’re seeing it to the point where people are placing data centers close to energy sources.”

Today, every rack gets AC
Today’s data center power setup has AC distributed to all the racks. The tray at the bottom of each rack contains a power supply unit (PSU) that converts the AC to a nominal 48 VDC. Some systems are implemented at 54 VDC, if necessary, to charge batteries to 48 V.

“The power is converted down to DC, with several conversions to get down to a usable voltage for a high-performance chip,” said John Dinh, director of product marketing for computing at Amkor Technology.

That 48 V supplies all the upper servers, as well as the top-of-rack (ToR) switch. On each board, the voltage drops in two stages, and driver MOS (DrMOS) chips bring the final voltage to the chips being powered. “There’s a power module for the first stage where they convert from 48 V down to 12 V, or sometimes down to 6 V,” Dinh explained. “The second stage, where they would have an additional controller and DrMOS chips, will convert further down to 1 V, 3 V, 0.8 V, or whatever a chip requires.”

Fig. 1: Current high-level data-center power architecture. AC feeds a power-supply unit (PSU) at the bottom of each rack, where the AC is converted to 48 VDC (nominally). Further voltage drops occur on each server or board. Source: Bryon Moyer/Semiconductor Engineering

Because different chips may have different power-supply voltage requirements, the second stage is replicated multiple times. The DrMOS chips must supply sufficient current in addition to the necessary voltages. Each one has a current limit, so it may be necessary to gang multiples in parallel for the same power line to reach the specified current.

For example, NVIDIA has many such chips on its Grace/Hopper and Grace/Blackwell boards. “There are tons of DrMOS chips used in the second stage,” said Dinh. “GH200 uses 19 DrMOS for Grace and 56 DrMOS for Hopper, while GB200 uses 19 DrMOS for Grace and 130 DrMOS for Blackwell.”

This speaks to the challenge of the existing setup. “They have to change the architecture,” he said. “It’s no longer just populating more DrMOS around the GPU. It’s not sustainable.”

This is similar in spirit to a change in recent generations of DRAM, where modules now host their own PMICs, and it’s no accident. “The notion is like what they do in data centers, said John Eble, vice president of product marketing, memory interface chips at Rambus. “‘Let’s deliver a higher voltage and lower current to the point of use [in this case, a memory], and then let’s have a power management IC that can take that in and then generate all the necessary voltages.'”

Not the first voltage boost
The power architecture for data centers serves one purpose — to take delivered AC power and get as much of that power to the chips and other components as possible. That speaks to power efficiency, but that efficiency is relative. Early on, data center racks lived off 12 VDC, but as rack power grew beyond 15 kW, the industry moved to 48 V.

The reason at the time is the same as for the next set of proposed changes. Since power is calculated as VI, increasing the voltage results in proportionally less current for a given amount of power. And the amount of current has several implications for data centers.

First, the current determines the size of the wire necessary to connect power. Today’s setup is expected to require around 200 kg of copper per rack to support the enormous power that AI and other computationally intensive workloads demand. By dropping the current, data centers can be wired with less copper, which is a non-trivial investment.

The second impact relates to the difference in current-carrying capacity in a given wire for AC versus DC. The skin effect means that for AC, much of the current moves near the surface of the wire, with less flowing internally.

“At the system level, with the same amount of copper used for AC, you can deliver higher power using DC current,” said Dinh. That alone would provide more power through existing wires.

But reducing the amount of AC also could mean that a given current could travel in a smaller wire — the whole wire, not just the outer edges. This also can result in less copper.

Finally, conduction losses are proportional to the current. That means reducing the current results in less loss, and therefore higher efficiency. Keeping voltages higher for as much of the wiring as possible means dropping voltages as close to the chips as possible. That saves power, and it keeps power integrity high.

The new data-center power configuration
The proposed power architecture now moves the PSU out to the periphery of the data center, or at least into its own rack — sometimes called a sidecar rack — at the end of a row of racks.

“The next generation of rack-scale computing introduces the centralized power architecture,” explained Dinh. “Power’s getting its own rack. No longer is it just a tray at the bottom of the rack. In this architecture, thousands of volts of AC are converted to DC at the periphery of the data center. The voltage coming out is 800 VDC, and that will be delivered to the hall or to the row of racks.”

That removes the AC that now feeds each rack. Instead, the racks receive a high DC voltage and then drop that down to what the chips want, in multiple stages, on the boards near the load.

Fig. 2: The new data-center power architecture. AC is converted to 800 VDC, either near the edge of the building or at the end of a rack. That high voltage is then dropped down close to the chips it powers, reducing losses along the way. Source: Bryon Moyer/Semiconductor Engineering

This setup is called high-voltage direct current (HVDC) and leverages some of the work done for EV chargers. “Because of EV charging technology, this setup is cheaper and ready to deploy,” noted Dinh. “The transition to HVDC in data centers coincidentally intersects with the growth of EV charging infrastructure.”

The 800 V isn’t sacrosanct, and some believe 400 V is a more attainable number. But NVIDIA already publicly said its Kyber rack architecture, due to hit the market in the 2027 timeframe, will employ the 800 V setup and a sidecar rack to house the PSU and cooling infrastructure.

Of course, this will result in one AC-to-DC conversion per building or row rather than one per rack. With the existing setup, if one PSU fails, it only takes down a rack. If the new PSU fails, it takes down much more. “Having fewer components can potentially create a single point of failure. To address that, they have to provide redundancy,” Dinh added.”

Power FETs and power integrity
Today’s arrangement means that the semiconductors dropping the voltage from the PSU must be able to support voltages more than 50 V. That’s a far cry from the upcoming 800 V, which will require high-voltage semiconductors. It should raise demand for SiC transistors, which can handle the high voltages far better than silicon and GaN. Effectively, some of the designs in place for EVs now should be adaptable, easing the transition.

“We have seen a surge in the requirements for power devices,” said Marc Swinnen, director of product marketing at Ansys. “Power FETs are ubiquitous. There’s so much power to be switched that it requires a power FET.”

The need for power-integrity verification becomes even stronger for such chips because slight changes in design parameters can make a big difference.

“There are specific tools designed specifically to analyze power transistors in great detail,” said Swinnen. “They use 2D meshing because the amount of power they’re carrying is so great that the resistance, the R_DSon, is the most important parameter. The R_DSon has to be calculated very accurately, and you want to identify exactly where the voltage drops are so there are no hot spots, where too much voltage drops around a corner or something and causes local heating. It’s not the traditional power integrity analysis with just RC.”

Temperature plays an important role here, owing to the positive feedback loop it can stoke. “Power is often a proxy for thermal,” continued Swinnen. “When people say, ‘I’m worried about power,’ what they’re really worried about is that it’s going to get too hot. And there’s feedback, because the power that a chip produces depends on its temperature. So a hotter chip will draw more power. And, of course, more power means a hotter chip. It is a chicken-and-egg thing.”

Not coming to every data center
Given the number of data centers currently in operation, it would be prohibitive to convert them all to this new architecture. That’s both impractical and unnecessary. Few workloads demand the kind of power this architecture provides. Rather, it’s become an issue as a result of extrapolating the kind of power that future AI workloads are expected to require. Applications other than AI and high-performance computing (HPC) can continue on the current infrastructure.

Meanwhile, makers of power-delivery infrastructure (electrical and mechanical) are gearing up to support this architecture. NVIDIA’s commitment removes some of the risk, and installation is expected in a couple of years.

Related Reading
New Approaches To Power Decoupling
Strategic placement of decoupling capacitors and regulators is critical to keeping power stable, but no one approach is sufficient.
Crisis Ahead: Power Consumption In AI Data Centers
Four key areas where chips can help manage AI’s insatiable power appetite.
Backside Power Delivery Nears Production
Breakthrough approach delivers better scaling and power efficiency, but at the cost of new processes like wafer thinning, bonding, and advanced debug.