Voltage Regulation Moves Into The Package

Integrated circuits require a variety of voltages and a wide range of currents, typically supplied by voltage regulators. But increasing power density is resulting in higher power delivery losses.

Moving those regulators closer to the chips they power can reduce those losses. Co-packaging them holds the most promise, but it comes with challenges.

“What people have been talking about, even folks at OCP last year, is that they want to integrate the voltage regulator on the substrate with the compute load,” said John Dinh, director of product marketing for computing at Amkor Technology.

The farther away the power supply is from the chip it powers, the higher the resistance, capacitance, and inductance — and consequently, the higher the losses in delivering that power. Increased currents further exacerbate the situation. Bringing power closer not only reduces the loop size (and hence inductance), but also reduces the total resistance and capacitance of the power lines.

Fig. 1: Shorter current loops. Current in a conventional package must run from the regulator through the PC board and, from there, into the processor and back, making for a long, inductive loop. Placing the regulator within the package dramatically shortens that loop. Source: Ferric

Ever higher power
The motivation for moving regulators originates in the continual increase in power density in data center servers for high-intensity workloads such as HPC and AI. All stops are out in efforts to build new sources of energy and to reduce power. Of course, power is the rate of energy use, and reducing it still can result in greater energy consumption if the power reduction results in larger workloads. Since workloads show no signs of slowing their growth, efforts continue to reduce power.

“As is evident from leading device-maker roadmaps, performance and power requirements will continue to grow in the foreseeable future,” said Vikas Gupta, director of engineering and technical promotion at ASE Group. “Total board power is increasing drastically, with device TGP (total graphics power) already well over 1,000 watts. AI servers are projected to consume over 1,000 kW of power. Current AI servers use 130 to 250 kW, but future AI servers will require 250 to 900 kW, with up to 576 GPUs per rack by 2026 to 2027. As device TGP continues to scale well beyond 1 kW, advanced packaging becomes mission-critical for sustaining performance gains in AI, HPC, and next-gen GPUs.”

Trends are similar for current. “Today, currents are in the 1,000 to 1,500 ampere range, and they’re talking about easily 2× going forward,” said Mukund Krishna, senior manager, product marketing at Empower Semiconductor. “At 3,000 A, you’re starting to hit the electromigration limits.”

Poor power delivery, however, can undermine efforts to save power. Each new generation of processor attempts to improve its power efficiency, if not outright power. But power losses elsewhere might counterbalance those savings.

“Essentially, you’re burning so much power getting [power] into the processor that this processor is no longer more energy efficient,” noted Noah Sturcken, CEO of Ferric. “It doesn’t provide better performance per watt than the previous generation. We’ve had customers who have told us, ‘If we don’t figure this out, I’m concerned my company is going to stop making these processors, and I’m going to have to find something else to do.’”

Ever closer power
Systems used to have a single power supply for the entire unit, much as small consumer gadgets tend to have today. But systems have become large and unwieldy with ever-growing power consumption. A single power supply clearly is inadequate for data centers, which explains why power delivery now takes place in trays in a rack (for AC-to-DC conversion), and in individual servers or boards (DC-to-DC).

“The development here is like the shift to point-of-load regulators that happened more than 50 years ago,” explained Sturcken. “You would have previously had a board and cables bringing power into that board. And then people realized that the conduction losses and the challenges with regulation were so significant that they needed to bring those power converters onto that board. With microprocessors, the power density has become so high that the package substrate for the microprocessor is now too far away from the power converters on the motherboard.”

That distance creates a long current loop between the regulator and load chip, which raises impedance, and it becomes particularly lossy with high currents at high frequencies.

“It’s a long, circuitous path with a lot of impedance, resistance, and inductance, and so you have resistive losses as well as resonance and transients,” said Sturcken. Moving regulators closer to the load shrinks that loop. “If you’ve miniaturized the power converter so that it’s small enough to go into the packaging of the microprocessor, then you dramatically reduce that resistance and inductance between the VR and the load.”

Switching regulators also require additional components. “The majority of the converters that we’re talking about are switching converters,” said Krishna. “And those need other passive components.”

Traditionally, those passives have been placed on the PCB next to the regulator. But delivering the kind of power density necessary for high-intensity workloads means shrinking the regulator. Faster switching can make do with smaller passives, so newer regulators integrate the passives into the regulator package. The industry has termed these integrated voltage regulators, or IVRs.

“IVRs are an attempt to integrate all of that into ideally a single-chip solution,” said Empower’s Krishna, noting they should be easier for designers to employ than older models. “The burden is no longer on the system implementer to choose their inductors or capacitors.”

Two DC stages
DC-to-DC conversion typically starts with 48 V (or sometimes 54 V), and first drops it to 12 or 6 V. Then it is further dropped to the specific DC voltage that a chip requires. These two voltage-drop stages may sit in different places on the board.

In general, one wants to get power delivery as close to the load (in this case, the chip) as possible. So high-power processors are typically surrounded by a large number of driver MOS (DrMOS) chips. These integrate the driver MOSFETs and power FETs and form part of an overall voltage regulator or power-management IC (PMIC). The power FETs carry the current that will be delivered to the load, whereas the drivers turn the power FETs on or off per control signals while boosting the current and voltage necessary for turning on and off the power FET quickly.

As an example, NVIDIA’s CPU/GPU boards require dozens of DrMOS chips, according to Amkor. The high number is necessary partly to provide different voltages, but mostly to provide sufficient current since they can be ganged in parallel. In general, regulators work most efficiently below their maximum current, so the most efficient boards may have more regulators than ones that try to save money by reducing the number of regulators and then running them full bore.

These power chips take up a fair bit of space on the board, but at the anticipated current densities, they’re still too far from their load. Moving them closer is happening in two phases — changing from horizontal to vertical delivery, and then co-packaging the regulators.

From horizontal to vertical
The typical power architecture today mounts regulator chips on the boards next to the processors they power. The current travels from the regulator into the board, through traces toward the chip, and then up from the board to the chip again. This is called horizontal delivery.

Fig. 2: Horizontal power delivery. The voltage regulators (VRs) are mounted on the same surface as the processor. Power goes vertically into the PCB, then horizontally under the processor, and vertically back up. Source: Bryon Moyer/Semiconductor Engineering

The challenge is that these processor chips are very large, so the distance from the regulator to the power bumps on the processor is growing. In general, the larger the processor die becomes, the lower the current density that might be expected since the current spreads out over a larger area. But it’s not playing out that way.

“The dimensions of the processors themselves are getting bigger, but the density of the logic within those processors is getting bigger at an even faster rate,” said Sturcken, resulting in a net increase in power density. “Microprocessors are going to consume more and more power, and the voltages they use are going to continue gradually decreasing. The current levels are going to go up at a fast rate, and eventually, the losses that you have from conduction, the regulation losses because of the high current levels, and the impedance of the power delivery network, are going to become prohibitive bottlenecks for microprocessors.”

That means more current traveling a longer distance, with increased power delivery losses as a result. “We have customers who are pushing their power density to beyond 5 A/mm², and the total processor power consumption is exceeding 5 kW,” said Sturcken. “They realize they could be losing 10% to 20% of the total power just in conduction and power-delivery losses.”

An alternative being implemented now is vertical power delivery. This depends on thinner regulator chips that allow them to be mounted on the backside of the processor (or even embedded into the core of the PCB), reducing the distance to the load from many millimeters (or more) to less than 5mm.

Fig. 3: In vertical power delivery, the voltage regulators (VRs) are mounted on the opposite side of the PCB, directly under the processor, shortening the distance that current travels. Source: Bryon Moyer/Semiconductor Engineering

“For lateral power-delivery solutions, PCB and package resistance result in power loss,” said ASE’s Gupta. “Vertical power delivery solutions address the routing losses by reducing the resistance attributed to shorter trace lengths in vertical power delivery configurations. Vertical power-delivery configurations coupled with final voltage step-downs as close to the point of load further improve power-delivery efficiency.”

Down to micron-length delivery
Vertical delivery brings power as close as physically possible to packaged chips. The only way to get closer is to integrate the regulator onto — or into — the package containing the chips to be powered. Co-packaged IVR schemes place the regulators on the package substrate, making this an advanced packaging technique.

If the package has an interposer, the regulator can be mounted on the interposer, but the lateral lines aren’t suitable for delivering power. “The interposer interconnect is typically too resistive to move power laterally,” explained Sturcken. “We do have customers that place an IVR on their silicon interposer, but they need to ‘via down’ to the organic package substrate in order to efficiently move the power laterally.”

Although the integration may sound straightforward, at least two issues complicate matters. First is simply the size of the regulator chips. Granted, as bare dies on a substrate, they’ll be physically smaller than the packaged versions being soldered to a board. But with the numbers we’re talking about, it still must be further shrunk to minimize the area per ampere delivered.

Such a small chip can be integrated in a number of ways. It can be mounted outside the bottom of the package in a tighter vertical-delivery loop, mounted atop the substrate in a tighter horizontal-delivery loop, or embedded within the substrate itself.

Fig. 4: Regulator placement at the package level. The regulators can be mounted on the bottom of the package substrate, placed on top of the substrate inside the package, or even embedded within the substrate. Source: Ferric

As with all the other active elements being co-housed in a single package, voltage regulators themselves dissipate power and generate heat. That becomes yet another thermal challenge in an advanced package.

“One kilowatt at 90% efficiency still dissipates 100 W,” said Krishna. “All these solutions are themselves dissipating a lot of power. How do you get that heat out?”

Ferric believes that, assuming the IVR is placed on the substrate, the heat will dissipate straight down. “It’s an efficient power converter, so even though a lot of power is going through it, not that much power is being dissipated in it, and we can dissipate that power through the board,” said Sturcken.

Inductors are a challenge
The second issue for co-packaged IVR is less straightforward — inductors. Switching regulators require capacitors and inductors. Capacitors have already been miniaturized, and in some cases they can be integrated monolithically on the chip. That hasn’t been the case with inductors.

Inductors need loops, and more inductance means more loops (classically in a helical structure). That makes an inductor a 3D element that’s not easy to build monolithically since chips tend to prefer planar structures. For this reason — along with the fact that today’s inductors are too large to integrate — inductors have always been implemented as separate components on circuit boards.

“We want to miniaturize this inductor, and we also want to operate the whole system at a faster switching frequency, which allows us to miniaturize the required inductance value and capacitance values,” said Sturcken.

This has become a challenge for more than one company. Ferric developed a new material that allows it to get the necessary inductance values with inductors small enough to integrate. Empower Semiconductor worked with an inductor partner to reduce losses on small inductors. Both consider their technologies to be proprietary, and neither has disclosed details.

“This is providing 10, 20, sometimes 50× miniaturization by volume relative to the next best alternative,” said Sturcken when discussing Ferric’s new inductor. “This is attractive for products that have a lot of different domains, something like an FPGA that has a lot of reconfigurable I/O on the perimeter. It has such a low profile that you can implement it in PCIe and OCP accelerator modules that typically don’t provide enough profile for conventional vertical power delivery.”

New materials may allow for smaller inductors, in general, but system builders are also experimenting with ways to integrate inductance as part of the packaging process. Details of the latter are unavailable at present because these solutions originate with the systems companies, not the packaging companies, and they’re considered proprietary techniques.

Coming to some systems…but not all
The extra effort necessary for IVR makes sense only for applications suffering from increasing power density, which largely means chips destined for the data center. Recent interest in this kind of solution has ballooned. “We’ve had a lot of customer engagement, but in a lot of cases, the customer said that [closer regulation] would be a nice feature to have, but they didn’t need it right then,” noted Sturcken. “And so adoption was gradual. That’s changed just within the past 12 months.”

Whether this becomes a cost-effective solution also depends on implementation. “There’s an opportunity to reduce the bill of materials cost, but it will depend on how the customer puts together the rest of the system,” Sturcken explained. “Our products will have a max output current and an output current level that’s a little bit less than the max output current, where the efficiency is best. So a customer could choose to go with the cheapest solution, in which case they’re using the fewest number of parts and operating those parts at their maximum load current. Or, they could get better conversion efficiency and maybe put two parts down instead of one so they’re each able to operate at their peak efficiency points.”

Ferric says its chips are also being designed into industrial and aerospace systems. The company said it reduced the number of external regulators for an FPGA from 30 to 1.

The other processor category that one might expect to benefit from the low profiles this solution provides would be mobile phone chips. But the extra effort for integrating the power with the processor may not be worthwhile. “Cell phones dissipate most of their power in the display and the radio,” noted Sturcken. “You’re not able to make as much of an impact on battery life.”

More generally, advanced packaging is an expensive endeavor and will be employed only where necessary due to high currents. “If you can keep the IVR on the board, that’s actually the most straightforward thing,” noted Krishna. “You don’t need to burden the package.”

Co-packaged IVR is part of an overall power-architecture rethink that’s happening in the data center. Within a couple of years, new data-center power configurations should start to appear. Although much of it has nothing to do with co-packaged IVR specifically, that’s a similar timeframe for systems that leverage the technology to hit the market.

Related Reading
Crisis Ahead: Power Consumption In AI Data Centers
Four key areas where chips can help manage AI’s insatiable power appetite.
Three-Way Race To 3D-ICs
Intel, TSMC, and Samsung are developing a broad set of technologies and relationships that will be required for the next generation of AI chips.