Difficult Memory Choices In AI Systems


The number of memory choices and architectures is exploding, driven by the rapid evolution in AI and machine learning chips being designed for a wide range of very different end markets and systems. Models for some of these systems can range in size from 10 billion to 100 billion parameters, and they can vary greatly from one chip or application to the next. Neural network training and infer... » read more

System-Level Packaging Tradeoffs


Leading-edge applications such as artificial intelligence, machine learning, automotive, and 5G, all require high bandwidth, higher performance, lower power and lower latency. They also need to do this for the same or less money. The solution may be disaggregating the SoC onto multiple die in a package, bringing memory closer to processing elements and delivering faster turnaround time. But ... » read more

Productivity Keeping Pace With Complexity


Designs have become larger and more complex and yet design time has shortened, but team sizes remain essentially flat. Does this show that productivity is keeping pace with complexity for everyone? The answer appears to be yes, at least for now, for a multitude of reasons. More design and IP reuse is using more and larger IP blocks and subsystems. In addition, the tools are improving, and mo... » read more

New Architectures, Much Faster Chips


The chip industry is making progress in multiple physical dimensions and with multiple architectural approaches, setting the stage for huge performance increases based on more modular and heterogeneous designs, new advanced packaging options, and continued scaling of digital logic for at least a couple more process nodes. A number of these changes have been discussed in recent conferences. I... » read more

Scaling AI/ML Training Performance With HBM2E Memory


In my April SemiEngineering Low Power-High Performance blog, I wrote: “Today, AI/ML neural network training models can exceed 10 billion parameters, soon it will be over 100 billion.” “Soon” didn’t take long to arrive. At the end of May, OpenAI unveiled a new 175-billion parameter GPT-3 language model. This represented a more that 100X jump over the size of GPT-2’s 1.5 billion param... » read more

The Race To Much More Advanced Packaging


Momentum is building for copper hybrid bonding, a technology that could pave the way toward next-generation 2.5D and 3D packages. Foundries, equipment vendors, R&D organizations and others are developing copper hybrid bonding, which is a process that stacks and bonds dies using copper-to-copper interconnects in advanced packages. Still in R&D, hybrid bonding for packaging provides mo... » read more

EDA On Board With New Package Options


A groundswell of activity around multi-die integration and advanced packaging is pushing EDA companies to develop integration strategies that speed up time to sign-off, increase confidence that a design will work as expected, while still leaving enough room for highly customized solutions. Challenges range from how to architect a design, how to explore the best options and configurations, ho... » read more

Power Impact At The Physical Layer Causes Downstream Effects


Data movement is rapidly emerging as one of the top design challenges, and it is being complicated by new chip architectures and physical effects caused by increasing density at advanced nodes and in multi-chip systems. Until the introduction of the latest revs of high-bandwidth memory, as well as GDDR6, memory was considered the next big bottleneck. But other compute bottlenecks have been e... » read more

ML Opening New Doors For FPGAs


FPGAs have long been used in the early stages of any new digital technology, given their utility for prototyping and rapid evolution. But with machine learning, FPGAs are showing benefits beyond those of more conventional solutions. This opens up a hot new market for FPGAs, which traditionally have been hard to sustain in high-volume production due to pricing, and hard to use for battery-dri... » read more

The Next Advanced Packages


Packaging houses are readying their next-generation advanced IC packages, paving the way toward new and innovative system-level chip designs. These packages include new versions of 2.5D/3D technologies, chiplets, fan-out and even wafer-scale packaging. A given package type may include several variations. For example, vendors are developing new fan-out packages using wafers and panels. One is... » read more

← Older posts