A Chiplet-Based Supercomputer For Generative LLMs That Optimizes Total Cost of Ownership


A technical paper titled "Chiplet Cloud: Building AI Supercomputers for Serving Large Generative Language Models" was published by researchers at University of Washington and University of Sydney. Abstract: "Large language models (LLMs) such as ChatGPT have demonstrated unprecedented capabilities in multiple AI tasks. However, hardware inefficiencies have become a significant factor limiting ... » read more

How To Justify A Data Center


The breadth of cloud capabilities and improvements in cost and licensing structures is prompting chipmakers to consider offloading at least some of their design work into the cloud. Cloud is a viable business today for semiconductor design. Over the past decade, the interest in moving to cloud computing has grown from an idea that was fun to talk about — but which no one was serious about ... » read more

Will PAYGO Shake Up How We Pay for Chips?


System builders are used to buying integrated circuits on a simple transactional basis — the chip has a price, and that’s what you pay. But some application spaces may have a wide variety of capabilities that need hardware support, and each feature may not be used for every instance. Traditionally, one would design different chips for different feature mixes and price points. But a new p... » read more

Sweeping Changes Ahead For Systems Design


Data centers are undergoing a fundamental change, shifting from standard processing models to more data-centric approaches based upon customized hardware, less movement of data, and more pooling of resources. Driven by a flood of web searches, Bitcoin mining, video streaming, data centers are in a race to provide the most efficient and fastest processing possible. But because there are so ma... » read more

Improving Power & Performance Beyond Scaling


Steven Woo, Rambus fellow and distinguished inventor, discusses architectural changes inside of servers and data centers to allow pooling of resources such as memory. That has a big impact on power efficiency and overall performance, but it also allows data centers to customize their architectures and prioritized resources with much more granularity than they can do today. » read more

Power Markets


There has been an ongoing discussion in the industry about the importance of power and performance and which is more important. I submit that the real question is: How much performance can be squeezed out of the power budget for any given market segment? Figure 1. Processor Market Segment Power Budgets Figure 1 shows a rough breakdown of the different market segments for processors, alo... » read more