PCIe Benefits From AI, Despite Scaling Protocols


Key takeaways: PCIe remains a critical technology for non-AI processing. For AI, PCIe will be strengthened by scale-out, agentic AI, and even some scale-up. CXL is seeing uptake, and some even think it could participate in AI processing. PCIe has been the go-to network for most data traffic moving from a processor to devices located elsewhere, which is also what the new data... » read more

Memory For AI At The Edge


Inferencing at the edge has very different needs than training large language models or large-scale inferencing in AI data centers. Many edge devices run on a battery. They're price-sensitive, and they are constrained by the physical area of the device. As a result, the amount of memory that can be packed into these devices is also limited. Steve Woo, Rambus fellow and distinguished inventor, t... » read more

Security in Data Centers for AI Applications


AI data centers are the engines of the new data revolution, transforming data lakes and extracting meaningful insights guided by user queries. In this white paper, we revisit the security problem and highlight that AI data centers pose specific risks whose impact extends far beyond initial expectations. Starting from the premise that the AI is “only as good as the data that comes in/out”, w... » read more

Chiplets 2026: Where Are We Today?


Jim Handy of Objective Analysis and Jawad Nasrullah from Palo Alto Electron kicked off last week's Chiplet Summit with predictions about where the chiplet market is headed and why chiplets are needed to accelerate AI. Handy noted that in the 1990s, multi-chip modules (MCMs) led to mid-'90s multi-chip packages (MCPs), and then progressed to NAND flash stacking, stacked die, big chips (e.g., X... » read more

Liquid Cooling Gains Traction In Data Centers


All electronics generate heat, and that heat must be removed to ensure those electronics don’t overheat. Moving air has been the predominant approach for decades, with liquid cooling limited to particularly intense computing workloads, largely in the supercomputing domain. With the rise in AI, data-center power density has grown to the point where liquid cooling is now seeing a larger buil... » read more

Tracking Your Preferences


I like to use my last blog of the year to focus on you, the reader. You provide valuable feedback to me and the rest of the team at Semiconductor Engineering. What do you want to see us write about? How in-depth should things be? This is always a balance between the amount of information provided and the rate at which readers tire with an article. My focus is the channels I write for – Sys... » read more

Ebook: The Impact of AI On Data Center Design


AI is reshaping the data center industry. Rising power demands, advanced cooling needs, and digital twin technology are redefining how facilities are designed and operated. Download our free ebook on AI-optimized data centers to learn: How AI workloads are driving massive increases in power and cooling requirements Why liquid cooling is becoming essential for AI infrastructure ... » read more

Implementing Power Dynamic Response For Greener AI Data Centers (Univ. of Cambridge, Nyobolt, Nanyang Tech)


A new technical paper titled "Improving AI Efficiency in Data Centres by Power Dynamic Response" was published by researchers at University of Cambridge, Nyobolt Limited and Nanyang Technological University. Abstract "The steady growth of artificial intelligence (AI) has accelerated in the recent years, facilitated by the development of sophisticated models such as large language models and... » read more

In-System Test For AI Data Centers


Testing inside the fab or packaging house can determine whether a chip or package meets all the functional requirements at time zero, but how that chip behaves in the field during its lifetime and under different workloads and environmental conditions may be very different. This is particularly true in AI data centers, where utilization of one or more dies may be significantly higher than in pr... » read more

Challenges In Stacking HBM


AI data centers are pushing for higher density in high-bandwidth memory. Today, the maximum number of layers that can be stacked is 8, but that increases to as many as 24 layers by 2030. The big challenge will be in the interconnects, and making sure the microbumps align. At 16 layers, the bump pitch will be less than 10 microns, and the dies will be thinner. Damon Tsai, head of product marketi... » read more

← Older posts