Thanks For The Memories!


“I want to maximize the MAC count in my AI/ML accelerator block because the TOPs rating is what sells, but I need to cut back on memory to save cost,” said no successful chip designer, ever. Emphasis on “successful” in the above quote. It’s not a purely hypothetical quotation. We’ve heard it many times. Chip architects — or their marketing teams — try to squeeze as much brag-... » read more

Dramatic Changes Ahead For Chips And Systems


Early this year, most people had never heard of generative AI. Now the entire world is racing to capitalize on it, and that's just the beginning. New markets, such as spatial computing, quantum computing, 6G, smart infrastructure, sustainability, and many more are accelerating the need to process more data faster, more efficiently, and with much more domain specificity. Compared to the days ... » read more

Vision Transformers Change The AI Acceleration Rules


Transformers were first introduced by the team at Google Brain in 2017 in their paper, "Attention is All You Need". Since their introduction, transformers have inspired a flurry of investment and research which have produced some of the most impactful model architectures and AI products to-date, including ChatGPT which is an acronym for Chat Generative Pre-trained Transformer. Transformers a... » read more

Making Tradeoffs With AI/ML/DL


Machine learning, deep learning, and AI increasingly are being used in chip design, and they are being used to design chips that are optimized for ML/DL/AI. The challenge is understanding the tradeoffs on both sides, both of which are becoming increasingly complex and intertwined. On the design side, machine learning has been viewed as just another tool in the design team's toolbox. That's s... » read more

Disaggregating And Extending Operating Systems


The push toward disaggregation and customization in hardware is starting to be mirrored on the software side, where operating systems are becoming smaller and more targeted, supplemented with additional software that can be optimized for different functions. There are two main causes for this shift. The first is rising demand for highly optimized and increasingly heterogeneous designs, which... » read more

Chiplets Taking Root As Silicon-Proven Hard IP


Chiplets are all the rage today, and for good reason. With the various ways to design a semiconductor-based system today, IP reuse via chiplets appears to be an effective and feasible solution, and a potentially low-cost alternative to shrinking everything to the latest process node. To enable faster time to market, common IP or technology that already has been silicon-proven can be utilized... » read more

Will Floating Point 8 Solve AI/ML Overhead?


While the media buzzes about the Turing Test-busting results of ChatGPT, engineers are focused on the hardware challenges of running large language models and other deep learning networks. High on the ML punch list is how to run models more efficiently using less power, especially in critical applications like self-driving vehicles where latency becomes a matter of life or death. AI already ... » read more

What’s The Difference Between An NPU And A GPNPU?


To understand the difference between an NPU (neural processing unit) and a GPNPU (general-purpose neural processing unit) let’s start with the NPU, a processing engine that accelerates machine learning (ML) workloads in System on Chip (SoC) designs. Click here to read more. » read more

Week In Review: Auto, Security, Pervasive Computing


Automotive, mobility Automaker Toyota and Texas-based electricity distributor Oncor Electric Delivery (Oncor) are embarking on a vehicle-to-grid (V2G) pilot project to explore the feasibility of transferring energy from BEVs’ batteries back to the grid. Toyota and Oncor want to better understand the interconnectivity between BEVs and utilities. The project will start testing at Oncor’s res... » read more

ML Focus Shifting Toward Software


New machine-learning (ML) architectures continue to garner a huge amount of attention as the race continues to provide the most effective acceleration architectures for the cloud and the edge, but attention is starting to shift from the hardware to the software tools. The big question now is whether a software abstraction eventually will win out over hardware details in determining who the f... » read more