Small Language Models: A Solution To Language Model Deployment At The Edge?


While Large Language Models (LLMs) like GPT-3 and GPT-4 have quickly become synonymous with AI, LLM mass deployments in both training and inference applications have, to date, been predominately cloud-based. This is primarily due to the sheer size of the models; the resulting processing and memory requirements often overwhelm the capabilities of edge-based systems. While the efficiency of Exped... » read more

To (B)atch Or Not To (B)atch?


When evaluating benchmark results for AI/ML processing solutions, it is very helpful to remember Shakespeare’s Hamlet, and the famous line: “To be, or not to be.” Except in this case the “B” stands for Batched. Batch size matters There are two different ways in which a machine learning inference workload can be used in a system. A particular ML graph can be used one time, preced... » read more

Extending The DDR5 Roadmap With MRDIMM


Given the voracious memory bandwidth and capacity demands of Gen AI and other advanced workloads, we’ve seen a rapid progression through the generations of DDR5 memory. Multiplexed Registered DIMMs (MRDIMMs) offer a new memory module architecture capable of extending the DDR5 roadmap and expanding the capabilities of server main memory. MRDIMM reuses the lion’s share of existing DDR5 infras... » read more

Harnessing Computational Storage For Faster Data Processing


By Ujjwal Negi and Prashant Dixit In the evolving landscape of data storage, computational storage devices (CSDs) are revolutionizing how we process and store data. By embedding processing capabilities within storage units, these devices enable in-situ data manipulation, minimizing data movement between storage and CPUs and dramatically improving performance and efficiency. This paradigm shi... » read more

Simulating Multiple DSPs As Multiple x86 Processes


An increasing number of embedded designs are multi-core systems. At the pre-silicon stage, customers use a simulation platform for architectural exploration and software development. Architects want to quantify the impact of the number of cores, local memory size, system memory latency, and interconnect bandwidth. Software teams wish to have a practical development platform that is not excrucia... » read more

Building Safe And Secure Software With Rust On Arm


The Rust Programming Language has gained the attention of government security agencies, and even the White House, due to its unique blend of safety, performance and productivity. Rust is designed to remove common programming burdens and handle issues like use-after-free errors at compile time. Remarkably, it achieves this without using a garbage collector, generating machine code that rivals th... » read more

Powering Mechanical Simulations: AMD Vs. Intel


When selecting the right central processing unit (CPU) for optimizing Ansys Mechanical structural finite element analysis (FEA) software performance, there are two major players to consider: Intel and AMD. Both have made significant advancements in recent years, but choosing between them depends on several factors that directly affect simulation speeds, scalability, and overall performance fo... » read more

Boost High-Performance IC Design Flows With Early Interactive Symmetry Checking


In the realm of high-performance IC (integrated circuit) design, symmetry is not just an aesthetic preference—it’s a critical factor for ensuring proper device functionality, especially in analog and RF designs. Achieving symmetry early in the design process helps to ensure consistent electrical behavior, which is essential for meeting performance goals and maintaining device reliability. H... » read more

In Memory, At Memory, Near Memory: What Would Goldilocks Choose?


The children’s fairy tale of ‘Goldilocks and the Three Bears’ describes the adventures of Goldi as she tries to choose among three choices for bedding, chairs, and bowls of porridge. One meal is “too hot,” the other “too cold,” and finally one is “just right.” If Goldi were faced with making architecture choices for AI processing in modern edge/device SoCs, she would also face... » read more

Green ICT: The IoT World Also Needs Ecological Awareness


By Volkhard Beyer and Dirk Mayer Microelectronics support the fight against climate change and for a better environment; for example, by intelligently controlling drive units, ensuring optimal operation of energy systems, and monitoring the resource requirements of production processes. Unfortunately, however, the ICT sector itself is one of the fastest-growing emitters of greenhouse gase... » read more

← Older posts