tag: inferencing

Data Formats For Inference On The Edge
By Brian Bailey - 14 Dec, 2023 - Comments: 0

AI/ML training traditionally has been performed using floating point data formats, primarily because that is what was available. But this usually isn't a viable option for inference on the edge, where more compact data formats are needed to reduce area and power. Compact data formats use less space, which is important in edge devices, but the bigger concern is the power needed to move around... » read more

AI Accelerator Architectures Poised For Big Changes
By Ann Mutschler - 04 Dec, 2023 - Comments: 5

AI is driving a frenzy of activity in the chip world as companies across the semiconductor ecosystem race to include AI in their product lineup. The challenge now is how to make AI run faster, use less energy, and to be able to leverage it from the edge to the data center — particularly with the rollout of large language models. On the hardware side, there are two main approaches for accel... » read more

Improving Image Resolution At The Edge
By Ed Sperling - 30 May, 2023 - Comments: 0

How much cameras see depends on how accurately the images are rendered and classified. The higher the resolution, the greater the accuracy. But higher resolution also requires significantly more computation, and it requires flexibility in the design to be able to adapt to new algorithms and network models. Jeremy Roberson, technical director and software architect for AI/ML at Flex Logix, talks... » read more

Optimizing Projected PCM for Analog Computing-In-Memory Inferencing (IBM)
By Technical Paper Link - 22 May, 2023 - Comments: 0

A new technical paper titled "Optimization of Projected Phase Change Memory for Analog In-Memory Computing Inference" was published by researchers at IBM Research. "A systematic study of the electrical properties-including resistance values, memory window, resistance drift, read noise, and their impact on the accuracy of large neural networks of various types and with tens of millions of wei... » read more

Low-Power Heterogeneous Compute Cluster For TinyML DNN Inference And On-Chip Training
By Technical Paper Link - 09 Apr, 2023 - Comments: 0

A new technical paper titled "DARKSIDE: A Heterogeneous RISC-V Compute Cluster for Extreme-Edge On-Chip DNN Inference and Training" was published by researchers at University of Bologna and ETH Zurich. Abstract "On-chip deep neural network (DNN) inference and training at the Extreme-Edge (TinyML) impose strict latency, throughput, accuracy, and flexibility requirements. Heterogeneous clus... » read more

Review of Tools & Techniques for DL Edge Inference
By Technical Paper Link - 16 Feb, 2023 - Comments: 0

A new technical paper titled "Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review" was published in "Proceedings of the IEEE" by researchers at University of Missouri and Texas Tech University. Abstract: Successful integration of deep neural networks (DNNs) or deep learning (DL) has resulted in breakthroughs in many areas. However, deploying thes... » read more

Where And Why AI Makes Sense In Cars
By Ed Sperling - 02 Feb, 2023 - Comments: 0

Experts at the Table: Semiconductor Engineering sat down to talk about where AI makes sense in automotive and what are the main challenges, with Geoff Tate, CEO of Flex Logix; Veerbhan Kheterpal, CEO of Quadric; Steve Teig, CEO of Perceive; and Kurt Busch, CEO of Syntiant. What follows are excerpts of that conversation, which were held in front of a live audience at DesignCon. Part two of this... » read more

Complex Tradeoffs In Inferencing Chips
By John Koon - 17 Oct, 2022 - Comments: 0

Designing AI/ML inferencing chips is emerging as a huge challenge due to the variety of applications and the highly specific power and performance needs for each of them. Put simply, one size does not fit all, and not all applications can afford a custom design. For example, in retail store tracking, it's acceptable to have a 5% or 10% margin of error for customers passing by a certain aisle... » read more

Techniques For Improving Energy Efficiency of Training/Inference for NLP Applications, Including Power Capping & Energy-Aware Scheduling
By Technical Paper Link - 17 Aug, 2022 - Comments: 0

This new technical paper titled "Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models" is from researchers at MIT and Northeastern University. Abstract: "The energy requirements of current natural language processing models continue to grow at a rapid, unsustainable pace. Recent works highlighting this problem conclude there is an urgent need ... » read more

Can Analog Make A Comeback?
By Brian Bailey - 09 Jun, 2022 - Comments: 3

We live in an analog world dominated by digital processing, but that could change. Domain specificity, and the desire for greater levels of optimization, may provide analog compute with some significant advantages — and the possibility of a comeback. For the last four decades, the advantages of digital scaling and flexibility have pushed the dividing line between analog and digital closer ... » read more

← Older posts

  • Sponsors

About

Navigation

Connect With Us

Copyright ©2013-2024 SMG   |  Terms of Service  |  Privacy Policy