Modeling AI Inference Performance


The metric in AI Inference that matters to customers is either throughput/$ for their model and/or throughput/watts for their model. One might assume throughput will correlate with TOPS, but you’d be wrong. Examine the table below: The Nvidia Tesla T4 gets 7.4 inferences/TOP, Xavier AGX 15 and InferX 1 34.5. And InferX X1 does it with 1/10th to 1/20th of the DRAM bandwidth of the ... » read more

Revving Up For Edge Computing


The edge is beginning to take shape as a way of limiting the amount of data that needs to be pushed up to the cloud for processing, setting the stage for a massive shift in compute architectures and a race among chipmakers for a stake in a new and highly lucrative market. So far, it's not clear which architectures will win, or how and where data will be partitioned between what needs to be p... » read more

eFPGA as Silicon Debugger


A variety of debugging features are available by implementing the Flex Logix embedded FPGA cores. This includes: • High observability—the many eFPGA IOs allow for thousands of signals to be monitored • Event detection—the eFPGA logic can be configured to detect very complex signal patterns to help identify meaningful events • High visibility—data can be logged by the eLA inside t... » read more

Using Multiple Inferencing Chips In Neural Networks


Geoff Tate, CEO of Flex Logix, talks about what happens when you add multiple chips in a neural network, what a neural network model looks like, and what happens when it’s designed correctly vs. incorrectly. » read more

Advantages Of BFloat16 For AI Inference


Essentially all AI training is done with 32-bit floating point. But doing AI inference with 32-bit floating point is expensive, power-hungry and slow. And quantizing models for 8-bit-integer, which is very fast and lowest power, is a major investment of money, scarce resources and time. Now BFloat16 (BF16) offers an attractive balance for many users. BFloat16 offers essentially t... » read more

Week In Review: Design, Low Power


eSilicon debuted its 7nm high-bandwidth interconnect (HBI)+ PHY IP, a special-purpose hard IP block that offers a high-bandwidth, low-power and low-latency wide-parallel, clock-forwarded PHY interface for 2.5D applications such as chiplets. HBI+ PHY delivers a data rate of up to 4.0Gbps per pin. Flexible configurations include up to 80 receive and 80 transmit connections per channel and up to 2... » read more

Open ISAs Gaining Traction


Open instruction set architectures are starting to gain a foothold, often in combination with other processors, as chipmakers begin to add more specialized compute elements and more flexibility into their designs. There are a number of these open ISAs available today, including Power, MIPS, and RISC-V, and there are a number of permutations and tools available for sale based on those archite... » read more

Memory Subsystems In Edge Inferencing Chips


Geoff Tate, CEO of Flex Logix, talks about key issues in a memory subsystem in an inferencing chip, how factors like heat can affect performance, and where these kinds of chips will be used. » read more

FPGA Design Tradeoffs Getting Tougher


FPGAs are getting larger, more complex, and significantly harder to verify and debug. In the past, FPGAs were considered a relatively quick and simple way to get to market before committing to the cost and time of developing an ASIC. But today, both FPGAs and eFPGAs are being used in the most demanding applications, including cloud computing, AI, machine learning, and deep learning. In some ... » read more

Autonomous Vehicles Are Reshaping The Tech World


The effort to build cars that can drive themselves is reshaping the automotive industry and its supply chain, impacting everything from who defines safety to how to ensure quality and reliability. Automakers, which hardly knew the names of their silicon suppliers a couple of years ago, are now banding together in small groups to share the costs and solve technical challenges that are well be... » read more

← Older posts