Software Stack For Edge AI Performance

Developing an agile software stack is important for successful AI deployment on the edge. We regularly encounter new machine learning models created from multiple AI frameworks that leverage the latest primitives and state-of-the-art ML model topologies. This Cambrian explosion has resulted from a fertile open-source community that has embraced AI and is now fueling a wide proliferation of ML models on the edge.

Models are being created by academia and industry alike and produced using cutting-edge tools and concepts; hence, running these models efficiently on a high-performance, up-to-date AI software stack can be a complex undertaking in any organization.

Cadence produces foundational design IPs built from years of domain expertise. Tensilica IPs are prevalent in applications ranging from audio, vision, radar, automotive, and microcontroller-based sub-systems.

Given the reach of Cadence IP and because AI inferencing is proliferating across various market segments, we believe in a common “one-tool” solution that provides customers with a uniform, scalable, and configurable ML software stack. Our one-tool philosophy is at the forefront of our overall AI software offerings.

Providing one common software interface and tooling will significantly improve the time to market for our customers and empower them to be better prepared for a continuously evolving AI market.

Different use cases can be enabled on the edge, ranging from very low-power, always-on use cases to the high-end automotive class of applications for ADAS and autonomy. These platforms could have AI compute requirements ranging from a few giga operations per second (GOPS) to hundreds of terra operations per second (TOPS).

Fig. 1: Common AI software tooling across all market verticals.

We aim to improve our overall software offering in all aspects expected of a high-performance machine-learning compiler stack. These range from uniform front-end design, common high-level intermediate representation, support of mixed precision quantization formats, lossless compression techniques, graph and tensor optimizations, etc.

These techniques will help customers in various AI deployment scenarios, such as ahead-of-time compilation, just-in-time compilation, and/or runtime interpreter frameworks using Delegate-based frameworks.

Fig. 2: Pillars of AI software stack.

Customers will be able to do early pre-silicon ML workload simulation and extract key KPIs much in advance of their silicon tape-out. Providing these accurate metrics is very valuable during the hardware-software co-design phase. Iterating through various ML models, fine-tuning configurable knobs to extract the maximum performance from the hardware IP is the fundamental goal of every SoC architect.

Fig. 3: Enabling hardware-software co-design.

Cadence AI software stack not only leverages open-source tools, but we also actively participate in open forums and frequently contribute to open-source development. As a player in various embedded market IP segments, we work with partners across the globe, and our goal is to provide the best-in-class AI solution across all market segments.