Driving AI, ML To New Levels On MCUs

One of the most dramatic impacts of technology of late has been the implementation of artificial intelligence and machine learning on small edge devices, the likes of which are forming the backbone of the Internet of Things.

At first, this happened through sheer engineering willpower and innovation. But as the drive towards a world of a trillion connected devices accelerates, we must find ways to efficiently expand the compute capabilities on a greater number of constrained devices at the far edge of the network.

Increasing the compute capabilities in these devices will immediately open the door for even more developers to write machine learning (ML) applications directly for the device for decision-making at the source, thus enhancing data security while cutting down on network energy consumption, latency and bandwidth usage. In fact, it’s already happening. Today ML is being implemented on Arm Cortex-M devices and at the edge on devices like Raspberry PIs.  For example, OpenMV has built a camera board for machine vision applications around the Cortex-M7. In addition, a recent survey by Arm found that 42 percent of respondents who are running AI or ML are doing so on CPUs or MCUs, compared with 26 percent on GPUs.

Against this backdrop, Arm has introduced Arm Helium technology, the M-Profile Vector Extension (MVE) for the Arm Cortex-M series processors that will enhance the compute performance of the Armv8.1-M architecture including the built-in security of TrustZone. Helium will deliver up to 15x more ML performance and up to 5x boost to signal processing for future Arm Cortex-M processors, unlocking new market opportunities for our partners where performance challenges have limited the use of low-cost and highly energy-efficient devices.

Advanced digital signal processing (DSP) is available today through Arm Neon technology in richer Cortex-A based devices. In fact, when Arm was asked to increase the DSP performance of Arm Cortex‑M processors, naturally the first thought was to just add the existing Neon technology. However, the need to support a range of performance points within the area constraints of typical Cortex‑M applications meant Arm had to start from scratch.

As a lighter noble gas, Helium seemed an apt name for the research project, made perfect by the fact that the nominal goals (for a mid-range processor) where a 4x performance increase for a 2x increase in data path width, coincides with Helium’s atomic weight and number. As it turns out, Arm managed to beat the 4x target on many digital signal processing (DSP) and machine learning (ML) kernels. Needless to say, the name Helium stuck, and was adopted as the branding for the MVE for the Arm Cortex-M processor series.

Underlying factors
As Arm looks to incorporate more ML capabilities on to these devices, several existing SoC development challenges loom large and require a higher level of expertise in utilizing different toolchains, programming, debugging and working with complex proprietary security solutions.

Armv8.1-M with Helium eliminates these challenges by delivering real-time control code, ML and DSP execution without compromising efficiency. In turn, millions of software developers will be able to securely scale intelligent applications that take advantage of DSP capabilities across a wider range of devices, enabling enhanced support for emerging applications across three key categories; vibration and motion, voice and sound, and vision and image processing. This will improve the user experience in future devices such as sensor hubs, wearables, audio devices and industrial applications powered by next-generation SoCs based on Cortex-M with Helium technology.

In addition to the added performance and lower development costs, SoC design and development teams will immediately recognize other benefits including:

• Optimized cost, power and design efforts by consolidating functionality
• Simple deployment of TrustZone due to Armv8.1-M being designed to Platform Security Architecture (PSA) specifications
• Single toolchain for both control and signal processing software development
• Easier software development due to a comprehensive set of tools, models and libraries from the well-established Helium ecosystem, many of which are already being used by Cortex-M developers today.

Simplifying software development
Software development will be made simpler due to Helium’s unified tool chain, libraries and models. The Helium toolchain includes the Arm Development Studio, encompassing Arm Keil MDK, Arm Models (which are immediately available to developers for code modelling) and various software libraries including CMSIS-DSP and CMSIS-NN, allowing developers to choose the best fit for their needs. And for signal processing applications Arm made it even simpler by removing the need for a dedicated DSP or function accelerator and eliminating another layer of design complexity.

Powering the next generation of embedded and IoT devices
Helium is the latest example of the value Arm’s Project Trillium brings to ML applications by supporting frameworks and libraries right down to the hardware. There is no one product that fits all as all SoC developers must innovate within different performance, area, power and cost constraints.

Tool chain and models for Helium are available today and Arm expects Helium to be available in silicon within the next two years. Find out about all Armv8.1-M architecture enhancements in this whitepaper. Read more about how to get access to the Arm tools supporting Helium.