Neural Nets In ADAS And Autonomous Driving SoC Designs

Automotive electronics has ushered in a new wave of semiconductor design innovation and one new technology gaining a lot of attention is neural networks (NNs). Advanced driving assistance systems (ADAS) and autonomous car designs now rely on NNs to meet the real-time requirements of complex object-recognition algorithms.

The concept of NNs has been around since World War II, promising a future of brain-inspired algorithms capable of learning based on training and environment rather than explicit if-then programming. The technology has only recently entered the commercial realm as a way to implement machine learning (ML), primarily due to three factors:

• The availability of huge datasets (“big data”) necessary for training
• Hardware powerful enough for “back-end” and near-real-time inference/decision-making
• New software that efficiently executes the algorithms

Stepping back, ML is a subset of artificial intelligence that relies on experiential learning rather than programming to make decisions. Deep learning (DL) is a subset of ML that is able to take raw information without any meaning and construct meaningful hierarchal representations of information that are actionable by a system.

NNs have emerged as the primary means of implementing DL in autonomous driving systems, with specialized hardware accelerators running in-vehicle NNs to classify objects like pedestrians and road signs. When an in-vehicle NN system has difficulty recognizing an object, it can chronicle the car’s environment at that time and send that information wirelessly to a data center, which uses the new information to (re-)train the system by recalculating and updating the weights in the car’s NN.

Fig. 1: An example of NN operation for object detection and analysis. (Source: Lee et al., Communications of the ACM 2011)

Take, for instance, a NN training operation for recognizing objects in front of a vehicle (Fig. 1). The training process determines the weight for each neuron input or object, and once scaled up and trained, NNs capable of recognizing objects classify inputs or objects. To do this in a vehicle at near-real-time latencies, NNs are implemented using specialized processing and data flow functions in large, complex system-on-chip (SoC) designs.

In many ways, automotive electronics are driving the advancement of ML and NN technologies. Below is a sneak peek into what really matters when incorporating NNs into automotive SoC designs.

Interconnect and memory architecture
In ML-based SoC designs, the ADAS and autonomous car architects slice the specialized processing and data flow functions more and more finely by adding more hardware accelerators to increase the efficiency of the NNs. As the types and numbers of processing elements increase, the SoC interconnect and memory architecture connecting these processing elements become critical.

Because of the need for near-real-time system response in ADAS and autonomous vehicles as well as the need to meet vehicle power consumption budgets, the extensive use of relatively slow and power-hungry off-chip DRAM access is a non-starter for NN architectures. To maintain data locality near processing elements and reduce power consumption, automotive chip developers usually rely on two design techniques: internal SRAMs closely coupled to hardware elements and hardware cache coherency.

Fig. 2: Memory architectures and interconnects are key in implementing NN accelerators in automotive SoCs. (Source: Arteris IP)

For smaller designs with few processing elements, dedicated internal SRAMs used as closely coupled memories often suffice. Problems arise when architects scale up the system to tens or hundreds of hardware accelerators. As the number of connections among elements grows, memory area increases, and the software complexity required to manually manage memory accesses explodes.

Therefore, for larger designs, architects use hardware cache coherency for some or all of the NN hardware architecture. Cache coherence also allows the processing elements in an automotive SoC design to share data without the overhead of direct software management.

Interconnect’s data flow leverage
Next, beyond the memory access mechanisms, the overall data flow must be optimized in NN SoCs to ensure that bandwidth and latency requirements are fulfilled in the ADAS and autonomous vehicle designs. So, whether they’re designing a cache-coherent memory architecture or a non-coherent memory design, SoC developers must properly configure the on-chip interconnect.

Before we go into further details about the interconnect fabric, it’s worth mentioning that cache coherence can be implemented even on non-coherent SoC designs by using the configurable proxy caches. The association, connectivity, and behavior of each proxy cache can be optimized for a specific use, such as object classification in ADAS and autonomous car applications.

Now, coming back to on-chip interconnect, which is crucial in handling different data requirements of processing elements, including size, access patterns, and frequency, and in optimizing these requirements to meet overall SoC performance goals. Data flow optimization is also crucial here in ensuring the quality of service (QoS) to meet bandwidth and latency requirements and avoid performance degradation.

Interconnect and functional safety
SoC interconnect is also vital in ensuring functional safety because it sees all the data being flown across the chip. The on-chip interconnect can discover and, in some cases, fix errors to improve the safety of the chip and thereby facilitate diagnostic coverage as per the ISO 26262 automotive functional safety standard.

Fig. 3: A view of how the interconnect fabric ensures data protection for functional safety. (Source: Arteris IP)

The interconnect technology can help automotive SoCs implement functional safety via data protection techniques in two ways: first, via error code correction (ECC) and parity protection for all data portions of the chip having safety or reliability requirements and second, by using intelligent hardware duplication of the parts of the interconnect that affect the content of a packet.

These safety mechanisms are integrated with automatically generated ECC generators and data checkers. Furthermore, they are linked with a configurable on-chip fault controller, which manages interconnect-wide built-in self-test (BIST) functions and communicates error status to runtime software.

Why interconnect matters in NNs
NNs are one of the latest additions to the complex world of SoC designs for ADAS and autonomous cars. Understanding the capabilities of on-chip interconnects is key for automotive SoC developers in navigating this complex world while effectively managing the hardware accelerators running the ML algorithms. Choosing the right memory architecture and interconnect technology is crucial in efficiently running NN operations. The specialized processing and data flow features make hardware accelerators highly suitable for NNs. Here, the interconnect backbone becomes central in integrating these hardware accelerators for NN functions.

Find out more about memory technologies like cache coherency and on-chip interconnect for implementing machine learning within automotive SoCs using the neural networks.