Enabling local execution of inferencing tasks on streams of incoming video.
The future of smart camera technology brings with it profound transformations in the way we interact with each other and the world around us. From smart cities that are safer and more efficient to rainforests that are monitored for illegal logging, the increasing need for advanced vision technology is growing. Diverse and complex use cases leveraging artificial intelligence (AI) and machine learning (ML) are driving the demand for increased processing capabilities and, increasingly, it makes sense, for scalability if nothing else, to provide that compute in the camera itself. Put simply, to run advanced AI workloads at the edge or on the endpoint camera itself, there is a need for higher performance processing, to address applications such as high-definition imaging to enhanced security.
Following the migration seen in laptops and smartphones, smart cameras are moving towards 64-bit processing to leverage more advanced capabilities and enable more leading-edge applications. As an example, in smart cities across the world we see cameras capable of running AI algorithms to detect people, pets, packages, license plates, and other objects. And, as compute is increased and advances in AI use cases in smart cameras continue to scale, it is possible to run these workloads locally on the CPU. With these advancements it will be possible to detect hundreds of people at a time or recognize license plates on speeding cars, all without needing to transmit large volumes of data to the cloud in real-time.
Moving to 64-bit processing brings advancements in the CPU hardware, increasing performance, and efficiency. Most notably, 64-bit CPU architectures come with enhanced register support and larger memory maps. The increase in both number and width of registers mean that larger data sets can be processed with a reduced number of memory accesses, resulting in faster data processing. The increased register support also means that the developer can take advantage of the state of the art in compiler optimizations, which will further boost performance. When the benefits of improved register support are combined with the increased memory map of 64-bit architectures, the software has direct access to more local data for processing, reducing need for the memory swapping, which impacts performance as data sets are swapped in and out of local memory.
Imaging requirements are increasing, smart cameras are embracing 4K resolution in the low-end cameras and 8K resolution in the mid- and high-end cameras. Moving from 1080p to 4K doubles the data rate of the incoming video streams from the cameras with the same encoding schemes. Furthermore, if the frame rate needs to increase from 15fps to 30fps to 60fps to better identify objects, then the need for data rate and performance would continue to scale accordingly. Higher fps will enable cameras to detect, identify, and recognize smaller objects and fast-moving objects with much better precision. The need for simultaneous 4K encoding from multiple streams further increases the demand for faster CPU performance. 64-bit processors are well suited to meet these demands.
Furthermore, to meet such a need for increasing performance and bandwidth requirements for imaging and ML workloads, smart cameras need support for floating point operations. The Armv8-A 64-bit architecture increases the FLOPS¹-per-cycle performance for single-precision by 2x and double-precision by 5x, which improves the user experience.
[1] FLOPS are defined as fused-MAC operations that contain a multiply portion and an accumulate portion. One fused-MAC operation equates to 2 FLOPs.
It is also critical to highlight the advancements that Arm has been making on imaging and ML through extensions to Neon starting with SVE in the Armv8 architecture and SVE2 in Armv9 architecture.
Fig. 1: Arm 64-bit CPUs for current and future smart cameras.
The faster and larger memory accesses delivered by the Arm 64-bit architecture, known as AArch64, enables ML models to be loaded into memory much more quickly. This increase in performance enables execution of inferencing tasks on streams of incoming video from the smart cameras much quicker, thus providing faster detection, recognition, and identification of objects in the image.
To enable efficient data transfer with on-chip accelerators such as video encoder/decoders and ML hardware external to processor core, memory bandwidth must be optimized. Improving system memory streaming performance speeds the movement of data in and out of the memory to be used by many different hardware elements. The following chart shows the most common memory streaming benchmark and illustrates the 3.75x improvement in performance when moving from a 32-bit to 64-bit CPU.
Fig. 2: Memory Streaming Performance improvement from Cortex-A7 32 bit CPU to Cortex-A35 64 bit CPU.
The 64-bit ARMv8-A architecture provides native security capabilities in its CPUs, where the crypto instructions AES, SHA, and CRC can provide 3x-11x better performance in software encryption that could be used for small granule cryptography. As Arm continues to improve the performance of key cryptography operations, smart cameras are able to continue to improve their cryptography algorithms to support the continued need for privacy and security demanded from IoT applications.
Fig. 3: Encryption improvement from Cortex-A7 32 bit CPU to Cortex-A35 64 bit CPU with Crypto extensions.
As in all markets, software and tools continue to play a major role in the smart cameras sector, enabling developers to deliver innovative new features and applications. Support for 64-bit Linux distributions continues to expand, with more companies adding support for various new features such as running container orchestrators enabled by newer Arm architectures.
Furthermore, Arm continues to enhance the open-source GNU compiler toolchain with the new features available in the latest CPUs. Arm has also added support for OpenCV, one of the leading open-source libraries for computer vision, image processing, and ML, thus empowering more developers to easily deploy smart camera applications and products.
Arm continues to deliver enhancements in newer 64-bit architecture specification releases related to digital signal processing, vector processing, ML, and security, all of which are key compute capabilities needed for future generations of smart camera.
There are many advantages of moving all smart cameras to 64-bit Arm based processors, and one of the key factors is the breadth of support from the ecosystem. For example, Amazon Web Services has added support for ML inferencing on any device supporting 64-bit Armv8 64-A (https://docs.aws.amazon.com/sagemaker/latest/dg/neo-supported-devices-edge-devices.html) with its AWS Sagemaker Neo service, benefitting companies that plan to use Amazon Web Services for ML deployments.
Ecosystem initiatives such as Project Cassini tackle two major roadblocks to IoT deployment: scalability and fragmentation. For silicon partners, ODMs, OEMs, ISVs, system integrators and developers, Project Cassini unlocks the edge and IoT compute opportunity by accelerating deployment of cloud-native applications across diverse Arm 64-bit platforms. To take advantage of Project Cassini and the associated SystemReady initiative, it is essential to have 64-bit as the base computing platform.
To accelerate the development and deployment of edge devices, Arm recently announced Arm Total Solutions for IoT. On the roadmap is a Total Solution for Vision which includes Arm Virtual Hardware to enable customers to kick off their software development ahead of silicon availability.
For companies in the smart camera ecosystem that want to leap ahead of competitors, it is important to take advantages of the power of 64-bit processing. Arm is here to support the developer community and partners to help address any technical challenges as part of the transition towards 64-bit.
The move towards 64-bit is a ‘win-win’ all around. It will provide a number of performance, efficiency, and security benefits to the entire ecosystem as well as getting more cameras ready for future innovation that will transform the world around us.
 |
 |
 |
 |
 |
 |
 |
 |
 |
Leave a Reply