Multitasking For Modern GPUs

Originally GPUs were all about one thing, 3D graphics, and specifically fill-rate. Creating 3D triangles, calculating their position, coloring them in, processing the right ones (thank you tile-based deferred rending), and outputting them to the screen. Nowadays GPUs need to do more – it’s called “compute.” Indeed, we’ve been talking about running this on low-power GPUs for a long time.

In other words, the modern GPU needs to multi-task. To enable this, in an efficient and secure manner, we have created HyperLane Technology. Inside every A-Series, from the smallest to the very largest are eight individual hardware control lanes, each isolated in memory, enabling different tasks to be submitted to the GPU simultaneously, for fully secure GPU multitasking. It means we can have eight completely different workloads and run them simultaneously; it’s a feature that’s unique in the market.

The IMG-A-Series is a significant leap forward for GPU-kind and will provide the basis for future generations of devices. To give some perspective, the Helio P95 SoC, just been announced by our friends at MediaTek is a strong mid-range chipset, featuring a PowerVR GM9446 GPU and will soon lie at the heart of some excellent, affordable devices. However, as always at Imagination, we’re looking to the future and compared to the P95, the A-Series will deliver around 2.5x the graphics performance, 8x faster AI processing or up to 60% lower power.

Dedicated hardware to maximize utilization
Thanks to what we call Dynamic Performance Control, we can ensure that the GPU spreads its performance across these multiple tasks, executing them all while maximizing GPU utilization. It’s a concept that’s been well explored in the desktop processor space, with one physical core appearing as multiple logical cores, with multiple tasks in flight to improve hardware utilization and multitasking performance.

Our unique solution accelerates this with our firmware processor, which ensures that multiple graphics or compute tasks are scheduled and prioritized by the hardware with no overhead, this ensures that the GPU is fully utilized at all times, increasing real-world performance.

HyperLane for graphics and AI
Now let’s focus on some use cases to see how it might be deployed in the real world. We’ll start with the classic mobile use case in a smartphone or a tablet.

You might want to sort or categorize the images on your phone – (yes, all 14,984 of them). While half of them might be food, family, friends and pets, the other half will probably be of your face, so your device might use that GPU compute power to apply that perfect filter. Again, thanks to HyperLanes it can conduct all these tasks in the background, even if your browsing the web or using an app, with no loss of performance.

Or you might have an augmented reality game, where the physics required to locate objects in 3D space is done using GPU compute at the same time as rending the graphics, making for a much smoother user experience.

HyperLanes for security
DTV: Another great use for HyperLanes in the consumer space is the digital television market and here the advantage it brings is security. A key issue that concerns content providers is ensuring their valuable IP, be it video or audio, is secure through digital rights management, something that is vital to secure licensing agreements. While competitive GPUs offer a “TrustZone,” this is an “all your eggs in one basket” approach, with all applications residing in the single secure area. With HyperLane Technology and memory isolation, each service or application can be placed in an entirely separate, secure container, so if one application is compromised by a hack, it does not provide a back door to the content.

Surveillance: While it is a topic of some debate over privacy concerns there’s no getting away from the fact that AI is increasingly being used in surveillance. Neural networks are being used for analytics to identify suspicious body movements using models such as pose estimation. Whether its airport security or just your home doorbell, AI can cut down on a lot of false positives, by not alerting you to everything – just things such as concerning behavior or someone coming right up to your door, rather than just walking past your house. Or AI could be used to perform image unwarping, at the same time as image analysis.

Data center: The GPU’s ability as a highly parallel mathematical co-processor makes it is ideal for accelerating many tasks other than 3D graphics alone. Imagination GPUs today are designed to meet the requirements of compute and graphics in the cloud. A-Series GPUs will be powering cloud-based AI training and inferencing as well as the cloud gaming experiences of tomorrow. Data centers are power efficiency sensitive, so the low-power design of the A-Series makes it a great fit for such scenarios. HyperLane Technology ensures further power savings by enabling multiple tasks to be run on a single GPU instance while maximizing utilization for even greater efficiencies.

Automotive: The best modern cars use all-digital dashboards, powered by a single Imagination high-performance GPU. The main cluster delivers information that is vital for the driver, so it is critical performance is always maintained. For efficiency, other tasks, such as infotainment, or sat-nav, also run on the same GPU but at a lower priority. Imagination’s hardware virtualization technology is our industry-proven automotive solution to enable a single GPU to run all these tasks and meet quality of service requirements. We integrate our HyperLane Technology with multi-operating system virtualization to enable all these tasks from different OSs to be run on the same GPU simultaneously, using prioritization techniques to ensure the quality of service for the main task. This means our virtualization with HyperLane Technology solution provides full isolation, for example protecting cluster data from being affected in the event of issues with other workloads.

AI synergy
HyperLane Technology is also the basis of a new feature called AI Synergy. This powerful, flexible AI solution turns SoCs containing an Imagination A-Series GPU and our dedicated neural network accelerator into AI powerhouses. It enables AI workloads to be run on the GPU at the same time as graphics while offloading fixed-function AI to a dedicated neural network accelerator for ultimate AI performance. Look out for a blog post covering AI Synergy in more depth in the near future!

Conclusion
As we can see from all these use cases, HyperLane Technology in IMG A-Series provides SoC designs and application developers with a whole new range of options that are unique in the market. Thanks to dynamic performance control, the technology can ensure maximum GPU utilization, along with workload prioritization to ensure quality of service (QoS) for the prioritized task. It also delivers full hardware-based isolation, to ensure security and robustness.