Eye-Catching Innovations In Display Subsystems

The mobile devices we hold in our hands today are nothing short of astonishing in what they deliver and how seamlessly they deliver it, and it’s not just geolocation and context-mapping applications that have made these devices so transformational. Hand in hand with processor and bandwidth improvements have come leaps in display capabilities, which means new entertainment and information applications have enriched our lives.

However, there’s a constant tension between the increasing amount of information that flows from our devices and how it’s rendered and viewed. Indeed, the content that we see every day on our smartphones is increasingly complex, and the ability to view it in all environments, indoors and out, can be a challenge. From VR games to HDR video playback, the quality display of content is vital in delivering a rich user experience.

This would simply be a fascinating engineering conversation about design tradeoffs and techniques between CPUs and GPUs, hardware and software if it weren’t for the fact that mobile display technologies and the visual experience are now a major — if not the major — selling point for new devices. It’s another important tipping point in mobile history. Think about it: there was a time when data capability was nice to have but not vital as long as your cellular voice service was reliable. Then things changed overnight.

The same is happening with displays, and it’s prompted by a number of trends:

Virtual reality
VR headsets require displays that are close to the eyes, so to maintain the perceived quality of these images, we need more pixels in the same area. In addition, rates of up to 120 frames per second (fps) are now needed to reduce the overall latency and achieve lower persistence on LCD VR panels. This demands a pixel throughput rate an order of magnitude higher today than in the days before VR applications. This, of course, significantly increases design complexity and system bandwidth and represents a major challenge to the power and area budgets for mobile devices.

High Dynamic Range (HDR)
HDR is an essential step on the road to improved picture fidelity. It provides a more realistic feel to images with better color and brightness capabilities, and a more realistic user experience. The demands of viewing HDR content on both HDR and SDR displays create a major challenge for the display sub-system due to the different capabilities of panels at the end of the display pipeline.

Multi-window
Multi-window mode, previously the preserve of desktop, allows multiple activities to be seen on the screen, all at the same time. The windows are resizable and reconfigurable depending on limitations defined by both the app and system controls. In terms of system design, enabling multi-window functionality is not trivial. The enhanced Android Window Composition capabilities of a premium display solution allow users to multi-task on their mobile device with the same efficiency we see on desktop.

Panel variety
The variety of panels and interfaces that need to be supported by display technology continues to grow. This is driven by more complex content and optimized functionality between the application processor and the panel’s display driver interconnect. From HDR to SDR, if media is able to display the vibrant color and crisp images we expect, the display pipeline must output the content to the correct performance capabilities of the panel, and also do so intelligently to avoid wasted bandwidth.

These types of system complexities are difficult to address with one-off approaches. At Arm, our engineers have spent many years working on holistic solutions to address evolving graphics and display requirements to help partners drive user experience more quickly into the future.

Recently we built on this rich history of display work when we unveiled Komeda, an architectural foundation for our new integrated display solution composed of the Mali-D71, CoreLink MMU-600, and Assertive Display 5. We think the innovations included in Komeda address the latest challenges created by relentless demand for performance improvements in mobile devices.

Here are some of the highlights:

4K120 performance—guaranteed
Premium mobile display requirements are on the rise. 4K+ resolutions and higher frame rates are creating system performance issues, requiring a strong need for an optimized display processor. The first DPU built on the Komeda architecture is Mali-D71, which:

• Cuts system power by 30%, reducing GPU workloads by performing composition, rotation, high-quality scaling, and other imaging processing in fixed function hardware.
• Doubles area efficiency. When driving a single display, for example, it can re-use the resources of a secondary display, resulting in more full frame layers being processed.
• Doubles pixel throughput in a new side-by-side mode that achieves VR 4K120 performance within a low power envelope.

Secure, efficient data flow
Ultimately, these new mobile display challenges are really about data management. Arm engineers focused on tightly integrating a specialized version of CoreLink MMU-600 with the Mali-D71. This coupling leads to a 55% area saving for the combined solution and a 50% latency improvement by hiding MMU latency from the real-time path. CoreLink MMU-600 also solves another real and necessary problem: secure content protection for your premium media.

High-quality HDR regardless of lighting conditions
Since its launch, Assertive Display has been widely adopted, shipping in more 1 billion premium devices because of its ability to closely reflect human visual perception on a mobile display. Assertive Display 5 builds on its predecessor’s sunlight visibility improvements, offering HDR management for display pipelines, improved color and gamut management, and enhanced power savings.

Assertive Display 5 can present HDR content (HDR10 or HLG) to any characteristic of display (any dynamic range and gamut), delivering an HDR experience even on an SDR display. Ambient light compensation is even more important for HDR content since it will have more valuable information in dark areas. Assertive Display’s advanced features enable users to see the real details in an image, just as they were intended.

These three components work together to enable Arm partners and OEMs to deliver an innovative solution that addresses today’s display technology disruptors. With the Arm Display Solution, display IP now delivers a real, competitive advantage for device manufacturers by improving user experience, reducing power and increasing battery life. This solution represents Arm’s commitment to delivering the best VR and HDR experiences possible and we look forward to continuing our leadership in the ongoing innovation for display technologies.