Embedded Display IP Solution For 4K Resolutions

By Licinio Sousa, Synopsys, and Vassilis Androutsopoulos, Arm

To meet consumers’ demands for high-resolution content and visual quality, high-end smartphones are moving from wide quad HD (WQHD) displays to ultra-high-resolution 4K displays. For AR/VR applications, since the display is closer to the eyes and must maintain visual quality, more pixels and high refresh rates are needed. Higher refresh rates minimize motion sickness and provide an immersive visual experience by reducing the discrepancy between what your eye sees and what your brain expects to see. This article illustrates the different display subsystem architectures, and describes an interoperable display processing unit (DPU) and MIPI Display Serial Interface (DSI) IP solution that enables 4K embedded displays for smartphones and AR/VR devices.

Display subsystem in an application processor
The multimedia pipeline in an application processor architecture encompasses multiple IP blocks such as a graphics processing unit (GPU), video processing unit (VPU), DPU, and MIPI DSI, shown in Figure 1. As the displayed content becomes more complex in terms of resolution and frame rate, the GPU/CPU must offload more functionality to meet increasing graphics performance within the ever-shrinking silicon area and thermal design power requirements.

Figure 1: Two key display IP blocks in an application processor: Arm Mali-D71 and Synopsys DesignWare MIPI DSI Host Controller IP with DSC encoder

The Arm Mali-D71 DPU is a fundamental processor and GPU companion in a multimedia subsystem that works seamlessly behind the Android Hardware Composer HAL (HWC). Mali-D71 offers a super-efficient, multi-layer hardware composition solution in terms of system power and performance. It also performs in-line orthogonal rotation, high quality upscaling and downscaling, ambient light adaptation, and high quality HDR local and global tone mapping and color/gamut management when combined with Arm Assertive Display 5. It performs all of the above operations within a single pass through memory, translating to significant system bandwidth (power) savings. The latest version of Arm Frame Buffer Compression (AFBC), version 1.2, is supported throughout the multimedia pipeline, adding significant performance and system bandwidth benefits. Arm Corelink MMU-600 is also an essential component of the Arm display solution. It is optimized to work with Mali-D71 on 4KB paged memory to guarantee real-time performance for latency-sensitive applications, such as VR. Mali-D71 supports all major display industry standards for all kinds of panels (mobile, tablet, TV, notebooks, etc.), including MIPI DSI.

Mali-D71 connects to the display device through the display serial interface like the MIPI DSI. The interface must support all the integral requirements of a 4K display in terms of bandwidth and speed. According to MIPI Alliance, the DSI specification provides “brilliant color rendering for the most demanding imagery and video scenes and supports transmission of stereoscopic content” (https://www.mipi.org/specifications/dsi). Designers can integrate the MIPI DSI specification into their application processor or system-on-chip (SoC) with highly-optimized IP like the Synopsys DesignWare MIPI DSI Controller.

Synopsys’ MIPI DSI Controller is a fully verified and configurable IP that converts the incoming pixel data, which in this case is Arm’s DPU, into MIPI DSI packets which are transmitted to the MIPI D-PHY link connecting to the embedded display. The Synopsys DSI IP supports dual DSI link use-cases by providing additional bandwidth for ultra-high-resolution mobile systems. It also supports the Video Electronics Standards Association (VESA) Display Stream Compression (DSC) standard to reduce the required data transmission bandwidth for 4K resolution.

From WQHD to 4K displays
System Integrators must select a DPU and a DSI solution that not only delivers the bandwidth, speed and power required for 4K embedded displays, but also allow for seamless interoperability with one another. Figure 2 shows an example of a WQHD display at 60Hz with 24 bits per pixel (bpp) needing 6.3Gbps to transfer data from the application processor to the display device.

Figure 2: Simplified diagram of a WQHD resolution display in a smartphone application

In this example, one link of one display output unit connects to one MIPI DSI host controller, which then connects to the display device using a 4-lane MIPI D-PHY. The DSI host controller integrates a VESA DSC encoder with the capability to perform visually lossless compression by a factor of 2x or 3x, reducing the required bandwidth to 3.16Gbps. The compressed stream converts into MIPI DSI packets passed down to the D-PHY link. Using a 4-lane operation at 1Gbps per lane results in a total available bandwidth of 4Gbps, enough to transmit the compressed stream. This entire process reverses on the display device side. The DSI packets are recovered by the receiver controller and a VESA DSC decoder decodes the data before the image is displayed on the panel.

In contrast, shifting to a 4K display requires a different architecture, shown in Figure 3. A 4K display resolution at 90Hz with 30 bpp requires 26.6Gbps bandwidth with two display drivers and associated MIPI display interfaces. The display drivers synchronize to enable the two halves of the image to display properly.

Figure 3: Simplified diagram of a 4K resolution display in a smartphone or AR/VR application

In this example, two links of one display output unit connect to two distinct MIPI DSI host controllers, which then connect to the display device using 2 by 4-lane MIPI D-PHYs. The DSC, with a compression factor of 3x, reduces the required bandwidth for each DSI link to 4.4Gbps. The compressed stream converts into MIPI DSI packets that are passed down to the D-PHY link. Using 2 sets of 4 lanes operating at 1.5Gbps per lane for a total available bandwidth of 6Gbps per DSI link is enough to transfer the compressed stream to each display port.

Both examples show the advantages of using a DSI solution with an integrated VESA DSC encoder. The encoder reduces: the amount of data transmission sent to the display device, the D-PHY operation frequency, the number of D-PHY lanes, and the display frame buffer size. This combination results in significant power, area, and cost savings.

A fully interoperable embedded display solution
MIPI-enabled displays support video mode, command mode or both. In video mode, the host must constantly refresh the display. Synchronizing the information and image data transmit over the MIPI bus as it uses DSI packets happens in real-time. Because the host must refresh the display, the display does not need a frame buffer. In command mode, the MIPI host transmits pixel data stream to the display using display command set (DCS), which is transmitted over the MIPI bus using DSI packets. The display has a full frame buffer to store the entire pixel data.

Once the data is in the display’s frame buffer, a local timing controller on the display retrieves data from the frame buffer and shows it on the display. The MIPI host does not need to refresh the display constantly.

Arm and Synopsys have collaborated to ensure Arm Mali DPU and Synopsys DesignWare DSI Controller IP, with DSC encoder, are fully interoperable for both video and command mode displays. Besides the fundamental clock and reset control, the companies provide the detailed interface connectivity, including all the IP-specific slide band signals for each display mode in both single and dual link mode of operation. Dual link operations double the available bandwidth, which is particularly important for ultra-high-resolution applications. Table 1 shows the numerous resolutions that the interoperable embedded display solution supports.

Table 1: The interoperable display solution supports numerous resolutions and configurations 

The combined display solution also supports several advanced features such as variable refresh rate, dynamic resolution change, and partial update.

Variable refresh rate enables changes of total frame time during frame transfers and therefore effective frame rate. This dynamic refresh rate change is especially useful when displaying static content (i.e., static screen scenarios) and playing back video of fixed frame rate (i.e., 24 fps or 48 fps).

Dynamic resolution change is a solution that enables the transmission of multi-resolution video streams between the Mali-D71 DPU and Synopsys DesignWare MIPI DSI Host Controller IP without the need for reprogramming or reconfiguring the IP components.

If the display peripheral supports partial update, then there is the option to program the Mali-D71 DPU and Synopsys DesignWare MIPI DSI Host Controller IP to update only a selected part of a frame buffer from the display peripheral. This feature is especially useful when only part of a screen is updated from frame-to-frame and the rest of the screen is static during most of the display time, which is during most frames. Figure 4 shows video playback displayed together with the status bar, navigation bar, and background. Only the video content changes between the frames, while other display layers remain static. With partial update support, only the video region of the display frame transmits to the display resulting in overall power savings.

Figure 4: Use case for partial frame buffer update – Video playback with static status/navigation bar

A complete and optimized display solution
For smartphones and AR/VR devices, there is a shift in embedded display resolution from WQHD to 4K. However, each application requires its own unique set of features when it comes to high bandwidth and refresh rates. Synopsys and Arm have collaborated to provide a complete embedded display solution with a fully interoperable DPU and MIPI DSl interface that enables designers to minimize integration risk. The solution supports not only command and video modes, but also incorporates several advanced features like variable refresh rate, dynamic resolution change, and partial update. The companies have drafted a comprehensive application note (available under NDA) that reduces SoC designers’ effort and risk of building 4K displays for high-end smartphones and AR/VR devices.

Contact us for more information or visit DesignWare MIPI DSI Host Controllers and the Mali-D71 DPU web pages.

Vassilis Androutsopoulos is a senior product manager at Arm.