Designing Immersive AR/VR Displays

Last month, we looked at how VESA (Video Electronics Standards Association) video compression codecs enable cutting-edge displays. Over the next couple of articles, we will take a closer look at some of the markets where VESA DSC (Display Stream Compression) and VDC-M (VESA Display Compression) compression offer significant benefits for designers working on display-based applications.

Demand for AR (augmented reality) and VR (virtual reality) devices has been growing steadily in recent years. No longer just for gamers, we are starting to see AR/VR devices in other contexts including industrial, medical, and educational settings. The past year has also seen a huge surge of interest in AR/VR in general as major players, including Meta, Microsoft, Google, and many others, have begun to focus on the Metaverse.

Head-mounted displays (HMDs) present several unique challenges when it comes to their design. Unlike a mobile phone or a TV, there is not just one display but two, and these displays are located at remarkably close proximity to the human eye. To enable a truly immersive AR/VR experience, designers need to consider field of view, display resolution, as well as pixel density and pixels per degree. A headset essentially needs to offer a large field of view combined with displays that have a high pixel density.

Transferring all the pixels needed for HMDs is challenging for both tethered and wireless devices. For tethered devices, the amount of bandwidth needed puts stress on the current physical interface technologies available. Wireless devices bring additional challenges as power consumption and miniaturization are huge obstacles for designers.

VESA DSC and VDC-M compression can be applied to many areas within an AR/VR system. This includes images captured by image sensors, video and graphics processed by the AR processor and GPU (graphics processing unit), video streams sent to micro-displays, and video stored inside the micro-display driver IC (integrated circuit). Using video compression enables designers to reduce the memory buffer size and bandwidth, leading to significant power savings and a smaller footprint, all while keeping latency extremely low. Keeping latency low for AR/VR applications is extremely important as it avoids potential motion sickness associated with the brain detecting small delays between the body’s physical movement and the image in the head-mounted device.

You might be asking whether all this compression going on behind the scenes will interfere will a user’s experience? The simple answer is no. As we mentioned in last month’s article, VESA DSC and VDC-M are visually lossless compression codecs, meaning that when used, no perceptible difference can be detected by the human eye. Both codecs have been rigorously tested and in 2021, additional research conducted by VESA validated the visually lossless performance of both codecs for stereoscopic 3D use cases, including for AR/VR applications.

Developed by Hardent, Rambus offers encoder and decoder IP cores supporting the VESA DSC and VDC-M compression standards. These IP solutions are silicon-proven and tailored for use in AR/VR applications.

Additional resource
Website: Rambus VESA DSC & VDC-M IP Cores