Big Push For 3D Sensing With iPhone X

3D sensing is a buzzword that has been thrown around quite a bit this year in connection with the rumors surrounding the tenth-anniversary iPhone. Although the iPhone X will be the first large-scale consumer push for 3D sensing, the technology has been around for years, particularly in industrial applications such as machine vision. 3D sensing is already used in the PC – think Intel’s RealSense 3D camera. For the iPhone X, the TrueDepth 3D camera will be used for facial recognition, referred to as Face ID, replacing Touch ID as a method for unlocking the phone and for mobile payments.

3D sensing is accomplished with either Time of Flight (ToF) or Structured Light technologies. Gesture recognition is achieved by passing an infrared light source through an optical device to spread the light through an entire space, such as a room, to measure the light reflected off of objects. It enables users to control a device without touching it or using an input device like a remote control. In order to be precise enough to track small movements anywhere in a room, “time-of-flight” techniques are used. Time of flight has the advantage over other ways of tracking reflected light that rely on the quantity of light reflected back; black targets will reflect back less light than white ones, even though the distance from the sensor is the same. Time of flight tracks how long it takes the light to reflect off the target and back to the sensor.

This whole-room tracking could be used for augmented reality applications, and Apple is making a big push in this area. Apple’s brand-new iOS 11 includes ARKit, Apple’s AR platform. According to Apple, “ARKit uses Visual Inertial Odometry (VIO) to accurately track the world around it. VIO fuses camera sensor data with CoreMotion data. These two inputs allow the device to sense how it moves within a room with a high degree of accuracy, and without any additional calibration.” ARKit can find horizontal planes in a room like tables and floors, and track smaller objects, as well as correcting the lighting for VR objects based on the actual light in the scene.

ST’s FlightSense ToF product has shipped 15 million units, and the company spent 8 years developing the technology. The product is a module including a proximity sensor, an ambient light sensor, and a laser IR light source, which can be placed under glass. In its most recent earnings call, ST announced they have shipped over $300 million in time-of-flight sensors, and they are in more than 80 smartphone models from 15 different OEMs. ST’s latest generation of FlightSense adds multi-object detection and multi-array scanning to mobile applications. To keep up with increased demand for this product, ST plans to spend an additional $125M to $200M in capex.

The other 3D sensing technology is Structured Light, in which a pattern is projected onto an object, and the depth of the target can be inferred based on the deformation of the projection. Himax has its own take on the Structured Light with advanced wafer-level optics (WLO) in its SLiM (Structured Light Module) product line, which is a turnkey module solution for OEMs. However, the company is not targeting mass production of its 3D sensing module until early 2018. Qualcomm and Himax have collaborated to develop a solution that combines Qualcomm’s Spectra computer vision technology with Himax’s SLiM product.

Figure: A 3D Sensing Block Diagram

Source: Finisar at myvcsel.com

Vertical Cavity Surface Emitting Lasers (VCSEL) are one component of the 3D sensing solution and are shown in the diagram above. VCSEL arrays have traditionally been used for gesture recognition and optical sensing. Low-end VCSELs are used in optical mice and laser printers. Higher-end products have been used in products like Microsoft’s Xbox Kinect, which allows consumers to control the console with gestures. Lumentum is a large producer of VCSELs and provided the technology for the Kinect, so they have the advantage of experience in the consumer marketplace. Lumentum’s 3D sensing revenues will jump from $5 million in 2Q17 to approximately $200 million in the third quarter. The destination of that order is expected to be the new flagship iPhone X, but this can’t be proven until teardowns reveal what’s inside the new phone. Lumentum has said sales are only limited by their capacity. Although Lumentum has its own wafer fab, its 3D sensing products are produced at a foundry.

Companies in the 3D sensing supply chain include Lumentum, Finisar, STMicroelectronics, Himax, Viavi (optical filters), Jabil Optics, ChipMOS, Xintec, Tong Hsing and probably TSMC.

Improvements in 3D sensing technology are still being made. For example, Chirp Microsystems has an ultra-low-power ultrasonic ToF solution targeted at wearables and smart home products that are always-on. Chirp’s solution is based on measuring the Time of Flight of sound waves rather than light.

All this points ultimately to an ever-increasing use of sensors in cell phones, a trend Semico has been predicting for some time. We think 3D sensing will have a positive impact on the semiconductor industry. Semico publishes a MEMS and Sensor Database which allows users to take a deeper dive into over 30 different product types and twenty different target markets, including smartphones, tablets, wearables, robots, and various automotive and medical segments. Contact Rick Vogelei at [email protected] for more information.