MIPI Standards Gaining Traction In New Markets

An explosion of low-cost, high-performance image sensors for a growing number of applications is propelling the MIPI interface into a variety of new markets, where standardized signal protocols and characteristics are becoming essential.

For years, MIPI has been almost synonymous with mobile phones. But as higher-resolution image sensors increasingly are deployed in automotive, AI, IoT, and medical devices, interest in MIPI is spreading well outside of its core market.

“These are very small, very high-resolution sensors, and they produce fantastic images,” said Marc Greenberg, group director of product marketing for the IP group at Cadence. “Mobile devices have two, three or four of these sensors, and they are being designed into a range of devices, from home security cameras to robotics to industrial IoT as well as military devices. These sensors are at a low enough cost point that they are being applied to lots of different types of devices that didn’t have image sensors before.”

Alongside of this increase in image sensors, there is a convergence on the neural network side as interpretation of those images happens closer to the source. “We’re at the point that these neural networks are really quite good at understanding what they’re seeing,” Greenberg said. “Is it a person? Is it an object? How is that person moving? Is that movement normal or abnormal? Is that movement a gesture of instruction, such as to change the channel? This convergence is about the availability of sensors and the ability of the neural networks to understand what the image sensor is viewing pointing to the trend of ‘vision everywhere,’ and sometimes coupled with AI. For example, there is a microwave oven that takes an image of your food, both before and during heating, and decides the optimal heating algorithm with 93% accuracy.”

There are multiple applications where image sensors are being combined with some level of intelligence. Among them:

• Cars are using imaging for collision avoidance, and increasingly they are being used to detect road conditions such as black ice even if the driver can’t see it.
• Security systems increasingly are differentiating between routine and unusual activity, including everything from facial recognition to identifying suspicious movements.
• Safety detectors in a variety of wearables can detect when a person falls and call for help, if necessary.
• Retail cameras can identify regular shoppers and predict gender, age, height, and weight to show targeted advertisements, like the scene in the movie “Minority Report.”

MIPI is an essential component for all of these applications. “To service mobile [phones] you need to be low power, low EMI, low pin count, and compact,” said Hezi Saar, director of product marketing for mobile, automotive and consumer IP at Synopsys. “MIPI has been establishing itself as the way to achieve these specifications, which are being implemented outside of mobile. Automotive followed mobile adoption, along with IoT, and that’s where we’re starting to see more applications using it. The majority of adoption is in the standard for the camera multimedia. In fact, there is progression all the time with the protocol. The PHYs provide the bandwidth needed on the C-PHY, D-PHY, and the display, as well as additional bandwidth, features, and capabilities, with compression using DSC (Display Stream Compression), for example.”

Other applications include military-grade sensors, which need to send huge amounts of data, noted Mark Lewis, senior principal design engineer in Cadence’s IP Group. “This leads to then trying to determine how to get that data into the system in a power-efficient way. That’s where the standards groups seem to be plugging away. Those are the folks who’ve got to keep an eye on compatibility with existing standards.”

For instance, other capabilities can be layered atop the MIPI camera serial interface (CSI) to transmit data in a more power-efficient way. “There’s also lots of information coming from companies about transferring a limited amount of data, rather than sending everything, which takes the weight off the application processor,” Lewis said.

One recent addition to the CSI standard includes the Always On Sentinel Controller (AOSC), which is used for always-on/always-aware applications. Traditionally, the CSI Specification defined how MIPI CSI-2 on a sensor interfaces with the MIPI physical layer (D/C-PHY). AOSC takes this further by defining an interface between MIPI CSI-2 and MIPI-I3C to transport just enough pixel information across this relatively low power physical interface to a DSP, which requests data from the sensor in order to detect an event. The detection of this event would then switch to transporting the full image data via MIPI C/DPHY to the application processor.

Another addition is the Smart Region of Interest (SROI), which provides a mechanism for transferring only selected areas of interest in an image rather than the entire image. This allows for an increased frame rate, while reducing the amount of data that needs to be moved, as well as the amount of energy needed to move it. It also means that higher-resolution images of the important area can be processed without sending redundant data.

MIPI CSI-2 supports the transport of compressed data, and although not defined by the MIPI Specifications, it can be used to reduce the data bandwidth requirement between the application processor and camera module. This has long been available in the MIPI specifications through the use of RAW data types.

Fig. 1: Evolving MIPI imaging solutions. Source: MIPI Alliance

Fig. 2: Unified imaging concept for MIPI CSI-2. Source: MIPI Alliance

MIPI in automotive
Today in automotive, there are a number of connectivity options, many of which are proprietary. But that is changing as vehicles become increasingly electrified with advanced imaging sensors and other features, and MIPI — which already is commonly used — is expanding even further.

“MIPI has been used in automotive for many, many years, and a good portion of our customers are in the automotive space for cameras and displays, as well as lidar,” said Ashraf Takla, CEO of Mixel. “Anywhere you see a sensor, there is a good chance it’s using MIPI, even in automotive. And that’s been the case for quite a few years.”

MIPI is particularly well suited for automotive environments. “It’s asymmetrical communication,” Takla said. “You’re sending a lot of data in one direction, very little data in the opposite direction. For the same reason MIPI was very good for mobile, it’s the same reasons for sensors — and the same holds true for automotive applications. While automotive is a different market than mobile, the sensors are there, and MIPI has been successful with sensors. All the sensor manufacturers use it, so it was a natural fit for automotive.”

As automotive OEMs and Tier 1, 2, and 3 technology providers focus increasingly on security — particularly where it impacts safety — the MIPI security working group [established March 2021] has ramped up efforts to address external threats. According to the MIPI Alliance, efforts will focus on defining how to map industry security frameworks, protocols, and algorithms to protect MIPI interfaces, as well as how to incorporate inputs from other MIPI working groups into future specifications and supporting materials.

“Listening in on analog signals in order to hack into a system is much more difficult than doing it on the digital side,” said Takla. “If you have the data on digital side, and it’s much more accessible. If you want to do malicious things, then of course that’s where you go. For security, we are always worried about the digital data because that’s what people are going to be monitoring and trying to get into.”

Cadence’s Lewis agreed. “Security has been built into the protocol standards because you have to be able to pass this information across,” he said. “The protocol standards dictate where that information goes, what that information is, what it contains. Security is end-to-end, and includes additional information such as what is needed to make sure this data gets from A to B, that it’s not been messed around with, that it has not been interrupted, and that the data came from the sensor you expected it to come from.”

In addition to security, MIPI activities in automotive include long-reach interfaces with MIPI A-PHY, a long-reach SerDes physical layer specification targeting ADAS, ADS, and other automotive applications such as cameras and in-vehicle infotainment displays.

Transmitting data over longer reaches requires these new physical standards.

Fig. 3: Different MIPI interconnect options. Source: MIPI Alliance

More displays in the vehicle, both large ones and smaller ones, such as e-mirrors, are good candidates for a MIPI-based display serial interface (DSI), noted Synopsys’ Saar. “In the last two years, MIPI Alliance has started to invest in the A-PHY specification in the automotive SerDes area,” he said. “It’s a fairly new specification, and this is really the basis for providing the long-reach standard.”

The vision for A-PHY is to unify automotive data, enabling a sensor to transmit a signal to the ECU, where it would be processed along with data from other sensors. From there, the processed data would be sent to one or more displays. C-PHY and D-PHY can still be used for the short reach.

MIPI A-PHY competes with other standards, including one from the Automotive SerDes Alliance (ASA). “Ethernet can be considered here, as well, even though it doesn’t compete directly, but there is still market sharing,” Takla said. “It’s not settled whether MIPI A-PHY will be the leading standard adopted. The jury’s still out on that. ASA is trying to do the best they can to capture that market. MIPI A-PHY and ASA are head-on competitors. They are going after the same market. Ethernet, of course, is a whole different ballgame, but it’s not head-to-head competition. There is overlap between MIPI and Ethernet in long reach. MIPI Alliance has been thinking about long reach for quite a while, spurred by BMW. It was really a shame that this bifurcation happened between the A-PHY and ASA, because at the beginning, it was only within the MIPI Alliance. Some companies, including BMW, were not happy with the direction it was taking so they formed ASA, and that’s when the competition came about.”

Even worse are competing interfaces developed by OEMs. “The reality is that today almost 100% of the market is controlled or serviced by proprietary interfaces, and those will continue to dominate and continue to service that market unless something happens,” said Saar. “A-PHY and ASA are attempts to standardize this toward one standard. This market is not coming into agreement what will be the right interface, what will be the single interface that needs to be used, and right now this is being seen by OEMs and IP providers as a battle between proprietary interfaces and standard A and standard B. For that to resolve itself, we either need to wait to see which technology is superior, or to use bridges to the technologies that are coming, like A-PHY or ASA.”

For an OEM, any right turn or left turn is very competitive. OEMs don’t want to hint toward where they’re going, which is complicated by the fact that OEMs are also now beginning to design their own silicon so they can determine their fate.

“The changes will not happen as rapidly as we see in the mobile industry, because the automotive industry moves slowly,” said Saar. “Both OEMs and their tiered suppliers will look at the risks and probably not jump in immediately. Instead, they will try to rely on bridges, so as not to take the risk of this automotive SerDes decision. The cameras, displays, and associated technology already exist, and the various automotive players just want to leverage whatever they can.”

Conclusion
MIPI has evolved well under the radar for most electronics. But as more industries deal with more data — particularly more image data — its usefulness is increasing, and so is its adoption.

This is certainly evident in mobile phones with cameras, but it also is becoming a mainstay in other markets — especially automotive, where the number of cameras and screens is increasing rapidly.

“While there are still a lot of unknowns in automotive, the transformation happening in the car is going to drive a lot of designs,” said Saar. “When it comes to MIPI, even with the challenges around the Automotive SerDes Alliance and other market unknowns stemming from ADAS and the changing automotive landscape, there is a major transformation underway involving MIPI.”