Auto Displays: Bigger, Brighter, More Numerous

Displays are rapidly becoming more critical to the central brains in automobiles, accelerating the adoption and evolution of this technology to handle multiple types of audio, visual, and other data traffic coming into and flowing throughout the vehicle.

These changes are having a broad impact on the entire design-through-manufacturing flow for display chip architectures. In the past, these chips were largely considered consumer electronics. But as they are integrated into safety-critical systems, where security is a big concern, they are undergoing a fundamental shift in how they are architected, tested, and deployed inside of vehicles. These chips also are being called upon to do more as displays take on a more central role in how drivers and passengers interact with vehicles.

The most visible sign of these changes is that the displays themselves are getting bigger. Many now exceed nine inches, and they have better resolution and a variety of enhanced features. Along with increased safety, the displays play a key role in personalization of the in-cabin user experience. In fact, future vehicles are expected to have many more displays of all sizes.

“These new touch features include hover and proximity sensing, force sensing and haptic / audio feedback, sleek designs with rotary and slider functions, all while working seamlessly and consistently under challenging use cases, e.g. glove usage, condensation, in a hostile EMC environment.  All of these new features present unique challenges to display makers and system integrators, such as low power mode requirements, styling and reliability. For instance, enabling a reliable hover feature involves display manufacturers adding a shield layer to reduce display noise by a factor of 5-20 to improve hover distances,” said Ray Notarantonio, senior director automotive vehicle user experience and EE architecture at Infineon Technologies America. He noted they have to work properly even when drivers are wearing gloves and when there is condensation on the screen, and they have to be electromagnetically compatible with everything else in the cabin.

Others agree. “Display technology is going to the next level,” said Willard Tu, senior director of automotive at Xilinx. “OEMs are liking the display trend. There are multiple displays, different shapes and sizes, higher resolutions, and improved lighting control. An instrument cluster in the past was mechanical analog gauges in nature, and then added segment LCD displays. Now they are TFT displays. Not only is the nature of the display technology changing, but the design and shape of the display — continuous, curved, retractable, rolled displays are in the works. What you see in consumer homes will end up in a vehicle at some point in time. Consumer electronics is definitely reshaping automotive displays technologies. We see a potential of 10X displays in a vehicle. 4X driver information (cluster, IVI/controls, navigation, message center, augmented reality), 1X passenger, 3X rear view/side mirrors, 2X rear seat entertainment.”

All of this technology is expected to show up in new vehicles within the next few years. There may be as many as 14 different displays inside the car, with physical mirrors gradually replaced by displays.

“There will be instrumentation clusters, infotainment, backseat entertainment, climate control, displays on the passenger side, displays for side mirrors, a display for the rear view mirror, a heads-up display, then displays in the back seats for the kids to watch videos,” observed Alain Legault, vice president of IP products at Hardent. “The problem is the number of cables. The cable harness inside a car is the third most expensive, and the third heaviest component inside a car. Automakers don’t want to see more copper inside the cars so they’re willing to pay extra for silicon to save on the cables.”

Making it all work
Mustafa Ali, senior product manager for the Automotive & IoT Line of Business at Arm, also sees the trend toward an increase in the number and resolution of screens per vehicle. But he noted that OEMs want these multiple screens to be managed by a single ECU, which in turn needs to have built-in functional safety capability.

One of the challenges SoC vendors face is there is no single standard for the video interface. That depends on which devices will be used in the car and the resolution of the display, and it can vary depending upon the distance from the SoC. All of this needs to be considered at the architectural level.

“Deciding on a protocol, speed, compression, topology are critical items for the SoC vendors right now,” said Hezi Saar, director of product marketing for mobile, automotive and consumer IP at Synopsys. “In addition to interfaces, another effort being injected into the display and camera markets is what long-reach solution will be used. That changes the whole topology. As displays replace mirrors, how do you get the bandwidth of images transmitted from the SoC to the multiple displays that are there? Currently, a chain topology is being looked to as the way to reduce wiring, and the cost of having so many SoCs driving different displays. In order for us to implement something like that, it requires the ecosystem to change. It requires us to think, ‘Do I use an SoC that has, let’s say, a DisplayPort on it, and does it drive 8K?’ That 8K is effectively driving multiple smaller displays, and one 4K display in the middle. We need to look at how the whole topology works out.”

Fig. 1: Automotive display implementations. Source: Synopsys

Display ecosystem differences
Within the automotive ecosystem, display subsystems are becoming more integrated. It’s a good way to re-use IP, but the level of technology integration can vary by geography. In fact, there are differences in how Japanese, European, and U.S. display supply chains are dealing with display advancements, said Kurt Shuler, vice president of marketing at Arteris IP.

“A lot of these companies were derived from OEMs,” Shuler said. “Denso, for instance, was part of Toyota before it was spun out. We are seeing differences as far as who is actually creating these chips. Is it the design team? Is it a traditional semiconductor manufacturer? Or is it a Tier 1? It may or may not be the traditional chip vendor. In Japan, there are companies like Fujitsu, and NEC that went into Panasonic, along with companies such as Denso that are designing their own chips, as well as running an IP business. There are other big players like NSITEXE [fully funded by Denso] and Renesas Electronics, the latter of which has been an incumbent in car displays for a long period of time worldwide. In Europe, companies like NXP have long been big players there, but Bosch and Continental also are creating their own chips. In every situation, there is recognition that the displays are becoming critical to the overall brain of the car, so the business is changing.”

As cameras take in data, process it for the driver, and then display it, that display becomes an integrated component rather than just a screen hooked up to a back-up camera.

“It’s a system. In the case of an ADAS system, it could operate in such a way where it sees something in front of the car, and says, ‘I’m going to tap the brakes for you, change your speed for you.’ That part of the system is most likely ASIL-D or ASIL-C,” Shuler said. “The display part might be ASIL-B, because that part of the system can still work, it’s displayed on the screen, and you see the bumper almost getting hit early. But that may not be as safety-related as, ‘You better slow down before you hit that car.’ So there will be mixes of ASIL levels in that, too. Aspects of the system that are the human-machine interface (HMIs) are usually at a lower ASIL because of the amount of diagnostic coverage for random errors that occur in that. As these display chips are becoming more of integrated systems, there are more HMIs. It’s a display, but it may have a touchscreen or may take inputs from all kinds of different controllers. There are latency requirements, and redundancy requirements because the controller is connected to the transmission so there must be redundancy, etc. It’s becoming a lot more complex.”

With many different types of data coming into the vehicle, data management is more important than ever, and automotive developers are trying to keep data associated together more than was seen in the past, according to Simon Rance, head of marketing at Cliosoft. “In the past, we saw a lot more focus on the design data. Now, there is more focus on the fact that engineering teams have the IP data that’s being integrated into the design, and how it’s all associated. Currently, we’re getting involved a lot more in the system level data, which takes the design, the IP integration of it all, and at an SoC level, how does that all come together with everything associated with it from the design data side to the documentation, traceability, and all pieces and assets that go along with it. In that way, it’s an asset management type of activity as well what’s getting associated with it.”

Particularly in automotive designs, asset management, and documentation are critical because of the functional safety aspect, traceability, and the long life cycles that these vehicles, their chips, and onboarding system have. “This is important because these systems are still designed years before they actually reach a dealership. For example, typically these things are designed years before that. Let’s say it takes three to five years to design the system, and they’ve used the data management technology to do the designs, and manage all of the assets. Then it goes into service, and it’s going to be operational for at least another 5 to 20 years. The data management that we’re seeing, and the impact it has there even for just automotive is the fact that everything has to be traced. It has to be kept for that long of a lifecycle because if something goes wrong, they’ve got to do a root cause analysis to find what piece of the data, or which asset or IP may have caused a potential failure, along with who was involved. So it’s not just the data, it’s the people involved with that design, the data, and the integration of it,” Rance explained.

Display tradeoffs
Including displays in safety-critical systems also increases the number of technical tradeoffs.

“With the display trend brings a slew of technical issues, such as interface options and lighting control,” said Xilinx’s Tu. “Programmable logic allows you to configure your own interface if it is not already available. But more importantly, we see a need for lighting control. Automakers want to use OLED technologies. However, there is a premium to this technology, and OLED suppliers largely focused on the consumer market right now. So designers are still relying on TFT displays. Many of the TFT panels other than the instrument cluster are not being recessed anymore because they are now touch screens. These displays are subject to ambient lighting washing them out, making them difficult to see. Hence, the concept of TFT+. This is where you are using two lighting panels to do the illumination, which makes the lights lighter and darks darker.”

With safety considerations come other tradeoffs, as well. “There was a long debate when DSC was introduced about display stream compression,” said Synopsys’ Saar. “Here, is ‘visually lossless,’ just a marketing term?”

With a consumer video, if an image is faded or a pixel is lost, it’s probably not noticeable. But with an automotive application, that could make a difference. Compression can affect the reliability of a display, which in turn has implications for the amount of data that needs to be moved throughout a vehicle.

“Would you support raw transmissions across the board, or would you support compression?” What type of compression, and how heavy will the compression be? Some of the displays are just entertainment,” Saar said. “But some of the displays are borderline safety or true safety, like e-mirrors and the cluster, where the driver sees all the information about where the car is driving and all the warning signs. These displays require higher safety ASIL certification, and the rest don’t really need that. The tradeoffs are between the bandwidth and compression, between how safety features are playing out, and also power. You need to think about which interface will give you the best bang for the buck for the complexity you’re looking to support.”

It’s also important to consider what happens in the dashboard console in the context of functional safety. “We’re getting more customers saying, ‘We’ve been doing dashboard displays for a while, but now we need to be ISO 26262 certified,’” said Arteris IP’s Shuler. “It’s usually ASIL-B compliant, and ISO certification adds hardware and software safety requirements to the design.”

Joe Rodriguez, product marketing manager at Rambus, also sees ASIL-B as a fairly common safety deliverable. “This is an overlay to the RTL, so we have to meet all of the fault coverage and related items associated with that.”

As display technology is more tightly integrated into safety-related functions, the demands for compliance will grow. Arm’s Ali sees strong interest in functional safety graphics as the focus shifts from physical dials and lights toward digital instrument clusters on LED screens. “This demands smooth, glitch-free rendering. Another area that continues to evolve is 360° surround vision, where the driver is able to see a bird’s-eye view of the car and its surroundings. Both of these trends require complex graphic processing with emphasis on safety.”

This has a big impact throughout the design cycle, as well. “The classic play for IP in automotive until two or three years ago was mostly with the traditional SoC vendors,” said Synopsys’ Saar. “Today, there are more players with more startups across the world. The automotive market is transforming heavily. We work with Tier 1s that are typically just putting the system together and are looking at, and developing ICs and SoCs now. Carmakers are considering or entering that market too. Semiconductor development is happening across the ecosystem. It doesn’t stop. It’s still a slow-moving industry, but it’s much much faster than what it used to be. And those players would like to either push their optimized architecture, or have their own vision of how the car should look. They want a sense of ownership on the services they would like to add, not just relying on an SoC vendor that will deliver a chip in three years and will put the control in the hands of this SoC.”

Also, while the zonal architecture is being adopted now in high-end automotive design, there are other designs that are not necessarily jumping into the zonal architecture immediately. “Hybrid architectures that still maintain the cockpit/IVI SoCs or ECU separately, and ADAS SoCs/ECUs that are separate will probably be mainstream for the next couple of years. That’s why you need to have multiple designs and multiple flavors. For an SoC maker, it’s important for them to decide which niche market to go after — or whether they will go after a broad market — and how do they satisfy it with the least number of interfaces so they are cost-effective for multiple kinds of applications,” he added.

Conclusion
With so many different ways to architect compute today, the tradeoffs seem endless. Rambus’ Rodriguez said he has seen MIPI used to transfer wirelessly, even though that’s not the original intent of it. “Some engineers and architects see it’s just a conduit. MIPI C-PHY is pretty efficient, and has a low pin count, so they’ve attached their IP, which is a wireless transfer. It’s kind of crazy how people are adapting to the needs in the market space, and that’s the way life is when there are creative people driving these industries.”

Related
Battle Brewing Over Automotive Display Protocols
Different options and standards will take years to sort out.
MicroLEDs Moving From Lab To Fab
Bringing the cost down and yield up on microLED is proving to be formidable, but display companies and LED suppliers are working together toward production-worthy solutions.