Testing Cars In Context

The choices for companies developing systems or components that will work in autonomous vehicles is to road test them for millions of miles or to simulate them, or some combination of both.

Simulation is much quicker, and it has worked well in the semiconductor world for decades. Simulating a chip or electronic system in context is hard enough. But simulating a system of systems in the real world is unprecedented. At this point no one is quite sure of what that context will be, how large it needs to be, and whether that is really sufficient for everything from self-driving cars to drones.

There are few landmarks to guide the tech world here. In fact, the closest example of simulation on this scale is the Boeing 787, which was delayed for years because simulations didn’t catch all of the problems, notably problems with overheating lithium-ion batteries. Boeing’s simulations showed the batteries would function well for 10 million hours, which turned out to be significantly optimistic.

While commercial jets are extremely complex, they are an evolutionary extension of systems that have been in the air for decades. That’s not true for autonomous cars, and the simulations required for cars will be significantly more complex. Air traffic can be annoying for passengers trying to get into busy airports, but that looks rather simple compared to a traffic jam in Los Angeles, Mumbai or Bangkok. There may be hundreds of jets in the vicinity of a major airport, but there are hundreds of thousands of cars heading in all directions in major cities at rush hour or on holiday weekends.

How a car will navigate those conditions isn’t entirely clear. The image presented by carmakers of passengers lounging comfortably in a well-appointed cabin listening to music or typing on a mobile device could easily be transformed into a mob of irate people trapped in a sea of immobilized carriages with no steering wheels, no gas pedals, and no immediate way out—helplessly watching their batteries run down. It could take weeks to clear the roads and reset everything, and it likely will make people very reluctant to get back into those vehicles.

Carmakers are correct in seeking zero defectivity. If one simulation goes wrong, it can gum up the entire traffic flow beyond anything ever witnessed before. But it’s not clear whether simulations will ever be complete enough to make all of this work properly. There will always be corner cases, and the only way to find those corner cases sometimes is to actually go looking for them, such as what caused battery fires in the 787.

There are tools such as formal verification, which can help route out problems once they are found. But identifying all of the problems up front is impossible. And on the road, in various regions around the world, there will always be unexpected problems.

Fig. 1. Can you simulate a sinkhole? Source: NBC News

No matter how good the planning, there will always be the unexpected. And while simulations can help to develop emergency systems, those simulations probably will require far more real-world data than is available today, and significantly more processing power than even large cloud operations can provide today.

In effect, simulations will require all of the possible interactions of people, vehicles, and unforeseen acts of nature within a given radius. The solution may build far more intelligent AI systems than are available today, and that may not happen so quickly. It’s hard enough for these systems to distinguish between a dog and a cat. Understanding that a tree branch is falling from above or that the road ahead is sinking is a whole different level of problem. Simulation is a great tool and a huge jump start for designers, but it won’t solve everything.

Related Stories
How To Test Autonomous Vehicles
New technologies, methodologies and flows will be required to guarantee self-driving cars are safe.