More Uses For Hypervisors

Hypervisors are showing up in more places than ever before as a quick and inexpensive way to utilize multiple cores and multiple chips more effectively and more securely.

This marks an interesting twist for a technology that originally was developed as a way of enabling virtualization on a PC, allowing users to run multiple incompatible applications on the same computer. That was followed in the early part of the millennium by the rapid adoption of hypervisors in data centers as a way of increasing server utilization, allowing multiple applications to run on the same machine as a way of reducing energy bills for powering and cooling those servers.

The latest incarnation of this technology is to more effectively utilize multiple cores on a chip, or across multiple chips in a much larger, connected system such as a car. Despite numerous reports over the years about the enduring value of this technology, it remains the fastest and cheapest solution, and the first choice for many design teams.

“One simple reason is that they are migrating from an old environment where they had only one core, and now they moved to a multicore device but they still have a configuration where they have—for safety or security reasons—three different cards with each card doing a different thing,” said Felix Baum, product manager, embedded virtualization at Mentor Graphics’ Embedded Software Division. “Many times when they try to use these parts, they cannot run just one software stack on top of them for a number of different reasons. Now they are running on a three- or four-core SoCs but they cannot combine all their applications for those reasons because of the security, safety or other reasons.”

The software team ends up running different tasks natively and independently, he said. One core runs one application that was running on a separate board, another thing runs on a second core that used to be a separate board. “Imagine you have a VME or PCI rack, and now you are trying to run it on an SoC. Inasmuch as folks may claim that it is do-able and possible, it requires a lot of discipline and testing because once you combine things, whether you want it or not, you’ll run into a situation where the memory is shared, the serial ports are shared, the Ethernet is shared. Sooner or later you’re going to run into a situation where one application that used to run on a card or on a board in an old system, now happens to run on a core trying to reach out and touch a certain chunk of memory or a peripheral, and another application that used to run on a secondary board but now runs on a second core on the same SoC, trying to touch the same resource. They collapse and they hurt each other. You get into timing issues and the system becomes not as robust as developers would have liked it to be.”

This is where hypervisors fit in. Steve Wilson, vice president for core infrastructure at Citrix, said that the first time he was exposed to the idea of hypervisors was more than 10 years ago during a meeting on Wall Street with some IT guys from a large investment bank. “One of them said, ‘I have a major problem because I can’t install any more computers. It doesn’t matter whether I could pay for them. I have plenty of money. I just can’t run any more computers. My datacenter is on the 25th floor of this building. It’s the entire 25th floor of the building and I have filled every conduit running from the ground floor up to the 25th floor with power cables. I’m out of power and I’m out of space.’ We talked about what he’d been trying to do to alleviate this. He said his first plan was to build another data center in New Jersey, so he went and priced that out, went to his boss and said he needed $150 million to build a new data center. Not surprisingly, his boss told him no. So we started having to look at creative alternatives.”

They determined the average utilization of the CPUs on these servers, on average across the day, across all the systems, was 5%. All those computers filling up that whole floor, burning power, burning air conditioning, were idle 95% of the time. “Why is this? If you look at a typical computer program, some of them just don’t do that much on a given point in a day, or during the month, but at the time there was no way to combine those,” he said.

Hypervisors to the rescue
Hypervisors suddenly became indispensable. “The use in IT grew along with the need to run multiple computer programs on the same physical piece of computer server hardware, because the problem is if there is one server per program, there will be a horrible rate of efficiency, and the cost will be 20 times too much for the computing power,” Wilson said. “And while the idea of running multiple programs on one computer might sound like a good idea, they are designed for different operating systems, they are written at different times and don’t coexist well. So an extreme case might be that one was written to run on Windows, and one might be written to run on Linux, and those can’t run at the same time.”

Enter the hypervisor, which is essentially a special piece of software that generates virtual machines. The hypervisor’s job is to create multiple virtual pieces of hardware.

“Let’s say I’m building a car,” he explained. “If you look at the auto industry these days, a huge amount of the intellectual property that goes into a car has nothing to do with internal combustion engines. It’s software. If you look at a Tesla, it’s running a fairly off-the-shelf computer at its core, but it’s running a gigantic collection of software. Some of that may be developed internally, but some of the applications may come from third parties and will be integrated into the system. And they don’t want to have to buy a new computer for every application that they are going to run in the car, such as a mapping computer, an auto-drive computer, engine-control computer. They just want one.”

However, each of these applications might be written to a different type of software environment. And, in fact, there are good reasons to keep them separate from each other, particularly for security reasons. “I don’t necessarily want the infotainment system to be able to reach over and control the auto-drive system from a security perspective, I want to keep them isolated. Hypervisors have been designed to allow a clean separation between different pieces of software, keep it secure, and make it so the applications can work together cooperatively without even knowing about each other because each think they have their own physical piece of hardware they are running on,” Wilson offered.

Drew Wingard, CTO at Sonics, agrees: “In embedded systems, it’s a way of firewalling open from protected use. There are more efficient ways of doing that, but they require enormous amounts of work.”

Wingard noted that a constant concern is how much damage these multiple uses can have on the system. Nevertheless, he said that for security reasons — and particularly from a cost-benefit standpoint — hypervisors make a lot of sense

One size does not fit all
With embedded systems, where engineering teams have used 8- or 16-bit microcontrollers, then migrated to 32-bit processors, and now have multicore processors, these embedded applications still try to run independent, isolated tasks. But the hypervisors used for the last 10 or 20 years may not be the right fit because the things that embedded software developers are concerned about are different than things that networking router development teams are concerned, such as throughput

“In the embedded space you care about isolation, you care about safety, you care about security,” said Baum. “A lot of engineering teams that come to us and start talking about hypervisors don’t really talk about virtualization. They are talking about separations and guaranteeing the separation/isolation between things.”

As such, it is important to get a handle on the different types of hypervisors.

First, Baum explained, there are Type 1 (also referred to as Type 0) hypervisors. “You can have a hypervisor that, when you turn on the power, this piece of software comes up and all it does is partition the hardware. It says, this chunk of memory is assigned to this resource and this core, and that chunk of memory to that resource. It sets up hardware-based separation features, and then it just lets go, meaning that it doesn’t get involved in anything, it just lets the operating system boot up. The only time that hypervisor Type 0/Type 1 gets to run, once everything is separated, is if one of the operating systems tries to touch something that it shouldn’t. Because of that, these Type 0/Type 1 hypervisors are real fast, really deterministic, and they provide very, very little overhead as far as the performance.” Vertical market segments that are being targeted for Type1/Type 0 hypervisors include automotive, industrial, medical, IoT, and gateways.

There also are Type 2 hypervisors, such as those made by VMWare, KVM, Xen project and a few others. A good example of this is on an Apple Mac, which can run both Mac OS and a simulated Windows environment. Another distinction for the Xen Project is that this type of hypervisor runs on x86, Intel and ARM instruction sets.

All of this speaks to the overarching idea that software teams will deal with bigger challenges going forward, said Simon Davidmann, CEO of Imperas Software. “A hardware processor can go from one to four or eight cores, and either they are put into silos doing different things like image analysis, or running an SMP version of Linux where everything is separate jobs. They’re not trying to do big parallel things in typical embedded systems. The challenge that they’ve got now is that there are multiple interacting subsystems, which are being more and more distributed. For instance, in a car each of the blocks in an ECU is running software, but they’re all interconnected, they’re talking across CAN, but actually it is one system and there’s a lot to be designed and verified across the whole system not just the small things. The issue is that there are multiple processes happening in parallel on top of a new generation of challenges where there are different silos. On one piece of hardware you might have three different operating systems running on that processor, all in different silos, and using hypervisors. So that it’s no good just to have Linux on a box.”

On the other hand, he asserts that hardware teams have almost got it sorted. “Going to smaller processes and doing mixed signal, absolutely you can’t underestimate how complex chips are. And the cost and the effort and the skills and the expertise and the very deep people you’ve got to have. It’s tremendous getting a big chip built, but actually the software is still unanswered. People really don’t know how to do it all. They use threads and then they find the things lock up two years later, then they find that there are vulnerabilities and people can break in even though they can’t. The majority of people in the embedded world have no clue about any of this. And then somehow their boss says to them, ‘We’ve got to have a USB stack, Ethernet, it’s got to do some Wi-Fi, it’s got to have a GUI, and you’ve got three months to do it.’ So they say, ‘We’ll get an ARM board from ST or Freescale or whomever it is and put Linux on it.’ But actually they really have no idea how all the vulnerabilities and difficulties and challenges and the complexities of this stuff. They are trying to re-use software but it’s very tough. So I think the software guys have a huge nightmare.”

These new challenges require new tools and new technologies. “The old way of debugging applications was you ran something like a gdbserver [a computer program that makes it possible to remotely debug other programs] in your product so that you could then control your application from outside,” Davidmann explained. “But the problem in sophisticated, real-time, multicore stuff is that that perturbs the system, and you lose determinism and controllability, and you need all of that. As things get more complex, we have to have better abstraction to the tools, and the paradigm we had at GDB is 20 years old and very simple. The current solutions need something far more sophisticated than that.”

The good news is that everybody is getting smarter, Baum asserted. “We see that newer processors, chips and architectures are more sophisticated and add certain hardware-enabled virtualization technologies. For instance, the ARM Cortex A9 has pretty much nothing compared to the Cortex A15, which has some rudimentary features of virtualization. Now you look at Cortex A53 and above, and there is much more support for this virtualization to speed things up and to optimize things.”

Related Stories
Will Hypervisors Protect Us?
They may not be a silver bullet, but they are a good first step when it comes to securing cars and the Internet of Things. Problems start when people believe the job is complete.
Hypervisors: Help Or Hindrance?
Almost everything is a tradeoff and tipping the scales is usually influenced by the end product goals. Hypervisors have a few such parameters.