The Multicore Processing Conundrum

We drive relentlessly into our technological future and often it seems like we’re upgrading our high-performance vehicle as it speeds forward. That’s no easy task, to be sure.

We were roaring along fine, observing Moore’s Law, and then we hit a speed bump. So design teams quickly adopted multi-core designs to compensate for the fact that pushing up speeds on single-core CPUs was a meltdown dead end (absent cryo-cooling).

Multicore explosion
We’re now in the era of quad and even octa-core processors not just in laptops but in smart phones! That and 64-bit processor architectures are expected to overwhelm the mobile space next year.

But is that a great thing? Not necessarily right now, according to a longtime industry analyst.

Mike Demler, senior analyst for The Linley Report, a semiconductor technology analysis firm, popped out on the speaker’s circuit last month to keynote MemCon 2014 in Santa Clara. “Do you really need eight cores?” he asked the audience. At first it seemed like a naïve question from an electronics industry veteran who should know better.

My reaction was along the lines of “Of course we need them! It’s what we do! We deliver more for less on schedule, every 18 months!” And given the applications that are elbowing their way into the mobile experience—think console-class gaming software and 4K video quality—it’s only natural that octa-cores will be a standard configuration soon.

Perhaps way in the future, but now?

Demler said he doesn’t haven’t eight processor cores in his home computer, so what would he do with them in his phone?

Dark silicon
More precisely, the question is what are engineers doing with four-core configurations today? The answer is: not as much as you’d think.

Often many-core designs operate with most of the cores “unlit” most of the time, Demler said. He clicked up a slide from Chinese mobile system benchmarking vendor. The slide showed two sets of tests that cycled through the top 10 applications that users have downloaded from Google Play to run on Android devices. In a quad-core configuration, one core is lit up about half the operating time; the second core “may a third to half the time and third and fourth cores (are) hardly lit up at all,” Demler said.

Sviral, a startup run by former Chips and Technologies CEO Gordon Campbell, goes a step further. Analyzing a 2011 MacBook Pro, the company estimates that less than 1% percent of the system’s 503 processor cores is used at any given time.

“Yes, people are putting more cores on chip but … what are we going to do with them and do we really need them?” Demler said in his keynote.

Programming is a big challenge. “Today all silicon ‘performance gains’ result from adding these additional processing cores, yet with few exceptions, the world’s 25 million programmers cannot program them efficiently,” according to Sviral.

A lot of smart people are attacking the problem, and it won’t remain a problem forever. (Demler noted that quad- and octa-core designs aren’t the only paths to pursue to higher performance mobile systems as 64-bit architectures ramp up, notably with the ARM v8 processor architecture).

The virtues of darkness
Indeed, some smart observers don’t think it’s a problem at all.

“I don’t know that it is such an awful thing that we have so much dark silicon,” said Chris Rowen, chief technologist for Cadence’s IP Group and Tensilica founder. “After all, what we want is systems that are responsive to our needs. If we have just occasional needs, then the resources shouldn’t be wasting power when we don’t need them.”

In Rowen’s world view of computing, there are three basic ways for thinking about multi-core systems:

1. Computing pools with homogeneous ensembles of cores with smooth OS-based load balancing of large workloads of more-or-less independent sets of tasks.
2. Computing hierarchies – heterogeneous sets of cores implementing “cognitive layering” (where cores compute the data most relevant to their capabilities).
3. Independent subsystems that implement different classes of compute (for example 3D graphics, audio, vision, storage, wireless networking.

“What we want is systems that are ‘always ready’ NOT ‘always on,’ with the right kind of processing capacity for the demand de jour,” Rowen said.

The outlook
Demler, peering into the near future, said single-core designs are being supplanted by multicore designs. Quad core implementation will be the average next year, he added.

“The 64-bit architectures are going to take over in the next year,” he said, adding high-performance graphics processing units bringing desktop like gaming experiences “will become the norm, even in midrange smart phones.”

We continue to tweak and improve our high-octane vehicle as it roars down the highway. Imagine what we’ll have when we drive over the programming roadblock.