New Architectures Redefining The Data Center

By Ed Sperling
The cost of powering and cooling data centers, coupled with a better understanding of how enterprise-level applications can utilize hardware more effectively, are spawning a new wave of changes inside of data centers.

Data centers are always evolving, but in this sector that evolution is deliberate and sometimes painstakingly slow. In fact, each major shift tends to last a decade or more. In the 1990s, the big shift was from monolithic mainframes and minicomputers to racks of commodity PC servers. And as processor density increased and the cost of powering up and cooling those racks escalated, the target in the early 2000s was improving server utilization to reduce those costs, ushering in a wave of virtualization.

The current focus is microservers, where number crunching is far less important than further reducing the power bill. That moves the emphasis from faster and faster processors to equivalent or better performance by splitting the job across multiple servers—in effect a more rationalized utilization of server resources for less power.

Power is the key in this equation. Inside large data centers, the energy bill literally can cost millions of dollars each year. Virtualization was the first major stab at reducing that cost. The latest shift is being driven by new applications that can work more efficiently. Open-source Hadoop is a good example. The analytics software effectively parses the processing into discrete chunks, which in turn can be handled by multiple servers. There are other applications in the pipeline that take a similar approach, particularly for storage and networking.

Both ARM and Intel are betting big on these low-power server architectures, and all of the major server vendors are lining up offerings in this area as the cost of power emerges as the key differentiator.

“There’s a big push to re-architect the data center,” said Lakshmi Mandyam, director of server systems and ecosystems at ARM. “Even with virtualization, utilization is still at most 40%. That’s a lot of power to pay for if you’re using it less than half the time.”

Mandyam noted that in conjunction with this shift, even the definition of a server is changing. “Storage, networking and computing are all converging, enabling highly integrated architectures for better performance,” she said. “In the emerging world of the Internet of Things, what we previously would have called an embedded node can now take advantage of the ARM server energy efficiency and cost effectiveness to offer new and interesting services.”

Design points
These are more than just terminology changes, though. A push toward right-sized servers for specific uses, which can range from video streaming to generic Web servers running various flavors of Linux, has created a scramble for differentiation among vendors selling this new class of boxes. It marks a fundamental shift away from server hardware that has focused almost exclusively on performance, instead putting good-enough performance where it is needed. There will still be a demand for powerful servers of all sorts, from Intel-based blades to IBM mainframes, but alongside those devices there is plenty of room for more efficient and targeted machines.

For server makers, this is the biggest opportunity to emerge in years. They’re all scrambling to differentiate their designs, and that means far more than just the ARM or Intel processor inside these boxes. In some cases, it’s the software bundled with the hardware. In others, it can include everything from the I/O architecture to accelerators for specific applications.

“One of the differentiators is the number and performance of the cores,” said Ron Giuseppe, senior strategic marketing manager for Synopsys’ DesignWare IP for Enterprise/Data Center Applications. “That can include different mixes of DDR3 and DDR4. But in addition, for power and performance, we’re seeing companies adding in protocol acceleration engines. So what you’re getting is more heterogeneous processing. It’s primarily processing that implements logic, 32-bit protocol engines, and hardware-software partitioning. Then, for protocol acceleration, such as with an e-commerce server, they’ll add something like an SSL application. If it’s just software running on there, it will use 100% of the processor. But with secure cryptography, you may be able to offload 70% of that to optimize it further.”

Microservers by design run significantly cooler. Giuseppe said that the typical pizza-box server runs at 200 to 400 watts, while a microserver may use as little as 10 watts of power. “They’re lower power, they create less noise, and they’re more reliable.”

Noise is a big issue inside of data centers. The amount of air being blown through racks of blade servers for cooling purposes, and the fans on the individual servers, are so loud that sometimes it’s difficult to hear. Microservers are remarkably quiet in comparison.

So why not use them in more places? A lot of people are beginning to ask that question, with some interesting possible ramifications. Rather than just networking together boxes using the same technology, the individual servers can be used with their own memory or with shared memory, such as a collection of Hybrid Memory Cubes.

“We’re seeing the beginning of a phase where rather than having a main CPU with memory, you can build CPUs with multiport memory,” said Joe Rash, senior director of business development at Open-Silicon. “So you design a mesh network of memory and processors surrounding that memory. That doesn’t require as much movement of data.”

Big consumers of servers, such as Facebook, Google and Amazon, are particularly keen on cutting energy costs because they have a direct bearing on profitability. Amazon and Google view their architectures as competitive secrets, but Facebook has given its open-rack specification to the Open Compute Project (www.opencompute.org) in an attempt to get more outside input and reduce total cost of ownership. That organization is actively developing new ideas to further improve efficiency.

“You’ll still need number crunching for certain applications,” said Rash. “But for others, such as search or some data applications, a centralized CPU can break up the job. You don’t need all those CPUs communicating through memory anymore.”

New opportunities
All of this is just a starting point, though. Shifting direction in server design needs to be set in context, both at the data center level and at the chip level. On the semiconductor side, small server processors have been almost exclusively rooted in CMOS because they have been driven by Intel’s almost laser focus on Moore’s Law and reducing the cost per transistor. But in a data center, the cost of the equipment in many cases is less expensive than the cost of powering and cooling that equipment, which means there is some new resiliency in the price.

This has attracted interest from lots of new players that previously had little interest in this market. Soitec, for one, which makes silicon-on-insulator wafers, has been pushing its FD-SOI at 28nm as a way of reducing power in SoCs. Steve Longoria, senior vice president at Soitec, said a different chip substrate alone could reduce power or improve performance by 25% to 30% in the microserver world. Add in different architectures and more efficient software and the savings goes significantly higher.

“We’re having discussions with those kinds of players, as well as the makers of commercial wireless infrastructure,” said Longoria. “This isn’t a market that needs bleeding-edge performance, which means they could reduce power significantly.”

Network on chip vendors are watching this space closely, as well. While the initial reaction has been, ‘Not yet,’ the opportunity changes with more heterogeneous processors working with various memory architectures.

“The server chip has been a one-trick pony,” said Kurt Shuler, vice president of marketing at Arteris. “And with homogeneous architectures, the overhead has been nothing. But when you start getting 64-bit CPUs and fixed functions, lots of inputs and all of this is very characterized, that starts getting much more interesting.”

Next steps
There is still much work to be done in this area. Commercial OSes such as Red Hat, Ubuntu and CentOS still need to be optimized for these new architectures. And software on all of these new platforms needs to be tested to make sure it runs 24 x 7 x 365, as well as to make sure it is fully compatible with virtualization software from companies such as Citrix and VMware. Work also needs to be done to optimize some of the applications that distribute processing across multiple servers rather than multiple cores. And cache coherency in microservers, particularly those used for storage, needs to be fully tested.

Nevertheless, the push for more efficient architectures and potentially different materials in the data center is a subtle but significant shift. As with any changes inside data centers, it will take years to unfold because companies are extremely careful about protecting their IT assets. But the direction appears to be fairly straightforward, even if the path to get there isn’t always quick.

“The focus now is on integration and optimization and total cost of ownership,” said ARM’s Mandyam. “We will optimize and optimize and deploy that to thousands of servers.”