New Battleground In The Data Race

For the past couple years, giant commercial data centers have been grabbing as much data as possible. The big question now is whether that investment will pay off.

Companies such as AWS, Google, Microsoft, Alibaba and Baidu are not necessarily the best equipped to leverage that data—or at least not yet. In fact, most of what they’ve been focusing on is a narrow slice of the data being collected, primarily for marketing reasons. The basic idea is that marketing firms and retailers will pay handsomely for information about buying habits of certain slices of that data, such as teenagers in Palo Alto, Calif., in households earning $250,000 to $350,000 per year.

This is just scratching the surface, though. The real value in data runs much deeper, and in the future that value will probably be realized much closer to where it originates, both within the devices themselves and within industry segments that develop those devices. And that has profound implications. If more data is processed or pre-processed by end devices such as cars, medical devices, manufacturing equipment, and utilized by the manufacturers or to add resilience into systems in operation, then what becomes of the billions of dollars spent on setting up these massive cloud operations?

In the case of Amazon, that investment is still core to the company’s business as the world’s largest retailer. And in China, where the government has deemed all data will be centralized, including the data from sensors in autonomous vehicles, there is little risk of change. But for a broad swath of other cloud providers, both large and small, it’s not clear how any of this will shake out. The cloud isn’t going away, but the size and definition of what constitutes the cloud may undergo some serious shifts.

For one thing, big industrial companies and carmakers aren’t going to share data with cloud vendors. They’re going to use it internally to figure out how consumers are using their products, where they need to improve their processes to reduce defects, and what they can trim because it’s not cost-effective. At one point the idea was that everything would be processed in the commercial cloud, but that approach is widely recognized these days as unrealistic. For one thing, it takes too long and requires too much energy to send huge quantities of data back and forth. Despite the promise of 5G millimeter wave, widespread rollouts to ship more data to the cloud are still years away, and the cost of streaming video from billions of sensors doesn’t make sense on any level.

As more data is processed at the source, there is less data that needs to be stored for each device (even though the number of sensors is rising), and more intrinsic value in that data. But the value is to the manufacturers and others in a particular market slice, not to the data center. And that’s becoming more evident as data analytics are combined with domain expertise.

In most new markets, the initial phase begins with a flood of startups, followed by consolidation, specialization and disaggregation. The data market is something of an aberration because the common link is processing and storage of that data. But as more companies begin collecting data themselves and developing their own clouds—and there will be many more of them—rather than storing everything in a commercial cloud, data increasingly will gravitate to the industry in which it was created. This is no longer about parking data in a convenient storage facility and allowing access to anyone willing to pay for it. The real money is in the data, and the real value is for those who have a deep understanding of what to do with it.