Microsoft Extends Big Data Management Scope on Azure

How the Cloud Is Changing the Way We Work

Microsoft this week at the Microsoft TechEd Europe 2014 conference announced significant expansions to its portfolio of tools for managing and analyzing Big Data running on the Microsoft Azure cloud.

Herain Oberoi, director of product management for Microsoft Data Platform, says Azure Data Factory is available in preview and provides a framework for managing and orchestrating multiple types of Big Data being processed on an Azure cloud. Azure Stream Analytics, also available in preview, then provides the streaming analytics to analyze that data in real time.

In addition, Microsoft has announced that Azure Event Hubs, a service that makes it possible to collect data from millions of devices connected to the Internet of Things (IoT) is now generally available.

Collectively, Oberoi says all three offerings are part of an evolving portfolio of tools that Microsoft is pulling together to manage and analyze data on its cloud. The simple fact of the matter is that most data is going to live where it is originally born. As such, Microsoft needs to develop a portfolio of Big Data management and analytics services in the cloud. Over time, those services will then become federated with data management and analytics capabilities that run on premise to create hybrid cloud computing environments, which Oberoi says should be thought of as Information Management 2.0 in the enterprise.

The challenge facing IT organizations going forward, says Oberoi, is making sure they partner with cloud service providers that can provide those types of information management frameworks versus leaving customers to roll their own data management frameworks on an infrastructure-as-a-service (IaaS) platform.

Of course, Oberoi also notes that the more data that runs in the cloud, the simpler it is to manage. In the cloud, there is usually only one single-tenant instance of that data, versus in an on-premise environment, where multiple copies of the same data tend to proliferate across the enterprise.

While it’s unlikely that IT organizations will ever move all of their data to a public cloud service, over time a lot of that data is going to be processed and stored outside the four walls of the traditional data center anyway. The challenge is going to be figuring out how manage all the data regardless of its physical location.