The Opportunities, and Challenges, of Big Data in the Cloud

Arthur Cole
Slide Show

5 Health Care Data Analytics Myths Debunked

It almost seems like a match made in heaven: Big Data requires lots of scale and the ability to push processing close to the edge; the cloud has scale in abundance, and is much closer to enterprise data users than most enterprises themselves.

So what’s not to love? Why isn’t the enterprise community falling over itself to ramp up Big Data operations in the cloud? It turns out that the Big Data-cloud connection is not quite as cut-and-dry as it seems, and there are good reasons why organizations would want to build cloud-based analytics solutions carefully, not haphazardly.

To be sure, there is no lack of Big Data solutions in the cloud. As CloudTweaks’ Jennifer Klostermann notes, everyone from Amazon to Microsoft to the corner cloud provider are deploying Hadoop, Spark, R and other highly scalable analytics platforms at a rapid clip. Microsoft has even succumbed to the lure of open source, namely Linux, as it seeks to draw more of the enterprise Big Data workload. At the moment, most of these deployments are geared toward social media applications, drawing feedback from customers, partners and employees that can be parsed to find hidden opportunities. Going forward, however, expect much of the work to center on machine intelligence and IoT-related data streams, which can then be tied to automated analysis platforms to provide real-time, predictive results in fast-moving market environments.


Leading organizations are already moving in this direction, says ZDnet’s Mark Samuels. Companies like Uber have put the fear in most executives that their long-standing business models can be upended at any time by a digital-savvy start-up, so they are pushing hard to refocus their cloud architectures away from traditional enterprise functions and more toward new data sources. Even municipalities around the globe are using advanced cloud-based analytics for things like traffic management, service delivery and even to fight crime.

But as mentioned above, this transition is not without its challenges. IBM’s Prat Moghe characterizes the divide between internal and external resources as “the cloud is from Venus, the data center is from Mars.” Clouds operate on different standards, different processes and different configurations than data centers, so migrating, integrating and conditioning data between the two is a perpetual challenge. As well, moving from test environments to production in the cloud is difficult in a world where you don’t have full control of the entire infrastructure stack. And making full use of cloud resources often requires a fair amount of IT staff retraining.

And even if you do manage to get a cloud-based Big Data analytics environment up and running, you are still a long way off from being a fully digital enterprise, says Forbes’ Joe McKendrick. The hard part is reworking all of your business processes to weed out the inefficiencies and duplication of effort that exist in most workflows. These can range from cross-subsidization requirements of your customers, lack of product design flexibility, poor product and service delivery, convoluted delivery chains and high-margin business models that are ripe for undercutting. Until established enterprises complete the transition on this level, they will continue to be easy prey for well-heeled start-ups that don’t have decades of legacy IT baggage to deal with.

The cloud is undoubtedly an invaluable tool when it comes to Big Data, but it is by no means the trouble-free environment that die-hard supporters claim it to be. By itself, it cannot pave the way to Big Data nirvana and paradigm-shifting insights into business models and markets.

But it can give you the scale and operational flexibility needed to do the real legwork of revamping the enterprise for an increasingly competitive digital economy.

Arthur Cole writes about infrastructure for IT Business Edge. Cole has been covering the high-tech media and computing industries for more than 20 years, having served as editor of TV Technology, Video Technology News, Internet News and Multimedia Weekly. His contributions have appeared in Communications Today and Enterprise Networking Planet and as web content for numerous high-tech clients like TwinStrata and Carpathia. Follow Art on Twitter @acole602.



Add Comment      Leave a comment on this blog post
Apr 15, 2016 11:22 AM CyberH CyberH  says:
Arthur, big data is a great opportunity! With the explosion of big data, companies are faced with data challenges in three different areas. First, you know the type of results you want from your data but it’s computationally difficult to obtain. Second, you know the questions to ask but struggle with the answers and need to do data mining to help find those answers. And third is in the area of data exploration where you need to reveal the unknowns and look through the data for patterns and hidden relationships. The open source HPCC Systems big data processing platform can help companies with these challenges by deriving insights from massive data sets quickly and simply. Designed bydata scientists, it is a complete integrated solution from data ingestion and data processing to data delivery. Their built-in Machine Learning Library and Matrix processing algorithms can assist with business intelligence and predictive analytics. More at http://hpccsystems.com Reply

Post a comment

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

null
null

 

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.