Big Data's Need for Big Solutions

Arthur Cole
Slide Show

Big Data Analytics

The first steps toward achieving a lasting competitive edge with big data analytics.

Big Data may be a fact of life for many enterprises, but that doesn't mean we are all fated to drown under giant waves of unintelligible and incomprehensible information.

In fact, the very definition of Big Data holds that loads can only acquire the term if they exceed the organization's ability to analyze and manage it. Therefore, Big Data is determined not by what is coming at you, but by your ability to handle it.

The answer, then, is to increase your ability to handle it. However, the speed at which Big Data is mounting and the complexity in building a suitable infrastructure add both cost and configuration hurdles to any Big Data program. IDC reports that storage companies shipped a total of 5,429 PB of capacity in the second quarter alone, a 30.7 percent increase. At the same time, data is hitting the enterprise from a plethora of sources, particularly as new generations of mobile and handheld devices produce a deluge of unstructured data that defies easy analysis.

But before you think this is just a storage and analytics challenge, consider the impact this is having on the network. As WAN optimization provider Infineta Systems showed recently, data center-to-data center connectivity is likely to suffer the most in Big Data environments. As enterprises adopt Hadoop and other tools to increase capacity to the petabyte level, reliance on wide area networks will increase as disparate resources are drawn together to process the load. Most current WAN environments are not tailored for extremely large data sets, leading to bottlenecks and performance deterioration.

Elsewhere in the stack, Big Data presents more of an opportunity than a challenge in light of changing data center platform requirements. Red Hat, for example, sees a chance to expand its footprint in enterprise environments through large-volume storage enhancements. The company's new Storage Software Appliance leverages technology recently acquired from Gluster to provide scale into the petabyte range. As an open source, distributed file system, the system provides for virtualized, pooled storage that can be layered across physical and cloud environments.

At the same time, Big Data presents new opportunities for some long-standing enterprise solutions. As HP's Srinivasan Rajan points out, COBOL offers a number of advantages when it comes to handling large data sets. For one, Big Data analytics can take advantage of COBOL's batch infrastructure support and complex algorithms. As well, COBOL's Job Control Language (JCL) is highly adept at scheduling large jobs into smaller ones, helping to lessen the impact on underlying resources.

Big Data, it would seem, is neither a net negative nor a net positive for the enterprise. Yes, it presents a challenge both for analysis and physical/virtual resource allocation. But at the same time, there is likely to be tremendous value in all that noise. The trick will be to identify and leverage the crucial bits of information without overwhelming your data infrastructure or your IT budget.

Add Comment      Leave a comment on this blog post
May 9, 2015 3:02 AM Prem Prem  says:
Very interesting and informative article!! Big Data is definitely next big thing in technical world and it is constantly revolutionizing the working culture of companies.Big Data is an opportunity for every industry to understand the issues concerning the vast amounts of content it creates. Companies like Tungsten Information Management are constantly working in various verticals such as Banking and Financial Services, Healthcare, Retail, Telecom, Media, Manufacturing and Internet Technology. Tungsten Information Management Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.