For a lot of people, the phrase Big Data has become synonymous with Apache Hadoop, the open source framework originally developed by Yahoo to manage massive amounts of data. But as more organizations become familiar with the nuances of Big Data, it’s starting to look like there will be a much greater appreciation for different approaches to managing Big Data in 2013.
According to RainStor CEO John Bantleman, one of the things that IT organizations need to remember is that while Hadoop may be the darling technology of the moment, Big Data is not a technology but rather an IT problem that needs to be effectively solved. While there’s no doubt that Hadoop will be broadly applied in the enterprise, Bantleman says that as IT organizations get more familiar with the challenges associated with Big Data, they will be just as likely to deploy any number of other platforms to handle certain types of Big Data applications.
For example, RainStor makes a database that is well-suited to cost effectively handle Big Data applications made up of petabytes of data at speeds that are significantly faster than a batch-oriented Hadoop cluster. Specifically, RainStor makes use of data compression and de-duplication techniques to store data in large blocks called partitions that reduce the storage footprint of Big Data applications by 95 percent. Data retained in RainStor can be queried directly using SQL, a BI tool or MapReduce without restoring or re-inflating the data.
Because RainStor supports traditional SQL applications, versus requiring IT organizations to master MapReduce or some other hybrid instance of SQL that has been layered on top of Hadoop, IT organizations don’t necessarily have to hire specialists and a data scientist to run Hadoop or invest in expensive massively parallel database systems that are complex to manage.
Perhaps the best thing about Hadoop is that it has shown a spotlight on the potential business value of Big Data. But far from being the only answer as the New Year progresses, as IT organizations struggle to master Hadoop many will come to the conclusion that Hadoop is but one weapon in an arsenal of platforms for managing Big Data. The end result will be a proliferation of database platforms throughout the enterprise — the adoption of which will be driven by the nature of the Big Data application workload versus merely assuming that Hadoop is the only right answer to every Big Data problem.