One of the supposed benefits of data frameworks such as Hadoop is that they provide a cost-effective way to manipulate so-called "Big Data."
The concept of Big Data is hardly new. What makes Hadoop interesting is that rather than having to buy data warehousing appliances from companies such as Oracle to manipulate large amounts of data, Hadoop lets customers work with large amounts of data on lower cost servers. Of course, the processing speeds are nowhere near what you would experience with a dedicated data warehouse appliance, but for a lot of organizations, speed isn't everything. They just want to work with larger sets of data so that they can have more confidence in the conclusions they come to because they worked with a large set of data, rather than just a comparatively small sample.
The challenge with Hadoop, however, is that its native interface is MapReduce. In the corporate world, SQL is king. So it will be interesting to see how business analysts in corporate America react to Karmasphere Analyst, a new tool from Karmasphere released today that allows business analysts to use SQL to query a variety of Hadoop implementations.
According to Karmasphere CEO Martin Hall, the business analyst community has been pretty much bypassed by Hadoop. There are tools for developers to create applications that need to access both SQL and MapReduce. But a tool specifically designed for business analysts has been the missing Hadoop link in the enterprise, said Hall.
The reason that all this matters is that there is an increased desire to rely more on analytics. After the downturn, many business executives were surprised by the extent of the economic collapse. They are now keenly interested in examining the possibility of using analytics software to get a better handle on trends in the business. But it's hard to be confident of a particular piece being analyzed if the data is based on a relatively small sample. Hadoop makes it possible for more organizations to cost-effectively work with larger sets of data, which in turn should result in business analysts having more confidence in the results of their queries.
Of course, that's the theory. How it plays out in the real business world remains to be seen. But one thing that is for certain is that for the cost of a $2,495 business analyst tool, you might not want to find out the hard way that it works because your competitors have been leveraging it at your expense.