The most fundamental problem with Big Data is that there is just too much of it to make any sense out of it.
To address that particular Big Data paradox, Nodeable has come up with what is essentially a pre-processing analytics engine for Hadoop environments, dubbed StreamReduce, that can be deployed on top of the Amazon cloud computing service.
Launched today, Nodeable takes advantage of the Amazon Web Services Elastic Map Reduce platform to stream data that has been pre-processed into Hadoop. While Hadoop is a cost-effective platform for storing massive amounts of information, its batch-oriented processing model makes it less than ideal for compute-intensive analytics applications. According to Nodeable CEO Dave Rosenberg, Nodeable solves this problem by applying a streaming analytics engine that pre-processes data prior to storing in Hadoop.
That approach not only reduces the sheer amount of data that the IT organization has to manage, but it makes it easier for the organization to organize and access the data that has the most value to the organization.
Rosenberg says one of the biggest challenges with Big Data is that a vast majority of it is of uncertain value. Nodeable provides a way of identifying and organizing all that data at scale as it is being collected. That in turn not only reduces the cost of transferring that data across the network, it lessens the organization's dependency on hiring data scientists to make sense of all that data, says Rosenberg.
There is not doubt that as the size and types of data being collected continue to mushroom, new approaches to processing that data are going to be required. The challenge facing IT organizations is figuring out exactly how to best go about doing that without breaking the IT bank.