Conventional wisdom holds that as a batch-oriented platform, Hadoop is not well suited for running high-performance analytics applications. This week, however, Actian unveiled the Actian Analytics Platform, which enables a massively parallel columnar database to natively process Hadoop data in a way that boosts analytics applications performance by a factor of ten.
Actian CTO Mike Hoskins says by leveraging an Actian Matrix columnar database running on the same x86 cluster as Hadoop, IT organizations can not only run Hadoop applications in real time, the total cost of deploying that analytics environment becomes less because of the columnar database. The Actian Analytics Platform provides facilities for managing data flow between Hadoop and the Actian Matrix database.
Rather than having to rely on separate appliances to process analytics, Hoskins says that multicore processors provide more than enough horsepower to process analytics in real time. The issue facing IT organizations is finding a way to augment Hadoop performance in a way that doesn’t require investments in anything other than standard x86 servers that can easily scale out over time.
Hoskins says that with the rise of next-generation MapReduce interfaces in the form of YARN and delivery of Big Data 2.0 and data warehouse services such as Redshift that are being offered by Amazon Web Services (AWS) in the cloud, instead of bringing data somewhere else to be processed, the focus needs to be on bringing compute resources to where the data is actually stored.
Hadoop is clearly emerging as a primary source of Big Data. But as a batch-oriented platform, Hadoop needs some help when dealing with data that needs to be processed in real time. The challenge facing IT organizations is finding a way to turn all the information into something that approaches actionable intelligence. Unfortunately, a large percentage of that intelligence only has value when it can be cost-effectively delivered in real time, which ultimately means processing everything as much as possible on the same base set of x86 servers.