"Hadoop will democratize big data," it's been said. But whoever said it neglected to consider just how hard Hadoop is - and how few people know how to manage the complexities of Hadoop.
That chasm is where Hortonworks may find its sweet spot.
Hortonworks is comprised of Yahoo's former Hadoop group - a spin-off company whose business model is to offer support and training for Hadoop. The company seems to be off to a strong start. It simultaneously announced a new Hortonworks Data Platform based on Apache Hadoop, and revealed a "growing ecosystem" of support partners, including Informatica (a Hadoop data parallel parser), Pervasive (solution for energy efficiency) and Datameer (BI solution for Hadoop) - a sort of three-legged race to the enterprise (my words), if you will, with Hortonworks partnering rather than trying to provide "a la carte services," says Read Write Web.
One new development in Hortonwork Hadoop distribution is Ambari, an installation and management system for Hadoop that should make installations and upgrades easier. Read Write Web also reports that Hortonworks is dedicated to the open source model for its work - with all code written directly in Apache.
Of course, Hortonworks isn't the only company trying to make Hadoop more accessible to businesses. Cloudera, Map R Technologies and Revolution Analytics - which focuses on R as a means of applying analytics to Hadoop stores - are among the key names in the Hadoop market.
Every day, you read about how some vendor or other is adding support for Hadoop - usually by way of a connector. But Forrester Analyst James Kobielus says adoption isn't quite as fast as you might think - actually, most are holding Hadoop "at arm's length," offering only partial interfaces, according to a recent SearchDataManagement article.
He expects data vendors will embrace Hadoop more completely during the next year, possibly through acquisitions, which should be fun to watch. It's hard for me to believe that they haven't already embraced Hadoop, but Kobielus says only a few data warehouse vendors have really integrated it fully into their core products.
So I'll say this: The vendor's marketing and PR teams certainly seem to have fully embraced Hadoop.
TechTarget does a good job of cutting through some of the hype to offer a more honest assessment of where companies are with Hadoop. One thing I found particularly interesting is that application developers are the ones dabbling in Hadoop, which isn't surprising. But what is surprising is that they're doing so without involving the rest of IT or data warehousing managers.
That is expected to change and one would certainly hope so. The real value will come from integrating whatever Hadoop's storing and processing with the rest of the enterprise's data. As I believe I've mentioned a time or two previously: silos bad, integration good.
The article also includes an excellent point by BI expert Wayne Eckerson that Hadoop advocates need to figure out how to de-junk Hadoop's data stores.