Eight Ways to Put Hadoop to Work in Any IT Department
We’re leaving behind the early days of Big Data, when you had to build your own Hadoop cluster and then figure out what to do with it.
From Big Data as a service to business analytics tools built on in-memory platforms, vendors are finding new ways to simplify Big Data.
The timing seems right. Marketing companies and departments are showing keen interest in using Big Data — particularly social media data — to enhance their BI abilities. Likewise, there’s interest from the IT crowd: A recent Gartner survey of IT leaders found that 42 percent have either invested in Big Data technology or are planning to do so within a year.
There are primarily four options for lowering the technology barriers to Big Data adoption.
1. Big Data in the cloud. In this InfoWorld article, IBM Big Data Evangelist and former Forrester Big Data analyst James Kobielus does a great job of explaining four situations perfect for Big Data in the cloud:
- Your enterprise applications — and thus most of your transactional data — is already in the public cloud.
- Preprocessing high-volume external data (think: Twitter feeds). It’s worth noting, too, that there are point solutions for specific types of Big Data. For instance, there are several SaaS solutions for marketing, including Salesforce.com’s Marketing Cloud or BrandsEye.
- Tactical applications beyond your current on-premise Big Data capabilities. “In fact, a public cloud offering might be the only feasible option if you need petabyte-scale, streaming, multi-structured, big data capability ASAP,” he writes.
- Elastic provisioning of very large, but short-lived analytic sandboxes.
2. Big Data Apps. Big Data technology isn’t just changing how we store and process data; it’s also transforming how applications are developed. I recently wrote an article, “How Big Data is Changing Enterprise Applications,” exploring this more fully, but basically what’s happening is BI, ERP, CRM and other enterprise apps are being built on top of in-memory databases, like HANA. It’s also possible with Hadoop and other NoSQL tools, but that’s not something people are really pursuing right now. The advantage of this is you can store large amounts of data in the apps themselves, and then use that data in more complex analysis.
3. Big Data in a box — and then some. This is perhaps the more expensive route to Big Data, and it’s been around for a while. You’re basically getting a Big Data solution — either Hadoop or an in-memory based system — that is on a pre-configured box.
What’s changed is that vendors are starting to offer additional capabilities. For instance, this week IBM unveiled its PureData System for Hadoop, which includes archival capabilities and a “family of analytic accelerators” that will simplify using Hadoop for social data, text analytics and machine data.
4. Analytics and visualization tools. Perhaps the fastest-growing area for Big Data solutions is in analytics. Technically, some of these fall either into the Big-Data-in-a-Box or Big Data Apps categories, but they’re such a hot topic right now, it’s worth mentioning separately. Drew Robb offered a rundown of some of the most recent solutions to hit the market in last week’s Enterprise Apps Today article, “5 Buzz-worthy Big Data Analytics Apps.”
