Sorting Through What's Really Going on in the Hadoop Stack

Share  
1  |  2  |  3  |  4  |  5  |  6  |  7  |  8  |  9  |  10  |  11
Previous Next

Click through for an overview of the Hadoop stack and gain a better understanding of its capabilities, as identified by Loraine Lawson.

Loraine LawsonEveryone tends to focus on the “big” in Big Data, so much so that it’s easy to lose focus on the fact that Hadoop is really about data. Let’s regroup for a minute and really look at what’s going on with the data on Hadoop.

First, there’s the core. When people say “Hadoop,” they’re usually referring to the Hadoop core, which Loraine Lawson explained:

The Hadoop Distributed File System. What’s it doing with the data? It’s distributing it on nodes and storing it there.

MapReduce. This does the real work in the Hadoop core. If you want to run a process or computation on the data, it “maps” that out to the nodes and then runs the process, and “reduces” the results to your answer. So, it’s processing the data.

Now, if you’re familiar with data at all, you’ll notice there are a whole lot of things missing from that equation, such as:

  • Modeling
  • Metadata
  • Job scheduling
  • Workflow
  • Data management

This is where the growing list of Apache Hadoop-related projects comes into play.

These projects go by an odd assortment of names: Pig, Hive, Flume, Zookeeper, but they’re often short-changed when we talk about Hadoop. Loraine has seen them referred to as the “Hadoop stack,” though some programmers prefer “Hadoop ecosystem." Forrester refers to them as “functional layers.”

For the most part, they’re of interest to developers more than executives, but hopefully a high-level view of these solutions will add some depth to your understanding of Hadoop and its capabilities.

Here are a few of the more common names you’ll hear.

 

Related Topics : Vulnerabilities and Patches, Resellers, Broadcom, Broadband Services, Supercomputing

 
More Slideshows

IT_Man37-290x195 How Sales Professionals Can Get the Most Out of Their BI Technology

Business information technology has the ability to contribute significantly to a company's bottom line, but it will be a wasted expense if your team does not understand how to use it properly. ...  More >>

JibeJobAppMistakes0x Five Reasons Your Best Applicants Move On

If you want to hire the best and the brightest candidates, you need make sure you're successfully bringing them to the end of the application process. ...  More >>

YeswareEmailMyths0x Debunking Five Common Email Myths

Everyone wants to work smarter and get out of the office faster; check out when the best time of day is to send an email, whether to use "cc," or what subject line length will get you the fastest reply. ...  More >>

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


 
Thanks for your registration, follow us on our social networks to keep up-to-date