Hadoop Skills: Not the Same Old Java

Susan Hall

I've been asking my colleague Loraine Lawson a lot of questions lately about what Hadoop is and isn't, trying to better understand it and the skills required. Probably a lot of folks are in the same boat.

She kindly passed along this SearchSOA piece, which says that though Hadoop requires Java skills, Hadoop probably will be a different type of Java development project. It says Big Data analytics shares similarities with traditional SOA and batch-oriented development.

Helena Schwenk, analyst at MWD Advisors, told the site:

Hadoop is "not about real time operational [business intelligence], but more about the discovery, exploration and analysis of large amounts of multistructured data."

She said it calls for more than a single platform or engine, but rather a variety of technologies, components and architectures. Among the skills and experience she mentions:


  • Experience in large-scale distributed systems
  • Knowledge of languages such as Java, C++, Pig Latin and HiveQL
  • Predictive modeling, natural language processing and text analysis
  • Data management
  • Integration of structured and unstructured data
  • A range of data-latency demands
  • Architectural support for scalability and high-speed processing


It's unlikely you'll find all those things in a single person and it's quite likely you'll need to broaden skills on your staff with more training, she says.

Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.