Hadoop Skills: Not the Same Old Java

Susan Hall

I've been asking my colleague Loraine Lawson a lot of questions lately about what Hadoop is and isn't, trying to better understand it and the skills required. Probably a lot of folks are in the same boat.

She kindly passed along this SearchSOA piece, which says that though Hadoop requires Java skills, Hadoop probably will be a different type of Java development project. It says Big Data analytics shares similarities with traditional SOA and batch-oriented development.

Helena Schwenk, analyst at MWD Advisors, told the site:

Hadoop is "not about real time operational [business intelligence], but more about the discovery, exploration and analysis of large amounts of multistructured data."

She said it calls for more than a single platform or engine, but rather a variety of technologies, components and architectures. Among the skills and experience she mentions:


  • Experience in large-scale distributed systems
  • Knowledge of languages such as Java, C++, Pig Latin and HiveQL
  • Predictive modeling, natural language processing and text analysis
  • Data management
  • Integration of structured and unstructured data
  • A range of data-latency demands
  • Architectural support for scalability and high-speed processing


It's unlikely you'll find all those things in a single person and it's quite likely you'll need to broaden skills on your staff with more training, she says.

Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Thanks for your registration, follow us on our social networks to keep up-to-date