Happy Thanksgiving! To commemorate this national holiday, I present my annual five data- and integration-related reasons to be grateful this year.
- Coming soon: Pay raises—Staffing firm Robert Half is predicting big salary raises for some technologists next year, with an average increase of 5.6 percent for IT workers. Out of the top 12, two are related to data. Business intelligence analysts could average a 7.4 percent increase next year, while data architect salaries are expected to increase an average of 7.2 percent. Network World published a slideshow featuring the rest of the top 12, if you’d like to read more.
- Help with IT integration backlog—Integration work once always required custom code, but tools have come a long way to reduce custom code with pre-built connectors, GUIs and re-usable integration processes. In fact, the technology has matured so much that some tools are now designed with business analysts in mind. That means IT can off-load some of its more mundane integration work to business users who, in turn, will not have to wait in the project queue to access their data.
- Job growth—Your pay raise probably isn’t a random act of kindness: Big Data is driving job demand, too. A study by e-skills UK found that six out of 10 companies last year encountered problems hiring those with the technical skills they needed. Overall, the number of people employed as Big Data specialists is expected to increase by 49 percent each year, with demand doubling by 2017, according to CNME Online. The top data-focused roles in demand include BI consultants, data architects, business analysts, BI architects and BI analysts.
- For Rent: Big Data and processing power—Amazon, Google and IBM are in an arms race of sorts when it comes to offering raw processing power in the cloud. Who stands to win? You and anyone else who wants to crunch Big Data without building a supporting infrastructure. That’s not exactly news, but another thing to be thankful for this year: With Watson, we’re now seeing access to large, pre-built data stores added to the deal.
- Stinger, a.k.a. better SQL for Hadoop—Earlier this year, the Apache community and Hortonworks launched the Stinger Initiative. Its goal is to improve the SQL interface with Hadoop. It’s a targeted effort in three phases, and two are already complete. What this will mean is that all those SQL programmers will have an easier time using Hive to query Hadoop Distributed File System data stores. It has also significantly improved Hive’s performance: Queries that took 1,500 seconds a few years ago are now running in 10 to 12 seconds.