Big Data Analytics, Big Skills Gap

Susan Hall

There's a cool graphic on this Technology Review piece showing the role of the data scientist. It's a Venn diagram by Hilary Mason, chief scientist at bit.ly, showing the intersection of engineering, math, computer science and hacking. The result? Awesome nerds.


Indeed, it's one of the hottest jobs around. Not only are companies drowning in data, but politicians including President Obama are seeking to unlock the secrets to success contained in their data. And it's not just a matter of being able to access the data, but to gain valuable insights from it.


A survey of practitioners of Big Data analytics by TDWI (report free with registration), puts the amount of data being analyzed at terabytes, but noted that it's "on the cusp of petabytes." As the report explains:

First, there's big data for massive amounts of detailed information. Second, there's advanced analytics, which is actually a collection of different tool types, including those based on predictive analytics, data mining, statistics, artificial intelligence, natural language processing, and so on. Put them together and you get big data analytics, the hottest new practice in BI today.
... big data analytics explores granular details of business operations and customer interactions that seldom find their way into a data warehouse or standard report.

The survey also asked for the job titles of those who do this data analysis. The results broke down this way:

  • Business analyst, 14 percent.
  • Director or manager of analytics, 14 percent.
  • Data architect, 10 percent.
  • Engineer, 10 percent.
  • Data analyst, 6 percent.
  • BI director, 5 percent.
  • Data scientist, 4 percent.
  • BI specialist, 3 percent.
  • Research analyst, 3 percent.
  • R&D specialist, 2 percent.
  • Other, 29 percent.


According to the report:

Almost one-third of survey responses (29 percent) were a mixed bag, describing marketers, consultants, statisticians, data governors, risk managers, and so on. This breadth of job titles is significant because it shows that analytics is not just for an analytic specialist. On the contrary, analytics is becoming a standard competency for a wide range of business and technology people.

This only underscores the prevalence of IT/business hybrids that analyst David Foote so often writes about. Yet companies will tell you that it's tough to find people with right the blend of business and technology skills. I interviewed Barbara Wixom, an associate professor at the University of Virginia's McIntire School of Commerce, about what universities are doing to address this problem and with IBM's Steve Gold about the company's collaboration with universities on analytics.


According to the Technology Review article, though:

This re-branding of statistical literacy as "data science" points out a larger trend - disciplines that were formerly the domain of the specialist, such as statistics, are now more important to a larger segment of the business world than ever. The fact that so few students view even a fraction of this level of mastery as necessary - and that schools often do not offer even a basic statistical education to non-math majors until the post-graduate level - suggests that in this area, perhaps even more than other areas associated with engineering, there is a yawning gap between the skills our workforce possesses and the skills employers require.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.