Lack of Trained Staff Hobbles Big Data


I was totally freaked out when I called the bank and in order to verify my identity, was asked to state my height. How would the bank know my height if I hadn't given that information? From my driver's license records, of course.


We'll be seeing more and more of that, experts say, despite all our privacy fears. Check out this eWEEK article about a research center in London devoted to commercializing public data.

Slide Show

Nine Predictions for the Analytics Industry in 2011

The competitive gap between analytical innovators and those who do not invest in analytics will widen over the coming 12 months.


I keep seeing articles on the rise of analytics in all sorts of things and have seen this called the move from the information age to the analysis age. As Howard Anderson at InformationWeek put it:

What are the relationships that are not evident but are powerful? If you work for a bank, are heads of households whose last name ends in a vowel and who were born between 1965 to 1975 40 percent less likely to have a bad loan than the rest of the population?

He's writing about executives' resistance, which he calls futile. He's foreseeing a future in which companies name a chief analytics officer, something the International Institute for Analytics predicted would happen this year.


There's gold in that data, according to the McKinsey Global Institute:

For example, a retailer using big data to the full could increase its operating margin by more than 60 percent. ... If US health care were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300 billion in value every year. Two-thirds of that would be in the form of reducing US health care expenditure by about 8 percent. In the developed economies of Europe, government administrators could save more than 100 billion ($149 billion) in operational efficiency improvements alone by using big data, not including using big data to reduce fraud and errors and boost the collection of tax revenues. And users of services enabled by personal location data could capture $600 billion in consumer surplus.

As I've written before, that means jobs. And with any profession where demand outstrips supply, salaries will reflect that. But for Big Data to really become widespread, many, many more people have to be trained. According to the McKinsey report:

The United States alone faces a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of big data.

So just as in the case of mobile apps, though companies might want to go in that direction, a skills shortage - as well as issues of privacy and security - could prevent that from happening as quickly as they'd like.