Should Companies Worry About Analytics Skills Shortage?

Ann All

With practically every new technology, there's an accompanying shortage of professionals equipped with the right skills. We've seen skills shortages with service-oriented architecture, and companies clamoring for folks with virtualization skills.


Now it appears they may soon be shaking the trees for statisticians, a position that Google's Hal Varian predicts will be the "next sexy job" in IT.


Companies would do well to consider a possible shortfall in statisticians before making any big investments in analytics software, advises Intelligent Enterprise Editor-in-Chief Doug Henschen. While vendors provide prebuilt models and off-the-shelf analytics applications, many companies find they need to customize to gain insights that are truly relevant to their business. He cites one, wholesale auto lender Dealer Services, which tried off-the-shelf analytics but is now building its own predictive model incorporating dealer traits and transaction parameters that have worked well for the company in the past.


These kinds of data-related jobs made a good showing in summer job listings. While July marked a record low for listed openings, according to the Labor Department's Job Openings and Labor Turnover survey, the Conference Board tracked an increase in jobs involving computer and mathematical sciences, from 398,000 in July to 406,800 in August. The Associated Press story specifically mentions IBM hiring or retraining people to work on data analysis projects, something I also referenced in my statistician post.


Interestingly, as Henschen points out, companies don't appear to be unduly concerned by the prospect of a shortage. Thirty-four percent of the 534 participants in a recent InformationWeek/Intelligent Enterprise survey said they "already have skilled analytics professionals on staff." Forty-eight percent said they expect to "train in-house BI experts and power users on analytic tools."


Henschen suggests paying close attention to users' comfort (or lack of comfort) with analytics software in any pilot projects and determining whether they can be trained to use analytics apps without lots of ongoing intervention from consultants. That point is echoed in a post I wrote last year. Responding to an earlier post of mine, reader Simon Mortimore warns that not performing a skills assessmentand providing training when needed will simply result in an "efficient information hole."


I also cited a company, publisher Meredith Corp., which recruited "super users" to train on the more advanced features of a new business intelligence system and then share them with colleagues. Users create about 95 percent of BI reports at that company.

Add Comment      Leave a comment on this blog post
Sep 15, 2009 3:04 AM Vincent Granville Vincent Granville  says: in response to Peter Simons

I think the issue is about hiring someone for (say) $60K per year, and not finding any candidates. If the company has to pay $80K instead, the job will disappear because the company can afford $60K only.

This is because de-leveraging has already happened at the company level, but not yet at the individual level. All the company has to do is wait another 12 months, then they will find tons of highly qualified people willing to work for $40K per year.

Sep 15, 2009 5:35 AM Peter Simons Peter Simons  says:

When you use the phrase "labor shortage" or "skills shortage" you're speaking in a sentence fragment.  What you actually mean to say is:  "There is a labor shortage at the salary level I'm willing to pay."  That statement is the correct phrase; the complete sentence and the intellectually honest statement.

Don't speak about shortages as though they represent some absolute, readily identifiable lack of desirable services. Price is rarely accorded its proper importance in this sort of "worker/skill shortage" rhetoric.

If you start raising your wages and improving working conditions, and continue to do so, you'll solve your 'shortage' and will shortly have people lining up around the block to work for you even if you need to have huge piles of steaming manure hand-scooped on a blazing summer afternoon. 

Re:  Shortage caused by employees retiring out of the workforce:  With the majority of retirement accounts down about 50% or more, people entering retirement age are being forced to work well into their sunset years.  So, you won't be getting a worker shortage anytime soon due to retirees exiting the workforce. 

Okay, fine.  Some specialized jobs require training and/or certification, again, the solution is to raise your wages and improve benefits. People will self-fund their re-education so that they can enter the industry in a work-ready state.  The attractive wages, working conditions and career prospects of technology during the 1980's and 1990's was a prime example of people's willingness to self-fund their own career re-education.

There is never enough of any good or service to satisfy all wants or desires. A customer must give up something to get something. The buyer must pay the market price and forego whatever else he could have for the same price. The forces of supply and demand determine these prices -- and the price of a skilled workman is no exception. The buyer can take it or leave it. However, those who choose to leave it (because of lack of funds or personal preference) must not cry shortage. The good is available at the market price. All goods and services are scarce, but scarcity and shortages are by no means synonymous. Scarcity is a regrettable and unavoidable fact.

Shortages are purely a function of price. The only way in which a shortage has existed, or ever will exist, is in cases where the "going price" has been held below the market-clearing price. Evidence does not suggest that such is the case for the "shortages" which have recently been advanced.

Sep 21, 2009 12:30 PM Michael Stagar Michael Stagar  says:

Dear Readers:

My Ph.D. and 25 years industrial and medical experience in statistics has awarded me some very interesting insights into software, decision making, and training:

1. Software will 'not' create analytic decision making for those who lack the necessary skills of interpretation and statistical analysis.

2. General statistical software will not have the judgmental power or the significance in validity and reliability needed in decision making, unless it is of such simplicity that very few other issues exit.

3. Statistics is a tool, nothing more. You have to know how, when, where, why, and for what that tool is needed.

4. Statistics is always changing their methodology and application based upon 'real-time' events.

5. Most Statisticians cannot teach. Find someone who has educational and industrial training skills. Merely teaching stats at a local college will not do. Industrial training is a must.

6. The reason that most individuals have a difficult time with stats is simple: POOR - VERY POOR INSTRUCTORS! Statistics must be applied to solve problems. It must be relevant - period! At one local community college, I replaced a stats prof. who had accrued an intolerable success rate of 35%. My first day, I ordered NO ONE to open up the books (since they were all crying any how). And we physically moved people around the room, used an old marker board, and dug right into why the selection of the proper stats tool (hammer to nail or wrench to socket - I actually brought in some Craftsman tools) can be understood with a little common sense and guts. My success rate was 95% with a skewed curve (sorry about that).

7. No one should try to re-invent the wheel. Don't try to create a statistical package out of Excel. Excel is a 'fair' data input format; it is not a statistical package. Nor is a car an airplane, even though they both have wheels.

8. Everyone in industry works in teams -so should stats! I always handed back poor quality stats projects in industry; went over the issues; and demanded that the teams re-submit before I would even consider a final evaluation.

Just remember that stats is merely a Craftsman tool set that will help you solve a problem. And please don't re-invent the wrench!


Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.