Questioning the Assumptions in Predictive Analytics

Six Big Business Intelligence Mistakes

Of all the analytics buzz words today, I have to think predictive analytics would be the most puzzling for business managers. One reason I say this is that executives tend to trust their own instincts over analytics, despite research showing they’re wrong to do so.

A little mistrust may not be a bad thing, according to Big Data and analytics expert Tom Davenport. In fact, Davenport suggests that business leaders ask some serious questions about what’s going on behind the predictive analytics curtain.

 “The big assumption in predictive analytics is that the future will continue to be like the past,” Davenport explains in a recent Harvard Business Review post. “As Charles Duhigg describes in his book The Power of Habit, people establish strong patterns of behavior that they usually keep up over time. Sometimes, however, they change those behaviors, and the models that were used to predict them may no longer be valid.”

The assumptions might be incorrect for a number of reasons, and he lists a few. Time or changing market circumstances, for instance, can alter the assumptions.

How they change isn’t actually the point. Whether the assumptions are right or wrong, business leaders need to understand what those assumptions are, he warns.

“Managers should always ask analysts what the key assumptions are, and what would have to happen for them to no longer be valid,” Davenport writes. “And both managers and analysts should continually monitor the world to see if key factors involved in assumptions might have changed over time.”

This isn’t a new message, really. For at least a decade, experts have been lecturing business leaders to look behind the curtains to understand what’s really going on with their technology investments.

That message is more urgent today than ever, though, because today it’s about the data, not the technology. The shift toward data-driven decision making creates a more direct correlation today between the coding (or, in this case, the data models and algorithms) and business strategy. In fact, experts such as IT Business Edge’s Don Tennant argue that predictive analytics will be a “game changer” for businesses.

It’s incumbent upon business managers to dig deeper into what’s going on behind the results. Is the data trustworthy? Is it representative? What are the business assumptions driving the analytics models or underlying algorithms?

Davenport recommends five specific questions business leaders should ask when using predictive analytics in his post. He also does a nice job of explaining how predictive analytics works, what common business problems it addresses, and a bit about the statistical math it performs.

But his discussion of assumptions in predictive analytics is why I rate this a “must-read” article for any business manager.

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.