Is ROI a Bad Way to Measure Analytics Success?

Five Ways to Use Data to Be More Relevant to Customers

Will analytics be more successful if you focus on ROI or should you instead focus on key performance indicators that “move the need” toward your business goals?

Recently, I’ve seen both positions exposed as a way to ensure success with analytics. Maybe it’s a bit of a trick question. After all, anything that improves ROI may well improve profitability goals, but they actually strike me as two very different starting points.

“Selecting the right critical metrics is a cornerstone of success as it brings focus and clarity on what matters most to the business,” writes Brian McCarthy, managing director, Information & Analytics Strategy, for Accenture Analytics, in a recent Harvard Business Review post.

McCarthy suggests that the senior management team should decide on the business goal for an analytics project first, and then identify the key performance indicators that will get them there. He uses an Accenture client, “a high-performing retailer,” as an example. The client’s goal was to drive growth and profitability. The leading KPIs for that goal were customer retention, product availability, labor scheduling, product assortment and employee engagement.

Contrast this to the suggestions coming out of this month’s Predictive Analytics World 2014. Sri Srikanth, Cisco program manager, Analytics, Digital Strategy & Enablement, writes that a key starting point for success is to “focus on ROI measures.”

“Being driven by ROI implies understanding which variables are controllable by the business, which data observations are of real interest and sometimes making adjustments to accomplish that,” Srikanth writes. “This may mean considering variables that are otherwise not significant, or oversampling certain data observations and so forth. But a relentless focus on ROI will yield the desired results.”

So, which is the path to success: KPIs as determined by business goals or “relentless focus on ROI?”

Maybe it depends on where you are. For instance, McCarthy is writing about the expansion and integration of analytics into the enterprise at large, but Srikanth’s piece is geared to a more technical audience dealing with predictive analytics — a relatively new approach. It’s certainly hard to argue with either man’s other suggestions, which are more tailored to the situation.

ROI seems like a logical starting point, and it’s certainly a common piece of advice — almost to the point of being a “throw-away comment.” I suppose, then, we could assume that focusing on ROI measurements is best when you’re testing the waters with predictive analytics, but as you expand analytics, you’ll need to go back to business goals.

Here’s the problem: I’m not sure reality bears out starting with ROI on analytics, even predictive analytics.

First, ROI tends to be a major talking point for IT — until the project is under way. After that, experts say ROI is seldom calculated once the project is finished, making it fairly useless as anything other than a sales tactic.

Yet even if you plan to calculate ROI, is it really appropriate to do so on a limited project when the technique and tools can be used across the enterprise? Certainly, the ROI will change quickly as more parts of the organization adopt predictive analytics.

Finally, if you want to show that analytics of any type are useful to the company, it makes more sense to demonstrate how analytics helps with actual business goals.

ROI is a business-wide term, but strategically, the metric positions IT as a cost center defending its expenses, rather than a business-enabler that’s showing movement toward concrete business goals.  Certainly, if your CFO insists, then ROI could be incorporated into the KPI.

As I see it, the bottom line is this: Focusing on ROI is an old standby for IT, but when it comes to data, it’s just not an appropriate measurement. If we’re going to talk about data as “the new oil,” and rely on data as a strategic necessity, you can’t measure it like it’s an IT project.

Instead, in my opinion, data needs to be measured as it’s used: a cost of doing business in our data-driven age.

Loraine Lawson is a veteran technology reporter and blogger. She currently writes the Integration blog for IT Business Edge, which covers all aspects of integration technology, including data governance and best practices. She has also covered IT/Business Alignment and IT Security for IT Business Edge. Before becoming a freelance writer, Lawson worked at TechRepublic as a site editor and writer, covering mobile, IT management, IT security and other technology trends. Previously, she was a webmaster at the Kentucky Transportation Cabinet and a newspaper journalist. Follow Lawson at Google+ and on Twitter.