Creating effective IT metrics is kind of like making piecrust. It doesn't seem like it would be too hard, in fact it sounds downright simple. Yet many people go through lots of failed versions before getting it right, and some folks never manage to produce a good crust.
And so it goes with metrics. Chris Lockhart discusses a common problem in creating metrics, coming up with ones that aren't connected to any real business value, on his Chris on EA blog. His example: A former client devoted a good amount of time and effort to collecting and presenting monthly data on projects created using an internal system development lifecycle (SDLC), with the project management office cheering every time the numbers went up. Yet there were no accompanying numbers showing whether this reduced costs or resulted in other benefits.
Soon, Lockhart writes, the company realized it needed to ask how much variance was reduced by using the SDLC, and how much that saved the company. Did quality increases promote profitability? Was it about money saved, costs avoided or revenue increased? He says:
Once these questions were asked and measurements taken to answer them, it became apparent that the overhead of running their custom developed SDLC (including training and enforcement and checkpoints and governance) was greater than the measured dollar benefit of the program.
Ouch. There's yet another lesson in the ineffectiveness of creating metrics just for the sake of doing so and becoming more caught up in creating the metrics than in actually assessing what they mean.
This is my chance to trot out some of the great tips on metrics I've shared in the past.
In May I shared a list of 12 characteristics of effective metrics written by TDWI's Wayne Eckerson. Among my favorite characteristics from his list: simple, actionable, standardized, game-proof and strategic. In order to make their metrics strategic, Eckerson advised, organizations "must start at the end point-with the goals, objectives or outcomes you want to achieve-and then work backwards." Doing this would help avoid scenarios like the one offered by Lockhart in his post.
In another post, I mentioned the common problem of IT trying to find the right balance in its metrics and offered some great advice from Jim Quick of Diamond Management & Technology Consultants on how to do so. For example, one tip for IT organizations that manage too many metrics: Go down the list of reports and ask, "What do we do when we look at this report?" Eliminate those for which the answer is "nothing." And here's a tip for those with not enough metrics: Identify an inefficient behavior you want to change and develop metrics so you can make a fact-based case that will persuade people to change it.
I've also cited some suggestions from Colin Fletcher, BMC Software's solutions marketing manager for BMC Atrium, on how IT can produce metrics that are relevant to business user. Hint: First focus on customer perceptions rather than hard data.