Business intelligence quality is different from the return on investment (ROI) for BI. A business intelligence project or system is designed to deliver specific information. BI quality is, most often, an IT-based assessment of the results of a business intelligence project. Measuring the quality of the BI system includes the basic logistics: getting the right information to the right people when needed and meeting the stated business goals for the BI system. Thus, BI quality is determined by assessing:

• The correctness and integrity of the data.
• The translation of the data into usable information.
• The speed and format of the delivery.
• How well the information meets the design criteria and business requirements in the preliminary design. (Do the users approve it?).

On the other hand, the ROI for BI is a business-management question. The ROI assessment is perhaps even more important to the organization. However, there are no industry standards for measuring the benefits of business intelligence. But listing them as “intangible” is no longer an acceptable answer.

Can We Calculate ROI for BI?

Placing a tangible value on the benefits of BI has been frustrating. In most cases, organizations have had to accept on intuition and faith that the benefits of BI are worth the costs, and that the risks of not making those investments are too great. What if your competitors are translating all their data into some fantastic information that will give them the edge?

We can start with the total cost of ownership for BI, which can be calculated with some level of reasonable accuracy. Deciding what should be included in those costs may be a matter of interpretation. However, there can be some standard and relatively straightforward decisions that will allow project-by-project comparisons and overall resource estimates. These costs include those for the data warehouse; information delivery; data gathering and management; and all the associated infrastructures, software, tools and support resources.

In addition, the BI project development, management and delivery costs, including the infrastructure, are part of the cost equation. What is most likely not being considered in current cost calculations are those involved in the training and experience base of the people who are a critical component of the BI product. Deciding what should be included in BI investment costs may be a matter of interpretation, but all these costs can be calculated, or at least identified, prorated and estimated.

Much more difficult is placing a value on the benefits received. How well are we using the information to make better decisions? There are metrics, such as comparisons of operational efficiencies, before and after, that are relatively standard for some of the most simple of BI applications. However, projecting and calculating tangible values for returns, especially on more complex BI investments, is not simple. The process can be frustrating and has often seemed impossible and, perhaps, pointless. But if we can assess these benefits and provide some tangible ratings, we can provide a basis for management decision-making about BI investments for the organization.

Think about it this way:

Good BI is the fusion of the right information, the right time, the right format, and the right human and/or system resources. If we wish to improve BI, we ask these questions:

• Do the people (or intelligent systems) have the information needed, when they need it, to make decisions?
• Do those people have the expertise, training and mindset to use that information in the best way for the good of the organization?
• Are they doing their job better because of the information being delivered?
• How much difference does that information make to them?

A Benefits Audit for BI

The most effective way to assess the benefits of BI is to ask the people who are involved. We can use questionnaires and surveys administered at regular intervals. This is a simple and direct approach. However, such a benefits audit must also be rigorous and structured to give tangible, realistic assessments of the relative benefits of BI projects, BI capsules (designated BI groupings), and the total organizational BI investment.

Using formal, well-constructed metrics and patterns, we can turn the opinions of experts into a trustworthy assessment of the benefits of BI for the organization. Those experts are the people who use the information and the managers who daily assess the value of the performance of those people who use this information. We may also wish to add competitive analysis and opinions from customers, marketing teams and consultants. Thus, we can turn expert evaluations into a practical, meaningful assessment tool for management.

There are a number of valuable features to be gained from such a benefits audit. We can:

• Make better decisions regarding commitment of resources to BI. The relative values of BI projects can be identified before spending the money on development. Opportunity costs and risks can be evaluated up front.
• Improve management planning, both strategic and tactical. Competitive, legal and regulatory requirements can more easily be assessed and planned for, and associated risks can be reduced.
• Encourage better decision-making.
• Improve the quality of BI and BI projects through a better understanding of the results. We will be able to feed results back into the processes for BI design, development and use.
• The organization’s culture can evolve and improve through an increased awareness of BI, integration and communication.
• Provide encouragement and support for our people. Just administering such audits will send a message that the organization cares about them, respects their opinions and is open to new ideas.
• Encourage creativity and broader use of BI  through recognition and associated education, training and awareness.

The questions, formats and process of the BI benefits audit will provide increased awareness, understanding and training for those who develop and implement the audit, as well as for those who participate.