Real Questions for BI Vendors
Click through to see the questions Ann discovered that can make a tangible difference in your diligence.
He told me:
When you look at the products out there, it's just like the old spreadsheet, there are so many features that are almost never used. The open source products that are out there, whether it's the databases or the integration tools or the BI tools, they don't provide as much as the big, mature, proprietary solutions, but they cover the "good enough." They hit what 80 percent of the market needs. ... There's a point where you say, "Why do I want to spend $1,000 per seat when I can spend significantly less than that?
Madsen isn't the first person who has shared this idea with me. I've heard it a lot, from consultants, from analysts, from users who find their organization's BI software intimidating, from just about everyone but vendors. So if this is true, what do users want? I found some possible answers in a BeyeNetwork column by Craig Schiff, president and CEO of BPM Partners. (Schiff writes about business performance management, but I think the line between BI and business performance management is a bit fuzzy.)
When Schiff surveyed 500 users for its annual BPM Pulse survey, he found a unified front end or single user interface was the biggest priority. Schiff asked respondents to rate a list of 14 potential technology capabilities on a 1-5 importance scale (1 equals not important, 5 equals very important). Seventy-one percent of respondents gave the unified front end a rating of 4 or 5.
This isn't a surprise, writes Schiff, since ease of use is a big benefit for users who appreciate encountering similar terminology and menu choices as they use different functions of a BI system.While all vendors "understand the value of a unified system; some are further along than others though in actually achieving it," he writes.
The second and third most popular choices among respondents were management of both structured and unstructured data and data visualization, both of which fundamentally also seem to relate to improving user experience. Many current BI systems support management of structured and unstructured data, says Schiff, but support of data visualization isn't as common.
The fourth most popular choice is tight integration with Microsoft Office. Even if the BI solution isn't based on Microsoft technologies, users want the ability to use Office (Excel, I assume) for ad hoc analysis and presentation, says Schiff. Microsoft is hoping to capitalize on users' longtime reliance on spreadsheets with its new PowerPivot BI product.
Like Schiff, I found the least popular choices more interesting than the most popular. As he notes, the four bottom choices, XBRL support, software-as-a-service/cloud computing, hardware appliance and open source version, are less pragmatic and obviously friendly to the end user than the top four. Writes Schiff:
None of those technologies, unlike the most popular set, are going to make it easier to analyze and understand their performance. It is also not obvious to the end user that those technologies are going to make their jobs any easier either. Until direct business benefits can be demonstrated, these technologies will probably remain fairly low on the list.
But are companies really paying attention to user desires or just giving lip service to the idea of an improved experience for BI users? I wonder after hearing anecdotes like one shared with me by John Kitchen, chief marketing officer for Datawatch Corp. One of his company's clients, a Global 500 consumer goods company, bought 300 copies of a BI product and found after three months that just four licenses were being used. This suggests they didn't do a good enough job of surveying users about their BI wish list items and/or didn't do a good job of matching those desired features with products that could meet them.
That's why Kitchen, one of several contributions to a slideshow of questions to ask potential BI vendors, suggested asking "how much training will it take to get an 'average' non-technical person successfully using your software?"
Schiff wrote about several other aspects of his BPM Pulse study in earlier columns, and they're worth checking out. In part one, he examined which companies are doing BPM (more big companies than small) and which functions interest them most (budgeting/forecasting leads the pack). Part two focused on BI vendor selection and satisfaction and included a question about whether respondents would consider buying a SaaS solution (45 percent weren't sure). Part 3 focused on the business benefits respondents were seeking and impact of the down economy on the use of BI (more focus on profitability optimization).