When our Ann All began researching questions to ask BI vendors, she was surprised that after a little prodding they produced some great questions, ones she’d want to ask any potential BI vendor before she got up from the table. Several of them offered clever suggestions on how to move past generic questions to more specific ones that should yield more accurate answers. You should definitely read her full blog post on the topic.
For instance, instead of asking "what kind of support can I expect from you?" Barney Finucane, an analyst at The BI Verdict, a vendor-independent BI product selection service, suggested companies should ask vendors if you can look at their technical and support Web sites. This will give you "a good idea of what they might be doing wrong" as well as what they do right, he told Ann.
In another angle on service, Forrester Research analyst Boris Evelson told Ann it’s important to ask about a vendor’s ability to help you with your overall BI strategies, not just implementing BI products. Why? "Any problems will likely involve integration, support, organizational structure and/or underlying business processes rather than technology. Only experienced consultants can help with those kinds of problems,” he said. It’s true that technology tunnel vision can derail BI initiatives — and plenty of other technology projects as well.
Click through for questions you should be asking potential BI vendors.
It’s a good idea to get references with BI needs similar to yours, a similar user base, in the same vertical industry, etc. If you can find an organization that has achieved the kind of deployment you have in mind using certain product(s), it ‘proves it’s actually feasible,’ says Barney Finucane, an analyst at The BI Verdict, a vendor-independent BI product selection service.
In reading through the sites, look for any common ‘red flag’ issues and see how the vendor resolved them.
Answers will vary, says John Kitchen, chief marketing officer for Datawatch Corporation, a provider of enterprise information management solutions. If you are implementing a full stack of software, you should expect it to take a fair amount of time and resources. Be suspicious of vendors who tell you it won’t. A simple workgroup application for an individual department to produce its own reports should be far quicker and easier. You want solutions for which the amount of time and effort required is commensurate with your expected results.
Plan to buy less than the vendor recommends. Most companies buy too many, says Barney Finucane, an analyst at The BI Verdict, a vendor-independent BI product selection service, because they are ‘too optimistic’ about their project or don’t listen to what end users are saying. During negotiations, vendors will often offer to make some kind of a volume deal, but it’s no bargain if licenses go unused.
This will help you determine where the vendor plays in the industry and if you’re even looking at the right product(s), says Wayne Eckerson, director of education and research for The Data Warehousing Institute (TDWI).
(Ask this under a non-disclosure agreement, of course.) Wayne Eckerson, director of education and research for The Data Warehousing Institute (TDWI), says this will help you understand what current customers do not like in the product and want to see changed.
Wayne Eckerson, director of education and research for The Data Warehousing Institute (TDWI), says this may help you avoid sticker shock when you determine-at least in the case of traditional on-premise software-that the cost of software will likely be the smallest part of your investment.
You want to know a vendor’s financial viability, especially startups. Even if it’s a private company with no public filings, try to get some kind of financial statements and/or speak to the company’s CFO (under non-disclosure, of course). Ask how much cash the vendor has on hand. A possible red flag with established companies is one that now sells something very different from its original products. ‘They’re basically a startup,’ says Wayne Eckerson, director of education and research for The Data Warehousing Institute (TDWI).
This is important, says Forrester Research analyst Boris Evelson, because any problems will likely involve integration, support, organizational structure and/or underlying business processes rather than technology. ‘Only experienced consultants can help with those kinds of problems,’ he says.
Much as when you hit a potential employee with this job interview question, you’re looking for a vendor whose future direction matches up well with your own. ‘You want to know that the vendor will be there for you as your business needs and computing environment evolve,’ says Seth Grimes, owner and principal consultant for business analytics strategy consulting company Alta Plana.
This may include data integration, data cleansing, data warehousing, performance management and portals, in addition to reports, queries, OLAP and dashboards. If a vendor doesn’t provide all of these, you may need supplementary tools from other vendors, which will need to be integrated, advises Forrester Research analyst Boris Evelson.
This may include report writer, query builder, OLAP engine, dashboard/data visualization tool, real time reporting/analysis, text analytics, BI workspace/sandbox, and the ability to analyze data without a data model. If not, you may need to purchase supplementary tools from other vendors and integrate them, says Forrester Research analyst Boris Evelson.
Specify your top vendors and applications, such as SAP ERP, IBM DB2 and Oracle’s Hyperion Essbase. Red flag: If they mention moving data to a warehouse, ask again. Anyone can report against a data warehouse, and the question is about direct access, says Jake Freivald, VP of marketing for BI provider Information Builders. If they keep talking about the warehouse, Freivald says it’s a warning sign-even if your project primarily uses the data warehouse-because it shows potential inflexibility.
This is important, says John Kitchen, chief marketing officer for Datawatch Corporation, a provider of enterprise information management solutions, because many companies outsource their data processing to outside parties, so will need to integrate data in different formats. An increasing amount of data also comes directly from the Web and thus may be in formats like XML and HTML.
Don’t take “easily” for an answer. Dig in. Ask which additional technologies or modules, if any, you may need to buy. Will anything run as a separate server? (Separate servers indicate different architecture and possible integration headaches.) What about integration software?
A cautionary tale from John Kitchen, chief marketing officer for Datawatch Corporation, a provider of enterprise information management solutions: One of his company’s clients, a Global 500 consumer goods company, bought 300 copies of a BI product and found after three months that just four licenses were being used, suggesting a poor match between user needs and the product and/or a need for more training. Corollary suggested question from Kitchen: ‘Can you employ an Excel front end to access the product?’ Companies can cut down on the need for training by using Excel, an interface already popular among many users.
In which environments can it be used: traditional on-premise, a virtualized server environment, as a service (so different departments can ‘subscribe’ to it) and/or in the cloud? The answer will vary. You’re looking for a product that matches up well with your current infrastructure and any infrastructure changes in the pipeline, says Brian Gentile, CEO of open source BI software provider Jaspersoft.
The answers you seek will depend upon your needs and may counterbalance the answers to some of the other questions. For instance, if a vendor offers a BI solution closely targeted to your company’s vertical industry, you may be willing to be more flexible on some of your other desired requirements.
