A data warehouse lies at the base of any business intelligence (BI) implementation project. And if you are designing your data warehouse to help you visualize your company’s most relevant data (as well as streamline workflows — which can help your company cut down on redundancies significantly), then you need to map out all the areas where there is a potential for your project to fail, before you begin building.
In this article, Himanshu Sareen, CEO at Icreon Tech, has identified five design mistakes that companies should avoid.
Data Warehouse Design Mistakes
Click through for five data warehouse design mistakes organizations should avoid, as identified by Himanshu Sareen, CEO of Icreon Tech.
Shortsightedness
Mistake 1: Not looking toward the future
Data warehouse design is not a short-term investment and it’s important to understand that the process will rarely show results in a limited time frame. By looking toward the future, you’ll be more likely to implement a platform that fits your business and that can easily scale as your workflows and other software integrations scale. You don’t want to design your data warehouse based on the volume of your current data scale — but based on the hypothetical dramatic increase of volume and variety of data that you’ll receive in the future. When creating the data warehouse design, you need to pay as much as attention as you can to a long-term business strategy, with at least a three- to five-year roadmap for the information organization.
Lack of Documentation
Mistake 2: Creating haphazard metadata layer documentation
The metadata is the integrator between data processes extract, transform, load (ETL) and BI. Unfortunately, a problem arises when the metadata layer is designed solely to fit shortsighted data criteria, which results in haphazard documentation.
In order to avoid this mistake, you must add descriptions to the tables or columns at the data design stage itself. If a business user rejects a BI report, it’s most likely because of a poorly designed data model that lacks an accessible description and uses inconsistent naming conventions. Prevent this by creating a metadata strategy at the data modeling stage of data warehouse design.
Data Shadow Systems
Mistake 3: Ignoring data shadow systems
It’s easy to focus on an enterprise’s transactional systems of record and BI, but not all the important information is located there. Key info is also located in “data shadow systems” dispersed throughout the organization. Spreadsheets in Excel contain many of the most useful reports and analytics. Groups of spreadsheets and customized databases make up data shadow systems, especially programs like Microsoft Access and other similar statistical databases. Business processes, like budgeting, forecasting and other reporting tasks are supported by data shadow systems. You will always need to use reporting from your enterprise application and BI, but it would be a grave mistake to ignore the reporting and analysis from data shadow systems.
Staffing
Mistake 4: Not having the right team
In any integration project, it’s vital that everyone knows their roles and is able to communicate their pain points effectively. To that end, the primary data warehouse team will build data marts rapidly to achieve high ROI, making use of reusable components such as conformed dimensions, conformed facts and high-level transformation objects.
The team should include ETL engineers, who write processes and load data — ultimately, they are responsible for the quality of the data that enters and leaves the data warehouse. You will also need reports/dashboard engineers who build reports and dashboards, as well as a program manager, and data modelers who build out data models as per best practice. It’s also important to have a separate maintenance and enhancement team that responds to user requests for enhancement to completed data marts. Additional team members should include data warehouse administrators, DBAs, source system experts, data stewards, training support personnel and business analysts.
Visual Appeal vs. Speed
Mistake 5: Finding a balance between speed and visual appeal
Accessibility and speedy action is what drives a data warehouse forward. Though fancy charts and reports might tempt you, do not sacrifice speed! In order to gain popularity among users, IT teams have noted that speedy response times are the prime swaying factor. A report may take a few seconds to load, while a chart could take three minutes. It would be a mistake to prioritize visual appeal over speed when designing the data warehouse process.
Conclusion
A data warehouse can fail even before you begin designing. If the foundation for your data warehouse is flawed, then your business intelligence can’t be reliable. By looking to the future for your design, having the correct team, protecting the integrity of the metadata layer and prioritizing speed over visual appeal, you can create an almost flawless data warehouse design.
