Nine Key Data Warehousing Trends for the CIO in 2011 and 2012
Optimization, flexible designs and alternative strategies will become more important as the demand for BI and business analytics increases.
Primarily, when you think about data integration you think about, well, data-not the applications. In part, that's because the way data is processed tends to separate out the two - you pull the data into a data warehouse, data mart, or whatever (hence, the "extract" in ETL) - and then you do whatever it is you want to do and you load it.https://o1.qnsr.com/log/p.gif?;n=203;c=204663295;s=11915;x=7936;f=201904081034270;u=j;z=TIMESTAMP;a=20410779;e=i
But it's not really so cut and dried, is it? Applications are intimately tied to data creation and use. What's more, a recent study by Ovum found that good data management may be critical to the performance of your applications.
Ovum surveyed IT executives at 146 large enterprises in North America, Australia and the UK on application performance and management. The survey points to a strong connection between bad management of data and problems with applications. Specifically, 85 percent of the companies complained of application performance problems and the leading culprits trace back to bad data practices such as a lack of standardization, inadequate archiving practices and too many point interfaces.
The study also found that 20-30 percent of data is duplicated across applications, which increases application maintenance costs and creates problems for data migration, synchronization and retention.
"All efforts to improve application management and delivery will be in vain if the underlying data and its management strategy are flawed - no matter how well architected the application platform, or how effective the development team," Ovum warned.
Really, none of that should surprise anyone. We all know what happens when an application has trouble loading or processing too much data. Heck, we've wasted entire days on that problem. But it's not actually an issue that gets a lot of attention. However, I suspect it's time to pay attention, especially as organizations are moving away from an application-centric view toward the a "data-centric enterprise," with its focus on Big Data and analytics.
Here's another issue organizations should start thinking about: How are you going to shift your existing data infrastructure to a data architecture that's planned and fits the modern demands of a data-centric enterprise?
There's a lot of information about what that data management should look like - from best practices to new products from integration vendors - but precious little discussion about how you make that shift without disrupting business or spending a fortune.
If you want to shift your data integration architecture - or, really, any architecture - but aren't sure how to do it without ripping and replacing everything, I recommend you check out "Create a Realistic Data Integration Strategy," a short video by Amy Kunz, a research analyst with Info Tech Research Group.
Now, there are no quick fixes to the chaos of most data layers. So Kunz suggests you focus on creating a short- and long-term plan that will steadily move you toward your desired data integration strategy. What I love about this video is that she manages to explain how to do that in just four minutes and 30 seconds, which is about all the patience I have for videos, to be honest.
In the short term, she suggests you:
- Identify where the current infrastructure causes pain.
- Document the existing architecture.
- Decide on a model architecture you will work toward - federated, hub-and-spoke, distributed or a combination - by thinking about the most critical interfaces and choose a model that best suits those needs.
- Since 82 percent of organizations say integration is not a standalone project, you'll want to identify business projects you can leverage as a way to shift your data integration strategy.
- Define standards for integration development - best practices.
But remember, even with short-term goals, you're talking S-L-O-W change. It can take years to complete the shift, she warns. So, long-term, you'll need to:
- Work toward your chosen model architecture.
- Look for integration consolidation opportunities. "Sometimes, the easiest way to fix an integration is to rebuild it completely, but with a new model architecture in mind, you may be able to replace three problematic integrations with one rebuild-an easy way to simplify," Kunz says.
- Build executive support by choosing a vocal and influential business sponsor who is experiencing the problems caused by poor data integration.
- Enforce the standards you defined in your short-term steps.
Like I said, it's a very concise presentation, but I think it's a smart and doable approach that's well worth the four and a half minutes you'll spend watching it. If you'd also like to read the Ovum study, you can download it from Informatica's UK site.