Take the Strategic View on Data Integration Projects

There are a number of technological factors driving data-integration projects within organizations right now. Service-oriented architecture, modernizing or replacing legacy systems, Master Data Management and adding complex or data exchange projects all require data integration.

So, it's easy to view data integration as one technical component of an overall technology initiative. On the face of it, it's about making data accessible from one system or application to another. Technology can do that -- whether you choose to hand-code point-to-point or publish data in a data services layer, it's technology.

Yes, but ... it turns out when IT divisions take this approach to data integration, it creates unforeseen problems and misses opportunities to add business value.

I think part of the consternation about Master Data Management is that, possibly more than any other data-integration technology, MDM requires you to think beyond the technology -- a point well made in recent blog posts by Forrester's Rob Karel and Informatica's Rick Sherman, who writes:

"My contention has always been that MDM is not a product solution, but a process with the key ingredients being people and politics."

It's a point you'll see made time and time again with MDM: You simply can't do it without involving the business and answering some serious business questions about who owns the data and who's responsible for quality.

MDM makes it impossible to take a technology-only approach to data integration, but the fact remains it was always a bad approach -- as this column, "Ten common mistakes companies make in data integration," demonstrates.

It's written by Marcia Kaufman, a partner at the research and consulting firm Hurwitz & Associates. Kaufman does identify 10 common mistakes, but I would say that most of the 10 mistakes amount to one big mistake: Taking a technical, rather than strategic, approach to data integration.

Kaufman's list isn't about blaming IT or preaching about IT/business alignment. In fact, Kaufman points out in mistake number 9 -- "Business owners are reluctant to give up ownership of data" -- the business users can be just as guilty of ignoring the big pictures as IT.

No, the point isn't to cast blame for creating the problem. The point is to figure out how you can contribute to finding a better solution. And Kaufman's list is a good starting point for doing that.

If you're involved in B2B data exchange or dealing with complex and unstructured data, you should also check out this recent The Data Warehousing Institute (TDWI) whitepaper, "Complex Data: A New Challenge for Data Integration."

This 12-page paper takes an in-depth look at why organizations need to expand data warehouses to include complex data, which essentially includes all forms of data not contained in your typical tabular structure. This means both unstructured data, such as e-mails and text files; semi-structured data, which contains some metdata; and complex-structued data, such as the data found in XML structures.

This paper does an excellent job of addressing both the technical and the business challenges you'll encounter in a data integration project.