Involving the Business in Data Integration, Migration and Governance

Loraine Lawson
Slide Show

Strategic Integration: 10 Business-Building Tips

Ten ways that companies can use integration and integration-related strategies to build business.

Teradata's data warehousing specialist, Rob Armstrong, shared a humorous story with me this week that says a lot about what's wrong with data and how it's being managed. He was working with a company that told him their data was fine-until they did a data warehousing project. The data warehouse, they explained, caused their data to be unclean.


It must have been hard to keep a straight face. Fortunately, I didn't have to bother. It's like looking at gray bath water and saying you were perfectly clean before someone made you take a bath.


Correlation does not imply causality, of course, but Armstrong had a tricky time convincing them that their data was already dirty; they just didn't know it because they'd never put it in one place and compared it.


Data projects, whether integration, migration, analytics or even governance, often bring to light problems that were previously hidden. And it's easy to want to blame the technology or the IT department; but increasingly, it's becoming clear to IT and to business leaders that if you want to stop the blame game and actually address the underlying issues, it's going to take involvement from the business.


As I shared yesterday, it's a fallacy to see IT as "in charge" of the data or of the problems associated with it. And as Armstrong pointed out to me, while IT certainly has some areas it needs to address, many of the problems of data integration require the business to solve questions-including questions of how data is defined (e.g. What is a "customer"?).


But it's easier said than done and leaves a lot of questions unanswered. For instance, when should the business be more involved and how do you get them to actually participate in processes that, until recently, IT "owned"?


I've found three resources recently that can help answer these questions at different stages of the data management process:


Let's start with the data migration process-a key component of data warehousing projects, and one you may not think would involve the business in any significant way. Dylan Jones, the founder of Data Quality Pro and Data Migration Pro, recently tackled this unusual area for IT/business alignment in a piece for DataFlux's blog site. I should add that, while he's talking about data migration projects, much of what he says could also hold true for data integration projects.


Jones contends there are three areas where IT should involve the business, but typically doesn't:


  1. Data cleansing. Business stakeholders and subject matters should be central to this process, which is all-too-often outsourced, writes Jones. "It is their responsibility to define what quality means and to work proactively in ensuring that when external resources are used, they deliver high quality data that meets the needs of the target system," he says. Certainly, this advice can apply to integration projects.
  2. In creating a transition plan-or during what Jones calls the "window of opportunity," which is the time between moving from legacy to the new target system. Often, IT is so focused on the technical issues, it forgets to consider how this will impact the business-particularly if there are problems. "Instead, the business should take control of this process; they should dictate what they need to transition, what quality controls they need to see, what confidence measures have to be reported in order to transition from the old world to the new," advises Jones.
  3. During validation and testing of the data. It's common to involve users in testing in a very "click here, click there," low-level kinda of way. But you're missing an opportunity to identify real problems early on if you don't have the business users deeply involved in all phases of validation and testing, he argues. "This goes far beyond user acceptance testing; they need to help design the testing use cases and overall test strategy as they know the data better than anyone else," he writes. Again, you'd do well to apply this step to integration projects as well.


Data governance is another area where business leaders need to step up involvement, and, fortunately, they seem to realize this, according to a recent article by Andy Hayler, published in the UK-based site, CIO. Hayler is the founder of Kalido, a data warehousing and business intelligence (BI) company, and now does consulting and blogging.


The article isn't heavy on concrete ways to involve the business, but it does include a lot of statistics about how more business leaders are becoming involved in data governance and why that matters. His company surveyed 257 large companies-with just over half from North America-and found that business involvement in data governance and quality is on the rise. Specifically, it found:


  • 79 percent of data governance initiatives were led either by the business or jointly by the business and IT.
  • 80 percent claimed to be measuring data quality.
  • The median number of employees dedicated to data governance is six full-time business staff and four IT staff.


The trick, it turns out, isn't always getting business leaders and users involved, but holding their interest. When business departments resist participating or become fatigued with the process, the whole thing falls apart and you wind up with distrust in the data and the reports you produce with it, he says:

Yet all too often such initiatives falter because the business people that sponsor such projects are insufficiently engaged, and after an initial burst of enthusiasm return to the old ways of doing things. ... Only by convincing the business that they own this data and are partly responsible for this mess will large organizations be able to get a true picture of their global business performance at a level of granularity and reliability that they need.

Finally, we come to my personal favorite, involving users in data integration. It turns out, there is a new generation of data integration platforms that is addressing the very question of how to encourage collaboration and give business managers more control over integration and data, according to an upcoming webinar featuring the senior manager of TDWI Research, Philip Russom.


The title for the Dec. 2 event is very telling: "Data Integration Platforms: A paradigm shift towards Business-to-IT Collaboration." The write-up explains that these new platforms are emerging from both a user-driven trend to squeeze more value from data and a vendor-driven trend to support multiple user types, metadata and other reusable artifacts.


It sounds like this webinar will give you both the talking points and some concrete ways to involve the business more in the data integration process. Russom will discuss both the business and technical reasons for collaborative data integration, as well as use cases for these collaborative platforms, and more.


The event is free, but you may want to pre-register. It begins at 9 a.m. PT. I suspect it'll be well worth your time, especially when you consider the high cost everyone's paid for IT being the only one responsible for data initiatives thus far.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post
Nov 25, 2010 3:44 PM Dylan Jones Dylan Jones  says:

Great article Loraine, thanks for the link also, appreciated.

This is a really timely piece, I recently ran a survey of Data Quality Pro members and secretly I was expecting a bias towards the IT end of the spectrum, I was really blown away by the fact that it was a reverse, far more people from the business end, despite the fact we cover a lot of technology, technical tutorials etc.

Whilst this is encouraging I still see the flip side when I talk to our members. So many data quality and data migration projects get passed directly to IT or even worse, to external IT outsourcers. It seldom works out and they invariably end up with a substandard product at the end.

Great point about keeping leadership interest too, on data migration projects we do this by actually forcing them to sit centre stage and manage the data quality elements of the project so they have no-one to blame. Obviously they have support, tools, resources etc. but the only way to sustain buy-in I find is to shift accountability. Easier said than done of course!


Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.