Best Practices for Integration and Re-Imagining the ICC

Share it on Twitter  
Share it on Facebook  
Share it on Linked in  

A reader survey at SearchDataManagement.com revealed that 40 percent of data management professionals say data integration will be one of their top challenges this year.


It's hardly shocking, particularly when you consider everything that relies on data integration and how few systems are truly integrated. Plus, the article points out, big data and real-time BI are generating even more demand for integration.


Mark Brunelli, SearchDataManagement's senior news editor, asked Steve Yaskin, CTO of data virtualization and cloud integration firm Queplix, and Bill Gassman, a Gartner research director, about the five top best practices for integration.


I've been trying to decide if others would agree that these are the "top" best practices when it comes to integration. See what you think:


  1. Get visibility into your data. This recommendation is from Yaskin, which is not surprising since Queplix has a tool for automating data inventory. Still, it's an undeniable issue-the article quotes an Institute of Electrical and Electronics Engineers (IEEE) survey, which found discovery takes up 40 percent of the time required for a typical data integration project.
  2. Make data quality part of the integration process. As the piece points out-and as those in the industry have known for some time-vendors are consolidating data quality and data integration tools. The good news is, if you have any kind of updated data integration platform, this isn't going to be an issue since the tool will support you. Still, I suspect data quality experts would hasten to remind you that data quality also transcends integration and reaches into all aspects of data management.
  3. Form an integration competency center. Definite ditto on that one. As organizations (finally) embrace the idea of data as an enterprise asset, there are more hands on-deck when it comes to integration. The best way to make sure everyone's not reinventing (or reinvesting in) the wheel is to establish an ICC. No, really. Go now and form an ICC.
  4. Retain control of your data when dealing with cloud integration firms. Or, I would think, any cloud-based firm of any kind. Does that really rank as a best practice? Because it seems to me it's more of a basic no-duh practice.
  5. Adopt data integration tools that business users can handle. Excellent suggestion, and I would think a definite new best practice, as cloud and BI demands move more integration work out of IT and into the hands of business analysts.


Looking at this list, three -- gain visibility into your data, retain control of your data and address data quality -- fall into the "you must do or there will be dire consequences" category. As I mentioned, they seem more like no-duh practices to me, but then again I kind of live in an ivory blog tower. I'm not sure how you'll deliver on an integration project without first addressing data quality-though I can see how you might get away with not practicing data quality once the integration is done.


The remaining two-start an ICC and adopt tools designed to include business users-strike me as true best practices.


In fact, three years ago, I called an <strong>ICC "the best practice that companies ignore</strong>." As data becomes more of a strategic focus, it's time to take that recommendation to heart.


I'm not sure why organizations haven't embraced ICCs. Perhaps the problem is the name."Integration Competency Center" does conjure up images of pointy-haired bureaucrats taking their sweet time debating what's "best" while Dilbert beats his head against the table.


I suggest we rethink the ICC as an elite strategic strike force. Gassman recommended assembling a small team: your best integration techie, the smartest business user, and a quick-thinking decision maker. Their goal: To identify the best tools and practices so integration can be fast, focused and inexpensive.


If your integration work is NOT fast, focused and inexpensive, it's a sure sign you need an ICC Ken Vollmer, a Forrester Research analyst and ICC expert I interviewed in 2008, offered these four signs you need an ICC:

  1. Multiple, overlapping integration solutions in use
  2. Staff members-and now, business users -- don't have time to master the intricacies of the more advanced tools
  3. No guidelines for how common integration problems, like defining user interfaces, are solved
  4. Little reuse of existing interfaces


Let's redefine the ICC as a lean, mean integration machine.