Why Businesses Keep Building Data Silos - Even When Everyone Knows Better


To me, one of IT's great paradoxes is that even as it tackles data integration projects, it continues to create new data silos.


Rick Sherman, a consultant with Athena IT Solutions, recently addressed this issue in his blog, The Data Doghouse. He concludes there are two reasons we still have data silos, even when we're striving for that ever-elusive "single version of the truth":

  1. Business intelligence and datawarehousing projects are typically organized on a tactical, project-by-project basis.
  2. There's no overall information architecture to guide that blueprint.


Okay-that's what causes them, but it still doesn't address the why. Why not change to a more strategic approach?


Sherman thought that question through as well, and lists five factors at work in favoring a tactical over a strategic approach:

  1. Group-based funding for projects.
  2. Jargon confusion. "There are many similarities between CRM (customer relationship management), SCM (supply chain management), budgeting and forecasting, performance management and balanced scorecards. Many people do not see that each has data, data integration and business intelligence that can be shared across projects," Sherman explains.
  3. Technology silos-in other words, they treat different integration technologies (ETL, EAI, data virtualization, etc.) as different applications and projects-when it's all just different approaches to data integration.
  4. IT's organizational disconnect, which keeps "data people" from talking to and working with "application people" -- a political problem also noticed by Judy Ko, Informatica's VP of Product Management and Marketing.
  5. No architecture helping to shape tactical projects.


This piece is Sherman's second on the topic of how people, process and politics impact data integration. He's promised more, and at some point I'm sure he'll come back to one solution he's written about previously: Integration Competency Centers.


Informatica, for whom Sherman sometimes writes, is huge on the concept of ICCs. In fact, the data integration company recently commissioned a study on the the potential ROI of Integration Competency Centers. Forrester, another proponent of ICCs as a best practice, conducted the research.


Forrester concludes ICCs can pay off in a big way. It found running an ICC can cost about $2.7 million over a five-year period, but the typical enterprise can save more than $5.5 million in IT and business benefits over five years. So, for large enterprises, that's double your money back-not a bad ROI.


Alas, there's nothing in the free report about how much an ICC would cost or whether it would payoff for smaller companies.


Joe McKendrick, who wrote about the report's findings for Informatica's blog, noted there's another potential payoff for ICCs: An ICC can help with SOA.