7 Steps to Smarter Integration
Sometimes, change can be worthwhile. The key is knowing what's worth pursuing and what's not.
There are so many reasons for data integration that it's easy to forget the fact that there are also legitimate business drivers for segregated data.
Jim Harris, a data management consultant who runs the Obsessive-Compulsive Data Quality blog, recently took a look at this topic from the perspective of data governance. He pointed out that there are good reasons for data silos. For instance, sometimes security requires data to be segregated from other enterprise data, as when you're dealing with sensitive or private data that should not be shared.
But, as you well know, silos are often created because some department just "felt" it needed a separate copy of the data. And then there are those situations where the silo served a purpose at one time, but is no longer needed.
One of the early goals of a new data governance program should be to provide the organization with a substantially improved view of how it is using its data - including data silos - to support its operational, tactical, and strategic business activities. Data governance can help the organization catalog existing data sources, build a matrix of data usage and related business processes and technology, identify potential external reference sources to use for data enrichment, as well as help define the metrics that meaningfully measure data quality using business-relevant terminology.
The goal, he writes, is to clarify whether or not there's an existing and legitimate reason for a silo to exist, and, if so, to make that reason transparent.
Privacy, security concerns, legal compliance - these are all good litmus tests for whether or not you should keep data in a silo. Certainly, this may take priority over other business drivers.
Those examples are no-brainers. What's hard are the silos that purport to serve a business purpose but actually do just the opposite. These silos can be tricky to identify and as sticky as a July afternoon in Georgia.
Netflix very generously provided us with a high-profile example of this type of wrong-headed silo thinking, as Jamie Fitzergerald of Dataversity recently pointed out.
Netflix's Reed Hasting was convinced he had a legitimate business reason for spinning off a new company called Quickster: He wanted to focus on growing the company's streaming business. But as Fitzgerald points out, Netflix failed to consider the customer experience this would create:
To put it bluntly, their plans would have intentionally created data silos (something most savvy companies strive to eliminate!) and as a result would have reduced benefits to customers from existing data assets, technology, and processes. Ironically, the same company that sponsored the pioneering Netflix Challenge - effectively crowd-sourcing the optimization of their movie recommendation model - was planning to intentionally create a less integrated, less customer-friendly version of itself.
Creating two operational data silos would've degraded customers' experience and Netflix's main drivers of customer values, Fitzgerald continued. "While most firms are striving to integrate their data, the Netflix plan would have 'ripped apart' an existing data ecosystem of significant value," he wrote.
After much public outrage and bad press, Netflix backed down from its spin-off plan.
I'm sure it was humiliating, but at the same time, at least there was an intervention to stop the silos. It's easy to cast stones, but have you looked at your own silos?
Silos aren't just a data management or data governance issue. Silos are, as Fitzgerald writes, "a sure-fire path to bad customer experiences."
Are you pulling a Netflix by allowing silos that frustrate and undermine the value your organization provides to customers? So, yes, consider the privacy and security issues of silos, but weigh any other alleged "business reason" for a data silo against the ultimate litmus: Does it enhance or detract from the customer experience?