The Never Ending War for Data Quality

Share it on Twitter  
Share it on Facebook  
Share it on Linked in  

One of those issues that is always a source of frustration in the enterprise is the quality of the data we spend so much time and money processing. The quest to make sure we have high quality data is nothing short of a never-ending battle between the forces of order and the chaos that envelopes every attempt to organize anything.

In the last couple of years, however, fighting the good fight over data quality has been getting more attention for a few reasons. The first is simply the amount of time IT organizations have to throw at the problem. With fewer major IT projects being launched in a down economy, spending more time on existing application maintenance and renewal is only a logical consequence.

But the other major factor at work here is that business wants to share that information much more aggressively than they ever have before. And not only do they want to do that internally, they also want to share that information with customers and suppliers as they move to integrate various business processes involving multiple applications. This can come as a shock to the internal IT department because it forces it to come clean about just how disorganized, and frequently wrong, the data actually is.

The good news is that companies such as Information Builders are reporting brisk sales of their data quality tools, which implies there is a lot of activity going on. The better news is that most business executives already suspected that there was something wrong with a lot of the data residing in various applications. It's one of the primary reasons they don't always have a lot of faith in the IT department. But for the most part, many of them are supportive of any effort to clean up the mess.

Alas, the war over data quality can never really be won. What can be done is that the number of instances where we have conflicting data and outright errors can be sharply reduced. There's no shame in having bad data; everybody does. The only real sin is not trying to do anything about it.