CIOs Must Change Approach to Data Quality

Data Integration Remains a Major IT Headache

Study shows that data integration is still costly and requires a lot of manual coding.

Riddle me this CIOs: Nine out of 10 executives describe their business as "data-driven." And 71 percent say they struggle with data inaccuracies on a daily basis.

How exactly is that going to work?

For IT, data quality tends to mean all your rows and columns matched up when you integrated your data. It's been a fairly specific, technical term, but that's changed in recent years. Now, data quality must address a series of concerns: Is it good? Is it accurate? Is it recent and reliable? And perhaps most importantly, can I depend upon it for making strategic and daily business decisions?

Data quality can make or break a company, or at least its quarterly earnings statement.

This should raise some serious questions for CIOs and IT departments about how you're going to provide that level of data quality, because you're probably not going to be able to do it with what you have now.

"With so much business success depending on the condition of enterprise data, it behooves us all to give that data the best quality, content, and standards possible," writes Philip Russom, the director of TDWI Research, in a recent article. "Most DQ solutions are in need of an update, expansion, or replacement. Most of these are siloed, with a focus on a single data domain, department, or application.

"Hence, the greatest challenge to next generation data quality is to extend its reach into more of the enterprise and its data."

It's a different data world today than when many data quality tools were created. And while the tools have evolved, just consider how our demands have changed:

• More companies and manufacturers need data to be real-time or near-real time.
• There are more compliance issues, which require better data governance and data stewardship. Data quality can play a key role in enforcing and automating governance, Russom points out.
• There's a growing business hunger for external, third-party data, which can be used to improve data quality, but which also brings its own quality concerns.
• Data has grown faster than Kudzu in Georgia, and companies need to be able to access and analyze it faster than they currently can.
• More types of data are being collected, including unstructured Web data, social media data and sensor data.

In "Ten Goals for Next Generation Data Quality," Russom outlines how things have to change, from tools to team structures and mindsets, before data quality will be able to support these new demands, and he even offers guidance for what you can do now.

While technology will play a role, it won't be enough without the disciplines to support it. This makes Russom's piece a must-read. The article appears in the most recent edition of What Works in Data Management Vol. 33, a best practices journal published online by TDWI. It's available as a free download and includes case studies and articles by industry experts and vendors.