The trouble with data quality is that the whole process of managing it is disassociated from how the data is actually used, so what winds up happening is that it never gets done.
Instead, data quality becomes another exercise in IT good housekeeping that nobody ever quite gets around to doing. Against that backdrop, it's worth noting how SnapLogic, a provider of application integration software, has teamed with Trillium Software, a provider of data quality software that is provided via a hosting service, to integrate data quality management directly into the application integration process.
According to Clark Newby, SnapLogic senior vice president of marketing, IT organizations can set up a SnapLogic server to automatically route data being transferred through the Trillium service, which essentially makes the whole data quality management process transparent to the average end user.
The Trillium integration is part of the summer 2011 release of the SnapLogic server, which also includes an instance of the open-source CouchDB database for managing documents and improvement to the way SnapLogic can run in-memory to boost overall performance.
As Newby points out, new technologies such as cloud computing and Big Data tend to turn bad problems like data quality into full-fledged epidemics. If IT organizations want to cure data quality issues, or at least make the problem manageable, they are going to have to find new ways to automate the process in a way that, as far as the rest of the business is concerned, makes the problem essentially go away. After all, rightly or wrongly, the rest of the business thinks data quality is an IT issue so the IT department might as well take it upon itself to solve the problem in the least-obtrusive way possible.