Lance Speck, vice president and general manager of Integration Products at Pervasive Software, explains to IT Business Edge’s Loraine Lawson why organizations need to incorporate data quality into regular work processes, rather than a one-time task, if they want to fix the data quality problems created by nearly a decade of neglect.
Lawson: When it comes to how people can improve data quality, what are some of the eternal issues with data quality that you see in terms of processes?
Speck: We’ve moved pretty fast in the last five, six, seven years, whether it be on-boarding new SaaS applications, new trading partners or moving things to the cloud. It’s all been great and faster than ever, but data quality has not been at the forefront of a lot of these moves. People are starting to experience that they're not getting the value out of all the investments that have been made and one of the reasons is because they didn’t really put quality, not just at the forefront, but as part of a way of life, a way of integrating that into all of their processes on a daily or weekly basis, and not just taking one really good shower a year.
Lawson: Is that what you see organizations do: Once a year data quality?
Speck: Sadly, the first part, in my opinion, is who owns it? Is it IT? IT sometimes are the gatekeepers. Is it the business? Is it marketing? Is it sales? Who’s the ultimate throat to choke for the quality problem?
And correct me if I’m wrong, but I never get a clear answer to that and I certainly don’t get a consistent answer to that. There’s certainly a lot of people that complain about it, but the ownership is debatable as to where and who is going to actually take responsibility for doing something about it.
Lawson: You're out there talking to people and selling your software and obviously you have a vested interest in convincing them that you can help them with this. But to what extent do they understand that this is not just a technology problem? To what extent do they expect you to fix it and how do you manage those expectations?
Speck: I don’t know that they expect us to fix it, but they expect that we can provide something that will help them fix it. I don’t know that they look to a company like ours and say it’s our fault, but if we couldn’t provide something to be part of the solution, then they would look at us as incomplete or as an inadequate partner. Whereas it used to be data quality was its own separate thing, now it’s very much part of integration. It’s merged together into the data and application integration space. It’s just a core component of it.
Lawson: Right, so what sort of mistakes do you see, other than no one’s in charge, which I think is a pretty big issue.
Speck: No one’s in charge is one. The other is it’s only now getting to the point where people are realizing the cost. I don’t know if you want to call it ostriching or what, but there certainly have been people who tried to ignore or not really tried to measure what the impact of bad data is. So it just got worse and worse and worse, and now they're looking back at all the different fantastic cloud and on-premise systems they’ve built and all of the things they’ve done and they're wondering why they're not getting what they need out of it.
It’s always been this problem for BI. Right? So for BI, if the quality was bad 15 years ago, then you would say, “Well, then our BI is going to be pretty bad.”
Now it’s everything. If you think about our core areas where we’re strong, if you think about billing, if you think about tax, if you think about marketing automation, well, those are just three areas where people certainly can’t afford to do faulty billing. It wouldn’t be wise for them to have problems when it comes to tax or just bad information. Then marketing automation, well, if you're not marketing to the right place, if you're not able to know who your customer is and what they do and who, then you're going to be wasting a lot of cycles.
You always hear the cliché where they say, “Well, if your data’s not right, then you're going to spend X number of dollars in waste.” But take a CFO, for example. I believe at one point the CFO might have thought, “Well, that just sounds like its noise.
Marketing always comes up with noise.” Now, because they’ve invested hundreds of thousands or millions of dollars in these various systems, they're asking, “Where’s my return?” And a lot of times, they're not seeing that return because of the quality issue that they’ve ignored for so long. So now they're interested.
Lawson: Are there other processes that you need to deal with when it comes to data quality?
Speck: Just to put it into something that’s a little bit easier to consume, there are several pieces to it.
First, when you first implement, are you dumping garbage in at the beginning? What is your quality assessment? What’s the process for you to do the scrubbing before you even implement a new system or a new application?
Most of the time, those things are behind and they're over budget, so they just say, “Well, just dump it all in and we’ll figure it out after the fact, because we have certain MBOs based on launching this thing on time.” So getting in and being able to do that in what’s already a high-pressure situation is critical. If we can help them get in front of that and tell them how important it is and how much it’ll save them over time, then that’s big.
The second part is in-process data validation. You have multiple systems. They're all talking to each other in some type of batch or real-time way and they're coming from different lines of business that have different rules and you need to make sure that’s being handled. That’s where it gets really hard, because then you start talking about master data management and who is the data steward and where do you store this data and what is the system of record and who wins in the case of a conflict?
(Third), you have to get everybody together and agree that it’s important. Then get a mediator. If it’s the CFO, fine, but it’s usually not. It’s usually somebody in IT or a panel of the business owners from different departments who agree that it’s a big enough problem where they may have to make some concessions. That’s also hard, but the pain has gotten so bad that I believe people are actually willing to come to the table and figure out a way to fix it.