Could Where You Place MDM Tools Affect Data Quality?

Loraine Lawson spoke with Marty Moseley, the CTO of master data management vendor Initiate Systems, who says companies are making a mistake by always coupling MDM tools with the data warehouse. He explains why moving MDM “upstream” can improve data quality.

Lawson: I was really interested in what you said about it mattering where you place your master data management solution. Where do people typically place MDM solutions?
Moseley: I think if people are in the data warehouse mindset, they usually think of master data management as a step in the cleansing process or as a feature or a function of the data warehouse. And that's not an invalid place to put it, certainly.

But if you think a main premise of master data management (is) to cleanse and to be the current version of the truth for some of your most important data, then you want to use that to influence transactions as they happen on your enterprise. That means putting it way upstream of the data warehouse, because by the time transactions happen, if you don't have MDM, then you're running the risk of capturing low-quality or inaccurate or redundant data. And once that transaction has happened and it's flowed through a half a dozen or a dozen different systems, then trying to fix it in your data warehouse is extremely difficult, if not impossible.

So I believe it's better to use MDM as a preventative measure to prevent bad data from entering the mainstream than it is as a reactive kind of a measure. But a lot of people are trying to make it part of the data warehouse and that's a fundamentally different use case for MDM than what I've been advocating, which is implemented at the edges and ensures higher qualities and standards of data, you know, from the outset.

Lawson: What would be the use case for putting it where it's typically put?
Moseley: What they do in data warehousing is they'll say, since I'm the data warehouse I'm de facto the standard for customer and product or item or vendor location, and therefore it makes sense that, as the data warehouse, I'm the current version of truth. And there's truth to that. The problem is it's not real time and it's not interactive. And so the rationale is, gosh, I've already got all this data from throughout my enterprise and it's already in the data warehouse and therefore why would you think to put your master data management any place but the data warehouse? And of course, its whole DNA is very, very different from what you need in a real-time service that guarantees data quality at any touch point in the organization or the ecosystem.

Lawson: So is this a decision people are kind of making because it's organizationally or politically easier or is it a technology decision?
Moseley: There are probably organizational aspects to it. There are probably some technology aspects. But in my opinion the reason it happens that way is that over the last 15 years or so, since we've been doing data warehouses in earnest - and I started doing data warehouses actually in '91 before it was really called that - what has happened is most of the skill sets for people that are worried about data quality, about data modeling, about information management, they’ve all migrated to the business intelligence world. That's been the center of gravity for data quality and data management and enterprise data management.

So it's natural that the people who would care about MDM happen to have been working in the data warehouse and business intelligence world for the last five, 10, 15 years of their career. And that's where you have the preponderance of folks who think about MDM. So, it's natural for them to think about MDM in context of the problems that they’ve been solving instead of jumping out of the data warehouse world, which is really a post-event, analytical kind of a framework, and jump into the real-time services world. They typically haven’t been involved in that part of the IT infrastructure.

Lawson: Are some MDM solutions set up to go a particular place or are they placement neutral?
Moseley: A couple of the vendors we see out there have applications that are called MDM applications, but they tend to be almost like ERP applications. They have lots of screens and forms and they're fairly monolithic in terms of their architecture. Although they would like for them to be a real-time, agile service that you call, they're pretty monstrous in size. And so they would typically see themselves as the center of the universe that every other system would tie in to. So MDM as a monolithic application would be one style.

The other style that is closer aligned to what we do is MDM as a service, in terms of a service-oriented architecture. We would see it as an autonomous, discrete application. It may be something that is living in the infrastructure that any system in the universe can call in real time to vet the quality of their master data. In that case, if I have a transaction that is happening in a CRM system, an e-commerce system or an ERP system, those systems can call this MDM service in real time to improve the quality of the data that they manage. So as those data flow through the enterprise, they're automatically of higher quality. That's more of a service-oriented approach.

Now, of course, MDM is a hot enough area that all kinds of data quality vendors are in there - all claiming that they do a major piece of the MDM puzzle. Other large companies, even though they're a data warehouse company, are saying, "Yes, yes, yes, we do MDM too." And of course, they do it attached to the data warehouse. So pretty much every vendor out there has some MDM solution.

Lawson: What is Initiate's MDM heritage? CDI?
Moseley: We actually got started before CDI. We got started in health care doing EMPI, electronic master person index or master patient index, solutions.

So we got started in matching patient records, where it was a matter of life or death. In some cases where we're hosting nationwide pharmacy records, we had to be able to scale to hundreds of millions of rows of data and yet be able to have performance. If somebody is checking into an emergency room or they're getting some kind of emergency surgery or procedure, you can't wait around for records to show up in the admissions or registration desk. So you have to be able to scale, you have to be able to perform and you have to be accurate.

We were doing that and then when CDI came along, we were actually looking for opportunities to get into the commercial market. It was kind of a natural for us. Then, as CDI was overshadowed by MDM, that also is a good fit for us from a market perspective.

Lawson: In the information I received from you, you mentioned the different architectural approaches to MDM, including hybrids. What do you mean by architectural approaches and what do you mean by “hybrid?”
Moseley: If I think of MDM architecture, I may choose to have all of the relevant data for the master data reside in all of the systems that house part of the data. And in that case, the service, the hub if you will, only has the key data required to go out to those pinpoint systems and gather all of the master data when there's a point of need. So that would be a registry style.

If I think of my master data as a monolithic app, kind of along the lines of an ERP system where every other system in the enterprise is subordinate, then that's what people think of as a centralized or a transactional style.

Now, there are pros and cons of each, right? The first style is very lightweight, it's very easy to change, that sort of thing. But whenever I make a query, I've got to go out and gather data from N number of different systems and pull that together. So it's got a little bit of a performance hit although it's very easy to get up and running.

To have an ERP-style system, then I can have all of my master data so that I never have to go out to any endpoint system, but it can also be difficult to scale and it can take a long, long time to get one of those up and running just because it's got so much data in it.

So a hybrid style is a hybridization of those. It has the registry approach, but it takes in enough common data into the hub so that you don't have to go out to the endpoint systems to gather the data when you need it. That gets over the disadvantage of the registry style but it lets other systems still be the masters of their own destiny. So other systems are not a slave to the hybrid style, but it has enough data in it so that when I want to inquire about a master record, I can go to that system and gather everything that I need. But it's not monolithic - it doesn’t store everything. So the hybrid system is a blend between the two and actually it turns out that it has the advantages of both and stays away from the disadvantages of both as well.