Loraine Lawson spoke with Andy Hayler, CEO of The Information Difference, a research firm dedicated to reporting on the master data management market. The firm will soon release its first study, which surveyed 115 companies about MDM adoption. Next week, Hayler discusses what role IT should - and should not - play in master data management and MDM's role at small and mid-sized companies.

Lawson: How much of MDM is about integration?
Hayler: Clearly, the whole point of master data is you're trying to take a consolidated view of master data across the enterprise. Obviously, there are different ways you can tackle that. You can try to put in a big honking repository, hub, and try to switch off other systems that are generating master data – that's a quite extreme approach. On the other hand, you can register where the various systems are that have to do with master data. In the middle, you have a more co-existing strategy where you would maintain what some people call a “golden copy” of master data, but you're not actually trying to switch off the operational systems, you're integrating it.

Very much in that particular scenario, which to be honest is by far the most common anyway, integration is a major point. So, therefore it does touch upon other areas. It's not purely a matter of just having a hub. You have to think about data quality, which I think is a major component of any MDM project. On the Web site, we've developed profiles of the most important data quality vendors for that reason. Often the data quality vendors partner with some of the MDM vendors platforms vendors. It varies – some of them have their own proprietary algorithms for merging and matching, and others partner with specialist vendors.

Finally, at the integration level, you've also got a physical data movement layer because you move this stuff around. And so there is definitely a relationship between MDM repository and tools like Tibco, WebSphere and Informatica.

Lawson: Do the technologies work together or can MDM replace them?
Hayler: The relationship between MDM vendors and EAI (enterprise application integration) vendors is quite complementary, because you're typically passing individual messages around. At the moment, MDM vendors don't want to build their own message bus. They're typically partnering with message bus vendors. MDM platform vendors will usually partner with an IBM or a TIBCO.

In the case of ETL - and these days that's pretty much Informatica since they're the last man standing - it's a little less clear. Informatica has slightly surprised me because I really expected them to come out with an MDM platform offering. If you look at, for example TIBCO, it has already done this. They bought a small MDM platform vendor (Velosel) a while ago and have done a good job of selling the benefits of the MDM platform in concert with their data movement technology. For Informatica, whose core business is to have all these transformation rules to integrate data, it would seem a very natural thing for them to have an MDM platform, and so far they've restricted themselves to buying two data quality vendors, Similarity Systems and Identity Systems. I would imagine this won't be their last move in this area because it would seem extremely natural that they would wish to have an MDM platform. The reason for a lot of the lock-in Informatica gets is because people encode all their transformation rules in Informatica scripts.

In fact, the reason why lots of companies are unhappy with ETL is all of these scripts are very opaque and the maintenance costs can be very high so a lot of the drive for MDM is to improve on that. If people start to do their integration without an ETL tool, then at least a chunk of Informatica's addressable market will go away. Therefore, I'm personally a little bit puzzled as to why Informatica hasn't just bought out one of the remaining independent vendors.

Lawson: Obviously, MDM impacts data integration – but can it also require application integration?
Hayler: It most definitely requires you to think about an integration of processes and that may cause some integration of applications. If you're a company, like Unilever – a marketing company – and you want to restructure the way you classify your products or brands, you're clearly not going to just go into SAP and click a button and create a new level of, say, the international product hierarchy without checking with a few people first. There's going to be a draft of the new version; people are going to send this out for review to people operating in different companies throughout the world. It's going to go through versions and feedback, and then at some point someone will sign off and it will be implemented. So all those processes, which perhaps today are done by e-mail and spreadsheets, really need to be supported as part of the MDM process.

Ideally, your MDM technology will support that kind of workflow and allow you to trigger events and allow authorization rules to be implemented and so on. The better MDM tools have quite elaborate workflow capabilities and allow you to define business rules for your company, deal with versions over time, etc. Not all of them do, and that is quite important because managing master data is as much about processes as it is about actual technical data.

Lawson: Okay – so if integration helps with MDM, what does MDM mean for integration once it's up and running?
Hayler: It's not going to replace technical integration, but if you could imagine a world where there was a single master data repository for your organization, you'd need a lot less integration work. Instead of having lots of point-to-point solutions managing transformations between systems, you'd have a single repository that supplies the golden copy out to all these different applications through a message bus. If that happens, each application just has to link to the service bus, instead of point-to-point between SAP and Oracle and Salesforce and all these separate applications.

You can then start to put together a much cleaner picture. Some of the rules embedded and hard coded into transformation scripts would instead be stored into the MDM repository and be open for people to see, and hopefully that would improve maintenance. If you talk to anyone who's been through [for example] a data warehouse project, often there are a lot of hard-coded integration rules encoded into the scripting language. It's not at all clear who's going to maintain those, and business people cannot easily get to them. When the business changes, all these hard-coded rules tend to break.

In a good MDM solution, all these business rules are stored in a repository in an open format. A lot of the work that's done in a very manual way today will be removed, because it will be possible to regenerate the necessary scripts rather than IT people having to go out and physically edit them. This is why I say the integration vendors need to understand what's happening in the MDM world, because a piece of their world is changing and I think they need to ideally become part of that solution. If MDM does continue to grow and be widely implemented, and you're an integration vendor and you don't participate in that world, you're going to lose at least some of your potential application market. This is why SAP, Oracle Microsoft and IBM have all bought MDM vendors. All four have realized if they don't have MDM at the heart at their own offerings, then some of their proprietary applications start to be hollowed out. I don't think it's fully dawned on all the integration vendors.

Lawson: It sounds as if MDM will do for data what SOA is supposed to do for applications.
Hayler: That's precisely right. One of the issues for SOA is you can have all these standards of interoperability, but it doesn't help you much if these applications can't understand the data they're referring to. A lot of SOA applications will struggle if there is poor master data. Anything that touches data – and let's face it, most applications touch data – if each of them have different versions of data, that's going to be an impediment to successful SOA applications. So, MDM can help enable SOA.