In a recent article, EAI and SOA expert David Linthicum proposes that we're about to see a new generation of data integration products. What does this mean for IT and users?
It means products that are easier to use, whether you're tapping on-premise or cloud-based data, Linthicum writes. It means products that use core standards and pre-built integration solutions. And it means quick, adaptable, dare he say agile, solutions. "In other words," writes Linthicum, "this technology finally lives up to the vision and the value of data integration."
To quote George Harrison in A Hard Day's Night, I'd be quite prepared for that eventuality. Aren't you?
Actually, Linthicum identifies a lot more essential qualities for third-generation integration products, including semantic mediation, security, data governance and better latency. And I must say, as pie-in-the-sky as that may sound to you, those are all things vendors are starting to bring together and offer.
To make his case, Linthicum traces the evolution of data integration. First-generation data integration used a wide range of approaches to solve the problem, including message brokers, single-pass transformations, and eventually EAI (enterprise application integration-a pattern-based aproach) and ETL. Obviously, these approaches had (and still do) their limitations, as Linthicum writes:
"The different approaches to data integration meant different results within the enterprise. In addition, the features of the technology were limited in terms of management, monitoring, and security. Moreover, many of these first generation technologies had performance issues."
The second generation of data integration tools focuses on the ESB (enterprise service bus), particularly as it's used with service-oriented architecture, Linthicum contends. As you no doubt suspected at the time, some of the first-generation vendors just rebranded their products, he writes, and many of these were "ineffective, being just too big and too slow."
And we already know the craziness that surrounded SOA, but Linthicum nicely sums up that brouhaha, observing:
"Many had trouble separating the concept of SOA from the concept of data integration. ... It was not really a failure of the pattern of architecture. It was a failure to understand the architecture."
All of which brings us to the third generation, which builds upon the first two generations, but with added solutions to address the unique challenges of new data challenges-specifically, the volumes of data we're dealing with, the focus on standards and, of course, the integration challenges of the cloud and other distributed systems.
While there are many reasons for these needs, the primary causes involve the emergence of cloud, SaaS, distributed and hybrid models of computing. Moreover, increasing volumes of data, the focus on standards, and drop-in enterprise applications contribute to the overall complexity of enterprise IT.
I have to say, the third generation looks great-but then again, these things always do. The question in my mind is how well the third-generation products will actual deliver on those promises. It's not a question I would expect Linthicum to answer in that article, however, largely because it appears on an integration vendor's website. Let's face it, there's a certain amount of credibility loss in any criticism - good or bad - on a vendor's site. Maybe we'll hear more in one of Linthicum's many other forums.
In the meantime, if you'd like to learn more about one type of a third-generation approach to data integration, you might want to sign up for TDWI's upcoming seminar on data federation. TDWI Research Director Wayne Eckerson will host the free event, which will explain the essentials, including defining data federation, identifying its use cases and limitations, and looking at possible tools and best practices. The event is scheduled for June 9 at 12 p.m. EDT. You should probably pre-register, however, since space is limited on this webinar.