Data Integration Remains a Major IT Headache
Study shows that data integration is still costly and requires a lot of manual coding.
The original idea for an ESB was sort of like a train: You have lots of messages, why not use an ESB to shuttle them all from here to there?https://o1.qnsr.com/log/p.gif?;n=203;c=204663295;s=11915;x=7936;f=201904081034270;u=j;z=TIMESTAMP;a=20410779;e=i
The reality is a bit different. Instead of one train, enterprises are running multiple buses to handle various types of messages. Part of the reason, it turns out, is ESBs are quite good at integration.
Now, ESB use has been on the rise for some time. Last year's Forrester's Q2 Wave on ESBs revealed that out of 167 application development and enterprise architects surveyed, 58 percent were using an ESB, and of those, 32 percent planned to expand their use of ESBs.
But a recent Enterprise Systems article looks at the proliferation of ESBs from a different angle: vendors. More vendors, particularly data integration vendors, are now offering ESBs as part of the data integration toolbox.
How's that work? The article goes into some detail about how and why ESBs are being used, but here's the crux of how it's used for integration:
At a very basic level, the message bus gives each vendor a means to shunt its own event traffic into an internetwork of enterprise event traffic. An Informatica ESB can use well-defined protocols (such as the Java Message Service) to communicate events to a TIBCO ESB. The ESB, in this context, becomes an integration interface. To use the language of traditional data integration, it becomes a connector' or adapter' to another application or platform.
Part of the reasons ESBs have fallen into this role is that most organizations have heterogeneous environments that must be mediated across applications. This, by the way, is also why ESBs are becoming a popular way to mediate between cloud-based and on-premise applications.
There are a couple of other interesting points made in this article. First, it discusses how integration solutions are seldom siloed as either "data" or "application" integration. Instead, their shift is toward events - as in complex event processing - which cuts across data and applications.
Second, Mark Madsen, a principal with BI and DW consultancy Third Nature, calls out ESBs as an outdated approach - a holdover from the client-server days.
"I suspect that part of the ESB challenge is that it's still sort of server-centric without meaning to be," Madsen told Enterprise Systems. "Today, SOA is in and ESB isn't quite the same approach and doesn't encapsulate the same way. there is no big message passing - an application calls services, maintains its state, and the remote invocations do the work."
As evidence, he points out that companies like Netflix, LinkedIn and Amazon manage to do large-scale, real-time applications without ESBs.