They say the only constant is change. But as Paige Roberts of Pervasive Software's Integration Division Product Marketing points out, businesses claim they want to be agile and prepared for change, but when it comes to their integration strategy, many are still pretending the applications, systems and data sources will remain constant.
Talk about planning to fail. If there's one guarantee in integration, it's that at some point, an integration point will break. As programmer Alberto Gutierrez recently wrote on his blog, Making Good Software:
The problem with any integration point is that you don't have any control over them, which means that they are not reliable; this requires the developer to have in mind additional considerations to successfully develop robust integration points.
Gutierrez addresses the problem from a programmer's point of view, but after reading Roberts' post, I think there are reasons to consider this issue further up the management chain. First, you need to make sure your programmers are, in fact, doing what they can to prepare for the inevitable integration breaks-Gutierrez's post offers four tips on what they should be doing. Second, in this interconnected world, it can pose a bigger business problem when integration fails, which means you need to have a plan in place for when these problems do occur.
Fortunately, as Roberts-a former software engineer-explains, there are design strategies you can use to plan for change. And, you can ensure that whatever data integration solution or platform you choose supports those change strategies.
7 Steps to Smarter Integration
Sometimes, change can be worthwhile. The key is knowing what's worth pursuing and what's not.
In other words, you need an enterprise-wide plan for managing integration change, whether you're focused on internal or external data integration.
What do you need to do at a top level to prepare for change? Here are some of the strategies Roberts and Gutierrez say you and your development team should put into place:
Plan for failure. "Every single integration point may fail; keep always this [in] mind when developing," warns Gutierrez. So, isolate the volatile bits, warns Roberts. "For integration projects, the volatile bits include paths, business rules, and data transformations among other things. Those things change with the weather."
How you do it: "Don't ever get caught with a proprietary system that won't let you modify those yourself, or where you have to pay an expensive consultant to come in and do it each time, when he can get around to it. Good integration design puts these in a location where the business user can get to them fast, and change them easily," advises Roberts.
Plan for modifying integration. You don't want an ERP upgrade to mean you have to redo your integration framework, warns Roberts. Likewise, Gutierrez cautions you don't want one integration failure to affect an entire application.
How you do it: Go modular, advises Roberts. "If a chunk of a solution is no longer wanted, you need to be able to pull it out, and plug in a new one, without breaking everything," she writes. Another option would be to use a service-based approach to data. Gutierrez suggests you minimize the amount of interactions with integration points, wrap the interaction logic in transactions, allow for retries and separate the business logic from the logic handling the integration.
Plan to find problems before they're business problems. "Every integration ever built will break, no matter how elegantly and flexibly designed," writes Roberts. "No matter how solid you build that puppy, some trading partner will switch formats without telling you, some server will crash, or some nasty, messed up data will sneak past your data quality watchdogs and wreak havoc." Therefore, you've got to monitor for broken points before they become major pain points.
How you do it: Roberts advises you opt for a solution with automated monitoring that will alert you if something changes.
On the development front, Gutierrez suggests you thoroughly test any integration and use robust logs that will answer three questions:
Both pieces are good reads. I also recommend you check out Roberts' Twitter feed, which frequently focuses on data integration.