A recent study by IBM user group SHARE shows at least 50 percent of companies still use hand-coded scripts to move data from mainframes to other databases or platforms. The problem with this type of manual integration is the scrips are hard to maintain -- especially when programmers leave the organization, the study found.
This is no small problem, given that organizations with 10,000 or more employees report that anywhere from 51 percent to 75 percent of their organization's data is managed and stored on mainframes. Given that many legacy programmers are mere years from retirement, this issue is becoming the big elephant in the server room, don't you think?
I must say, I wasn't surprised to see that hand-coding stat. That very issue came up recently when I interviewed Philip Russom, an analyst with The Data Warehousing Institute, about data integration trends for this year.
We were sidetracked when Russom mentioned that data integration solutions are very mature -- 15 years of development will do that for a technology -- but then quickly added that "roughly half of data integration solutions are built from scratch with hand coding."
This surprised me, so I asked, "Why are they hand-coding if you can buy a solution?"
It turns out, Russom's been grappling with that question for some time.
In Russom's words, hand-coded data integration is a "fairly old and arcane practice." His case is simple: Data integration tools really came into their own about 2000, right after the big Y2K push. Data integration requires a senior-level programmer typically, he said, which means you'll be paying at least one programmer six figures to spend months coding data integration from scratch.
By comparison, you could build a comparable solution in two to three months with a vendor tool for less money, he said.
I dryly asked if there was a trend for people to figure that out.
Russom stammered a bit, saying:
"Oh boy, well you've touched a nerve here. You know, I've just gotten tired of explaining this to people for about the last eight years now."
Whenever he explains his case, people tell him: Wow, I get the economics. Wow, thank you for enlightening me. BUT we still have this culture of hand-coding.
And, if I may be so bold, apparently they also have a culture of hard-headedness.
So I can see why many hope service-oriented architecture will be able to shake things up a bit. According to the study, about 23 percent of respondents reported their company is involved with an SOA project, with an additional third in the planning or review stages of SOA.
Certainly, vendors offering SOA solutions are looking at legacy systems. Software vendors are releasing adapters based on the Web Services Interoperability and SOAP Simple Object Access Protocol, according to a recent CIO Today article, "Legacy Data Systems: Forklift or Facelift?"
The article is a great resource for learning how old systems can be given new life through Web tools and SOA, as well as when you should simply pull the plug on legacy systems.
SOA is really about a systemic change in the way we do IT, and not really a 'special project.'
If you work in an IT divisions that's done things the same way for 40 years, SOA is your opportunity to revisit not only how the business works, but how your IT organization functions. Take a clue from Russom and start by evaluating how your programmers are spending their time.