Testing is one area of SOA that you simply cannot underfund or underestimate. SOA will require a complete overhaul of your IT testing process - from whom you include in the testing, to when it starts, to what it covers.
In fact, after reading this white paper on SOA Test Methodology from Torry Harris Business Solutions, I'm willing to bet this is where most organizations will make their big SOA mistakes.
And believe me, I'm not into betting. In fact, I don't even like candy bar bingo. I figure, why give away what I know I have - be it a dollar or a Snickers bar - on a possibility? My friends think I missed my calling by not going into risk management.
Why do I think testing will prove to be a stumbling block? Well, first of all, testing is where application design typically falters - and that's when testing is a fairly straightforward process: You build an application and you test it.
But SOA requires a much more complex testing approach, (you say methodology, I say why spend a quarter when a good dime word will do?) because the possibilities are more complicated.
Here are the big reasons SOA changes testing:
This 21-page white paper doesn't just outline the problem - in fact, it spends a small amount of time on that. Instead, it focuses on how to structure your testing to address these problems.
It advocates a V test methodology model and explains how you'll need to expand and change your testing to accommodate SOA. It's vendor neutral, though it does list the main vendor and open source test tools for SOA environments at the end.
This is not an overnight fix. It's going to require retraining your staff, getting your application development people to talk - gasp! - to business users and managers on a regular basis, new tools and, yes, most likely more money.
But addressing testing now could save you months of trouble and make the difference between a successful or a mock-able SOA deployment.
You might want to read this older, but still useful article on testing from SOA Web Services Journal, SOA Calls for New Test Approach, for more information on the problem.