Testing is one area of SOA that you simply cannot underfund or underestimate. SOA will require a complete overhaul of your IT testing process - from whom you include in the testing, to when it starts, to what it covers.
In fact, after reading this white paper on SOA Test Methodology from Torry Harris Business Solutions, I'm willing to bet this is where most organizations will make their big SOA mistakes.
And believe me, I'm not into betting. In fact, I don't even like candy bar bingo. I figure, why give away what I know I have - be it a dollar or a Snickers bar - on a possibility? My friends think I missed my calling by not going into risk management.
Why do I think testing will prove to be a stumbling block? Well, first of all, testing is where application design typically falters - and that's when testing is a fairly straightforward process: You build an application and you test it.
But SOA requires a much more complex testing approach, (you say methodology, I say why spend a quarter when a good dime word will do?) because the possibilities are more complicated.
Here are the big reasons SOA changes testing:
- SOA services will not have a user interface. Generally, interfaces are one of the key test points for standard application development.
- Applications are fairly contained. There's one development team that builds one project and delivers it through a specific type of server and browser, and then maintains it. That's out the door. When you build an application with services, you could be using services design from multiple teams, even teams outside your company - and unless you work for a very small company that never upgrades, you'll need to design for cross-platform use.
- Since the use of services is never ending - unlike when you finish an application - you'll need to perform testing at all levels of development and deployment.
- Since the point of services is the agility - the quick reuse of the service in other places - teams will need to offer a guarantee of service quality.
- Since services may be crossing the firewall, your development teams will need to understand network security.
- Given the difference in how SOA services are deployed, you'll need to actively involve business - the managers and the users - throughout the project lifecycle.
This 21-page white paper doesn't just outline the problem - in fact, it spends a small amount of time on that. Instead, it focuses on how to structure your testing to address these problems.
It advocates a V test methodology model and explains how you'll need to expand and change your testing to accommodate SOA. It's vendor neutral, though it does list the main vendor and open source test tools for SOA environments at the end.https://o1.qnsr.com/log/p.gif?;n=203;c=204663295;s=11915;x=7936;f=201904081034270;u=j;z=TIMESTAMP;a=20410779;e=i
This is not an overnight fix. It's going to require retraining your staff, getting your application development people to talk - gasp! - to business users and managers on a regular basis, new tools and, yes, most likely more money.
But addressing testing now could save you months of trouble and make the difference between a successful or a mock-able SOA deployment.
You might want to read this older, but still useful article on testing from SOA Web Services Journal, SOA Calls for New Test Approach, for more information on the problem.