Is all open source software good? No, of course not. The pros of OSS are plentiful, but the cons are just as numerous. With often a myriad of competing OSS projects -- many not professionally supported -- little certainty around future releases, rapid updates requiring constant upkeep and buggy early versions, how can one justifiably take the risk? How do we mitigate these cons and filter the wheat from the chaff?

The following assessment criteria should start the evaluation of any open source product/platform:

• Investigate the licensing/legal situation of the software.
• Evaluate whether the software is standards compliant.
• Contact references to confirm product viability.
• Find products with a supporting or stable developer.
• Find products that use an open/industry standard implementation language.
• Reference third-party reviews of the software.
• Reference books published about the software (the more the better).
• Reference industry analysts, such as Gartner, Forrester or IDC.

In general, a mature open source platform will meet the following criteria:

1. Project extensions are available.
2. The project has reached a one-year maturity mark.
3. Security patches, bug fixes and new features/enhancements are delivered separately.
4. The core development team has rigid criteria for participation.
5. The software has reasonable automated unit and functional tests with code coverage in the 30 percent - 80 percent range.
6. The software easily integrates with external services.
7. The component’s bug database is kept up-to-date with revision numbers for each product enhancement.
8. The solution has been ported across multiple platforms (Linux, Windows, Solaris and Mac).
9. The community is organized into groups, each responsible for separate tasks (the maintainer, the documentation group, the development group, the evangelism group).
10. The project’s license is acknowledged by the Open Source Initiative.
11. Large-scale adoption, including both public and well-known large-scale organizational deployments exists.
12. There is separation of documentation: user documentation, installation documentation, admin documentation, and development documentation.

And remember to always weigh target usage of the application (mission-critical, departmental or beta) against the OSS product's readiness.

Many of these points were reiterated by our recent discovery of Business Readiness Rating. Now that's an example of an open source project that lost its legs, but what was left behind was some great info.