Open Source Does Not Assure Quality

Rob Enderle

You may be surprised to know that I never intended to be as controversial as I once was but was thrust into the spotlight when my very first column, initially titled, "Making Measured Decisions," was renamed by my editor "Linux is Not Ready for the Enterprise." To this day, I'm amazed at how many people criticized me without reading what I wrote.


The piece was intended to be a discussion of what folks should think about, and it used a column I'd intended for another purpose as its core. I was concerned at the time that the hype was leading the product (this was five years ago) and wanted folks to think a bit more about what they were doing. This lack of thinking seems to happen more often when something is new and trendy. I'd done something similar when Windows NT was new, but with a vastly different reaction.


The result of the column was that I received threats on my job and even my life. This got me really angry; I saw it as a freedom of speech issue and, for awhile, went to war. In some ways, I'm kind of proud of this because I put my personal safety and even my income needs aside, but I was also letting people manipulate me. That, for someone of my experience, is incredibly stupid.


I say this all as a preamble because I don't intend to declare war on open source again and honestly feel it fills a unique need in the marketplace, but as it is with proprietary products, often the hype exceeds the reality. I'm not saying that open source means bad quality; I'm saying that the only way to assure quality is to actually do the work. Open source is not an assurance of quality.


Assuring Quality


The problem, for me, with this idea that open source has higher quality is that I've worked inside one of the largest software companies in the world. I know, for a fact, that even though we clearly had access to our own source code, we were often surprised by what people did with that code -- things that they weren't supposed to. I also watched products that were supposed to be fully tested and quality assured go out the door, bypassing many of these tests in order to hit dates. I haven't seen anything with open source that makes these problems go away.


Firefox has had a number of problems that match anything Opera, Safari or IE have had. I just got off a flight to Germany where the entertainment system was crashing every other hour and watched folks blame the crashes on Windows, even though the actual platform was Red Hat Linux.


My biggest "ah hah" moment, however, was back at IBM, where I watched executive management disband Quality Assurance for Software because "they already knew what the problems were" and decided that quality should be the responsibility of the group building the product. The result was a massive reduction in both quality and customer satisfaction, and the termination of the executives who made that ill-fated decision.


My conclusion is that people don't like to do quality assurance work and open source doesn't change people. It is tedious, doesn't invite much praise, and generally results in someone else getting the credit. So, open source or not, unless someone is tasked and measured on assuring the quality of something, the resulting quality will be questionable. Successful open source companies, like Red Hat, or Novell, I contend, have a quality assurance program that looks a lot like what a proprietary company has. That is how they maintain quality. It isn't because their customers are taking on any more of this load then they otherwise would.




Let me use an analogy that I see in Wikipedia. Unlike code, the average person can read and understand Wikipedia entries. With code, it generally takes someone who understands how to code, and even old coders like me, who are woefully out of date, have to depend on others to help them decipher what they're looking at. Most of us, and I include myself, couldn't point out bad code if it bit us. But we can figure out whether a bio of a public person or our own bio is accurate. I've been watching my Wikipedia biography morph around over the last several years. At one time, it had me married to and divorced from someone I'd never met, and that evidently was redacted without leaving a trace after I wrote about it. I recently went to the Wikipedia folks to complain. They were kind enough to stub the thing out so that it could be properly rebuilt but the process made me wonder, particularly after I looked at the bios of a number of CEOs, just how much money some people are spending to ensure the information about them is positive. I'll bet if we aggregated it, the money being spent runs into the triple digit millions. In other words, the quality assurance effort for Wikipedia may actually exceed (at least for the targeted bios) the entire annual budget for a normal encyclopedia by several times.


Wikipedia has a strong policy on this but it appears, at least in my instance, that few read or followed it.


For instance, if you go to the biographies of either of the presidential candidates, Obama or McCain, you will see that both men are perfect (I'll bet you disagree with at least one). Neither has apparently made any mistakes and, even though they are public officials who have been in nasty campaigns, there is no mention of the nasty comments that they have made or that have been made about them. Nobody is this perfect, not even your own candidate. Given how many people look at these bios, you have to wonder how much it is costing each campaign to maintain these bios in this pristine fashion per day. You have to believe a lot of people on both sides would love to edit these things and probably try on a regular basis (unless it has been blocked).


By the way, you may want to make sure someone who doesn't like you, your spouse, or one of your kids hasn't decided to take exception to them in Wikipedia. This could actually be rather damaging to someone, if timed right.


Wrapping Up


I maintain that if something that is this public and can be read by anyone can be so easily subverted, then for code it is even more easily done. This means for those who think open source is some kind of quality assurance in and of itself, they are mistaken. Open source products only mean people can take an extra quality step -- it doesn't mean they do. If you want this extra quality benefit, then the buyer must resource the quality assurance effort. If they don't, they need to assure themselves that the quality assurance effort the vendor takes is adequate, just as they would any other product.


Open source does not change the rule in the book Japan Inc. that says someone has to own quality or no one owns it. Or, if you don't know who owns quality there is a chance no one does.


One final thought, it is clear a lot of people have spent a lot of money assuring their version of quality in Wikipedia, and this free product seems to have a substantial hidden cost. Something to think about while you check and assure the quality of your own free open source Wikipedia entry.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.