Open Source Does Not Assure Quality

Rob Enderle

You may be surprised to know that I never intended to be as controversial as I once was but was thrust into the spotlight when my very first column, initially titled, "Making Measured Decisions," was renamed by my editor "Linux is Not Ready for the Enterprise." To this day, I'm amazed at how many people criticized me without reading what I wrote.

 

The piece was intended to be a discussion of what folks should think about, and it used a column I'd intended for another purpose as its core. I was concerned at the time that the hype was leading the product (this was five years ago) and wanted folks to think a bit more about what they were doing. This lack of thinking seems to happen more often when something is new and trendy. I'd done something similar when Windows NT was new, but with a vastly different reaction.

 

The result of the column was that I received threats on my job and even my life. This got me really angry; I saw it as a freedom of speech issue and, for awhile, went to war. In some ways, I'm kind of proud of this because I put my personal safety and even my income needs aside, but I was also letting people manipulate me. That, for someone of my experience, is incredibly stupid.

 

I say this all as a preamble because I don't intend to declare war on open source again and honestly feel it fills a unique need in the marketplace, but as it is with proprietary products, often the hype exceeds the reality. I'm not saying that open source means bad quality; I'm saying that the only way to assure quality is to actually do the work. Open source is not an assurance of quality.

 

Assuring Quality


 

The problem, for me, with this idea that open source has higher quality is that I've worked inside one of the largest software companies in the world. I know, for a fact, that even though we clearly had access to our own source code, we were often surprised by what people did with that code -- things that they weren't supposed to. I also watched products that were supposed to be fully tested and quality assured go out the door, bypassing many of these tests in order to hit dates. I haven't seen anything with open source that makes these problems go away.

 

Firefox has had a number of problems that match anything Opera, Safari or IE have had. I just got off a flight to Germany where the entertainment system was crashing every other hour and watched folks blame the crashes on Windows, even though the actual platform was Red Hat Linux.

 

My biggest "ah hah" moment, however, was back at IBM, where I watched executive management disband Quality Assurance for Software because "they already knew what the problems were" and decided that quality should be the responsibility of the group building the product. The result was a massive reduction in both quality and customer satisfaction, and the termination of the executives who made that ill-fated decision.

 

My conclusion is that people don't like to do quality assurance work and open source doesn't change people. It is tedious, doesn't invite much praise, and generally results in someone else getting the credit. So, open source or not, unless someone is tasked and measured on assuring the quality of something, the resulting quality will be questionable. Successful open source companies, like Red Hat, or Novell, I contend, have a quality assurance program that looks a lot like what a proprietary company has. That is how they maintain quality. It isn't because their customers are taking on any more of this load then they otherwise would.

 

Wikipedia

 

Let me use an analogy that I see in Wikipedia. Unlike code, the average person can read and understand Wikipedia entries. With code, it generally takes someone who understands how to code, and even old coders like me, who are woefully out of date, have to depend on others to help them decipher what they're looking at. Most of us, and I include myself, couldn't point out bad code if it bit us. But we can figure out whether a bio of a public person or our own bio is accurate. I've been watching my Wikipedia biography morph around over the last several years. At one time, it had me married to and divorced from someone I'd never met, and that evidently was redacted without leaving a trace after I wrote about it. I recently went to the Wikipedia folks to complain. They were kind enough to stub the thing out so that it could be properly rebuilt but the process made me wonder, particularly after I looked at the bios of a number of CEOs, just how much money some people are spending to ensure the information about them is positive. I'll bet if we aggregated it, the money being spent runs into the triple digit millions. In other words, the quality assurance effort for Wikipedia may actually exceed (at least for the targeted bios) the entire annual budget for a normal encyclopedia by several times.

 

Wikipedia has a strong policy on this but it appears, at least in my instance, that few read or followed it.

 

For instance, if you go to the biographies of either of the presidential candidates, Obama or McCain, you will see that both men are perfect (I'll bet you disagree with at least one). Neither has apparently made any mistakes and, even though they are public officials who have been in nasty campaigns, there is no mention of the nasty comments that they have made or that have been made about them. Nobody is this perfect, not even your own candidate. Given how many people look at these bios, you have to wonder how much it is costing each campaign to maintain these bios in this pristine fashion per day. You have to believe a lot of people on both sides would love to edit these things and probably try on a regular basis (unless it has been blocked).

 

By the way, you may want to make sure someone who doesn't like you, your spouse, or one of your kids hasn't decided to take exception to them in Wikipedia. This could actually be rather damaging to someone, if timed right.

 

Wrapping Up

 

I maintain that if something that is this public and can be read by anyone can be so easily subverted, then for code it is even more easily done. This means for those who think open source is some kind of quality assurance in and of itself, they are mistaken. Open source products only mean people can take an extra quality step -- it doesn't mean they do. If you want this extra quality benefit, then the buyer must resource the quality assurance effort. If they don't, they need to assure themselves that the quality assurance effort the vendor takes is adequate, just as they would any other product.

 

Open source does not change the rule in the book Japan Inc. that says someone has to own quality or no one owns it. Or, if you don't know who owns quality there is a chance no one does.

 

One final thought, it is clear a lot of people have spent a lot of money assuring their version of quality in Wikipedia, and this free product seems to have a substantial hidden cost. Something to think about while you check and assure the quality of your own free open source Wikipedia entry.



Add Comment      Leave a comment on this blog post

Sep 24, 2008 1:02 AM Nihiltres Nihiltres  says:

" the process made me wonder, particularly after I looked at the bios of a number of CEOs, just how much money some people are spending to ensure the information about them is positive. Ill bet if we aggregated it, the money being spent runs into the triple digit millions."If this is the case, where is that money going? It's certainly not going to the Wikimedia Foundation, which scrapes by in the lower single-digit millions with around half of the budget devoted solely to technical maintenance and development. Either a) people are spending to hire people to fix up their entry or b) your statement is nonsense. It makes much more sense to apply Occam's razor and assume only that Wikipedia's biographies of living persons policy (intended to prevent libel) is being followed too carefully, rather than making the assumptions that Wikipedia is so important to many CEOs that they are willing to pay thousands to keep it looking slightly nicer than it would otherwise, and that these editors-for-pay are not being found out and banned from the project.Further, I find your Obama & McCain argument to be particularly weak: those articles are exceptional in that they are the focus of much controversy and media attention. This is besides the obvious point that the articles do include "mistakes" on the part of the men: read those articles carefully and you might realize that criticism is included objectively as those facts which are known, rather than by specifically saying "X is a bad man because he did Y" or some similar statement. For example, the Obama article says that "he used marijuana, cocaine, and alcohol", and the McCain one that 'The Arizona Republic would write that the McCain-Bush primary contest in South Carolina "has entered national political lore as a low-water mark in presidential campaigns "' (referring to sleaziness). Each merely raises the facts of the matter and some of what has been said about it, and allows readers to make their own judgements. If you think that the articles portray these men as "perfect", you are probably lacking reading skills or at least reading lazilyI found those examples within seconds while skimming the articles.I don't understand how you can find that "it is clear that a lot of people have spent a lot of money assuring their version of quality in Wikipedia" without any concrete evidence, as far as I can tell. Please, if you're going to make a point, try to use examples that actually illustrate it.

Reply
Sep 26, 2008 3:53 AM Rob Enderle Rob Enderle  says:
My point is the money is going to assure the BIO is accurate and it is paid by the person the BIO is about. My own experience is a case in point. I had a number of friends spend hours trying to fix the errors in my own BIO or, at least make it BIO and not a rant against me. I'm not a political figure it must be very expensive to protect their BIOs. How about CEOs who have pissed ex-employees, or anyone that has said anything negative about Apple. Go back into archives and see what my BIO went through and then we can talk. Reply
Oct 3, 2008 2:44 AM Ken Holmes Ken Holmes  says:
Rob, I agree that open source doesn't automatically assure quality. And, as you alluded to, neither does closed source. Open source permits much greater access and opens possibilities. Of course, it matters who takes advantage of that access and what their goals and level of expertise are.You have written before about Wikipedia and open source. I must respectfully disagree with you. Wikipedia is not open source in the way that code can be open source. Wikipedia and word processors are software that are the result of coding. However, you are unlikely to compare the quality of the word processor with the writings someone cranks out using that word processor. The same is true for Wikipedia. The issues of quality are akin to those of news papers and magazines. Even journalists, from time to time, have been found to fabricate stories. I believe the level of published writings and published code different.You have also written previously about Red Hat on an airliner. It does sadden me that the issue has not been resolved. Still, I don't think this is a problem of open source code. I believe some person or persons have failed to research the issue and deal with it. That, whether open or closed source, is a matter of humans rather than machines and machine language.Open source is another model of development. There is no perfect model. Humans are all imperfect and some are just mean or stupid, e.g. authoring and spreading malware and spam.I have been around long enough to know that (as in politics) there is no shortage of "lies, damn lies and statistics" in business and definitely in the computer/software realm. Caveat Emptor.Best to you, as always.Ken Reply
Oct 8, 2008 8:26 AM Rob Enderle Rob Enderle  says:
The lay person can't read code and many seem to think the "many eyes = high quality thing is true. I just want to make sure people are assuring a true quality process like both Netscape and Red Hat actually have. Reply
Oct 10, 2008 5:39 AM Ken Holmes Ken Holmes  says:
Touche! I must add OpenSuse and Novell SLED and SLES to your list. But yes, "too many cooks spoil the broth". Reply

Post a comment

 

 

 

 


(Maximum characters: 1200). You have 1200 characters left.

 

 

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


 

Resource centers

Business Intelligence

Business performance information for strategic and operational decision-making

SOA

SOA uses interoperable services grouped around business processes to ease data integration

Data Warehousing

Data warehousing helps companies make sense of their operational data