Why Yelp-Like Websites for the Enterprise Products Don’t Work

Rob Enderle
Slide Show

The State of Enterprise Crowdsourcing

I was on Amazon earlier this week and noticed that some people appear to be working overtime in an effort to keep people from reading the new book "Haunted Empire: Apple After Steve Jobs," which is about Apple post-Steve Jobs. When the book first came out, I was surprised to see five reviews on Amazon, all indicating that the book was horrid. (Since then, however, reviews have clearly trended more positive.) When I looked at the people who wrote the reviews, though, they seemed to either have never reviewed a book before or had never reviewed anything before, which was really suspicious.

My experience with Amazon book reviews dovetailed with concerns I’d heard about on Yelp, where folks have discovered that a large percentage of the reviews are from people who aren’t legitimate. And both are good examples to support the theory I read in an article on Forbes that suggests that enterprises can’t trust software review websites.

The Forbes article resonated with me, because I know that getting IT folks to write tech reviews is really difficult. For one, IT staff usually doesn’t have the time, plus IT skills and writing skills don’t often go hand-in-hand.

Yelp’s Issues

The problem with Yelp that the firm is desperately trying to correct is that too many reviewers fall into three categories.

  • Company shills, which range from employees to paid contributors who say nice things about the product or service but have never actually used it
  • Company enemies, which can include disgruntled employees, employees of competitors or unreasonable customers
  • Trolls, or people who just love to go on the Web and cause folks pain because they can

Yelp is trying really hard to better ensure the quality of reviews, but that has proven to be a very difficult task. The sad thing is that the real reviewers don’t have the writing skills of the fake ones, so often the best “read” is from a person who isn’t honest. Even after a significant effort to improve review quality, a recent report indicates that false Yelp reviews have jumped from 5 percent to 20 percent and this is probably an understatement, because it is very likely that some of the false reviews haven’t been caught.

Enterprise Review Websites

If this is bad news for Yelp, it is horrid for sites trying to provide similar review services for enterprise products. Unlike consumer services and products, where the use case is generally easy and comparatively similar, enterprise deployments are neither. For instance, you can have a great experience with a vendor or product when the vendor puts their best people on it and partially funds the deployment in order to get a customer blessing or to work out an early adopter problem. In addition, products are deployed very differently based on unique requirements of each company. Plans are dependent upon what the company needs to have integrated and also the training and quality of the team doing the deployment.

In short, even if you could get an honest review, you will find that your experience is very different from the reviews you’ve read unless your environment is nearly identical to the one you read about.

I do a lot of interviews with IT folks, but one stands out. It was years ago, during an event called Scalability Day, which was held to showcase that Microsoft’s Windows NT could scale to mainframe loads. The company that I called raved about the offering; however, under protracted questioning, they admitted that if they, rather than Microsoft, had paid for the deployment, it would have cost them three times what a mainframe would have cost. The product worked and the review was extremely positive and authentic, but the result had no relation to what would happen if another company had tried to emulate what had been done.

What I’ve found over the years is that the success of a product or service often has more to do with the quality of the vendor team, the readiness of the IT shop to accept the technology, the business type, and the experience of the IT team doing the deployment. These often trump the quality of the product or service, and yet these points aren’t typically captured in a review at a level that is granular enough for another company to emulate.

Wrapping Up: Compare Like to Like

The best solution to have true, online reviews for IT products would likely be to ensure the independence and knowledge of the reviewer and also to group reviews by geography, business type and technology mix. At least then you could get more of an apples-to-apples view, but you might not get enough data points to put your mind at ease. You could also, if your enterprise was large enough, engage other folks in the company that use products like the one you are researching. For products and services that are used widely, this process would provide validated reviews that would apply to your industry and business and that reflect better what your experience would likely be.

In the end, though, it will be difficult for consumer services of online review sites to maintain adequate quality. In fact, it will likely be impossible for a company to provide enterprise-level product reviews online without some kind of analytics tool that ensures quality, accuracy and applicability.

Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.


Add Comment      Leave a comment on this blog post
Mar 20, 2014 6:53 PM Rby22 Rby22  says:
Hi Rob, Great article! We at Casetree.com completely agree and have developed this site to provide real consumer reviews backed by legal lawsuits, hence eliminating fake reviews, posts, or monthly memberships. It will be an honor if you can check out the site and send us your feedback!! (www.casetree.com) Reply
Mar 20, 2014 7:20 PM HateYelp HateYelp  says:
If anybody is going to publish something about someone's livelihood that may potentially be damaging the complaint must first be seen and arbitrated by actual real human person eyes and brain, the system the BBB uses always has been the best system, they cut through all the crap and will only ask a business to rectify real situations where the customer was not treated fairly by the business and the business must address it or their rating will be hurt. I can agree with this. But obviously There is no electronic lazy mans algorithm that can possibly figure out the complexity of human communication, I wish all these sites would give up the notion that algorithms work for this purpose, this practice should be made illegal due to all the financial damage inflicted, its painfully obvious by the massive feedback that they are doing more harm than good and algorithms for this purpose simply don't work. Reply
May 14, 2014 5:39 PM supersnapp supersnapp  says: in response to HateYelp
I am not sure about this comment that BBB does it correctly. BBB is something that companies like to talk about being a member of, but BBB is far too producer friendly. So as a result BBB is not all that relevant for consumers. This is a problem when the entity is captured by the group it is reviewing. If you are a company, then the BBB sounds like a great deal -- and then the BBB is built up as a reliable source when in fact they are a captured entity. Yelp is not perfect, but many of their reviews or most of their reviews are legitimate. You get information you could get no where else but on Yelp. When I use Yelp, I end up being more satisfied than if I do not use it. Company webpages are not going to tell you anything but the positive. Furthermore, disgruntled employees have just a legitimate opinion as do those that had a great experience -- by calling them disgruntled, we apply a test -- that is if the review is positive, then its legitimate. If its negative, we apply some ad hominem label to it, and that way we can make it illegitimate, without providing any evidence to the contrary. Reply
Nov 23, 2016 11:01 AM Simplicity Simplicity  says:
Hello everybody, review websites are not perfect and yet the best solution to get at least a gut feeling if a product or service fit your needs and expectations. However, I agree that ensuring a good review quality is not an easy job which in my oppinion depends much more on building a feedback culture than on analytic tools. Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.




Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.

Subscribe Daily Edge Newsletters

Sign up now and get the best business technology insights direct to your inbox.