|

A user’s guide to user reviews

How to avoid the hype and get the most from Yelp, the Better Business Bureau, and other online review sites

Published: April 2014

Ninety percent of consumers say that they read online user reviews before they make buying decisions, and why not? What could be more helpful than the opinion of millions of Americans about the products and services they’ve tried? But what sounds great in theory seems a lot less valuable in reality, because the trustworthiness of user reviews is increasingly being called into question.

Good reviews can boost sales and profits; negative ones can do just the opposite. A one-star increase in user ratings on Yelp, for example, can boost revenues by 5 to 9 percent for a restaurant, according to a 2011 Harvard Business Review study. A similar jump on Travelocity and TripAdvisor can push a hotel’s room rates up 11 percent, Cornell University researchers say. That gives unscrupulous advertisers, businesses, and reviewers all the financial motive they need to game the system. Those fabrications “are the 21st century’s version of false advertising,” says Attorney General Eric Schneiderman of New York.

Given that reality, how can you wisely tap this often useful information? Here’s our advice.

1. Be skeptical

You don’t know whether the strangers who write the reviews have actually used the product and are telling the truth or whether they’re shills paid $1 to $10 per review, such as those uncovered in an investigation last year by Schneiderman’s office. It snared 19 companies that pumped out thousands of fake consumer reviews on such websites as Yelp, Google+ Local, Yahoo Local, CitySearch, and InsiderPages.com.

Also bear in mind that user reviews, whether bogus or real, tend to skew toward the positive. Researchers at the University of Texas in Dallas and California State University in Long Beach found that people who rented movies online were five times more likely to post a positive review than a negative one. About 67 percent of Yelp’s reviews for the last quarter of 2013 were four or five stars.

2. Don’t try to guess which reviews are true

Some well-intentioned consumer advice recommends that you try to be a truth detective and scrutinize reviews for signs of fakery. But when Cornell researchers asked undergraduates to determine which of the 800 user reviews for 20 Chicago hotels were phony and which were real, the students showed no ability to do so. One reason: People suffer from a “truth bias” that leads them to trust what they read—until they discover evidence to the contrary.

3. Don’t rely on reviewer reviews

Publishers of user reviews have come up with a variety of safeguards to prevent bogus write-ups. But we found them to be a mixed bag. When Nitin Jindal and Bing Liu of the University of Illinois in Chicago analyzed 5.8 million user reviews of electronics, books, music, and DVDs on Amazon.com, for example, they noticed something unusual. Many of them were from “trusted reviewers” who had written hundreds or thousands of them. The researchers concluded that such an output was “unlikely for an ordinary consumer.” They also found that if a product had only one review, it was likely to be fabricated, but that reviews of products with high sales volume were less likely to be made up. A representative from Yelp told us last year that its computer algorithm for filtering out false reviews had never been objectively tested. But Liu says that he tested it himself and found that people could “probably trust” the reviews that pass for publication.

4. Check the criteria

Root around sites with user reviews to find out exactly how the publishers manipulate, filter, and use them. OpenTable, for example, says that its reviews must “pass our standards and guidelines for publication.” The site also says that the user reviews aim to benefit diners “as well as our restaurant partners.” The partners are 31,000 restaurants across the U.S. that pay $190 million in fees for the review and booking services, and get to nominate a “featured review” for placement at the top of OpenTable’s ratings and review tab.

5. Look for verified standards

To get eBay’s Top Rated Plus seal, which pushes its best merchants to the top of its rankings, sellers must maintain a good track record with known users. They must also have a significant volume of sales and adhere to pro-consumer standards, including a 14-day money-back return policy and same-day or one-business-day shipping.

6. Rely most on objective evaluations

Your best sources of information are those based on scientifically sampled surveys, such as those produced by the nonprofit Consumers’ Checkbook, which rates local service companies in seven U.S. metropolitan areas, comparative product testing such as that done by Consumer Reports, and other objective measures, as you would find with ratings from the Better Business Bureau.

7. Use reviews for anecdotal wisdom

User reviews can provide helpful insight into product and service attributes that you might not have thought about on your own or that aren’t spelled out in product descriptions or photos. A user might be able to tell you that, say, the bed linens you’re considering aren’t the shade of white advertised but are really more of an ivory, or that the jacket you’re thinking about buying tends to run small, so you should order a size larger.

Editor's Note: This article also appeared in the May 2014 issue of Consumer Reports magazine.
   

E-mail Newsletters

FREE e-mail Newsletters!
Choose from cars, safety, health, and more!
Already signed-up?
Manage your newsletters here too.

Money News

Connect

and safety with
subscribers and fans

Follow us on:

Cars

Cars New Car Price Report
Find out what the dealers don't want you to know! Get dealer pricing information on a new car with the New Car Price Report.

Order Your Report

Mobile

Mobile Get Ratings on the go and compare
while you shop

Learn more