|

How We Test

Overview

Welcome to Consumer Reports, the largest independent consumer-testing organization in the world, where hundreds of people work every day to test products, inform you of the results, and protect your consumer interests. In our dozens of state-of-the-art labs, we test everything from air conditioners to ziti.

Appliances & Home

Photo: Whirpool

Readers of Consumer Reports and ConsumerReports.org have plenty to say—every year, thousands of our readers call, write, or e-mail our customer-relations department with comments and questions about the thousands of products we test each year. In the home-and-garden area, they wondered about refrigerators that don't keep food cold, dishwashers that drown out conversations, and funny-looking twisted light bulbs that claim to save energy.

The occasional rhetorical question arrives in our inboxes, as in "What the heck were you thinking?" On our appliances and home products testing page we address some of the common inquiries we get about how we review products for the home.

 

Cars

Consumer Reports operates the largest and most sophisticated independent automobile testing center devoted to the consumer interest anywhere in the world. Situated on 327 acres in rural Connecticut, the Consumer Reports Auto Test Center is home to more than 20 staff members, including automotive engineers, technicians, and support staff. Consumer Reports buys, anonymously, all the cars it tests, about 80 per year, and drives each for thousands of miles.

Formal testing is done at the track and on surrounding public roads. The evaluation regimen consists of more than 50 individual tests. Some are objective, instrumented track tests using state-of-the-art electronic gear that yield empirical findings. Some are subjective evaluations-jury tests done by the experienced engineering staff. These videos will provide further insights into the ways that Consumer Reports evaluates new cars to help its readers make smart, informed choices. (Watch our car-review videos.) See our Guide to Consumer Reports Ratings.

Electronics

When it comes to testing products, Consumer Reports stands alone. You might know us best for our Ratings, which enable you to compare the performance of many models at a glance. Here's a behind-the-scenes look at the unique testing process that helps you get the most for your money.

How we test
Products are tested by engineers and technicians with years and sometimes decades of expertise in their field. They live with the products for several weeks, putting them through a battery of objective tests using scientific measurements, along with subjective tests that replicate the user experience. We test products against existing industry or government standards and develop our own benchmarks when we encounter new technologies or issues that require further testing. All models within a category go through exactly the same tests, side by side, so they're judged on a level playing field, and test results can be compared.

Testers focus on a product's primary function (evaluating image quality for TVs and cameras, for example) and some secondary functions, too (sound quality for TVs or photos shot with a cell-phone camera). They evaluate whether features add to usability or just make an item more complicated. They check ergonomic functions (how intuitive controls are and how comfortable a keyboard is) and, where appropriate, consider battery life, speed, and other attributes.

We use a product as any consumer would. For example, we assess how long a laptop computer's battery will last when running everyday applications, such as word processing and photo editing, or how quickly a digital camera can shoot photos at a fast-moving soccer game.

Visit our electronics-testing page for details on what, when, and where we test and what we report on our findings.

Food & Sensory

Many of Consumer Reports tests involve the use of sensitive instruments. A liquid chromatograph determines how much caffeine is in coffee, and an atomic absorption spectrophotometer determines the amount of heavy metals in plastics and toys. A digital photometer measures light and color of TV displays. To evaluate a food's nutrition, we use sophisticated laboratory instruments. But how do we evaluate its sensory quality—the characteristics of its ingredients, the balance of its flavors? To do this, we use a very sensitive instrument called the human palate.

Consumer Reports has a panel of people who have been carefully screened. During their initial interview, they let us know, among other things, whether they're willing to eat foods they dislike (Visitors to Consumer Reports invariably volunteer to taste ice cream, but our testers are also required to taste buttery spreads—straight-up.) The people we hire have normal taste and odor acuity, but they've also shown the ability to recall and identify various flavors and textures, and to communicate—in precise terms—what their taste buds are telling them.

Our in-house sensory experts, who have studied food science, nutrition, statistics, and psychology, give the panelists basic training in evaluating foods. The tasters learn that personal preference must play no part in taste testing and that they should ignore irrelevant cues like color, which can make a bright red sauce seem more tomatoey than a dull orange sauce, even when it's not. Some food categories—such as wine—require a specific expertise or knowledge, so experts in that category are used and they adhere to the same testing principles as our in-house panel.

Visit our food-and-sensory testing page for more details.

Health

Consumer Reports rates many different kinds of health-care products and services, including:

  • Exercise equipment, such as ellipticals, treadmills, pedometers, and home gyms.
  • Home medical devices, such as blood-pressure monitors, blood-glucose meters, and fever thermometers.
  • Personal-care items, such as sunscreens, moisturizers, electric shavers, electric toothbrushes, and bathroom scales.
  • Drugs, including prescription and over-the-counter medication.
  • Health-insurance plans, including HMO, PPOs, and Medicare Advantage plans.
  • Hospitals, on measures such as safety, infection rates, and patient satisfaction.
  • Heart-surgery groups, based on success and complication rates for heart-bypass surgery.
  • Medical tests and treatments, such as screening tests for heart disease.

In some cases, such as in our Ratings of exercise equipment and home medical devices, most of our testing is done in our Yonkers facility, with our in-house staff of exercise and medical experts, engineers, and technicians.

In other cases, such as for drugs, health insurance, hospitals, and doctors, our Ratings are based mostly on data that we get from external sources, such as federal and state databases or partner organizations, as well as from our own surveys, which are conducted by the Consumer Reports National Research Center. That data is then analyzed by our in-house physicians and health statisticians and presented by our health writers and editors.

Visit our how we rate health-care products and services page for more details.