How do we gather the data?

Every year, we send our Annual Questionnaire to all Consumer Reports subscribers. This is one of the largest consumer surveys in the world. Some subscribers receive a print version in the mail, others receive an e-mail invitation. We verify that no one sends more than one questionnaire and that all responses are sent from the appropriate mail or e-mail address. A team of social scientists analyzes the hundreds of thousands of responses and develops the brand repair indexes that are reported in our publications.

What is a meaningful difference between brand repair rates?

Elementary statistics ensure that, given our often huge sample sizes, very small differences between brands can be statistically significant. A stated meaningful difference in our repair histories means that all such differences (or more) are statistically significant. Our standard for statistical significance is p<.01. In other words, the probability that differences in brand repair rates reported as meaningful were simply a function of chance is only 1 out of 100.

Could brand share impact on brand repair rates?

No. Provided that a brand attains adequate sample size, the size of a brand's share in the market has no impact at all on the reported repair rate. Our minimum sample size per brand is 100 cases for each full year in the analysis. Most analyses cover at least 3 years of historical data.

What do we mean by eliminating differences solely due to age and usage?

The published repair index reflects a statistical standardization that controls for the effects of age (and usually) usage on failure rates. We have found that the percent of models that have ever failed generally increases with both age and usage. Since brands in our survey had different age and usage profiles, we standardized data by applying a constant set of weights to repair rates for each brand year and usage category. The resulting index can be interpreted as the percent of models in the analysis that had ever been repaired or had a serious problem that was not repaired -- standardized by the typical age and usage distribution.

How should a reader interpret our stated caution that models within a brand may vary, and changes in design or manufacture may affect reliability?

The repair index is both brand-specific and historical. For example, even within a brand, one model may have contained a specific repair-prone component, while another model may not have. Moreover, a given manufacturer may decide to either redesign or outsource components. Still, although we carefully recalibrate our repair indexes each year and look for trends in the data, we have found that, in general, brands that have been reliable in the past tend to be reliable in the future. So, over the years we have found that using the repair history does increase the chances that our readers' next purchase will be a reliable model.

Is it correct to compare repair rates from one product to another?

We advise against doing this. For example, the data we use to create a brand repair rate for 30- to 36-inch TVs includes proportionately more new models than the data for 26- and 27-inch TVs. Also, some products have far greater usage rates than others (tractor mowers versus walk-behind mowers, for example. When repair rates are directly comparable, we include them in the same chart, or sets of charts (such as the various configurations of refrigerators).