Learning from our mistake

	ONE STEP  We’ll add an extra level of scrutiny when we get unusual results.

On this Web site, you'll read an article that’s unusual for Consumer Reports. It’s about tests and results that were wrong.

In the February 2007 issue, we published a report on infant car seats that we later withdrew when we learned of problems with the test data. The details are presented in “How our car seat tests went wrong.”

We take the mistake very seriously, and so do our readers. Robert Gutierrez of Bedford, Texas, was one of many who wrote us asking for an explanation. “I will also be looking for a statement indicating how you intend to prevent future test procedures from such gross miscalculations,” he said.

We’re committed to correcting the mistake and preventing similar ones. Beginning now, here’s what we’ll do differently:

Confer more regularly with outside experts when we’re developing complex tests. The mistakes in car seat testing might have been avoided if we’d called on additional safety experts to advise us on new protocols.

In some cases, we already do that. We spent months consulting with independent experts and with manufacturers of headlights, bulbs, and vehicles to determine the best ways to test headlights. We know that outside expertise can be invaluable, and we’ll now regularly consult experts from academia, government, and industry when we’re contemplating complex new tests. That said, once we have valid protocols, we’ll continue to limit contact with manufacturers, government, and other agencies during testing and analysis to avoid compromising our independence.

Refine our procedures concerning outside labs. We occasionally use contract labs when we don’t have the equipment or special expertise in a given area. As a matter of course, we monitor and visit any lab that’s working for us. Where appropriate, we’ll now hire a consultant with expertise in the subject area to review procedures and results. If called for, we’ll retest at a second lab. On a related note, many readers said they were surprised that we tested car seats at an outside lab. While that fact was included in the report (albeit in a note below the Ratings), it isn’t always. In the future, we’ll tell you every time, and more prominently.

Redouble scrutiny when our findings are unusual. Whether we’re performing lab tests or investigating services, we’ll ratchet up our already formidable review process. Now, any report that calls a product Not Acceptable or that raises questions about an entire group of products must have my sign-off. In addition, when we get unusual results, we’ll step back, take a closer look at data on real-world experience, and do more to challenge our methodology and conclusions. Still other changes are forthcoming.

For 71 years, Consumer Reports has enjoyed the support of readers like you. With these changes and other advances, we are determined to continue serving your needs and earning your trust.