Donate |

Critics chew over our diet Ratings

Consumer Reports News: May 18, 2011 06:04 PM

We’ve been doing diet Ratings since 2005, but this year’s installment—our third—is suddenly controversial. The main bone of contention: our inclusion of a 2010 study of our top-rated diet, Jenny Craig.

First, let’s acknowledge where the critics have a point. We should have prominently mentioned that participants in the 2010 study, which had a remarkably low dropout rate, didn’t have to pay for the prepackaged meals, snacks, and desserts that are the backbone of Jenny Craig’s program—a freebie that the investigators valued at $6,240 over the course of the two-year study. The monetary value of the free food and counseling in the Jenny Craig trial is indeed a lot higher than, say, the monetary value of the Slim-Fast products, or the cost of membership to Weight Watchers, which were also free to participants in clinical trials of those diets.

We didn’t delve into all this in our three-page article because freebies are standard for clinical trials. Participants don’t normally pay for the value of the products and services, such as drugs, medical devices, counseling, and examinations, that are part of the trial. And many trials, like the 2010 Jenny Craig study, are funded by industry (though run by outside researchers). But we took many dings for not pointing that out in our story (see this example from Salon) so we obviously should have mentioned it.

The "free food" diet?
But was it only the free food that put Jenny Craig on top of this year’s Ratings? The study’s authors state that the free food likely helped the Jenny Craig study achieve a dropout rate of just 8 percent at the end of two years.

We were curious to see whether, without the free-food thumb on the scale, Jenny Craig would have fared as well as it did in our Ratings. So we asked our statistics gurus to re-run our diet analysis, leveling the playing field by omitting the dropout rates of all the studies we used and focusing solely on the other aspects we considered: weight loss and nutritional quality. Jenny Craig still came out on top.

Weight Watchers objected that the Jenny Craig trial was not a “real-life scenario.” We agree. Clinical trials are more of a “best-case” scenario. They cost a fortune to run, and investigators typically do everything but stand on their heads to keep volunteers from quitting.

In our dream world, someone with several million dollars to spend would fund a clinical trial pitting all the major commercial diets head-to-head. But that hasn’t happened, so the best alternative is to use the randomized clinical trials that do exist, most of them studies of individual diets. These types of trials carefully select subjects (excluding, for instance, people who have medical conditions that would make dieting dangerous), randomly assign them to whatever diets are being studied or to a control group, and monitor them over the course of the study.

These clinical trials were the only type of medical study we considered in putting together this year’s Ratings. And to raise the bar still higher, we excluded any studies that hadn’t been published in full in a reputable peer-reviewed journal.

Real-world choices
Finally, the New York Times faulted us for ignoring an earlier study of what happens to average people who enroll in Jenny Craig. We did exclude this study, because it’s not a clinical trial like the ones we considered for the other plans, but a tracking study that observed what people naturally did over time. But it’s interesting nevertheless.

The study, which was published in 2007 in the International Journal of Obesity, reviewed the records of 60,164 people who voluntarily signed up for Jenny Craig over a one-year period and paid for everything themselves.

The dropout rate was high: 27 percent had quit after a month, and only 6.6 percent lasted a full year. But among those stalwarts, weight loss after a year was significant: The women lost an average of about 16 percent of their starting weight, around 34 pounds. And men lost about 13 percent of their weight, or 37 pounds.

Those results were better than the weight loss in any of the clinical trials that went into our Ratings. Just to see what would happen if we had included this study in our calculations, our statistical specialists ran the numbers again using the dropout and weight-loss data from the 2007 study and assigning it to be 40 percent of Jenny Craig’s overall diet score. Again, Jenny Craig came out on top.

It would be great if we had such robust observational studies of other diets, but so far we don’t. Based on a brief abstract on its website, it looks like Weight Watchers has completed such a study of its customers in Germany. We look forward to reporting the results when and if they’re published in full.

Bottom line: No matter how we look at the numbers, Jenny Craig is still our winner, and still not for everyone. All of our research continues to point to the fact that the best diet for you is the one that you can follow for months or even years.

We’d love to hear about your experiences with these and other diets.

Nancy Metcalf

   

E-mail Newsletters

FREE e-mail Newsletters! Choose from safety, health, cars, and more!
Already signed-up?
Manage your newsletters here too.

Health News

Connect

and safety with
subscribers and fans

Follow us on:

Cars

Cars New Car Price Report
Find out what the dealers don't want you to know! Get dealer pricing information on a new car with the New Car Price Report.

Order Your Report

Mobile

Mobile Get Ratings on the go and compare
while you shop

Learn more