Here’s a nice essay about the Women’s Health Initiative, a nine-year mega-million-dollar experiment to measure the effect of “healthy eating” especially a low-fat diet.
48,835 postmenopausal women . . . were randomly assigned . . . to either their regular unrestricted diet or to a “healthy” diet that was low-fat (20% fat) and high fiber, with at least 5 servings of fruits and vegetables, and 6 servings of grains a day. The “healthy” eaters endured an “intense behavioral modification program by specially trained and certified professionals” to keep them on their diets. While they backslide a little, they did surprisingly well in sticking to the diet — as good as dietary prescripts will ever get and money can buy — at a cost of $8,498 spent per person!
Oops, no effect. “The results of this huge study, despite the hundreds of millions of dollars of taxpayer money spent on it, were quietly buried.”
I conclude two things: 1. People in charge of spending vast sums on nutrition research don’t know very much about what constitutes a healthy diet. 2. The same people know very little about how to do experiments. The most basic lesson is to do the smallest experiment possible.
Sandy Szwarc, the author of this essay, concludes:
When we enjoy a variety of foods from all of the food groups — as most everyone naturally does when they’re not trying to control their eating — and trust our bodies, we’ll get the nutrients we need to prevent deficiencies. And that is the only thing that nutritional science can credibly support.
There is some truth to this, both (a) we instinctively eat to avoid certain deficiencies and (b) nutrition science has found conclusive evidence that we need certain chemicals. But she is quite wrong in the sense that most Americans appear to suffer from huge omega-3 deficiencies (my posts about this). Many of them, probably most of them “enjoy a variety of foods from all of the food groups.”
Thanks to Dave Lull.
Another advantage of small studies is that the political power of the investigators and other vested interests in any given study is likely to be lower. That would make the spinning of results more difficult.
Interesting comment, David. Along similar lines, smaller studies would allow more even-handed interpretation of the results, since results opposite to expectations would be less embarrassing.
“The results of this huge study, despite the hundreds of millions of dollars of taxpayer money spent on it, were quietly buried.”
please, the paper was published in JAMA and covered in the new york times, for pete’s sake! its hard for a scientific article to get less buried than that.
Rich, coverage in the NY Times (once?) doesn’t make your point. Was it covered on the Today show? Good Morning America? Oprah? A hundred other places?
How much coverage did it deserve? Well, the notion that Americans have been given poor nutrition advice for a long time is very important — had the advice been better, lots of lives would be better. I think the results deserved lots of discussion about “where we went wrong” or “where the experts went wrong” or “where the funding system went wrong” — whatever you want to call it. There was none of that.
“When we enjoy a variety of foods from all of the food groups — as most everyone naturally does when they’re not trying to control their eating — and trust our bodies, we’ll get the nutrients we need to prevent deficiencies.”
There is something pleasantly Jane Jacobs like or Hayekian in that statement (very anti-central planning/anti-expertise).
But sadly, in my personal experience, if I let myself spontaneously choose what I want to eat, I’ll eat very unhealthy.
Unhealthy foods exist as a result of engineering and mass production.
If they didn’t exist, we would be able to trust our bodies and eat well.
That’s why they shouldn’t exist.
Seth,
The original quote was that the results were “quietly buried.” you may believe the results deserved wider coverage, but that’s quite different than suggesting they were “quietly buried.” I do think publication in one of the pre-eminent medical journals and coverage in a pre-eminent newspapers is ample evidence against quiet burial.
Reasonable point. I quoted too selectively and misled you about what the author meant. You are right, by one meaning of “quietly buried” they were not. The author meant they received far less coverage than they would have had they come out differently. She meant “by comparison, they were quietly buried.” That isn’t clear from the single sentence I quoted.
Does anyone know why they chose 49,000 participants, which is an astoundingly large number? It seems that kind of number would pop out of a power calculation if you expected the effect to be very small and/or the background variation to be very large. If its the former, it makes the interesting point that the study cost will be decreasing in the magnitude of the effect you expect to find. This isn’t to say that some very small effects might have a big social benefit and that you would want to know about them, but it is still a perverse property of research!
Great point David.