Epidemiology has lots of critics. In this article, for example, it is called “lying on a grand scale.” Every critique I have read has ignored history. Epidemiologists have been right about two major issues: 1. Heavy smoking causes lung cancer. 2. Folate deficiency causes birth defects. In both cases, the first evidence was epidemiological. Another example is John Snow’s conclusion about the value of clean water. In my experience, epidemiologists often overstate the strength of their evidence (as do most of us) but overstatement is quite different from having nothing worth saying.
Let’s look at an example. Many people think osteoporosis is due to lack of calcium. Bones are made of calcium, right? The epidemiology of hip fractures is clear. In spite of the conventional idea, the rate of hip fracture has been highest in places where people eat a lot of calcium, such as Sweden, and lowest in places where they eat little, such as Hong Kong. (For example.) In other words, the epidemiology flatly contradicted the conventional idea. This was apparently ignored by nutrition experts (everyone knows correlation does not equal causation) who advised millions of people, especially women, to take calcium supplements to avoid osteoporosis. Millions of people followed (and follow) that advice.
Thanks to a recent meta-analysis we now know that experiments and better data firmly support the earlier epidemiology, which suggested that calcium supplements are dangerous. Here are its main conclusions:
In meta-analyses of placebo controlled trials of calcium or calcium and vitamin D, complete trial-level data were available for 28,072 participants from eight trials of calcium supplements and the WHI CaD participants not taking personal calcium supplements. . . .Calcium or calcium and vitamin D increased the risk of myocardial infarction (relative risk 1.24 (1.07 to 1.45), P = 0.004) and the composite of myocardial infarction or stroke (1.15 (1.03 to 1.27), P = 0.009). . . . A reassessment of the role of calcium supplements in osteoporosis management is warranted.
If the epidemiology had been taken more seriously, many heart attacks might have been avoided.
Is this an “anecdote” — a single example — proving nothing? Here’s how you can check. Randomly select a meta-analysis of epidemiological studies. Thousands have been done. Then ask if the results summarized in the meta-analysis appear random. Better yet, randomly pick two meta-analyses. Suppose the first summarizes 5 studies and the second summarizes 6. If the 11 results were shuffled together, how well could you assign them correctly?
I think epidemiology is a field of study that is particularly ill suited to the modern research environment. The pressure to continuously publish results, especially results pertinent to public health policy, produces mass false positives. Government policy can help entrench these false positives. This is a “boy who cried wolf” situation. People will ignore important results like the calcium result because they’re used to epidemiological results being false positives.
Hmm… One reason why there are more fractures in Sweden than in Hong Kong might be climate. More snow and ice in the winter?
While this isn’t directly relevant to the main point of your post, readers might be interested to know that current thinking leans toward the idea that both osteoporosis and arterial calcification are problems not of insufficient/excessive calcium, but of the body’s failure to distribute calcium to its proper places. Likely candidates for the driving factors include hyperinsulinemia and deficiencies of the fat-soluble vitamins D, K2, and A. To avoid osteoporosis or heart attacks, there seems to be a good case for supplementing D (or sunlight) and K2.
When your metabolism is putting calcium into the blood vessels instead of the bones, it seems like a reasonable hypothesis that more calcium would increase the risk of heart attack.
Regarding epidemiology: I think most of the correlations found by epidemiologists do not reflect direct causation; rather than A->B or B->A, they usually mean that both A and B have a common cause that was not measured. Unfortunately, many epidemiologists and nearly all readers of popular science articles have not grasped this.
Supposedly, most of the forms of Calcium supplements that people use actually remove calcium from the bones. Calcium citrate, etc.
Calcium carbonate (like from eggshells, oysters) works correctly though..
I have read that only 20% of epidemiological findings turn true in controlled experiments. 80% are refuted.
It is a question of the cost of different mistakes. + whether people can accept information with the idea that it has a very low probability of being true. Many people cannot handle informatiothat way
My dad has a nice batch of mustard greens in the garden. I’ll stick with them!
I am curious why you think this. Your belief seems much stronger than the available evidence warrants. “Most of the correlations” is a very large set. My impression is that most of the correlations discovered by epidemiologists are still unexplained — that is, we are not sure why they happened.
To think they suggest cause and effect simply means they make a cause-effect statement (e.g., A causes B) more plausible. Almost always they do. I see nothing wrong with epidemiologists pointing this out. It is also true that an A/B association makes a common cause explanation more plausible. Epidemiologists often point this out, too.
I agree (although I have no idea about the number of false positives). Is epidemiology worse than any other area of health research? I tend to think it is better because it is not distorted by vast amounts of drug company money. All researchers are under pressure to publish.
Gary Taubes had an interesting article about epidemiology and some of the problems associated with it:
https://www.nytimes.com/2007/09/16/magazine/16epidemiology-t.html?pagewanted=all
Thanks for the Taubes reference, AC: a very useful summary.
It’s good to do some research about things like calcium. Not only do you need calcium, but magnesium is important too, but most people do not know that they work together. People get enough calcium, but common calcium sources like milk have very little magnesium, so instead, people should focus on getting more magnesium to improve their bone strength.
If anyone would like to read more about this, here is an article that explains how to make the most from calcium: https://faevia.com/2012/02/getting-the-most-out-of-calcium/