Category: scientific method

Who Is Listened To? Science and Science Journalism

Seth Roberts6 Comments

This book review of Spillover by David Quammen is quite unfavorable about Laurie Garrett, the Pulitzer-Prize-winning science journalist. Several years ago, at the UC Berkeley journalism school, I heard her talk. During the question period, I made a comment something like this: “It seems to me there is kind of a conspiracy between the science journalist and the scientist. Both of them want the science to be more important than it really is. The scientist wants publicity. The science journalist wants their story on the front page. The effect is that things get exaggerated, this or that finding is claimed to be more important than it really is.” Garrett didn’t agree. She did not give a reason. This was interesting, since I thought my point was obviously true.

The book review, by Edward Hooper, author of The River, a book about the origin of AIDS, makes a more subtle point. It is about how he has been ignored.

When I wrote The River, I did my level best to interview each of the major living protagonists involved in the origins-of-AIDS debate. This amounted to well over 600 interviews, mostly of two hours or more, and about 500 of which were done face-to-face rather than down the phone. Although the authors of the three aforementioned books (Pepin, Timberg and Halperin, Nattrass) all devote time and several pages to The River, and to claims that I definitely got it wrong, not one of them bothered to contact me at any point – either to challenge my findings, or to ask me questions. However, I have been contacted by someone through my website (a lawyer and social scientist) who asked me several questions, to all of which I responded. Later, this man read the first two of these three pro-bushmeat books and contacted the authors of each by email, to ask them one or two simple questions about their dismissal of the OPV hypothesis [= the AIDS virus came from an oral polio vaccine]. His letters to Pepin, Timberg and Halperin (which he later forwarded to me) were courteous and non-confrontational, and in two instances he sent three separate letters, but apparently not one of the authors could be bothered to reply to any of these approaches.

In other words, there is a kind of moat. Inside the moat, are the respected people — the “real” scientists. Outside the moat are the crazy people, whom it is a good idea to ignore. Even if they have written a book on the topic. Hooper and those who agreed with him were outside the moat.

Hooper quotes Quammen:

“Hooper’s book was massive”, Quammen writes, “overwhelmingly detailed, seemingly reasonable, exhausting to plod through, but mesmerizing in its claims…”

I look forward to the day that the Shangri-La Diet is called “seemingly reasonable”. Quammen and Garrett (whose Coming Plague has yet to come) write about science for a living. I have a theory about their behavior. To acknowledge misaligned incentives (scientists, like journalists, care about other things than truth ) and power relationships (some scientists are in a position to censor other scientists and points of view they dislike) would make their jobs considerably harder. They are afraid of what would happen to them — would they be kicked out, placed on the other side of the moat? — if they took “crazy” views seriously. It is also time-consuming to take “crazy” views seriously (“massive . . . exhausting”). So they ignore them.

Elements of Personal Science

Seth Roberts4 Comments

To do personal science well, what should you learn?

Professional scientists learn how to do science mostly in graduate school, mostly by imitation, although they might take a statistics class. Personal scientists rarely have anyone to imitate, so have more need to understand basic principles. There are five skills/dimensions that matter. Here are a few comments about each one:

1. Motivation. In conventional science, the scientist does it as part of a job and subjects are paid. Neither works here: It isn’t a job and you can’t pay yourself. My original motivation was wanting to learn how to do experiments (for my job — experimental psychologist). After I discovered how useful it could be, I started doing personal science to solve actual problems, including early awakening and overweight. On these two subjects (sleep and weight control) conventional scientists seemed to have made and be making little progress, with a few exceptions (such as Sclafani, Cabanac, and Ramirez) in the area of weight control. Here my motivation was lack of plausible alternatives. Now I now see personal science like playing the lottery, except it costs almost nothing. Most of the time nothing happens, once in a long while there is a big payoff. An example of the lottery-like payoff is that for ten years I measured my sleep, trying to figure out what was causing my early awakening. One day it suddenly got worse (when I changed my breakfast). That led me to realize many things. Another example is I measured my brain function with an arithmetic test for several years. One day it suddenly improved (due to butter).

2. Measurement. Conventional scientists almost always use already-established measures because they improve communication. In contrast, a personal scientist wants a measure that is especially sensitive to the problem (e.g., insomnia) to be solved or the question to be answered (e.g., did flaxseed oil improve my balance?). Communication is much less important. Psychologists use Likert scales (rating scales with 5 or 7 possible answers) to measure internal states but they almost always use inexperienced and unmotivated subjects. When I’ve measured internal states (e.g., mood), I have a lot of motivation and eventually have a lot of experience and find I can make much finer distinctions. Unlike conventional research, I care enormously about the convenience of the measurement. For example, it should be brief.

3. Treatment choice. You don’t want to do a lot of experiments that don’t find any effect, so you need to choose wisely the treatments you test. Scanning the internet (what has cured insomnia?) and reading scientific papers (what are standard treatments for insomnia?) hasn’t worked for me, although it’s better to try anything than to try nothing. One thing that’s worked is to test large surprising effects I hear about. An example is Tara Grant’s discovery that restricting her Vitamin D to the morning improved her sleep. Also successful is measuring the problem for a long time, in search of outliers. When the problem suddenly gets better or worse, I test whatever unusual happened just before that. For example, when I switched from oatmeal breakfast to fruit breakfast, my early awakening suddenly got worse. I started testing various breakfasts. A third successful strategy is to combine the first two strategies with evolutionary thinking, giving bonus points if the treatment I’m thinking of testing provides something present in Stone Age life but absent now. For example, this is one reason I decided to test the effect of standing a lot. Stone Age people must have been on their feet more than most of us.

4. Experimental design. The hard part is knowing how fast the treatment effect rises and falls. If it rises and falls quickly, your experiment should be very different than if it rises and falls slowly. In most cases, what I study rises and falls slowly and the best design is some variation of ABA. Do A for several days, do B for several days, do A for several days. It is much easier to do a condition for too few days than too many so I try to err on the side of too many days. The hardest lesson to learn was to realize how little I know and avoid complex designs with untested assumptions.

5. Data analysis. Statistics books and classes emphasize statistical tests, whereas in practice what matters are simple graphs (e.g., what you measure versus time). I make one or more new graphs every time I collect new data (e.g., I make a plot of my weight versus time every time I weigh myself) but rarely do t tests and the like. I’ve learned to make several graphs at different time scales (e.g., last week, last month, etc.), not just one graph.

I believe these factors combine in a multiplicative way to determine how much you learn. If any is poor, you will learn little. They provide a way of asking yourself what you’ve learned after you’ve done some personal science. For example, where did I get the idea for the treatment? Presumably, with experience, you slowly get better at each of them.

Thanks to Brian Toomey for encouraging me to write this.

Why Quantified Self Matters

Seth Roberts2 Comments

Why Quantified Self Matters is the title of a talk I gave yesterday at a Quantified Self conference in Beijing. I gave six examples of things I’d discovered via self-tracking and self-experiment (self-centered moi?), such as how to lose weight (the Shangri-La Diet) and be in a better mood. I said that the Quantified Self movement matters because it supports that sort of thing, i.e., personal science, which has several advantages over professional science. The Quantified Self movement supports learning from data, in contrast to trusting experts.

If I’d had more time, I would have said that personal science and professional science have different strengths. Personal science is good at both the beginning of research (when a new idea has not yet been discovered) and the end of research (when a new idea, after having been confirmed, is applied in everyday life). It is a good way to come up with plausible new ideas and a good way to develop them (assess their plausibility when they are still not very plausible, figure out the best dose, the best treatment details). That’s the beginning of research. Personal science is also a good way to take accepted ideas and apply them in everyday life (e.g., a medical treatment, an idea about deficiency disease) because it fully allows for human diversity (e.g., a medicine that works for most people doesn’t work for you, you have an allergy, whatever). That’s the end of research.

Professional science works well, better than personal science, when an idea is in a middle range of plausibility — quite plausible but not yet fully accepted. At that point it fits a professional scientist’s budget. Their research must be expensive (Veblen might have coined the term conspicuous research, in addition to “conspicuous consumption” and “conspicuous leisure”) and only quite plausible ideas are worth expensive tests. It also fits their other needs, such as avoidance of “crazy” ideas and a steady stream of publishable results (because ideas that are quite plausible are likely to produce usable results when tested). Professional science is also better than personal science for studying all sorts of “useless” topics. They aren’t actually useless but the value is too obscure and perhaps the research too expensive for people to study them on their own (e.g., I did research on how rats measure time).

In other words, the Quantified Self movement matters because it gives all of us a new scientific tool. A way to easily see where the scientific tools we already have cannot easily see.

 

 

Bayesian Shangri-La Diet

Seth Roberts7 Comments

In July, a Cambridge UK programmer named John Aspden wanted to lose weight. He had already lost weight via a low-carb (no potatoes, rice, bread, pasta, fruit juice) diet. That was no longer an option. He came across the Shangri-La Diet. It seemed crazy but people he respected took it seriously so he tried it. It worked. His waist shrank by four belt notches in four months. With no deprivation at all.

Before he started, he estimated the odds (i.e., his belief) of three different outcomes predicted by three different theories. What would happen if he drank 300 calories (2 tablespoons) per day of unflavored olive oil (Sainsbury’s Mild Olive Oil)? Aspden considered the predictions of three theories.

I called my three ideas of what would happen [= three theories that make different predictions] if I started eating extra oil Willpower, Helplessness and Shangri-La. (1) Willpower (W) is the conventional wisdom. If you eat an extra 300 calories a day you should get fatter. This was the almost unanimous prediction of my friends. Your appetite shouldn’t be affected. (2) Helplessness (H) was my own best guess. If you eat more, it will reduce your appetite and so you’ll eat less at other times to compensate, and so your weight won’t move. Whether this appetite loss would be consciously noticeable I couldn’t guess. This was my own best guess. (3) Shangri-La (S) is your theory. The oil will drop the set point for some reason, and as a result, you should see a very noticeable loss of appetite.

More about these theories. His original estimate of the likelihood of each prediction being true: W 39%, H 60%, S 1%. He added later, “I think I was being generous with the 1%”. After the prediction of the S theory turned out to be true, the S theory became 50 times more plausible, Aspden decided.

I like this a lot. Partly because of the quantification. If you were a high jumper in a world without exact measurement, people could only say stuff like “you jumped very high.” It would be more satisfying to have a more precise metric of accomplishment. It is a scientist’s dream of making an unlikely prediction that turns out to be true. The more unlikely, the more progress you have made. Here is quantification of what I accomplished. Although Aspden could find dozens of online reports that following the diet caused weight loss, he still believed that outcome very unlikely. Given that (a) the obesity epidemic has lasted 30-odd years and (b) people hate being fat, you might think that conventional wisdom about weight control should be assigned a very low probability of being correct.

I also like this because it is the essence of science: choosing between theories (including no theory) based on predictions. The more unlikely the outcome, the more you learn. You’d never know this from 99.99% of scientific papers, which say nothing about how unlikely the actual outcome was a priori — at least, nothing numerical. I can’t say why this happens (why an incomplete inferential logic, centered on p values, remains standard), but it has the effect of making good work less distinguishable from poor work. Maybe within the next ten years, a wise journal editor will begin to require both sorts of logic (Bayesian and p value). You need both. In Aspden’s case, the p value — which would indicate the clarity of the belt-tightening — was surely very large. This helped Aspden focus on the Bayesian aspect — the change in belief. This example shows how much you lose by ignoring the Bayesian aspect, as practically all papers do. In this case, you lose a lot. Anyone paying attention understands that the conventional wisdom about weight control must be wrong. Here is guidance towards a better theory. If not mine, you at least want a theory that predicts this result.

 

 

 

 

 

Measuring Yourself to Improve Your Health? Want to Guest-Blog?

Seth Roberts6 Comments

What surprised me most about my self-experimental discoveries was that they were outside my area of expertise (animal learning). I discovered how to sleep better but I’m not a sleep researcher. I discovered how to improve my mood but I’m not a mood researcher. I discovered that flaxseed oil improved brain function but I’m not a nutrition researcher. And so on. This is not supposed to happen. Chemistry professors are not supposed to advance physics. Long ago, this rule was broken. Mendel was not a biologist, Wegener (continental drift) was not a geologist. It hasn’t been broken in the last 100 years. As knowledge increases, the “gains due to specialization” — the advantage of specialists over everyone else within their area of expertise — is supposed to increase. The advantage, and its growth, seem inevitable. It occurs, say economists, because specialized knowledge (e.g., what physicists know that the rest of us, including chemists, don’t know) increases. My theory of human evolution centers on the idea that humans have evolved to specialize and trade. In my life I use thousands of things made by specialists that I couldn’t begin to make myself.

Here we have two things. 1. A general rule (specialists have a big advantage, within their specialty, over the rest of us) that is overwhelmingly true. 2. An exception (my work). How can this be explained? What can we learn from it? I’ve tried to answer these questions but I can add to what I said in that paper. The power of specialization is clearly enormous. Adam Smith, who called specialization “division of labor”, was right. The existence of an exception to the general rule suggests there are forces pushing in the opposite direction (toward specialists being worse than the rest of us in their area of expertise) that can be more powerful than the power of specialization. Given the power of specialization, the countervailing forces must be remarkably strong. Can we learn more about them? Can we harness them? Can we increase them? The power of specialization has been increasing for thousands of years. How strong the countervailing forces may become is unclear.

The more you’ve read this blog, the more you know what I think the countervailing forces are. Some of them weaken specialists: 1. Professors prefer to be useless rather than useful (Veblen). 2. A large fraction (99%?) of health care workers have no interest in remedies that do not allow them to make money. 3. Medical school professors are terrible scientists. 4. Restrictions on research. Some of them strengthen the rest of us: 1. Data storage and analysis have become very cheap. 2. It is easier for non-scientists to read the scientific literature. 3. No one cares more about your health than you. These are examples. The list could be much longer. What’s interesting is not the critique of health care, which is pretty obvious, but the apparent power of these forces, which isn’t obvious at all.

I want to learn more about this. I want learn how to use these opposing forces and, if possible, increase them. One way to do this is find more exceptions to the general rule, that is, find more people who have improved their health beyond expert advice. I have found some examples. To find more, to learn more about them, and to encourage this sort of thing (DIY Health), I offer the opportunity to guest-blog here.

I think the fundamental reason you can improve on what health experts tell you is that you can gather data. Health experts have weakened their position by ignoring vast amounts of data. Three kinds of data are helpful: (a) other people’s experiences, (b) scientific papers and (c) self-measurement (combined with self-experimentation). No doubt (c) is the hardest to collect and the most powerful. I would like to offer one or more people the opportunity to guest-blog here about what happens when they try to do (c). In plain English, I am looking for people who are measuring a health problem and trying to improve on expert advice. For example, trying to lower blood pressure without taking blood pressure medicine. Or counting pimples to figure out what’s causing your acne. Or measuring your mood to test alternatives to anti-depressants. I don’t care what’s measured, so long as it is health-related. (Exception: no weight-loss stories) and you approach these measurements with an open mind (e.g., not trying to promote some product or theory). I am not trying to collect success stories. I am trying to find out what happens when people take this approach.

Guest-blogging may increase your motivation, push you to think more (“ I blog, therefore I think“) and give you access to the collective wisdom of readers of this blog (in the comments). If guest-blogging about your experiences and progress (or lack of it) might interest you, contact me with details of what you are doing or plan to do.

Posit Science: Does It Help?

Seth Roberts5 Comments

Tim Lundeen pointed me to the website of Posit Science, which sells ($10/month) access to a bunch of exercises that supposedly improve various brain functions, such as memory, attention, and navigation. I first encountered Posit Science at a booth at a convention for psychologists about five years ago. They had reprints available. I looked at a study published in the Proceedings of the National Academy of Sciences. I was surprised how weak was the evidence that their exercises helped.

Maybe the evidence has improved. Under the heading “world class science” the Posit Science website emphasizes a few of the 20-odd published studies. First on their list of “peer-reviewed research” is “the IMPACT study”, which has its own web page.

With 524 participants, the IMPACT study is the largest clinical trial ever to examine whether a specially designed, widely available cognitive training program significantly improves cognitive abilities in adults. Led by distinguished scientists from Mayo Clinic and the University of Southern California, the IMPACT study proves that people can make statistically significant gains in memory and processing speed if they do the right kind of scientifically designed cognitive exercises.

The study compared a few hundred people who got the Posit Science exercises with a few hundred people who got an “active control” treatment that is poorly described. It is called “computer-based learning”. I couldn’t care less that people who spend an enormous amount of time doing laboratory brain tests (1 hour/day, 5 days/week, 8-10 weeks) thereby do better on other laboratory brain tests. I wanted to know if the laboratory training produced improvement in everyday life. This is what most people want to know, I’m sure. The study designers seem to agree. The procedure description says “to be of real value to users, improvement on a training program must generalize to improvement on real-world activities”.

On the all-important question of real-world improvement, the results page said very little. I looked for the published paper. I couldn’t find it on the website. Odd. I found it on Scribd.

Effect of the training on real-world activities was measured like this:

The CSRQ-25 consists of 25 statements about cognition and mood in everyday life over the past 2 weeks, answered using a 5-point Likert scale.

Mood? Why was that included? In any case, the training group started with an average score of 2.23 on the CSRQ-25. After training, they improved by 0.07. (Significantly more than the control group.) Not only is that a tiny improvement (percentage-wise) it is unclear what it means. The measurement scale is not well-described. Was the range of possible answers 1 to 5? Or 0 to 4? What does 2 mean? What does 3 mean? It is clear, however, that on a scale where the greatest possible improvement was either 1.23 (assuming 1 was the best possible score) or 2.23 (assuming 0 was the best possible score), the actual improvement was 0.07. Not much for 50-odd hours of practice. Although the website seems proud of the large sample size (“largest clinical trial ever”), it is now clear why it was so large: With a smaller sample the tiny real-world improvement would have been undetectable. Because the website treats this as the best evidence, I assume the other evidence is even less impressive. The questions about mood are irrelevant to the website claims, which are all about cognition. Why weren’t the mood questions removed from the analysis? It is entirely possible that, had the mood questions been removed, the training would have produced no improvement.

The first author of the IMPACT study is Glenn Smith, who works at the Mayo Clinic. I emailed him to ask (a) why the assessment of real-world effects included questions about mood and (b) what happens if the mood questions are removed. I predict he won’t answer. A friend predicts he will.

More questions for Posit Science

Quantified Self Utopia: What Would It Look Like?

Seth Roberts1 Comment

On the QS forums, Christian Kleineidam asked:

While doing Quantified Self public relations I lately meet the challenge of explaining how our lives are going to change if everything in QS goes the way we want. A lot of what I do in quantified self is about boring details. . . . Let’s imagine a day 20 years in the future and QS is successful. How will that day be different than [now]?

Self-measurement has helped me two ways. One is simple and clear. It has helped me be healthy. Via QS, I have found new ways to sleep better, lose weight, be in a better mood, have fewer colds (due to better immune function), reduce inflammation in my body, have better balance, have a better-functioning brain, have better blood sugar, and so on. I am not an expert in any of these areas — I am not a professional sleep researcher, for example. I believe that this will be a large part of the long-term importance of QS: it will help non-experts make useful discoveries about health and it will help spread those discoveries. Non-experts have important advantages over professional researchers. The non-experts (the personal scientists) are only concerned with helping themselves, not with pleasing their colleagues or winning grants, promotions, or prizes; they can take as long as necessary; and they can test “crazy” ideas. In a QS-successful world, many non-experts would make such discoveries and what they learned would reach a wide audience. Lots of people would know about them and take them seriously. As a result, people would be a lot healthier.

Self-measurement has also helped me in a more subtle way. It made me believe I have more power over my health than I thought. This change began when I studied my acne. I did not begin with any agenda, any point I wanted to make, I just wanted to practice experimentation. I counted my pimples (the QS part) and did little experiments. My results showed that one of the drugs my dermatologist had prescribed (tetracycline, an antibiotic) didn’t work. My dermatologist hadn’t said this was possible. Either he had done nothing to learn if worked or he had reached the wrong answer. What stunned me was how easy it had been to find out something important a well-trained experienced expert didn’t know. My dermatologist was not an original thinker. He did what he was told to do by med school professors (antibiotics are a very common treatment for acne). It was the fact that I could improve on their advice that stunned me. I didn’t have a lab. I didn’t have a million-dollar grant. Yet I had learned something important about acne that dermatology professors with labs and grants had failed to learn (antibiotics may not work, be sure to check).

Skepticism about mainstream medicine is helpful, yes, but only a little bit. More useful is finding a better way. For example, it’s useful to point out that antidepressants don’t work well. It’s more useful to find new ways to combat depression. Two years ago, the psychiatrist Daniel Carlat came out with a book called Unhinged that criticized modern psychiatry: too much reliance on pills. No kidding. Carlat recommended more talk therapy, as if that worked so well. As far as I could tell, Carlat had no idea that you need better research to find better solutions and had no idea what better research might be. This is where QS comes in. By encouraging people to study themselves, it encourages study of a vast number of possible depression treatments that will never (or not any time soon) be studied by mainstream researchers. By providing a way to publicize what people learn by doing this, it helps spread encouraging results. In the case of depression, I found that seeing faces in the morning produced an oscillation in my mood (high during the day, low at night). This has obvious consequences for treating depression. This sort of thing will not be studied by mainstream researchers any time soon but it can easily be studied by someone tracking their mood.

In a QS-successful world, many people would have grasped the power that they have to improve their own health. (You can’t just measure yourself, you have to do experiments and choose your treatments wisely, but measuring yourself is a good start.) They would have also grasped the power they have to improve other people’s health because (a) they can test “crazy” solutions mainstream researchers will never test, (b) they can run more realistic tests than mainstream researchers, (c) they can run longer tests than mainstream researchers, and (d) no matter what the results, they can publicize them. In a QS-successful world, there will be a whole ecosystem that supports that sort of thing. Such an ecosystem is beginning to grow, no doubt about it.

Big Diet and Exercise Study Fails to Find Benefit

Seth Roberts34 Comments

Persons with Type 2 diabetes have an increased risk of heart disease and stroke. They are usually overweight. A study of about 5000 persons with Type 2 diabetes who were overweight or worse asked if eating less and exercise — causing weight loss — would reduce the risk. of heart disease and stroke. The difficult treatment caused a small amount of weight loss (5%), which was enough to reduce risk factors. The study ended earlier than planned because eating less and exercise didn’t help: “11 years after the study began, researchers concluded it was futile to continue — the two groups had nearly identical rates of heart attacks, strokes and cardiovascular deaths.”

Heart disease and stroke are major causes of death and disability. Failure of such an expensive study ($20 million?) to produce a clearly helpful result is an indication that mainstream health researchers don’t understand what causes heart disease and stroke. Another indication is that the treatment being studied (eating less and exercise) was popular in the 1950s. Mainstream thinking about weight control is stuck in the 1950s. It is entirely possible that greater weight loss — which mainstream thinking is unable to achieve — would have reduced heart disease and stroke. If you understand what causes heart disease and stroke, your understanding may lead you to lines of reasoning less obvious than people with diabetes are overweight –> weight loss treatments).

One of the study organizers – Rena Wing, a Brown University professor who studies weight control — told a journalist “you do a study because you don’t know the answer.” She failed to add, I’m sure, that wise people do not give a super-expensive car to someone who can’t drive. You should learn to drive with a cheap car. Allowing ignorant researchers to do a super-expensive study was a mistake. To learn something, do the cheapest easiest study that will help. (As I have said many times.) You should not simply do “a study”. This principle was the most helpful thing I learned during my first ten years as a scientist. In this particular case, I doubt that a $20 million study was the cheapest easiest way to learn how to reduce heart disease and stroke.

I made progress on weight control, sleep, and other things partly because studying myself allowed me to learn quickly and cheaply. If researchers understood what causes major health problems, they would be able to invent treatments with big benefits. That the Nobel Prize in Physiology or Medicine is given year after year to work that makes no progress on major health problems is another sign of the lack of understanding reflected in the failure of this study. I have never seen this lack of understanding — which has great everyday consequences — pointed out by any science blogger or science columnist or science journalist, many of whom describe themselves as “skeptical” and complain about “bad science.”

 

 

JAMA Jumps to Conclusions About Vitamin D

Seth Roberts10 Comments

A recent experiment published in JAMA, one of the most prestigious medical journals in the world, found that giving people a very large dose of Vitamin D (100,000 IU) once/month did not prevent colds, even though it greatly increased blood levels of Vitamin D. This finding supports my view that it is important to take Vitamin D in the morning. (Because a study in which this wasn’t done found no effect.) My view implies that blood levels may not matter — you can get high levels of Vitamin D by taking it at what I consider the wrong times of day. The usual thinking about Vitamin D has been that blood level is all that matters.

The editors of JAMA considered the Vitamin D study so important that they asked someone (Dr. Jeffrey Linder, associated with Harvard Medical School) to write a commentary — an associated editorial that puts the new finding in context.

Linder’s commentary (might be gated) is important because (a) it is a kind of random sample of how top research doctors think (he was selected to write it) and (b) he completely fails to grasp that the time of day Vitamin D is taken might matter. Colds, the immune system, sleep, time of day — it’s not far-fetched. When you do an experiment to see if X causes Y, and find no effect, I believe that there are usually many possible reasons other than X never causes Y. Something was wrong with the equipment, something was wrong with your X (e.g., it was stale), something was wrong with your measurements (e.g., ceiling effect), and on and on. Linder did not see it this way.

The 2011 IOM report called for additional research to determine whether vitamin D therapy reduces the incidence of respiratory tract infections. The VIDARIS trial [= the new study] has rigorously addressed this question. Results suggest that vitamin D should join the therapies listed in the Cochrane reviewsas being ineffective for preventing or treating upper respiratory tract infections in healthy adults.

He seriously thinks one null result proves something. Sure, the new study is “rigorous” in certain ways. But it was far from exhaustive. It did not explore the many ways Vitamin D may be given, for example. It did not consider the possibility that blood levels don’t matter. Linder’s combination of (a) interest in rigor and (b) failure to understand the importance of exhaustive reminds me of a friend. When she was in 1st grade she had a pile of pennies. She knew how many she had — she had counted them. However, she did not know how to subtract. When she spent some of her pennies, to find out how many she had left she had to count them all over again.

My friend had half the skills an accountant needs. Linder’s commentary reflects only half the skills a scientist needs. To the extent that he is representative of top research doctors, this is shocking. It is as if most accountants at Arthur Andersen didn’t know how to subtract.

I have asked Dr. Linder if he has any response. If he does, I will post it.

Vitamin D3 Eliminated Colds and Improved Sleep When Taken in the Morning (Stories 24 and 25)

Seth Roberts12 Comments

A year and a half ago, the father of a friend of mine started taking Vitamin D3, 5000 IU/day at around 7 am — soon after getting up. That his regimen is exactly what I’d recommend (good dose, good time of day) is a coincidence — he doesn’t read this blog. He used to get 3 or 4 terrible colds every year, year after year. Since he started the Vitamin D3, he hasn’t gotten any. “A huge lifestyle improvement,” said my friend. His dad studied engineering at Caltech and is a considerable skeptic about new this and that.

Much more recently his mother changed the time of day she took her usual dose of Vitamin D3. For years she had been taking half in the morning (with a calcium supplement) and half at night. Two weeks ago she started taking the whole dose in the morning. Immediately — the first night — her sleep improved. She used to wake up every 2 hours. Since taking the Vitamin D3 in the morning, she has been waking up only every 3-6 hours. A few days ago, my friend reports she had “her best sleep in years”.

Sleep and immune function are linked in many ways beyond the fact that we sleep more when we’re sick. A molecule that promotes sleep turned out to be very close to a molecule that produces fever, for example. I found that when I did two things to improve my sleep (more standing, more morning light) I stopped getting colds. So it makes sense that a treatment that improves one (sleep or immune function) would also improve the other (immune function or sleep).

A few days ago I posted a link about a recent Vitamin D study that found no effect of Vitamin D on colds. The study completely neglected importance of time of day by giving one large injection of Vitamin D (100,000 IU) per month at unspecified time. I commented: “One more Vitamin D experiment that failed to have subjects take the Vitamin D early in the morning — the time it appears most likely to have a good effect.” These two stories, which I learned about after that post, support my comment. What’s interesting is that the researchers who do Vitamin D studies keep failing to take time of day into account and keep failing to find an effect and keep failing to figure out why. I have gathered 23 anecdotes that suggest that their studies are failing because they are failing to make sure their subjects take their Vitamin D early in the morning. Yet these researchers, if they resemble most medical researchers, disparage anecdotes. (Disparagement of anecdotes reaches its apotheosis in “evidence-based medicine”.) The same anecdotes that, I believe, contain the information they need to do a successful Vitamin D clinical trial. Could there be a serious problem with how Vitamin D researchers are trained to do research? A better approach would be to study anecdotes to get ideas about causation and then test those ideas. This isn’t complicated or hard to understand, but I haven’t heard of it being taught. If you understand this method, you treasure anecdotes rather than dismiss them (“anecdotal evidence”).