Month: June 2012

The Difficulty of Finding a Good Experiment to Do

Seth Roberts5 Comments

My self-experimentation began because of one tiny thing — an article I noticed in the Brown University Science Library about teaching mathematics to college students. “The best way to learn is to do,” it started. Which made sense. To learn how to do experiments — one of my goals as a graduate student — I started doing self-experiments. Let’s imagine I had never seen that article, which is entirely possible. In this parallel universe, I become a psychology professor and then one day notice that someone else has done the personal science I have actually done. Has written “ Self-experimentation as a source of new ideas,” The Shangri-La Diet, and so on, including the experiments I’ve described on this blog. How would I react?

Many things about it would not impress me. You devised an arithmetic test to measure your brain function — so what? You measured yourself for a long time — big deal. You did an experiment — yawn. I might be slightly impressed by the experimental designs, which are simple and effective. Most experimental psychology uses more complex designs. What would baffle me would be the discovery of safe powerful beneficial treatments. How did this guy find these treatments? For example, the first experiment in “Self-experimentation as a source of new ideas” is about the effect of breakfast on early awakening. Eliminating breakfast reduced the fraction of days with early awakening from about 50% to 10%. Not eating breakfast is easy and perfectly safe. I don’t know of anything like this in all sleep research.

To a psychology professor, doing an experiment on one’s sleep is nothing. Finding something naturalistic (= not a drug) and sustainable that caused a big improvement, however, would be . . . unprecedented? Seemingly impossible? Psychology professors study everyday topics of great interest, such as memory and problem-solving and happiness, quite often. They would love to find easy safe sustainable non-drug ways of improving these things by large amounts. But I can’t think of a single example.

I thought about how hard it is to find big beneficial experimental effects (it’s easy to make things worse) when I read this post by the economist Yanis Varoufakis. He is excited about working for an online game company (Valve) because the nature of their game will allow experimental study of economics.

Econometrics is a travesty! . . . Econometrics purports to test economic theories by statistical means. And yet what it ends up testing is whether some ‘reduced form’, an equation (or system of equations), that is consistent with one’s theory, is also consistent with the data. The problem of course is that the ‘reduced form’ under test can be shown to be consistent with an infinity of competing theories. Thus, econometrics can only pretend to discriminate between mutually contradictory theories. All it does is to discover empirical regularities lacking any causal meaning. [Why is he sure they lack any causal meaning? — Seth]. . . The reason for this unavoidable failure? None other than our inability to run experiments on a macroeconomy such as rewinding time to, say, 1932, in order to see whether the US would have rebounded without the New Deal (or to 2009 to see what would have happened to the US economy without Ben Bernanke’s Quantitative Easing). Even at the level of the microeconomy, keeping faith with the ceteris paribus assumption (i.e. keeping all other things equal in order to measure, e.g., the relationship between the price of and the demand for milk) is impossible (as opposed to just hard).

In sharp contrast to our incapacity to perform truly scientific tests in ‘normal’ economic settings, Valve’s digital economies are a marvelous test-bed for meaningful experimentation. . . . We can change the economy’s underlying values, rules and settings, and then sit back to observe how the community responds, how relative prices change, the new behavioural patterns that evolve. An economist’s paradise indeed…

I find this baffling. It’s like thinking: Now I can write. Soon I will be writing stuff that the world wants to read! Okay, now he can do experiments. Good. After a few of them, I suppose, he will learn what every experimental scientist knows and confronts every working day: it is incredibly hard to do interesting experiments. The “sharp contrast” between the new setting and the old one has yet to be demonstrated.

Okay, how did I find a bunch of big beneficial safe sustainable effects? I am now finishing a paper in which I try to answer this question. To be brief: 1. As I’ve said, I believe that the distribution of surprise/observation follows a power-law-like distribution. Almost all observations, very little surprise, a tiny fraction of observations, great surprise. Which is pretty obvious. 2. The “slope” (parameter) of the distribution depends on subject-matter knowledge (more knowledge = more favorable slope, i.e., “chance favors the prepared mind”), scientific skill (more skill = more favorable slope), and novelty (more novelty = more favorable slope). I was in good shape on all three. For example, when I studied sleep, I knew a lot about sleep. Novelty is enormously important. In my personal science I could easily study treatments (e.g., not eating breakfast) and dimensions (e.g., how rested I felt when I awoke) that had rarely if ever been studied before. I could do this again and again, keeping novelty high and thus keeping the slope very favorable. (Varoufakis will get a burst of novelty when he begins experimentation (the situation is new) but forced to use that situation for all his experiments the novelty will run down, making the slope of the distribution less favorable.) 3. My cost/observation was very low and the benefit/observation remarkably high (I was improving my own health). So I was very motivated to make observations. My answer in the paper is a little more complicated but that’s most of it.

Who Watches the Watchdogs? The Myths of Journalism

Seth Roberts1 Comment

In a great essay, Edward Jay Epstein points out, at least by implication, that the Pulitzer Prize committee is not terribly interested in the truth of things:

A sustaining myth of journalism holds that every great government scandal is revealed through the work of enterprising reporters who by one means or another pierce the official veil of secrecy. . . This view of journalistic revelation is propagated by the press even in cases where journalists have had palpably little to do with the discovery of corruption. Pulitzer Prizes were thus awarded this year to the Wall Street journal for “revealing” the scandal which forced Vice President Agnew to resign and to the Washington Star/News for “revealing” the campaign contribution that led to the indictments of former cabinet officers Maurice Starts and John N. Mitchell, although reporters at neither newspaper in actual fact had anything to do with uncovering the scandals. . . . Yet to perpetuate the myth that the members of the press were the prime movers in such great events as the conviction of a Vice President and the indictment of two former cabinet officers, the Pulitzer Prize committee simply chose the news stories nearest to these events and awarded them its honors.

The Nobel Prize in Biology committee operates the same way, except with the disadvantage that there is not one important (= useful in a big way) biology discovery per year. There are far fewer than that. So almost every year the Nobel Prize in Biology goes to discoveries with little practical importance that are described as having great practical importance. The profession (in this case, biology) is credited with much more power than it actually has.

Why does this happen? One possible reason is that no one points it out. (Epstein’s essay, still relevant today, was published in 1974.) When a powerful journalistic institution does bad things, it is incredibly dangerous (to your career) to point this out. This is why the Murdoch scandal is so big — it went on so long. Spy magazine had a column called Review of Reviewers. It was hilarious because the misdeeds were great. Unlike almost anything else in Spy, the author was anonymous. Brilliant writing that the author did not take credit for because it was dangerous to criticize the watchdogs. Likewise, hardly anyone except Epstein criticizes the prize committees (who resemble watchdogs) so they can be profoundly inaccurate.

 

Double Interview on the Benefits of Probiotics

Seth Roberts2 Comments

This curious 2006 article has an interview with one researcher in one column and an interview with another researcher in another column. Their results differed.

Pro probiotic. “Children with [infectious acute diarrhea] who took Lactobacillus [various strains and species, in nutritional supplement form, not in yogurt form] had a shorter duration of diarrhea (on average 0.7 days shorter) than those who took placebo. Also, they had fewer episodes of diarrhea, i.e. fewer stools, on the second day of treatment than those in the placebo group. Interestingly, the children who took higher doses of Lactobacillus had shorter duration of diarrhea, and it seems that a daily dose of at least 10 billion viable bacteria is necessary to have a beneficial effect.”

Anti probiotic. “I published a big study looking at Lactobacillus GG in kids with Crohn’s disease who were already doing fairly well on medication. We put them on the probiotic or a placebo for two years. We followed them for two years and looked for whether the probiotic group had a lower rate of relapse and whether there were any differences between the two groups. We didn’t find any differences.”

Assorted Links

Seth Roberts7 Comments
  • New study shows that a Yakult probiotic drink helps people with lactose intolerance and the benefits persist 3 months after one month of drinking it. Yakult is common in Chinese and Japanese supermarkets but rare in American ones. Until I read this article, I didn’t realize that people drink it because of lactose intolerance, which is much more common in Asia than America. Via Cooling Inflammation.
  • news from the Human Microbiome Project. “To the scientists’ surprise, they also found genetic signatures of disease-causing bacteria lurking in everyone’s microbiome. But instead of making people ill, or even infectious, these disease-causing microbes simply live peacefully among their neighbors.” You may recall that a Nobel Prize was given for the discovery that ulcers are caused by a certain species of bacteria. However, almost everyone with the “disease-causing” bacteria does not get ulcers. Apparently the “surprise[d]” scientists studying the human microbiome did not know that. If it were better known that you don’t need to kill bacteria to make them harmless, antibiotic usage would be less attractive.
  • Air pollution epidemiologist fired from UCLA after his research contradicts claims about the danger of air pollution.
  • How to conduct a personal experiment: biphasic sleeping

Thanks to Melissa McEwen, Peter Spero, Tim Beneke, Dave Lull and Bryan Castañeda.

MIT Professor Reenacts the Movie Groundhog Day

Seth Roberts2 Comments

A friend of mine went to college at MIT. “One of my professors repeated himself,” she said. “Every lecture was the same.”

The class was introductory physics. “You mean he gave the same lecture year after year?” I said.

“No. Every lecture.” Hard to believe, but yes, every lecture was the same. The professor was replaced in the middle of the term.

How Useful is Personal Genomics? A Case Study

Seth Roberts12 Comments

How much can you help yourself by getting your genome sequenced? A lot, a little, not at all? Scenario 1 (big help): You discover you have a greatly elevated risk of Disease X. You do various things to reduce that risk that actually reduce it. Scenario 2: (a little help): You discover you have a greatly elevated risk of Rare Disease X. You do various things to reduce that risk but they don’t help. At least, when Disease X starts, you will be less upset. Scenario 3 (no help): You discover that you have a greatly elevated risk for a common easily-noticed disease (such as obesity). You already watched your weight, this changes nothing. Scenario 4 (harm): You discover that you have a greatly elevated risk of Scary Disease X (e.g., bipolar disorder). It is depressing news. Later studies show that the gene/disease association was a mistake. (Many gene/disease associations have failed to replicate.)

A recent Wired article tries to answer this question for one person: Raymond McCauley, a bioinformatics scientist who had his genome sequenced four years ago and learned he was “four or five times more likely than most people to develop age-related macular degeneration (AMD)”. The article says “of all the ailments described in the 23andme profile, AMD has one of the strongest genetic associations”. If I found this in my genetic profile, I would want to know the confidence interval of the increased risk. Is it a factor of 4.5 plus or minus 1? Or 4.5 plus or minus 8? This isn’t easy to figure out. In addition to the question of variability, there can easily be bias (= estimate is too high). Let’s say I do 100 gene/disease association studies. Then I scan these studies to pick the one with the strongest gene/disease association. It should be obvious that this particular association is likely to be too high and, depending on the details, could plausibly be pure chance (i.e., true association is zero). I have been unable to find out how replicable the gene/AMD association is. According to Wikipedia, “the lifetime risk of developing late-stage macular degeneration is 50% for people that have a relative with macular degeneration, versus 12% for people that do not have relatives with macular degeneration.” (Until it was eliminated via better diet, pellagra also ran in families.) The Wired article does not say whether any of McCauley’s relatives have/had AMD — a huge omission, given the uncertainty of gene/disease associations.

It wasn’t obvious what McCauley should do, according to the article:

McCauley read that there were a few preventative measures he could take to reduce the chances of AMD one day rendering him blind: don’t smoke and avoid ultraviolet light, for instance. Also, it seemed, he could try taking a special combination of vitamins, including B12 and lutein. But when he consulted the research, he could find little evidence to support the effectiveness of the regime, based on his genotype.

The article says nothing about quitting smoking but he does wear glasses that reduce ultraviolet light and takes certain vitamins. It is very hard for him to determine whether they help.

Here is a study that found greater omega-3 consumption associated with lower risk of AMD. Here is a study that found AMD associated with inflammation (too little omega-3 increases inflammation). Here is a study that found no association between vitamin and mineral intake and AMD. Based on this, if 23andme told me I had an increased risk of AMD, I would make sure to optimize my intake of flaxseed oil (or other omega-3 source) using some sort of brain test. I have documented in other posts that brain function is sensitive to omega-3 intake and (probably) most people don’t get enough. Of course, just as it is foolish to smoke (a lot) regardless of whether you have genetic risk of AMD, it is foolish to not optimize one’s omega-3 intake, whether or not you have genetic risk of AMD. In other words: everyone should optimize their omega-3 intake. If the 23andme results cause McCauley to do something wise like this that he would otherwise not have done, they have helped him.

The omega-3 study appeared after the Wired article so I don’t know how McCauley reacted to it. A puzzle about the story is that it isn’t even clear that the gene/AMD associations are true. Consider McCauley’s older relatives: parents, grandparents. Did/do any of them have AMD? If not, it is more plausible that all of them were at 12% risk of the disease than at 50% risk. Suppose all of them had, according to 23andme, the same increased risk as McCauley (at least some of them have the risk-bearing genes). Now it becomes more plausible that something is wrong with the 23andme risk estimate. If some of McCauley’s older relatives do have AMD, it is not clear why the 23andme results would make much difference. He should have already have known he was at increased risk of AMD.

The upshot is that in this particular case, I cannot even rule out Scenario 4 (does harm). All four scenarios strike me as plausible. Based on this article, we are a long way from learning the value of personal genomics.

Previously I used the example of Aaron Blaisdell to make the possibly counter-intuitive point that if you have a genetic disease something is wrong with your environment. Well, I do not have any obvious genetic disease. But I discovered, via self-experimentation, that my environment was terrible — meaning it could be improved in all sorts of ways: stop eating breakfast, drink flaxseed oil, eat butter, look at faces in the morning, take Vitamin D in the morning, and so on, not to mention eat fermented foods (which I figured out via psychology, not self-experimentation). My findings about what is optimal are so different than the way anyone now lives (except people who read this blog) that I believe everyone‘s environment can be vastly improved. If so, the value of discovering you have a genetically elevated risk of this or that is not obvious — you should already be trying to improve your environment. At least that is what my data has taught me. On the other hand, maybe genetic info (even wrong genetic info!) will give you a kick in the pants. Maybe that has happened with McCauley.

 

Variation in Abbott Blood Sugar Test Strips: A Warning

Seth Roberts7 Comments

I’ve measured my fasting blood sugar (= blood sugar before breakfast) for about four years. I began out of curiosity but became alarmed when the values approached “pre-diabetic” (> 100 mg/dl, diabetic is 126 mg/dl or so). Eventually I learned that walking an hour/day put them in the 80′s consistently. Perhaps 84 is optimal, who knows.

I have used Abbott test meters and strips. They need so little blood that testing is painless. Recently (January 2012?) Abbott introduced new “butterfly” test strips that “wick” the blood. The meters stayed exactly the same. The new test strips are certainly better. I started to use them. I started using them after a gap (a month?). All of a sudden my scores were about 5 mg/dl better — for example, 84 instead of 89. I assumed this was due to lifestyle changes on my part. I was walking more, I was more muscular, whatever. These were plausible explanations. Surely Abbott had not corrected a huge mistake (given the size of the business, the importance of diabetes, and the need for accurate test strips, to be consistently off by 5 mg/dl would be a huge mistake).

Now I wonder. I recently found some old-style test strips, barely expired (2012/04). I have compared them to the new-style strips (expiration 2013/06). Here are my results:

Morning 1. New: 81, 84. Old: 99, 91, 100.

Morning 2. New: 80. Old: 96, 94, 95.

Morning 3. New: 84. Old: 104, 97, 105.

Morning 4. New: 86. Old: 101, 100, 100.

These results involve three different meters. The old strips come from three separate vials. It is clear that the old strips produce readings much higher (about 15 mg/dl higher) than the new strips.

The old strips are expired but I doubt they got 15 mg/dl worse in 1-2 months. I expect they are accurate when they leave the factory and slowly get worse. Now I have some idea of how much worse (and in what direction). Apparently there is a big increase in bias with little increase in variability. I’ve gone from batch to batch before and never noticed a difference. Only when comparing the new strips with the old has a difference been clear. The earlier comparison, with a 5 mg/dl difference, compared unexpired old strips with the new strips.

I conclude that with the old strips, deterioration with age is worse than I expected and I should pay more attention to test strip age.

 

Assorted Links

Seth Roberts3 Comments

The Glacially-Slow Conquest of Scurvy And Its Relevance to Modern Life

Seth Roberts9 Comments

Scurvy is a disease of civilization because you need civilization to make long ocean voyages. It is the first disease of civilization to be understood and eliminated. In a paper called “Innovation and Evaluation” (gated), Frederick Mosteller, a professor of statistics at Harvard, noted how long it took. In 1601, James Lancaster, a sea captain, did an experiment involving four ships on a long voyage. Men on one ship got lemon juice, men on the other three ships did not. The men given lemon juice were far less likely to get scurvy. In 1747, James Lind, a doctor, compared six purported cures for scurvy. Lemons and oranges (one cure) were much better than the other five (as Lind expected). In 1795 the British Navy started using citrus juice regularly and wiped out scurvy on their ships. In 1865, the British Board of Trade recommended citrus juice for commercial ships. It took more than 200 years for a simple and effective remedy — discovered before Lancaster — to spread widely.

The sailors at risk of scurvy did not control what they ate. The people who controlled what they ate never got scurvy. Sure, the people who controlled what sailors ate did not want them to get scurvy (high rates of scurvy were a big problem) but they also had other concerns. The lesson I draw from this story is do not let anyone else (doctor, expert, etc.) solve your health problems for you. Sure, other people, as part of their job, will sell you something, provide advice, write a prescription, provide therapy, do surgery, whatever. It might work. They want to help you — the more they help you, the better they look, the more business they attract. But it is entirely possible, this bit of history teaches, that they are slow on the uptake or have conflicts of interest and a much better solution is available.

Thanks to Steve Hansen.

More Examples of Mainstream Health Care Ignoring the Immune System

Seth Roberts19 Comments

In a recent post I made an obvious point. If our immune systems were stronger, we would need antibiotics less often and antibiotic resistance would become less of a problem. I hadn’t heard this point made (for example, this WHO report fails to say it). This was one example, I said, of how mainstream health care ignores the immune system. Perfectly obvious things, such as this idea about antibiotic resistance, fail to be noticed. I gave five more examples. Since then I have come across even more examples:

1. Hospitals do little to help patients sleep and often interrupt sleep, Nancy Lebovitz pointed out (better sleep –> better immune function). This article describes the problem. One way to improve hospital sleep — beyond don’t wake patients up — would be to provide exposure to strong sunlight-like light in the morning and prevent exposure to sunlight-like light after dark. I found that an hour of sunlight or similar light from fluorescent lamps in the morning improved my sleep. Most fluorescent light resembles sunlight (both have strong bluish components), incandescent light (reddish) does not. Until they install dual lighting systems (bluish light during the day, reddish light at night), hospitals can provide blue-blocker glasses to wear after dark.

2. The book Immortal Bird (sent me by the publisher) tells how Damon Weber, born with a defective heart, had a heart transplant when he was a teenager. After the transplant, problems arose. The doctors involved (at NewYork-Presbyterian ­Hospital/Columbia University Medical Center) took the problems to be signs of transplant rejection. In fact they were due to infection. Drugs given to deal with the mistakenly-assumed rejection suppressed Damon’s immune system. They reduced his ability to fight off the infection and he died. The author of the book, Damon’s father, sued the doctors and hospital for malpractice. The doctors did not exactly “ignore” the immune system, but they apparently failed to fully grasp the danger of immune suppression, even though the infection that killed Damon is common in transplant cases. (Although Columbia Presbyterian charged half a million dollars for the transplant, “three years into the lawsuit the [hospital’s] medical director claimed Damon’s post-op records couldn’t be located.”)

3. I asked a UCSF medical student what she’d been taught about the immune system. “We cover it!” she said. In a section called “Infectious Disease, Immunology, and Inflammation”. What makes the immune system work better or worse? I asked. “If you’re stressed out, it doesn’t work well,” she said. If you’re malnourished, like in Bangladesh. You need “nutrients and vitamins”. (A booklet I got telling me to take less antibiotics told me to “eat healthy”.) She also said the students get entire lectures on how to treat diseases so rare they might never be encountered. There is a whole section on genetics. Sure, they cover it. So superficially that they don’t remember the most basic idea: Better sleep –> better immune function. I said our health care system is built around first, let them get sick. That’s right, she said. Ignoring the immune system is an excellent way to allow people to get sick.

4. Melissa McEwen pointed out that proton pump inhibitors, such as Nexium, reduce the body’s ability to fight infection. They are prescribed for acid reflux and reduce how much acid the stomach makes. Because stomach acid kills bacteria, there should have been far more concern about their safety. “Proton pump inhibitors (PPIs) are among the most widely prescribed medications worldwide [billions of prescriptions]. . . . The collective body of information overwhelmingly suggests an increased risk of infectious complications,” says this article. Because the drugs are so common, the damage is great and, because of more infection, not restricted to those who take them. It could have been avoided by research into treatments that do not harm the immune system.