Category: self-experimentation

Elements of Personal Science

Seth Roberts4 Comments

To do personal science well, what should you learn?

Professional scientists learn how to do science mostly in graduate school, mostly by imitation, although they might take a statistics class. Personal scientists rarely have anyone to imitate, so have more need to understand basic principles. There are five skills/dimensions that matter. Here are a few comments about each one:

1. Motivation. In conventional science, the scientist does it as part of a job and subjects are paid. Neither works here: It isn’t a job and you can’t pay yourself. My original motivation was wanting to learn how to do experiments (for my job — experimental psychologist). After I discovered how useful it could be, I started doing personal science to solve actual problems, including early awakening and overweight. On these two subjects (sleep and weight control) conventional scientists seemed to have made and be making little progress, with a few exceptions (such as Sclafani, Cabanac, and Ramirez) in the area of weight control. Here my motivation was lack of plausible alternatives. Now I now see personal science like playing the lottery, except it costs almost nothing. Most of the time nothing happens, once in a long while there is a big payoff. An example of the lottery-like payoff is that for ten years I measured my sleep, trying to figure out what was causing my early awakening. One day it suddenly got worse (when I changed my breakfast). That led me to realize many things. Another example is I measured my brain function with an arithmetic test for several years. One day it suddenly improved (due to butter).

2. Measurement. Conventional scientists almost always use already-established measures because they improve communication. In contrast, a personal scientist wants a measure that is especially sensitive to the problem (e.g., insomnia) to be solved or the question to be answered (e.g., did flaxseed oil improve my balance?). Communication is much less important. Psychologists use Likert scales (rating scales with 5 or 7 possible answers) to measure internal states but they almost always use inexperienced and unmotivated subjects. When I’ve measured internal states (e.g., mood), I have a lot of motivation and eventually have a lot of experience and find I can make much finer distinctions. Unlike conventional research, I care enormously about the convenience of the measurement. For example, it should be brief.

3. Treatment choice. You don’t want to do a lot of experiments that don’t find any effect, so you need to choose wisely the treatments you test. Scanning the internet (what has cured insomnia?) and reading scientific papers (what are standard treatments for insomnia?) hasn’t worked for me, although it’s better to try anything than to try nothing. One thing that’s worked is to test large surprising effects I hear about. An example is Tara Grant’s discovery that restricting her Vitamin D to the morning improved her sleep. Also successful is measuring the problem for a long time, in search of outliers. When the problem suddenly gets better or worse, I test whatever unusual happened just before that. For example, when I switched from oatmeal breakfast to fruit breakfast, my early awakening suddenly got worse. I started testing various breakfasts. A third successful strategy is to combine the first two strategies with evolutionary thinking, giving bonus points if the treatment I’m thinking of testing provides something present in Stone Age life but absent now. For example, this is one reason I decided to test the effect of standing a lot. Stone Age people must have been on their feet more than most of us.

4. Experimental design. The hard part is knowing how fast the treatment effect rises and falls. If it rises and falls quickly, your experiment should be very different than if it rises and falls slowly. In most cases, what I study rises and falls slowly and the best design is some variation of ABA. Do A for several days, do B for several days, do A for several days. It is much easier to do a condition for too few days than too many so I try to err on the side of too many days. The hardest lesson to learn was to realize how little I know and avoid complex designs with untested assumptions.

5. Data analysis. Statistics books and classes emphasize statistical tests, whereas in practice what matters are simple graphs (e.g., what you measure versus time). I make one or more new graphs every time I collect new data (e.g., I make a plot of my weight versus time every time I weigh myself) but rarely do t tests and the like. I’ve learned to make several graphs at different time scales (e.g., last week, last month, etc.), not just one graph.

I believe these factors combine in a multiplicative way to determine how much you learn. If any is poor, you will learn little. They provide a way of asking yourself what you’ve learned after you’ve done some personal science. For example, where did I get the idea for the treatment? Presumably, with experience, you slowly get better at each of them.

Thanks to Brian Toomey for encouraging me to write this.

Is Jimmy Moore’s Ketosis Diet the Shangri-La Diet in Disguise?

Seth Roberts17 Comments

I have recently encountered three examples that suggest low-carb diets don’t work well long-term:

1. Alex Chernavsky tried a low-carb diet in 2002. Starting at 270 pounds, he lost 70 pounds. A year later, he started to rapidly regain the lost weight. He stopped the diet.

2. A “medical professional” started at about 260 pounds (she’s 5’3″). After reading Wheat Belly, she gave up wheat. “After several months of being wheat free I lost 10 lbs. But that’s where it stopped.” Then she did full low-carb. “From May to July I did what basically was Atkins induction. I lost 20 lbs but then the weight loss stopped.”

3. Jimmy Moore lost a lot of weight eating low-carb. Starting in 2004 at 410 pounds, he lost 180 pounds. Then he gained half of it back, ending up near 300 pounds in early 2012.

The theory behind the Shangri-La Diet (SLD) says unfamiliar food will cause weight loss because its smell is not (yet) associated with calories. As the food becomes familiar, its smell becomes associated with calories. Weight loss due to unfamiliarity will disappear. Going low-carb usually involves eating unfamiliar foods. They become familiar. This explains low-carb weight regain. The theory explains partial low-carb success (e.g., Jimmy Moore didn’t regain all the lost weight) by assuming that the high-carb foods (e.g., soft drinks) given up produced stronger smell-calorie associations than the low-carb foods (e.g., steak) that replaced them.

Recently Jimmy Moore has been losing weight again. Starting at 306 pounds, over 7 months he has lost 60 pounds. He believes that to lose weight with a low-carb diet, there must be sufficient ketones in your blood — you must be at the optimal level of ketosis. “In order to be fully keto-adapted and to start burning stored body fat for fuel, ketone levels must be between 0.5 to 3.0 millimolar,” he wrote. To be fully keto-adapted, he began measuring his ketone level regularly. His first test showed that his ketone level was 0.3. “Holy cow, that could be one of the reasons why I’m not seeing my weight go down!” he wrote. He began adjusting his diet to put his ketone level between 0.5 and 3.0 millimolar, which involved changing protein intake as well as carb intake.

He changed his diet in various ways (mainly protein reduction) and started losing weight. In what I’ve read, he does not describe his current diet or earlier diet in detail, but does say this:

I will tell you that I’ve drank liberal amounts of water and 2 Tbs Carlson’s liquid fish oil daily along with my regular daily vitamins during this experiment.

Which sounds exactly like the Shangri-La Diet. Alex Chernavsky lost considerable weight and has kept it off doing almost the same thing with flaxseed oil.

My guess is that he is losing weight because of the fish oil. The theory behind SLD makes two predictions: 1. If Jimmy stops the fish oil and continues the ketone level adjustment, he will stop losing weight. 2. If Jimmy stops the ketone level adjustment but continues the fish oil, he will continue losing weight.

I asked Jimmy for comment. Here’s what he said:

It’s an interesting theory, but not one I want to particularly test out since I’m still doing so well at accomplishing what I am aiming for right now–fat loss, mental acuity and great overall health [all due to the fish oil, I believe — Seth]. Perhaps once this period of testing NK [nutritional ketosis] is over in May, I can add in your suggestion as another testing point.

The theory behind low-carb dieting has never made any correct predictions, as far as I know. It does not explain why the lost weight is often regained. If it turns out Jimmy Moore’s weight loss is due to his ketone adjustment, that will be the first correct prediction of the theory.

In contrast, the theory behind SLD led me to five new ways to lose weight (eating bland food, eating slowly-digested food, drinking unflavored sugar water, drinking oil with no smell, eating food nose-clipped). That’s roughly the same as five correct predictions, two of them (drinking sugar water, drinking oil with no smell) counter-intuitive.

Jimmy Moore’s weight loss may eventually show you can lose weight via SLD even when you don’t realize you’re doing SLD.

Bayesian Shangri-La Diet

Seth Roberts7 Comments

In July, a Cambridge UK programmer named John Aspden wanted to lose weight. He had already lost weight via a low-carb (no potatoes, rice, bread, pasta, fruit juice) diet. That was no longer an option. He came across the Shangri-La Diet. It seemed crazy but people he respected took it seriously so he tried it. It worked. His waist shrank by four belt notches in four months. With no deprivation at all.

Before he started, he estimated the odds (i.e., his belief) of three different outcomes predicted by three different theories. What would happen if he drank 300 calories (2 tablespoons) per day of unflavored olive oil (Sainsbury’s Mild Olive Oil)? Aspden considered the predictions of three theories.

I called my three ideas of what would happen [= three theories that make different predictions] if I started eating extra oil Willpower, Helplessness and Shangri-La. (1) Willpower (W) is the conventional wisdom. If you eat an extra 300 calories a day you should get fatter. This was the almost unanimous prediction of my friends. Your appetite shouldn’t be affected. (2) Helplessness (H) was my own best guess. If you eat more, it will reduce your appetite and so you’ll eat less at other times to compensate, and so your weight won’t move. Whether this appetite loss would be consciously noticeable I couldn’t guess. This was my own best guess. (3) Shangri-La (S) is your theory. The oil will drop the set point for some reason, and as a result, you should see a very noticeable loss of appetite.

More about these theories. His original estimate of the likelihood of each prediction being true: W 39%, H 60%, S 1%. He added later, “I think I was being generous with the 1%”. After the prediction of the S theory turned out to be true, the S theory became 50 times more plausible, Aspden decided.

I like this a lot. Partly because of the quantification. If you were a high jumper in a world without exact measurement, people could only say stuff like “you jumped very high.” It would be more satisfying to have a more precise metric of accomplishment. It is a scientist’s dream of making an unlikely prediction that turns out to be true. The more unlikely, the more progress you have made. Here is quantification of what I accomplished. Although Aspden could find dozens of online reports that following the diet caused weight loss, he still believed that outcome very unlikely. Given that (a) the obesity epidemic has lasted 30-odd years and (b) people hate being fat, you might think that conventional wisdom about weight control should be assigned a very low probability of being correct.

I also like this because it is the essence of science: choosing between theories (including no theory) based on predictions. The more unlikely the outcome, the more you learn. You’d never know this from 99.99% of scientific papers, which say nothing about how unlikely the actual outcome was a priori — at least, nothing numerical. I can’t say why this happens (why an incomplete inferential logic, centered on p values, remains standard), but it has the effect of making good work less distinguishable from poor work. Maybe within the next ten years, a wise journal editor will begin to require both sorts of logic (Bayesian and p value). You need both. In Aspden’s case, the p value — which would indicate the clarity of the belt-tightening — was surely very large. This helped Aspden focus on the Bayesian aspect — the change in belief. This example shows how much you lose by ignoring the Bayesian aspect, as practically all papers do. In this case, you lose a lot. Anyone paying attention understands that the conventional wisdom about weight control must be wrong. Here is guidance towards a better theory. If not mine, you at least want a theory that predicts this result.

 

 

 

 

 

Assorted Links

Seth Roberts1 Comment

Thanks to Dave Lull.

50 Years of Knuckle Cracking Did Not Produce Arthritis

Seth Roberts8 Comments

Warned by relatives that knuckle cracking causes arthritis, Donald Unger decided to crack only the knuckles of his left hand. For 50 years he frequently cracked his left hand, never his right. Finally he wrote a letter to a scientific journal (in which he calls himself “the author”) pointing out that he did not have arthritis in either hand, supporting the conclusion of another study which studied a much smaller amount of knuckle cracking.

Thanks to Bryan Castañeda via Now I Know.

Why Self-Track? The Possibility of Hard-to-Explain Change

Seth Roberts8 Comments

My personal science introduced me to a research method I have never seen used in research articles or described in discussions of scientific method. It might be called wait and see. You measure something repeatedly, day after day, with the hope that at some point it will change dramatically and you will be able to determine why. In other words: 1. Measure something repeatedly, day after day. 2. When you notice an outlier, test possible explanations. In most science, random (= unplanned) variation is bad. In an experiment, for example, it makes the effects of the treatment harder to see. Here it is good.

Here are examples where wait and see paid off for me:

1. Acne and benzoyl peroxide. When I was a graduate student, I started counting the number of pimples on my face every morning. One day the count improved. It was two days after I started using benzoyl peroxide more regularly. Until then, I did not think benzoyl peroxide worked well — I started using it more regularly because I had run out of tetracycline (which turned out not to work).

2. Sleep and breakfast. I changed my breakfast from oatmeal to fruit because a student told me he had lost weight eating foods with high water content (such as fruit). I did not lose weight but my sleep suddenly got worse. I started waking up early every morning instead of half the time. From this I figured out that any breakfast, if eaten early, disturbed my sleep.

3. Sleep and standing (twice). I started to stand a lot to see if it would cause weight loss. It didn’t, but I started to sleep better. Later, I discovered by accident that standing on one leg to exhaustion made me sleep better.

4. Brain function and butter. For years I measured how fast I did arithmetic. One day I was a lot faster than usual. It turned out to be due to butter.

5. Brain function and dental amalgam. My brain function, measured by an arithmetic test, improved over several months. I eventually decided that removal of two mercury-containing fillings was the likely cause.

6. Blood sugar and walking. My fasting blood sugar used to be higher than I would like — in the 90s. (Optimal is low 80s.) Even worse, it seemed to be increasing. (Above 100 is “pre-diabetic.”) One day I discovered it was much lower than expected (in the 80s). The previous day I had walked for an hour, which was unusual. I determined it was indeed cause and effect. If I walked an hour per day, my fasting blood sugar was much better.

This method and examples emphasize the point that different scientific methods are good at different things and we need all of them (in contrast to evidence-based medicine advocates who say some types of evidence are “better” than other types — implying one-dimensional evaluation). One thing we want to do is test cause-effect ideas (X causes Y). This method doesn’t do that at all. Experiments do that well, surveys are better than nothing. Another thing we want to do is assess the generality of our cause-effect ideas. This method doesn’t do that at all. Surveys do that well (it is much easier to survey a wide range of people than do an experiment with a wide range of people), multi-person experiments are better than nothing. A third thing we want to do is come up with cause-effect ideas worth testing. Most experiments are a poor way to do this, surveys are better than nothing. This method is especially good for that.

The possibility of such discoveries is a good reason to self-track. Professional scientists almost never use this method. But you can.

How Martha Rotter Cured Her Acne By Self-Experimentation

Seth Roberts3 Comments

Several months ago I posted about how Martha Rotter figured out that her acne was caused by cow dairy products. Now a longer version of her story (by me) is on Boing Boing. There is a ton of useful information in the comments. Some examples:

“ Dairy is what caused my acne.” Someone replied: “Same here, specifically milk. I switched to soy milk in high school and my moderately-bad acne went away very suddenly. . . . If I eat a lot of cheese at once, like having pizza more than a couple days a week, my backne gets worse and I get acne inside my ears.” Someone else misunderstands genetics: “I do have tumor-forming disease (fortunately stable, and partially corrected with surgery) so I do have some sympathy when it comes to this sort of thing, but my condition is so well established as genetic I never even saw hope in trying to control it with diet.” Aaron Blaisdell had a well-established genetic condition (porphyria) that went away when he changed his diet.

Someone else found that dairy mattered:

I had terrible acne as a teenager and I drank almost a carton of milk every day. . . . When I moved out on my own, I no longer had milk delivered at the door and I fell out of the habit of drinking it altogether, switching to tea and water instead. My face cleared within weeks. . . . Whenever I indulged in cheese, the break-outs returned.

Someone else discovered multiple causes:

I have had strikingly similar experiences with a very particular form of acne, for years. Multiple doctors with no results until I got frustrated with it. I heard that the four most common causes of skin reactions can be wheat, milk, peanut butter and eggs – so I took all of them out *and* meat.

And watched my skin slowly return to normal.

After playing with my food by putting one thing in, seeing what happened, and then taking that out and trying something else, I found that wheat in particular is the trigger for me with dairy as a close second.

Someone else: “I took wheat from my diet, and my skin cleared up. If I allow wheat back in for one day, the next day I have acne.”

Not all solutions were dietary:

My wife and I found the only thing that worked reliably–even including a couple of different kinds of antibiotics–was “the regimen” as described on acne.org. Basically you use a low-strength (2.5%) benzoyl peroxide every day and moisturise like mad afterwards.

These are just examples. There are many more helpful comments.

Assorted Links

Seth Roberts1 Comment

 

Thanks to Ken Feinstein.

More About Pork Fat and Sleep

Seth Roberts22 Comments

One day in 2009, I ate a large amount of pork belly (very high in fat — pork belly is the cut used to make bacon). That night I slept an unusually long time. The next day I had more energy than usual. This led me to do an experiment in which I ate a pork belly meal (with lots of pork belly, about 250 g) on some days but not others. I compared my sleep after the two sorts of days. I kept constant the number of one-legged stands I did each day because that has an effect. During the first half of the experiment I kept this constant at 4; during the second half, at 2. I originally posted the results only from the first half.

Now I’ve analyzed the results from both halves. Here are ratings of how rested I felt when I woke up, on a scale where 0 = 0% = not rested at all and 100 = 100% = completely rested.

The two halves were essentially the same: pork belly produced a big improvement. Here are the results for sleep duration.

No clear effect of pork belly in either half of the experiment.

The main thing I learned was that pork fat really helps. The effect is remarkably clear. With micronutrients, such as Vitamin C, the body has considerable storage. It may take months without the nutrient to become noticeably deficient. With omega-3, which is between a micronutrient and a macronutrient, my experiments found that it takes about two days to start to see deficiency. With pork fat there seems to be no storage at all. I needed to eat lots of pork fat every day to get the best sleep. That repletion and depletion are fast made this experiment easy. How curious we are so often told animal fat is bad when an easy experiment shows it is good, at least for me.