Category: self-experimentation

Omega-3 and Arithmetic (several analyses)

Seth Roberts4 Comments

In a recent post I described Tim Lundeen’s arithmetic data. He found that increasing his daily dose of DHA seemed to increase the speed at which he did simple arithmetic. Here is the graph:

Tim Lundeen's arithmetic data

I didn’t bother to do any statistical tests because I thought the DHA effect was obvious. However, someone in the comments said it wasn’t obvious to them. Fair enough.

If DHA has no effect, then the scores with more DHA should be the same as the just-preceding scores with less DHA. There are practice effects, of course, so I analyzed the data after practice stopped having an effect: After about Day 40. (And I left out days preceded by a gap in testing — e.g., a day preceded by a week off.) Thousands of learning experiments have found that practice makes a difference at first and then the effect goes away — additional practice doesn’t change behavior.

If I do a t-test comparing low-DHA days (after Day 40) with high-DHA days, I get a huge t value — about 9. If you’re familiar with real-life t values, I’m sure you’ll agree that’s a staggeringly high value for a non-trivial effect. The model corresponding to this test is indicated by the lines in this figure:

Tim Lundeen's data

The red (”more DHA”) points don’t fit the line very well, which suggests doing an analysis where the slopes can vary:

Tim Lundeen's arithmetic data

There is still a huge effect of DHA, now split between two terms in the model — a difference-in-level term (t = 4) and a difference-in-slope term (t = 3).

But this analysis can be improved because based on thousands of experiments I don’t believe that the less-DHA line could have a positive slope, as it does in the model. Or at least I believe that is very unlikely. So I will constrain the less-DHA line to have a slope of zero:

Tim Lundeen's arithmetic data

Now I get t = 8 for the difference in slopes and t = 4 for the difference in level. This is interesting because it implies that more DHA not only caused immediate improvement but also opened the door to more gradual improvement (indicated by the slope difference). DHA changed something that allowed practice to have more effect.

That’s a new way of thinking about the effects of omega-3 — actually, I have never seen any data with the feature that a treatment caused a practice effect to resume — so I have to thank the person who claimed the difference wasn’t obvious.

Omega-3 and Arithmetic (continued)

Seth Roberts14 Comments

Tim Lundeen, a Bay Area software developer, previously posted here about what happened when he increased his daily dose of DHA (an omega-3 fat in fish oil) from 400 mg/day to 800 mg/day: The next day, the speed with which he did simple arithmetic (e.g., 7 + 3) increased. At that point he had only four days from the high-DHA condition. Now he has two months. Here it is:

Tim Lundeen's arithmetic data

The y axis is the total time taken to do a set of 100 simple arithmetic problems.

Bottom line: The improvement continued, at roughly the same level. Very good evidence for an effect.

Tim had earlier found that doses of 200 and 400 mg/day of DHA had no apparent effect.

My main posts about omega-3.

Interview about Self-Experimentation (part 2 of 2)

Seth Roberts2 Comments

8. How do you verify your results?

Repetition — first by me, later by others.

9. It seems your whole life is nothing but a self-experiment – how can your friends handle this?

Well, I spend a few hours in the morning differently than anyone else. I go to sleep earlier and wake up earlier than most people around me. And I eat less than most people. I like to think I make up for it by being in a better mood.

10. How do your colleagues react to your self-experiments?

Most of them think self-experimentation is a mistake, a waste of time. A few think it is creative and important.

11. Your most recent research is dealing with the effects of omega-3 on dental health. What is this research exactly about?

It’s not about dental health – that effect (omega-3 improved my gums) was an accident. It’s about the effects of omega-3 on my brain. I am varying the omega-3 in my diet in various ways and measuring how well my brain works in various ways. I began this research when I discovered that swallowing flaxseed oil capsules improved my balance. I was surprised but the effect makes sense: balance is controlled by the brain and the brain is more than half fat. Maybe you need the right fats in your diet if you want your brain to work as well as possible.

12. How did you get the idea of searching for the relation between omega-3 and dental health?

See answer to previous question.

13. How did you get the idea of taking oil to lose weight?

It was a three-step process. Step 1: I came up with a new theory of weight control. Step 2: I accidentally lost my appetite during a trip to Paris. I guessed that the cause was the unfamiliar sugar-sweetened soft drinks I’d been drinking because of the heat. This led me to discover that drinking small amounts of sugar water cause a lot of weight loss. Step 3: A friend pointed out that my theory predicted that flavorless oil should be just as effective as sugar water.

14. Are you going to search for a medical explanation for the effects of omega-3 fats?

No. Just convincing most people that there are effects is hard enough. It will also take a long time to learn how to maximize the effects. For example, what oils are best? How much oil is best? Other people are in a better position than me to try to explain the effects. But I don’t think it is terribly mysterious or surprising that dietary omega-3 should improve brain function: the brain is more than half fat. Surely the type of fat matters. My discovery is how big and fast the effect is. That’s not obvious.

15. When you consider your work as a whole, what is the most important result of your scientific research via self-experimentation?

Discovery of the effect of morning faces on mood. I believe depression is a deficiency disease, caused by too little exposure to morning faces. (See this paper for details.) No doubt that sounds very odd — even odder than the Shangri-La Diet — but consider this. In a wonderful book called The Good Women of China, the author, a Chinese radio host named Xinran Xue, wrote about her travels all over China to learn how different women lived. The last chapter is about visiting an extremely poor and backward community called Shouting Hill where an egg is a luxury and each women has multiple husbands because two or three girls are traded for a wife. She comes back to the radio station and tells her colleagues what she has seen. One of them asks, “Are they happy?” Another says, “Don’t be ridiculous, how could they be happy?” Because they were so poor — very poor even by Chinese standards. Xue answered:

I said to Mengxing that, out of the hundreds of Chinese women I had spoken to over nearly ten years of broadcasting and journalism, the women of Shouting Hill were the only ones to tell me they were happy.

It is pretty clear they saw plenty of faces in the morning.

Durian and SLD

Seth RobertsLeave a comment

The obvious connection between durian, the big smelly spiky Asian fruit, and the Shangri-La Diet is that both rely on flavor-calorie learning. We come to like the initially unpleasant smell and flavor of durian because we learn to associate it with the calories in the fruit. Here’s what happens:

“To anyone who doesn’t like durian it smells like a bunch of dead cats,” said Bob Halliday, a food writer in based Bangkok. “But as you get to appreciate durian, the smell is not offensive at all. It’s attractive.

From an article in today’s NY Times. The theory that led me to the SLD centers on flavor-calorie learning.

A less obvious connection is a principle that helped me discover that drinking sugar water causes weight loss. I was in Paris and lost my appetite — a rare event. The principle is that rare events are usually due to rare events. So I wondered what else unusual had happened. Well, there was something: I had been drinking several sugar-sweetened unfamiliar soft drinks per day. When I got back to Berkeley I started to test the possibility that sugar-sweetened water can cause weight loss and SLD was born.

For a fruit, durian has three rare properties:

    1. very strong, unpleasant smell
    2. very big
    3. hard to handle (because spiky)

Following the Rare-Causes-Rare principle, these should have a common explanation. Lightning does not strike thrice in one place for different reasons. According to Wikipedia,

The thorny armored covering of the fruit may have evolved because it discourages smaller animals, since larger animals are more likely to transport the seeds far from the parent tree.

That’s a good explanation of #3 and it explains the other two rare features (#1 and #2) as well. The reason for the strong smell (#1) is so that the signal will be broadcast a long distance: Large animals are less dense than small animals. We think of the smell of ripe durian as very unpleasant but perhaps almost all unfamiliar smells are unpleasant; so any random strong smell will seem very unpleasant. Big fruit (#2) means big tree and big tree means that seeds must be carried far away so as to be placed in soil where they will not compete with the mother tree. Coconuts are big and hard to eat. Pineapples are big and spiky.

The Rare-Causes-Rare principle also helped me discover the effect of morning faces on my mood and the effect of omega-3 on my balance.

Interview about Self-Experimentation (part 1 of 2)

Seth RobertsLeave a comment

For a German magazine, I’ve been answering some questions about self-experimentation. Here are the first seven questions and my answers:

1. When and why did you came up with the idea of performing a self-experiment for the first time Mr. Roberts?

I started self-experimentation as a graduate student. My field of study was experimental psychology so it was important to learn how to do experiments. “The best way to learn is to do,” I had read. So the more experiments I did the more I would learn. Self-experiments were easy and fast. So I started doing them to increase how quickly I learned about experimentation.

2.Now, self-experimentation must be considered as an inherent part of your scientific work – or is it rather a bauble?

Self-experimentation has been the most influential work of mine by far. Lots of surprises and practical applications.

3. Your self-experiments always deal with very personal concerns like sleep disorders, depressions, procrastination or weight control. Has self-experimentation changed your life?

Yes. Sleep, weight, mood, general health, brain – all better. And it is very satisfying to help people. Thousands of people have used my ideas (described in The Shangri-La Diet) to lose weight.

4. What is the role of coincidence in your self-experimentation?

Most of my self-experimentation has started with an unexpected change. I changed my breakfast; my sleep got worse. I started taking flaxseed oil capsules; my balance improved. I started to stand a lot and my sleep got better. I started walking outside in the morning; my sleep improved. I watched TV in the morning; my mood improved the next day. I drank unfamiliar soft drinks; I lost my appetite. Each of these surprises led to lots of self-experimentation.

5. By coincidence for example you found a relation between watching TV in the morning and your mood the following day. What made you looking at this?

I was hoping to improve my sleep. When we sleep is affected by when we have contact with other people. If you have contact with other people late at night, you will be awake later the following night. I knew about research that suggested that watching TV has the same effect on sleep as human contact. I wondered if my sleep was bad because I didn’t have contact with other people in the morning. Maybe TV could substitute for that, I thought. So I watched TV early one morning.

6. When experimenting on yourself, aren’t you taking a big risk for your health? Have there been self-experiments you would now describe as risky?

Doctors have done risky self-experiments. I haven’t. I have studied the effects of very common things – watching TV, not eating breakfast, standing a lot. Millions of people have done these things without harm. They’re not dangerous.

7. Which of your experiments did you enjoy most?

Seeing faces in the morning. The effects are wonderful: I feel happy, serene, and energetic the next day. I’ve done several experiments about sleep. It feels great to wake up feeling very rested.

Omega-3 and Dental Health (part 2 of 2)

Seth Roberts4 Comments

I looked at my gums this morning. I had never seen them so pink (that is, non-red). They looked just like the picture of healthy gums at the dentist. As I explained yesterday, my gums are in good shape because I am drinking 4 tablespoons/day of flaxseed oil, which contains a lot of omega-3.

Meta-analyses of the effects of omega-3 have had trouble finding an effect. A meta-analysis about mood found a barely-reliable effect and concluded “the evidence available provides little support for the use of n–3 PUFAs to improve depressed mood.” (They should have said “ a little support.”) A meta-analysis about heart disease concluded “Long chain and shorter chain omega 3 fats do not have a clear effect on total mortality, combined cardiovascular events, or cancer.” The effect on total mortality was close to significant and there was evidence of heterogeneity (i.e., studies varied) so their results were not completely negative, as the authors noted in response to comments. The effect is just weak, apparently.

In other words, after combining many experiments, each experiment with dozens or hundreds of subjects, meta-analyses can barely see an effect of omega-3. Yet I found a perfectly clear effect with one subject? An effect I wasn’t even looking for? That seems discrepant, and worth trying to explain.

My explanation is this: What I had in my favor and all those other studies did not were the benefits of self-experimentation. In particular,

1. The effect on balance was so clear that I used it to find the best dose. I found that 3 tablespoons/day was better than 2 tablespoons/day and even at 3 T/day there was an effect of time of day. So I went to 4 T/day. It seemed no better than 3/T day, so I stopped there. Conventional studies have not been able to do anything like this.

2. The effect on balance was so clear that I could use it for quality control. If I happened to buy a bad bottle of flaxseed oil I would have noticed — the results would not have been consistent, starting from when I started the new bottle. (I have gone through about six bottles.) Previous studies have had little or no quality control. If half their omega-3 went bad, they would have had no way of knowing.

3. I was strongly motivated to take the flaxseed oil. I know it is beneficial. This is not the case in any double-blind experiment when treatment is compared to placebo. In such experiments, every subject has reason to doubt that taking the pill will make a difference.

4. Dosage in nutrition, as in these mood and heart disease studies, has been built around avoiding failure — for example, what dose will avoid heart disease? Whereas I was looking for the optimum. My brain does not fail in any obvious way if I don’t have enough omega-3; it just functions worse. The amounts needed to avoid obvious failure are probably (a) different for different parts of the body and (b) less than optimal. For example, the amount of omega-3 needed to avoid dementia may be 1 T/day whereas the amount needed to avoid heart disease may be 2 T/day. The optimal amount, the amount needed for best performance, is likely to be greater than all of these failure thresholds. It is a better target.

Something else in my favor, not related to self-experimentation is that I studied the effect of omega-3 on my balance — how long before I lost my balance, a measure that can have many values. In contrast, most omega-3 research has involved binary measures like mortality or heart attacks. Someone either dies or does not die, for example. Binary measures tell you less than many-valued measures.

Given these advantages, it makes sense that I could find a much clearer effect.

Science in Action: Omega-3 (old data re-analysed)

Seth Roberts4 Comments

A few months ago I did a little experiment to test my belief that omega-3 was affecting my balance. I replaced fats high in omega-3 (flaxseed oil and walnut oil) with a fat low in omega-3 (sesame oil). Here is a new analysis of the data:

walnut oil and flaxseed oil versus sesame oil

The raw data are the same. The new analysis differs from the earlier analysis in two ways: 1. How the number for each day is computed. The old analysis dropped the first 5 trials and took the mean of the rest. The new analysis fits a regression line to balance as a function of trial to estimate an effect of trial and subtract it, then takes a mean of all the trials. 2. Allowance for improvement. The new analysis, as the graph shows, fits a slope to all the data. The improvement over days is subtracted from each day’s score before the two conditions are compared.

The old analysis gave t = 4.1 (p = very tiny). The new one gives t = 6.3 (p = very very tiny). Big improvement!

Directory of my omega-3 posts.

Science in Action: Omega-3 (what’s the best dose?)

Seth Roberts3 Comments

With a better understanding of how to measure balance, I looked again at my data about the effects of flaxseed oil. Here is a new, improved comparison of 2 tablespoons/day and 3 tablespoons/day:

2 vs 3 tablespoons/day

Very clear difference: one-tailed p = .004.

Here is a messy comparison between 3 and 4 tablespoons/day:

3 vs 4 tablespoons/day

I compared 3 tablespoons/day at 2 different times with 4 tablespoons/day divided between those 2 times. I didn’t want to take 4 tablespoons at one time and I wanted to have at least 2 tablespoons in the evening because of the sleep benefits. The graph shows that 4 tablespoons/day has about the same effect as 3.

The big picture: Earlier data convinced me there is probably an effect. Before doing more subtle, convincing, publishable experiments, I have been trying to make the effect as large as possible. For two reasons: 1. To make the effect as clear as possible. 2. To have the most beneficial possible baseline (a baseline to which I will return many times). I foresee doing an experimental design like this: baseline (n days), something else 1 (n days), baseline (n days), something else 2 (n days), baseline (n days), something else 3, and so on. During those many baseline days I want the effect to be as strong as possible.

Science in Action: Omega-3 (measurement improvement)

Seth Roberts2 Comments

I’ve learned a few things. As some of you may know, I’ve been measuring my balance by standing on a board that is balanced on a tiny platform (a pipe plug) — pictures here. Now and then the board would slip off the platform. I supposed this was a failure of balance but I wasn’t sure, especially if it happened as soon as I stood on it. So I got another board into which my brother-in-law kindly drilled the perfect-size hole so that the plug will never slip:

New board (with hole for plug)

To see if this made a difference I did an experiment with a design I have never used before but that I really like: ABABABAB… (one day per condition). In other words, Monday I tested my balance with the old board, Tuesday with the new board, Wednesday with the old board, Thursday with the new board, etc. Simple, efficient, well-balanced. Here are the results:

new board vs. old board

The red line is fit to the red points, the blue line to the blue points. The two lines are constrained to have the same slope.

Well, that’s clear. I expected my balance to be better with the new board, actually.

Speaking of the unexpected, I made another measurement improvement that truly surprises me — the surprise is that I never did it before. When I looked at my early balance data (the first 10 or so days of data) I saw that my balance improved for the first 5 trials and was roughly constant after that. Each session was 20 trials so I dropped (excluded) the first 5 trials from my analyses — considering them “warm-up” trials. I took the mean of the last 15 trials. That seemed very reasonable and I thought nothing of it.

Recently I asked again how performance changes over a session. The answer was a bit different: I found that performance improved for the first 10 trials. Now there are 30 trials in a session, so dropping the first 10 of them seemed okay. And that’s what I did.

But then I looked at how variability changed over a session. I expected the earliest trials to be more variable than the rest but the data didn’t show that. Variability was pretty constant from the first trials to the last. Hmm. Maybe I am losing valuable information by not including those early trials in my averages. It occurred to me: why not allow for the warmup effect by modelling it, rather than by excluding it? (Modelling it meaning estimating it and then subtracting it.) I did that, and then I looked at the size of the standard errors of the means (standard errors based on the residuals from the fit) for the most recent 40 days — essentially, the error in measurement. Here is what I found. Median standard errors:

First 10 trials (out of 30) excluded: 0.073
First 5 trials excluded: 0.064
First trial excluded: 0.061
No trials excluded: 0.059

My eyes opened wide when I saw these numbers. Oh my god! I was throwing away so much! A reduction in error from 0.073 to 0.059 — that’s 20% better.

How To Do Experiments That Generate Ideas

Seth Roberts2 Comments

A few days ago a graduate student in economics asked me what I thought of behavioral economics. On the positive side, I said, some of the phenomena are impressive. For example, the endowment effect, which is so strong I would demonstrate it in class. On the negative side, none of the researchers use experiments to generate ideas. They don’t merely not do it; they seem unaware of the possibility of doing it. The graduate student wondered how it can be done. I said there were three main ways:

1. Do something extra. Do a little more than necessary so that your experiment tells you about something that isn’t the focus of interest. For example, vary a factor that you think is not important. This is Saul Sternberg’s idea. I did this in my peak-procedure experiments: measured how long rats held down the bar. This was irrelevant to the purpose of the experiments, which was to understand how rats measured time. These measurements greatly surprised me. For years, I misunderstood them. Eventually they led to a new line of research about the control of variability.

2. Measure a function, not a point. Ask how your treatment changes a whole function, not just this or that numerical measure. This is what I did in my peak procedure experiments: The experiments generated for every condition an entire function showing response rate as a function of time. I saw how treatments changed the entire function. This talk describes some of the new ideas this led to.

3. Make your experiment easy and fast. The easier and faster it is, the more you can do it in lots of variations. Our ignorance of behavior being great, some fraction of these are likely to generate unexpected – and therefore inspiring — results. This is one reason self-experimentation is good for generating ideas: It is easy and fast.

I am not aware of any other written answers to this question, strangely enough.