In a recent post I described Tim Lundeen’s arithmetic data. He found that increasing his daily dose of DHA seemed to increase the speed at which he did simple arithmetic. Here is the graph:
I didn’t bother to do any statistical tests because I thought the DHA effect was obvious. However, someone in the comments said it wasn’t obvious to them. Fair enough.
If DHA has no effect, then the scores with more DHA should be the same as the just-preceding scores with less DHA. There are practice effects, of course, so I analyzed the data after practice stopped having an effect: After about Day 40. (And I left out days preceded by a gap in testing — e.g., a day preceded by a week off.) Thousands of learning experiments have found that practice makes a difference at first and then the effect goes away — additional practice doesn’t change behavior.
If I do a t-test comparing low-DHA days (after Day 40) with high-DHA days, I get a huge t value — about 9. If you’re familiar with real-life t values, I’m sure you’ll agree that’s a staggeringly high value for a non-trivial effect. The model corresponding to this test is indicated by the lines in this figure:
The red (”more DHA”) points don’t fit the line very well, which suggests doing an analysis where the slopes can vary:
There is still a huge effect of DHA, now split between two terms in the model — a difference-in-level term (t = 4) and a difference-in-slope term (t = 3).
But this analysis can be improved because based on thousands of experiments I don’t believe that the less-DHA line could have a positive slope, as it does in the model. Or at least I believe that is very unlikely. So I will constrain the less-DHA line to have a slope of zero:
Now I get t = 8 for the difference in slopes and t = 4 for the difference in level. This is interesting because it implies that more DHA not only caused immediate improvement but also opened the door to more gradual improvement (indicated by the slope difference). DHA changed something that allowed practice to have more effect.
That’s a new way of thinking about the effects of omega-3 — actually, I have never seen any data with the feature that a treatment caused a practice effect to resume — so I have to thank the person who claimed the difference wasn’t obvious.
I’m glad to see that fitting some regression lines was helpful. It looks like the positive slope you find so strange is largely due to the two outliers toward the end of the 400 mg/day period. If you toss those or switch to some form of robust regression, the slope will probably flatten out. (But that point you will have thrown out quite a bit of data)
Seth, fyi the last photo in this post gets a “this photo is currently unavailable” error from flickr.
I had also noticed that my scores seem to be improving over time since increasing DHA, and my sense is that this will continue; partly because I do’t get “hung up” on individual problems as much, and I am able to pipeline more effectively (e.g. look ahead and start the next problem while writing down the answer to the current one).
The only other change I’ve made in the same period is to start taking 20mg/day of resveratrol from Life Extensions, which I started about 60 days before the increase in DHA. So seems like the improvement in learning rate is due to either DHA (most likely) or resveratrol (which, because it improves blood glucose management, could improve brain function over time). Anyway, it is certainly nice, and I am subjectively more effective in my work as well.
Your analysis is provocative, and it’s convincing enough to make me try it myself.
To really be convinced, though, I would want to see the trial periods shortened to a few weeks instead of a few months, the order of the trial periods randomized, a placebo oil to substitute for lower dose of flaxseed if he didn’t do that this time, and blinding as to which dose of flax seed in each time period. Since the trial periods are so long, the effect could be explained by some type of seasonal variation (e.g., daylight length, some other difference in the experimenter’s life during the two 3 month periods), and since that he expected better results on higher dose, and we know that people improve their test performance if they expect they’ll do better (e.g., stereotype threat), blinding would prevent that.
Btw, what R command do you use to fit the two lines for different range of the x axis?
Those are good points, except how can someone expect an effect (namely, the downward slope with more DHA) that has never been seen before? (Not by me and not by Tim, at least.)
Re R. I got the fit with
fit = lm(time~day)
and plotted it with
lines(day,fit$fitted.values)