Two days ago I explained why the test I was using to measure my mental function many times/day had room for improvement. I wanted a new test much like the old test but with which my accuracy was higher.
I was more accurate with the simple arithmetic test (e.g., 3 + 6) than with the memory test I described two days ago. The crucial difference might have been the number of possible answers. The arithmetic test had 40-odd possible answers; the memory test had 2 (yes and no). Saul Sternberg did a reaction-time experiment in which the number of possible answers was varied from 2 to 8. I don’t know what the accuracy data were but the variance of the reaction times was lower with 8 possible answers even though reaction times were longer. A plausible explanation is that there was much more anticipation with 2 possible answers than with 8. Anticipation can cause errors.
The new test I am trying consists of typing how many letters from the set {A, B, C, D} are among a set of four letters chosen from a much larger set (most of the alphabet). The possible answers are “1″, “2″, “3″, and “4,” each equally likely. For example, I might see T B X A. The correct answer is “2″. I am using R (the programming language) to run this test so I type “2″ with one hand and hit Enter with the other as fast as possible.
Here are the results so far from the new test — the training phase.
These values are taken from fit of a linear model; they are similar to means. As I gain experience with the test I am getting faster. The new test is slower than the old test (which is good — more mental processing).
Consistent with what Sternberg found, variation in reaction times is less with the new test than with the old test even though average reaction times are greater:
This graph shows the standard deviation of residuals from the fitted model. The units are reciprocal seconds (x 10) because I did a reciprocal transformation before fitting the model. The reciprocal transformation made the reaction times close to normally distributed.
Here is accuracy:
The new test feels easier than the old test, but so far there is little difference.
Overall it seems to be a step in the right direction. Reduction in variation of reaction times means more sensitive measurements.
The experiments I am planning are very simple: Test myself regularly (say, every half-hour), eat something. If the measurements are steady, it is very easy to see an effect. As far as I know, such experiments have never been done. One reason, I think, is that they require self-experimentation: It is no trouble for me to do the test (which takes 4 minutes) 100 times in a week and thereby reach a steady state. But to have someone else do the test 100 times as preparation — especially if the test were done in a lab — would be very difficult.