At the latest Quantified Self Meetup, I described my recent arithmetic results. Gary Wolf asked why I used arithmetic to measure brain function.
The simplest answer is that it won a competition. I tried eight or so different ways of measuring brain function and it was the best. I had high hopes for all of the tests and spent a lot of time on most of them. I hoped to find a test that was as fun as playing a game yet provided the detailed and controlled info of a typical cognitive psychology task. I didn’t quite get there.
The other tasks had several problems:
1. Long learning curve. My belief that our environments might be having big and unnoticed effects on how well our brains work began when I noticed that my balance was better the morning after taking 6 flaxseed oil capsules. A small amount of flaxseed oil substantially improved how well my brain worked — at least on one dimension. The test I devised to measure this effect confirmed its existence but after doing it for a year my score was still improving. Obvious learning during testing makes it harder to interpret the results — you need to correct for the learning. I wanted a task where there was less room for learning. I already know arithmetic well so an arithmetic test took less practice until learning stopped.
2. Not so portable. Some of the tasks required props (e.g., a printed sheet of paper for a paper-and-pencil test).
3. Trial-to-trial interference. Wanting to cram as many trials as possible into a short time, I started with an arithmetic task with 5 blocks of 20 trials each. There were pauses between blocks. During a block, however, as soon as I finished one question (e.g., 3+4), the next one would appear. At first, this was fine. As I got faster, though, I started to make a large number of mistakes quite often. I required 85% accuracy and it started to take longer and longer to do 5 blocks with acceptable accuracy. The arithmetic task I use now has separated trials. Instead of 100 trials in 3 minutes I do about 32 trials in 3 minutes, but there has been no increase in the amount of time they take. (I found that 3 minutes was as long as I would comfortably do the test; 32 is simply the number of trials I can do in 3 minutes.)
4. Carpal Tunnel Syndrome. After doing a lot of an earlier task that involved just two fingers, my wrists started to hurt. The arithmetic task uses eight fingers (excluding the thumbs) equally. 1, 2, 3, 4, 7, 8, 9, and 0 are possible answers; 5 and 6 are never answers.
5. Subjective. One of the tasks I tried involved making a single motion on the touch pad. It wasn’t easy to be sure I’d made one motion rather than two.
6. Small number of measures. One task I tried only generated five scores in four minutes. The arithmetic task I use generates 32 scores in three minutes. Each of the 32 scores can be corrected for the difficulty of the problem (e.g., 0+0 is faster than 6*7).
7. Unpleasant outcomes. Some of the tasks were frustrating. When I didn’t do well it was irritating. For some reason that doesn’t happen with the arithmetic task.
A more game-like task would be better but for that I may need a new input device, like a joystick. R — the language I use to collect the data — is great for data analysis but poor for data collection. it is unlikely to work with joysticks anytime soon. My arithmetic task needs an R function that only works with Windows, which shows how bad the problem is. It’s a miracle that R function exists. Maybe it was written by an experimental psychologist.
Any chance you would be willing to post the source to your R program(s)?
I was wondering the same thing as Mike … step-by-step instructions would be much appreciated.
I really enjoyed your talk at the QS meetup Professor Roberts.
Reaction time with simple mental tasks measure peak performance. Whereas arguably one may find average performance more directly relevant for day-to-day work. And statistically it may enjoy smaller measurement variations for making inferences.
It was the first QS event I have been to. Hope to hear more of your findings in the future. Thanks!
Justin & Mike, thanks for your interest. Okay, I will post the R code.
Ming-Chi, I would think that peak performance is less variable than average performance. In peak performance, everything is pushed up against a ceiling, not so in average performance. I also don’t understand how average performance could be measured.
What were some of the other tasks you rejected?
I had a friend who played a great deal of blitz chess on ICC. They give you a quantified rating. He had a lot of variation between 2100 and 1600. Thinking back, I wonder if that wasn’t a good measure of his brain function. He had a lot of late nights. On a particularly late (early?) morning even I beat him once.
I think the only factor I would control for is how much chess I played recently. I noticed that my rating would go up if I played a lot and go down quickly if I quit. Maybe that’s just too big of an effect.
Anyway… what did you reject?
I rejected: 1. a two finger classification task. 2. a balance task. 3. a short-term memory task (digit span). 4. a paper-and-pencil classification task. 5. an arithmetic task where the problems are blocked rather than separated. 6. a one-finger typing task with preview (the string to be typed is shown before the signal to start typing). 7. a one-finger typing task without preview. 8. A time-discrimination task (the interval between clicks). 9. A task requiring me to move the pointer a precise distance using the touchpad. 10. A task where I move the pointer from place to place on the screen as quickly as possible.
Task 9 was the most fun and I’m sorry I couldn’t use it.
I’d also be interested in seeing the source code for this. Pretty please
i see. sounds interesting. keep us posted.