From Andrew Gelman’s blog:
On page xxi [of Nassim Taleb’s new book The Black Swan], Taleb says how almost no great discovery came from design and planning.
I said something similar to a graduate student last week: Really useful discoveries are almost never the result of trying to do something useful; they are almost always due to accidents. Penicillin, for example. If you notice something by accident, it must be a big effect otherwise you wouldn’t have noticed it. That’s a great place to start: A big effect you didn’t know about.
I’ll have to see what else The Black Swan says about this. It makes self-experimentation look really good: (a) It’s much easier to to do a self-experiment than to do a conventional experiment so there is more chance of accidents; and (b) because we pay close attention to ourselves, it’s much easier to notice the unexpected with self-experimentation than with conventional research. Every useful finding in my long self-experimentation paper — breakfast, morning faces, standing, morning light, sugar water — came from an accidental discovery. In four of the five cases, the accident happened during a self-experiment; I varied something to see if X would change and noticed that Y changed. The exception was sugar water, whose appetite-suppressing effects I noticed while traveling. Hmm. Maybe travel is a type of self-experimentation. Or self-experimentation a type of travel. Certainly they are closely related.