3. Be minimal. In other words, do the easiest, simplest thing that will that will tell you something important and new, that will provide significant progress. This I learned by failure. In the early days of my self-experimentation — and of my rat research, too — I constantly tried to do experiments that broke this rule, and again and again they failed to work. (Slow learner.) The more complex the design, the more untested assumptions it makes. And untested assumptions, at least mine, are often wrong. I’ve been good about following this rule in my own research in recent years so to give an example of how it is broken I will describe someone else’s research. I sat in on a planning session for an experiment about asthma at a highly-ranked school of public health. The experiment was expensive — the grant to pay for it was many hundreds of thousands of dollars. There were to be about 50 families in the treatment group and 50 families in the control group. They had done some pilot work involving three families. They proposed to begin the full experiment. I suggested that they do a larger pilot experiment — maybe four families in each group. There were several professors and several more people with Ph.D.’s at the meeting. No one agreed with me. Several people explicitly disagreed: “I don’t think we need to do any more pilot work.” As it turned out, I was right. They began the full experiment and it failed miserably because recruitment turned out to be far more difficult than expected.
Almost all proposed research I hear about breaks this rule, which is fascinating in a train-wreck kind of way. I have never seen a book about research design that makes this point. As a result, I suspect that books about research design are often counter-productive: The student would have been better off if he or she hadn’t read them. The textbook teaches this or that complication to people who can barely do basic stuff. The poor student wastes time using complex designs that fail in cases where a simpler design would have succeeded.
Part 1 is here. Part 3 is here. You no longer need to register to comment.