The most important thing I learned in graduate school — or ever — about research is: Better to do than to think. By do I mean collect data. It is better to do an experiment than to think about doing an experiment, in the sense that you will learn more from an hour spent doing (e.g., doing an experiment) than from an hour thinking about what to do. Because 99% of what goes on in university classrooms and homework assignments is much closer to thinking than doing, and because professors often say they teach “thinking” (“I teach my students how to think”) but never say they teach “doing”, you can see this goes against prevailing norms. I first came across this idea in an article by Paul Halmos about teaching mathematics. Halmos put it like this: “The best way to learn is to do.” When I put it into practice, it was soon clear he was right.
I have never heard a scientist say this. But I recently heard a story that makes the same point. A friend wrote me:
I met Kary Mullis after high school. I knew that PCR was already taught in some high schools (like mine) and was curious how he discovered it. He said that he had some ideas about how to make the reaction work and discussed them with others, who explained why it wouldn’t work. He wasn’t insightful enough to understand their explanations so he had to go to the lab and see for himself why it wouldn’t work. It turned out it worked.
An example of better to do than to think.
Better to do than to think is not exactly anti-authoritarian but it is close. I was incredibly lucky to learn it from Halmos. It isn’t obvious how else I might have learned it. It took me many years to learn Research Lesson #2: Do the smallest easiest thing. And I learned this only because of all my self-experimentation. I started doing self-experimentation because of better to do than to think.
The first sentence in the Halmos article you linked is something I pound on constantly with tennis coaches:
“The best way to learn is to do; the worst way to teach is to talk.”
The current number two downloaded paper (it was number one for quite some time) on the SSRN is by an outsider, Mebane Faber. He has a bachelor’s degree in, I think, biochemistry, but the paper is in the field of finance, in which he now works. He appears to be self-taught in finance.
https://hq.ssrn.com/rankings/Ranking_display.cfm?TRN_gID=10&requesttimeout=900
As an artist, the same thing applies. Your imagination is in your fingers.
As with Mullis, most of the great discoveries have come from creative people tinkering, often using new tools. Therefore, academic science is organized so that in order to “do”, you must first write a grant application (i.e. lots of “thinking” and “writing”), which must be scored by a committee (i.e. a minimal amount of “thinking” and lots of “reading” and “talking”), and then if you’re one of the one in ten or so lucky ones who get funded you may finally get a chance for a small amount of “doing” (except it won’t really be tinkering since you’ll be limited by what you said you were going to do in the application). Hmm — perhaps if we had more committees to “set direction” and tell researchers what subjects they should be “thinking” about . . . that could work . . .
Completely agree. I tutor inmates trying to get their GED in math. The only method I’m seen have any success is for them to do tons of problems, and then do even more problems. So when I’m teaching someone how to multiply fractions, I’ll spend maybe five minutes showing him the rule for multiplying fractions, and then we spend the rest of the hour just multiplying fractions. The next time we meet, I’ll have him do even more problems. I used to try to explain what fractions are, why the rule is the way it is, but I quickly found that was a waste of everyone’s time.
Thing is, after doing several hundred problems with fractions, most are able to give a pretty good explanation of what fractions are, and why the multiplication rule is what it is.
Seth, I would disagree with your use of PCR as a supporting example. Let’s think about how PCR came to be and the influence of thinking vs doing. PCR was conceived in a theoretical sense by Mullis. He likes to tell the story of figuring out PCR, a thought that came to him while driving in California. It was from “thinking” about this problem not “doing” something.
You are right about the importance of Mullis persevering when others told him it wouldn’t work, but that is illustrating something closer to perseverance or believing in oneself.
Mullis probably would never have figured out PCR if he was pursuing a solution by “doing.” I’m not sure what that would even mean in this context but trying various experiments probably wasn’t going to lead him to this solution or at least not as fast.
In the field of health, with so many opportunities for false positives and negatives, confounding, correlation not being causation, etc I would prefer if people engaged in even more thinking about what they are doing, not less. I think the examples given in the above comments are instructive; teaching tennis, doing art or learning fractions. For those examples I would agree with the quote “the best way to learn is to do.”
These are very different from attempting to show cause and effect in factors relating to human health. For that, the “doing” can make you better at the techniques, such as collecting data. But while trying to determine truths one can easily be lead astray by not having thought enough.
JeffR, yes, you need to do a little thinking — I agree with you there. A more subtle version of my “doing-better-than-thinking” point would be about the optimal mix of thinking and doing. It would be that the optimum doing-to-thinking ratio (time spent doing compared to time spent thinking) is a lot more than you probably think or practice. (That is, most people spend far too much time thinking.) But the rest of what you say I disagree with. For example, the notion that “correlation is not causation” is an anti-doing way of looking at correlations. It discourages people from using correlations to come up with testable ideas; testable ideas encourage doing.
It’s important also to keep in mind the opportunity costs for various kinds of things under discussion. I love the math problems idea — of course the best mix would be lots and lots of doing, because the cost of doing is cheap and the price for failure is low: if you get a problem wrong, do another one. Simple. Who could argue? Same with a lot of the self-tracking stuff: if an important insight can be had, and the cost of bad experimental design, or confounded data, is low, then who cares? Do the experiment. More data is better.
But this is not always the case, and in fact is probably not often the case, at least with respect to the scientific enterprise that Seth likes to kick around. I’m in the process of doing an fMRI study, and I can tell you that a kind of wild west “just do it” mentality is about the best way I can think of to waste a hundred thousand dollars and never ever get the opportunity to do another study. A host of other examples come to mind: launching a satellite; building a house; building a highway, or a shopping center; figuring out what regions around a brain tumor can be resected with the least chance of turning someone into a cauliflower.
The “correlation vs. causation” canard is getting kind of tired, too. Yes, the mantra “correlation is not causation” is probably overused and wrongly used. And yes, Seth is right to point out that following correlations can be an incredibly fruitful tool for finding interesting questions. But why can’t you be satisfied with making an insightful and reasonable comment without barreling headfirst into a tree? Correlation is _not_ causation; and when the scientific question is subtle, potentially misleading, and expensive to misdiagnose, you damn well better understand why the difference matters.
The goal with all of this shouldn’t be to get everyone to renounce the distinction between the two (thinking vs. doing, correlation vs. causation) but rather to know when to care about that distinction, and how much.
Shane, I was talking about research, not building (“building a highway or shopping mall”) or brain surgery. With an fMRI experiment, you are forced to spend a lot of time thinking since the cost is so high.
“Correlation is _not_ causation.” That’s a truism. Not helpful. In this post I was summarizing my experiences. If you have had an experience that suggested a correlation was overinterpreted, that would be good to know.
“Shane, I was talking about research, not building or brain surgery.”
Brain surgery is research. Where someone cuts depends on a large body of research. If that research is sloppy, the putative function of some bit of brain may turn out to be rather more important wrt some particular functionality than was thought.
““Correlation is _not_ causation.” That’s a truism. Not helpful.”
Activity, or lack thereof, in anterior temporal lobe correlates nicely with progression of semantic dementia. The same activity also correlates nicely with social processing. So what can you say about social processing based on ATL activation? Not too damn much. What can you say about ATL wrt general semantic processing, the kind lost in SD? Also not too damn much. What you can say, based on these very nice correlations in both cases, is that ATL does something, at some point, during the course of both types of processing, and yet that hasn’t stopped a lot of overprediction based on correlation worshiping.
In public health school, an example they use to teach us that correlation is not necessarily due to causation (to fancy up the expression a bit) is of initial research that indicated coffee was a cause of pancreatic cancer. Additional research lead to the realization that pancreatic cancer is caused by cigarettes and smokers drink coffee more often. It’s a little more complicated than that, more available
heret.
Erik La Gattuta Says “As an artist, the same thing applies. Your imagination is in your fingers.”
Exactly my thoughts when I read the post.
Either I have the perfect image in my head in the moment I hear the assignment, or there is no other way then to pick up a pencil. I really tried the tough way – sitting half a day around thinking, trying to get an image in my head. But it just doesn’t work. Starting to draw does the trick. I nearly suspect humans are just not very good at thinking…
Your text just puts it more poetic than I could have said it. Kinda adapted on my page, hope thats ok.
To me the difference between doing and thinking is the scale. When we think something physical is happening so it could be considered that we are doing something. What concerns me about doing on a larger scale is there is always a reaction of which we are only assuming what the results will be. I do not feel well informed enough to make a decision as to whether or not it is good or bad, it just is. For me I hope all that I do is good, but all I can trust is my gut instinct which has been too conditioned since birth to completely trust. I think the answer lies deeper than we could possibly be aware of, but I’ve noticed plenty of signs along the way that make we question the expectations I’ve been following. Its usually when I stop and reflect on past actions. This may reason that no action would be the answer, however with no action I’m assuming we wouldn’t exist. This leads me to believe one again there is a balance and its going to vary for every single person at different times.
How ironic (yeah, I know) that the ad on the top-right is “Anything’s possible. Keep thinking.”
Do I really have to point out that to think is in the subset of verbs that are to do? My experience has been, for anything sufficiently complicated thinking comes first.
I should also point out that there are plenty of studies that suggests that thinking about doing something is as good or better than practicing that thing. Of course, there is some physical practice required for this to work, and maybe that is what the poster is trying to get at.
As someone who does *a lot* of troubleshooting and is very good at it, I can tell you how I do this one activity where thinkers outperform doers. I get some information about a system that may be broken. I then time thinking about what could cause the symptoms, that is, I simulate the system in my head, and twist and turn knobs until I can see in my head something that might have symptoms similar to the knowledge I already have. Now I have a hypothesis. Now I can test my hypothesis. I gather more data, then I adjust the simulation in my head (occasionally I even get such new information that I actually need to adjust the model in my head of the system I am working on). Now I think some more, running some more simulations until I have a new hypothesis to test. Rinse. Repeat.
There’s always a trade off here though. Thinking is usually cheap, but it can take you places that could never happen. Doing is usually expensive, but you’ll never do something impossible.
Thinking prevents you from doing something that won’t work. Doing prevents you from thinking about what can’t work. Each can save you time and effort. And the ratio changes depending on their relative costs for a given person and a given problem space.
In this post I am talking about scientific research at the low end of the cost/hour scale. When something is well-understood theoretically, as is usually true in engineering (e.g., electrical engineering), I agree you should use the theory (“thinking”) to guide what to do.
I think you should set some time for thinking and if you still don’t get it, then it’s time to start doing it.
Yeah, that makes sense. I suppose I usually “don’t get it”.
Very relevant: https://www.brepettis.com/blog/2009/3/3/the-cult-of-done-manifesto.html
(Bre Pettis is a cofounder of the NYC Resistor hackerspace, and of MakerBot Industries)
I agree with what you write here, but on the other hand I also keep in mind the adage that “Two years in the lab can save you as much as two weeks in the library!” – there needs to be a balance between the effort spent thinking and the effort spent doing.
I wouldn’t call time spent in the library “thinking”. I call that reading. And I have learned plenty of useful things from reading–my reading has led me to many self-experiments that had useful results.