Martin Wolf relays what passes for wisdom:
Albert Einstein is reported to have said that insanity consists of doing the same thing over and over again and expecting different results.
Which, if true, shows that Einstein was a theorist.
Call me insane. Based on many years of data collection, I believe scientific progress has a power-law distribution. You sample from this distribution when you collect data. You collect data again and again — “doing the same thing over and over again”. Almost all the data you collect produces little progress; a tiny fraction produces great progress. The secret to scientific progress is doing the same thing over and over — and being wise enough to grasp that the results will vary greatly. (Nassim Taleb understands this.) In the short term, it seems like you are getting nowhere.
I learned this lesson from my sleep research. For ten years I tried various solutions to my problem of early awakening. Nothing worked. All my ideas were wrong. Eventually I got “lucky” but actually I made my own luck by persisting so long.
Once you realize the distribution of progress, you grasp that the secret of success is making the cost per sample as low as possible. Few scientists, in my experience, have figured this out. They prefer expensive experiments because larger grants signal higher status. Won’t fancy equipment tell me more? they rationalize. Grant givers, also failing to understand the basic point, are happy to oblige the status-seekers: Much easier to administer one $200,000 grant than 10 $20,000 grants. And progress slows to a crawl.
More Rita Mae Brown is a more likely source of this saying than Albert Einstein.
It depends what you mean by “the same thing.” Does repeatedly collecting data about a problem while trying various solutions count as doing the same thing? I’d say no – the various solutions that you’re trying are different things that you’re doing, and the data that you’re collecting are the results which you’re checking to see if they’ve come out different.
Lots of quotes are falsely attributed to Einstein (and Mark Twain, Abraham Lincoln, Samuel Johnson…).
I second Vince. I think you’re confusing perseverance with “doing the same thing”. I don’t think Einstein (or whoever else) would’ve dissed perseverance. That’s just anathema to research, and pretty much excellence in any sphere.
https://www.telegraph.co.uk/health/8594293/Crash-course-diet-reverses-Type-2-diabetes-in-a-week.html
What is the difference between perseverance and doing the same thing again and again?
One difference between perseverance and doing the same thing again and again is that perseverance includes doing slightly different things again and again.
Would you describe what’s required for the focused practice needed for the 10K hours to mastery as perseverance?
“perseverance includes doing slightly different things again and again.” It does? I have never heard that definition of perseverance. Is it in any dictionary?
Nancy, no, the long practice you need to become good at something is inevitably varied.
I always read this quote if you are doing exactly the same (e.g. collect the same data and do the SAME experiment) then you are insane. For example, in trying to lose weight, trying the same diet (e.g. chocolate-chip cookies), but each time expect to lose weight. This is insanity.
I think that the difference here in the process. The quote says that if you want different results, you need to change something in the process.
So to answer this question: “What is the difference between perseverance and doing the same thing again and again?” Perseverance is in the goal (e.g. lose weight, find inner piece, etc.). Doing the same thing again and again – is in the process (trying the same method over and over).
It looks like you have shown perseverance without insanity. You have tried different methods to achieve your goal.
I agree with the others: you’re misinterpreting Einstein’s quote.
To understand how, examine Thomas Edison: “While working on the nickel/iron storage battery, he performed 10,295 failed experiments before achieving success.”
https://www.yeartosuccess.com/members/y2s/blog/VIEW/00000021/00000257/Inspiration-from-Thomas-Edison.html
That’ 10,295 *different* experiments. If he’d performed the same experiment 10,295 times, expecting a different outcome, he would have clearly been insane. Edison was known as the “Godfather of Perseverance”.
Repetition is never exact, in my experience. If Einstein’s quote is to mean anything — is to apply anywhere — it must refer to inexact repetition. In the article by Wolf from which I took it, it refers to inexact repetition.
But the exact meaning of “repetition” and “perseverance” is a digression. The point of this post is that I learned something from my self-experimentation (the power-law distribution of progress) that I hadn’t known, that makes a difference in what scientists should do, and that I have never heard anyone else say. I did know about Edison (“genius is 99% perspiration”), but thought of him as an engineer, not a scientist. If the power law distribution of scientific progress is well-known, I would be happy to be corrected.
To me the salient point in this post is “the secret of success is making the cost per sample as low as possible”. Now that’s a great quote, Seth!
In my world of web analytics, there is a lot of interest in doing AB testing/controlled experiments. The challenge is to get some successes from time to time, otherwise people will lose interest. And the only way to get there is to do lots of experiments, which is only possible if the implementation of these AB tests is quick and cheap.
Dear Professor–
If you take a narrow view of ‘doing the same thing over and over again’, then Einstein’s own statement would define Einstein as insane. After all, it took him 10 years to formulate general relativity after the special case (1905 to 1915) and he spent most of this time thinking and occasionally writing it down. But I think, and I’m sure you agree, what Einstein means is that insanity is attempting the same thing over and over again and expecting different results, all other things being equal. I think Olga is correct when she gives the example of eating chocolate chip cookies.
Now things get a little messy for me, so maybe you can help me out: it is certainly true that the statement is impotent if it only applies to noise-free cases, but I think a fair reading would suggest that in cases where noise and variability are low, trying the same thing over and over and expecting different results is foolish. Certainly, it’s hard to draw lines here, but I think we would agree that collecting data over a sufficiently long period of time, finding that your approach produces insignificant results, and attempting it again under the same circumstances with no modification, is foolish.
brilliant post
Good observation.
Would a fair summary be: “Self-experimention is antifragile; it benefits from volatility rather than suffering because of it”?
Perhaps Taleb already includes personal science under “aggressive tinkering/convex bricolage”. Ask him next time he’s around.
I’m certainly in favor of doing lots of cheap, easy experiments. But I’m not completely convinced that it’s always the best thing. Thinking back say 30 or 40 years ago when it was much harder to do lab measurements (in physics labs, for example, especially pre-computer) because the equipment cost a lot, budgets were smaller, and generally speaking it was harder to make measurements, to analyze the results you generally had to think a lot harder before setting up an experiment, and you had to work harder to understand what the results were telling you.
The result was you learned more at both ends (before and after) of the measurements. The drawback was that it was harder to discover if you had observed a special case or exception. I’m not sure that this trade off has changed in favorable ways over the years.
Tom, you seem to be saying that if you slow down research you are forced to think harder and therefore do more productive research. This is exactly what I thought BEFORE I started doing self-experimentation. Before self-experimentation, I believed it was a good idea to think long and hard before doing an experiment. When I started doing more frequent experiments, I came to see that I was completely wrong about this. I made more progress in one month of doing fast “unthinking” “sloppy” experiments than I had in a year of doing careful thoughtful ones. I think your idea that slower is better is an illusion. If you were actually able to compare different speeds of research I predict you would find that faster is better.
That’s certainly how I program, and I don’t know how I’d be able to do it any other way, not with effectiveness. And it’s how I *like* to work in other areas, too. It suits me pretty well. But I’m not convinced it’s a one size fits all thing.
I’m not exactly saying if you slow down, you will automatically do better work. I was reflecting on when our circumstances or equipment forced us to be slower and more thoughful. It’s not quite the same thing. I think – just based on what I noticed in myself and others – that when it became easier to work quickly, people tended to lose touch with the phenomena (or the world) – sometimes, anyway. It takes a different attitude to make good use of little throwaway steps, and maybe that doesn’t work well for everyone or for all areas.
But personally, it suits me fine!