Big public works projects inevitably cost far more than the original budget. I heard a talk about this a few years ago. The speaker gave many examples, including Boston’s Big Dig. His explanation was that these projects would not be approved if voters were told the truth. The German newspaper magazine Der Spiegel has just published an interview with several architects responsible for recent German projects with especially large discrepancies between what people were told at the beginning and the unfolding reality — Berlin’s new airport, for example. The article’s headline calls them “debacles”. One architect gives the same explanation as the speaker I heard: “The pure truth doesn’t get you far in this business. The opera house in Sydney would never have been approved if they had known how much it would cost from the start.”
I disagree. I see the same massive underestimation of time and effort in projects that I do and that my colleagues and friends do, projects we do for ourselves that require no one’s approval. I think something will take an hour. It takes five hours. Plainly the world is more complicated than our mental model of it, sure, but there is more to it than that. Someone did a survey of people in Maryland who had been in a car accident so bad they had had to go to the hospital. Within only a year, a large fraction of them (half?) had forgotten about it. When asked if within the last year they had had an accident so bad they were hospitalized, they said no. Apparently we forget difficulties, even extreme ones, really fast. If you forget difficulties, you will underestimate them.
If I had realized how difficult everything would be, I couldn’t have done any of it is one explanation, which I’ve heard attributed to Gregory Bateson. From Malcolm Gladwell’s excellent review in this week’s New Yorker of a biography of Albert Hirschman, the economist, I learned that Hirschman — had he realized that this was human nature — would have had a different evolutionary explanation: We underestimate difficulties because this way of thinking increases innovation. Debacle . . . or opportunity? Difficulty is the mother of invention.
Indeed, what would humanity be without the optimism bias? Everything we’ve done since deviating from other primates has been kind of insanely optimistic.
Seth: Maybe there is a compensating bias in the other direction (pessimism bias) because if you are unrealistically optimistic in life-endangering situations, you may die. If I underestimate how long a project will take, nobody dies. If planners underestimate how much an airport will cost, nobody dies.
Do you have a cite for the Maryland car accident study? I study self-report/recall accuracy, and I’ve never heard of it. I want to add it to my list.
Seth: Sorry, I don’t. I learned about it from a talk by Elizabeth Loftus, maybe around 1992. I think the talk is online somewhere.
Without having seen the Maryland study, I note that the standard KABC0 injury scale for car crashes has a very broad range for “A,” anything from “taken in, checked, and released” to “permanently paralyzed.” Police might also code A for cases where someone refused treatment, and comparisons of police and medical reports have shown B injuries that look bad (visible abrasions, any complaint of neck or head trauma) coded as As.
Of course people forget things, but if researches simply compared self-reporting to crash reports, they were conflating two types of error. As I said, this is without having seen the study, so it might have taken into account reporting problems.
Minor correction: Der Spiegel is a magazine, not a newspaper.
On the substantial issue, I found Elizer Yudkowsky’s post useful, though too wordy:
https://lesswrong.com/lw/jg/planning_fallacy/
I think there’s a more direct way that evolution favors that optimism bias and amnesia of the negative. Without it there would be hardly any second pregnancies. When I think about it objectively, having children has brought about some of the most grueling hours of my life, with weeks of feeding a newborn round the clock, hours of colicky howling (and that’s after pregnancy and childbirth). But those memories feel remote. The highlights feel much more real. Thank you, dopamine!
Seth: That used to be my explanation. I can’t say I have evidence against it.
FYI: https://www.npr.org/blogs/health/2013/06/18/191279201/3-d-printer-brings-dexterity-to-children-with-no-fingers?uidt=1371584750#commentBlock
An example of innovation coming from outside the establishment. Now, will the establishment embrace it, ignore it or fight to shut it down?
This is the basis for the whole venture capital industry today, and can be seen in many commerical industries at their infancy. Examples: automobiles at the start of the 20th century, dot-coms at the very tail end, plus many in between. Most VC backed companies fail or flounder and only a small few are successful, but when the capital was invested, every company in a VC’s portfolio was forecast to have a chance to be a total success.
This optimism actually creates much consumer surplus, as in the process of innovation, nobody is sure what exactly will fit the consumer need, so proliferation of ideas and oversupply a) helps identify the best idea and b) helps drive down the price quickly so that more people can benefit from the new invention. Three cheers for myopia!
Seth: Your comment reminds me of something I figured out about how to do science: To make progress, it isn’t necessary to have a correct explanation of something. Any explanation is helpful, so long as it is testable, because doing the test will be more enlightening that you realize. (Scientists are too pessimistic, engineers too optimistic.) What’s important is motivation, not understanding.
Literal myopia may also be a spur to innovation: Buckminster Fuller’s interest in structures is said to have developed early as he concentrated on what he could touch and manipulate, since the world beyond was a bewildering blur, and James Thurber reputedly derived many surreal ideas from his brain’s attempts to interpret the fuzzy images his eyes supplied to it.
Mario Puzo famously remarked that had he known that The Godfather would be so popular, he would have written it better.
Seth: One of the first managing editors of The New Yorker had a saying: “don’t get it right, get it written.”