Via Proinvests.com, Nassim Taleb said this:
I was in Korea last week with a collection of suit-wearing hotshots. On a panel sat Takatoshi Kato, IMF Deputy Managing Director. Before the discussion he gave us a powerpoint lecture showing the IMF projections for 2010, 2011, …, 2014. I could not control myself and got into a state of rage. I told the audience that the next time someone from the IMF shows you projections for some dates in the future, to show us what they PROJECTED for 2008 and 2009 in 2004, 2005, …, and 2007. They would then verify that Mr. Takatoshi and his colleagues provide a prime illustration to the “expert problem”: they serve as experts while offering the scientific reliability of astrologers. Anyone relying on them is a turkey.
This allowed me to show the urgency of my idea of robustness. We cannot get rid of charlatans. My point is that we need to build a society robust to charlatanism and expert-error [emphasis added], one in which Mr. Takatoshi and his staff can be as incompetent as they want without endangering the general public. We need less reliance on these people and the Obama administration has been making us more dependent on the “expert problem.”
I completely agree. This highlights two hidden strengths of self-experimentation.
First, the more you can rely on data about yourself, the less you need to rely on data from other people, which until recently — internet forums, CureTogether.org, PatientsLikeMe.com — almost always came to you through experts (usually doctors), who filtered it to suit their purposes. When I was a grad student, I had acne. My dermatologist prescribed tetracycline, a powerful and dangerous antibiotic. Studying myself, I quickly figured out tetracycline didn’t work. My dermatologist had failed to figure that out — that it didn’t work in at least some cases. In his practice, he must have encountered examples of this, but he ignored them. It served his purposes to think it worked. That’s one sort of filtering: Ignoring inconvenient data. Self-experimentation made me less reliant on my dermatologist.
Second, self-experimentation allows researchers such as myself to do innovative research in some area without getting permission from experts in that area. Self-experimentation is very cheap; no grant is required. A self-experimenter can be as heretical as he cares to be. My research on weight control has been breezily dismissed by nutrition professors, for example. Obviously they wouldn’t fund it. The Animal Care and Use Committee at UC Berkeley turned down my application to do rat research about it — my ideas couldn’t possibly be true, they said. My research on mood isn’t just utterly different than what clinical psychologists and psychiatrists say to each other in meetings and papers; it also, at first glance, sounds absurd. Self-experimentation allowed me to do it. That’s another sort of filtering: control of what research gets done.
I don’t think conventional research in nutrition, clinical psychology, or psychiatry is worthless — far from it. I think it is very valuable. (For one thing, it helped me see that my self-experimental conclusions, as unorthodox as they were, had plenty of empirical support.) What is hard for outsiders to grasp is how what they see — what they read in magazines and newspapers and even books — is heavily filtered to conform to a party line. Plenty of research supports the Shangri-La Diet, for example (such as research about the set point theory of weight control), but you are unlikely to read about it in, say, The New Yorker because it doesn’t fit conventional ideas. Plenty of conventional research supports my ideas about mood, but you are unlikely to read about that research because it doesn’t support the party line of “dopamine imbalance” causing depression or whatever. This is what Leonard Syme taught his public-health students — that the party line was a lot more questionable than an outsider would ever guess. They hadn’t heard that before. (And it was unpleasant: Uncertainty is unpleasant.) This is a third sort of filtering: What data reaches outsiders.
I never had a teacher like Leonard Syme — I’ve never even heard of someone else doing what he did — but self-experimentation taught me the same thing. I came to see the fragility of mainstream claims about all sorts of things related to health. As Taleb says, we are used to thinking the charlatans are on the fringes. But they’re not — there’s plenty of them at the centers of power.
Thanks to Dave Lull. Frontline’s recent show “The Warning” makes the same point as Taleb, that there is great incompetence at the highest levels of power.