Nassim Taleb on Research Strategy

In Forbes, Nassim Taleb, author of The Black Swan, made some comments I like:

Things, it turns out, are all too often discovered by accident. . . . Academics are starting to realize that a considerable component of medical discovery comes from the fringes, where people find what they are not exactly looking for. It is not just that hypertension drugs led to Viagra or that angiogenesis drugs led to the treatment of macular degeneration, but that even discoveries we claim come from research are themselves highly accidental. They are the result of undirected tinkering narrated after the fact, when it is dressed up as controlled research. The high rate of failure in scientific research should be sufficient to convince us of the lack of effectiveness in its design. If the success rate of directed research is very low, though, it is true that the more we search, the more likely we are to find things “by accident,” outside the original plan.

If the success rate per test is low, a good research strategy is to start with low-cost tests. Ants do this: They search with low-cost tests (single ants), exploit with high-cost tests (many ants). I don’t think the need to use different tools at different stages in the scientific process is well understood. John Tukey used the terms exploratory data analysis and confirmatory data analysis to make this point about data analysis but distinguishing exploratory and confirmatory experimental design is much less common.

I think my self-experimentation has been productive partly because it is a low-cost way of testing. All my interesting discoveries were accidents. My latest omega-3 research started with an accidental observation.

Can Professors Say the Truth? (radio show)

Part of today’s KQED Forum program with Michael Krasny was devoted to the attacks on Michael Bailey and his book. Here is the webpage. Joan Roughgarden, a professor of biology at Stanford, was one of the guests. After Bailey gave a talk at Stanford in 2003, Roughgarden wrote an op-ed in the student newspaper that contained the following sentence:

To many observers, Bailey appears to be a rather dumb, stubborn, dense and possibly deceptive regular guy with some experience in locker-room humor.

This sort of comment would go over poorly on KQED, so what would she say? It turns out that she calls Bailey’s book a “fraud.” It is fradulent because it is not “science” — by which she means a scientific article — in spite of having the word science in the title. Apparently Roughgarden thinks that if you write a book about science it is fraud to use the word science in the title. She also complains that Bailey uses stories based on transsexuals he had met to illustrate Blanchard’s theory. She calls those stories Bailey’s evidence for the theory, ignoring the evidence in Blanchard’s papers. This is not quite the incisive criticism we might expect from a Stanford professor.

Jane Jacobs and Traffic Tickets

Rexford Township, Michigan, has started to pay police officers according to the number of tickets they write. In Systems of Survival, a book about moral systems, Jane Jacobs criticized something similar: ticket quotas for police. Treating guardians (such as police) as if they were in commerce doesn’t work well, she wrote. There are two ways of making a living (taking and trading). Both have value, but they need to follow different rules of conduct (which we may grandly call morals) to work well.

What Goes Around Comes Around

One of my favorite stories in Discover Your Inner Economist by Tyler Cowen:

During one riot in Michigan, one woman sold stones to rioters. . . . Small stones went for $1, larger stones brought in $5 a piece. Most of the rocks were thrown at police. . . . The woman claimed that she collected about $70 from her efforts, but she stopped when she was hit by a rock herself.

A perfect illustration of the title of this post.

What makes a good story? Perhaps 1. Hero. 2. Villain. 3. Struggle. 4. Details. 5. Humor. 6. Message. 7. Goodness is rewarded or sin is punished. 8. Truth (it actually happened). The rock story has six of these.

A Curious View of Obesity

From a recent issue of JAMA:

Several lines of evidence from the study of patients with brain diseases converge on the prefrontal cortex (PFC), especially in the right hemisphere, as a critical area involved in the cognitive control of food intake. The PFC is the part of the brain that has undergone the biggest expansion during evolution, accounting for approximately one third of the surface of the human brain (Figure). Many complex aspects of behavior that distinguish humans from other species originate here, through the confluence of sensory, limbic, and autonomic information. Current theories on the PFC posit a crucial role for this region in the top-down control of behavior, especially under conflicting situations, when inappropriate responses need to be inhibited.

Diverse findings suggest a crucial role of the PFC in obesity. In the mid-1900s, overeating and weight gain were a common side effect in patients who underwent frontal leukotomy, a psychosurgical procedure that disconnects the frontal lobe from the rest of the brain. Damage to the right frontal lobe can cause a passion for eating and a specific preference for fine food, the so-called gourmand syndrome. In patients with degenerative dementia, the presence of hyperphagia correlates positively with right frontal atrophy and negatively with left frontal atrophy. . . . Hyperactivity of the right PFC can lead to anorexia-like symptoms, for example, in patients with right prefrontal focal epilepsy, in which the eating disorder can cease after initiation of anticonvulsant therapy.

Additional data support a link between the right PFC and spontaneous physical activity. . . . The right PFC is preferentially involved in guiding decision making according to social conduct and comprehension of bodily information at a higher level. . . . A dysfunction of the right PFC may represent a central event in the etiology of human obesity. . . . Increasing the activity of the right PFC might decrease appetite and reestablish inhibitory mechanisms controlling eating, as well as improve long-term adherence to interventions such as diet or exercise therapy, which is a major barrier that limits the success of any attempt to treat obesity.

No data to support this prediction are given. I think exercise and diet therapies usually fail because they are based on too-simple ideas about weight control. What does the gourmand syndrome reveal, I wonder.

Reference: The Right Brain Hypothesis for Obesity. Miguel Alonso-Alonso, MD, MPhil; Alvaro Pascual-Leone, MD, PhD. JAMA. 2007;297:1819-1822.

Can Professors Say the Truth? (part 4)

Deidre McCloskey and Lynn Conway — the subjects of my previous post on this topic — are both powerful persons. McCloskey is Distinguished Professor of Economics, History, English, and Communication, a title created just for her. In October 2007, she will receive an honorary degree from Goteborg University. Conway is a member of the National Academy of Engineering. McCloskey and Conway abused their power when they attacked Bailey.

As awful as their actions were, even worse is what Northwestern University administrators (led by Provost Lawrence Dumas) did: Let themselves be used as tools in the attack. McCloskey and Conway master-minded the filing of an absurd human-subjects complaint against Bailey — and Northwestern took it seriously! As Bailey says, it was “obvious to Northwestern officials” what McCloskey and Conway were trying to do (ruin Bailey) and why. It was like the teacher in a playground taking the side of the bully. Except worse, because Bailey could have been fired.

Kudos to Alice Dreger for shining light on a very unsavory episode in American academic history.

Dreger’s paper. Part 1. Part 2. Part 3.

Can Professors Say the Truth? (part 3)

After The Man Who Would Be Queen: The Science of Gender-Bending and Transsexualism by Michael Bailey was published, several transsexuals started an extraordinary defamation campaign against Bailey. The story of this campaign, including interviews, is told in the new paper by Alice Dreger that I mentioned in earlier posts on this topic (Part 1, Part 2).

The defamation campaign was led by professors. They claimed Blanchard’s typology of transsexuals was false, of course, but never clearly explained why. Bailey’s crime wasn’t that his book spread falsehoods; it was that it spread a truth they didn’t want spread.

One of those professors was Deidre McCloskey, the author of Crossing. She wrote an amazing review of Bailey’s book. From her review:

Almost everyone in the scientific study of sex and gender has checked and balanced and resisted the Clarke Institute’s [Blanchard worked at the Clarke Institute] theory. It has proven to be wrong and has been laid aside by the mainstream of gender researchers.

Who are these “almost everyone”? McCloskey never says. And it’s a long review.

Lynn Conway, a professor of electrical engineering at the University of Michigan and a member of the National Academy of Engineering, constructed a website called “An investigation into the publication of J. Michael Bailey’s book on transsexualism by the National Academies”. This big website has little to say about Blanchard’s typology other than this, written by Conway:

It is unfalsifiable (note: any trans woman who reports that she doesn’t fit the classifications is explained by the “theory” as being a “liar”). Furthermore, the scheme has no predictive capabilities. Thus it is thus untestable.

Well, which is it? “Proven wrong” by “almost everyone” (McCloskey) or “unfalsifiable” and without “predictive capabilities” and “untestable” (Conway)? McCloskey and Conway must have talked many times. This discrepancy in how they attacked Blanchard’s theory shows how little they cared about its truth — or that they knew it was true.

For people engaged in what they called a noble cause (defending transsexuals), McCloskey and Conway showed a remarkable disinclination to tell Dreger what they had done. Dreger tried hard to interview both of them.

McCloskey gave Dreger some brief email answers and then

refused to tell me anything more substantial unless I first proved to her, by showing her what I was writing, that I agreed with her position.

As for Conway, Dreger was unable to reach her at the University of Michigan. Finally she called Conway at home:

We had a phone call that lasted about a minute (August 16, 2006). She surprised me by being extremely hostile at the outset. She also would not answer a question about whether she was willing to speak to me on the record. This confused me — why would she not just tell me whether or not she wanted to speak on the record. I said as much. She responded that it was very strange that I would call her at home. I told her how many other ways I had tried to reach her with no response before finally calling her home. She then said that I was stalking her and added that she would circulate this fact widely.