Author: Seth Roberts

Spider Science

Seth Roberts3 Comments

The success of my self-experimentation has puzzled me. The individual discoveries (a new way to lose weight, a new way to improve mood, sleep-related stuff, the fast effects of omega-3) seem normal; someone would have found them. It’s their combination that’s strange. Scientists who study weight control do not discover anything about mood, for example. But I did.

An ancient (2001) essay by Paul Graham is about how the future lies with web-based applications. No more Microsoft Word. One of Graham’s stories sheds light on my puzzle:

I studied click trails of people taking the test drive [of Graham’s web-based application] and found that at a certain step they would get confused and click on the browser’s Back button. . . . So I added a message at that point, telling users that they were nearly finished, and reminding them not to click on the Back button. , . . The number of people completing the test drive rose immediately from 60% to 90%. . . . Our revenue growth increased by 50%, just from that change.

I studied click trails. He examined a rich data set, looking for hypotheses to test. I practiced what I’ll call spider science: I waited for something to happen. When it did, I started to study it, just as a spider moves to the part of the web with the fly. Here are examples:

1. A change in what I ate for breakfast caused me to wake up early much more often. I did many little experiments to find out why.

2. Watching TV early one morning seemed to have improved my mood the next day. This led to a lot of research to figure out why and how to control the effect.

3. After I started to stand more, my sleep improved. I made many measurements to see if this was cause and effect and if so what the function looked like (the function relating hours of standing to sleep improvement).

4. In Paris I lost my appetite. This started the research that led to the Shangri-La Diet.

5. The morning after I took some omega-3 capsules, my balance improved. This led to experiments to see if it was cause and effect and if so what the function (balance vs. amount of omega-3) looked like.

6. One day I took flaxseed oil at an unusual time. My mental scores suddenly improved. I started to study these short-term effects.

7. While studying these short-term effects, I noticed improvements shortly after exercise. I started to study the effect of exercise.

Graham studied click trails partly because he could so easily act on anything he learned, partly because it was his company and he was so committed to its success. The seven examples I have given all came about partly because I could easily act on what I noticed and partly because I would directly benefit from learning more.

Conventional scientists do not practice spider science. They do not continuously monitor or search out large rich data sets hoping to find something they can act on. They can’t afford to, it’s unconventional, it’s too risky, it won’t pay off soon enough, they probably couldn’t act on what they found, etc. Later in Graham’s essay he marvels that big companies develop any software at all. Microsoft is like “a mountain that can walk.” Likewise, I’m impressed that scientists operating under the usual constraints manage to discover anything. You might think tenure allows them to relax, wait, take chances, and do things they weren’t trained to do, but it doesn’t work out that way.

How Accurate is Epidemiology? (part 3)

Seth RobertsLeave a comment

To my previous post about Gary Taubes’s NY Times article, Andrew Gelman adds that it is good to see public discussion of these issues. I agree. I also like seeing them raised in a dramatic context: Who’s right? Powerful people making serious mistakes. How will we know? Health at risk! That sort of thing.

Speaking of drama and epidemiology . . . For many years the introductory epidemiology class for graduate students in the UC Berkeley School of Public Health was taught by Leonard Syme. I learned about this class at party. I spoke to someone who had taken it and, as a result, had switched from public policy to epidemiology. Very impressive. I knew Syme slightly. I went to his office to learn more about how he had managed to influence someone so much. “Lots of students have said that,” he told me as I entered his office. Lots of students, after taking his class, had decided to become epidemiologists. The list included Michael Marmot, one of the most important epidemiologists in the world, who studies the social gradient in health — the tendency for the people at the top to be healthier than the people at the bottom, even after controlling for all sorts of things.

The class met once/week. Every week there was a new topic. For every topic Syme would assign a paper laying out the conventional wisdom — that high cholesterol causes heart disease, for example — plus three or four papers that cast doubt on that conclusion. I think he even had American Heart Association internal emails. Several students would present the material and then there would be debate — what’s to be believed? The debates were intense. If ever the students seemed to be reaching agreement, he would say something to derail it. “You know, there was a study that found . . . ”

Practically all classes make you think you know more at the end of them than you knew when they began. Practically all professors believe this is proper and good and cannot imagine anything else. With Syme’s class, the opposite happened: Your beliefs were undermined. You walked out knowing less than when you walked in. You had been sure that X causes Y; now you were unsure. At first, Syme said, many students found it hard to take. A three-hour debate with no resolution. They did not like the uncertainty that it produced. But eventually they got used to it.

The overall effect of Syme’s class was to make students think that epidemiology was important and difficult — even exciting. It was important because we really didn’t know the answers to big questions, like how to reduce heart disease; and it was difficult and exciting because the answers were not nearly as obvious as we had been told. This is why many students switched careers.

Marmot on Syme: “I have never come across anyone in the academic world who had quite the powerful influence on students that Syme did.” Nor have I. That meeting with Syme, about five years ago, was one of two conversations in my life that really taught me something about how to teach. I was the only person at Berkeley to ever ask him about his teaching, Syme said. What a pity.

Syme on how his research began.

Fear-Mongering?

Seth RobertsLeave a comment

This post by Dr. Erika Schwartz, complaining about a breast cancer story in the NY Times, makes important points. When politicians — such as Joe McCarthy or Jean-Marie Le Pen — try to scare us, most of us appreciate the psychology involved: The more fearful we become, the more we will look to them to protect us, thus increasing their power. Our fear = their power. Schwartz is saying that respected doctors and journalists do the same thing. How prophetic was The Coming Plague (1994) by Laurie Garrett?

Schwartz’s post has too little detail to convince me that this particular story is guilty. Nor do I agree with her that statistics are “totally meaningless when applied to the individual.” Her contribution is to ask: how can we discuss these issues without fear-mongering?

The Academic Stockholm Syndrome

Seth Roberts1 Comment

Today I went to a talk about terrorism. After the talk, I asked a question: What’s the evidence for the recommendations you made at the end? The speaker, a professor at Harvard, began her answer by apologizing: I can only tell you case-by-case anecdotes, she said. She repeated this apology a little later. Well, of course no one has done a controlled experiment (or any experiment) on how to deal with terrorism. Of course all we have is a story here, a story there. The speaker, whose talk was good, had heard the pervasive dismissiveness I criticize here so many times that not only did she expect it, she accepted it. The academic Stockholm Syndrome.

A Bayesian Tries SLD

Seth Roberts2 Comments

Bayesian data analysis, which Andrew Gelman has pioneered, is about taking one’s beliefs into account when doing data analysis. When I wrote The Shangri-La Diet, I was being a kind of Bayesian: I realized that the facts I had gathered so far did not establish the diet as any sort of panacea. Based on the facts in the book, it was hard to say how widely helpful the diet would turn out to be. I wrote the book anyway because the facts I had gathered so far were so surprising, so inconsistent with what almost everyone said about how to lose weight. From a Bayesian point of view — taking prior beliefs into account — they were impressive. If conventional views were right, no one should lose weight following SLD. But several people had. Some of them, such as Tim Beneke, had lost a lot of weight. To complain that there was no clinical trial, no certainty, was to miss the point that the book includes data that should have been impossible.

Whoever blogs at 4d2.org says something similar:

My first reaction to [SLD] was, of course, that it was one of the stupidest things I’d ever seen. Then I started reading the forums on the creator’s (Seth Roberts) site, and then I did some Googling. And would you believe that, in the absence of anything that I would call scientific evidence, this thing seems to work for most people that try it. . . . Five days ago I honestly believed the Shangri-La Diet to be hooey — interesting hooey, maybe, but still hooey. . . I decided I’d try it for myself and report on the results. I want to make it really clear that I approached this diet with a very healthy dose of skepticism. You should also understand that I’m a staunch advocate of the “eat right and exercise, stupid” philosophy of weight loss. I have never followed a prepackaged diet strategy. Having said all that: it works. I do not know why or how it works, but it works.

How Accurate is Epidemiology? (part 2)

Seth Roberts6 Comments

Because Gary Taubes is probably the country’s best health journalist, his article in today’s NY Times Magazine (”Do We Really Know What Makes Us Healthy?”) about the perils of epidemiology especially interested me. It’s the best article on the subject I’ve read. He does a good job explaining what’s called the healthy-user bias — people who take Medicine X tend to make other healthy choices as well. Does wine reduce heart attacks? Well, probably — but people who drink more wine also eat more fruits and vegetables.

The article falls short in two big ways. Taubes does a terrible job presenting the case for epidemiology. He mentions the discovery that smoking causes lung cancer but then disparages it by quoting someone calling it “turkey shoot” epidemiology. Actually, that discovery did more for public health than any clinical trial or laboratory experiment I can think of. Taubes fails to mention the discovery that too-little folate in a pregnant woman’s diet causes neural-tube and other birth defects. As the dean of a school of public health put it in a talk, that one discovery justified all the money ever spent on schools of public health (where epidemiology is taught). Taubes also fails to mention that some sorts of epidemiology are much less error-prone than the studies he talks about. For example, a county-by-county study of cancer rates in the United States showed a big change across a geological fault line. People on one side of the line were eating more selenium than people on the other side. Experiments have left no doubt that too-little selenium in your diet causes cancer.

Even worse, Taubes shows no understanding of the big picture. Above all, epidemiology is a way to generate new ideas. Clinical trials are a way to test new ideas. To complain that epidemiology has led to many ideas that turned out to be wrong — or to write a long article about it — is like complaining that you can’t take a bike on the highway. That’s not what bikes are for. If only 10% of the ideas generated by epidemiology turn out to be correct, well, 10% is more than zero. Taubes should have asked everyone he interviewed “Is there a better way to generate new ideas?” Judging from his article, he asked no one.

Now excuse me to take a selenium pill . . .

Regent Blum, Meet Provost Dumas

Seth RobertsLeave a comment

Richard Blum, chairman of the UC Board of Regents, rescinded a speaking invitation to Larry Summers after some UC Davis faculty complained:

After a group of UC Davis women faculty began circulating a petition, UC regents rescinded an invitation to Larry Summers, the controversial former president of Harvard University, to speak at a board dinner Wednesday night in Sacramento. The dinner comes during the regents’ meeting at UCD next week. Summers gained notoriety for saying that innate differences between men and women could be a reason for under-representation of women in science, math and engineering. . . Professor Maureen Stanton, one of the petition organizers, was delighted by news of the change this morning, saying it’s “a move in the right direction.”

Northwestern University Provost Lawrence Dumas is responsible for allowing Lynn Conway and Deirdre McCloskey to use his university’s considerable power to try to silence Michael Bailey. At Marginal Revolution, Alex Tabarrok calls the UC Regents’ action “shameful.” I agree.

Thanks to Matthew Pearson.

Abstracts from The New Yorker

Seth Roberts3 Comments

The New Yorker now has online abstracts, just like scientific journals. From the abstract of an article by Patricia Marx:

The writer spies from her living-room window a multitude of colorful puffy parkas from Pucci (24 East 64th Street). The writer then calls Dr. Andrej Romanovsky to ask how the body detects cold. New York is the city of coats. Real coats, not car coats, for in this town, we walk. . . . Still worried about the coming cold? There is always one thing left to do: Miami (U.S. Airways; flights as low as $59 one-way).

Surely this is better than the article itself. Just as brandy is better than the wine it is distilled from.

Why Are Medical Costs So High?

Seth Roberts4 Comments

At Cato Unbound, David Cutler, a Harvard public policy professor whose research I used in The Shangri-La Diet, writes:

The most important reason why medical costs increase over time is because we develop new ways of treating patients and provide that care to ever more people.

At least in his essay, Cutler fails to consider an alternative explanation: Medical costs have increased a lot because we have become a lot more sick — more in need of help. Over the last 50 years, obesity has greatly increased. Diabetes has greatly increased. Depression has greatly increased. Depression, including subclinical depression, is now common and has so many bad effects or correlates — less activity, less socializing, less sunlight, poor sleep, less compliance with everything — that its impact on health must be great.