Author: Seth Roberts
A Different Sort of Scientific Progress: Toward Utility
From a excellent column by Tim Hartford:
Esther Duflo, a French economics professor at MIT, wondered whether there was anything that could be done about absentee teachers in rural India, which is a large problem for remote schoolhouses with a single teacher. Duflo and her colleague Rema Hanna took a sample of 120 schools in Rajasthan, chose 60 at random, and sent cameras to teachers in the chosen schools. The cameras had tamper-proof date and time stamps, and the teachers were asked to get a pupil to photograph the teacher with the class at the beginning and the end of each school day.
It was a simple idea, and it worked. Teacher absenteeism plummeted, as measured by random audits, and the class test scores improved markedly.
Another young economist, Ben Olken of Harvard, used a similar randomisation technique to work out whether corruption in Indonesian road-building projects was best fought top-down, using audits, or bottom-up, soliciting comments from local villagers about whether money was being embezzled. One challenge was to work out how much embezzlement was taking place. Olken enlisted engineers to take samples of the road’s structure and to estimate how much it should have cost to build; he compared that estimate with how much spending was claimed in the project’s accounts. The missing funds were a rough guide to the amount embezzled.
In contrast to Duflo’s results, Olken found that the bottom-up monitoring was not effective – it shifted the embezzlement from something the villagers cared about (wages) to something they did not (building materials). The threat of a guaranteed audit – a threat that was later carried out – was much more effective, reducing the estimates of missing funds by a third.
A chapter in The Theory of the Leisure Class by Thorstein Veblen is about academia and the tendency toward uselessness, which Veblen explained as a way of signalling that one doesn’t need to work. As a general rule at research universities, useful = low status. A few years ago, I had lunch with an engineering professor. By far the most useful thing to come out of the UC Berkeley Electrical Engineering Department in the last 20 years, he told me, was a circuit analysis program (SPICE). Used everywhere. A big contribution to the field. Who did it? I asked. He didn’t know. That’s how low-status it was — no professor wanted to be closely associated with it.
The curious thing about the two examples that Hartford describes is that they are happening at the same time. Is this a coincidence? Or is there an explanation?
Peter Pronovost’s research on ICU checklists is far more useful than one would expect from a medical school professor; likewise my self-experimentation about everyday problems (e.g., poor sleep) was far more useful than one would expect from a psychology professor. So perhaps there is some sort of larger discipline-spanning force at work.
Jonathan Schwarz, Philosopher of Science
In his New Year’s Resolutions, Jonathan Schwarz vowed to “accentuate the positive”:
At all times in history, there have been zillions of people doing wonderful things with little recognition. 99% of the attention goes to various monsters. Even when the attention is extremely negative (i.e., people like us yowling about Dick Cheney or Thomas Friedman) it suggests the monsters are the only ones doing anything important, and the rest of us have nothing better to do than talk about them.
This is empirically wrong. And it saps our capability for independent thought, because it orients us toward reacting to the powerful, rather than acting ourselves.
This is especially pernicious in a period when technology is opening up ways to build new and better institutions. While I understand the visceral appeal of dumping a bucket of pig excrement on Fred Hiatt, this takes time away from what will have a longer-term impact: nurturing our own fledgling efforts.
This is similar to what I wrote — in the context of a NY Times review of a book about “pseudoscience” — about skeptics being a dime a dozen and what’s really lacking is sophisticated appreciation.
Interview with Gary Taubes (part 2)
INTERVIEWER What do you think about prions?
TAUBES Here’s the problem with prions: the claim is that here’s a radical discovery — an infectious agent that doesn’t have nucleic acid — and it’s based fundamentally on a negative result, which is that when researchers have gone looking for the nucleic acids they failed to find them. Therefore, so the logic goes, they must not be there. The original claim, by Stan Pruisner, another Nobel Prize winner, was premature. He made some claims in his early papers that were definitively wrong. Yet everything he’s done since then supports his initial claim, which suggests he’s was either remarkably lucky to begin with, or he’s only capable of interpreting his results so that they agree with his preconceptions. One of the themes in all of my work is that if you go public on premature data, what happens is that the motivation to do really good science ceases. By “really good science”, what you’re supposed to do, as brutally as you can, is to try to come up with tests that would refute your own hypothesis. The idea is that if your hypothesis survives every rigorous test you can imagine, and all those that everyone else can imagine, then you can start believing itss true. But once you’ve staked a claim based on premature data — once you’ve gone out on a limb without doing any of those rigorous tests — now your motivation becomes to prove that you were right., which you can never do in any case. But the point is that you stop trying to refute your hypothesis, and you start trying to accumulate evidence that supports it and the latter isn’t science. It’s more like what happens in religions.
INTERVIEW That’s what happened with Peter Duesberg. He was a good scientist until he started making claims about HIV.
TAUBES When I wrote this prion article in 1987, the science was so bad that it was a joke. Still, I never said that the prion concept wasn’t correct; I just said there was excruciatingly little evidence to support it, and there were plenty of reasons to believe it was wrong. How do you get strains of an infectious agent without nucleic acids (RNA or DNA) to encode the information in the strains? If you actually look today, even though Prusiner has won the Nobel Prize, if you go to the WHO website or the NIH website and you read up on prions, you’ll see that it’s still considered a hypothesis. There’s still no way to explain how you can get strains without a virus. Prusiner has these ideas, but they’re along the lines of now “a miracle happens”. It’s another long story, but one of the problems (and this is a theme in my book), when you let an untested hypothesis grow and infect the science to the point where people start to believe it’s true, even though it’s never been rigorously tested, the obstacles against ever overturning it get bigger and bigger. It’s like the dietary fat hypothesis: you let it sit around for 40 years, and it evolves to the point that people consider it dogma; it’s virtually impossible to overturn it. The situation with prions isn’t so bad because the public doesn’t care about prions the way that they care about diet, but once the Nobel Prize is awarded, even though it’s still considered a hypothesis, people tend to ignore the studies that suggest it’s wrong. There’s one researcher from Yale who is constantly publishing evidence in major journals that she’s found the nucleic acids, and people just ignore her. They believe the question has already been answered.
INTERVIEWER What’s her name?
TAUBES Laura Manuelidis.
Interview directory.
Interview with Gary Taubes (part 3)
INTERVIEWER You wrote that New York Times piece, and from my take on it, you had a bunch of evidence, and then you got a book contract. Is it fair to say that you found out that what you wrote in the piece was mostly right?
TAUBES It’s a difficult question. I had actually pitched the New York Times piece on fat as an attempt to determine the cause of the obesity epidemic. The proposal was very open ended. I had several ideas. I actually believed, going in to the story, that the answer was going to be that high-fructose corn syrup was responsible for Americans getting fatter over the last 30 years.
INTERVIEWER I’m glad to hear that.
TAUBES The thing about the obesity epidemic is that we can say when it starts, give or take five years: sometime between the mid-1970s and late 1980s. So we have a starting point, and that happens to coincide with the introduction of a type of high-fructose corn syrup known as HFCS-55, which was developed to taste exactly like sugar when it’s put in sodas and juices. In fact, it is effectively identical to sugar, as far as the body is concerned — sugar (sucrose) is 50-50 glucose and fructose and HFCS-55 is 45 percent glucose and 55 percent fructose — although I didn’t know that when I pitched the article. But I thought that high-fructose corn syrup is so cheap. Basically this is an idea that Greg Critser in a book called Fatland picked up on, and subsequently Michael Pollan, too, that high fructose corn syrup allows you to saturate the market with sugar, without any fear that price fluctuations will cause you to go out of business, or lose you a lot of money. If the international price of sugar suddenly spikes, as it did in the 70s, and you’re committed to fulfilling this enormous demand for sugar you’ve created, then you’re in trouble. But if you have a cheap reliable source of sugar, at a price that won’t change from year to year, then you can create an enormous market without fear. This was, more or less, my naive idea of how the economics of HFCS might have caused an entire nation to get fat. Once they had this dependable low-cost sugar substitute, the sugar industry and the soda industry could then expand their production and sell Big Gulps, etc. Then I did the reporting. I talked with industry analysts, and they said that was nonsense; that the primary cost of selling sodas and fruit juices is the bottling and the shipping, and that the cost of the sweeteners is such a tiny portion of the cost of the end product that it wouldn’t have made any difference whether it was sugar or high-fructose corn syrup. So I moved to my next idea, which was based on the fact that the beginning of the obesity epidemic coincided with the institutionalization of the low-fat dogma. As I’m doing that reporting, I stumbled upon what was, at that time, five trials of the Atkins diet, all of which had been finished, but not yet published. At one point, when I was doing the reporting, I actually got worried that some other journalist would beat me to the punch.
INTERVIEWER What was it about the Atkins diet that made these trials so important to your article?
TAUBES: Well, remember, my background, as a journalist and in school, was more or less in physics. In the kind of physics I used to write about, you’ve got some complicated detector that’s looking at particles and atoms smashing together inside it and you’re looking for some byproduct of a collision that you’ve never seen before. A new particle. But the first thing you have to do is make sure you understand your detector. Can you believe what it’s telling you. So you to have to calibrate it. If you want to know how much you weigh, for instance, one thing you might do before you step on the scale is you calibrate that. You make sure that when you’re not on it, the little arrow on the scale is pointing to zero. If it’s registering one or two pounds when you’re not standing on it, then it might be off by five or ten pounds or more when you are. So you want to calibrate your equipment. You want to know that when you set it to zero, it says zero. That’s an idea that’s always resonated with me. Measure what happens at an extreme, make sure you understand that, and then see what happens from there. So here’s the Atkins diet: in theory, you’re removing virtually all of the carbohydrates, but you don’t tell people to eat less. You tell them to eat as much as they want. It’s like you’re setting the diet to zero carbohydrates, and as much fat as possible. According to conventional wisdom, you should certainly not lose weight and you might even gain it. But here were five studies saying that, lo and behold, people really do lose weight when you remove the carbohydrates from the diet, and they lose more weight than they do when you tell them to keep the carbohydrates but eat less calories. What’s more, their cholesterol profiles actually improve, so how can fat or saturated fat be bad for your cholesterol, if these high fat, high saturated fat diets make your cholesterol levels better. To me that had to tell you something about the validity of the low-fat dogma and about the underlying physiology. What do carbohydates do, and what does their removal do. So once I learned about those five studies, I was confident that I had a story that was now worth writing. As for your original question, about whether I found out most of what I originally wrote was right, obviously the book supports the message of the article, but I no longer believe a fair number of things I believed when I wrote that story. For instance, when I wrote the Times article I inherently believed that the key was still calories consumed.
INTERVIEWER You mean things that you believed then, that you don’t believe now?
TAUBES Yes, that I don’t believe now. In that original article, I discussed what David Ludwig has argued — that easily digestible carbohydrates cause these blood sugar and insulin spikes, and that in turn causes blood sugar to plummet, and the result is blood sugar so low a few hours later that this in turn makes you hungry. So you eat more and that’s why you get fat with carb-rich diets. Ludwig works with obese children at Harvard and I believed that his hypothesis was probably true. Then I also talked about Michael Schwartz’s research at the University of Washington. Schwartz believes that insulin’s primary role is to suppress hunger in the brain, but that somehow we become resistant to that effect and so, once again, we eat too much and that’s why we get fat. Both these theories are predicated on the notion that we get fat because we eat too much and that’s what I believed. We consume more calories than we expend and we get fat; something about carbohydrates facilitates that excess consumption. Now I believe the causality is reversed, and that’s what I discuss in the book and in the lecture. Carbohydrates make us accumulate calories in our fat tissue, and that in turn makes us eat too much. It’s all about the regulation of fat metabolism. All those things that Ludwig and Schwartz were talking about might have been true (I mean, they are true, on some level), but they’re not the driving force of why we get fat, or why removing the carbohydrates makes us lean.
INTERVIEWER I see. So that’s a good summing up of what was in your article that you believe, and what you don’t believe anymore.
TAUBES There are other related facts, as well. I never imagined when I wrote that original article that I would come to believe that exercise won’t make you lose weight, even though I’ve been an athlete my entire life and it’s never helped me. So it’s fair to say that when I wrote that New York Times article five years ago, I had an entirely different conception about the causes and cures of obesity and overweight. Carbohydrates were key, but my understanding of the mechanisms was completely different. That’s the kicker with research and reporting: you don’t know what you’ll find until you do it.
Worried About Prostate Cancer?
Many studies have linked dairy consumption with an increased risk of prostate cancer. Two studies in a recent issue of the American Journal of Epidemiology looked at this connection in detail and reached the same surprising conclusion: it’s low-fat dairy that is problematic. Here are their abstracts:
High intakes of calcium and dairy products have been suggested to be related to prostate cancer risk. Such associations were examined in the Multiethnic Cohort Study (1993–2002) among 82,483 men who completed a detailed quantitative food frequency questionnaire. During a mean follow-up of 8 years, 4,404 total cases of prostate cancer were identified. In Cox proportional hazards models, no association was found between calcium and vitamin D intake and total, advanced, or high-grade prostate cancer risk, whether for total intake, intake from foods, or intake from supplements, among all male participants or among nonusers of supplemental calcium. No association of calcium or vitamin D intake was seen across racial/ethnic groups. In analyses of food groups, dairy product and total milk consumption were not associated with prostate cancer risk. However, low-/nonfat milk was related to an increased risk and whole milk to a decreased risk of total prostate cancer; after stratification, these effects were limited to localized or low-grade tumors. Although the findings from this study do not support an association between the intakes of calcium and vitamin D and prostate cancer risk, they do suggest that an association with milk consumption may vary by fat content, particularly for early forms of this cancer.
Calcium and dairy foods in relation to prostate cancer were examined in the National Institutes of Health (NIH)-AARP (formerly known as the American Association of Retired Persons) Diet and Health Study (1995/1996–2001). Diet was assessed with a food frequency questionnaire at baseline. Multivariate relative risks and 95% confidence intervals were estimated by Cox regression. During up to 6 years of follow-up (n = 293,888), the authors identified 10,180 total prostate cancer cases (8,754 nonadvanced, 1,426 advanced, and 178 fatal cases). Total and supplemental calcium were unrelated to total and nonadvanced prostate cancer. However, a statistically nonsignificant positive association with total calcium was observed for advanced (≥2,000 vs. 500–<750 mg/day: relative risk (RR) = 1.25, 95% confidence interval (CI): 0.91, 1.71; ptrend = 0.06) and fatal (≥1,000 vs. 500–<750 mg/day: RR = 1.39, 95% CI: 0.92, 2.09; ptrend = 0.10) prostate cancer. Skim milk, but not other dairy foods, was associated with increased risk of advanced prostate cancer (≥2 vs. zero servings/day: RR = 1.23, 95% CI: 0.99, 1.54; ptrend = 0.01). In contrast, calcium from nondairy foods was associated with lower risk of nonadvanced prostate cancer (≥600 vs. <250 mg/day: RR = 0.82, 95% CI: 0.68, 0.99; ptrend = 0.04). Although the authors cannot definitively rule out a weak association for aggressive prostate cancer, their findings do not provide strong support for the hypothesis that calcium and dairy foods increase prostate cancer risk.
Emphasis added. The differences between these two abstracts should interest people trying to learn how to write abstracts.
Interview with Gary Taubes (part 1)
I interviewed Gary Taubes shortly after he gave a talk about the main ideas of his new book — Good Calories, Bad Calories – at UC Berkeley. The interview lasted 2 hours.
INTERVIEWER I just spoke to someone who reduced the carbohydrate in his diet, for various reasons, including your book. He found that his performance on mental problems started improving again. It had stopped improving; it had been constant for a long time, and then it started getting better. So it may be that when you reduce the carbohydrate in your diet, your brain starts working better.
TAUBES Well, there is evidence that your brain works more efficiently on ketones, as does your heart. So if he reduced his carbohydrate consumption sufficiently, he probably increased the level of ketones in his blood. But I’m just speculating here.
INTERVIEWER: The book seems to have had an unusual beginning. You’d been writing about salt, and you learned that a scientist you didn’t trust about salt was also talking about obesity?
TAUBES Well, I’ve spent over 20 years now writing about controversial science. In the mid-1980s, I lived at CERN for ten months, the big physics lab outside Geneva, watching physicists discover non-existent elementary particles. Then I wrote a somewhat infamous story about prions, the supposed causative agents of Mad Cow Disease. I wrote a book about cold fusion: I got obsessed with this question of how it happened, because it was so obviously wrong. After all that, I developed what I believe is a very good feel for who’s a good scientist, and who’s a bad scientist, just by talking to them. There are certain ways that good scientists describe their data, describe the caveats, and describe the conditions by which they may or may not be right. I had also, obviously, with cold fusion, interviewed some of the worst scientists in the world. I used to joke with my friends in the physics community that if you want to cleanse your discipline of the worst scientists in it, every three or four years, you should have someone publish a bogus paper claiming to make some remarkable new discovery — infinite free energy or ESP, or something suitably cosmic like that. Then you have it published in a legitimate journal ; it shows up on the front page of the New York Times, and within two months, every bad scientist in the field will be working on it. Then you just take the ones who publish papers claiming to replicate the effect, and you throw them out of the field. A way of cleaning out the bottom of the barrel.
INTERVIEWER I thought your NY Times article, “What if It’s All Been a Big Fat Lie,” sort of did that. The people who came out against it, they were the bad journalists. Just throw them out!
TAUBES Well, how I got onto that: I was doing this story for Science on salt and blood pressure, looking into the controversy about whether salt consumption plays any role at all in raising blood pressure and causing hypertension. One of the prime players in this salt/blood pressure controversy was obviously one of the worst scientists I’d ever met — one of the five worst…you can’t say, in that five, who is the very worst, but they’re all pretty bad. This is a group that includes guys like Stan Pons and Martin Fleischman who claimed to have discovered cold fusion. While I’m on the phone with this guy, interviewing him, he takes credit for getting Americans to eat less fat and fewer eggs. I literally finished the interview, called my editor at Science, and I said “You know, one of the worst scientists I’ve ever interviewed just took credit for getting Americans to eat less fat and fewer eggs, and I don’t know what the story is, but when I’m done with this salt story, I’m going to look into fat, cholesterol, and saturated fat.” I had a great relationship with Science. My editors had faith in me. If I said there was a story there, they’d give me the support I needed to pursue it. A year later, I ended up with that first story in Science, saying that there’s no evidence that reducing the total fat in the diet makes a damned bit of difference in our health. The evidence that saturated fat and monounsaturated fats are players is, at best, marginal. And that led to the N.Y. Times article.
INTERVIEWER What did that scientist say that made you rank him so low?
TAUBES There are all kinds of signs. He told me there was no controversy, when there was obviously a controversy. His side might have been right, but to deny there as a controversy was ludicrous. He talked about the legitimacy of throwing out negative data. You measure salt consumption one way; you don’t see any effect on blood pressure, and so you decide that’s obviously the wrong way to measure it. The implication of everything he told me was that he knew what the answer was before he did his experiments, and then he adjusted his experimental techniques and methodology until he got the answer that he wanted. And he believed this was legitimate science. There are other signs. I’m a stickler about the use of words like “evidence” and “proof”. So if someone tells you there’s no evidence for some controversial belief, you can be fairly confident that they’re a bad scientist. There’s always evidence, or there wouldn’t be a controversy. If somebody says that “we proved that this was true” or “we set out to prove that this was true” that’s another bad sign. The point here, as Popper noted, among others, is that you can never prove anything is true; you can only refute it. So researchers who talk about proving a hypothesis is true rather than testing it make me worried.
INTERVIEWER Yeah, I see what you’re saying. They overstate; they twist things around to make it come out the way they want. They are way too sure of what they…
TAUBES Yes, and the really good scientists are the ones, almost by definition, who are most skeptical of evidence that seems to support their beliefs. They’re most aware of how they could have been fooled, how they could have screwed up, or how they might have missed artifacts in their experiment that could have explained what they observed. They’re very careful about what they say. If you ask them to do play devil’s advocate, and tell you how they could have screwed up, then at the very least, they’ll say “Well, if I knew how I could have done it, I would have checked it before I made the claim”. So when I’m talking about discerning the difference between a good scientist and a bad scientist, I’m talking about how they speak about their research, the evidence itself, it’s presence or absence. My friends in journalism would often ask me this question: by what right do I think make decisions about who’s a good scientist and who’s not. I’d say “Well, when you’re an English major, you can be confident that Norman Mailer was a better writer than John Grisham, even though John Grisham makes 10 to 100 times more money”. It’s just a feel for what you do; I don’t know how else to describe it. I know a good scientist when I talk with one. I might be fooled, on occasion, but….
INTERVIEWER It’s not particularly well-correlated with how famous they are, or how many Nobel Prizes they’ve won.
TAUBES My first book was about a Nobel Prize winner who discovered non-existent elementary particles.
INTERVIEWER Who was that?
TAUBES An Italian physicist named Carlo Rubbia.
Advances in SLD: Easing the Burden of Sugar Water
One way to do the Shangri-la Diet is to drink sugar water. This can cause your blood sugar to go too high if you don’t drink it slowly. Dr. Edward Pooley, a UK doctor, has noted that in the UK you can buy FibreSure, which is powdered inulin, a flavorless soluble fiber. Adding it to the sugar drink should slow digestion of the sugar. It is also sold in the United States.
Advances in SLD: Eating Lots of Nose-Clipped Food
At the Shangri-La Diet forums, Heidi 555 wrote (excerpts):
I highly recommend nose-clipping a high percentage of food. My weight has been steadily dropping down and my body is shrinking. But best of all I feel really good in my body. For the past five years I’ve been dedicated to getting in shape. I gradually built some muscles but found it impossible to lose those last 10 pounds of fat. Working out made me look stockier. Now I am in the enviable position of trying to figure out what I want to weigh. Imagine that – pick what I want to weigh! I can’t believe that I’m trying to figure that out.
For the most part, I don’t mind nose clipping lots of food. I enjoy eating even with nose clips on. Yesterday, I made some delicious cream of mushroom vegetable soup. I raved about how delicious it tasted. My husband looked at me as if I was slightly nuts. I was wearing nose clips and couldn’t taste a damn thing. Yet, I enjoyed every mouthful and raved about how great it “tasted”. The healthy, creamy, warm, texture, umami elements were deliciously satisfying.
I have found the same thing — that there is a lot of pleasure to be gotten from the non-smell elements of food (creamy, sweet, salty, sour, etc.) and that the overall effect, when these elements are present in good amounts, is that the food tastes delicious, even without smell. When I have nose-clipped chicken, for example, I sprinkle it with salt, sugar, vinegar, and hot sauce.
I don’t worry about a two hour window. I’m also flexible each day with what percentage of food I nose clip. I think in general I nose clip somewhere between 40 and 90 percent of what I eat. But it’s not as bad as it sounds. With strong appetite suppression, I often don’t care if I taste what I eat. I try to nose clip extremely healthy food. But sometimes I have a fridge to mouth nose clipped binge. The weirdest thing is that I always feel like I’m eating a lot. Maybe eating as much as you want, of whatever you want, always feels like a lot.
It’s especially interesting she doesn’t worry about a two-hour window.
With flavorless oil and unflavored sugar water, there is a dosage limit, of course: you’ll probably want to stay under 400 calories/day so that the rest of your diet provides good nutrition. With nose-clipping there is no obvious calorie limit. Everything we know about nutrition suggests you could eat all your food this way. Given the right choice of foods — foods that are adequately creamy, salty, sweet, etc. — you’ll still enjoy everything.
The Best Way to Learn is to Do (Jonathan Schwarz edition)
“The best way to learn is to do,” wrote the late Paul Halmos at the beginning of an article about how to teach college math that inspired me to start self-experimenting. Jonathan Schwarz says something similar:
America is so completely depoliticized that I support people doing pretty much anything (except forming neighborhood fascist gangs, and even that doesn’t worry me too much). Perhaps I’m foolishly optimistic, but I believe people will learn from the horrendous mistakes they’ll surely make. And even if they don’t, giving it a shot is the only way they have even a possibility of doing so.
Well put.
A subtle defense of the Iraq War? If Halmos were alive I like to think he’d agree with this:
Lesson 1: The best way to learn is to do.
Lesson 2: And the best thing to do is something small.