Category: scientific method

Climategate: Its Educational Value (continued)

Seth Roberts7 Comments

In a response to the comments on my previous post, I say that the primary attitude of science isn’t to be skeptical, it is to think for yourself. (Which, in practice, means ignoring what fancy hot-shot scientists at prestigious universities tell you to think.) Funny that fancy hot-shot scientists at prestigious universities never teach that.

Or almost never. In another comment on that post, Andrew Gelman mentions the Feynman Lectures as books from which you can learn about science. Having read Volume 1, I have no idea what he means. I was a freshman at Caltech. Feynman was a professor there at the time. The Feynman Lectures had been published but they were judged too difficult for most of the freshmen! I am not kidding. The faculty had learned that they were too hard to understand. They didn’t teach what the faculty called “problem-solving” — that is, deriving predictions from theories. So there were two tracks of Intro Physics at Caltech: the Feynman track (fewer students) and the non-Feynman track (more students). I was in the Feynman track. He wasn’t the professor, but we used his book. That’s how I came to read Volume 1. I liked it, but it didn’t teach me anything important about science.

Yet — during the exact same time, freshman year — Feynman himself did teach me something important about how to be a scientist. He taught me (= encouraged me) to think for myself. Not in any obvious way. On Wednesdays at 11:00 am, Feynman would answer questions for an hour. Anything except textbook problem-solving questions. There were more than a thousand students at Tech but maybe 20-30 attended these little sessions. One day I asked: “I’ve read some philosophy. It doesn’t make sense. Yet lots of people say it’s important. Am I missing something, or does it have as little value as I think it has?” Feynman’s answer: He agreed with me. There was one book of philosophy he liked, a survey by Bertrand Russell, but for the rest of it, it was people talking and talking and saying nothing.

Wow, he agrees with me, I thought. I had reached what I thought was a very minority opinion — an opinion I’d read nowhere else, had heard nowhere else — and this famous person who I respected agreed with me! It certainly taught me to think for myself.

Climategate: Its Educational Value

Seth Roberts25 Comments

Before the printing press, there were very few books. It was extremely hard to learn math; you had to pay a tutor. Of course literacy was very low — but all knowledge that could be transmitted through books (such as math) was very low.

Science cannot be taught through books. You can learn a lot about calculus by reading books. You can learn almost nothing important about science. Science is not a collection of facts, it is a method, a way of gathering knowledge. Almost always it is taught by doing — by working in a lab, for example. Just as, before printed books, almost no one could do any math, it is true today that almost no one can do any science. (Most doctors think the bigger the sample size, the better.)

If you look at a biology textbook, it is full of conclusions. It says practically nothing about the process by which those conclusions were reached. For some reason biologists have decided not to teach that — perhaps because it is difficult and messy to teach. And someone might be offended. Whatever the reason, the process goes undescribed. And it’s all sciences, not just biology. (Until recently, economists avoided teaching data. At least in introductory economics, data was too messy for them.)

As long as you have to learn science by doing it practically no one will understand it — just as almost no one did math when you had to hire a tutor to learn it. But now we have the Internet. And blogs. Two new things have entered the picture: a great deal of emotion (blogs are full of emotion, unlike textbooks); and unlimited space. Now science can begin to be taught without actually doing an apprenticeship. If you add enough emotion, anything becomes riveting. And there is now plenty of room for all the false starts and messy details. I suppose most scientists who blog are too worried about being dignified to say anything emotional or messy, but that doesn’t matter because there are so many bloggers.

According to Stephen Dubner, “if you are fan of science, this [Climategate] is a pretty grim day.” I think it’s a great day. As great as the day the first math text was printed. It’s the first time a large number of people are getting a real lesson in science. Mainstream media coverage is pathetic but there are so many bloggers it doesn’t matter. You can read about it endlessly. As you do, you will painlessly and unforgettably learn what Leonard Syme taught his students for years, and what I blogged about a few weeks ago: The apparent consensus on any difficult issue is more fragile than it looks. You are learning how conclusions are actually arrived at. It isn’t pretty — which textbook writers and professors, seeking dignity above all else, fail to mention.

Congratulations, Andrew Rivkin

Seth Roberts21 Comments

Andrew Rivkin writes about climate change for the New York Times. One of the stolen emails says:

At 17:07 27/10/2009, Michael Mann wrote:

Hi Phil,
…
p.s. be a bit careful about what information you send to Andy and what emails you copy him in on. He’s not as predictable as we’d like

In other words: Most reporters are predictable. Meaning they repeat what they are told instead of thinking for themselves. Otherwise there would be no need to say this.

Think about it. Michael Mann, a respected climate scientist, thinks that whatever line he and Phil Jones, another respected climate scientist, are pushing is so poorly supported by the evidence that they worry about a New York Times reporter finding holes in it! Independent thinking, even by someone without technical training, worries them! Really, it’s hard to avoid concluding that these guys are clowns, propped up by all sorts of people (journalists, Al Gore, many others) who benefit from a false certainty about this stuff.

Please, someone tell me: Why should I believe climate models? Have their predictions (not their fits) been compared to what actually happened?

More About What Causes MS

Seth Roberts3 Comments

In an earlier post I linked to a poll at This Is MS that asked if there is a correlation between getting red in the face after exercise and having multiple sclerosis. Such a correlation would support Paulo Zamboni’s idea that MS is due to poor blood circulation in the brain.

A poll at This Is MS is likely to be answered by people who have MS. Nancy Lebovitz realized she could help get answers from people who don’t have MS — crucial to learning if there is a correlation — by posting the poll on her LiveJournal page.

The two polls taken together show a strong correlation. Out of 40 people with MS, 72% get red-faced. Out of 27 people without MS, 22% get red-faced. Thanks, Nancy.

A Fourth Thing Elizabeth Kolbert Didn’t Know

Seth Roberts10 Comments

Elizabeth Kolbert, the New Yorker staff writer, did not know that Phil Jones, a climate-change scientist, manuevered to keep hidden information that disagreed with his conclusions. Here is what one of the damning emails gathered from the University of East Anglia’s Climate Research Unit said:

From Phil Jones [head of the Climate Research Unit]. To: Michael Mann. Date: May 29, 2008
“Can you delete any emails you may have had with Keith re AR4? Keith will do likewise.”

To keep them from being exposed via a Freedom of Information law. Robin Hanson and Tyler Cowen think this is no big deal. I disagree. Yes, I said before this happened that the consensus was likely to appear stronger than it is and that bloggers were a powerful force toward truth — both of which this episode merely supports rather than reveals. And, yeah, it’s just email; the really damning info is the tree-ring data reanalyzed by Stephen McIntyre.

The reason I think this is important is two-fold. First, this is not a smoking gun. Global warming does not equal the honesty of Phil Jones. But it is a powerful piece of evidence that climate skeptics can use to convince anyone that the consensus isn’t as consensus-y as it appears. Second, it exposes what Kevin Trenberth (a proponent of man-made global warming) really thinks. This is something that few knew until now. Here is what he really thinks:

The fact is that we can’t account for the lack of warming at the moment and it is a travesty that we can’t. The CERES data published in the August BAMS 09 supplement on 2008 shows there should be even more warming: but the data are surely wrong. Our observing system is inadequate.

The data are surely wrong. Trenberth, being human, is going to put the best possible spin on things, the spin most consistent with what he has said many times . . . and this is what he comes up with. Support for the idea of global warming is entirely based on climate models. No one has created a mini-Earth and done experiments. If the data and models don’t agree, there is no reason to believe the models. And if you don’t believe the models you have no reason to believe in global warming. Is Trenberth an ignoramus whose honest assessment of the situation (the models and the data profoundly disagree) should be ignored? Of course not. He doesn’t draw the obvious conclusion (the models are wrong) but nothing prevents the rest of us from doing so.

Just to be clear: I completely agree with Robin’s larger point that this sort of thing supports prediction markets. And I think reduced reliance on fossil fuel would be a very good thing.

Three Things Elizabeth Kolbert Doesn’t Know.

The Parable of the Wii

Seth Roberts5 Comments

For exercise (Dance Dance Revolution) and self-tracking, I decided to buy a Wii. My first attempt, I was scammed. It arrived in August. With difficulty, I took it and accessories unopened to China. That was hard. It was even harder — for no obvious reason — to install it in China. The box sat unopened next to my TV, easily visible, for two months.

Finally I opened the box, took out the parts, put them together, added batteries, plugged it into the TV in my apartment. And nothing happened! Was my TV at fault? Or the Wii? Wii’s aren’t sold in China. I imagined bringing it back to America to get the problem fixed. After a few days, I tested my TV using video output from a neighbor’s Apple computer. My TV worked. After the test, my Wii also worked. When I replaced the Apple input with the Wii input I saw the Wii input for the first time. I don’t understand it, but that’s what happened.

In my experience, this is how science works. It is much harder than expected, then it pays off in ways that defy understanding. The concept of self-experimentation is simple: I will measure X (sleep, productivity) about myself. I will test different ways to improve X, learn what works, and thereby improve X. The reality is different. For years I measured my sleep and tried to improve it. It was hard to deal with the data. Even worse, every idea I had was wrong. That seemed like a huge obstacle — like my Wii needing repair. But I kept plugging away, because it was better than doing nothing, and . . . got somewhere. Out of nowhere and nothing. Not only did I improve my sleep, I arrived at a broader idea about health that turned out to be very helpful (that our bodies are designed for Stone-Age conditions and self-experimentation can help determine those conditions, which aren’t obvious). Just as we overvalue big steps (e.g., well-funded prestigious research), we undervalue small ones (e.g., cheap research with no prestige).

Science is basically a bunch of little steps. Many little experiments that explore cause-effect space. If you find a new example of cause and effect, the payoff is unpredictably large. Scientists don’t like thinking of themselves as wandering ants. But that’s how they are most effective. This goes against human psychology because wandering (Nassim Taleb calls it “tinkering”) is low status and lonely. The payoff is too rare and too unclear. It isn’t supported by powerful institutions, such as research universities and medical schools. Imagine an ant who says “I know where food is!” This is a way to get many ants to follow him, to feel important, to have high status, to get support from his employer. That’s why he does it. But he doesn’t know. The effect on the rest of us, the potential beneficiaries of progress, is that instead of having a thousand ants wandering everywhere, we have a thousand ants following one ant who doesn’t know what he’s doing.

Mistaken Consensus in Physics?

Seth Roberts18 Comments

Steven Sheets writes:

I can’t really think of an area in physics where a consensus has been achieved only to be shown to be completely wrong.

Good point. I know little about physics but I tend to agree. Work awarded the Nobel Prize in Physics is more trustworthy than work awarded the Nobel Prize in Medicine, for example. I think a big reason that a consensus presented to the public is wrong is because there is outside pressure to get it right — pressure to find a way to lose weight, pressure to find how to reduce heart disease, and so on. Whereas there is no public pressure to get this or that physics question right. Less is at stake and the physics community can take as long as it wants.

Still, physicists make mistakes and other physicists go along with those mistakes. I can think of three examples:

1. When calculating the charge on an electron, Millikan famously used the wrong value for the viscosity of air. This didn’t prevent those using other methods from getting the same answer.

2. It was a rather bold title: How Nature Works (1996) by Per Bak. Yet the sand avalanche models on which the whole thing was based turned out to be wrong. Actual sand didn’t behave as predicted. There wasn’t consensus in the physics community that Bak was right, but many physicists took him seriously. (As far I can tell from a distance.) I’ve worked on explaining power-law data (the subject of Bak’s book) and the ideas in that book weren’t helpful.

3. Long ago, lots of physicists – if astronomy is part of physics — believed that the sun revolved around the earth.

There’s plenty of pressure and a lot at stake to get climate predictions right. So I think climate models are in the territory where big consensus mistakes are made. As Patrik points out, the story of the Yamal tree-ring data — which I wasn’t thinking of when I wrote the Elizabeth Kolbert post, or I would have mentioned it — is a very good reason to think that what Kolbert writes about climate is less certain than Kolbert thinks.

Three Things Elizabeth Kolbert Doesn’t Know

Seth Roberts28 Comments

A staff position at The New Yorker is the best journalistic job in the world. Elizabeth Kolbert, a very good writer and reporter, has one of them. In the current issue, criticizing Superfreakonomics, she writes:

To be skeptical of climate models and credulous about things like carbon-eating trees and cloudmaking machinery and hoses that shoot sulfur into the sky is to replace a faith in science with a belief in science fiction.

I cannot discuss engineering (“carbon-eating trees”, etc.) but I can discuss science (“climate models”). Here Kolbert shows the same limitation that practically every science journalist shows (the big exceptions are Gary Taubes and John Crewdson): They take the consensus view too seriously. In case after case — so many that it’s hard not to draw sweeping conclusions — the consensus view about difficult topics is more fragile than an outsider would ever guess. It’s not necessarily wrong, just less certain.

Kolbert places too much faith in those climate models. Here are three things Kolbert doesn’t know:

1. For years, as I’ve blogged, Leonard Syme, an epidemiology prof at Berkeley, taught his students to distrust one mainstream public-health conclusion after another. Maybe 12 examples in all. He showed them facts they didn’t know. All of a sudden the picture wasn’t so clear any more. That he could do this in so many cases, one case per week, is what’s telling.

2. If you believe mainstream ideas about weight control, the Shangri-La Diet is absurd. It can’t possibly work. Since it has actually worked in countless cases — more than half the time, as far as I can judge — the experts, it appears, got it utterly wrong. Long before me, Michel Cabanac, a professor of physiology at Laval University, was saying the same thing — that the consensus view about how to lose weight was wrong. No matter how many millions of times journalists repeated it. The Shangri-La Diet merely makes it vividly clear he was right.

3. Hal Pashler and I wrote a paper about how mental models based on fitting data were delusional. The data that supposedly supported them did not. To take seriously a model because it could fit data was a mistake, we pointed out; what matters is correct predictions. It isn’t easy to figure out the predictions of a model with many adjustable parameters; and the modelers in these cases never did. These models were accepted professionally for half a century; perhaps they still are.

It is possible that climate modelers have a different psychology than scientists in other areas — that the evidence for the consensus presented to outsiders is as strong as the scientists involved say it is — but it seems highly unlikely. For example, I doubt the climate models Kolbert places such faith in have been tested (their predictions, not just their fits, compared with reality).

There’s no doubt that carbon dioxide concentration and global temperature are correlated, but you may not know that carbon dioxide concentration lagged temperature for a long time. Because of this, I’m sure the temperature change caused the carbon-dioxide change. It isn’t mysterious; as water changes temperature, the amount of carbon dioxide it can dissolve changes. As water heats, carbon dioxide is released into the air.

This means that something powerful — not carbon dioxide — has been producing changes in global temperature so large they cause carbon dioxide to rise and fall in amounts as large as those we are now worried about. Until we know what this is there is no way to allow for it. To subtract it from observed carbon dioxide and temperature changes, see what remains, and try to draw conclusions from the residuals. And we don’t know what it is, no matter how closely this or that climate model fits data. (How closely they fit data depends on how many parameters they have, not merely how truthful they are. More adjustable parameters –> closer fit.) Until we know what it is, it is entirely possible that this force, not man-made emissions, is behind recent increases in global temperature and carbon dioxide. If man-made emissions are not causing the change in temperature, reducing them is unlikely to do much. (Sure, there are a hundred blog posts dismissing the inconvenient backward lag. I’ve been unable to find even one that addresses the point I’m making here.)

This is like what Richard Herrnstein and Charles Murray failed to understand in The Bell Curve. They had a whole chapter on the Flynn Effect (the large increase in IQ over years) but they failed to grasp that until the Flynn Effect was correctly explained — until we knew what caused it — there was a big environmental contribution to IQ that they didn’t understand. Perhaps it was this powerful environmental factor that caused the between-race differences in IQ that they attributed to genes. They were unable to equate different races for this factor — to take its effect into account.

Herrnstein and Murray might have been smart enough to see the problem — but, in any case, they ignored it. Kolbert is smart enough to understand that the climate scientists she talks to have a vested interest in overstating their case — but, at least in her writing, she ignores this. If she stopped ignoring the vested-interest problem and tried to think for herself — to sort out for herself conflicting claims, to stop believing everything a mainstream thinker tells her — her job would be much harder. (It took Gary Taubes seven long years to write Good Calories Bad Calories.) Given Kolbert’s lack of scientific background (at The New Yorker she originally covered politics), perhaps her job would be impossible. Kolbert’s faith is not in science, as she pompously says, but in scientists.

How to Base Medicine on Evidence

Seth RobertsLeave a comment

The thing to notice about what the New York Times calls “ the evidence-based medicine practiced at Intermountain hospital” is how different it is than the movement called evidence-based medicine. The Intermountain stuff, above all, is not black-and-white thinking. It is a good example of what the opposite looks like. The rules aren’t simple, they are complex, and not fixed. It is what engineers in other areas have been doing since Deming.

So many scientists — not to mention everyone else — are completely paralyzed, rendered completely useless, by their black-and-white thinking. It feels good to them — they love the certainty of it, and the power it gives them to look down on others — and they never quite realize what it has done to them. The notion of using evidence to improve health care made perfect sense — until black-and-white thinkers got a hold of it.

Any class in scientific method should be at least half about avoiding black-and-white thinking. They never are.

Black-and-White Thinking: An Example

Seth Roberts6 Comments

I’ve complained many times on this blog about scientists who engage in black-and-white thinking. Here is an example of such thinking outside of science:

At the mosque, Haniyeh addressed the campers on the importance of reciting the Koran. “There are two kinds of people,” he advised them. “Those who know the Koran is right and who follow it, and those who turn their backs on the Koran.”