The Freakonomics blog posted a letter from reader Jeffrey Mindich about an interesting traffic experiment in Taiwan. Timers were installed alongside red and green traffic lights:

At 187 intersections which had the timers installed, those that counted down the remaining time on green lights saw a doubling in the number of reported accidents . . . while those that counted down until a red light turned green saw a halving in . . . the number of reported accidents.

Great research! Unexpected results. Simple, easy-to-understand design. Large effects — to change something we care about (such as traffic accidents) by a factor of two in a new way is a great accomplishment. This reveals something important — I don’t know what — about what causes accidents. I expect it can be used to reduce accidents in other situations.

It’s another example (in addition to obstetrics) of what I was talking about in my twisted skepticism post — the downside of “correctness”. There’s no control group, no randomization (apparently), yet the results are very convincing (that adding the timers caused the changes in accidents). The evidence-based medicine movement says treatment decisions should be guided by results from controlled randomized trials, nothing less. This evidence would fail their test. Following their rules, you would say: “This is low-quality evidence. Controlled experiment needed.” The Taiwan evidence is obviously very useful — it could lead a vast worldwide decrease in traffic accidents — so there must be something wrong with their rules, which would delay or prevent taking this evidence as seriously as it deserves.

Writing about advances in obstetrics, Atul Gawande, like me, suggests there is a serious downside to being methodologically “correct”:

Ask most research physicians how a profession can advance, and they will talk about the model of “evidence-based medicine”—the idea that nothing ought to be introduced into practice unless it has been properly tested and proved effective by research centers, preferably through a double-blind, randomized controlled trial. But, in a 1978 ranking of medical specialties according to their use of hard evidence from randomized clinical trials, obstetrics came in last. Obstetricians did few randomized trials, and when they did they ignored the results. . . . Doctors in other fields have always looked down their masked noses on their obstetrical colleagues. Obstetricians used to have trouble attracting the top medical students to their specialty, and there seemed little science or sophistication to what they did. Yet almost nothing else in medicine has saved lives on the scale that obstetrics has. In obstetrics . . . if a strategy seemed worth trying doctors did not wait for research trials to tell them if it was all right. They just went ahead and tried it, then looked to see if results improved. Obstetrics went about improving the same way Toyota and General Electric did: on the fly, but always paying attention to the results and trying to better them. And it worked.

Is there a biological metaphor for this? A perfectly good method (say, randomized trials) is introduced into the population of medical research methods. Unfortunately for those in poor health, the new method becomes the tool of a dogmatic tendency, which uses it to reduce medical progress.