Month: October 2010

How to Lie with Meta-Analysis

Seth Roberts6 Comments

Michael Constans drew my attention to a Consumer Reports article about spinal surgery. According to the article, a popular type of spinal surgery called vertroplasty (involving cementing vertebrae together) doesn’t work and should be stopped:

Despite the popularity of the procedure, the American Academy of Orthopedic Surgeons has just released a guideline [actually a meta-analysis of available data] saying it doesn’t work, and shouldn’t be used.

They reviewed all the literature about the procedure, and found two good-quality studies (randomized controlled trials) that show vertebroplasty works no better than a fake (placebo) procedure. There were no clinically significant differences in pain or disability, they say.

No significant difference means it “doesn’t work”? At best, it’s absence of evidence. And it’s not even that because we don’t know what the rest of the research suggests. The meta-analysis might have ignored thirty studies; it doesn’t say how many studies were ignored. Nor does it say what “placebo” means. Patients are interested in pain relief. Whether the pain relief is “all in their head” or whatever hardly matters. From a patient’s view, and a clinician’s view, a better comparison is a group that gets another plausible treatment or no treatment. The meta-analysis reports one study that compared vertrebroplasty to “conservative” treatment, which makes more sense.

So how come vertebroplasty has been used so often? Other experts have recommended the treatment in the past, including the United Kingdom’s National Institute for Health and Clinical Excellence, which said most people got some pain relief from the procedure.

This suggests that ignoring of evidence and poor choice of comparison group made a difference.

In this case, it probably comes down to what you mean by significant pain relief. The US surgeons set a strict definition—they said a difference in pain relief of less than 2 points on a 10-point scale was not meaningful for patients. Smaller differences in pain relief were recorded in the studies, but, say the surgeons, they were not big enough to make a real difference.

I think patients would prefer to decide for themselves what degree of pain relief is meaningful.

The surgeons say the evidence against vertebroplasty is strong, and they don’t expect future studies to overturn their recommendations.

Haha! Two studies with inferior comparison groups that failed to find a difference — and ignoring an unknown number of studies and arbitrarily raising the bar — adds up “strong evidence against”!

Still the Guideline and Evidence Report is useful. It reviews a range of treatments, provides citations (so you can look much further), and isn’t wedded to the placebo-comparison group. It includes studies with other comparison groups; the article just doesn’t mention them.

And I completely agree with the article’s conclusion:

If you’re considering any type of surgery, ask your surgeon to show you data about how likely it is to solve your problem.

And don’t take your surgeon’s word for it that such data exists.

Strong Light and Cancer

Seth Roberts5 Comments

From an excellent article about light pollution (not online) by David Owen in the 20 Aug 2007 issue of The New Yorker:

Richard Stevens, a cancer epidemiologists at the University of Connecticut Health Center, in Farmington, has suggested a link between cancer and the “circadian disruption” of hormones caused by artificial lighting. Early in his career, Stevens was one of many researchers struck by the markedly high incidence of breast cancer among women in the industrial world, in comparison with those in developing countries, and he at first supported the most common early hypothesis, which was that the cause must be dietary. Yet repeated studies found no clear link to food. In the early eighties, Stevens told me recently, “I literally woke up in the middle of the night — there was a street lamp outside the window, and it was so bright that I could almost read in my bedroom — and I thought, Could it be that? A few years later, he persuaded the [directors] of the Nurses’ Health Study . . . to add questions about nighttime employment, and the study subsequently revealed a strong association between working the night shift and an increased risk of breast cancer. [The researchers] wrote, “We hypothesize that the potential primary culprit for this observed association is the lack of melatonin, a cancer-protective agent whose production is severely diminished in people exposed to light at night.”

Exposure to strong light at night reduces the amplitude of your circadian rhythms. That causes a thousand changes. To decide that one of them (“lack of melatonin”) is the one that matters is highly premature. If reducing circadian amplitude increases cancer, it follows that getting more light during the day — which surely increases circadian amplitude — will reduce cancer.

The article also says:

Growing numbers of us pass most of our waking hours “in a box, looking at a box,” as Dave Crawford put it . . . Fewer and fewer of us spend much time outside at all, except in automobiles.

I have measured the light inside cars (front seats) several times and found it is quite strong (you are close to a big window). If the article is arguing that night light is bad and causes cancer, I am unconvinced. Night light exposure and daylight exposure are confounded — people who work night shifts get more night light and less daylight.
The Nurses Study paper: E Schernhammer, K Schulmeister. Light at night and cancer risk. Photochemistry and Photobiology, 2004, Vol 79, Iss 4, pp 316-318.

Where Does Oil Come From?

Seth Roberts6 Comments

This fascinating article describes two ideas about oil production that were new to me: 1. It is made by microbes a long way down inside the Earth. 2. It is made by nuclear reactions going on in the middle of the earth.

The first idea came from a Cornell geologist named Thomas Gold. According to the article, “some geologists were so incensed by Gold’s ideas they petitioned to have the government remove all mention of it from the nation’s libraries.” That is so strange (and no source cited) you might think the whole article is made-up but Gold explained his ideas here (short) and here (long).

Thanks to Carl Willat.

Chinese Text-Message Censorship

Seth Roberts4 Comments

An American friend wondered about Chinese reaction to Liu Xiaobo winning the Nobel Peace Prize. So I sent a text message to four Chinese friends: “Did u know chinese dissident won nobel peace prize?”

Six hours later I hadn’t received any replies. I phoned one of them. She hadn’t received my message. I saw it hadn’t gone through: “Unable to send message”. I tried again. Failed again. Then I tried to send “did u know chinese dissident won prize?” Success. I tried the earlier message, with “nobel peace prize”. Fail. Tried the shorter message. Success.

Messages almost never fail, so it was clearly censorship. One of my friends said the same about messages in Chinese: “We can’t send email or text messages” about it. She had heard about the prize from her classmates. She didn’t know how they had learned about it. Another Chinese friend read it on a website. Later it was gone.

I thought of the boy who cried wolf. One day there was a wolf. But no one believed him.

Dairy Consumption and Health

Seth Roberts6 Comments

Two studies of the effect of dairy consumption on health have recently appeared. Both suggest it is healthy. One of them– a prospective study where about 1500 people were followed for 16 years — found no association of dairy intake with overall mortality but did find a protective effect of full-fat dairy against heart disease. The study considered lots of possibilities and the authors write ” it is important to take into account the large number of comparisons considered in this study and thus we cannot rule out the possibility that the protective association between full-fat dairy intake and cardiovascular mortality was due to chance.”

I mentioned this study earlier. It gains more credence because of the other study, which is a meta-analysis. The second study found protective effects of dairy products on several outcomes, including overall mortality:

Meta-analyses suggest a reduction in risk in the subjects with the highest dairy consumption relative to those with the lowest intake: 0.87 (0.77, 0.98) for all-cause deaths, 0.92 (0.80, 0.99) for ischaemic heart disease, 0.79 (0.68, 0.91) for stroke and 0.85 (0.75, 0.96) for incident diabetes.

This is good news for me since I eat yogurt and butter every day.

Thanks to Peter Spero.

Research Fraud in China

Seth Roberts9 Comments

From the New York Times:

Last December, a British journal that specializes in crystal formations announced that it was withdrawing more than 70 papers by Chinese authors whose research was of questionable originality or rigor. . . . “Even fake papers count because nobody actually reads them,” said Mr. Fang, who is more widely known by his pen name, Fang Zhouzi, and whose Web site, New Threads, has exposed more than 900 instances of fakery, some involving university presidents and nationally lionized researchers.

Recently a Tsinghua colleague asked me to fix the English in his paper. Most paragraphs required a few changes every sentence but here and there were whole paragraphs with no mistakes. Presumably he copied them from somewhere else. The material in them was boring — it was like copying from the phone book — so it was hard to care (he wasn’t taking credit for anyone else’s ideas) but I wonder if he realized how obvious it was. I don’t mean this is typical. I have looked at several other papers by Chinese authors and found no patches of perfect English.

The article begins with a false claim by a Chinese doctor — and of course these are truly damaging. In my experience, false claims by American doctors are common. An example is my surgeon recommending an operation that, she said, evidence showed would benefit me. There was no such evidence. One value of self-experimentation is that you can find out if a medicine works, rather than take your doctor’s word for it. I became impressed with self-experimentation when it showed me that an acne medicine (tetracycline, an antibiotic) my dermatologist had prescribed didn’t work. Not at all. He didn’t express any doubts when he prescribed it. Call it forensic DNA testing (e.g., The Innocence Project) for the rest of us.

Perhaps the Chinese people, faced with even more false claims than Americans, can benefit even more from self-experimentation.

Thanks to Tim Beneke.

How to Eat a Lot of Butter

Seth Roberts25 Comments

Since I discovered that butter makes my brain work better, I have been eating half a stick (60 g) per day. Usually half in the morning and half in the evening. It is hard to eat by itself but easy to eat with other foods. I’ve tried a dozen ways of doing this. My top three additions:

1. Pu’er tea. The most convenient. As convenient as drinking tea. Put the butter in hot tea, wait till it melts. I can eat at least 20 g of butter in one cup of tea. Butter tea is common in Tibet. Thanks again to Robin Barooah.

2. Cherry tomatoes. The healthiest and fastest. Slice the tomatoes in half lengthwise, eat each half with a similar-sized piece of butter. It is like that classic Italian combination, mozzarella and tomatoes.

3. Thin-sliced roast beef. The most delicious. Wrap a piece of butter with the roast beef. However, I already eat plenty of meat, it is hard to get thin-sliced roast beef in Beijing, and it is so delicious I end up buying a lot of thin-sliced roast beef.

None of these additions affects brain function (measured by arithmetic score), as far as I can tell, although I suppose the tea wakes me up.

Quantified Self in National Post

Seth Roberts7 Comments

The National Post, a large Canadian newspaper, has a long article about quantitative self-tracking. Overall I like it. It looks at the subject in five or six ways, it focuses on examples of self-tracking rather than people generalizing about it, and, best of all, it includes actual data.

I wasn’t so pleased with the treatment of my work. First, the graph showing my butter data was wrongly labeled and the dividing line between before butter and during butter put in the wrong place. (These mistakes have been fixed.)Â Second, the description of my acne experiments — my dermatologist prescribed Medicines A and B, I found that only B worked — misses the point. True, I found that B worked better than A but far more interesting is that Medicine A (an antibiotic) didn’t work at all. Contrary to what I believed. Antibiotics are dangerous. How many people are taking dangerous drugs with no benefit? Third, the written description of my butter research doesn’t say the main point: butter improved my brain function in the sense that I did arithmetic faster. Instead it says I found butter was better than “standing on something painful”. A billion people would like better-functioning brains. None of them care whether butter is better than standing on something painful.

I pointed out the last two problems to Kathryn Carlson, the author of the article. She replied that in the future she would call me to go over the accuracy of the relevant parts of the article. I had considered asking the people who made the graph to show it to me but had thought because they had my graph of the same data in front of them, they couldn’t go wrong.

A good lesson for me.

This Year’s Nobel Prize in Medicine

Seth Roberts5 Comments

I applaud it. The winner developed in vitro fertilization, which has helped millions of parents. In contrast to last year’s prize for telomere research, which has helped no one. Notice what in vitro fertilization is not: It is not taking a powerful poorly-tested drug for the rest of your life — the drug industry’s preferred answer to all problems. It is not expensive (given the benefits). Unlike health care in general. It is not dangerous, unlike many drugs and surgeries. It is not molecular biology. It is barely science (uncovering cause and effect). If the prize were given for research like this year after year, many biologists who now dream of winning a Nobel Prize would stop dreaming. It is not a typical Nobel Prize. They waited so long to award it that the winner became demented. Above all, the prize-winning work was not mainstream medical research. The winner and his collaborator endured “an unremitting barrage of criticism”, unlike almost any other medical researcher.

The award is unflattering to medical ethicists, who did a lot to try to prevent the prize-winning work.

Out-of-Control Drug Companies and Their Consorts

Seth Roberts2 Comments

The increase in pediatric bipolar diagnosis, [Dr. Joseph Biederman’s] lawyer said, “cannot be attributed solely to Dr. Biederman’s work.”

This article is a long list of drug-company abuses, such as

Lilly produced a video called “The Myth of Diabetes” to sell Zyprexa, which became its all-time best-selling drug, even though evidence showed that Zyprexa could cause diabetes.

And

Pfizer paid more than 250 child psychiatrists to promote its antipsychotic, Geodon, at a time when it was approved only for adults, according to a government filing with the Pfizer settlement last year. High-prescribing doctors pocketed extra money in the form of research payments, speaking fees, gifts, meals and junkets.

Pharma Gossip. Thanks to Ken Feinstein.