Category: How Things Begin

Testing Treatments: Nine Questions For the Authors

Seth Roberts14 Comments

From this comment (thanks, Elizabeth Molin) I learned of a British book called Testing Treatments (pdf), whose second edition has just come out. Its goal is to make readers more sophisticated consumers of medical research. To help them distinguish “good” science from “bad” science. Ben Goldacre, the Bad Science columnist, fulsomely praises it (“I genuinely, truly, cannot recommend this awesome book highly enough for its clarity, depth, and humanity”). He wrote a foreword. The main text is by Imogen Evans (medical journalist), Hazel Thornton (writer), Iain Chalmers (medical researcher), and Paul Glaziou (medical researcher, editor of Journal of Evidence-Based Medicine).

To me, as I’ve said, medical research is almost entirely bad. Almost all medical researchers accept two remarkable rules: (a) first, let them get sick and (b) no cheap remedies. These rules severely limit what is studied. In terms of useful progress, the price of these limits has been enormous: near total enfeeblement. For many years the Nobel Prize in Medicine has documented the continuing failure of medical researchers all over the world to make significant progress on all major health problems, including depression, heart disease, obesity, cancer, diabetes, stroke, and so on. It is consistent with their level of understanding that some people associated with medicine would write a book about how to do something (good science) the whole field manifestly can’t do. Testing Treatments isn’t just a fat person writing a book about how to lose weight, it’s the author failing to notice he’s fat.

In case the lesson of the Nobel Prizes isn’t clear, here are some questions for the authors:

1. Why no chapter on prevention research? To fail to discuss prevention, which should be at least half of health care, at length is like writing a book using only half the letters of the alphabet. The authors appear unaware they have done so.

2. Why are practically all common medical treatments expensive?

3. Why should some data be ignored (“clear rules are followed, describing where to look for evidence, what evidence can be included”)? The “systematic reviews” that Goldacre praises here (p. 12) may ignore 95% of available data.

4. The book says: “Patients with life-threatening conditions can be desperate to try anything, including untested ‘treatments’. But it is far better for them to consider enrolling in a suitable clinical trial in which a new treatment is being compared with the current best treatment.” Really? Perhaps an ancient treatment (to authors, untested) would be better. Why are there never clinical trials that compare current treatments (e.g., drugs) to ancient treatments? The ancient treatments, unlike the current ones, have passed the test of time. (The authors appear unaware of this test.) Why is the comparison always one relatively new treatment versus another even newer treatment?

5. Why does all the research you discuss center on reducing symptoms rather than discovering underlying causes? Isn’t the latter vastly more helpful than the former?

6. In a discussion of how to treat arthritis (pp. 170-172), why no mention of omega-3? Many people (with good reason, including this) consider omega-3 anti-inflammatory. Isn’t inflammation a major source of disease?

7. Why is there nothing about how to make your immune system work better? Why is this topic absent from the examples? The immune system is mentioned only once (“Bacterial infections, such as pneumonia, which are associated with the children’s weakened immune system, are a common cause of death [in children with AIDS]“).

8. Care to defend what you say about “ghostwriting” (where med school professors are the stated authors of papers they didn’t write)? You say ghostwriting is when “a professional writer writes text that is officially credited to someone else” (p. 124). Officially credited? Please explain. You also say “ghostwritten material appears in academic publications too – and with potentially worrying consequences” (p. 124). Potentially worrying consequences? You’re not sure?

9. Have you ever discovered a useful treatment? No such discoveries are described in “About the Authors” nor does the main text contain examples. If not, why do you think you know how? If you’re just repeating what others have said, why do you think your teachers are capable of useful discovery? The authors dedicate the book to someone “who encouraged us repeatedly to challenge authority.” Did you ever ask your teachers for evidence that evidence-based medicine is an improvement?

The sad irony of Testing Treatments is that it glorifies evidence-based medicine. According to that line of thinking, doctors should ask for evidence of effectiveness. They should not simply prescribe the conventional treatment. In a meta sense, the authors of Testing Treatments have made exactly the mistake that evidence-based medicine was supposed to fix: Failure to look at evidence. They have failed to see abundant evidence (e.g., the Nobel Prizes) that, better or not, evidence-based medicine is little use.

Above all, the authors of Testing Treatments and the architects of evidence-based medicine have failed to ask: How do new ideas begin? How can we encourage them? Healthy science is more than hypothesis testing; it includes hypothesis generation — and therefore includes methods for doing so. What are those methods? By denigrating and ignoring and telling others to ignore what they call “low-quality evidence” (e.g., case studies), the architects of evidence-based medicine have stifled the growth of new ideas. Ordinary doctors cannot do double-blind clinical trials. Yet they can gather data. They can write case reports. They can do n=1 experiments. They can do n=8 experiments (“case series”). There are millions of ordinary doctors, some very smart and creative (e.g., Jack Kruse). They are potentially a great source of new ideas about how to improve health. By denigrating what ordinary doctors can do (the evidence they can collect) — not to mention what the rest of us can do — and by failing to understand innovation, the architects of evidence-based medicine have made a bad situation (the two rules I mentioned earlier) even worse. They have further reduced the ability of the whole field to innovate, to find practical solutions to common problems.

Evidence-based medicine is religion-like in its emphasis on hierarchy (grades of evidence) and rule-following. In the design of religions, these features made sense (to the designers). You want unquestioning obedience (followers must not question leaders) and you want the focus to be on procedure (rules and rituals) rather than concrete results. Like many religions, evidence-based medicine draws lines (on this side “good”, on that side “bad”) where no lines actually exist. Such line-drawing helps religious leaders because it allows their followers to feel superior to someone (to people outside their religion). When it comes to science, however, these features make things worse. Good ideas can come from anybody, high or low in the hierarchy, on either side of any line. And every scientist comes to realize, if they didn’t already know, that you can’t do good science simply by following rules. It is harder than that. You have to pay close attention to what happens and be flexible. Evidence-based medicine is the opposite of flexible. “ There is considerable intellectual tyranny in the name of science,” said Richard Feynman.

Testing Treatments has plenty of stories. Here I agree with the authors — good stories. It’s the rest of the book that shows their misunderstanding. I would replace the book’s many pages of advice and sermonizing with a few simple words: Ask your doctor for the evidence behind their treatment recommendation. He or she may not want to tell you. Insist. Don’t settle for vague banalities (“It’s good to catch these things early”). Don’t worry about being “difficult”. You won’t find this advice anywhere in Testing Treatments. If I wanted to help patients, I would find out what happens when it is followed.

More Two of the authors respond in the comments. And I comment on their response.

Brain Surprise! Why Did I Do So Well?

Seth Roberts9 Comments

For the last four years or so I have daily measured how well my brain is working by means of balance measurements and mental tests. For three years I have used a test of simple arithmetic (e.g, 7 * 8, 2 + 5). I try to answer as fast as possible. I take faster answers to indicate a better-functioning brain.

Yesterday my score was much better than usual. This shows what happened.

My usual average is about 550 msec or more; my score yesterday was 525 msec. An unexplained improvement of 25 msec.

What caused the improvement? I came up with a list of ways that yesterday was much different than usual, that is, was an outlier in other ways. These are possible causes. From more to less plausible:

1. I had 33 g extra flaxseed last night. (By mistake. I’m not sure about this.)

2. The test came at the perfect time after I had my afternoon yogurt with 33 g flaxseed. When I took flaxseed oil (now I eat ground flaxseed), it was clear that there was a short-term improvement for a few hours.

3. Many afternoons I eat 33 g ground flaxseed with yogurt. Yesterday I ground the afternoon flaxseed an unusually long time, making made the omega-3 more digestible.

4. I did kettlebells swings and a kettlebell walk about 2 hours before the test. These exercises are not new but usually I do them on different days. Yesterday was the first time I’ve done them on the same day. I’m sure ordinary walking improves performance for perhaps 30 minutes after I stop walking.

5. I had duck and miso soup a half-hour before the test. Almost never eat this.

6. I had a fermented egg (“thousand-year-old egg”) at noon. I rarely eat them.

7. I had peanuts with my yogurt and ground flaxseed. Peanuts alone seem to have no effect. Perhaps something in the peanuts improves digestion of the omega-3 in the flaxseed.

8. I started watching faces at 7 am that morning instead of 6:30 am or earlier.

Here are eight ideas to test. Perhaps one or two will turn out to be important. Perhaps none will.

After I made this list, I read student papers. The assignment was to comment on a research article. One of the articles was about the effect of holding a warm versus cold coffee cup. Holding a warm coffee cup makes you act “warmer,” said the article. Commenting on this, a student said she thought it was ridiculous until she remembered going to the barber. She sees the person who washes her hair (in warm water) as friendly, the barber as cold. Maybe this is due to the warm water used to wash her hair, she noted. This made me realize another unusual feature of yesterday: I had washed my hair in warm water longer than usual. I think I did it at least 30 minutes before the arithmetic test but I’m not sure. In any case, here is another idea to test. I found earlier that cold showers slowed down my arithmetic speed.

This illustrates a big advantage of personal science (science done for personal gain) over professional science (science done because it’s your job): The random variation in my life may suggest plausible new ideas. As far as I can tell, professional scientists have learned almost nothing about practical ways to make your brain work better. You can find many lists of “brain food” on the internet. Inevitably the evidence is weak. I’d be surprised if any of them helped more than a tiny amount (in my test, a few msec). The real brain foods, in my experience, are butter and omega-3. Perhaps my tests will merely confirm the value of omega-3 (Explanations 1-3). But perhaps not (Explanations 4-8 and head heating).

Jane Jacobs and Amazon.com

Seth RobertsLeave a comment

How did air-breathing evolve? In The Nature of Economies (p. 87), Jane Jacobs uses it to illustrate the developmental pattern she calls “bifurcation” (air-breathing isn’t a refinement of water-breathing). She speculates on how it started:

Lungfish had both gills and a primitive lung, suggesting that their habitat was swampland. The earliest to take to dry land may have inhabited swamps subject to severe droughts or perhaps they were escaping fearsomely-jawed predators who couldn’t follow them to dry land.

According to Steve Yegge’s already-famous “psst, Googlers” memo, something much like this was why Amazon started selling web computing services, which wasn’t a refinement of their earlier business (selling books, toys, etc.):

Amazon was a product company too, so it took an out-of-band force to make Bezos understand the need for a platform. That force was their evaporating margins; he was cornered and had to think of a way out. But all he had was a bunch of engineers and all these computers… if only they could be monetized somehow… you can see how he arrived at AWS [Amazon Web Services], in hindsight.

People say necessity is the mother of invention. That isn’t even close to true. Trial and error is the mother of true, profound invention. The Bezos story, and Jacobs’s generalization of it, suggest what is actually true: necessity is the mother of development. Necessity pushes people to use, and thereby develop, inventions they had ignored.

Chapter 1 of The Nature of Economies.

More News about Liberation Therapy

Seth RobertsLeave a comment

An Italian surgeon, Paolo Zamboni, claimed that he found low blood flow from the brain in 100% of patients with multiple sclerosis (MS). He began by studying his wife.

A new study supports the connection:

The Canadian researchers analyzed eight studies from Italy, Germany, Jordan and the U.S. that involved 664 MS patients in total. The studies looked at how frequently CCSVI [chronic cerebrospinal venous insufficiency] was found in people with MS compared to healthy people or those with other neurological disorders such as Parkinson’s disease.

One of the studies — Zamboni’s — found CCSVI in 100 per cent of people with MS, and zero per cent of people without the disease. Other studies found the vein abnormalities in people who didn’t have MS.

Overall, when the results were combined, people with MS were 13.5 times more likely to have CCSVI. Even when the study by Zamboni — which generated the excitement about CCSVI — was removed, the syndrome was 3.7 times more common in people with MS.

Thanks to Anne Weiss.

Google Yes, Wikipedia Yes, Aaron Swartz No?

Seth Roberts6 Comments

We praise Google and Wikipedia for making knowledge more available — consider them two of the best innovations of the last 50 years — but after Aaron Swartz, a friend of mine, apparently tried to do the same thing he was charged with wire and computer fraud and faces up to 35 years in jail and a $1 million fine.

The prosecutor, U.S. Attorney Carmen Ortiz, made an interesting statement:

Stealing is stealing, whether you use a computer command or a crowbar, and whether you take documents, data or dollars. It is equally harmful to the victim whether you sell what you have stolen or give it away.

In my experience, tautological statements such as “stealing is stealing” or “correlation is not causation” do not bode well for that side of the argument. As Thorstein Veblen might say, the reason for the tautology was the need for it.

Ortiz’s statement shows that she, like the rest of us, thinks that what matters is amount of harm. Harm is hard to find here. The only clear harm is that MIT access to JSTOR was shut down for a few days. This is so minor that JSTOR’s statement about the case (which includes “it was the government’s decision whether to prosecute, not JSTOR’s. . . . We [have] no interest in this becoming an ongoing legal matter”) doesn’t mention it. I don’t think many people will agree that this amount of harm justifies the charges that Ortiz has brought.

Sign a petition supporting Aaron.

How Things Begin (Time Out)

Seth Roberts1 Comment

Time Out magazine was started in 1968 in London by Tony Elliot, who was 22 at the time. The original title was Where It’s At. There were all sorts of new cultural stuff, such as a concert by the Who, that the mainstream media didn’t notice. The fringe-y alternative media weren’t interested in the attention to detail required to put out a list of events. That was the gap Time Out filled. Elliot borrowed a small amount of money (70 pounds) to start it. He and his co-workers worked without pay for the first three or four months. It was hard to get distribution, so they went around to parks passing it out. At a Beijing talk, Elliot said he didn’t remember the first paid advertiser (maybe a music store) but he did remember when he got an unsolicited advertising order from the prestigious London Film Museum. They understand what we’re trying to do, he thought.

I asked what some of his biggest mistakes had been. Both involved not saying no when he should have said no.

How Things Begin: The Fleming Fund

Seth Roberts1 Comment

Better to light a candle than curse the darkness, the saying goes. What if there are no candles?

Ken Rousseau, a software manager in Silicon Valley, went to Caltech in the late 70s. He didn’t have a good time. He was a physics major He took a required course on electricity and magnetism where the average score on the final was 15 out of 100. As he took it, he thought, I guess I can’t be a physics major. He got a 16 — a solid B. That a professor would design such a demoralizing test revealed, he believed, that the professor didn’t care about students. At Tech, lack of caring for students was shown in big things and small. Every building on campus was air-conditioned except the student houses, and Pasadena gets really hot in the summer. The graduation rate around that time — the fraction of entering students who graduate in four years — was 59%. At MIT it was 80 or 90%. When a student drops out of Tech, it’s a lost opportunity on both sides, Rousseau felt. It was/is very difficult to get into Tech. To send 41% of admitted students away struck him as a terrible thing.

He did graduate. For many years, when Tech would ask him for money, he would say no, sometimes with a letter about why. But he kept in touch with other students who had lived in the same undergraduate house (Fleming House), one of the seven student houses. Every year, a bunch of them would have a weekend-long beach party. At one of them the idea arose: Let’s start a Fleming Fund. To help the students buy beer, that sort of thing. Tech is a tough place, let’s help them get through it.

In the 1990s, Rousseau got a letter from the president of Caltech that made him angry. Tech was #4 in the U.S. News rankings, it said, mainly because of the low fraction of alumni giving. Let’s make Tech #1 by giving more, wrote the president. Rousseau responded with a five-page letter that made one simple point: Alumni giving is so low because the people in charge cared so little about students. Their lack of concern is being reciprocated.

By 2003 or 2004 Rousseau had enough money that he got a personal visit from the development office. His visitor knew his wife’s name, the approximate ages of his children, and the high points of his professional career. Rousseau told him of his residual bitterness. “You’ve obviously benefited a lot from your Tech experience,” said the development officer. “Why have you only given $163 over the years?” He had it wrong, Rousseau said. He had given $1. His wife, who had also gone to Tech, had given $162.

He told the development officer he was interested in helping Tech students — particularly Fleming House residents. In essence, he wanted to bring the Fleming Fund into existence. Around this time, Frank Bernstein, another Caltech alum who was working as a patent attorney in Silicon Valley, was also solicited. “Frank, I’m looking for a really significant donation,” said the same development officer who had approached Rousseau. Bernstein, who’d also lived in Fleming, told Rousseau about the conversation and they again resurrected the idea of the Fleming Fund.

The development officer came back to them with ideas. Maybe you could fund a lecturer, he suggested. Or graduate student salaries. Helping undergraduates was clearly a new and difficult concept for the development office. They were looking for contributions that, in their words, “directly benefited the Institute.” Bernstein pointed out to them that this was a narrow and self-defeating view. They want alumni to contribute. They want to get them in the habit of contributing. A Fleming Fund will help with that.

Because Rousseau’s daughter, a high school student, was considering going to Tech, Rousseau visited the campus in 2006. He met with Tom Mannion, the administrator for student affairs, and came to believe that the administration cared more about students than they had in the past. A new incoming president, Jean-Lou Chameau, appeared to genuinely care about undergrads. (Later events have validated that view. Chameau has made a point of discussing student life in his public discussions and has started to push administration officials to discuss what they’re doing with regards to student life.) After that, Rousseau and Bernstein met with the development officer who had solicited them and started working on the details. The Institute set a minimum of $100,000. Once the fund reached this level, income from the fund would be given to the students to spend.

In 2008 the details were hammered out. There would be two sort of restrictions: 1. Obvious limits on what the money could be spent on (no bail, no illegal drugs, etc.). 2. An oversight committee of three people, including the past president of Fleming House. The oversight committee only gets involved when the amount of money is more than the house’s usual budget. The income, at least at first, would be about $10,000 year for a house of about 120 students.

In May 2009, the fund was announced during a Fleming House reunion dinner at Tom Mannion’s house. Many undergrads came up to Rousseau and told him it was a “really cool idea.” They were touched that someone out there cared about them. The Institute is thinking of repeating it with the other student houses.

Yogurt Popularizer Dies: Note How Old He Was

Seth Roberts3 Comments

Daniel Carasso, who popularized yogurt worldwide via the Dannon brand, died on Sunday. He was 103. From the obituary in the NY Times:

In 1916 his father took the family back to Spain, where he [the father] became disturbed by the high incidence of intestinal disorders, especially among children. Isaac Carasso [Daniel’s father] began studying the work of Elie Metchnikoff, the Russian microbiologist who believed that human life could be extended by introducing lactic-acid bacilli, found in yogurt and sour milk, into the digestive system. Using cultures developed at the Pasteur Institute in Paris, Isaac began producing Danone. . . in 1941 the arrival of the Nazis forced [Daniel Carasso] to flee to the United States. There he formed a partnership with two family friends, Joe Metzger, a Swiss-born Spanish businessman, and his son Juan, whose flair for marketing would make Dannon a household name in the United States. . .The little company operated at a loss until 1947, when, in a concession to the American sweet tooth, strawberry jam was added to the yogurt. Sales took off, new flavors were added to the product line, and Dannon yogurt made the leap from specialty product to snack food and dessert.

Thanks to Marian Lizzi.