Thanks to JR Minkel.
In response to my post about the trouble with randomized controlled trials (RCTs), Bruce Charlton, the editor of Medical Hypotheses, wrote me:
The golden age of medical discovery came before the widespread usage of RCTs. This golden age was all but over by the end of the 1960s; since then the rate of progress has declined (see refs such as Horrobin, Le Fanu and Wurtman in https://www.hedweb.com/bgcharlton/funding.html).
The earliest big and influential RCT in psychiatry was in the mid 1960s, and it was – in retrospect – misleading wrt MAOIs due to too low a dosage. Now that RCTs are regarded as indispensible, medical research is captive to Big Pharma
https://www.guardian.co.uk/commentisfree/2009/aug/08/seroxat-pharmaceutical-birth-defect
Another area of medicine [in addition to obstetrics] that has made big progress without being RCT-led is anesthetics. Dentistry is a third. These specialties are instead technology-led.
He also pointed me to an article by David Horrobin, the founder of Medical Hypotheses, titled “Are large clinical trials in rapidly lethal diseases usually unethical?” His answer was that some of their aspects are unethical: Prospective subjects (sick persons) are not told the low chance of benefit, the high chance of bad side effects, and the great financial benefit of such trials to the institutions that run them.
Horrobin’s article also made the point I made: The emphasis on RCTs suppresses innovation because only big well-established companies can afford them:
50 years ago, good scientific evidence of a potential therapeutic effect would quickly have generated a small clinical trial in one or two centers with perhaps 30 or 40 patients. Such a trial would have cost almost nothing. It would certainly have missed small or marginal effects, but it would not have missed the sort of large effect that most patients want. Unfortunately, now, such an approach has become impossible. . . . The escalation of costs has therefore drastically reduced the range of compounds from which new treatments can be drawn.
My reading of history is that suppression of innovation can last a long time but eventually change comes from the outside and the system collapses. Detroit, for example, has collapsed. General Motors was once as dominant as big drug companies are now.
In an email to a friend, I compared the obsession of med school professors with methodological purity (e.g., efficacy must be demonstrated with an RCT, randomized controlled trial) to religious ritual. More concern with appearances (ritual), I said, is linked to less understanding of substance. My friend replied:
I am actually a believer in this particular religion (The Cult of RCT)! Seriously: I think the medical world is quite right to put a huge premium on RCTs, because RCTs so often prove that things they are doing don’t work. While sometimes the RCT may provide a negative verdict on something that does work, this seems to me an unusual case, and generally avoidable if one considers statistical power, possible subgroup responses, etc and avoids overgeneralizing the conclusions.
I replied:
Are RCTs better than what prevailed before? Probably. But I would say the same about religion, which has its benefits.
I think the medical world has turned off a large fraction of its brain via insistence on RCTs and failure to understand their weaknesses and the strengths of alternatives. It isn’t just that “RCTs may return a negative verdict on something that works,” it’s also that such a requirement for very expensive research suppresses innovation — testing things via cheaper ways. Atul Gawande wrote about how obstetricians made a lot of progress by ignoring this requirement:
https://www.newyorker.com/archive/2006/10/09/061009fa_fact
Other areas of medicine, which followed the RCT requirement, made less progress during the same period, it can be argued.
Let’s say I told you that the only way you can travel to work is via an armed escort — you would be appalled, even though it’s true you would be safer. An insistence on RCTs is overreaction. Given the lack of innovation in medicine/health care, for which I believe they (or at least the lack of understanding they embody) are partly responsible, very expensive overreaction.
The best way to learn is to do. The best way to learn about health is to do as many experiments as possible. Not slow, expensive RCTs. Not slow, expensive surveys, which don’t involving “doing” to the extent that an experiment does. This is a big reason my self-experiments taught me a lot — because I could do so many of them.
Doenjang is a fermented soybean paste often served in Korean restaurants — as a vegetable dip, for example. This study found that the longer it’s fermented, the more powerful its anti-cancer action:
Doenjang fermented for 24 mo exhibited a two- to three-fold increase in antitumor effects on sarcoma-180-injected mice and antimetastatic effects in colon 26-M 3.1 cells in mice compared with the 3- or 6-mo fermented doenjang. The 24-mo fermentation was the most effective in preventing cancer by decreasing tumor formation and increasing natural killer cell activity in spleens and glutathione S-transferase activity in livers of mice.
Many things about doenjang stay roughly the same during fermentation. This study shows that what’s increasing (bacteria, etc.) is responsible for the anti-cancer effect, which supports my umami hypothesis (that we need fermented foods or something similar to be healthy).
I make yogurt rather than buy it so that I can ferment it a long time (e.g., 24 hours in a yogurt machine). The yogurt I make is much sourer than commercial yogurt.
Thanks to Michael Bowerman.
In a wiser world, we would see genetics research as we see astronomy: worth supporting, but without expecting practical benefit. In this world, however, genetics research is far better funded than astronomy and is expected to have practical benefits.
Unfortunately, the benefits have been slight. A New York Times article by Nicholas Wade makes this clear:
The primary goal of the $3 billion Human Genome Project — to ferret out the genetic roots of common diseases like and and then generate treatments — remains largely elusive. Indeed, after 10 years of effort, geneticists are almost back to square one in knowing where to look for the roots of common disease.
“Largely” elusive? Completely elusive is more accurate, as far as I know. Not one treatment has come from this work.
In spite of ten years of failure, geneticists appear no wiser than before:
With most diseases, the common variants have turned out to explain just a fraction of the genetic risk. It now seems more likely [to prominent geneticists] that each common disease is mostly caused by large numbers of rare variants.
I know of no examples where a common (or any) disease has been shown to be caused by “large numbers of rare variants.” Perhaps these estimates of “genetic risk” are as misleading as asking what percentage of the area of a rectangle is determined by its width.
History repeats. Ten years ago, geneticists had zero examples of how mapping the human genome would help anyone with a common disease. Absence of any examples didn’t prevent such vast claims as human genome mapping will ““revolutionize the diagnosis, prevention and treatment of most, if not all, human diseases”. From zero, they extrapolated to “most”.
It’s a sad comment on science journalism that, at the time, no one pointed out the absence of examples, as far as I know, and a sad comment on Wade, holder of a powerful and prestigious job, that he has not pointed it out now. He simply repeats a claim. At least he has noticed a gigantic failure after it happens, even if he inaccurately describes it (“largely” rather than “completely”).
Lack of examples of the practical value of genetic mapping didn’t keep a huge amount of money from being spent.
With the catalog [of common genetic variants] in hand, the second stage was to see if any of the variants were more common in the patients with a given disease than in healthy people. These studies required large numbers of patients and cost several million dollars apiece. Nearly 400 of them had been completed by 2009.
Ten failures would have been plenty; 400 failures shows the resistant-to-evidence nature of the whole enterprise. It’s an example of how a little biochemical-mechanism research goes a long way; a lot of biochemical-mechanism research goes a little way.
For geneticists, to acknowledge the lack of examples is scary. Their funding might be cut! So they don’t. But nothing prevents journalists from thinking for themselves and asking a supposedly “tough” question (“what’s an example?”) — although asking for examples is the most basic question there is.
Thanks to Alex Chernavsky. More about the cargo-cult nature of modern biology. If you don’t believe me, read this: “Of the roughly 50 companies at the conference, not one is focused on approaches related to tracking down new genes. . . . The one corner of the genome-focused biotech industry that’s thriving is the one churning out equipment and services to support researchers in their endless hunt for gene links.”
A reader of this blog reported the following conversation with his doctor:
us: We want to put our autistic daughter on a gluten-free, casein-free diet. We have heard that some autistic kids have gotten some benefit from it.
Dr: It’s not something I know about, but there is no harm in it so feel free to give it a try. You know, I had a patient whose parents put her on a ketogenic diet to treat her seizures, and it seemed to help. That goes against everything I was taught in med school, but if it works, I think that’s great.
What’s telling here is the word everything. In medical school, the doctor seems to say, he was taught in a dozen ways that the sun revolves around the earth (or its health-science equivalent). If the alternative — the earth revolves around the sun — explains more of the data, well, “that’s great.” Again, it sounds like 1984: Part of the doctor’s brain has been turned off by repetition of something that supports the status quo.
In Beijing, sky-high housing prices have left many couples who want to get married without the traditional requirement: an apartment they own. There’s a term for those who get married anyway: naked marriage. They are literally unprotected against rent increases. More abstractly, they are less comfortable than a couple rich enough to buy housing.
There is a high school in Osaka called (in Japanese) Ocean Science High School. It specializes in training students for fish industry jobs. During my visits to Japan, what’s most impressed me hasn’t been high-end restaurant food, as great as it is, but the way everyone seems to take pride in their job and doing it well. At one point a friend’s car got a flat tire. We limped to a service station. The attendant fixed the flat in 3 minutes, running around as if we were in a race. Typical for Japan, but unlike anywhere else I’ve been. I hope someday I can learn how this attitude is taught. Surely it has something to do with schools like Ocean Science — not Fish Industry — High School.
Black market = illegal. Grey market = “the trade of a commodity through distribution channels . . . unofficial, unauthorized, or unintended.”
In the evening, near the Wudaokou subway station in Beijing (where lots of students live), dozens of street vendors sell paperbacks ($1 each), jewelry, dresses, socks, scarves, electronic accessories, fruit, toys, shoes, cooked food, stuffed animals, and many other things. No doubt it’s illegal. When a police car approaches, they pick up and leave. Once I saw a group of policemen confiscate a woman’s goods.
What’s curious is how far vendors move when police approach. Once I saw the vendors on a corner, all 12 of them, each with a cart, move to the middle of the intersection — the middle of traffic — where they clustered. At the time I thought the traffic somehow protected them. Now I think they wanted to move back fast when the police car went away. Tonight, like last night, there’s a police car at that corner, the northeast corner of the intersection. No vendors there. The vendors who’d usually be there were now at the northwest corner. In other words, if a policeman got out of his car and walked across the street, he’d encounter all the vendors that he’d displaced.