Category: Twilight of Expertise

The Twilight of Expertise (medical doctors)

Seth Roberts10 Comments

Long ago the RAND Corporation ran an experiment that found that additional medical spending provided no additional health benefit (except in a few cases). People who didn’t like the implication that ordinary medical care was at least partly worthless could say that it was only at the margin that the benefits stopped. This was unlikely but possible. Now a non-experimental study has found essentially the same thing:

To that end, Orszag has become intrigued by the work of Mitchell Seltzer, a hospital consultant in central New Jersey. Seltzer has collected large amounts of data from his clients on how various doctors treat patients, and his numbers present a very similar picture to the regional data. Seltzer told me that big-spending doctors typically explain their treatment by insisting they have sicker patients than their colleagues. In response he has made charts breaking down the costs of care into thin diagnostic categories, like “respiratory-system diagnosis with ventilator support, severity: 4,” in order to compare doctors who were treating the same ailment. The charts make the point clearly. Doctors who spent more — on extra tests or high-tech treatments, for instance — didn’t get better results than their more conservative colleagues. In many cases, patients of the aggressive doctors stay sicker longer and die sooner because of the risks that come with invasive care.

Perhaps the doctors who ordered the high-tech treatments, when questioned about their efficacy, would have responded as my surgeon did to a similar question about the surgery she recommended (and would make thousands of dollars from): The studies are easy to find, just use Google. (There were no studies.)

It’s like the RAND study: Defenders of doctors will say that some of them didn’t know what they were doing but the rest did. But that’s the most doctor-friendly interpretation. A more realistic interpretation is that a large fraction of the profession doesn’t care much about evidence. In everyday life, evidence is called feedback. If you are driving and you don’t pay attention to and fix small deviations from the middle of the road, eventually you crash. You don’t need a double-blind clinical trial not to crash your car — a lesson the average doctor, the average medical school professor, and the average Evidence-Based-Medicine advocate haven’t learned.

The Twilight of Expertise (part 16: opticians)

Seth Roberts3 Comments

These glasses can help everyone, not just the poor:

The wearer adjusts a dial on the syringe to add or reduce amount of fluid in the membrane, thus changing the power of the lens. When the wearer is happy with the strength of each lens the membrane is sealed by twisting a small screw, and the syringes removed. The principle is so simple, the team has discovered, that with very little guidance people are perfectly capable of creating glasses to their own prescription.

[Josh] Silver [a retired professor of physics] calls his flash of insight a “tremendous glimpse of the obvious” – namely that opticians weren’t necessary to provide glasses

Speaking of not needing opticians and making glasses more affordable, a year ago I discovered by accident something extremely useful: Wearing one contact lens is better than wearing two.

Wearing just one contact lens, I get good distance vision from the lensed eye and and good close-up vision from the unlensed eye. Wearing two contact lenses, I have poor close-up vision. Another benefit of one rather than two contact lenses is that one eye is contact-lens-free for a long time. And I go through contact lenses half as fast. I wear lenses that last one month so I switch monthly which eye has the lens.

No optician told me this. No optician has even figured this out, as far as I know.

The Case of the Missing Evidence

Seth Roberts38 Comments

The most telling detail in Robin Hanson’s lecture about doctors was about a nurse assigned to measure hand-washing rates among surgeons at her hospital. After she measured the hand-washing rates, she — as ordered — correlated them with death rates. It turned out that the surgeon who washed his hands the least had the highest death rate. For reporting this — as she was ordered to — the nurse was fired. Robin learned this story from his wife, who was a friend of the ex-nurse.

I was very impressed by Robin’s lecture, which was both accessible and profound, and it was one reason that during my next encounter with a doctor I was more skeptical than most patients. As I blogged earlier:

I have a tiny hernia that I cannot detect but one day my primary-care doctor did. He referred me to Dr. [Eileen] Consorti, a general surgeon [in Berkeley]. She said I should have surgery for it. Why? I asked. Because it could get worse, she said. Eventually I asked: Why do you think it’s better to have surgery than not? Surgery is dangerous. (Not to mention expensive and time-consuming.) She said there were clinical trials that showed this. Just use google, you’ll find them, she said. I tried to find them. I looked and looked but failed to find any relevant evidence. My mom, who does medical searching for a living, was unable to find any completed clinical trials. One was in progress (which implied the answer to my question wasn’t known). I spoke to Dr. Consorti again. I can’t find any studies, I said, nor can my mom. Okay, we’ll find some and copy them for you, she said, you can come by the office and pick them up. She sounded completely sure the studies existed. I waited. Nothing from Dr. Consorti’s office. After a few weeks, I phoned her office and left a message. No reply. I waited a month, phoned again, and left another message. No reply.

Yesterday Dr. Consorti finally got back to me, by posting a comment:

Seth, While I am in the process of finding papers in the literature to satisfy your scientific curiosity on why this hernia should or should not be fixed I am additionally trying to care for around 30 new patients referred to me for their new cancer diagnosis in the last 3 months. This may or may not explain why I have not been motivated to answer your call regarding your ambivalence about fixing your hernia. Yes, it is small and runs the risk of incarceration at some time. I will call you once I clear my desk and do my own literature search. Thanks for the update. Eileen Consorti

Fair enough. She’s busy. And I am glad to have her reply and her view of the situation. On the other hand, I am pretty sure the studies she was so sure existed — that justified the surgery — don’t exist. To call my curiosity about whether the proposed surgery would do more good than harm “scientific” has a bit of truth: No doubt scientists understand better than others that you can test claims such as “you need this surgery”. But it isn’t “scientific” in the least to worry that a medical procedure will do more harm than good. Everyone, not just scientists, worries about that. Surgery is scary. Let’s set aside the death rate, which is low but non-zero. How many brain cells are killed by general anesthesia? Dr. Consorti doesn’t know, nor do I. The number is plausibly more than zero. I suspect a power-law distribution: Most instances of general anesthesia kill a small number, a small fraction kill a large number.

I pointed Robin to Dr. Consorti’s response. He replied:

I wonder if she even realizes that she in fact doesn’t know why you should get surgery.

What I know and Dr. Consorti, very reasonably, doesn’t know, is that my mom was a librarian at the UCSF medical library and has done a vast amount of medical-literature searching. If she can’t find any relevant studies, it is very likely they don’t exist. And my mom did find a study in progress, which, to repeat myself, shows that my question about cost versus benefit is a good one. Others had the same question and launched a study to answer it. Robin’s lecture helped me ask it. Thanks, Robin.

More. Robin’s version of the fired-nurse story is here. Thanks to Charles Williams.

The Twilight of Expertise (fugu liver removal)

Seth Roberts1 Comment

Fugu is a puffer fish prized by Japanese fish connoisseurs. Its liver is poisonous, thus only specially-trained chefs can serve it. A episode of The Simpsons featured Homer poisoned by fugu.

Recently, however, researchers determined that fugu liver is poisonous because fugu eat poisonous food. When fugu is farmed, and given non-poisonous food, its liver is harmless, and the fish tastes almost as good. No more need for special processing. Unsurprisingly, the National Fugu Association wants to preserve the status quo. But you can now buy fugu liver in the town of Useki.

Masataka Kinashi, the head of the tourism association in Usuki and a fugu dealer himself, suddenly stared down at his desk when asked about the widespread sale of fugu liver.

“Officially, you can never eat it here,” Mr. Kinashi said. “Well, it’s not that you can’t eat it, but, no, you can’t eat it. That’s the only answer I can give you.”

The Twilight of Expertise (part 13: ICU doctors)

Seth Roberts2 Comments

The other shoe drops. A year ago Atul Gawande wrote in The New Yorker about the Apgar score, a low-tech measurement of newborn viability that led to vast improvements in obstetrics. That’s the “how to improve?” side of things. Now Gawande has written about something equally simple and powerful on the “here’s how to improve” side of medicine: the use of checklists to improve ICU treatment. The first article was called “The Score”; this one is called “The Checklist”.

Checklists are the idea of Peter Pronovost, an ICU doctor at Johns Hopkins Hospital. His first checklist, in 2001, was designed to prevent infections on tubes inserted into patients. Nurses made sure that doctors followed the checklist. It’s like the Ten Commandments: the top and bottom getting together to improve the behavior of people in the middle. Checklists involved the empowerment of nurses (bottom) by hospital administrators (top) to improve the performance of doctors (middle). No coincidence, I’m sure, that the Apgar score also involved female empowerment: Virginia Apgar was one of the first powerful women in medicine.

Pronovost told Gawande:

The tasks of medical science fall into three buckets. One is understanding disease biology. One is finding effective therapies. And one is insuring those therapies are delivered effectively. That third bucket has been almost totally ignored by research funders, government, and academia. It’s viewed as the art of medicine. That’s a mistake, a huge mistake. And from a taxpayer’s perspective it’s outrageous.

Not to mention a sick person’s perspective. I completely agree. Several years ago I heard an industrial designer give a talk to an interface design group. He said that new high-tech products go through three stages: (a) used only by gadgeteers and professional engineers (e.g., the first home computers); (b) used by experts (e.g., billing software for lawyers); and (c) mass market (e.g., cell phones). The discipline of engineering, he said, was good at designing for the first two stages but not the third.

The similarities suggest a common explanation. I think one reason goes back to Veblen: It is low status to do useful work. It may also have to do with male dominance of medical research and engineering. When balancing status versus usefulness, men may weigh status more highly.

More innovation in the delivery of medicine: house calls. No kidding. More about Peter Pronovost.

The Twilight of Expertise (part 12: Super Crunchers)

Seth Roberts4 Comments

Ian Ayres’ interesting new book, Super Crunchers, has a chapter about expert prediction versus predictions from math models. Almost always, the math models do better than the experts. I learned about this in graduate school when I read stuff by Paul Meehl, a psychology professor who compared the predictions of clinicians and regression equations in the 1950s. The idea has gathered strength since then and now the persons in some jobs — such as loan officers — are required to follow an algorithm for making decisions. Their expertise is ignored. Obviously they no longer derive as much self-worth from their job, Ayres points out.

It’s like the beginning of agriculture. Lots has been written about the physical problems caused by the change to agriculture. Stature decreased, tooth decay increased, and so on. I’ve never read about the mental problems it must have caused. I can only speculate, of course, but here’s an possible example: Hunters derived self-worth from bringing meat to their families. Taking that away caused problems. (Watching Once Were Warriors, a terrific movie, should make this more plausible.)

I have never read anything about how to reintroduce into everyday jobs crucial mental elements that hunting had and farming lacked. Nutrition education, vitamin supplements, dietary fortification, and other nutrition programs push us toward a pre-agricultural diet, which was far more diverse and better balanced. There is no similar set of things that move us closer to pre-agricultural ways of making a living. My self-experimental research is all about the value stuff that ancient life had but modern life lacks — such as seeing lots of faces in the morning — but I have never figured out how to simulate elements of hunting, beyond being on one’s feet a lot.

The Twilight of Expertise (part 11: journalists)

Seth Roberts2 Comments

Philip Weiss has written an excellent (as usual) article about Matt Drudge.

“Matt Drudge is just about the most powerful journalist in America,” said Pat Buchanan.

And he’s self-employed. He started way down:

This is an incredibly lonely kid, [said a friend]. He doesn’t have a sister, his mother is in and out of hospitals [diagnosed with schizophrenia], the father was beside himself. In high school they treated him like shit. He was starting to lose his hair in high school; think what that does to a kid.

The Twilight of Expertise (part 10: book reviewers)

Seth RobertsLeave a comment

According to Publisher’s Weekly, a new program at amazon.com called

Amazon Vine rewards the site’s elite reviewers by giving them access to advance copies. According to a representative at Amazon, invitations have gone out to the site’s “top reviewers,” deemed so by their review rankings, to become Vine Voices.

I once read about a Los Angeles catering business that wasn’t doing so well until they doubled their prices. This is the opposite of that.

From Seth Godin:

When the Times switched from 10 books on the Hardcover [Best Sellers] list, they created a list of 15 Hardcover [Best Sellers] and a list of 5 Advice, How To and Miscellaneous [Best Sellers]. I wrote in and asked the editor why they only had 5 titles on this list and 15 on the others. She wrote back and said,

“Because we don’t want people to read those books.”

Pride goeth before a fall.

The Twilight of Expertise (part 9: clinical trials again)

Seth Roberts2 Comments

An article in this week’s BMJ about problems with clinical trials makes some of the points I made in a recent post. The article is based on a London conference held last week. In my post, I said the evaluation of the Shangri-La Diet going on at the SLD forums was in many ways better than a clinical trial.

At the conference, a speaker complained that

key groups of participants were often excluded from clinical studies

I pointed out that anyone could post at the SLD forums.

Doug Altman, professor of statistics in medicine at Oxford University, said that the presentation of statistical results of clinical trials “lacked transparency and precluded any further analysis.”

I said that the forums are more transparent.

Paul Glasziou, director of the Centre for Evidence Based Medicine at Oxford University, warned that many clinical trials described treatments that were difficult to replicate in normal clinical settings.

I said that the forums were more realistic — meaning that the treatments being tested were closer to what actually could happen.

The Twilight of Expertise (part 8: spiritual experts)

Seth Roberts2 Comments

“Religion is extremely important to the Tibetans,” says Wikipedia, but what does that mean? The Tibetan Buddhism entry is no help. Last night at dinner, however, I did learn what it means, at least in part. Tibetans spend a vast amount of time on religious observances — what the observer (Bryan Ng, a Berkeley engineer) called a “religion tax.” One example was a well-observed month-long annual religious festival. Another was a sensationally slow method of travel: Take a step or two, bow down, lie down on the ground, get up, take another step, bow down, and so on. This method is used to cover long distances, such as 20 miles or more. The extremely devout do this along highways.

The Chinese government wants to reduce the influence of religion, he said. Goods imported into Tibet from China via the new railway should increase commerce, for example. The power of the Chinese government makes it likely they will succeed.