Author: Seth Roberts

How to Find a Doctor

Seth Roberts5 Comments

David A. Pfister, a Bay Area oncologist, was named “Best of the Bay Oncologist” in 2010 by KRON-TV, according to a Yelp reviewer. He was named one of America’s “Top Doctors” by US News, based on a “peer nomination process.” The biggest doctor rating site, at least in America, is HealthGrades. A HealthGrades survey of Dr. Pfister’s patients (n = 31) asked Would you recommend Dr. Pfister to family and friends? Dr. Pfister’s score — halfway between “mostly yes” and “definitely yes” — put him close to the national average.

The “Best of Bay” comment was one of seven Yelp reviews of Dr. Pfister that filtered out (= downgraded) by Yelp’s filtering algorithm. The filtered-out reviews were much more positive than the reviews that passed the filtering process. In the five passing reviews, Dr. Pfister received an average rating of 1 out of 5, with comments to match:

He was chronically late, and had poor time-management skills. . . . This was the third and final time that he’s made me wait at least an hour past my scheduled appointment time (requiring me to leave before seeing him). [2008]

He was 30+ min late, unfriendly and unapologetic. His bedside manner is horrific and he talked me into having a procedure that ended up being painful and unnecessary. The office is completely disorganized. There are records of deceased patients out in the open in the bathroom. [2011]

When I visit his office, the only thing he wishes to discuss with me are the results of my recent labs. If it were up to him, my appointment would last 2 minutes. . . All my other doctors have told me for years I should get my care elsewhere. Typical visits consists of 2 hours waiting, 5 minutes with the doctor. [2010]

He is consistently late, as much as two hours, to his first appointments of the day. He arrives completely disheveled, hair sticking up and shirt untucked as if he was up half the night drinking. He also forgets your history and has to be reminded who you are, despite continual and regular appointments. Finally, if you ask questions he becomes very defensive and has even yelled at me for asking questions. [2009]

Which view of Dr. Pfister is more accurate, KRON-TV or Yelp? In March 2012, his license was suspended. He “admitted he has a psychiatric problem and a substance abuse problem.” The Yelp reviews that passed the filtering algorithm, with their complaints about lateness, poor grooming, and disorganization, predicted the suspension (assuming that doctors with low yelp scores are more likely to be disciplined). HealthGrades has yet to figure out there is anything unusual about Dr. Pfister. He is not listed on vitals.com.

I came across Dr. Pfister while glancing through yelp ratings of Berkeley doctors. His low rating surprised me. A yelp reviewer linked to the license suspension.

My conclusion: When looking for a doctor, check yelp. Yelp’s filtering algorithm, which emphasized the low reviews, really works. In California, you can search state records for licensing board disciplinary actions but such actions are very rare.

Thanks to Bryan Castañeda for a long conversation about detecting bad doctors. In Unaccountable (which should have been on my Best Books of 2012 list), Marty Makary says that hospitals and surgeons are in many ways unaccountable for their mistakes. Yelp is a countervailing force.

Interview with Doron Weber, Author of Immortal Bird, About What He Learned From a Hospital Tragedy

Seth RobertsLeave a comment

Immortal Bird by Doron Weber, a program director at the Sloan Foundation, is about his son, Damon, who had a rare medical condition, and his son’s heart transplant operation (cost = $500,000) at New York Presbyterian/Columbia University Medical Center. Damon died after the operation. The post-operative care was so bad his father sued. “Three years into the lawsuit, the medical director [of the hospital] claimed Damon’s post-op records couldn’t be located,” said the New York Times.

How can such tragedies be prevented? To find out, I interviewed Doron Weber by email.

SETH Let’s say someone lives in a different part of the country — Los Angeles, for instance. What would you tell them about picking doctors to do a difficult expensive operation?

DORON I believe the key step before making any major medical decision is to gather as much information as possible. In my son’s case, we talked to everyone we knew at his regular New York hospital (New York Presbyterian) for their recommendation, and then we compared that information with experts at half a dozen other hospitals in New York and across the country who had a good reputation for his operation. I had established contacts at many of these hospitals, usually through physicians or scientists who I knew, either personally or professionally. But sometimes I would just get the name of a leading doctor and call him or her cold. They didn’t always respond but often they did, especially if you could make the case sound interesting. And I found that most doctors are very decent people who will try to share their knowledge, albeit succintly. I got the best results by being polite but determined and I didn’t require a long conversation–though some physicians were truly generous with their time–because in the end, you just want to know what they would do or who they would go see if it was their son or daughter.

I also traveled with my son to meet many of these experts at places like Children’s Hospital of Philadelphia, Boston Children’s, and the Mayo Clinic in Minnesota. During my son’s long illness, I found 3-4 key advisers–medical people who I respected and trusted, who would take my calls (one was my cousin, another the friend of a friend), and who were willing to work with me as my son’s case developed. These wonderful physicians would not just give intelligent medical advice seasoned by experience but they would send me the latest medical journals and articles for any possible leads. And they would direct me to other experts. Good people tend to know other good people.

If there was one mistake I made, it was to rely too much on data and statistics–they do matter, and they worked to extend and enhance my son’s life for several years–and not to listen to my own instincts. The physician whom I consider responsible for my son’s death–and against whom I have a still-pending lawsuit–was someone whom I had a bad feeling about from the start. (See Immortal Bird for examples.) But she had a great reputation, everyone kept extolling her and her hospital had the best outcome data for my son’s operation. Also my son wanted to stay at that hospital. So I suppressed my doubts and reservations and made the correct statistical calculation but a disastrous human one.

SETH What about screening doctors by asking about their legal record? For example, “Have you ever been sued for malpractice?” If so, going down the list of cases and learning about each one. And: “Have you ever been disciplined by a medical board?”

DORON Before my son’s wrongful death, despite all my information gathering, it never occurred to me to inquire about a physician’s legal record and whether he or she had ever been sued for malpractice. Now I know better. It would be very helpful to know if, and how many times, a physician has been sued before, even if it not definitive, because many doctors and hospital insurers settle out of court with strict confidentiality rules. But at least it gives you a preliminary context. And of course there are also frivolous lawsuits but if the same doctor was charged three times for the same alleged infraction, it is worth heeding. I have been most amazed at how many people, when I tell them about my medical lawsuit, describe how they or a loved one were horribly mistreated by a physician or hospital and came close to filing a lawsuit–but they didn’t go though with it because of the stress and the long, uphill battle and the years and expense involved. (Our own lawsuit has been active for six years but is on a contingency basis because we could not have afforded it otherwise.) Almost everyone has a personal hospital horror story–if a conversation ever flags, just bring up this subject–but most people shy away from challenging the hospital and the doctors with their big reputations and deep pockets. I also found people who did not understand that they had been mistreated because it was too painful to confront and they preferred to accept the hospital’s misleading explanation. I think beyond a record of being sued, every physician should have to post a record of all patient histories, which minimally would include diagnosis, length and type of treatment, and outcome for each case. In no other field does the consumer have less information on which to base a decision, and yet in no other field are the stakes so high.

SETH Based on your experience with your son, what are the first things we should change about our health care system?

DORON For me the greatest problem with our health care system is that it is no longer about health care but about the health business. Many hospitals have been taken over by private equity firms while even the non-profits are under pressure to reduce costs at the expense of patient outcomes. So I think we have to find a way to return the patient to the center of the health care system and ensure that everything else revolves around his or her well-being. Efficiency and controlling costs matters but health care is not just another business and should not be run by business managers. I like the Mayo Clinic model where doctors are under salary so can take their time and not worry about insurance and where physicians at the same hospital consult with one another and take a more holistic, multidisciplinary approach. I also think continuity of care is absolutely critical and each patient needs one assigned physician who will take full responsibility and oversight for his/her care and be held accountable, regardless of how many specialists or other doctors the patient sees.

Never Be Alone in a Hospital

Seth Roberts6 Comments

The Health Care Blog post titled “The Empowered Patient” by Maggie Mahar exists, as far as I can tell, because much hospital care has considerable room for improvement and many mistakes are made — for example, patients are given the wrong drug. One commenter (MD as Hell) said he has worked in hospitals more than 30 years and has some advice, including

  1. Never be alone in a hospital
  2. Never go to a hospital unless you have no alternative
  3. Do not let fear motivate you to be a consumer of any part of healthcare

In the comments, several doctors expressed their dislike of the whole idea of “patient participation”. For example,

Patients manage the process. Really? I’m sure your plumber or mechanic love you and this philosophy so much they hug you when you greet them.

Plumber and mechanic errors are not the #3 cause of death in America, as Marty Makary says about medical errors.

Here is another argument against patient participation:

The huge problem that barely anyone wants to talk about is [the assumption] that patient (and family ) participation are always (or even just mostly) beneficial. This is a completely unfounded assumption. Please read Dr. Brawley’s book “How we do harm” to read 2 long and IMHO representative anecdotes of patient/family centeredness resulting in net harm. . . . Lack of patient involvement and medical errors are hardly on top of the list of pressing flaws of the US health care system . . . Profit centeredness resulting in overtreatment of the insured and undertreatment of the underinsured are the main issues.

If medical errors are the #3 cause of death in America, they are one of the most serious flaws of the US health care system. The doctors who dislike patient participation in this comment section do not propose a better way to reduce mistakes, a better way to spend the time and mental energy required by patient participation. Maybe their annoyance is a good thing. Maybe they will be so annoyed they will reduce errors in other ways.

It is bizarre that patient involvement cannot be easily dismissed. I cannot think of another profession (accountants, bus drivers, carpenters, dentists, elementary school teachers, and so on) where anyone says never be alone with them. Sure, hospital patients are highly vulnerable but that vulnerability is no secret. It could have led to a system, similar to flying (airplane passengers are highly vulnerable), with an extremely low rate of fatal error. My own experience supports patient involvement. The biggest motivation for my self-experimentation, at least at first, was my self-experimental discovery that a powerful acne medicine my dermatologist had prescribed (tetracycline, an antibiotic) was no help. My dermatologist had shown no signs of considering this a possibility. When I told him about my experiment (varying the dose of the antibiotic) and the results (no change in acne), he said, “Why did you do that?” Later a surgeon I consulted about a tiny hernia was completely misleading about the evidence for her recommendation that I have surgery for it.

Podcast: Play in new window | Download

Assorted Links

Seth Roberts3 Comments

Thanks to Alex Chernavsky and Tim Beneke.

Department of Self-Presentation: The GiveWell Mistakes Page

Seth Roberts1 Comment

The GiveWell website has a page (“Our Shortcomings”) that is a list of mistakes. A good idea, sure, what about execution?

It starts badly. Here is the stated reason for the page:

Because we are a startup organization working in areas we have little experience with, it is particularly important that we constantly recognize and learn from our shortcomings. We make this log public so as to be up front with any potential supporters about ways in which we need to improve.

The second sentence alone would have been fine.

The first item is called “overaggressive and inappropriate marketing.” I’d call it “dishonest marketing”.

I once attended a short talk, before PowerPoint, in which the speaker, Herb Terrace, a Columbia University psychology professor, put a slide in backwards. He struggled to fix it. It was funny and memorable. Maybe I should make similar mistakes on purpose, I thought. I have no idea if the GiveWell mistakes page is a reasonable summary of their mistakes. As Renata Adler pointed out, the New York Times corrects trivial mistakes and leaves major blunders uncorrected (“there are, as a rule, no genuine corrections. These departments are cosmetic”). But the GiveWell mistakes page does three things well. (a) It’s a readable summary of what they do and their goals. In contrast, I found their “About” pages unhelpful. (b) It makes them more attractive. As confessions of difficulties and problems and weaknesses usually do. (c) It draws attention to them. It is an original and thought-provoking thing to do. The next time I teach a class, should I include “mistakes I made last time I taught this class”? Maybe.

 

Assorted Links

Seth Roberts2 Comments

Thanks to Casey Manion.

Elements of Personal Science

Seth Roberts4 Comments

To do personal science well, what should you learn?

Professional scientists learn how to do science mostly in graduate school, mostly by imitation, although they might take a statistics class. Personal scientists rarely have anyone to imitate, so have more need to understand basic principles. There are five skills/dimensions that matter. Here are a few comments about each one:

1. Motivation. In conventional science, the scientist does it as part of a job and subjects are paid. Neither works here: It isn’t a job and you can’t pay yourself. My original motivation was wanting to learn how to do experiments (for my job — experimental psychologist). After I discovered how useful it could be, I started doing personal science to solve actual problems, including early awakening and overweight. On these two subjects (sleep and weight control) conventional scientists seemed to have made and be making little progress, with a few exceptions (such as Sclafani, Cabanac, and Ramirez) in the area of weight control. Here my motivation was lack of plausible alternatives. Now I now see personal science like playing the lottery, except it costs almost nothing. Most of the time nothing happens, once in a long while there is a big payoff. An example of the lottery-like payoff is that for ten years I measured my sleep, trying to figure out what was causing my early awakening. One day it suddenly got worse (when I changed my breakfast). That led me to realize many things. Another example is I measured my brain function with an arithmetic test for several years. One day it suddenly improved (due to butter).

2. Measurement. Conventional scientists almost always use already-established measures because they improve communication. In contrast, a personal scientist wants a measure that is especially sensitive to the problem (e.g., insomnia) to be solved or the question to be answered (e.g., did flaxseed oil improve my balance?). Communication is much less important. Psychologists use Likert scales (rating scales with 5 or 7 possible answers) to measure internal states but they almost always use inexperienced and unmotivated subjects. When I’ve measured internal states (e.g., mood), I have a lot of motivation and eventually have a lot of experience and find I can make much finer distinctions. Unlike conventional research, I care enormously about the convenience of the measurement. For example, it should be brief.

3. Treatment choice. You don’t want to do a lot of experiments that don’t find any effect, so you need to choose wisely the treatments you test. Scanning the internet (what has cured insomnia?) and reading scientific papers (what are standard treatments for insomnia?) hasn’t worked for me, although it’s better to try anything than to try nothing. One thing that’s worked is to test large surprising effects I hear about. An example is Tara Grant’s discovery that restricting her Vitamin D to the morning improved her sleep. Also successful is measuring the problem for a long time, in search of outliers. When the problem suddenly gets better or worse, I test whatever unusual happened just before that. For example, when I switched from oatmeal breakfast to fruit breakfast, my early awakening suddenly got worse. I started testing various breakfasts. A third successful strategy is to combine the first two strategies with evolutionary thinking, giving bonus points if the treatment I’m thinking of testing provides something present in Stone Age life but absent now. For example, this is one reason I decided to test the effect of standing a lot. Stone Age people must have been on their feet more than most of us.

4. Experimental design. The hard part is knowing how fast the treatment effect rises and falls. If it rises and falls quickly, your experiment should be very different than if it rises and falls slowly. In most cases, what I study rises and falls slowly and the best design is some variation of ABA. Do A for several days, do B for several days, do A for several days. It is much easier to do a condition for too few days than too many so I try to err on the side of too many days. The hardest lesson to learn was to realize how little I know and avoid complex designs with untested assumptions.

5. Data analysis. Statistics books and classes emphasize statistical tests, whereas in practice what matters are simple graphs (e.g., what you measure versus time). I make one or more new graphs every time I collect new data (e.g., I make a plot of my weight versus time every time I weigh myself) but rarely do t tests and the like. I’ve learned to make several graphs at different time scales (e.g., last week, last month, etc.), not just one graph.

I believe these factors combine in a multiplicative way to determine how much you learn. If any is poor, you will learn little. They provide a way of asking yourself what you’ve learned after you’ve done some personal science. For example, where did I get the idea for the treatment? Presumably, with experience, you slowly get better at each of them.

Thanks to Brian Toomey for encouraging me to write this.

Assorted Links

Seth Roberts5 Comments
  • Unusual fermented foods, such as shio koji (fermented salt, sort of)
  • David Healy talk about problems with evidence-based medicine. Example of Simpson’s paradox in suicide rates.
  • The ten worst mistakes of DSM-5. This is miserably argued. The author has two sorts of criticisms: 1. Narrow a diagnosis (e.g., autism): People who need treatment won’t get it! 2. Widen a diagnosis (e.g., depression) or add a new one (many examples): This will cause fads and over-medication! It isn’t clear how to balance the two goals (helping people get treatment, avoiding fads and over-medication) nor why the various changes being criticized will produce more bad than good. Allen Frances, the author, was chair of the committee in charge of DSM-4. He could have written: “When we wrote DSM-4, we made several mistakes . . . . The committee behind DSM-5 has not learned from our mistakes. . . .” That would have been more convincing. That the chair of the committee behind DSM-4, in spite of feeling strongly about it, cannot persuasively criticize DSM-5 speaks volumes.
  • The Lying Dutchman. “Very few social psychologists make stuff up, but he was working in a discipline where cavalier use of data was common. This is perhaps the main finding of the three Dutch academic committees which investigated his fraud. The committees found many bad practices: researchers who keep rerunning an experiment until they get the right result, who omit inconvenient data, misunderstand statistics, don’t share their data, and so on.”

A Brief History of Antibiotics

Seth Roberts7 Comments

This excellent article by Carl Zimmer gives a brief history of the development of antibiotics. It makes the usual points that the microbes within us improve our health and killing them (with antibiotics) can have bad effects. One study found that children given antibiotics had a higher risk of developing inflammatory bowel disease (IBD) later in life. Giving antibiotics to a child younger than one year was especially dangerous — the risk of IBD increased by a factor of 6.

The article makes the minor mistake of taking seriously what researchers say about number of species:

Each of us is home to several thousand [bacterial] species. . . . My own belly button, I’ve been reliably informed, contains at least 53 species.

Counting the number of species inside us is like measuring the length of the coast of England. The more closely you look (in the case of coastlines, the shorter the ruler you use), the larger the number you will arrive at. I’d be surprised if the researchers who count bacterial species adjust for this.

What I found most interesting about the article is it says nothing about fermented foods. Apparently the connection is not so obvious.