The Best Blogs Are Very Good

From It’s Raining Noodles:

Before the next batch of snapshots and corresponding captions [on a screen at a fast food restaurant in Singapore] were revealed, a simple message against a plain background read, “To all the great and wonderful mothers out there…” as a sort of enigmatic prelude to the marvels to come, and because I am a cynical bitch with a very gloomy worldview, I asked out loud, “What about the other mothers?” To which Maria put on an affected frown as if she’d been hurt that her loved one was being left out by the onscreen message, and gleefully responded, “What about MY mom?”

More. Not to mention this:

The past three hours can only be adequately described by the word SIGH. Thinking that I was taking a step towards improving my relationship with God, no this is not a long preachy thing so please stay with me, I agreed to sit through a session thingy with a very Christian Mrs. In-Law and a very reluctant Neptune. . . . The sharing session was led by a heterosexual couple (why I am highlighting the heterosexual part will become evident later), and it was interesting because they had a different take on the religion from most that I’ve heard. In many ways it felt like a lecture in literature class, and for the most part I enjoyed it because of the different perspectives on the same ol’ concepts. . . . The only revelation for me, though, arrived when the female half of the heterosexual couple went on to preach that God gives up on people who insist on pursuing sin, such as idolatry and YES YOU GUESSED IT, homosexuality. She was all, “YES, HOMOSEXUALITY IS WRONG! God has given up on people like that.”

And I realised in that moment that God had probably just punished me by making me sit through three hours of this thingy thinking that MAYBE I had a shot at heaven when actually? ACTUALLY? NO CHANCE AT ALL. God has abandoned me to begin with right from the start. I’m damned forever. So I looked at Neptune and said, “God has given up on us!” and I was very sad. I find it very difficult to wrap my head around exactly why I deserve to go straight to hell when all I’ve done is fall in love with another person.

Suppose You Write the Times to Fix an Error (part 2)

The Roberts-Schwartz correspondence continued. I replied to Schwartz:

“Dining establishments”? [His previous email stated: “Four restaurants simply cannot represent the variety of dining establishments in New York City”] I thought the survey was about sushi restaurants. Places where raw fish is available.

Quite apart from that, I am sorry to see such a fundamental error perpetuated in a science section. If you don’t believe me that the teenagers’ survey was far better than you said, you might consult a friend of mine, Andrew Gelman, a professor of statistics at Columbia.

John Tukey — the most influential statistician of the last half of the 20th century — really did say that a well-chosen of sample of 3 was worthwhile when it came to learning about sexual behavior. Which varies even more widely than sushi restaurants. A sample of 4 is better than a sample of 3.

Schwartz replied:

The survey included 4 restaurants and 10 stores.

The girls would not disclose the names of any of the restaurants, and only gave me the name of one store whose samples were not mislabeled. Their restaurants and stores might have been chosen with exquisite care and scientific validity, but without proof of that I could not say it in the article.

I wrote:

I realize the NY Times has an “answer every letter” policy and I am a little sorry to subject you to it. Except that this was a huge goof and you caused your subjects damage by vastly undervaluing their work. Yes, I knew the survey included 4 restaurants and 10 stores. That was clear.

As a reader I had no need to know the names of the places; I realized the girls were trying to reach broad conclusions. They were right not to give you the names because to do so might have obscured the larger point. It was on your side that the big failing occurred, as far as I can tell. Did you ask the girls about their sampling method? That was crucial info. Apparently The Times doesn’t correct errors of omission but that was a major error in your article: That info (how they sampled) wasn’t included.

He replied:

I could have been more clear on the subject of sample size, but I did not commit an error. Neither do my editors. That is why they asked me to write a letter to you instead of writing up a correction.

I don’t feel I have been “subjected to” anything, or that this is some kind of punishment. This is an interesting collision between the precise standards of someone with deep grounding in social science and statistical proof and someone who tries to write intelligible stories about science for a daily newspaper and a general interest audience. But I am not sorry that you wrote to me, even a little sorry.

i wrote:

“I did not commit an error.” Huh? What am I missing? Your article had two big errors:

1. An error of commission. You stated the study should be not taken seriously because the sample size was too small. For most purposes, especially those of NY Times readers, the sample size was large enough.

2. An error of omission. You failed to describe the sampling protocol — how those 10 stores and 4 restaurants were chosen. This was crucial info for knowing to what population the results should be generalized.

If you could explain why these aren’t errors, that would be a learning experience.

Did you ask the girls how they sampled?

His full reply:

We’re not getting anywhere here.

Not so. After complaining he didn’t have “proof” that the teenagers used a good sampling method, he won’t say if he asked them about their sampling method. That’s revealing.

Something similar happened with a surgeon I was referred to, Dr. Eileen Consorti, in Berkeley. I have a tiny hernia that I cannot detect but one day my primary-care doctor did. He referred me to Dr. Consorti, a general surgeon. She said I should have surgery for it. Why? I asked. Because it could get worse, she said. Eventually I asked: Why do you think it’s better to have surgery than not? Surgery is dangerous. (Not to mention expensive and time-consuming.) She said there were clinical trials that showed this. Just use google, you’ll find them, she said. I tried to find them. I looked and looked but failed to find any relevant evidence. My mom, who does medical searching for a living, was unable to find any completed clinical trials. One was in progress (which implied the answer to my question wasn’t known). I spoke to Dr. Consorti again. I can’t find any studies, I said, nor can my mom. Okay, we’ll find some and copy them for you, she said, you can come by the office and pick them up. She sounded completely sure the studies existed. I waited. Nothing from Dr. Consorti’s office. After a few weeks, I phoned her office and left a message. No reply. I waited a month, phoned again, and left another message. No reply.

More. In spite of Dr. Consorti’s statement in the comments (see below) that “I will call you once I clear my desk and do my own literature search,” one year later (August 2009) I haven’t heard from her.

Science in Action: Why Did I Sleep So Well? (part 9)

From Nassim Taleb’s web notebook:

I was going to have dinner with Seth Roberts in San Francisco. So, out of curiosity, I tried his diet [ clipping my nose and consuming two large tablespoons of flaxseed oil ] . . . When someone who observed me with a noseclip asked: “what are you doing?” , I gave my answer “trying to be healthier.” It elicited a smile: “Why don’t you dance outside on one leg for ten minutes? That too may work very well.”

Strange strange coincidence.

Why Did I Sleep So Well? directory.

Suppose You Write the Times to Fix an Error (part 1)

Recently the New York Times published a fascinating article by John Schwartz in the science section about how two teenagers discovered that a lot of raw fish sold in New York is mislabeled. Unfortunately, the article contained two big mistakes: 1. The teenagers’ results were dismissed as unconvincing because the sample size (10 stores and 4 sushi restaurants) was, according to Schwartz, too small. For many purposes the sample was large enough, if their sampling method was good. 2. The sampling method wasn’t described. Without knowing how the stores and restaurants were chosen, it’s impossible to know to what population the results apply. This was like reviewing a car and not saying the price.

In an email to the Times I pointed out the first mistake:

Your article titled “Fish Tale Has DNA Hook” by John Schwartz, which appeared in your August 22, 2008 issue, has two serious errors:

1. The article states: “The sample size is too small to serve as an indictment of all New York fishmongers and restaurateurs.” To whom the results apply — whom they “indict” — depends on the sampling method used — how the teenagers decided what businesses to check. Sample size has almost nothing to do with it. This was the statistician John Tukey’s complaint about the Kinsey Report. The samples were large but the sampling method was terrible — so it didn’t matter that the samples were large.

2. The article states: “the results are unlikely to be a mere statistical fluke.” It’s unclear what this means. In particular, I have no idea what it would mean that the results are “a mere statistical fluke.” The error rate of the lab where the teenagers sent the fish to be identified is probably very low.

In retrospect the second error is “serious” only if incomprehensibility is serious. Maybe not. I should have pointed out the failure to describe the sampling protocol) but didn’t.

I got the following reply from Schwartz:

Thank you for your note about my article, “Fish Tale Has DNA Hook,” which appeared in the newspaper on Friday. You state that the story misstated the importance of sampling size as “an indictment of all New York fishmongers and restaurateurs.” Although you are certainly correct in stating that poor methodology can undercut work performed using even the largest samples, it is also ill advised to try to establish broad conclusions from a very small sample. The fact that mislabeling occurred one in four pieces of seafood from 14 restaurants and shops in no way allows us to conclude that 25 percent of fish sold in New York or in the United States is mislabeled. And that is all I was trying to say with the reference to sample size was that while the girls’ experiment shows that some mislabeling has occurred, their work cannot say how much of it goes on or whether any given restaurant or shop is mislabeling its products. Similarly, when I wrote that it is unlikely the findings are a “statistical fluke,” I merely meant that while it is possible that Kate and Louisa found the only 8 restaurants and shops in New York City that mislabel their products, that is not likely, and so the possibility that the practice is widespread should not be discounted. And, of course, I hope you can forgive the pun.

Thanks again for taking the time read the article and respond to it, and I hope that you will find more to like in other stories that I write.

Uh-oh. The email was as mistaken as the article, although it did clear up what “statistical fluke” meant. I wrote again:

Thanks for your reply. I’m sorry to say that you still have things more or less completely wrong.

“Their work cannot say how much of it goes on or whether any given restaurant or shop is mislabeling its products.” Wrong. [Except for the obvious point that the survey does not supply info about particular places.] I don’t know what sampling protocol they used — how they chose the restaurants and fish sellers. (This is another big problem with your article, that you didn’t state how they sampled.) Maybe they used a really good sampling protocol, one that gave each restaurant and fish seller an equal chance of being in the sample. If so, then their work can indeed “say how much [mislabeling] goes on.” They can give an estimate and put confidence intervals around that estimate. Just like the Gallup poll does.

Somewhere you got the idea that big samples are a lot better than small ones. Sometimes you do need a big sample — if you want to predict the outcome of a close election, for example. But for many things you don’t need a big sample to answer the big questions. And this is one of those cases. There is no need to know with any precision how much mislabeling goes on. If it’s above 50%, it’s a major scandal, if it’s 10-50% it’s a minor scandal, if it’s less than 10%, it’s not a scandal at all. And the study you described in your article probably puts the estimate firmly in the minor scandal category. In contrast to your “it’s cute but doesn’t really tell us anything” conclusion quite the opposite is probably true (if their sampling procedure was good): It probably tells us most of what we want to know. You’re making the same mistake Alfred Kinsey made: He thought a big sample was wonderful. As John Tukey told him, he was completely wrong. Tukey said he’d rather have a sample of 3, well-chosen.

Thanks for explaining what you meant by “statistical fluke.” You may not realize you are breaking new ground here. Scientists wonder all the time if their results are “a statistical fluke.” What they mean by this is that they’ve done an experiment and have two groups, A (treated) and B (untreated) and wonder if the measured difference between them — there is always some difference — could be due to chance, that is, is a statistical fluke. In your example of the mislabeled fish there are not two groups — this is why your usage is mysterious. I have never seen the phrase used the way you used it. And I think that the readers of the Times already realized, without your saying so, that it is exceptionally unlikely that these were the only fish sellers in New York that mislabeled fish.

Schwartz replied:

I understand your points, and certainly see the difference between a small-but-helpful sample and a large-but-useless sample. but four restaurants simply cannot represent the variety of dining establishments in New York City. Four restaurants, ten markets.

I also realize that you must think I am thickheaded to keep at this, but I will certainly keep in mind your points in the future and will try not make facile references to small and large samples when the principles are, as you state, more complicated than that.

To be continued. My original post about this article.

Science in Action: Why Did I Sleep So Well? (part 8)

My recent experience suggests that if I stand on one foot until it becomes slightly difficult about four times/day I will sleep much better. Two days ago I measured how long those four bouts of one-foot standing actually were: 6.2 (left foot), 4.3 (right), 4.8 (left), and 5.2 (right) minutes. The median is 5.0 minutes. When I started doing this, about two weeks ago, each bout was about 2 minutes.

It doesn’t seem to matter when I do them. Now I do two in the morning and two in the evening. Fits perfectly with a subway commute. You’ll want to be forced to stand.

In the evening I have a pleasant sense of anticipation: I will fall asleep and wake up feeling really good. I have never before felt this way. I have slept this well before, when I stood 9 or 10 hours/day. The sheer difficulty and all-consumingness of doing that, I now realize, got in the way of anticipating the benefits.

Something else curious is that one-foot standing leaves no mark — I can’t tell at 3 pm how many bouts I’ve done so far just by noticing how I feel. Unlike water or calorie consumption: If I don’t drink anything I’ll get thirsty. If I don’t eat anything I’ll get hungry. But if I don’t get enough of this particular byproduct of exercise I’ll never notice.

Suppose Your Book Gets a Great Review in the Times

Few books, including Lolita (“highbrow pornography”), get great reviews in the New York Times. One that did is Ammon Shea’s Reading the OED: One Man, One Year, 21,730 Pages. According to Nicholson Baker, the reviewer, “Shea has walked the wildwood of our gnarled, ancient speech and returned singing incomprehensible sounds in a language that turns out to be our own.” Someday — 20 years from now? — every review in the Times archives will be retrospectively assigned an Entertainment-Weekly-style grade by computer analysis and Baker’s review will be determined to have given an A to Shea’s book. I interviewed Shea about the experience.

Few of us will ever get such a positive review in the NY Times , so we must live vicariously. What were the effects on you of Baker’s review?

I have a sneaking suspicion that he liked the book more than I did, which is fine by me. I’m an enormous fan of both his writing and his perspective on things, and there is no one who I would rather have had read it. It did not change my feeling on who I am or what might lie in store for me in the future, but it did make me feel deeply and improbably happy.

What were the effects on your editor and publisher?

My editor [Marion Lizzi, who also edited The Shangri-La Diet] says she is quite happy with it as well, and I see no reason to disbelieve her.

What was the effect on sales?

I don’t know what the exact figures were, although I understand that they were significantly higher after the review came out. I understand that the publisher is preparing another printing, which I suppose is to be credited at least somewhat to the effects of the review.

Did any friends/family contact you about the review?

Some of them did call or write – but both my family and my circle of friends are fairly small, so there was not so much hullabaloo.

How long did it take for the effect of the review to wear off?

It hasn’t worn off in some ways – I’m still delighted that Baker enjoyed reading the book. However, in some other ways I’d say as soon as I began to seriously think about writing the next book that the incipient terror of that process nudged the residual celebratory feelings of the review somewhat to the side.

Earlier interview with Shea.

Everything I Know I Learned from Japanese Curry Instructions

I got this in a Japanese supermarket:

back of food mix box

Translation:

How to make soup curry:

Ingredients-
1 packet of soup mix
1 packet of flavorful oil
1 packet of spicy flavoring
80g (3 oz.) of chicken thigh meat cut into bite sized pieces
1/4 medium sized carrot
1/2 medium sized potato (cut in half)
400ml water

1. Boil water in a small pan. Â Add chicken, potato, and carrot, cook until vegetables become soft, about 20 minutes on med-low heat.
2. Turn off the heat, add the soup mix and mix thoroughly, turn the heat back on and cook a little longer until the flavor penetrates the meat and vegetables.
3. Pour the flavorful oil onto a plate and pour the finished curry on top.
4. Add a desired amount of the spice flavoring.

the spice flavoring is fairly spicy, so please use caution when adding
please cook the chicken thoroughly before adding the soup mix
-to make a double portion, double the meat and vegetables, and increase water to 700ml.
-the black things in the soup are basil

To make a dish like the picture on the box: Add sauteed japanese eggplant, shimeji mushrooms, green peppers, and hard boiled egg to the dish. Use boned chicken meat.

How to eat: Using a spoon, scoop rice and add a small amount of curry to the spoon. Â Please keep the curry and rice in separate dishes to prevent the rice from getting soggy.

Caution: Please use the entire contents of the packets after opening. Â Cannot be preserved for later use.

I have bolded the interesting parts: 1. The use of please. 2. The explanation (“the spice flavoring is fairly spicy”). You won’t find them in the instructions on most American products. I became aware of this aspect of Japanese life when I read T. R. Reid’s wonderful book Confucius Lives Next Door: What Living in the East Teaches Us about Living in the West, which was based on six years Reid spent in Japan as a correspondent for the Washington Post. At one point Reid quoted from a sign in a park. The sign had a list of prohibitions: No littering, no music, and so on. But instead of saying, as an American sign would, “no littering”, the sign said something like: “So that others can enjoy the beauty of the park, please put your litter in the proper receptacle.”

A few years ago I taught a class called Psychology and the Real World in which students did some sort of off-campus work of their choosing. (An example of my teaching philosophy.) One student volunteered in a hospital. One day he told a story about being treated rudely by a nurse. I said, yeah, we live in a pretty rude culture. Japan is different, I said, and told the class about Reid’s Japanese park signs.

My student was impressed. He had a part-time job monitoring parking in front of a San Francisco hotel. People would often try to park in an area that needed to be kept clear and it was his job to get them to move. His method — pre-Reid — had been to go over to the offending car and say “sorry, you can’t park here.” Post-Reid, he was elaborately polite: “Please forgive me for disturbing you, but we need to keep this area clear so that taxis can pick up and drop off passengers. I’m sorry for inconveniencing you, but would you be kind enough to move your car?” Something like that. Pre-Reid, about half the time the driver would argue or cause some sort of difficulty. Post-Reid, there were no problems.

Thanks to Pearl Alexander.

Science in Action: Why Did I Sleep So Well? (part 7)

I’ve continued to sleep extremely well. I’m sure there’s something to this. I’m almost sure it’s because of the one-legged standing.

Here are some technical details. I usually do four bouts of one-legged standing, two in the morning and two later. During each bout I stand on one leg, pulling my other leg up behind me. Sometimes I touch something to balance myself. Usually I watch or read something at the same time. Each bout lasts until it’s hard to continue — until it becomes slightly painful. At first the bouts lasted about two minutes, now they last about four minutes. I enjoy it more when I time it with a stopwatch.

I haven’t yet systematically varied the number of bouts but I suspect one is too few to get the full effect and four is plenty. I’m still trying different ways of arranging them throughout the day. Doing all four at once is too tiring — it takes too long to recover. Maybe it’s best to do two whenever’s convenient during the day and then do two more in the evening when it’s okay to be tired.

Directory.

The Emperor’s New Clothes: Meta-Analysis

In an editorial about the effect of vitamin-mineral supplements in the prestigious American Journal of Clnicial Nutrition, the author, Donald McCormick, a professor of nutrition at Emory University, writes:

This study is a meta-analysis of randomized controlled trials that were previously reported. Of 2311 trials identified, only 16 met the inclusion criteria.

That’s throwing away a lot of data! Maybe, just maybe, something could be learned from other 2295 randomized controlled trials?

Evidence snobs.