This semester at Tsinghua University — the most selective college in China — I taught a freshman seminar about recent psychology research. Three weeks ago I gave my students a choice of five articles from Psychological Science, all published in 2008. They were to read one of them and comment.
Mostly I try to teach appreciation but three weeks ago we focused on how articles could be improved. I have never tried to teach this, yet the students made some very good points. Here are some of their comments:
1. This article said that we believe women make better leaders when there is within-group conflict and that men make better leaders when there is between-group conflict. One student pointed out that Rwanda was a good example. After the genocide (within-group conflict), far more women were elected to office.
2. This article studied the effect of cleanliness on moral judgments. One experiment compared two groups: subjects in one group had recently washed their hands, subjects in the other group had not. Before the time when the handwashing happened, both groups saw unpleasant scenes from a movie. Students pointed out an important confounding not mentioned in the article: The two groups differed not only in handwashing but also in the time from movie to test (because handwashing took time). Perhaps subjects who washed their hands remembered the movie less well.
3. The name-letter effect is a tendency to favor outcomes (broadly defined) that involve the first letter of your name. This study involved Belgium workers. The authors found that workers were more likely to be employed by a company whose first letter matched the first letter of their name. The correlation was small but reliable. Two students pointed out that this might reflect the company’s choice of whom to hire rather than the employee’s choice of where to work. One student pointed out that the correlation might be due to name-place correlations across Belgium. Perhaps certain regions favor certain names for both people and companies. As you move closer to the French border, perhaps French names become more common among both people and companies.
In all of these cases, had I been the editor, I would have required the authors to change their article appropriately.
James Surowiecki’s The Wisdom of Crowds described cases where averages of estimates made by non-experts did very well, sometimes out-performing experts. These three examples don’t involve numerical judgments nor averaging, but they do show non-experts (freshmen) doing better than experts (journal editors and reviewers) in certain ways. Each paper was read by about eight students.
More It isn’t easy to convey how impressed I was. The comments about Rwanda and about name localization certainly deserve a letter to the editor (if Psychological Science published them). Both of them are sophisticated methodological comments. The Rwandan one says that after you write an experimental article, try to find out if real-world events support your findings. That may be a helpful lesson in many cases. The name localization one suggests that psychologists who use survey data should be learning more about how to analyze survey data. Several other times my students surprised me with how good their comments were. One was during a discussion of possible reasons for the Holocaust, another was about why women in ancient China bound their feet, a third involved proposals for Mindless-Eating-type experiments.