This BMJ research asked how well authors responded to criticism in emailed letters to the editor. A highly original subject, but the researchers, one of whom (Fiona Godlee) is the top BMJ editor, appear lost. They summarize the results but appear to have no idea what to learn from them, ending their paper with this:
Editors should ensure that authors take relevant criticism seriously and respond adequately to it.
Which was perfectly reasonable before any data was collected. So that’s not a good conclusion.
The real conclusion is this: The letters to the editor were far better than nothing because authors responded to their criticisms about half the time.
One time the editor of The Guardian came to my university to give a talk on the unhealthy relationship/symbiosis between politicians and the press.
At the end he concluded that the press shouldn’t suck up to politicians just so they get the stories first, and that politicans shouldn’t manipulate the press so cynically; then the floor was opened for questions. I asked him what systemic changes would he recommend, rather than just telling people to ‘be good’ at the end of a long cause-effect explanation of exactly why people aren’t being good. He irritably restated his conclusion then tried to turn the question around, asking what I would do (though I was able to give an opinion, he was the expert, not me).
Yes, the letters to the editor are better than nothing, just like the faltering professional-ethics-culture of journalists and politicians is better than nothing, but a better conclusion would have been: we should create a culture of expectation that every convincing-sounding criticism should be answered satisfactorily.
Doesn’t have to be 100% – people have lives to live; fall and break their ankle, feel to hungover to answer letters, etc – but there should be some stigma attached to ducking hard questions.
G, that’s a great story. As if the editor had had brain surgery that kept him from thinking useful thoughts. Since he ran an important paper, that’s not trivial.
Perhaps some criticisms were not answered satisfactorily because there was no satisfactory answer. Or because the criticisms were unclear. To have a sensible idea on how to improve things I’d want to look closely at what the problems were.
It is interesting how often people think in terms of conclusions they want to see result instead of methods to get to the conclusion. Dieting and weight loss is generally like that.