New refereeing

In a recent article, the New York Times describes the new approach to refereeing adopted by the Shakespeare Quarterly, namely to turn the refereeing process into a public on-line discussion.

“Mixing traditional and new methods, the journal posted online four essays not yet accepted for publication, and a core group of experts — what Ms. Rowe called “our crowd sourcing” — were invited to post their signed comments on the Web site MediaCommons, a scholarly digital network. Others could add their thoughts as well, after registering with their own names. In the end 41 people made more than 350 comments, many of which elicited responses from the authors. The revised essays were then reviewed by the quarterly’s editors, who made the final decision to include them in the printed journal.”

I find the move quite interesting, as well as worth repeating within our community. For one thing, nowadays, most papers are already available on arXiv. And sometimes the journal they are submitted to is also indicated on the abstract page. So introducing a (possibly monitored) forum of opinions about the paper [on arXiv or on the journal own webpage] would not be a major difficulty. And could come as a complement to the usual refereeing process. As posted in an earlier post (?), I actually am in favour of removing anonymity for the usual referees as well, possibly on a voluntary basis, as I do not see a huge risk of a bias coming from this move, while it should speed up the refereeing process. The double-blind policy adopted by JASA and JCGS, among others, does not strike me as efficient.

2 Responses to “New refereeing”

  1. […] be detected by the community. (I am not advocating the end of academic journals, far from it!, but an evolution towards a wider range of evaluations via Internet discussions, as for the DREAM paper recently.) […]

  2. X:

    Maybe most of _your_ papers are already on Arxiv. But I don’t think that’s standard in statistics. When it comes to applied statistics or social science, I associate Arxiv with goofy papers by physicists that aren’t usually so closely connected to serious research. I’m sure things are different in pure math and even in some areas of statistical computation.

    I agree that the whole double-blind thing is a joke. People seem much more conserved about hurt feelings of authors of bad papers than about the progress of science.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s