Double-Blind Bind

An editorial on peer review was published today in Nature, and the text is reprinted in their Peer-to-Peer blog. The editors are soliciting comments and suggestions, and so far the count is up to 25 48.

Working double-blind

Should there be author anonymity in peer review?

Double-blind peer review, in which both authors and referees are anonymous, is apparently much revered, if not much practised. The Publishing Research Consortium (PRC) has assessed attitudes towards peer review among 3,000 academics in an international survey across the sciences and humanities. ...71% have confidence in double-blind peer review and 56% prefer it to other forms of review. Support is highest with those who have experienced it (the humanities and social sciences) or where it is perceived to do the most good (among female authors). The least enthusiastic group is editors. So is it time for editors, and those at Nature in particular, to reconsider their position?

If referees know the authors' identities, it may leave the latter vulnerable to biases about them or their previous work, their gender, their nationality or their being new to an area of research. But the PRC survey supports the contention of Nature and others that identifying authors stimulates referees to ask appropriate questions (for example, differentiating between a muddy technical explanation and poor experimental technique). Knowing author identities also makes it easier to compare the new manuscript with the authors' previously published work, to ensure that a true advance is being reported. And knowing rather than guessing the identities of authors encourages reviewers to raise potential conflicts of interest to the editors.

The Neurocritic has been a proponent of completely open peer review, where the identity of the authors and the reviewers is known (see Anonymous Peer Review Means Never Having to Say You're Sorry and Peer Review Trial and Debate at Nature).

Commenter Nina Papavasiliou puts forth another interesting proposal:

Is double blind (or even open) review a cure-all? Not necessarily, and there are definitely good reasons to argue for/against both options (including but not limited to the reasons you mention). But here's another thought, that comes from being on NIH review panels: What if review is changed by having each reviewer identity and their comments blogged to one another before a decision on the paper is reached? That will a) stop reviewers from making silly comments (from asking for ridiculous experiments to killing the paper with a couple of unsubstantiated lines like "I see nothing new here" - as they will not be hiding behind anonymity in their reviewer group) and b) may even increase the quality of papers (as reviewers could, in real time, convince one another and reach consensus prior to a decision).

She also describes the disadvantages of being a junior investigator, one who would benefit from double-blind review.

Peer review can be more or less biased depending on field. I "live" in a field where it is indeed extremely biased but not against sex - rather there is direct bias (both by editors and also by reviewers) against junior faculty combined with a relatively free ride for senior faculty. Of course, this is anecdotal, but a study on the issue broken down by field will, I think, be far more informative.

Many junior faculty in my field have direct experience of this bias in peer review. How else would you call it when a paper with a set of data but junior authors is rejected by the journal (e.g. Nature) without review, when the IDENTICAL results (authored by very senior authors) are reviewed by the same Editor and published with much fanfare? (This has happened to me twice, and if you are interested I can substantiate with exact publications, dates etc).

CLARIFICATION: Dr. Papavasiliou doesn't think that being female is as great a disadvantage in her field as seniority. However, a recent paper showed that in another field (ecology and evolutionary biology), double-blind review benefits women to a significant extent (Budden et al., 2008 -- read a good summary here).

References

Budden A, Tregenza T, Aarssen L, Koricheva J, Leimu R, Lortie C. (2008). Women, Science and Writing. Trends in Ecology & Evolution 23:4-6.

Editorial. Working double-blind. Nature 451; 605-606; 2008.