[Insight-developers] SCIENTIFIC PUBLISHING: The End of the "Review Nightmare" (Part 2)

[Luis Ibanez]

Editorial
"What are journals for?"


Journal of Biology 2009, 8:1doi:10.1186/jbiol111

The electronic version of this article is the complete one and can be
found online at: http://jbiol.com/content/8/1/1
Published: 27 January 2009

© 2009 BioMed Central Ltd
Editorial


Here is a truism. The function of a scientific journal is to disseminate
the results of research. It also of course has a responsibility to
ensure as far as possible that the results reported are valid. But it
would seem from the comments of many biologists – sometimes in print
[1-3] – that the policing function of journals (especially but not
exclusively the high-profile journals) is in danger of overwhelming
their primary function as publishers.

The chief source of frustration seems to be with increasingly frequent,
time-consuming and sometimes iterative re-review which, it is commonly
felt, makes little difference to the eventual validity or quality of the
paper.

For that reason, Journal of Biology will embark, starting now, on an
experimental policy of allowing authors to opt out of re-review. This is
in keeping with the premise of open-access publishing, of which our
publisher, BioMed Central, was the pioneer, that freedom from the
practical constraints imposed by paper and printing should remove all
needless barriers to access to research results.

The decision to launch the experiment was taken after consultation with
members of our Editorial Board, who were on balance (by about four to
one), and often emphatically, in favor of it. Box 1 contains a highly
selective sample of the raw data. Even those members of the Board who
had misgivings (on which more below) often expressed themselves with
some passion about difficulties with reviewers: one of the most
impassioned of the six we have quoted in fact felt that the need for
rigorous review should nonetheless transcend the wish to escape the
horrors of current reviewing practice. – Not that we are proposing to
abandon review.


Briefly, the proposition is as follows.

All papers submitted to Journal of Biology will first be screened by a
member of our Editorial Board, or an appropriate expert of equivalent
sagacity, for suitability in principle for the journal, so that referees
are asked to judge only whether the data adequately support the
conclusions: this policy is already in operation.

If the conclusions are not adequately supported by the data the paper
will be rejected.

If the paper is deemed perfect as it stands, it will be published
without delay (and with an accompanying commentary, as is our normal
practice).

If one or more of our referees has suggestions or demands for revision,
including the addition of data, authors will be requested to respond to
the referees and revise the manuscript, and will be asked at that point
to say whether they wish the referees to look again. (It is probably
worth saying that authors will not be invited to select which referees
see their paper again: if they opt for re-review the paper will be seen
by all of the original referees – who will also see one another's
comments and the responses of the authors: this is our normal procedure
and we find it useful, for reasons I probably need not spell out.)

Where authors opt out of re-review their responses and revised
manuscript will be carefully scrutinized by the editorial staff, and if
it seems clear to us that substantive issues have not been addressed the
manuscript may be rejected. Otherwise it will be published, with an
accompanying commentary whose author will have access to the referees'
reports and the authors' response.
The case against

There are obvious problems with this proposition. People may be
reluctant to referee papers if they think there is a risk that their
careful and important comments may be disregarded: about one in five of
our Editorial Board members indicated that this would apply to them.
Losing 20% of the possible referees for a paper would be a very serious
problem because it is hard enough to find good referees to look at
papers quickly as it is. On the other hand, if a substantial number of
authors opted out of re-review, this would release more time for the
remaining 80%.

Seriously flawed papers might find their way into the journal. This of
course happens anyway from time to time with most, if not all journals,
and probably especially with high-profile journals aspiring to publish
particularly newsworthy results. Referees do not always see what turn
out to be serious flaws; or, in the case of iterative re-review where
the authors cannot come to terms with referees, reviewers sometimes just
give up. It is not clear that the end result would be any worse under
re-review opt-out.

These are the two most egregious potential problems. If I tried to cover
all eventualities and contingencies, possible modifications to the
present plan, and other strategies for addressing refereeing problems
(see for example Biology Direct, also published by BioMed Central:
http://www.biology-direct.com/info/about/ webcite), this editorial would
exceed any reasonable expectation of its readers' patience. A fuller
discussion can be found in question-and-answer form on the Blog site:
http://blogs.openaccesscentral.com/blogs/bmcblog/entry/q_a_the_re_review
webcite; but I will answer here the most important question: since there
are clearly other possible ways for editors to resolve the problem of
unfairness and iteration in the review process, why take this arguably
risky step?
Whose risk?

Just as it is the job of journal editors to promote the dissemination of
research results rather than to obstruct it, it is the author who is in
the end accountable for the quality and validity of the paper that is
published. Editors are responsible for finding, and referees for giving,
thoughtful and unbiased advice to help the author to ensure that the
conclusions of the paper are indeed valid and that the text achieves
reasonable standards of scholarliness and intelligibility. (Earlier work
in the field should be properly acknowledged and the paper should be
readable without undue pain.)

If there are no flaws in a paper at the time of submission that should
clearly prohibit publication, it seems to us reasonable to leave the
authors to decide how much more help they need to make sure that the
published paper will not damage their reputation (see Robert Horvitz's
remark in Box 1). It is often not clear that the revisions demanded of
authors, in particular by the more selective journals, substantially
increase their value. When they do, it seems that it is sometimes by
insisting on additional data that arguably justify a second paper (see
also [4]), which may have more to do with enhancing the reputation of
the journal than with helping the authors to communicate their research.

Moreover there are strong arguments against a blanket insistence on
absolutely rigorous validation of data (some of them are rehearsed for
genome-scale analysis in a comment from Tim Hughes in this issue of
Journal of Biology [2]). This applies especially in the case of
genuinely novel research, which is quite likely to raise issues that
cannot immediately be resolved. Tim Hunt will not mind if I cite him as
having remarked that the paper reporting the discovery of cyclin, the
cell-cycle regulatory protein for which he later won (with Paul Nurse
and Leland Hartwell) the Nobel Prize in Physiology or Medicine, was
messy and flawed in a number of ways. Which is not to say that journals
should be recklessly publishing all flawed and messy papers with
striking claims in the hope that they will win Nobel prizes. There is no
escape from hard judgement calls, for referees and for editors, and we
shall not always get it right. The question is whether iterative
reviewing materially affects how often we get it wrong.

Journals also have a responsibility to their readers. If they should be
willing to publish results that are not definitive, they should also
provide a broader perspective for readers who may not be equipped to
judge the merit of the paper for themselves. We already publish
commentaries with our research articles, and it is common practice for
general journals to publish commentaries that may draw attention to the
limitations of papers making important claims. (Of course such
commentaries may also serve to explain for non-specialists how important
the paper is.) Under our experimental re-review opt-out scheme, the
commentaries will become an important mechanism for ensuring that
readers are alerted to issues that have not been resolved before
publication. It is not clear how often these will prove damaging in the
end. (One of the reviewers of the first cyclin paper apparently said it
was "...wild speculation, based on faulty logic".)
What else is new?

Journal of Biology last published an editorial about itself on the
occasion of its fifth birthday [5]. What is new since then? Let me start
with what is not new. Our experiment with peer review continues a fine
tradition of promoting the interests of authors and to which indeed some
have felt moved to testify (see [5]). A second good thing that hasn't
changed is the strong link with our sister journals Genome Biology, BMC
Biology and the more specialized journals of the BMC series, any of
which may provide a painless route to publication of papers that after
review don't quite meet the criteria for publication in Journal of Biology.

Meanwhile our major new development is a greatly expanded range of
commissioned content. You will find in this issue a brief explanation
from James Ferrell, in question-and-answer format, of what systems
biology is [6]: this is the first in a series in the same format on
topics of current interest or fundamental importance on which readers
may feel bemused. As well as our minireviews exploring topics arising
from papers published in our sister journals (see for example Whited and
Tabin on limb regeneration in this issue [7]), we shall be publishing
full reviews: in this issue, by Peter Moore on the cell's most important
ribozyme, the ribosome [8]. We have occasionally in the past published
comment: in this issue Tim Hughes sounds off, in a restrained way, about
the futility of 'validation' in genomics papers [2].

Although I am new to Journal of Biology as Editor, Penelope Webb, in
whose editorial care it has been for the past two years, remains as
Consulting Editor, and Elizabeth Moylan as Deputy Editor, to help uphold
the journal's honorable tradition of friendliness and courtesy to
authors. New to the journal since its fifth birthday are Sandra Le
(Senior Editor), who has built the minireview series, and Elizabeth
Benson (Assistant Editor) who has just joined us. Penelope and Elizabeth
Moylan in particular embody the blood ties to our sibling journals in
the BMC series.

Miranda Robertson, Editor