, , ,

Peer review is central to academic publishing; it is what enures that the work that is published has merit.  Peer-reviewed journals are the main way that academic knowledge is disseminated.  There are, however, many problems with this system, one being the cost of journal articles to the non-institutional reader.  Whilst university libraries will have subscriptions to many journals and make them available to students and staff, members of the general public have to pay a large sum (such as £30 per article) to gain access to the article without a subscription.  To attempt to remedy this situation there has been a massive increase in the number of open access journals where content is freely available online.

Many open access journals require a fee before an article is accepted for submission, whilst more require a fee for the publication of a successfully submitted article.  This model means that the publishers get their money from the authors rather than those accessing the articles.  However, it seems that that are problems with the rigorousness of the peer-review system for some of these open access journals.

A science journalist at Harvard University, John Bohannon, has conducted a fascinating study where he submitted a ficititious scientific paper to 304 open access journals over 10 months. The journals were worldwide and included some published by respected academic publishers such as Elsevier, Wolters Kluwer, and Sage.

Bohannon used fabricated names and fabricated academic institutions, for example, Ocorrafoo Cobange, a biologist at the Wassee Institute of Medicine in Asmara.  The paper was written to be superficially credible, but with flaws that any competent reviewer should be able to see.  Different variations of the article were submitted to the 304 journals, but they all took the format: Molecule X from lichen species Y inhibits the growth of cancer cell Z.

Bohannon wrote several flaws into the paper, an obvious one being that the caption of a data plot claims that it shows a “dose-dependent” effect on cell growth (which is central to the point of the paper), however, the actual data clearly shows that there is no such effect. Secondly, there was no effective control for the experiments; control cells were not exposed to radiation like the test cells were.  Any suggestion, therefore, that molecule X has any effect cannot be concluded from the experiment.

The final flaw should be evident to most people, not just specialists; the paper reads:

In the next step, we will prove that molecule X is effective against cancer in animal and human. We conclude that molecule X is a promising new drug for the combined-modality treatment of cancer.

Such a drastic conclusion from such minimal data, combined with exaggerated claims for the future of molecule X should raise multiple red flags about the author and the research as a whole.

To date, the paper has been accepted by 157 of the journals and rejected by 98. Of these 255 versions that went through the entire editing process to either acceptance or rejection 106 went through peer review (whilst the remainder did not). 70% of the peer reviewed journals accepted the paper; this does not bode well. To its credit, the only journal that drew attention to potential ethical problems with the research and rejected the paper was Public Library of Science, PLOS ONE.

So that the articles were not actually published, Bohannon withdrew them after they were accepted. It is, however, very worrying that such substandard research was almost published and brings the open-access journals into disrepute. It shows that peer-review is by no means a perfect system for ensuring that only valid research is published and draws attention to the plethora of poor quality open access journals.

See Bohannon’s article in Science for more on his study.