New idea for driving scientific debate in your journal club: PREreview

The eLife folks directed me to a very interesting link the other day — an article published on eLife Labs that encouraged scientists across the world to engage with research articles before they have been peer reviewed. (For those who do not know eLife Labs, it’s the place where all the new and cool things that try to change the scientific publishing racket are brought to the fore — I encourage a regular look-see!)

Preprint servers such as arXiv and bioRxiv are all the rage these days — they get research out to the community much faster and allow researchers to stake a claim on a novel technology or piece of research to avoid getting scooped during a sometimes lengthy peer review process. I won’t discuss the merits/drawbacks of preprint servers here today, but I did want to draw our readers’ attention to an idea that emerged from the eLife Labs article. Authors Samantha Hindle and Daniela Saderi lobby for the already existing “scientific journal club” structure to be a little bolder than average…  start discussing papers before submission to journals and have a say before the editors and reviewers do!

Secondly, the article makes the excellent point that too much discussion gets locked away in the journal clubs and never sees the light of day:

“it can be frustrating when those detailed critiques remain within the walls of the journal club and are unlikely to reach the authors or the wider community.”

Indeed, I would go a step further and say that it’s not just about the paper as a standalone piece of work, but the context of the discussion as well — journal clubs aren’t simply “Is this paper good or bad?” Rather, they often stimulate tangential discussion that drives new ideas. Some of the best journal clubs I have been to talk about the actual paper for a minority of the session, instead focusing on teaching points, historical context, or new ideas that could be pursued as a result. Liberating that strand of thinking is key to moving forward. Biomedical science often lacks the in-depth secondary criticism that connects the literature that our colleagues in the humanities and social sciences are so used to — journal clubs are one of the few places where it still takes place.

Wading through some murky waters

Some of the biggest pushback I’ve had from colleagues when I discuss this idea is that the preprint articles are “not up to publication standard.” No doubt there will be some papers that suffer from poor editing, absent controls, or premature conclusions. These make it tough to wade through, but perhaps this is the whole point of getting the material out there earlier (to know how a paper stands up when submitted to a jury of peers). We are already inundated with too many articles to read, and preprints threaten to increase that burden and the hopeless attempt to “keep up” with the literature.

If enough of these PREreviews were completed, it could have numerous potential benefits:

1. Authors fix the easy things before submission.
2. Metrics could be established for “quality” of the paper (e.g., positive reviews lead to a ranking of some sort).
3. Journals might be able to avoid external blind peer review (e.g., that article has been reviewed, we like it, we are going to publish/publicise it).

If the scientific community got behind this style of review in strong numbers, the possibilities are endless. At the very least there would be some incredible (and public!) discussion about scientific research and its impacts, although part of me worries about the practicalities (what if papers are never reviewed, do they become unimportant? Does posting prepublication make the minimum publishable unit problem even more acute?) — time will tell.

As with any new idea though — don’t knock it too hard until you try it…  so our journal club is going to tackle a bioRxiv paper in the coming weeks — I’ll be sure to report back!