‘Liquid Journals’ Use the Web to Upend Peer Review

No one goes into science in order to spend every waking hour thinking about how to squeeze as many publications out of every experimental result as possible, rather than doing actual research, but that’s the game they’re forced to play.

Cutthroat competition for funding and positions has turned peer review, the traditional method for deciding what gets published in scientific journals, into an adversarial system that is fundamentally broken, according to those who study it.

The solution, say an increasingly vocal chorus of gadflies, is to supplement peer review with something less dysfunctional. The problem is: what?

Fabio Casati and his collaborators at the University of Trento, in Italy, think they have an answer.

Just as news, books and all other forms of printed material are undergoing their own revolutionary transformation from dead-tree media to the infinitely more malleable formats made possible by computers and the Internet, so too have scientific journals begun their slow, painful transition away from old, hidebound thinking.

Liquid Journals follow the disintermediated tendencies of the web to their logical conclusion: Liquid Journals do not rely on peer review. Instead, they are assembled by individuals or groups of scientists and experts using the Liquid Journals platform.

The Liquid Journals platform does not discriminate between peer reviewed and non peer reviewed papers, raw data sets and blog posts. The idea is that smart scientists can decide for themselves what belongs in their own liquid journal, and influential leaders and groups in the movement will organically accrue a readership to their journal according to the quality of the work they select.

If that sounds nuts, keep in mind that many have observed that there is little correlation between the judgment of the initial group of peer reviewers for a paper and its ultimate impact.

Breaking the Choke-Hold of Anonymous Peer Review

The zealous, relatively youthful programmers of Liquid Journal have concluded, and they’re not alone in this, that the autocratic nature of “peer” review – in which just three reviewers can, without fear their identities will ever be exposed, reject a paper for whatever reason they please, including a personal dislike of the submitter – does not add value to the scientific process.

Like the blogosphere itself, Liquid Journals accrue readers not because they have a choke-hold on distribution, as is the case with traditional journals, but because their readers find them to be uniquely qualified to filter a particular field.

On top of this is the open commenting model that has grown up around pre-print servers like arXiv.org, which serves the physics and mathematical community. In this model, paper are reviewed by everyone who cares to contribute – the difference is that, in Liquid Journals, algorithms will track various measures of the skill and reputation of reviewers, allowing new ways to filter for scientific resources by the quality an entire community has decided they possess.

Data that Lives, Rather than Dying on the Page

Liquid Journals will also turn scientific publications into living documents, rather than static informational cul-de-sacs. Already scientist have the ability to search for papers by the number of times they’ve been cited – imagine if that up-to-date information were an inherent part of the metadata of every scientific paper.

Liquid Journals could even solve the issue of publication quantity vs. quality. In a Liquid Journal, where results are grouped by third parties, there is no incentive to break the results of experiments into as many publications as possible. (A practice common among scientists whose value is measured by the length of their publication record.)

Ultimately, whether or not Liquid Journals are the solution to the problems of peer review is beside the point – the system is broken, and in no way respects the incredible diversity of distribution and filtering methods made possible by the web, much less the benefits that rapidly sharing data could have for the scientific enterprise as a whole.

Anyone can start their own liquid journal: just sign in via Facebook (naturally) at LiquidJournal.org.

Follow Mims on Twitter or contact him via email.