Peer review is an academic term used to describe the process of evaluating scientific research. However, due to the generally confidential nature of scientific evaluation, the exact process and what it means for research to be ‘peer reviewed’ can be unclear for some.
In this article, we’ll explore what ‘peer reviewed’ actually means, why peer review exists, its types, what the process does and does not guarantee, and how preprints and peer review compare.
What does ‘peer reviewed’ mean?
Peer review is a standardised method of assessing the integrity and originality of scientific research. It typically involves one or two external reviewers with expertise in the relevant subject areas providing detailed feedback on scholarly articles that have been submitted to journals.
For a scholarly article to be classed as ‘peer reviewed’, it needs to have successfully passed this period of evaluation. This whole period can take anywhere from a few months to a year, depending on factors such as rounds of assessment needed and reviewer availability.
Why it exists
The peer review model exists first and foremost as a quality control mechanism. Editors and reviewers are provided with a recognised process and standard for assessing the validity of scientific research. As a reuslt, the quality of information entering the scholarly ecosystem improves. Through the standardisation of the reviewing process, editorial workflows are also optimised for efficiency.
Ultimately, the peer review process serves as a mark of integrity, helping to build public and industry trust in science and scientific discoveries.
Types and definitions
Peer review has evolved alongside scientific communication. This evolution of the peer review process has resulted in many distinct types.
Single-blind peer review
This type is perhaps most closely aligned with what is commonly known as ‘peer review’. Single-blind peer review means the reviewers know the authors, but the authors do not know the identity of the reviewers.
This arrangement ensures authors cannot influence reviewers, but it may also carry risk of bias. Since the reviewers know the identity of the authors, their nationality, institutional prestige, and scholarly reputation could influence the decisions of reviewers, although situations like this are rare and typically contained by ethical parameters.
Double-blind peer review
In the double-blind peer review arrangement, neither the authors nor the reviewers know the identities of the other parties. This is arguably the most neutral reviewing model, with affiliations, names, and positions removed, leaving only the research to be evaluated.
Open peer review
Open peer review is an umbrella term for the various open practices used during the peer review process. Examples of open peer review practices include:
- Open reports: The review process is made public along with the published research.
- Open identities: The identities of the author and reviewer are known to each other, and the reader.
- Open participation: The scientific community is encouraged to leave comments on research post-publication.
- Open interaction: Authors, reviewers, and other researchers are free to interact and discuss.
Open participation and community feedback also feature in the reviewing of preprints. However, preprint-based research assessment is its own distinct process, and will be touched upon in further detail later.
What peer review does and does not guarantee
The peer review process, as we’ve discussed, is a quality control mechanism that improves quality and catches errors. It guarantees that individual experts have evaluated a study’s methodologies, data and results, and technical soundness to ensure it meets academic standards and disciplinary expectations.
What the peer review process does not guarantee is that a research paper is infallible. For example, reviewers tend not to independently verify raw data by repeating the experiments of a paper they are assessing; it is assumed that the data is factual. Errors, retractions, and long-term reproducibility can still occur in peer-reviewed literature.
Preprints and peer review
Preprints are distinct from peer-reviewed scholarly articles in that they don’t undergo a traditional, lengthy review process. Before they are posted online on Preprints.org, they instead undergo a small screening process, which usually takes 24 hours to complete.
Whilst this speed of dissemination is a big draw for researchers, this process may initially seem like a limitation in terms of ensuring academic quality and integrity. However, the preprinting process opens up different, more flexible ways of reviewing and improving research than a closed model of assessment.
Preprint-based research assessment
Preprint-based research assessment is the process of evaluating preprint research after posting and during the dissemination stage. It is an umbrella term, generally encompassing three types of preprint-based research output assessment taking place across different online services:
- Community commenting
- Organised community feedback
- Journal-like peer review
Community commenting
The first type, community commenting, involves members of the scholarly community leaving feedback on preprints hosted on preprint servers such as Preprints.org. This opens up the reviewing process beyond a handful of experts, allowing fellow scholars and interested researchers or even members of the public with an interest in scientific research to contribute constructive, nuanced, and broad criticism of the work.
There are no formal guidelines for commenting on preprints. However, Preprints.org recommends the “FAST” principles for feedback etiquette. This framework emphasizes that comments should be focused, appropriate, specific, and transparent.
Organised community feedback
Platforms like PREreview crowdsource preprints reviews written by the community. Structured feedback is published with its own Digital Object Identifier (DOI), making it easily discoverable and citable.
Journal-like peer review
In these cases, those scholars who would like to give a more in-depth response can do so.
This may be done through a specific institution, platform, or publication. For example, Peer Community In (PCI) provides organised peer review and formal recommendations for preprints, with review reports and editorial decisions made publicly available.
Supporting, not replacing, traditional peer review
Preprint-based research assessment does not replace the traditional peer review process. In fact, preprints provide a way for researchers to receive feedback before formal evaluation, leverage the wisdom of the crowd, and try out research methods in a space that is conducive to refinement rather than getting it perfect first time.
To read more about how preprints support academic integrity, read our post on how preprints pave the way for peer review.
