Prerpints.org logo
Preprint
Article

Roles and Responsibilities for Referees of International Peer Reviewed Journals

Altmetrics

Downloads

161

Views

67

Comments

0

A peer-reviewed article of this preprint also exists.

This version is not peer-reviewed

Submitted:

27 February 2023

Posted:

27 February 2023

You are already at the latest version

Alerts
Abstract
Scholarly publication in international journals depends on qualified, unbiased and available referees: (1) qualified in their ability to assume each role inherent to being a peer reviewer and in willingly and competently accepting the responsibilities that coincide with these roles; (2) unbiased in focusing on the submitted research content irrespective of their own research programs in judging the submission’s merit; and (3) available to devote time to read and understand the paper, check the accuracy and relevance of references, and write a comprehensive review commenting on the strengths and weaknesses of the manuscript, the ability of the research to be replicated, and the contribution of the work to the discipline. This study investigates the range of reviewer’s roles and responsibilities in relation to author’s own assessment as a frequent reviewer for fourteen journals representing five publishing houses—and as an active researcher—in comparison with a 2019 comprehensive study of the views of 224 authors on peer review. Based on this investigation, advice will be provided to potential reviewers regarding what is expected of them in undertaking their work. Recommendations will be offered for peer review to mitigate weaknesses in the process and increase the pool of qualified peer reviewers.
Keywords: 
Subject: Social Sciences  -   Sociology

1. Introduction

Referees [1], or peer reviewers, of international journals engage in a process of detailed examination by which authors’ submitted manuscripts are scrutinized, improved, and evaluated by these academics for the authors’ ability to satisfy the receiving journal’s editors and guidelines [2]. The process of refereeing is synonymous with that of peer reviewing [3]. To referee is the older term, used exclusively until after the mid 1960s when peer review became the more commonly used expression, coinciding with this activity being deemed essential for rigorous academic publications [4].
Although previously considered ambiguous and recognized as a concept open to change [5], what defines an international journal is the language of publication is one employed by researchers in a number of countries, the editorial board and pool of reviewers include members from a variety of countries, and the journal maintains comprehensive, clear and public standards for publication—practices that result in greater visibility and higher citation rates regarding international journals [6]. International peer reviewed journals are generally held to a higher standard of review than what has been referred to as “megajournals”—open-access journals publishing manuscripts presenting scientifically trustworthy empirical results, yet doing so without considering the potential scientific contribution prior to publication [7]. That peer reviewers of international journals be qualified, unbiased and available is the foundation of legitimacy regarding publication of the research conducted [4,6]. As such, it is important that peer reviewers—in volunteering their time to maintain a high standard for research publication—be able to assume each of the roles inherent to being a referee and they willingly and competently accept the responsibilities that coincide with them.
As a social concept, what is considered a role with respect to peer review mirrors the definition provided by the Open Education Sociology Dictionary—a role is a position represented by a set of socially defined attributes and expectations determining appropriate behavior for an individual based on their status in relation to the group [8]. In contrast, responsibilities, with respect to specific roles, according to the Oxford English Dictionary are the state or fact of being in charge of or of having a duty towards a person or thing [9]. In other words, responsibilities of peer reviewers represent a subset of their roles, developed and understood in relation to them.
Although peer review is ubiquitous and fundamental, and the number of articles regarding issues related to peer review was found to have advanced from 50 in 1974 to approximately 120 in 2014 [10] and to have doubled between 2005 in 2015 [11], there has been a paucity of recent research to clarify the roles and responsibilities of peer reviewers. What has been written has lamented that the journals have been insufficiently diligent in defining peer reviewers’ roles and responsibilities [12].
One of the controversies surrounding the roles and responsibilities of peer reviewers is the appropriate length of time it should take to complete an adequate peer review, with the assumption that peer reviews are necessarily less effective if they are conducted quickly [10,[13], rather than considering that the expeditious peer reviewer may instead be efficient and skilled in this regard. This type of assumption—that peer review is inevitably a long process [3]—in part encourages substantial durations between the time of submission and that of publication, decreasing the timeliness of research while concurrently indicating to reviewers that conducting reviews is necessarily burdensome.
A peer review should take an average of 3 hours with only very complex topics taking hours more according to one author [13]. Another, based on a survey by Publons, specified that 5 hours is the average reported by peer reviewers [14]. A further author on peer review used research published over ten years ago for a reference claiming that the average reviewer spends approximately 10 hours reviewing a submission [15]. As such, there is no agreement on how long peer review actually takes. From the perspective of this author as a frequent reviewer, peer review generally takes 3 hours except when the submission is over twenty-five pages or the author’s writing demonstrates poor language skills, requiring additional time to decipher the meaning of the text. These reasons are comparable to, though slightly different from, those put forward in a review of reviewers which found four reasons determining the length of time for completion of the review: (1) scientific quality and content of the paper, (2) reviewer expertise, (3) presence of supplementary material, and (4) clarity of the language and structure [16]. Yet, regardless of which of the estimations is most accurate in time taken to review an article, Publons found a median of 16.4 days for reviewers to get their reviews back to the journal [17] (p. 32). Such a median was unexpected by this author—as someone who returns reviews the same day the review process is begun—especially given that perception of review speed is a significant predictor of the author’s assessment of the quality of the review [14].
After receiving scrutiny in the last few years [11,18], the publishing houses of journals have become responsive to the past criticism [19] regarding the obscurity of their responsibilities expected of peer reviewers. As a result, in the estimation of this frequent peer reviewer, guidelines have become very clear in the information they provide to peer reviewers on their roles and responsibilities. This is evidenced by the various journals associated with their publishing houses now having a separate, detailed page on each of their websites devoted to the guidelines for reviewers (examples of these website are cited below, in the Method section). Nevertheless, there are researchers who continue to criticize journals for not being clear in their roles and responsibilities, yet basing their conclusion on research conducted before these changes were made to improve the transparency of the roles and responsibilities of peer reviewers [20]. That these criticisms continue, in spite of the evidence that publishing houses are now making their guidelines obvious to peer reviewers and to authors, may be due in part to evidence demonstrating that few peer reviewers are themselves active researchers [16]. As such, these continuing assertions regarding problems with the roles and responsibilities of peer reviewers are at best questionable.
Some authors have pondered the necessity of peer review [21]. Yet, researchers continue to recognize its value—as was made clear in the 2018 Publons report that found 98% of respondents consider peer review to be either important (31.2%) or extremely important (66.8%) [17] (p. 9). Still, regardless of the improvement to the guidelines that have been provided to reviewers by journals, with the number of international peer reviewed journals increasing [4,16], the pool of qualified peers reviewers is diminishing [22]. The previous solutions that have been adopted to increase the pool of referees have concentrated on rewards to peer reviewers with respect to their role as peer reviewers [23]. This has not had the type of success necessary to increase the pool of effective reviewers who can attend to the guidelines [24]. Therefore, what is needed is a direct and more meaningful way to increase the pool of competent reviewers as those most qualified to assume the roles and responsibilities of peer review in relation to the guidelines of these journals. Since it is active researchers seeking publication who are most committed to publication in international journals, this author suggests that recognizing the peer reviewer foremost as a researcher is likely a more productive way to increase the pool of qualified peer reviewers to international journals than viewing them from the perspective of being a referee. Such recognition does not focus on the success of rewards in relation to the role of referee—a role that is not the primary focus of academics. Instead, the more substantial concern of scholars is the publication of their own research, so increasing the pool should be based on promoting academic scholarship, not professionalizing the role of reviewer.
Following from this understanding of the context of peer review, the aims of this study are to delineate the roles and responsibilities of peer reviewers in such a way as to decrease the lengthy period to publication and, as well, to increase the number of peer reviewers for maintaining international standards of journal publication. This study is valuable because it is important that research on the roles and responsibilities of peer reviewers of international journals be current—within the last five years—a period in which there has been little research on this topic. As such, the range of the roles and responsibilities of peer reviewers will be investigated in relation to the author’s own standards as an active researcher and frequent peer reviewer for various international journals of five distinct publishing houses. This will be undertaken in comparison with a 2019 publication evaluating the roles and responsibilities of referees based on the views of authors publishing on peer review [12]. The major findings of this comparison will be noted and, following from them, advice will be provided to potential reviewers concerning what is expected of referees in undertaking their work. Recommendations will be offered for improving peer review with regard to both mitigating weaknesses in the process with respect to function and time taken to completion, and increasing the pool of qualified peer reviewers—recognized a decade ago as the fundamental issues with respect to peer review [25].

2. Materials and Method

Regarding the materials and method to be used in investigating the roles and responsibilities of peer reviewers, it would seem reasonable that in order for a researcher to make a fair assessment of the roles and responsibilities expected of peer reviewers of international journals the researcher should be both a frequent peer reviewer and one who has current research experience to judge what should be the expectations. In meeting these proposed requirements, the author of this article undertakes this examination after having reviewed ninety manuscripts for fourteen different journals from 1 July 2022 to 31 December 2022 while concurrently having published five articles in international peer reviewed journals, with two others under review, over the same period. The journals for which the reviews were conducted are associated with one of the following five international publishing houses: Springer (specifically BMC [25], a division of Springer Nature), Dove Press, Frontiers, MDPI, and Wiley. It will be the guidelines of these five publishing houses that will be compared and contrasted for this evaluation of international journals, as these are the publishing houses with which the author is familiar. The guidelines for reviewers for each of the separate publishing houses are found as part of the publishing houses’ websites: Dove [26], Frontiers [27], MDPI [28], Springer [29], and Wiley [30].

2.1. Materials

In assessing the expectations of peer reviewers for each of these publishing houses, there are two materials that will be used: the data of Table 1 and of Table 2 to follow—both constructed by the author for the purpose of this study. Table 1 has been created based on the author’s own experience as a reviewer while Table 2 is the result of investigating, selecting from, and combining the content of two tables from a 2019 publication on peer review [12]. No research was conducted on human subjects for this study; as a result, an ethical review of the research was not required.

2.1.1. Author’s View on the Most Relevant Responsibilities in Conducting Peer Reviews

Table 1 has been constructed regarding the roles and responsibilities that the author has found most relevant in conducting the ninety reviews mentioned above. The content of this table thus represent the first material for this study and will be used to evaluate the roles and responsibilities of peer reviewers with respect to each of the five publishing houses of international journals for which the author has conducted reviews over the six month period between 1 July 2022 and 31 December 2022.
Table 1. Roles and corresponding responsibilities of reviewers for international peer reviewed journals recognized as important by the author for conducting a review.
Table 1. Roles and corresponding responsibilities of reviewers for international peer reviewed journals recognized as important by the author for conducting a review.
Roles Responsibilities
Ensure suitability as a reviewer Declare any conflict of interest
Determine if deadline can be met
Establish expertise to do the review
Maintain confidentiality
Use the journal guidelines for reviewers Read the guidelines
Follow the guidelines
Examine content of submission Read the submission carefully
Note clarity of writing
Check references for accuracy
Identify possible plagiarism
Evaluate submission Establish title/abstract/keywords suitability
Determine validity of question
Assess correctness of method
Gauge clarity of tables and figures
Identify if limitations are provided
Judge if conclusions are supported
Establish adequacy of references
Assess the originality of the work
Consider if there are ethical concerns
Determine if work is reproducible
Consider suitability for journal’s audience
Write the report Summarize the argument presented
State the strengths and weaknesses
Detail comments of suggested edits
Help improve written presentation
Comment without personal bias
Provide advice to editors Indicate ethical concerns to editor
Advise on further expertise required
Accept, reject or call for revisions
The material of Table 1, based on the author’s own experiences, incorporates and refines previous research findings on tips for reviewers [31], dos and don’ts of peer review [32] as well as the Committee on Publication Ethics (COPE) ethical guidelines for peer review [33]. Developed in this way, the material recognizes seven roles of reviewers and, for each role, there are corresponding responsibilities in relation to the order in which each role and responsibility becomes evident when a reviewer is in the process of conducting a review. The various roles of being a reviewer during the process of conducting the review can be considered in connection with the differing relationships the reviewer has to the journal, editors and authors [34]; the responsibilities concern specific tasks associated with the role [35] where a task is defined in the Oxford English Dictionary as a piece of work undertaken as a duty [36].
Regarding the details of Table 1, the initial role of the reviewer concerns the reviewer’s self-assessment of whether a positive response can be provided to the editors regarding the reviewer’s ability to accept the invitation to review. The responsibilities associated with this role are in regard to four separate aspects: ethical, temporal, disciplinary and personal—i.e., whether the reviewer has a conflict of interest, can meet the deadline, has the expertise to conduct the review effectively, and can guarantee the prudence to refrain from discussing the content of the review.
The second role concerns the reviewer’s relationship to the journal, that is, the willingness of the reviewer to provide the necessary time and concentration to study the guidelines of the journal with respect to peer review. Once the guideless are read, the responsibility of the reviewer is to agree to follow those guidelines when conducting the review.
In the third role, the reviewer acts as an agent for the journal in examining the authors’ manuscript to assess whether it meets the minimum requirements for a submission. This assessment is done in reading through the manuscript carefully. The reviewer notes the clarity of the writing. Depending on the difficulty in deciphering the paper, an unclear presentation may be reason for rejection. However, the authors’ limited knowledge of the language employed by the journal should not be a reason in itself for rejection. The type of problem with clarity that would point to obvious rejection is if the arguments presented did not follow from one another or from the data presented. References that are cited and listed inaccurately would also point to rejection if the references do not state what the authors’ claim. A necessary point to establish quickly is if the reviewer identifies plagiarism in the submission. A finding of plagiarism must be brought to the attention of the editor expeditiously as this represents grounds for immediate rejection of the manuscript. Regarding this last reason for immediate rejection, this author reports never having come across a manuscript that was plagiarized in any review personally conducted; however, this is known to occur [37].
Following from the third role, the fourth role of reviewers is to judge the adequacy of the submission—a role that bridges the intent of the authors with the expectations of the journal. This bridging is done in relation to the order of the manuscript’s different sections. The reviewer should begin by checking the front matter to establish the suitability of the title, abstract, and the keywords. The relevance of the title should be evident from the abstract and each of the keywords should be found in the abstract and listed in relation to the order in which they there appear. Reading through the introduction, the reviewer should check if it ends with a valid question based on the points made, and on the research presented in the introduction. The reviewer should read through the method to assess its correctness, both with respect to how it is applied and that it is the appropriate method, given the research question. Any tables or figures included should be gauged for their clarity of presentation and information provided. The discussion should include identified limitations and the conclusions should be supported by the evidence provided. In finalizing reading through the paper, the reviewer needs to be cognizant of establishing the adequacy of references—especially whether the work cited has been published within the last five years as references older than this, unless the work is seminal in the field or is supporting a historical point, are likely out of date. Once the submission is examined fully, the reviewer then has the responsibility of assessing the originality of the work, considering if there are ethical concerns and determining if work is reproducible. If any one of these three is lacking, this would involve the reviewer calling for a rejection of the article as originality, meeting ethical standards, and reproducibility are the foundation of research suitable for publication in international peer reviewed journals. Although the importance of originality may be questioned, editors of journals see this as a key concern [38]. With studies intended to determine the reproducibility of a previous study, the originality of the paper would be judged in relation to the authors’ assessment of the results from a current standpoint in relation to when the initial research results were published.
At this point in the review process, the reviewer assumes the fifth role—writing the report. This is a role that involves the relationship between the reviewer and the authors and includes responsibilities undertaken in the following order. The reviewer should summarize the argument presented in one paragraph then state the paper’s strengths and weaknesses in a second paragraph. After this, detailed comments and suggested edits should be provided line by line, if line numbers are available. Otherwise, these comments and suggested edits should be referred to page by page (a way of reducing the time it takes to review a submission recommended by this author is for these line by line suggested edits to be noted as the reviewer conducts the reading, rather than starting these for the first time once the entire paper is read). The purpose of these comments and suggestions is to improve the paper, it is neither to criticize nor embarrass the authors. The reviewer must be sure to maintain an objectively helpful manner in making comments and that they are made without personal bias.
The final role of the reviewer corresponds to the relationship of the reviewer to the editor and involves providing advice directly to the editor, some of which remains unseen by the authors. Those points that are private between the reviewer and the editor regard two things. The first is whether the reviewer has ethical concerns about the submission. These can range from a lack of informed consent obtained by the authors in conducting the research, to an inappropriate relationship recognized by the reviewer between the funding agency and the authors, to improper self-citation by the authors to references that are irrelevant to the submission (although a rare occurrence, this author has come across improper self-citation as a reviewer). Advice provided to the editors by the reviewer on whether further expertise is required to give a balanced review is also unseen the authors. This type of advice generally involves whether the reviewer is deficient in some area, such as not being a statistician (this author’s own deficiency as a reviewer). The advice to the editors that is seen by the authors is the final decision of the reviewer to accept, reject or call for revisions of the submission. This decision will be made in relation to the results of each of the previous responsibilities conducted by the reviewer. The first time a submission is assessed, it is unlikely to be accepted outright. Common problems are that the references chosen are either out of date for the current claims made by the authors, certain claims are not referenced, and/or there are no, or insufficient, limitations noted by the authors. In each case, the call will be for revision. Rejection is the assessment if there are major problems with the submission that can’t be fixed as a result of revision. Some major problems in this regard are plagiarism, the inappropriate use of a research method, the conclusions not following from the results, or the authors not following ethical protocol for a research study involving human subjects. In this author’s estimation as a reviewer, it is advisable to provide sufficient information to authors so that a paper might be revised even if there are a significant number of issues, rather than moving hastily to call for a rejection.

2.1.2. Ranking of Views of the 224 Authors Regarding Referees’ Roles and Responsibilities

Table 2 represents the second material to be used in assessing the roles and responsibilities of peer reviewers for international journals. It has been constructed by the author based on the results of a 2019 study of 224 authors of articles published on the roles and tasks of peer reviewers [12]. Table 2 combines the data from two separate tables in relation to (1) the roles and (2) the tasks of peer reviewers as viewed by various authors. Although these two tables from the 2019 publication are differentiated by their authors as roles and tasks, in examining the content of the tables, each relates rather to responsibilities of a peer reviewer, as roles in the case of this 2019 article are merely more general responsibilities and tasks are more specific. In this respect, the 2019 publication does not actually refer to roles of peer reviewers as has been defined above in the Introduction. What has been extracted from those two tables from the 2019 publication is the thirty most common replies of these 224 authors. Table 2 lists the number of authors who mentioned a particular responsibility providing the rank of that responsibility as a result.
Table 2. Data extracted from Table 1 and Table 2 of the 2019 publication, A Scoping Review on the Roles and Tasks of Peer Reviewers in the Manuscript Review Process in Biomedical Journals [12], in relation to ranking the 30 most common replies with respect to the number of replies by 224 authors of articles on the peer review process corresponding to the responsibilities of peer reviewers.
Table 2. Data extracted from Table 1 and Table 2 of the 2019 publication, A Scoping Review on the Roles and Tasks of Peer Reviewers in the Manuscript Review Process in Biomedical Journals [12], in relation to ranking the 30 most common replies with respect to the number of replies by 224 authors of articles on the peer review process corresponding to the responsibilities of peer reviewers.
Rank Responsibilities of Peer Reviewers According to Authors #
1 Determine if interpretation is supported by the data 92
2 Provide constructive criticism 87
3 Improve manuscript 84
4 Timeliness: meet journal deadline 81
5 Recommendations on publication (e.g. no/minor/major revisions, reject) 74
6 Be expert in the subject area/matter/field and/or be familiar with/trained in research methods and statistics 70
7 Determine validity/quality/technical merit/rigor 69
8 Declare/avoid potential or actual conflict of interest 66
9 Assess adequacy of methods in general 65
10 Evaluate study design 56
11 Assess originality 55
11 Consider general ethical aspects and report on any specific ethical concerns (including manipulation of data, plagiarism, duplicate publication, inappropriate treatment of animal or human subjects) 55
12 Determine clarity of tables 54
12 Assess novelty 54
13 Consider Adequacy of discussion (general) 53
14 Comment on interest to journal readership/relevance for journal scope 52
14 Appropriateness and accuracy of references 52
14 Maintain confidentiality of the manuscript, avoiding disclosure/discussion with others 52
15 Comment on clarity of study purpose and hypothesis 50
16 Assess importance/significance 48
17 Consider adequacy of results (general) 46
18 Comment upon relevance to practice/science (clinical relevance) 45
19 Comment upon contribution to the field 42
19 Assess data analysis (methods and tests) 42
19 Consider use of statistics 42
20 Be polite/courteous/respectful in the communication with authors 41
20 Determine overall readability 41
21 Assess presentation (general) 40
21 Advise editors on the merits of manuscripts 40
22 Be familiar with journal’s mission, review process, review criteria, guidelines (i.e. both author and reviewer guidelines) and forms prior to starting the review 39
22 Be fair: evaluate manuscript in a fair manner 39
23 Evaluate adequacy of introduction (in general) 37
23 Assess coherence/clarity and logical flow of the text 37
24 Be objective: objectively judge all aspects of the manuscript 36
24 Consider one’s time availability prior to accepting review request 36
25 Highlight whether current literature is covered 35
25 Prior to accepting review request, determine whether the manuscript is within one’s area of expertise (only review manuscripts in one’s own field of expertise) 35
25 Be thorough/comprehensive/detailed/accurate 35
26 Assess sampling strategy 34
27 Consider clarity and validity of statistical methods 33
27 Determine how data were collected/reproducibility of methods 33
28 Be unbiased in their assessment: peer reviewers should have an unbiased attitude towards an author’s gender, previous work, institution and nationality 32
28 Provide confidential comments to editor 32
29 Identify strengths and weaknesses 31
30 Assess grammar and spelling 30

2.2. Method

To compare the reponsibilities peer reviewers of the five publishing houses for which the author has reviewed in the last six months, the responsibilities itemized in Table 1 will be listed in Table 3 to follow, as they appear in Table 1, representing the order of these responsibilities in the manner they arise during the review process. The horizonal lines dividing the table group the responsibilities in relation to the seven different roles of referees noted in Table 1. Whether each of the publishing houses clearly states a particular responsiblity for reviewers in their website guidelines for reviewers will be indicted by a “Yes”. If the responsibility is not clearly stated, the space is left blank, as it would be misleading to conclude that the answer should be “No” merely because it is not definitely stated. The point is only to determine if the responsibility is clearly indicated, not whether it is assumed by the publishing house that authors will understand the importance of the unstated responsibility. In this regard, the publishing house may expect or prefer a particular responsibility, but not state it directly. The rank order established in Table 2 then represents the rank number in the first column of Table 3.
The purpose of this method is to both compare the five publishing houses on their stated responsibilties of peer reviewers, but to do this from the standpoint of those responsibilities the author has found the most compelling as a frequent reviewer. Additionally, these responsibilities are compared to the rank in importance of the particular responsibility to that which was identified in 2019 of the 224 authors of articles on the peer review process. The point is to compare both the five publishing houses regarding responsibilities of peer reviewers and to do so with respect to what authors on peer review think should be the importance of a particular responsibility. The publishing companies’ information is obtained from reading the guidelines on their respective webpages [26-30].
Table 3. Responsibilities of peers reviewers, grouped by roles set out in Table 1, and whether the responsibilities are identified by five publishing houses by the order in which they occur in the review process, ranked in relation to the number of times authors on peer review recognize the responsibilities of peer reviewers as per Table 2.
Table 3. Responsibilities of peers reviewers, grouped by roles set out in Table 1, and whether the responsibilities are identified by five publishing houses by the order in which they occur in the review process, ranked in relation to the number of times authors on peer review recognize the responsibilities of peer reviewers as per Table 2.
Rank Responsibility of Peer Reviewers Dove Frontiers MDPI Springer Wiley
8 Declare any conflict of interest Yes Yes Yes Yes Yes
24 Determine if deadline can be met Yes Yes Yes Yes Yes
25 Establish expertise to do the review Yes Yes Yes Yes
14 Maintain confidentiality Yes Yes Yes Yes
22 Read the guidelines Yes Yes Yes Yes
— Follow the guidelines Yes Yes
— Read the submission carefully Yes Yes Yes Yes
20 Note clarity of writing Yes Yes Yes Yes Yes
14 Check references for accuracy Yes Yes
11 Identify possible plagiarism Yes Yes Yes Yes Yes
— Establish title/abstract/keywords suitability Yes
15 Determine validity of question Yes Yes Yes Yes
9 Assess correctness of method Yes Yes Yes Yes Yes
12 Gauge clarity of tables and figures Yes Yes Yes Yes
17 Judge if conclusions are supported Yes Yes Yes Yes Yes
25 Establish adequacy of references Yes Yes Yes
11 Assess the originality of the work Yes Yes Yes Yes Yes
11 Consider if there are ethical concerns Yes Yes Yes Yes Yes
— Identify if limitations are provided Yes
27 Determine if work is reproducible Yes Yes Yes
14 Consider suitability for journal’s audience Yes Yes
— Summarize the argument presented Yes Yes
29 State the strengths and weaknesses Yes Yes Yes Yes Yes
25 Detail comments of suggested edits Yes Yes Yes Yes Yes
3 Help improve written presentation Yes Yes Yes Yes Yes
28 Comment without personal bias Yes Yes Yes Yes
28 Indicate ethical concerns to editor Yes Yes Yes Yes
— Advise on further expertise required Yes Yes
5 Accept, reject or call for revisions Yes Yes Yes Yes Yes

3. Results

The results to be presented will be with respect to Table 3 and be divided into two parts. The first will be regarding the ways in which the guidelines for peer reviewers of the five publishing houses explicitly state a particular responsibility for peer reviewers that this author has deemed important. The second part will be comparing the rank of the responsibilities with respect to the views of the 224 authors on the responsibilities of peer reviewers. This will be done in order to see both how what the author deems important ranks with these authors and which of the responsibilities expected of peer reviewers by the five publishing houses is given clear mention by them.

3.1. Comparing Responsibilities of Peer Reviewers for the Five Publishing Houses

In comparing the “Yes” results of the five publishing houses of journals with which this author has reviewed in the final six months of 2022, it is evident that most of the responsibilities the author considers important are mentioned specifically in the responsibilities of each publishing house’s guidelines for peer reviewers. However, none of the publishing houses mentioned all of the responsibilities; yet, there was no responsibility that was mentioned by none of the publishing houses.
There are a total of 29 individual responsibilities listed in Table 3. Of those , the number each the publishing house mentions explicitly is as follows: Dove—17, Frontiers—24, MDPI—24, Springer—22, Wiley—23. There were only 12 responsibilities that were mentioned by every publishing house, representing 41% of all of the responsibilities the author considers important regarding the responsibilities of peer reviewers. Of these unanimous mentions, the two roles from Table 1 most likely to have unanimous mentions among them are “Evaluate the submission” and “Write the report”. There were 8 responsibilities mentioned by four of the publishing houses, 2 indicated by three of them, 5 stated explicitly by two of them, and 2 that were clearly recognized by only one of the publishing houses.

3.2. Comparing the Rank of Responsibilities by the Five Publishing Houses

Before comparing the mentions of responsibilities by the five publishing houses with respect to the rank they are seen to have in amalgamating the views of 224 authors on peer review, it should be noted that these 224 authors (see Table 2) had a low agreement among themselves. Although there were 224 authors, the highest number of those who agreed on the importance of any particular responsibility was 92. In other words, at best, less than half of the total number were in agreement. This means there is little concurrence in what responsibilities authors on peer review think should be included in the guidelines for reviewers. Furthermore, it is interesting to note also that 6 of the responsibilities thought important by this author were not mentioned by any of the 224 authors.
Of the 11 unanimous mentions of responsibilities by the five publishing houses, their ranks from highest to lowest in comparison with the views of the 224 authors are as follows: 3, 5, 8, 9, 11 (with three ranking as 11), 17, 20, 24, 25, 29. The responsibilities that were not unanimous, that ranked somewhere in between those that were unanimous, included the following by ranks and the number of publishing houses that mentioned the particular ranked responsibility: 12—4, 14—4, 14—2, 14—2, 15—4, 22—4, 25—4, 25—3, 27—3, 28—4, 28—4 (two of the responsibilities had the same rank and same number of publishing houses that mentioned the particular responsibility).

4. Discussion

This discussion, with respect to the results, will be divided into two parts. The first will be a comparison of the guidelines of the five different publishing houses. The second will examine the ranking of the responsibilities of peer reviewers with respect to the views of the 224 authors who have written on peer review. The limitations of this examination will then follow.

4.1. Guidelines of the Five Publishing Houses

Of the five publishing houses representing journals for which the author has reviewed articles in the last half year of 2022, Dove is the least explicit in its guidelines for reviewers while Frontiers and MDPI both are most specific. How the latter two differ is that Frontiers is clear in telling reviewers they should follow the guidelines while MDPI does not provide this advice directly. Instead MDPI focuses on advising reviewers to read the submission carefully, which Frontiers does not. Nevertheless, Frontiers, unlike MDPI, states that the references should be checked for accuracy. In this regard, the way that Frontiers and MDPI differ seems to be with respect to their focus—Frontiers details a systematic approach to reviewing in all respects while MDPI seems to want reviewers to attend foremost to assessing the value of the content of the manuscript. This hypothesis regarding the differing focus of these two publishing houses with respect to peer review is further supported by Frontiers providing no explicit responsibility to summarize the paper when writing the review nor in reminding reviewers to comment without personal bias. Further justification is provided for concluding that MDPI is less concerned with reviewers taking an entirely systematic approach to reviewing in that it does not directly state that reviewers should let editors know when additional expertise is required to fully evaluate the submission.
It is only with respect to two responsibilities of peer reviewers thought important by this author that neither Frontiers nor MDPI provide explicit mention in their guidelines to peer reviewers: “Establish title/abstract/keywords suitability”, and “Identify if limitations are provided”. That reviewers should establish the suitability of the different parts of the front matter has become evident to this author in noting that there are instances when the title does not conform to the content of the submission, the abstract is poorly written, and/or the keywords are either not mentioned in the abstract or they are identified out of order (all keywords should be found in the abstract and listed in the order in which they there appear). These are problems the reviewer can bring to the attention of the authors during the review process and the importance of doing so has been recognized by one author in 2017 [39]. If they are left for the editors to improve then the authors are not provided with an explanation regarding why the front matter is required to be changed—they are merely advised to make the changes by the copy editor. Still, the front matter will be attended to by the editors if the reviewer does not bring the proposed changes to the attention of the author. The importance of this responsibility differs from that of missing limitations. If the reviewer does not mention the need for a limitations section, the editors will not necessarily expect that the authors include one. This problem is a greater concern than that of the front matter deficiencies,. Without a limitations section, the authors have not provided a balanced approach to the assessment of their research. This is a major flaw of any submission. It is for this reason that—although it is helpful if reviewers identify the need to correct the front matter of a paper—it is imperative that they request a limitations section if none is provided. This lack of providing direct instructions to reviewers in the guidelines of both the Frontiers and MDPI publishing houses concerning the need for a limitations section is an important omission that requires amendment. Of the five publishing houses, only Springer unequivocally states in its guidelines that reviewers should determine if a limitations section is provided.
Regarding the responsibility that reviewers should “Establish title/abstract/keywords suitability”, there is only one publishing house that states directly that reviewers must be mindful of examining these aspects of the front matter—Wiley. The Wiley publishing house in most respects is the most particular in the responsibilities it expects through its guidelines for reviewers. Why it falls a little short of Frontiers and MDPI in this regard is that, unlike these two publishing houses, it neither pointedly tells reviewers to read the guidelines nor to follow them. Perhaps this publishing house, which is very detailed regarding its responsibilities in other ways, considers the need to read the guidelines and follow them to be self-evident. In contrast—from this author’s experience in reading the reviews conducted by the other reviewers of the same paper the author has been charged with reviewing (a privilege only available once the reviewer has submitted their own contribution to the review process)—many reviewers do not follow the guidelines, and this appears to be because they have not read them. Reviews based on evidence suggesting a lack of familiarity with the evaluating journal’s guidelines generally are not helpful to either the authors who have submitted the manuscript or the editors to whom the review is intended to provide guidance. In most other respects, the guidelines provided by Wiley take into consideration the necessary responsibilities as recognized by this author. What is missing is pointedly telling reviewers to: “Identify if limitations are provided”, “Consider suitability for journal’s audience”, “Summarize the argument presented” and “Advise on further expertise required”. In this regard, the Wiley publishing house appears to recognize the submission as an independent entity specific to one journal rather than a contribution to the larger world of academic publishing.
Although, as mentioned, Springer is the only publishing house that clearly states in its guidelines that the reviewers should check for the limitations section, in other regards, this publishing house, and Dove, are both more relaxed about the requirements for the review process they specify in their guidelines. For Springer, this flexibility with respect to what is required by the reviewer extends primarily to the supporting details of the submission. Checking references for accuracy and adequacy is not distinctly stated, nor is gauging the quality of the tables and figures. That the adequacy of the references is not necessarily checked means that there is a potential for work to be published with dated references leading to claims that cannot be currently supported. It should also be mentioned that although Springer and Dove don’t indicate checking the accuracy of references, the other publishing houses in most cases do not state that references, unless seminal, must be from the last five years. Only MDPI explicitly states references should be from the last five years—Wiley, in contrast, specifies the last ten years (much too broad a timeline for scientific articles, from this author’s perspective). This lack of focus on the details of the review process extends to there being no direct comment on the need for reviewers to determine if the study is reproducible. Dove, in being more informal in the guidelines to reviewers in general, does not direct reviewers to follow the guidelines, advise the editors on the ethical concerns of the paper, or indicate the reviewer’s own limitations as a reviewer. It is the only publishing house that does not ask that reviewers “Determine validity of question”. Reviewers are instead expected to provide a more general account of the strengths and weaknesses of the paper from a disciplinary perspective.

4.2. Ranking of Responsibilities of Peer Reviewers

In comparing the results of Table 2 with those of Table 3, it is immediately evident that the rankings that the 224 authors of papers on peer review gave to specific responsibilities considered necessary for peer reviewers differs substantially from the responsibilities this author considers important that are also found important by the five publishing houses. As pointed out previously, the 224 authors had little agreement among themselves. There is no responsibility that even half of the authors agreed was necessary. The largest agreement was by 92 authors. What is interesting with respect to Table 3 is that the two tasks that the 224 authors thought most important, “Determine if interpretation is supported by the data” and “Provide constructive criticism”, are not part of Table 3. The reason these points are not included is that they are both too broad and too vague. It is unclear in what way reviewers are to determine if the interpretation is supported by the data. Rather than provide this ambiguous statement, the data of Table 3 instead breaks down, into a number of different steps, how this task might be completed throughout the process of conducting the review. This breakdown includes each of the responsibilities under the role of “Evaluate Submission” as laid out in Table 1. Similarly, “Provide constructive criticism” is too obscure a task for reviews to know what is wanted and what should be avoided. In this respect, Table 1, under the role of “Write the report”, outlines exactly what the responsibilities should be for the reviewer in this regard. When reviewers concentrate on their roles in evaluating the submission and writing the report as detailed in Table 1, there is no need for the extraneous responsibilities that have first and second place in the rankings of Table 2.
Nevertheless, there is important agreement among the rankings of the 224 authors and the five publishing houses if the rankings on Table 3 that appear in the top 11 of Table 2 are examined as they are the few where all the publishing houses explicitly include the responsibility as part of their guidelines. That said, only the ranks of 3, 5, 8, 9 and 11 appear. The others between 1 and 11 from Table 2 are not found on Table 3. For 4, 6, 7 and 10, the reason they don’t appear has more to do with how the authors (of the 2019 article from which the results were found) have made particular distinctions. Regarding the responsibility ranked as 4, “Timeliness: meet journal deadline”, this is very similar to “Determine if deadline can be met”, which ranks as item 24. As the publishing houses refer the need for authors to determine if the deadline can be met, this coincides with the responsibility set out by this author in Table 1, rather than the idea of timeliness specified by the authors of the 2019 paper. However, they likely represent the same point and should not be differentiated as they are in the 2019 paper. The responsibility ranked as 6 in Table 2, “Be expert in the subject area/matter/field and/or be familiar with/trained in research methods and statistics”, similarly is very close to “Establish expertise to do the review”, which ranks 25 in Table 2. Given that the point ranked as 6 conflates a number of issues together and should have been broken down by the 2019 authors of the study—yet it contains the idea of the reviewer establishing their expertise to do the review—the responsibility of establishing expertise is one considered important in both Table 2 and Table 3. Again, “Determine validity/quality/technical merit/rigor”, which represents rank 7 on Table 2 contains more than one task and should not have been listed as one point alone by the 2019 authors. With more than one task as part of rank 7, it is difficult to tell how this task differs from “Assess adequacy of methods in general”, which ranks as 9. Finally, “Evaluate study design’, which ranks as 10 in Table 2, is too general to be a helpful instruction to peer reviewers. As with the item ranking 1 discussed above, responsibility 10 is too broad and reviewers are better informed to undertake each of the responsibilities in the role of “Evaluate Submission” from Table 1.
There are six responsibilities listed on Table 3 that have no corresponding rankings from Table 2. These are, “Follow the guidelines”, “Read the submission carefully”, “Establish title/abstract/keywords suitability”, “Identify if limitations are provided”, “Summarize the argument presented”, and “Advise on further expertise required”. Generally, these responsibilities initially laid out in Table 1 are not ones expressly stated in the guidelines and are found in fewer of the five publishing houses, with only one or two of these publishing houses directly advising reviewers to undertake these responsibilities. The exception is “Read the submission carefully”—four of the five publishing houses make of point of telling this to their peer reviewers in their guidelines. That none of the 224 authors on peer review thought it important to mention that the guidelines for peer review of the particular journal should be followed, and that few of the publishing houses include this advice in their guidelines, perpetuates the idea voiced by authors over the last fifteen years [6,12,14,16,19,20,23,25,26,28,40,41,42,43], that peer reviewers need training (an idea that is not supported by most journal editors [30]). This is in contrast to expecting that they merely are obligated to read the guidelines to know what is important in conducting a peer review. Similar to the responsibility of “Establish title/abstract/keywords suitability”, neglecting to provide the editors with “Advise on further expertise required” prolongs the process of peer review. For this reason, even though neither the 224 authors nor most of the publishing houses exactly state these as responsibilities of peer reviewers (in that they don’t explicitly state them) there is insufficient guidance on including these ways of reducing the time between submission and decision. The neglect of most publishing houses in this study to decidedly tell reviewers in their guidelines that they should “Summarize the argument presented” means that the authors of the submission and the editors cannot be sure that the reviewer has understood the content of the manuscript—potentially leading to additional confusion and wasted time in the review process. This is why it is problematic that this responsibility did not receive a ranking in the study of 224 authors. Yet, by far the biggest omission by both those 224 authors in the 2019 study and the various publishing houses is regarding “Identify if limitations are provided”. Only Springer specifies that reviewers need to ensure that there is a limitations section to the submission under review. That none of the 224 authors considered this important and four of the publishing houses do not explicitly include this information in their guidelines means there is a substantial amount of research published that may not have been scrutinized to ensure limitations are covered in the discussion.

4.3. Limitations

The primary limitation of this study is that the material of Table 1 is based on the author’s own perception as a frequent reviewer of various international journals. Although this author’s view is informed by a number of sources [31-33}, it still represents the summation of only one reviewer. Why this is a problem is that reviewers, by the nature of their task, work alone. As such, they don’t necessarily communicate their process of reviewing to others and receive feedback on the strengths and weaknesses of their established process. That said, when reviewers finish a review they are given the opportunity by the journal to read the reviews of the other reviewers who have refereed the same manuscript. Usually, this means the reviewer is permitted to read two additional reviews. It is from always taking the opportunity to compare reviews that this author considers the material created for this study in Table 1 represented the most useful summation of the roles and responsibilities of reviewers.
Another limitation of this work is that research related to the views of authors on peer reviews concerning their roles and responsibilities was pertaining to one study conducted in 2019 of 224 authors of articles on peer review. This reliance on only one source for data was especially problematic because the authors of that paper differentiated the responsibilities and tasks of peer reviewers and did not consider the roles (although they believed they did in recognizing their distinction between roles and tasks). As such, by here evaluating the tasks and roles voiced by these authors as each being subsumed under responsibilities for the purpose of this evaluation, this author may have extended the context of responsibilities further than the 224 authors would have assumed. Furthermore, without the 2019 paper actually identifying the roles of reviewers (as defined in this paper’s Introduction), it remains unclear how the 224 authors conceptualized the roles of reviewers, and this could not be assessed. Further research in this regard could be to go back to the individual articles of the 224 authors and specifically look for what each has said about the roles of peer reviewers in contrast to responsibilities (or tasks).
The author acknowledged in the Materials section not being a statistician. Determining the accuracy and validity of statistical results takes expert knowledge and time when conducting a review. As such, given that this author does not possess the expertise to review statistical results, the author’s thought that reviews can be conducted in a timely manner under 16.4 days may be naïve. It’s possible that in certain cases the reviewer must design and create a specialized computer program to assess the accuracy of the manuscript’s results under review. If parameter definitions in the paper are vague, this may take time to decipher on the part of the reviewer. Therefore, this advice on how to decrease the time to a decision on the submission may be deficient in not having taken into consideration what may possibly be the most time-consuming aspect of peer review— representing a limitation of this assessment.
One aspect of the peer review process that was not considered was whether it makes a difference to the roles and responsibilities of peer reviewers for the journal to have a blind review or if the journal expects open identities and/or open reports [17] (p.48). This possibility was not considered because, with respect to the journals for which this author reviews, most have the option of choosing whether the reviewer would prefer an open review or does not want to make their identity known. In this author’s own experience, the transparency of the review has been irrelevant to the roles and responsibilities the reviewer should assume—a view that has been supported by research which, however, also notes there may be a difference in how scholars in the humanities and social sciences view open journals in comparison with the sciences [44]. As such, some of the 224 reviewers who participated in the 2019 study may not have held the same view regarding whether open review changes the roles and responsibilities of referees. That their view on this matter is unknown is a limitation.

5. Conclusions

Maintaining the quality of peer review, especially when the number of peer reviewers in comparison to submissions is decreasing, is necessary if the peer review process is to remain effective. Various authors since 2007, as noted in the Discussion, have considered that the best way to do this is to require all potential peer reviewers to undertake peer review training. Although this suggestion may establish consistency in the requirements for peer review, if the curriculum for peer review were standardized—and following it demanded—it neglects to recognize the efforts of journals in making their requirements for peer review readily accessible, transparent, extensive, detailed, and particular regarding their journal’s needs. As such, rather than necessitating that peer reviewers be trained, journals need only make their guidelines required reading for peer reviewers and that they advise peer reviewers to follow them when conducting the review.
Nevertheless, although there is much to be commended about the guidelines for reviewers offered by each of the five publishing houses examined, in some cases these guidelines still do not highlight a few key components of peer review. The first: the importance of author’s identifying research from the last five years has not been stressed sufficiently to peer reviewers by most publishing houses. Furthermore, peer reviewers have not been asked to focus on the vital importance of the limitation section to submissions. In this regard, peer reviewers have scrutinized submissions without effective attention being given to requiring that authors assess the weaknesses of their research. In following a recommendation to publishing houses that they include a focus on these important aspects, the current value of international publications and their veracity will be increased.
With respect to increasing the pool of reviewers—although one author has argued that there should be no obligation to review papers, as this demand may produce ill-conceived reports that are “misleading and useless” [43]—this present author suggests, in contrast, that each corresponding author should review at least one other paper in their field before their accepted publication goes to press. Although this idea is in opposition to the above mentioned author, it is not a new idea. Furthermore, it has even been proposed that authors of a submission should review three other papers to compensate for the three reviews they have experienced in regards to their submission during the review process [15,19]. However, such a stringent requirement, although it would significantly increase the number of peer reviewers available, would at the same time increase the time to decision to a degree that likely would be detrimental to prompt publication in international peer reviewed journals. Since the actual purpose of peer review is to determine suitability for publication in a timely manner, rather than to calculate a fair distribution of hours devoted to peer reviewing, it would seem reasonable that the publication of research not be slowed down unnecessarily by expecting all authors with an accepted manuscript to then each review what would normally be three other manuscripts to match the number of peer reviews they experienced in the acceptance of their own submission. As a growing amount of scientific research is conducted in large teams [45], there can be a very large number of authors for a particular publication. If, before an accepted paper is published every one of these authors were expected to conduct a peer review, the focus of the role of the journal would change from publishing to administering peer reviews. Therefore, the suggestion is to have only the corresponding author of an accepted publication conduct a peer review before publication to keep administration of these peer reviews manageable. This is a suggestion supported by recent research published in this journal [46]. In this way, the pool of qualified peers reviewers would necessarily be increased while maintaining the aim of expeditious publication.
A median of 16.4 days for time to completion of peer reviews from the Publons survey of peer reviewers [17] (p. 32) is unexpectedly long from this author’s perspective. In promoting timely research, every effort should be made by publishing houses to improve their guidelines so that this median time is reduced. As such, it would be helpful if these guidelines included the following responsibilities that this author has found useful in reducing the number of times that a submission requires revision: “Establish title/abstract/keywords suitability”, “Summarize the argument presented” and “Advise on further expertise required”. If all publishing houses asked reviewers to, (1) examine the front matter for errors and omissions, (2) summarize the content of the paper in the first paragraph of the review to indicate that the reviewer has understood the argument presented by the submission’s authors, and (3) provide advice to the editors that the reviewer is lacking in some area, the time to completion of reviews would likely be reduced.
The roles and responsibilities of referees of international peer reviewed journals have, in response to previous criticism by researchers, become detailed, comprehensive and transparent, as evidenced by the guidelines for peer review being specified on their own webpage for each of the publishing houses investigated. As such, the most effective thing a peer reviewer can do in understanding these roles and responsibilities—rather than undergoing general training on the peer review process—is to read these guidelines and follow them. Nevertheless, there remains room for improvement in these guidelines. The most important changes that should be made are that all publishing houses should specify the references of submitted work must be current within the last five years and that each submission be scrutinized to determine if it includes a limitations section. With the number of submissions continuing to increase, in order for these roles and responsibilities to be undertaken in a timely manner the pool of reviewers must be increased in a way that is relevant to these referees as researchers. By requiring each corresponding author to conduct a peer review before publication of an accepted submission, not only will the pool of qualified researchers be increased, it will be increased in a way that is relevant to the entire process of peer review in international journals.
In this tenth anniversary issue of Publications, it is hoped that this focus on the roles and responsibilities of peer viewers for international journals will exemplify the importance the peer review process continues to have for international journal publication, and this effort presented will be identified as an effective means for both making these roles and responsibilities evident while at the same time decreasing the time to completion and increasing the pool of qualified peer reviewers.

Funding

This research received no external funding.

Data Availability Statement

The data for Table 1 is available through this publication. The data for Table 2 can be extracted from Table 1 and Table 2 of the 2019 publication, A Scoping Review on the Roles and Tasks of Peer Reviewers in the Manuscript Review Process in Biomedical Journals [12].

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Dondio, P.; Casnici, N.; Grimaldo, F.; Gilbert, N.; Squazzoni, F. The “invisible hand” of peer review: The implications of author-referee networks on peer review in a scholarly journal. J. Informetr. 2019, 13, 708–716. [Google Scholar] [CrossRef]
  2. Hoffman, A.J. A modest proposal to the peer review process: A collaborative and interdisciplinary approach in the assessment of scholarly communication. Res. Ethics 2022, 18, 84–91. [Google Scholar] [CrossRef]
  3. Jana, S. A History and Development of Peer-Review Process. Ann. Lib. Inform. Stud. 2019, 66, 152–162. Available online: http://eprints.rclis.org/39332/1/26964-465493240-1-PB.pdf (accessed on 8 February 2023).
  4. Fyfe, A.; Squazzoni, F.; Torny, D.; Dondio, P. Managing the growth of peer review at the Royal Society journals, 1865-1965. Sci. Technol. Human Values 2020, 45, 405–429. [Google Scholar] [CrossRef]
  5. Buela-Casal, G.; Perakakis, P.; Taylor, M.; Checa, P. Measuring internationality: Reflections and perspectives on academic journals. Scientometrics 2006, 67, 45–65. [Google Scholar] [CrossRef]
  6. Tennant, J.P.; Ross-Hellauer, T. The limitations to our understanding of peer review. Res. integrity peer rev. 2020, 5, 6. [Google Scholar] [CrossRef]
  7. Björk, B.C.; Catani, P. Peer review in megajournals compared with traditional scholarly journals: Does it make a difference? Learn. Publ. 2016, 29, 9–12. [Google Scholar] [CrossRef]
  8. Bell, K. (Ed.) Role. Open Edu. Sociol. Dict. 2013. Available online: https://sociologydictionary.org/role/ (accessed on 19 February 2023).
  9. Responsibility, n. OED Online Oxford University Press. 2022. Available online: www.oed.com/view/Entry/163862 (accessed on 19 February 2023).
  10. Willis, M. Why do peer reviewers decline to review manuscripts? A study of reviewer invitation responses. Learn. Publ. 2016, 29, 5–7. [Google Scholar] [CrossRef]
  11. Grimaldo, F.; Marušić, A.; Squazzoni, F. Fragments of peer review: A quantitative analysis of the literature (1969–2015). PLoS ONE 2018, 13, e0193148. [Google Scholar] [CrossRef]
  12. Glonti, K.; Cauchi, D.; Cobo, E.; Boutron, I.; Moher, D.; Hren, D. A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals. BMC Med. 2019, 17, 118. [Google Scholar] [CrossRef]
  13. Castelo-Branco, C. Peer review? No thanks. Climacteric 2023, 26, 3–4. [Google Scholar] [CrossRef] [PubMed]
  14. Drvenica, I.; Bravo, G.; Vejmelka, L.; Dekanski, A.; Nedić, O. Peer Review of Reviewers: The Author’s Perspective. Publications 2019, 7, 1. [Google Scholar] [CrossRef]
  15. Schultz, D.M. How to Be a More Effective Reviewer, Mon. Wea. Rev. 2022, 150, 1201–1205. [Google Scholar] [CrossRef]
  16. Pomponi, F.; D’Amico, B.; Rye, T. Who Is (Likely) Peer-Reviewing Your Papers? A Partial Insight into the World’s Top Reviewers. Publications 2019, 7, 15. [Google Scholar] [CrossRef]
  17. Publons|Clarivate Analytics. Global State of Peer Review. 2018. Available online: https://publons.com/static/Publons-Global-State-Of-Peer-Review-2018.pdf (accessed on 9 February 2023).
  18. Brezis, E.S.; Birukou, A. Arbitrariness in the peer review process. Scientometrics 2020, 123, 393–411. [Google Scholar] [CrossRef]
  19. Grainger, D.W. Peer review as professional responsibility: A quality control system only as good as the participants. Biomaterials 2007, 28, 5199–5203. [Google Scholar] [CrossRef]
  20. Hoffman, A.J. A modest proposal to the peer review process: A collaborative and interdisciplinary approach in the assessment of scholarly communication. Res. Ethics Rev. 2022, 18, 84–91. [Google Scholar] [CrossRef]
  21. Blockeel, C.; Drakopoulos, P.; Polyzos, N.P.; Tournaye, H.; García-Velasco, J.A. Review the ‘peer review’. Reproduct. Biomed. Online 2017, 35, 747–749. [Google Scholar] [CrossRef]
  22. Fox, C.W.; Albert, A.Y.; Vines, T.H. Recruitment of reviewers is becoming harder at some journals: A test of the influence of reviewer fatigue at six journals in ecology and evolution. Res. Integrity Peer Rev. 2017, 2, 1–6. [Google Scholar] [CrossRef]
  23. Warne, V. Rewarding reviewers–sense or sensibility? A Wiley study explained. Learn. Publ. 2016, 29, 41–50. [Google Scholar] [CrossRef]
  24. Zaharie, M.A.; Seeber, M. Are non-monetary rewards effective in attracting peer reviewers? A natural experiment. Scientometrics 2018, 117, 1587–1609. [Google Scholar] [CrossRef] [PubMed]
  25. Hope, A.A.; Munro, C.L. Criticism and Judgment: A Critical Look at Scientific Peer Review. Am. J. Crit. Care 2019, 28, 242–245. [Google Scholar] [CrossRef] [PubMed]
  26. Writing a Peer Review. Available online: https://www.dovepress.com/peer-review-guidelines/writing-peer-reviews (accessed on 21 January 2023).
  27. Peer Review. Available online: https://www.frontiersin.org/about/peer-review (accessed on 21 January 2023).
  28. Guidelines for Reviewers. Available online: https://www.mdpi.com/reviewers#_bookmark1 (accessed on 21 January 2023).
  29. Advancing Peer Review at BMC. Available online: https://www.biomedcentral.com/about/advancing-peer-review (accessed on 21 January 2023).
  30. Step by Step Guide for Reviewing a Manuscript. Available online: https://authorservices.wiley.com/Reviewers/journal-reviewers/how-to-perform-a-peer-review/step-by-step-guide-to-reviewing-a-manuscript.html (accessed on 21 January 2023).
  31. Azer, S.A.; Ramani, S.; Peterson, R. Becoming a peer reviewer to medical education journals. Med. Teacher. 2012, 34, 698–704. [Google Scholar] [CrossRef] [PubMed]
  32. Rees, L.; Flynn, J. Supportive peer review: Why and how to constructively review a paper. Pediatr. Nephrol. 2022, 37, 1701–1703. [Google Scholar] [CrossRef]
  33. COPE Council. COPE Ethical guidelines for Peer Reviewers—English; COPE Council: Mudgee, Australia, 2017. [Google Scholar] [CrossRef]
  34. Glonti, K.; Hren, D. Editors’ perspectives on the peer-review process in biomedical journals: Protocol for a qualitative study. BMJ Open 2018, 8, e020568. [Google Scholar] [CrossRef] [PubMed]
  35. Glonti, K.; Boutron, I.; Moher, D.; Hren, D. Journal editors’ perspectives on the roles and tasks of peer reviewers in biomedical journals: A qualitative study. BMJ Open 2019, 9, e033421. [Google Scholar] [CrossRef]
  36. Task, n. OED Online Oxford University Press. Available online: www.oed.com/view/Entry/198017 (accessed on 20 February 2023).
  37. lse, H.; Van Noorden, R. The fight against fake-paper factories that churn out sham science. Nature 2021, 591, 516–519. [Google Scholar] [CrossRef]
  38. Warner, T.A. How to write an effective peer-review report: An editor’s perspective. Inter. J. Remote Sensing 2019, 40, 4871–4875. [Google Scholar] [CrossRef]
  39. Pytynia, K.B. Why Participate in Peer Review as a Journal Manuscript Reviewer: What’s in It for You? Otolaryngol. -Head Neck Surg. 2017, 156, 976–977. [Google Scholar] [CrossRef]
  40. Teplitskiy, M.; Acuna, D.; Elamrani-Raoult, A.; Körding, K.; Evans, J. The sociology of scientific validity: How professional networks shape judgement in peer review. Res. Pol. 2018, 47, 1825–1841. [Google Scholar] [CrossRef]
  41. Barroga, E. Innovative Strategies for Peer Review. KAMJE 2020, 35, e138. [Google Scholar] [CrossRef] [PubMed]
  42. Moher, D.; Altman, D.G. Four proposals to help improve the medical research literature. PLoS Med. 2015, 12, e1001864. [Google Scholar] [CrossRef] [PubMed]
  43. Cintas, P. Peer review: From recognition to improved practices. FEMS Microbio. Let. 2016, 363, fnw115. [Google Scholar] [CrossRef]
  44. Bravo, G.; Grimaldo, F.; López-Iñesta, E.; Mehmani, B.; Squazzoni, F. The effect of publishing peer review reports on referee behavior in five scholarly journals. Nat. Commun. 2019, 10, 322. [Google Scholar] [CrossRef] [PubMed]
  45. Wu, L.; Wang, D.; Evans, J.A. Large teams develop and small teams disrupt science and technology. Nature 2019, 566, 378–382. [Google Scholar] [CrossRef] [PubMed]
  46. Raoult, V. How Many Papers Should Scientists Be Reviewing? An Analysis Using Verified Peer Review Reports. Publications 2020, 8, 4. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated