the importance of peer review in research

You are using an outdated browser . Please upgrade your browser today !

What Is Peer Review and Why Is It Important?

It’s one of the major cornerstones of the academic process and critical to maintaining rigorous quality standards for research papers. Whichever side of the peer review process you’re on, we want to help you understand the steps involved.

This post is part of a series that provides practical information and resources for authors and editors.

Peer review – the evaluation of academic research by other experts in the same field – has been used by the scientific community as a method of ensuring novelty and quality of research for more than 300 years. It is a testament to the power of peer review that a scientific hypothesis or statement presented to the world is largely ignored by the scholarly community unless it is first published in a peer-reviewed journal.

It is also safe to say that peer review is a critical element of the scholarly publication process and one of the major cornerstones of the academic process. It acts as a filter, ensuring that research is properly verified before being published. And it arguably improves the quality of the research, as the rigorous review by like-minded experts helps to refine or emphasise key points and correct inadvertent errors.

Ideally, this process encourages authors to meet the accepted standards of their discipline and in turn reduces the dissemination of irrelevant findings, unwarranted claims, unacceptable interpretations, and personal views.

If you are a researcher, you will come across peer review many times in your career. But not every part of the process might be clear to you yet. So, let’s have a look together!

Types of Peer Review

Peer review comes in many different forms. With single-blind peer review , the names of the reviewers are hidden from the authors, while double-blind peer review , both reviewers and authors remain anonymous. Then, there is open peer review , a term which offers more than one interpretation nowadays.

Open peer review can simply mean that reviewer and author identities are revealed to each other. It can also mean that a journal makes the reviewers’ reports and author replies of published papers publicly available (anonymized or not). The “open” in open peer review can even be a call for participation, where fellow researchers are invited to proactively comment on a freely accessible pre-print article. The latter two options are not yet widely used, but the Open Science movement, which strives for more transparency in scientific publishing, has been giving them a strong push over the last years.

If you are unsure about what kind of peer review a specific journal conducts, check out its instructions for authors and/or their editorial policy on the journal’s home page.

Why Should I Even Review?

To answer that question, many reviewers would probably reply that it simply is their “academic duty” – a natural part of academia, an important mechanism to monitor the quality of published research in their field. This is of course why the peer-review system was developed in the first place – by academia rather than the publishers – but there are also benefits.

Are you looking for the right place to publish your paper? Find out here whether a De Gruyter journal might be the right fit.

Besides a general interest in the field, reviewing also helps researchers keep up-to-date with the latest developments. They get to know about new research before everyone else does. It might help with their own research and/or stimulate new ideas. On top of that, reviewing builds relationships with prestigious journals and journal editors.

Clearly, reviewing is also crucial for the development of a scientific career, especially in the early stages. Relatively new services like Publons and ORCID Reviewer Recognition can support reviewers in getting credit for their efforts and making their contributions more visible to the wider community.

The Fundamentals of Reviewing

You have received an invitation to review? Before agreeing to do so, there are three pertinent questions you should ask yourself:

• Does the article you are being asked to review match your expertise?
• Do you have time to review the paper?
• Are there any potential conflicts of interest (e.g. of financial or personal nature)?

If you feel like you cannot handle the review for whatever reason, it is okay to decline. If you can think of a colleague who would be well suited for the topic, even better – suggest them to the journal’s editorial office.

But let’s assume that you have accepted the request. Here are some general things to keep in mind:

Please be aware that reviewer reports provide advice for editors to assist them in reaching a decision on a submitted paper. The final decision concerning a manuscript does not lie with you, but ultimately with the editor. It’s your expert guidance that is being sought.

Reviewing also needs to be conducted confidentially . The article you have been asked to review, including supplementary material, must never be disclosed to a third party. In the traditional single- or double-blind peer review process, your own anonymity will also be strictly preserved. Therefore, you should not communicate directly with the authors.

When writing a review, it is important to keep the journal’s guidelines in mind and to work along the building blocks of a manuscript (typically: abstract, introduction, methods, results, discussion, conclusion, references, tables, figures).

After initial receipt of the manuscript, you will be asked to supply your feedback within a specified period (usually 2-4 weeks). If at some point you notice that you are running out of time, get in touch with the editorial office as soon as you can and ask whether an extension is possible.

Some More Advice from a Journal Editor

• Be critical and constructive. An editor will find it easier to overturn very critical, unconstructive comments than to overturn favourable comments.
• Justify and specify all criticisms. Make specific references to the text of the paper (use line numbers!) or to published literature. Vague criticisms are unhelpful.
• Don’t repeat information from the paper , for example, the title and authors names, as this information already appears elsewhere in the review form.
• Check the aims and scope. This will help ensure that your comments are in accordance with journal policy and can be found on its home page.
• Give a clear recommendation . Do not put “I will leave the decision to the editor” in your reply, unless you are genuinely unsure of your recommendation.
• Number your comments. This makes it easy for authors to easily refer to them.
• Be careful not to identify yourself. Check, for example, the file name of your report if you submit it as a Word file.

Sticking to these rules will make the author’s life and that of the editors much easier!

Explore new perspectives on peer review in this collection of blog posts published during Peer Review Week 2021

[Title image by AndreyPopov/iStock/Getty Images Plus

David Sleeman

David Sleeman worked as a Senior Journals Manager in the field of Physical Sciences at De Gruyter.

You might also be interested in

Academia & Publishing

Taming the Dragon: Librarians, Creativity and AI

Guardians of scholarship: librarians and copyright in the age of ai, 16 holiday reads for the intellectually curious, visit our shop.

De Gruyter publishes over 1,300 new book titles each year and more than 750 journals in the humanities, social sciences, medicine, mathematics, engineering, computer sciences, natural sciences, and law.

Pin It on Pinterest

Why is peer review important in science?

Why is Peer Review Important in Science?

Peer review is an essential component of the scientific process, ensuring that research is rigorously evaluated and held to the highest standards of quality and accuracy. The importance of peer review cannot be overstated, as it is a mechanism that allows the scientific community to verify and validate research findings, ensuring that only the most reliable and accurate information is disseminated to the public. In this article, we will delve into the reasons why peer review is crucial in science, examining its numerous benefits, advantages, and importance.

What is Peer Review?

Before we explore the significance of peer review, it is necessary to understand what it entails. Peer review is a process in which experts in the same field as the researcher review and evaluate the quality, validity, and relevance of research, often before publication. This review is typically conducted by independent, impartial experts who are not directly affiliated with the research institution or project. The goal of peer review is to ensure that the research is sound, well-designed, and accurately reported, thereby maintaining the integrity of the scientific process.

Peer review is important for several reasons:

1. Quality Control

Peer review serves as a quality control mechanism, guaranteeing that research meets the highest standards of quality and rigor. Independent reviewers have the expertise to identify potential flaws, errata, and errors, ensuring that the research is accurately reported and free from significant biases.

2. Academic Integrity

Peer review helps maintain the integrity of the scientific process by ensuring that research is subjected to rigorous scrutiny and evaluation. This helps to prevent plagiarism, fabrication, and other forms of academic dishonesty, thereby preserving the credibility of the scientific community.

3. Ensuring Transparency

Peer review facilitates transparency by making the research process more accessible and transparent. Reviewers provide constructive feedback, which can lead to improvements in the research design, methodology, and presentation, thereby enhancing the overall quality of the study.

4. Building Consensus

Peer review helps to build consensus among experts, as different perspectives and evaluations are incorporated into the research. This consensus-building process contributes to the development of a more comprehensive understanding of the research topic.

5. Discovery and Innovation

Peer review encourages innovation and discovery by fostering a dialogue between the researcher and the reviewers. This exchange of ideas and knowledge can lead to new insights, identify potential avenues for future research, and stimulate breakthroughs in various fields.

The Benefits of Peer Review

The benefits of peer review are numerous:

• More accurate and reliable research : Peer review ensures that research is accurate, reliable, and free from significant errors.
• Increased credibility : The scientific community places a high value on peer-reviewed research, recognizing it as a key hallmark of high-quality, rigorous, and trustworthy research.
• Improved research design : Peer review provides valuable feedback, which can lead to improvements in research design, methodology, and presentation.
• Faster dissemination of knowledge : Peer review accelerates the dissemination of knowledge, as published research is widely available and accessible to the scientific community.

The Current State of Peer Review

While peer review is an essential component of the scientific process, its effectiveness is not without its challenges. There are concerns about the current state of peer review, including:

• Biomimicry : The increasing pressure to publish, coupled with a lack of resources, can lead to a culture of brevity and superficial analysis, compromising the quality of review.
• Time constraints : Reviewers often face time constraints, making it difficult to provide thorough and comprehensive reviews.
• Lack of transparency : The peer review process can be opaque, with little or no feedback provided to authors, leading to frustration and a lack of accountability.

In conclusion, peer review is a vital component of the scientific process, ensuring that research is rigorously evaluated and held to the highest standards of quality and accuracy. With its numerous benefits, including quality control, academic integrity, transparency, building consensus, and driving discovery and innovation, peer review plays a critical role in maintaining the integrity and legitimacy of scientific research. As the scientific community continues to evolve, it is essential that we address the challenges faced by peer review, ensuring that this mechanism remains robust and effective in promoting the highest standards of scientific excellence.

• How to download YouTube videos with captions?
• How can I remove all friends on Facebook?
• How to create new user in MySQL?
• How to put music on itunes on iphone?
• How come I canʼt see my profile views on TikTok?
• Does qa analyst easy to be instead by AI?
• How to turn off comments on my Facebook post?
• Is 7 too young for a training bra?

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Science-Based Medicine

Exploring issues and controversies in the relationship between science and medicine

The Importance and Limitations of Peer-Review

Peer-review is a critical part of the functioning of the scientific community, of quality control, and the self corrective nature of science. But it is no panacea. It is helpful to understand what it is, and what it isn’t, its uses and abuses.

When the statement is made that research is “peer-reviewed” this is usually meant to refer to the fact that it has been published in a peer-reviewed journal. Different scientific disciplines have different mechanisms for determining which journals are legitimately peer-reviewed. In medicine the National Library of Medicine (NLM) has rules for peer-review and they decide on a case by case basis which journals get their stamp of approval. Such journals are then listed as peer-reviewed.

The basic criterion is that there is a formalized process of peer-review prior to publication – so this presents a barrier to publication that acts as a quality control filter. Typically, the journal editor will give a submitted paper to a small number of qualified peers – recognized experts in the relevant field. The reviewers will then submit detailed criticism of the paper along with a recommendation to reject, accept with major revisions, accept with minor revisions, or accept as is. It is rare to get an acceptance as is on the first round.

The editor also reviews the paper, and may break a tie among the reviewers or add their own comments. The process, although at times painful, is quite useful in not only checking the quality of submitted work, but improving the quality. A reviewer, for example, may point out prior research the authors did not comment on, or may point our errors in the paper which can be fixed.

It is typical for authors to submit a paper to a prestigious journal first, and then if they get rejected to work their way down the food chain until they find a journal that will accept it. This does not always mean that the paper was of poorer quality – the most prestigious journals have tons of submissions and can pick and choose the most relevant or important studies. But sometimes it does mean the paper is mediocre or even poor.

The limitations of Peer-Review

It is important to realize that not all peer-reviewed journals are created equal. Small or obscure journals may follow the rules and gain recognized peer-reviewed status, but be desperate for submissions and have a low bar for acceptance. They also have a harder time getting world-class experts to review their submissions, and have to find reviewers that are also farther down the food chain. The bottom line is that when a study is touted as “peer-reviewed” you have to consider where it was reviewed and published.

Even at the best journals, the process is only as good as the editors and reviewers, who are people who make mistakes. A busy reviewer may give a cursory read through a paper that superficially looks good, but miss subtle mistakes. Or they may not take the time to chase down every reference, or check all the statistics. The process generally works, and is certainly better than having no quality control filter, but it is also no guarantee of correctness, or even the avoidance of mistakes.

Peer-reviewers also have biases. They may be prejudiced against studies that contradict their own research or their preferred beliefs. They may therefore bias the published studies in their favored direction, and may be loath to give a pass to a submission that would directly contradict something they have published. For this reasons editors often allow authors to request or recommend reviewers, or to request that certain people not be asked to be reviewers. Each journal has their own policy. Sometimes an editor will specifically use a reviewer that the authors request not be used, thinking they may be trying to avoid legitimate criticism.

The process can be quite messy, and full of politics. But in the end it more or less works. If an author thinks they were treated unfairly by one journal, they can always go to another or they can talk directly to the editor to appeal a decision and try to make their case.

Perhaps the biggest weakness of peer-review, however, is when an entire discipline of peers is lacking in some fundamental way. For example, there are now many journals dedicated to so-called “alternative medicine” ( CAM ).  The editors of such journals tend to have a pro- CAM bias, and they find reviewers with a pro- CAM bias. So pretty much any pro- CAM article can get published. Some have enough ideological friends at the NLM that they can get approved as peer-reviewed, despite glaring biases in their editorial policy.

Post Publication Peer Review

The term peer-review is sometimes used to refer to the fact that papers are read and reviewed by the broader scientific community once they are published. However, this post-production review should not be confused with “peer-reviewed” and that term should not be used to refer to post-publication review, to avoid confusion.

The process, however, is even more critical to quality control in science. Now, instead of one editor and 2-3 reviewers looking at a study, dozens or hundreds (maybe even thousands) of scientists can pick over a study, dissect the statistics and the claims, bring to bear knowledge from related areas or other research, and provide detailed criticism. This is the real “meat grinder” of science. Hundreds of reviewers are more likely to find problems than the few pre-publication reviewers. Arguments can be tested in the unforgiving arena of the scientific community, weeding out bad arguments, honing others, so that only the best survive.

Here is the bottom line – peer-review is a necessary component of quality control in science, but is no guarantee of quality, and you have to know the details of the journal that is providing the peer-review.

Founder and currently Executive Editor of Science-Based Medicine Steven Novella, MD is an academic clinical neurologist at the Yale University School of Medicine. He is also the host and producer of the popular weekly science podcast, The Skeptics’ Guide to the Universe , and the author of the NeuroLogicaBlog , a daily blog that covers news and issues in neuroscience, but also general science, scientific skepticism, philosophy of science, critical thinking, and the intersection of science with the media and society. Dr. Novella also has produced two courses with The Great Courses , and published a book on critical thinking - also called The Skeptics Guide to the Universe .

• Posted in: Clinical Trials , Science and Medicine

Posted by Steven Novella

What is Peer Review: Importance and Types of Peer Review

In academic publishing, peer review stands as both a guardian and a guide. The peer review process ensures that academic manuscripts meet the rigorous standards of quality, accuracy, and relevance before they are published and shared with the wider world. In this article, we explain what is peer review, delve into its importance in scholarly publishing, and outline the various types of peer review that underpin the credibility and integrity of research. As the ongoing Peer Review Week 2023 focuses on the future of peer review, we also speculate on the changes and evolution of peer review over the coming years.

Table of Contents

What is peer review, the importance of peer review, the different types of peer review, peer review challenges and potential solutions, the future of peer review.

Peer review has its roots in ancient Greece and has evolved over 300 years to become the bedrock of research publishing today. 1 At its core, peer review is a systematic evaluation process where experts in a specific field critically assess the quality, validity, and significance of research contributions well before they are published. Peer reviews are employed by academic journals to ensure the preservation and improvement of the caliber of the content they publish. This essential step in the scholarly publishing journey serves as a quality control mechanism, ensuring that research is not only accurate but also meets the highest standards of credibility and ethical integrity.

One cannot overstate the significance of peer review in the world of academia. Here are some key aspects of its importance:

Quality Assurance: Peer review acts as the first line of defense against research flaws, errors, or inaccuracies. It ensures that research published is of the highest quality.

Credibility and Trust: When research undergoes peer review, it gains the trust of the academic community and the wider public. This trust is essential for building the reputation of both the research and the researcher.

Validation of Findings: Peer review validates research findings, providing assurance that the methods and results are sound and reliable.

Identification of Flaws: It allows experts to identify potential flaws, biases, or limitations in the research, prompting authors to address them before publication.

There are various types of peer review, 2,3 and each comes with its own strengths and limitations. Here are the most common forms of peer review:

Single-Blind Peer Review: In this type of peer review, the identity of the reviewers is hidden from the author, but the reviewers know the identity of the author. This reduces potential bias but may not eliminate it entirely.

Double-Blind Peer Review: Both the reviewers and the author remain anonymous to each other in this type of peer review. This is considered one of the most impartial forms of peer review, reducing biases based on author identity.

Open Peer Review: In open peer review, the identities of both reviewers and authors are disclosed. This type of peer review promotes transparency but can sometimes deter reviewers from providing honest critiques.

Post-Publication Peer Review: 4 This type of peer review has gained momentum in recent years with the popularity of preprints. Research is published first, and then the academic community assesses and reviews it. This type of peer review allows for immediate feedback and discussion once research is in the public domain. It helps encourage transparency and open dialogue, and the broader community of experts involved means it is more likely to uncover potential issues. However, it also introduces complexities regarding the timeline of research validation, as publication and review are no longer sequential but concurrent.

Peer review, while essential, is not without its challenges. Here are some of the main hurdles in the peer review process:

• Potential Bias: Reviewers may inadvertently introduce biases based on various factors, including author demographics, research topic, or personal opinions. Solutions involve diversifying reviewer pools and promoting double-blind or open peer review.
• Review Timelines: Traditional peer review can be time-consuming, delaying the dissemination of research. Solutions include streamlining review processes, embracing post-publication peer review, and setting clear submission-to-publication timelines.
• Availability of Reviewers: Finding willing and qualified reviewers can be a challenge, particularly for niche topics. Solutions involve building more robust reviewer databases and offering incentives for reviewers.
• Quality Control: Ensuring consistent and thorough peer review can be challenging for journal editors. Solutions involve providing clearer guidelines for reviewers and promoting reviewer training.

The world of scholarly publishing is continually evolving, and the peer review process is likely to also undergo some changes to adapt to the digital age. In the future, we can expect to see:

Technological Advancements: The use of artificial intelligence (AI) and machine learning to assist in the peer review process, improving efficiency and reducing bias.

Hybrid Models: A blending of traditional and post-publication peer review, striking a balance between rigorous pre-publication evaluation and open discourse post-publication.

Greater Transparency: Increasing transparency in peer review processes, including sharing reviewer reports and decision rationale with published papers.

Global Collaboration: Enhanced international collaboration in peer review, with reviewers and authors from diverse backgrounds and regions.

In conclusion, peer review, in all its forms, remains the cornerstone of academic excellence, ensuring that research maintains the highest standards of quality, credibility, and integrity. While challenges persist, innovative solutions and the evolution of peer review models promise a future where research is more rigorously evaluated, transparent, and accessible than ever before. As scholars, we must adapt to this evolving landscape, embracing the transformative potential of peer review in the pursuit of knowledge and discovery.

References:

• Mulligan, A. Is peer review in crisis? Perspectives in Publishing, Elsevier. August 2004. Available online at https://assets.ctfassets.net/o78em1y1w4i4/1NSvvQobvw8cD6gjNJsAia/ca8416702cdcc5c513899457b36bc93a/PerspPubl2.pdf
• What is peer review?, Elsevier website. Available online at https://www.elsevier.com/reviewers/what-is-peer-review
• Peer Review: Types of Peer Review. Icahn School of Medicine at Mount Sinai website. Available online at https://libguides.mssm.edu/peerreview/types
• O’Sullivan, L. Ma, L. Doran, P. An Overview of Post-Publication Peer Review. Scholarly Assessment Reports. Available online at https://scholarlyassessmentreports.org/articles/10.29024/sar.26

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$12 a month!  

Related Reads:

• 5 Reasons for Rejection After Peer Review
• How to Write an Academic Essay with References
• PhD Thesis First Draft: 8 Practical Writing Tips for PhD Students
• Manuscript Withdrawal: Reasons, Consequences, and How to Withdraw Submitted Manuscripts

AI Content Detectors: Unfairly Tagged Research and the Quest for Authenticity

Life sciences papers: 9 tips for authors writing in biological sciences, you may also like, online ai writing tools: cost-efficient help for dissertation..., phd qualifying exam: tips for success , ai in education: it’s time to change the..., 9 steps to publish a research paper, self-plagiarism in research: what it is and how..., 6 tips for post-doc researchers to take their..., 8 most effective ways to increase motivation for..., how to make your thesis supervision work for..., how to write a conclusion for research papers..., ethical research practices for research with human subjects.

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

The peer review process

Dmitry tumin, joseph drew tobias.

• Author information
• Copyright and License information

Address for correspondence: Dr. Joseph Drew Tobias, Department of Anesthesiology and Pain Medicine, Nationwide Children's Hospital, 700 Children's Drive, Columbus, Ohio 43205, USA. E-mail: [email protected]

This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.

The peer review process provides a foundation for the credibility of scientific findings in medicine. The following article discusses the history of peer review in scientific and medical journals, the process for the selection of peer reviewers, and how journal editors arrive at a decision on submitted manuscripts. To aid authors who are invited to revise their manuscripts for further consideration, we outline steps for considering reviewer comments and provide suggestions for organizing the author's response to reviewers. We also examine ethical issues in peer review and provide recommendations for authors interested in becoming peer reviewers themselves.

Key words: Manuscript review process, manuscript submission, peer review

Introduction

The review of research articles by peer experts prior to their publication is considered a mainstay of publishing in the medical literature.[ 1 , 2 ] This peer review process serves at least two purposes. For journal editors, peer review is an important tool for evaluating manuscripts submitted for publication. Reviewers assess the novelty and importance of the study, the validity of the methods, including the statistical analysis, the quality of the writing, the presentation of the data, and the connections drawn between the study findings and the existing literature. For authors, peer review is an important source of feedback on scientific writing and study design, and may aid in professionalization of junior researchers still learning the conventions of their field. Nevertheless, peer review can be frustrating, intimidating, or mysterious. This can deter authors from publishing their work or lead them to seek publication in less credible venues that use less rigorous peer review or do not subject manuscripts to peer review at all. In this article, we trace the origins of the scientific peer review system, explain its contemporary workings, and present authors with a brief guide on shepherding their manuscripts through peer review in medical journals.

The History of Scientific Peer Review

The introduction of peer review has been popularly attributed to the Royal Society of Edinburg, which compiled a collection of articles that had undergone peer review in 1731.[ 2 , 3 ] However, this initial process did not meet the criteria of peer review in its modern form, and well into the twentieth century, external and blinded peer review was still far from a requisite for scientific publication. Albert Einstein protested to the editor of an American journal in 1936 that his article was sent out for review, whereas this was not the practice of the German journals to which he had previously contributed.[ 4 ] Nevertheless, by the 1960s, the scientific value of peer review was becoming widely accepted, and in recent years, publication in a peer-reviewed journal has become a standard metric of scientific productivity (for the researchers) and validity (for the study).[ 5 , 6 ] In fact, publication in peer-reviewed quality journals is used to evaluate the quality of research during the academic promotion process. Today, peer review continues to evolve with the introduction of open review (reviewer comments posted publicly with the final article), postpublication review (reviews solicited from readers in an open forum after article publication), and journal review networks (where reviews are transferred from one journal to another when an article is rejected).[ 7 , 8 , 9 ] The constant at the center of this change remains the individual reviewer, who is asked to contribute their expertise to evaluating a manuscript that may or may not ever be shared with a wider scientific audience.

Reviewer Selection

The opacity of the peer review process is due, in part, to the anonymity of the reviewers and authors' lack of familiarity with how reviewers are selected. Typically, reviewers are selected by an editor of the journal, although depending on the size and organization of the journal, this may be the Editor-in-Chief, an Associate Editor, a Managing Editor, or an Editorial Assistant. Some journals permit authors to suggest their own reviewers, although the extent to which editors use these suggestions is variable. Authors may also be asked specifically or allowed to oppose reviewers, if they feel that certain scholars cannot grant their manuscript an unbiased hearing. Again, it is at the editors' discretion whether these requests are heeded. It has been suggested that these “opposed” reviewers may even be deliberately selected to ensure critical evaluation of a controversial manuscript. Alternatively, for very specific and narrow subject areas, there may be a limited number of appropriately qualified reviewers.

In general, reviewers may be of any academic rank and from a wide range of medical disciplines. A reviewer may be selected for their expertise in the topic of the study, but also for their general methodological expertise, or because they have been a reliable reviewer for the journal in the past. Qualified reviewers may not be invited if they cannot be reached by the editorial team, if they tend to submit late or uninformative reviews, or if they are too closely connected with the manuscript authors (e.g., colleagues at the same institution) and therefore may not provide an unbiased review. The reviewers initially selected by the editors may decline the invitation to review, mandating that the editors seek other reviewers. Unfortunately, this process of waiting for a response from the initial invitation to review (aside from the time taken to review) is one of the more common causes resulting in a delay in getting a response from the journal when a manuscript is submitted. The invited reviewer may pass the review on to a junior faculty member to allow them to participate and experience the academic peer review process. This may be performed with the permission of the editor, and noted after the review is submitted to the editor when the invited reviewer identifies that another person has participated in the process.

The initially received reviews may conflict with one another, leading the editors to cast a wider net for experts who will agree to review a submission. Because many factors may delay the completion of the review process, editors may proactively invite more reviews than they require and decide on the manuscript after a minimum number of reviews have been completed. The use of email and the internet has greatly facilitated communication for the review process, which used to be accomplished via telephone and postal mail. In most instances, an initial email is sent to the reviewer inquiring regarding their availability and interest. They are then asked to agree to review, at which time, a secondary email with a link to the journal site, the manuscript, and the review forms is sent.

How Reviewers Assess a Manuscript

From the reviewer's perspective, participation in the review process begins with an invitation from the journal editors to consider reviewing a submitted manuscript. If they accept, the reviewers will be able to access the submitted manuscript files, and sometimes the authors' cover letter, and other article metadata (e.g., the authors' list of preferred reviewers, figures, tables, etc.). Some journals ask reviewers to complete a structured questionnaire regarding the manuscript, rating its attributes on a numeric scale, or answering specific questions about each article section. All journals permit the submission of free-response evaluations. It is these evaluations that typically carry the greatest weight in the editors' final decision. The free-text reviewer reports also give the authors specific instructions about revising their manuscript and responding to the concerns that are raised. Reviewers may also submit confidential free-response comments to the editors (not seen by the authors) and indicate to the editors if they would be willing to review a revised version of the manuscript. In the end, the reviewer is asked to indicate their final recommendation to accept the manuscript without changes, accept after minor revisions, reconsider after major revisions, or reject. Some journals may offer additional variations on these recommendations, such as “reject but allow resubmission,” discussed below.

Regardless of the requested format for reviews, reviewers will typically evaluate several key aspects of submitted manuscripts. For original research studies, these will include the importance of the research question, the rigor of the methods, the completeness, accuracy, and novelty of the study and its results, and the validity of conclusions drawn from the data. The presentation of the manuscript, including the writing style, structure, grammar, and syntax also determine how the manuscript is received by the reviewers. Although the study design and results may be valid, these findings may be lost if the presentation is not precise or if there are grammar and spelling errors.

Reviewers also consider whether the study adds to existing knowledge in the field, whether it was ethically conducted, and whether it may be subject to any conflicts of interest. The editor and the reviewers also evaluate the study content and decide whether it is valuable and relevant to the readers of the journal. Although the study may be valid and well performed, it may be decided that the subject matter fits more appropriately in a journal of a different specialty. Along those lines, there may be overlap in the interests and fit of journals in different specialties, so that common topics in anesthesiology research may be of interest to journals from surgical specialties, pain medicine, or healthcare quality and patient safety, depending on the article content.

Some reviewers may submit their comments in paragraph form, building a narrative of the study's strengths and weaknesses section by section, whereas others may submit a short summary of the study followed by a list of criticisms or suggested corrections. Less commonly, reviewers may annotate the original manuscript with specific changes and questions or using the track-changes function of the word-processing software. Although the reviewers may recommend a specific editorial decision (e.g., recommend accepting an article with revisions, recommend rejection) in their comments to authors, this is generally discouraged by most journals and does not override the final decision reached by the editorial team. The ultimate decision generally resides with the section editor or the editor-in-chief, once they have seen and evaluated the comments of the reviewers. Depending on the format of the journal, the manuscript may be reviewed by one to five individuals. When there are specific statistical questions or advanced methods used, a separate review of the analytic methods may be required. For high-profile journals with high Impact Factors, a recommendation to accept may be required from all reviewers to receive a favorable editorial decision. At times, if there is a split decision, an additional reviewer or member of the editorial board may be asked to evaluate the manuscript to break the tie.

Almost all journals practice blinded review, where the reviewers' identities are not revealed to the authors. Double-blind review, where authors' identities are concealed from reviewers, although previously uncommon in medical journals, has been increasingly used. The editors communicate their decision and reviewers' evaluations to the authors in a decision letter (e-mail), informing of manuscript acceptance or rejection.

Reviews and the Editorial Decision

The comments submitted by external reviewers are collected by the editorial team and considered when determining the overall decision on the submitted article. The reviews may be read directly by the Editor-in-Chief, or by one or more Associate or Section Editors. The first editor reading the reviews might provide a recommendation that is then considered by the more senior editor; or the editors may convene to discuss the reviews and reach a decision as a group. In some journals, editors may write their own summary of the reviewers' criticism (sometimes adding their own) or may point out the critiques they consider most important to their decision. In other journals, editors weigh the number of positive and negative reviews or may reject an article unless all reviewers endorse its acceptance or revision.

Based on the external reviews and their own reading of the manuscript, the editors will reach one of several options regarding the manuscript. Unconditional acceptance of an article on its first submission to a journal (without any requested revisions) is very rare. Sometimes, articles will be conditionally accepted or accepted with minor revisions, meaning that the editors wish the authors to make changes to their manuscript based on the reviewers' comments but will not send the revised manuscript for a further round of external review. Rather, if the comments are generally minor, the editor will ensure that the comments are appropriately addressed in the authors' revision. The more common decision is “major revision,” where editors are willing to consider a revised version of the article but will subject it to further external review, by the original reviewers, a new set of reviewers, or a combination of both. Some journals also use a “reject and resubmit” decision, indicating lower enthusiasm for a resubmitted version of the article but still permitting resubmission, perhaps in an alternative format (e.g., brief report or letter to the editor, vs. full article) or with extensive revisions. For this latter decision, a full review will be accomplished as the revised manuscript is handled in much the same way as a new submission.

If the editors feel an article is a poor fit for their journal or falls too far below its standards, they may reject submissions outright without sending the manuscript for external review. This “desk reject” should not be confused with articles being “unsubmitted” by a managing editor or editorial assistant. The latter can happen due to style or formatting issues with the initial submission, which the author is asked to correct before the manuscript proceeds to review. Having a manuscript “unsubmitted” does not preclude resubmission of a corrected manuscript and is unlikely to affect reviewer assessment and, eventually, editorial decision.

Revising the Manuscript

When the initial editorial decision is positive, but not an unconditional acceptance, authors may elect to revise their manuscript and resubmit it to the same journal with a point-by-point response to the reviewers (discussed in the next section). The primary aim of the authors for this revision should be to address the criticisms and concerns raised during the initial review. Yet, this may be easier said than done when faced with conflicting recommendations, hostile reviews, or simply a large number of suggestions to be accommodated within a strict manuscript word limit. To streamline the process of responding to reviews, we offer the following roadmap as a suggestion.

Address the “fatal flaws”

Reviewers or editors may point out critical weaknesses of the study that prevent it from drawing the intended conclusions or even any conclusions at all. For example, an inaccuracy in the data, a bias in patient recruitment, a limitation of sample size, or a lack of follow-up may be so severe that the manuscript cannot provide credible evidence on the treatment or exposure it is meant to study. In particular, a lack of appropriate ethical approval would disqualify a study from publication, no matter how methodologically rigorous it may have been. In systematic reviews and analyses of existing databases, prior publication of a near-identical paper by a different group may also fundamentally preclude a paper from acceptance. On the rare occasions when the paper's central conclusions are found invalid and cannot be corrected through new analysis or a different framing of the authors' argument, reconceiving the study may be a better approach than attempting to revise and resubmit. At other times, some of these issues may be approached and the editor and reviewers satisfied by adding text to the discussion outlining the limitations of the current study. This may allow authors to acknowledge the concerns expressed by the reviewers and yet not redo their study from the beginning.

Amend the data analysis

More commonly, reviewers ask for changes to the data analysis without implying that these requests invalidate the entire study. We recommend making these changes before any further edits to the manuscript, because the intent is often to see if the paper's original findings are robust. In the best case scenario, any additional analysis will only confirm and strengthen the central conclusions. However, additional analyses sometimes reveal contradicting findings, which the authors should frankly address in the revised manuscript, by pointing out the contradiction and speculating about why different analyses of their data may have reached different conclusions. Especially when the study design was prospectively registered, the authors should explain in the manuscript which analyses were planned a priori and which were added post hoc . In these studies, authors should also avoid changing the pre-specified primary outcome, which would have been used for any a priori power or sample size calculation.

Decline infeasible or inappropriate suggestions

Some requests may not be feasible, for example, when requested data were not collected for a prospective study, or when collecting the data would mean starting chart review from scratch for a retrospective study. At other times, it may not be feasible to comply with the reviewers' requests if they disagree with the study type, the study cohort, or make other requests that would require a new or different study to address. Reviewers could also request changes to the statistical analysis that are not appropriate for the data at hand or for the study aims. In these cases, authors have the choice of rebutting the reviewers' comments while making no change in their manuscript, but an argumentative revision that leans too heavily on this option may be received poorly on re-review, resulting in rejection of the manuscript. In our experience, authors may be successful in responding to the reviews while rebutting one or two of the reviewers' suggestions, but a legitimate argument must be made for the rebuttal, and the reasons clearly stated.

Explain the study rationale and methods

Having completed the revision of the data analysis, authors should check that their methods section includes a complete and correct explanation of how the data were collected and explains how the analysis was performed. It may be appropriate to end the introduction by stating the hypothesis of the study. In the methods section, reviewers will often ask about the ethical committee approval of the study, the site(s) where the study was conducted, patient inclusion and exclusion criteria, the consent process, the procedures involved and the protocol for anesthetic management, and the specific data points that were collected during the study. For prospective clinical studies, authors should also indicate whether the study was submitted to a trial registry (such as ClinicalTrials.gov), and whether this was done before or after study enrollment had started. Clearly stated ethical approval and trial registration information must be provided for all submissions. Explanations may be sought if the editors and reviewers believe that the study did not meet standards for ethical approval, patient consent, or trial registration. Other requests related to methods may ask to clarify how the primary and secondary aims outlined in the introduction were addressed in the analysis, and how the sample size was determined, whether based on a statistical power analysis or logistical considerations (e.g., how many patients could be recruited with available resources). When a statistical power analysis is performed, reviewers may ask for more detail about the assumptions of this analysis and any supporting data from pilot studies or previous publications.

Check the conclusions and limitations

Having revised the introduction, methods, and results, the authors should revise the discussion to make any changes to the conclusions required by new or different study findings. We recommend that authors start the discussion with a review of what the study found, and then discuss how the study findings relate to similar work that has been previously published. An excessively long discussion does not ensure that a study will be published and, in fact, may detract from the quality of the manuscript. For a scientific study (retrospective or prospective), the discussion should not read like a comprehensive review of the literature. Typically, the discussion of study limitations will be expanded in the revised manuscript to include additional study weaknesses pointed out by reviewers, acknowledge suggested changes that could not be made to the study methods, and mention other suggestions for future studies that would build on the current results or answer questions left unanswered by the current study. Reviewers may ask that the conclusions be more specific in addressing the primary aim or hypothesis of the study (stated in the introduction), but they may also encourage authors to go further afield in their discussion, connecting their findings to results from previous publications and describing how their findings support or challenge current clinical practice.

Writing the Response to Reviewers

As seen above, manuscript revision can require more writing and (re)analysis than even the initial submission. Therefore, the aim of the revision memo (response to reviewers) is to summarize for the editors and reviewers how each change addresses the concerns raised on the initial review. This document is handled differently by different journals; some require it to be uploaded as a separate file, others require that the revision memo be entered in a text-box during the online submission process, and still others require that the response to review be included in the cover letter for the resubmitted manuscript. Therefore, authors should pay close attention to the decision letter and its instructions as to how they should submit their response to reviewers and how they should refer to manuscript edits in the revision memo (e.g., by page number, by line number, or copying sections of the revised manuscript into the memo).

Typically, the reviewers' comments should be copied and entered in the response memo so that each comment is numbered and the response clearly listed after it, in a different font style or color. It is equally important to determine how the journal would like the changes tracked in the revised manuscript. Some journals will ask that the authors use the track-changes mode in the word processing software, whereas others may ask for changes to be highlighted or be added in a different color font. Deleted manuscript text may need to be shown in strike-through font or simply removed from the revised submission, depending on the journal. Journals may ask for two copies of the revised manuscript: one showing the changes and one in a clean format that is ready for copyediting.

A typical revision memo will include a short paragraph acknowledging the editorial decision and reviewer comments and briefly summarizing key changes made to the manuscript. This would be followed by a numbered list of comments from the editors and reviewers (as received in the decision letter), with the authors' response to each one. Although not all reviewers and editors submit their comments as a numbered list, the authors may want to break up long sections or paragraphs of the reviews into shorter, numbered comments, to separately describe how each one was addressed in the revision. The authors' responses need not be excessively ingratiating but should respect the reviewers' effort in evaluating the manuscript, and concisely explain what was changed or why a change was not or could not be made. Different reviewers may have conflicting recommendations for revision. This may be as simple as one asking for a more concise definition of a method while another asking for a more detailed explanation. With conflicting reviews, the authors may consider taking the recommendation that is endorsed in the editor's comments (if this is provided), the one that is best aligned with the study aims, or the one that best matches the methods and writing style used in other contemporary papers in the field; and explaining this rationale when responding to the reviewers.

What to Do with a Rejected Manuscript

Based on reviewer reports or their own judgment, editors may reject a manuscript with no option to resubmit. It is essential to read the decision letter closely as some journals will state that they cannot publish a manuscript in its current form but offer to consider a new submission of a substantially revised manuscript (“reject and resubmit,” as mentioned above, in contrast to “revise and resubmit”). When the manuscript is rejected with no option of resubmission, authors may appeal this decision, but this option is rarely exercised and may not change the editors' decision. Appeals are also generally only successful when made by experienced and recognized scholars in the field.

Unless the study is discovered to be so flawed as to preclude publication in any venue, authors will usually consider submitting it to another journal after the initial rejection. Taking a single rejection and tabling a manuscript without further submission is rarely a good option. It is possible that multiple rejections will precede an eventual acceptance for valuable work. Given the amount of time taken to devise, implement, and up a study, we encourage authors to consider resubmission to a new journal, if the study is well conceived and addresses an important problem or question. In this case, the criticisms in the initial review are not binding, but still worth the authors' consideration. Particularly, authors should address any major flaws in the study's approach and conclusions (distinct from reviewers' preferences for additional data analysis unrelated to the primary aims), and correct any factual, spelling, or grammatical errors prior to resubmission. Adding recommended secondary analyses could sometimes strengthen the next submission, although just as often, the reviewers at the next journal may find these additional analyses superfluous, and will have their own set of analyses to recommend.

Becoming a Reviewer

Like any complex skill, navigating the peer review process is best learned through repetition. Becoming a peer reviewer for scientific journals is an important way to hone this skill, as well as providing a service to the scientific community, and adding to one's academic credentials as an expert whose opinion is sought by journal editors. The most common entry point to becoming a reviewer is through scientific publication; the authors of published articles can be contacted by another journal to provide a review on a related submission. One's expertise in a specific area may be noted by the editor who performs a topic search of key words when looking for reviewers. Alternatively, editors and associate editors may call on colleagues who they know are recognized experts in a particular field. Academic mentorship is also important, as mentors may ask junior colleagues and faculty to help them with reviewing article submissions, or may pass their name along to journal editors to be considered for inclusion in the reviewer pool. Once one has successfully reviewed for a journal, they are frequently called upon to review other submissions, especially if their review was returned in a timely manner. Many journals will give a specific timeframe within which the review is to be completed, while others will not. In most cases, a response within 2–4 weeks is considered acceptable. Some journals have now started editorial fellowships that aim to provide an immersive experience in the peer review and publishing process for early-career scientists. Lastly, researchers wishing to become peer reviewers may contact journal editors themselves, or register reviewer accounts in journal online submission systems. Although the general structure of peer review reports is described above, more specific guidance on performing peer review is available in other publications.[ 10 , 11 ]

Peer Review Ethics

Authors, reviewers, and editors have a shared responsibility for the ethical conduct of peer review. This is necessary to sustain the professional and public trust in peer review, as a system of evaluation that is accurate, constructive, and free from bias. Recently reported ethics violations have included authors misrepresenting the identity of suggested reviewers, reviewers plagiarizing a manuscript sent to them for review or recommending its rejection and then conducting a similar study, and editors inappropriately pressuring authors to cite articles published in their journal.[ 12 , 13 , 14 ] Some journals and publishers have also been criticized for circumventing the peer review process for submitted manuscripts.[ 15 ] For reviewers, it is most important that they be unbiased and not have any hidden agendas or personal vendettas to settle. For authors, ethical conduct in peer review includes disclosing the study's ethics committee approval, trial registration, and consent process; disclosing any related or overlapping prior publications; disclosing any actual or potential conflicts of interest; and submitting the manuscript only to one journal. These requirements are typically stated in the journal's guidelines for authors, and may need to be acknowledged in the cover letter accompanying the manuscript. In responding to reviews, authors should also carefully consider whether their revisions still fall within the scope of the ethics committee approval for the study and the informed consent that was obtained, and whether the revised manuscript remains faithful to the aims and study design of any pre-registered trial protocol.

Scientific research is not complete until it is published, but not all research can or should be published. It falls to peer-review to determine the difference. By engaging with the process of peer review, authors can improve the quality of their work as well as gain confidence that it is published in a reputable medium. Furthermore, the fact that a study has been peer reviewed will increase its stature and potential for recognition. However, the peer review process does not assure this. Although responding to reviews can be challenging, we hope that the suggestions sketched out in this article will help authors plan their approach to manuscript revision and resubmission. We also encourage authors to participate in this process as reviewers, so that the labor of peer review is properly shared among the community of scientists.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• 1. Bornmann L. Scientific peer review. Ann Rev Inf Sci Technol. 2011;45:197–245. [ Google Scholar ]
• 2. Manchikanti L, Kaye AD, Boswell MV, Hirsch JA. Medical journal peer review: Process and bias. Pain Physician. 2015;18:E1–14. [ PubMed ] [ Google Scholar ]
• 3. Csiszar A. Peer review: Troubled from the start. Nature. 2016;532:306–8. doi: 10.1038/532306a. [ DOI ] [ PubMed ] [ Google Scholar ]
• 4. Kennefick D. Einstein versus the physical review. Phys Today. 2005;58:43–8. [ Google Scholar ]
• 5. Pagel PS, Hudetz JA. Scholarly productivity and national institutes of health funding of foundation for anesthesia education and research grant recipients: Insights from a bibliometric analysis. Anesthesiology. 2015;123:683–91. doi: 10.1097/ALN.0000000000000737. [ DOI ] [ PubMed ] [ Google Scholar ]
• 6. Geyer ED, Tumin D, Miller R, Cartabuke RS, Tobias JD. Progress to publication of survey research presented at anesthesiology society meetings. Paediatr Anaesth. 2018 doi: 10.1111/pan.13466. [Epub ahead of print] [ DOI ] [ PubMed ] [ Google Scholar ]
• 7. Callaway E. Open peer review finds more takers. Nature. 2016;539:343. doi: 10.1038/nature.2016.20969. [ DOI ] [ PubMed ] [ Google Scholar ]
• 8. Teixeira da Silva JA, Dobránszki J. Problems with traditional science publishing and finding a wider niche for post-publication peer review. Account Res. 2015;22:22–40. doi: 10.1080/08989621.2014.899909. [ DOI ] [ PubMed ] [ Google Scholar ]
• 9. Kirk AD, Malchesky PS, Shapiro R, Webber SA, Hirsch HH, Marty FM, et al. Introducing the Wiley transplant peer review network. Am J Transplant. 2016;16:2505–7. doi: 10.1111/ajt.13965. [ DOI ] [ PubMed ] [ Google Scholar ]
• 10. Spigt M, Arts IC. How to review a manuscript. J Clin Epidemiol. 2010;63:1385–90. doi: 10.1016/j.jclinepi.2010.09.001. [ DOI ] [ PubMed ] [ Google Scholar ]
• 11. McDermott S, Turk MA. The art of manuscript review - and manuscript development. Disabil Health J. 2017;10:1–2. doi: 10.1016/j.dhjo.2016.10.009. [ DOI ] [ PubMed ] [ Google Scholar ]
• 12. Qi X, Deng H, Guo X. Characteristics of retractions related to faked peer reviews: An overview. Postgrad Med J. 2017;93:499–503. doi: 10.1136/postgradmedj-2016-133969. [ DOI ] [ PubMed ] [ Google Scholar ]
• 13. Gross C. Scientific misconduct. Annu Rev Psychol. 2016;67:693–711. doi: 10.1146/annurev-psych-122414-033437. [ DOI ] [ PubMed ] [ Google Scholar ]
• 14. Ioannidis JP. A generalized view of self-citation: Direct, co-author, collaborative, and coercive induced self-citation. J Psychosom Res. 2015;78:7–11. doi: 10.1016/j.jpsychores.2014.11.008. [ DOI ] [ PubMed ] [ Google Scholar ]
• 15. Bohannon J. Who's afraid of peer review? Science. 2013;342:60–5. doi: 10.1126/science.2013.342.6154.342_60. [ DOI ] [ PubMed ] [ Google Scholar ]
• View on publisher site
• PDF (388.4 KB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections