systematic review research meaning

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

Formulate a research question
Develop a protocol
Search for all relevant studies
Apply the selection criteria
Extract the data
Synthesize the data
Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

Academic style
Vague sentences
Style consistency

See an example

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
Access to databases and journal archives. Often, your educational institution provides you with access.
Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

They minimize research bias by considering all available evidence and evaluating each study for bias.
Their methods are transparent , so they can be scrutinized by others.
They’re thorough : they summarize all available evidence.
They can be replicated and updated by others.

Systematic reviews also have a few cons .

They’re time-consuming .
They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

Allow you to more effectively communicate your research to other researchers and practitioners
Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

Population(s) or problem(s)
Intervention(s)
Comparison(s)

You can rearrange these four components to write your research question:

What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

Type of study design(s)
The population of patients with eczema
The intervention of probiotics
In comparison to no treatment, placebo , or non-probiotic treatment
The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
Randomized control trials, a type of study design

Their research question was:

What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

Background information : Provide the context of the research question, including why it’s important.
Research objective (s) : Rephrase your research question as an objective.
Selection criteria: State how you’ll decide which studies to include or exclude from your review.
Search strategy: Discuss your plan for finding studies.
Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
Handsearch: Conference proceedings and reference lists of articles
Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

Abstract : A summary of the review
Introduction : Including the rationale and objectives
Methods : Including the selection criteria, search method, data extraction method, and synthesis method
Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
Discussion : Including interpretation of the results and limitations of the review
Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Student’s t -distribution
Normal distribution
Null and Alternative Hypotheses
Chi square tests
Confidence interval
Quartiles & Quantiles
Cluster sampling
Stratified sampling
Data cleansing
Reproducibility vs Replicability
Peer review
Prospective cohort study

Research bias

Implicit bias
Cognitive bias
Placebo effect
Hawthorne effect
Hindsight bias
Affect heuristic
Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts , with an introduction , a main body, and a conclusion .

An annotated bibliography is a list of source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a paper .

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved June 29, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Systematic Reviews

What is a Systematic Review?

A systematic review is an evidence synthesis that uses explicit, reproducible methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Key Characteristics of a Systematic Review:

Generally, systematic reviews must have:

a clearly stated set of objectives with pre-defined eligibility criteria for studies
an explicit, reproducible methodology
a systematic search that attempts to identify all studies that would meet the eligibility criteria
an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias
a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

A meta-analysis is a systematic review that uses quantitative methods to synthesize and summarize the pooled data from included studies.

Additional Information

How-to Books
Beyond Health Sciences

Cochrane Handbook For Systematic Reviews of Interventions Provides guidance to authors for the preparation of Cochrane Intervention reviews. Chapter 6 covers searching for reviews.
Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care From The University of York Centre for Reviews and Dissemination: Provides practical guidance for undertaking evidence synthesis based on a thorough understanding of systematic review methodology. It presents the core principles of systematic reviewing, and in complementary chapters, highlights issues that are specific to reviews of clinical tests, public health interventions, adverse effects, and economic evaluations.
Cornell, Sytematic Reviews and Evidence Synthesis Beyond the Health Sciences Video series geared for librarians but very informative about searching outside medicine.
<< Previous: Getting Started
Next: Levels of Evidence >>
Getting Started
Levels of Evidence
Locating Systematic Reviews
Searching Systematically
Developing Answerable Questions
Identifying Synonyms & Related Terms
Using Truncation and Wildcards
Identifying Search Limits/Exclusion Criteria
Keyword vs. Subject Searching
Where to Search
Search Filters
Sensitivity vs. Precision
Core Databases
Other Databases
Clinical Trial Registries
Conference Presentations
Databases Indexing Grey Literature
Web Searching
Handsearching
Citation Indexes
Documenting the Search Process
Managing your Review

Research Support

Last Updated: Jun 6, 2024 9:14 AM
URL: https://guides.library.ucdavis.edu/systematic-reviews

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

Knowledge Base
Methodology
Systematic Review | Definition, Examples & Guide

Systematic Review | Definition, Examples & Guide

Published on 15 June 2022 by Shaun Turney . Revised on 17 October 2022.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

What is a systematic review, systematic review vs meta-analysis, systematic review vs literature review, systematic review vs scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias . The methods are repeatable , and the approach is formal and systematic:

Formulate a research question
Develop a protocol
Search for all relevant studies
Apply the selection criteria
Extract the data
Synthesise the data
Write and publish a report

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism, run a free check.

Systematic reviews often quantitatively synthesise the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
Access to databases and journal archives. Often, your educational institution provides you with access.
Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

They minimise research b ias by considering all available evidence and evaluating each study for bias.
Their methods are transparent , so they can be scrutinised by others.
They’re thorough : they summarise all available evidence.
They can be replicated and updated by others.

Systematic reviews also have a few cons .

They’re time-consuming .
They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

Allow you to more effectively communicate your research to other researchers and practitioners
Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

Population(s) or problem(s)
Intervention(s)
Comparison(s)

You can rearrange these four components to write your research question:

What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fourth component, the type of study design . In this case, the acronym is PICOT .

Type of study design(s)
The population of patients with eczema
The intervention of probiotics
In comparison to no treatment, placebo , or non-probiotic treatment
The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
Randomised control trials, a type of study design

Their research question was:

What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

Background information : Provide the context of the research question, including why it’s important.
Research objective(s) : Rephrase your research question as an objective.
Selection criteria: State how you’ll decide which studies to include or exclude from your review.
Search strategy: Discuss your plan for finding studies.
Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of grey literature.
Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
Handsearch: Conference proceedings and reference lists of articles
Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

You should apply the selection criteria in two phases:

Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram .

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
Your judgement of the quality of the evidence, including risk of bias .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.

Step 6: Synthesise the data

Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:

Narrative ( qualitative ): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
Quantitative : Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

Abstract : A summary of the review
Introduction : Including the rationale and objectives
Methods : Including the selection criteria, search method, data extraction method, and synthesis method
Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
Discussion : Including interpretation of the results and limitations of the review
Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

To familiarise yourself with the current state of knowledge on your topic
To ensure that you’re not just repeating what others have already done
To identify gaps in knowledge and unresolved problems that your research can address
To develop your theoretical framework and methodology
To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Turney, S. (2022, October 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 24 June 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/

Is this article helpful?

Shaun Turney

Other students also liked, what is a literature review | guide, template, & examples, exploratory research | definition, guide, & examples, what is peer review | types & examples.

Systematic Review

Library Help
What is a Systematic Review (SR)?
Steps of a Systematic Review
Framing a Research Question
Developing a Search Strategy
Searching the Literature
Managing the Process
Meta-analysis
Publishing your Systematic Review

Introduction to Systematic Review

Introduction
Types of literature reviews
Other Libguides
Systematic review as part of a dissertation
Tutorials & Guidelines & Examples from non-Medical Disciplines

A "high-level overview of primary research on a focused question" utilizing high-quality research evidence through:

Source: Kysh, Lynn (2013): Difference between a systematic review and a literature review. [figshare]. Available at:

Depending on your learning style, please explore the resources in various formats on the tabs above.

For additional tutorials, visit the SR Workshop Videos from UNC at Chapel Hill outlining each stage of the systematic review process.

Know the difference! Systematic review vs. literature review

It is common to confuse systematic and literature reviews as both are used to provide a summary of the existent literature or research on a specific topic. Even with this common ground, both types vary significantly. Please review the following chart (and its corresponding poster linked below) for a detailed explanation of each as well as the differences between each type of review.

Source: Kysh, L. (2013). What’s in a name? The difference between a systematic review and a literature review and why it matters. [Poster] Retrieved from .

Check the website from UNC at Chapel Hill,

Types of literature reviews along with associated methodologies

JBI Manual for Evidence Synthesis . Find definitions and methodological guidance.

- Systematic Reviews - Chapters 1-7

- Mixed Methods Systematic Reviews - Chapter 8

- Diagnostic Test Accuracy Systematic Reviews - Chapter 9

- Umbrella Reviews - Chapter 10

- Scoping Reviews - Chapter 11

- Systematic Reviews of Measurement Properties - Chapter 12

Systematic reviews vs scoping reviews -

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal , 26 (2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1 (28). htt p s://doi.org/ 10.1186/2046-4053-1-28

Munn, Z., Peters, M., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review ? Guidance for authors when choosing between a systematic or scoping review approach. BMC medical research methodology, 18 (1), 143. https://doi.org/10.1186/s12874-018-0611-x. Also, check out the Libguide from Weill Cornell Medicine for the differences between a systematic review and a scoping review and when to embark on either one of them.

Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: Exploring review types and associated information retrieval requirements . Health Information & Libraries Journal , 36 (3), 202–222. https://doi.org/10.1111/hir.12276

Temple University. Review Types . - This guide provides useful descriptions of some of the types of reviews listed in the above article.

UMD Health Sciences and Human Services Library. Review Types . - Guide describing Literature Reviews, Scoping Reviews, and Rapid Reviews.

Whittemore, R., Chao, A., Jang, M., Minges, K. E., & Park, C. (2014). Methods for knowledge synthesis: An overview. Heart & Lung: The Journal of Acute and Critical Care, 43 (5), 453–461. https://doi.org/10.1016/j.hrtlng.2014.05.014

Differences between a systematic review and other types of reviews

Armstrong, R., Hall, B. J., Doyle, J., & Waters, E. (2011). ‘ Scoping the scope ’ of a cochrane review. Journal of Public Health , 33 (1), 147–150. https://doi.org/10.1093/pubmed/fdr015

Kowalczyk, N., & Truluck, C. (2013). Literature reviews and systematic reviews: What is the difference? Radiologic Technology , 85 (2), 219–222.

White, H., Albers, B., Gaarder, M., Kornør, H., Littell, J., Marshall, Z., Matthew, C., Pigott, T., Snilstveit, B., Waddington, H., & Welch, V. (2020). Guidance for producing a Campbell evidence and gap map . Campbell Systematic Reviews, 16 (4), e1125. https://doi.org/10.1002/cl2.1125. Check also this comparison between evidence and gaps maps and systematic reviews.

Rapid Reviews Tutorials

Rapid Review Guidebook by the National Collaborating Centre of Methods and Tools (NCCMT)

Hamel, C., Michaud, A., Thuku, M., Skidmore, B., Stevens, A., Nussbaumer-Streit, B., & Garritty, C. (2021). Defining Rapid Reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews. Journal of clinical epidemiology , 129 , 74–85. https://doi.org/10.1016/j.jclinepi.2020.09.041

Image: by WeeblyTutorials

under the tab on the left side menu.

Müller, C., Lautenschläger, S., Meyer, G., & Stephan, A. (2017). Interventions to support people with dementia and their caregivers during the transition from home care to nursing home care: A systematic review . International Journal of Nursing Studies, 71 , 139–152. https://doi.org/10.1016/j.ijnurstu.2017.03.013
Bhui, K. S., Aslam, R. W., Palinski, A., McCabe, R., Johnson, M. R. D., Weich, S., … Szczepura, A. (2015). Interventions to improve therapeutic communications between Black and minority ethnic patients and professionals in psychiatric services: Systematic review . The British Journal of Psychiatry, 207 (2), 95–103. https://doi.org/10.1192/bjp.bp.114.158899
Rosen, L. J., Noach, M. B., Winickoff, J. P., & Hovell, M. F. (2012). Parental smoking cessation to protect young children: A systematic review and meta-analysis . Pediatrics, 129 (1), 141–152. https://doi.org/10.1542/peds.2010-3209

Scoping Review

Hyshka, E., Karekezi, K., Tan, B., Slater, L. G., Jahrig, J., & Wild, T. C. (2017). The role of consumer perspectives in estimating population need for substance use services: A scoping review . BMC Health Services Research, 171-14. https://doi.org/10.1186/s12913-017-2153-z
Olson, K., Hewit, J., Slater, L.G., Chambers, T., Hicks, D., Farmer, A., & ... Kolb, B. (2016). Assessing cognitive function in adults during or following chemotherapy: A scoping review . Supportive Care In Cancer, 24 (7), 3223-3234. https://doi.org/10.1007/s00520-016-3215-1
Pham, M. T., Rajić, A., Greig, J. D., Sargeant, J. M., Papadopoulos, A., & McEwen, S. A. (2014). A scoping review of scoping reviews: Advancing the approach and enhancing the consistency . Research Synthesis Methods, 5 (4), 371–385. https://doi.org/10.1002/jrsm.1123
Scoping Review Tutorial from UNC at Chapel Hill

Qualitative Systematic Review/Meta-Synthesis

Lee, H., Tamminen, K. A., Clark, A. M., Slater, L., Spence, J. C., & Holt, N. L. (2015). A meta-study of qualitative research examining determinants of children's independent active free play . International Journal Of Behavioral Nutrition & Physical Activity, 12 (5), 121-12. https://doi.org/10.1186/s12966-015-0165-9

Videos on systematic reviews


This video lecture explains in detail the steps necessary to conduct a systematic review (44 min.)	Here's a brief introduction to how to evaluate systematic reviews (16 min.)

Systematic Reviews: What are they? Are they right for my research? - 47 min. video recording with a closed caption option.

More training videos on systematic reviews:

from Yale University

(approximately 5-10 minutes each)

with Margaret Foster

(approximately 55 min each)

Books on Systematic Reviews

Books on Meta-analysis

University of Toronto Libraries - very detailed with good tips on the sensitivity and specificity of searches.
Monash University - includes an interactive case study tutorial.
Dalhousie University Libraries - a comprehensive How-To Guide on conducting a systematic review.

Guidelines for a systematic review as part of the dissertation

Guidelines for Systematic Reviews in the Context of Doctoral Education Background by University of Victoria (PDF)
Can I conduct a Systematic Review as my Master’s dissertation or PhD thesis? Yes, It Depends! by Farhad (blog)
What is a Systematic Review Dissertation Like? by the University of Edinburgh (50 min video)

Further readings on experiences of PhD students and doctoral programs with systematic reviews

Puljak, L., & Sapunar, D. (2017). Acceptance of a systematic review as a thesis: Survey of biomedical doctoral programs in Europe . Systematic Reviews , 6 (1), 253. https://doi.org/10.1186/s13643-017-0653-x

Perry, A., & Hammond, N. (2002). Systematic reviews: The experiences of a PhD Student . Psychology Learning & Teaching , 2 (1), 32–35. https://doi.org/10.2304/plat.2002.2.1.32

Daigneault, P.-M., Jacob, S., & Ouimet, M. (2014). Using systematic review methods within a Ph.D. dissertation in political science: Challenges and lessons learned from practice . International Journal of Social Research Methodology , 17 (3), 267–283. https://doi.org/10.1080/13645579.2012.730704

UMD Doctor of Philosophy Degree Policies

Before you embark on a systematic review research project, check the UMD PhD Policies to make sure you are on the right path. Systematic reviews require a team of at least two reviewers and an information specialist or a librarian. Discuss with your advisor the authorship roles of the involved team members. Keep in mind that the UMD Doctor of Philosophy Degree Policies (scroll down to the section, Inclusion of one's own previously published materials in a dissertation ) outline such cases, specifically the following:

" It is recognized that a graduate student may co-author work with faculty members and colleagues that should be included in a dissertation . In such an event, a letter should be sent to the Dean of the Graduate School certifying that the student's examining committee has determined that the student made a substantial contribution to that work. This letter should also note that the inclusion of the work has the approval of the dissertation advisor and the program chair or Graduate Director. The letter should be included with the dissertation at the time of submission. The format of such inclusions must conform to the standard dissertation format. A foreword to the dissertation, as approved by the Dissertation Committee, must state that the student made substantial contributions to the relevant aspects of the jointly authored work included in the dissertation."

by CommLab India

Cochrane Handbook for Systematic Reviews of Interventions - See Part 2: General methods for Cochrane reviews
Systematic Searches - Yale library video tutorial series
Using PubMed's Clinical Queries to Find Systematic Reviews - From the U.S. National Library of Medicine
Systematic reviews and meta-analyses: A step-by-step guide - From the University of Edinsburgh, Centre for Cognitive Ageing and Cognitive Epidemiology

by Vinova

Bioinformatics

Mariano, D. C., Leite, C., Santos, L. H., Rocha, R. E., & de Melo-Minardi, R. C. (2017). A guide to performing systematic literature reviews in bioinformatics . arXiv preprint arXiv:1707.05813.

Environmental Sciences

Collaboration for Environmental Evidence. 2018. Guidelines and Standards for Evidence synthesis in Environmental Management. Version 5.0 (AS Pullin, GK Frampton, B Livoreil & G Petrokofsky, Eds) www.environmentalevidence.org/information-for-authors .

Pullin, A. S., & Stewart, G. B. (2006). Guidelines for systematic review in conservation and environmental management. Conservation Biology, 20 (6), 1647–1656. https://doi.org/10.1111/j.1523-1739.2006.00485.x

Engineering Education

Borrego, M., Foster, M. J., & Froyd, J. E. (2014). Systematic literature reviews in engineering education and other developing interdisciplinary fields. Journal of Engineering Education, 103 (1), 45–76. https://doi.org/10.1002/jee.20038

Public Health

Hannes, K., & Claes, L. (2007). Learn to read and write systematic reviews: The Belgian Campbell Group . Research on Social Work Practice, 17 (6), 748–753. https://doi.org/10.1177/1049731507303106
McLeroy, K. R., Northridge, M. E., Balcazar, H., Greenberg, M. R., & Landers, S. J. (2012). Reporting guidelines and the American Journal of Public Health’s adoption of preferred reporting items for systematic reviews and meta-analyses . American Journal of Public Health, 102 (5), 780–784. https://doi.org/10.2105/AJPH.2011.300630
Pollock, A., & Berge, E. (2018). How to do a systematic review. International Journal of Stroke, 13 (2), 138–156. https://doi.org/10.1177/1747493017743796
Institute of Medicine. (2011). Finding what works in health care: Standards for systematic reviews . https://doi.org/10.17226/13059
Wanden-Berghe, C., & Sanz-Valero, J. (2012). Systematic reviews in nutrition: Standardized methodology . The British Journal of Nutrition, 107 Suppl 2, S3-7. https://doi.org/10.1017/S0007114512001432

Social Sciences

Bronson, D., & Davis, T. (2012). Finding and evaluating evidence: Systematic reviews and evidence-based practice (Pocket guides to social work research methods). Oxford: Oxford University Press.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide . Malden, MA: Blackwell Pub.
Cornell University Library Guide - Systematic literature reviews in engineering: Example: Software Engineering
Biolchini, J., Mian, P. G., Natali, A. C. C., & Travassos, G. H. (2005). Systematic review in software engineering . System Engineering and Computer Science Department COPPE/UFRJ, Technical Report ES, 679 (05), 45.
Biolchini, J. C., Mian, P. G., Natali, A. C. C., Conte, T. U., & Travassos, G. H. (2007). Scientific research ontology to support systematic review in software engineering . Advanced Engineering Informatics, 21 (2), 133–151.
Kitchenham, B. (2007). Guidelines for performing systematic literature reviews in software engineering . [Technical Report]. Keele, UK, Keele University, 33(2004), 1-26.
Weidt, F., & Silva, R. (2016). Systematic literature review in computer science: A practical guide . Relatórios Técnicos do DCC/UFJF , 1 .

by Day Translations

Resources for your writing

Academic Phrasebank - Get some inspiration and find some terms and phrases for writing your research paper
Oxford English Dictionary - Use to locate word variants and proper spelling
<< Previous: Library Help
Next: Steps of a Systematic Review >>
Last Updated: May 8, 2024 1:44 PM
URL: https://lib.guides.umd.edu/SR

Jump to navigation

Bahasa Malaysia

What are systematic reviews?

Watch this video from Cochrane Consumers and Communication to learn what systematic reviews are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone.

Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does.

Search our Plain Language Summaries of health evidence
Learn more about Cochrane and our work

Introduction to Systematic Reviews

Reference work entry
First Online: 20 July 2022
pp 2159–2177
Cite this reference work entry

Tianjing Li 3 ,
Ian J. Saldanha 4 &
Karen A. Robinson 5

268 Accesses

A systematic review identifies and synthesizes all relevant studies that fit prespecified criteria to answer a research question. Systematic review methods can be used to answer many types of research questions. The type of question most relevant to trialists is the effects of treatments and is thus the focus of this chapter. We discuss the motivation for and importance of performing systematic reviews and their relevance to trialists. We introduce the key steps in completing a systematic review, including framing the question, searching for and selecting studies, collecting data, assessing risk of bias in included studies, conducting a qualitative synthesis and a quantitative synthesis (i.e., meta-analysis), grading the certainty of evidence, and writing the systematic review report. We also describe how to identify systematic reviews and how to assess their methodological rigor. We discuss the challenges and criticisms of systematic reviews, and how technology and innovations, combined with a closer partnership between trialists and systematic reviewers, can help identify effective and safe evidence-based practices more quickly.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Get 10 units per month
Download Article/Chapter or Ebook
1 Unit = 1 Article or 1 Chapter
Cancel anytime
Available as PDF
Read on any device
Instant download
Own it forever
Available as EPUB and PDF
Durable hardcover edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

What Is the Difference Between a Systematic Review and a Meta-analysis?

Systematic Reviewing

AHRQ (2015) Methods guide for effectiveness and comparative effectiveness reviews. Available from https://effectivehealthcare.ahrq.gov/products/cer-methods-guide/overview . Accessed on 27 Oct 2019

Andersen MZ, Gülen S, Fonnes S, Andresen K, Rosenberg J (2020) Half of Cochrane reviews were published more than two years after the protocol. J Clin Epidemiol 124:85–93. https://doi.org/10.1016/j.jclinepi.2020.05.011

Article Google Scholar

Berkman ND, Lohr KN, Ansari MT, Balk EM, Kane R, McDonagh M, Morton SC, Viswanathan M, Bass EB, Butler M, Gartlehner G, Hartling L, McPheeters M, Morgan LC, Reston J, Sista P, Whitlock E, Chang S (2015) Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol 68(11):1312–1324

Borah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545. https://doi.org/10.1136/bmjopen-2016-012545

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JP, Oliver S (2014) How to increase value and reduce waste when research priorities are set. Lancet 383(9912):156–165. https://doi.org/10.1016/S0140-6736(13)62229-1

Clarke M, Chalmers I (1998) Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA 280(3):280–282

Cooper NJ, Jones DR, Sutton AJ (2005) The use of systematic reviews when designing studies. Clin Trials 2(3):260–264

Djulbegovic B, Kumar A, Magazin A, Schroen AT, Soares H, Hozo I, Clarke M, Sargent D, Schell MJ (2011) Optimism bias leads to inconclusive results-an empirical study. J Clin Epidemiol 64(6):583–593. https://doi.org/10.1016/j.jclinepi.2010.09.007

Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J (2017) Living systematic review network. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 91:23–30

Equator Network. Reporting guidelines for systematic reviews. Available from https://www.equator-network.org/?post_type=eq_guidelines&eq_guidelines_study_design=systematic-reviews-and-meta-analyses&eq_guidelines_clinical_specialty=0&eq_guidelines_report_section=0&s=+ . Accessed 9 Mar 2020

Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ, Akl EA, Beyene J, Chang S, Churchill R, Dearness K, Guyatt G, Lefebvre C, Liles B, Marshall R, Martínez García L, Mavergames C, Nasser M, Qaseem A, Sampson M, Soares-Weiser K, Takwoingi Y, Thabane L, Trivella M, Tugwell P, Welsh E, Wilson EC, Schünemann HJ (2016) Panel for updating guidance for systematic reviews (PUGs). When and how to update systematic reviews: consensus and checklist. BMJ 354:i3507. https://doi.org/10.1136/bmj.i3507 . Erratum in: BMJ 2016 Sep 06 354:i4853

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ (2011) GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4):383–394

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) (2019a) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester

Google Scholar

Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (2019b) Standards for the conduct of new Cochrane intervention reviews. In: JPT H, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R (eds) Methodological expectations of Cochrane intervention reviews. Cochrane, London

IOM (2011) Committee on standards for systematic reviews of comparative effectiveness research, board on health care services. In: Eden J, Levit L, Berg A, Morton S (eds) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC

Jonnalagadda SR, Goyal P, Huffman MD (2015) Automating data extraction in systematic reviews: a systematic review. Syst Rev 4:78

Krnic Martinic M, Pieper D, Glatt A, Puljak L (2019) Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol 19(1):203. Published 4 Nov 2019. https://doi.org/10.1186/s12874-019-0855-0

Lasserson TJ, Thomas J, Higgins JPT (2019) Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC (1992) Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 327(4):248–254

Lau J (2019) Editorial: systematic review automation thematic series. Syst Rev 8(1):70. Published 11 Mar 2019. https://doi.org/10.1186/s13643-019-0974-z

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100. https://doi.org/10.1371/journal.pmed.1000100

Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I (2016) Towards evidence based research. BMJ 355:i5440. https://doi.org/10.1136/bmj.i5440

Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC (2018) Machine learning for identifying randomized controlled trials: an evaluation and practitioner’s guide. Res Synth Methods 9(4):602–614. https://doi.org/10.1002/jrsm.1287

Michelson M, Reuter K (2019) The significant cost of systematic reviews and meta-analyses: a call for greater involvement of machine learning to assess the promise of clinical trials. Contemp Clin Trials Commun 16:100443. https://doi.org/10.1016/j.conctc.2019.100443 . Erratum in: Contemp Clin Trials Commun 2019 16:100450

Moher D, Liberati A, Tetzlaff J (2009) Altman DG; PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 151(4):264–269. W64

Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4(1):1. https://doi.org/10.1186/2046-4053-4-1

NIHR HTA Stage 1 guidance notes. Available from https://www.nihr.ac.uk/documents/hta-stage-1-guidance-notes/11743 ; Accessed 10 Mar 2020

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D (2016) Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 13(5):e1002028. https://doi.org/10.1371/journal.pmed.1002028

Page MJ, Higgins JPT, Sterne JAC (2019) Chapter 13: assessing risk of bias due to missing results in a synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ et al (eds) Cochrane handbook for systematic reviews of interventions, 2nd edn. Wiley, Chichester, pp 349–374

Chapter Google Scholar

Robinson KA (2009) Use of prior research in the justification and interpretation of clinical trials. Johns Hopkins University

Robinson KA, Goodman SN (2011) A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med 154(1):50–55. https://doi.org/10.7326/0003-4819-154-1-201101040-00007

Rouse B, Cipriani A, Shi Q, Coleman AL, Dickersin K, Li T (2016) Network meta-analysis for clinical practice guidelines – a case study on first-line medical therapies for primary open-angle glaucoma. Ann Intern Med 164(10):674–682. https://doi.org/10.7326/M15-2367

Saldanha IJ, Lindsley K, Do DV et al (2017) Comparison of clinical trial and systematic review outcomes for the 4 most prevalent eye diseases. JAMA Ophthalmol 135(9):933–940. https://doi.org/10.1001/jamaophthalmol.2017.2583

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA (2017) AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358:j4008. https://doi.org/10.1136/bmj.j4008

Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007) How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 147(4):224–233

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JP (2016) ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 355:i4919. https://doi.org/10.1136/bmj.i4919

Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, Emberson JR, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JPT (2019) RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 366:l4898. https://doi.org/10.1136/bmj.l4898

Thomas J, Kneale D, McKenzie JE, Brennan SE, Bhaumik S (2019) Chapter 2: determining the scope of the review and the questions it will address. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (eds) Cochrane handbook for systematic reviews of interventions version 6.0 (updated July 2019). Cochrane. Available from www.training.cochrane.org/handbook

USPSTF U.S. Preventive Services Task Force Procedure Manual (2017). Available from: https://www.uspreventiveservicestaskforce.org/uspstf/sites/default/files/inline-files/procedure-manual2017_update.pdf . Accessed 21 May 2020

Whitaker (2015) UCSF guides: systematic review: when will i be finished? https://guides.ucsf.edu/c.php?g=375744&p=3041343 , Accessed 13 May 2020

Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J (2016) Churchill R; ROBIS group. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 69:225–234. https://doi.org/10.1016/j.jclinepi.2015.06.005

Download references

Author information

Authors and affiliations.

Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

Tianjing Li

Department of Health Services, Policy, and Practice and Department of Epidemiology, Brown University School of Public Health, Providence, RI, USA

Ian J. Saldanha

Department of Medicine, Johns Hopkins University, Baltimore, MD, USA

Karen A. Robinson

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianjing Li .

Editor information

Editors and affiliations.

Department of Surgery, Division of Surgical Oncology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Steven Piantadosi

Department of Epidemiology, School of Public Health, Johns Hopkins University, Baltimore, MD, USA

Curtis L. Meinert

Section Editor information

Department of Epidemiology, University of Colorado Denver Anschutz Medical Campus, Aurora, CO, USA

The Johns Hopkins Center for Clinical Trials and Evidence Synthesis, Johns Hopkins University, Baltimore, MD, USA

Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

Rights and permissions

Reprints and permissions

Copyright information

About this entry

Cite this entry.

Li, T., Saldanha, I.J., Robinson, K.A. (2022). Introduction to Systematic Reviews. In: Piantadosi, S., Meinert, C.L. (eds) Principles and Practice of Clinical Trials. Springer, Cham. https://doi.org/10.1007/978-3-319-52636-2_194

Download citation

DOI : https://doi.org/10.1007/978-3-319-52636-2_194

Published : 20 July 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-52635-5

Online ISBN : 978-3-319-52636-2

eBook Packages : Mathematics and Statistics Reference Module Computer Science and Engineering

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

Duke NetID Login
919.660.1100
Duke Health Badge: 24-hour access
Accounts & Access
Databases, Journals & Books
Request & Reserve
Training & Consulting
Request Articles & Books
Renew Online
Reserve Spaces
Reserve a Locker
Study & Meeting Rooms
Course Reserves
Pay Fines/Fees
Recommend a Purchase
Access From Off Campus
Building Access
Computers & Equipment
Wifi Access
My Accounts
Mobile Apps
Known Access Issues
Report an Access Issue
All Databases
Article Databases
Basic Sciences
Clinical Sciences
Dissertations & Theses
Drugs, Chemicals & Toxicology
Grants & Funding
Interprofessional Education
Non-Medical Databases
Search for E-Journals
Search for Print & E-Journals
Search for E-Books
Search for Print & E-Books
E-Book Collections
Biostatistics
Global Health
MBS Program
Medical Students
MMCi Program
Occupational Therapy
Path Asst Program
Physical Therapy
Researchers
Community Partners

Conducting Research

Archival & Historical Research
Black History at Duke Health
Data Analytics & Viz Software
Data: Find and Share
Evidence-Based Practice
NIH Public Access Policy Compliance
Publication Metrics
Qualitative Research
Searching Animal Alternatives

Systematic Reviews

Test Instruments

Using Databases

JCR Impact Factors
Web of Science

Finding & Accessing

COVID-19: Core Clinical Resources
Health Literacy
Health Statistics & Data
Library Orientation

Writing & Citing

Creating Links
Getting Published
Reference Mgmt
Scientific Writing

Meet a Librarian

Request a Consultation
Find Your Liaisons
Register for a Class
Request a Class
Self-Paced Learning

Search Services

Literature Search
Systematic Review
Animal Alternatives (IACUC)
Research Impact

Citation Mgmt

Other Software

Scholarly Communications

About Scholarly Communications
Publish Your Work
Measure Your Research Impact
Engage in Open Science
Libraries and Publishers
Directions & Maps
Floor Plans

Library Updates

Annual Snapshot
Conference Presentations
Contact Information
Gifts & Donations

What is a Systematic Review?

Types of Reviews
Manuals and Reporting Guidelines
Our Service
1. Assemble Your Team
2. Develop a Research Question
3. Write and Register a Protocol
4. Search the Evidence
5. Screen Results
6. Assess for Quality and Bias
7. Extract the Data
8. Write the Review
Additional Resources
Finding Full-Text Articles

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. The key characteristics of a systematic review are:

a clearly defined question with inclusion and exclusion criteria;
a rigorous and systematic search of the literature;
two phases of screening (blinded, at least two independent screeners);
data extraction and management;
analysis and interpretation of results;
risk of bias assessment of included studies;
and report for publication.

Medical Center Library & Archives Presentations

The following presentation is a recording of the Getting Started with Systematic Reviews workshop (4/2022), offered by the Duke Medical Center Library & Archives. A NetID/pw is required to access the tutorial via Warpwire.

<< Previous: Overview
Next: Types of Reviews >>
Last Updated: Jun 18, 2024 9:41 AM
URL: https://guides.mclibrary.duke.edu/sysreview
Duke Health
Duke University
Duke Libraries
Medical Center Archives
Duke Directory
Seeley G. Mudd Building
10 Searle Drive
[email protected]

Library Services

UCL LIBRARY SERVICES

Guides and databases
Library skills
Systematic reviews

What are systematic reviews?

Types of systematic reviews
Formulating a research question
Identifying studies
Searching databases
Describing and appraising studies
Synthesis and systematic maps
Software for systematic reviews
Online training and support
Live and face to face training
Individual support
Further help

Systematic reviews are a type of literature review of research which require equivalent standards of rigour as primary research. They have a clear, logical rationale that is reported to the reader of the review. They are used in research and policymaking to inform evidence-based decisions and practice. They differ from traditional literature reviews particularly in the following elements of conduct and reporting.

Systematic reviews:

use explicit and transparent methods
are a piece of research following a standard set of stages
are accountable, replicable and updateable
involve users to ensure a review is relevant and useful.

For example, systematic reviews (like all research) should have a clear research question, and the perspective of the authors in their approach to addressing the question is described. There are clearly described methods on how each study in a review was identified, how that study was appraised for quality and relevance and how it is combined with other studies in order to address the review question. A systematic review usually involves more than one person in order to increase the objectivity and trustworthiness of the reviews methods and findings.

Research protocols for systematic reviews may be peer-reviewed and published or registered in a suitable repository to help avoid duplication of reviews and for comparisons to be made with the final review and the planned review.

History of systematic reviews to inform policy (EPPI-Centre)
Six reasons why it is important to be systematic (EPPI-Centre)
Evidence Synthesis International (ESI): Position Statement Describes the issues, principles and goals in synthesising research evidence to inform policy, practice and decisions

A 'non-systematic review' might use some of the same methods as systematic reviews, such as systematic approaches to identify studies or quality appraise the literature. There may be times when this approach can be useful. In a student dissertation, for example, there may not be the time to be fully systematic in a review of the literature if this is only one small part of the thesis. In other types of research, there may also be a need to obtain a quick and not necessarily thorough overview of a literature to inform some other work (including a systematic review). Another example, is where policymakers, or other people using research findings, want to make quick decisions and there is no systematic review available to help them. They have a choice of gaining a rapid overview of the research literature or not having any research evidence to help their decision-making.

Just like any other piece of research, the methods used to undertake any literature review should be carefully planned to justify the conclusions made.

Finding out about different types of systematic reviews and the methods used for systematic reviews, and reading both systematic and other types of review will help to understand some of the differences.

Typically, a systematic review addresses a focussed, structured research question in order to inform understanding and decisions on an area. (see the Formulating a research question section for examples).

Sometimes systematic reviews ask a broad research question, and one strategy to achieve this is the use of several focussed sub-questions each addressed by sub-components of the review.

Another strategy is to develop a map to describe the type of research that has been undertaken in relation to a research question. Some maps even describe over 2,000 papers, while others are much smaller. One purpose of a map is to help choose a sub-set of studies to explore more fully in a synthesis. There are also other purposes of maps: see the box on systematic evidence maps for further information.

Reporting standards specify minimum elements that need to go into the reporting of a review. The reporting standards refer mainly to methodological issues but they are not as detailed or specific as critical appraisal for the methodological standards of conduct of a review.

A number of organisations have developed specific guidelines and standards for both the conducting and reporting on systematic reviews in different topic areas.

PRISMA PRISMA is a reporting standard and is an acronym for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. The Key Documents section of the PRISMA website links to a checklist, flow diagram and explanatory notes. PRISMA is less useful for certain types of reviews, including those that are iterative.
eMERGe eMERGe is a reporting standard that has been developed for meta-ethnographies, a qualitative synthesis method.
ROSES: RepOrting standards for Systematic Evidence Syntheses Reporting standards, including forms and flow diagram, designed specifically for systematic reviews and maps in the field of conservation and environmental management.

Useful books about systematic reviews

Systematic approaches to a successful literature review

An introduction to systematic reviews

Cochrane handbook for systematic reviews of interventions

Systematic reviews: crd's guidance for undertaking reviews in health care.

Finding what works in health care: Standards for systematic reviews

Systematic Reviews in the Social Sciences

Meta-analysis and research synthesis.

Research Synthesis and Meta-Analysis

Doing a Systematic Review

Literature reviews.

What is a literature review?
Why are literature reviews important?
<< Previous: Systematic reviews
Next: Types of systematic reviews >>
Last Updated: May 30, 2024 4:38 PM
URL: https://library-guides.ucl.ac.uk/systematic-reviews

Help us improve our Library guides with this 5 minute survey . We appreciate your feedback!

UOW Library
Key guides for researchers

Systematic Review

What is a systematic review.

Five other types of systematic review
How is a literature review different?
Search tips for systematic reviews
Controlled vocabularies
Grey literature
Transferring your search
Documenting your results
Support & contact

A systematic review is an authoritative account of existing evidence using reliable, objective, thorough and reproducible research practices.

It is a method of making sense of large bodies of information and contributes to the answers to questions about what works and what doesn't.

Systematic reviews map areas of uncertainty and identify where little or no relevant research has been done, but where new studies are needed.

It is a good idea to familiarise yourself with the systematic review process before beginning your review. You can do this by searching for other systematic reviews to look at as examples, by reading a glossary of commonly used terms , and by learning how to distinguish between types of systematic review.

Characteristics of a systematic review

Some characteristics, or features, of systematic reviews are:

Clearly stated set of objectives with pre-defined eligibility criteria
Explicit, reproducible methodology
A systematic search that attempts to identify all studies that would meet the eligibility criteria
Assesses the validity of the findings, for example assessing the risk of bias
Systematic presentation and synthesis of the findings of the included studies. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 6).

Watch this video from the Cochrane Library for more information about systematic reviews.

Previous: Introduction to systematic reviews
Next: Five other types of systematic review
Last Updated: Jun 20, 2024 12:04 PM
URL: https://uow.libguides.com/systematic-review

Insert research help text here

LIBRARY RESOURCES

Library homepage

Library SEARCH

A-Z Databases

STUDY SUPPORT

Academic Skills Centre

Referencing and citing

Digital Skills Hub

MORE UOW SERVICES

UOW homepage

Student support and wellbeing

IT Services

On the lands that we study, we walk, and we live, we acknowledge and respect the traditional custodians and cultural knowledge holders of these lands.

1.2.2 What is a systematic review?

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993) . The key characteristics of a systematic review are:

a clearly stated set of objectives with pre-defined eligibility criteria for studies;

an explicit, reproducible methodology;

a systematic search that attempts to identify all studies that would meet the eligibility criteria;

an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and

a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies (Glass 1976). By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3 ). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.

A-Z Publications

Annual Review of Psychology

Volume 70, 2019, review article, how to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses.

Andy P. Siddaway 1 , Alex M. Wood 2 , and Larry V. Hedges 3
View Affiliations Hide Affiliations Affiliations: 1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected] 2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom 3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected]
Vol. 70:747-770 (Volume publication date January 2019) https://doi.org/10.1146/annurev-psych-010418-102803
First published as a Review in Advance on August 08, 2018
Copyright © 2019 by Annual Reviews. All rights reserved

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Article metrics loading...

Full text loading...

Literature Cited

APA Publ. Commun. Board Work. Group J. Artic. Rep. Stand. 2008 . Reporting standards for research in psychology: Why do we need them? What might they be?. Am. Psychol . 63 : 848– 49 [Google Scholar]
Baumeister RF 2013 . Writing a literature review. The Portable Mentor: Expert Guide to a Successful Career in Psychology MJ Prinstein, MD Patterson 119– 32 New York: Springer, 2nd ed.. [Google Scholar]
Baumeister RF , Leary MR 1995 . The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychol. Bull. 117 : 497– 529 [Google Scholar]
Baumeister RF , Leary MR 1997 . Writing narrative literature reviews. Rev. Gen. Psychol. 3 : 311– 20 Presents a thorough and thoughtful guide to conducting narrative reviews. [Google Scholar]
Bem DJ 1995 . Writing a review article for Psychological Bulletin. Psychol . Bull 118 : 172– 77 [Google Scholar]
Borenstein M , Hedges LV , Higgins JPT , Rothstein HR 2009 . Introduction to Meta-Analysis New York: Wiley Presents a comprehensive introduction to meta-analysis. [Google Scholar]
Borenstein M , Higgins JPT , Hedges LV , Rothstein HR 2017 . Basics of meta-analysis: I 2 is not an absolute measure of heterogeneity. Res. Synth. Methods 8 : 5– 18 [Google Scholar]
Braver SL , Thoemmes FJ , Rosenthal R 2014 . Continuously cumulating meta-analysis and replicability. Perspect. Psychol. Sci. 9 : 333– 42 [Google Scholar]
Bushman BJ 1994 . Vote-counting procedures. The Handbook of Research Synthesis H Cooper, LV Hedges 193– 214 New York: Russell Sage Found. [Google Scholar]
Cesario J 2014 . Priming, replication, and the hardest science. Perspect. Psychol. Sci. 9 : 40– 48 [Google Scholar]
Chalmers I 2007 . The lethal consequences of failing to make use of all relevant evidence about the effects of medical treatments: the importance of systematic reviews. Treating Individuals: From Randomised Trials to Personalised Medicine PM Rothwell 37– 58 London: Lancet [Google Scholar]
Cochrane Collab. 2003 . Glossary Rep., Cochrane Collab. London: http://community.cochrane.org/glossary Presents a comprehensive glossary of terms relevant to systematic reviews. [Google Scholar]
Cohn LD , Becker BJ 2003 . How meta-analysis increases statistical power. Psychol. Methods 8 : 243– 53 [Google Scholar]
Cooper HM 2003 . Editorial. Psychol. Bull. 129 : 3– 9 [Google Scholar]
Cooper HM 2016 . Research Synthesis and Meta-Analysis: A Step-by-Step Approach Thousand Oaks, CA: Sage, 5th ed.. Presents a comprehensive introduction to research synthesis and meta-analysis. [Google Scholar]
Cooper HM , Hedges LV , Valentine JC 2009 . The Handbook of Research Synthesis and Meta-Analysis New York: Russell Sage Found, 2nd ed.. [Google Scholar]
Cumming G 2014 . The new statistics: why and how. Psychol. Sci. 25 : 7– 29 Discusses the limitations of null hypothesis significance testing and viable alternative approaches. [Google Scholar]
Earp BD , Trafimow D 2015 . Replication, falsification, and the crisis of confidence in social psychology. Front. Psychol. 6 : 621 [Google Scholar]
Etz A , Vandekerckhove J 2016 . A Bayesian perspective on the reproducibility project: psychology. PLOS ONE 11 : e0149794 [Google Scholar]
Ferguson CJ , Brannick MT 2012 . Publication bias in psychological science: prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychol. Methods 17 : 120– 28 [Google Scholar]
Fleiss JL , Berlin JA 2009 . Effect sizes for dichotomous data. The Handbook of Research Synthesis and Meta-Analysis H Cooper, LV Hedges, JC Valentine 237– 53 New York: Russell Sage Found, 2nd ed.. [Google Scholar]
Garside R 2014 . Should we appraise the quality of qualitative research reports for systematic reviews, and if so, how. Innovation 27 : 67– 79 [Google Scholar]
Hedges LV , Olkin I 1980 . Vote count methods in research synthesis. Psychol. Bull. 88 : 359– 69 [Google Scholar]
Hedges LV , Pigott TD 2001 . The power of statistical tests in meta-analysis. Psychol. Methods 6 : 203– 17 [Google Scholar]
Higgins JPT , Green S 2011 . Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0 London: Cochrane Collab. Presents comprehensive and regularly updated guidelines on systematic reviews. [Google Scholar]
John LK , Loewenstein G , Prelec D 2012 . Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 23 : 524– 32 [Google Scholar]
Juni P , Witschi A , Bloch R , Egger M 1999 . The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 282 : 1054– 60 [Google Scholar]
Klein O , Doyen S , Leys C , Magalhães de Saldanha da Gama PA , Miller S et al. 2012 . Low hopes, high expectations: expectancy effects and the replicability of behavioral experiments. Perspect. Psychol. Sci. 7 : 6 572– 84 [Google Scholar]
Lau J , Antman EM , Jimenez-Silva J , Kupelnick B , Mosteller F , Chalmers TC 1992 . Cumulative meta-analysis of therapeutic trials for myocardial infarction. N. Engl. J. Med. 327 : 248– 54 [Google Scholar]
Light RJ , Smith PV 1971 . Accumulating evidence: procedures for resolving contradictions among different research studies. Harvard Educ. Rev. 41 : 429– 71 [Google Scholar]
Lipsey MW , Wilson D 2001 . Practical Meta-Analysis London: Sage Comprehensive and clear explanation of meta-analysis. [Google Scholar]
Matt GE , Cook TD 1994 . Threats to the validity of research synthesis. The Handbook of Research Synthesis H Cooper, LV Hedges 503– 20 New York: Russell Sage Found. [Google Scholar]
Maxwell SE , Lau MY , Howard GS 2015 . Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?. Am. Psychol. 70 : 487– 98 [Google Scholar]
Moher D , Hopewell S , Schulz KF , Montori V , Gøtzsche PC et al. 2010 . CONSORT explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 340 : c869 [Google Scholar]
Moher D , Liberati A , Tetzlaff J , Altman DG PRISMA Group. 2009 . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339 : 332– 36 Comprehensive reporting guidelines for systematic reviews. [Google Scholar]
Morrison A , Polisena J , Husereau D , Moulton K , Clark M et al. 2012 . The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28 : 138– 44 [Google Scholar]
Nelson LD , Simmons J , Simonsohn U 2018 . Psychology's renaissance. Annu. Rev. Psychol. 69 : 511– 34 [Google Scholar]
Noblit GW , Hare RD 1988 . Meta-Ethnography: Synthesizing Qualitative Studies Newbury Park, CA: Sage [Google Scholar]
Olivo SA , Macedo LG , Gadotti IC , Fuentes J , Stanton T , Magee DJ 2008 . Scales to assess the quality of randomized controlled trials: a systematic review. Phys. Ther. 88 : 156– 75 [Google Scholar]
Open Sci. Collab. 2015 . Estimating the reproducibility of psychological science. Science 349 : 943 [Google Scholar]
Paterson BL , Thorne SE , Canam C , Jillings C 2001 . Meta-Study of Qualitative Health Research: A Practical Guide to Meta-Analysis and Meta-Synthesis Thousand Oaks, CA: Sage [Google Scholar]
Patil P , Peng RD , Leek JT 2016 . What should researchers expect when they replicate studies? A statistical view of replicability in psychological science. Perspect. Psychol. Sci. 11 : 539– 44 [Google Scholar]
Rosenthal R 1979 . The “file drawer problem” and tolerance for null results. Psychol. Bull. 86 : 638– 41 [Google Scholar]
Rosnow RL , Rosenthal R 1989 . Statistical procedures and the justification of knowledge in psychological science. Am. Psychol. 44 : 1276– 84 [Google Scholar]
Sanderson S , Tatt ID , Higgins JP 2007 . Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int. J. Epidemiol. 36 : 666– 76 [Google Scholar]
Schreiber R , Crooks D , Stern PN 1997 . Qualitative meta-analysis. Completing a Qualitative Project: Details and Dialogue JM Morse 311– 26 Thousand Oaks, CA: Sage [Google Scholar]
Shrout PE , Rodgers JL 2018 . Psychology, science, and knowledge construction: broadening perspectives from the replication crisis. Annu. Rev. Psychol. 69 : 487– 510 [Google Scholar]
Stroebe W , Strack F 2014 . The alleged crisis and the illusion of exact replication. Perspect. Psychol. Sci. 9 : 59– 71 [Google Scholar]
Stroup DF , Berlin JA , Morton SC , Olkin I , Williamson GD et al. 2000 . Meta-analysis of observational studies in epidemiology (MOOSE): a proposal for reporting. JAMA 283 : 2008– 12 [Google Scholar]
Thorne S , Jensen L , Kearney MH , Noblit G , Sandelowski M 2004 . Qualitative meta-synthesis: reflections on methodological orientation and ideological agenda. Qual. Health Res. 14 : 1342– 65 [Google Scholar]
Tong A , Flemming K , McInnes E , Oliver S , Craig J 2012 . Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med. Res. Methodol. 12 : 181– 88 [Google Scholar]
Trickey D , Siddaway AP , Meiser-Stedman R , Serpell L , Field AP 2012 . A meta-analysis of risk factors for post-traumatic stress disorder in children and adolescents. Clin. Psychol. Rev. 32 : 122– 38 [Google Scholar]
Valentine JC , Biglan A , Boruch RF , Castro FG , Collins LM et al. 2011 . Replication in prevention science. Prev. Sci. 12 : 103– 17 [Google Scholar]
Article Type: Review Article

Most Read This Month

Most cited most cited rss feed, job burnout, executive functions, social cognitive theory: an agentic perspective, on happiness and human potentials: a review of research on hedonic and eudaimonic well-being, sources of method bias in social science research and recommendations on how to control it, mediation analysis, missing data analysis: making it work in the real world, grounded cognition, personality structure: emergence of the five-factor model, motivational beliefs, values, and goals.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
J Family Med Prim Care
v.2(1); Jan-Mar 2013

Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare

S. gopalakrishnan.

Department of Community Medicine, SRM Medical College, Hospital and Research Centre, Kattankulathur, Tamil Nadu, India

P. Ganeshkumar

Healthcare decisions for individual patients and for public health policies should be informed by the best available research evidence. The practice of evidence-based medicine is the integration of individual clinical expertise with the best available external clinical evidence from systematic research and patient's values and expectations. Primary care physicians need evidence for both clinical practice and for public health decision making. The evidence comes from good reviews which is a state-of-the-art synthesis of current evidence on a given research question. Given the explosion of medical literature, and the fact that time is always scarce, review articles play a vital role in decision making in evidence-based medical practice. Given that most clinicians and public health professionals do not have the time to track down all the original articles, critically read them, and obtain the evidence they need for their questions, systematic reviews and clinical practice guidelines may be their best source of evidence. Systematic reviews aim to identify, evaluate, and summarize the findings of all relevant individual studies over a health-related issue, thereby making the available evidence more accessible to decision makers. The objective of this article is to introduce the primary care physicians about the concept of systematic reviews and meta-analysis, outlining why they are important, describing their methods and terminologies used, and thereby helping them with the skills to recognize and understand a reliable review which will be helpful for their day-to-day clinical practice and research activities.

Introduction

Evidence-based healthcare is the integration of best research evidence with clinical expertise and patient values. Green denotes, “Using evidence from reliable research, to inform healthcare decisions, has the potential to ensure best practice and reduce variations in healthcare delivery.” However, incorporating research into practice is time consuming, and so we need methods of facilitating easy access to evidence for busy clinicians.[ 1 ] Ganeshkumar et al . mentioned that nearly half of the private practitioners in India were consulting more than 4 h per day in a locality,[ 2 ] which explains the difficulty of them in spending time in searching evidence during consultation. Ideally, clinical decision making ought to be based on the latest evidence available. However, to keep abreast with the continuously increasing number of publications in health research, a primary healthcare professional would need to read an insurmountable number of articles every day, covered in more than 13 million references and over 4800 biomedical and health journals in Medline alone. With the view to address this challenge, the systematic review method was developed. Systematic reviews aim to inform and facilitate this process through research synthesis of multiple studies, enabling increased and efficient access to evidence.[ 1 , 3 , 4 ]

Systematic reviews and meta-analyses have become increasingly important in healthcare settings. Clinicians read them to keep up-to-date with their field and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research and some healthcare journals are moving in this direction.[ 5 ]

This article is intended to provide an easy guide to understand the concept of systematic reviews and meta-analysis, which has been prepared with the aim of capacity building for general practitioners and other primary healthcare professionals in research methodology and day-to-day clinical practice.

The purpose of this article is to introduce readers to:

The two approaches of evaluating all the available evidence on an issue i.e., systematic reviews and meta-analysis,
Discuss the steps in doing a systematic review,
Introduce the terms used in systematic reviews and meta-analysis,
Interpret results of a meta-analysis, and
The advantages and disadvantages of systematic review and meta-analysis.

Application

What is the effect of antiviral treatment in dengue fever? Most often a primary care physician needs to know convincing answers to questions like this in a primary care setting.

To find out the solutions or answers to a clinical question like this, one has to refer textbooks, ask a colleague, or search electronic database for reports of clinical trials. Doctors need reliable information on such problems and on the effectiveness of large number of therapeutic interventions, but the information sources are too many, i.e., nearly 20,000 journals publishing 2 million articles per year with unclear or confusing results. Because no study, regardless of its type, should be interpreted in isolation, a systematic review is generally the best form of evidence.[ 6 ] So, the preferred method is a good summary of research reports, i.e., systematic reviews and meta-analysis, which will give evidence-based answers to clinical situations.

There are two fundamental categories of research: Primary research and secondary research. Primary research is collecting data directly from patients or population, while secondary research is the analysis of data already collected through primary research. A review is an article that summarizes a number of primary studies and may draw conclusions on the topic of interest which can be traditional (unsystematic) or systematic.

Terminologies

Systematic review.

A systematic review is a summary of the medical literature that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue. It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and random errors.[ 7 ] To this end, systematic reviews may or may not include a statistical synthesis called meta-analysis, depending on whether the studies are similar enough so that combining their results is meaningful.[ 8 ] Systematic reviews are often called overviews.

The evidence-based practitioner, David Sackett, defines the following terminologies.[ 3 ]

Review: The general term for all attempts to synthesize the results and conclusions of two or more publications on a given topic.
Overview: When a review strives to comprehensively identify and track down all the literature on a given topic (also called “systematic literature review”).
Meta-analysis: A specific statistical strategy for assembling the results of several studies into a single estimate.

Systematic reviews adhere to a strict scientific design based on explicit, pre-specified, and reproducible methods. Because of this, when carried out well, they provide reliable estimates about the effects of interventions so that conclusions are defensible. Systematic reviews can also demonstrate where knowledge is lacking. This can then be used to guide future research. Systematic reviews are usually carried out in the areas of clinical tests (diagnostic, screening, and prognostic), public health interventions, adverse (harm) effects, economic (cost) evaluations, and how and why interventions work.[ 9 ]

Cochrane reviews

Cochrane reviews are systematic reviews undertaken by members of the Cochrane Collaboration which is an international not-for-profit organization that aims to help people to make well-informed decisions about healthcare by preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of healthcare interventions.

Cochrane Primary Health Care Field is a systematic review of primary healthcare research on prevention, treatment, rehabilitation, and diagnostic test accuracy. The overall aim and mission of the Primary Health Care Field is to promote the quality, quantity, dissemination, accessibility, applicability, and impact of Cochrane systematic reviews relevant to people who work in primary care and to ensure proper representation in the interests of primary care clinicians and consumers in Cochrane reviews and review groups, and in other entities. This field would serve to coordinate and promote the mission of the Cochrane Collaboration within the primary healthcare disciplines, as well as ensuring that primary care perspectives are adequately represented within the Collaboration.[ 10 ]

Meta-analysis

A meta-analysis is the combination of data from several independent primary studies that address the same question to produce a single estimate like the effect of treatment or risk factor. It is the statistical analysis of a large collection of analysis and results from individual studies for the purpose of integrating the findings.[ 11 ] The term meta-analysis has been used to denote the full range of quantitative methods for research reviews.[ 12 ] Meta-analyses are studies of studies.[ 13 ] Meta-analysis provides a logical framework to a research review where similar measures from comparable studies are listed systematically and the available effect measures are combined wherever possible.[ 14 ]

The fundamental rationale of meta-analysis is that it reduces the quantity of data by summarizing data from multiple resources and helps to plan research as well as to frame guidelines. It also helps to make efficient use of existing data, ensuring generalizability, helping to check consistency of relationships, explaining data inconsistency, and quantifies the data. It helps to improve the precision in estimating the risk by using explicit methods.

Therefore, “systematic review” will refer to the entire process of collecting, reviewing, and presenting all available evidence, while the term “meta-analysis” will refer to the statistical technique involved in extracting and combining data to produce a summary result.[ 15 ]

Steps in doing systematic reviews/meta-analysis

Following are the six fundamental essential steps while doing systematic review and meta-analysis.[ 16 ]

Define the question

This is the most important part of systematic reviews/meta-analysis. The research question for the systematic reviews may be related to a major public health problem or a controversial clinical situation which requires acceptable intervention as a possible solution to the present healthcare need of the community. This step is most important since the remaining steps will be based on this.

Reviewing the literature

This can be done by going through scientific resources such as electronic database, controlled clinical trials registers, other biomedical databases, non-English literatures, “gray literatures” (thesis, internal reports, non–peer-reviewed journals, pharmaceutical industry files), references listed in primary sources, raw data from published trials and other unpublished sources known to experts in the field. Among the available electronic scientific database, the popular ones are PUBMED, MEDLINE, and EMBASE.

Sift the studies to select relevant ones

To select the relevant studies from the searches, we need to sift through the studies thus identified. The first sift is pre-screening, i.e., to decide which studies to retrieve in full, and the second sift is selection which is to look again at these studies and decide which are to be included in the review. The next step is selecting the eligible studies based on similar study designs, year of publication, language, choice among multiple articles, sample size or follow-up issues, similarity of exposure, and or treatment and completeness of information.

It is necessary to ensure that the sifting includes all relevant studies like the unpublished studies (desk drawer problem), studies which came with negative conclusions or were published in non-English journals, and studies with small sample size.

Assess the quality of studies

The steps undertaken in evaluating the study quality are early definition of study quality and criteria, setting up a good scoring system, developing a standard form for assessment, calculating quality for each study, and finally using this for sensitivity analysis.

For example, the quality of a randomized controlled trial can be assessed by finding out the answers to the following questions:

Was the assignment to the treatment groups really random?
Was the treatment allocation concealed?
Were the groups similar at baseline in terms of prognostic factors?
Were the eligibility criteria specified?
Were the assessors, the care provider, and the patient blinded?
Were the point estimates and measure of variability presented for the primary outcome measure?
Did the analyses include intention-to-treat analysis?

Calculate the outcome measures of each study and combine them

We need a standard measure of outcome which can be applied to each study on the basis of its effect size. Based on their type of outcome, following are the measures of outcome: Studies with binary outcomes (cured/not cured) have odds ratio, risk ratio; studies with continuous outcomes (blood pressure) have means, difference in means, standardized difference in means (effect sizes); and survival or time-to-event data have hazard ratios.

Combining studies

Homogeneity of different studies can be estimated at a glance from a forest plot (explained below). For example, if the lower confidence interval of every trial is below the upper of all the others, i.e., the lines all overlap to some extent, then the trials are homogeneous. If some lines do not overlap at all, these trials may be said to be heterogeneous.

The definitive test for assessing the heterogeneity of studies is a variant of Chi-square test (Mantel–Haenszel test). The final step is calculating the common estimate and its confidence interval with the original data or with the summary statistics from all the studies. The best estimate of treatment effect can be derived from the weighted summary statistics of all studies which will be based on weighting to sample size, standard errors, and other summary statistics. Log scale is used to combine the data to estimate the weighting.

Interpret results: Graph

The results of a meta-analysis are usually presented as a graph called forest plot because the typical forest plots appear as forest of lines. It provides a simple visual presentation of individual studies that went into the meta-analysis at a glance. It shows the variation between the studies and an estimate of the overall result of all the studies together.

Forest plot

Meta-analysis graphs can principally be divided into six columns [ Figure 1 ]. Individual study results are displayed in rows. The first column (“study”) lists the individual study IDs included in the meta-analysis; usually the first author and year are displayed. The second column relates to the intervention groups and the third column to the control groups. The fourth column visually displays the study results. The line in the middle is called “the line of no effect.” The weight (in %) in the fifth column indicates the weighting or influence of the study on the overall results of the meta-analysis of all included studies. The higher the percentage weight, the bigger the box, the more influence the study has on the overall results. The sixth column gives the numerical results for each study (e.g., odds ratio or relative risk and 95% confidence interval), which are identical to the graphical display in the fourth column. The diamond in the last row of the graph illustrates the overall result of the meta-analysis.[ 4 ]

An external file that holds a picture, illustration, etc.
Object name is JFMPC-2-9-g001.jpg

Interpretation of meta-analysis[ 4 ]

Thus, the horizontal lines represent individual studies. Length of line is the confidence interval (usually 95%), squares on the line represent effect size (risk ratio) for the study, with area of the square being the study size (proportional to weight given) and position as point estimate (relative risk) of the study.[ 7 ]

For example, the forest plot of the effectiveness of dexamethasone compared with placebo in preventing the recurrence of acute severe migraine headache in adults is shown in Figure 2 .[ 17 ]

An external file that holds a picture, illustration, etc.
Object name is JFMPC-2-9-g002.jpg

Forest plot of the effectiveness of dexamethasone compared with placebo in preventing the recurrence of acute severe migraine headache in adults[ 17 ]

The overall effect is shown as diamond where the position toward the center represents pooled point estimate, the width represents estimated 95% confidence interval for all studies, and the black plain line vertically in the middle of plot is the “line of no effect” (e.g., relative risk = 1).

Therefore, when examining the results of a systematic reviews/meta-analysis, the following questions should be kept in mind:

Heterogeneity among studies may make any pooled estimate meaningless.
The quality of a meta-analysis cannot be any better than the quality of the studies it is summarizing.
An incomplete search of the literature can bias the findings of a meta-analysis.
Make sure that the meta-analysis quantifies the size of the effect in units that you can understand.

Subgroup analysis and sensitivity analysis

Subgroup analysis looks at the results of different subgroups of trials, e.g., by considering trials on adults and children separately. This should be planned at the protocol stage itself which is based on good scientific reasoning and is to be kept to a minimum.

Sensitivity analysis is used to determine how results of a systematic review/meta-analysis change by fiddling with data, for example, what is the implication if the exclusion criteria or excluded unpublished studies or weightings are assigned differently. Thus, after the analysis, if changing makes little or no difference to the overall results, the reviewer's conclusions are robust. If the key findings disappear, then the conclusions need to be expressed more cautiously.

Advantages of Systematic Reviews

Systematic reviews have specific advantages because of using explicit methods which limit bias, draw reliable and accurate conclusions, easily deliver required information to healthcare providers, researchers, and policymakers, help to reduce the time delay in the research discoveries to implementation, improve the generalizability and consistency of results, generation of new hypotheses about subgroups of the study population, and overall they increase precision of the results.[ 18 ]

Limitations in Systematic Reviews/Meta-analysis

As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers’ ability to assess the strengths and weaknesses of those reviews.[ 5 ]

Even though systematic review and meta-analysis are considered the best evidence for getting a definitive answer to a research question, there are certain inherent flaws associated with it, such as the location and selection of studies, heterogeneity, loss of information on important outcomes, inappropriate subgroup analyses, conflict with new experimental data, and duplication of publication.

Publication Bias

Publication bias results in it being easier to find studies with a “positive” result.[ 19 ] This occurs particularly due to inappropriate sifting of the studies where there is always a tendency towards the studies with positive (significant) outcomes. This effect occurs more commonly in systematic reviews/meta-analysis which need to be eliminated.

The quality of reporting of systematic reviews is still not optimal. In a recent review of 300 systematic reviews, few authors reported assessing possible publication bias even though there is overwhelming evidence both for its existence and its impact on the results of systematic reviews. Even when the possibility of publication bias is assessed, there is no guarantee that systematic reviewers have assessed or interpreted it appropriately.[ 20 ]

To overcome certain limitations mentioned above, the Cochrane reviews are currently reported in a format where at the end of every review, findings are summarized in the author's point of view and also give an overall picture of the outcome by means of plain language summary. This is found to be much helpful to understand the existing evidence about the topic more easily by the reader.

A systematic review is an overview of primary studies which contains an explicit statement of objectives, materials, and methods, and has been conducted according to explicit and reproducible methodology. A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way. Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the reviews were valid and reliable.

High-quality systematic reviews and meta-analyses take great care to find all relevant studies, critically assess each study, synthesize the findings from individual studies in an unbiased manner, and present balanced important summary of findings with due consideration of any flaws in the evidence. Systematic review and meta-analysis is a way of summarizing research evidence, which is generally the best form of evidence, and hence positioned at the top of the hierarchy of evidence.

Systematic reviews can be very useful decision-making tools for primary care/family physicians. They objectively summarize large amounts of information, identifying gaps in medical research, and identifying beneficial or harmful interventions which will be useful for clinicians, researchers, and even for public and policymakers.

Source of Support: Nil

Conflict of Interest: None declared.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Easy guide to conducting a systematic review

Affiliations.

1 Discipline of Child and Adolescent Health, University of Sydney, Sydney, New South Wales, Australia.
2 Department of Nephrology, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
3 Education Department, The Children's Hospital at Westmead, Sydney, New South Wales, Australia.
PMID: 32364273
DOI: 10.1111/jpc.14853

A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a predefined search strategy to identify and appraise all available published literature on a specific topic. The meticulous nature of the systematic review research methodology differentiates a systematic review from a narrative review (literature review or authoritative review). This paper provides a brief step by step summary of how to conduct a systematic review, which may be of interest for clinicians and researchers.

Keywords: research; research design; systematic review.

PubMed Disclaimer

Publication types

Search in MeSH

Related information

Linkout - more resources, full text sources.

Ovid Technologies, Inc.

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Evidence Synthesis and Systematic Reviews

Question Formulation

Systematic Reviews

Rapid reviews, scoping reviews.

Other Review Types
Resources for Reviews by Discipline and Type
Tools for Evidence Synthesis
Grey Literature

Definition : A systematic review is a summary of research results (evidence) that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue. It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and errors.

When to use : If you want to identify, appraise, and synthesize all available research that is relevant to a particular question with reproduceable search methods.

Limitations : It requires extensive time and a team

Resources :

Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare
The 8 stages of a systematic review
Determining the scope of the review and the questions it will address
Reporting the review

Definition : Rapid reviews are a form of evidence synthesis that may provide more timely information for decision making compared with standard systematic reviews

When to use : When you want to evaluate new or emerging research topics using some systematic review methods at a faster pace

Limitations : It is not as rigorous or as thorough as a systematic review and therefore may be more likely to be biased

Cochrane guidance for rapid reviews
Steps for conducting a rapid review
Expediting systematic reviews: methods and implications of rapid reviews

Definition : Scoping reviews are often used to categorize or group existing literature in a given field in terms of its nature, features, and volume.

When to use : Label body of literature with relevance to time, location (e.g. country or context), source (e.g. peer-reviewed or grey literature), and origin (e.g. healthcare discipline or academic field) It also is used to clarify working definitions and conceptual boundaries of a topic or field or to identify gaps in existing literature/research

Limitations : More citations to screen and takes as long or longer than a systematic review. Larger teams may be required because of the larger volumes of literature. Different screening criteria and process than a systematic review

PRISMA-ScR for scoping reviews
JBI Updated methodological guidance for the conduct of scoping reviews
JBI Manual: Scoping Reviews (2020)
Equator Network-Current Best Practices for the Conduct of Scoping Reviews
<< Previous: Question Formulation
Next: Other Review Types >>
Last Updated: Jun 11, 2024 10:02 AM
URL: https://guides.temple.edu/systematicreviews

Temple University

University libraries.

See all library locations

Library Directory
Locations and Directions
Frequently Called Numbers

Need help? Email us at [email protected]

Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Systematic Reviews

Describes what is involved with conducting a systematic review of the literature for evidence-based public health and how the librarian is a partner in the process.

Several CDC librarians have special training in conducting literature searches for systematic reviews. Literature searches for systematic reviews can take a few weeks to several months from planning to delivery.

Fill out a search request form here or contact the Stephen B. Thacker CDC Library by email [email protected] or telephone 404-639-1717.

Campbell Collaboration

Cochrane Collaboration

Eppi Centre

Joanna Briggs Institute

McMaster University

PRISMA Statement

Systematic Reviews – CRD’s Guide

Systematic Reviews of Health Promotion and Public Health Interventions

The Guide to Community Preventive Services

Look for systematic reviews that have already been published.

To ensure that the work has not already been done.
To provides examples of search strategies for your topic

Look in PROSPERO for registered systematic reviews.

Search Cochrane and CRD-York for systematic reviews.

Search filter for finding systematic reviews in PubMed

Other search filters to locate systematic reviews

A systematic review attempts to collect and analyze all evidence that answers a specific question. The question must be clearly defined and have inclusion and exclusion criteria. A broad and thorough search of the literature is performed and a critical analysis of the search results is reported and ultimately provides a current evidence-based answer to the specific question.

Time: According to Cochrane , it takes 18 months on average to complete a Systematic Review.

The average systematic review from beginning to end requires 18 months of work. “…to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months…” ( Cochrane Collaboration )

Review Team: Team Members at minimum…

Content expert
2 reviewers
1 tie breaker
1 statistician (meta-analysis)
1 economist if conducting an economic analysis
*1 librarian (expert searcher) trained in systematic reviews

“Expert searchers are an important part of the systematic review team, crucial throughout the review process-from the development of the proposal and research question to publication.” ( McGowan & Sampson, 2005 )

*Ask your librarian to write a methods section regarding the search methods and to give them co-authorship. You may also want to consider providing a copy of one or all of the search strategies used in an appendix.

The Question to Be Answered: A clearly defined and specific question or questions with inclusion and exclusion criteria.

Written Protocol: Outline the study method, rationale, key questions, inclusion and exclusion criteria, literature searches, data abstraction and data management, analysis of quality of the individual studies, synthesis of data, and grading of the evidience for each key question.

Literature Searches: Search for any systematic reviews that may already answer the key question(s). Next, choose appropriate databases and conduct very broad, comprehensive searches. Search strategies must be documented so that they can be duplicated. The librarian is integral to this step of the process. Before your librarian creates a search strategy and starts searching in earnest you should write a detailed PICO question , determine the inclusion and exclusion criteria for your study, run a preliminary search, and have 2-4 articles that already fit the criteria for your review.

What is searched depends on the topic of the review but should include…

At least 3 standard medical databases like PubMed/Medline, CINAHL, Embase, etc..
At least 2 grey literature resources like Clinicaltrials.gov, COS Conference Papers Index, Grey Literature Report, etc…

Citation Management: EndNote is a bibliographic management tools that assist researchers in managing citations. The Stephen B. Thacker CDC Library oversees the site license for EndNote.

To request installation: The library provides EndNote to CDC staff under a site-wide license. Please use the ITSO Software Request Tool (SRT) and submit a request for the latest version (or upgraded version) of EndNote. Please be sure to include the computer name for the workstation where you would like to have the software installed.

EndNote Training: CDC Library offers training on EndNote on a regular basis – both a basic and advanced course. To view the course descriptions and upcoming training dates, please visit the CDC Library training page .

For assistance with EndNote software, please contact [email protected]

Vendor Support and Services: EndNote – Support and Services (Thomson Reuters) EndNote – Tutorials and Live Online Classes (Thomson Reuters)

Getting Articles:

Articles can be obtained using DocExpress or by searching the electronic journals at the Stephen B. Thacker CDC Library.

IOM Standards for Systematic Reviews: Standard 3.1: Conduct a comprehensive systematic search for evidence

The goal of a systematic review search is to maximize recall and precision while keeping results manageable. Recall (sensitivity) is defined as the number of relevant reports identified divided by the total number of relevant reports in existence. Precision (specificity) is defined as the number of relevant reports identified divided by the total number of reports identified.

Issues to consider when creating a systematic review search:

All concepts are included in the strategy
All appropriate subject headings are used
Appropriate use of explosion
Appropriate use of subheadings and floating subheadings
Use of natural language (text words) in addition to controlled vocabulary terms
Use of appropriate synonyms, acronyms, etc.
Truncation and spelling variation as appropriate
Appropriate use of limits such as language, years, etc.
Field searching, publication type, author, etc.
Boolean operators used appropriately
Line errors: when searches are combined using line numbers, be sure the numbers refer to the searches intended
Check indexing of relevant articles
Search strategy adapted as needed for multiple databases
Cochrane Handbook: Searching for Studies See Part 2, Chapter 6

A step-by-step guide to systematically identify all relevant animal studies

Materials listed in these guides are selected to provide awareness of quality public health literature and resources. A material’s inclusion does not necessarily represent the views of the U.S. Department of Health and Human Services (HHS), the Public Health Service (PHS), or the Centers for Disease Control and Prevention (CDC), nor does it imply endorsement of the material’s methods or findings. HHS, PHS, and CDC assume no responsibility for the factual accuracy of the items presented. The selection, omission, or content of items does not imply any endorsement or other position taken by HHS, PHS, and CDC. Opinion, findings, and conclusions expressed by the original authors of items included in these materials, or persons quoted therein, are strictly their own and are in no way meant to represent the opinion or views of HHS, PHS, or CDC. References to publications, news sources, and non-CDC Websites are provided solely for informational purposes and do not imply endorsement by HHS, PHS, or CDC.

Exit Notification / Disclaimer Policy

The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
You will be subject to the destination website's privacy policy when you follow the link.
CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
Write for Us
BMJ Journals

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

A high-quality systematic review is described as the most reliable source of evidence to guide clinical practice. The purpose of a systematic review is to deliver a meticulous summary of all the available primary research in response to a research question. A systematic review uses all the existing research and is sometime called ‘secondary research’ (research on research). They are often required by research funders to establish the state of existing knowledge and are frequently used in guideline development. Systematic review findings are often used within the …

Competing interests None.

Read the full text or download the PDF:

Teaching and Research guides

Systematic reviews.

Starting the review

Systematic review - Definition

Evidence based practice, systematic review vs. literature review, types of reviews.

Research question
Plan your search
Sources to search
Search example
Screen and analyse
Guides and software
Further help

A systematic review is a type of secondary evidence that summarises research that has already been published.

Systematic reviews, according to Wright, et al. , are defined as a:

“review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review.”

What are Systematic Reviews? (3:23 min) by Cochrane ( YouTube )

When conducting a systematic review it is important to ask a question that can be answered through use of evidence, rather than subjective judgment. In evidence based practice, systematic reviews are considered one of the highest levels of information.

Pyramid of evidence based practice. Case reports, case studies, cohort studies, randomized control trials and systematic reviews at the top of the pyramid because they have stronger evidence and fewer articles

Image: What's In a Name?: the Difference Between a Systematic Review and a Literature Review and Why It Matters (pdf) by Lynn Kysh / CC-BY 4.0

A systematic review may sometimes include a meta-analysis . A meta-analysis is the use of statistical methods to combine data from studies included in a systematic review. Not all systematic reviews include a meta-analysis.

Diagram of circles showing all review articles as the outer circle, systematic reviews as a much smaller circle within that and meta-analyses as an even smaller circle within the systemaic reviews circle

Image: What's In a Name?: the Difference Between a Systematic Review and a Literature Review and Why It Matters (pdf) by Lynn Kysh / CC-BY 4.0

A systematic review is a type of literature review. The differences between a systematic review and a literature review are outlined in the following table.

Systematic reviews differ from literature reviews in terms of defintion, goals, the question, components, number of authors, requirements and value.

Image: What's In a Name?: the Difference Between a Systematic Review and a Literature Review and Why It Matters (pdf) by Lynn Kysh / CC-BY 4.0

There are a number of different types of reviews, following is a brief description of some review types.

Literature review - A literature review, also known as a narrative review, is the most common review type. Generally, it provides an examination of current literature of a topic area. The completeness and comprehensiveness of the literature included is subjective. Searches are usually limited and do not include quality assessment of the material. Typically, the synthesis is narrative, and analysis may be chronological, conceptual, thematic, etc. The library guide Literature reviews provides guidance on starting a literature review, including resources, techniques and approaches to searching the literature and writing the review.
Systematic review - A systematic review seeks to systematically search for, appraise, and synthesise research evidence, adhering to guidelines on the conduct of a review. Aims for exhaustive, comprehensive searching. Quality assessment is carried out. Typically, narrative with tabular accompaniment. Synthesis outlines what is known and recommendations for practice, as well as recommendations for future research. The library guide Systematic reviews outlines the process for conducting a systematic review.
Meta-analysis - A meta-analysis is a technique that statistically combines the results of quantitative studies to provide a more precise effect of the results. Aims for exhaustive, comprehensive searching. May use a funnel plot to assess completeness. Quality assessment is carried out. Synthesis is graphical and tabular with narrative commentary. Numerical analysis of measures of effect are provided. The library guide Systematic reviews briefly refers to meta-analyses.
Scoping review - A scoping review is a preliminary assessment of potential size and scope of available research literature. Aims to identify the nature and extent of research evidence by characterising quantity and quality of literature, perhaps by study design. Completeness of searching determined by time/scope constraints. No formal quality assessment is carried out. Synthesis is typically tabular with some narrative commentary. Attempts to specify a viable review.
Rapid review - A rapid review is an assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research. Completeness of searching determined by time constraints. Time‐limited formal quality assessment. Synthesis is typically tabular with some narrative commentary.

Systematic Reviews: What is a systematic review?

What is a systematic review.

Planning your review
Additional resources

A systematic review is a tightly structured literature review that focuses on a topic with strict research parameters. The methodology used to collect research has to be consistent in order to reduce misinterpretation and misrepresentation of the data.

To help you understand and conduct your systematic review we have produce a number of posts to help you:

Getting started with your systematic review
Introduction to Health Science databases
Planning your systematic review
Using frameworks to structure your search
Advanced search: making use of Boolean operators
Systematic review: organising your keywords and subject terms
Making use of MeSH and Suggested Subject Terms

You can access these and more from the Specialist Library Support online resources page .

What is a Systematic literature search?

A systematic literature search is a literature review on a database (such as Medline) which demonstrates that you have compiled a list of appropriate search terms and includes the structure of your search history which provides the evidence on which your assignment is based.

This is a less rigorous process than a systematic review. A systematic review usually covers a wider scope; you would be expected to look at all the available research in the area in question. For example, you would be expected to visit the Library if articles were only held in hard copy format, and where necessary obtain articles not held by the Library via the Inter-Library Loan service .

You may be told that you need to conduct a systematic review when in fact you just need to perform a literature search in a systematic manner.

If you are unsure about the differences between a systematic review and a literature review take a look at this guide: What’s in a Name? The difference between a Systematic Review and a Literature Review and Why it Matters .

<< Previous: Home
Next: Planning your review >>
Last Updated: Feb 6, 2024 3:18 PM
URL: https://subjects.library.manchester.ac.uk/systematic-reviews

Open access
Published: 22 June 2024

Considering planetary health in health guidelines and health technology assessments: a scoping review protocol

Thomas Piggott ORCID: orcid.org/0000-0003-1643-5386 1 , 2 ,
Maheen Raja 1 ,
Charlotte T. J. Michels 3 ,
Alina Herrmann 4 , 5 ,
Karolina Anna Scahill 6 , 7 ,
Andrea J. Darzi 1 , 8 ,
Laura Jewell 2 ,
KM Saif-Ur-Rahman 9 , 10 ,
Hendrik Napierala 11 ,
Ruben Heuer 12 ,
Rebecca L. Morgan 1 , 13 ,
Grigorios I. Leontiadis 14 ,
Ignacio Neumann 1 , 15 , 16 ,
Holger Schünemann 1 , 17 &
Fiona A. Miller 18

Systematic Reviews volume 13 , Article number: 163 ( 2024 ) Cite this article

327 Accesses

1 Altmetric

Metrics details

This protocol outlines a scoping review with the objective of identifying and exploring planetary health considerations within existing health guidelines and health technology assessments (HTA). The insights gained from this review will serve as a basis for shaping future Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) guidance on planetary health.

We will adhere to the JBI methodology for scoping reviews. We will conduct a comprehensive search and screening of results in all languages across various databases including MEDLINE, EMBASE, CINAHL, Global Health, Health Systems Evidence, Greenfile, and Environmental Issues. Additionally, we will supplement this search with resources such as the GIN library, BIGG database, Epistemonikos , GRADE guidelines repository, GRADEpro Guideline Development Tool Database, MAGICapp, NICE website, WHO websites, and a manual exploration of unpublished relevant documents using Google incognito mode. Two independent reviewers will screen and assess the full texts of identified documents according to the eligibility criteria. The following information from each full text will be extracted: document title; first author’s name; publication year; language; document type; document as a guideline or HTA; the topic/discipline; document purpose/study objective; developing/sponsoring organization; the country in which the study/guideline/HTA report was conducted; definition of planetary health or related concept provided; types of planetary health experts engaged; study methods; suggested methods to assess planetary health; use of secondary data on planetary health outcomes; description for use of life cycle assessment; description for assessing the quality of life cycle; population/intended audience; interventions; category; applicable planetary health boundaries; consideration of social justice/global equity; phase of intervention in life cycle related to planetary health addressed; the measure of planetary health impact; impact on biodiversity/land use; one health/animal welfare mention; funding; and conflict of interest. Data analysis will involve a combination of descriptive statistics and directed content analysis, with results presented in a narrative format and displayed in tables and graphs.

The final review results will be submitted to open-access peer-reviewed journals for publication when they become available. The research findings will also be disseminated at relevant planetary health conferences and workshops.

Systematic review registration

Open Science Framework ( https://osf.io/3jmsa ).

Peer Review reports

Methodological developments in health guidelines and technology assessments have achieved notable progress over the past decades, supporting health decision-making, practice, and the population for which the guidance serves. However, improvements in health status and health care delivery have caused overexploitation of our planet’s resources, accompanied by pollution and the disturbance of the Earth’s vital systems [ 1 ]. These unintended consequences have driven climate change and impacted planetary health. Health system interventions make significant contributions to greenhouse gas (GHG) emissions: 5.2% of global GHGs [ 2 ], up to 10% of GHGs in the USA [ 3 ], and other negative consequences for the planet’s health [ 4 ].

There is increasing interest in planetary health as demonstrated by new journals and fields of research. This work builds on a long-standing history of considering environmental and sustainability-related considerations [ 5 ]. Planetary health extends this work to argue that human health, animal health, and the planet’s health are inextricably linked [ 6 ]. In particular, the use of Life Cycle Assessments (LCA) to assess the environmental impacts of health interventions is growing [ 7 ]. However, the term planetary health has only grown substantially within opinion pieces, rather than in published original research [ 8 ]. To date, there has been little attention paid to the integration of planetary health considerations in health technology assessments (HTAs) and health guideline decision-making. This lack of consideration and negligence of the health system, and society more broadly, towards planetary health fails to recognize the interconnectedness between human health, animal health, and the planet’s health [ 6 ]. As the climate crisis accelerates it is becoming increasingly clear that health guidelines and HTAs should consider planetary health [ 9 ].

Global guideline developers are progressively employing the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) methodology in the pursuit of guideline development. This approach, recognized for its reliability and logical underpinnings, serves as a robust framework for transitioning from evidence to the establishment of recommendations [ 10 ]. The GRADE approach yields comprehensive summaries of evidence (accompanied by evaluations of evidence certainty) and graded recommendations (entailing evaluations of recommendation strength and comprehensive evidence certainty). The GRADE approach for developing recommendations using the evidence-to-decision (EtD) framework addresses a broad range of criteria including benefits, harms, the balance of effects, certainty of evidence, resources required, cost-effectiveness, equity, acceptability, and feasibility. The EtD framework allows flexibility in the criteria considered to inform decision-making by either adding or broadening existing criteria (e.g., considering political and health system factors within acceptability and feasibility) [ 11 ], modifying criteria based on user perspective (e.g., clinical, individual, diagnostic), or by truncating the criteria [ 12 ] considered during the decision-making process. However, the current EtD criteria do not explicitly prompt consideration of planetary health.

Guidelines have recently begun incorporating environmental and planetary health considerations [ 13 , 14 ]. By addressing planetary health, HTA reporters and guideline developers will be supported to assess the potential differential impacts of health interventions more comprehensively on both human health and the environment. This integration may prompt guideline developers and HTA reporters to proactively address and mitigate negative planetary health effects resulting from healthcare practices, such as the overuse of single-use plastics, carbon emissions from transportation, or other environmentally harmful impacts. This scoping review will thoroughly investigate and identify planetary health considerations in current health guidelines and HTAs. It will ultimately inform the development of future GRADE guidance on consideration of planetary health to inform decision-making [ 15 ].

The proposed scoping review will follow the methodology outlined by the JBI (formerly Joanna Briggs Institute) for conducting scoping reviews [ 16 , 17 ] and will adhere to the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping reviews (PRISMA-ScR [ 18 ]). Recent updates of PRISMA 2020 [ 19 ], specifically the scoping review modifications as detailed in the scoping review chapter of the JBI manual on evidence synthesis [ 20 ], will also be taken into consideration (Additional file 1).

The primary purpose of this protocol is to predefine the review’s objectives, review questions, eligibility criteria, methodologies, and reporting guidelines, to ensure transparency. The protocol functions as a plan for the scoping review and is designed to mitigate potential reporting biases. Any deviations from the established protocol during the review will be clearly addressed and explained within the complete scoping review [ 20 ]. The protocol was registered in the Open Science Framework ( https://osf.io/3jmsa ).

A collaborative effort was initiated, bringing together a team of global experts specializing in guideline methodology and planetary health, with the purpose of contributing to this scoping review. To enhance our application of the JBI scoping review guidance and PRISMA-ScR, and to refine our screening, study selection, and data extraction, pilot testing phases were undertaken at each step. The pilot phase aimed to achieve the following objectives: (a) evaluate and enhance the methods outlined in the protocol; (b) establish and train a team of reviewers (c) establish essential group procedures; (d) create and enhance the required tools; (e) fine-tune the process and content of data extraction; and (f) ascertain the scope of our work. During the pilot, we employed an adapted approach, and after its conclusion, we further refined the methods for the proposed scoping review. This section outlines the fundamental methods employed in the proposed scoping review.

Review question

Our goal will be to address the following review questions:

What are dimensions of planetary health that have been considered in health guidelines and HTAs?

How have guideline development methodologies suggested that health guidelines or HTAs consider the environment, climate change, or planetary health?

What are the methods that health guidelines and HTAs have used to incorporate evidence and assess the certainty of the evidence of planetary health outcomes?

Eligibility criteria

According to The Lancet definition, planetary health is defined as “The achievement of the highest attainable standard of health, wellbeing, and equity worldwide through judicious attention to the human systems—political, economic, and social—that shape the future of humanity and the Earth's natural systems that define the safe environmental limits within which humanity can flourish. Put simply, planetary health is the health of human civilization and the state of the natural systems on which it depends” [ 8 ]. The intricate mechanisms underpinning the linkage between human health and the surrounding natural systems remain multifaceted and occasionally elusive. This discourse shifts its focal point primarily towards outcomes that transcend human-centric considerations. The concept of focus here will predominantly be on non-human-centric outcomes because past HTAs and health guidelines have historically concentrated on direct human outcomes. Thus, the imperative emerges to place substantial emphasis on the well-being of animals and the equilibrium of natural systems as measured by the planetary boundaries concept, with the recognition that the ultimate preservation of human health is fundamentally interconnected between all [ 21 , 22 , 23 ].

We will include any human health guidelines or HTAs covering the breadth of clinical, health system, and public health topics, that also address planetary health outcomes (including a focus on animal health and natural systems). We will include the most recent version of the report if there are multiple versions. We will include methodological papers or handbooks that provide insight into how to address planetary health in the guideline or HTA process. We will include guidelines and HTAs published in any language, employing translation tools when studies are included beyond the languages of our team. We will not restrict eligibility based on geography or the level of government/region a guideline is focused on.

Types of sources

This scoping review will focus on health guidelines and HTAs that address planetary health outcomes including those that have a focus on animal health, natural systems, and the environment.

Exclusion criteria

In our exclusion criteria, we will exclude any studies or reports not directly related to planetary health. This includes reports or studies unrelated to health guidelines or health technology assessments. Additionally, we will exclude LCA modeling studies that do not form a part of a guideline or HTA. Furthermore, if an abstract does not explicitly reference planetary health, one health, ecosystem health, climate change, or related concepts we will exclude it from consideration for full-text screening. Concepts related to sustainability that are not relevant to environmental or planetary health, such as sustainable financing, will also be excluded. Lastly, studies focusing solely on the impacts of climate change or the environment on health (i.e., not the opposing direction of health interventions on the environment) will not be included in our review.

Search strategy

The search strategy was developed in consultation with a health sciences librarian at McMaster University. The search will identify guidelines and HTAs that have addressed planetary health outcomes or considerations. We will complete a primary search of the literature using the following databases: Ovid MEDLINE, EMBASE, CINAHL, Global Health, Health Systems Evidence, Greenfile, and Environmental Issues. We will complement this search with the GIN international guideline library and registry of guidelines in development [ 24 ], BIGG international database of GRADE guidelines [ 25 ], Epistemonikos GRADE guidelines repository [ 26 ], GRADEpro GDT Database of GRADE EtD’s and Guidelines [ 27 ], MAGICapp [ 28 ], National Institute for Health and Care Excellence (NICE) website [ 29 ], and the World Health Organization website [ 30 ]. We will also complement this with a search for online content on Google incognito mode with similar search terms to identify any unpublished documents of relevance. In addition, the search will be further complemented by utilizing another ongoing scoping review of handbooks on guideline development, that identified 120 international guideline development handbooks by diverse organizations, to search for references to planetary health in these handbooks [ 31 ]. The search strategy for different databases has been provided in the Additional file 2: Appendix 1 .

Evidence selection

A web-based software platform Covidence (Covidence Systematic Review Software VHI, Melbourne, Australia) will be used to automatically remove duplicates and to screen the retrieved articles. The screening of citations will be carried out in two phases, both of which will go through pilot testing. Two review authors will independently screen the title and abstract of the citations based on the prespecified inclusion and exclusion criteria. A simple screening algorithm as outlined in Additional file 3: Appendix 2 will be employed. Any discrepancies will be resolved through a third reviewer.

Two reviewers will individually assess the full texts of the identified eligible documents based on the predetermined criteria for eligibility. Any conflicts will be resolved by a third reviewer. The reasons for exclusion will be documented in the full-text screening phase. A simple screening algorithm as outlined in Additional file 4: Appendix 3 will be employed. All pertinent complete guideline texts and HTA reports will be gathered and shared using an online folder. This folder will encompass associated documents and supplementary materials. The comprehensive details of the search results and the study inclusion process will be thoroughly documented in the final scoping review. This information is visually presented using a Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram [ 19 ] in Fig. 1 and will continue to be updated in the final scoping review once the data extraction process begins. Additional details can be found in the PRISMA-P checklist (see Additional file 1).

PRISMA flow diagram depicting search results before data extraction

Data extraction

Using a pilot-tested and standardized extraction form, a team of experienced reviewers will be responsible for extracting data from the guidelines and HTA reports incorporated in this scoping review. One reviewer will perform preliminary data extraction, and this will be verified by a second reviewer. Any disagreements will be resolved through discussion or involving another reviewer. The team will convene for regular meetings to address any potential concerns that might arise. These meetings will also serve the purpose of maintaining consistency and providing training to all reviewers. The primary investigator of the project will then review and refine the extracted data for each guideline and/or HTA. The following data will be extracted: document title; first author’s names; publication year; language; document type (peer review publication, guideline handbook, report, grey literature, other); whether the document is a guideline or HTA; the topic/discipline (respirology, gastroenterology, nutrition, anesthesia, other); document purpose/study objective; developing/sponsoring organization; country in which the study/guideline/HTA report was conducted (USA, UK, Canada, Australia, Germany, other); definition of planetary health or related concept provided; types of planetary health experts engaged (engineer, one health/veterinary, economist, modelling expert, geologist/earth scientist, other); study methods; suggested methods to assess planetary health (LCA, other modelling approach, use of existing databases, direct measurement, expert input on impact, other); whether the study uses secondary data on planetary health outcomes; description for use of LCA; description for assessing quality of life cycle; population/intended audience; interventions; category (human health, animal health/one health, natural systems, environment, other); applicable planetary health boundaries (climate change, change in biosphere integrity, stratospheric ozone depletion, ocean acidification, biogeochemical flows, land-system change, freshwater use, atmospheric aerosol loading, introduction of novel entities); consideration of social justice/global equity; phase of intervention in life cycle related to planetary health addressed (goods production, goods transport, patient/staff travel, diagnostic tests, facility requirements, infection prevention and control (IPAC) requirements, disposal impacts, other); measure of planetary health impact (carbon dioxide emissions, methane emissions, other greenhouse gas emission, nitrogen/phosphorous inputs, energy input, water use, waste production, monetary equivalents of impact, other); impact on biodiversity/land use and one health/animal welfare mentioned; funding (not transparent, public funding, private funding, both public and private funding, other), conflict of interest (reported or not reported).

Data analysis and presentation

Data analysis will involve a combination of content analysis using both deductive and inductive approaches, along with the utilization of descriptive statistics. Part of the data collected, specifically the yes/no items, will be converted into quantitative form, and basic descriptive statistical methods will be employed to examine their distribution.

We will employ directed content analysis [ 32 ] to examine the textual data, with the flexibility to uncover emergent codes as well. To examine the data extracted from the guidelines/HTAs, two coders will each perform iterative rounds of analysis. Following the initial extraction, the case studies will be elucidated, and distinct themes will be identified within different categories of studies (e.g., guideline topics, and methods papers). As we go through the coding process, they will underline the important and relevant sections in the text and choose a term, phrase, or description to best capture its meaning. The codes developed during the pilot phase will serve as predetermined codes in the analysis, with additional codes being added as necessary. Codes sharing similar concepts will be organized into categories. When appropriate, explanations and examples from the text will accompany the codes or categories. The coders will not aim to quantify the frequency of code occurrences. These codes and categories will be utilized to populate the following predefined themes: goods production, goods transport, patient/staff travel, diagnostic tests, facility requirements, IPAC requirements, and disposal impacts. Additional themes will be introduced as required.

The risk of bias and the quality of the studies will be assessed for primary research papers, health guidelines/HTAs using the Risk of Bias In Non-randomized Studies of Exposures/Interventions (ROBINS I/E) tool and Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument respectively [ 33 , 34 , 35 ]. The quality of the theoretical or commentary papers included in the scoping review will not be assessed, as it is not typically done in scoping reviews [ 17 , 36 ].

In our final conclusive assessment, we will present the results of our search using an adapted PRISMA flowchart [ 19 ]. We will also provide a concise overview of the fundamental attributes of the guidelines included, along with the outcomes of the screening procedure. Results of the analysis will be presented narratively, with codes and classifications for each topic presented in a table, as appropriate, and the quantified data presented in graphs. Any discrepancies or modifications from the established protocol will be duly documented within the final scoping review.

Discussion and preliminary results (pilot phase)

Our team, comprised of ten reviewers, undertook a comprehensive three-part pilot exercise encompassing title and abstract screening, full-text screening, and a data extraction pilot exercise. Throughout the pilot phase, we implemented an adapted approach allowing us to address unforeseen challenges and incorporate valuable insights from our team’s collective experience. Upon the conclusion of each pilot exercise, we further refined the methods for the proposed scoping review. We provide the dates of key milestones in the scoping review process in Table 1 .

Title-abstract screening pilot exercise

In the title-abstract pilot exercise that was completed in October 2023, 50 articles were randomly selected from the compiled search results. Ten independent reviewers assessed inclusion and exclusion criteria by screening titles and abstracts of each using the Title-Abstract Screening Form (Additional file 3: Appendix 2 ). Reviewers’ assessments, showing variability, were compiled for discussion in a scheduled meeting to resolve discrepancies. By resolving discrepancies amongst the results, the discussions and group feedback led to refinements clarifying the inclusion/exclusion criteria, which were as follows. There would be no exclusion based on study design, but we acknowledged that a focus on guidelines or HTAs (question 2) might be limiting. Life Cycle Assessment modeling studies or other modeling studies that were not part of a guideline or HTA will be excluded. Studies lacking explicit references to planetary health, one health, ecosystem health, climate change, or related concepts in the abstract will also be excluded. Additionally, concepts unrelated to environment/planetary health, such as sustainable financing, will also be excluded, along with studies describing the impacts of climate change or the environment on health. This refinement process allowed for a more targeted and precise selection of articles for subsequent phases.

Full-text screening pilot exercise

In the full-text pilot exercise that was completed in November 2023, 10 articles were strategically selected by the study leads to encompass both inclusion and exclusion scenarios, providing a basis for discussing common mistakes. Ten independent reviewers were tasked with evaluating the inclusion and/or exclusion criteria for all ten references by evaluating the full text using the Full-Text Screening Form (Additional file 4: Appendix 3 ). This form directed reviewers to assess whether the study addressed planetary health, the impact of an intervention on planetary health, and its relation to health guidelines or HTAs. Additionally, reviewers were asked to evaluate whether the study served as an example/case study of implementing planetary health considerations in guideline or HTA decision-making, and if it offered advice or suggestions, without an example/case study, on addressing planetary health considerations. The results of the exercise exhibited variability and were compiled for discussion in a scheduled meeting to reconcile discrepancies. In response to group discussions and feedback, specific criteria were clarified including what constitutes a guideline. For this study, guidelines encompass any clinical, health system, or public health guideline offering actionable statements based on evidence reviews, panel recommendations, consensus statements, position or policy statements, scientific statements, or other clear processes. Furthermore, studies focusing on policy or decision-making considerations of planetary health that lack a focus on guidelines or HTAs will be excluded.

Data extraction pilot exercise

In the data extraction pilot exercise that was completed in January 2024, five articles meeting the inclusion criteria were selected by the study leads. Ten independent reviewers were tasked with extracting data using Additional file 5: Appendix 4 : Data Extraction Form. Following data extraction, a scheduled meeting with the review team was organized to address potential inconsistencies in extraction. The form underwent significant modifications based on feedback from the team, incorporating additional items such as the topic/discipline of the study, type of planetary health experts engaged, planetary health boundaries, consideration of social justice/global equity, impact on biodiversity/land use, and impact on one health/animal welfare. These pilot exercises allowed for the refinement of the data extraction methodology for the following stages of the study (Additional file 6: Appendix 5 ; Additional file 7: Appendix 6 ).

This protocol provides a description of the objectives, inclusion and exclusion criteria, methods, and analysis of a scoping review to be undertaken by an international interdisciplinary group of experts. The comprehensive findings of this scoping review will be made accessible and published in a peer-reviewed journal once they become available. To inform future guidance on planetary health within the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) criteria, the primary aim of our scoping review is to identify and examine planetary health considerations in current health guidelines and health technology assessments.

The review is currently in progress and we are targeting finalizing the analysis and submitting it in 2024. Preliminary work towards GRADE guidance on planetary health was presented at the GRADE Working Group meeting in Miami in May 2024. The scoping review actively informs the ongoing development of guidance on the consideration of planetary health in health guidelines and HTAs.

Addressing planetary health in health technology assessment (HTA) reports and guidelines will provide essential support for a more comprehensive evaluation of the potential impacts of health interventions on both human health and the environment. This holistic approach will encourage guideline developers and HTA reporters to take proactive measures in addressing and mitigating adverse effects on planetary health caused by healthcare practices, including issues such as excessive use of single-use plastics, transportation-related carbon emissions, and other environmentally detrimental impacts. Hence, promoting a healthier future for all life forms and the Earth.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Appraisal of Guidelines for Research and Evaluation II

Evidence to Decision

Grading of Recommendations, Assessment, Development, and Evaluations

Greenhouse Gas

Health Technology Assessments

Infection Prevention and Control

Life Cycle Assessments

Risk of Bias in Non-randomized Studies of Exposures/Interventions

MacNeill AJ, McGain F, Sherman JD. Planetary health care: a framework for sustainable health systems. Lancet Planet Health. 2021;5(2):e66–8.

Article PubMed Google Scholar

Romanello M, Di Napoli C, Drummond P, Green C, Kennard H, Lampard P, et al. The 2022 report of the Lancet Countdown on health and climate change: health at the mercy of fossil fuels. Lancet. 2022;400(10363):1619–54.

Eckelman MJ, Sherman J. Environmental Impacts of the U.S Health Care System and Effects on Public Health. PLoS One. 2016;11(6):e0157014.

Article PubMed PubMed Central Google Scholar

Lenzen M, Malik A, Li M, Fry J, Weisz H, Pichler PP, et al. The environmental footprint of health care: a global assessment. Lancet Planet Health. 2020;4(7):e271–9.

Buse CG, Oestreicher JS, Ellis NR, Patrick R, Brisbois B, Jenkins AP, et al. Public health guide to field developments linking ecosystems, environments and health in the Anthropocene. J Epidemiol Community Health. 2018;72(5):420–5.

Whitmee S, Haines A, Beyrer C, Boltz F, Capon AG, de Souza Dias BF, et al. Safeguarding human health in the Anthropocene epoch: report of The Rockefeller Foundation-Lancet Commission on planetary health. Lancet. 2015;386(10007):1973–2028.

Drew J, Christie SD, Rainham D, Rizan C. HealthcareLCA: an open-access living database of health-care environmental impact assessments. Lancet Planet Health. 2022;6(12):e1000–12.

Rossa-Roccor V, Acheson ES, Andrade-Rivas F, Coombe M, Ogura S, Super L, et al. Scoping review and bibliometric analysis of the term “planetary health” in the peer-reviewed literature. Front Public Health. 2020;8:343.

Herrmann A, Lenzer B, Muller BS, Danquah I, Nadeau KC, Muche-Borowski C, et al. Integrating planetary health into clinical guidelines to sustainably transform health care. Lancet Planet Health. 2022;6(3):e184–5.

Schünemann HB, J.; Guyatt, G.; Oxman, A. GRADE Handbook https://gdt.gradepro.org/app/handbook/handbook.html2013 [

Morgan RL, Kelley L, Guyatt GH, Johnson A, Lavis JN. Decision-making frameworks and considerations for informing coverage decisions for healthcare interventions: a critical interpretive synthesis. J Clin Epidemiol. 2018;94:143–50.

Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.

Nordic Council of Ministes. Nordic Nutrition Recommendations 2023: Integrating Environmental Aspects. 2023. Available: https://norden.diva-portal.org/smash/get/diva2:1769986/FULLTEXT06.pdf .

Schmiemann G, Dörks M. DEGAM S1-Guideline: Climate-conscious prescription of inhaled medications. German College of General Practitioners and Family Physicians. 2022. Available: https://register.awmf.org/assets/guidelines/053_D_Ges_fuer_Allgemeinmedizin_und_Familienmedizin/053-059eng_S1_Climate-conscious-prescription-of-inhaled-medications_2022-09.pdf .

Schunemann HJ, Brennan S, Akl EA, Hultcrantz M, Alonso-Coello P, Xia J, et al. The development methods of official GRADE articles and requirements for claiming the use of GRADE - A statement by the GRADE guidance group. J Clin Epidemiol. 2023;159:79–84.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119–26.

Peters MDJ, Godfrey C, McInerney P, Khalil H, Larsen P, Marnie C, et al. Best practice guidance and reporting items for the development of scoping review protocols. JBI Evid Synth. 2022;20(4):953–68.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. J Clin Epidemiol. 2021;134:178–89.

Peters MDJ, Godfrey CM, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. https://synthesismanual.jbi.global : JBI; 2020.

Richardson K, Steffen W, Lucht W, Bendtsen J, Cornell SE, Donges JF, et al. Earth beyond six of nine planetary boundaries. Sci Adv. 2023;9(37):eadh2458.

Steffen W, Richardson K, Rockstrom J, Cornell SE, Fetzer I, Bennett EM, et al. Sustainability Planetary boundaries: guiding human development on a changing planet. Science. 2015;347(6223):1259855.

Rockström J, Steffen W, Noone K, Persson Å, Chapin, III FS, Lambin E, et al. Planetary Boundaries: Exploring the Safe Operating Space for Humanity. Ecol Soc. 2009;14(2):472–5.

Article Google Scholar

GIN International Guideline Library and Registry of Guidelines in Development [Available from: https://guidelines.ebmportal.com/ .

BIGG International Database of GRADE guidelines [Available from: https://sites.bvsalud.org/bigg/en/biblio/ .

Epistemonikos GRADE Guidelines Repository [Available from: https://www.epistemonikos.org/en/groups/grade_guideline .

GRADEpro GDT Database of GRADE EtD’s and Guidelines [Available from: https://guidelines.gradepro.org/ .

MAGICapp [Available from: https://app.magicapp.org/#/guidelines .

National Institute for Health and Care Excellence (NICE) [Available from: https://www.nice.org.uk/guidance/published?type=ph .

World Health Organization [Available from: https://www.who.int/publications/i .

Khabsa J, Nour Eldine M, Yaacoub S, El-Khoury R, El Yaman N, Wiercioch W, Schünemann HJ, Akl EA. (submitted). Guideline development methods based on a survey of handbooks: 1. Introduction to the series.

Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Sterne JA, Hernan MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.

ROBINS-E Development Group (Higgins J MR, Rooney A, Taylor K, Thayer K, Silva R, Lemeris C, Akl A, Arroyave W, Bateson T, Berkman N, Demers P, Forastiere F, Glenn B, Hróbjartsson A, Kirrane E, LaKind J, Luben T, Lunn R, McAleenan A, McGuinness L, Meerpohl J, Mehta S, Nachman R, Obbagy J, O'Connor A, Radke E, Savović J, Schubauer-Berigan M, Schwingl P, Schunemann H, Shea B, Steenland K, Stewart T, Straif K, Tilling K, Verbeek V, Vermeulen R, Viswanathan M, Zahm S, Sterne J). Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E). 2023.

Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421–4.

Pollock D, Davies EL, Peters MDJ, Tricco AC, Alexander L, McInerney P, et al. Undertaking a scoping review: A practical guide for nursing and midwifery students, clinicians, researchers, and academics. J Adv Nurs. 2021;77(4):2102–13.

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

Department of Health Research Methods, Evidence, and Impact, McMaster University, HSC-2C 1280 Main Street West Hamilton, Hamilton, ON, L8N 3Z5, Canada

Thomas Piggott, Maheen Raja, Andrea J. Darzi, Rebecca L. Morgan, Ignacio Neumann & Holger Schünemann

Department of Family Medicine, Queens University, Kingston, Canada

Thomas Piggott & Laura Jewell

Knowledge Institute of the Dutch Association of Medical Specialists, Mercatorlaan 1200, Postbus 3320, Utrecht, 3502 GH, The Netherlands

Charlotte T. J. Michels

Institute of Global Health, Heidelberg University, Heidelberg University Hospital, Im Neuenheimer Feld 324, 69120, Heidelberg, Germany

Alina Herrmann

Institute of General Medicine, Cologne University, Cologne University Hospital, Cologne, Germany

College of Medicine and Veterinary Medicine, University of Edinburgh, 49 Little France Crescent, Edinburgh, EH16 4SB, UK

Karolina Anna Scahill

Evidensia Södra Djursjukhuset Kungens Kurva, Kungens Kurva, Sweden

Department of Anesthesia, McMaster University, Hamilton, Canada

Andrea J. Darzi

College of Medicine, Nursing and Health Sciences, University of Galway, University Road, Galway, H91TK33, Ireland

KM Saif-Ur-Rahman

Evidence Synthesis Ireland and Cochrane Ireland, University of Galway, University Road, Galway, H91TK33, Ireland

Institute of General Practice and Family Medicine, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt Universität Zu Berlin, Charitéplatz 1, 10117, Berlin, Germany

Hendrik Napierala

Division of Evidence-Based Medicine (dEBM), Department of Dermatology, Venereology and Allergology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin Und Humboldt Universität zu Berlin, Berlin, Germany

Ruben Heuer

School of Medicine, Case Western Reserve University, Cleveland, OH, USA

Rebecca L. Morgan

Department of Medicine, McMaster University, Hamilton, Canada

Grigorios I. Leontiadis

School of Medicine, Universidad San Sebastián, Santiago, Chile

Ignacio Neumann

GRADE Conosur, Universidad San Sebastián, Santiago, Chile

Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, 20090, Pieve Emanuele (Milan), Italy

Holger Schünemann

Institute of Health Policy, Management & Evaluation, Dalla Lana School of Public Health; Collaborative Centre for Climate, Health & Sustainable Care, University of Toronto, Toronto, ON, Canada

Fiona A. Miller

You can also search for this author in PubMed Google Scholar

Contributions

TP made substantial contributions to the study conceptualization, design, data collection, pilot analysis, and decision to publish. MR made substantial contributions to the study design, data collection, pilot analysis, and protocol writing. CM, AH, KAS, AJD, LJ, KMS, HN, RH, and RLM made contributions to the data collection and pilot analysis. GIL, IN, HS, and FAM made contributions to the study design. All authors read and approved the study protocol.

Corresponding author

Correspondence to Thomas Piggott .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

Alina Herrmann is a member of the German Alliance of Climate Change and Health (KLUG), and an honorary spokesperson on climate change and health at the German Society of General and Family Medicine (DEGAM). Holger Schünemann is the GRADE working group co-chair.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: prisma-p 2015 checklist., additional file 2: appendix 1. search strategy approach., additional file 3: appendix 2. title-abstract screening form., additional file 4: appendix 3. full text screening guide (in covidence)., additional file 5: appendix 4. data extraction form., additional file 6: appendix 5. sample included studies for validating search., additional file 7: appendix 6. search notes., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Piggott, T., Raja, M., Michels, C.T.J. et al. Considering planetary health in health guidelines and health technology assessments: a scoping review protocol. Syst Rev 13 , 163 (2024). https://doi.org/10.1186/s13643-024-02577-2

Download citation

Received : 08 February 2024

Accepted : 30 May 2024

Published : 22 June 2024

DOI : https://doi.org/10.1186/s13643-024-02577-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Planetary health
Health guidelines
Health technology assessments

Systematic Reviews

ISSN: 2046-4053

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

May 5-8, 2024

Past Conferences

A Non-Systematic Literature Review of Literature Reviews: Evaluation of Methods Implemented and Language Used to Describe Systematic and Non-Systematic Literature Reviews

Brown AE 1 , Los AK 1 , Goldsmith-Martin G 1 , Phalguni A 1 , Bowes K 1 , Briggs JA 1 , Bajko P 1 , Kilty RLH 1 , Khankhel ZS 2 1 Genesis Research Group, Newcastle upon Tyne, Tyne and Wear, UK, 2 Genesis Research Group, Hoboken, NJ, USA

OBJECTIVES: Systematic literature reviews (SLRs) are a widely used research tool with established best practices. Non-systematic literature reviews (nSLRs) encompass a range of review types and implement varying approaches to searching, screening, and data extraction. We sought to collate established guidance for SLRs/nSLRs and to understand whether there was consistency in the language and methods used to describe and conduct published SLRs/nSLRs.

METHODS: MEDLINE (via PubMed) was searched (5 Dec 2023) to identify SLRs/nSLRs published on any topic; records were restricted to those published in English, within the prior week. Title/abstract (TIAB) screening was conducted by a single researcher; includes were tagged as either SLR or nSLR. A random sample of TIAB-level includes (100 SLRs, 100 nSLRs) underwent full-text screening, involving high-level data extraction and conducted by two independent researchers. Separate searches of established evidence synthesis organizations (i.e. Cochrane, University of York, PRISMA), a selection of universities, and other research organizations were also conducted to collate recommendations or established guidance for conducting SLRs and nSLRs.

RESULTS: 1,233/1,271 TIABs identified from MEDLINE met the inclusion criteria. Of the 100 reviews defined by authors as SLRs, 93 followed established guidance; 7 did not clearly report whether established guidance for conducting SLRs was followed. Of the 100 nSLRs, authors defined their research using varying terms: review (n=55), scoping review (n=19), literature review (n=11), narrative review (n=10), comprehensive review (n=2), rapid review (n=1), structured integrative review (n=1), and tutorial review (n=1). Upon review of methods implemented in the nSLRs, 12 utilized methods sufficient for SLR definition; the remaining either leveraged elements of SLR methods or did not clearly report these details.

Study Approaches

Topic Subcategory

Literature Review & Synthesis

No Additional Disease & Conditions/Specialized Treatment Areas

Frontiers in Allergy
Drug, Venom & Anaphylaxis
Research Topics

Anaphylaxis Challenges: Idiopathic and Rare Causes

Total Downloads

Total Views and Downloads

About this Research Topic

The main first step in managing anaphylaxis is establishing a correct diagnosis. Many factors concerning definition, diagnostic criteria, and lack of confirmatory testing, makes the diagnosis a challenge, not only to the treating or first line medical responders but also to the allergologists who are ...

Keywords : Anaphylaxis, diagnostic challenges, allergic reactions, systemic reactions, rare causes, idiopathic anaphylaxis

Important Note : All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Topic Editors

Topic coordinators, recent articles, submission deadlines.

	Manuscript
	Manuscript Extension

Participating Journals

Manuscripts can be submitted to this Research Topic via the following journals:

total views

Demographics

No records found

total views article views downloads topic views

Top countries

Top referring sites, about frontiers research topics.

With their unique mixes of varied contributions from Original Research to Review Articles, Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author.

IMAGES

The Systematic Review Process
systematic literature review steps
Systematic Reviews: What They Are, Why They Are Important, and How to
The importance of meta-analysis and systematic review: How research
Before you begin
Systematic Review: essential steps and practical tips

VIDEO

Everything about Systematic Review| Meaning| Steps
Statistical Procedure in Meta-Essentials
تحديد موضوع ومجال البحث بطريقة ممنهجة |Determine the topic and field of research systematically
Introduction to Systematic Review of Research
سؤل البحث عند كتابة ورقة بطريقة ممنهجة Research question for systematic review paper
Systematic Review Part I

COMMENTS

Systematic Review
A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review. In 2008, Dr. Robert Boyle and his colleagues published a systematic review in ...
Systematic review
A systematic review is a scholarly synthesis of the evidence on a clearly presented topic using critical methods to identify, define and assess research on the topic. A systematic review extracts and interprets data from published studies on the topic (in the scientific literature), then analyzes, describes, critically appraises and summarizes interpretations into a refined evidence-based ...
Introduction to systematic review and meta-analysis
A systematic review collects all possible studies related to a given topic and design, and reviews and analyzes their results [ 1 ]. During the systematic review process, the quality of studies is evaluated, and a statistical meta-analysis of the study results is conducted on the basis of their quality. A meta-analysis is a valid, objective ...
What is a Systematic Review?
an explicit, reproducible methodology. a systematic search that attempts to identify all studies that would meet the eligibility criteria. an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias. a systematic presentation, and synthesis, of the characteristics and findings of ...
Systematic Review
A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review. In 2008, Dr Robert Boyle and his colleagues published a systematic review in ...
What is a Systematic Review (SR)?
Systematic Reviews in the Social Sciences by Roberts, H., & Petticrew, M. Such diverse thinkers as Lao-Tze, Confucius, and U.S. Defense Secretary Donald Rumsfeld have all pointed out that we need to be able to tell the difference between real and assumed knowledge. The systematic review is a scientific tool that can help with this difficult task.
Systematic reviews: Structure, form and content
Topic selection and planning. In recent years, there has been an explosion in the number of systematic reviews conducted and published (Chalmers & Fox 2016, Fontelo & Liu 2018, Page et al 2015) - although a systematic review may be an inappropriate or unnecessary research methodology for answering many research questions.Systematic reviews can be inadvisable for a variety of reasons.
What are systematic reviews?
What are systematic reviews? Watch on. Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does. About Cochrane.
How to Do a Systematic Review: A Best Practice Guide for Conducting and
Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question.
Introduction to Systematic Reviews
Abstract. A systematic review identifies and synthesizes all relevant studies that fit prespecified criteria to answer a research question. Systematic review methods can be used to answer many types of research questions. The type of question most relevant to trialists is the effects of treatments and is thus the focus of this chapter.
LibGuides: Systematic Reviews: What is a Systematic Review?
A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. The key characteristics of a systematic review are: a clearly defined question with inclusion and exclusion criteria; a rigorous and systematic search of the literature;
What are systematic reviews?
Sometimes systematic reviews ask a broad research question, and one strategy to achieve this is the use of several focussed sub-questions each addressed by sub-components of the review. Another strategy is to develop a map to describe the type of research that has been undertaken in relation to a research question. Some maps even describe over ...
Guides: Systematic Review: What is a systematic review?
A systematic review is an authoritative account of existing evidence using reliable, objective, thorough and reproducible research practices. It is a method of making sense of large bodies of information and contributes to the answers to questions about what works and what doesn't. Systematic reviews map areas of uncertainty and identify where ...
1.2.2 What is a systematic review?
A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman ...
How to Do a Systematic Review: A Best Practice Guide ...
Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to ...
Systematic Reviews and Meta-analysis: Understanding the Best Evidence
Systematic reviews can also demonstrate where knowledge is lacking. This can then be used to guide future research. Systematic reviews are usually carried out in the areas of clinical tests (diagnostic, screening, and ... The steps undertaken in evaluating the study quality are early definition of study quality and criteria, setting up a good ...
Easy guide to conducting a systematic review
A systematic review is a type of study that synthesises research that has been conducted on a particular topic. Systematic reviews are considered to provide the highest level of evidence on the hierarchy of evidence pyramid. Systematic reviews are conducted following rigorous research methodology. To minimise bias, systematic reviews utilise a ...
Evidence Synthesis and Systematic Reviews
Definition: A systematic review is a summary of research results (evidence) that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue.It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and errors. When to use: If you want to identify, appraise, and synthesize all ...
CDC Library
Several CDC librarians have special training in conducting literature searches for systematic reviews. Literature searches for systematic reviews can take a few weeks to several months from planning to delivery. Fill out a search request form or contact the Stephen B. Thacker CDC Library by email [email protected] or telephone 404-639-1717.
What is a systematic review?
A high-quality systematic review is described as the most reliable source of evidence to guide clinical practice. The purpose of a systematic review is to deliver a meticulous summary of all the available primary research in response to a research question. A systematic review uses all the existing research and is sometime called 'secondary research' (research on research). They are often ...
Systematic Review Process: Definitions of a Systematic Review
A systematic review is defined as "a review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review." The methods must be reproducible and transparent.
About systematic reviews
A systematic review is a type of secondary evidence that summarises research that has already been published. Systematic reviews, according to Wright, et al., are defined as a: "review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the ...
Systematic Reviews: What is a systematic review?
A systematic review is a tightly structured literature review that focuses on a topic with strict research parameters. The methodology used to collect research has to be consistent in order to reduce misinterpretation and misrepresentation of the data. To help you understand and conduct your systematic review we have produce a number of posts ...
Unravelling effectiveness in intelligence: a systematic review
A systematic literature review is a methodical and comprehensive analysis of existing research on a specific topic. Footnote 12 This systematic review is based on contributions to journals that discuss effectiveness of intelligence and sets out to explore concepts of intelligence effectiveness in intelligence studies.
Considering planetary health in health guidelines and health technology
The final review results will be submitted to open-access peer-reviewed journals for publication when they become available. The research findings will also be disseminated at relevant planetary health conferences and workshops. Systematic review registration. Open Science Framework (https://osf.io/3jmsa).
ISPOR
OBJECTIVES: Systematic literature reviews (SLRs) are a widely used research tool with established best practices. Non-systematic literature reviews (nSLRs) encompass a range of review types and implement varying approaches to searching, screening, and data extraction. We sought to collate established guidance for SLRs/nSLRs and to understand whether there was consistency in the language and ...
Anaphylaxis Challenges: Idiopathic and Rare Causes
Manuscripts can include opinion and local experience using the current definition and diagnostic criteria, as well as studies and research conducted in respect to testing, diagnosis, and management.Suitable articles include but are not limited to case reports, case studies, original research articles, mini reviews, reviews, or systematic ...

Have a language expert improve your writing

Systematic Review | Definition, Example & Guide

Table of contents

Receive feedback on language, structure, and formatting

Step 1: Formulate a research question

Step 2: Develop a protocol

Step 3: Search for all relevant studies

Step 4: Apply the selection criteria

Step 5: Extract the data

Step 6: Synthesize the data

Step 7: Write and publish a report

Cite this Scribbr article

Is this article helpful?

Shaun Turney

Systematic Reviews

Key Characteristics of a Systematic Review:

Additional Information

Research Support

Have a language expert improve your writing

Systematic Review | Definition, Examples & Guide

Table of contents

Prevent plagiarism, run a free check.

Step 1: Formulate a research question

Step 2: Develop a protocol

Step 3: Search for all relevant studies

Step 4: Apply the selection criteria

Step 5: Extract the data

Step 6: Synthesise the data

Step 7: Write and publish a report

Cite this Scribbr article

Is this article helpful?

Shaun Turney

Systematic Review

Introduction to Systematic Review

Scoping Review

Qualitative Systematic Review/Meta-Synthesis

What are systematic reviews?

Introduction to Systematic Reviews

Access this chapter

Similar content being viewed by others

What Is the Difference Between a Systematic Review and a Meta-analysis?

Systematic Reviewing

Author information

Corresponding author

Editor information

Section Editor information

Rights and permissions

Copyright information

About this entry

Download citation

Share this entry

Conducting Research

Systematic Reviews

Using Databases

Finding & Accessing

Writing & Citing

Meet a Librarian

Search Services

Citation Mgmt

Scholarly Communications

Library Updates

What is a Systematic Review?

Medical Center Library & Archives Presentations

UCL LIBRARY SERVICES

What are systematic reviews?

On this page

Useful books about systematic reviews

Systematic approaches to a successful literature review

An introduction to systematic reviews

Cochrane handbook for systematic reviews of interventions

Finding what works in health care: Standards for systematic reviews

Systematic Reviews in the Social Sciences

Research Synthesis and Meta-Analysis

Doing a Systematic Review

Systematic Review

Characteristics of a systematic review

1.2.2 What is a systematic review?

Annual Review of Psychology

Most Read This Month

Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare