U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • Indian J Anaesth
  • v.60(9); 2016 Sep

Literature search for research planning and identification of research problem

Anju grewal.

Department of Anaesthesiology, Dayanand Medical College and Hospital, Ludhiana, Punjab, India

Hanish Kataria

1 Department of Surgery, Government Medical College and Hospital, Chandigarh, India

2 Department of Cardiac Anaesthesia, All India Institute of Medical Sciences, New Delhi, India

Literature search is a key step in performing good authentic research. It helps in formulating a research question and planning the study. The available published data are enormous; therefore, choosing the appropriate articles relevant to your study in question is an art. It can be time-consuming, tiring and can lead to disinterest or even abandonment of search in between if not carried out in a step-wise manner. Various databases are available for performing literature search. This article primarily stresses on how to formulate a research question, the various types and sources for literature search, which will help make your search specific and time-saving.

INTRODUCTION

Literature search is a systematic and well-organised search from the already published data to identify a breadth of good quality references on a specific topic.[ 1 ] The reasons for conducting literature search are numerous that include drawing information for making evidence-based guidelines, a step in the research method and as part of academic assessment.[ 2 ] However, the main purpose of a thorough literature search is to formulate a research question by evaluating the available literature with an eye on gaps still amenable to further research.

Research problem[ 3 ] is typically a topic of interest and of some familiarity to the researcher. It needs to be channelised by focussing on information yet to be explored. Once we have narrowed down the problem, seeking and analysing existing literature may further straighten out the research approach.

A research hypothesis[ 4 ] is a carefully created testimony of how you expect the research to proceed. It is one of the most important tools which aids to answer the research question. It should be apt containing necessary components, and raise a question that can be tested and investigated.

The literature search can be exhaustive and time-consuming, but there are some simple steps which can help you plan and manage the process. The most important are formulating the research questions and planning your search.

FORMULATING THE RESEARCH QUESTION

Literature search is done to identify appropriate methodology, design of the study; population sampled and sampling methods, methods of measuring concepts and techniques of analysis. It also helps in determining extraneous variables affecting the outcome and identifying faults or lacunae that could be avoided.

Formulating a well-focused question is a critical step for facilitating good clinical research.[ 5 ] There can be general questions or patient-oriented questions that arise from clinical issues. Patient-oriented questions can involve the effect of therapy or disease or examine advantage versus disadvantage for a group of patients.[ 6 ]

For example, we want to evaluate the effect of a particular drug (e.g., dexmedetomidine) for procedural sedation in day care surgery patients. While formulating a research question, one should consider certain criteria, referred as ‘FINER’ (F-Feasible, I-Interesting, N-Novel, E-Ethical, R-Relevant) criteria.[ 5 ] The idea should be interesting and relevant to clinical research. It should either confirm, refute or add information to already done research work. One should also keep in mind the patient population under study and the resources available in a given set up. Also the entire research process should conform to the ethical principles of research.

The patient or study population, intervention, comparison or control arm, primary outcome, timing of measurement of outcome (PICOT) is a well-known approach for framing a leading research question.[ 7 , 8 ] Dividing the questions into key components makes it easy and searchable. In this case scenario:

  • Patients (P) – What is the important group of patients? for example, day care surgery
  • Intervention (I) – What is the important intervention? for example, intravenous dexmedetomidine
  • Comparison (C) – What is the important intervention of comparison? for example, intravenous ketamine
  • Outcome (O) – What is the effect of intervention? for example, analgesic efficacy, procedural awareness, drug side effects
  • Time (T) – Time interval for measuring the outcome: Hourly for first 4 h then 4 hourly till 24 h post-procedure.

Multiple questions can be formulated from patient's problem and concern. A well-focused question should be chosen for research according to significance for patient interest and relevance to our knowledge. Good research questions address the lacunae in available literature with an aim to impact the clinical practice in a constructive manner. There are limited outcome research and relevant resources, for example, electronic database system, database and hospital information system in India. Even when these factors are available, data about existing resources is not widely accessible.[ 9 ]

TYPES OF MEDICAL LITERATURE

(Further details in chapter ‘Types of studies and research design’ in this issue).

Primary literature

Primary sources are the authentic publication of an expert's new evidence, conclusions and proposals (case reports, clinical trials, etc) and are usually published in a peer-reviewed journal. Preliminary reports, congress papers and preprints also constitute primary literature.[ 2 ]

Secondary literature

Secondary sources are systematic review articles or meta-analyses where material derived from primary source literature are infererred and evaluated.[ 2 ]

Tertiary literature

Tertiary literature consists of collections that compile information from primary or secondary literature (eg., reference books).[ 2 ]

METHODS OF LITERATURE SEARCH

There are various methods of literature search that are used alone or in combination [ Table 1 ]. For past few decades, searching the local as well as national library for books, journals, etc., was the usual practice and still physical literature exploration is an important component of any systematic review search process.[ 10 , 11 ] With the advancement of technology, the Internet is now the gateway to the maze of vast medical literature.[ 12 ] Conducting a literature review involves web-based search engines, i.e., Google, Google Scholar, etc., [ Table 2 ], or using various electronic research databases to identify materials that describe the research topic or those homologous to it.[ 13 , 14 ]

Methods of literature search

An external file that holds a picture, illustration, etc.
Object name is IJA-60-635-g001.jpg

Web based methods of literature search

An external file that holds a picture, illustration, etc.
Object name is IJA-60-635-g002.jpg

The various databases available for literature search include databases for original published articles in the journals [ Table 2 ] and evidence-based databases for integrated information available as systematic reviews and abstracts [ Table 3 ].[ 12 , 14 ] Most of these are not freely available to the individual user. PubMed ( http://www.ncbi.nlm.nih.gov/pubmed/ ) is the largest available resource since 1996; however, a large number of sources now provide free access to literature in the biomedical field.[ 15 ] More than 26 million citations from Medline, life science journals and online books are included in PubMed. Links to the full-text material are included in citations from PubMed Central and publisher web sites.[ 16 ] The choice of databases depends on the subject of interest and potential coverage by the different databases. Education Resources Information Centre is a free online digital library of education research and information sponsored by the Institute of Education Sciences of the U.S. Department of Education, available at http://eric.ed.gov/ . No one database can search all the medical literature. There is need to search several different databases. At a minimum, PubMed or Medline, Embase and the Cochrane central trials Registry need to be searched. When searching these databases, emphasis should be given to meta-analysis, systematic reviews randomised controlled trials and landmark studies.

Electronic source of Evidence-Based Database

An external file that holds a picture, illustration, etc.
Object name is IJA-60-635-g003.jpg

Time allocated to the search needs attention as exploring and selecting data are early steps in the research method and research conducted as part of academic assessment have narrow timeframes.[ 17 ] In Indian scenario, limited outcome research and accessibility to data leads to less thorough knowledge of nature of research problem. This results in the formulation of the inappropriate research question and increases the time to literature search.

TYPES OF SEARCH

Type of search can be described in different forms according to the subject of interest. It increases the chances of retrieving relevant information from a search.

Translating research question to keywords

This will provide results based on any of the words specified; hence, they are the cornerstone of an effective search. Synonyms/alternate terms should be considered to elicit further information, i.e., barbiturates in place of thiopentone. Spellings should also be taken into account, i.e., anesthesia in place of anaesthesia (American and British). Most databases use controlled word-stock to establish common search terms (or keywords). Some of these alternative keywords can be looked from database thesaurus.[ 4 ] Another strategy is combining keywords with Boolean operators. It is important to keep a note of keywords and methods used in exploring the literature as these will need to be described later in the design of search process.

‘Medical Subject Heading (MeSH) is the National Library of Medicine's controlled hierarchical vocabulary that is used for indexing articles in PubMed, with more specific terms organised underneath more general terms’.[ 17 ] This provides a reliable way to retrieve citations that use different terminology for identical ideas, as it indexes articles based on content. Two features of PubMed that can increase yield of specific articles are ‘Automatic term mapping’ and ‘automatic term explosion’.[ 4 ]

For example, if the search keyword is heart attack, this term will match with MeSH transcription table heading and then explode into various subheadings. This helps to construct the search by adding and selecting MeSH subheadings and families of MeSH by use of hyperlinks.[ 4 ]

We can set limits to a clinical trial for retrieving higher level of evidence (i.e., randomised controlled clinical trial). Furthermore, one can browse through the link entitled ‘Related Articles’. This PubMed feature searches for similar citations using an intricate algorithm that scans titles, abstracts and MeSH terms.[ 4 ]

Phrase search

This will provide pages with only the words typed in the phrase, in that exact order and with no words in between them.

Boolean operators

AND, OR and NOT are the three Boolean operators named after the mathematician George Boole.[ 18 ] Combining two words using ‘AND’ will fetch articles that mention both the words. Using ‘OR’ will widen the search and fetch more articles that mention either subject. While using the term ‘NOT’ to combine words will fetch articles containing the first word but not the second, thus narrowing the search.

Filters can also be used to refine the search, for example, article types, text availability, language, age, sex and journal categories.

Overall, the recommendations for methodology of literature search can be as below (Creswell)[ 19 ]

  • Identify keywords and use them to search articles from library and internet resources as described above
  • Search several databases to search articles related to your topic
  • Use thesaurus to identify terms to locate your articles
  • Find an article that is similar to your topic; then look at the terms used to describe it, and use them for your search
  • Use databases that provide full-text articles (free through academic libraries, Internet or for a fee) as much as possible so that you can save time searching for your articles
  • If you are examining a topic for the first time and unaware of the research on it, start with broad syntheses of the literature, such as overviews, summaries of the literature on your topic or review articles
  • Start with the most recent issues of the journals, and look for studies about your topic and then work backward in time. Follow-up on references at the end of the articles for more sources to examine
  • Refer books on a single topic by a single author or group of authors or books that contain chapters written by different authors
  • Next look for recent conference papers. Often, conference papers report the latest research developments. Contact authors of pertinent studies. Write or phone them, asking if they know of studies related to your area of interest
  • The easy access and ability to capture entire articles from the web make it attractive. However, check these articles carefully for authenticity and quality and be cautious about whether they represent systematic research.

The whole process of literature search[ 20 ] is summarised in Figure 1 .

An external file that holds a picture, illustration, etc.
Object name is IJA-60-635-g004.jpg

Process of literature search

Literature search provides not only an opportunity to learn more about a given topic but provides insight on how the topic was studied by previous analysts. It helps to interpret ideas, detect shortcomings and recognise opportunities. In short, systematic and well-organised research may help in designing a novel research.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

  • Search for relevant literature
  • Evaluate sources
  • Identify themes, debates, and gaps
  • Outline the structure
  • Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

upload-your-document-ai-proofreader

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

  • Quick Run-through
  • Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

  • Demonstrate your familiarity with the topic and its scholarly context
  • Develop a theoretical framework and methodology for your research
  • Position your work in relation to other researchers and theorists
  • Show how your research addresses a gap or contributes to a debate
  • Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Literature review guide

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

use of research literature

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

  • Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
  • Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
  • Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
  • Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

  • Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
  • Body image, self-perception, self-esteem, mental health
  • Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

  • Your university’s library catalogue
  • Google Scholar
  • Project Muse (humanities and social sciences)
  • Medline (life sciences and biomedicine)
  • EconLit (economics)
  • Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

  • What question or problem is the author addressing?
  • What are the key concepts and how are they defined?
  • What are the key theories, models, and methods?
  • Does the research use established frameworks or take an innovative approach?
  • What are the results and conclusions of the study?
  • How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
  • What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

  • Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
  • Themes: what questions or concepts recur across the literature?
  • Debates, conflicts and contradictions: where do sources disagree?
  • Pivotal publications: are there any influential theories or studies that changed the direction of the field?
  • Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

  • Most research has focused on young women.
  • There is an increasing interest in the visual aspects of social media.
  • But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

  • Look at what results have emerged in qualitative versus quantitative research
  • Discuss how the topic has been approached by empirical versus theoretical scholarship
  • Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

  • Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically evaluate: mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

  • Sampling methods
  • Simple random sampling
  • Stratified sampling
  • Cluster sampling
  • Likert scales
  • Reproducibility

 Statistics

  • Null hypothesis
  • Statistical power
  • Probability distribution
  • Effect size
  • Poisson distribution

Research bias

  • Optimism bias
  • Cognitive bias
  • Implicit bias
  • Hawthorne effect
  • Anchoring bias
  • Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

  • To familiarize yourself with the current state of knowledge on your topic
  • To ensure that you’re not just repeating what others have already done
  • To identify gaps in knowledge and unresolved problems that your research can address
  • To develop your theoretical framework and methodology
  • To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved February 22, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Research Methods

  • Getting Started
  • Literature Review Research
  • Research Design
  • Research Design By Discipline
  • SAGE Research Methods
  • Teaching with SAGE Research Methods

Literature Review

  • What is a Literature Review?
  • What is NOT a Literature Review?
  • Purposes of a Literature Review
  • Types of Literature Reviews
  • Literature Reviews vs. Systematic Reviews
  • Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

  • Summarizes and analyzes previous research relevant to a topic
  • Includes scholarly books and articles published in academic journals
  • Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

  • Help gather ideas or information
  • Keep up to date in current trends and findings
  • Help develop new questions

A literature review is important because it:

  • Explains the background of research on a topic.
  • Demonstrates why a topic is significant to a subject area.
  • Helps focus your own research questions or problems
  • Discovers relationships between research studies/ideas.
  • Suggests unexplored ideas or populations
  • Identifies major themes, concepts, and researchers on a topic.
  • Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
  • Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
  • Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

  • provides thorough knowledge of previous studies; introduces seminal works.
  • helps focus one’s own research topic.
  • identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
  • suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
  • identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
  • helps the researcher avoid repetition of earlier research.
  • suggests unexplored populations.
  • determines whether past studies agree or disagree; identifies controversy in the literature.
  • tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

use of research literature

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

use of research literature

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

  • clearly stated objectives with pre-defined eligibility criteria for studies
  • explicit, reproducible methodology
  • a systematic search that attempts to identify all studies
  • assessment of the validity of the findings of the included studies (e.g. risk of bias)
  • systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

  • << Previous: Getting Started
  • Next: Research Design >>
  • Last Updated: Aug 21, 2023 4:07 PM
  • URL: https://guides.lib.udel.edu/researchmethods
  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

  • Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
  • Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
  • Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

  • << Previous: Theoretical Framework
  • Next: Citation Tracking >>
  • Last Updated: Feb 8, 2024 1:57 PM
  • URL: https://libguides.usc.edu/writingguide

Logo for BCcampus Open Publishing

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 2: Getting Started in Research

Reviewing the Research Literature

Learning Objectives

  • Define the research literature in psychology and give examples of sources that are part of the research literature and sources that are not.
  • Describe and use several methods for finding previous research on a particular research idea or question.

Reviewing the research literature means finding, reading, and summarizing the published research relevant to your question. An empirical research report written in American Psychological Association (APA) style always includes a written literature review, but it is important to review the literature early in the research process for several reasons.

  • It can help you turn a research idea into an interesting research question.
  • It can tell you if a research question has already been answered.
  • It can help you evaluate the interestingness of a research question.
  • It can give you ideas for how to conduct your own study.
  • It can tell you how your study fits into the research literature.

What Is the Research Literature?

The  research literature  in any field is all the published research in that field. The research literature in psychology is enormous—including millions of scholarly articles and books dating to the beginning of the field—and it continues to grow. Although its boundaries are somewhat fuzzy, the research literature definitely does not include self-help and other pop psychology books, dictionary and encyclopedia entries, websites, and similar sources that are intended mainly for the general public. These are considered unreliable because they are not reviewed by other researchers and are often based on little more than common sense or personal experience. Wikipedia contains much valuable information, but the fact that its authors are anonymous and may not have any formal training or expertise in that subject area, and its content continually changes makes it unsuitable as a basis of sound scientific research. For our purposes, it helps to define the research literature as consisting almost entirely of two types of sources: articles in professional journals, and scholarly books in psychology and related fields.

Professional Journals

Professional journals  are periodicals that publish original research articles. There are thousands of professional journals that publish research in psychology and related fields. They are usually published monthly or quarterly in individual issues, each of which contains several articles. The issues are organized into volumes, which usually consist of all the issues for a calendar year. Some journals are published in hard copy only, others in both hard copy and electronic form, and still others in electronic form only.

Most articles in professional journals are one of two basic types: empirical research reports and review articles.  Empirical research reports  describe one or more new empirical studies conducted by the authors. They introduce a research question, explain why it is interesting, review previous research, describe their method and results, and draw their conclusions. Review articles  summarize previously published research on a topic and usually present new ways to organize or explain the results. When a review article is devoted primarily to presenting a new theory, it is often referred to as a theoretical article .

Figure 2.6 Small Sample of the Thousands of Professional Journals That Publish Research in Psychology and Related Fields

Most professional journals in psychology undergo a process of  double-blind peer review . Researchers who want to publish their work in the journal submit a manuscript to the editor—who is generally an established researcher too—who in turn sends it to two or three experts on the topic. Each reviewer reads the manuscript, writes a critical but constructive review, and sends the review back to the editor along with his or her recommendations. The editor then decides whether to accept the article for publication, ask the authors to make changes and resubmit it for further consideration, or reject it outright. In any case, the editor forwards the reviewers’ written comments to the researchers so that they can revise their manuscript accordingly. This entire process is double-blind, as the reviewers do not know the identity of the researcher(s), and vice versa. Double-blind peer review is helpful because it ensures that the work meets basic standards of the field before it can enter the research literature. However, in order to increase transparency and accountability some newer open access journals (e.g., Frontiers in Psychology) utilize an open peer review process wherein the identities of the reviewers (which remain concealed during the peer review process) are published alongside the journal article.

Scholarly Books

Scholarly books  are books written by researchers and practitioners mainly for use by other researchers and practitioners. A  monograph  is written by a single author or a small group of authors and usually gives a coherent presentation of a topic much like an extended review article.  Edited volumes have an editor or a small group of editors who recruit many authors to write separate chapters on different aspects of the same topic. Although edited volumes can also give a coherent presentation of the topic, it is not unusual for each chapter to take a different perspective or even for the authors of different chapters to openly disagree with each other. In general, scholarly books undergo a peer review process similar to that used by professional journals.

Literature Search Strategies

Using psycinfo and other databases.

The primary method used to search the research literature involves using one or more electronic databases. These include Academic Search Premier, JSTOR, and ProQuest for all academic disciplines, ERIC for education, and PubMed for medicine and related fields. The most important for our purposes, however, is PsycINFO, which is produced by the APA. PsycINFO is so comprehensive—covering thousands of professional journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous with the research literature in psychology. Like most such databases, PsycINFO is usually available through your university library.

PsycINFO consists of individual records for each article, book chapter, or book in the database. Each record includes basic publication information, an abstract or summary of the work (like the one presented at the start of this chapter), and a list of other works cited by that work. A computer interface allows entering one or more search terms and returns any records that contain those search terms. (These interfaces are provided by different vendors and therefore can look somewhat different depending on the library you use.) Each record also contains lists of keywords that describe the content of the work and also a list of index terms. The index terms are especially helpful because they are standardized. Research on differences between women and men, for example, is always indexed under “Human Sex Differences.” Research on notetaking is always indexed under the term “Learning Strategies.” If you do not know the appropriate index terms, PsycINFO includes a thesaurus that can help you find them.

Given that there are nearly four million records in PsycINFO, you may have to try a variety of search terms in different combinations and at different levels of specificity before you find what you are looking for. Imagine, for example, that you are interested in the question of whether women and men differ in terms of their ability to recall experiences from when they were very young. If you were to enter “memory for early experiences” as your search term, PsycINFO would return only six records, most of which are not particularly relevant to your question. However, if you were to enter the search term “memory,” it would return 149,777 records—far too many to look through individually. This is where the thesaurus helps. Entering “memory” into the thesaurus provides several more specific index terms—one of which is “early memories.” While searching for “early memories” among the index terms returns 1,446 records—still too many too look through individually—combining it with “human sex differences” as a second search term returns 37 articles, many of which are highly relevant to the topic.

QR code that links to PsycINFO video

Depending on the vendor that provides the interface to PsycINFO, you may be able to save, print, or e-mail the relevant PsycINFO records. The records might even contain links to full-text copies of the works themselves. (PsycARTICLES is a database that provides full-text access to articles in all journals published by the APA.) If not, and you want a copy of the work, you will have to find out if your library carries the journal or has the book and the hard copy on the library shelves. Be sure to ask a librarian if you need help.

Using Other Search Techniques

QR code that links to Google Scholar video

In addition to entering search terms into PsycINFO and other databases, there are several other techniques you can use to search the research literature. First, if you have one good article or book chapter on your topic—a recent review article is best—you can look through the reference list of that article for other relevant articles, books, and book chapters. In fact, you should do this with any relevant article or book chapter you find. You can also start with a classic article or book chapter on your topic, find its record in PsycINFO (by entering the author’s name or article’s title as a search term), and link from there to a list of other works in PsycINFO that cite that classic article. This works because other researchers working on your topic are likely to be aware of the classic article and cite it in their own work. You can also do a general Internet search using search terms related to your topic or the name of a researcher who conducts research on your topic. This might lead you directly to works that are part of the research literature (e.g., articles in open-access journals or posted on researchers’ own websites). The search engine Google Scholar is especially useful for this purpose. A general Internet search might also lead you to websites that are not part of the research literature but might provide references to works that are. Finally, you can talk to people (e.g., your instructor or other faculty members in psychology) who know something about your topic and can suggest relevant articles and book chapters.

What to Search For

When you do a literature review, you need to be selective. Not every article, book chapter, and book that relates to your research idea or question will be worth obtaining, reading, and integrating into your review. Instead, you want to focus on sources that help you do four basic things: (a) refine your research question, (b) identify appropriate research methods, (c) place your research in the context of previous research, and (d) write an effective research report. Several basic principles can help you find the most useful sources.

First, it is best to focus on recent research, keeping in mind that what counts as recent depends on the topic. For newer topics that are actively being studied, “recent” might mean published in the past year or two. For older topics that are receiving less attention right now, “recent” might mean within the past 10 years. You will get a feel for what counts as recent for your topic when you start your literature search. A good general rule, however, is to start with sources published in the past five years. The main exception to this rule would be classic articles that turn up in the reference list of nearly every other source. If other researchers think that this work is important, even though it is old, then by all means you should include it in your review.

Second, you should look for review articles on your topic because they will provide a useful overview of it—often discussing important definitions, results, theories, trends, and controversies—giving you a good sense of where your own research fits into the literature. You should also look for empirical research reports addressing your question or similar questions, which can give you ideas about how to operationally define your variables and collect your data. As a general rule, it is good to use methods that others have already used successfully unless you have good reasons not to. Finally, you should look for sources that provide information that can help you argue for the interestingness of your research question. For a study on the effects of cell phone use on driving ability, for example, you might look for information about how widespread cell phone use is, how frequent and costly motor vehicle crashes are, and so on.

How many sources are enough for your literature review? This is a difficult question because it depends on how extensively your topic has been studied and also on your own goals. One study found that across a variety of professional journals in psychology, the average number of sources cited per article was about 50 (Adair & Vohra, 2003) [1] . This gives a rough idea of what professional researchers consider to be adequate. As a student, you might be assigned a much lower minimum number of references to use, but the principles for selecting the most useful ones remain the same.

Key Takeaways

  • The research literature in psychology is all the published research in psychology, consisting primarily of articles in professional journals and scholarly books.
  • Early in the research process, it is important to conduct a review of the research literature on your topic to refine your research question, identify appropriate research methods, place your question in the context of other research, and prepare to write an effective research report.
  • There are several strategies for finding previous research on your topic. Among the best is using PsycINFO, a computer database that catalogs millions of articles, books, and book chapters in psychology and related fields.
  • Practice: Use the techniques discussed in this section to find 10 journal articles and book chapters on one of the following research ideas: memory for smells, aggressive driving, the causes of narcissistic personality disorder, the functions of the intraparietal sulcus, or prejudice against the physically handicapped.
  • Watch the following video clip produced by UBCiSchool about how to read an academic paper (without losing your mind):

QR code that links to UBCiSchool video

Video Attributions

  • “ Sample PsycINFO Search on EBSCOhost ” by APA Publishing Training . Standard YouTube Licence.
  • “ Using Google Scholar (CLIP) ” by clipinfolit . CC BY (Attribution)
  • “ How to Read an Academic Paper ” by UBCiSchool . CC BY (Attribution)
  • Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58 , 15–23. ↵

All the published research in a particular field.

Periodicals that publish original research articles.

A type of research article which describes one or more new empirical studies conducted by the authors.

A type of research article that summarizes previously published research on a topic and usually presents new ways to organize or explain the results.

A type of review article primarily devoted to presenting a new theory.

Books written by researchers and practitioners mainly for sue by other researchers and practitioners.

Type of scholarly book written by a single author or small group of authors, coherently presents a topic much like an extended review article.

A type of scholarly book in which an editor or small group of editors recruit many authors to write separate chapters on different aspects of the same topic.

An electronic database covering thousands of professional journals and scholarly books produced by the APA.

Research Methods in Psychology - 2nd Canadian Edition by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

use of research literature

Harvey Cushing/John Hay Whitney Medical Library

  • Collections
  • Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

  • Biomedical Databases
  • Global (Public Health) Databases
  • Soc. Sci., History, and Law Databases
  • Grey Literature
  • Trials Registers
  • Data and Statistics
  • Public Policy
  • Google Tips
  • Recommended Books
  • Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

  • Explains the background of research on a topic.
  • Demonstrates why a topic is significant to a subject area.
  • Discovers relationships between research studies/ideas.
  • Identifies major themes, concepts, and researchers on a topic.
  • Identifies critical gaps and points of disagreement.
  • Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

Cover Art

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

  • Make sure your research question is not too broad or too narrow.  Is it manageable?
  • Begin writing down terms that are related to your question. These will be useful for searches later.
  • If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

  • This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

  • use the tabs on this guide
  • Find other databases in the Nursing Information Resources web page
  • More on the Medical Library web page
  • ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

  • Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
  • Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
  • Review the abstracts of research studies carefully. This will save you time.
  • Use the bibliographies and references of research studies you find to locate others.
  • Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
  • Ask your librarian for help at any time.
  • Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

  • What was the research question of the study you are reviewing? What were the authors trying to discover?
  • Was the research funded by a source that could influence the findings?
  • What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
  • Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
  • If there are conflicting studies, why do you think that is?
  • How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

  • Review the abstracts carefully.  
  • Keep careful notes so that you may track your thought processes during the research process.
  • Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
  • << Previous: Recommended Books
  • Last Updated: Jan 4, 2024 10:52 AM
  • URL: https://guides.library.yale.edu/YSNDoctoral
  • Methodology
  • Open access
  • Published: 11 October 2016

Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research

  • Stephen J. Gentles 1 , 4 ,
  • Cathy Charles 1 ,
  • David B. Nicholas 2 ,
  • Jenny Ploeg 3 &
  • K. Ann McKibbon 1  

Systematic Reviews volume  5 , Article number:  172 ( 2016 ) Cite this article

49k Accesses

26 Citations

13 Altmetric

Metrics details

Overviews of methods are potentially useful means to increase clarity and enhance collective understanding of specific methods topics that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness. This type of review represents a distinct literature synthesis method, although to date, its methodology remains relatively undeveloped despite several aspects that demand unique review procedures. The purpose of this paper is to initiate discussion about what a rigorous systematic approach to reviews of methods, referred to here as systematic methods overviews , might look like by providing tentative suggestions for approaching specific challenges likely to be encountered. The guidance offered here was derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research.

The guidance is organized into several principles that highlight specific objectives for this type of review given the common challenges that must be overcome to achieve them. Optional strategies for achieving each principle are also proposed, along with discussion of how they were successfully implemented in the overview on sampling. We describe seven paired principles and strategies that address the following aspects: delimiting the initial set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology used to describe specific methods topics, and generating rigorous verifiable analytic interpretations. Since a broad aim in systematic methods overviews is to describe and interpret the relevant literature in qualitative terms, we suggest that iterative decision making at various stages of the review process, and a rigorous qualitative approach to analysis are necessary features of this review type.

Conclusions

We believe that the principles and strategies provided here will be useful to anyone choosing to undertake a systematic methods overview. This paper represents an initial effort to promote high quality critical evaluations of the literature regarding problematic methods topics, which have the potential to promote clearer, shared understandings, and accelerate advances in research methods. Further work is warranted to develop more definitive guidance.

Peer Review reports

While reviews of methods are not new, they represent a distinct review type whose methodology remains relatively under-addressed in the literature despite the clear implications for unique review procedures. One of few examples to describe it is a chapter containing reflections of two contributing authors in a book of 21 reviews on methodological topics compiled for the British National Health Service, Health Technology Assessment Program [ 1 ]. Notable is their observation of how the differences between the methods reviews and conventional quantitative systematic reviews, specifically attributable to their varying content and purpose, have implications for defining what qualifies as systematic. While the authors describe general aspects of “systematicity” (including rigorous application of a methodical search, abstraction, and analysis), they also describe a high degree of variation within the category of methods reviews itself and so offer little in the way of concrete guidance. In this paper, we present tentative concrete guidance, in the form of a preliminary set of proposed principles and optional strategies, for a rigorous systematic approach to reviewing and evaluating the literature on quantitative or qualitative methods topics. For purposes of this article, we have used the term systematic methods overview to emphasize the notion of a systematic approach to such reviews.

The conventional focus of rigorous literature reviews (i.e., review types for which systematic methods have been codified, including the various approaches to quantitative systematic reviews [ 2 – 4 ], and the numerous forms of qualitative and mixed methods literature synthesis [ 5 – 10 ]) is to synthesize empirical research findings from multiple studies. By contrast, the focus of overviews of methods, including the systematic approach we advocate, is to synthesize guidance on methods topics. The literature consulted for such reviews may include the methods literature, methods-relevant sections of empirical research reports, or both. Thus, this paper adds to previous work published in this journal—namely, recent preliminary guidance for conducting reviews of theory [ 11 ]—that has extended the application of systematic review methods to novel review types that are concerned with subject matter other than empirical research findings.

Published examples of methods overviews illustrate the varying objectives they can have. One objective is to establish methodological standards for appraisal purposes. For example, reviews of existing quality appraisal standards have been used to propose universal standards for appraising the quality of primary qualitative research [ 12 ] or evaluating qualitative research reports [ 13 ]. A second objective is to survey the methods-relevant sections of empirical research reports to establish current practices on methods use and reporting practices, which Moher and colleagues [ 14 ] recommend as a means for establishing the needs to be addressed in reporting guidelines (see, for example [ 15 , 16 ]). A third objective for a methods review is to offer clarity and enhance collective understanding regarding a specific methods topic that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness within the available methods literature. An example of this is a overview whose objective was to review the inconsistent definitions of intention-to-treat analysis (the methodologically preferred approach to analyze randomized controlled trial data) that have been offered in the methods literature and propose a solution for improving conceptual clarity [ 17 ]. Such reviews are warranted because students and researchers who must learn or apply research methods typically lack the time to systematically search, retrieve, review, and compare the available literature to develop a thorough and critical sense of the varied approaches regarding certain controversial or ambiguous methods topics.

While systematic methods overviews , as a review type, include both reviews of the methods literature and reviews of methods-relevant sections from empirical study reports, the guidance provided here is primarily applicable to reviews of the methods literature since it was derived from the experience of conducting such a review [ 18 ], described below. To our knowledge, there are no well-developed proposals on how to rigorously conduct such reviews. Such guidance would have the potential to improve the thoroughness and credibility of critical evaluations of the methods literature, which could increase their utility as a tool for generating understandings that advance research methods, both qualitative and quantitative. Our aim in this paper is thus to initiate discussion about what might constitute a rigorous approach to systematic methods overviews. While we hope to promote rigor in the conduct of systematic methods overviews wherever possible, we do not wish to suggest that all methods overviews need be conducted to the same standard. Rather, we believe that the level of rigor may need to be tailored pragmatically to the specific review objectives, which may not always justify the resource requirements of an intensive review process.

The example systematic methods overview on sampling in qualitative research

The principles and strategies we propose in this paper are derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research [ 18 ]. The main objective of that methods overview was to bring clarity and deeper understanding of the prominent concepts related to sampling in qualitative research (purposeful sampling strategies, saturation, etc.). Specifically, we interpreted the available guidance, commenting on areas lacking clarity, consistency, or comprehensiveness (without proposing any recommendations on how to do sampling). This was achieved by a comparative and critical analysis of publications representing the most influential (i.e., highly cited) guidance across several methodological traditions in qualitative research.

The specific methods and procedures for the overview on sampling [ 18 ] from which our proposals are derived were developed both after soliciting initial input from local experts in qualitative research and an expert health librarian (KAM) and through ongoing careful deliberation throughout the review process. To summarize, in that review, we employed a transparent and rigorous approach to search the methods literature, selected publications for inclusion according to a purposeful and iterative process, abstracted textual data using structured abstraction forms, and analyzed (synthesized) the data using a systematic multi-step approach featuring abstraction of text, summary of information in matrices, and analytic comparisons.

For this article, we reflected on both the problems and challenges encountered at different stages of the review and our means for selecting justifiable procedures to deal with them. Several principles were then derived by considering the generic nature of these problems, while the generalizable aspects of the procedures used to address them formed the basis of optional strategies. Further details of the specific methods and procedures used in the overview on qualitative sampling are provided below to illustrate both the types of objectives and challenges that reviewers will likely need to consider and our approach to implementing each of the principles and strategies.

Organization of the guidance into principles and strategies

For the purposes of this article, principles are general statements outlining what we propose are important aims or considerations within a particular review process, given the unique objectives or challenges to be overcome with this type of review. These statements follow the general format, “considering the objective or challenge of X, we propose Y to be an important aim or consideration.” Strategies are optional and flexible approaches for implementing the previous principle outlined. Thus, generic challenges give rise to principles, which in turn give rise to strategies.

We organize the principles and strategies below into three sections corresponding to processes characteristic of most systematic literature synthesis approaches: literature identification and selection ; data abstraction from the publications selected for inclusion; and analysis , including critical appraisal and synthesis of the abstracted data. Within each section, we also describe the specific methodological decisions and procedures used in the overview on sampling in qualitative research [ 18 ] to illustrate how the principles and strategies for each review process were applied and implemented in a specific case. We expect this guidance and accompanying illustrations will be useful for anyone considering engaging in a methods overview, particularly those who may be familiar with conventional systematic review methods but may not yet appreciate some of the challenges specific to reviewing the methods literature.

Results and discussion

Literature identification and selection.

The identification and selection process includes search and retrieval of publications and the development and application of inclusion and exclusion criteria to select the publications that will be abstracted and analyzed in the final review. Literature identification and selection for overviews of the methods literature is challenging and potentially more resource-intensive than for most reviews of empirical research. This is true for several reasons that we describe below, alongside discussion of the potential solutions. Additionally, we suggest in this section how the selection procedures can be chosen to match the specific analytic approach used in methods overviews.

Delimiting a manageable set of publications

One aspect of methods overviews that can make identification and selection challenging is the fact that the universe of literature containing potentially relevant information regarding most methods-related topics is expansive and often unmanageably so. Reviewers are faced with two large categories of literature: the methods literature , where the possible publication types include journal articles, books, and book chapters; and the methods-relevant sections of empirical study reports , where the possible publication types include journal articles, monographs, books, theses, and conference proceedings. In our systematic overview of sampling in qualitative research, exhaustively searching (including retrieval and first-pass screening) all publication types across both categories of literature for information on a single methods-related topic was too burdensome to be feasible. The following proposed principle follows from the need to delimit a manageable set of literature for the review.

Principle #1:

Considering the broad universe of potentially relevant literature, we propose that an important objective early in the identification and selection stage is to delimit a manageable set of methods-relevant publications in accordance with the objectives of the methods overview.

Strategy #1:

To limit the set of methods-relevant publications that must be managed in the selection process, reviewers have the option to initially review only the methods literature, and exclude the methods-relevant sections of empirical study reports, provided this aligns with the review’s particular objectives.

We propose that reviewers are justified in choosing to select only the methods literature when the objective is to map out the range of recognized concepts relevant to a methods topic, to summarize the most authoritative or influential definitions or meanings for methods-related concepts, or to demonstrate a problematic lack of clarity regarding a widely established methods-related concept and potentially make recommendations for a preferred approach to the methods topic in question. For example, in the case of the methods overview on sampling [ 18 ], the primary aim was to define areas lacking in clarity for multiple widely established sampling-related topics. In the review on intention-to-treat in the context of missing outcome data [ 17 ], the authors identified a lack of clarity based on multiple inconsistent definitions in the literature and went on to recommend separating the issue of how to handle missing outcome data from the issue of whether an intention-to-treat analysis can be claimed.

In contrast to strategy #1, it may be appropriate to select the methods-relevant sections of empirical study reports when the objective is to illustrate how a methods concept is operationalized in research practice or reported by authors. For example, one could review all the publications in 2 years’ worth of issues of five high-impact field-related journals to answer questions about how researchers describe implementing a particular method or approach, or to quantify how consistently they define or report using it. Such reviews are often used to highlight gaps in the reporting practices regarding specific methods, which may be used to justify items to address in reporting guidelines (for example, [ 14 – 16 ]).

It is worth recognizing that other authors have advocated broader positions regarding the scope of literature to be considered in a review, expanding on our perspective. Suri [ 10 ] (who, like us, emphasizes how different sampling strategies are suitable for different literature synthesis objectives) has, for example, described a two-stage literature sampling procedure (pp. 96–97). First, reviewers use an initial approach to conduct a broad overview of the field—for reviews of methods topics, this would entail an initial review of the research methods literature. This is followed by a second more focused stage in which practical examples are purposefully selected—for methods reviews, this would involve sampling the empirical literature to illustrate key themes and variations. While this approach is seductive in its capacity to generate more in depth and interpretive analytic findings, some reviewers may consider it too resource-intensive to include the second step no matter how selective the purposeful sampling. In the overview on sampling where we stopped after the first stage [ 18 ], we discussed our selective focus on the methods literature as a limitation that left opportunities for further analysis of the literature. We explicitly recommended, for example, that theoretical sampling was a topic for which a future review of the methods sections of empirical reports was justified to answer specific questions identified in the primary review.

Ultimately, reviewers must make pragmatic decisions that balance resource considerations, combined with informed predictions about the depth and complexity of literature available on their topic, with the stated objectives of their review. The remaining principles and strategies apply primarily to overviews that include the methods literature, although some aspects may be relevant to reviews that include empirical study reports.

Searching beyond standard bibliographic databases

An important reality affecting identification and selection in overviews of the methods literature is the increased likelihood for relevant publications to be located in sources other than journal articles (which is usually not the case for overviews of empirical research, where journal articles generally represent the primary publication type). In the overview on sampling [ 18 ], out of 41 full-text publications retrieved and reviewed, only 4 were journal articles, while 37 were books or book chapters. Since many books and book chapters did not exist electronically, their full text had to be physically retrieved in hardcopy, while 11 publications were retrievable only through interlibrary loan or purchase request. The tasks associated with such retrieval are substantially more time-consuming than electronic retrieval. Since a substantial proportion of methods-related guidance may be located in publication types that are less comprehensively indexed in standard bibliographic databases, identification and retrieval thus become complicated processes.

Principle #2:

Considering that important sources of methods guidance can be located in non-journal publication types (e.g., books, book chapters) that tend to be poorly indexed in standard bibliographic databases, it is important to consider alternative search methods for identifying relevant publications to be further screened for inclusion.

Strategy #2:

To identify books, book chapters, and other non-journal publication types not thoroughly indexed in standard bibliographic databases, reviewers may choose to consult one or more of the following less standard sources: Google Scholar, publisher web sites, or expert opinion.

In the case of the overview on sampling in qualitative research [ 18 ], Google Scholar had two advantages over other standard bibliographic databases: it indexes and returns records of books and book chapters likely to contain guidance on qualitative research methods topics; and it has been validated as providing higher citation counts than ISI Web of Science (a producer of numerous bibliographic databases accessible through institutional subscription) for several non-biomedical disciplines including the social sciences where qualitative research methods are prominently used [ 19 – 21 ]. While we identified numerous useful publications by consulting experts, the author publication lists generated through Google Scholar searches were uniquely useful to identify more recent editions of methods books identified by experts.

Searching without relevant metadata

Determining what publications to select for inclusion in the overview on sampling [ 18 ] could only rarely be accomplished by reviewing the publication’s metadata. This was because for the many books and other non-journal type publications we identified as possibly relevant, the potential content of interest would be located in only a subsection of the publication. In this common scenario for reviews of the methods literature (as opposed to methods overviews that include empirical study reports), reviewers will often be unable to employ standard title, abstract, and keyword database searching or screening as a means for selecting publications.

Principle #3:

Considering that the presence of information about the topic of interest may not be indicated in the metadata for books and similar publication types, it is important to consider other means of identifying potentially useful publications for further screening.

Strategy #3:

One approach to identifying potentially useful books and similar publication types is to consider what classes of such publications (e.g., all methods manuals for a certain research approach) are likely to contain relevant content, then identify, retrieve, and review the full text of corresponding publications to determine whether they contain information on the topic of interest.

In the example of the overview on sampling in qualitative research [ 18 ], the topic of interest (sampling) was one of numerous topics covered in the general qualitative research methods manuals. Consequently, examples from this class of publications first had to be identified for retrieval according to non-keyword-dependent criteria. Thus, all methods manuals within the three research traditions reviewed (grounded theory, phenomenology, and case study) that might contain discussion of sampling were sought through Google Scholar and expert opinion, their full text obtained, and hand-searched for relevant content to determine eligibility. We used tables of contents and index sections of books to aid this hand searching.

Purposefully selecting literature on conceptual grounds

A final consideration in methods overviews relates to the type of analysis used to generate the review findings. Unlike quantitative systematic reviews where reviewers aim for accurate or unbiased quantitative estimates—something that requires identifying and selecting the literature exhaustively to obtain all relevant data available (i.e., a complete sample)—in methods overviews, reviewers must describe and interpret the relevant literature in qualitative terms to achieve review objectives. In other words, the aim in methods overviews is to seek coverage of the qualitative concepts relevant to the methods topic at hand. For example, in the overview of sampling in qualitative research [ 18 ], achieving review objectives entailed providing conceptual coverage of eight sampling-related topics that emerged as key domains. The following principle recognizes that literature sampling should therefore support generating qualitative conceptual data as the input to analysis.

Principle #4:

Since the analytic findings of a systematic methods overview are generated through qualitative description and interpretation of the literature on a specified topic, selection of the literature should be guided by a purposeful strategy designed to achieve adequate conceptual coverage (i.e., representing an appropriate degree of variation in relevant ideas) of the topic according to objectives of the review.

Strategy #4:

One strategy for choosing the purposeful approach to use in selecting the literature according to the review objectives is to consider whether those objectives imply exploring concepts either at a broad overview level, in which case combining maximum variation selection with a strategy that limits yield (e.g., critical case, politically important, or sampling for influence—described below) may be appropriate; or in depth, in which case purposeful approaches aimed at revealing innovative cases will likely be necessary.

In the methods overview on sampling, the implied scope was broad since we set out to review publications on sampling across three divergent qualitative research traditions—grounded theory, phenomenology, and case study—to facilitate making informative conceptual comparisons. Such an approach would be analogous to maximum variation sampling.

At the same time, the purpose of that review was to critically interrogate the clarity, consistency, and comprehensiveness of literature from these traditions that was “most likely to have widely influenced students’ and researchers’ ideas about sampling” (p. 1774) [ 18 ]. In other words, we explicitly set out to review and critique the most established and influential (and therefore dominant) literature, since this represents a common basis of knowledge among students and researchers seeking understanding or practical guidance on sampling in qualitative research. To achieve this objective, we purposefully sampled publications according to the criterion of influence , which we operationalized as how often an author or publication has been referenced in print or informal discourse. This second sampling approach also limited the literature we needed to consider within our broad scope review to a manageable amount.

To operationalize this strategy of sampling for influence , we sought to identify both the most influential authors within a qualitative research tradition (all of whose citations were subsequently screened) and the most influential publications on the topic of interest by non-influential authors. This involved a flexible approach that combined multiple indicators of influence to avoid the dilemma that any single indicator might provide inadequate coverage. These indicators included bibliometric data (h-index for author influence [ 22 ]; number of cites for publication influence), expert opinion, and cross-references in the literature (i.e., snowball sampling). As a final selection criterion, a publication was included only if it made an original contribution in terms of novel guidance regarding sampling or a related concept; thus, purely secondary sources were excluded. Publish or Perish software (Anne-Wil Harzing; available at http://www.harzing.com/resources/publish-or-perish ) was used to generate bibliometric data via the Google Scholar database. Figure  1 illustrates how identification and selection in the methods overview on sampling was a multi-faceted and iterative process. The authors selected as influential, and the publications selected for inclusion or exclusion are listed in Additional file 1 (Matrices 1, 2a, 2b).

Literature identification and selection process used in the methods overview on sampling [ 18 ]

In summary, the strategies of seeking maximum variation and sampling for influence were employed in the sampling overview to meet the specific review objectives described. Reviewers will need to consider the full range of purposeful literature sampling approaches at their disposal in deciding what best matches the specific aims of their own reviews. Suri [ 10 ] has recently retooled Patton’s well-known typology of purposeful sampling strategies (originally intended for primary research) for application to literature synthesis, providing a useful resource in this respect.

Data abstraction

The purpose of data abstraction in rigorous literature reviews is to locate and record all data relevant to the topic of interest from the full text of included publications, making them available for subsequent analysis. Conventionally, a data abstraction form—consisting of numerous distinct conceptually defined fields to which corresponding information from the source publication is recorded—is developed and employed. There are several challenges, however, to the processes of developing the abstraction form and abstracting the data itself when conducting methods overviews, which we address here. Some of these problems and their solutions may be familiar to those who have conducted qualitative literature syntheses, which are similarly conceptual.

Iteratively defining conceptual information to abstract

In the overview on sampling [ 18 ], while we surveyed multiple sources beforehand to develop a list of concepts relevant for abstraction (e.g., purposeful sampling strategies, saturation, sample size), there was no way for us to anticipate some concepts prior to encountering them in the review process. Indeed, in many cases, reviewers are unable to determine the complete set of methods-related concepts that will be the focus of the final review a priori without having systematically reviewed the publications to be included. Thus, defining what information to abstract beforehand may not be feasible.

Principle #5:

Considering the potential impracticality of defining a complete set of relevant methods-related concepts from a body of literature one has not yet systematically read, selecting and defining fields for data abstraction must often be undertaken iteratively. Thus, concepts to be abstracted can be expected to grow and change as data abstraction proceeds.

Strategy #5:

Reviewers can develop an initial form or set of concepts for abstraction purposes according to standard methods (e.g., incorporating expert feedback, pilot testing) and remain attentive to the need to iteratively revise it as concepts are added or modified during the review. Reviewers should document revisions and return to re-abstract data from previously abstracted publications as the new data requirements are determined.

In the sampling overview [ 18 ], we developed and maintained the abstraction form in Microsoft Word. We derived the initial set of abstraction fields from our own knowledge of relevant sampling-related concepts, consultation with local experts, and reviewing a pilot sample of publications. Since the publications in this review included a large proportion of books, the abstraction process often began by flagging the broad sections within a publication containing topic-relevant information for detailed review to identify text to abstract. When reviewing flagged text, the reviewer occasionally encountered an unanticipated concept significant enough to warrant being added as a new field to the abstraction form. For example, a field was added to capture how authors described the timing of sampling decisions, whether before (a priori) or after (ongoing) starting data collection, or whether this was unclear. In these cases, we systematically documented the modification to the form and returned to previously abstracted publications to abstract any information that might be relevant to the new field.

The logic of this strategy is analogous to the logic used in a form of research synthesis called best fit framework synthesis (BFFS) [ 23 – 25 ]. In that method, reviewers initially code evidence using an a priori framework they have selected. When evidence cannot be accommodated by the selected framework, reviewers then develop new themes or concepts from which they construct a new expanded framework. Both the strategy proposed and the BFFS approach to research synthesis are notable for their rigorous and transparent means to adapt a final set of concepts to the content under review.

Accounting for inconsistent terminology

An important complication affecting the abstraction process in methods overviews is that the language used by authors to describe methods-related concepts can easily vary across publications. For example, authors from different qualitative research traditions often use different terms for similar methods-related concepts. Furthermore, as we found in the sampling overview [ 18 ], there may be cases where no identifiable term, phrase, or label for a methods-related concept is used at all, and a description of it is given instead. This can make searching the text for relevant concepts based on keywords unreliable.

Principle #6:

Since accepted terms may not be used consistently to refer to methods concepts, it is necessary to rely on the definitions for concepts, rather than keywords, to identify relevant information in the publication to abstract.

Strategy #6:

An effective means to systematically identify relevant information is to develop and iteratively adjust written definitions for key concepts (corresponding to abstraction fields) that are consistent with and as inclusive of as much of the literature reviewed as possible. Reviewers then seek information that matches these definitions (rather than keywords) when scanning a publication for relevant data to abstract.

In the abstraction process for the sampling overview [ 18 ], we noted the several concepts of interest to the review for which abstraction by keyword was particularly problematic due to inconsistent terminology across publications: sampling , purposeful sampling , sampling strategy , and saturation (for examples, see Additional file 1 , Matrices 3a, 3b, 4). We iteratively developed definitions for these concepts by abstracting text from publications that either provided an explicit definition or from which an implicit definition could be derived, which was recorded in fields dedicated to the concept’s definition. Using a method of constant comparison, we used text from definition fields to inform and modify a centrally maintained definition of the corresponding concept to optimize its fit and inclusiveness with the literature reviewed. Table  1 shows, as an example, the final definition constructed in this way for one of the central concepts of the review, qualitative sampling .

We applied iteratively developed definitions when making decisions about what specific text to abstract for an existing field, which allowed us to abstract concept-relevant data even if no recognized keyword was used. For example, this was the case for the sampling-related concept, saturation , where the relevant text available for abstraction in one publication [ 26 ]—“to continue to collect data until nothing new was being observed or recorded, no matter how long that takes”—was not accompanied by any term or label whatsoever.

This comparative analytic strategy (and our approach to analysis more broadly as described in strategy #7, below) is analogous to the process of reciprocal translation —a technique first introduced for meta-ethnography by Noblit and Hare [ 27 ] that has since been recognized as a common element in a variety of qualitative metasynthesis approaches [ 28 ]. Reciprocal translation, taken broadly, involves making sense of a study’s findings in terms of the findings of the other studies included in the review. In practice, it has been operationalized in different ways. Melendez-Torres and colleagues developed a typology from their review of the metasynthesis literature, describing four overlapping categories of specific operations undertaken in reciprocal translation: visual representation, key paper integration, data reduction and thematic extraction, and line-by-line coding [ 28 ]. The approaches suggested in both strategies #6 and #7, with their emphasis on constant comparison, appear to fall within the line-by-line coding category.

Generating credible and verifiable analytic interpretations

The analysis in a systematic methods overview must support its more general objective, which we suggested above is often to offer clarity and enhance collective understanding regarding a chosen methods topic. In our experience, this involves describing and interpreting the relevant literature in qualitative terms. Furthermore, any interpretative analysis required may entail reaching different levels of abstraction, depending on the more specific objectives of the review. For example, in the overview on sampling [ 18 ], we aimed to produce a comparative analysis of how multiple sampling-related topics were treated differently within and among different qualitative research traditions. To promote credibility of the review, however, not only should one seek a qualitative analytic approach that facilitates reaching varying levels of abstraction but that approach must also ensure that abstract interpretations are supported and justified by the source data and not solely the product of the analyst’s speculative thinking.

Principle #7:

Considering the qualitative nature of the analysis required in systematic methods overviews, it is important to select an analytic method whose interpretations can be verified as being consistent with the literature selected, regardless of the level of abstraction reached.

Strategy #7:

We suggest employing the constant comparative method of analysis [ 29 ] because it supports developing and verifying analytic links to the source data throughout progressively interpretive or abstract levels. In applying this approach, we advise a rigorous approach, documenting how supportive quotes or references to the original texts are carried forward in the successive steps of analysis to allow for easy verification.

The analytic approach used in the methods overview on sampling [ 18 ] comprised four explicit steps, progressing in level of abstraction—data abstraction, matrices, narrative summaries, and final analytic conclusions (Fig.  2 ). While we have positioned data abstraction as the second stage of the generic review process (prior to Analysis), above, we also considered it as an initial step of analysis in the sampling overview for several reasons. First, it involved a process of constant comparisons and iterative decision-making about the fields to add or define during development and modification of the abstraction form, through which we established the range of concepts to be addressed in the review. At the same time, abstraction involved continuous analytic decisions about what textual quotes (ranging in size from short phrases to numerous paragraphs) to record in the fields thus created. This constant comparative process was analogous to open coding in which textual data from publications was compared to conceptual fields (equivalent to codes) or to other instances of data previously abstracted when constructing definitions to optimize their fit with the overall literature as described in strategy #6. Finally, in the data abstraction step, we also recorded our first interpretive thoughts in dedicated fields, providing initial material for the more abstract analytic steps.

Summary of progressive steps of analysis used in the methods overview on sampling [ 18 ]

In the second step of the analysis, we constructed topic-specific matrices , or tables, by copying relevant quotes from abstraction forms into the appropriate cells of matrices (for the complete set of analytic matrices developed in the sampling review, see Additional file 1 (matrices 3 to 10)). Each matrix ranged from one to five pages; row headings, nested three-deep, identified the methodological tradition, author, and publication, respectively; and column headings identified the concepts, which corresponded to abstraction fields. Matrices thus allowed us to make further comparisons across methodological traditions, and between authors within a tradition. In the third step of analysis, we recorded our comparative observations as narrative summaries , in which we used illustrative quotes more sparingly. In the final step, we developed analytic conclusions based on the narrative summaries about the sampling-related concepts within each methodological tradition for which clarity, consistency, or comprehensiveness of the available guidance appeared to be lacking. Higher levels of analysis thus built logically from the lower levels, enabling us to easily verify analytic conclusions by tracing the support for claims by comparing the original text of publications reviewed.

Integrative versus interpretive methods overviews

The analytic product of systematic methods overviews is comparable to qualitative evidence syntheses, since both involve describing and interpreting the relevant literature in qualitative terms. Most qualitative synthesis approaches strive to produce new conceptual understandings that vary in level of interpretation. Dixon-Woods and colleagues [ 30 ] elaborate on a useful distinction, originating from Noblit and Hare [ 27 ], between integrative and interpretive reviews. Integrative reviews focus on summarizing available primary data and involve using largely secure and well defined concepts to do so; definitions are used from an early stage to specify categories for abstraction (or coding) of data, which in turn supports their aggregation; they do not seek as their primary focus to develop or specify new concepts, although they may achieve some theoretical or interpretive functions. For interpretive reviews, meanwhile, the main focus is to develop new concepts and theories that integrate them, with the implication that the concepts developed become fully defined towards the end of the analysis. These two forms are not completely distinct, and “every integrative synthesis will include elements of interpretation, and every interpretive synthesis will include elements of aggregation of data” [ 30 ].

The example methods overview on sampling [ 18 ] could be classified as predominantly integrative because its primary goal was to aggregate influential authors’ ideas on sampling-related concepts; there were also, however, elements of interpretive synthesis since it aimed to develop new ideas about where clarity in guidance on certain sampling-related topics is lacking, and definitions for some concepts were flexible and not fixed until late in the review. We suggest that most systematic methods overviews will be classifiable as predominantly integrative (aggregative). Nevertheless, more highly interpretive methods overviews are also quite possible—for example, when the review objective is to provide a highly critical analysis for the purpose of generating new methodological guidance. In such cases, reviewers may need to sample more deeply (see strategy #4), specifically by selecting empirical research reports (i.e., to go beyond dominant or influential ideas in the methods literature) that are likely to feature innovations or instructive lessons in employing a given method.

In this paper, we have outlined tentative guidance in the form of seven principles and strategies on how to conduct systematic methods overviews, a review type in which methods-relevant literature is systematically analyzed with the aim of offering clarity and enhancing collective understanding regarding a specific methods topic. Our proposals include strategies for delimiting the set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology, and generating credible and verifiable analytic interpretations. We hope the suggestions proposed will be useful to others undertaking reviews on methods topics in future.

As far as we are aware, this is the first published source of concrete guidance for conducting this type of review. It is important to note that our primary objective was to initiate methodological discussion by stimulating reflection on what rigorous methods for this type of review should look like, leaving the development of more complete guidance to future work. While derived from the experience of reviewing a single qualitative methods topic, we believe the principles and strategies provided are generalizable to overviews of both qualitative and quantitative methods topics alike. However, it is expected that additional challenges and insights for conducting such reviews have yet to be defined. Thus, we propose that next steps for developing more definitive guidance should involve an attempt to collect and integrate other reviewers’ perspectives and experiences in conducting systematic methods overviews on a broad range of qualitative and quantitative methods topics. Formalized guidance and standards would improve the quality of future methods overviews, something we believe has important implications for advancing qualitative and quantitative methodology. When undertaken to a high standard, rigorous critical evaluations of the available methods guidance have significant potential to make implicit controversies explicit, and improve the clarity and precision of our understandings of problematic qualitative or quantitative methods issues.

A review process central to most types of rigorous reviews of empirical studies, which we did not explicitly address in a separate review step above, is quality appraisal . The reason we have not treated this as a separate step stems from the different objectives of the primary publications included in overviews of the methods literature (i.e., providing methodological guidance) compared to the primary publications included in the other established review types (i.e., reporting findings from single empirical studies). This is not to say that appraising quality of the methods literature is not an important concern for systematic methods overviews. Rather, appraisal is much more integral to (and difficult to separate from) the analysis step, in which we advocate appraising clarity, consistency, and comprehensiveness—the quality appraisal criteria that we suggest are appropriate for the methods literature. As a second important difference regarding appraisal, we currently advocate appraising the aforementioned aspects at the level of the literature in aggregate rather than at the level of individual publications. One reason for this is that methods guidance from individual publications generally builds on previous literature, and thus we feel that ahistorical judgments about comprehensiveness of single publications lack relevance and utility. Additionally, while different methods authors may express themselves less clearly than others, their guidance can nonetheless be highly influential and useful, and should therefore not be downgraded or ignored based on considerations of clarity—which raises questions about the alternative uses that quality appraisals of individual publications might have. Finally, legitimate variability in the perspectives that methods authors wish to emphasize, and the levels of generality at which they write about methods, makes critiquing individual publications based on the criterion of clarity a complex and potentially problematic endeavor that is beyond the scope of this paper to address. By appraising the current state of the literature at a holistic level, reviewers stand to identify important gaps in understanding that represent valuable opportunities for further methodological development.

To summarize, the principles and strategies provided here may be useful to those seeking to undertake their own systematic methods overview. Additional work is needed, however, to establish guidance that is comprehensive by comparing the experiences from conducting a variety of methods overviews on a range of methods topics. Efforts that further advance standards for systematic methods overviews have the potential to promote high-quality critical evaluations that produce conceptually clear and unified understandings of problematic methods topics, thereby accelerating the advance of research methodology.

Hutton JL, Ashcroft R. What does “systematic” mean for reviews of methods? In: Black N, Brazier J, Fitzpatrick R, Reeves B, editors. Health services research methods: a guide to best practice. London: BMJ Publishing Group; 1998. p. 249–54.

Google Scholar  

Cochrane handbook for systematic reviews of interventions. In. Edited by Higgins JPT, Green S, Version 5.1.0 edn: The Cochrane Collaboration; 2011.

Centre for Reviews and Dissemination: Systematic reviews: CRD’s guidance for undertaking reviews in health care . York: Centre for Reviews and Dissemination; 2009.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700–0.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9(1):59.

Article   PubMed   PubMed Central   Google Scholar  

Kastner M, Tricco AC, Soobiah C, Lillie E, Perrier L, Horsley T, Welch V, Cogo E, Antony J, Straus SE. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012;12(1):1–1.

Article   Google Scholar  

Booth A, Noyes J, Flemming K, Gerhardus A. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. In: Integrate-HTA. 2016.

Booth A, Sutton A, Papaioannou D. Systematic approaches to successful literature review. 2nd ed. London: Sage; 2016.

Hannes K, Lockwood C. Synthesizing qualitative research: choosing the right approach. Chichester: Wiley-Blackwell; 2012.

Suri H. Towards methodologically inclusive research syntheses: expanding possibilities. New York: Routledge; 2014.

Campbell M, Egan M, Lorenc T, Bond L, Popham F, Fenton C, Benzeval M. Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health. Syst Rev. 2014;3(1):1–11.

Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reportingqualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Article   PubMed   Google Scholar  

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78.

Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365(9465):1159–62.

Alshurafa M, Briel M, Akl EA, Haines T, Moayyedi P, Gentles SJ, Rios L, Tran C, Bhatnagar N, Lamontagne F, et al. Inconsistent definitions for intention-to-treat in relation to missing outcome data: systematic review of the methods literature. PLoS One. 2012;7(11):e49163.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gentles SJ, Charles C, Ploeg J, McKibbon KA. Sampling in qualitative research: insights from an overview of the methods literature. Qual Rep. 2015;20(11):1772–89.

Harzing A-W, Alakangas S. Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics. 2016;106(2):787–804.

Harzing A-WK, van der Wal R. Google Scholar as a new source for citation analysis. Ethics Sci Environ Polit. 2008;8(1):61–73.

Kousha K, Thelwall M. Google Scholar citations and Google Web/URL citations: a multi‐discipline exploratory analysis. J Assoc Inf Sci Technol. 2007;58(7):1055–65.

Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–72.

Booth A, Carroll C. How to build up the actionable knowledge base: the role of ‘best fit’ framework synthesis for studies of improvement in healthcare. BMJ Quality Safety. 2015;24(11):700–8.

Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol. 2013;13(1):37.

Carroll C, Booth A, Cooper K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11(1):29.

Cohen MZ, Kahn DL, Steeves DL. Hermeneutic phenomenological research: a practical guide for nurse researchers. Thousand Oaks: Sage; 2000.

Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. Newbury Park: Sage; 1988.

Book   Google Scholar  

Melendez-Torres GJ, Grant S, Bonell C. A systematic review and critical appraisal of qualitative metasynthetic practice in public health to develop a taxonomy of operations of reciprocal translation. Res Synthesis Methods. 2015;6(4):357–71.

Article   CAS   Google Scholar  

Glaser BG, Strauss A. The discovery of grounded theory. Chicago: Aldine; 1967.

Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. Integrative approaches to qualitative and quantitative evidence. In: UK National Health Service. 2004. p. 1–44.

Download references

Acknowledgements

Not applicable.

There was no funding for this work.

Availability of data and materials

The systematic methods overview used as a worked example in this article (Gentles SJ, Charles C, Ploeg J, McKibbon KA: Sampling in qualitative research: insights from an overview of the methods literature. The Qual Rep 2015, 20(11):1772-1789) is available from http://nsuworks.nova.edu/tqr/vol20/iss11/5 .

Authors’ contributions

SJG wrote the first draft of this article, with CC contributing to drafting. All authors contributed to revising the manuscript. All authors except CC (deceased) approved the final draft. SJG, CC, KAB, and JP were involved in developing methods for the systematic methods overview on sampling.

Authors’ information

Competing interests.

The authors declare that they have no competing interests.

Consent for publication

Ethics approval and consent to participate, author information, authors and affiliations.

Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Stephen J. Gentles, Cathy Charles & K. Ann McKibbon

Faculty of Social Work, University of Calgary, Alberta, Canada

David B. Nicholas

School of Nursing, McMaster University, Hamilton, Ontario, Canada

Jenny Ploeg

CanChild Centre for Childhood Disability Research, McMaster University, 1400 Main Street West, IAHS 408, Hamilton, ON, L8S 1C7, Canada

Stephen J. Gentles

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stephen J. Gentles .

Additional information

Cathy Charles is deceased

Additional file

Additional file 1:.

Submitted: Analysis_matrices. (DOC 330 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Gentles, S.J., Charles, C., Nicholas, D.B. et al. Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research. Syst Rev 5 , 172 (2016). https://doi.org/10.1186/s13643-016-0343-0

Download citation

Received : 06 June 2016

Accepted : 14 September 2016

Published : 11 October 2016

DOI : https://doi.org/10.1186/s13643-016-0343-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Systematic review
  • Literature selection
  • Research methods
  • Research methodology
  • Overview of methods
  • Systematic methods overview
  • Review methods

Systematic Reviews

ISSN: 2046-4053

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

use of research literature

Duke University Libraries

Literature Reviews

  • Getting started

What is a literature review?

Why conduct a literature review, stages of a literature review, lit reviews: an overview (video), check out these books.

  • Types of reviews
  • 1. Define your research question
  • 2. Plan your search
  • 3. Search the literature
  • 4. Organize your results
  • 5. Synthesize your findings
  • 6. Write the review
  • Thompson Writing Studio This link opens in a new window
  • Need to write a systematic review? This link opens in a new window

use of research literature

Contact a Librarian

Ask a Librarian

use of research literature

Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject.

Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field.

Analysis: Involves critically evaluating and summarizing key findings, methodologies, and debates found in academic literature.

Identifying Gaps: Aims to pinpoint areas where there is a lack of research or unresolved questions, highlighting opportunities for further investigation.

Contextualization: Enables researchers to understand how their work fits into the broader academic conversation and contributes to the existing body of knowledge.

use of research literature

tl;dr  A literature review critically examines and synthesizes existing scholarly research and publications on a specific topic to provide a comprehensive understanding of the current state of knowledge in the field.

What is a literature review NOT?

❌ An annotated bibliography

❌ Original research

❌ A summary

❌ Something to be conducted at the end of your research

❌ An opinion piece

❌ A chronological compilation of studies

The reason for conducting a literature review is to:

use of research literature

Literature Reviews: An Overview for Graduate Students

While this 9-minute video from NCSU is geared toward graduate students, it is useful for anyone conducting a literature review.

use of research literature

Writing the literature review: A practical guide

Available 3rd floor of Perkins

use of research literature

Writing literature reviews: A guide for students of the social and behavioral sciences

Available online!

use of research literature

So, you have to write a literature review: A guided workbook for engineers

use of research literature

Telling a research story: Writing a literature review

use of research literature

The literature review: Six steps to success

use of research literature

Systematic approaches to a successful literature review

Request from Duke Medical Center Library

use of research literature

Doing a systematic review: A student's guide

  • Next: Types of reviews >>
  • Last Updated: Feb 15, 2024 1:45 PM
  • URL: https://guides.library.duke.edu/lit-reviews

Duke University Libraries

Services for...

  • Faculty & Instructors
  • Graduate Students
  • Undergraduate Students
  • International Students
  • Patrons with Disabilities

Twitter

  • Harmful Language Statement
  • Re-use & Attribution / Privacy
  • Support the Libraries

Creative Commons License

  • UConn Library
  • Literature Review: The What, Why and How-to Guide
  • Introduction

Literature Review: The What, Why and How-to Guide — Introduction

  • Getting Started
  • How to Pick a Topic
  • Strategies to Find Sources
  • Evaluating Sources & Lit. Reviews
  • Tips for Writing Literature Reviews
  • Writing Literature Review: Useful Sites
  • Citation Resources
  • Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

  • To develop a theory or evaluate an existing theory
  • To summarize the historical or existing state of a research topic
  • Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

  • A research paper assigned in a course
  • A thesis or dissertation
  • A grant proposal
  • An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

  • Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

  • Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

  • Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

  • Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

  • UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
  • << Previous: Getting Started
  • Next: How to Pick a Topic >>
  • Last Updated: Sep 21, 2022 2:16 PM
  • URL: https://guides.lib.uconn.edu/literaturereview

Creative Commons

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

  • Planning the Review
  • The Research Question
  • Choosing Where to Search
  • Organizing the Review
  • Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

  • Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

Cover Art

  • Next: Planning the Review >>
  • Last Updated: Jan 17, 2024 10:05 AM
  • URL: https://libguides.northwestern.edu/literaturereviews

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

News alert: UC Berkeley has announced its next university librarian

Secondary menu

  • Log in to your Library account
  • Hours and Maps
  • Connect from Off Campus
  • UC Berkeley Home

Search form

Conducting a literature review: why do a literature review, why do a literature review.

  • How To Find "The Literature"
  • Found it -- Now What?

Besides the obvious reason for students -- because it is assigned! -- a literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed.

You identify:

  • core research in the field
  • experts in the subject area
  • methodology you may want to use (or avoid)
  • gaps in knowledge -- or where your research would fit in

It Also Helps You:

  • Publish and share your findings
  • Justify requests for grants and other funding
  • Identify best practices to inform practice
  • Set wider context for a program evaluation
  • Compile information to support community organizing

Great brief overview, from NCSU

Want To Know More?

Cover Art

  • Next: How To Find "The Literature" >>
  • Last Updated: Dec 8, 2023 10:11 AM
  • URL: https://guides.lib.berkeley.edu/litreview

Purdue University

  • Ask a Librarian

Research: Overview & Approaches

  • Getting Started with Undergraduate Research
  • Planning & Getting Started
  • Building Your Knowledge Base
  • Locating Sources
  • Reading Scholarly Articles
  • Creating a Literature Review

Finding and Completing a Literature Review

Intro to creating a literature review.

  • Productivity & Organizing Research
  • Scholarly and Professional Relationships
  • Empirical Research
  • Interpretive Research
  • Action-Based Research
  • Creative & Experimental Approaches

  • Palgrave's Study Guide to Carrying Out a Literature Review Your research is seen as a contribution to knowledge in the field and it needs to indicate, therefore, that there is an awareness of what that knowledge comprises. Read this guide to getting started.
  • Purdue OWL's Guide to Writing a Literature Review A literature review requires the writer to perform extensive research on published work in one’s field in order to explain how one’s own work fits into the larger conversation regarding a particular topic. This task requires the writer to spend time reading, managing, and conveying information; the complexity of literature reviews can make this section one of the most challenging parts of writing about one’s research. This handout will provide some strategies for revising literature reviews.

Every time you conduct research, you will need to make it clear where you got your evidence from. This work of citing our sources is absolutely essential for a couple of reasons.

  • It demonstrates to the readers of our own research that we have evidence to back up our claims.
  • A complete and correct citation directs readers to the original source for them to verify our claims and learn more.
  • It gives credit to the researchers whose intellectual work helped form our own research.
  • << Previous: Reading Scholarly Articles
  • Next: Productivity & Organizing Research >>
  • Last Edited: Nov 10, 2023 3:32 PM
  • URL: https://guides.lib.purdue.edu/research_approaches

The Writing Center • University of North Carolina at Chapel Hill

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

  • Roughly how many sources should you include?
  • What types of sources (books, journal articles, websites)?
  • Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
  • Should you evaluate your sources?
  • Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

  • Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
  • Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
  • Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

  • Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
  • By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
  • By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
  • Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
  • Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

  • Current Situation: Information necessary to understand the topic or focus of the literature review.
  • History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

  • Systematic review
  • Open access
  • Published: 19 February 2024

‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

  • Annette Boaz   ORCID: orcid.org/0000-0003-0557-1294 1 ,
  • Juan Baeza 2 ,
  • Alec Fraser   ORCID: orcid.org/0000-0003-1121-1551 2 &
  • Erik Persson 3  

Implementation Science volume  19 , Article number:  15 ( 2024 ) Cite this article

990 Accesses

55 Altmetric

Metrics details

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions ( n  = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Peer Review reports

Contribution to the literature

Considerable time and money is invested in implementing and evaluating strategies to increase the implementation of research into clinical practice.

The growing body of evidence is not providing the anticipated clear lessons to support improved implementation.

Instead what is needed is better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice.

This would involve a more central role in implementation science for a wider range of perspectives, especially from the social, economic, political and behavioural sciences and for greater use of different types of synthesis, such as realist synthesis.

Introduction

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice [ 1 , 2 ]. In recent years researchers have worked to improve the consistency in the ways in which these interventions (often called strategies) are described to support their evaluation. One notable development has been the emergence of Implementation Science as a field focusing explicitly on “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” ([ 3 ] p. 1). The work of implementation science focuses on closing, or at least narrowing, the gap between research and practice. One contribution has been to map existing interventions, identifying 73 discreet strategies to support research implementation [ 4 ] which have been grouped into 9 clusters [ 5 ]. The authors note that they have not considered the evidence of effectiveness of the individual strategies and that a next step is to understand better which strategies perform best in which combinations and for what purposes [ 4 ]. Other authors have noted that there is also scope to learn more from other related fields of study such as policy implementation [ 6 ] and to draw on methods designed to support the evaluation of complex interventions [ 7 ].

The increase in activity designed to support the implementation of research into practice and improvements in reporting provided the impetus for an update of a review of systematic reviews of the effectiveness of interventions designed to support the use of research in clinical practice [ 8 ] which was itself an update of the review conducted by Grimshaw and colleagues in 2001. The 2001 review [ 9 ] identified 41 reviews considering a range of strategies including educational interventions, audit and feedback, computerised decision support to financial incentives and combined interventions. The authors concluded that all the interventions had the potential to promote the uptake of evidence in practice, although no one intervention seemed to be more effective than the others in all settings. They concluded that combined interventions were more likely to be effective than single interventions. The 2011 review identified a further 13 systematic reviews containing 313 discrete primary studies. Consistent with the previous review, four main strategy types were identified: audit and feedback; computerised decision support; opinion leaders; and multi-faceted interventions (MFIs). Nine of the reviews reported on MFIs. The review highlighted the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. MFIs claimed an improvement in effectiveness over single interventions, although effect sizes remained small to moderate and this improvement in effectiveness relating to MFIs has been questioned in a subsequent review [ 10 ]. In updating the review, we anticipated a larger pool of reviews and an opportunity to consolidate learning from more recent systematic reviews of interventions.

This review updates and extends our previous review of systematic reviews of interventions designed to implement research evidence into clinical practice. To identify potentially relevant peer-reviewed research papers, we developed a comprehensive systematic literature search strategy based on the terms used in the Grimshaw et al. [ 9 ] and Boaz, Baeza and Fraser [ 8 ] overview articles. To ensure optimal retrieval, our search strategy was refined with support from an expert university librarian, considering the ongoing improvements in the development of search filters for systematic reviews since our first review [ 11 ]. We also wanted to include technology-related terms (e.g. apps, algorithms, machine learning, artificial intelligence) to find studies that explored interventions based on the use of technological innovations as mechanistic tools for increasing the use of evidence into practice (see Additional file 1 : Appendix A for full search strategy).

The search was performed in June 2022 in the following electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched for articles published since the 2011 review. We searched from January 2010 up to June 2022 and applied no language restrictions. Reference lists of relevant papers were also examined.

We uploaded the results using EPPI-Reviewer, a web-based tool that facilitated semi-automation of the screening process and removal of duplicate studies. We made particular use of a priority screening function to reduce screening workload and avoid ‘data deluge’ [ 12 ]. Through machine learning, one reviewer screened a smaller number of records ( n  = 1200) to train the software to predict whether a given record was more likely to be relevant or irrelevant, thus pulling the relevant studies towards the beginning of the screening process. This automation did not replace manual work but helped the reviewer to identify eligible studies more quickly. During the selection process, we included studies that looked explicitly at interventions designed to turn research evidence into practice. Studies were included if they met the following pre-determined inclusion criteria:

The study was a systematic review

Search terms were included

Focused on the implementation of research evidence into practice

The methodological quality of the included studies was assessed as part of the review

Study populations included healthcare providers and patients. The EPOC taxonomy [ 13 ] was used to categorise the strategies. The EPOC taxonomy has four domains: delivery arrangements, financial arrangements, governance arrangements and implementation strategies. The implementation strategies domain includes 20 strategies targeted at healthcare workers. Numerous EPOC strategies were assessed in the review including educational strategies, local opinion leaders, reminders, ICT-focused approaches and audit and feedback. Some strategies that did not fit easily within the EPOC categories were also included. These were social media strategies and toolkits, and multi-faceted interventions (MFIs) (see Table  2 ). Some systematic reviews included comparisons of different interventions while other reviews compared one type of intervention against a control group. Outcomes related to improvements in health care processes or patient well-being. Numerous individual study types (RCT, CCT, BA, ITS) were included within the systematic reviews.

We excluded papers that:

Focused on changing patient rather than provider behaviour

Had no demonstrable outcomes

Made unclear or no reference to research evidence

The last of these criteria was sometimes difficult to judge, and there was considerable discussion amongst the research team as to whether the link between research evidence and practice was sufficiently explicit in the interventions analysed. As we discussed in the previous review [ 8 ] in the field of healthcare, the principle of evidence-based practice is widely acknowledged and tools to change behaviour such as guidelines are often seen to be an implicit codification of evidence, despite the fact that this is not always the case.

Reviewers employed a two-stage process to select papers for inclusion. First, all titles and abstracts were screened by one reviewer to determine whether the study met the inclusion criteria. Two papers [ 14 , 15 ] were identified that fell just before the 2010 cut-off. As they were not identified in the searches for the first review [ 8 ] they were included and progressed to assessment. Each paper was rated as include, exclude or maybe. The full texts of 111 relevant papers were assessed independently by at least two authors. To reduce the risk of bias, papers were excluded following discussion between all members of the team. 32 papers met the inclusion criteria and proceeded to data extraction. The study selection procedure is documented in a PRISMA literature flow diagram (see Fig.  1 ). We were able to include French, Spanish and Portuguese papers in the selection reflecting the language skills in the study team, but none of the papers identified met the inclusion criteria. Other non- English language papers were excluded.

figure 1

PRISMA flow diagram. Source: authors

One reviewer extracted data on strategy type, number of included studies, local, target population, effectiveness and scope of impact from the included studies. Two reviewers then independently read each paper and noted key findings and broad themes of interest which were then discussed amongst the wider authorial team. Two independent reviewers appraised the quality of included studies using a Quality Assessment Checklist based on Oxman and Guyatt [ 16 ] and Francke et al. [ 17 ]. Each study was rated a quality score ranging from 1 (extensive flaws) to 7 (minimal flaws) (see Additional file 2 : Appendix B). All disagreements were resolved through discussion. Studies were not excluded in this updated overview based on methodological quality as we aimed to reflect the full extent of current research into this topic.

The extracted data were synthesised using descriptive and narrative techniques to identify themes and patterns in the data linked to intervention strategies, targeted behaviours, study settings and study outcomes.

Thirty-two studies were included in the systematic review. Table 1. provides a detailed overview of the included systematic reviews comprising reference, strategy type, quality score, number of included studies, local, target population, effectiveness and scope of impact (see Table  1. at the end of the manuscript). Overall, the quality of the studies was high. Twenty-three studies scored 7, six studies scored 6, one study scored 5, one study scored 4 and one study scored 3. The primary focus of the review was on reviews of effectiveness studies, but a small number of reviews did include data from a wider range of methods including qualitative studies which added to the analysis in the papers [ 18 , 19 , 20 , 21 ]. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. In this section, we discuss the different EPOC-defined implementation strategies in turn. Interestingly, we found only two ‘new’ approaches in this review that did not fit into the existing EPOC approaches. These are a review focused on the use of social media and a review considering toolkits. In addition to single interventions, we also discuss multi-faceted interventions. These were the most common intervention approach overall. A summary is provided in Table  2 .

Educational strategies

The overview identified three systematic reviews focusing on educational strategies. Grudniewicz et al. [ 22 ] explored the effectiveness of printed educational materials on primary care physician knowledge, behaviour and patient outcomes and concluded they were not effective in any of these aspects. Koota, Kääriäinen and Melender [ 23 ] focused on educational interventions promoting evidence-based practice among emergency room/accident and emergency nurses and found that interventions involving face-to-face contact led to significant or highly significant effects on patient benefits and emergency nurses’ knowledge, skills and behaviour. Interventions using written self-directed learning materials also led to significant improvements in nurses’ knowledge of evidence-based practice. Although the quality of the studies was high, the review primarily included small studies with low response rates, and many of them relied on self-assessed outcomes; consequently, the strength of the evidence for these outcomes is modest. Wu et al. [ 20 ] questioned if educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes. Although based on evaluation projects and qualitative data, their results also suggest that positive changes on patient outcomes can be made following the implementation of specific evidence-based approaches (or projects). The differing positive outcomes for educational strategies aimed at nurses might indicate that the target audience is important.

Local opinion leaders

Flodgren et al. [ 24 ] was the only systemic review focusing solely on opinion leaders. The review found that local opinion leaders alone, or in combination with other interventions, can be effective in promoting evidence‐based practice, but this varies both within and between studies and the effect on patient outcomes is uncertain. The review found that, overall, any intervention involving opinion leaders probably improves healthcare professionals’ compliance with evidence-based practice but varies within and across studies. However, how opinion leaders had an impact could not be determined because of insufficient details were provided, illustrating that reporting specific details in published studies is important if diffusion of effective methods of increasing evidence-based practice is to be spread across a system. The usefulness of this review is questionable because it cannot provide evidence of what is an effective opinion leader, whether teams of opinion leaders or a single opinion leader are most effective, or the most effective methods used by opinion leaders.

Pantoja et al. [ 26 ] was the only systemic review focusing solely on manually generated reminders delivered on paper included in the overview. The review explored how these affected professional practice and patient outcomes. The review concluded that manually generated reminders delivered on paper as a single intervention probably led to small to moderate increases in adherence to clinical recommendations, and they could be used as a single quality improvement intervention. However, the authors indicated that this intervention would make little or no difference to patient outcomes. The authors state that such a low-tech intervention may be useful in low- and middle-income countries where paper records are more likely to be the norm.

ICT-focused approaches

The three ICT-focused reviews [ 14 , 27 , 28 ] showed mixed results. Jamal, McKenzie and Clark [ 14 ] explored the impact of health information technology on the quality of medical and health care. They examined the impact of electronic health record, computerised provider order-entry, or decision support system. This showed a positive improvement in adherence to evidence-based guidelines but not to patient outcomes. The number of studies included in the review was low and so a conclusive recommendation could not be reached based on this review. Similarly, Brown et al. [ 28 ] found that technology-enabled knowledge translation interventions may improve knowledge of health professionals, but all eight studies raised concerns of bias. The De Angelis et al. [ 27 ] review was more promising, reporting that ICT can be a good way of disseminating clinical practice guidelines but conclude that it is unclear which type of ICT method is the most effective.

Audit and feedback

Sykes, McAnuff and Kolehmainen [ 29 ] examined whether audit and feedback were effective in dementia care and concluded that it remains unclear which ingredients of audit and feedback are successful as the reviewed papers illustrated large variations in the effectiveness of interventions using audit and feedback.

Non-EPOC listed strategies: social media, toolkits

There were two new (non-EPOC listed) intervention types identified in this review compared to the 2011 review — fewer than anticipated. We categorised a third — ‘care bundles’ [ 36 ] as a multi-faceted intervention due to its description in practice and a fourth — ‘Technology Enhanced Knowledge Transfer’ [ 28 ] was classified as an ICT-focused approach. The first new strategy was identified in Bhatt et al.’s [ 30 ] systematic review of the use of social media for the dissemination of clinical practice guidelines. They reported that the use of social media resulted in a significant improvement in knowledge and compliance with evidence-based guidelines compared with more traditional methods. They noted that a wide selection of different healthcare professionals and patients engaged with this type of social media and its global reach may be significant for low- and middle-income countries. This review was also noteworthy for developing a simple stepwise method for using social media for the dissemination of clinical practice guidelines. However, it is debatable whether social media can be classified as an intervention or just a different way of delivering an intervention. For example, the review discussed involving opinion leaders and patient advocates through social media. However, this was a small review that included only five studies, so further research in this new area is needed. Yamada et al. [ 31 ] draw on 39 studies to explore the application of toolkits, 18 of which had toolkits embedded within larger KT interventions, and 21 of which evaluated toolkits as standalone interventions. The individual component strategies of the toolkits were highly variable though the authors suggest that they align most closely with educational strategies. The authors conclude that toolkits as either standalone strategies or as part of MFIs hold some promise for facilitating evidence use in practice but caution that the quality of many of the primary studies included is considered weak limiting these findings.

Multi-faceted interventions

The majority of the systematic reviews ( n  = 20) reported on more than one intervention type. Some of these systematic reviews focus exclusively on multi-faceted interventions, whilst others compare different single or combined interventions aimed at achieving similar outcomes in particular settings. While these two approaches are often described in a similar way, they are actually quite distinct from each other as the former report how multiple strategies may be strategically combined in pursuance of an agreed goal, whilst the latter report how different strategies may be incidentally used in sometimes contrasting settings in the pursuance of similar goals. Ariyo et al. [ 35 ] helpfully summarise five key elements often found in effective MFI strategies in LMICs — but which may also be transferrable to HICs. First, effective MFIs encourage a multi-disciplinary approach acknowledging the roles played by different professional groups to collectively incorporate evidence-informed practice. Second, they utilise leadership drawing on a wide set of clinical and non-clinical actors including managers and even government officials. Third, multiple types of educational practices are utilised — including input from patients as stakeholders in some cases. Fourth, protocols, checklists and bundles are used — most effectively when local ownership is encouraged. Finally, most MFIs included an emphasis on monitoring and evaluation [ 35 ]. In contrast, other studies offer little information about the nature of the different MFI components of included studies which makes it difficult to extrapolate much learning from them in relation to why or how MFIs might affect practice (e.g. [ 28 , 38 ]). Ultimately, context matters, which some review authors argue makes it difficult to say with real certainty whether single or MFI strategies are superior (e.g. [ 21 , 27 ]). Taking all the systematic reviews together we may conclude that MFIs appear to be more likely to generate positive results than single interventions (e.g. [ 34 , 45 ]) though other reviews should make us cautious (e.g. [ 32 , 43 ]).

While multi-faceted interventions still seem to be more effective than single-strategy interventions, there were important distinctions between how the results of reviews of MFIs are interpreted in this review as compared to the previous reviews [ 8 , 9 ], reflecting greater nuance and debate in the literature. This was particularly noticeable where the effectiveness of MFIs was compared to single strategies, reflecting developments widely discussed in previous studies [ 10 ]. We found that most systematic reviews are bounded by their clinical, professional, spatial, system, or setting criteria and often seek to draw out implications for the implementation of evidence in their areas of specific interest (such as nursing or acute care). Frequently this means combining all relevant studies to explore the respective foci of each systematic review. Therefore, most reviews we categorised as MFIs actually include highly variable numbers and combinations of intervention strategies and highly heterogeneous original study designs. This makes statistical analyses of the type used by Squires et al. [ 10 ] on the three reviews in their paper not possible. Further, it also makes extrapolating findings and commenting on broad themes complex and difficult. This may suggest that future research should shift its focus from merely examining ‘what works’ to ‘what works where and what works for whom’ — perhaps pointing to the value of realist approaches to these complex review topics [ 48 , 49 ] and other more theory-informed approaches [ 50 ].

Some reviews have a relatively small number of studies (i.e. fewer than 10) and the authors are often understandably reluctant to engage with wider debates about the implications of their findings. Other larger studies do engage in deeper discussions about internal comparisons of findings across included studies and also contextualise these in wider debates. Some of the most informative studies (e.g. [ 35 , 40 ]) move beyond EPOC categories and contextualise MFIs within wider systems thinking and implementation theory. This distinction between MFIs and single interventions can actually be very useful as it offers lessons about the contexts in which individual interventions might have bounded effectiveness (i.e. educational interventions for individual change). Taken as a whole, this may also then help in terms of how and when to conjoin single interventions into effective MFIs.

In the two previous reviews, a consistent finding was that MFIs were more effective than single interventions [ 8 , 9 ]. However, like Squires et al. [ 10 ] this overview is more equivocal on this important issue. There are four points which may help account for the differences in findings in this regard. Firstly, the diversity of the systematic reviews in terms of clinical topic or setting is an important factor. Secondly, there is heterogeneity of the studies within the included systematic reviews themselves. Thirdly, there is a lack of consistency with regards to the definition and strategies included within of MFIs. Finally, there are epistemological differences across the papers and the reviews. This means that the results that are presented depend on the methods used to measure, report, and synthesise them. For instance, some reviews highlight that education strategies can be useful to improve provider understanding — but without wider organisational or system-level change, they may struggle to deliver sustained transformation [ 19 , 44 ].

It is also worth highlighting the importance of the theory of change underlying the different interventions. Where authors of the systematic reviews draw on theory, there is space to discuss/explain findings. We note a distinction between theoretical and atheoretical systematic review discussion sections. Atheoretical reviews tend to present acontextual findings (for instance, one study found very positive results for one intervention, and this gets highlighted in the abstract) whilst theoretically informed reviews attempt to contextualise and explain patterns within the included studies. Theory-informed systematic reviews seem more likely to offer more profound and useful insights (see [ 19 , 35 , 40 , 43 , 45 ]). We find that the most insightful systematic reviews of MFIs engage in theoretical generalisation — they attempt to go beyond the data of individual studies and discuss the wider implications of the findings of the studies within their reviews drawing on implementation theory. At the same time, they highlight the active role of context and the wider relational and system-wide issues linked to implementation. It is these types of investigations that can help providers further develop evidence-based practice.

This overview has identified a small, but insightful set of papers that interrogate and help theorise why, how, for whom, and in which circumstances it might be the case that MFIs are superior (see [ 19 , 35 , 40 ] once more). At the level of this overview — and in most of the systematic reviews included — it appears to be the case that MFIs struggle with the question of attribution. In addition, there are other important elements that are often unmeasured, or unreported (e.g. costs of the intervention — see [ 40 ]). Finally, the stronger systematic reviews [ 19 , 35 , 40 , 43 , 45 ] engage with systems issues, human agency and context [ 18 ] in a way that was not evident in the systematic reviews identified in the previous reviews [ 8 , 9 ]. The earlier reviews lacked any theory of change that might explain why MFIs might be more effective than single ones — whereas now some systematic reviews do this, which enables them to conclude that sometimes single interventions can still be more effective.

As Nilsen et al. ([ 6 ] p. 7) note ‘Study findings concerning the effectiveness of various approaches are continuously synthesized and assembled in systematic reviews’. We may have gone as far as we can in understanding the implementation of evidence through systematic reviews of single and multi-faceted interventions and the next step would be to conduct more research exploring the complex and situated nature of evidence used in clinical practice and by particular professional groups. This would further build on the nuanced discussion and conclusion sections in a subset of the papers we reviewed. This might also support the field to move away from isolating individual implementation strategies [ 6 ] to explore the complex processes involving a range of actors with differing capacities [ 51 ] working in diverse organisational cultures. Taxonomies of implementation strategies do not fully account for the complex process of implementation, which involves a range of different actors with different capacities and skills across multiple system levels. There is plenty of work to build on, particularly in the social sciences, which currently sits at the margins of debates about evidence implementation (see for example, Normalisation Process Theory [ 52 ]).

There are several changes that we have identified in this overview of systematic reviews in comparison to the review we published in 2011 [ 8 ]. A consistent and welcome finding is that the overall quality of the systematic reviews themselves appears to have improved between the two reviews, although this is not reflected upon in the papers. This is exhibited through better, clearer reporting mechanisms in relation to the mechanics of the reviews, alongside a greater attention to, and deeper description of, how potential biases in included papers are discussed. Additionally, there is an increased, but still limited, inclusion of original studies conducted in low- and middle-income countries as opposed to just high-income countries. Importantly, we found that many of these systematic reviews are attuned to, and comment upon the contextual distinctions of pursuing evidence-informed interventions in health care settings in different economic settings. Furthermore, systematic reviews included in this updated article cover a wider set of clinical specialities (both within and beyond hospital settings) and have a focus on a wider set of healthcare professions — discussing both similarities, differences and inter-professional challenges faced therein, compared to the earlier reviews. These wider ranges of studies highlight that a particular intervention or group of interventions may work well for one professional group but be ineffective for another. This diversity of study settings allows us to consider the important role context (in its many forms) plays on implementing evidence into practice. Examining the complex and varied context of health care will help us address what Nilsen et al. ([ 6 ] p. 1) described as, ‘society’s health problems [that] require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies’. This will help us shift implementation science to move, ‘beyond a success or failure perspective towards improved analysis of variables that could explain the impact of the implementation process’ ([ 6 ] p. 2).

This review brings together 32 papers considering individual and multi-faceted interventions designed to support the use of evidence in clinical practice. The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been conducted. As a whole, this substantial body of knowledge struggles to tell us more about the use of individual and MFIs than: ‘it depends’. To really move forwards in addressing the gap between research evidence and practice, we may need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of perspectives, especially from the social, economic, political and behavioural sciences in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed. Harvey et al. [ 53 ] suggest that when context is likely to be critical to implementation success there are a range of primary research approaches (participatory research, realist evaluation, developmental evaluation, ethnography, quality/ rapid cycle improvement) that are likely to be appropriate and insightful. While these approaches often form part of implementation studies in the form of process evaluations, they are usually relatively small scale in relation to implementation research as a whole. As a result, the findings often do not make it into the subsequent systematic reviews. This review provides further evidence that we need to bring qualitative approaches in from the periphery to play a central role in many implementation studies and subsequent evidence syntheses. It would be helpful for systematic reviews, at the very least, to include more detail about the interventions and their implementation in terms of how and why they worked.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Before and after study

Controlled clinical trial

Effective Practice and Organisation of Care

High-income countries

Information and Communications Technology

Interrupted time series

Knowledge translation

Low- and middle-income countries

Randomised controlled trial

Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362:1225–30. https://doi.org/10.1016/S0140-6736(03)14546-1 .

Article   PubMed   Google Scholar  

Green LA, Seifert CM. Translation of research into practice: why we can’t “just do it.” J Am Board Fam Pract. 2005;18:541–5. https://doi.org/10.3122/jabfm.18.6.541 .

Eccles MP, Mittman BS. Welcome to Implementation Science. Implement Sci. 2006;1:1–3. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:2–14. https://doi.org/10.1186/s13012-015-0209-1 .

Article   Google Scholar  

Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10:1–8. https://doi.org/10.1186/s13012-015-0295-0 .

Nilsen P, Ståhl C, Roback K, et al. Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Sci. 2013;8:2–12. https://doi.org/10.1186/1748-5908-8-63 .

Rycroft-Malone J, Seers K, Eldh AC, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implementation Sci. 2018;13:1–15. https://doi.org/10.1186/s13012-018-0811-0 .

Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS). Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011;4:212. https://doi.org/10.1186/1756-0500-4-212 .

Article   PubMed   PubMed Central   Google Scholar  

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior – an overview of systematic reviews of interventions. Med Care. 2001;39 8Suppl 2:II2–45.

Google Scholar  

Squires JE, Sullivan K, Eccles MP, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:1–22. https://doi.org/10.1186/s13012-014-0152-6 .

Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc. 2021;109:561–74. https://doi.org/10.5195/jmla.2021.1223 .

Thomas JM. Diffusion of innovation in systematic review methodology: why is study selection not yet assisted by automation? OA Evid Based Med. 2013;1:1–6.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. epoc.cochrane.org/epoc-taxonomy . Accessed 9 Oct 2023.

Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag. 2009;38:26–37. https://doi.org/10.1177/183335830903800305 .

Menon A, Korner-Bitensky N, Kastner M, et al. Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009;41:1024–32. https://doi.org/10.2340/16501977-0451 .

Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–8. https://doi.org/10.1016/0895-4356(91)90160-b .

Article   CAS   PubMed   Google Scholar  

Francke AL, Smit MC, de Veer AJ, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:1–11. https://doi.org/10.1186/1472-6947-8-38 .

Jones CA, Roop SC, Pohar SL, et al. Translating knowledge in rehabilitation: systematic review. Phys Ther. 2015;95:663–77. https://doi.org/10.2522/ptj.20130512 .

Scott D, Albrecht L, O’Leary K, Ball GDC, et al. Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012;7:1–17. https://doi.org/10.1186/1748-5908-7-70 .

Wu Y, Brettle A, Zhou C, Ou J, et al. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ Today. 2018;70:109–14. https://doi.org/10.1016/j.nedt.2018.08.026 .

Yost J, Ganann R, Thompson D, Aloweni F, et al. The effectiveness of knowledge translation interventions for promoting evidence-informed decision-making among nurses in tertiary care: a systematic review and meta-analysis. Implement Sci. 2015;10:1–15. https://doi.org/10.1186/s13012-015-0286-1 .

Grudniewicz A, Kealy R, Rodseth RN, Hamid J, et al. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:2–12. https://doi.org/10.1186/s13012-015-0347-5 .

Koota E, Kääriäinen M, Melender HL. Educational interventions promoting evidence-based practice among emergency nurses: a systematic review. Int Emerg Nurs. 2018;41:51–8. https://doi.org/10.1016/j.ienj.2018.06.004 .

Flodgren G, O’Brien MA, Parmelli E, et al. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD000125.pub5 .

Arditi C, Rège-Walther M, Durieux P, et al. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017. https://doi.org/10.1002/14651858.CD001175.pub4 .

Pantoja T, Grimshaw JM, Colomer N, et al. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD001174.pub4 .

De Angelis G, Davies B, King J, McEwan J, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2:e16. https://doi.org/10.2196/mededu.6288 .

Brown A, Barnes C, Byaruhanga J, McLaughlin M, et al. Effectiveness of technology-enabled knowledge translation strategies in improving the use of research in public health: systematic review. J Med Internet Res. 2020;22:e17274. https://doi.org/10.2196/17274 .

Sykes MJ, McAnuff J, Kolehmainen N. When is audit and feedback effective in dementia care? A systematic review. Int J Nurs Stud. 2018;79:27–35. https://doi.org/10.1016/j.ijnurstu.2017.10.013 .

Bhatt NR, Czarniecki SW, Borgmann H, et al. A systematic review of the use of social media for dissemination of clinical practice guidelines. Eur Urol Focus. 2021;7:1195–204. https://doi.org/10.1016/j.euf.2020.10.008 .

Yamada J, Shorkey A, Barwick M, Widger K, et al. The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open. 2015;5:e006808. https://doi.org/10.1136/bmjopen-2014-006808 .

Afari-Asiedu S, Abdulai MA, Tostmann A, et al. Interventions to improve dispensing of antibiotics at the community level in low and middle income countries: a systematic review. J Glob Antimicrob Resist. 2022;29:259–74. https://doi.org/10.1016/j.jgar.2022.03.009 .

Boonacker CW, Hoes AW, Dikhoff MJ, Schilder AG, et al. Interventions in health care professionals to improve treatment in children with upper respiratory tract infections. Int J Pediatr Otorhinolaryngol. 2010;74:1113–21. https://doi.org/10.1016/j.ijporl.2010.07.008 .

Al Zoubi FM, Menon A, Mayo NE, et al. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18:2–11. https://doi.org/10.1186/s12913-018-3253-0 .

Ariyo P, Zayed B, Riese V, Anton B, et al. Implementation strategies to reduce surgical site infections: a systematic review. Infect Control Hosp Epidemiol. 2019;3:287–300. https://doi.org/10.1017/ice.2018.355 .

Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci. 2015;10:1–11. https://doi.org/10.1186/s13012-015-0306-1 .

Cahill LS, Carey LM, Lannin NA, et al. Implementation interventions to promote the uptake of evidence-based practices in stroke rehabilitation. Cochrane Database Syst Rev. 2020. https://doi.org/10.1002/14651858.CD012575.pub2 .

Pedersen ER, Rubenstein L, Kandrack R, Danz M, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implement Sci. 2018;13:1–30. https://doi.org/10.1186/s13012-018-0788-8 .

Jenkins HJ, Hancock MJ, French SD, Maher CG, et al. Effectiveness of interventions designed to reduce the use of imaging for low-back pain: a systematic review. CMAJ. 2015;187:401–8. https://doi.org/10.1503/cmaj.141183 .

Bennett S, Laver K, MacAndrew M, Beattie E, et al. Implementation of evidence-based, non-pharmacological interventions addressing behavior and psychological symptoms of dementia: a systematic review focused on implementation strategies. Int Psychogeriatr. 2021;33:947–75. https://doi.org/10.1017/S1041610220001702 .

Noonan VK, Wolfe DL, Thorogood NP, et al. Knowledge translation and implementation in spinal cord injury: a systematic review. Spinal Cord. 2014;52:578–87. https://doi.org/10.1038/sc.2014.62 .

Albrecht L, Archibald M, Snelgrove-Clarke E, et al. Systematic review of knowledge translation strategies to promote research uptake in child health settings. J Pediatr Nurs. 2016;31:235–54. https://doi.org/10.1016/j.pedn.2015.12.002 .

Campbell A, Louie-Poon S, Slater L, et al. Knowledge translation strategies used by healthcare professionals in child health settings: an updated systematic review. J Pediatr Nurs. 2019;47:114–20. https://doi.org/10.1016/j.pedn.2019.04.026 .

Bird ML, Miller T, Connell LA, et al. Moving stroke rehabilitation evidence into practice: a systematic review of randomized controlled trials. Clin Rehabil. 2019;33:1586–95. https://doi.org/10.1177/0269215519847253 .

Goorts K, Dizon J, Milanese S. The effectiveness of implementation strategies for promoting evidence informed interventions in allied healthcare: a systematic review. BMC Health Serv Res. 2021;21:1–11. https://doi.org/10.1186/s12913-021-06190-0 .

Zadro JR, O’Keeffe M, Allison JL, Lembke KA, et al. Effectiveness of implementation strategies to improve adherence of physical therapist treatment choices to clinical practice guidelines for musculoskeletal conditions: systematic review. Phys Ther. 2020;100:1516–41. https://doi.org/10.1093/ptj/pzaa101 .

Van der Veer SN, Jager KJ, Nache AM, et al. Translating knowledge on best practice into improving quality of RRT care: a systematic review of implementation strategies. Kidney Int. 2011;80:1021–34. https://doi.org/10.1038/ki.2011.222 .

Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10Suppl 1:21–34. https://doi.org/10.1258/1355819054308530 .

Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: illustrating the method for implementation research. Implementation Sci. 2012;7:1–10. https://doi.org/10.1186/1748-5908-7-33 .

Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5:e008592. https://doi.org/10.1136/bmjopen-2015-008592 .

Metz A, Jensen T, Farley A, Boaz A, et al. Is implementation research out of step with implementation practice? Pathways to effective implementation support over the last decade. Implement Res Pract. 2022;3:1–11. https://doi.org/10.1177/26334895221105585 .

May CR, Finch TL, Cornford J, Exley C, et al. Integrating telecare for chronic disease management in the community: What needs to be done? BMC Health Serv Res. 2011;11:1–11. https://doi.org/10.1186/1472-6963-11-131 .

Harvey G, Rycroft-Malone J, Seers K, Wilson P, et al. Connecting the science and practice of implementation – applying the lens of context to inform study design in implementation research. Front Health Serv. 2023;3:1–15. https://doi.org/10.3389/frhs.2023.1162762 .

Download references

Acknowledgements

The authors would like to thank Professor Kathryn Oliver for her support in the planning the review, Professor Steve Hanney for reading and commenting on the final manuscript and the staff at LSHTM library for their support in planning and conducting the literature search.

This study was supported by LSHTM’s Research England QR strategic priorities funding allocation and the National Institute for Health and Care Research (NIHR) Applied Research Collaboration South London (NIHR ARC South London) at King’s College Hospital NHS Foundation Trust. Grant number NIHR200152. The views expressed are those of the author(s) and not necessarily those of the NIHR, the Department of Health and Social Care or Research England.

Author information

Authors and affiliations.

Health and Social Care Workforce Research Unit, The Policy Institute, King’s College London, Virginia Woolf Building, 22 Kingsway, London, WC2B 6LE, UK

Annette Boaz

King’s Business School, King’s College London, 30 Aldwych, London, WC2B 4BG, UK

Juan Baeza & Alec Fraser

Federal University of Santa Catarina (UFSC), Campus Universitário Reitor João Davi Ferreira Lima, Florianópolis, SC, 88.040-900, Brazil

Erik Persson

You can also search for this author in PubMed   Google Scholar

Contributions

AB led the conceptual development and structure of the manuscript. EP conducted the searches and data extraction. All authors contributed to screening and quality appraisal. EP and AF wrote the first draft of the methods section. AB, JB and AF performed result synthesis and contributed to the analyses. AB wrote the first draft of the manuscript and incorporated feedback and revisions from all other authors. All authors revised and approved the final manuscript.

Corresponding author

Correspondence to Annette Boaz .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a., additional file 2: appendix b., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Boaz, A., Baeza, J., Fraser, A. et al. ‘It depends’: what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice. Implementation Sci 19 , 15 (2024). https://doi.org/10.1186/s13012-024-01337-z

Download citation

Received : 01 November 2023

Accepted : 05 January 2024

Published : 19 February 2024

DOI : https://doi.org/10.1186/s13012-024-01337-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Implementation
  • Interventions
  • Clinical practice
  • Research evidence
  • Multi-faceted

Implementation Science

ISSN: 1748-5908

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

use of research literature

The independent source for health policy research, polling, and news.

Building on the Evidence Base: Studies on the Effects of Medicaid Expansion, February 2020 to March 2021

Madeline Guth and Meghana Ammula Published: May 06, 2021

  • Introduction
  • Bibliography

Themes in Recent Research

Several key themes emerge from recent literature investigating the effects of Medicaid expansion (Figure 1). In contrast to earlier research , which largely focused on the impacts of Medicaid expansion for the general population or for low-income populations, recent research has increasingly focused on outcomes for specific populations, such as people with cancer or behavioral health needs. Recent research has also focused on specific outcomes such as mortality and social determinants of health. Although overall findings across these themes generally show positive effects of Medicaid expansion, a smaller number of studies find no impact of expansion on specific outcomes for specific populations; however, very few studies suggest any negative effects.

Many studies published between February 2020 and March 2021 and cited throughout this report have findings across multiple of these themes and are thus cited in multiple sections. Additionally, many studies on expansion published prior to February 2020 also have findings related to these themes but are not cited in this report; however, these can be found cited in an earlier literature review and are also included in the Bibliography and Appendices to this report.

use of research literature

Figure 1: Recent studies find positive effects of the ACA Medicaid expansion across a range of categories.​

A growing body of research finds that Medicaid expansion has improved overall mortality rates as well as mortality rates associated with some specific health conditions. These findings are consistent with earlier research identifying that expansion contributed to declines in overall and some specific mortality rates, but had no effect on mortality rates associated with other specific conditions.

  • Overall mortality. A 2020 national study found that expansion was associated with a significant 3.6% decrease in all-cause mortality, the majority of which was accounted for by a significant 1.93% decrease in health care amenable mortality. Another study found that expansion was associated with reductions in health care amenable mortality and in mortality not due to drug overdose. 1 , 2
  • Studies find that expansion was associated with significant declines in mortality related to certain specific conditions, in some instances limited to certain subgroups. These findings include decreased mortality associated with different types of cancer, cardiovascular disease, and liver disease. Studies also find decreased maternal mortality, and one study found a decrease in infant mortality among Hispanics only. 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10
  • However, other studies suggest no effect of expansion on mortality among safety-net hospital patients, individuals with glottic cancer, individuals with glioblastoma, patients undergoing hemodialysis, and overall infant mortality. One study found no significant difference between COVID-19 mortality rates in expansion versus non-expansion states, despite lower incidence rates in expansion states. One study concluded that available data was insufficient to adequately identify the impact of expansion on opioid mortality. 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19

Cancer, Chronic Disease, and Disabilities

Recent research finds largely positive impacts of expansion on coverage and access to care among populations with cancer, chronic disease, and/or disabilities. However, findings on utilization of care and health outcomes are more mixed, with some studies suggesting improvements and others finding no effect of expansion. These studies build on prior research indicating generally positive effects of expansion for populations with cancer and other health conditions. Recent research also provides additional evidence on expansion’s impacts across a range of chronic conditions considered by the CDC to put people at higher risk of severe illness and death from COVID-19 (such as diabetes, obesity, and lung and heart conditions).

  • Coverage of people with cancer . Studies overwhelmingly find that Medicaid expansion has increased insurance coverage rates among cancer patients and survivors. Research also finds changes in payer mix of care for patients with cancer, with declines in the proportion of uninsured patients and increases in the proportion of Medicaid-insured patients. 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43
  • Cancer diagnosis, treatment, and outcomes. Most studies find an association between expansion and increases in early-stage diagnosis rates among cancer patients, suggesting that expansion has facilitated earlier utilization of care for these patients. Findings on utilization of cancer treatment services and on access to timely treatment are mixed, though more studies find improvements as compared to studies that find no effect of expansion. Of studies that consider cancer mortality, three suggest improvements for patients with certain types of cancer, while three suggest no effect for patients with other types of cancer. 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75
  • Cancer screening and prevention. Several studies find that expansion increased receipt of cancer screenings such as mammograms, though a similar number of studies find no effect of expansion on screening rates for certain cancers. Two studies identified an association between expansion and increased rates of human papillomavirus (HPV) vaccines (overall and among teenagers specifically), while a third found no effect of expansion on HPV vaccination rates among female community health center patients. 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84
  • Diabetes . Studies find that expansion increased insurance coverage rates among adults and teenagers with diabetes. Although research indicates that expansion increased affordability of health care for populations with diabetes, findings on utilization of preventive care and treatment are more mixed (between studies finding improvements and studies finding no effect). Two studies identified improvements in diabetes biomarkers among community health center patients following expansion. Two studies that considered women of reproductive age found that expansion did not affect the prevalence of diabetes prior to or during pregnancy. 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96
  • Other chronic disease . In addition to cancer and diabetes, research also considers a range of other chronic conditions including cardiovascular and pulmonary diseases, obesity, and liver disease. Studies find that among those with chronic disease, expansion contributed to increased insurance coverage and improvements in payer mix, improved access to care, and better health outcomes including disease management and mortality. Findings on effects on treatment utilization and quality of care were mixed (between studies finding improvements and studies finding no effect). Finally, studies generally suggest that expansion increased screening for chronic conditions but did not reduce the prevalence of these conditions except for smoking. 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123
  • People with disabilities . A small number of recent studies consider the impacts of expansion for people with disabilities. One study found increased coverage options for people with disabilities in expansion states, while other studies suggested no effect of expansion on utilization of care or employment among this population. One study found that expansion improved mental health outcomes for caregivers of people with disabilities. 124 , 125 , 126 , 127 , 128

Sexual and Reproductive Health

Recent research finds that expansion has contributed to improvements in a number of outcomes related to sexual and reproductive health. This body of research includes findings related to women’s health and HIV/AIDS outcomes, both areas of health care that have faced increased challenges during the coronavirus pandemic. Building on prior research finding positive impacts among people of reproductive age, recent research indicates that expansion has improved measures including coverage rates before, during, and after pregnancy; maternal mortality and infant health outcomes; utilization of the most effective contraceptive methods; and screening for HIV/AIDS.

  • Maternal and infant health outcomes. Studies find that expansion significantly increased access to and utilization of health care for pregnant women and mothers. Two studies found significant declines in maternal mortality, in contrast to one study which found no impact of expansion on certain health outcomes during pregnancy. Studies generally suggest an association between expansion and improvements in birth outcomes such as low birthweight, but find no impact on infant mortality (except for one study which found a reduction in Hispanic infant mortality only). 129 , 130 , 131 , 132 , 133 , 134 , 135 , 136 , 137 , 138 , 139 , 140
  • Postpartum insurance coverage. Although the American Rescue Plan Act of 2021 created a new option to expand postpartum coverage to 12 months via a State Plan Amendment, current federal statute requires that pregnancy-related Medicaid coverage continue through just 60 days. Research indicates that ACA Medicaid expansion has decreased coverage loss after this 60-day period ends: all recent studies that consider rates of insurance coverage after pregnancy find that expansion significantly increased postpartum coverage. Studies also suggest an association between expansion and increased coverage prior to and during pregnancy. 141 , 142 , 143 , 144 , 145 , 146 , 147 , 148 , 149
  • Access to contraception. Most studies find that expansion increased utilization of the most effective contraception methods (long-acting reversible contraception, which includes IUDs and implants ); however, studies generally find no effect on overall contraception use. One study found an association between expansion and improved payer mix for contraceptive visits at safety net clinics, with a decline in the proportion of uninsured patients and an increase in the proportion of publicly-insured patients. 150 , 151 , 152 , 153 , 154 , 155
  • HIV/AIDS screening and outcomes. Studies suggest that expansion increased overall rates of HIV screening, including one study that found that increases in HIV test and diagnosis rates occurred despite no change in actual HIV incidence. Research also indicates higher insurance coverage rates among people with or at risk of HIV, increased utilization of Pre-Exposure Prophylaxis (PrEP) to treat HIV, and improved quality of care for patients with HIV. 156 , 157 , 158 , 159 , 160 , 161 , 162 , 163 , 164 , 165 , 166 , 167

Behavioral Health

A growing body of research finds that expansion is associated with improvements in access to care and outcomes related to substance use disorder (SUD) as well as other mental health care. These findings are consistent with prior research indicating positive effects of expansion on behavioral health care access and outcomes. Recent research on SUD largely focuses on opioid use disorder (OUD) specifically, which is more prevalent among Medicaid enrollees as compared to the general population. Given the impacts of the coronavirus pandemic on mental health and substance use , Medicaid expansion coverage is likely to continue to serve as a significant source of coverage for behavioral health care.

  • Access to care and outcomes for SUD. Studies find that Medicaid expansion was associated with increased insurance coverage among adults with SUD and improved payer mix of SUD-related visits (declines in uninsured patients and/or increases in Medicaid-covered patients). Studies also find that expansion increased the receipt of medication assisted treatment (MAT) prescriptions for the treatment of OUD, and that following expansion opioid treatment facilities were more likely to offer MAT and comprehensive mental health services. In contrast, a small number of studies found no effect of expansion on utilization of certain health care services for SUD. One study found no effect of expansion on drug-overdose deaths, while a second concluded that available data was insufficient to adequately identify the impact of expansion on drug-related mortality. 168 , 169 , 170 , 171 , 172 , 173 , 174 , 175 , 176 , 177 , 178 , 179 , 180 , 181
  • Mental health care access and outcomes . Studies find that expansion increased access to care for adults with mental health conditions such as depression, including by increasing the likelihood that mental health care providers accepted Medicaid. Findings on utilization of mental health care are more mixed, with some studies suggesting increased utilization of services such as mental health care via telehealth , and others finding no effect of expansion on other mental health services. Findings on mental health outcomes are also mixed: one study found that expansion was associated with improvements in self-reported mental health among low-income adults, while two other studies found no impact on similar measures among near-elderly adults and among women of reproductive age. 182 , 183 , 184 , 185 , 186 , 187 , 188 , 189 , 190 , 191 , 192 , 193

Economic Impacts on States and Providers

Building on prior research, recent studies identify positive financial impacts of Medicaid expansion for states, hospitals, and other providers. These studies join a body of prior research finding overwhelmingly positive effects of expansion on economic outcomes (see Appendix A, Figure 5). These economic findings are particularly relevant given fiscal stress experienced by both states and Medicaid providers during the coronavirus pandemic.

  • State budgets and economies. All recent studies that consider the financial impacts of expansion for states find positive effects. Studies find that expansion states experienced increased federal Medicaid spending. One study found that through 2018, Medicaid expansion led to increased federal spending in expansion states but very small (<1%), insignificant increases in spending from state sources (including in 2017 and 2018 when states began paying 5% and 6% of expansion costs respectively, a rate that was subsequently phased to 10% in 2020 and beyond). In addition, Medicaid expansion did not crowd out other areas of state spending and states that did not expand passed up $43 billion in federal funds in 2018. Research also finds that expansion resulted in increased revenue as well as net state savings by offsetting state costs in other areas, such as state spending on substance use disorder (SUD) treatment and on the traditional Medicaid program. One study found that the mortality reductions associated with expansion resulted in between $20.97 and $101.8 billion in annual welfare gains, implying that mortality-related savings alone may offset the entire net cost of expansion. 194 , 195 , 196 , 197 , 198
  • Payer mix. Studies overwhelmingly find that Medicaid expansion has resulted in payer mix improvements (declines in uninsured patients and/or increases in Medicaid-covered patients). Findings include payer mix improvements for hospitalizations, emergency department visits, and visits to community health centers and other safety-net clinics. Studies identify payer mix improvements among patients hospitalized for a range of specific conditions including traumatic injuries, surgeries, and treatment for substance use disorder. In line with payer mix improvements, studies also find decreased uncompensated care costs (UCC) overall and for specific types of hospitals, including those in rural areas. 199 , 200 , 201 , 202 , 203 , 204 , 205 , 206 , 207 , 208 , 209 , 210 , 211 , 212 , 213 , 214 , 215 , 216 , 217 , 218 , 219 , 220 , 221 , 222 , 223 , 224 , 225 , 226 , 227 , 228 , 229 , 230
  • Financial performance of hospitals and other providers. Research finds that expansion contributed to increased hospital revenue overall and from specific services. Although studies find that expansion has improved provider operating margins and profitability, these findings vary by hospital type. For example, one study found that despite declines in UCC and increases in Medicaid revenue across all hospital types, only hospitals in non-metropolitan areas and small hospitals experienced improved profit margins; another study similarly found gains in overall revenue only for rural and small hospitals. A few studies suggest that improvements in payer mix and UCC at hospitals may have been partially offset by increases in unreimbursed Medicaid care and declines in commercial revenue. One recent study found that expansion reduced the number of annual hospital closures. 231 , 232 , 233 , 234 , 235 , 236 , 237 , 238 , 239 , 240 , 241

Disparities

A growing body of research considers the impact of Medicaid expansion on disparities in different outcomes by race/ethnicity, socioeconomic status, and other categories. These studies build on an earlier literature review finding that expansion has helped to narrow racial disparities in coverage and certain health outcomes, with more limited evidence suggesting reduced racial disparities in access to and use of care. Some studies on racial/ethnic disparities cited here are also included in this earlier review, which included studies published through July 2020.

  • Disparities by race/ethnicity. Findings on expansion’s impact on racial disparities in health coverage, access, and outcomes are mixed and generally mirror findings from a previous literature review , with evidence of decreased racial disparities for some populations in measures including coverage rates, affordability of care, utilization of surgery and other services, and health outcomes including maternal and infant mortality. However, similar numbers of studies identify no effect of expansion on racial disparities in these and other measures. A very small number of studies find evidence of increased racial disparities (in coverage rates for specific populations and in breast cancer mortality). Across outcomes, most research focuses on disparities for Black and Hispanic individuals, with limited findings on impacts for other groups of color. 242 , 243 , 244 , 245 , 246 , 247 , 248 , 249 , 250 , 251 , 252 , 253 , 254 , 255 , 256 , 257 , 258 , 259 , 260 , 261 , 262 , 263 , 264 , 265 , 266 , 267 , 268 , 269 , 270 , 271 , 272 , 273 , 274 , 275 , 276 , 277 , 278 , 279 , 280
  • Disparities by socioeconomic status (income and/or education). In contrast to research on racial disparities, recent studies that consider socioeconomic disparities all find improvements. Studies find that expansion has reduced disparities in coverage by income and/or education status, including for populations with certain cancer diagnoses. A smaller number of studies also find decreased socioeconomic disparities in utilization of care, certain health outcomes such as maternal mortality, and individual financial stability. 281 , 282 , 283 , 284 , 285 , 286 , 287 , 288 , 289 , 290 , 291
  • Disparities by other categories. A few recent studies identify an association between expansion and reduced coverage disparities by age, sex, and marital status, but no effect on coverage disparities by work status and obesity. One study found that expansion reduced age disparities in individual financial stability. Another study found no effect of expansion on disparities by sex in receipt of HIV tests. 292 , 293 , 294 , 295 , 296 , 297 , 298

Social Determinants of Health

Recent research indicates largely positive impacts of expansion associated with different social determinants of health. These recent studies are consistent with prior research on expansion’s effect on social determinants of health and also contribute new evidence on effects for certain measures. Social determinants of health are the conditions in which people are born, grow, live, work, and age. Improvements in these measures associated with expansion could help to mitigate increased hardship due to the coronavirus pandemic.

  • Access to care in rural areas. Studies find that expansion was associated with greater improvements in access to care in rural areas, including increased HIV diagnosis rates and access to mental health care. In contrast, one study found that utilization of tobacco cessation treatment remained limited in rural Appalachia even after Medicaid expansion in Kentucky. Research also suggests that rural hospitals experienced particularly substantial improvements in financial performance following expansion. 299 , 300 , 301 , 302 , 303 , 304 , 305 , 306 , 307
  • Impacts on economic stability, employment, and educational outcomes. Studies find that expansion decreased catastrophic health expenditures (health care spending as a percentage of family income). One study found that expansion was associated with greater increases in income among low-income individuals and contributed to decreased levels of income inequality. One study found an association between expansion and decreased odds of job loss, though two other studies found no effect of expansion on employment among people with disabilities. Finally, a national study found significant reductions in high school dropout rates in the first year of expansion implementation, which would translate to an 11.2% reduction in drop-out rates in non-expansion states if they adopted the expansion. 308 , 309 , 310 , 311 , 312 , 313 , 314
  • Outcomes for justice-involved and individuals experiencing homelessness. One study found that although pregnant women referred by criminal justice agencies to opioid use disorder (OUD) treatment facilities received medication as treatment at lower rates than women referred by other sources, expansion mitigated this effect by increasing receipt of medication for these women. Another study found that expansion resulted in decreased rates of recidivism in some geographic areas. A study in Arkansas found a spike in utilization of acute care among adults experiencing homelessness who gained coverage through expansion, suggesting a pent-up demand that stabilized in the years following expansion implementation. 315 , 316 , 317  

Looking Ahead

This literature review builds on a prior report and summarizes new evidence on more specific outcomes for certain populations. The full body of Medicaid expansion research includes over 600 studies (summarized in Appendix A) and indicates overall positive effects across a range of outcomes for patients, providers, and states. These findings suggest that Medicaid expansion could help mitigate adverse impacts of the coronavirus pandemic at the patient, provider, and state level; although research to date on Medicaid expansion and COVID-19 remains limited, future studies will likely further consider these impacts. Additionally, continued research cited in this report demonstrating positive economic impacts may help inform states still debating whether to adopt the expansion , particularly given the new ARPA financial incentive that would more than offset state expansion costs for two years (after which states would continue to bear 10% of the cost). Future policy proposals at the state and federal level could further affect Medicaid expansion coverage and options for people in the coverage gap .

The authors thank Diana Park for her assistance reviewing studies for inclusion and compiling supplemental materials.

  • Affordable Care Act
  • Access to Care
  • State Budgets
  • Medicaid's Future

Also of Interest

  • The Effects of Medicaid Expansion under the ACA: Studies from January 2014 to January 2020
  • New Incentive for States to Adopt the ACA Medicaid Expansion: Implications for State Spending
  • Status of State Medicaid Expansion Decisions: Interactive Map
  • What Does the Recent Literature Say About Medicaid Expansion?: Economic Impacts on Providers

Research on country-of-origin perceptions: review, critical assessment, and the path forward

  • Review Article
  • Published: 20 February 2024

Cite this article

  • Saeed Samiee 1 ,
  • Leonidas C. Leonidou 2 ,
  • Constantine S. Katsikeas 3 &
  • Bilge Aykol 4  

14 Accesses

Explore all metrics

Despite the volume of research and significant advancements in the country-of-origin (CO) area, the topic remains contentious in two key areas. One area is the presence of tensions and contradictions associated with origin-related research. The second relates to an over-reliance on relatively narrow theories that can neither address disparities nor capture a range of CO ecosystem considerations critical to the effective use of results in addressing firm-level planning and outcomes. Our examination of the business-to-consumer CO literature details the characteristics of published work and highlights the substantive contributions of the 50 most influential publications, with the overarching goal of accommodating meaningful future research. We examine 417 journal articles (551 studies) published from 1962 to 2022 to extract important granular characteristics of the literature and to summarize the findings of the most influential CO contributions. We also report the results of two surveys of academic researchers and US exporters and importers examining CO’s role in research and practice. Finally, we propose a theoretical lens, the paradox theory, as a basis for considering and framing competing aspects of the CO ecosystem and recommend the use of multilevel modeling to link future studies to marketing strategy and performance outcomes, thus advancing CO research.

Plain Language Summary

The research study examines the effect of a product's country of origin (CO – the country where a product is made) on consumer behavior. Despite thorough research on this subject since the 1960s, there are still areas of uncertainty regarding how consumers perceive products and brands from different countries, and how these perceptions guide their purchasing choices. The study's goal is to critically evaluate existing research on CO effects on consumer behavior and to provide summaries of the top 50 most influential contributions. The study conducted a thorough review of published research on CO effects, supplemented by two surveys to gather the opinions of both CO researchers and industry professionals. The research examines the theoretical foundations of the CO literature, key methodological aspects of CO research, major empirical findings on the topic, and offers suggestions for future research. To identify and extract prior CO articles, the study utilized several key datasets, including ABI Global, EBSCO, Scopus, and the Web of Science. In general, CO research shows that consumers have different perceptions of products and brands from different countries, which can guide their purchasing decisions. However, there are ongoing questions and concerns about the precision and relevance of CO research, including weak or nonexistent theoretical and conceptual framing, conflicting findings, and limited managerial relevance. The study asserts that the CO literature lacks a comprehensive theoretical framework that includes competing aspects within the CO ecosystem. The research concludes that future CO studies should incorporate consumer sentiments that are linked to marketing strategy and performance outcomes. The study also suggests the use of multilevel conceptualizations that concurrently involve consumers, suppliers, and the context of the consuming nation, inclusive of marketing strategy elements and outcomes. The results of this research have substantial implications for researchers, managers, and policymakers in understanding and leveraging the effect of a product's country of origin on consumer behavior.

This text was initially drafted using artificial intelligence, then reviewed by the author(s) to ensure accuracy.

Malgré le volume de recherche et les avancées significatives dans le domaine du pays d’origine ( Country-of-Origin - CO), le sujet reste controversé sur deux aspects clés. Le premier réside dans la présence de tensions et de contradictions associées à la recherche portée sur l’origine. Le second est lié à une dépendance excessive des théories relativement étroites lesquelles ne peuvent ni aborder les disparités ni saisir une série de considérations liées à l’écosystème du CO essentielles à l’utilisation efficace des résultats dans la résolution de la planification et des conséquences au niveau de l’entreprise. Notre examen de la littérature du «Business-to-Consumer» CO détaille les caractéristiques des travaux publiés et met en lumière les contributions substantielles des 50 publications les plus influentes, et ce dans le but ultime de soutenir de futures recherches importantes. Nous examinons 417 articles scientifiques (551 travaux) publiés durant la période 1962–2022 afin d’extraire des caractéristiques granulaires importantes de la littérature et de résumer les résultats des contributions les plus influentes dans le domaine du CO. Nous spécifions également les résultats de deux enquêtes menées auprès de chercheurs universitaires et d’exportateurs et importateurs américains visant à examiner le rôle du CO dans la recherche et la pratique. Enfin, nous proposons une perspective théorique, la théorie du paradoxe, comme base pour considérer et encadrer les aspects concurrents de l'écosystème du CO, et nous recommandons l'utilisation d'une modélisation multiniveaux pour relier de futures recherches à la stratégie de marketing et aux conséquences en matière de performance, faisant ainsi progresser la recherche sur le CO.

A pesar del volumen de investigación y los avances significativos del área de país de origen (CO por las siglas en inglés usada por los autores), el tema sigue siendo polémico en dos áreas clave. Un área es la presencia de tensiones y contradicciones asociadas con la investigación relacionada con el origen. La segunda se basa excesivamente en teorías relativamente restringidas que no pueden ni abordar disparidades ni capturar un rango de consideraciones críticas de ecosistemas de país de origen para el uso efectivo de resultados en abordar la planeación y los resultados a nivel de la empresa. Nuestro estudio de la bibliografía sobre el país de origen de las empresas y consumidores detalla las características de los trabajos publicados y destaca las contribuciones sustantivas de las 50 publicaciones más influyentes, con el objetivo general de dar cabida a futuras investigaciones significativas. Examinamos 417 artículos de revistas (551 estudios) publicados entre 1962 y 2022 para extraer importantes características granulares de la bibliografía y resumir las conclusiones de las contribuciones sobre país de origen más influyentes. También presentamos los resultados de dos encuestas realizadas a investigadores académicos y a exportadores e importadores estadounidenses, examinando el papel del país de origen en la investigación y la práctica. Por último, proponemos un lente teórico, la teoría de la paradoja, como base para considerar y enmarcar los aspectos contrapuestos del ecosistema del país de origen y recomendamos la modelación multinivel para vincular futuros estudios con la estrategia de marketing y los resultados de rendimiento, avanzando de esta manera la investigación sobre el país de origen.

Apesar do volume de pesquisa e avanços significativos na área do país de origem (CO), o tema permanece controverso em duas áreas principais. Uma área é a presença de tensões e contradições associadas à pesquisa relacionada com a origem. A segunda diz respeito a uma dependência excessiva de teorias relativamente estreitas que não conseguem abordar disparidades nem captar uma série de considerações sobre o ecossistema do CO críticas para a utilização eficaz de resultados na abordagem do planeamento e dos resultados a nível da empresa. Nossa análise da literatura sobre CO entre empresas e consumidores detalha as características de trabalhos publicados e destaca as contribuições substantivas das 50 publicações mais influentes, com o objetivo geral de acomodar significativas pesquisas futuras. Examinamos 417 artigos de periódicos (551 estudos) publicados de 1962 a 2022 para extrair características granulares importantes da literatura e resumir as descobertas das contribuições mais influentes do CO. Também relatamos os resultados de duas pesquisas realizadas com pesquisadores acadêmicos e exportadores e importadores dos Estados Unidos que examinam o papel do CO na pesquisa e na prática. Finalmente, propomos uma lente teórica, a teoria do paradoxo, como uma base para considerar e enquadrar aspectos concorrentes do ecossistema de CO e recomendamos o uso de modelagem multinível para associar estudos futuros à estratégia de marketing e resultados de desempenho, dessa forma avançando a pesquisa de CO.

尽管在原产国 (CO) 领域有大量研究并取得了重大进展, 该主题在两个关键领域仍然存在着争议。其中一个领域是与起源相关的研究有冲突和矛盾。第二个与过度依赖相对狭隘的理论有关, 这些理论既不能解决差异又不能捕捉一系列的对于有效利用结果来解决公司层面的规划和成果至关重要的关于CO生态系统的考量。我们对企业对消费者的 CO 文献的研究详细介绍了所发作品的特征, 并强调了 50 种最有影响的出版物的实质贡献, 其总体目标是为有意义的未来研究提供支持。我们研究了 1962 至 2022 年间发表的 417 篇期刊文章 (551 项研究) , 以提取文献的重要细节特征, 并总结对 CO最具影响力的贡献的研究发现。我们还报告了两项针对学术研究人员和美国进出口商的问卷调查结果, 以考察CO在研究和实践中的作用。最后, 我们提出了一种理论视角, 即悖论, 作为考虑和框架 CO 生态系统竞争方面的基础, 并建议使用多层次建模将未来研究与营销策略和绩效结果联系起来, 从而推进 CO 研究。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

use of research literature

The EU, for example, grants three geographic indications (GI) if a product has unique characteristics linked to a region: (1) protected designation of origin (PDO: food and wine), (2) protected geographical indication (PGI: food and wine), and (3) geographical indication (GI: spirit drinks). PDO has the strongest link to origin (e.g., Kalamata olive oil). PGI is associative based on the connection between a region and product (e.g., Bordeaux and Champagne) (European Commission, 2023 ).

The Academic Journal Guide (Chartered Association of Business Schools) classifies most business publications, including about 70 marketing journals.

Search terms are available from the authors upon request.

We define an adapted theory as one that modifies its original form by taking into consideration the specific context (or problem) at hand, inclusive of the theory’s associated constructs.

A recent collection of contributions by Al-Sulaiti and Al-Sulaiti ( 2023 ) is indicative of scholars’ attention to this gap in the CO literature.

Although we commissioned 35 responses each from importers and exporters, Qualtrics delivered 13 additional responses (9 exporters and 4 importers).

Group means for exporters and importers are not statistically different and are only slightly above the midpoint of our 7-point scale. However, importers’ means are invariably lower across all items (exporters = 4.70–5.34; importers = 4.05–5.15, where 1 = strongly disagree and 7 = strongly agree).

We use “foreign-linked” to convey all origin-related associations with a product, brand, or service, inclusive of foreign ownership of locally produced products or brands, and service/tourism brands.

The “overlapping demand theory” (Linder, 1961 ) asserts that greater similarity of end-user preferences across countries leads to greater similarity across products and services demanded and, hence, the greater the trade potential between nations.

Some CO studies use “multilevel” to refer to consumer responses to, e.g., product and brand (e.g., Leonidou, Palihawadana, & Talias, 2007 ; Zhou, Yang, & Hui, 2010 ), but do not use multilevel designs.

Abdellah-Kilani, F., & Zorai, R. (2019). Brand Origin Recall Accuracy (BORECA): A new measure of brand origin salience. International Marketing Review, 36 (3), 464–482.

Article   Google Scholar  

Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior . Prentice Hall.

Google Scholar  

Al-Sulaiti, K. I., & Al-Sulaiti, I. K. (2023). Country of origin effects on service evaluation. Doha, Qatar: Qatar University Press. Available at: https://qspace.qu.edu.qa/bitstream/handle/10576/49727/Country%20of%20origin%20Effects%20on%20Service%20Evaluation%20OA.pdf?sequence=3&isAllowed=y

Albaum, G., & Peterson, R. A. (1984). Empirical research in international marketing: 1976–1982. Journal of International Business Studies, 15 (1), 161–173.

Balabanis, G., & Diamantopoulos, A. (2004). Domestic country bias, country-of-origin effects, and consumer ethnocentrism: A multidimensional unfolding approach. Journal of the Academy of Marketing Science, 32 (1), 80–95.

Balabanis, G., & Diamantopoulos, A. (2008). Brand origin identification by consumers: A classification perspective. Journal of International Marketing, 16 (1), 39–71.

Balabanis, G., Mueller, R., & Melewar, T. C. (2002). The human values’ lenses of country of origin images. International Marketing Review, 19 (6), 582–610.

Barney, J. B. (1996). Gaining and sustaining competitive advantage . Addison-Wesley.

Bartikowski, B., Fastoso, F., & Gierl, H. (2019). Luxury cars Made-in-China: Consequences for brand positioning. Journal of Business Research, 102 , 288–297.

Batra, R., Ramaswamy, V., Alden, D. L., Steenkamp, J. E. B. M., & Ramachander, S. (2000). Effects of brand local/non-local origin on consumer attitudes in developing countries. Journal of Consumer Psychology, 9 , 83–95.

Bhaskaran, S., & Sukumaran, N. (2007). Contextual and methodological issues in COO studies. Marketing Intelligence & Planning, 25 (1), 66–81.

Bloemer, J., Brijs, K., & Kasper, H. (2009). The CoO-ELM model: A theoretical framework for the cognitive processes underlying country of origin-effects. European Journal of Marketing, 43 (1/2), 62–89.

Cakici, N. M., & Shukla, P. (2017). Country-of-origin misclassification awareness and consumers’ behavioral intentions: Moderating roles of consumer affinity, animosity, and product knowledge. International Marketing Review, 34 (3), 354–376.

Denison, D. R., Hooijberg, R., & Quinn, R. E. (1995). Paradox and performance: Toward a theory of behavioral complexity in managerial leadership. Organization Science, 6 (5), 524–540.

Diamantopoulos, A., Florack, A., Halkias, G., & Palcu, J. (2017). Explicit versus implicit country stereotypes as predictors of product preferences: Insights from the stereotype content model. Journal of International Business Studies, 48 , 1023–1036.

Diamantopoulos, A., & Zeugner-Roth, K. P. (2011). Country-of-origin as brand element. In D. C. Bello & D. A. Griffith (Eds.), Wiley International Encyclopedia of Marketing (Vol. 6, pp. 18–22). Wiley.

Dichter, E. (1962). The world customer. Harvard Business Review, 40 (4), 113–122.

Eroglu, S. A., & Machleit, K. A. (1989). Effects of individual and product-specific variables on utilising country of origin as a product quality cue. International Marketing Review, 6 (6), 27–41.

European Commission. (2023). Geographical indications and quality schemes explained . EU Geographic Indication Rules. Accessed April 21, 2023.

Funk, C. A., Arthurs, J. D., Treviño, L. J., & Joireman, J. (2010). Consumer animosity in the global value chain: The effect of international production shifts on willingness to purchase hybrid products. Journal of International Business Studies, 41 (4), 639–651.

Gineikiene, J., Schlegelmilch, B. B., & Ruzeviciute, R. (2016). Our apples are healthier than your apples: Deciphering the healthiness bias for domestic and foreign products. Journal of International Marketing, 24 (2), 80–99.

Gong, W. (2003). Chinese consumer behavior: A cultural framework and implications. Journal of American Academy of Business, 3 (1/2), 373–380.

Graetz, F., & Smith, A. C. (2009). Duality theory and organizing forms in change management. Journal of Change Management, 9 (1), 9–25.

Håkansson, H., & Ford, D. (2002). How should companies interact in business networks? Journal of Business Research, 55 (2), 133–139.

Halimi, T. A., D’Souza, C., & Sullivan-Mort, G. (2017). Examining the role of empathy on third-country nationals’ foreign product purchase behavior: A study of the growing international boycott against Israel. International Marketing Review, 34 (6), 760–786.

Han, C. M. (1989). Country image: Halo or summary construct? Journal of Marketing Research, 26 (2), 222–229.

Han, C. M., & Terpstra, V. (1988). Country-of-origin effects for uni-national and bi-national products. Journal of International Business Studies, 19 (2), 235–255.

Herz, M. F., & Diamantopoulos, A. (2013a). Activation of country stereotypes: Automaticity, consonance, and impact. Journal of the Academy of Marketing Science, 41 , 400–417.

Herz, M. F., & Diamantopoulos, A. (2013b). Country-specific associations made by consumers: A dual-coding theory perspective. Journal of International Marketing, 21 (3), 95–121.

Herz, M. F., & Diamantopoulos, A. (2017). I use it but will tell you that I don’t: Consumers’ country-of-origin cue usage denial. Journal of International Marketing, 25 (2), 52–71.

Hofmann, D. A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23 (6), 723–744.

Hu, Y., & Wang, X. (2010). Country-of-origin premiums for retailers in international trades: Evidence from eBay’s international markets. Journal of Retailing, 86 (2), 200–207.

Insch, A., & Jackson, E. (2014). Consumer understanding and use of country-of-origin in food choice. British Food Journal, 116 (1), 62–79.

Jo, M., Nakamoto, K., & Nelson, J. E. (2003). The shielding effects of brand image against lower quality countries-of-origin in global manufacturing. Journal of Business Research, 56 (8), 637–646.

Johansson, J. K., & Thorelli, H. B. (1985). International product positioning. Journal of International Business Studies, 16 (3), 57–75.

Josiassen, A., & Harzing, A. W. (2008). Descending from the ivory tower: Reflections on the relevance and future of country-oforigin research. European Management Review, 5 (4), 264–270.

Katsikeas, C. S., Madan, S., Brendl, C. M., Candler, B., Lehmann, D., Baumgartner, H., Weijters, B., Wang, M., Huang, C., & Huber, J. (2023). Commentaries on ‘Scale use and abuse: Toward best practices in the deployment of scales.’ Journal of Consumer Psychology, 33 (1), 244–258.

Kaynak, E., Kucukemiroglu, O., & Hyder, A.S. (2000). Consumers’ country‐of‐origin (COO) perceptions of imported products in a homogenous less‐developed country. European Journal of Marketing , 34 (9/10), 1221–1241.

Khanna, S. R. (1986). Asian companies and the country stereotyping paradox: An empirical study. Columbia Journal of World Business, 21 (2), 29–38.

Klein, J. G., Ettenson, R., & Morris, M. D. (1998). The animosity model of foreign product purchase: An empirical test in the People’s Republic of China. Journal of Marketing, 62 (1), 89–100.

Kock, F., Josiassen, A., & Assaf, A. G. (2019). Toward a universal account of country-induced predispositions: integrative framework and measurement of country-of-origin images and country emotions. Journal of International Marketing, 27 (3), 43–59.

Kotabe, M., & Jiang, C. X. (2008). Contemporary research trends in international marketing: The 2000s. In A. Rugman (Ed.), Oxford Handbook of International Business, Ch. 17 (2nd ed., pp. 447–501). Oxford University Press.

Kuhn, T. S. (1962). The structure of scientific revolutions . University of Chicago Press.

Laroche, M., Papadopoulos, N., Heslop, L. A., & Mourali, M. (2005). The influence of country image structure on consumer evaluations of foreign products. International Marketing Review, 22 (1), 96–115.

Leonidou, L. C., Katsikeas, C. S., Samiee, S., & Aykol, B. (2018). International marketing research: A state-of-the-art review and the way forward. In L. C. Leonidou, C. S. Katsikeas, S. Samiee, L. C. Leonidou, C. S. Katsikeas, S. Samiee, & B. Aykol (Eds.), Advances in global marketing: A research anthology (pp. 3–33). Springer.

Chapter   Google Scholar  

Leonidou, L. C., Palihawadana, D., & Talias, M. A. (2007). British consumers’ evaluations of US versus Chinese goods: A multi-level and multi-cue comparison. European Journal of Marketing, 41 (7/8), 786–820.

Liefeld, J. P. (2004). Consumer knowledge and use of country-of-origin information at the point of purchase. Journal of Consumer Behaviour: An International Research Review, 4 (2), 85–87.

Linder, S. B. (1961). An Essay on Trade and Transformation (pp. 82–109). Almqvist & Wiksell.

Ma, J., Yang, J., & Yoo, B. (2020). The moderating role of personal cultural values on consumer ethnocentrism in developing countries: The case of Brazil and Russia. Journal of Business Research, 108 , 375–389.

MacRoberts, M., & MacRoberts, B. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40 (5), 342–349.

Magnusson, P., Westjohn, S. A., & Zdravkovic, S. (2011). “What? I thought Samsung was Japanese”: accurate or not, perceived country of origin matters. International Marketing Review, 28 (5), 454–472.

Maheswaran, D. (1994). Country of origin as a stereotype: Effects of consumer expertise and attribute strength on product evaluations. Journal of Consumer Research, 21 (2), 354–365.

Mandler, T., Won, S., & Kim, K. (2017). Consumers’ cognitive and affective responses to brand origin misclassifications: Does confidence in brand origin identification matter? Journal of Business Research, 80 , 97–209.

Misangyi, V. F., Elms, H., Greckhamer, T., & Lepine, J. A. (2006). A new perspective on a fundamental debate: a multilevel approach to industry, corporate, and business unit effects. Strategic Management Journal, 27 (6), 571–590.

Molina-Azorín, J. F., Pereira-Moliner, J., & Claver-Cortés, E. (2010). The importance of the firm and destination effects to explain firm performance. Tourism Management, 31 , 22–28.

Nes, E. B. (2019). The role of country images in international marketing: Country-of-origin effects. In D. Ingenhoff, C. White, A. Buhmann, & S. Kiousis (Eds.), Bridging Disciplinary Perspectives of Country Image Reputation, Brand, and Identity (pp. 33–48). Routledge.

Obermiller, C., & Spangenberg, E. R. (1989). Exploring the effects of country-of-origin labels: An information processing framework. In T. K. Srull (Ed.), Advances in Consumer Research, Vol. 16, pp. 454–459.

Ozanne, L. K., Phipps, M., Weaver, T., Carrington, M., Luchs, M., Catlin, J., Gupta, S., Santos, N., Scott, K., & Williams, J. (2016). Managing the tensions at the intersection of the triple bottom line: A paradox theory approach to sustainability management. Journal of Public Policy & Marketing, 35 (2), 249–261.

Peterson, R. A., & Jolibert, A. J. P. (1995). A meta-analysis of country-of-origin effects. Journal of International Business Studies, 26 , 883–900.

Poole, M. S., & Van de Ven, A. H. (1989). Using paradox to build management and organization theories. Academy of Management Review, 14 (4), 562–578.

Roth, K. P., & Diamantopoulos, A. (2009). Advancing the country image construct. Journal of Business Research, 62 (7), 726–740.

Roth, M. S., & Romeo, J. B. (1992). Matching product category and country image perceptions: A framework for managing country-of-origin effects. Journal of International Business Studies, 23 (3), 477–497.

Samiee, S. (1994). Customer evaluation of products in a global market. Journal of International Business Studies, 25 (3), 579–604.

Samiee, S. (2011). Resolving the impasse regarding research on the origins of products and brands. International Marketing Review, 28 (5), 473–485.

Samiee, S., & Chabowski, B. R. (2021). Knowledge structure in product-and brand origin–related research. Journal of the Academy of Marketing Science, 49 (5), 947–968.

Samiee, S., & Leonidou, L. C. (2011). Relevance and rigor in international marketing research: Developments in product and brand origin line of inquiry. In S. C. Jain & D. A. Griffith (Eds.), Handbook of research in international marketing (pp. 68–87). Edward Elgar.

Samiee, S., Shimp, T., & Sharma, S. (2005). Brand origin recognition accuracy: Its antecedents and consumers’ cognitive limitations. Journal of International Business Studies, 36 , 379–397.

Schooler, R. D. (1965). Product bias in the Central American Common Market. Journal of Marketing Research, 2 (4), 394–397.

Semaan, R. W., Gould, S., Chao, M. C., & Grein, A. F. (2019). “We don’t all see it the same way” The biasing effects of country-of-origin and preference reversals on product evaluation. European Journal of Marketing, 53 (5), 989–1014.

Sharma, P. (2011). Country of origin effects in developed and emerging markets: Exploring the contrasting roles of materialism and value consciousness. Journal of International Business Studies, 42 , 285–306.

Shimp, T. A., Samiee, S., & Madden, T. J. (1993). Countries and their products: A cognitive structure perspective. Journal of the Academy of Marketing Science, 21 (4), 323–330.

Shimp, T. A., & Sharma, S. (1987). Consumer ethnocentrism: Construction and validation of the CETSCALE. Journal of Marketing Research, 24 (3), 280–289.

Smith, W., & Lewis, M. (2011). Toward a theory of paradox: A dynamic equilibrium model of organizing. Academy of Management Review, 36 (2), 381–403.

Usunier, J. (2011). The shift from manufacturing to brand origin: Suggestions for improving COO relevance. International Marketing Review, 28 (5), 486–496.

Usunier, J. C., & Cestre, G. (2008). Further considerations on the relevance of country-of-origin research. European Management Review, 5 (4), 271–274.

Vafeas, M. (2021). Client–agency briefing: Using paradox theory to overcome challenges associated with client resource deployment. Journal of Advertising, 50 (3), 299–308.

Verlegh, P. W. J., & Steenkamp, J. B. E. M. (1999). A review and meta-analysis of country-of-origin research. Journal of Economic Psychology, 20 (5), 521–546.

Zeithaml, V. A., Jaworski, B. J., Kohli, A. K., Tuli, K. R., Ulaga, W., & Zaltman, G. (2020). A theories-in-use approach to building marketing theory. Journal of Marketing, 84 (1), 32–51.

Zhou, L., Yang, Z., & Hui, M. K. (2010). Non-local or local brands? A multi-level investigation into confidence in brand origin identification and its strategic implications. Journal of the Academy of Marketing Science, 38 (2), 202–218.

Zeugner-Roth, K. P., Žabkar, V., & Diamantopoulos, A. (2015). Consumer ethnocentrism, national identity, and consumer cosmopolitanism as drivers of consumer behavior: A social identity theory perspective. Journal of International Marketing , 23 (2), 25–54.

Download references

Author information

Authors and affiliations.

University of Tulsa, Tulsa, OK, USA

Saeed Samiee

University of Cyprus, Nicosia, Cyprus

Leonidas C. Leonidou

University of Leeds, Leeds, UK

Constantine S. Katsikeas

Dokuz Eylül University, Izmir, Turkey

Bilge Aykol

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Saeed Samiee .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Accepted by Masaaki Kotabe, Consulting Editor, 9 November 2023. This article has been with the authors for two revisions.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOC 605 KB)

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Samiee, S., Leonidou, L.C., Katsikeas, C.S. et al. Research on country-of-origin perceptions: review, critical assessment, and the path forward. J Int Bus Stud (2024). https://doi.org/10.1057/s41267-023-00678-z

Download citation

Received : 23 September 2022

Revised : 27 October 2023

Accepted : 09 November 2023

Published : 20 February 2024

DOI : https://doi.org/10.1057/s41267-023-00678-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Country-of-origin
  • CO ecosystem
  • Literature review
  • Paradox theory
  • Multilevel modeling
  • International marketing

Advertisement

  • Find a journal
  • Publish with us
  • Track your research
  • Open access
  • Published: 17 February 2024

The complexity of leadership in coproduction practices: a guiding framework based on a systematic literature review

  • Sofia Kjellström 1 , 2 ,
  • Sophie Sarre 2 &
  • Daniel Masterson 1  

BMC Health Services Research volume  24 , Article number:  219 ( 2024 ) Cite this article

378 Accesses

Metrics details

As coproduction in public services increases, understanding the role of leadership in this context is essential to the tasks of establishing relational partnerships and addressing power differentials among groups. The aims of this review are to explore models of coproduction leadership and the processes involved in leading coproduction as well as, based on that exploration, to develop a guiding framework for coproduction practices.

A systematic review that synthesizes the evidence reported by 73 papers related to coproduction of health and welfare.

Despite the fact that models of coleadership and collective leadership exhibit a better fit with the relational character of coproduction, the majority of the articles included in this review employed a leader-centric underlying theory. The practice of coproduction leadership is a complex activity pertaining to interactions among people, encompassing nine essential practices: initiating, power-sharing, training, supporting, establishing trust, communicating, networking, orchestration, and implementation.

Conclusions

This paper proposes a novel framework for coproduction leadership practices based on a systematic review of the literature and a set of reflective questions. This framework aims to help coproduction leaders and participants understand the complexity, diversity, and flexibility of coproduction leadership and to challenge and enhance their capacity to collaborate effectively.

Peer Review reports

Introduction

For more than 40 years, scholars and practitioners have sought to identify and understand various aspects of coproduction with the goal of improving services as well as equalizing (or at least reorganizing) power relations in service design and delivery [ 1 ]. More recently, such discussion has focused on the roles of leaders and leadership in coproduction, seeking to describe and assess the various types of leaders and leadership that might maximize the goals of coproduction processes and outcomes. Leaders can act to make coproduction, in all its forms, happen [ 2 , 3 ]. Leaders can enhance coproduction by providing resources, establishing inviting structures, and prioritizing the involvement of various stakeholders. Conversely, they can inhibit coproduction by perpetuating conservative administrative cultures, failing to provide training, or being reluctant to share power [ 3 ]. Coproduction relies on leadership at all levels, ranging from senior managers to local “champions” and including the citizens and third-sector organizations that participate in coproduction activities and practices.

This review presents a synthesis of research on the leadership of coproduction, which has been recognized for its scarcity [ 3 , 4 , 5 , 6 ]. The review provides new knowledge regarding the fact that coproduction leadership must become more deliberately (in)formed by collective leadership models. It also illustrates the multiplicity and complexity associated with coproduction leadership activities by outlining practices in which leaders must engage to ensure success. This review can inform a framework that offers guiding insights on which commissioners, evaluators, managers and leaders of coproduction can reflect as well as suggestions and directions for future research.

  • Coproduction

Coproduction is a broad concept that is associated with different meanings across a range of contexts [ 1 ]. Many definitions and uses of the term coproduction and codesign have been identified [ 7 ]. Throughout this paper, although we acknowledge the distinctions associated with the concepts and origins of the notion of codesign, we use the broad term coproduction to refer to some form of collaboration or partnership between service providers and service users or citizens. For this review, we follow the definitions provided by Osborne and Strokosch [ 8 ], who identified ‘ consumer coproduction’ as an inevitable component of value creation in interactions among service providers; ‘participatory coproduction’, in which context participation is deliberative and occurs at the strategic level of service design and planning; and ‘enhanced coproduction’, which represents a potential mechanism for transforming organizational processes and boundaries.

Power is inevitably central to coproduction. Schlappa and Ymani claimed that the coproduction process is “inherently negotiated, emergent and reliant on a range of actors who may have both common and contrasting motivations, and are able to exercise power, which in turn is moderated by the context in which these relations occur” [ 6 ]. This sensitivity to motivation, context and power is helpful for our understanding of leadership in coproduction.

Leadership models

Most conceptualizations of leadership have been based on the claim that leadership is a kind of inherent characteristic exhibited by human beings, such that leaders are depicted as heroes with unique traits, styles or behaviours [ 9 ]. However, research on leadership in coproduction is important in relation to an emerging body of research that focuses on the notion of “leadership in the plural” [ 10 ] or “collective leadership” [ 11 , 12 ]. These phrases act as umbrella terms that refer to overlapping concepts such as shared, collaborative, distributed, pooled and relational leadership. A core feature of these models is that leadership is not (only) viewed as a property of individuals and their behaviours but rather as a collective phenomenon that is distributed or shared among different people [ 10 ]. A distinction can be made between two types of collective leadership. Leadership can be shared in interpersonal relationships; for example, it can be pooled among duos or trios at the top of an organization, or shared leadership can be exercised within teams working on a project. This notion is based upon the assumption that people have different skills that complement each other. The second kind of collective leadership is a more radical version of this notion, according to which leadership emerges as a result of direction, alignment, and commitment within a group [ 11 ] or can be observed to reside within the system, for example, in the form of distributed leadership across interorganizational and intraorganizational boundaries and networks [ 10 , 12 ]. In cross-sectoral collaboration, leadership is distributed across time and space, which requires structures to guide how leadership is shared and organized. It has been argued that collective leadership is best suited to the analysis of coproduction practices [ 4 , 6 , 13 , 14 ].

It is important to note that distinctions have been made between management (planning, monitoring and controlling) and leadership (creating a vision, inspiring and changing) based on behaviours [ 15 ]. However, many authors have not made such a distinction, and the terms have frequently been used interchangeably. We therefore adopt the practice employed in the papers included in this review and use the terms leadership and leader as catch-all terms; we only use the words management or manager when the papers refer to job titles or ‘public management’.

Leadership models can be regarded as resembling a colour palette that offers a variety of choices, and similar to colours, some models fit a situation better than others. This paper investigates the use and fit of various leadership models for coproduction.

Leadership of coproduction research

Extant research on the leadership of coproduction has been described as “sparse” [ 4 ], a “neglected area” [ 5 ] and “overlooked” [ 3 , 6 ]. Despite a recent resurgence of interest in the potential of coproduction as a means of maintaining and improving the quality of health and social care, significant questions regarding how coproduction can and should be led in this context remain unanswered. Most reviews of coproduction have not addressed this issue [ 2 , 16 , 17 , 18 ]. Clarke et al.’s (2017) review identified the lack of managerial authority and leadership as a key barrier to the implementation of coproduced interventions but did not explore the implications of this finding for future practice. The review conducted by Bussu and Galanti (2018) stands alone in its focus on leadership, although the empirical cases explored by those authors were restricted to the context of local government in the UK. Recent empirical case studies that have explored leadership [ 13 , 14 , 15 , 19 ] have focused on public managers [ 3 , 5 , 14 ] or on identifying the consequences of different models of leadership. This review contributes to the literature by providing knowledge regarding how to make deliberate choices pertaining to coproduction leadership in terms of how it is conceptualized and shared and the activities that are necessary for leading coproduction.

Coproduction leadership practices

The leadership of coproduction poses a number of challenges. A proposed aim of coproduction is to drive change within services and in traditional state-citizen relationships by establishing equal and reciprocal relationships among professionals, the people using services, and their families and neighbours. This task requires a restructuring of health and welfare services to equalize power between providers and other stakeholders with an interest in the design and provision of these services. However, it has been suggested that coproduction runs the risk of reproducing existing inequalities in power rather than mitigating them since coproduction is inevitably saturated with unequal power relations that must be acknowledged but cannot be managed away [ 20 ].

In this paper, we present the findings of a systematic review of the literature on leadership in coproduction. The purpose of this review is to explore models of coproduction leadership and the practices involved in leading coproduction in the context of health and social care sectors [ 7 ]. The results are synthesized to develop a framework for actors who seek to commission, design, lead or evaluate coproduction processes. This framework emphasizes the need to make more deliberate choices regarding the underlying conceptualization of leadership and the ways in which such a conceptualization is related to the activities necessary for leading coproduction. Based on the framework, we also propose specific guiding questions for individuals involved in coproduction in practice and make suggestions for future research.

This systematic literature review is based on a study protocol on coproduction research in the context of health and social care sectors [ 21 ], and data were obtained from a published scoping review, where the full search strategy is provided [ 7 ]. The scoping review set out to identify ‘what is out there’ and to explore the definitions of the concepts of coproduction and codesign. In brief, the following search terms for the relevant concept (co-produc* OR coproduc* OR co-design* OR codesign*) and context (health OR social OR & “public service*” OR “public sector”) were used to query the following databases: CINAHL with Full Text (EBSCOHost), Cochrane Central Register of Controlled Trials (Wiley), MEDLINE (EBSCOhost), PsycINFO (ProQuest), PubMed (legacy), and Scopus (Elsevier). This paper focused on leadership. All titles and abstracts included in the scoping review ( n  = 979) were obtained and searched for leadership concepts (leader* OR manage*) ( n  = 415). These materials were reviewed independently by SK and SS using the following inclusion criterion: conceptual, empirical and reflection papers that included references to the management and/or leadership of coproduction. Study protocols were excluded because we wanted to capture lessons drawn from implementation, and conference papers were excluded because they lacked sufficient detail. Articles focusing on the context of individual-level coproduction (i.e., cases in which an individual client or patient was the focus of coproduction) were excluded, as we were interested in the leadership processes involved in collective coproduction. Conflicts were resolved through discussion and further consideration of disputed papers. This process led to the inclusion of 73 articles (Fig.  1 – PRISMA flow chart).

figure 1

PRISMA flow chart

The method used for this research was a systematic review with qualitative synthesis. The strength of this approach lies in its ability to complement research evidence with user and practitioner considerations [ 22 ]. In the process of examining the full texts of the papers, two researchers (SK and SS) extracted background data independently. To promote coproduction, four stakeholders were strategically selected through the personal networks of one of the authors, SK. These stakeholders exhibited diverse expertise in the leadership of coproduction. One was a leadership developer and family member of an individual with 24/7 care needs. Another was a physician. The third worked in peer support and had personal experience with mental health services. The fourth was a health care leader. Four key articles were chosen due to the diversity of leadership ideas they exhibit and the depth of the explicit text on leadership they provided. During the analysis by stakeholders, no themes were changed or refined; instead, the analysis confirmed the relevance of the initially identified themes, thus emphasizing the robustness of our findings based on a process that involved reading four key articles and identifying the perceived key implications for our research aim.

A qualitative synthesis unites the findings of individual studies in a different arrangement, thereby constructing new knowledge that is not apparent from the individual studies in isolation [ 23 ]. This fact is particularly evident in this review, since leadership was seldom the main focus of the included articles. Accordingly, we employed multiple pieces of information to construct a pattern. The process of synthesis started at a very broad level with the goal of understanding which aspects of leadership were addressed in the literature. This process then separated into two strands. One such strand focused on interpreting the data from the perspective of current leadership models, while the other focused on interpreting leadership practices – i.e., the activities and relationships that are part of the process of leading coproduction. We searched for themes both within and across individual articles, and our goal was interpretative rather than purely aggregative. This process resulted in three themes pertaining to coproduction leadership models and nine coproduction leadership practices. We present these findings together in the form of a framework because consideration of both leadership models and practices prompts better and more conscious choices, which can improve the quality of coproduction. Persons one and two from the stakeholder group also provided feedback on a draft of this paper, and their insights were integrated into this research.

Sample description

We included 73 papers (Additional file 1 ) dating from 1994 to 2019 (the year in which the initial search was performed). Most of these papers were empirical ( n  = 54), and more than half of them were case studies ( n  = 30). Fifteen articles were conceptual papers, and four were literature reviews. The setting or focus of the papers was predominantly on services ( n  = 66), while the remainder of the papers were on research ( n  = 4) or policy ( n  = 3). The papers drew on evidence collected from 13 countries, and the most common national setting was the UK ( n  = 29). Nine cross-national papers were also included. Issues related to leadership were rarely the focus of the papers.

Results: A coproduction leadership framework

The synthesis consists of three parts (roles, models and practices), which are combined to develop an overarching and integrative framework for essential issues pertaining to coproduction leadership [ 4 , 24 ].

People and roles

The way in which the leadership of coproduction has been conceptualized in the literature suggests that a range of actors are involved in the coproduction of health and wellbeing and that these actors can take on different leadership roles and functions. Service users, community members and community representatives can play a vital role in the task of deliberatively coproducing or even transforming services, as can third-sector organizations, external experts, politicians, mid-level facilitators, managers, and senior leaders.

It has been argued that it is important to involve leaders from diverse backgrounds who have personal experiential knowledge of public involvement to encourage involvement from a broader population [ 25 , 26 , 27 ]. Service users and community members play leadership roles in coproduction initiatives related to health or well-being. These roles involve shared decision-making and accountability at various levels, ranging from the personal to the systemic.

Senior leaders include formal representatives of organizations (executives, politicians, or formal managers) and formal or respected leaders of communities. They play an important role throughout this process. During the initiation stage, by implementing and sustaining the outcomes of coproduction, they play a crucial role in the provision of resources such as time, money, materials, and access to networks. In the interim stages, their commitment to coproduction, sponsorship, and engagement is vital.

Champions and ambassadors use their expertise and passion to drive coproduction efforts. In particular, "insider" champions can establish trust among participants and help service providers understand the importance of coproduction. These champions advocate for coproduction and actively support initiatives [ 28 , 29 , 30 , 31 ]. Ambassadors are individuals who have expertise and volunteer their time to train others or work with clients in coproduced services. They play a crucial role in the tasks of supporting and promoting coproduction [ 28 , 32 , 33 ].

Project leaders and facilitators are individuals who are responsible for guiding and supporting coproduction projects, thereby ensuring their smooth operation and collaborative nature. Project leaders are responsible for overall project management, including the setting of goals, objectives, and timelines. They play a pivotal role in ensuring that projects remain on track, and they facilitate accessible and transparent dialogue among stakeholders and ensure equal representation [ 34 , 35 ]. Facilitators focus on supporting the group involved in coproduction, maintaining respectful interactions, empowering service users and carers, and addressing any tensions that may arise during the collaborative process [ 36 , 37 ].

In summary, senior leaders sponsor and support coproduction. Champions and ambassadors are individuals who advocate for and support coproduction initiatives, while project leaders and facilitators are responsible for managing and guiding coproduction projects themselves, thereby ensuring effective collaboration among stakeholders. All of these roles can be played by people drawn from various backgrounds, including senior staff, health care professionals, experts in coproduction, researchers, citizens, or volunteers.

Three models of leadership in coproduction

These actors play different leadership roles, and leadership can be exercised by individuals or groups. Three leadership models have been proposed: leadership as enacted by individual leaders, coleadership and collective leadership.

Leadership by individual leaders

A leader-centric view has been the dominant interpretation of leadership in the field of coproduction. Many references were made to “senior leaders”. This term was used to describe formal representatives of organizations or services (senior managers, executives), formally appointed community leaders (policy-makers, local government leaders), or respected leaders of communities. Senior support was described as an important success factor in coproduction [ 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ]. Other leadership roles included project leaders, facilitators, ambassadors, and champions – as described in the previous section.

Some papers referred to traits and characteristics exhibited by leaders that facilitate coproduction. These factors included innovativeness, personability, action orientation [ 46 ], courage [ 47 ], passion [ 32 , 46 ], and empathy [ 25 , 46 , 48 ]. “Strong leadership” was often mentioned, albeit without elaboration [ 49 , 50 , 51 , 52 , 53 , 54 , 55 ]. By implication, “strong leadership” appeared to include providing clear direction and guidance, having a clear vision [ 53 ], holding onto a vision [ 34 ], and keeping the vision alive for the team [ 43 ].

Other researchers noted a more collaborative and democratic leadership style that is characterized by listening, transparency, deliberation, and nurturing coproductive behaviours [ 27 , 30 , 48 ]. Senior leaders could use a “top-down” approach to promote user involvement. Alternatively, they could “learn to manage horizontally not top down; embrace ground up initiatives; [and] aim to empower partners” [ 32 , 45 , 51 ] and be “open to changes that would disturb traditional relationships and power disparities between service users and providers” [ 41 ]. Respondents to a survey of participants in a peer-led support network favoured a traditional directive model of leadership alongside a more facilitative and enabling style [ 56 ]. However, they found it challenging to transition to a more distributed and collective leadership approach.

Co-leadership

The terms “co-lead”, “co-leadership” and “dual leadership” refer to situations in which a formal leadership role is allocated to more than one person, in which context the relevant people may represent different institutions or different groups, e.g., different professional groups, researchers and service users/citizens, or teachers and students [ 28 , 31 , 40 , 41 , 57 , 58 ]. Coleads were defined as “individuals who led and made joint decisions” [ 59 ]. Some papers explored the leadership role of service users or community members in the coproduction of research related to health or wellbeing [ 35 , 60 , 61 ]. In these studies, areas of research were proposed by patients/community members, who then collaborated with academic researchers, thereby playing an equal or leading role. Coleadership was reported to result in shared learning.

Collective leadership

Few discernible differences among “ shared”, “distributed” and “collective” leadership were found in the papers included in this review. The approaches examined in this context were characterized by distributed roles and responsibilities in which different individuals’ skills and expertise were identified as best suited to the task at hand. Shared leadership depends on willingness on the part of leaders (implicitly non-community leaders) to be challenged and directed by community members rather than rigidly maintaining their previous conceptions of the issues and the appropriate means of addressing them [ 36 ].

Ward, De Brún, Beirne, Conway, Cunningham, English, Fitzsimons, Furlong, Kane and Kelly [ 62 ] referred to collective leadership as an emergent and dynamic team phenomenon. Other authors argued for a more structured approach to shared leadership [ 36 , 41 ] or distributed leadership [ 28 , 42 , 56 , 59 , 63 ]. Such an approach could involve allocating specific roles to service users, engaging them in a formal structure and/or enabling them to set an agenda [ 41 ], specifying shared roles and responsibilities [ 36 ], and/or providing dedicated support to lay “champions” in research studies [ 28 ]. Various benefits were attributed to collective leadership, such as empowering people to speak up [ 36 , 51 ] and feel engaged.

Nine practices associated with leading coproduction

We identified nine processes that encompass wide-ranging activities and interactions between individuals and groups with regard to leading the coproduction of health and wellbeing. As Farr noted, “Coproduction and codesign […] involves facilitating, managing and co-ordinating a complex set of psychological, social, cultural and institutional interactions” [ 64 ]. In some cases, these processes naturally align with certain actors—for instance, senior leaders play key roles in the tasks of initiating coproduction and implementing and sustaining its results—but other processes (championing coproduction, establishing trusting relationships, and ensuring good communication) are applicable to any and all participants in the coproduction process. Similarly, some of these practices occur at particular timepoints in a coproduction arc (namely, during the stages of initiation or implementation), while others can occur at any or all timepoints (i.e., during the assimilation stage or beyond). Deliberately considering the most suitable leadership model with regard to the aims and context of an initiative is useful at the start, but reflecting on the operation and appropriateness of the model is always salient.

Initiating coproduction

The initiation of coproduction entails recognizing the need for coproduction, dedicating resources, inviting and establishing relevant multi-stakeholder coproduction networks, and coproducing a vision and goals.

It has been argued that senior leaders act as gatekeepers for coproduction because they must recognize the need for it [ 45 ]. Senior leaders play a role in the task of determining the extent to which communities are given the opportunity to influence service design and integration [ 38 , 51 ]. Coproduction requires resources (principally time and money but also networks), which can be used to take advantage of other resources such as skills [ 29 , 31 , 34 , 40 ]. Senior leaders often control or provide access to such resources, which means that they are best positioned to initiate coproduction initiatives [ 41 , 65 ]. However, the findings of a cross-national study on the coproduction of policy showed that, in practice, senior leaders’ control over resources meant that they tended to define the means, methods and forms of participation [ 65 ].

In the task of establishing a conducive environment for coproduction, it is important to pay attention to which actors (organizations or individuals) are participating in the process [ 33 , 42 , 64 , 66 ] and to factors that may delimit those participants or their involvement [ 36 , 42 , 67 ]. Several papers emphasized the need to ensure that all stakeholders are involved from the outset [ 37 , 38 , 41 , 48 , 51 ]. In the initiation stages, a shared vision should be created [ 36 , 61 , 68 ], goals should be coproduced, and responsibilities should be clearly allocated [ 65 ]. Role clarity, ability, and motivation have been identified as determinants of coproductive behaviour, and leaders must implement arrangements to achieve these goals for coproducers [ 69 ].

Power sharing

It has been argued that coproduction leadership must attend to issues pertaining to power redistribution [ 60 , 61 , 63 , 64 ] and uphold the ideology of coproduction by promoting the values of democracy and transparency [ 30 , 32 , 70 , 71 , 72 , 73 , 74 ]. This process can occur at different levels.

At the macro system level, several cultural shifts have been implicated in the redistribution of power – a shift in current professional and stakeholder identities; more fluid, flattened and consensus-based ways of working; and a willingness to accommodate ‘messy’ issues [ 75 ]. The last of those issues was highlighted by Hopkins, Foster and Nikitin [ 29 , s 192], who suggested that coproduction requires service providers to “sit more easily with the unknown, to be comfortable in not having all the answers.” Similarly, “The challenge is that to be transformative, power must be shared with health service users. To do this entails building new relationships and fostering a new culture in health-care institutions that is supportive of participatory approaches” [ 42 , p 379].

At the meso level, several practices could be used to share power. Greenhalgh, Jackson, Shaw and Janamian [ 30 ] identified the importance of equitable decision-making practices and “evenly distributed power constellations.” This goal can be achieved, for instance, by ensuring that service users represent a majority on the project management committee or in codesign events with the goal of challenging dominant professional structures and discourses [ 37 ]. Other scholars called for clear roles and responsibilities [ 38 , 59 , 65 ]. Mulvale, Moll, Miatello, Robert, Larkin, Palmer, Powell, Gable and Girling [ 36 ] recommended the establishment of shared roles and responsibilities, the creation of a representative expert panel to resolve stalemates, and possibly the implementation of formal agreements regarding data and reporting. Importantly, however, Greenhalgh, Jackson, Shaw and Janamian [ 30 ] noted that governance structures and processes alone do not automatically overcome the subtle and inconspicuous uses of power. Farr [ 64 ] recommended the constant practice of critical reflection and dialogue and posed several questions for participants to consider: who is involved, what the interactions are like, how coproduction efforts are implemented within and across structures, and what changes are made.

Although sharing power has been described as an essential component in coproduction, the involvement of stakeholders does not necessarily entail empowerment [ 47 ], and case studies have demonstrated that service improvement initiatives that involve citizens or service users can be instrumental and effective with regard to improving services without enhancing or sharing power or political consciousness if stakeholders are invited but power is not shared [ 32 ]. Farr [ 64 ] noted that rather than coproduction being inherently emancipatory, coproduction and codesign processes can have either dominating or emancipatory effects [ 33 ], and the exclusion of vulnerable groups from coproduction has the potential to reinforce existing inequities [ 75 ].

Training and development for emerging leadership

The importance of appropriate training and mutual learning was noted in several papers [ 36 , 42 , 48 , 63 , 69 , 76 , 77 ]. Implicitly, training for professionals was framed in terms of training in the process of sharing power with service users or facilitating collaboration, whereas training for service users was framed as capacity-building in terms of collaboration and/or leadership. In one case study focusing on coproduced research, participants rejected the notion of “training” from academic researchers with the aim of avoiding suggesting that a certain level of “expertise” needed to be transferred [ 60 ].

Playing a leadership role can be empowering [ 51 , 71 ], but for some individuals, it can be overwhelming [ 71 ]. Leading coproduction requires practice and the development of skills and capacities [ 26 , 48 ]. In some initiatives, lay partners were initially involved in limited roles and gradually took on more responsible leadership tasks over time [ 28 , 42 , 78 ]. In addition, community members’ level of involvement was flexible—they could be participants or take on additional roles as volunteers, paid staff members or directors of organizations. This flexibility offered participants the opportunity to "begin sharing, as opposed to shouldering, the burden of involvement” [ 71 ].

The provision of support

Support is necessary throughout the coproduction process from its outset to the stages of implementation and sustainment [ 25 , 34 , 68 ]. Key dimensions of support include facilitating, advocating for, and championing coproduction. Project management is instrumental to the smooth operation and facilitation of coproduction [ 34 , 35 , 37 , 44 ]. Several facilitation activities are conducted by project leaders and facilitators [ 41 , 42 , 59 , 61 , 78 ]. These activities include holding onto a vision and keeping it alive for the team, ensuring that the project remains on track, and helping maintain momentum. In one codesign case study, facilitators helped people focus on quick wins with the goal of maintaining motivation and engagement; they "needed to support movement from inaction to action, by sifting through group ideas to fix a plan" [ 34 ]. Although these authors acknowledged that this approach may have limited coproduction, they argued that such initiatives would not be sustainable if they were perceived to be “unfeasible.”

Another key function entails advocating for and championing coproduction initiatives to ensure that the process remains ongoing [ 25 , 28 , 29 , 30 , 31 , 32 , 37 , 41 , 74 , 79 ]. Senior leaders play an important role in the task of championing coproduction, and their support has often been described as an important success factor [ 34 , 38 , 39 , 43 , 80 ]. However, effective champions could equally include health care professionals [ 37 ], experts in coproduction [ 51 ], researchers [ 35 , 60 , 61 ], volunteers [ 51 ] or other citizens [ 41 , 61 ]. Champions with lived experience can gain the confidence of their peers and help create understanding among service providers [ 28 , 36 ].

Establishing trusting relationships

Coproduction is essentially relational and requires concerted efforts to establish trusting relationships and a sense of commitment. The importance of trust among stakeholders in coproduction has been noted in several papers [ 28 , 30 , 36 , 37 , 38 , 46 , 48 , 64 , 74 , 81 , 82 ]. In the field of health research, it is difficult to secure funding for the process of establishing relationships and working in the context of partnerships during the early stages of development [ 25 ]. It can therefore be helpful to base recruitment for coproduction initiatives on pre-existing trusting relationships [ 36 ]. If such pre-existing trusting relationships do not exist, policy-makers and senior leaders play a role in the creation of frameworks that can facilitate the development of trust both among organizations and between organizations and citizens, such as political and bureaucratic commitment on the part of regional and local governments and the engagement of actors who play a “boundary-spanning” role in the relationships between service providers, non-government organizations and communities [ 38 ]. Trust is established based on clear responsibilities [ 38 ] and adherence to the principles of engagement in coproduction. In addition to these frameworks, individual leaders must develop trust through interactions with coproducers, using collaborative skills such as those pertaining to communication and listening [ 48 ]. In one case study, through the frank sharing of the organizational, financial, and governance challenges and opportunities faced by stakeholders, people reached a growing understanding and appreciation of each other’s positions, which engendered trust [ 30 ]. Mulvale, Moll, Miatello, Robert, Larkin, Palmer, Powell, Gable [ 36 ] highlighted the importance of understanding and responding to participants’ histories, contexts, and cultural differences.

Commitment can be viewed as more important than resources [ 59 ]. The commitment to and engagement in coproduction exhibited by an organization’s senior leaders demonstrate organizational commitment and lend credibility to coproduction initiatives [ 25 , 34 , 38 , 41 , 47 , 59 , 80 , 83 ]. On some occasions, coproduction initiatives are reported to senior leaders, while on other occasions, the senior leaders were part of the coproduction team. Senior leaders who adopt a more hands-on approach serve as role models [ 25 ], advocating for patient engagement and engendering commitment on the part of staff and patients [ 28 ]. In public health initiatives, buy-in from community leaders confers legitimacy on innovations, helps ensure community trust [ 61 , 78 ], increases the engagement of community members [ 78 ] and is key to a project’s success [ 83 ].

Communication

Communication is a key activity in coproduction, and leaders must establish an environment that is conducive to “epistemological tolerance” [ 47 ], such that different perspectives are valued and appreciated. Such environments facilitate dialogue among partners [ 28 , 30 , 35 , 51 ] and allow critical voices to be heard [ 42 ] . Open dialogue among stakeholders is a starting point for the task of identifying the sources of assumptions and stereotypes, which is itself a prerequisite for change in attitudes and practice [ 28 ]. Project leaders must also facilitate accessible and transparent dialogue and ensure the equal representation of all stakeholders, including those who are less able to communicate verbally [ 57 , 71 ]. Professional leaders are responsible for critically reviewing their professional norms, organizational/institutional processes and past and present policies and practices [ 55 , 75 ].

Dealing with multiple stakeholders, which is inevitably required in coproduction, requires addressing multiple perspectives in an attempt to bring them together. This task frequently involves a degree of conflict and peace negotiation [ 30 , 34 , 41 , 48 , 61 , 64 ]. Leaders should be alert to conflict and power dynamics [ 34 , 36 ]. It may be necessary for meeting chairs to encourage participants to move on from their familiar, entrenched positions to avoid descending into circular arguments and stalemates (Chisholm et al. 2018). This task could require the injection of a critical voice, as Greenhalgh explained:

“Meeting chairs were selected for their leadership qualities, ability to identify and rise above “groupthink” (bland consensus was explicitly discouraged), and commitment to ensuring that potential challenges to new ideas were identified and vigorously discussed. They set an important ethos of constructive criticism and creative innovation, with the patient experience as the central focus. They recognized that if properly handled, conflict was not merely healthy and constructive, but an essential process in achieving successful change in a complex adaptive system. ” [ 30 ]

Leaders must acknowledge the facts that discomfort can arise when more equitable relationships are established [ 61 ] and that challenges to professional identity [ 84 ] and the loss of control [ 72 ] are factors in this process.

Networking refers to the practice of establishing and maintaining relationships with various stakeholders both within and outside the coproduction initiative. Since coproduction involves working with different stakeholders in networks, several papers have discussed the vital mediating processes associated with this context.

“Bridging, brokering and boundary spanning roles have a key role in cross fertilization of ideas between groups, for generating new ideas and for increasing understanding and cooperation” [ 32 , 53 ].

In policy-making, it is helpful to develop coordination structures and processes such as cross-sector working groups and committees, intersector communication channels [ 65 ], and relationship and dialogue structures [ 42 ]. Community representatives can play a mediating role between individuals and public organizations and may alleviate professionals’ concerns regarding the transaction costs of coproduction in the planning and management of services [ 26 , 81 ]. However, these representatives may or may not use this power to amplify the voices of individual coproducers [ 81 ].

An important role of project leaders is that of the “broker” [ 32 , 85 ], who focuses on mediating among different stakeholders in an attempt to align their perspectives [ 26 ,  37 ,  72 , 86 ]. Another role focuses on spanning the boundaries across sites [ 50 ], between local service providers [ 68 ], or among local services, non-government organizations and the community [ 38 ]. Bovaird, drawing on a number of cases of coproduction, came to the following conclusion:

“ there is a need for a new type of public service professional: the coproduction development officer, who can help to overcome the reluctance of many professionals to share power with users and their communities and who can act internally in organizations (and partnerships) to broker new roles for coproduction between traditional service professionals, service managers, and the political decision-makers who shape the strategic direction of the service system.” [ 81 ]

Orchestration

This practice involves reflecting on and improving coproduction itself. It includes activities such as evaluating the effectiveness of coproduction efforts, assessing the impact of coproduction on outcomes, and making adjustments to improve the coproduction process. Several papers have addressed the roles of local government or public managers or health professionals in overseeing and (as we refer to this process) ‘orchestrating’ the networks involved in coproduction at the community or local government level [ 30 , 33 , 65 , 74 , 87 ]. Orchestration involves recruiting the appropriate actors as noted above as well as directing and coordinating activities, thereby ensuring that the whole is more than the sum of its parts. As part of their orchestration work, leaders play a role in the task of managing risk in service innovation [ 55 , 87 ] and must commit to self-reflexivity and a critical review of norms, policies and practices to alert themselves to any unintended negative consequences and strive to counteract them [ 55 ]. Sturmberg, Martin and O’Halloran [ 88 ] used the metaphor of ‘conducting’ to describe the function of leadership in health care – i.e., leading the orchestra through inspiration and empowerment rather than control, leading to the provision of feedback as the performance unfolds.

From a public service perspective, Powers and Thompson [ 69 ] argued that coproduction requires the leader (“usually a public official”) to mobilize the community on behalf of the public good, organize the provision of the good, create incentives, and supervise the enforcement of community norms. Sancino [ 74 ] argued that local governments play a ‘meta-coproduction role’ that requires them to maximize the coproduction and peer-production of community outcomes by taking into account community contributions and deciding which services should be commissioned or decommissioned (a point that was also made by Wilson [ 87 ]) and to promote coproduction and peer-production in such a way as to promote the coproduction of outcomes that have been decided through a democratic process. In this way, he argued,

"the local government becomes the pivot of different kinds of relationships and networks made up of different actors who collectively assume the responsibility for implementing an overall strategic plan of the community beyond their specific roles and interests." [ 74 ]

Sancino [ 74 ] attempted to draw out the leadership implications of this situation, arguing that rather than focusing on service delivery, public managers must create appropriate conditions for such meta-coproduction. This task entails a directing role based on framing shared scenarios for change in the community through sense-making; an activator role based on activating, mobilizing and consolidating the social capital of the community to promote diffused public leadership; a convenor role based on serving as a meta-manager in the process of self-organizing the knowledge, resources and competencies pertaining to the community in question; and an empowering role based on creating conditions in which peer production and coproduction can be combined to create the corresponding added value (i.e., higher levels of community outcomes) [ 74 ]. This practice essentially focuses on self-assessment and continuous improvement within the coproduction framework.

Implementation

It has been argued that coproduction in services [ 30 , 79 ] or policy-making [ 65 ] may improve implementation. The role of leadership in supporting the implementation of the outcomes of coproduction is essential [ 37 , 41 , 49 , 52 , 64 , 65 , 85 , 86 ]. Leaders can argue for the legitimacy of coproduced innovations [ 89 ] and implement mechanisms aimed at acting on the issues thus raised and continuing to promote patients’ involvement [ 28 , 41 ]. Implementing the outcomes of coproduction relies on outcome-focused leadership [ 30 ]. The results of coproduction initiatives must be transformed into strategic plans and policies [ 41 ], and patient perspectives must be translated into actionable quality improvement initiatives [ 49 ]. Conversely, implementation can be blocked by leaders who fail to respond to the results of coproduction initiatives or who implement policies or procedures that are poorly aligned with the recommendations arising from coproduction [ 30 , 41 ]. It should also be acknowledged that not all demands thus generated can always be met [ 61 ]. Failures of implementation run the risk of stakeholder disillusionment; thus, the management of expectations is important.

A framework for coproduction leadership

When coproduction is initiated, it is possible to consider the actors involved and to imagine various forms of coproduction. In the design process, it is possible to make a deliberate choice with regard to the most appropriate model of leadership, and depending on the leadership model selected (leader-centric, coleadership, or collective leadership), different leadership practices emerge. The nine leadership practices identified can be enacted by different people and in different ways. The leadership of coproduction that thus emerges is shaped by issues such as the model of coproduction, the stakeholders involved, participants’ motivations and the context of coproduction. A main concern lies in the need to design project structures and work practices that are aligned and that enable leadership to emerge. We thus created a table (Table  1 ) that illustrates potential reflective questions in this context.

This discussion highlights and problematizes the two main findings of this systematic review, namely, the need to deliberately consider underlying models of leadership and the complex character of leading coproduction.

The need for the deliberate use of leadership in the plural

A focus on leader-centric approaches and the quality of leaders has characterized public leadership research [ 90 ]. Such a focus is echoed in our findings on coproduction leadership, first with regard to the prominence of senior leaders and, to a lesser extent, facilitators. Politicians were rarely identified in the papers included in our review despite representing some of the main actors identified in a previous review [ 4 , 91 ]. Second, many papers referenced the need for “strong” leaders, and the skills and behaviours of individual leaders were noted. As other researchers have found, despite the focus of this field on relationships and interactions, its emphasis has frequently remained on the individual leader and their ability to engage and inspire followers [ 13 ]. Furthermore, even in papers that emphasized ‘coleadership’ or ‘collective leadership,’ the focus remained on public managers, service managers and facilitators. Very little evidence has been reported concerning individual service users’ or citizens’ leadership of (as distinct from involvement in ) coproduction. Although the involvement of community leaders was reported to play a role in project success, no articles explored this issue.

However, some important exceptions should be noted. For example, some studies exhibited a preference for mixed models, employing both a directive approach (particularly in the beginning) and a more facilitative and distributed leadership approach [ 56 ]. Rycroft-Malone, Burton, Wilkinson, Harvey, McCormack, Baker, Dopson, Graham, Staniszewska and Thompson [ 53 ] concluded that consideration should be given to models that combine hierarchical, directive structures with distributed facilitative forms of leadership.

One explanation for this rather narrow view of leadership is that despite the rapidly increasing number of publications in the general field of coproduction [ 7 , 18 ], empirical studies have still lacked depth with regard to investigations of the leadership of this process. Most empirical studies included in this review mentioned leadership only in passing or derived some conclusions regarding leadership from case studies focusing on other aspects of the coproduction process.

Another explanation for this situation is that although coproduction focuses on partnership, in most cases, senior leaders have control over resources and the power to define the means, methods, extent and forms of participation [ 65 ]. Even shared leadership models seem to rely on traditional leaders’ willingness to share power [ 10 ], as leaders are the actors who invite, facilitate, and support the participation of coleaders. However, some signs of change towards a broader view should be noted. Recent publications have theorized the leadership of coproduction and included case studies that have demonstrated leadership to be a social, collective and relational phenomenon that emerges as a property of interactions among individuals in given contexts [ 13 , 19 ].

The complexity of coproduction leadership practices

Our findings indicate that the leadership of coproduction practices entails challenging and complex tasks. Complexity emerges in cases in which many parts are interrelated in multiple ways. Different kinds of leadership activities may be necessary depending on the stakeholders involved [ 92 ], the context [ 13 ], and the mode, level, and phase of coproduction [ 93 ]. A complexity perspective based on systems thinking is therefore useful [ 13 , 19 ]. All actors involved in coproduction are potential leaders, but for that potential to be realized, the coproduction initiative and its leadership must be framed and comprehended in a more plural way. A recent study on systems thinking and complex adaptive thinking as means of initiating coproduction advocated a collective leadership approach [ 19 ].

Our findings highlight the need for a complex way of making meaning of leadership throughout the coproduction process, such as the ability to be flexible due to circumstances and employ both strong leadership and more facilitative approaches when necessary. Leaders must also promote the values of democracy, transparency and the redistribution of power among stakeholders throughout the process [ 64 , 94 ]. These practices and tasks are complex, which must be matched by an inner mental complexity [ 95 , 96 ]. Several practices identified in this research, such as genuinely valuing diverse perspectives, promoting mutual transformative power sharing and welcoming conflicts, require a complex mode of meaning-making that results from psychological development. These issues warrant further exploration. Future studies featuring a thoughtful choice of leadership and complexity models as well as a broader methodological repertoire are thus necessary (see Table  2 for an overview).

Methodological strengths and limitations

A strength of this review lies in its integration of research on the sparse and overlooked issue of leadership in coproduction. Our search strategy, which involved using the key words manag* and lead*, may have excluded some relevant papers. To verify that this approach did not represent an excessively blunt exclusion criterion, we checked 10% of the articles that were excluded based on this criterion. All of these articles would also have been excluded for failing to include any exploration of the management or leadership of coproduction. We therefore determined that this exclusion criterion was justifiable. Many papers did not have an explicit focus on leadership; however, by synthesizing the data, all data were treated as reflections that jointly created a larger pattern, similar to a kaleidoscope. The exclusion of non-peer-reviewed papers is likely to have led to the exclusion of coproduced outputs, which may have offered important insights into the leadership of coproduction, particularly with regard to the experiences of service users and citizens playing leadership roles. In the reporting of this review, the PRISMA guidelines were followed (Additional file 2 ). It should be noted that the lack of reporting bias assessment and certainty assessment represents a limitation of this study.

Future research

Future research (see Table  2 ) should focus on under-represented roles, such as those of politicians and community leaders, and explore emerging collective leadership models based on real-time observational studies. It should also investigate the balance between strong and shared leadership by using qualitative and participatory research methods. Incorporating systems thinking and relevant leadership models can offer new perspectives on collective leadership practices.

Practical implications

This paper explored coproduction leadership practices and revealed that they require a deliberate and plural understanding of leadership roles and tasks. We proposed a framework for coproduction leadership that takes into account the actors involved, the models of leadership, and the leadership practices that emerge in different contexts and during different phases of coproduction. We also provided a set of reflective considerations that can help all actors involved in this process make more deliberate choices regarding the parties involved, leadership models of coproduction, and practices (Table  1 ).

Our systematic review revealed some gaps in the literature on coproduction leadership, such as the lack of attention to the mental complexity of coproduction leaders, the under-representation of service users and citizens as leaders, and the need for more empirical studies that use appropriate models and methods to capture the complexity of coproduction leadership. We suggest that future research should address these gaps, thus contributing to the advancement of coproduction theory and practice.

Our framework also has some practical implications for coproduction leaders and participants. At the start of coproduction process, all people, particularly leaders, must learn more about different models of leadership and how power is shared. Throughout this process, flexibility is necessary because leadership constellations change over time; they emerge and fade away, thus implying different underlying leadership models. A multitude of practices must be implemented throughout the coproduction process. People in leader roles must be aware of their personal strengths and limitations, not only with the goal of sharing leadership but also with the aim of establishing partnerships with others who have competence in certain practices, such as facilitation or addressing conflicts. Reflecting upon the guiding questions can also help illustrate the extent to which power and leadership are being shared. In conclusion, to create more equal power relations over time, we must challenge our current practices and work deliberately to enhance the capacity of individuals and groups to effectively engage in coproduction leadership.

Osborne SP, Radnor Z, Strokosch K. Co-production and the co-creation of value in public services: a suitable case for treatment? Public Manag Rev. 2016;18(5):639–53.

Article   Google Scholar  

Voorberg WH, Bekkers VJJM, Tummers LG. A Systematic Review of Co-Creation and Co-Production: Embarking on the social innovation journey. Public Manag Rev. 2015;17(9):1333–57.

Gassner D, Gofen A. Coproduction investments: Street-level management perspective on coproduction. Cogent Business Management. 2019;6(1):1617023.

Bussu S, Tullia Galanti M. Facilitating coproduction: the role of leadership in coproduction initiatives in the UK. Policy Society. 2018;37(3):347–67.

Brown PR, Head BW. Navigating tensions in co-production: A missing link in leadership for public value. Public Administration (London). 2019;97(2):250–63.

Schlappa H, Imani Y: Who is in the lead? New perspectives on leading service co-production. In: Co-production and co-creation. edn.: Routledge; 2018: 99–108.

Masterson D. Mapping definitions of co-production and co-design in health and social care:systematic scoping review providing lessons for the future. Health Expect. 2022;25(3):902–13.

Article   PubMed   PubMed Central   Google Scholar  

Osborne SP, Strokosch K. It takes Two to Tango? Understanding the C o-production of Public Services by Integrating the Services Management and Public Administration Perspectives. Br J Manag. 2013;24:S31–47.

Parry KW, Bryman A: Leadership in organizations. In: The SAGE Handbook of Organization Studies. edn. Edited by Clegg SR, Hardy C, Lawrence TB, Nord WR. London: SAGE; 2006: 446–468.

Denis J-L, Langley A, Sergi V. Leadership in the plural. Acad Manag Ann. 2012;6(1):211–83.

Drath WH, McCauley CD, Palus CJ, Van Velsor E, O’Connor PMG, McGuire JB. Direction, alignment, commitment: Toward a more integrative ontology of leadership. Leadersh Q. 2008;19(6):635–53.

Ospina SM, Foldy EG, Fairhurst GT, Jackson B. Collective dimensions of leadership: Connecting theory and method. Human Relations. 2020;73(4):441–63.

Schlappa H, Imani Y, Nishino T. Relational leadership: an analytical lens for the exploration of co-production. In: Loeffler E, Bovaird T, editors. The Palgrave Handbook of Co-Production of Public Services and Outcomes. edn.: Springer; 2021. p. 471–90.

Loeffler E. Co-production of public services and outcomes. 1st ed. In: The Palgrave Handbook of Co-Production of Public Services and Outcomes. Cham, Switzerland: Palgrave Macmillan; 2021. p. 387–408. https://link.springer.com/book/10.1007/978-3-030-55509-2 .

Kotter JP. Force for change: how leadership differs from management: Simon and Schuster; 2008.

Palumbo R. Contextualizing co-production of health care: a systematic literature review. Intern J Public Sector Management. 2016;29(1):72–90.

Palumbo R, Manesh MF: Travelling along the public service co-production road: a bibliometric analysis and interpretive review. Public management review 2021:1–37.

Fusco F, Marsilio M, Guglielmetti C. Co-production in health policy and management: a comprehensive bibliometric review. BMC Health Serv Res. 2020;20(1):504–504.

Docherty K: Exploring Collective Leadership and Co-Production: An Empirical Study. In: Processual Perspectives on the Co-Production Turn in Public Sector Organizations. edn.: IGI Global; 2021: 130–155.

Turnhout E, Metze T, Wyborn C, Klenk N, Louder E. The politics of co-production: participation, power, and transformation. Current Opinion Environmental Sustainability. 2020;42:15–21.

Kjellström S, Areskoug-Josefsson K, Andersson Gäre B, Andersson A-C, Ockander M, Käll J, McGrath J, Donetto S, Robert G. Exploring, measuring and enhancing the coproduction of health and well-being at the national, regional and local levels through comparative case studies in Sweden and England: the “Samskapa” research programme protocol. BMJ Open. 2019;9(7):e029723–e029723.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108.

Article   PubMed   Google Scholar  

Denyer D, Tranfield D: Producing a Systematic Review. In: The Sage handbook of organizational research methods. edn. Edited by Buchanan D, Bryman A: Sage Publications Ltd; 2009.

Sicilia M, Sancino A, Nabatchi T, Guarini E. Facilitating co-production in public services: management implications from a systematic literature review. Public Money Management. 2019;39(4):233–40.

Staniszewska S, Denegri S, Matthews R, Minogue V. Reviewing progress in public involvement in NIHR research: developing and implementing a new vision for the future. BMJ Open. 2018;8(7):e017124.

Byrne L, Stratford A, Davidson L. The global need for lived experience leadership. Psychiatr Rehabil J. 2018;41(1):76.

Nicol E, Sang B. A co-productive health leadership model to support the liberation of the NHS. J R Soc Med. 2011;104(2):64–8.

Hafford-Letchfield T, Simpson P, Willis PB, Almack K. Developing inclusive residential care for older lesbian, gay, bisexual and trans (LGBT) people: An evaluation of the Care Home Challenge action research project. Health Soc Care Community. 2018;26(2):e312–20.

Hopkins L, Foster A, Nikitin L. The process of establishing Discovery College in Melbourne. Ment Health Soc Incl. 2018;22(4):187–94.

Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94(2):392–429.

Green S, Beveridge E, Evans L, Trite J, Jayacodi S, Evered R, Parker C, Polledri L, Tabb E, Green J. Implementing guidelines on physical health in the acute mental health setting: a quality improvement approach. Int J Ment Heal Syst. 2018;12(1):1–9.

Poocharoen O-o. Ting B: Collaboration, co-production, networks: Convergence of theories. Public Manag Rev. 2015;17(4):587–614.

Topp SM, Sharma A, Chileshe C, Magwende G, Henostroza G, Moonga CN. The health system accountability impact of prison health committees in Zambia. International journal for equity in health. 2018;17(1):74–74.

Chisholm L, Holttum S, Springham N. Processes in an experience-based co-design project with family carers in community mental health. SAGE Open. 2018;8(4):2158244018809220.

Mader LB, Harris T, Kläger S, Wilkinson IB, Hiemstra TF. Inverting the patient involvement paradigm: defining patient led research. Res Involv Engagem. 2018;4(1):1–7.

Mulvale G, Moll S, Miatello A, Robert G, Larkin M, Palmer VJ, Powell A, Gable C, Girling M. Codesigning health and other public services with vulnerable and disadvantaged populations: Insights from an international collaboration. Health Expect. 2019;22(3):284–97.

Larkin M, Boden ZV, Newton E. On the brink of genuinely collaborative care: experience-based co-design in mental health. Qual Health Res. 2015;25(11):1463–76.

Farooqi SA. Co-production: what makes co-production work? Evidence from Pakistan. Int J Public Sect Manag. 2016;29(4):381–95.

Murphy L, Wells JS, Lachman P, Bergin M. A quality improvement initiative in community mental health in the Republic of Ireland. Health Sci J. 2015;9(1):1.

Google Scholar  

Redwood S, Brangan E, Leach V, Horwood J, Donovan JL. Integration of research and practice to improve public health and healthcare delivery through a collaborative’Health Integration Team’model-a qualitative investigation. BMC Health Serv Res. 2016;16(1):1–13.

Bombard Y, Baker GR, Orlando E, Fancott C, Bhatia P, Casalino S, Onate K, Denis J-L, Pomey M-P. Engaging patients to improve quality of care: a systematic review. Implement Sci. 2018;13(1):1–22.

Marston C, Hinton R, Kean S, Baral S, Ahuja A, Costello A, Portela A. Community participation for transformative action on women’s, children’s and adolescents’ health. Bull World Health Organ. 2016;94(5):376–82.

Burhouse A, Lea C, Ray S, Bailey H, Davies R, Harding H, Howard R, Jordan S, Menzies N, White S. Preventing cerebral palsy in preterm labour: a multiorganisational quality improvement approach to the adoption and spread of magnesium sulphate for neuroprotection. BMJ open quality. 2017;6(2):e000189.

Burhouse A, Rowland M, Niman HM, Abraham D, Collins E, Matthews H, Denney J, Ryland H. Coaching for recovery: a quality improvement project in mental healthcare. BMJ Open Quality. 2015;4(1):u206576–w202641.

Millenson ML, DiGioia AM III, Greenhouse PK, Swieskowski D. Turning patient-centeredness from ideal to real: lessons from 2 success stories. J Ambul Care Manage. 2013;36(4):319–34.

Yokota F, Biyani M, Islam R, Ahmed A, Nishikitani M, Kikuchi K, Nohara Y, Nakashima N. Lessons learned from Co-Design and Co-Production in a portable health clinic research project in Jaipur district, India (2016–2018). Sustainability. 2018;10(11):4148.

Cooke J, Langley J, Wolstenholme D, Hampshaw S. Seeing" the difference: the importance of visibility and action as a mark of" authenticity" in co-production: comment on" collaboration and co-production of knowledge in healthcare: opportunities and challenges. Int J Health Policy Manag. 2017;6(6):345.

Sicilia M, Guarini E, Sancino A, Andreani M, Ruffini R. Public services management and co-production in multi-level governance settings. Int Rev Adm Sci. 2016;82(1):8–27.

Bak K, Moody L, Wheeler SM, Gilbert J. Patient and Staff Engagement in Health System Improvement: A Qualitative Evaluation of the Experience-Based Co-design Approach in Canada. Healthcare Quarterly (Toronto, Ont). 2018;21(2):24–9.

Bagot KL, Cadilhac DA, Kim J, Vu M, Savage M, Bolitho L, Howlett G, Rabl J, Dewey HM, Hand PJ. Transitioning from a single-site pilot project to a state-wide regional telehealth service: The experience from the Victorian Stroke Telemedicine programme. J Telemed Telecare. 2017;23(10):850–5.

Bruce G, Wistow G, Kramer R. Connected Care Re-visited: Hartlepool and Beyond. J Integr Care. 2011;19(2):13–21.

Hogan MJ, Johnston H, Broome B, McMoreland C, Walsh J, Smale B, Duggan J, Andriessen J, Leyden KM, Domegan C. Consulting with citizens in the design of wellbeing measures and policies: lessons from a systems science application. Soc Indic Res. 2015;123(3):857–77.

Rycroft-Malone J, Burton CR, Wilkinson J, Harvey G, McCormack B, Baker R, Dopson S, Graham ID, Staniszewska S, Thompson C. Collective action for implementation: a realist evaluation of organisational collaboration in healthcare. Implement Sci. 2015;11(1):1–17.

Sorrentino M, Guglielmetti C, Gilardi S, Marsilio M. Health care services and the coproduction puzzle: filling in the blanks. Administration Society. 2017;49(10):1424–49.

Williams BN, Kang S-C, Johnson J. (Co)-contamination as the dark side of co-production: Public value failures in co-production processes. Public Manag Rev. 2016;18(5):692–717.

Gillard S, Foster R, Turner K. Evaluating the Prosper peer-led peer support network: a participatory, coproduced evaluation. Ment Health Soc Incl. 2016;20(2):80–92.

Greenwood DA, Litchman ML, Ng AH, Gee PM, Young HM, Ferrer M, Ferrer J, Memering CE, Eichorst B, Scibilia R. Development of the intercultural diabetes online community research council: codesign and social media processes. J Diabetes Sci Technol. 2019;13(2):176–86.

Lindsay C, Pearson S, Batty E, Cullen AM, Eadson W. Co-production as a route to employability: Lessons from services with lone parents. Public Administration. 2018;96(2):318–32.

Bell T, Vat LE, McGavin C, Keller M, Getchell L, Rychtera A, Fernandez N. Co-building a patient-oriented research curriculum in Canada. Res Involv Engagem. 2019;5(1):1–13.

Cox N, Clayson A, Webb L. A safe place to reflect on the meaning of recovery: a recovery community co-productive approach using multimedia interviewing technology. Drugs Alcohol Today. 2016;16(1):4–15.

Haynes E, Marawili M, Marika BM, Mitchell AG, Phillips J, Bessarab D, Walker R, Cook J, Ralph AP. Community-based participatory action research on rheumatic heart disease in an Australian Aboriginal homeland: Evaluation of the ‘On track watch’project. Eval Program Plann. 2019;74:38–53.

Ward ME, De Brún A, Beirne D, Conway C, Cunningham U, English A, Fitzsimons J, Furlong E, Kane Y, Kelly A. Using co-design to develop a collective leadership intervention for healthcare teams to improve safety culture. Int J Environ Res Public Health. 2018;15(6):1182.

McGregor J, Repper J, Brown H. “The college is so different from anything I have done”. A study of the characteristics of Nottingham Recovery College. J Mental Health Training, Education Practice. 2014;9(1):3–15.

Farr M. Power dynamics and collaborative mechanisms in co-production and co-design processes. Crit Soc Policy. 2018;38(4):623–44.

Hämäläinen R-M, Aro AR, Lau CJ, Rus D, Cori L, Syed AM. Cross-sector cooperation in health-enhancing physical activity policymaking: more potential than achievements? Health Res Policy Syst. 2016;14(1):33–33.

Farmer J, Taylor J, Stewart E, Kenny A. Citizen participation in health services co-production: a roadmap for navigating participation types and outcomes. Aust J Prim Health. 2017;23(6):509–15.

Morton M, Paice E. Co-Production at the Strategic Level: Co-Designing an Integrated Care System with Lay Partners in North West London. England Int J Integr Care. 2016;16(2):2–2.

Jeffs L, Merkley J, Sinno M, Thomson N, Peladeau N, Richardson S. Engaging Stakeholders to Co-design an Academic Practice Strategic Plan in an Integrated Health System: The Key Roles of the Nurse Executive and Planning Team. Nurs Adm Q. 2019;43(2):186–92.

Powers KJ, Thompson F. Managing Coprovision: Using Expectancy Theory to Overcome the Free-Rider Problem. J Public Adm Res Theory. 1994;4(2):179–96.

Andrews R, Brewer GA. Social capital, management capacity and public service performance: Evidence from the US states. Public Manag Rev. 2013;15(1):19–42.

Budge G, Mitchell A, Rampling T, Down P, Collective B. “It kind of fosters a culture of interdependence”: A participatory appraisal study exploring participants’ experiences of the democratic processes of a peer-led organisation. J Community Applied Social Psychology. 2019;29(3):178–92.

Farmer J, Currie M, Kenny A, Munoz S-A. An exploration of the longer-term impacts of community participation in rural health services design. Soc Sci Med. 2015;141:64–71.

Ersoy A. The spread of coproduction: How the concept reached the northernmost city in the UK. Local Econ. 2016;31(3):410–23.

Article   MathSciNet   Google Scholar  

Sancino A. The meta co-production of community outcomes: Towards a citizens’ capabilities approach. International J Voluntary Nonprofit Organizations. 2016;27(1):409–24.

Nies H. Communities as co-producers in integrated care. Int J Integr Care. 2014;(14):1–4.

Terp M, Laursen BS, Jørgensen R, Mainz J, Bjørnes CD. A room for design: Through participatory design young adults with schizophrenia become strong collaborators: A Room for Design. Int J Ment Health Nurs. 2016;25(6):496–506.

Seid M, Dellal G, Peterson LE, Provost L, Gloor PA, Fore DL, Margolis PA. Co-designing a Collaborative Chronic Care Network (C3N) for inflammatory bowel disease: development of methods. JMIR Hum Factors. 2018;5(1):e8083.

Schaaf M, Topp SM, Ngulube M. From favours to entitlements: community voice and action and health service quality in Zambia. Health Policy Plan. 2017;32(6):847–59.

Oertzen A-S, Odekerken-Schröder G, Brax SA, Mager B. Co-creating services—conceptual clarification, forms and outcomes. J Serv Manag. 2018;29(4):641–79.

Nimegeer A, Farmer J, West C, Currie M. Addressing the problem of rural community engagement in healthcare service design. Health Place. 2011;17(4):1004–6.

Bovaird T. Beyond engagement and participation: User and community coproduction of public services. Public Adm Rev. 2007;67(5):846–60.

Macaulay B. Considering social enterprise involvement in the commissioning of health services in Shetland. Local Econ. 2016;31(5):650–9.

Miller A, Young EL, Tye V, Cody R, Muscat M, Saunders V, Smith ML, Judd JA, Speare R. A community-directed integrated Strongyloides control program in Queensland, Australia. Trop Med Infect Dis. 2018;3(2):48.

Walsh M, Kittler MG, Mahal D. Towards a new paradigm of healthcare: Addressing challenges to professional identities through Community Operational Research. Eur J Oper Res. 2018;268(3):1125–33.

Windrum P. Third sector organizations and the co-production of health innovations. Manag Decis. 2014;5(6):1046–56.

Farmer J, Carlisle K, Dickson-Swift V, Teasdale S, Kenny A, Taylor J, Croker F, Marini K, Gussy M. Applying social innovation theory to examine how community co-designed health services develop: using a case study approach and mixed methods. BMC Health Serv Res. 2018;18(1):1–12.

Wilson G. Co-Production and Self-Care: New Approaches to Managing Community Care Services for Older People1. Social Policy Administration. 1994;28(3):236–50.

Sturmberg JP, Martin CM, O’Halloran D. Music in the Park. An integrating metaphor for the emerging primary (health) care system: Music in the Park. J Eval Clin Pract. 2010;16(3):409–14.

Vennik FD, van de Bovenkamp HM, Putters K, Grit KJ. Co-production in healthcare: rhetoric and practice. Int Rev Adm Sci. 2016;82(1):150–68.

Crosby BC, Bryson JM. Why leadership of public leadership research matters: and what to do about it. Public Manag Rev. 2018;20(9):1265–86.

Clarke D, Jones F, Harris R, Robert G. What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings? A rapid evidence synthesis. BMJ Open. 2017;7(7):e014650.

Gallan AS, Jarvis CB, Brown SW, Bitner MJ. Customer positivity and participation in services: an empirical test in a health care context. J Acad Mark Sci. 2013;41(3):338–56.

Nabatchi T, Sancino A, Sicilia M. Varieties of Participation in Public Services: The Who, When, and What of Coproduction. Public Adm Rev. 2017;77(5):766–76.

Larsen T, Karlsen JE, Sagvaag H: Keys to unlocking service provider engagement in constrained co-production partnerships. Action Research 2020:1476750320925862.

Kegan R, Laskow Lahey L. How the way we talk can change the way we work : seven languages for transformation. San Francisco: Jossey-Bass Pfeiffer; Wiley; 2001.

Hill R. Thinking like a round table leader: how mental complexity enables leaders to succeed in a complex environment. Journal of Character and Leadership Development. 2021;8(1):116–30.

Download references

Acknowledgements

The authors wish to thank Forte, the Swedish Research Council for Health, Working Life and Welfare. In particular, we would like to thank Mary McCall for valuable help.

Open access funding provided by Jönköping University. The study of Samskapa, a coproduction research programme, received funding from Forte, the Swedish Research Council for Health, Working Life and Welfare, under grant agreement no. 2018–01431.

Author information

Authors and affiliations.

The Jönköping Academy for Improvement of Health and Welfare, School of Health and Welfare, Jönköping University, Barnarpsgatan 39, Jönköping, Sweden

Sofia Kjellström & Daniel Masterson

Florence Nightingale Faculty of Nursing, Midwifery & Palliative Care, King’s College London, London, UK

Sofia Kjellström & Sophie Sarre

You can also search for this author in PubMed   Google Scholar

Contributions

SK and SS performed the data extraction, qualitative synthesis and drafted the manuscript and Table 1 . SK finalized the manuscript. D.M. screened the data from a previous scoping review, provided the search strategy (Additional file 1 : Appendix 1) and constructed the Prisma flowchart. SS compiled sample description in Additional file 2 : Appendix 2. All authors reviewed and approved the manuscript and agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Sofia Kjellström .

Ethics declarations

Ethics approval and consent to participate.

Not applicable; according to Swedish law research, ethical approval is not needed for research that does not involve human persons.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix 1. .

Description of included papers. 

Additional file 2: Appendix 2. 

PRISMA_2020_checklist - Mangement review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Kjellström, S., Sarre, S. & Masterson, D. The complexity of leadership in coproduction practices: a guiding framework based on a systematic literature review. BMC Health Serv Res 24 , 219 (2024). https://doi.org/10.1186/s12913-024-10549-4

Download citation

Received : 11 April 2023

Accepted : 03 January 2024

Published : 17 February 2024

DOI : https://doi.org/10.1186/s12913-024-10549-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Health and welfare

BMC Health Services Research

ISSN: 1472-6963

use of research literature

  • Research article
  • Open access
  • Published: 22 February 2024

Fenfluramine for the treatment of status epilepticus: use in an adult with Lennox–Gastaut syndrome and literature review

  • Adam Strzelczyk   ORCID: orcid.org/0000-0001-6288-9915 1 ,
  • Hendrik Becker 1 ,
  • Lisa Tako 1 ,
  • Susanna Hock 1 ,
  • Elke Hattingen 2 ,
  • Felix Rosenow 1 &
  • Catrin Mann 1  

Neurological Research and Practice volume  6 , Article number:  10 ( 2024 ) Cite this article

Metrics details

Novel treatments are needed to control refractory status epilepticus (SE). This study aimed to assess the potential effectiveness of fenfluramine (FFA) as an acute treatment option for SE. We present a summary of clinical cases where oral FFA was used in SE.

A case of an adult patient with Lennox–Gastaut syndrome (LGS) who was treated with FFA due to refractory SE is presented in detail. To identify studies that evaluated the use of FFA in SE, we performed a systematic literature search.

Four case reports on the acute treatment with FFA of SE in children and adults with Dravet syndrome (DS) and LGS were available. We report in detail a 30-year-old woman with LGS of structural etiology, who presented with generalized tonic and dialeptic seizures manifesting at high frequencies without a return to clinical baseline constituting the diagnosis of SE. Treatment with anti-seizure medications up to lacosamide 600 mg/d, brivaracetam 300 mg/d, valproate 1,600 mg/d, and various benzodiazepines did not resolve the SE. Due to ongoing refractory SE and following an unremarkable echocardiography, treatment was initiated with FFA, with an initial dose of 10 mg/d (0.22 mg/kg body weight [bw]) and fast up-titration to 26 mg/d (0.58 mg/kg bw) within 10 days. Subsequently, the patient experienced a resolution of SE within 4 days, accompanied by a notable improvement in clinical presentation and regaining her mobility, walking with the assistance of physiotherapists. In the three cases reported in the literature, DS patients with SE were treated with FFA, and a cessation of SE was observed within a few days. No treatment-emergent adverse events were observed during FFA treatment in any of the four cases.

Conclusions

Based on the reported cases, FFA might be a promising option for the acute treatment of SE in patients with DS and LGS. Observational data show a decreased SE frequency while on FFA, suggesting a potentially preventive role of FFA in these populations.

We summarize four cases of refractory status epilepticus (SE) successfully treated with fenfluramine.

Refractory SE resolved after 4–7 days on fenfluramine.

Swift fenfluramine up-titration was well-tolerated during SE treatment.

Treatment-emergent adverse events on fenfluramine were not observed.

Fenfluramine might be a valuable acute treatment option for SE in Dravet and Lennox–Gastaut syndromes.

Refractory status epilepticus (SE) is characterized by unresponsiveness to initial therapy with benzodiazepines and anti-seizure medications (ASMs). It occurs in approximately 40–50% of all cases of SE [ 1 ]. Status epilepticus is associated with many fatalities and high morbidity [ 2 , 3 , 4 ], and treatment is difficult in refractory cases due to the failure of first- and second-line therapies. Therapeutic management beyond the use of benzodiazepines and common ASMs is based on clinical reports, expert opinion, and pathophysiological assumptions from experimental data. Patients with developmental and epileptic encephalopathies (DEEs) such as Dravet syndrome (DS) and Lennox–Gastaut syndrome (LGS) frequently experience convulsive and non-convulsive SE [ 5 , 6 ], and their treatment is more challenging because some ASMs may exacerbate seizures and SE, including sodium channel blockers in DS [ 7 , 8 ].

Recently, fenfluramine (FFA) was licensed in Europe and the US as an adjunctive treatment for epileptic seizures in DS and LGS patients over 2 years of age. FFA is derived from amphetamine and has different modes of action than other ASMs. FFA modulates serotonergic neurotransmission and thus produces an overall increase in serotonin levels in the synaptic cleft by increasing central serotonin release and simultaneously reducing presynaptic serotonin reuptake [ 9 , 10 ]. FFA acts directly on some specific serotonin receptors (HTR) including 5-HT1D, 5-HT2A, and 5-HT2C receptors through its main metabolite, norfenfluramine. It additionally seems to positively modulate the sigma-1 receptor, which belongs to a group of chaperone proteins. On activation, the sigma-1 receptor modulates various voltage-gated ion channels (Ca2 +, Na +, and K +) and NMDA receptors, which influence excitatory and inhibitory neurotransmission [ 9 , 10 ]. The efficacy of FFA has been demonstrated in four double-blind, placebo-controlled, randomized phase III studies in DS and LGS patients [ 11 , 12 , 13 , 14 ]. Details on the use of FFA and pharmacological properties are presented in Table  1 [ 15 ].

Given the severity of illness and unfavorable outcomes in patients with refractory SE, therapies with different modes of action are critical to stopping ongoing seizure activity. The objective of this study was to determine the use, efficacy, and tolerability of FFA in patients with refractory SE.

Since the availability of FFA in our department (early 2020 within the Early Access Program), we recorded in detail all patients treated with it. Between April 2020 and November 2023, FFA was started de novo in one adult patient during refractory SE. CARE guidelines for reporting case reports were followed [ 17 ].

To identify further studies evaluating the use of FFA in SE, we performed a systematic literature search in the MEDLINE, Cochrane Central Register of Controlled Trials, and Excerpta Medica databases from 1996 until November 2023, using a combined search strategy including the following keywords: “fenfluramine”, “epilepsy”, “seizure”, “Dravet”, “Lennox–Gastaut”, “epileptic encephalopathy”, and “status epilepticus”. We examined the reference lists of all identified studies and review articles on FFA [ 9 , 18 ] for additional relevant studies and reports.

The classifications of seizure type, epilepsy type and syndrome, and SE were adopted based on the latest definitions proposed by the International League Against Epilepsy (ILAE) [ 19 , 20 , 21 ]. Regarding seizure duration, the ILAE SE definition was applied and considered all tonic–clonic seizures lasting for more than 5 min, as well as focal seizures with impaired consciousness and absence seizures lasting for more than 10 min [ 21 ]. Refractory SE refers to recurrent seizure activity notwithstanding the administration of two ASMs appropriately selected and dosed, including a benzodiazepine [ 22 , 23 ].

The data collected included demographics, clinical diagnosis, etiology, semiology, history of seizures or SE, total length of stay in hospital, modified Rankin Scale (mRS), and Status Epilepticus Severity Score [ 24 ] on admission. The cessation of SE and further seizure freedom were the primary outcome measures used to demonstrate ASM benefits. Treatment was considered successful when no further ASMs were administered until cessation of SE.

Regarding the efficacy of FFA, the duration of SE before the initiation of FFA and the number of ASMs used before FFA initiation were analyzed. The timing of FFA with respect to SE onset and cessation and the presence of adverse events were also collected.

The maximum recommended daily dose of FFA for the treatment of DS or LGS is 0.7 mg per kg bodyweight or a total maximum of 26 mg (0.4 mg/kg bw/day and a total of 17 mg maximum daily dose in comedication with stiripentol), divided into two doses during the day. The recommended treatment initiation is 0.2 mg per kg body weight per day and a gradual increase over 2 weeks to the maximum dose [ 25 ].

Case presentation

This 30-year-old woman had LGS with tonic (associated with falls), generalized tonic–clonic, and focal impaired awareness seizures with gaze deviation to the right side, mostly occurring as clusters of four to six seizures over a few days every 4–6 weeks since the age of 6 months. Furthermore, at the age of 27 years, two episodes of SE occurred: one associated with infection and another with an undetermined cause. Febrile seizures were not reported during childhood. Genetic testing for DEEs including SCN1A, SCN1A (MLPA), PCDH19, SCN2A, CHD2, GABRA1, STXBP1, and HCN1 was negative [ 26 ]. Initial imaging studies did not show an underlying structural cause of the epilepsy. Besides seizures, autism spectrum disorder, severe communication impairment, ataxia, and mild tetraparesis were present since childhood. The patient maintained mobility and was able to walk a few kilometers. No seizure remission had been achieved on various combinations of ASMs.

At the age of 30, the patient presented with generalized tonic and focal impaired awareness seizures manifesting at high frequencies without a return to clinical baseline, constituting the diagnosis of SE. On admission, the ASM regimen consisted of lacosamide 350 mg/d, brivaracetam 100 mg/d, and clonazepam 0.75 mg/d. No clinical or laboratory hints of infection existed. The EEG showed generalized slowing, focal and generalized sharp waves, and 11 seizure patterns with generalized paroxysmal fast activity, coinciding with clinical manifestations of a tonic seizure within 20 min of recording.

Subsequent adjustments to the ASM regimen, with increases of brivaracetam to 300 mg/d and lacosamide to 600 mg/d, a switch to lorazepam 2 mg/d, and the introduction of valproate 1,600 mg/d, did not result in a resolution of SE. Due to elevated ammonia levels, valproate had to be reduced to 800 mg/d, and lorazepam was replaced by clobazam 15 mg/d. The patient’s clinical course was complicated by a urinary tract infection. Seizure and SE were recorded on continuous EEG monitoring. A brain MRI showed various pathologies corresponding with a structural etiology of LGS, while FLAIR and DWI show ictal changes in the right temporoparietal region (Fig.  1 ).

figure 1

Etiology of Lennox–Gastaut syndrome was assumed due to various structural pathologies. Coronal PD-weighted image shows hippocampal sclerosis and atrophy of the temporal lobe, whereas the right superior temporal gyrus is swollen and white matter is hyperintense ( A ). The axial FLAIR ( B ) and corresponding DWI ( C ) show ictal swelling of the right parietal gyri with white matter edema and cortical diffusion restriction, while the contralateral gyri in the left hemisphere are atrophic. Coronal surface reformatting of the 3D FLAIR ( D ) shows how the ictal changes extend across the temporoparietal brain region. See again atrophy in the left hemisphere

Due to ongoing refractory SE and following an unremarkable echocardiography, treatment was initiated with FFA, with an initial dose of 10 mg/d (0.22 mg/kg bw), and fast up-titration to 26 mg/d (0.58 mg/kg bw) within 10 days. FFA solution (Fintepla® 2.2 mg/ml, UCB Pharma S.A., Bruxelles, Belgium) was given through a nasogastric tube twice per day.

No adverse events were observed. Subsequently, the patient experienced a resolution of SE within 4 days, accompanied by a notable improvement in clinical presentation and regaining her mobility, walking with the assistance of physiotherapists. Valproate could be withdrawn from the therapy after SE cessation, and the patient was discharged with brivaracetam 300 mg/d, lacosamide 400 mg/d, clonazepam 0.5 mg/d, and fenfluramine 26 mg/d.

At 2 months follow-up, the caregiver reported an almost seizure-free outcome, with only one tonic seizure since discharge. Mobility improved, with a walking distance of 500 to 1,000 m. The caregivers reduced ASMs to brivaracetam 200 mg/d, lacosamide 400 mg/d, and fenfluramine 17.6 mg/d, while clonazepam has been discontinued.

Literature review

Three additional cases of treatment of SE with FFA were identified and are summarized along with the current case in Table  2 . The first report, published by Specchio et al., was on the treatment of an 8-year-old boy with DS in whom FFA was started in a non-convulsive SE [ 27 ]. Trowbridge et al. reported on a 5-year-old boy with frequent SE episodes in whom FFA stopped the series of SE [ 28 ]. The first adult with DS treated with FFA during a super-refractory SE was reported by Millet et al. [ 29 ]. The number of prior failed ASMs for the treatment of SE before FFA was started ranged between 2 and 14, the SE was resolved in EEG within 4–7 days following the start of FFA. Follow-up data were reported for periods between 1 month and 2 years and 10 months. In all cases, FFA was maintained in the long term.

We present data on four patients who received acute treatment with FFA for SE. In all cases, a resolution of SE within a few days after starting administration of FFA was observed, and no treatment-emergent adverse events were reported. Although we describe a limited case series of patients, the data suggest that FFA may be a useful and well-tolerated acute therapeutic approach in treating refractory SE in patients with DS and LGS.

We describe an adult who fulfilled the diagnostic criteria for LGS [ 30 ], and the MRI revealed structural changes as underlying cause. Despite a prolonged treatment course of SE, the patient returned to baseline functioning and achieved near seizure freedom at the last follow-up, and the ASM drug load could be reduced. The other three cases with DS reported stable ASM drug regimens. A more extended follow-up would be desirable in all cases.

In addition to these four case reports, several publications have demonstrated a diminished incidence of SE during treatment with FFA in real-world settings. A German multicenter analysis of 78 DS patients treated in the Early Access Programme showed a significantly decreased number of SE episodes, from 28% of patients in the 6 months before FFA to only 14% during a 6-month follow-up while on FFA. In addition, 45% of patients were able to discontinue concomitant ASMs, and a further 23% reduced their intake. The retention rate was 92% in adults and 83% in children [ 31 ].

Zhu et al. reported the successful treatment of an 8-month-old boy diagnosed with early infantile epileptic encephalopathy (EIEE), Dravet-like syndrome, and global developmental delays due to biallelic SCN1B genetic variants [ 32 ]. Seizures began at the age of 3 weeks, and by the age of 2 months, myoclonic twitching had evolved to nearly continuous myoclonic seizures. By the age of 4 months, the patient repeatedly presented with convulsive SE and responded only for a short time to multiple ASMs at high doses. Pyridoxine, topiramate, clobazam, and a ketogenic diet also failed. At the age of 8 months, FFA titrated up to 0.7 mg/kg/d gradually reduced the seizure frequency. The patient attained a status-free condition and approximately 50% reduction in seizure frequency, resulting in decreased hospital admissions and costs [ 32 ].

Another case of EIEE linked to biallelic SCN1B variants was reported by Aeby et al. [ 33 ]. The girl had focal and focal-to-bilateral clonic seizures and refractory myoclonic SE since the age of 3 months, not responding to ASMs such as levetiracetam, valproate, topiramate, and clobazam or a ketogenic diet. By the age of 28 months, treatment with FFA (0.6 mg/kg/d) was started and resulted in a significant reduction in seizure frequency and full resolution of recurrent SE episodes for over 2 years of follow-up [ 33 ].

Overall, SE and the sudden unexpected death in epilepsy (SUDEP) are the leading seizure-related causes of death in patients with epilepsy and DEEs such as DS [ 34 , 35 ] and LGS [ 36 ]. A recent pooled analysis of phase III clinical trials and long-term open-label observational studies showed a reduction of all-cause (including SE-related) and SUDEP mortality rates in DS patients treated with FFA [ 37 ]. Cautiously, the use of FFA might be assumed to potentially lead to a reduction in mortality. However, a limitation of our review is that the collected cases may be subject to publication bias since successful cases are more likely to be reported.

Based on the four case reports of acute SE treatment with FFA, and reduced rates of SE during FFA treatment in observational studies and case reports, FFA may prove useful in treating and preventing SE. Treatments that suppress refractory SE are critically needed. Further studies should investigate whether earlier treatment with FFA leads to better and earlier control of SE.

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Antiseizure medication

CAse Reports guidelines

Charlson Comorbidity Index

Magnetic resonance imaging

Electroencephalography

International League Against Epilepsy

Modified Rankin Scale

Non-convulsive status epilepticus

Standard deviation

Status epilepticus

Status Epilepticus Severity Score

Beuchat, I., Rosenow, F., Kellinghaus, C., Trinka, E., Unterberger, I., Ruegg, S., et al. (2022). Refractory status epilepticus: Risk factors and analysis of intubation in the multicenter SENSE registry. Neurology, 99 (16), e1824–e1834. https://doi.org/10.1212/WNL.0000000000201099

Article   PubMed   Google Scholar  

Guterman, E. L., Betjemann, J. P., Aimetti, A., Li, J. W., Wang, Z., Yin, D., et al. (2021). Association between treatment progression, disease refractoriness, and burden of illness among hospitalized patients with status epilepticus. JAMA Neurology, 78 (5), 588–595. https://doi.org/10.1001/jamaneurol.2021.0520

Kellinghaus, C., Rossetti, A. O., Trinka, E., Lang, N., May, T. W., Unterberger, I., et al. (2019). Factors predicting cessation of status epilepticus in clinical practice: Data from a prospective observational registry (SENSE). Annals of Neurology, 85 (3), 421–432. https://doi.org/10.1002/ana.25416

Article   CAS   PubMed   Google Scholar  

Strzelczyk, A., Ansorge, S., Hapfelmeier, J., Bonthapally, V., Erder, M. H., & Rosenow, F. (2017). Costs, length of stay, and mortality of super-refractory status epilepticus: A population-based study from Germany. Epilepsia, 58 (9), 1533–1541. https://doi.org/10.1111/epi.13837

Bauer, K., Rosenow, F., Knake, S., Willems, L. M., Kamppi, L., & Strzelczyk, A. (2023). Clinical characteristics and outcomes of patients with recurrent status epilepticus episodes. Neurological Research and Practice, 5 (1), 34. https://doi.org/10.1186/s42466-023-00261-9

Article   PubMed   PubMed Central   Google Scholar  

Donnan, A. M., Schneider, A. L., Russ-Hall, S., Churilov, L., & Scheffer, I. E. (2023). Rates of status epilepticus and sudden unexplained death in epilepsy in people with genetic developmental and epileptic encephalopathies. Neurology, 100 (16), e1712–e1722. https://doi.org/10.1212/WNL.0000000000207080

Strzelczyk, A., & Schubert-Bast, S. (2022). A practical guide to the treatment of dravet syndrome with anti-seizure medication. CNS Drugs, 36 (3), 217–237. https://doi.org/10.1007/s40263-022-00898-1

Strzelczyk, A., Schubert-Bast, S. (2021). Expanding the treatment landscape for Lennox-Gastaut syndrome: current and future strategies. CNS Drugs , 35 (1), 61–83. https://doi.org/10.1007/s40263-020-00784-8

Frampton, J. E. (2023). Fenfluramine: A review in dravet and lennox-gastaut syndromes. Drugs, 83 (10), 923–934. https://doi.org/10.1007/s40265-023-01881-w

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sourbron, J., & Lagae, L. (2023). Fenfluramine: A plethora of mechanisms? Frontiers in Pharmacology, 14 , 1192022. https://doi.org/10.3389/fphar.2023.1192022

Knupp, K. G., Scheffer, I. E., Ceulemans, B., Sullivan, J. E., Nickels, K. C., Lagae, L., et al. (2022). Efficacy and safety of fenfluramine for the treatment of seizures associated with lennox-gastaut syndrome: A randomized clinical trial. JAMA Neurology, 79 (6), 554–564. https://doi.org/10.1001/jamaneurol.2022.0829

Lagae, L., Sullivan, J., Knupp, K., Laux, L., Polster, T., Nikanorova, M., et al. (2020). Fenfluramine hydrochloride for the treatment of seizures in Dravet syndrome: A randomised, double-blind, placebo-controlled trial. Lancet, 394 (10216), 2243–2254. https://doi.org/10.1016/s0140-6736(19)32500-0

Article   CAS   Google Scholar  

Nabbout, R., Mistry, A., Zuberi, S., Villeneuve, N., Gil-Nagel, A., Sanchez-Carpintero, R., et al. (2020). Fenfluramine for treatment-resistant seizures in patients with dravet syndrome receiving stiripentol-inclusive regimens: A randomized clinical trial. JAMA Neurology, 77 (3), 300–308. https://doi.org/10.1001/jamaneurol.2019.4113

Sullivan, J., Lagae, L., Cross, J. H., Devinsky, O., Guerrini, R., Knupp, K. G., et al. (2023). Fenfluramine in the treatment of Dravet syndrome: Results of a third randomized, placebo-controlled clinical trial. Epilepsia, 64 (10), 2653–2666. https://doi.org/10.1111/epi.17737

Muhle, H., Kurlemann, G., Lehmann, I., Hamer, H., Mayer, T., Potschka, H., et al. (2024) Fenfluramine in clinical practice: new therapy option for Dravet and Lennox–Gastaut syndromes. Clinical Epileptology . https://doi.org/10.1007/s10309-023-00646-5

Li, J., Nelis, M., Sourbron, J., Copmans, D., Lagae, L., Cabooter, D., et al. (2021). Efficacy of fenfluramine and norfenfluramine enantiomers and various antiepileptic drugs in a zebrafish model of dravet syndrome. Neurochemical Research, 46 (9), 2249–2261. https://doi.org/10.1007/s11064-021-03358-2

Riley, D. S., Barber, M. S., Kienle, G. S., Aronson, J. K., von Schoen-Angerer, T., Tugwell, P., et al. (2017). CARE guidelines for case reports: Explanation and elaboration document. Journal of Clinical Epidemiology, 89 , 218–235. https://doi.org/10.1016/j.jclinepi.2017.04.026

Samanta, D. (2022). Fenfluramine: A review of pharmacology, clinical efficacy, and safety in epilepsy. Children (Basel), 9 , 8. https://doi.org/10.3390/children9081159

Article   Google Scholar  

Fisher, R. S., Cross, J. H., French, J. A., Higurashi, N., Hirsch, E., Jansen, F. E., et al. (2017). Operational classification of seizure types by the international league against epilepsy: Position paper of the ILAE commission for classification and terminology. Epilepsia, 58 (4), 522–530. https://doi.org/10.1111/epi.13670

Scheffer, I. E., Berkovic, S., Capovilla, G., Connolly, M. B., French, J., Guilhoto, L., et al. (2017). ILAE classification of the epilepsies: Position paper of the ILAE commission for classification and terminology. Epilepsia, 58 (4), 512–521. https://doi.org/10.1111/epi.13709

Trinka, E., Cock, H., Hesdorffer, D., Rossetti, A. O., Scheffer, I. E., Shinnar, S., et al. (2015). A definition and classification of status epilepticus—report of the ILAE task force on classification of status epilepticus. Epilepsia, 56 (10), 1515–1523. https://doi.org/10.1111/epi.13121

Shorvon, S., & Ferlisi, M. (2011). The treatment of super-refractory status epilepticus: A critical review of available therapies and a clinical treatment protocol. Brain, 134 (Pt 10), 2802–2818. https://doi.org/10.1093/brain/awr215

Betjemann, J. P., & Lowenstein, D. H. (2015). Status epilepticus in adults. The Lancet Neurology, 14 (6), 615–624. https://doi.org/10.1016/S1474-4422(15)00042-3

Rossetti, A. O., Logroscino, G., Milligan, T. A., Michaelides, C., Ruffieux, C., & Bromfield, E. B. (2008). Status epilepticus severity score (STESS): A tool to orient early treatment strategy. Journal of Neurology, 255 (10), 1561–1566. https://doi.org/10.1007/s00415-008-0989-1

European Medicines Agency. Fintepla (fenfluramine). 2023. https://www.ema.europa.eu/en/medicines/human/EPAR/fintepla . Accessed 5 Nov 2023

Boßelmann, C., Borggräfe, I., Fazeli, W., Klein, K.-M., Kluger, G. J., Müller-Schlüter, K., et al. (2023). Genetische Diagnostik der Epilepsien: Empfehlung der Kommission Epilepsie und Genetik der Deutschen Gesellschaft für Epileptologie (DGfE). Clinical Epileptology, 36 (3), 224–237. https://doi.org/10.1007/s10309-023-00580-6

Specchio, N., Pietrafusa, N., Ferretti, A., Trivisano, M., & Vigevano, F. (2020). Successful use of fenfluramine in nonconvulsive status epilepticus of Dravet syndrome. Epilepsia, 61 (4), 831–833. https://doi.org/10.1111/epi.16474

Trowbridge, S., Poduri, A., & Olson, H. (2021). Early diagnosis and experimental treatment with fenfluramine via the Investigational New Drug mechanism in a boy with Dravet syndrome and recurrent status epilepticus. Epileptic Disorders, 23 (6), 954–956. https://doi.org/10.1684/epd.2021.1345

Millett, D., & Pach, S. (2021). Fenfluramine in the successful treatment of super-refractory status epilepticus in a patient with Dravet syndrome. Epilepsy & Behavior Reports, 16 , 100461. https://doi.org/10.1016/j.ebr.2021.100461

Specchio, N., Wirrell, E. C., Scheffer, I. E., Nabbout, R., Riney, K., Samia, P., et al. (2022). International League Against Epilepsy classification and definition of epilepsy syndromes with onset in childhood: Position paper by the ILAE Task Force on Nosology and Definitions. Epilepsia, 63 (6), 1398–1442. https://doi.org/10.1111/epi.17241

Strzelczyk, A., Pringsheim, M., Mayer, T., Polster, T., Klotz, K. A., Muhle, H., et al. (2021). Efficacy, tolerability, and retention of fenfluramine for the treatment of seizures in patients with Dravet syndrome: Compassionate use program in Germany. Epilepsia, 62 (10), 2518–2527. https://doi.org/10.1111/epi.17034

Zhu, Z., Bolt, E., Newmaster, K., Osei-Bonsu, W., Cohen, S., Cuddapah, V. A., et al. (2022). SCN1B genetic variants: A review of the spectrum of clinical phenotypes and a report of early myoclonic encephalopathy. Children (Basel)., 9 , 10. https://doi.org/10.3390/children9101507

Aeby, A., Sculier, C., Bouza, A. A., Askar, B., Lederer, D., Schoonjans, A. S., et al. (2019). SCN1B-linked early infantile developmental and epileptic encephalopathy. Annals of Clinical Translational Neurology, 6 (12), 2354–2367. https://doi.org/10.1002/acn3.50921

Strzelczyk, A., Lagae, L., Wilmshurst, J. M., Brunklaus, A., Striano, P., Rosenow, F., et al. (2023). Dravet syndrome: A systematic literature review of the illness burden. Epilepsia Open, 8 (4), 1256–1270. https://doi.org/10.1002/epi4.12832

Cooper, M. S., McIntosh, A., Crompton, D. E., McMahon, J. M., Schneider, A., Farrell, K., et al. (2016). Mortality in Dravet syndrome. Epilepsy Research, 128 , 43–47. https://doi.org/10.1016/j.eplepsyres.2016.10.006

Strzelczyk, A., Zuberi, S. M., Striano, P., Rosenow, F., & Schubert-Bast, S. (2023). The burden of illness in Lennox-Gastaut syndrome: A systematic literature review. Orphanet Journal of Rare Diseases, 18 (1), 42. https://doi.org/10.1186/s13023-023-02626-4

Cross, J. H., Galer, B. S., Gil-Nagel, A., Devinsky, O., Ceulemans, B., Lagae, L., et al. (2021). Impact of fenfluramine on the expected SUDEP mortality rates in patients with Dravet syndrome. Seizure, 93 , 154–159. https://doi.org/10.1016/j.seizure.2021.10.024

Download references

Acknowledgements

This study was supported by the State of Hessen with a LOEWE grant to the CePTER Consortium ( https://www.uni-frankfurt.de/67689811 ).

This data analysis was supported by the State of Hessen with a LOEWE Grant to the CePTER Consortium ( https://www.uni-frankfurt.de/67689811 ).

Author information

Authors and affiliations.

Goethe-University Frankfurt, Epilepsy Center Frankfurt Rhine-Main, Department of Neurology, University Hospital Frankfurt, Schleusenweg 2-16 (Haus 95), 60528, Frankfurt am Main, Germany

Adam Strzelczyk, Hendrik Becker, Lisa Tako, Susanna Hock, Felix Rosenow & Catrin Mann

Goethe-University Frankfurt, Department of Neuroradiology, University Hospital Frankfurt, Frankfurt am Main, Germany

Elke Hattingen

You can also search for this author in PubMed   Google Scholar

Contributions

AS conceived and supervised the analysis. AS, HB, LT, SH, and CM contributed to the recruitment, collection, and collation of data. LT and EH designed the figures. AS wrote the first draft. AS, FR, and CM edited and revised the first draft. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Adam Strzelczyk .

Ethics declarations

Ethics approval and consent for publication.

The retrospective analysis on the use of anti-seizure medications had ethical approval, and CARE guidelines were followed.

Consent for publication

Not applicable.

Competing interests

The authors declare the following financial interests or personal relationships that may be considered competing interests: AS received personal fees and grants from Angelini Pharma, Biocodex, Desitin Arzneimittel, Eisai, Jazz (GW) Pharmaceuticals, Marinus Pharma, Precisis, Takeda, UCB (Zogenix) Pharma, and UNEEG medical. AS reports that he is part of the editorial board of Neurological Research and Practice . HB, LT, SH, and EH have no conflicts of interest. FR received personal fees from Angelini Pharma/Arvelle Therapeutics, Eisai GmbH, GW Pharmaceuticals/Jazz Pharma, Roche Pharma, and UCB Pharma and grants from the Detlev-Wrobel-Fonds for Epilepsy Research, the Deutsche Forschungsgemeinschaft (DFG), the Federal Ministry of Education and Research (BMBF), the LOEWE Programme of the State of Hesse, and the European Union. FR reports that he is part of the editorial board of Neurological Research and Practice . CM reports speakers’ honoraria from Eisai and UCB and travel support from Jazz Pharmaceuticals.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Strzelczyk, A., Becker, H., Tako, L. et al. Fenfluramine for the treatment of status epilepticus: use in an adult with Lennox–Gastaut syndrome and literature review. Neurol. Res. Pract. 6 , 10 (2024). https://doi.org/10.1186/s42466-023-00306-z

Download citation

Received : 18 December 2023

Accepted : 28 December 2023

Published : 22 February 2024

DOI : https://doi.org/10.1186/s42466-023-00306-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Anticonvulsants
  • Encephalopathy

Neurological Research and Practice

ISSN: 2524-3489

  • Submission enquiries: Access here and click Contact Us
  • General enquiries: [email protected]

use of research literature

U.S. flag

An official website of the United States government.

Here’s how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • American Job Centers
  • Apprenticeship
  • Demonstration Grants
  • Farmworkers
  • Federal Bonding Program
  • Foreign Labor Certification
  • Indians and Native Americans
  • Job Seekers
  • Layoffs and Rapid Response
  • National Dislocated Worker Grants
  • Older Workers
  • Skills Training Grants
  • Trade Adjustment Assistance
  • Unemployment Insurance
  • Workforce Innovation and Opportunity Act (WIOA)
  • WIOA Adult Program
  • Advisories and Directives
  • Regulations
  • Labor Surplus Area
  • Performance
  • Recovery-Ready Workplace Resource Hub
  • Research and Evaluation
  • ETA News Releases
  • Updates for Workforce Professionals
  • Regional Offices
  • Freedom of Information Act

The Role of the Workforce System in Addressing the Opioid Crisis: A Literature Review

Publication info, research methodology, description, other products.

This literature review describes findings from studies on various employment and training interventions to 1) assist individuals in recovery, 2) provide assistance to employers preventing opioid use disorder and creating a recovery-friendly workplace, and 3) develop the health care workforce to address the opioid crisis. The review was developed as part of an implementation evaluation of six Dislocated Worker Demonstration Grants to address the National Health Emergency (NHE) of the opioid crisis. Products form the study also include a resource guide, final report, and four short briefs on promising strategies.

As this review notes, the evidence base for employment interventions specifically aimed at or tested with people with opioid use disorder is limited and, that, while some of the approaches have been rigorously tested, others have not yet been evaluated but are seen as potentially promising practices. The research reviewed covers such approaches as:

  • Intensive case management, as found in various models, such as the individual placement and support (IPS) model, a counseling model based on the interpersonal cognitive problem solving (ICPS) method; and a strategy based on the customized employment support (CES) vocational model;
  • Use of "contingency management," a treatment approach that provides privileges or rewards to participants who exhibit desired behaviors;
  • "Lighter-touch" employment or vocational services for people receiving substance use disorder treatment;
  • Workplace prevention initiatives, employee assistance programs, recovery-friendly workplace initiatives, and modifications in workplace drug testing; and
  • Innovative methods to increase the reach and breadth of training for health care professionals, strategies to support provider training on using medication-assisted treatment, and use of nontraditional providers (such as peer recovery specialists).

The literature notes that, overall, the research on employment-related interventions for people with opioid use disorder is still in its infancy, and for that reason, opportunities for building evidence should be capitalized upon by any organization providing services to address it, and in so doing, lay the groundwork for more rigorous studies.

IMAGES

  1. Helping You in Writing a Literature Review Immaculately

    use of research literature

  2. Sample Literature Review Of An Action Research / A literature review is

    use of research literature

  3. Research Process- Objective, Hypothesis (Lec2)

    use of research literature

  4. How to write a literature review in research paper

    use of research literature

  5. Example Literature Review Research Proposal by

    use of research literature

  6. Scholarly Literature (Databases) ARTICLES

    use of research literature

VIDEO

  1. Systematic literature review

  2. Write Your Literature Review FAST

  3. Academic Writing Workshop

  4. Research Methods: Writing a Literature Review

  5. Review of literature for Research BY DR.NISHIKANT C.DHANDE

  6. Research (Literature Review) Groups 9 & 10

COMMENTS

  1. Literature search for research planning and identification of research problem

    Literature search is a systematic and well-organised search from the already published data to identify a breadth of good quality references on a specific topic. [ 1]

  2. How to Write a Literature Review

    It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic. There are five key steps to writing a literature review: Search for relevant literature Evaluate sources Identify themes, debates, and gaps

  3. Literature review as a research methodology: An ...

    In addition, a literature review is an excellent way of synthesizing research findings to show evidence on a meta-level and to uncover areas in which more research is needed, which is a critical component of creating theoretical frameworks and building conceptual models.

  4. Literature Review Research

    Literature Review is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

  5. 5. The Literature Review

    Definition A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated.

  6. Reviewing the Research Literature

    Describe and use several methods for finding previous research on a particular research idea or question. Reviewing the research literature means finding, reading, and summarizing the published research relevant to your question. An empirical research report written in American Psychological Association (APA) style always includes a written ...

  7. Steps in Conducting a Literature Review

    1. Choose a topic. Define your research question. Your literature review should be guided by your central research question. The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

  8. Reviewing the research methods literature: principles and strategies

    First, reviewers use an initial approach to conduct a broad overview of the field—for reviews of methods topics, this would entail an initial review of the research methods literature. This is followed by a second more focused stage in which practical examples are purposefully selected—for methods reviews, this would involve sampling the ...

  9. Getting started

    Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject. Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field. Analysis: Involves critically evaluating and summarizing key findings, methodologies, and ...

  10. Methodological Approaches to Literature Review

    A literature review is defined as "a critical analysis of a segment of a published body of knowledge through summary, classification, and comparison of prior research studies, reviews of literature, and theoretical articles." (The Writing Center University of Winconsin-Madison 2022) A literature review is an integrated analysis, not just a summary of scholarly work on a specific topic.

  11. Literature Review: The What, Why and How-to Guide

    What kinds of literature reviews are written? Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified.

  12. Research Guides: Literature Reviews: What is a Literature Review?

    A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the ...

  13. Writing a Literature Review

    Divide the research by sociological, historical, or cultural sources; Theoretical: In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts.

  14. Guidance on Conducting a Systematic Literature Review

    Literature review is an essential feature of academic research. Fundamentally, knowledge advancement must be built on prior existing work. To push the knowledge frontier, we must know where the frontier is. By reviewing relevant literature, we understand the breadth and depth of the existing body of work and identify gaps to explore.

  15. Conducting a Literature Review: Why Do A Literature Review?

    Besides the obvious reason for students -- because it is assigned! -- a literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed. You identify: core research in the field experts in the subject area methodology you may want to use (or avoid)

  16. Writing a literature review

    The dissemination of your research findings, whether by publication in a peer-reviewed paper or by oral presentation, will use the information gathered for a literature review, thus providing reference points for your new data and helping to identify and deliver the potential impact of your research.

  17. Creating a Literature Review

    A literature review requires the writer to perform extensive research on published work in one's field in order to explain how one's own work fits into the larger conversation regarding a particular topic. This task requires the writer to spend time reading, managing, and conveying information; the complexity of literature reviews can make ...

  18. PDF Conducting a Literature Review

    The Literature Research Workflow Web of Science The world's largest and highest quality publisher-neutral citation index. Essential Science Indicators Reveals emerging science trends as well as influential individuals, institutions, papers, journals, and countries across 22 categories of research. Journal Citation Reports

  19. Literature Reviews

    But how is a literature review different from an academic research paper? The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute.

  20. PDF Literature Review and Focusing the Research

    3. Research may have been conducted on a different population than the one in which you are interested, thus justifying your work with the different population. For example, Schirmer and McGough (2005) reviewed research literature on reading development and reading instruction and found that there was a lack of research of this type on students

  21. PDF Use of Libraries, Literature Search and Review

    3.6 Chapter Summary. The use of the library and scholarly web-based search may offer quick access to the archival literature, which is always ordered according to its date of publication and disciplines. A literature search and review is an essential component in the design and implementation of a research project.

  22. 'It depends': what 86 systematic reviews tell us about what strategies

    We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos.

  23. Building on the Evidence Base: Studies on the Effects of Medicaid

    Themes in Recent Research. Several key themes emerge from recent literature investigating the effects of Medicaid expansion (Figure 1). In contrast to earlier research, which largely focused on ...

  24. Research on country-of-origin perceptions: review, critical ...

    The research examines the theoretical foundations of the CO literature, key methodological aspects of CO research, major empirical findings on the topic, and offers suggestions for future research. To identify and extract prior CO articles, the study utilized several key datasets, including ABI Global, EBSCO, Scopus, and the Web of Science.

  25. The complexity of leadership in coproduction practices: a guiding

    This systematic literature review is based on a study protocol on coproduction research in the context of health and social care sectors [], and data were obtained from a published scoping review, where the full search strategy is provided [].The scoping review set out to identify 'what is out there' and to explore the definitions of the concepts of coproduction and codesign.

  26. Fenfluramine for the treatment of status epilepticus: use in an adult

    Background Novel treatments are needed to control refractory status epilepticus (SE). This study aimed to assess the potential effectiveness of fenfluramine (FFA) as an acute treatment option for SE. We present a summary of clinical cases where oral FFA was used in SE. Methods A case of an adult patient with Lennox-Gastaut syndrome (LGS) who was treated with FFA due to refractory SE is ...

  27. The Role of the Workforce System in Addressing the Opioid Crisis: A

    This literature review describes findings from studies on various employment and training interventions to 1) assist individuals in recovery, 2) provide assistance to employers preventing opioid use disorder and creating a recovery-friendly workplace, and 3) develop the health care workforce to address the opioid crisis. The review was developed as part of an implementation evaluation of six ...

  28. Sustainability

    The steel industry, which relies heavily on primary energy, is one of the industries with the highest CO2 emissions in China. It is urgent for the industry to identify ways to embark on the path to "green steel". Hydrogen metallurgy technology uses hydrogen as a reducing agent, and its use is an important way to reduce CO2 emissions from long-term steelmaking and ensure the green and ...