Library Homepage

Research Methods and Design

  • Action Research
  • Case Study Design

Literature Review

  • Quantitative Research Methods
  • Qualitative Research Methods
  • Mixed Methods Study
  • Indigenous Research and Ethics This link opens in a new window
  • Identifying Empirical Research Articles This link opens in a new window
  • Research Ethics and Quality
  • Data Literacy
  • Get Help with Writing Assignments

A literature review is a discussion of the literature (aka. the "research" or "scholarship") surrounding a certain topic. A good literature review doesn't simply summarize the existing material, but provides thoughtful synthesis and analysis. The purpose of a literature review is to orient your own work within an existing body of knowledge. A literature review may be written as a standalone piece or be included in a larger body of work.

You can read more about literature reviews, what they entail, and how to write one, using the resources below. 

Am I the only one struggling to write a literature review?

Dr. Zina O'Leary explains the misconceptions and struggles students often have with writing a literature review. She also provides step-by-step guidance on writing a persuasive literature review.

An Introduction to Literature Reviews

Dr. Eric Jensen, Professor of Sociology at the University of Warwick, and Dr. Charles Laurie, Director of Research at Verisk Maplecroft, explain how to write a literature review, and why researchers need to do so. Literature reviews can be stand-alone research or part of a larger project. They communicate the state of academic knowledge on a given topic, specifically detailing what is still unknown.

This is the first video in a whole series about literature reviews. You can find the rest of the series in our SAGE database, Research Methods:

Videos

Videos covering research methods and statistics

Identify Themes and Gaps in Literature (with real examples) | Scribbr

Finding connections between sources is key to organizing the arguments and structure of a good literature review. In this video, you'll learn how to identify themes, debates, and gaps between sources, using examples from real papers.

4 Tips for Writing a Literature Review's Intro, Body, and Conclusion | Scribbr

While each review will be unique in its structure--based on both the existing body of both literature and the overall goals of your own paper, dissertation, or research--this video from Scribbr does a good job simplifying the goals of writing a literature review for those who are new to the process. In this video, you’ll learn what to include in each section, as well as 4 tips for the main body illustrated with an example.

Cover Art

  • Literature Review This chapter in SAGE's Encyclopedia of Research Design describes the types of literature reviews and scientific standards for conducting literature reviews.
  • UNC Writing Center: Literature Reviews This handout from the Writing Center at UNC will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.
  • Purdue OWL: Writing a Literature Review The overview of literature reviews comes from Purdue's Online Writing Lab. It explains the basic why, what, and how of writing a literature review.

Organizational Tools for Literature Reviews

One of the most daunting aspects of writing a literature review is organizing your research. There are a variety of strategies that you can use to help you in this task. We've highlighted just a few ways writers keep track of all that information! You can use a combination of these tools or come up with your own organizational process. The key is choosing something that works with your own learning style.

Citation Managers

Citation managers are great tools, in general, for organizing research, but can be especially helpful when writing a literature review. You can keep all of your research in one place, take notes, and organize your materials into different folders or categories. Read more about citations managers here:

  • Manage Citations & Sources

Concept Mapping

Some writers use concept mapping (sometimes called flow or bubble charts or "mind maps") to help them visualize the ways in which the research they found connects.

literature review research design

There is no right or wrong way to make a concept map. There are a variety of online tools that can help you create a concept map or you can simply put pen to paper. To read more about concept mapping, take a look at the following help guides:

  • Using Concept Maps From Williams College's guide, Literature Review: A Self-guided Tutorial

Synthesis Matrix

A synthesis matrix is is a chart you can use to help you organize your research into thematic categories. By organizing your research into a matrix, like the examples below, can help you visualize the ways in which your sources connect. 

  • Walden University Writing Center: Literature Review Matrix Find a variety of literature review matrix examples and templates from Walden University.
  • Writing A Literature Review and Using a Synthesis Matrix An example synthesis matrix created by NC State University Writing and Speaking Tutorial Service Tutors. If you would like a copy of this synthesis matrix in a different format, like a Word document, please ask a librarian. CC-BY-SA 3.0
  • << Previous: Case Study Design
  • Next: Quantitative Research Methods >>
  • Last Updated: Feb 6, 2024 9:20 AM

CityU Home - CityU Catalog

Creative Commons License

Grad Coach

How To Structure Your Literature Review

3 options to help structure your chapter.

By: Amy Rommelspacher (PhD) | Reviewer: Dr Eunice Rautenbach | November 2020 (Updated May 2023)

Writing the literature review chapter can seem pretty daunting when you’re piecing together your dissertation or thesis. As  we’ve discussed before , a good literature review needs to achieve a few very important objectives – it should:

  • Demonstrate your knowledge of the research topic
  • Identify the gaps in the literature and show how your research links to these
  • Provide the foundation for your conceptual framework (if you have one)
  • Inform your own  methodology and research design

To achieve this, your literature review needs a well-thought-out structure . Get the structure of your literature review chapter wrong and you’ll struggle to achieve these objectives. Don’t worry though – in this post, we’ll look at how to structure your literature review for maximum impact (and marks!).

The function of the lit review

But wait – is this the right time?

Deciding on the structure of your literature review should come towards the end of the literature review process – after you have collected and digested the literature, but before you start writing the chapter. 

In other words, you need to first develop a rich understanding of the literature before you even attempt to map out a structure. There’s no use trying to develop a structure before you’ve fully wrapped your head around the existing research.

Equally importantly, you need to have a structure in place before you start writing , or your literature review will most likely end up a rambling, disjointed mess. 

Importantly, don’t feel that once you’ve defined a structure you can’t iterate on it. It’s perfectly natural to adjust as you engage in the writing process. As we’ve discussed before , writing is a way of developing your thinking, so it’s quite common for your thinking to change – and therefore, for your chapter structure to change – as you write. 

Need a helping hand?

literature review research design

Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components – an  introduction , a  body   and a  conclusion . 

Let’s take a closer look at each of these.

1: The Introduction Section

Just like any good introduction, the introduction section of your literature review should introduce the purpose and layout (organisation) of the chapter. In other words, your introduction needs to give the reader a taste of what’s to come, and how you’re going to lay that out. Essentially, you should provide the reader with a high-level roadmap of your chapter to give them a taste of the journey that lies ahead.

Here’s an example of the layout visualised in a literature review introduction:

Example of literature review outline structure

Your introduction should also outline your topic (including any tricky terminology or jargon) and provide an explanation of the scope of your literature review – in other words, what you  will   and  won’t   be covering (the delimitations ). This helps ringfence your review and achieve a clear focus . The clearer and narrower your focus, the deeper you can dive into the topic (which is typically where the magic lies). 

Depending on the nature of your project, you could also present your stance or point of view at this stage. In other words, after grappling with the literature you’ll have an opinion about what the trends and concerns are in the field as well as what’s lacking. The introduction section can then present these ideas so that it is clear to examiners that you’re aware of how your research connects with existing knowledge .

Free Webinar: Literature Review 101

2: The Body Section

The body of your literature review is the centre of your work. This is where you’ll present, analyse, evaluate and synthesise the existing research. In other words, this is where you’re going to earn (or lose) the most marks. Therefore, it’s important to carefully think about how you will organise your discussion to present it in a clear way. 

The body of your literature review should do just as the description of this chapter suggests. It should “review” the literature – in other words, identify, analyse, and synthesise it. So, when thinking about structuring your literature review, you need to think about which structural approach will provide the best “review” for your specific type of research and objectives (we’ll get to this shortly).

There are (broadly speaking)  three options  for organising your literature review.

The body section of your literature review is the where you'll present, analyse, evaluate and synthesise the existing research.

Option 1: Chronological (according to date)

Organising the literature chronologically is one of the simplest ways to structure your literature review. You start with what was published first and work your way through the literature until you reach the work published most recently. Pretty straightforward.

The benefit of this option is that it makes it easy to discuss the developments and debates in the field as they emerged over time. Organising your literature chronologically also allows you to highlight how specific articles or pieces of work might have changed the course of the field – in other words, which research has had the most impact . Therefore, this approach is very useful when your research is aimed at understanding how the topic has unfolded over time and is often used by scholars in the field of history. That said, this approach can be utilised by anyone that wants to explore change over time .

Adopting the chronological structure allows you to discuss the developments and debates in the field as they emerged over time.

For example , if a student of politics is investigating how the understanding of democracy has evolved over time, they could use the chronological approach to provide a narrative that demonstrates how this understanding has changed through the ages.

Here are some questions you can ask yourself to help you structure your literature review chronologically.

  • What is the earliest literature published relating to this topic?
  • How has the field changed over time? Why?
  • What are the most recent discoveries/theories?

In some ways, chronology plays a part whichever way you decide to structure your literature review, because you will always, to a certain extent, be analysing how the literature has developed. However, with the chronological approach, the emphasis is very firmly on how the discussion has evolved over time , as opposed to how all the literature links together (which we’ll discuss next ).

Option 2: Thematic (grouped by theme)

The thematic approach to structuring a literature review means organising your literature by theme or category – for example, by independent variables (i.e. factors that have an impact on a specific outcome).

As you’ve been collecting and synthesising literature , you’ll likely have started seeing some themes or patterns emerging. You can then use these themes or patterns as a structure for your body discussion. The thematic approach is the most common approach and is useful for structuring literature reviews in most fields.

For example, if you were researching which factors contributed towards people trusting an organisation, you might find themes such as consumers’ perceptions of an organisation’s competence, benevolence and integrity. Structuring your literature review thematically would mean structuring your literature review’s body section to discuss each of these themes, one section at a time.

The thematic structure allows you to organise your literature by theme or category  – e.g. by independent variables.

Here are some questions to ask yourself when structuring your literature review by themes:

  • Are there any patterns that have come to light in the literature?
  • What are the central themes and categories used by the researchers?
  • Do I have enough evidence of these themes?

PS – you can see an example of a thematically structured literature review in our literature review sample walkthrough video here.

Option 3: Methodological

The methodological option is a way of structuring your literature review by the research methodologies used . In other words, organising your discussion based on the angle from which each piece of research was approached – for example, qualitative , quantitative or mixed  methodologies.

Structuring your literature review by methodology can be useful if you are drawing research from a variety of disciplines and are critiquing different methodologies. The point of this approach is to question  how  existing research has been conducted, as opposed to  what  the conclusions and/or findings the research were.

The methodological structure allows you to organise your chapter by the analysis method  used - e.g. qual, quant or mixed.

For example, a sociologist might centre their research around critiquing specific fieldwork practices. Their literature review will then be a summary of the fieldwork methodologies used by different studies.

Here are some questions you can ask yourself when structuring your literature review according to methodology:

  • Which methodologies have been utilised in this field?
  • Which methodology is the most popular (and why)?
  • What are the strengths and weaknesses of the various methodologies?
  • How can the existing methodologies inform my own methodology?

3: The Conclusion Section

Once you’ve completed the body section of your literature review using one of the structural approaches we discussed above, you’ll need to “wrap up” your literature review and pull all the pieces together to set the direction for the rest of your dissertation or thesis.

The conclusion is where you’ll present the key findings of your literature review. In this section, you should emphasise the research that is especially important to your research questions and highlight the gaps that exist in the literature. Based on this, you need to make it clear what you will add to the literature – in other words, justify your own research by showing how it will help fill one or more of the gaps you just identified.

Last but not least, if it’s your intention to develop a conceptual framework for your dissertation or thesis, the conclusion section is a good place to present this.

In the conclusion section, you’ll need to present the key findings of your literature review and highlight the gaps that exist in the literature. Based on this, you'll  need to make it clear what your study will add  to the literature.

Example: Thematically Structured Review

In the video below, we unpack a literature review chapter so that you can see an example of a thematically structure review in practice.

Let’s Recap

In this article, we’ve  discussed how to structure your literature review for maximum impact. Here’s a quick recap of what  you need to keep in mind when deciding on your literature review structure:

  • Just like other chapters, your literature review needs a clear introduction , body and conclusion .
  • The introduction section should provide an overview of what you will discuss in your literature review.
  • The body section of your literature review can be organised by chronology , theme or methodology . The right structural approach depends on what you’re trying to achieve with your research.
  • The conclusion section should draw together the key findings of your literature review and link them to your research questions.

If you’re ready to get started, be sure to download our free literature review template to fast-track your chapter outline.

Literature Review Course

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

Literature review 101 - how to find articles

27 Comments

Marin

Great work. This is exactly what I was looking for and helps a lot together with your previous post on literature review. One last thing is missing: a link to a great literature chapter of an journal article (maybe with comments of the different sections in this review chapter). Do you know any great literature review chapters?

ISHAYA JEREMIAH AYOCK

I agree with you Marin… A great piece

Qaiser

I agree with Marin. This would be quite helpful if you annotate a nicely structured literature from previously published research articles.

Maurice Kagwi

Awesome article for my research.

Ache Roland Ndifor

I thank you immensely for this wonderful guide

Malik Imtiaz Ahmad

It is indeed thought and supportive work for the futurist researcher and students

Franklin Zon

Very educative and good time to get guide. Thank you

Dozie

Great work, very insightful. Thank you.

KAWU ALHASSAN

Thanks for this wonderful presentation. My question is that do I put all the variables into a single conceptual framework or each hypothesis will have it own conceptual framework?

CYRUS ODUAH

Thank you very much, very helpful

Michael Sanya Oluyede

This is very educative and precise . Thank you very much for dropping this kind of write up .

Karla Buchanan

Pheeww, so damn helpful, thank you for this informative piece.

Enang Lazarus

I’m doing a research project topic ; stool analysis for parasitic worm (enteric) worm, how do I structure it, thanks.

Biswadeb Dasgupta

comprehensive explanation. Help us by pasting the URL of some good “literature review” for better understanding.

Vik

great piece. thanks for the awesome explanation. it is really worth sharing. I have a little question, if anyone can help me out, which of the options in the body of literature can be best fit if you are writing an architectural thesis that deals with design?

S Dlamini

I am doing a research on nanofluids how can l structure it?

PATRICK MACKARNESS

Beautifully clear.nThank you!

Lucid! Thankyou!

Abraham

Brilliant work, well understood, many thanks

Nour

I like how this was so clear with simple language 😊😊 thank you so much 😊 for these information 😊

Lindiey

Insightful. I was struggling to come up with a sensible literature review but this has been really helpful. Thank you!

NAGARAJU K

You have given thought-provoking information about the review of the literature.

Vakaloloma

Thank you. It has made my own research better and to impart your work to students I teach

Alphonse NSHIMIYIMANA

I learnt a lot from this teaching. It’s a great piece.

Resa

I am doing research on EFL teacher motivation for his/her job. How Can I structure it? Is there any detailed template, additional to this?

Gerald Gormanous

You are so cool! I do not think I’ve read through something like this before. So nice to find somebody with some genuine thoughts on this issue. Seriously.. thank you for starting this up. This site is one thing that is required on the internet, someone with a little originality!

kan

I’m asked to do conceptual, theoretical and empirical literature, and i just don’t know how to structure it

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

  • Print Friendly
  • USC Libraries
  • Research Guides

Organizing Your Social Sciences Research Paper

  • 5. The Literature Review
  • Purpose of Guide
  • Design Flaws to Avoid
  • Independent and Dependent Variables
  • Glossary of Research Terms
  • Reading Research Effectively
  • Narrowing a Topic Idea
  • Broadening a Topic Idea
  • Extending the Timeliness of a Topic Idea
  • Academic Writing Style
  • Choosing a Title
  • Making an Outline
  • Paragraph Development
  • Research Process Video Series
  • Executive Summary
  • The C.A.R.S. Model
  • Background Information
  • The Research Problem/Question
  • Theoretical Framework
  • Citation Tracking
  • Content Alert Services
  • Evaluating Sources
  • Primary Sources
  • Secondary Sources
  • Tiertiary Sources
  • Scholarly vs. Popular Publications
  • Qualitative Methods
  • Quantitative Methods
  • Insiderness
  • Using Non-Textual Elements
  • Limitations of the Study
  • Common Grammar Mistakes
  • Writing Concisely
  • Avoiding Plagiarism
  • Footnotes or Endnotes?
  • Further Readings
  • Generative AI and Writing
  • USC Libraries Tutorials and Other Guides
  • Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

  • Give a new interpretation of old material or combine new with old interpretations,
  • Trace the intellectual progression of the field, including major debates,
  • Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
  • Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

  • Place each work in the context of its contribution to understanding the research problem being studied.
  • Describe the relationship of each work to the others under consideration.
  • Identify new ways to interpret prior research.
  • Reveal any gaps that exist in the literature.
  • Resolve conflicts amongst seemingly contradictory previous studies.
  • Identify areas of prior scholarship to prevent duplication of effort.
  • Point the way in fulfilling a need for additional research.
  • Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

  • An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
  • Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
  • An explanation of how each work is similar to and how it varies from the others,
  • Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

  • Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
  • Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
  • Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
  • Persuasiveness -- which of the author's theses are most convincing or least convincing?
  • Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

  • Current Situation : Information necessary to understand the current topic or focus of the literature review.
  • Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
  • History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
  • Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
  • Standards : Description of the way in which you present your information.
  • Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

  • Sources in your literature review do not clearly relate to the research problem;
  • You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
  • Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
  • Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
  • Does not describe the search procedures that were used in identifying the literature to review;
  • Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
  • Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

  • How are they organizing their ideas?
  • What methods have they used to study the problem?
  • What theories have been used to explain, predict, or understand their research problem?
  • What sources have they cited to support their conclusions?
  • How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Research Methods: Literature Reviews

  • Annotated Bibliographies
  • Literature Reviews
  • Scoping Reviews
  • Systematic Reviews
  • Scholarship of Teaching and Learning
  • Persuasive Arguments
  • Subject Specific Methodology

A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. A literature review helps the author learn about the history and nature of their topic, and identify research gaps and problems.

Steps & Elements

Problem formulation

  • Determine your topic and its components by asking a question
  • Research: locate literature related to your topic to identify the gap(s) that can be addressed
  • Read: read the articles or other sources of information
  • Analyze: assess the findings for relevancy
  • Evaluating: determine how the article are relevant to your research and what are the key findings
  • Synthesis: write about the key findings and how it is relevant to your research

Elements of a Literature Review

  • Summarize subject, issue or theory under consideration, along with objectives of the review
  • Divide works under review into categories (e.g. those in support of a particular position, those against, those offering alternative theories entirely)
  • Explain how each work is similar to and how it varies from the others
  • Conclude which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of an area of research

Writing a Literature Review Resources

  • How to Write a Literature Review From the Wesleyan University Library
  • Write a Literature Review From the University of California Santa Cruz Library. A Brief overview of a literature review, includes a list of stages for writing a lit review.
  • Literature Reviews From the University of North Carolina Writing Center. Detailed information about writing a literature review.
  • Undertaking a literature review: a step-by-step approach Cronin, P., Ryan, F., & Coughan, M. (2008). Undertaking a literature review: A step-by-step approach. British Journal of Nursing, 17(1), p.38-43

literature review research design

Literature Review Tutorial

  • << Previous: Annotated Bibliographies
  • Next: Scoping Reviews >>
  • Last Updated: Feb 29, 2024 12:00 PM
  • URL: https://guides.auraria.edu/researchmethods

1100 Lawrence Street Denver, CO 80204 303-315-7700 Ask Us Directions

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

  • Advanced Search
  • Journal List
  • Perspect Clin Res
  • v.11(2); Apr-Jun 2020

Study designs: Part 7 – Systematic reviews

Priya ranganathan.

Department of Anaesthesiology, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Rakesh Aggarwal

1 Director, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India

In this series on research study designs, we have so far looked at different types of primary research designs which attempt to answer a specific question. In this segment, we discuss systematic review, which is a study design used to summarize the results of several primary research studies. Systematic reviews often also use meta-analysis, which is a statistical tool to mathematically collate the results of various research studies to obtain a pooled estimate of treatment effect; this will be discussed in the next article.

In the previous six articles in this series on study designs, we have looked at different types of primary research study designs which are used to answer research questions. In this article, we describe the systematic review, a type of secondary research design that is used to summarize the results of prior primary research studies. Systematic reviews are considered the highest level of evidence for a particular research question.[ 1 ]

SYSTEMATIC REVIEWS

As defined in the Cochrane Handbook for Systematic Reviews of Interventions , “Systematic reviews seek to collate evidence that fits pre-specified eligibility criteria in order to answer a specific research question. They aim to minimize bias by using explicit, systematic methods documented in advance with a protocol.”[ 2 ]

NARRATIVE VERSUS SYSTEMATIC REVIEWS

Review of available data has been done since times immemorial. However, the traditional narrative reviews (“expert reviews”) do not involve a systematic search of the literature. Instead, the author of the review, usually an expert on the subject, used informal methods to identify (what he or she thinks are) the key studies on the topic. The final review thus is a summary of these “selected” studies. Since studies are chosen at will (haphazardly!) and without clearly defined criteria, such reviews preferentially include those studies that favor the author's views, leading to a potential for subjectivity or selection bias.

In contrast, systematic reviews involve a formal prespecified protocol with explicit, transparent criteria for the inclusion and exclusion of studies, thereby ensuring completeness of coverage of the available evidence, and providing a more objective, replicable, and comprehensive overview it.

META-ANALYSIS

Many systematic reviews use an additional tool, known as meta-analysis, which is a statistical technique for combining the results of multiple studies in a systematic review in a mathematically appropriate way, to create a single (pooled) and more precise estimate of treatment effect. The feasibility of performing a meta-analysis in a systematic review depends on the number of studies included in the final review and the degree of heterogeneity in the inclusion criteria as well as the results between the included studies. Meta-analysis will be discussed in detail in the next article in this series.

THE PROCESS OF A SYSTEMATIC REVIEW

The conduct of a systematic review involves several sequential key steps.[ 3 , 4 ] As in other research study designs, a clearly stated research question and a well-written research protocol are essential before commencing a systematic review.

Step 1: Stating the review question

Systematic reviews can be carried out in any field of medical research, e.g. efficacy or safety of interventions, diagnostics, screening or health economics. In this article, we focus on systematic reviews of studies looking at the efficacy of interventions. As for the other study designs, for a systematic review too, the question is best framed using the Population, Intervention, Comparator, and Outcome (PICO) format.

For example, Safi et al . carried out a systematic review on the effect of beta-blockers on the outcomes of patients with myocardial infarction.[ 5 ] In this review, the Population was patients with suspected or confirmed myocardial infarction, the Intervention was beta-blocker therapy, the Comparator was either placebo or no intervention, and the Outcomes were all-cause mortality and major adverse cardiovascular events. The review question was “ In patients with suspected or confirmed myocardial infarction, does the use of beta-blockers affect mortality or major adverse cardiovascular outcomes? ”

Step 2: Listing the eligibility criteria for studies to be included

It is essential to explicitly define a priori the criteria for selection of studies which will be included in the review. Besides the PICO components, some additional criteria used frequently for this purpose include language of publication (English versus non-English), publication status (published as full paper versus unpublished), study design (randomized versus quasi-experimental), age group (adults versus children), and publication year (e.g. in the last 5 years, or since a particular date). The PICO criteria used may not be very specific, e.g. it is possible to include studies that use one or the other drug belonging to the same group. For instance, the systematic review by Safi et al . included all randomized clinical trials, irrespective of setting, blinding, publication status, publication year, or language, and reported outcomes, that had used any beta-blocker and in a broad range of doses.[ 5 ]

Step 3: Comprehensive search for studies that meet the eligibility criteria

A thorough literature search is essential to identify all articles related to the research question and to ensure that no relevant article is left out. The search may include one or more electronic databases and trial registries; in addition, it is common to hand-search the cross-references in the articles identified through such searches. One could also plan to reach out to experts in the field to identify unpublished data, and to search the grey literature non-peer-reviewednon-peer-reviewed. This last option is particularly helpful non-pharmacologic (theses, conference abstracts, and non-peer-reviewed journals). These sources are particularly helpful when the intervention is relatively new, since data on these may not yet have been published as full papers and hence are unlikely to be found in literature databases. In the review by Safi et al ., the search strategy included not only several electronic databases (Cochrane, MEDLINE, EMBASE, LILACS, etc.) but also other resources (e.g. Google Scholar, WHO International Clinical Trial Registry Platform, and reference lists of identified studies).[ 5 ] It is not essential to include all the above databases in one's search. However, it is mandatory to define in advance which of these will be searched.

Step 4: Identifying and selecting relevant studies

Once the search strategy defined in the previous step has been run to identify potentially relevant studies, a two-step process is followed. First, the titles and abstracts of the identified studies are processed to exclude any duplicates and to discard obviously irrelevant studies. In the next step, full-text papers of the remaining articles are retrieved and closely reviewed to identify studies that meet the eligibility criteria. To minimize bias, these selection steps are usually performed independently by at least two reviewers, who also assign a reason for non-selection to each discarded study. Any discrepancies are then resolved either by an independent reviewer or by mutual consensus of the original reviewers. In the Cochrane review on beta-blockers referred to above, two review authors independently screened the titles for inclusion, and then, four review authors independently reviewed the screen-positive studies to identify the trials to be included in the final review.[ 5 ] Disagreements were resolved by discussion or by taking the opinion of a separate reviewer. A summary of this selection process, showing the degree of agreement between reviewers, and a flow diagram that depicts the numbers of screened, included and excluded (with reason for exclusion) studies are often included in the final review.

Step 5: Data extraction

In this step, from each selected study, relevant data are extracted. This should be done by at least two reviewers independently, and the data then compared to identify any errors in extraction. Standard data extraction forms help in objective data extraction. The data extracted usually contain the name of the author, the year of publication, details of intervention and control treatments, and the number of participants and outcome data in each group. In the review by Safi et al ., four review authors independently extracted data and resolved any differences by discussion.[ 5 ]

Handling missing data

Some of the studies included in the review may not report outcomes in accordance with the review methodology. Such missing data can be handled in two ways – by contacting authors of the original study to obtain the necessary data and by using data imputation techniques. Safi et al . used both these approaches – they tried to get data from the trial authors; however, where that failed, they analyzed the primary outcome (mortality) using the best case (i.e. presuming that all the participants in the experimental arm with missing data had survived and those in the control arm with missing mortality data had died – representing the maximum beneficial effect of the intervention) and the worst case (all the participants with missing data in the experimental arm assumed to have died and those in the control arm to have survived – representing the least beneficial effect of the intervention) scenarios.

Evaluating the quality (or risk of bias) in the included studies

The overall quality of a systematic review depends on the quality of each of the included studies. Quality of a study is inversely proportional to the potential for bias in its design. In our previous articles on interventional study design in this series, we discussed various methods to reduce bias – such as randomization, allocation concealment, participant and assessor blinding, using objective endpoints, minimizing missing data, the use of intention-to-treat analysis, and complete reporting of all outcomes.[ 6 , 7 ] These features form the basis of the Cochrane Risk of Bias Tool (RoB 2), which is a commonly used instrument to assess the risk of bias in the studies included in a systematic review.[ 8 ] Based on this tool, one can classify each study in a review as having low risk of bias, having some concerns regarding bias, or at high risk of bias. Safi et al . used this tool to classify the included studies as having low or high risk of bias and presented these data in both tabular and graphical formats.[ 5 ]

In some reviews, the authors decide to summarize only studies with a low risk of bias and to exclude those with a high risk of bias. Alternatively, some authors undertake a separate analysis of studies with low risk of bias, besides an analysis of all the studies taken together. The conclusions from such analyses of only high-quality studies may be more robust.

Step 6: Synthesis of results

The data extracted from various studies are pooled quantitatively (known as a meta-analysis) or qualitatively (if pooling of results is not considered feasible). For qualitative reviews, data are usually presented in the tabular format, showing the characteristics of each included study, to allow for easier interpretation.

Sensitivity analyses

Sensitivity analyses are used to test the robustness of the results of a systematic review by examining the impact of excluding or including studies with certain characteristics. As referred to above, this can be based on the risk of bias (methodological quality), studies with a specific study design, studies with a certain dosage or schedule, or sample size. If results of these different analyses are more-or-less the same, one can be more certain of the validity of the findings of the review. Furthermore, such analyses can help identify whether the effect of the intervention could vary across different levels of another factor. In the beta-blocker review, sensitivity analysis was performed depending on the risk of bias of included studies.[ 5 ]

IMPORTANT RESOURCES FOR CARRYING OUT SYSTEMATIC REVIEWS AND META-ANALYSES

Cochrane is an organization that works to produce good-quality, updated systematic reviews related to human healthcare and policy, which are accessible to people across the world.[ 9 ] There are more than 7000 Cochrane reviews on various topics. One of its main resources is the Cochrane Library (available at https://www.cochranelibrary.com/ ), which incorporates several databases with different types of high-quality evidence to inform healthcare decisions, including the Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials (CENTRAL), and Cochrane Clinical Answers.

The Cochrane Handbook for Systematic Reviews of Interventions

The Cochrane handbook is an official guide, prepared by the Cochrane Collaboration, to the process of preparing and maintaining Cochrane systematic reviews.[ 10 ]

Review Manager software

Review Manager (RevMan) is a software developed by Cochrane to support the preparation and maintenance of systematic reviews, including tools for performing meta-analysis.[ 11 ] It is freely available in both online (RevMan Web) and offline (RevMan 5.3) versions.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement is an evidence-based minimum set of items for reporting of systematic reviews and meta-analyses of randomized trials.[ 12 ] It can be used both by authors of such studies to improve the completeness of reporting and by reviewers and readers to critically appraise a systematic review. There are several extensions to the PRISMA statement for specific types of reviews. An update is currently underway.

Meta-analysis of Observational Studies in Epidemiology statement

The Meta-analysis of Observational Studies in Epidemiology statement summarizes the recommendations for reporting of meta-analyses in epidemiology.[ 13 ]

PROSPERO is an international database for prospective registration of protocols for systematic reviews in healthcare.[ 14 ] It aims to avoid duplication of and to improve transparency in reporting of results of such reviews.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

(Stanford users can avoid this Captcha by logging in.)

  • Send to text email RefWorks EndNote printer

Literature review and research design : a guide to effective research practice

Available online, at the library.

literature review research design

Education Library (Cubberley)

The Education Library is closed for construction. Request items for pickup at another library.

More options

  • Find it at other libraries via WorldCat
  • Contributors

Description

Creators/contributors, contents/summary.

  • Introduction
  • Acknowledgements
  • Part One: On Research
  • Chapter 1. Research Philosophy
  • Chapter 2. Research Practice
  • Part Two: Reading Literature
  • Chapter 3. Attitude
  • Chapter 4. Managing the Literature
  • Chapter 5. Deep Reading
  • Part Three: Writing About Literature
  • Chapter 6. Writing with Literature
  • Chapter 7. Writing a Literature Review
  • Select References, Annotated.
  • (source: Nielsen Book Data)

Bibliographic information

Browse related items.

Stanford University

  • Stanford Home
  • Maps & Directions
  • Search Stanford
  • Emergency Info
  • Terms of Use
  • Non-Discrimination
  • Accessibility

© Stanford University , Stanford , California 94305 .

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

  • Knowledge Base

Methodology

  • Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

  • Formulate a research question
  • Develop a protocol
  • Search for all relevant studies
  • Apply the selection criteria
  • Extract the data
  • Synthesize the data
  • Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Prevent plagiarism. Run a free check.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

  • A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
  • If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
  • Access to databases and journal archives. Often, your educational institution provides you with access.
  • Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
  • Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

  • They minimize research bias by considering all available evidence and evaluating each study for bias.
  • Their methods are transparent , so they can be scrutinized by others.
  • They’re thorough : they summarize all available evidence.
  • They can be replicated and updated by others.

Systematic reviews also have a few cons .

  • They’re time-consuming .
  • They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

  • Allow you to more effectively communicate your research to other researchers and practitioners
  • Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

  • Population(s) or problem(s)
  • Intervention(s)
  • Comparison(s)

You can rearrange these four components to write your research question:

  • What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

  • Type of study design(s)
  • The population of patients with eczema
  • The intervention of probiotics
  • In comparison to no treatment, placebo , or non-probiotic treatment
  • The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
  • Randomized control trials, a type of study design

Their research question was:

  • What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

  • Background information : Provide the context of the research question, including why it’s important.
  • Research objective (s) : Rephrase your research question as an objective.
  • Selection criteria: State how you’ll decide which studies to include or exclude from your review.
  • Search strategy: Discuss your plan for finding studies.
  • Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

  • Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
  • Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
  • Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
  • Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

  • Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
  • Handsearch: Conference proceedings and reference lists of articles
  • Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
  • Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

  • Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
  • Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

  • Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
  • Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

  • Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
  • Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

  • Abstract : A summary of the review
  • Introduction : Including the rationale and objectives
  • Methods : Including the selection criteria, search method, data extraction method, and synthesis method
  • Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
  • Discussion : Including interpretation of the results and limitations of the review
  • Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

  • Student’s  t -distribution
  • Normal distribution
  • Null and Alternative Hypotheses
  • Chi square tests
  • Confidence interval
  • Quartiles & Quantiles
  • Cluster sampling
  • Stratified sampling
  • Data cleansing
  • Reproducibility vs Replicability
  • Peer review
  • Prospective cohort study

Research bias

  • Implicit bias
  • Cognitive bias
  • Placebo effect
  • Hawthorne effect
  • Hindsight bias
  • Affect heuristic
  • Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved March 25, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, what is your plagiarism score.

In order to utilize all of the features of this web site, JavaScript must be enabled in your browser.

Logos Bible Software

Literature Review and Research Design: A Guide to Effective Research Practice

literature review research design

Designing a research project is possibly the most difficult task a dissertation writer faces. It is fraught with uncertainty: what is the best subject? What is the best method? For every answer found, there are often multiple subsequent questions, so it’s easy to get lost in theoretical debates and buried under a mountain of literature.

This book looks at literature review in the process of research design, and how to develop a research practice that will build skills in reading and writing about research literature—skills that remain valuable in both academic and professional careers. Literature review is approached as a process of engaging with the discourse of scholarly communities that will help graduate researchers refine, define, and express their own scholarly vision and voice. This orientation on research as an exploratory practice, rather than merely a series of predetermined steps in a systematic method, allows the researcher to deal with the uncertainties and changes that come with learning new ideas and new perspectives.

The focus on the practical elements of research design makes this book an invaluable resource for graduate students writing dissertations. Practicing research allows room for experiment, error, and learning, ultimately helping graduate researchers use the literature effectively to build a solid scholarly foundation for their dissertation research project.

This is a Logos Reader Edition. Learn more.

Key Features

  • Looks at literature review in the process of research design.
  • Allows the researcher to deal with the uncertainties that come with learning new ideas.
  • Helps graduate researchers use the literature effectively.

Praise for Literature Review and Research Design

Unlike other books on research, this book does not prescribe methods or recipes. Rather, it feels like one is sitting with an experienced dissertation coach, having a series of short conversations about the tacit knowledge that underlies the various aspects of research practice. After reading this book, novice researchers will have a better understanding of how the literature supports and brings out a researcher’s own voice

--Arnold Wentzel, author of Creative Research in Economics (Routledge, 2016) and A Guide to Argumentative Research Writing and Thinking (Routledge, 2017)

Product Details

  • Title : Literature Review and Research Design: A Guide to Effective Research Practice
  • Author : Dave Harris
  • Publisher : Routledge
  • Print Publication Date: 2020
  • Logos Release Date: 2023
  • Era: era:contemporary
  • Language : English
  • Resources: 1
  • Format : Digital › Logos Reader Edition
  • Subjects : Research › Methodology; Information resources › Evaluation; Dissertations, Academic
  • ISBNs : 9780367250379 , 9780429285660 , 0367250373 , 0429285663
  • Resource ID: LLS:LTRVWRSRCHPRCTC
  • Resource Type: Monograph
  • Metadata Last Updated: 2023-12-13T22:13:09Z

About Dave Harris

Dave Harris  is a writing coach who helps authors develop productive writing practices, using principles from design methods, philosophy of science, and cognitive science. With Jean-Pierre Protzen, he is author of  The Universe of Design  (2010, Routledge), and, alone, author of  Getting the Best of Your Dissertation  (2015, Thought Clearing).

Sign in with your Faithlife account

Book cover

Conference on Systems Engineering Research

CSER 2023: The Proceedings of the 2023 Conference on Systems Engineering Research pp 691–705 Cite as

Literature Review and Research Design for Systems Integration: Case Study in Defense Systems

  • Gaute Tetlie 5 ,
  • Gerrit Muller 5 &
  • Satyanarayana Kokkula 5  
  • Conference paper
  • First Online: 26 March 2024

Part of the book series: Conference on Systems Engineering Research Series ((CSERS))

This chapter analyses existing literature to identify an integration strategy suitable for a Norwegian defense contractor. Various types of unknowns cause uncertainties in the system design. These uncertainties manifest as problems discovered during later project phases. To mitigate such uncertainties, a criterion-driven integration strategy is suggested. Adding to this strategy, we recommend also identifying test-to-design areas. By doing so, uncertainties not directly captured by the chosen criterion may also be captured. Lastly, a research design with three iterations is recommended to validate the proposed integration strategy. This research shall be executed in Spring 2023, and the findings shall be published later.

  • Systems integration
  • Defense systems
  • Integration strategy
  • Research design
  • System quality

This is a preview of subscription content, log in via an institution .

D.D. Gransberg, J.E. Koch, K.R. Molennar, Preparing for Design-Build Projects: A Primer for Owners, Engineers, and Contractors (ASCE Press, 2006), pp. 1–10. ISBN: 978-0-7844-7139-5

Book   Google Scholar  

Q. Chen et al., Time and cost performance of design-build projects. J. Constr. Eng. Manag. 142 (2), 04015074 (2016). https://doi.org/10.1061/(ASCE)CO.1943-7862.0001056

Article   MathSciNet   Google Scholar  

T. Raz, A.J. Shenhar, D. Dvir, Risk management, project success, and technological uncertainty. R&D Manag., 101–109 (2002). https://doi.org/10.1111/1467-9310.00243

K.A. Kjeldaas, R.A. Haugen, E. Syverud, Challenges in detecting emergent behavior in system testing. INCOSE Int. Symp. 31 , 1211–1228 (2021). https://doi.org/10.1002/j.2334-5837.2021.00896.x

Article   Google Scholar  

G. Bammer, M. Smithson, Uncertainty and Risk: Multidisciplinary Perspectives (Earthscan, 2012), pp. 293–295. ISBN: 978-1-8497-7360-7

D. Snowden, Complex acts of knowing: Paradox and descriptive self-awareness. J. Knowl. Manag., 100–111 (2002). https://doi.org/10.1108/13673270210424639

N. Taleb, Silent risk: Lectures on fat tails, (anti)fragility, and asymmetric exposures. SSRN Electron. J., 31–35 (2014). https://statics.teams.cdn.office.net/evergreen-assets/safelinks/1/atp-safelinks.html

Google Scholar  

C.M. Martin, J.P. Sturmberg, General practice — Chaos, complexity and innovation. Med. J. Aust. 183 (2), 106–109 (2005). https://doi.org/10.5694/j.1326-5377.2005.tb06943.x

C.F. Kurtz, D. Snowden, The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Syst. J. 42 , 462–483 (2003). https://doi.org/10.1147/sj.423.0462

A. Sols, Systems Engineering Theory and Practice (ASCE Press, 2014), pp. 219–230. ISBN: 978-8-4846-8539-5

S. Sharma, D. Sarkar, D. Gupta, Agile processes and methodologies: A conceptual study. Int. J. Comput. Sci. Eng. 4 (5), 892–898 (2012)

G. Muller, System Integration How-To . URL: https://gaudisite.nl/SystemIntegrationHowToPaper.pdf

J.C. Mankins, Technology Readiness Levels . (1995), URL: https://aiaa.kavi.com/apps/group_public/download.php/2212/TRLs_MankinsPaper_1995.pdf

Department of Defense, Department of Defense Standard Practice: System Safety, in SAFT – System Safety E (May 2012)

M.E. Conway, How Do Committees Invent? (1968), pp. 28–31. URL: https: // hashingit.com/elements/research-resources/1968-04-committees.pdf

N. Slack, A. Brandon-Jones, R. Johnston, Operations Management (Pearson Books, 2013), pp. 495–521. ISBN: 978-0-273-77620-8

G. Langford, Engineering Systems Integration: Theory, Metrics, and Methods (Taylor & Francis Group, 2012), pp. 20–21. ISBN: 978-1-4398-5288-0

SEBoK, System Integration . URL: https://sebokwiki.org/wiki/System_Integration#Boundary_of_Integration_Activity

R. van Ommering, Software reuse in product populations. IEEE Trans. Softw. Eng. 31 (7), 537–550 (2005). https://doi.org/10.1109/TSE.2005.84

Download references

Author information

Authors and affiliations.

Systems Engineering, Univeristy of South-Eastern Norway, Kongsberg, Norway

Gaute Tetlie, Gerrit Muller & Satyanarayana Kokkula

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Stevens Institute of Technology, Hoboken, NJ, USA

Dinesh Verma

Astronautics Aerospace and Mechanical Engineering Department, University of Southern California, Los Angeles, CA, USA

Azad M. Madni

Endevor, Wilmington, DE, USA

Steven Hoffenson

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Tetlie, G., Muller, G., Kokkula, S. (2024). Literature Review and Research Design for Systems Integration: Case Study in Defense Systems. In: Verma, D., Madni, A.M., Hoffenson, S., Xiao, L. (eds) The Proceedings of the 2023 Conference on Systems Engineering Research. CSER 2023. Conference on Systems Engineering Research Series. Springer, Cham. https://doi.org/10.1007/978-3-031-49179-5_47

Download citation

DOI : https://doi.org/10.1007/978-3-031-49179-5_47

Published : 26 March 2024

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-49178-8

Online ISBN : 978-3-031-49179-5

eBook Packages : Engineering Engineering (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research

REVIEW article

Investigating machine learning and natural language processing techniques applied for detecting eating disorders: a systematic literature review.

Ghofrane Merhbene

  • Applied Machine Intelligence, Bern University of Applied Sciences, Biel/Bienne, Switzerland

Recent developments in the fields of natural language processing (NLP) and machine learning (ML) have shown significant improvements in automatic text processing. At the same time, the expression of human language plays a central role in the detection of mental health problems. Whereas spoken language is implicitly assessed during interviews with patients, written language can also provide interesting insights to clinical professionals. Existing work in the field often investigates mental health problems such as depression or anxiety. However, there is also work investigating how the diagnostics of eating disorders can benefit from these novel technologies. In this paper, we present a systematic overview of the latest research in this field. Our investigation encompasses four key areas: (a) an analysis of the metadata from published papers, (b) an examination of the sizes and specific topics of the datasets employed, (c) a review of the application of machine learning techniques in detecting eating disorders from text, and finally (d) an evaluation of the models used, focusing on their performance, limitations, and the potential risks associated with current methodologies.

1 Introduction

Recent reports in broad media about the latest conversational chatbots, which can generate human-like texts in response to user questions have made natural language processing (NLP) famous to the broad public. Yet the possibilities of this field go far beyond text generation and chatbots. Classifying texts into two (or more) groups and automatically extracting indicators that suggest that a text snippet belongs to either of the groups is also a common task. In particular, when using machine learning, this allows the identification of patterns that might differ from what a human might detect that are nonetheless effective in separating the two groups.

Meanwhile, in clinical practice in mental health, inventories with scaling questions are often used for diagnosis. Such inventories have limitations, including for example defensiveness (the denial of symptoms) or social bias that can influence the results of the questionnaires ( 1 ). In these cases, an automated text analysis applied to specific open questions or interview transcripts can provide further source of information indicating the patient’s condition that is more resistant to manipulations such as those arising from defensiveness.

Defensiveness is common amongst those afflicted with eating disorders (EDs). Respondents to a survey investigating the denial and concealment of EDs ( 2 ) reported a variety of attempts to hide the respective ED. Furthermore, the authors of the study state that such methods were described as deliberate strategies. This makes it challenging to use clinical instruments where an inventory item contains obvious indications for which options to choose in order to obtain a specific result.

EDs generally occur in the form of unhealthy eating habits, disturbances in behaviors, thoughts, and attitudes towards food, causing in some cases extreme weight loss or gain. These disorders not only impact mental health but also have physical effects ( 3 ). EDs are classified in the category F50 of the ICD-10 and can refer to different disorders including anorexia, bulimia or overeating 1 . A study conducted by Mohler-Kuo et al. ( 4 ) in Switzerland discovered that the lifetime prevalence for any ED is 3.5%. Another survey investigating the lifetime prevalence of EDs in English and French studies from 2000 to 2018 found that the weighted means were 8.4% for women, and 2.2% for men ( 5 ).

The power of natural language processing (NLP) has already been applied to the field of mental health, especially in research. Feelings and written expression are closely correlated: An analysis of student essays has shown that students suffering from depression use more negatively valenced 2 words and more frequently use the word “I” ( 6 ). Different approaches have been applied to explore how to use automated text analysis on tasks such as the detection of burnout ( 7 ), depression ( 8 , 9 ), the particular case of post-partum depression ( 10 , 11 ), anxiety ( 12 ), and suicide risk assessment ( 13 ), ( 14 ). Often, such methods are based on anonymized publicly available online data. Only little work makes use of clinical data. Furthermore, the English language has been the primary focus, even though these methods can be highly language-dependent, meaning that data and methods should be carefully reviewed when adapting to local languages. This is relevant, as it has been shown that adapting to the patient’s language is beneficial in mental health diagnostics and treatment ( 15 ). In our view, one aim of such technologies should be to explore ways to support clinical practitioners in their daily work, and provide them with additional sources of information to consider. Therefore, we often refer to such solutions as Augmented Intelligence 3 , rather than Artificial Intelligence, as they aim to empower humans rather than replacing them.

Despite existing work in the field of ML and NLP for depression, anxiety or suicide risk assessment, there has been a lack of a detailed systematic literature comparison on the automatic detection of EDs using NLP technologies for both clinical and non-clinical data. A recent survey ( 16 ) investigated the use of natural language processing applied to mental illness detection. The majority of the identified results (45%) had worked on depression, whereas only 2% were about eating disorders in general and 3% about anorexia. Whereas the broad scope of the survey provides a generous overview of the research landscape, it does not compare the case of eating disorders in detail.

In this paper, we have undertaken a systematic literature review to address this research gap, following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines ( 17 ) to ensure a well-structured and transparent methodology.

We contribute to the field by (a) analyzing the metadata of published papers to understand the current trends and methodologies, (b) examining the sizes and targeted topics of the datasets used in these studies, (c) reviewing how machine learning techniques are applied to detect eating disorders from textual data, and (d) evaluating the performance, limitations, and potential risks of the models deployed in this domain.

Our research is guided by specific questions, structured around four distinct perspectives, which collectively form the core of our investigative approach.

● Demographical Questions (DemRQ): Focus on metadata aspects of the paper:

● • DemRQ1: When was the paper published?

● • DemRQ2: From which countries were the contributors of the papers included in this study?

● Input Questions (InputRQ): Focus on the format and topic of the input data:

● • InputRQ1: Which languages were taken into consideration?

● • InputRQ2: What was the size of the dataset used?

● • InputRQ3: Which data sources were used for data collection in the case of both clinical and non-clinical data?

● • InputRQ4: What types of eating disorders were addressed in these studies?

● Architectural Questions (ArchRQ): Focus on the experimental architecture:

● • ArchRQ1: Which feature extraction technique was used?

● • ArchRQ2: Which machine learning techniques in the field of NLP have been used for ED detection?

● Evaluation Questions (EvalRQ): Focus on the evaluation aspects of the trained model:

● • EvalRQ1: How did the model perform?

● • EvalRQ2: What are the limitations and risks of the existing methods, and how can they be improved?

The article is structured as follows: First, we describe our methodology such as the study design and the paper selection process. We then describe the results of the literature search and describe the findings of our review. Finally, we summarize our results and describe perspectives for future research in the field.

2.1 Study design

To answer our research questions, we conducted a structured literature review (SLR) following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines ( 17 ). This includes standards for literature search strategies and setting criteria for the inclusion or exclusion of gathered works in the final review.

2.2 Literature search strategy

In accordance with PRISMA standards, we have set an 8-year time span for searching for documents (2014-2022) related to our research scope. We consider the year 2014 mainly because Bellows et al. ( 18 ) conducted a study on automatically detecting binge Eating disorder using clinical data, which we deem to be the initial research in the field. We then compiled a list of all databases to be searched. The list included the following databases:

● Google Scholar

● IEEE Xplore

● Pubmed

In addition, in order to efficiently conduct our database search we have compiled a list of keywords and conditions. These keywords are relevant to the research topic of EDs and their detection using NLP and machine learning techniques. Furthermore, the list included specific terms related to social media and online social networks in order to enable the identification of studies that explore the use of social media for the early detection of EDs, which is an ongoing research interest. The final query is presented below:

(eating disorder OR anorexia OR binge eating OR bulimia OR overeating) AND (natural language processing OR NLP OR text mining OR inventories OR machine learning OR artificial intelligence OR automatic detection OR early detection OR social media OR online social network OR clinical).

Using the aforementioned search keywords and conditions, we retrieved research articles where NLP techniques have been used for the detection of EDs from clinical and non-clinical data. The detailed workflow is depicted in Figure 1 , and the corresponding PRISMA flow diagram for this SLR is shown in Figure 2 .

www.frontiersin.org

Figure 1 Methodology for document collection.

www.frontiersin.org

Figure 2 PRISMA Flow diagram. Based on: Page et al., ( 17 ).

With the initially proposed search query, a large number of papers was identified. With manual analysis we explored options to define a more restrictive query, still making sure to capture the relevant papers, which turned out challenging. We therefore adapted our method to consider the first 100 elements returned by the search query on each database, sorted by relevance. This furthermore allowed to apply the same methodology for all three data sources, including especially Google Scholar, where the search functionalities are limited compared to databases like PubMed, and thus we had to make a selection on the number of items to be reviewed. Given the interidisciplinarity of our approach, we wanted to include Google Scholar to target a vast number of sources and ensure the most relevant work can be included.

A Python script was used to screen the articles for duplicates. As a result, 1 article was excluded from further consideration, leaving a total of 299 articles for further analysis (see Figure 2 ). To refine the results further, a manual title scan was performed to exclude articles that were not pertinent to the research topic. This resulted in the exclusion of 237 articles, leaving a total of 62 for further analysis. Additionally, a manual scan of the abstracts from the remaining 62 articles was performed to exclude any that were not relevant to the study. This process resulted in the exclusion of an additional 30 articles, leaving a total of 32 for inclusion in the final analysis. After thoroughly reading and evaluating 32 articles, 27 were selected as relevant for the researched topic (according to the criteria from Table 1 ). These chosen articles were deemed to possess high relevance and reliability for this SLR. Finally, we scanned the references section of the articles included in our survey and identified any relevant literature that may have been missed in the initial database search. This added n=18 articles to the studies that were finally included in the review (n=45). The process is illustrated in Figure 2 .

www.frontiersin.org

Table 1 SLR study selection of literature using inclusion and exclusion criteria.

2.3 Inclusion and exclusion criteria

Table 1 outlines the predefined exclusion and inclusion criteria that were used to guide the selection of related studies for the review. These criteria were established in advance to help simplify the process of identifying and selecting relevant papers. In particular, papers that focused solely on the psychological aspects of EDs and did not consider the use of automated text analysis technologies were excluded from the review. By adhering to these criteria, we were able to more effectively and efficiently select the relevant papers.

In this section, we provide a thorough review and analysis of the research studies included in this systematic literature review.

3.1 Terminology

● Bag of Words (BoW) is a fundamental technique used in NLP for text representation. It involves representing text data by counting the frequency of occurrence of each word in a document.

● Term Frequency-Inverse Document Frequency (TF-IDF) is a numerical statistic used to evaluate the importance of a word in a document within a collection or corpus. It combines two metrics: term frequency (TF), which measures the frequency of a word in a document, and inverse document frequency (IDF), which penalizes words that are common across the entire corpus.

● Bidirectional Encoder Representations from Transformers (BERT) ( 19 ) is a pretrained deep learning model introduced by Google in 2018. It belongs to the Transformer architecture and is designed to understand the context of words in a sentence by considering both left and right context simultaneously

● Word2Vec ( 20 ) is a technique for learning word embeddings. Word2Vec represents each word as a vector, with similar words having vectors that are closer together in the vector space.

● Global Vectors for Word Representation (GloVe) ( 21 ) is another technique for learning word embeddings. GloVe also generates vector representations of words based on their co-occurrence statistics in a corpus. However, GloVe considers the global context of the entire corpus to learn word embeddings, unlike Word2Vec, which focuses on local context.

● Embeddings from Language Models (ELMO) ( 22 ) is a deep contextualized word representation model. It generates word embeddings by considering the entire input sentence and capturing its contextual information.

● Doc2Vec ( 23 ) also known as Paragraph Vector, is an unsupervised learning algorithm to generate vector representations for pieces of texts like sentences and documents, it extends the Word2Vec methodology to larger blocks of text, capturing the context of words in a document.

● Bidirectional Long Short-Term Memory (Bi-LSTM) ( 24 ) is a type of Recurrent Neural Network (RNN) that processes data in both forward and backward directions. This architecture is particularly effective in understanding the context in sequence data like text or time series, as it captures information from both past (backward) and future (forward) states.

● Linguistic Inquiry and Word Count (LIWC) ( 25 ) is a text analysis program that counts words in psychologically meaningful categories.

3.2 Demographical research questions

Figure 3 shows the yearly distribution of the selected research work (DemRQ1). The data suggests a growing interest in this topic in recent years. This is in line with the findings of Zhang et al. ( 16 ) that found that there has been an upward trend over the last years in using NLP and machine learning methods to detect mental health problems. Notably, we highlight a prominent peak in 2018 and 2019, which coincides with the emergence of tasks related to EDs in eRisk competitions.

www.frontiersin.org

Figure 3 Yearly distribution of all research articles.

We also observed the geographical distribution of the authors’ affiliations of the selected studies (DemRQ2). As visualized in the heat-map in Figure 4 , 7 of the selected studies were from the USA and Spain, 5 from Mexico and France.

www.frontiersin.org

Figure 4 Geographic distribution of all institutions involved in the selected research articles.

From the 45 selected studies, 24 were results from the eRisk lab 4 , hosted by the CLEF Conference since 2017. This academic research competition focuses on the development and evaluation of text-based risk prediction models for social media. Each year, the lab provides a shared task framework where teams of participants are tasked with developing NLP techniques to automatically identify and predict the risk of different mental illness behaviors from social media data, including Eating Disorders. Participants are provided with a training dataset and a test dataset, and the performance of their models is evaluated based on two categories: performance and latency. The eRisk lab provides a unique opportunity for researchers to collaborate and innovate in the field of NLP and mental health, aiming to improve the detection and prevention of mental health issues in online communities. The datasets used in the eRisk lab are primarily sourced from the social media platform Reddit.

Since 2017, the challenge has included two tasks pertaining to the early detection of Eating Disorders. In both 2018 and 2019, the task involved the early detection of signs of anorexia [see e.g., Losada et al. ( 26 )]. In contrast, the 2022 iteration introduced a novel task centered on measuring the severity of eating disorders ( 27 ). This task diverged from the previous ones in that no labeled training data was supplied to participants, meaning that participants could not evaluate the quality of their models’ predictions until test time. The task objective was to assess a user’s level of eating disorder severity through analysis of their Reddit posting history. In order to achieve this, participants were required to predict users’ responses to a standard eating disorder questionnaire (EDE-Q) 5 ( 28 ).

3.3 Input research questions

Our first input research question (InputRQ1) investigates the different languages that are considered in the studies included in this SLR. Research has shown that only a small number of the over 7000 languages used worldwide are represented in recent technologies from the field of natural language processing ( 29 ). We wanted to investigate whether this is also the case for the detection of eating disorders. Text analysis, naturally, depends on the specific language and can typically not be transferred from one language to another without specific adaptions.

Table 2 gives indication about the language of data used, its size, its source, and the type of eating disorder that was investigated in the selected studies (excluding studies from eRisk). 18 of the 21 studies used English data, 2 used Polish and 1 Spanish data. The 24 papers from the eRisk lab challenges all relied on English data from the platform Reddit. Overall, only 3 out of 45 studies used a language other than English (7%). This confirms the need for further work in applying the latest technological developments to non-English texts.

www.frontiersin.org

Table 2 Datasets characteristics.

The dataset size is another crucial factor we took into account in our analysis ((InputRQ2). As depicted in Figure 5 , the distribution of dataset sizes used in the studies reveals that datasets ranging from 1k to 10k instances are the most frequently used.

www.frontiersin.org

Figure 5 Dataset sizes distribution based on Table 2 excluding articles from eRisk.

The distribution of dataset sizes across different research topics, as illustrated in Figure 6 , offers insightful perspectives. Notably, Anorexia research displays the most significant variance in dataset sizes, spanning from less than 1K to over 1 million data points. In contrast, binge eating research predominantly employs datasets within a narrower range of 1K to 10K data points. For broader Eating Disorders, 6 studies leverage datasets between 10K and 100K, while 3 others operate with datasets in the 100K to 1 million range. Finally, research on Mental Disorders encompasses datasets varying from 1K to more than 1 million data points.

www.frontiersin.org

Figure 6 Dataset sizes distribution by targeted ED based on Table 2 excluding articles from eRisk.

Table 2 also gives an overview of the data sources (InputRQ3). From the 45 studies, the used datasets can be classified as follows in four groups:

● eRisk lab datasets: 24 studies

● Other online forums and social media: 17

● Medical data: 3

● SMHD dataset ( 50 ): 1

The distribution of the primary focus of these studies is illustrated in Figure 7 (InputRQ4) The majority of the studies (n=29) we collected focused on anorexia, while 12 studies conducted a broader investigation of EDs in general rather than focusing on a specific type. Additionally, three studies had a more extensive scope, delving into various mental disorders, including but not limited to EDs, while one study focused on binge eating.

www.frontiersin.org

Figure 7 Research distribution of all research articles.

3.4 Architectural and evaluations research questions

3.4.1 erisk challenge.

Table 3 summarizes all the papers that we identified following our strategy, including the ones from eRisk. In 2018 and 2019, the eRisk papers focused on a text classification task aimed at developing an early detection system for eating disorders on social media using the history of users’ writings data. The aim was to train a text classifier that could effectively identify and flag potential cases of anorexia based on users’ social media content. For the eRisk challenge resulting in papers from 2022, the task was different. Participants were provided with the social media history of specific users and had to predict their answers to questions 1-12 and 19-28 from the Eating Disorder Examination Questionnaire (EDE-Q) 7 ( 28 ).

www.frontiersin.org

Table 3 Overview of machine learning methods and performance metrics of the studies included in this systematic literature review.

(ArchRQ1) The complexity of this task, along with the development in the field of NLP over the years 2019 to 2022, explains the choice of word2vec, GloVe ( 72 ) or transformer-based models ( 62 , 66 , 73 ) for vectorization/feature representation. For the remaining entries, very different approaches were used, ranging from anorexia specific vocabulary and LIWC ( 58 ) to more general approaches like Bag of Words (BoW) ( 52 , 53 ) or TF-IDF ( 51 , 57 ). (ArchRQ2) The choices of methods for prediction were also heterogeneous, ranging from cosine similarity ( 72 ) to linear models ( 52 , 54 , 58 , 66 , 71 ), to neural networks ( 51 , 53 , 56 ).

(EvalRQ1) For the 2018-2019 eRisk papers, we report F1 values corresponding to the binary classification task, whereas for the 2022 paper we report mean average error (MAE), corresponding to the average deviation between user’s predicted questionnaire responses and the ground truth responses.

3.4.2 Non-eRisk studies

Table 3 shows the feature representation, tasks studied, machine learning techniques, and performance metrics of all studies included in this SLR. In this section we focus on Non-eRisk studies. We grouped these studies into the following categories with regard to the feature extraction techniques they apply (ArchRQ1):

● Bag of Words (BoW)

● Word embeddings

● TF-IDF

● BERT representations

● and other feature representations

Furthermore, it is worth noting that the machine learning methods used in these studies span various categories (ArchRQ2), including:

● Classical machine learning (ML) methods such as Support Vector Machine (SVM), Naive Bayes, Logistic Regression, etc.

● Deep learning (DL) methods, e.g., recurrent neural networks.

● Combination of different methods from classical ML and DL.

● Large language models (LLMs), e.g., BERT.

● Other approaches.

Additionally, the tasks addressed in these studies can be broadly grouped into categories such as:

● Classification

● Topic modeling

● Sentiment analysis

In terms of feature extraction techniques employed across the 21 studies, a variety of methods were utilized. Among these, three studies ( 33 , 46 , 78 ) relied on TF-IDF. Four studies, including Zhang et al. ( 16 ) Benítez-Andrades et al. ( 38 ) Villegas et al. ( 48 ), and Jiang et al. ( 44 ), opted for BERT representations. Notably, Jiang et al. ( 44 ) combined BERT with LIWC.

Moreover, Bag of Words (BoW) and various types of Word Embeddings, including GloVe ( 35 , 48 ), FastText ( 35 ), and Word2Vec ( 35 , 36 ), were widely employed as feature extraction techniques in these studies.

It is pertinent to note that some studies, like Chancellor et al. ( 79 ) and Benítez-Andrades et al. ( 38 ), did not provide comprehensive details on this aspect in their papers. Conversely, other articles adopted a more personalized approach to construct their features. For instance, some represented each data point as a vector within certain categories ( 39 , 40 ), while others used rule-based methods ( 18 ) or leveraged algorithms like decision trees ( 41 ) and topic modeling ( 42 ) to determine feature selection.

Our results show that from the 21 studies, 8 make use of classical machine learning methods, 1 uses deep learning, 5 use a combination of classical ML and DL, 4 use large-language models and 3 use other approaches.

When using classical machine learning, some studies compare different methods. For example, López Úbeda et al. ( 33 ) apply 5 different supervised machine learning models: SVM, multilayer peceptron classifier, naive bayes, decision tree and logistic regression, and Villegas et al. ( 48 ) compare naive bayes, random forest, logistic regression and SVM. Along with the classical machine learning methods, the studies apply different feature representations ranging from Bag of Words (BoW) to TF-IDF ( 33 , 78 ), up to contextualized embeddings such as BERT ( 48 ).

Other studies compared both classical machine learning as well as deep learning methods. For example, in the case of Tébar and Gopalan ( 42 ), a so-called feature fusion model that includes both deep learning (a convolutional neural network (CNN) and a BiGRU model), as well as a classical machine learning model (logistic regression classifier with handcrafted features) is used.

For the studies using transformer-based large language models, different models including the BERT ( 19 ) model and its variations have been used. For example, Benítez-Andrades et al. ( 32 ) applied five variations of the BERT model. The paper from Dinu and Moldovan ( 43 ) uses BERT, RoBERTa and XLNET, whereas Jiang et al. ( 44 ) use BERT and REALM. The work from Zhang et al. ( 45 ) focusing on different mental illnesses used the BERT model, as well as the MBERT variation.

(EvalRQ1) The performance of each study is also reported in Table 3 .

(EvalRQ2) Finally, we investigated the limitations of the proposed studies (RQ4) in order to provide a structured outlook for future work in the field.

In many cases, there were limitations in terms of the datasets. For example, Yan et al. ( 78 ) cites the limited availability of labeled data. They used a dataset of 50 posts, which they expect to be labeled correctly. Also Zhou et al. ( 34 ) mention that their study is limited by the number of collected tweets, which may result in some irrelevant topics arising from noise for their topic modeling task.

In many studies, social media data is used. The nature of such data is seen as a potential limitation for the resulting methods ( 37 ). Other studies indicated as a limitation that only one social media platform was used to gather their data ( 38 , 42 ). For example, a study from ( 35 ) points out that their work did not take into account the potential biases in the data that may exist, such as underrepresented population or lack of diverse perspectives. In addition, one of the notable constraints arises from the fundamental disparity between social media data and traditional clinical text data, often used in healthcare and medical research. Clinical records encompass detailed information on patients’ medical histories, diagnoses, treatments, and outcomes, rendering them fundamentally distinct from the informal, user-generated content prevalent on social media platforms. Several studies point out that the involvement of clinical professionals would be beneficial. For example, Choudhury ( 30 ) states that their method could be more successful with the involvement of clinicians.

Different studies rely on anonymous data, which makes it difficult to ensure a good distribution within the training data over different populations and underrepresented groups. For example, Ragheb et al. ( 62 ) sees potential to optimize the model for different use cases and populations. Manual labeling by humans is also considered a source of bias since limited information about the users writing them is available to the annotators. This limited information may not encompass the full context of the users’ lives, beliefs, or backgrounds. Annotators may make subjective judgments based solely on the content of the post, which can be influenced by their own biases and interpretations. Thus, limited context can lead to misinterpretations or mislabeling, potentially distorting the research results ( 38 ).

In the limitations, it is also discussed how texts written by laypeople and ED promotional 8 and educational materials can be hard to classify ( 34 ). This can be partly explained by the short length of texts, for example in the case of tweets, and the semantic similarity of the two types of texts.

Whereas many studies achieved good performance in terms of accuracy or f1-scores, they see a potential limitation in this matter. For example, Wang et al. ( 40 ) discusses that the validation was done only with a small sample of the data, and thus further validation is required with larger samples. In another study, the authors were concerned about the problem of overfitting ( 52 ).

4 Discussion

In this systematic literature survey we have discussed the use of machine learning and natural language processing methods for the detection of eating disorders. Our survey was conducted using the PRISMA framework ( 17 ). Our results have shown that many studies focus on the detection of anorexia, or eating disorders in general (see Figure 7 ). We have also seen that there was more work over the last couple of years, indicating a growing interest in the topic (as shown in Figure 3 ). Whereas most publications were from institutions in the USA and Spain, work from other countries including Mexico, France and Canada was also identified, as shown in Figure 4 . Nevertheless, our work has shown that most research efforts have only been applied to the English language. Given the relevance of local languages for mental health diagnostics and treatment ( 15 ), it is thus necessary for future research to address other languages. With regard to the machine learning and feature extraction methods being applied, a comparison turned out to be challenging due to the diverse nature of the datasets and approaches used. The proposed approaches were classified into different categories, including classical machine learning, deep learning, a combination of classical and deep learning, the use of large language models, as well as other approaches. Several studies used f1-score as a common measure, reaching different performances ranging from 0.67 to 0.93. Overall, having a sufficient data quality and quantity was often seen as a major limitation of the approaches. Since 2017, the eRisk challenge has included two tasks pertaining to the early detection of Eating Disorders. In both 2018 and 2019, the task involved the early detection of signs of anorexia [see e.g., Losada et al. ( 26 )]. In contrast, the 2022 iteration introduced a novel task centered on measuring the severity of eating disorders ( 27 ). This task diverged from the previous ones in that no labeled training data was supplied to participants, meaning that participants could not evaluate the quality of their models’ predictions until test time. The objective task was to assess a user’s level of eating disorder severity through analysis of their Reddit posting history.

Given the composition of both the eRisk lab and the SMHD dataset ( 50 ) predominantly with social media data, it is notable that an overwhelming majority (93%) of the studies in our analysis employ this data type. This underscores the widespread reliance on social media sources in modern research methodologies. This finding confirms the results of Zhang et al. ( 16 ) who found that among 399 papers applying NLP methods for the identification of mental health problems, 81% consisted of social media data.

It is worth mentioning that we came across two types of use cases in the studies. Many studies focus on the individual’s expression of their behavior and feelings with regard to eating disorders. Some studies, namely Choudhury ( 30 ) and Chancellor et al. ( 49 ), investigate the wording of pro-anorexia or pro-eating disorders communities on social media and online forums. Such communities promote disordered eating habits as acceptable alternative lifestyles ( 49 ). Whereas in many of the studies the technologies target support for clinical professionals, in these cases other applications such as content moderation are in the foreground.

In the realm of data collection for eating disorder research, manual labeling of datasets has been a common approach, with various strategies employed. For instance, Zhang et al. ( 45 ) relied on the voluntary efforts of 31 individuals to meticulously annotate 8554 data points encompassing 38 symptoms related to MD (Mental Disorders). Other studies took different routes, combining expert knowledge with input from non-expert annotators 9 ( 38 ), or solely relying on domain experts ( 46 ). In some cases, researchers have employed machine learning algorithms to automatically annotate their datasets and subsequently validated the results with input from human labelers ( 44 ). The majority of datasets underwent annotation by non-expert human annotators, as seen in studies conducted by ( 79 , 40 , 34 , 41 ).

Our review revealed few instances of Large Language Models (LLMs) application ( 10 , 11 , 19 , 30 , 38 , 43 , 44 , 45 , 49 , 50 , 61 , 67 , 73 , 74 , 79 , 80 ). Despite this, the rising adoption of technologies like MentalBERT ( 77 ) and MentaLLama ( 81 ), alongside traditional machine and deep learning approaches, is notable. This trend, driven by the impressive efficacy of LLMs in natural language processing, is expected to continue on. As these technologies evolve and become more accessible, we anticipate their increased utilization in this field of research, enhancing computational model accuracy and efficiency.

Based on the identified limitations in the selected studies, we infer the following focus topics that we suggest for future work in the field of using natural language processing and machine learning in ED research:

● Data Quantity and Quality: how can more high-quality data be created and shared, while respecting the ethical and privacy limitations of such sensitive data?

● Involvement of Clinical Professionals: how can machine learning engineers and clinical professionals work together more closely?

● More Diversity in Data: How can the diversity of the population in the used datasets be increased to avoid bias in the classification?

● Local Languages: How can the proposed methods be extended to local languages other than English?

In conclusion, based on the studies investigated in this literature survey, there is potential for further development and in the long-term a novel tool support for clinical professionals based on text data.

Author contributions

GM: Formal analysis, Writing – review & editing, Writing – original draft, Visualization, Investigation, Data curation. AP: Formal analysis, Writing – review & editing, Writing – original draft, Validation, Supervision, Methodology, Conceptualization. MK-B: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing.

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The authors gratefully acknowledge the support of the Inventus Bern Foundation for our research in the field of augmented intelligence for the detection of eating disorders.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

  • ^ https://icd.who.int/browse10/2019/en#/F50
  • ^ Valence is a measure of the emotional intensity or positivity/negativity associated with a word.
  • ^ See e.g., https://digitalreality.ieee.org/publications/what-is-augmented-intelligence
  • ^ https://erisk.irlab.org/
  • ^ https://www.corc.uk.net/media/1273/ede-qquesionnaire.pdf
  • ^ A content or an activity that promotes or encourages eating disorders (EDs).
  • ^ individuals who lack specialized domain knowledge or expertise in the subject matter.

1. Williams CL, Butcher JN, Paulsen JA. 13 - overview of multidimensional inventories of psychopathology with a focus on the mmpi-2. In: Goldstein G, Allen DN, DeLuca J, editors. Handbook of Psychological Assessment , 4th ed. Academic Press, San Diego (2019). 397–417. doi: 10.1016/B978-0-12-802203-0.00013-4

CrossRef Full Text | Google Scholar

2. Vandereycken W, Van Humbeeck I. Denial and concealment of eating disorders: a retrospective survey. Eur Eating Disord Review: Prof J Eating Disord Assoc . (2008) 16:109–14.

Google Scholar

3. Smink FR, van Hoeken D, Hoek HW. Epidemiology, course, and outcome of eating disorders. Curr Opin Psychiatry . (2013) 26:543–8. doi: 10.1097/yco.0b013e328365a24f

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Mohler-Kuo M, Schnyder U, Dermota P, Wei W, Milos G. The prevalence, correlates, and help-seeking of eating disorders in Switzerland. psychol Med . (2016) 46:2749–58. doi: 10.1017/S0033291716001136

5. Galmiche M, Déchelotte P, Lambert G, Tavolacci MP. Prevalence of eating disorders over the 2000–2018 period: a systematic literature review. Am J Clin Nutr . (2019) 109:1402–13.

PubMed Abstract | Google Scholar

6. Rude S, Gortner E-M, Pennebaker J. Language use of depressed and depression-vulnerable college students. Cogn Emotion . (2004) 18:1121–33.

7. Merhbene G, Nath S, Puttick AR, Kurpicz-Briki M. Burnoutensemble: Augmented intelligence to detect indications for burnout in clinical psychology. Front Big Data . (2022) 4.

8. Moreno MA, Jelenchick LA, Egan KG, Cox E, Young H, Gannon KE, et al. Feeling bad on facebook: Depression disclosures by college students on a social networking site. Depress. Anxiety . (2011) 28:447–55. doi: 10.1002/da.20805

9. Schwartz HA, Eichstaedt J, Kern ML, Park G, Sap M, Stillwell D, et al. (2014). “Towards assessing changes in degree of depression through facebook,” In: Resnik P, Resnik R, Mitchell M. editors. Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: from linguistic Signal to Clinical Reality, (Baltimore, Maryland, USA).

10. De Choudhury M, Counts S, Horvitz E. “Predicting postpartum changes in emotion and behavior via social media,” In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery. (2013). pp. 3267–76. doi: 10.1145/2470654.2466447

11. De Choudhury M, Counts S, Horvitz EJ, Hoff A. “Characterizing and predicting postpartum depression from shared facebook data”. In: Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing. New York, NY, USA: Association for Computing Machinery). 2014). pp. 626–38. doi: 10.1145/2531602.2531675

12. Tadesse MM, Lin H, Xu B, Yang L. Detection of depression-related posts in reddit social media forum. IEEE Access . (2019) 7:44883–93. doi: 10.1109/ACCESS.2019.2909180

13. Morales M, Dey P, Theisen T, Belitz D, Chernova N. “An investigation of deep learning systems for suicide risk assessment.” In: Niederhoffer K, Hollingshead K, Resnik P, Resnik R, Loveys K editors Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology . Minneapolis, Minnesota: Association for Computational Linguistics (2019). p. 177–81. doi: 10.18653/v1/W19-3023

14. Just MA, Pan L, Cherkassky VL, McMakin DL, Cha C, Nock MK, et al. Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth. Nat Hum Behav . (2017) 1:911–9. doi: 10.1038/s41562-017-0234-y

15. Griner D, Smith TB. Culturally adapted mental health intervention: A meta-analytic review. Psychotherapy: Theory research practice Training . (2006) 43:531.

16. Zhang T, Schoene AM, Ji S, Ananiadou S. Natural language processing applied to mental illness detection: a narrative review. NPJ digital Med . (2022) 5:46.

17. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The prisma 2020 statement: An updated guideline for reporting systematic reviews. BMJ . (2021) 372. doi: 10.1136/BMJ.N71

18. Bellows BK, LaFleur J, Kamauu AWC, Ginter T, Forbush TB, Agbor S, et al. Automated identification of patients with a diagnosis of binge eating disorder from narrative electronic health records. J Am Med Inform Assoc . (2014) 21(e1):e163–8. doi: 10.1136/amiajnl-2013-001859

19. Devlin J, Chang M, Lee K, Toutanova K. “BERT: pre-training of deep bidirectional transformers for language understanding.” In: Burstein J, Doran C, Solorio T, editors. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019 , vol. 1 (Long and Short Papers). Association for Computational Linguistics (2019). p. 4171–86. doi: 10.18653/V1/N19-1423

20. Mikolov T, Chen K, Corrado G, Dean J. “Efficient estimation of word representations in vector space.” In: International Conference on Learning Representations . (2013).

21. Pennington J, Socher R, Manning C. GloVe: Global vectors for word representation. In: Moschitti A, Pang B, Daelemans W, editors. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Doha, Qatar (2014). p. 1532–43. doi: 10.3115/v1/D14-1162

22. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, et al. “Deep contextualized word representations.” In Walker M, Ji H, Stent A editors. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long Papers). New Orleans, Louisiana: Association for Computational Linguistics (2018) 2227–37. doi: 10.18653/v1/N18-1202

23. Le QV, Mikolov T. “Distributed representations of sentences and documents.” In: Xing EP, Jebara T, editors. Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research . Beijing, China: PMLR (2014) 32(2):1188–96. Available at: http://proceedings.mlr.press/v32/le14.pdf .

24. Schuster M, Paliwal K. Bidirectional recurrent neural networks. IEEE Trans Signal Process . (1997) 45:2673–81. doi: 10.1109/78.650093

25. Pennebaker JW, Francis ME, Booth RJ. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates (2001) 71(2001):2001.

26. Losada DE, Crestani F, Parapar J. “Overview of eRisk: Early Risk Prediction on the Internet.” In: Bellot P, Trabelsi C, Mothe J, Murtagh F, Nie JY, Soulier L, et al editors. Experimental IR Meets Multilinguality, Multimodality, and Interaction. Cham: Springer International Publishing. (2018) 343–61.

27. Parapar J, Martín-Rodilla P, Losada DE, Crestani F. Overview of eRisk 2022: Early risk prediction on the internet. In: Experimental IR Meets Multilinguality, Multimodality, and Interaction: 13th International Conference of the CLEF Association, CLEF 2022, Bologna, Italy, September 5–8, 2022, Proceedings . Springer (2022). p. 233–56.

28. Fairburn CG, Beglin SJ. Eating disorder examination questionnaire (ede-q) Database record, APA PsycTests. (1994).

29. Joshi P, Santy S, Budhiraja A, Bali K, Choudhury M. “The state and fate of linguistic diversity and inclusion in the nlp world.” In: Jurafsky D, Chai J, Schluter N, Tetreault J, editors. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020) 6282–6293. doi: 10.18653/v1/2020.acl-main.560

30. De Choudhury M. Anorexia on tumblr: A characterization study. In: Proceedings of the 5th International Conference on Digital Health 2015 . New York, NY, USA: Association for Computing Machinery (2015) 43–50. doi: 10.1145/2750511.2750515

31. Yan H, Fitzsimmons-Craft E, Goodman M, Krauss M, Das S, Cavazos-Rehg P. Automatic detection of eating disorder-related social media posts that could benefit from a mental health intervention. Int J Eating Disord . (2019) 52:1150–6. doi: 10.1002/eat.23148

32. Benítez-Andrades JA, Alija-Pérez JM, García-Rodríguez I, Benavides C, Alaiz-Moretón H, Vargas RP, et al. BERT model-based approach for detecting categories of tweets in the field of eating disorders (ED). In: 2021 IEEE 34th International Symposium on Computer-Based Medical Systems (CBMS) (New York, USA: IEEE) (2021). p. 586–90. Available at: https://api.semanticscholar.org/CorpusID:236095644 .

33. López Úbeda P, Plaza del Arco FM, Díaz Galiano MC, Urena Lopez LA, Martin M. “Detecting anorexia in Spanish tweets.” In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019). Varna, Bulgaria: INCOMA Ltd. (2019) 655–63. doi: 10.26615/978-954-452-056-4077

34. Zhou S, Zhao Y, Bian J, Haynos AF, Zhang R. Exploring eating disorder topics on twitter: Machine learning approach. JMIR Med Inform. (2020) 8(10):e18273. doi: 10.2196/18273

35. Aguilera J, Hernández Farías DI, Ortega-Mendoza RM, Montes-y-Gómez M. Depression and anorexia detection in social media as a one-class classification problem. Applied Intelligence . (2021) 51:6088–103. doi: 10.1007/s10489-020-02131-2

36. Spinczyk D, Bas M, Dzieciatko M, Maćkowski M, Rojewska K, Maćkowska S. Computer-aided therapeutic diagnosis for anorexia. BioMed Eng OnLine (2020) 19:53. doi: 10.1186/s12938-020-00798-9

37. Aragón ME, López-Monroy AP, González-Gurrola LC, Montes-y-Gómez M. Detecting mental disorders in social media through emotional patterns-the case of anorexia and depression. EEE Transactions on Affective Computing . (2021) 14(1):211–22. doi: 10.1109/TAFFC.2021.3075638

38. Benítez-Andrades JA, Alija-Pérez J-M, Vidal M-E, Pastor-Vargas R, Vidal ME, García-Ordás T. Traditional machine learning models and bidirectional encoder representations from transformer (BERT)-based automatic classification of tweets about eating disorders: Algorithm development and validation study. JMIR Medical Informatics (2022) 10(2):e34492. doi: 10.2196/34492

39. Ramiandrisoa F, Mothe J. Early Detection of Depression and Anorexia from Social Media: A Machine Learning Approach. In: Cantador I, Chevalier M, Melucci M, Mothe J, editors. Circle 2020 , vol. 2621 . Proceedings of the Conference CIRCLE 2020, Samatan, France (2020).

40. Wang T, Brede M, Ianni A, Mentzakis E. “Detecting and characterizing eating-disorder communities on social media.” In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM '17) . New York, NY, USA: Association for Computing Machinery (2017) 91–100. doi: 10.1145/3018661.3018706

41. He L, Luo J. “What makes a pro eating disorder hashtag: Using hashtags to identify pro eating disorder tumblr posts and Twitter users.” In: 2016 IEEE International Conference on Big Data (Big Data) . IEEE (2016). 3977–9. doi: 10.1109/BigData.2016.7841081

42. Tébar B, Gopalan A. “Early Detection of Eating Disorders using Social Media.” 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) , Washington, DC, USA (2021). pp. 193–8. doi: 10.1109/CHASE52844.2021.00042.

43. Dinu A, Moldovan A-C. “Automatic detection and classification of mental illnesses from general social media texts.” In: Mitkov R, Angelova G Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021) . Held Online: INCOMA Ltd. (2021). p. 358–66. Available at: https://aclanthology.org/2021.ranlp-1.41

44. Jiang Z, Levitan SI, Zomick J, Hirschberg J. Detection of mental health from Reddit via deep contextualized representations. In: Holderness E, Jimeno Yepes A, Lavelli A, Minard A-L, Pustejovsky J, Rinaldi F, et al editors. Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis . Online: Association for Computational Linguistics (2020) 147–56. doi: 10.18653/v1/2020.louhi-1.16

45. Zhang Z, Chen S, Wu M, Zhu KQ. Symptom identification for interpretable detection of multiple mental disorders. In: Goldberg Y, Kozareva Z, Zhang Y, editos. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . Abu Dhabi, United Arab Emirates: Association for Computational Linguistics (2022) 9970–85. doi: 10.18653/v1/2022.emnlp-main.677

46. Hwang Y, Kim H, Choi H, Lee J. Exploring abnormal behavior patterns of online users with emotional eating behavior: Topic modeling study. J Med Internet Res . (2020) 22:e15700. doi: 10.2196/15700

47. Rojewska K, Maćkowska S, Maćkowski M, Różańska A, Barańska K, Dzieciatko M, et al. Natural language processing and machine learning supporting the work of a psychologist and its evaluation on the example of support for psychological diagnosis of anorexia. Appl Sci . (2022) 12. doi: 10.3390/app12094702

48. Villegas MP, Errecalde ML, Cagnina LC. “A comparison of text representation approaches for early detection of anorexia.” In: Memorias del Congreso Argentino en Ciencias de la Computación - CACIC 2021 , Workshop: WBDMD - Base de Datos y MinerÍa de Datos. (2021). 301–10.

49. Chancellor S, Pater JA, Clear T, Gilbert E, De Choudhury M. #thyghgapp: Instagram content moderation and lexical variation in pro-eating disorder communities (New York, NY, USA: Association for Computing Machinery). CSCW . (2016) 16:1201–13. doi: 10.1145/2818048.2819963

50. Cohan A, Desmet B, Yates A, Soldaini L, MacAvaney S, Goharian N. SMHD: a large-scale resource for exploring online language usage for multiple mental health conditions. In: Bender EM, Derczynski L, Isabelle P editors. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA. Association for Computational Linguistics (2018). 1485–97. Available at: https://aclanthology.org/C18-1126

51. Wang Y-T, Huang H-H, Chen H-H. “A neural network approach to early risk detection of depression and anorexia on social media text.” In: Conference and Labs of the Evaluation Forum (CLEF) . Aachen, Germany: CEUR-WS.org (2018). Available at: https://api.semanticscholar.org/CorpusID:51940589

52. Paul S, Jandhyala SK, Basu T. Early detection of signs of anorexia and depression over social media using effective machine learning frameworks. In: Conference and Labs of the Evaluation Forum (CLEF) (2018). Available at: https://api.semanticscholar.org/CorpusID:51942457 .

53. Trotzek M, Koitka S, Friedrich CM. Word embeddings and linguistic metadata at the clef 2018 tasks for early detection of depression and anorexia. In: Conference and Labs of the Evaluation Forum (CLEF) . (2018). Available at: https://api.semanticscholar.org/CorpusID:51939971 .

54. Ramiandrisoa F, Mothe J, Benamara F, Moriceau V. IRIT at e-Risk 2018. In: 9th Conference and Labs of the Evaluation Forum, Living Labs (CLEF 2018), Avignon, France: CEUR-WS.org. (2018). pp. 1–12. Available at: https://hal.science/hal-02290007 .

55. Ortega-Mendoza RM, López-Monroy AP, Franco-Arcega A, Montes-y-Gómez M. PEIMEX at eRisk2018: Emphasizing personal information for depression and anorexia detection. In: Conference and Labs of the Evaluation Forum (CLEF) . (2018). Available at: https://api.semanticscholar.org/CorpusID:51939864 .

56. Ragheb W, Moulahi B, Azé J, Bringay S, Servajean M. Temporal mood variation: at the CLEF eRisk-2018 tasks for early risk detection on the internet. In: CLEF 2018 - Conference and Labs of the Evaluation Forum . Avignon, France. Aachen, Germany: CEUR Workshop Proceedings (2018) 2125(78). Available at: https://hal-lirmm.ccsd.cnrs.fr/lirmm-01989632/file/paper_78.pdf .

57. Liu N, Zhou Z, Xin K, Ren F. TUA1 at eRisk 2018. In: Cappellato L, Ferro N, Nie J, Soulier L, editors. Working Notes of CLEF 2018 - Conference and Labs of the Evaluation Forum, Avignon, France, September 10-14, 2018 Aachen, Germany: CEUR-WS.org. CEUR Workshop Proceedings (2018). Available at: https://ceur-ws.org/Vol-2125/paper_121.pdf .

58. Ramírez-Cifuentes D, Freire A. UPF’s participation at the clef eRisk 2018: Early risk prediction on the internet. In: Conference and Labs of the Evaluation Forum (CLEF) . (2018).

59. Funez DG, Ucelay MJG, Villegas MP, Burdisso SG, Cagnina LC, Montes-y-Gómez M, et al. UNSL’s participation at eRisk 2018 lab. In: Conference and Labs of the Evaluation Forum (CLEF) . Aachen, Germany: CEUR-WS.org (2018). Available at: https://api.semanticscholar.org/CorpusID:198489135 .

60. Aragón ME, Lopez-Monroy AP, Montes-y-Gómez M. (2019). “INAOE-CIMAT at eRisk 2019: Detecting Signs of Anorexia using Fine-Grained Emotions,” in: Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes. Aachen, Germany: CEUR-WS.org. Available at: https://api.semanticscholar.org/CorpusID:198489135 .

61. Burdisso SG, Errecalde ML, Montes-y-Gómez M. UNSL at eRisk 2019: a Unified Approach for Anorexia, Self-harm, and Depression Detection in Social Media. In: Cappellato L, Ferro N, Losada DE, Muller H. Conference and Labs of the Evaluation Forum (CLEF) . Aachen, Germany: CEUR-WS.org (2019). Available at: https://api.semanticscholar.org/CorpusID:198490018 .

62. Ragheb W, Azé J, Bringay S, Servajean M. Attentive Multi-stage Learning for Early Risk Detection of Signs of Anorexia and Self-harm on Social Media. In: Cappellato L, Ferro N, Losada DE, Müller H, editors. CLEF 2019 - Conference and Labs of the Evaluation Forum , vol. 2380 . CEUR Workshop Proceedings, Lugano, Switzerland (2019).

63. Fano E, Karlgren J, Nivre J. “Working Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum, Lugano, Switzerland, September 9-12, 2019.” Vol. 2380. Cappellato L, Ferro N, Losada DE, Muller H. editors. Vol. 2380. Aachen, Germany: CEUR Workshop Proceedings (2019).

64. Masood R, Ramiandrisoa F, Aker A. “UDE at eRisk 2019: Early risk prediction on the internet.” In: Cappellato L, Ferro N, Losada DE, Müller H, editors. Working Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum, Lugano, Switzerland, September 9-12, 2019 Aachen, Germany: CEUR Workshop Proceedings (2019). 2380.

65. Naderi N, Gobeill J, Teodoro D, Pasche E, Ruch P. A baseline approach for early detection of signs of anorexia and self-harm in reddit posts, in: Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes, Lugano, Switzerland: CEUR-WS.org. (2019). CEUR Workshop Proceedings.

66. Mohammadi E, Amini H, Kosseim L. Quick and (maybe not so) easy detection of anorexia in social media posts. Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes (2019). CEUR Workshop Proceedings.

67. Plaza del Arco FM, López-Úbeda P, Díaz-Galiano MC, Ureña López LA, Martín Valdivia MT. “Integrating UMLS for Early Detection of Signs of Anorexia,” In: Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes. Aachen, Germany: CEUR-WS.org. (2019). Available at: https://api.semanticscholar.org/CorpusID:198489706 .

68. Ranganathan A, Haritha A, Thenmozhi D, Aravindan C. “Early detection of anorexia using rnn-lstm and svm classifiers,” In: Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes. Aachen, Germany: CEUR-WS.org. (2019). Available at: https://api.semanticscholar.org/CorpusID:198488874 .

69. Ferdowsi S, Knafou J, Borissov N, Vicente Alvarez D, Mishra R, Amini P, et al. Deep learning-based risk prediction for interventional clinical trials based on protocol design: A retrospective study. Patterns . (2023) 4:100689. doi: 10.1016/j.patter.2023.100689

70. Trifan A, Oliveira JL. (2019). “BioInfo@UAVR at eRisk 2019: Delving into Social Media Texts for the Early Detection of Mental and Food Disorders,” In: Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes. Aachen, Germany: CEUR-WS.org. Available at: https://api.semanticscholar.org/CorpusID:198488663 .

71. Ortega-Mendoza RM, Irazú D, Farías H, Montes-Y-Gómez M. “LTL-INAOE's Participation at eRisk 2019: Detecting Anorexia in Social Media through Shared Personal Information.” In: Cappellato L, Ferro N, Losada DE, Müller H. editors. Working Notes of CLEF 2019 - Conference and Labs of the Evaluation Forum. CEUR Workshop Proceedings Vol. 2380. Lugano, Switzerland, September 9-12, 2019. CEUR-WS.org (2019). Available at: https://ceur-ws.org/Vol-2380/paper_75.pdf .

72. Hosseini Saravani SH, Normand L, Maupomé D, Rancourt F, Soulas T, Besharati S, et al. Measuring the severity of the signs of eating disorders using similarity-based models. CLEF (Working Notes) (2022). 936–46.

73. Mármol-Romero AM, Jiménez-Zafra SM, Plaza-Del-Arco FM, Molina-González MD, Martín-Valdivia M-T, Montejo-Ráez A. “SINAI at eRisk@CLEF 2022: Approaching Early Detection of Gambling and Eating Disorders with Natural Language Processing.“ In: Faggioli G, Ferro N, Hanbury A, Potthast M editors. Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. CEUR Workshop Proceedings (2022) . Bologna, Italy. September 5th - to - 8th, 2022. CEUR-WS.org. 3180:961–71. Available at: https://ceur-ws.org/Vol-3180/paper-76.pdf .

74. Srivastava H, Lijin NS, Sruthi S, Basu T. “Exploring the potential of bag of words, document embeddings and transformer based framework for early prediction of eating disorder, depression and pathological gambling over social media.” In: Faggioli G, Ferro N, Hanbury A, Potthast M, editors. Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, Bologna, Italy, September 5th - to - 8th Aachen, Germany: CEUR Workshop Proceedings (2022). p. 972–86.

75. Li W, Han J, Pei J. CMAR: accurate and efficient classification based on multiple classassociation rules, in: Proceedings 2001 IEEE International Conference on Data Mining. San Jose, CA, USA: IEEE. (2001). 369–76. doi: 10.1109/ICDM.2001.989541

76. Guu K, Lee K, Tung Z, Pasupat P, Chang M-W. REALM: retrieval-augmented language model pre-training. In: Proceedings of the 37th International Conference on Machine Learning. ICML'20. JMLR.org . (2020).

77. Ji S, Zhang T, Ansari L, Fu J, Tiwari P, Cambria E. MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. In: Calzolari N, Béchet F, Blache P, Choukri K, Cieri C, Declerck T, et al editors. Proceedings of the Thirteenth Language Resources and Evaluation Conference . Marseille, France: European Language Resources Association (2022). 7184–90. Available at: https://aclanthology.org/2022.lrec-1.778 .

78. Yan H, Phd EEF-C, Goodman M, Krauss M, Das S, Cavazos-Rehg P. (2019). doi: 10.1002/eat.23148

79. Chancellor S, Kalantidis Y, Pater JA, De Choudhury MD, Shamma DA. Multimodal Classification of Moderated Online Pro-Eating Disorder Content, In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery. (2017). 3213–26. doi: 10.1145/3025453.3025985

80. Burdisso SG, Errecalde M, Montes-y-Gómez M. A text classification framework for simple and effective early depression detection over social media streams. Expert Syst Appl . (2019) 133:182–97. doi: 10.1016/j.eswa.2019.05.023

81. Yang K, Zhang T, Kuang Z, Xie Q, Ananiadou S. Mentalllama: Interpretable mental health analysis on social media with large language models. arXiv . (2023) arXiv preprint arXiv:2309.13567.

Keywords: natural language processing, machine learning, eating disorders, mental health, artificial intelligence, anorexia, bulimia, binge eating

Citation: Merhbene G, Puttick A and Kurpicz-Briki M (2024) Investigating machine learning and natural language processing techniques applied for detecting eating disorders: a systematic literature review. Front. Psychiatry 15:1319522. doi: 10.3389/fpsyt.2024.1319522

Received: 11 October 2023; Accepted: 05 March 2024; Published: 26 March 2024.

Reviewed by:

Copyright © 2024 Merhbene, Puttick and Kurpicz-Briki. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Mascha Kurpicz-Briki, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

IMAGES

  1. Chapter 5 Performing a literature review

    literature review research design

  2. Helping You in Writing a Literature Review Immaculately

    literature review research design

  3. Sample of Research Literature Review

    literature review research design

  4. Literature Review Methodology Example

    literature review research design

  5. 50 Smart Literature Review Templates (APA) ᐅ TemplateLab

    literature review research design

  6. Literature Reviews

    literature review research design

VIDEO

  1. Write Your Literature Review FAST

  2. Part 03: Literature Review (Research Methods and Methodology) By Dr. Walter

  3. Research Methods

  4. Literature Review Research Methodology

  5. Approaches , Analysis And Sources Of Literature Review ( RESEARCH METHODOLOGY AND IPR)

  6. Literature Review

COMMENTS

  1. Literature review as a research methodology: An overview and guidelines

    As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.

  2. How to Write a Literature Review

    Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.

  3. Literature Review and Research Design

    This book looks at literature review in the process of research design, and how to develop a research practice that will build skills in reading and writing about research literature—skills that remain valuable in both academic and professional careers. Literature review is approached as a process of engaging with the discourse of scholarly ...

  4. Literature Review Research Design

    Literature review research. part of every research paper (staggered design with literature review as one stage) and stand-alone research design. aims at summarizing the existing body of knowledge and identifying the gaps in it. different forms of literature review research design available to address the objective.

  5. Literature Review

    Literature Review. A literature review is a discussion of the literature (aka. the "research" or "scholarship") surrounding a certain topic. A good literature review doesn't simply summarize the existing material, but provides thoughtful synthesis and analysis. The purpose of a literature review is to orient your own work within an existing ...

  6. How To Structure A Literature Review (Free Template)

    Demonstrate your knowledge of the research topic. Identify the gaps in the literature and show how your research links to these. Provide the foundation for your conceptual framework (if you have one) Inform your own methodology and research design. To achieve this, your literature review needs a well-thought-out structure.

  7. Writing a literature review

    Writing a literature review requires a range of skills to gather, sort, evaluate and summarise peer-reviewed published data into a relevant and informative unbiased narrative. Digital access to research papers, academic texts, review articles, reference databases and public data sets are all sources of information that are available to enrich ...

  8. How-to conduct a systematic literature review: A quick guide for

    Method details Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12].An SLR updates the reader with current literature about a subject [6].The goal is to review critical points of current knowledge on a ...

  9. Reviewing the literature: choosing a review design

    Many health professionals, students and academics including health researchers will have grappled with the challenges of undertaking a review of the literature and choosing a suitable design or framework to structure the review. For many undergraduate and master's healthcare students their final year dissertation involves undertaking a review of the literature as a way of assessing their ...

  10. Literature Review and Research Design A Guide to Effective Research

    Literature review is approached as a process of engaging with the discourse of scholarly communities that will help graduate researchers refine, define, and express their own scholarly vision and voice. This orientation on research as an exploratory practice, rather than merely a series of predetermined steps in a systematic method, allows the ...

  11. 5. The Literature Review

    A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...

  12. PDF Reviewing the literature: choosing a review design

    Reviewing the literature is a crucial skill for health professionals, students and researchers, but it can be daunting to choose the right review design for the research question. This article provides a practical guide to the different types of review designs and methods, with examples and tips to help you conduct a rigorous and comprehensive literature review.

  13. Literature Review and Research Design

    This book looks at literature review in the process of research design, and how to develop a research practice that will build skills in reading and writing about research literature—skills that remain valuable in both academic and professional careers.

  14. Chapter 9 Methods for Literature Reviews

    9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.

  15. PDF 13 Literature Review Research Design

    13.2 Particularities of Literature Review Research Design. In this section, we specifically address the elements that make literature review research a discrete research design differentiated from others. Next to the characteristics of litera-ture review research, we address the main issues and decisions to be made within this research design ...

  16. Literature Reviews, Theoretical Frameworks, and Conceptual Frameworks

    The first element we discuss is a review of research (literature reviews), which highlights the need for a specific research question, study problem, or topic of investigation. ... Common questions concern determining which literature pertains to the topic of study or the role of the literature review in the design of the study. This section ...

  17. Research Methods: Literature Reviews

    A literature review involves researching, reading, analyzing, evaluating, and summarizing scholarly literature (typically journals and articles) about a specific topic. The results of a literature review may be an entire report or article OR may be part of a article, thesis, dissertation, or grant proposal. ... Research: locate literature ...

  18. Study designs: Part 7

    Systematic reviews are a type of study design that aim to synthesize the best available evidence on a specific research question. In this article, which is the seventh part of a series on study designs, we explain the rationale, steps, methods, and challenges of conducting systematic reviews. We also provide examples and resources for further learning.

  19. Literature review and research design : a guide to effective research

    This book looks at literature review in the process of research design, and how to develop a research practice that will build skills in reading and writing about research literature-skills that remain valuable in both academic and professional careers. ... The focus on the practical elements of research design makes this book an invaluable ...

  20. Systematic Review

    Systematic review vs. literature review. A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method. ... Type of study design(s) Example: Formulate a research ...

  21. Literature Review and Research Design: A Guide to Effective Research

    Praise for Literature Review and Research Design Unlike other books on research, this book does not prescribe methods or recipes. Rather, it feels like one is sitting with an experienced dissertation coach, having a series of short conversations about the tacit knowledge that underlies the various aspects of research practice.

  22. Literature Review and Research Design: A Guide to Effective Research

    Education, Computer Science. 2021. TLDR. This paper aims to review current literature from a variety of disciplines on the effect of the digital age on students' deep reading skills and provide teachers with effective strategies to encourage and sustain students' deep reading in the online world. Expand.

  23. Literature Review Research Design

    This chapter addresses the literature review research design's peculiarities, characteristics, and significant fallacies. Conducting and writing poor literature reviews is one way to lower academic work's value. State-of-the-art literature reviews are valuable and publishable scholarly documents. Too many new scholars think that empirical ...

  24. ResearchGate

    ResearchGate

  25. A Contrastive Study of Lexical Bundles Expressing Gratitude in

    Literature Review Research on Lexical Bundles (LB) Most LB research analyzes LBs in terms of form, frequency, structure, and so on. The structural analysis usually uses pre-established taxonomies to first categorize and then examine LB structures. ... Research Design Corpora.

  26. Literature Review and Research Design for Systems ...

    Lastly, a research design with three iterations is recommended to validate the proposed integration strategy. This research shall be executed in Spring 2023, and the findings shall be published later. ... Muller, G., Kokkula, S. (2024). Literature Review and Research Design for Systems Integration: Case Study in Defense Systems. In: Verma, D ...

  27. Recurrence of post-traumatic stress disorder: systematic review of

    Many people will experience a potentially traumatic event in their lifetime and a minority will go on to develop post-traumatic stress disorder (PTSD). A wealth of literature explores different trajectories of PTSD, focusing mostly on resilient, chronic, recovered and delayed-onset trajectories. Less is known about other potential trajectories such as recurring episodes of PTSD after initial ...

  28. Frontiers

    2.1 Study design. To answer our research questions, we conducted a structured literature review (SLR) following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines . This includes standards for literature search strategies and setting criteria for the inclusion or exclusion of gathered works in the final ...