analytical research methodology

What Is Research Methodology? A Plain-Language Explanation & Definition (With Examples)

By Derek Jansen (MBA)  and Kerryn Warren (PhD) | June 2020 (Last updated April 2023)

If you’re new to formal academic research, it’s quite likely that you’re feeling a little overwhelmed by all the technical lingo that gets thrown around. And who could blame you – “research methodology”, “research methods”, “sampling strategies”… it all seems never-ending!

In this post, we’ll demystify the landscape with plain-language explanations and loads of examples (including easy-to-follow videos), so that you can approach your dissertation, thesis or research project with confidence. Let’s get started.

Research Methodology 101

• What exactly research methodology means
• What qualitative , quantitative and mixed methods are
• What sampling strategy is
• What data collection methods are
• What data analysis methods are
• How to choose your research methodology
• Example of a research methodology

What is research methodology?

Research methodology simply refers to the practical “how” of a research study. More specifically, it’s about how  a researcher  systematically designs a study  to ensure valid and reliable results that address the research aims, objectives and research questions . Specifically, how the researcher went about deciding:

• What type of data to collect (e.g., qualitative or quantitative data )
• Who  to collect it from (i.e., the sampling strategy )
• How to  collect  it (i.e., the data collection method )
• How to  analyse  it (i.e., the data analysis methods )

Within any formal piece of academic research (be it a dissertation, thesis or journal article), you’ll find a research methodology chapter or section which covers the aspects mentioned above. Importantly, a good methodology chapter explains not just   what methodological choices were made, but also explains  why they were made. In other words, the methodology chapter should justify  the design choices, by showing that the chosen methods and techniques are the best fit for the research aims, objectives and research questions. 

So, it’s the same as research design?

Not quite. As we mentioned, research methodology refers to the collection of practical decisions regarding what data you’ll collect, from who, how you’ll collect it and how you’ll analyse it. Research design, on the other hand, is more about the overall strategy you’ll adopt in your study. For example, whether you’ll use an experimental design in which you manipulate one variable while controlling others. You can learn more about research design and the various design types here .

Need a helping hand?

What are qualitative, quantitative and mixed-methods?

Qualitative, quantitative and mixed-methods are different types of methodological approaches, distinguished by their focus on words , numbers or both . This is a bit of an oversimplification, but its a good starting point for understanding.

Let’s take a closer look.

Qualitative research refers to research which focuses on collecting and analysing words (written or spoken) and textual or visual data, whereas quantitative research focuses on measurement and testing using numerical data . Qualitative analysis can also focus on other “softer” data points, such as body language or visual elements.

It’s quite common for a qualitative methodology to be used when the research aims and research questions are exploratory  in nature. For example, a qualitative methodology might be used to understand peoples’ perceptions about an event that took place, or a political candidate running for president. 

Contrasted to this, a quantitative methodology is typically used when the research aims and research questions are confirmatory  in nature. For example, a quantitative methodology might be used to measure the relationship between two variables (e.g. personality type and likelihood to commit a crime) or to test a set of hypotheses .

As you’ve probably guessed, the mixed-method methodology attempts to combine the best of both qualitative and quantitative methodologies to integrate perspectives and create a rich picture. If you’d like to learn more about these three methodological approaches, be sure to watch our explainer video below.

What is sampling strategy?

Simply put, sampling is about deciding who (or where) you’re going to collect your data from . Why does this matter? Well, generally it’s not possible to collect data from every single person in your group of interest (this is called the “population”), so you’ll need to engage a smaller portion of that group that’s accessible and manageable (this is called the “sample”).

How you go about selecting the sample (i.e., your sampling strategy) will have a major impact on your study.  There are many different sampling methods  you can choose from, but the two overarching categories are probability   sampling and  non-probability   sampling .

Probability sampling  involves using a completely random sample from the group of people you’re interested in. This is comparable to throwing the names all potential participants into a hat, shaking it up, and picking out the “winners”. By using a completely random sample, you’ll minimise the risk of selection bias and the results of your study will be more generalisable  to the entire population. 

Non-probability sampling , on the other hand,  doesn’t use a random sample . For example, it might involve using a convenience sample, which means you’d only interview or survey people that you have access to (perhaps your friends, family or work colleagues), rather than a truly random sample. With non-probability sampling, the results are typically not generalisable .

To learn more about sampling methods, be sure to check out the video below.

What are data collection methods?

As the name suggests, data collection methods simply refers to the way in which you go about collecting the data for your study. Some of the most common data collection methods include:

• Interviews (which can be unstructured, semi-structured or structured)
• Focus groups and group interviews
• Surveys (online or physical surveys)
• Observations (watching and recording activities)
• Biophysical measurements (e.g., blood pressure, heart rate, etc.)
• Documents and records (e.g., financial reports, court records, etc.)

The choice of which data collection method to use depends on your overall research aims and research questions , as well as practicalities and resource constraints. For example, if your research is exploratory in nature, qualitative methods such as interviews and focus groups would likely be a good fit. Conversely, if your research aims to measure specific variables or test hypotheses, large-scale surveys that produce large volumes of numerical data would likely be a better fit.

What are data analysis methods?

Data analysis methods refer to the methods and techniques that you’ll use to make sense of your data. These can be grouped according to whether the research is qualitative  (words-based) or quantitative (numbers-based).

Popular data analysis methods in qualitative research include:

• Qualitative content analysis
• Thematic analysis
• Discourse analysis
• Narrative analysis
• Interpretative phenomenological analysis (IPA)
• Visual analysis (of photographs, videos, art, etc.)

Qualitative data analysis all begins with data coding , after which an analysis method is applied. In some cases, more than one analysis method is used, depending on the research aims and research questions . In the video below, we explore some  common qualitative analysis methods, along with practical examples.  

Moving on to the quantitative side of things, popular data analysis methods in this type of research include:

• Descriptive statistics (e.g. means, medians, modes )
• Inferential statistics (e.g. correlation, regression, structural equation modelling)

Again, the choice of which data collection method to use depends on your overall research aims and objectives , as well as practicalities and resource constraints. In the video below, we explain some core concepts central to quantitative analysis.

How do I choose a research methodology?

As you’ve probably picked up by now, your research aims and objectives have a major influence on the research methodology . So, the starting point for developing your research methodology is to take a step back and look at the big picture of your research, before you make methodology decisions. The first question you need to ask yourself is whether your research is exploratory or confirmatory in nature.

If your research aims and objectives are primarily exploratory in nature, your research will likely be qualitative and therefore you might consider qualitative data collection methods (e.g. interviews) and analysis methods (e.g. qualitative content analysis). 

Conversely, if your research aims and objective are looking to measure or test something (i.e. they’re confirmatory), then your research will quite likely be quantitative in nature, and you might consider quantitative data collection methods (e.g. surveys) and analyses (e.g. statistical analysis).

Designing your research and working out your methodology is a large topic, which we cover extensively on the blog . For now, however, the key takeaway is that you should always start with your research aims, objectives and research questions (the golden thread). Every methodological choice you make needs align with those three components. 

Example of a research methodology chapter

In the video below, we provide a detailed walkthrough of a research methodology from an actual dissertation, as well as an overview of our free methodology template .

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

199 Comments

Thank you for this simple yet comprehensive and easy to digest presentation. God Bless!

You’re most welcome, Leo. Best of luck with your research!

I found it very useful. many thanks

This is really directional. A make-easy research knowledge.

Thank you for this, I think will help my research proposal

Thanks for good interpretation,well understood.

Good morning sorry I want to the search topic

Thank u more

Thank you, your explanation is simple and very helpful.

Very educative a.nd exciting platform. A bigger thank you and I’ll like to always be with you

That’s the best analysis

So simple yet so insightful. Thank you.

This really easy to read as it is self-explanatory. Very much appreciated…

Thanks for this. It’s so helpful and explicit. For those elements highlighted in orange, they were good sources of referrals for concepts I didn’t understand. A million thanks for this.

Good morning, I have been reading your research lessons through out a period of times. They are important, impressive and clear. Want to subscribe and be and be active with you.

Thankyou So much Sir Derek…

Good morning thanks so much for the on line lectures am a student of university of Makeni.select a research topic and deliberate on it so that we’ll continue to understand more.sorry that’s a suggestion.

Beautiful presentation. I love it.

please provide a research mehodology example for zoology

It’s very educative and well explained

Thanks for the concise and informative data.

This is really good for students to be safe and well understand that research is all about

Thank you so much Derek sir🖤🙏🤗

Very simple and reliable

This is really helpful. Thanks alot. God bless you.

very useful, Thank you very much..

thanks a lot its really useful

in a nutshell..thank you!

Thanks for updating my understanding on this aspect of my Thesis writing.

thank you so much my through this video am competently going to do a good job my thesis

Thanks a lot. Very simple to understand. I appreciate 🙏

Very simple but yet insightful Thank you

This has been an eye opening experience. Thank you grad coach team.

Very useful message for research scholars

Really very helpful thank you

yes you are right and i’m left

Research methodology with a simplest way i have never seen before this article.

wow thank u so much

Good morning thanks so much for the on line lectures am a student of university of Makeni.select a research topic and deliberate on is so that we will continue to understand more.sorry that’s a suggestion.

Very precise and informative.

Thanks for simplifying these terms for us, really appreciate it.

Thanks this has really helped me. It is very easy to understand.

I found the notes and the presentation assisting and opening my understanding on research methodology

Good presentation

Im so glad you clarified my misconceptions. Im now ready to fry my onions. Thank you so much. God bless

Thank you a lot.

thanks for the easy way of learning and desirable presentation.

Thanks a lot. I am inspired

Well written

I am writing a APA Format paper . I using questionnaire with 120 STDs teacher for my participant. Can you write me mthology for this research. Send it through email sent. Just need a sample as an example please. My topic is ” impacts of overcrowding on students learning

Thanks for your comment.

We can’t write your methodology for you. If you’re looking for samples, you should be able to find some sample methodologies on Google. Alternatively, you can download some previous dissertations from a dissertation directory and have a look at the methodology chapters therein.

All the best with your research.

Thank you so much for this!! God Bless

Thank you. Explicit explanation

Thank you, Derek and Kerryn, for making this simple to understand. I’m currently at the inception stage of my research.

Thnks a lot , this was very usefull on my assignment

excellent explanation

I’m currently working on my master’s thesis, thanks for this! I’m certain that I will use Qualitative methodology.

Thanks a lot for this concise piece, it was quite relieving and helpful. God bless you BIG…

I am currently doing my dissertation proposal and I am sure that I will do quantitative research. Thank you very much it was extremely helpful.

Very interesting and informative yet I would like to know about examples of Research Questions as well, if possible.

I’m about to submit a research presentation, I have come to understand from your simplification on understanding research methodology. My research will be mixed methodology, qualitative as well as quantitative. So aim and objective of mixed method would be both exploratory and confirmatory. Thanks you very much for your guidance.

OMG thanks for that, you’re a life saver. You covered all the points I needed. Thank you so much ❤️ ❤️ ❤️

Thank you immensely for this simple, easy to comprehend explanation of data collection methods. I have been stuck here for months 😩. Glad I found your piece. Super insightful.

I’m going to write synopsis which will be quantitative research method and I don’t know how to frame my topic, can I kindly get some ideas..

Thanks for this, I was really struggling.

This was really informative I was struggling but this helped me.

Thanks a lot for this information, simple and straightforward. I’m a last year student from the University of South Africa UNISA South Africa.

its very much informative and understandable. I have enlightened.

An interesting nice exploration of a topic.

Thank you. Accurate and simple🥰

This article was really helpful, it helped me understanding the basic concepts of the topic Research Methodology. The examples were very clear, and easy to understand. I would like to visit this website again. Thank you so much for such a great explanation of the subject.

Thanks dude

Thank you Doctor Derek for this wonderful piece, please help to provide your details for reference purpose. God bless.

Many compliments to you

Great work , thank you very much for the simple explanation

Thank you. I had to give a presentation on this topic. I have looked everywhere on the internet but this is the best and simple explanation.

thank you, its very informative.

Well explained. Now I know my research methodology will be qualitative and exploratory. Thank you so much, keep up the good work

Well explained, thank you very much.

This is good explanation, I have understood the different methods of research. Thanks a lot.

Great work…very well explanation

Thanks Derek. Kerryn was just fantastic!

Great to hear that, Hyacinth. Best of luck with your research!

Its a good templates very attractive and important to PhD students and lectuter

Thanks for the feedback, Matobela. Good luck with your research methodology.

Thank you. This is really helpful.

You’re very welcome, Elie. Good luck with your research methodology.

Well explained thanks

This is a very helpful site especially for young researchers at college. It provides sufficient information to guide students and equip them with the necessary foundation to ask any other questions aimed at deepening their understanding.

Thanks for the kind words, Edward. Good luck with your research!

Thank you. I have learned a lot.

Great to hear that, Ngwisa. Good luck with your research methodology!

Thank you for keeping your presentation simples and short and covering key information for research methodology. My key takeaway: Start with defining your research objective the other will depend on the aims of your research question.

My name is Zanele I would like to be assisted with my research , and the topic is shortage of nursing staff globally want are the causes , effects on health, patients and community and also globally

Thanks for making it simple and clear. It greatly helped in understanding research methodology. Regards.

This is well simplified and straight to the point

Thank you Dr

I was given an assignment to research 2 publications and describe their research methodology? I don’t know how to start this task can someone help me?

Sure. You’re welcome to book an initial consultation with one of our Research Coaches to discuss how we can assist – https://gradcoach.com/book/new/ .

Thanks a lot I am relieved of a heavy burden.keep up with the good work

I’m very much grateful Dr Derek. I’m planning to pursue one of the careers that really needs one to be very much eager to know. There’s a lot of research to do and everything, but since I’ve gotten this information I will use it to the best of my potential.

Thank you so much, words are not enough to explain how helpful this session has been for me!

Thanks this has thought me alot.

Very concise and helpful. Thanks a lot

Thank Derek. This is very helpful. Your step by step explanation has made it easier for me to understand different concepts. Now i can get on with my research.

I wish i had come across this sooner. So simple but yet insightful

really nice explanation thank you so much

I’m so grateful finding this site, it’s really helpful…….every term well explained and provide accurate understanding especially to student going into an in-depth research for the very first time, even though my lecturer already explained this topic to the class, I think I got the clear and efficient explanation here, much thanks to the author.

It is very helpful material

I would like to be assisted with my research topic : Literature Review and research methodologies. My topic is : what is the relationship between unemployment and economic growth?

Its really nice and good for us.

THANKS SO MUCH FOR EXPLANATION, ITS VERY CLEAR TO ME WHAT I WILL BE DOING FROM NOW .GREAT READS.

Short but sweet.Thank you

Informative article. Thanks for your detailed information.

I’m currently working on my Ph.D. thesis. Thanks a lot, Derek and Kerryn, Well-organized sequences, facilitate the readers’ following.

great article for someone who does not have any background can even understand

I am a bit confused about research design and methodology. Are they the same? If not, what are the differences and how are they related?

Thanks in advance.

concise and informative.

Thank you very much

How can we site this article is Harvard style?

Very well written piece that afforded better understanding of the concept. Thank you!

Am a new researcher trying to learn how best to write a research proposal. I find your article spot on and want to download the free template but finding difficulties. Can u kindly send it to my email, the free download entitled, “Free Download: Research Proposal Template (with Examples)”.

Thank too much

Thank you very much for your comprehensive explanation about research methodology so I like to thank you again for giving us such great things.

Good very well explained.Thanks for sharing it.

Thank u sir, it is really a good guideline.

so helpful thank you very much.

Thanks for the video it was very explanatory and detailed, easy to comprehend and follow up. please, keep it up the good work

It was very helpful, a well-written document with precise information.

how do i reference this?

MLA Jansen, Derek, and Kerryn Warren. “What (Exactly) Is Research Methodology?” Grad Coach, June 2021, gradcoach.com/what-is-research-methodology/.

APA Jansen, D., & Warren, K. (2021, June). What (Exactly) Is Research Methodology? Grad Coach. https://gradcoach.com/what-is-research-methodology/

Your explanation is easily understood. Thank you

Very help article. Now I can go my methodology chapter in my thesis with ease

I feel guided ,Thank you

This simplification is very helpful. It is simple but very educative, thanks ever so much

The write up is informative and educative. It is an academic intellectual representation that every good researcher can find useful. Thanks

Wow, this is wonderful long live.

Nice initiative

thank you the video was helpful to me.

Thank you very much for your simple and clear explanations I’m really satisfied by the way you did it By now, I think I can realize a very good article by following your fastidious indications May God bless you

Thanks very much, it was very concise and informational for a beginner like me to gain an insight into what i am about to undertake. I really appreciate.

very informative sir, it is amazing to understand the meaning of question hidden behind that, and simple language is used other than legislature to understand easily. stay happy.

This one is really amazing. All content in your youtube channel is a very helpful guide for doing research. Thanks, GradCoach.

research methodologies

Please send me more information concerning dissertation research.

Nice piece of knowledge shared….. #Thump_UP

This is amazing, it has said it all. Thanks to Gradcoach

This is wonderful,very elaborate and clear.I hope to reach out for your assistance in my research very soon.

This is the answer I am searching about…

realy thanks a lot

Thank you very much for this awesome, to the point and inclusive article.

Thank you very much I need validity and reliability explanation I have exams

Thank you for a well explained piece. This will help me going forward.

Very simple and well detailed Many thanks

This is so very simple yet so very effective and comprehensive. An Excellent piece of work.

I wish I saw this earlier on! Great insights for a beginner(researcher) like me. Thanks a mil!

Thank you very much, for such a simplified, clear and practical step by step both for academic students and general research work. Holistic, effective to use and easy to read step by step. One can easily apply the steps in practical terms and produce a quality document/up-to standard

Thanks for simplifying these terms for us, really appreciated.

Thanks for a great work. well understood .

This was very helpful. It was simple but profound and very easy to understand. Thank you so much!

Great and amazing research guidelines. Best site for learning research

hello sir/ma’am, i didn’t find yet that what type of research methodology i am using. because i am writing my report on CSR and collect all my data from websites and articles so which type of methodology i should write in dissertation report. please help me. i am from India.

how does this really work?

perfect content, thanks a lot

As a researcher, I commend you for the detailed and simplified information on the topic in question. I would like to remain in touch for the sharing of research ideas on other topics. Thank you

Impressive. Thank you, Grad Coach 😍

Thank you Grad Coach for this piece of information. I have at least learned about the different types of research methodologies.

Very useful content with easy way

Thank you very much for the presentation. I am an MPH student with the Adventist University of Africa. I have successfully completed my theory and starting on my research this July. My topic is “Factors associated with Dental Caries in (one District) in Botswana. I need help on how to go about this quantitative research

I am so grateful to run across something that was sooo helpful. I have been on my doctorate journey for quite some time. Your breakdown on methodology helped me to refresh my intent. Thank you.

thanks so much for this good lecture. student from university of science and technology, Wudil. Kano Nigeria.

It’s profound easy to understand I appreciate

Thanks a lot for sharing superb information in a detailed but concise manner. It was really helpful and helped a lot in getting into my own research methodology.

Comment * thanks very much

This was sooo helpful for me thank you so much i didn’t even know what i had to write thank you!

You’re most welcome 🙂

Simple and good. Very much helpful. Thank you so much.

This is very good work. I have benefited.

Thank you so much for sharing

This is powerful thank you so much guys

I am nkasa lizwi doing my research proposal on honors with the university of Walter Sisulu Komani I m on part 3 now can you assist me.my topic is: transitional challenges faced by educators in intermediate phase in the Alfred Nzo District.

Appreciate the presentation. Very useful step-by-step guidelines to follow.

I appreciate sir

wow! This is super insightful for me. Thank you!

Indeed this material is very helpful! Kudos writers/authors.

I want to say thank you very much, I got a lot of info and knowledge. Be blessed.

I want present a seminar paper on Optimisation of Deep learning-based models on vulnerability detection in digital transactions.

Need assistance

Dear Sir, I want to be assisted on my research on Sanitation and Water management in emergencies areas.

I am deeply grateful for the knowledge gained. I will be getting in touch shortly as I want to be assisted in my ongoing research.

The information shared is informative, crisp and clear. Kudos Team! And thanks a lot!

hello i want to study

Hello!! Grad coach teams. I am extremely happy in your tutorial or consultation. i am really benefited all material and briefing. Thank you very much for your generous helps. Please keep it up. If you add in your briefing, references for further reading, it will be very nice.

All I have to say is, thank u gyz.

Good, l thanks

thank you, it is very useful

Trackbacks/Pingbacks

• What Is A Literature Review (In A Dissertation Or Thesis) - Grad Coach - […] the literature review is to inform the choice of methodology for your own research. As we’ve discussed on the Grad Coach blog,…
• Free Download: Research Proposal Template (With Examples) - Grad Coach - […] Research design (methodology) […]
• Dissertation vs Thesis: What's the difference? - Grad Coach - […] and thesis writing on a daily basis – everything from how to find a good research topic to which…

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What Is a Research Methodology? | Steps & Tips

What Is a Research Methodology? | Steps & Tips

Published on 25 February 2019 by Shona McCombes . Revised on 10 October 2022.

Your research methodology discusses and explains the data collection and analysis methods you used in your research. A key part of your thesis, dissertation, or research paper, the methodology chapter explains what you did and how you did it, allowing readers to evaluate the reliability and validity of your research.

It should include:

• The type of research you conducted
• How you collected and analysed your data
• Any tools or materials you used in the research
• Why you chose these methods
• Your methodology section should generally be written in the past tense .
• Academic style guides in your field may provide detailed guidelines on what to include for different types of studies.
• Your citation style might provide guidelines for your methodology section (e.g., an APA Style methods section ).

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

How to write a research methodology, why is a methods section important, step 1: explain your methodological approach, step 2: describe your data collection methods, step 3: describe your analysis method, step 4: evaluate and justify the methodological choices you made, tips for writing a strong methodology chapter, frequently asked questions about methodology.

Prevent plagiarism, run a free check.

Your methods section is your opportunity to share how you conducted your research and why you chose the methods you chose. It’s also the place to show that your research was rigorously conducted and can be replicated .

It gives your research legitimacy and situates it within your field, and also gives your readers a place to refer to if they have any questions or critiques in other sections.

You can start by introducing your overall approach to your research. You have two options here.

Option 1: Start with your “what”

What research problem or question did you investigate?

• Aim to describe the characteristics of something?
• Explore an under-researched topic?
• Establish a causal relationship?

And what type of data did you need to achieve this aim?

• Quantitative data , qualitative data , or a mix of both?
• Primary data collected yourself, or secondary data collected by someone else?
• Experimental data gathered by controlling and manipulating variables, or descriptive data gathered via observations?

Option 2: Start with your “why”

Depending on your discipline, you can also start with a discussion of the rationale and assumptions underpinning your methodology. In other words, why did you choose these methods for your study?

• Why is this the best way to answer your research question?
• Is this a standard methodology in your field, or does it require justification?
• Were there any ethical considerations involved in your choices?
• What are the criteria for validity and reliability in this type of research ?

Once you have introduced your reader to your methodological approach, you should share full details about your data collection methods .

Quantitative methods

In order to be considered generalisable, you should describe quantitative research methods in enough detail for another researcher to replicate your study.

Here, explain how you operationalised your concepts and measured your variables. Discuss your sampling method or inclusion/exclusion criteria, as well as any tools, procedures, and materials you used to gather your data.

Surveys Describe where, when, and how the survey was conducted.

• How did you design the questionnaire?
• What form did your questions take (e.g., multiple choice, Likert scale )?
• Were your surveys conducted in-person or virtually?
• What sampling method did you use to select participants?
• What was your sample size and response rate?

Experiments Share full details of the tools, techniques, and procedures you used to conduct your experiment.

• How did you design the experiment ?
• How did you recruit participants?
• How did you manipulate and measure the variables ?
• What tools did you use?

Existing data Explain how you gathered and selected the material (such as datasets or archival data) that you used in your analysis.

• Where did you source the material?
• How was the data originally produced?
• What criteria did you use to select material (e.g., date range)?

The survey consisted of 5 multiple-choice questions and 10 questions measured on a 7-point Likert scale.

The goal was to collect survey responses from 350 customers visiting the fitness apparel company’s brick-and-mortar location in Boston on 4–8 July 2022, between 11:00 and 15:00.

Here, a customer was defined as a person who had purchased a product from the company on the day they took the survey. Participants were given 5 minutes to fill in the survey anonymously. In total, 408 customers responded, but not all surveys were fully completed. Due to this, 371 survey results were included in the analysis.

Qualitative methods

In qualitative research , methods are often more flexible and subjective. For this reason, it’s crucial to robustly explain the methodology choices you made.

Be sure to discuss the criteria you used to select your data, the context in which your research was conducted, and the role you played in collecting your data (e.g., were you an active participant, or a passive observer?)

Interviews or focus groups Describe where, when, and how the interviews were conducted.

• How did you find and select participants?
• How many participants took part?
• What form did the interviews take ( structured , semi-structured , or unstructured )?
• How long were the interviews?
• How were they recorded?

Participant observation Describe where, when, and how you conducted the observation or ethnography .

• What group or community did you observe? How long did you spend there?
• How did you gain access to this group? What role did you play in the community?
• How long did you spend conducting the research? Where was it located?
• How did you record your data (e.g., audiovisual recordings, note-taking)?

Existing data Explain how you selected case study materials for your analysis.

• What type of materials did you analyse?
• How did you select them?

In order to gain better insight into possibilities for future improvement of the fitness shop’s product range, semi-structured interviews were conducted with 8 returning customers.

Here, a returning customer was defined as someone who usually bought products at least twice a week from the store.

Surveys were used to select participants. Interviews were conducted in a small office next to the cash register and lasted approximately 20 minutes each. Answers were recorded by note-taking, and seven interviews were also filmed with consent. One interviewee preferred not to be filmed.

Mixed methods

Mixed methods research combines quantitative and qualitative approaches. If a standalone quantitative or qualitative study is insufficient to answer your research question, mixed methods may be a good fit for you.

Mixed methods are less common than standalone analyses, largely because they require a great deal of effort to pull off successfully. If you choose to pursue mixed methods, it’s especially important to robustly justify your methods here.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Next, you should indicate how you processed and analysed your data. Avoid going into too much detail: you should not start introducing or discussing any of your results at this stage.

In quantitative research , your analysis will be based on numbers. In your methods section, you can include:

• How you prepared the data before analysing it (e.g., checking for missing data , removing outliers , transforming variables)
• Which software you used (e.g., SPSS, Stata or R)
• Which statistical tests you used (e.g., two-tailed t test , simple linear regression )

In qualitative research, your analysis will be based on language, images, and observations (often involving some form of textual analysis ).

Specific methods might include:

• Content analysis : Categorising and discussing the meaning of words, phrases and sentences
• Thematic analysis : Coding and closely examining the data to identify broad themes and patterns
• Discourse analysis : Studying communication and meaning in relation to their social context

Mixed methods combine the above two research methods, integrating both qualitative and quantitative approaches into one coherent analytical process.

Above all, your methodology section should clearly make the case for why you chose the methods you did. This is especially true if you did not take the most standard approach to your topic. In this case, discuss why other methods were not suitable for your objectives, and show how this approach contributes new knowledge or understanding.

In any case, it should be overwhelmingly clear to your reader that you set yourself up for success in terms of your methodology’s design. Show how your methods should lead to results that are valid and reliable, while leaving the analysis of the meaning, importance, and relevance of your results for your discussion section .

• Quantitative: Lab-based experiments cannot always accurately simulate real-life situations and behaviours, but they are effective for testing causal relationships between variables .
• Qualitative: Unstructured interviews usually produce results that cannot be generalised beyond the sample group , but they provide a more in-depth understanding of participants’ perceptions, motivations, and emotions.
• Mixed methods: Despite issues systematically comparing differing types of data, a solely quantitative study would not sufficiently incorporate the lived experience of each participant, while a solely qualitative study would be insufficiently generalisable.

Remember that your aim is not just to describe your methods, but to show how and why you applied them. Again, it’s critical to demonstrate that your research was rigorously conducted and can be replicated.

1. Focus on your objectives and research questions

The methodology section should clearly show why your methods suit your objectives  and convince the reader that you chose the best possible approach to answering your problem statement and research questions .

2. Cite relevant sources

Your methodology can be strengthened by referencing existing research in your field. This can help you to:

• Show that you followed established practice for your type of research
• Discuss how you decided on your approach by evaluating existing research
• Present a novel methodological approach to address a gap in the literature

3. Write for your audience

Consider how much information you need to give, and avoid getting too lengthy. If you are using methods that are standard for your discipline, you probably don’t need to give a lot of background or justification.

Regardless, your methodology should be a clear, well-structured text that makes an argument for your approach, not just a list of technical details and procedures.

Methodology refers to the overarching strategy and rationale of your research. Developing your methodology involves studying the research methods used in your field and the theories or principles that underpin them, in order to choose the approach that best matches your objectives.

Methods are the specific tools and procedures you use to collect and analyse data (e.g. interviews, experiments , surveys , statistical tests ).

In a dissertation or scientific paper, the methodology chapter or methods section comes after the introduction and before the results , discussion and conclusion .

Depending on the length and type of document, you might also include a literature review or theoretical framework before the methodology.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, October 10). What Is a Research Methodology? | Steps & Tips. Scribbr. Retrieved 1 July 2024, from https://www.scribbr.co.uk/thesis-dissertation/methodology/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a literature review | guide, template, & examples, what is a theoretical framework | a step-by-step guide.

• Open access
• Published: 07 September 2020

A tutorial on methodological studies: the what, when, how and why

• Lawrence Mbuagbaw   ORCID: orcid.org/0000-0001-5855-5461 1 , 2 , 3 ,
• Daeria O. Lawson 1 ,
• Livia Puljak 4 ,
• David B. Allison 5 &
• Lehana Thabane 1 , 2 , 6 , 7 , 8  

BMC Medical Research Methodology volume  20 , Article number:  226 ( 2020 ) Cite this article

40k Accesses

53 Citations

60 Altmetric

Metrics details

Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Peer Review reports

The field of meta-research (or research-on-research) has proliferated in recent years in response to issues with research quality and conduct [ 1 , 2 , 3 ]. As the name suggests, this field targets issues with research design, conduct, analysis and reporting. Various types of research reports are often examined as the unit of analysis in these studies (e.g. abstracts, full manuscripts, trial registry entries). Like many other novel fields of research, meta-research has seen a proliferation of use before the development of reporting guidance. For example, this was the case with randomized trials for which risk of bias tools and reporting guidelines were only developed much later – after many trials had been published and noted to have limitations [ 4 , 5 ]; and for systematic reviews as well [ 6 , 7 , 8 ]. However, in the absence of formal guidance, studies that report on research differ substantially in how they are named, conducted and reported [ 9 , 10 ]. This creates challenges in identifying, summarizing and comparing them. In this tutorial paper, we will use the term methodological study to refer to any study that reports on the design, conduct, analysis or reporting of primary or secondary research-related reports (such as trial registry entries and conference abstracts).

In the past 10 years, there has been an increase in the use of terms related to methodological studies (based on records retrieved with a keyword search [in the title and abstract] for “methodological review” and “meta-epidemiological study” in PubMed up to December 2019), suggesting that these studies may be appearing more frequently in the literature. See Fig.  1 .

Trends in the number studies that mention “methodological review” or “meta-

epidemiological study” in PubMed.

The methods used in many methodological studies have been borrowed from systematic and scoping reviews. This practice has influenced the direction of the field, with many methodological studies including searches of electronic databases, screening of records, duplicate data extraction and assessments of risk of bias in the included studies. However, the research questions posed in methodological studies do not always require the approaches listed above, and guidance is needed on when and how to apply these methods to a methodological study. Even though methodological studies can be conducted on qualitative or mixed methods research, this paper focuses on and draws examples exclusively from quantitative research.

The objectives of this paper are to provide some insights on how to conduct methodological studies so that there is greater consistency between the research questions posed, and the design, analysis and reporting of findings. We provide multiple examples to illustrate concepts and a proposed framework for categorizing methodological studies in quantitative research.

What is a methodological study?

Any study that describes or analyzes methods (design, conduct, analysis or reporting) in published (or unpublished) literature is a methodological study. Consequently, the scope of methodological studies is quite extensive and includes, but is not limited to, topics as diverse as: research question formulation [ 11 ]; adherence to reporting guidelines [ 12 , 13 , 14 ] and consistency in reporting [ 15 ]; approaches to study analysis [ 16 ]; investigating the credibility of analyses [ 17 ]; and studies that synthesize these methodological studies [ 18 ]. While the nomenclature of methodological studies is not uniform, the intents and purposes of these studies remain fairly consistent – to describe or analyze methods in primary or secondary studies. As such, methodological studies may also be classified as a subtype of observational studies.

Parallel to this are experimental studies that compare different methods. Even though they play an important role in informing optimal research methods, experimental methodological studies are beyond the scope of this paper. Examples of such studies include the randomized trials by Buscemi et al., comparing single data extraction to double data extraction [ 19 ], and Carrasco-Labra et al., comparing approaches to presenting findings in Grading of Recommendations, Assessment, Development and Evaluations (GRADE) summary of findings tables [ 20 ]. In these studies, the unit of analysis is the person or groups of individuals applying the methods. We also direct readers to the Studies Within a Trial (SWAT) and Studies Within a Review (SWAR) programme operated through the Hub for Trials Methodology Research, for further reading as a potential useful resource for these types of experimental studies [ 21 ]. Lastly, this paper is not meant to inform the conduct of research using computational simulation and mathematical modeling for which some guidance already exists [ 22 ], or studies on the development of methods using consensus-based approaches.

When should we conduct a methodological study?

Methodological studies occupy a unique niche in health research that allows them to inform methodological advances. Methodological studies should also be conducted as pre-cursors to reporting guideline development, as they provide an opportunity to understand current practices, and help to identify the need for guidance and gaps in methodological or reporting quality. For example, the development of the popular Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) guidelines were preceded by methodological studies identifying poor reporting practices [ 23 , 24 ]. In these instances, after the reporting guidelines are published, methodological studies can also be used to monitor uptake of the guidelines.

These studies can also be conducted to inform the state of the art for design, analysis and reporting practices across different types of health research fields, with the aim of improving research practices, and preventing or reducing research waste. For example, Samaan et al. conducted a scoping review of adherence to different reporting guidelines in health care literature [ 18 ]. Methodological studies can also be used to determine the factors associated with reporting practices. For example, Abbade et al. investigated journal characteristics associated with the use of the Participants, Intervention, Comparison, Outcome, Timeframe (PICOT) format in framing research questions in trials of venous ulcer disease [ 11 ].

How often are methodological studies conducted?

There is no clear answer to this question. Based on a search of PubMed, the use of related terms (“methodological review” and “meta-epidemiological study”) – and therefore, the number of methodological studies – is on the rise. However, many other terms are used to describe methodological studies. There are also many studies that explore design, conduct, analysis or reporting of research reports, but that do not use any specific terms to describe or label their study design in terms of “methodology”. This diversity in nomenclature makes a census of methodological studies elusive. Appropriate terminology and key words for methodological studies are needed to facilitate improved accessibility for end-users.

Why do we conduct methodological studies?

Methodological studies provide information on the design, conduct, analysis or reporting of primary and secondary research and can be used to appraise quality, quantity, completeness, accuracy and consistency of health research. These issues can be explored in specific fields, journals, databases, geographical regions and time periods. For example, Areia et al. explored the quality of reporting of endoscopic diagnostic studies in gastroenterology [ 25 ]; Knol et al. investigated the reporting of p -values in baseline tables in randomized trial published in high impact journals [ 26 ]; Chen et al. describe adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement in Chinese Journals [ 27 ]; and Hopewell et al. describe the effect of editors’ implementation of CONSORT guidelines on reporting of abstracts over time [ 28 ]. Methodological studies provide useful information to researchers, clinicians, editors, publishers and users of health literature. As a result, these studies have been at the cornerstone of important methodological developments in the past two decades and have informed the development of many health research guidelines including the highly cited CONSORT statement [ 5 ].

Where can we find methodological studies?

Methodological studies can be found in most common biomedical bibliographic databases (e.g. Embase, MEDLINE, PubMed, Web of Science). However, the biggest caveat is that methodological studies are hard to identify in the literature due to the wide variety of names used and the lack of comprehensive databases dedicated to them. A handful can be found in the Cochrane Library as “Cochrane Methodology Reviews”, but these studies only cover methodological issues related to systematic reviews. Previous attempts to catalogue all empirical studies of methods used in reviews were abandoned 10 years ago [ 29 ]. In other databases, a variety of search terms may be applied with different levels of sensitivity and specificity.

Some frequently asked questions about methodological studies

In this section, we have outlined responses to questions that might help inform the conduct of methodological studies.

Q: How should I select research reports for my methodological study?

A: Selection of research reports for a methodological study depends on the research question and eligibility criteria. Once a clear research question is set and the nature of literature one desires to review is known, one can then begin the selection process. Selection may begin with a broad search, especially if the eligibility criteria are not apparent. For example, a methodological study of Cochrane Reviews of HIV would not require a complex search as all eligible studies can easily be retrieved from the Cochrane Library after checking a few boxes [ 30 ]. On the other hand, a methodological study of subgroup analyses in trials of gastrointestinal oncology would require a search to find such trials, and further screening to identify trials that conducted a subgroup analysis [ 31 ].

The strategies used for identifying participants in observational studies can apply here. One may use a systematic search to identify all eligible studies. If the number of eligible studies is unmanageable, a random sample of articles can be expected to provide comparable results if it is sufficiently large [ 32 ]. For example, Wilson et al. used a random sample of trials from the Cochrane Stroke Group’s Trial Register to investigate completeness of reporting [ 33 ]. It is possible that a simple random sample would lead to underrepresentation of units (i.e. research reports) that are smaller in number. This is relevant if the investigators wish to compare multiple groups but have too few units in one group. In this case a stratified sample would help to create equal groups. For example, in a methodological study comparing Cochrane and non-Cochrane reviews, Kahale et al. drew random samples from both groups [ 34 ]. Alternatively, systematic or purposeful sampling strategies can be used and we encourage researchers to justify their selected approaches based on the study objective.

Q: How many databases should I search?

A: The number of databases one should search would depend on the approach to sampling, which can include targeting the entire “population” of interest or a sample of that population. If you are interested in including the entire target population for your research question, or drawing a random or systematic sample from it, then a comprehensive and exhaustive search for relevant articles is required. In this case, we recommend using systematic approaches for searching electronic databases (i.e. at least 2 databases with a replicable and time stamped search strategy). The results of your search will constitute a sampling frame from which eligible studies can be drawn.

Alternatively, if your approach to sampling is purposeful, then we recommend targeting the database(s) or data sources (e.g. journals, registries) that include the information you need. For example, if you are conducting a methodological study of high impact journals in plastic surgery and they are all indexed in PubMed, you likely do not need to search any other databases. You may also have a comprehensive list of all journals of interest and can approach your search using the journal names in your database search (or by accessing the journal archives directly from the journal’s website). Even though one could also search journals’ web pages directly, using a database such as PubMed has multiple advantages, such as the use of filters, so the search can be narrowed down to a certain period, or study types of interest. Furthermore, individual journals’ web sites may have different search functionalities, which do not necessarily yield a consistent output.

Q: Should I publish a protocol for my methodological study?

A: A protocol is a description of intended research methods. Currently, only protocols for clinical trials require registration [ 35 ]. Protocols for systematic reviews are encouraged but no formal recommendation exists. The scientific community welcomes the publication of protocols because they help protect against selective outcome reporting, the use of post hoc methodologies to embellish results, and to help avoid duplication of efforts [ 36 ]. While the latter two risks exist in methodological research, the negative consequences may be substantially less than for clinical outcomes. In a sample of 31 methodological studies, 7 (22.6%) referenced a published protocol [ 9 ]. In the Cochrane Library, there are 15 protocols for methodological reviews (21 July 2020). This suggests that publishing protocols for methodological studies is not uncommon.

Authors can consider publishing their study protocol in a scholarly journal as a manuscript. Advantages of such publication include obtaining peer-review feedback about the planned study, and easy retrieval by searching databases such as PubMed. The disadvantages in trying to publish protocols includes delays associated with manuscript handling and peer review, as well as costs, as few journals publish study protocols, and those journals mostly charge article-processing fees [ 37 ]. Authors who would like to make their protocol publicly available without publishing it in scholarly journals, could deposit their study protocols in publicly available repositories, such as the Open Science Framework ( https://osf.io/ ).

Q: How to appraise the quality of a methodological study?

A: To date, there is no published tool for appraising the risk of bias in a methodological study, but in principle, a methodological study could be considered as a type of observational study. Therefore, during conduct or appraisal, care should be taken to avoid the biases common in observational studies [ 38 ]. These biases include selection bias, comparability of groups, and ascertainment of exposure or outcome. In other words, to generate a representative sample, a comprehensive reproducible search may be necessary to build a sampling frame. Additionally, random sampling may be necessary to ensure that all the included research reports have the same probability of being selected, and the screening and selection processes should be transparent and reproducible. To ensure that the groups compared are similar in all characteristics, matching, random sampling or stratified sampling can be used. Statistical adjustments for between-group differences can also be applied at the analysis stage. Finally, duplicate data extraction can reduce errors in assessment of exposures or outcomes.

Q: Should I justify a sample size?

A: In all instances where one is not using the target population (i.e. the group to which inferences from the research report are directed) [ 39 ], a sample size justification is good practice. The sample size justification may take the form of a description of what is expected to be achieved with the number of articles selected, or a formal sample size estimation that outlines the number of articles required to answer the research question with a certain precision and power. Sample size justifications in methodological studies are reasonable in the following instances:

Comparing two groups

Determining a proportion, mean or another quantifier

Determining factors associated with an outcome using regression-based analyses

For example, El Dib et al. computed a sample size requirement for a methodological study of diagnostic strategies in randomized trials, based on a confidence interval approach [ 40 ].

Q: What should I call my study?

A: Other terms which have been used to describe/label methodological studies include “ methodological review ”, “methodological survey” , “meta-epidemiological study” , “systematic review” , “systematic survey”, “meta-research”, “research-on-research” and many others. We recommend that the study nomenclature be clear, unambiguous, informative and allow for appropriate indexing. Methodological study nomenclature that should be avoided includes “ systematic review” – as this will likely be confused with a systematic review of a clinical question. “ Systematic survey” may also lead to confusion about whether the survey was systematic (i.e. using a preplanned methodology) or a survey using “ systematic” sampling (i.e. a sampling approach using specific intervals to determine who is selected) [ 32 ]. Any of the above meanings of the words “ systematic” may be true for methodological studies and could be potentially misleading. “ Meta-epidemiological study” is ideal for indexing, but not very informative as it describes an entire field. The term “ review ” may point towards an appraisal or “review” of the design, conduct, analysis or reporting (or methodological components) of the targeted research reports, yet it has also been used to describe narrative reviews [ 41 , 42 ]. The term “ survey ” is also in line with the approaches used in many methodological studies [ 9 ], and would be indicative of the sampling procedures of this study design. However, in the absence of guidelines on nomenclature, the term “ methodological study ” is broad enough to capture most of the scenarios of such studies.

Q: Should I account for clustering in my methodological study?

A: Data from methodological studies are often clustered. For example, articles coming from a specific source may have different reporting standards (e.g. the Cochrane Library). Articles within the same journal may be similar due to editorial practices and policies, reporting requirements and endorsement of guidelines. There is emerging evidence that these are real concerns that should be accounted for in analyses [ 43 ]. Some cluster variables are described in the section: “ What variables are relevant to methodological studies?”

A variety of modelling approaches can be used to account for correlated data, including the use of marginal, fixed or mixed effects regression models with appropriate computation of standard errors [ 44 ]. For example, Kosa et al. used generalized estimation equations to account for correlation of articles within journals [ 15 ]. Not accounting for clustering could lead to incorrect p -values, unduly narrow confidence intervals, and biased estimates [ 45 ].

Q: Should I extract data in duplicate?

A: Yes. Duplicate data extraction takes more time but results in less errors [ 19 ]. Data extraction errors in turn affect the effect estimate [ 46 ], and therefore should be mitigated. Duplicate data extraction should be considered in the absence of other approaches to minimize extraction errors. However, much like systematic reviews, this area will likely see rapid new advances with machine learning and natural language processing technologies to support researchers with screening and data extraction [ 47 , 48 ]. However, experience plays an important role in the quality of extracted data and inexperienced extractors should be paired with experienced extractors [ 46 , 49 ].

Q: Should I assess the risk of bias of research reports included in my methodological study?

A : Risk of bias is most useful in determining the certainty that can be placed in the effect measure from a study. In methodological studies, risk of bias may not serve the purpose of determining the trustworthiness of results, as effect measures are often not the primary goal of methodological studies. Determining risk of bias in methodological studies is likely a practice borrowed from systematic review methodology, but whose intrinsic value is not obvious in methodological studies. When it is part of the research question, investigators often focus on one aspect of risk of bias. For example, Speich investigated how blinding was reported in surgical trials [ 50 ], and Abraha et al., investigated the application of intention-to-treat analyses in systematic reviews and trials [ 51 ].

Q: What variables are relevant to methodological studies?

A: There is empirical evidence that certain variables may inform the findings in a methodological study. We outline some of these and provide a brief overview below:

Country: Countries and regions differ in their research cultures, and the resources available to conduct research. Therefore, it is reasonable to believe that there may be differences in methodological features across countries. Methodological studies have reported loco-regional differences in reporting quality [ 52 , 53 ]. This may also be related to challenges non-English speakers face in publishing papers in English.

Authors’ expertise: The inclusion of authors with expertise in research methodology, biostatistics, and scientific writing is likely to influence the end-product. Oltean et al. found that among randomized trials in orthopaedic surgery, the use of analyses that accounted for clustering was more likely when specialists (e.g. statistician, epidemiologist or clinical trials methodologist) were included on the study team [ 54 ]. Fleming et al. found that including methodologists in the review team was associated with appropriate use of reporting guidelines [ 55 ].

Source of funding and conflicts of interest: Some studies have found that funded studies report better [ 56 , 57 ], while others do not [ 53 , 58 ]. The presence of funding would indicate the availability of resources deployed to ensure optimal design, conduct, analysis and reporting. However, the source of funding may introduce conflicts of interest and warrant assessment. For example, Kaiser et al. investigated the effect of industry funding on obesity or nutrition randomized trials and found that reporting quality was similar [ 59 ]. Thomas et al. looked at reporting quality of long-term weight loss trials and found that industry funded studies were better [ 60 ]. Kan et al. examined the association between industry funding and “positive trials” (trials reporting a significant intervention effect) and found that industry funding was highly predictive of a positive trial [ 61 ]. This finding is similar to that of a recent Cochrane Methodology Review by Hansen et al. [ 62 ]

Journal characteristics: Certain journals’ characteristics may influence the study design, analysis or reporting. Characteristics such as journal endorsement of guidelines [ 63 , 64 ], and Journal Impact Factor (JIF) have been shown to be associated with reporting [ 63 , 65 , 66 , 67 ].

Study size (sample size/number of sites): Some studies have shown that reporting is better in larger studies [ 53 , 56 , 58 ].

Year of publication: It is reasonable to assume that design, conduct, analysis and reporting of research will change over time. Many studies have demonstrated improvements in reporting over time or after the publication of reporting guidelines [ 68 , 69 ].

Type of intervention: In a methodological study of reporting quality of weight loss intervention studies, Thabane et al. found that trials of pharmacologic interventions were reported better than trials of non-pharmacologic interventions [ 70 ].

Interactions between variables: Complex interactions between the previously listed variables are possible. High income countries with more resources may be more likely to conduct larger studies and incorporate a variety of experts. Authors in certain countries may prefer certain journals, and journal endorsement of guidelines and editorial policies may change over time.

Q: Should I focus only on high impact journals?

A: Investigators may choose to investigate only high impact journals because they are more likely to influence practice and policy, or because they assume that methodological standards would be higher. However, the JIF may severely limit the scope of articles included and may skew the sample towards articles with positive findings. The generalizability and applicability of findings from a handful of journals must be examined carefully, especially since the JIF varies over time. Even among journals that are all “high impact”, variations exist in methodological standards.

Q: Can I conduct a methodological study of qualitative research?

A: Yes. Even though a lot of methodological research has been conducted in the quantitative research field, methodological studies of qualitative studies are feasible. Certain databases that catalogue qualitative research including the Cumulative Index to Nursing & Allied Health Literature (CINAHL) have defined subject headings that are specific to methodological research (e.g. “research methodology”). Alternatively, one could also conduct a qualitative methodological review; that is, use qualitative approaches to synthesize methodological issues in qualitative studies.

Q: What reporting guidelines should I use for my methodological study?

A: There is no guideline that covers the entire scope of methodological studies. One adaptation of the PRISMA guidelines has been published, which works well for studies that aim to use the entire target population of research reports [ 71 ]. However, it is not widely used (40 citations in 2 years as of 09 December 2019), and methodological studies that are designed as cross-sectional or before-after studies require a more fit-for purpose guideline. A more encompassing reporting guideline for a broad range of methodological studies is currently under development [ 72 ]. However, in the absence of formal guidance, the requirements for scientific reporting should be respected, and authors of methodological studies should focus on transparency and reproducibility.

Q: What are the potential threats to validity and how can I avoid them?

A: Methodological studies may be compromised by a lack of internal or external validity. The main threats to internal validity in methodological studies are selection and confounding bias. Investigators must ensure that the methods used to select articles does not make them differ systematically from the set of articles to which they would like to make inferences. For example, attempting to make extrapolations to all journals after analyzing high-impact journals would be misleading.

Many factors (confounders) may distort the association between the exposure and outcome if the included research reports differ with respect to these factors [ 73 ]. For example, when examining the association between source of funding and completeness of reporting, it may be necessary to account for journals that endorse the guidelines. Confounding bias can be addressed by restriction, matching and statistical adjustment [ 73 ]. Restriction appears to be the method of choice for many investigators who choose to include only high impact journals or articles in a specific field. For example, Knol et al. examined the reporting of p -values in baseline tables of high impact journals [ 26 ]. Matching is also sometimes used. In the methodological study of non-randomized interventional studies of elective ventral hernia repair, Parker et al. matched prospective studies with retrospective studies and compared reporting standards [ 74 ]. Some other methodological studies use statistical adjustments. For example, Zhang et al. used regression techniques to determine the factors associated with missing participant data in trials [ 16 ].

With regard to external validity, researchers interested in conducting methodological studies must consider how generalizable or applicable their findings are. This should tie in closely with the research question and should be explicit. For example. Findings from methodological studies on trials published in high impact cardiology journals cannot be assumed to be applicable to trials in other fields. However, investigators must ensure that their sample truly represents the target sample either by a) conducting a comprehensive and exhaustive search, or b) using an appropriate and justified, randomly selected sample of research reports.

Even applicability to high impact journals may vary based on the investigators’ definition, and over time. For example, for high impact journals in the field of general medicine, Bouwmeester et al. included the Annals of Internal Medicine (AIM), BMJ, the Journal of the American Medical Association (JAMA), Lancet, the New England Journal of Medicine (NEJM), and PLoS Medicine ( n  = 6) [ 75 ]. In contrast, the high impact journals selected in the methodological study by Schiller et al. were BMJ, JAMA, Lancet, and NEJM ( n  = 4) [ 76 ]. Another methodological study by Kosa et al. included AIM, BMJ, JAMA, Lancet and NEJM ( n  = 5). In the methodological study by Thabut et al., journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al. used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine “high impact” [ 77 ]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [ 78 ]. We acknowledge that the term “generalizability” may apply differently for methodological studies, especially when in many instances it is possible to include the entire target population in the sample studied.

Finally, methodological studies are not exempt from information bias which may stem from discrepancies in the included research reports [ 79 ], errors in data extraction, or inappropriate interpretation of the information extracted. Likewise, publication bias may also be a concern in methodological studies, but such concepts have not yet been explored.

A proposed framework

In order to inform discussions about methodological studies, the development of guidance for what should be reported, we have outlined some key features of methodological studies that can be used to classify them. For each of the categories outlined below, we provide an example. In our experience, the choice of approach to completing a methodological study can be informed by asking the following four questions:

What is the aim?

Methodological studies that investigate bias

A methodological study may be focused on exploring sources of bias in primary or secondary studies (meta-bias), or how bias is analyzed. We have taken care to distinguish bias (i.e. systematic deviations from the truth irrespective of the source) from reporting quality or completeness (i.e. not adhering to a specific reporting guideline or norm). An example of where this distinction would be important is in the case of a randomized trial with no blinding. This study (depending on the nature of the intervention) would be at risk of performance bias. However, if the authors report that their study was not blinded, they would have reported adequately. In fact, some methodological studies attempt to capture both “quality of conduct” and “quality of reporting”, such as Richie et al., who reported on the risk of bias in randomized trials of pharmacy practice interventions [ 80 ]. Babic et al. investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [ 81 ]. Further, biases related to choice of outcomes can also be explored. For example, Tan et al investigated differences in treatment effect size based on the outcome reported [ 82 ].

Methodological studies that investigate quality (or completeness) of reporting

Methodological studies may report quality of reporting against a reporting checklist (i.e. adherence to guidelines) or against expected norms. For example, Croituro et al. report on the quality of reporting in systematic reviews published in dermatology journals based on their adherence to the PRISMA statement [ 83 ], and Khan et al. described the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals based on the CONSORT extension for harms [ 84 ]. Other methodological studies investigate reporting of certain features of interest that may not be part of formally published checklists or guidelines. For example, Mbuagbaw et al. described how often the implications for research are elaborated using the Evidence, Participants, Intervention, Comparison, Outcome, Timeframe (EPICOT) format [ 30 ].

Methodological studies that investigate the consistency of reporting

Sometimes investigators may be interested in how consistent reports of the same research are, as it is expected that there should be consistency between: conference abstracts and published manuscripts; manuscript abstracts and manuscript main text; and trial registration and published manuscript. For example, Rosmarakis et al. investigated consistency between conference abstracts and full text manuscripts [ 85 ].

Methodological studies that investigate factors associated with reporting

In addition to identifying issues with reporting in primary and secondary studies, authors of methodological studies may be interested in determining the factors that are associated with certain reporting practices. Many methodological studies incorporate this, albeit as a secondary outcome. For example, Farrokhyar et al. investigated the factors associated with reporting quality in randomized trials of coronary artery bypass grafting surgery [ 53 ].

Methodological studies that investigate methods

Methodological studies may also be used to describe methods or compare methods, and the factors associated with methods. Muller et al. described the methods used for systematic reviews and meta-analyses of observational studies [ 86 ].

Methodological studies that summarize other methodological studies

Some methodological studies synthesize results from other methodological studies. For example, Li et al. conducted a scoping review of methodological reviews that investigated consistency between full text and abstracts in primary biomedical research [ 87 ].

Methodological studies that investigate nomenclature and terminology

Some methodological studies may investigate the use of names and terms in health research. For example, Martinic et al. investigated the definitions of systematic reviews used in overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks [ 88 ].

Other types of methodological studies

In addition to the previously mentioned experimental methodological studies, there may exist other types of methodological studies not captured here.

What is the design?

Methodological studies that are descriptive

Most methodological studies are purely descriptive and report their findings as counts (percent) and means (standard deviation) or medians (interquartile range). For example, Mbuagbaw et al. described the reporting of research recommendations in Cochrane HIV systematic reviews [ 30 ]. Gohari et al. described the quality of reporting of randomized trials in diabetes in Iran [ 12 ].

Methodological studies that are analytical

Some methodological studies are analytical wherein “analytical studies identify and quantify associations, test hypotheses, identify causes and determine whether an association exists between variables, such as between an exposure and a disease.” [ 89 ] In the case of methodological studies all these investigations are possible. For example, Kosa et al. investigated the association between agreement in primary outcome from trial registry to published manuscript and study covariates. They found that larger and more recent studies were more likely to have agreement [ 15 ]. Tricco et al. compared the conclusion statements from Cochrane and non-Cochrane systematic reviews with a meta-analysis of the primary outcome and found that non-Cochrane reviews were more likely to report positive findings. These results are a test of the null hypothesis that the proportions of Cochrane and non-Cochrane reviews that report positive results are equal [ 90 ].

What is the sampling strategy?

Methodological studies that include the target population

Methodological reviews with narrow research questions may be able to include the entire target population. For example, in the methodological study of Cochrane HIV systematic reviews, Mbuagbaw et al. included all of the available studies ( n  = 103) [ 30 ].

Methodological studies that include a sample of the target population

Many methodological studies use random samples of the target population [ 33 , 91 , 92 ]. Alternatively, purposeful sampling may be used, limiting the sample to a subset of research-related reports published within a certain time period, or in journals with a certain ranking or on a topic. Systematic sampling can also be used when random sampling may be challenging to implement.

What is the unit of analysis?

Methodological studies with a research report as the unit of analysis

Many methodological studies use a research report (e.g. full manuscript of study, abstract portion of the study) as the unit of analysis, and inferences can be made at the study-level. However, both published and unpublished research-related reports can be studied. These may include articles, conference abstracts, registry entries etc.

Methodological studies with a design, analysis or reporting item as the unit of analysis

Some methodological studies report on items which may occur more than once per article. For example, Paquette et al. report on subgroup analyses in Cochrane reviews of atrial fibrillation in which 17 systematic reviews planned 56 subgroup analyses [ 93 ].

This framework is outlined in Fig.  2 .

A proposed framework for methodological studies

Conclusions

Methodological studies have examined different aspects of reporting such as quality, completeness, consistency and adherence to reporting guidelines. As such, many of the methodological study examples cited in this tutorial are related to reporting. However, as an evolving field, the scope of research questions that can be addressed by methodological studies is expected to increase.

In this paper we have outlined the scope and purpose of methodological studies, along with examples of instances in which various approaches have been used. In the absence of formal guidance on the design, conduct, analysis and reporting of methodological studies, we have provided some advice to help make methodological studies consistent. This advice is grounded in good contemporary scientific practice. Generally, the research question should tie in with the sampling approach and planned analysis. We have also highlighted the variables that may inform findings from methodological studies. Lastly, we have provided suggestions for ways in which authors can categorize their methodological studies to inform their design and analysis.

Availability of data and materials

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Abbreviations

Consolidated Standards of Reporting Trials

Evidence, Participants, Intervention, Comparison, Outcome, Timeframe

Grading of Recommendations, Assessment, Development and Evaluations

Participants, Intervention, Comparison, Outcome, Timeframe

Preferred Reporting Items of Systematic reviews and Meta-Analyses

Studies Within a Review

Studies Within a Trial

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

PubMed   Google Scholar  

Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

PubMed   PubMed Central   Google Scholar  

Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.

Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Bmj. 2017;358:j4008.

Lawson DO, Leenus A, Mbuagbaw L. Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review. Pilot Feasibility Studies. 2020;6(1):13.

Puljak L, Makaric ZL, Buljan I, Pieper D. What is a meta-epidemiological study? Analysis of published literature indicated heterogeneous study designs and definitions. J Comp Eff Res. 2020.

Abbade LPF, Wang M, Sriganesh K, Jin Y, Mbuagbaw L, Thabane L. The framing of research questions using the PICOT format in randomized controlled trials of venous ulcer disease is suboptimal: a systematic survey. Wound Repair Regen. 2017;25(5):892–900.

Gohari F, Baradaran HR, Tabatabaee M, Anijidani S, Mohammadpour Touserkani F, Atlasi R, Razmgir M. Quality of reporting randomized controlled trials (RCTs) in diabetes in Iran; a systematic review. J Diabetes Metab Disord. 2015;15(1):36.

Wang M, Jin Y, Hu ZJ, Thabane A, Dennis B, Gajic-Veljanoski O, Paul J, Thabane L. The reporting quality of abstracts of stepped wedge randomized trials is suboptimal: a systematic survey of the literature. Contemp Clin Trials Commun. 2017;8:1–10.

Shanthanna H, Kaushal A, Mbuagbaw L, Couban R, Busse J, Thabane L: A cross-sectional study of the reporting quality of pilot or feasibility trials in high-impact anesthesia journals Can J Anaesthesia 2018, 65(11):1180–1195.

Kosa SD, Mbuagbaw L, Borg Debono V, Bhandari M, Dennis BB, Ene G, Leenus A, Shi D, Thabane M, Valvasori S, et al. Agreement in reporting between trial publications and current clinical trial registry in high impact journals: a methodological review. Contemporary Clinical Trials. 2018;65:144–50.

Zhang Y, Florez ID, Colunga Lozano LE, Aloweni FAB, Kennedy SA, Li A, Craigie S, Zhang S, Agarwal A, Lopes LC, et al. A systematic survey on reporting and methods for handling missing participant data for continuous outcomes in randomized controlled trials. J Clin Epidemiol. 2017;88:57–66.

CAS   PubMed   Google Scholar  

Hernández AV, Boersma E, Murray GD, Habbema JD, Steyerberg EW. Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? Am Heart J. 2006;151(2):257–64.

Samaan Z, Mbuagbaw L, Kosa D, Borg Debono V, Dillenburg R, Zhang S, Fruci V, Dennis B, Bawor M, Thabane L. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–88.

Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59(7):697–703.

Carrasco-Labra A, Brignardello-Petersen R, Santesso N, Neumann I, Mustafa RA, Mbuagbaw L, Etxeandia Ikobaltzeta I, De Stio C, McCullagh LJ, Alonso-Coello P. Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary-of-findings tables with a new format. J Clin Epidemiol. 2016;74:7–18.

The Northern Ireland Hub for Trials Methodology Research: SWAT/SWAR Information [ https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/ ]. Accessed 31 Aug 2020.

Chick S, Sánchez P, Ferrin D, Morrice D. How to conduct a successful simulation study. In: Proceedings of the 2003 winter simulation conference: 2003; 2003. p. 66–70.

Google Scholar  

Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106(3):485–8.

Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: an update. Mount Sinai J Med New York. 1996;63(3–4):216–24.

CAS   Google Scholar  

Areia M, Soares M, Dinis-Ribeiro M. Quality reporting of endoscopic diagnostic studies in gastrointestinal journals: where do we stand on the use of the STARD and CONSORT statements? Endoscopy. 2010;42(2):138–47.

Knol M, Groenwold R, Grobbee D. P-values in baseline tables of randomised controlled trials are inappropriate but still common in high impact journals. Eur J Prev Cardiol. 2012;19(2):231–2.

Chen M, Cui J, Zhang AL, Sze DM, Xue CC, May BH. Adherence to CONSORT items in randomized controlled trials of integrative medicine for colorectal Cancer published in Chinese journals. J Altern Complement Med. 2018;24(2):115–24.

Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors' implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178.

The Cochrane Methodology Register Issue 2 2009 [ https://cmr.cochrane.org/help.htm ]. Accessed 31 Aug 2020.

Mbuagbaw L, Kredo T, Welch V, Mursleen S, Ross S, Zani B, Motaze NV, Quinlan L. Critical EPICOT items were absent in Cochrane human immunodeficiency virus systematic reviews: a bibliometric analysis. J Clin Epidemiol. 2016;74:66–72.

Barton S, Peckitt C, Sclafani F, Cunningham D, Chau I. The influence of industry sponsorship on the reporting of subgroup analyses within phase III randomised controlled trials in gastrointestinal oncology. Eur J Cancer. 2015;51(18):2732–9.

Setia MS. Methodology series module 5: sampling strategies. Indian J Dermatol. 2016;61(5):505–9.

Wilson B, Burnett P, Moher D, Altman DG, Al-Shahi Salman R. Completeness of reporting of randomised controlled trials including people with transient ischaemic attack or stroke: a systematic review. Eur Stroke J. 2018;3(4):337–46.

Kahale LA, Diab B, Brignardello-Petersen R, Agarwal A, Mustafa RA, Kwong J, Neumann I, Li L, Lopes LC, Briel M, et al. Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey. J Clin Epidemiol. 2018;99:14–23.

De Angelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJPM, et al. Is this clinical trial fully registered?: a statement from the International Committee of Medical Journal Editors*. Ann Intern Med. 2005;143(2):146–8.

Ohtake PJ, Childs JD. Why publish study protocols? Phys Ther. 2014;94(9):1208–9.

Rombey T, Allers K, Mathes T, Hoffmann F, Pieper D. A descriptive analysis of the characteristics and the peer review process of systematic review protocols published in an open peer review journal from 2012 to 2017. BMC Med Res Methodol. 2019;19(1):57.

Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002;359(9302):248–52.

Porta M (ed.): A dictionary of epidemiology, 5th edn. Oxford: Oxford University Press, Inc.; 2008.

El Dib R, Tikkinen KAO, Akl EA, Gomaa HA, Mustafa RA, Agarwal A, Carpenter CR, Zhang Y, Jorge EC, Almeida R, et al. Systematic survey of randomized trials evaluating the impact of alternative diagnostic strategies on patient-important outcomes. J Clin Epidemiol. 2017;84:61–9.

Helzer JE, Robins LN, Taibleson M, Woodruff RA Jr, Reich T, Wish ED. Reliability of psychiatric diagnosis. I. a methodological review. Arch Gen Psychiatry. 1977;34(2):129–33.

Chung ST, Chacko SK, Sunehag AL, Haymond MW. Measurements of gluconeogenesis and Glycogenolysis: a methodological review. Diabetes. 2015;64(12):3996–4010.

CAS   PubMed   PubMed Central   Google Scholar  

Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research. Stat Med. 2002;21(11):1513–24.

Moen EL, Fricano-Kugler CJ, Luikart BW, O’Malley AJ. Analyzing clustered data: why and how to account for multiple observations nested within a study participant? PLoS One. 2016;11(1):e0146721.

Zyzanski SJ, Flocke SA, Dickinson LM. On the nature and analysis of clustered data. Ann Fam Med. 2004;2(3):199–200.

Mathes T, Klassen P, Pieper D. Frequency of data extraction errors and methods to increase data extraction quality: a methodological review. BMC Med Res Methodol. 2017;17(1):152.

Bui DDA, Del Fiol G, Hurdle JF, Jonnalagadda S. Extractive text summarization system to aid data extraction from full text in systematic review development. J Biomed Inform. 2016;64:265–72.

Bui DD, Del Fiol G, Jonnalagadda S. PDF text classification to leverage information extraction from publication reports. J Biomed Inform. 2016;61:141–8.

Maticic K, Krnic Martinic M, Puljak L. Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience. BMC Med Res Methodol. 2019;19(1):32.

Speich B. Blinding in surgical randomized clinical trials in 2015. Ann Surg. 2017;266(1):21–2.

Abraha I, Cozzolino F, Orso M, Marchesi M, Germani A, Lombardo G, Eusebi P, De Florio R, Luchetta ML, Iorio A, et al. A systematic review found that deviations from intention-to-treat are common in randomized trials and systematic reviews. J Clin Epidemiol. 2017;84:37–46.

Zhong Y, Zhou W, Jiang H, Fan T, Diao X, Yang H, Min J, Wang G, Fu J, Mao B. Quality of reporting of two-group parallel randomized controlled clinical trials of multi-herb formulae: A survey of reports indexed in the Science Citation Index Expanded. Eur J Integrative Med. 2011;3(4):e309–16.

Farrokhyar F, Chu R, Whitlock R, Thabane L. A systematic review of the quality of publications reporting coronary artery bypass grafting trials. Can J Surg. 2007;50(4):266–77.

Oltean H, Gagnier JJ. Use of clustering analysis in randomized controlled trials in orthopaedic surgery. BMC Med Res Methodol. 2015;15:17.

Fleming PS, Koletsi D, Pandis N. Blinded by PRISMA: are systematic reviewers focusing on PRISMA and ignoring other guidelines? PLoS One. 2014;9(5):e96407.

Balasubramanian SP, Wiener M, Alshameeri Z, Tiruvoipati R, Elbourne D, Reed MW. Standards of reporting of randomized controlled trials in general surgery: can we do better? Ann Surg. 2006;244(5):663–7.

de Vries TW, van Roon EN. Low quality of reporting adverse drug reactions in paediatric randomised controlled trials. Arch Dis Child. 2010;95(12):1023–6.

Borg Debono V, Zhang S, Ye C, Paul J, Arya A, Hurlburt L, Murthy Y, Thabane L. The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement. BMC Anesthesiol. 2012;12:13.

Kaiser KA, Cofield SS, Fontaine KR, Glasser SP, Thabane L, Chu R, Ambrale S, Dwary AD, Kumar A, Nayyar G, et al. Is funding source related to study reporting quality in obesity or nutrition randomized control trials in top-tier medical journals? Int J Obes. 2012;36(7):977–81.

Thomas O, Thabane L, Douketis J, Chu R, Westfall AO, Allison DB. Industry funding and the reporting quality of large long-term weight loss trials. Int J Obes. 2008;32(10):1531–6.

Khan NR, Saad H, Oravec CS, Rossi N, Nguyen V, Venable GT, Lillard JC, Patel P, Taylor DR, Vaughn BN, et al. A review of industry funding in randomized controlled trials published in the neurosurgical literature-the elephant in the room. Neurosurgery. 2018;83(5):890–7.

Hansen C, Lundh A, Rasmussen K, Hrobjartsson A. Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality. Cochrane Database Syst Rev. 2019;8:Mr000047.

Kiehna EN, Starke RM, Pouratian N, Dumont AS. Standards for reporting randomized controlled trials in neurosurgery. J Neurosurg. 2011;114(2):280–5.

Liu LQ, Morris PJ, Pengel LH. Compliance to the CONSORT statement of randomized controlled trials in solid organ transplantation: a 3-year overview. Transpl Int. 2013;26(3):300–6.

Bala MM, Akl EA, Sun X, Bassler D, Mertz D, Mejza F, Vandvik PO, Malaga G, Johnston BC, Dahm P, et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol. 2013;66(3):286–95.

Lee SY, Teoh PJ, Camm CF, Agha RA. Compliance of randomized controlled trials in trauma surgery with the CONSORT statement. J Trauma Acute Care Surg. 2013;75(4):562–72.

Ziogas DC, Zintzaras E. Analysis of the quality of reporting of randomized controlled trials in acute and chronic myeloid leukemia, and myelodysplastic syndromes as governed by the CONSORT statement. Ann Epidemiol. 2009;19(7):494–500.

Alvarez F, Meyer N, Gourraud PA, Paul C. CONSORT adoption and quality of reporting of randomized controlled trials: a systematic analysis in two dermatology journals. Br J Dermatol. 2009;161(5):1159–65.

Mbuagbaw L, Thabane M, Vanniyasingam T, Borg Debono V, Kosa S, Zhang S, Ye C, Parpia S, Dennis BB, Thabane L. Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review. Contemporary Clin trials. 2014;38(2):245–50.

Thabane L, Chu R, Cuddy K, Douketis J. What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials. Int J Obes. 2007;31(10):1554–9.

Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evidence Based Med. 2017;22(4):139.

METRIC - MEthodological sTudy ReportIng Checklist: guidelines for reporting methodological studies in health research [ http://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-study-designs/#METRIC ]. Accessed 31 Aug 2020.

Jager KJ, Zoccali C, MacLeod A, Dekker FW. Confounding: what it is and how to deal with it. Kidney Int. 2008;73(3):256–60.

Parker SG, Halligan S, Erotocritou M, Wood CPJ, Boulton RW, Plumb AAO, Windsor ACJ, Mallett S. A systematic methodological review of non-randomised interventional studies of elective ventral hernia repair: clear definitions and a standardised minimum dataset are needed. Hernia. 2019.

Bouwmeester W, Zuithoff NPA, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, Altman DG, Moons KGM. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9(5):1–12.

Schiller P, Burchardi N, Niestroj M, Kieser M. Quality of reporting of clinical non-inferiority and equivalence randomised trials--update and extension. Trials. 2012;13:214.

Riado Minguez D, Kowalski M, Vallve Odena M, Longin Pontzen D, Jelicic Kadic A, Jeric M, Dosenovic S, Jakus D, Vrdoljak M, Poklepovic Pericic T, et al. Methodological and reporting quality of systematic reviews published in the highest ranking journals in the field of pain. Anesth Analg. 2017;125(4):1348–54.

Thabut G, Estellat C, Boutron I, Samama CM, Ravaud P. Methodological issues in trials assessing primary prophylaxis of venous thrombo-embolism. Eur Heart J. 2005;27(2):227–36.

Puljak L, Riva N, Parmelli E, González-Lorenzo M, Moja L, Pieper D. Data extraction methods: an analysis of internal reporting discrepancies in single manuscripts and practical advice. J Clin Epidemiol. 2020;117:158–64.

Ritchie A, Seubert L, Clifford R, Perry D, Bond C. Do randomised controlled trials relevant to pharmacy meet best practice standards for quality conduct and reporting? A systematic review. Int J Pharm Pract. 2019.

Babic A, Vuka I, Saric F, Proloscic I, Slapnicar E, Cavar J, Pericic TP, Pieper D, Puljak L. Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent. J Clin Epidemiol. 2019.

Tan A, Porcher R, Crequit P, Ravaud P, Dechartres A. Differences in treatment effect size between overall survival and progression-free survival in immunotherapy trials: a Meta-epidemiologic study of trials with results posted at ClinicalTrials.gov. J Clin Oncol. 2017;35(15):1686–94.

Croitoru D, Huang Y, Kurdina A, Chan AW, Drucker AM. Quality of reporting in systematic reviews published in dermatology journals. Br J Dermatol. 2020;182(6):1469–76.

Khan MS, Ochani RK, Shaikh A, Vaduganathan M, Khan SU, Fatima K, Yamani N, Mandrola J, Doukky R, Krasuski RA: Assessing the Quality of Reporting of Harms in Randomized Controlled Trials Published in High Impact Cardiovascular Journals. Eur Heart J Qual Care Clin Outcomes 2019.

Rosmarakis ES, Soteriades ES, Vergidis PI, Kasiakou SK, Falagas ME. From conference abstract to full paper: differences between data presented in conferences and journals. FASEB J. 2005;19(7):673–80.

Mueller M, D’Addario M, Egger M, Cevallos M, Dekkers O, Mugglin C, Scott P. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol. 2018;18(1):44.

Li G, Abbade LPF, Nwosu I, Jin Y, Leenus A, Maaz M, Wang M, Bhatt M, Zielinski L, Sanger N, et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol. 2017;17(1):181.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203.

Analytical study [ https://medical-dictionary.thefreedictionary.com/analytical+study ]. Accessed 31 Aug 2020.

Tricco AC, Tetzlaff J, Pham B, Brehaut J, Moher D. Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study. J Clin Epidemiol. 2009;62(4):380–6 e381.

Schalken N, Rietbergen C. The reporting quality of systematic reviews and Meta-analyses in industrial and organizational psychology: a systematic review. Front Psychol. 2017;8:1395.

Ranker LR, Petersen JM, Fox MP. Awareness of and potential for dependent error in the observational epidemiologic literature: A review. Ann Epidemiol. 2019;36:15–9 e12.

Paquette M, Alotaibi AM, Nieuwlaat R, Santesso N, Mbuagbaw L. A meta-epidemiological study of subgroup analyses in cochrane systematic reviews of atrial fibrillation. Syst Rev. 2019;8(1):241.

Download references

Acknowledgements

This work did not receive any dedicated funding.

Author information

Authors and affiliations.

Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada

Lawrence Mbuagbaw, Daeria O. Lawson & Lehana Thabane

Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph’s Healthcare—Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada

Lawrence Mbuagbaw & Lehana Thabane

Centre for the Development of Best Practices in Health, Yaoundé, Cameroon

Lawrence Mbuagbaw

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Department of Epidemiology and Biostatistics, School of Public Health – Bloomington, Indiana University, Bloomington, IN, 47405, USA

David B. Allison

Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON, Canada

Lehana Thabane

Centre for Evaluation of Medicine, St. Joseph’s Healthcare-Hamilton, Hamilton, ON, Canada

Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada

You can also search for this author in PubMed   Google Scholar

Contributions

LM conceived the idea and drafted the outline and paper. DOL and LT commented on the idea and draft outline. LM, LP and DOL performed literature searches and data extraction. All authors (LM, DOL, LT, LP, DBA) reviewed several draft versions of the manuscript and approved the final manuscript.

Corresponding author

Correspondence to Lawrence Mbuagbaw .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mbuagbaw, L., Lawson, D.O., Puljak, L. et al. A tutorial on methodological studies: the what, when, how and why. BMC Med Res Methodol 20 , 226 (2020). https://doi.org/10.1186/s12874-020-01107-7

Download citation

Received : 27 May 2020

Accepted : 27 August 2020

Published : 07 September 2020

DOI : https://doi.org/10.1186/s12874-020-01107-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Methodological study
• Meta-epidemiology
• Research methods
• Research-on-research

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

What are Analytical Study Designs?

• Research Process
• Peer Review

Analytical study designs can be experimental or observational and each type has its own features. In this article, you'll learn the main types of designs and how to figure out which one you'll need for your study.

Updated on September 19, 2022

A study design is critical to your research study because it determines exactly how you will collect and analyze your data. If your study aims to study the relationship between two variables, then an analytical study design is the right choice.

But how do you know which type of analytical study design is best for your specific research question? It's necessary to have a clear plan before you begin data collection. Lots of researchers, sadly, speed through this or don't do it at all.

When are analytical study designs used?

A study design is a systematic plan, developed so you can carry out your research study effectively and efficiently. Having a design is important because it will determine the right methodologies for your study. Using the right study design makes your results more credible, valid, and coherent.

Descriptive vs. analytical studies

Study designs can be broadly divided into either descriptive or analytical.

Descriptive studies describe characteristics such as patterns or trends. They answer the questions of what, who, where, and when, and they generate hypotheses. They include case reports and qualitative studies.

Analytical study designs quantify a relationship between different variables. They answer the questions of why and how. They're used to test hypotheses and make predictions.

Experimental and observational

Analytical study designs can be either experimental or observational. In experimental studies, researchers manipulate something in a population of interest and examine its effects. These designs are used to establish a causal link between two variables.

In observational studies, in contrast, researchers observe the effects of a treatment or intervention without manipulating anything. Observational studies are most often used to study larger patterns over longer periods.

Experimental study designs

Experimental study designs are when a researcher introduces a change in one group and not in another. Typically, these are used when researchers are interested in the effects of this change on some outcome. It's important to try to ensure that both groups are equivalent at baseline to make sure that any differences that arise are from any introduced change.

In one study, Reiner and colleagues studied the effects of a mindfulness intervention on pain perception . The researchers randomly assigned participants into an experimental group that received a mindfulness training program for two weeks. The rest of the participants were placed in a control group that did not receive the intervention.

Experimental studies help us establish causality. This is critical in science because we want to know whether one variable leads to a change, or causes another. Establishing causality leads to higher internal validity and makes results reproducible.

Experimental designs include randomized control trials (RCTs), nonrandomized control trials (non-RCTs), and crossover designs. Read on to learn the differences.

Randomized control trials

In an RCT, one group of individuals receives an intervention or a treatment, while another does not. It's then possible to investigate what happens to the participants in each group.

Another important feature of RCTs is that participants are randomly assigned to study groups. This helps to limit certain biases and retain better control. Randomization also lets researchers pinpoint any differences in outcomes to the intervention received during the trial. RTCs are considered the gold standard in biomedical research and are considered to provide the best kind of evidence.

For example, one RCT looked at whether an exercise intervention impacts depression . Researchers randomly placed patients with depressive symptoms into intervention groups containing different types of exercise (i.e., light, moderate, or strong). Another group received usual medications or no exercise interventions.

Results showed that after the 12-week trial, patients in all exercise groups had decreased depression levels compared to the control group. This means that by using an RCT design, researchers can now safely assume that the exercise variable has a positive impact on depression.

However, RCTs are not without drawbacks. In the example above, we don't know if exercise still has a positive impact on depression in the long term. This is because it's not feasible to keep people under these controlled settings for a long time.

Advantages of RCTs

• It is possible to infer causality
• Everything is properly controlled, so very little is left to chance or bias
• Can be certain that any difference is coming from the intervention

Disadvantages of RCTs

• Expensive and can be time-consuming
• Can take years for results to be available
• Cannot be done for certain types of questions due to ethical reasons, such as asking participants to undergo harmful treatment
• Limited in how many participants researchers can adequately manage in one study or trial
• Not feasible for people to live under controlled conditions for a long time

Nonrandomized controlled trials

Nonrandomized controlled trials are a type of nonrandomized controlled studies (NRS) where the allocation of participants to intervention groups is not done randomly . Here, researchers purposely assign some participants to one group and others to another group based on certain features. Alternatively, participants can sometimes also decide which group they want to be in.

For example, in one study, clinicians were interested in the impact of stroke recovery after being in an enriched versus non-enriched hospital environment . Patients were selected for the trial if they fulfilled certain requirements common to stroke recovery. Then, the intervention group was given access to an enriched environment (i.e. internet access, reading, going outside), and another group was not. Results showed that the enriched group performed better on cognitive tasks.

NRS are useful in medical research because they help study phenomena that would be difficult to measure with an RCT. However, one of their major drawbacks is that we cannot be sure if the intervention leads to the outcome. In the above example, we can't say for certain whether those patients improved after stroke because they were in the enriched environment or whether there were other variables at play.

Advantages of NRS's

• Good option when randomized control trials are not feasible
• More flexible than RCTs

Disadvantages of NRS's

• Can't be sure if the groups have underlying differences
• Introduces risk of bias and confounds

Crossover study

In a crossover design, each participant receives a sequence of different treatments. Crossover designs can be applied to RCTs, in which each participant is randomly assigned to different study groups.

For example, one study looked at the effects of replacing butter with margarine on lipoproteins levels in individuals with cholesterol . Patients were randomly assigned to a 6-week butter diet, followed by a 6-week margarine diet. In between both diets, participants ate a normal diet for 5 weeks.

These designs are helpful because they reduce bias. In the example above, each participant completed both interventions, making them serve as their own control. However, we don't know if eating butter or margarine first leads to certain results in some subjects.

Advantages of crossover studies

• Each participant serves as their own control, reducing confounding variables
• Require fewer participants, so they have better statistical power

Disadvantages of crossover studies

• Susceptible to order effects, meaning the order in which a treatment was given may have an effect
• Carry-over effects between treatments

Observational studies

In observational studies, researchers watch (observe) the effects of a treatment or intervention without trying to change anything in the population. Observational studies help us establish broad trends and patterns in large-scale datasets or populations. They are also a great alternative when an experimental study is not an option.

Unlike experimental research, observational studies do not help us establish causality. This is because researchers do not actively control any variables. Rather, they investigate statistical relationships between them. Often this is done using a correlational approach.

For example, researchers would like to examine the effects of daily fiber intake on bone density . They conduct a large-scale survey of thousands of individuals to examine correlations of fiber intake with different health measures.

The main observational studies are case-control, cohort, and cross-sectional. Let's take a closer look at each one below.

Case-control study

A case-control is a type of observational design in which researchers identify individuals with an existing health situation (cases) and a similar group without the health issue (controls). The cases and the controls are then compared based on some measurements.

Frequently, data collection in a case-control study is retroactive (i.e., backwards in time). This is because participants have already been exposed to the event in question. Additionally, researchers must go through records and patient files to obtain the records for this study design.

For example, a group of researchers examined whether using sleeping pills puts people at risk of Alzheimer's disease . They selected 1976 individuals that received a dementia diagnosis (“cases”) with 7184 other individuals (“controls”). Cases and controls were matched on specific measures such as sex and age. Patient data was consulted to find out how much sleeping pills were consumed over the course of a certain time.

Case-control is ideal for situations where cases are easy to pick out and compare. For instance, in studying rare diseases or outbreaks.

Advantages of case-control studies

• Feasible for rare diseases
• Cheaper and easier to do than an RCT

Disadvantages of case-control studies

• Relies on patient records, which could be lost or damaged
• Potential recall and selection bias

Cohort study (longitudinal)

A cohort is a group of people who are linked in some way. For instance, a birth year cohort is all people born in a specific year. In cohort studies, researchers compare what happens to individuals in the cohort that have been exposed to some variable compared with those that haven't on different variables. They're also called longitudinal studies.

The cohort is then repeatedly assessed on variables of interest over a period of time. There is no set amount of time required for cohort studies. They can range from a few weeks to many years.

Cohort studies can be prospective. In this case, individuals are followed for some time into the future. They can also be retrospective, where data is collected on a cohort from records.

One of the longest cohort studies today is The Harvard Study of Adult Development . This cohort study has been tracking various health outcomes of 268 Harvard graduates and 456 poor individuals in Boston from 1939 to 2014. Physical screenings, blood samples, brain scans and surveys were collected on this cohort for over 70 years. This study has produced a wealth of knowledge on outcomes throughout life.

A cohort study design is a good option when you have a specific group of people you want to study over time. However, a major drawback is that they take a long time and lack control.

Advantages of cohort studies

• Ethically safe
• Allows you to study multiple outcome variables
• Establish trends and patterns

Disadvantages of cohort studies

• Time consuming and expensive
• Can take many years for results to be revealed
• Too many variables to manage
• Depending on length of study, can have many changes in research personnel

Cross-sectional study

Cross-sectional studies are also known as prevalence studies. They look at the relationship of specific variables in a population in one given time. In cross-sectional studies, the researcher does not try to manipulate any of the variables, just study them using statistical analyses. Cross-sectional studies are also called snapshots of a certain variable or time.

For example, researchers wanted to determine the prevalence of inappropriate antibiotic use to study the growing concern about antibiotic resistance. Participants completed a self-administered questionnaire assessing their knowledge and attitude toward antibiotic use. Then, researchers performed statistical analyses on their responses to determine the relationship between the variables.

Cross-sectional study designs are ideal when gathering initial data on a research question. This data can then be analyzed again later. By knowing the public's general attitudes towards antibiotics, this information can then be relayed to physicians or public health authorities. However, it's often difficult to determine how long these results stay true for.

Advantages of cross-sectional studies

• Fast and inexpensive
• Provides a great deal of information for a given time point
• Leaves room for secondary analysis

Disadvantages of cross-sectional studies

• Requires a large sample to be accurate
• Not clear how long results remain true for
• Do not provide information on causality
• Cannot be used to establish long-term trends because data is only for a given time

So, how about your next study?

Whether it's an RCT, a case-control, or even a qualitative study, AJE has services to help you at every step of the publication process. Get expert guidance and publish your work for the world to see.

The AJE Team

See our "Privacy Policy"

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Data Analysis in Research: Types & Methods

Content Index

Why analyze data in research?

Types of data in research, finding patterns in the qualitative data, methods used for data analysis in qualitative research, preparing data for analysis, methods used for data analysis in quantitative research, considerations in research data analysis, what is data analysis in research.

Definition of research in data analysis: According to LeCompte and Schensul, research data analysis is a process used by researchers to reduce data to a story and interpret it to derive insights. The data analysis process helps reduce a large chunk of data into smaller fragments, which makes sense. 

Three essential things occur during the data analysis process — the first is data organization . Summarization and categorization together contribute to becoming the second known method used for data reduction. It helps find patterns and themes in the data for easy identification and linking. The third and last way is data analysis – researchers do it in both top-down and bottom-up fashion.

LEARN ABOUT: Research Process Steps

On the other hand, Marshall and Rossman describe data analysis as a messy, ambiguous, and time-consuming but creative and fascinating process through which a mass of collected data is brought to order, structure and meaning.

We can say that “the data analysis and data interpretation is a process representing the application of deductive and inductive logic to the research and data analysis.”

Researchers rely heavily on data as they have a story to tell or research problems to solve. It starts with a question, and data is nothing but an answer to that question. But, what if there is no question to ask? Well! It is possible to explore data even without a problem – we call it ‘Data Mining’, which often reveals some interesting patterns within the data that are worth exploring.

Irrelevant to the type of data researchers explore, their mission and audiences’ vision guide them to find the patterns to shape the story they want to tell. One of the essential things expected from researchers while analyzing data is to stay open and remain unbiased toward unexpected patterns, expressions, and results. Remember, sometimes, data analysis tells the most unforeseen yet exciting stories that were not expected when initiating data analysis. Therefore, rely on the data you have at hand and enjoy the journey of exploratory research. 

Create a Free Account

Every kind of data has a rare quality of describing things after assigning a specific value to it. For analysis, you need to organize these values, processed and presented in a given context, to make it useful. Data can be in different forms; here are the primary data types.

• Qualitative data: When the data presented has words and descriptions, then we call it qualitative data . Although you can observe this data, it is subjective and harder to analyze data in research, especially for comparison. Example: Quality data represents everything describing taste, experience, texture, or an opinion that is considered quality data. This type of data is usually collected through focus groups, personal qualitative interviews , qualitative observation or using open-ended questions in surveys.
• Quantitative data: Any data expressed in numbers of numerical figures are called quantitative data . This type of data can be distinguished into categories, grouped, measured, calculated, or ranked. Example: questions such as age, rank, cost, length, weight, scores, etc. everything comes under this type of data. You can present such data in graphical format, charts, or apply statistical analysis methods to this data. The (Outcomes Measurement Systems) OMS questionnaires in surveys are a significant source of collecting numeric data.
• Categorical data: It is data presented in groups. However, an item included in the categorical data cannot belong to more than one group. Example: A person responding to a survey by telling his living style, marital status, smoking habit, or drinking habit comes under the categorical data. A chi-square test is a standard method used to analyze this data.

Learn More : Examples of Qualitative Data in Education

Data analysis in qualitative research

Data analysis and qualitative data research work a little differently from the numerical data as the quality data is made up of words, descriptions, images, objects, and sometimes symbols. Getting insight from such complicated information is a complicated process. Hence it is typically used for exploratory research and data analysis .

Although there are several ways to find patterns in the textual information, a word-based method is the most relied and widely used global technique for research and data analysis. Notably, the data analysis process in qualitative research is manual. Here the researchers usually read the available data and find repetitive or commonly used words. 

For example, while studying data collected from African countries to understand the most pressing issues people face, researchers might find  “food”  and  “hunger” are the most commonly used words and will highlight them for further analysis.

LEARN ABOUT: Level of Analysis

The keyword context is another widely used word-based technique. In this method, the researcher tries to understand the concept by analyzing the context in which the participants use a particular keyword.  

For example , researchers conducting research and data analysis for studying the concept of ‘diabetes’ amongst respondents might analyze the context of when and how the respondent has used or referred to the word ‘diabetes.’

The scrutiny-based technique is also one of the highly recommended  text analysis  methods used to identify a quality data pattern. Compare and contrast is the widely used method under this technique to differentiate how a specific text is similar or different from each other. 

For example: To find out the “importance of resident doctor in a company,” the collected data is divided into people who think it is necessary to hire a resident doctor and those who think it is unnecessary. Compare and contrast is the best method that can be used to analyze the polls having single-answer questions types .

Metaphors can be used to reduce the data pile and find patterns in it so that it becomes easier to connect data with theory.

Variable Partitioning is another technique used to split variables so that researchers can find more coherent descriptions and explanations from the enormous data.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

There are several techniques to analyze the data in qualitative research, but here are some commonly used methods,

• Content Analysis:  It is widely accepted and the most frequently employed technique for data analysis in research methodology. It can be used to analyze the documented information from text, images, and sometimes from the physical items. It depends on the research questions to predict when and where to use this method.
• Narrative Analysis: This method is used to analyze content gathered from various sources such as personal interviews, field observation, and  surveys . The majority of times, stories, or opinions shared by people are focused on finding answers to the research questions.
• Discourse Analysis:  Similar to narrative analysis, discourse analysis is used to analyze the interactions with people. Nevertheless, this particular method considers the social context under which or within which the communication between the researcher and respondent takes place. In addition to that, discourse analysis also focuses on the lifestyle and day-to-day environment while deriving any conclusion.
• Grounded Theory:  When you want to explain why a particular phenomenon happened, then using grounded theory for analyzing quality data is the best resort. Grounded theory is applied to study data about the host of similar cases occurring in different settings. When researchers are using this method, they might alter explanations or produce new ones until they arrive at some conclusion.

LEARN ABOUT: 12 Best Tools for Researchers

Data analysis in quantitative research

The first stage in research and data analysis is to make it for the analysis so that the nominal data can be converted into something meaningful. Data preparation consists of the below phases.

Phase I: Data Validation

Data validation is done to understand if the collected data sample is per the pre-set standards, or it is a biased data sample again divided into four different stages

• Fraud: To ensure an actual human being records each response to the survey or the questionnaire
• Screening: To make sure each participant or respondent is selected or chosen in compliance with the research criteria
• Procedure: To ensure ethical standards were maintained while collecting the data sample
• Completeness: To ensure that the respondent has answered all the questions in an online survey. Else, the interviewer had asked all the questions devised in the questionnaire.

Phase II: Data Editing

More often, an extensive research data sample comes loaded with errors. Respondents sometimes fill in some fields incorrectly or sometimes skip them accidentally. Data editing is a process wherein the researchers have to confirm that the provided data is free of such errors. They need to conduct necessary checks and outlier checks to edit the raw edit and make it ready for analysis.

Phase III: Data Coding

Out of all three, this is the most critical phase of data preparation associated with grouping and assigning values to the survey responses . If a survey is completed with a 1000 sample size, the researcher will create an age bracket to distinguish the respondents based on their age. Thus, it becomes easier to analyze small data buckets rather than deal with the massive data pile.

LEARN ABOUT: Steps in Qualitative Research

After the data is prepared for analysis, researchers are open to using different research and data analysis methods to derive meaningful insights. For sure, statistical analysis plans are the most favored to analyze numerical data. In statistical analysis, distinguishing between categorical data and numerical data is essential, as categorical data involves distinct categories or labels, while numerical data consists of measurable quantities. The method is again classified into two groups. First, ‘Descriptive Statistics’ used to describe data. Second, ‘Inferential statistics’ that helps in comparing the data .

Descriptive statistics

This method is used to describe the basic features of versatile types of data in research. It presents the data in such a meaningful way that pattern in the data starts making sense. Nevertheless, the descriptive analysis does not go beyond making conclusions. The conclusions are again based on the hypothesis researchers have formulated so far. Here are a few major types of descriptive analysis methods.

Measures of Frequency

• Count, Percent, Frequency
• It is used to denote home often a particular event occurs.
• Researchers use it when they want to showcase how often a response is given.

Measures of Central Tendency

• Mean, Median, Mode
• The method is widely used to demonstrate distribution by various points.
• Researchers use this method when they want to showcase the most commonly or averagely indicated response.

Measures of Dispersion or Variation

• Range, Variance, Standard deviation
• Here the field equals high/low points.
• Variance standard deviation = difference between the observed score and mean
• It is used to identify the spread of scores by stating intervals.
• Researchers use this method to showcase data spread out. It helps them identify the depth until which the data is spread out that it directly affects the mean.

Measures of Position

• Percentile ranks, Quartile ranks
• It relies on standardized scores helping researchers to identify the relationship between different scores.
• It is often used when researchers want to compare scores with the average count.

For quantitative research use of descriptive analysis often give absolute numbers, but the in-depth analysis is never sufficient to demonstrate the rationale behind those numbers. Nevertheless, it is necessary to think of the best method for research and data analysis suiting your survey questionnaire and what story researchers want to tell. For example, the mean is the best way to demonstrate the students’ average scores in schools. It is better to rely on the descriptive statistics when the researchers intend to keep the research or outcome limited to the provided  sample  without generalizing it. For example, when you want to compare average voting done in two different cities, differential statistics are enough.

Descriptive analysis is also called a ‘univariate analysis’ since it is commonly used to analyze a single variable.

Inferential statistics

Inferential statistics are used to make predictions about a larger population after research and data analysis of the representing population’s collected sample. For example, you can ask some odd 100 audiences at a movie theater if they like the movie they are watching. Researchers then use inferential statistics on the collected  sample  to reason that about 80-90% of people like the movie. 

Here are two significant areas of inferential statistics.

• Estimating parameters: It takes statistics from the sample research data and demonstrates something about the population parameter.
• Hypothesis test: I t’s about sampling research data to answer the survey research questions. For example, researchers might be interested to understand if the new shade of lipstick recently launched is good or not, or if the multivitamin capsules help children to perform better at games.

These are sophisticated analysis methods used to showcase the relationship between different variables instead of describing a single variable. It is often used when researchers want something beyond absolute numbers to understand the relationship between variables.

Here are some of the commonly used methods for data analysis in research.

• Correlation: When researchers are not conducting experimental research or quasi-experimental research wherein the researchers are interested to understand the relationship between two or more variables, they opt for correlational research methods.
• Cross-tabulation: Also called contingency tables,  cross-tabulation  is used to analyze the relationship between multiple variables.  Suppose provided data has age and gender categories presented in rows and columns. A two-dimensional cross-tabulation helps for seamless data analysis and research by showing the number of males and females in each age category.
• Regression analysis: For understanding the strong relationship between two variables, researchers do not look beyond the primary and commonly used regression analysis method, which is also a type of predictive analysis used. In this method, you have an essential factor called the dependent variable. You also have multiple independent variables in regression analysis. You undertake efforts to find out the impact of independent variables on the dependent variable. The values of both independent and dependent variables are assumed as being ascertained in an error-free random manner.
• Frequency tables: The statistical procedure is used for testing the degree to which two or more vary or differ in an experiment. A considerable degree of variation means research findings were significant. In many contexts, ANOVA testing and variance analysis are similar.
• Analysis of variance: The statistical procedure is used for testing the degree to which two or more vary or differ in an experiment. A considerable degree of variation means research findings were significant. In many contexts, ANOVA testing and variance analysis are similar.
• Researchers must have the necessary research skills to analyze and manipulation the data , Getting trained to demonstrate a high standard of research practice. Ideally, researchers must possess more than a basic understanding of the rationale of selecting one statistical method over the other to obtain better data insights.
• Usually, research and data analytics projects differ by scientific discipline; therefore, getting statistical advice at the beginning of analysis helps design a survey questionnaire, select data collection methods , and choose samples.

LEARN ABOUT: Best Data Collection Tools

• The primary aim of data research and analysis is to derive ultimate insights that are unbiased. Any mistake in or keeping a biased mind to collect data, selecting an analysis method, or choosing  audience  sample il to draw a biased inference.
• Irrelevant to the sophistication used in research data and analysis is enough to rectify the poorly defined objective outcome measurements. It does not matter if the design is at fault or intentions are not clear, but lack of clarity might mislead readers, so avoid the practice.
• The motive behind data analysis in research is to present accurate and reliable data. As far as possible, avoid statistical errors, and find a way to deal with everyday challenges like outliers, missing data, data altering, data mining , or developing graphical representation.

LEARN MORE: Descriptive Research vs Correlational Research The sheer amount of data generated daily is frightening. Especially when data analysis has taken center stage. in 2018. In last year, the total data supply amounted to 2.8 trillion gigabytes. Hence, it is clear that the enterprises willing to survive in the hypercompetitive world must possess an excellent capability to analyze complex research data, derive actionable insights, and adapt to the new market needs.

LEARN ABOUT: Average Order Value

QuestionPro is an online survey platform that empowers organizations in data analysis and research and provides them a medium to collect data by creating appealing surveys.

MORE LIKE THIS

When You Have Something Important to Say, You want to Shout it From the Rooftops

Jun 28, 2024

The Item I Failed to Leave Behind — Tuesday CX Thoughts

Jun 25, 2024

Feedback Loop: What It Is, Types & How It Works?

Jun 21, 2024

QuestionPro Thrive: A Space to Visualize & Share the Future of Technology

Jun 18, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Choosing the Right Research Methodology: A Guide for Researchers

• 3 minute read
• 43.4K views

Table of Contents

Choosing an optimal research methodology is crucial for the success of any research project. The methodology you select will determine the type of data you collect, how you collect it, and how you analyse it. Understanding the different types of research methods available along with their strengths and weaknesses, is thus imperative to make an informed decision.

Understanding different research methods:

There are several research methods available depending on the type of study you are conducting, i.e., whether it is laboratory-based, clinical, epidemiological, or survey based . Some common methodologies include qualitative research, quantitative research, experimental research, survey-based research, and action research. Each method can be opted for and modified, depending on the type of research hypotheses and objectives.

Qualitative vs quantitative research:

When deciding on a research methodology, one of the key factors to consider is whether your research will be qualitative or quantitative. Qualitative research is used to understand people’s experiences, concepts, thoughts, or behaviours . Quantitative research, on the contrary, deals with numbers, graphs, and charts, and is used to test or confirm hypotheses, assumptions, and theories. 

Qualitative research methodology:

Qualitative research is often used to examine issues that are not well understood, and to gather additional insights on these topics. Qualitative research methods include open-ended survey questions, observations of behaviours described through words, and reviews of literature that has explored similar theories and ideas. These methods are used to understand how language is used in real-world situations, identify common themes or overarching ideas, and describe and interpret various texts. Data analysis for qualitative research typically includes discourse analysis, thematic analysis, and textual analysis. 

Quantitative research methodology:

The goal of quantitative research is to test hypotheses, confirm assumptions and theories, and determine cause-and-effect relationships. Quantitative research methods include experiments, close-ended survey questions, and countable and numbered observations. Data analysis for quantitative research relies heavily on statistical methods.

Analysing qualitative vs quantitative data:

The methods used for data analysis also differ for qualitative and quantitative research. As mentioned earlier, quantitative data is generally analysed using statistical methods and does not leave much room for speculation. It is more structured and follows a predetermined plan. In quantitative research, the researcher starts with a hypothesis and uses statistical methods to test it. Contrarily, methods used for qualitative data analysis can identify patterns and themes within the data, rather than provide statistical measures of the data. It is an iterative process, where the researcher goes back and forth trying to gauge the larger implications of the data through different perspectives and revising the analysis if required.

When to use qualitative vs quantitative research:

The choice between qualitative and quantitative research will depend on the gap that the research project aims to address, and specific objectives of the study. If the goal is to establish facts about a subject or topic, quantitative research is an appropriate choice. However, if the goal is to understand people’s experiences or perspectives, qualitative research may be more suitable. 

Conclusion:

In conclusion, an understanding of the different research methods available, their applicability, advantages, and disadvantages is essential for making an informed decision on the best methodology for your project. If you need any additional guidance on which research methodology to opt for, you can head over to Elsevier Author Services (EAS). EAS experts will guide you throughout the process and help you choose the perfect methodology for your research goals.

Why is data validation important in research?

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

Writing a good review article

Scholarly Sources: What are They and Where can You Find Them?

Input your search keywords and press Enter.

• How it works

Published by Nicolas at March 21st, 2024 , Revised On March 12, 2024

The Ultimate Guide To Research Methodology

Research methodology is a crucial aspect of any investigative process, serving as the blueprint for the entire research journey. If you are stuck in the methodology section of your research paper , then this blog will guide you on what is a research methodology, its types and how to successfully conduct one. 

Table of Contents

What Is Research Methodology?

Research methodology can be defined as the systematic framework that guides researchers in designing, conducting, and analyzing their investigations. It encompasses a structured set of processes, techniques, and tools employed to gather and interpret data, ensuring the reliability and validity of the research findings. 

Research methodology is not confined to a singular approach; rather, it encapsulates a diverse range of methods tailored to the specific requirements of the research objectives.

Here is why Research methodology is important in academic and professional settings.

Facilitating Rigorous Inquiry

Research methodology forms the backbone of rigorous inquiry. It provides a structured approach that aids researchers in formulating precise thesis statements , selecting appropriate methodologies, and executing systematic investigations. This, in turn, enhances the quality and credibility of the research outcomes.

Ensuring Reproducibility And Reliability

In both academic and professional contexts, the ability to reproduce research outcomes is paramount. A well-defined research methodology establishes clear procedures, making it possible for others to replicate the study. This not only validates the findings but also contributes to the cumulative nature of knowledge.

Guiding Decision-Making Processes

In professional settings, decisions often hinge on reliable data and insights. Research methodology equips professionals with the tools to gather pertinent information, analyze it rigorously, and derive meaningful conclusions.

This informed decision-making is instrumental in achieving organizational goals and staying ahead in competitive environments.

Contributing To Academic Excellence

For academic researchers, adherence to robust research methodology is a hallmark of excellence. Institutions value research that adheres to high standards of methodology, fostering a culture of academic rigour and intellectual integrity. Furthermore, it prepares students with critical skills applicable beyond academia.

Enhancing Problem-Solving Abilities

Research methodology instills a problem-solving mindset by encouraging researchers to approach challenges systematically. It equips individuals with the skills to dissect complex issues, formulate hypotheses , and devise effective strategies for investigation.

Understanding Research Methodology

In the pursuit of knowledge and discovery, understanding the fundamentals of research methodology is paramount. 

Basics Of Research

Research, in its essence, is a systematic and organized process of inquiry aimed at expanding our understanding of a particular subject or phenomenon. It involves the exploration of existing knowledge, the formulation of hypotheses, and the collection and analysis of data to draw meaningful conclusions. 

Research is a dynamic and iterative process that contributes to the continuous evolution of knowledge in various disciplines.

Types of Research

Research takes on various forms, each tailored to the nature of the inquiry. Broadly classified, research can be categorized into two main types:

• Quantitative Research: This type involves the collection and analysis of numerical data to identify patterns, relationships, and statistical significance. It is particularly useful for testing hypotheses and making predictions.
• Qualitative Research: Qualitative research focuses on understanding the depth and details of a phenomenon through non-numerical data. It often involves methods such as interviews, focus groups, and content analysis, providing rich insights into complex issues.

Components Of Research Methodology

To conduct effective research, one must go through the different components of research methodology. These components form the scaffolding that supports the entire research process, ensuring its coherence and validity.

Research Design

Research design serves as the blueprint for the entire research project. It outlines the overall structure and strategy for conducting the study. The three primary types of research design are:

• Exploratory Research: Aimed at gaining insights and familiarity with the topic, often used in the early stages of research.
• Descriptive Research: Involves portraying an accurate profile of a situation or phenomenon, answering the ‘what,’ ‘who,’ ‘where,’ and ‘when’ questions.
• Explanatory Research: Seeks to identify the causes and effects of a phenomenon, explaining the ‘why’ and ‘how.’

Data Collection Methods

Choosing the right data collection methods is crucial for obtaining reliable and relevant information. Common methods include:

• Surveys and Questionnaires: Employed to gather information from a large number of respondents through standardized questions.
• Interviews: In-depth conversations with participants, offering qualitative insights.
• Observation: Systematic watching and recording of behaviour, events, or processes in their natural setting.

Data Analysis Techniques

Once data is collected, analysis becomes imperative to derive meaningful conclusions. Different methodologies exist for quantitative and qualitative data:

• Quantitative Data Analysis: Involves statistical techniques such as descriptive statistics, inferential statistics, and regression analysis to interpret numerical data.
• Qualitative Data Analysis: Methods like content analysis, thematic analysis, and grounded theory are employed to extract patterns, themes, and meanings from non-numerical data.

The research paper we write have:

• Precision and Clarity
• Zero Plagiarism
• High-level Encryption
• Authentic Sources

Choosing a Research Method

Selecting an appropriate research method is a critical decision in the research process. It determines the approach, tools, and techniques that will be used to answer the research questions. 

Quantitative Research Methods

Quantitative research involves the collection and analysis of numerical data, providing a structured and objective approach to understanding and explaining phenomena.

Experimental Research

Experimental research involves manipulating variables to observe the effect on another variable under controlled conditions. It aims to establish cause-and-effect relationships.

Key Characteristics:

• Controlled Environment: Experiments are conducted in a controlled setting to minimize external influences.
• Random Assignment: Participants are randomly assigned to different experimental conditions.
• Quantitative Data: Data collected is numerical, allowing for statistical analysis.

Applications: Commonly used in scientific studies and psychology to test hypotheses and identify causal relationships.

Survey Research

Survey research gathers information from a sample of individuals through standardized questionnaires or interviews. It aims to collect data on opinions, attitudes, and behaviours.

• Structured Instruments: Surveys use structured instruments, such as questionnaires, to collect data.
• Large Sample Size: Surveys often target a large and diverse group of participants.
• Quantitative Data Analysis: Responses are quantified for statistical analysis.

Applications: Widely employed in social sciences, marketing, and public opinion research to understand trends and preferences.

Descriptive Research

Descriptive research seeks to portray an accurate profile of a situation or phenomenon. It focuses on answering the ‘what,’ ‘who,’ ‘where,’ and ‘when’ questions.

• Observation and Data Collection: This involves observing and documenting without manipulating variables.
• Objective Description: Aim to provide an unbiased and factual account of the subject.
• Quantitative or Qualitative Data: T his can include both types of data, depending on the research focus.

Applications: Useful in situations where researchers want to understand and describe a phenomenon without altering it, common in social sciences and education.

Qualitative Research Methods

Qualitative research emphasizes exploring and understanding the depth and complexity of phenomena through non-numerical data.

A case study is an in-depth exploration of a particular person, group, event, or situation. It involves detailed, context-rich analysis.

• Rich Data Collection: Uses various data sources, such as interviews, observations, and documents.
• Contextual Understanding: Aims to understand the context and unique characteristics of the case.
• Holistic Approach: Examines the case in its entirety.

Applications: Common in social sciences, psychology, and business to investigate complex and specific instances.

Ethnography

Ethnography involves immersing the researcher in the culture or community being studied to gain a deep understanding of their behaviours, beliefs, and practices.

• Participant Observation: Researchers actively participate in the community or setting.
• Holistic Perspective: Focuses on the interconnectedness of cultural elements.
• Qualitative Data: In-depth narratives and descriptions are central to ethnographic studies.

Applications: Widely used in anthropology, sociology, and cultural studies to explore and document cultural practices.

Grounded Theory

Grounded theory aims to develop theories grounded in the data itself. It involves systematic data collection and analysis to construct theories from the ground up.

• Constant Comparison: Data is continually compared and analyzed during the research process.
• Inductive Reasoning: Theories emerge from the data rather than being imposed on it.
• Iterative Process: The research design evolves as the study progresses.

Applications: Commonly applied in sociology, nursing, and management studies to generate theories from empirical data.

Research design is the structural framework that outlines the systematic process and plan for conducting a study. It serves as the blueprint, guiding researchers on how to collect, analyze, and interpret data.

Exploratory, Descriptive, And Explanatory Designs

Exploratory design.

Exploratory research design is employed when a researcher aims to explore a relatively unknown subject or gain insights into a complex phenomenon.

• Flexibility: Allows for flexibility in data collection and analysis.
• Open-Ended Questions: Uses open-ended questions to gather a broad range of information.
• Preliminary Nature: Often used in the initial stages of research to formulate hypotheses.

Applications: Valuable in the early stages of investigation, especially when the researcher seeks a deeper understanding of a subject before formalizing research questions.

Descriptive Design

Descriptive research design focuses on portraying an accurate profile of a situation, group, or phenomenon.

• Structured Data Collection: Involves systematic and structured data collection methods.
• Objective Presentation: Aims to provide an unbiased and factual account of the subject.
• Quantitative or Qualitative Data: Can incorporate both types of data, depending on the research objectives.

Applications: Widely used in social sciences, marketing, and educational research to provide detailed and objective descriptions.

Explanatory Design

Explanatory research design aims to identify the causes and effects of a phenomenon, explaining the ‘why’ and ‘how’ behind observed relationships.

• Causal Relationships: Seeks to establish causal relationships between variables.
• Controlled Variables : Often involves controlling certain variables to isolate causal factors.
• Quantitative Analysis: Primarily relies on quantitative data analysis techniques.

Applications: Commonly employed in scientific studies and social sciences to delve into the underlying reasons behind observed patterns.

Cross-Sectional Vs. Longitudinal Designs

Cross-sectional design.

Cross-sectional designs collect data from participants at a single point in time.

• Snapshot View: Provides a snapshot of a population at a specific moment.
• Efficiency: More efficient in terms of time and resources.
• Limited Temporal Insights: Offers limited insights into changes over time.

Applications: Suitable for studying characteristics or behaviours that are stable or not expected to change rapidly.

Longitudinal Design

Longitudinal designs involve the collection of data from the same participants over an extended period.

• Temporal Sequence: Allows for the examination of changes over time.
• Causality Assessment: Facilitates the assessment of cause-and-effect relationships.
• Resource-Intensive: Requires more time and resources compared to cross-sectional designs.

Applications: Ideal for studying developmental processes, trends, or the impact of interventions over time.

Experimental Vs Non-experimental Designs

Experimental design.

Experimental designs involve manipulating variables under controlled conditions to observe the effect on another variable.

• Causality Inference: Enables the inference of cause-and-effect relationships.
• Quantitative Data: Primarily involves the collection and analysis of numerical data.

Applications: Commonly used in scientific studies, psychology, and medical research to establish causal relationships.

Non-Experimental Design

Non-experimental designs observe and describe phenomena without manipulating variables.

• Natural Settings: Data is often collected in natural settings without intervention.
• Descriptive or Correlational: Focuses on describing relationships or correlations between variables.
• Quantitative or Qualitative Data: This can involve either type of data, depending on the research approach.

Applications: Suitable for studying complex phenomena in real-world settings where manipulation may not be ethical or feasible.

Effective data collection is fundamental to the success of any research endeavour. 

Designing Effective Surveys

Objective Design:

• Clearly define the research objectives to guide the survey design.
• Craft questions that align with the study’s goals and avoid ambiguity.

Structured Format:

• Use a structured format with standardized questions for consistency.
• Include a mix of closed-ended and open-ended questions for detailed insights.

Pilot Testing:

• Conduct pilot tests to identify and rectify potential issues with survey design.
• Ensure clarity, relevance, and appropriateness of questions.

Sampling Strategy:

• Develop a robust sampling strategy to ensure a representative participant group.
• Consider random sampling or stratified sampling based on the research goals.

Conducting Interviews

Establishing Rapport:

• Build rapport with participants to create a comfortable and open environment.
• Clearly communicate the purpose of the interview and the value of participants’ input.

Open-Ended Questions:

• Frame open-ended questions to encourage detailed responses.
• Allow participants to express their thoughts and perspectives freely.

Active Listening:

• Practice active listening to understand areas and gather rich data.
• Avoid interrupting and maintain a non-judgmental stance during the interview.

Ethical Considerations:

• Obtain informed consent and assure participants of confidentiality.
• Be transparent about the study’s purpose and potential implications.

Observation

1. participant observation.

Immersive Participation:

• Actively immerse yourself in the setting or group being observed.
• Develop a deep understanding of behaviours, interactions, and context.

Field Notes:

• Maintain detailed and reflective field notes during observations.
• Document observed patterns, unexpected events, and participant reactions.

Ethical Awareness:

• Be conscious of ethical considerations, ensuring respect for participants.
• Balance the role of observer and participant to minimize bias.

2. Non-participant Observation

Objective Observation:

• Maintain a more detached and objective stance during non-participant observation.
• Focus on recording behaviours, events, and patterns without direct involvement.

Data Reliability:

• Enhance the reliability of data by reducing observer bias.
• Develop clear observation protocols and guidelines.

Contextual Understanding:

• Strive for a thorough understanding of the observed context.
• Consider combining non-participant observation with other methods for triangulation.

Archival Research

1. using existing data.

Identifying Relevant Archives:

• Locate and access archives relevant to the research topic.
• Collaborate with institutions or repositories holding valuable data.

Data Verification:

• Verify the accuracy and reliability of archived data.
• Cross-reference with other sources to ensure data integrity.

Ethical Use:

• Adhere to ethical guidelines when using existing data.
• Respect copyright and intellectual property rights.

2. Challenges and Considerations

Incomplete or Inaccurate Archives:

• Address the possibility of incomplete or inaccurate archival records.
• Acknowledge limitations and uncertainties in the data.

Temporal Bias:

• Recognize potential temporal biases in archived data.
• Consider the historical context and changes that may impact interpretation.

Access Limitations:

• Address potential limitations in accessing certain archives.
• Seek alternative sources or collaborate with institutions to overcome barriers.

Common Challenges in Research Methodology

Conducting research is a complex and dynamic process, often accompanied by a myriad of challenges. Addressing these challenges is crucial to ensure the reliability and validity of research findings.

Sampling Issues

Sampling bias:.

• The presence of sampling bias can lead to an unrepresentative sample, affecting the generalizability of findings.
• Employ random sampling methods and ensure the inclusion of diverse participants to reduce bias.

Sample Size Determination:

• Determining an appropriate sample size is a delicate balance. Too small a sample may lack statistical power, while an excessively large sample may strain resources.
• Conduct a power analysis to determine the optimal sample size based on the research objectives and expected effect size.

Data Quality And Validity

Measurement error:.

• Inaccuracies in measurement tools or data collection methods can introduce measurement errors, impacting the validity of results.
• Pilot test instruments, calibrate equipment, and use standardized measures to enhance the reliability of data.

Construct Validity:

• Ensuring that the chosen measures accurately capture the intended constructs is a persistent challenge.
• Use established measurement instruments and employ multiple measures to assess the same construct for triangulation.

Time And Resource Constraints

Timeline pressures:.

• Limited timeframes can compromise the depth and thoroughness of the research process.
• Develop a realistic timeline, prioritize tasks, and communicate expectations with stakeholders to manage time constraints effectively.

Resource Availability:

• Inadequate resources, whether financial or human, can impede the execution of research activities.
• Seek external funding, collaborate with other researchers, and explore alternative methods that require fewer resources.

Managing Bias in Research

Selection bias:.

• Selecting participants in a way that systematically skews the sample can introduce selection bias.
• Employ randomization techniques, use stratified sampling, and transparently report participant recruitment methods.

Confirmation Bias:

• Researchers may unintentionally favour information that confirms their preconceived beliefs or hypotheses.
• Adopt a systematic and open-minded approach, use blinded study designs, and engage in peer review to mitigate confirmation bias.

Tips On How To Write A Research Methodology

Conducting successful research relies not only on the application of sound methodologies but also on strategic planning and effective collaboration. Here are some tips to enhance the success of your research methodology:

Tip 1. Clear Research Objectives

Well-defined research objectives guide the entire research process. Clearly articulate the purpose of your study, outlining specific research questions or hypotheses.

Tip 2. Comprehensive Literature Review

A thorough literature review provides a foundation for understanding existing knowledge and identifying gaps. Invest time in reviewing relevant literature to inform your research design and methodology.

Tip 3. Detailed Research Plan

A detailed plan serves as a roadmap, ensuring all aspects of the research are systematically addressed. Develop a detailed research plan outlining timelines, milestones, and tasks.

Tip 4. Ethical Considerations

Ethical practices are fundamental to maintaining the integrity of research. Address ethical considerations early, obtain necessary approvals, and ensure participant rights are safeguarded.

Tip 5. Stay Updated On Methodologies

Research methodologies evolve, and staying updated is essential for employing the most effective techniques. Engage in continuous learning by attending workshops, conferences, and reading recent publications.

Tip 6. Adaptability In Methods

Unforeseen challenges may arise during research, necessitating adaptability in methods. Be flexible and willing to modify your approach when needed, ensuring the integrity of the study.

Tip 7. Iterative Approach

Research is often an iterative process, and refining methods based on ongoing findings enhance the study’s robustness. Regularly review and refine your research design and methods as the study progresses.

Frequently Asked Questions

What is the research methodology.

Research methodology is the systematic process of planning, executing, and evaluating scientific investigation. It encompasses the techniques, tools, and procedures used to collect, analyze, and interpret data, ensuring the reliability and validity of research findings.

What are the methodologies in research?

Research methodologies include qualitative and quantitative approaches. Qualitative methods involve in-depth exploration of non-numerical data, while quantitative methods use statistical analysis to examine numerical data. Mixed methods combine both approaches for a comprehensive understanding of research questions.

How to write research methodology?

To write a research methodology, clearly outline the study’s design, data collection, and analysis procedures. Specify research tools, participants, and sampling methods. Justify choices and discuss limitations. Ensure clarity, coherence, and alignment with research objectives for a robust methodology section.

How to write the methodology section of a research paper?

In the methodology section of a research paper, describe the study’s design, data collection, and analysis methods. Detail procedures, tools, participants, and sampling. Justify choices, address ethical considerations, and explain how the methodology aligns with research objectives, ensuring clarity and rigour.

What is mixed research methodology?

Mixed research methodology combines both qualitative and quantitative research approaches within a single study. This approach aims to enhance the details and depth of research findings by providing a more comprehensive understanding of the research problem or question.

You May Also Like

Craft a compelling scholarship motivation letter by showcasing your passion, achievements, and future goals concisely and impactfully.

Are you in need of captivating and achievable research topics within the field of biology? Your quest for the best […]

Explore the essential elements in choosing effective control variables for robust and valid research outcomes.

Ready to place an order?

USEFUL LINKS

Learning resources, company details.

• How It Works

Automated page speed optimizations for fast site performance

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMJ Open Access

Analytical studies: a framework for quality improvement design and analysis

Conducting studies for learning is fundamental to improvement. Deming emphasised that the reason for conducting a study is to provide a basis for action on the system of interest. He classified studies into two types depending on the intended target for action. An enumerative study is one in which action will be taken on the universe that was studied. An analytical study is one in which action will be taken on a cause system to improve the future performance of the system of interest. The aim of an enumerative study is estimation, while an analytical study focuses on prediction. Because of the temporal nature of improvement, the theory and methods for analytical studies are a critical component of the science of improvement.

Introduction: enumerative and analytical studies

Designing studies that make it possible to learn from experience and take action to improve future performance is an essential element of quality improvement. These studies use the now traditional theory established through the work of Fisher, 1 Cox, 2 Campbell and Stanley, 3 and others that is widely used in biomedicine research. These designs are used to discover new phenomena that lead to hypothesis generation, and to explore causal mechanisms, 4 as well as to evaluate efficacy and effectiveness. They include observational, retrospective, prospective, pre-experimental, quasiexperimental, blocking, factorial and time-series designs.

In addition to these classifications of studies, Deming 5 defined a distinction between analytical and enumerative studies which has proven to be fundamental to the science of improvement. Deming based his insight on the distinction between these two approaches that Walter Shewhart had made in 1939 as he helped develop measurement strategies for the then-emerging science of ‘quality control.’ 6 The difference between the two concepts lies in the extrapolation of the results that is intended, and in the target for action based on the inferences that are drawn.

A useful way to appreciate that difference is to contrast the inferences that can be made about the water sampled from two different natural sources ( figure 1 ). The enumerative approach is like the study of water from a pond. Because conditions in the bounded universe of the pond are essentially static over time, analyses of random samples taken from the pond at a given time can be used to estimate the makeup of the entire pond. Statistical methods, such as hypothesis testing and CIs, can be used to make decisions and define the precision of the estimates.

Environment in enumerative and analytical study. Internal validity diagram from Fletcher et al . 7

The analytical approach, in contrast, is like the study of water from a river. The river is constantly moving, and its physical properties are changing (eg, due to snow melt, changes in rainfall, dumping of pollutants). The properties of water in a sample from the river at any given time may not describe the river after the samples are taken and analysed. In fact, without repeated sampling over time, it is difficult to make predictions about water quality, since the river will not be the same river in the future as it was at the time of the sampling.

Deming first discussed these concepts in a 1942 paper, 8 as well as in his 1950 textbook, 9 and in a 1975 paper used the enumerative/analytical terminology to characterise specific study designs. 5 While most books on experimental design describe methods for the design and analysis of enumerative studies, Moen et al 10 describe methods for designing and learning from analytical studies. These methods are graphical and focus on prediction of future performance. The concept of analytical studies became a key element in Deming's ‘system of profound knowledge’ that serves as the intellectual foundation for improvement science. 11 The knowledge framework for the science of improvement, which combines elements of psychology, the Shewhart view of variation, the concept of systems, and the theory of knowledge, informs a number of key principles for the design and analysis of improvement studies:

• Knowledge about improvement begins and ends in experimental data but does not end in the data in which it begins.
• Observations, by themselves, do not constitute knowledge.
• Prediction requires theory regarding mechanisms of change and understanding of context.
• Random sampling from a population or universe (assumed by most statistical methods) is not possible when the population of interest is in the future.
• The conditions during studies for improvement will be different from the conditions under which the results will be used. The major source of uncertainty concerning their use is the difficulty of extrapolating study results to different contexts and under different conditions in the future.
• The wider the range of conditions included in an improvement study, the greater the degree of belief in the validity and generalisation of the conclusions.

The classification of studies into enumerative and analytical categories depends on the intended target for action as the result of the study:

• Enumerative studies assume that when actions are taken as the result of a study, they will be taken on the material in the study population or ‘frame’ that was sampled.

More specifically, the study universe in an enumerative study is the bounded group of items (eg, patients, clinics, providers, etc) possessing certain properties of interest. The universe is defined by a frame, a list of identifiable, tangible units that may be sampled and studied. Random selection methods are assumed in the statistical methods used for estimation, decision-making and drawing inferences in enumerative studies. Their aim is estimation about some aspect of the frame (such as a description, comparison or the existence of a cause–effect relationship) and the resulting actions taken on this particular frame. One feature of an enumerative study is that a 100% sample of the frame provides the complete answer to the questions posed by the study (given the methods of investigation and measurement). Statistical methods such as hypothesis tests, CIs and probability statements are appropriate to analyse and report data from enumerative studies. Estimating the infection rate in an intensive care unit for the last month is an example of a simple enumerative study.

• Analytical studies assume that the actions taken as a result of the study will be on the process or causal system that produced the frame studied, rather than the initial frame itself. The aim is to improve future performance.

In contrast to enumerative studies, an analytical study accepts as a given that when actions are taken on a system based on the results of a study, the conditions in that system will inevitably have changed. The aim of an analytical study is to enable prediction about how a change in a system will affect that system's future performance, or prediction as to which plans or strategies for future action on the system will be superior. For example, the task may be to choose among several different treatments for future patients, methods of collecting information or procedures for cleaning an operating room. Because the population of interest is open and continually shifts over time, random samples from that population cannot be obtained in analytical studies, and traditional statistical methods are therefore not useful. Rather, graphical methods of analysis and summary of the repeated samples reveal the trajectory of system behaviour over time, making it possible to predict future behaviour. Use of a Shewhart control chart to monitor and create learning to reduce infection rates in an intensive care unit is an example of a simple analytical study.

The following scenarios give examples to clarify the nature of these two types of studies.

Scenario 1: enumerative study—observation

To estimate how many days it takes new patients to see all primary care physicians contracted with a health plan, a researcher selected a random sample of 150 such physicians from the current active list and called each of their offices to schedule an appointment. The time to the next available appointment ranged from 0 to 180 days, with a mean of 38 days (95% CI 35.6 to 39.6).

This is an enumerative study, since results are intended to be used to estimate the waiting time for appointments with the plan's current population of primary care physicians.

Scenario 2: enumerative study—hypothesis generation

The researcher in scenario 1 noted that on occasion, she was offered an earlier visit with a nurse practitioner (NP) who worked with the physician being called. Additional information revealed that 20 of the 150 physicians in the study worked with one or more NPs. The next available appointment for the 130 physicians without an NP averaged 41 days (95% CI 39 to 43 days) and was 18 days (95% CI 18 to 26 days) for the 20 practices with NPs, a difference of 23 days (a 56% shorter mean waiting time).

This subgroup analysis suggested that the involvement of NPs helps to shorten waiting times, although it does not establish a cause–effect relationship, that is, it was a ‘hypothesis-generating’ study. In any event, this was clearly an enumerative study, since its results were to understand the impact of NPs on waiting times in the particular population of practices. Its results suggested that NPs might influence waiting times, but only for practices in this health plan during the time of the study. The study treated the conditions in the health plan as static, like those in a pond.

Scenario 3: enumerative study—comparison

To find out if administrative changes in a health plan had increased member satisfaction in access to care, the customer service manager replicated a phone survey he had conducted a year previously, using a random sample of 300 members. The percentage of patients who were satisfied with access had increased from 48.7% to 60.7% (Fisher exact test, p<0.004).

This enumerative comparison study was used to estimate the impact of the improvement work during the last year on the members in the plan. Attributing the increase in satisfaction to the improvement work assumes that other conditions in the study frame were static.

Scenario 4: analytical study—learning with a Shewhart chart

Each primary care clinic in a health plan reported its ‘time until the third available appointment’ twice a month, which allowed the quality manager to plot the mean waiting time for all of the clinics on Shewhart charts. Waiting times had been stable for a 12-month period through August, but the manager then noted a special cause (increase in waiting time) in September. On stratifying the data by region, she found that the special cause resulted from increases in waiting time in the Northeast region. Discussion with the regional manager revealed a shortage of primary care physicians in this region, which was predicted to become worse in the next quarter. Making some temporary assignments and increasing physician recruiting efforts resulted in stabilisation of this measure.

Documenting common and special cause variation in measures of interest through the use of Shewhart charts and run charts based on judgement samples is probably the simplest and commonest type of analytical study in healthcare. Such charts, when stable, provide a rational basis for predicting future performance.

Scenario 5: analytical study—establishing a cause–effect relationship

The researcher mentioned in scenarios 1 and 2 planned a study to test the existence of a cause–effect relationship between the inclusion of NPs in primary care offices and waiting time for new patient appointments. The variation in patient characteristics in this health plan appeared to be great enough to make the study results useful to other organisations. For the study, she recruited about 100 of the plan's practices that currently did not use NPs, and obtained funding to facilitate hiring NPs in up to 50 of those practices.

The researcher first explored the theories on mechanisms by which the incorporation of NPs into primary care clinics could reduce waiting times. Using important contextual variables relevant to these mechanisms (practice size, complexity, use of information technology and urban vs rural location), she then developed a randomised block, time-series study design. The study had the power to detect an effect of a mean waiting time of 5 days or more overall, and 10 days for the major subgroups defined by levels of the contextual variables. Since the baseline waiting time for appointments varied substantially across practices, she used the baseline as a covariate.

After completing the study, she analysed data from baseline and postintervention periods using stratified run charts and Shewhart charts, including the raw measures and measures adjusted for important covariates and effects of contextual variables. Overall waiting times decreased 12 days more in practices that included NPs than they did in control practices. Importantly, the subgroup analyses according to contextual variables revealed conditions under which the use of NPs would not be predicted to lead to reductions in waiting times. For example, practices with short baseline waiting times showed little or no improvement by employing NPs. She published the results in a leading health research journal.

This was an analytical study because the intent was to apply the learning from the study to future staffing plans in the health plan. She also published the study, so its results would be useful to primary care practices outside the health plan.

Scenario 6: analytical study—implementing improvement

The quality-improvement manager in another health plan wanted to expand the use of NPs in the plan's primary care practices, because published research had shown a reduction in waiting times for practices with NPs. Two practices in his plan already employed NPs. In one of these practices, Shewhart charts of waiting time by month showed a stable process averaging 10 days during the last 2 years. Waiting time averaged less than 7 days in the second practice, but a period when one of the physicians left the practice was associated with special causes.

The quality manager created a collaborative among the plan's primary care practices to learn how to optimise the use of NPs. Physicians in the two sites that employed NPs served as subject matter experts for the collaborative. In addition to making NPs part of their care teams, participating practices monitored appointment supply and demand, and tested other changes designed to optimise response to patient needs. Thirty sites in the plan voluntarily joined the collaborative and hired NPs. After 6 months, Shewhart charts indicated that waiting times in 25 of the 30 sites had been reduced to less than 7 days. Because waiting times in these practices had been stable over a considerable period of time, the manager predicted that future patients would continue to experience reduced times for appointments. The quality manger began to focus on a follow-up collaborative among the backlog of 70 practices that wanted to join.

This project was clearly an analytical study, since its aim was specifically to improve future waiting-time performance for participating sites and other primary care offices in the plan. Moreover, it focused on learning about the mechanisms through which contextual factors affect the impact of NPs on primary care office functions, under practice conditions that (like those in a river) will inevitably change over time.

Statistical theory in enumerative studies is used to describe the precision of estimates and the validity of hypotheses for the population studied. But since these statistical methods provide no support for extrapolation of the results outside the population that was studied, the subject experts must rely on their understanding of the mechanisms in place to extend results outside the population.

In analytical studies, the standard error of a statistic does not address the most important source of uncertainty, namely, the change in study conditions in the future. Although analytical studies need to take into account the uncertainty due to sampling, as in enumerative studies, the attributes of the study design and analysis of the data primarily deal with the uncertainty resulting from extrapolation to the future (generalisation to the conditions in future time periods). The methods used in analytical studies encourage the exploration of mechanisms through multifactor designs, contextual variables introduced through blocking and replication over time.

Prior stability of a system (as observed in graphic displays of repeated sampling over time, according to Shewhart's methods) increases belief in the results of an analytical study, but stable processes in the past do not guarantee constant system behaviour in the future. The next data point from the future is the most important on a graph of performance. Extrapolation of system behaviour to future times therefore still depends on input from subject experts who are familiar with mechanisms of the system of interest, as well as the important contextual issues. Generalisation is inherently difficult in all studies because ‘whereas the problems of internal validity are solvable within the limits of the logic of probability statistics, the problems of external validity are not logically solvable in any neat, conclusive way’ 3 (p. 17).

The diverse activities commonly referred to as healthcare improvement 12 are all designed to change the behaviour of systems over time, as reflected in the principle that ‘not all change is improvement, but all improvement is change.’ The conditions in the unbounded systems into which improvement interventions are introduced will therefore be different in the future from those in effect at the time the intervention is studied. Since the results of improvement studies are used to predict future system behaviour, such studies clearly belong to the Deming category of analytical studies. Quality improvement studies therefore need to incorporate repeated measurements over time, as well as testing under a wide range of conditions (2, 3 and 10). The ‘gold standard’ of analytical studies is satisfactory prediction over time.

Conclusions and recommendations

In light of these considerations, some important principles for drawing inferences from improvement studies include 10 :

• The analysis of data, interpretation of that analysis and actions taken as a result of the study should be closely tied to the current knowledge of experts about mechanisms of change in the relevant area. They can often use the study to discover, understand and evaluate the underlying mechanisms.
• The conditions of the study will be different from the future conditions under which the results will be used. Assessment by experts of the magnitude of this difference and its potential impact on future events should be an integral part of the interpretation of the results of the intervention.
• Show all the data before aggregation or summary.
• Plot the outcome data in the order in which the tests of change were conducted and annotate with information on the interventions.
• Use graphical displays to assess how much of the variation in the data can be explained by factors that were deliberately changed.
• Rearrange and subgroup the data to study other sources of variation (background and contextual variables).
• Summarise the results of the study with appropriate graphical displays.

Because these principles reflect the fundamental nature of improvement—taking action to change performance, over time, and under changing conditions—their application helps to bring clarity and rigour to improvement science.

Acknowledgments

The author is grateful to F Davidoff and P Batalden for their input to earlier versions of this paper.

Competing interests: None.

Provenance and peer review: Not commissioned; externally peer reviewed.

• Privacy Policy

Home » Data Analysis – Process, Methods and Types

Data Analysis – Process, Methods and Types

Table of Contents

Data Analysis

Definition:

Data analysis refers to the process of inspecting, cleaning, transforming, and modeling data with the goal of discovering useful information, drawing conclusions, and supporting decision-making. It involves applying various statistical and computational techniques to interpret and derive insights from large datasets. The ultimate aim of data analysis is to convert raw data into actionable insights that can inform business decisions, scientific research, and other endeavors.

Data Analysis Process

The following are step-by-step guides to the data analysis process:

Define the Problem

The first step in data analysis is to clearly define the problem or question that needs to be answered. This involves identifying the purpose of the analysis, the data required, and the intended outcome.

Collect the Data

The next step is to collect the relevant data from various sources. This may involve collecting data from surveys, databases, or other sources. It is important to ensure that the data collected is accurate, complete, and relevant to the problem being analyzed.

Clean and Organize the Data

Once the data has been collected, it needs to be cleaned and organized. This involves removing any errors or inconsistencies in the data, filling in missing values, and ensuring that the data is in a format that can be easily analyzed.

Analyze the Data

The next step is to analyze the data using various statistical and analytical techniques. This may involve identifying patterns in the data, conducting statistical tests, or using machine learning algorithms to identify trends and insights.

Interpret the Results

After analyzing the data, the next step is to interpret the results. This involves drawing conclusions based on the analysis and identifying any significant findings or trends.

Communicate the Findings

Once the results have been interpreted, they need to be communicated to stakeholders. This may involve creating reports, visualizations, or presentations to effectively communicate the findings and recommendations.

Take Action

The final step in the data analysis process is to take action based on the findings. This may involve implementing new policies or procedures, making strategic decisions, or taking other actions based on the insights gained from the analysis.

Types of Data Analysis

Types of Data Analysis are as follows:

Descriptive Analysis

This type of analysis involves summarizing and describing the main characteristics of a dataset, such as the mean, median, mode, standard deviation, and range.

Inferential Analysis

This type of analysis involves making inferences about a population based on a sample. Inferential analysis can help determine whether a certain relationship or pattern observed in a sample is likely to be present in the entire population.

Diagnostic Analysis

This type of analysis involves identifying and diagnosing problems or issues within a dataset. Diagnostic analysis can help identify outliers, errors, missing data, or other anomalies in the dataset.

Predictive Analysis

This type of analysis involves using statistical models and algorithms to predict future outcomes or trends based on historical data. Predictive analysis can help businesses and organizations make informed decisions about the future.

Prescriptive Analysis

This type of analysis involves recommending a course of action based on the results of previous analyses. Prescriptive analysis can help organizations make data-driven decisions about how to optimize their operations, products, or services.

Exploratory Analysis

This type of analysis involves exploring the relationships and patterns within a dataset to identify new insights and trends. Exploratory analysis is often used in the early stages of research or data analysis to generate hypotheses and identify areas for further investigation.

Data Analysis Methods

Data Analysis Methods are as follows:

Statistical Analysis

This method involves the use of mathematical models and statistical tools to analyze and interpret data. It includes measures of central tendency, correlation analysis, regression analysis, hypothesis testing, and more.

Machine Learning

This method involves the use of algorithms to identify patterns and relationships in data. It includes supervised and unsupervised learning, classification, clustering, and predictive modeling.

Data Mining

This method involves using statistical and machine learning techniques to extract information and insights from large and complex datasets.

Text Analysis

This method involves using natural language processing (NLP) techniques to analyze and interpret text data. It includes sentiment analysis, topic modeling, and entity recognition.

Network Analysis

This method involves analyzing the relationships and connections between entities in a network, such as social networks or computer networks. It includes social network analysis and graph theory.

Time Series Analysis

This method involves analyzing data collected over time to identify patterns and trends. It includes forecasting, decomposition, and smoothing techniques.

Spatial Analysis

This method involves analyzing geographic data to identify spatial patterns and relationships. It includes spatial statistics, spatial regression, and geospatial data visualization.

Data Visualization

This method involves using graphs, charts, and other visual representations to help communicate the findings of the analysis. It includes scatter plots, bar charts, heat maps, and interactive dashboards.

Qualitative Analysis

This method involves analyzing non-numeric data such as interviews, observations, and open-ended survey responses. It includes thematic analysis, content analysis, and grounded theory.

Multi-criteria Decision Analysis

This method involves analyzing multiple criteria and objectives to support decision-making. It includes techniques such as the analytical hierarchy process, TOPSIS, and ELECTRE.

Data Analysis Tools

There are various data analysis tools available that can help with different aspects of data analysis. Below is a list of some commonly used data analysis tools:

• Microsoft Excel: A widely used spreadsheet program that allows for data organization, analysis, and visualization.
• SQL : A programming language used to manage and manipulate relational databases.
• R : An open-source programming language and software environment for statistical computing and graphics.
• Python : A general-purpose programming language that is widely used in data analysis and machine learning.
• Tableau : A data visualization software that allows for interactive and dynamic visualizations of data.
• SAS : A statistical analysis software used for data management, analysis, and reporting.
• SPSS : A statistical analysis software used for data analysis, reporting, and modeling.
• Matlab : A numerical computing software that is widely used in scientific research and engineering.
• RapidMiner : A data science platform that offers a wide range of data analysis and machine learning tools.

Applications of Data Analysis

Data analysis has numerous applications across various fields. Below are some examples of how data analysis is used in different fields:

• Business : Data analysis is used to gain insights into customer behavior, market trends, and financial performance. This includes customer segmentation, sales forecasting, and market research.
• Healthcare : Data analysis is used to identify patterns and trends in patient data, improve patient outcomes, and optimize healthcare operations. This includes clinical decision support, disease surveillance, and healthcare cost analysis.
• Education : Data analysis is used to measure student performance, evaluate teaching effectiveness, and improve educational programs. This includes assessment analytics, learning analytics, and program evaluation.
• Finance : Data analysis is used to monitor and evaluate financial performance, identify risks, and make investment decisions. This includes risk management, portfolio optimization, and fraud detection.
• Government : Data analysis is used to inform policy-making, improve public services, and enhance public safety. This includes crime analysis, disaster response planning, and social welfare program evaluation.
• Sports : Data analysis is used to gain insights into athlete performance, improve team strategy, and enhance fan engagement. This includes player evaluation, scouting analysis, and game strategy optimization.
• Marketing : Data analysis is used to measure the effectiveness of marketing campaigns, understand customer behavior, and develop targeted marketing strategies. This includes customer segmentation, marketing attribution analysis, and social media analytics.
• Environmental science : Data analysis is used to monitor and evaluate environmental conditions, assess the impact of human activities on the environment, and develop environmental policies. This includes climate modeling, ecological forecasting, and pollution monitoring.

When to Use Data Analysis

Data analysis is useful when you need to extract meaningful insights and information from large and complex datasets. It is a crucial step in the decision-making process, as it helps you understand the underlying patterns and relationships within the data, and identify potential areas for improvement or opportunities for growth.

Here are some specific scenarios where data analysis can be particularly helpful:

• Problem-solving : When you encounter a problem or challenge, data analysis can help you identify the root cause and develop effective solutions.
• Optimization : Data analysis can help you optimize processes, products, or services to increase efficiency, reduce costs, and improve overall performance.
• Prediction: Data analysis can help you make predictions about future trends or outcomes, which can inform strategic planning and decision-making.
• Performance evaluation : Data analysis can help you evaluate the performance of a process, product, or service to identify areas for improvement and potential opportunities for growth.
• Risk assessment : Data analysis can help you assess and mitigate risks, whether it is financial, operational, or related to safety.
• Market research : Data analysis can help you understand customer behavior and preferences, identify market trends, and develop effective marketing strategies.
• Quality control: Data analysis can help you ensure product quality and customer satisfaction by identifying and addressing quality issues.

Purpose of Data Analysis

The primary purposes of data analysis can be summarized as follows:

• To gain insights: Data analysis allows you to identify patterns and trends in data, which can provide valuable insights into the underlying factors that influence a particular phenomenon or process.
• To inform decision-making: Data analysis can help you make informed decisions based on the information that is available. By analyzing data, you can identify potential risks, opportunities, and solutions to problems.
• To improve performance: Data analysis can help you optimize processes, products, or services by identifying areas for improvement and potential opportunities for growth.
• To measure progress: Data analysis can help you measure progress towards a specific goal or objective, allowing you to track performance over time and adjust your strategies accordingly.
• To identify new opportunities: Data analysis can help you identify new opportunities for growth and innovation by identifying patterns and trends that may not have been visible before.

Examples of Data Analysis

Some Examples of Data Analysis are as follows:

• Social Media Monitoring: Companies use data analysis to monitor social media activity in real-time to understand their brand reputation, identify potential customer issues, and track competitors. By analyzing social media data, businesses can make informed decisions on product development, marketing strategies, and customer service.
• Financial Trading: Financial traders use data analysis to make real-time decisions about buying and selling stocks, bonds, and other financial instruments. By analyzing real-time market data, traders can identify trends and patterns that help them make informed investment decisions.
• Traffic Monitoring : Cities use data analysis to monitor traffic patterns and make real-time decisions about traffic management. By analyzing data from traffic cameras, sensors, and other sources, cities can identify congestion hotspots and make changes to improve traffic flow.
• Healthcare Monitoring: Healthcare providers use data analysis to monitor patient health in real-time. By analyzing data from wearable devices, electronic health records, and other sources, healthcare providers can identify potential health issues and provide timely interventions.
• Online Advertising: Online advertisers use data analysis to make real-time decisions about advertising campaigns. By analyzing data on user behavior and ad performance, advertisers can make adjustments to their campaigns to improve their effectiveness.
• Sports Analysis : Sports teams use data analysis to make real-time decisions about strategy and player performance. By analyzing data on player movement, ball position, and other variables, coaches can make informed decisions about substitutions, game strategy, and training regimens.
• Energy Management : Energy companies use data analysis to monitor energy consumption in real-time. By analyzing data on energy usage patterns, companies can identify opportunities to reduce energy consumption and improve efficiency.

Characteristics of Data Analysis

Characteristics of Data Analysis are as follows:

• Objective : Data analysis should be objective and based on empirical evidence, rather than subjective assumptions or opinions.
• Systematic : Data analysis should follow a systematic approach, using established methods and procedures for collecting, cleaning, and analyzing data.
• Accurate : Data analysis should produce accurate results, free from errors and bias. Data should be validated and verified to ensure its quality.
• Relevant : Data analysis should be relevant to the research question or problem being addressed. It should focus on the data that is most useful for answering the research question or solving the problem.
• Comprehensive : Data analysis should be comprehensive and consider all relevant factors that may affect the research question or problem.
• Timely : Data analysis should be conducted in a timely manner, so that the results are available when they are needed.
• Reproducible : Data analysis should be reproducible, meaning that other researchers should be able to replicate the analysis using the same data and methods.
• Communicable : Data analysis should be communicated clearly and effectively to stakeholders and other interested parties. The results should be presented in a way that is understandable and useful for decision-making.

Advantages of Data Analysis

Advantages of Data Analysis are as follows:

• Better decision-making: Data analysis helps in making informed decisions based on facts and evidence, rather than intuition or guesswork.
• Improved efficiency: Data analysis can identify inefficiencies and bottlenecks in business processes, allowing organizations to optimize their operations and reduce costs.
• Increased accuracy: Data analysis helps to reduce errors and bias, providing more accurate and reliable information.
• Better customer service: Data analysis can help organizations understand their customers better, allowing them to provide better customer service and improve customer satisfaction.
• Competitive advantage: Data analysis can provide organizations with insights into their competitors, allowing them to identify areas where they can gain a competitive advantage.
• Identification of trends and patterns : Data analysis can identify trends and patterns in data that may not be immediately apparent, helping organizations to make predictions and plan for the future.
• Improved risk management : Data analysis can help organizations identify potential risks and take proactive steps to mitigate them.
• Innovation: Data analysis can inspire innovation and new ideas by revealing new opportunities or previously unknown correlations in data.

Limitations of Data Analysis

• Data quality: The quality of data can impact the accuracy and reliability of analysis results. If data is incomplete, inconsistent, or outdated, the analysis may not provide meaningful insights.
• Limited scope: Data analysis is limited by the scope of the data available. If data is incomplete or does not capture all relevant factors, the analysis may not provide a complete picture.
• Human error : Data analysis is often conducted by humans, and errors can occur in data collection, cleaning, and analysis.
• Cost : Data analysis can be expensive, requiring specialized tools, software, and expertise.
• Time-consuming : Data analysis can be time-consuming, especially when working with large datasets or conducting complex analyses.
• Overreliance on data: Data analysis should be complemented with human intuition and expertise. Overreliance on data can lead to a lack of creativity and innovation.
• Privacy concerns: Data analysis can raise privacy concerns if personal or sensitive information is used without proper consent or security measures.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Research Design – Types, Methods and Examples

Problem Statement – Writing Guide, Examples and...

Scope of the Research – Writing Guide and...

Research Summary – Structure, Examples and...

Thesis – Structure, Example and Writing Guide

MANOVA (Multivariate Analysis of Variance) –...

Reference management. Clean and simple.

What is research methodology?

The basics of research methodology

Why do you need a research methodology, what needs to be included, why do you need to document your research method, what are the different types of research instruments, qualitative / quantitative / mixed research methodologies, how do you choose the best research methodology for you, frequently asked questions about research methodology, related articles.

When you’re working on your first piece of academic research, there are many different things to focus on, and it can be overwhelming to stay on top of everything. This is especially true of budding or inexperienced researchers.

If you’ve never put together a research proposal before or find yourself in a position where you need to explain your research methodology decisions, there are a few things you need to be aware of.

Once you understand the ins and outs, handling academic research in the future will be less intimidating. We break down the basics below:

A research methodology encompasses the way in which you intend to carry out your research. This includes how you plan to tackle things like collection methods, statistical analysis, participant observations, and more.

You can think of your research methodology as being a formula. One part will be how you plan on putting your research into practice, and another will be why you feel this is the best way to approach it. Your research methodology is ultimately a methodological and systematic plan to resolve your research problem.

In short, you are explaining how you will take your idea and turn it into a study, which in turn will produce valid and reliable results that are in accordance with the aims and objectives of your research. This is true whether your paper plans to make use of qualitative methods or quantitative methods.

The purpose of a research methodology is to explain the reasoning behind your approach to your research - you'll need to support your collection methods, methods of analysis, and other key points of your work.

Think of it like writing a plan or an outline for you what you intend to do.

When carrying out research, it can be easy to go off-track or depart from your standard methodology.

Tip: Having a methodology keeps you accountable and on track with your original aims and objectives, and gives you a suitable and sound plan to keep your project manageable, smooth, and effective.

With all that said, how do you write out your standard approach to a research methodology?

As a general plan, your methodology should include the following information:

• Your research method.  You need to state whether you plan to use quantitative analysis, qualitative analysis, or mixed-method research methods. This will often be determined by what you hope to achieve with your research.
• Explain your reasoning. Why are you taking this methodological approach? Why is this particular methodology the best way to answer your research problem and achieve your objectives?
• Explain your instruments.  This will mainly be about your collection methods. There are varying instruments to use such as interviews, physical surveys, questionnaires, for example. Your methodology will need to detail your reasoning in choosing a particular instrument for your research.
• What will you do with your results?  How are you going to analyze the data once you have gathered it?
• Advise your reader.  If there is anything in your research methodology that your reader might be unfamiliar with, you should explain it in more detail. For example, you should give any background information to your methods that might be relevant or provide your reasoning if you are conducting your research in a non-standard way.
• How will your sampling process go?  What will your sampling procedure be and why? For example, if you will collect data by carrying out semi-structured or unstructured interviews, how will you choose your interviewees and how will you conduct the interviews themselves?
• Any practical limitations?  You should discuss any limitations you foresee being an issue when you’re carrying out your research.

In any dissertation, thesis, or academic journal, you will always find a chapter dedicated to explaining the research methodology of the person who carried out the study, also referred to as the methodology section of the work.

A good research methodology will explain what you are going to do and why, while a poor methodology will lead to a messy or disorganized approach.

You should also be able to justify in this section your reasoning for why you intend to carry out your research in a particular way, especially if it might be a particularly unique method.

Having a sound methodology in place can also help you with the following:

• When another researcher at a later date wishes to try and replicate your research, they will need your explanations and guidelines.
• In the event that you receive any criticism or questioning on the research you carried out at a later point, you will be able to refer back to it and succinctly explain the how and why of your approach.
• It provides you with a plan to follow throughout your research. When you are drafting your methodology approach, you need to be sure that the method you are using is the right one for your goal. This will help you with both explaining and understanding your method.
• It affords you the opportunity to document from the outset what you intend to achieve with your research, from start to finish.

A research instrument is a tool you will use to help you collect, measure and analyze the data you use as part of your research.

The choice of research instrument will usually be yours to make as the researcher and will be whichever best suits your methodology.

There are many different research instruments you can use in collecting data for your research.

Generally, they can be grouped as follows:

• Interviews (either as a group or one-on-one). You can carry out interviews in many different ways. For example, your interview can be structured, semi-structured, or unstructured. The difference between them is how formal the set of questions is that is asked of the interviewee. In a group interview, you may choose to ask the interviewees to give you their opinions or perceptions on certain topics.
• Surveys (online or in-person). In survey research, you are posing questions in which you ask for a response from the person taking the survey. You may wish to have either free-answer questions such as essay-style questions, or you may wish to use closed questions such as multiple choice. You may even wish to make the survey a mixture of both.
• Focus Groups.  Similar to the group interview above, you may wish to ask a focus group to discuss a particular topic or opinion while you make a note of the answers given.
• Observations.  This is a good research instrument to use if you are looking into human behaviors. Different ways of researching this include studying the spontaneous behavior of participants in their everyday life, or something more structured. A structured observation is research conducted at a set time and place where researchers observe behavior as planned and agreed upon with participants.

These are the most common ways of carrying out research, but it is really dependent on your needs as a researcher and what approach you think is best to take.

It is also possible to combine a number of research instruments if this is necessary and appropriate in answering your research problem.

There are three different types of methodologies, and they are distinguished by whether they focus on words, numbers, or both.

➡️ Want to learn more about the differences between qualitative and quantitative research, and how to use both methods? Check out our guide for that!

If you've done your due diligence, you'll have an idea of which methodology approach is best suited to your research.

It’s likely that you will have carried out considerable reading and homework before you reach this point and you may have taken inspiration from other similar studies that have yielded good results.

Still, it is important to consider different options before setting your research in stone. Exploring different options available will help you to explain why the choice you ultimately make is preferable to other methods.

If proving your research problem requires you to gather large volumes of numerical data to test hypotheses, a quantitative research method is likely to provide you with the most usable results.

If instead you’re looking to try and learn more about people, and their perception of events, your methodology is more exploratory in nature and would therefore probably be better served using a qualitative research methodology.

It helps to always bring things back to the question: what do I want to achieve with my research?

Once you have conducted your research, you need to analyze it. Here are some helpful guides for qualitative data analysis:

➡️  How to do a content analysis

➡️  How to do a thematic analysis

➡️  How to do a rhetorical analysis

Research methodology refers to the techniques used to find and analyze information for a study, ensuring that the results are valid, reliable and that they address the research objective.

Data can typically be organized into four different categories or methods: observational, experimental, simulation, and derived.

Writing a methodology section is a process of introducing your methods and instruments, discussing your analysis, providing more background information, addressing your research limitations, and more.

Your research methodology section will need a clear research question and proposed research approach. You'll need to add a background, introduce your research question, write your methodology and add the works you cited during your data collecting phase.

The research methodology section of your study will indicate how valid your findings are and how well-informed your paper is. It also assists future researchers planning to use the same methodology, who want to cite your study or replicate it.

Learning Analytics Methods and Tutorials

A Practical Guide Using R

• Open Access
• © 2024

You have full access to this open access Book

• Mohammed Saqr 0 ,
• Sonsoles López-Pernas 1

School of Computing,, University of Eastern Finland, Joensuu, Finland

You can also search for this editor in PubMed   Google Scholar

School of Computing, University of Eastern Finland, Joensuu, Finland

• The book is a tutorial for learning analytics, data mining and quantitative methods in education
• The book is or beginners and experienced readers with step-by-step tutorials, real-life data and code
• The book presents the latest advances in learning analytics and educational research method
• This book is in open access which means you have free and unlimited access

8666 Accesses

10 Citations

15 Altmetric

Buy print copy

• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

About this book

This open access comprehensive methodological book offers a much-needed answer to the lack of resources and methodological guidance in learning analytics, which has been a problem ever since the field started. The book covers all important quantitative topics in education at large as well as the latest in learning analytics and education data mining. The book also goes deeper into advanced methods that are at the forefront of novel methodological innovations. Authors of the book include world-renowned learning analytics researchers, R package developers, and methodological experts from diverse fields offering an unprecedented interdisciplinary reference on novel topics that is hard to find elsewhere.

This is an open access book.

• learning analytics methods
• educational data mining
• quantitative methods in education
• social network analysis
• sequence analysis
• Process mining
• machine learning in education
• artificial intelligence in education
• temporal networks
• epistemic networks
• open access

Table of contents (22 chapters)

Front matter, capturing the wealth and diversity of learning processes with learning analytics methods.

• Sonsoles López-Pernas, Kamila Misiejuk, Rogers Kaliisa, Miguel Ángel Conde-González, Mohammed Saqr

Getting Started

A broad collection of datasets for educational research training and application.

• Sonsoles López-Pernas, Mohammed Saqr, Javier Conde, Laura Del-Río-Carazo

Getting Started with R for Education Research

• Santtu Tikka, Juho Kopra, Merja Heinäniemi, Sonsoles López-Pernas, Mohammed Saqr

An R Approach to Data Cleaning and Wrangling for Education Research

• Juho Kopra, Santtu Tikka, Merja Heinäniemi, Sonsoles López-Pernas, Mohammed Saqr

Introductory Statistics with R for Educational Researchers

Visualizing and reporting educational data with r.

• Sonsoles López-Pernas, Kamila Misiejuk, Santtu Tikka, Juho Kopra, Merja Heinäniemi, Mohammed Saqr

Machine Learning

Predictive modelling in learning analytics: a machine learning approach in r.

• Jelena Jovanovic, Sonsoles López-Pernas, Mohammed Saqr

Dissimilarity-Based Cluster Analysis of Educational Data: A Comparative Tutorial Using R

• Keefe Murphy, Sonsoles López-Pernas, Mohammed Saqr

An Introduction and R Tutorial to Model-Based Clustering in Education via Latent Profile Analysis

• Luca Scrucca, Mohammed Saqr, Sonsoles López-Pernas, Keefe Murphy

Temporal Methods

Sequence analysis in education: principles, technique, and tutorial with r.

• Mohammed Saqr, Sonsoles López-Pernas, Satu Helske, Marion Durand, Keefe Murphy, Matthias Studer et al.

Modeling the Dynamics of Longitudinal Processes in Education. A Tutorial with R for the VaSSTra Method

• Sonsoles López-Pernas, Mohammed Saqr

A Modern Approach to Transition Analysis and Process Mining with Markov Models in Education

• Jouni Helske, Satu Helske, Mohammed Saqr, Sonsoles López-Pernas, Keefe Murphy

Multi-Channel Sequence Analysis in Educational Research: An Introduction and Tutorial with R

• Sonsoles López-Pernas, Mohammed Saqr, Satu Helske, Keefe Murphy

The Why, the How and the When of Educational Process Mining in R

Network analysis, social network analysis: a primer, a guide and a tutorial in r.

• Mohammed Saqr, Sonsoles López-Pernas, Miguel Ángel Conde-González, Ángel Hernández-García

Editors and Affiliations

Mohammed Saqr

Sonsoles López-Pernas

About the editors

Mohammed Saqr is an Associate Professor of learning analytics and Academy of Finland Research Council researcher. Mohammed Saqr leads the Unit of learning analytics at the University of Eastern Finland (UEF), School of Computing which was, according to Scopus and Web of Science, Europe’s most productive learning analytics lab during the last five years (2019-2023). Mohammed Saqr has a PhD in learning analytics from Stockholm University, Sweden. Before joining UEF in Finland, Mohammed had a postdoc at Université Paris Cité, France, and obtained the title of Docent in learning analytics from the University of Oulu, Finland. Mohammed Saqr published more than 150 peer-reviewed interdisciplinary, methodological, and empirical studies about learning analytics, artificial intelligence, big data, network science, science of science, and medicine. Mohammed has several awards, e.g., the PhD was awarded the best thesis, he also got several international research awards (e.g., best papers), and obtained the University of Michigan Office of Academic Innovation fellowship. In 2023, the Society of Learning Analytics Research (SOLAR) granted Mohammed Europe’s Emerging Scholar Award for "noteworthy research leading to significant knowledge and understanding of learning analytics and the impact of research on learning analytics application, adoption, and professional development in Europe”. Mohammed got funding from prestigious institutions: Academy of Finland (as PI) for idiographic learning analytics and Swedish Research Council (as Co-PI) as well as several other grants. Mohammed is on the editorial board of several academic journals e.g., Transactions of Learning Technologies, British Journal of Education Technologies, and Plos One. Mohammed organized and contributed to several international conferences, and presented several invited keynotes and talks. Mohammed’s current collaboration network includes more than a hundred researchers from around the world including Finland, Spain, Sweden, Norway, Japan, Ireland, Germany, Serbia, Luxembourg, Bulgaria, Australia, France, Turkiye, Egypt, UK, USA, and the Netherlands.

López-Pernas Sonsoles is a Senior Researcher at University of Eastern Finland (UEF) since 2022 and holds the title of Docent (adjunct professor) in educational data mining. She obtained her Masters and PhD in Engineering from Universidad Politécnica de Madrid (Spain). During her career, she has developed several open-source software projects related to educational technology and big data analysis. She is skilled in quantitative methods that include learning analytics, machine learning, process and sequence mining, network analysis, complex event processing, and data visualization, which are proven by more than 100 empirical publications in the field of learning analytics, education technology, and big data as well as her impressive workplace achievements. Sonsoles has obtained several prestigious awards and recognitions that include a research award from theRoyal Academy of Doctors of Spain (RADE), the extraordinary PhD thesis award from Universidad Politécnica de Madrid, and two open source project awards in the university open source software contest (CUSL), as well as best paper awards in learning analytics and game-based learning. She has contributed to the organization of several scientific conferences and workshops (e.g., Finnish Learning Analytics and Artificial Intelligence in Education conference and Network Analysis workshop at LAK), and has been part of the program committee of LAK, AIED, ICCE, and Koli Calling. Sonsoles sits on the editorial board of IEEE Transactions on Education, and PloS One. Her collaboration network extends over the five continents.

Together, Mohammed and Sonsoles have extensively published in top journals in the field such as Computers & Education, International Journal of Computer Supported Collaborative Learning, Computers in Human Behavior, Journal of Learning Analytics and Educational Research Review as well as top conferences e.g., LAK, ECTEL, ICALT and TEEM.

Bibliographic Information

Book Title : Learning Analytics Methods and Tutorials

Book Subtitle : A Practical Guide Using R

Editors : Mohammed Saqr, Sonsoles López-Pernas

DOI : https://doi.org/10.1007/978-3-031-54464-4

Publisher : Springer Cham

eBook Packages : Computer Science , Computer Science (R0)

Copyright Information : The Editor(s) (if applicable) and The Author(s) 2024

Hardcover ISBN : 978-3-031-54463-7 Published: 25 June 2024

Softcover ISBN : 978-3-031-54466-8 Due: 26 July 2024

eBook ISBN : 978-3-031-54464-4 Published: 24 June 2024

Edition Number : 1

Number of Pages : XXXIV, 736

Number of Illustrations : 40 b/w illustrations, 202 illustrations in colour

Topics : Computers and Education , Education, general , Data Mining and Knowledge Discovery , Computer Applications

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 01 July 2024

Time of sample collection is critical for the replicability of microbiome analyses

• Celeste Allaband   ORCID: orcid.org/0000-0003-1832-4858 1 , 2 , 3 ,
• Amulya Lingaraju 2 ,
• Stephany Flores Ramos   ORCID: orcid.org/0000-0002-1918-9769 1 , 2 , 3 ,
• Tanya Kumar 4 ,
• Haniyeh Javaheri 2 ,
• Maria D. Tiu 2 ,
• Ana Carolina Dantas Machado 2 ,
• R. Alexander Richter 2 ,
• Emmanuel Elijah 5 , 6 ,
• Gabriel G. Haddad 3 , 7 , 8 ,
• Vanessa A. Leone 9 ,
• Pieter C. Dorrestein   ORCID: orcid.org/0000-0002-3003-1030 3 , 5 , 6 , 10 ,
• Rob Knight   ORCID: orcid.org/0000-0002-0975-9019 3 , 6 , 11 , 12 , 13 &
• Amir Zarrinpar   ORCID: orcid.org/0000-0001-6423-5982 2 , 6 , 13 , 14 , 15  

Nature Metabolism ( 2024 ) Cite this article

Metrics details

• Animal disease models
• Circadian regulation
• Research management

As the microbiome field moves from descriptive and associative research to mechanistic and interventional studies, being able to account for all confounding variables in the experimental design, which includes the maternal effect 1 , cage effect 2 , facility differences 3 , as well as laboratory and sample handling protocols 4 , is critical for interpretability of results. Despite significant procedural and bioinformatic improvements, unexplained variability and lack of replicability still occur. One underexplored factor is that the microbiome is dynamic and exhibits diurnal oscillations that can change microbiome composition 5 , 6 , 7 . In this retrospective analysis of 16S amplicon sequencing studies in male mice, we show that sample collection time affects the conclusions drawn from microbiome studies and its effect size is larger than those of a daily experimental intervention or dietary changes. The timing of divergence of the microbiome composition between experimental and control groups is unique to each experiment. Sample collection times as short as only 4 hours apart can lead to vastly different conclusions. Lack of consistency in the time of sample collection may explain poor cross-study replicability in microbiome research. The impact of diurnal rhythms on the outcomes and study design of other fields is unknown but likely significant.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Data availability

Literature review data are at https://github.com/knightlab-analyses/dynamics/data/ . Figure 1 , mock data are at https://github.com/knightlab-analyses/dynamics/data/MockData . Figure 2 (Allaband/Zarrinpar 2021) data are under EBI accession ERP110592 . Figure 3 data (longitudinal IHC) are under EBI accession ERP110592 and (longitudinal circadian TRF) EBI accession ERP123226 . Figure 4 data (Zarrinpar/Panda 2014) are in the Supplementary Excel file attached to the source paper 13 ; (Leone/Chang 2015) figshare for the 16S amplicon sequence data are at https://doi.org/10.6084/m9.figshare.882928 (ref. 63 ). Extended Data Fig. 2 data (Caporaso/Knight 2011) are at MG-RAST project mgp93 (IDs mgm4457768.3 and mgm4459735.3). Extended Data Fig. 3 data (Wu/Chen 2018) are under ENA accession PRJEB22049 . Extended Data Fig. 4 data (Tuganbaev/Elinav 2021) are under ENA accession PRJEB38869 .

Code availability

All relevant code notebooks are on GitHub at https://github.com/knightlab-analyses/dynamics/notebooks .

Schloss, P. D. Identifying and overcoming threats to reproducibility, replicability, robustness, and generalizability in microbiome research. mBio 9 , e00525–18 (2018).

Article   PubMed   PubMed Central   Google Scholar  

Gilbert, J. A. et al. Current understanding of the human microbiome. Nat. Med. 24 , 392–400 (2018).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Knight, R. et al. Best practices for analysing microbiomes. Nat. Rev. Microbiol. 16 , 410–422 (2018).

Article   CAS   PubMed   Google Scholar  

Ley, R. E. et al. Obesity alters gut microbial ecology. Proc. Natl Acad. Sci. USA 102 , 11070–11075 (2005).

Deloris Alexander, A. et al. Quantitative PCR assays for mouse enteric flora reveal strain-dependent differences in composition that are influenced by the microenvironment. Mamm. Genome 17 , 1093–1104 (2006).

Friswell, M. K. et al. Site and strain-specific variation in gut microbiota profiles and metabolism in experimental mice. PLoS ONE 5 , e8584 (2010).

Sinha, R. et al. Assessment of variation in microbial community amplicon sequencing by the Microbiome Quality Control (MBQC) project consortium. Nat. Biotechnol. 35 , 1077–1086 (2017).

Alvarez, Y., Glotfelty, L. G., Blank, N., Dohnalová, L. & Thaiss, C. A. The microbiome as a circadian coordinator of metabolism. Endocrinology 161 , bqaa059 (2020).

Frazier, K. & Chang, E. B. Intersection of the gut microbiome and circadian rhythms in metabolism. Trends Endocrinol. Metab. 31 , 25–36 (2020).

Heddes, M. et al. The intestinal clock drives the microbiome to maintain gastrointestinal homeostasis. Nat. Commun. 13 , 6068 (2022).

Leone, V. et al. Effects of diurnal variation of gut microbes and high-fat feeding on host circadian clock function and metabolism. Cell Host Microbe 17 , 681–689 (2015).

Thaiss, C. A. et al. Transkingdom control of microbiota diurnal oscillations promotes metabolic homeostasis. Cell 159 , 514–529 (2014).

Zarrinpar, A., Chaix, A., Yooseph, S. & Panda, S. Diet and feeding pattern affect the diurnal dynamics of the gut microbiome. Cell Metab. 20 , 1006–1017 (2014).

Liang, X., Bushman, F. D. & FitzGerald, G. A. Rhythmicity of the intestinal microbiota is regulated by gender and the host circadian clock. Proc. Natl Acad. Sci. USA 112 , 10479–10484 (2015).

Thaiss, C. A. et al. Microbiota diurnal rhythmicity programs host transcriptome oscillations. Cell 167 , 1495–1510 (2016).

Yu, F. et al. Deficiency of intestinal Bmal1 prevents obesity induced by high-fat feeding. Nat. Commun. 12 , 5323 (2021).

Leone, V. A. et al. Atypical behavioral and thermoregulatory circadian rhythms in mice lacking a microbiome. Sci. Rep. 12 , 14491 (2022).

Thaiss, C. A., Zeevi, D., Levy, M., Segal, E. & Elinav, E. A day in the life of the meta-organism: diurnal rhythms of the intestinal microbiome and its host. Gut Microbes 6 , 137–142 (2015).

Mukherji, A., Kobiita, A., Ye, T. & Chambon, P. Homeostasis in intestinal epithelium is orchestrated by the circadian clock and microbiota cues transduced by TLRs. Cell 153 , 812–827 (2013).

Weger, B. D. et al. The mouse microbiome is required for sex-specific diurnal rhythms of gene expression and metabolism. Cell Metab. 29 , 362–382 (2019).

Kaczmarek, J. L., Musaad, S. M. & Holscher, H. D. Time of day and eating behaviors are associated with the composition and function of the human gastrointestinal microbiota. Am. J. Clin. Nutr. 106 , 1220–1231 (2017).

Skarke, C. et al. A pilot characterization of the human chronobiome. Sci. Rep. 7 , 17141 (2017).

Jones, J., Reinke, S. N., Ali, A., Palmer, D. J. & Christophersen, C. T. Fecal sample collection methods and time of day impact microbiome composition and short chain fatty acid concentrations. Sci. Rep. 11 , 13964 (2021).

Collado, M. C. et al. Timing of food intake impacts daily rhythms of human salivary microbiota: a randomized, crossover study. FASEB J. 32 , 2060–2072 (2018).

Kohn, J. N. et al. Differing salivary microbiome diversity, community and diurnal rhythmicity in association with affective state and peripheral inflammation in adults. Brain. Behav. Immun. 87 , 591–602 (2020).

Takayasu, L. et al. Circadian oscillations of microbial and functional composition in the human salivary microbiome. DNA Res. 24 , 261–270 (2017).

Reitmeier, S. et al. Arrhythmic gut microbiome signatures predict risk of type 2 diabetes. Cell Host Microbe 28 , 258–272 (2020).

Allaband, C. et al. Intermittent hypoxia and hypercapnia alter diurnal rhythms of luminal gut microbiome and metabolome. mSystems https://doi.org/10.1128/mSystems.00116-21 (2021).

Tuganbaev, T. et al. Diet diurnally regulates small intestinal microbiome-epithelial-immune homeostasis and enteritis. Cell 182 , 1441–1459 (2020).

Wu, G. et al. Light exposure influences the diurnal oscillation of gut microbiota in mice. Biochem. Biophys. Res. Commun. 501 , 16–23 (2018).

Nelson, R. J. et al. Time of day as a critical variable in biology. BMC Biol. 20 , 142 (2022).

Dantas Machado, A. C. et al. Diet and feeding pattern modulate diurnal dynamics of the ileal microbiome and transcriptome. Cell Rep. 40 , 111008 (2022).

Morton, J. T. et al. Establishing microbial composition measurement standards with reference frames. Nat. Commun. 10 , 2719 (2019).

Caporaso, J. G. et al. Moving pictures of the human microbiome. Genome Biol. 12 , R50 (2011).

Bisanz, J. E., Upadhyay, V., Turnbaugh, J. A., Ly, K. & Turnbaugh, P. J. Meta-analysis reveals reproducible gut microbiome alterations in response to a high-fat diet. Cell Host Microbe 26 , 265–272.e4 (2019).

Kohsaka, A. et al. High-fat diet disrupts behavioral and molecular circadian rhythms in mice. Cell Metab. 6 , 414–421 (2007).

Hatori, M. et al. Time-restricted feeding without reducing caloric intake prevents metabolic diseases in mice fed a high-fat diet. Cell Metab. 15 , 848–860 (2012).

Baker, F. Normal rumen microflora and microfauna of cattle. Nature 149 , 220 (1942).

Article   Google Scholar  

Zhang, L., Wu, W., Lee, Y.-K., Xie, J. & Zhang, H. Spatial heterogeneity and co-occurrence of mucosal and luminal microbiome across swine intestinal tract. Front. Microbiol. 9 , 48 (2018).

Klymiuk, I. et al. Characterization of the luminal and mucosa-associated microbiome along the gastrointestinal tract: results from surgically treated preterm infants and a murine model. Nutrients 13 , 1030 (2021).

Kim, D. et al. Comparison of sampling methods in assessing the microbiome from patients with ulcerative colitis. BMC Gastroenterol. 21 , 396 (2021).

Tripathi, A. et al. Intermittent hypoxia and hypercapnia reproducibly change the gut microbiome and metabolome across rodent model systems. mSystems 4 , e00058–19 (2019).

Uhr, G. T., Dohnalová, L. & Thaiss, C. A. The Dimension of Time in Host-Microbiome Interactions. mSystems 4 , e00216–e00218 (2019).

Voigt, R. M. et al. Circadian disorganization alters intestinal microbiota. PLoS ONE 9 , e97500 (2014).

McDonald, D. et al. American gut: an open platform for citizen science microbiome research. mSystems 3 , e00031–18 (2018).

Borodulin, K. et al. Cohort profile: the National FINRISK Study. Int. J. Epidemiol. 47 , 696–696i (2018).

Article   PubMed   Google Scholar  

Ren, B. et al. Methionine restriction improves gut barrier function by reshaping diurnal rhythms of inflammation-related microbes in aged mice. Front. Nutr. 8 , 746592 (2021).

Beli, E., Prabakaran, S., Krishnan, P., Evans-Molina, C. & Grant, M. B. Loss of diurnal oscillatory rhythms in gut microbiota correlates with changes in circulating metabolites in type 2 diabetic db/db mice. Nutrients 11 , E2310 (2019).

Wang, L. et al. Methionine restriction regulates cognitive function in high-fat diet-fed mice: roles of diurnal rhythms of SCFAs producing- and inflammation-related microbes. Mol. Nutr. Food Res. 64 , e2000190 (2020).

Guo, T. et al. Oolong tea polyphenols ameliorate circadian rhythm of intestinal microbiome and liver clock genes in mouse model. J. Agric. Food Chem. 67 , 11969–11976 (2019).

Mistry, P. et al. Circadian influence on the microbiome improves heart failure outcomes. J. Mol. Cell. Cardiol. 149 , 54–72 (2020).

Shao, Y. et al. Effects of sleeve gastrectomy on the composition and diurnal oscillation of gut microbiota related to the metabolic improvements. Surg. Obes. Relat. Dis. 14 , 731–739 (2018).

Bolyen, E. et al. Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2. Nat. Biotechnol. 37 , 852–857 (2019).

Amir, A. et al. Deblur rapidly resolves single-nucleotide community sequence patterns. mSystems 2 , e00191–16 (2017).

Mirarab, S., Nguyen, N. & Warnow, T. in Biocomputing 2012 , 247–258 (World Scientific, 2011).

Lozupone, C., Lladser, M. E., Knights, D., Stombaugh, J. & Knight, R. UniFrac: an effective distance metric for microbial community comparison. ISME J. 5 , 169–172 (2011).

Lauber, C. L., Zhou, N., Gordon, J. I., Knight, R. & Fierer, N. Effect of storage conditions on the assessment of bacterial community structure in soil and human-associated samples: Influence of short-term storage conditions on microbiota. FEMS Microbiol. Lett. 307 , 80–86 (2010).

Marotz, C. et al. Evaluation of the effect of storage methods on fecal, saliva, and skin microbiome composition. mSystems 6 , e01329–20 (2021).

Song, S. J. et al. Preservation methods differ in fecal microbiome stability, affecting suitability for field studies. mSystems 1 , e00021–16 (2016).

Wu, G. D. et al. Sampling and pyrosequencing methods for characterizing bacterial communities in the human gut using 16S sequence tags. BMC Microbiol. 10 , 206 (2010).

Piedrahita, J. A., Zhang, S. H., Hagaman, J. R., Oliver, P. M. & Maeda, N. Generation of mice carrying a mutant apolipoprotein E gene inactivated by gene targeting in embryonic stem cells. Proc. Natl Acad. Sci. USA 89 , 4471–4475 (1992).

Chaix, A., Zarrinpar, A., Miu, P. & Panda, S. Time-restricted feeding is a preventative and therapeutic intervention against diverse nutritional challenges. Cell Metab. 20 , 991–1005 (2014).

Gibbons, S. Diel Mouse Gut Study (HF/LF diet) . figshare https://doi.org/10.6084/m9.figshare.882928 (2015).

Download references

Acknowledgements

C.A. was supported by NIH T32 OD017863. S.F.R. is supported by the Soros Foundation. A.L. is supported by the AHA Postdoctoral Fellowship grant. T.K. is supported by NIH T32 GM719876. A.C.D.M. is supported by R01 HL148801-02S1. G.G.H. and A.Z. are supported by NIH R01 HL157445. A.Z. is further supported by the VA Merit BLR&D Award I01 BX005707 and NIH grants R01 AI163483, R01 HL148801, R01 EB030134 and U01 CA265719. All authors receive institutional support from NIH P30 DK120515, P30 DK063491, P30 CA014195, P50 AA011999 and UL1 TR001442.

Author information

Authors and affiliations.

Division of Biomedical Sciences, University of California, San Diego, La Jolla, CA, USA

Celeste Allaband & Stephany Flores Ramos

Division of Gastroenterology, University of California, San Diego, La Jolla, CA, USA

Celeste Allaband, Amulya Lingaraju, Stephany Flores Ramos, Haniyeh Javaheri, Maria D. Tiu, Ana Carolina Dantas Machado, R. Alexander Richter & Amir Zarrinpar

Department of Pediatrics, University of California, San Diego, La Jolla, CA, USA

Celeste Allaband, Stephany Flores Ramos, Gabriel G. Haddad, Pieter C. Dorrestein & Rob Knight

Medical Scientist Training Program, University of California San Diego, La Jolla, CA, USA

Tanya Kumar

Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California, San Diego, La Jolla, CA, USA

Emmanuel Elijah & Pieter C. Dorrestein

Center for Microbiome Innovation, University of California, San Diego, La Jolla, CA, USA

Emmanuel Elijah, Pieter C. Dorrestein, Rob Knight & Amir Zarrinpar

Department of Neurosciences, University of California, San Diego, La Jolla, CA, USA

Gabriel G. Haddad

Rady Children’s Hospital, San Diego, CA, USA

Department of Animal and Dairy Sciences, University of Wisconsin-Madison, Madison, WI, USA

Vanessa A. Leone

Center for Computational Mass Spectrometry, University of California, San Diego, La Jolla, CA, USA

Pieter C. Dorrestein

Department of Computer Science and Engineering, University of California, San Diego, La Jolla, CA, USA

Halıcıoğlu Data Science Institute, University of California, San Diego, La Jolla, CA, USA

Shu Chien-Gene Lay Department of Bioengineering, University of California, San Diego, La Jolla, CA, USA

Rob Knight & Amir Zarrinpar

Division of Gastroenterology, Jennifer Moreno Department of Veterans Affairs Medical Center, La Jolla, CA, USA

Amir Zarrinpar

Institute of Diabetes and Metabolic Health, University of California, San Diego, La Jolla, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

C.A. and A.Z. conceptualized the work. C.A., E.E., P.C.D., R.K. and A.Z. determined the methodology. C.A., A.L., S.F.R., T.K., H.J., M.D.T., A.C.D.M. and R.A.R. were involved in data investigation. C.A., S.F.R., T.K., H.J., M.D.T., A.C.D.M. and R.A.R. created visualizations. A.Z. acquired funding and was the project administrator. R.K. and A.Z. supervised the work. G.G.H. and V.A.L. provided resources. C.A., A.L., S.F.R., T.K., H.J., M.D.T. and A.Z. wrote the first draft. All authors contributed to the review and editing of the manuscript.

Corresponding author

Correspondence to Amir Zarrinpar .

Ethics declarations

Competing interests.

A.Z. is a co-founder and a chief medical officer, and holds equity in Endure Biotherapeutics. P.C.D. is an advisor to Cybele and co-founder and advisor to Ometa and Enveda with previous approval from the University of California, San Diego. All other authors declare no competing interests.

Peer review

Peer review information.

Nature Metabolism thanks Robin Voigt-Zuwala, Jacqueline M. Kimmey, John R. Kirby and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Yanina-Yasmin Pesch, in collaboration with the Nature Metabolism team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1 microbiome literature review..

A ) 2019 Literature Review Summary. Of the 586 articles containing microbiome (16 S or metagenomic) data, found as described in the methods section, the percentage of microbiome articles from each of the publication groups. B ) The percentage of microbiome articles belonging to each individual journal in 2019. Because the numerous individual journals from Science represented low percentages individually, they were grouped together. C ) The percentage articles where collection time was explicitly stated (yes: 8 AM, ZT4, etc.), implicitly stated (relative: ‘before surgery’, ‘in the morning’, etc.), or unstated (not provided: ‘daily’, ‘once a week’, etc.). D ) Meta-Analysis Inclusion Criteria Flow Chart. Literature review resulting in the five previously published datasets for meta-analysis 11 , 13 , 28 , 29 , 30 .

Extended Data Fig. 2 Single Time Point (Non-Circadian) Example.

A ) Weighted UniFrac PCoA Plot - modified example from Moving Pictures Qiime2 tutorial data [ https://docs.qiime2.org/2022.11/tutorials/moving-pictures/ ]. Each point is a sample. Points were coloured by body site of origin. There are 8 gut, 8 left palm, 9 right palm, and 9 tongue samples. B ) Within-Condition Distances (WCD) boxplot/stripplot for each body site (n = 8–9 mouse per group per time point). C ) Between Condition Distances (BCD) boxplot/stripplot for each unique body site comparison (n = 8–9 mouse per group per time point). D ) All pairwise grouping comparisons, both WCD and BCD, are shown in the boxplots/stripplots (n = 8–9 mouse per group per time point). Only WCD to BCD statistical differences are shown. Boxplot centre line indicates median, edges of boxes are quartiles, error bars are min and max values. Significance was determined using a paired Mann-Whitney-Wilcoxon test two-sided with Bonferroni correction. Notation: ns (not significant) = p > 0.05, * = p < 0.05; ** = p < 0.01; *** = p < 0.001, **** = p < 0.00001.

Extended Data Fig. 3 Additional Analysis of Apoe-/- Mice Exposed to IHC Conditions.

A ) Weighted UniFrac PCoA stacked view (same as Fig. 2b but different orientation). Good for assessing overall similarity not broken down by time point. Significance determined by PERMANOVA (p = 0.005). B ) Weighted UniFrac PCoA of only axis 1 over time. C ) Boxplot/scatterplot of within-group weighted UniFrac distance values for the control group (Air, n = 3–4 samples per time point). Unique non-zero values in the matrix were kept. Dotted line indicates the mean of all values presented. No significant differences (p > 0.05) found. D ) Boxplot/scatterplot of within-group weighted UniFrac distance values for the experimental group (IHC, n = 3–4 samples per time point)). Unique non-zero values in the matrix were kept. Dotted line indicates the mean of all values presented. No significant differences (p > 0.05) found. E ) Boxplot/scatterplot of within-group weighted UniFrac distance values for both control (Air) and experimental (IHC) groups [n = 3–4 samples per group per time point]. Mann-Whitney-Wilcoxon test with Bonferroni correction used to determine significant differences between groups. Boxplot centre line indicates median, edges of boxes are quartiles, error bars are min and max values. Notation: ns = not significant, p > 0.05; * = p < 0.05; ** = p < 0.01; *** = p < 0.001.

Extended Data Fig. 4 Irregular differences in diurnal rhythm patterns leads to generally minor shifts in BCD when comparing LD vs DD mice.

A ) Experimental design. Balb/c mice were fed NCD ad libitum under 0:24 L:D (24 hr darkness, DD) experimental conditions and compared to 12:12 L:D (LD) control conditions. After 2 weeks, mice from each group were euthanized every 4 hours for 24 hours (N = 4–5 mice/condition) and samples were collected from the proximal small intestine (‘jejunum’) and distal small intestine (‘ileum’) contents. B ) BCD for luminal contents of proximal small intestine samples comparing LD to DD mice (N = 4–5 mice/condition). Dotted line is the average of all shown weighted UniFrac distances. Significance was determined using a paired Mann-Whitney-Wilcoxon test two-sided with Bonferroni correction; notation: **** = p < 0.00001. C ) BCD for luminal contents of distal small intestine samples comparing LD to DD mice (N = 4–5 mice/condition). Dotted line is the average of all shown weighted UniFrac distances. Boxplot centre line indicates median, edges of boxes are quartiles, error bars are min and max values.

Extended Data Fig. 5 Localized changes in BCD between luminal and mucosal contents.

A ) Experimental design and sample collection for a local site study. Small intestinal samples were collected every 4 hours for 24 hours (N = 4–5 mice/condition, skipping ZT8). Mice were fed ad libitum on the same diet (NCD) for 4 weeks before samples were taken. B ) BCD for luminal vs mucosal conditions (N = 4–5 mice/condition). The dotted line is the average of all shown weighted UniFrac distances. Significance is determined using the Mann-Whitney-Wilcoxon test two-sided with Bonferroni correction. C ) Heatmap of mean BCD distances comparing luminal and mucosal by time point (N = 4–5 mice/condition). Highest value highlighted in navy, lowest value highlighted in gold. Boxplot centre line indicates median, edges of boxes are quartiles, error bars are min and max values. Significance was determined using a paired Mann-Whitney-Wilcoxon test two-sided with Bonferroni correction. Notation: * = p < 0.05; ** = p < 0.01; *** = p < 0.001, **** = p < 0.00001. D ) Experimentally relevant log ratio, highlighting the changes seen at ZT20 (N = 4–5 mice/condition). Boxplot center line indicates median, edges of boxes are quartiles, error bars are min and max values. Significance was determined using a paired Mann-Whitney-Wilcoxon test two-sided with Bonferroni correction. Notation: * = p < 0.05; ** = p < 0.01; *** = p < 0.001, **** = p < 0.00001.

Supplementary information

Reporting summary, rights and permissions.

Reprints and permissions

About this article

Cite this article.

Allaband, C., Lingaraju, A., Flores Ramos, S. et al. Time of sample collection is critical for the replicability of microbiome analyses. Nat Metab (2024). https://doi.org/10.1038/s42255-024-01064-1

Download citation

Received : 27 October 2022

Accepted : 08 May 2024

Published : 01 July 2024

DOI : https://doi.org/10.1038/s42255-024-01064-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

This paper is in the following e-collection/theme issue:

Published on 1.7.2024 in Vol 26 (2024)

Exploring Relations Between Unique Patient Characteristics and Virtual Reality Immersion Level on Anxiety and Pain in Patients Undergoing Venipuncture: Secondary Analysis of a Randomized Control Trial

Authors of this article:

Original Paper

• Jeffrey I Gold 1, 2 , PhD   ; 
• Krystal M Akbar 3 , PhD   ; 
• Sandra Avila 3 , PsyD   ; 
• Nhat H Ngo 4 , BA, BS   ; 
• Margaret J Klein 2 , MS  

1 Departments of Anesthesiology, Pediatrics, and Psychiatry & Behavioral Sciences, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States

2 Department of Anesthesiology Critical Care Medicine, The Saban Research Institute, Children's Hospital Los Angeles, Los Angeles, CA, United States

3 The Saban Research Institute at Children's Hospital Los Angeles, Los Angeles, CA, United States

4 Department of Anesthesiology Critical Care Medicine, The Biobehavioral Pain Lab, The Saban Research Institute, Children's Hospital Los Angeles, Los Angeles, CA, United States

Corresponding Author:

Jeffrey I Gold, PhD

Department of Anesthesiology Critical Care Medicine

The Saban Research Institute

Children's Hospital Los Angeles

4650 Sunset Boulevard MS#12

Los Angeles, CA, 90027

United States

Phone: 1 3233616341

Fax:1 3233611022

Email: [email protected]

Background: Virtual reality (VR) is a well-researched digital intervention that has been used for managing acute pain and anxiety in pediatric patients undergoing various medical procedures. This study focuses on investigating the role of unique patient characteristics and VR immersion level on the effectiveness of VR for managing pediatric pain and anxiety during venipuncture.

Objective: The purpose of this study is to determine how specific patient characteristics and level of immersion during a VR intervention impact anxiety and pain levels for pediatric patients undergoing venipuncture procedures.

Methods: This study is a secondary data analysis of 2 combined, previously published randomized control trials on 252 pediatric patients aged 10-21 years observed at Children’s Hospital Los Angeles from April 12, 2017, to July 24, 2019. One randomized clinical trial was conducted in 3 clinical environments examining peripheral intravenous catheter placement (radiology and an infusion center) and blood draw (phlebotomy). Conditional process analysis was used to conduct moderation and mediation analyses to assess the impact of immersion level during the VR intervention.

Results: Significant moderation was found between the level of immersion and anxiety sensitivity when predicting postprocedural anxiety ( P =.01). Patients exhibiting the highest anxiety sensitivity within the standard of care yielded a 1.9 (95% CI 0.9-2.8; P <.001)-point elevation in postprocedural anxiety relative to individuals with high immersion levels. No other significant factors were found to mediate or moderate the effect of immersion on either postprocedural anxiety or pain.

Conclusions: VR is most effective for patients with higher anxiety sensitivity who report feeling highly immersed. Age, location of the procedure, and gender of the patient were not found to significantly impact VR’s success in managing levels of postprocedural pain or anxiety, suggesting that immersive VR may be a beneficial intervention for a broad pediatric population.

Trial Registration: ClinicalTrials.gov NCT04268901; https://clinicaltrials.gov/study/NCT04268901

Introduction

The subjective experience of pain results from complex interactions among biological, psychological, and social factors and is largely informed by early life experiences [ 1 ]. Thus, what a child remembers about initial painful experiences is a strong predictor of their response to subsequent painful experiences [ 2 - 5 ]. Anxiety has been shown to moderate children’s memory for procedural pain, increasing the likelihood of remembering more pain than they initially reported [ 6 ]. In addition to the unique interplay of pain and anxiety associated with routine medical procedures, the literature reflects that pain and anxiety associated with routine medical care can lead to adverse consequences that can affect lifelong health, such as a negative impact on one’s perception of health care, attempts to escape the distressing medical procedure, poor recovery outcomes, avoidance of preventative health care, and the risk for medical trauma [ 7 - 9 ].

Decreasing pain and anxiety during pediatric medical care is critical to ensuring optimal health and health care experiences for individuals across the lifespan. Best practice guidelines for the treatment of procedural pain and anxiety in pediatric populations have historically been cited as a combination of pharmacological and nonpharmacological interventions [ 10 - 13 ]. While pharmacological interventions (eg, sedatives and opioids) have become increasingly common, these analgesics have been linked to higher mortality risk and longer hospital admissions [ 14 ]. Additionally, these medication interventions have been associated with high tolerance, dependence, and unfavorable side effects [ 15 , 16 ]. In fact, when a study compared the efficacy of virtual reality (VR) and opioid therapy as pain management tools during thermal pain stimulation, results indicated that there was no significant difference in pain reduction between the 2 treatment groups [ 17 ]. This suggests that VR may be just as effective as routine pharmacological interventions in reducing the experience of pain and anxiety while mitigating the adverse impacts of tolerance, dependence, and other unfavorable side effects.

VR is an immersive and interactive computer-generated environment that has gained traction as a pain and anxiety management strategy within the medical field over the past few decades. In the past 5 years alone, studies have cited VR’s success in reducing pain and anxiety during dental procedures [ 18 ], cancer-related treatments [ 19 , 20 ], and burn wound care [ 21 ]. However, many children and adolescents view venipuncture—a medical procedure in which a needle is used to draw blood from a vein—as one of the most distressing aspects of attending hospital visits [ 22 , 23 ] due to a combination of fear, anxiety, and pain [ 24 ]. This finding holds important considerations for children with acute or chronic illnesses, as they are exposed to needle procedures at a much higher rate than similar-aged peers who do not have medical conditions. Further, children may experience these procedures as more distressing than adults due to receptive language, expressive language, and emotional regulation limitations that impact their understanding of the routine nature of medical procedures, their ability to communicate their experiences of pain, and their ability to cope with painful experiences, respectively [ 25 - 27 ].

Extensive research on the use of VR has established it as a safe, feasible, effective, and efficacious intervention for reducing pain and anxiety associated with routine painful medical procedures in pediatric populations compared with standard of care (SOC) [ 28 - 31 ]. Furthermore, pediatric studies have indicated that children report lower pain, distress, and anxiety scores when using VR, an immersive pain management tool, –than when using iPads (Apple Inc), a passive coping tool [ 32 - 36 ]. However, information regarding the impact of specific unique individual characteristics that affect VR’s level of efficacy has been limited to preliminary studies with outdated technologies. For example, researchers have found that a higher perception of feeling present in the VR simulation is associated with better outcomes [ 37 - 39 ], and recent studies that have explored the effects of age and sex characteristics across a range of pediatric specialty clinics have neither found any significant correlations with pain nor anxiety levels [ 40 , 41 ]. This suggests a need for further investigation into the demographic, medical, and psychological variables that may impact VR’s efficacy to further appreciate “who benefits” from VR interventions and why.

This study aims to identify individual factors that impact the degree of VR’s effectiveness in reducing pain and anxiety during routine painful medical venipuncture procedures (phlebotomy and peripheral intravenous catheter [PIVC] placement) in pediatric patients.

Study Design and Population

This study is a secondary data analysis of 2 combined, previously published randomized control trials on 252 pediatric patients aged 10-21 years observed at an urban pediatric academic medical center (Children’s Hospital Los Angeles) from April 12, 2017, to July 24, 2019 [ 29 , 30 ]. Both randomized control trials implemented the same protocol. A total of 3 clinical environments were used to examine PIVC placement (radiology and an infusion center) [ 30 ] and blood draw (phlebotomy) [ 29 ]. In total, 125 dyads (patient and caregiver) were randomized to receive the VR intervention while 125 received standard of care (SOC). Patients randomized to the VR group played a multisensory (visual and auditory) VR game, BearBlast, where users traveled on a preset path through a colorful and immersive 3D environment filled with animated landscapes, buildings, and clouds, during which the user’s gaze controlled the direction of a cannon fired to knock down teddy bears. The VR game is equipped with a head-tracking system, enabling the player to look around the virtual environment (VE), controlling the game with only the movement of their head (Oculus Gear VR). Patients and participants were English or Spanish-speaking.

Caregivers provided demographic information for patients younger than 18 years of age, while patients older than 18 years completed self-reported demographic questionnaires focused on age, gender, race or ethnicity, and relevant medical history.

The Faces Pain Scale-Revised (FPS-R) [ 42 ] was used to measure patient pain before and during the PIVC procedure, and uses a horizontal series of 6 faces displaying a range of facial expressions, from no pain (0 points) to significant pain (10 points). Patients and caregivers pointed to the face that indicated the patient’s level of pain. Across multiple studies, the FPS-R has been found to be both a reliable and valid measure of patient pain for children between the ages of 4 and 16 years of age [ 42 ].

A visual analog scale (VAS) was provided to patients and caregivers to measure patient anxiety prior to and after the PIVC procedure. The VAS provides the patient with a vertical image of a thermometer that shifts in color from yellow (accompanied by an image of a neutral face) at the bottom to dark red (accompanied by an image of a distressed face) at the top. Patients and caregivers were asked to point to the specific part of the thermometer that rated the patient’s level of anxiety, with the neutral face scoring as 0 points and the distressed face scoring 10 points. Many studies have evaluated the effectiveness of a VAS and concluded that this type of scale is subject to less bias when compared to categorical scales [ 43 ].

Anxiety Sensitivity

The Childhood Anxiety Sensitivity Index (CASI) was used to measure the patient’s anxiety sensitivity, and it is comprised of a 3-point Likert scale that assesses the patient’s belief that their anxiety will result in a negative consequence, such as sickness, embarrassment, or loss of control. On the CASI, 1 indicates no negative consequences, 2 indicates some negative consequences, and 3 indicates a lot of negative consequences (range 18-54) [ 30 ]. Studies have found that the CASI has high internal consistency (α=.87) and good test-retest reliability in both clinical ( r =0.79) and nonclinical ( r =0.76) samples [ 44 ]. Patients in our study who were missing 1 of the 18 items had their total CASI sum imputed by adding the mean of their 17 complete items to their 17-item sum. Patients missing more than 1 item of the CASI were excluded from CASI analyses.

VR Immersion

Patients in the VR group completed the Gold-Rizzo Immersion and Presence (GRIP) inventory, which is a 16-item measure that asks the patients to indicate their degree of immersion in the game, with 0 indicating no immersion, 1 indicating little immersion, and 2 indicating a lot of immersion. This measure is comprised of 3 domains—sense of involvement, perceived realism of the VR game, and sense of transportation into the experience [ 30 ]. The scores on the GRIP inventory range from 0 to 32 points, with higher scores indicating higher levels of immersion. Patients in the SOC group were given a score of 0 for the GRIP immersion score since they never experienced the VR intervention. Patients in the VR group who were missing 1 or 2 of the 16-item GRIP had their missing items imputed with their nonmissing median to create a total 16-item score. These final scores were categorized into no immersion (score=0), low (score 1-19), medium (20-25), and high immersion (>25) based on our sample’s distribution.

Data Analysis

Patient demographic and preprocedural characteristics were summarized using the median with IQR for continuous variables while frequency and percentage were used for categorical variables. Group differences were tested using the chi-square or Mann-Whitney U test.

Conditional process analysis was used for the moderation and mediation analyses [ 45 ]. Separate models were run for the proposed mediation and moderation effects of child characteristics (age, gender, procedure location, and anxiety sensitivity) and level of immersion on the outcomes of postprocedural anxiety VAS and pain FPS-R. All models were controlled for preprocedural pain or anxiety.

When pairwise comparisons were analyzed for significant differences, the Bonferroni multiple comparison adjustment was applied and simultaneous 95% CIs were presented. All P values were assessed at the α level of .05. Statistical analyses and data visualization were carried out with SAS (version 9.4; SAS Institute) for Windows and SPSS Statistics (version 28.0; IBM).

Ethical Considerations

All activities and procedures were approved by the local institutional review board at Children’s Hospital Los Angeles (CHLA-15-00549). All patients older than 18 years provided written informed consent. Caregivers of patients younger than 18 years provided patient assent and written informed consent. All activities and procedures were approved by the local institutional review board at Children’s Hospital Los Angeles.

A total of 250 patients with complete data were included in the current secondary data analysis. Of the participants, 46% (n=115) were female with a median age of 15 (IQR 13-17.3) years and 51.2% (n=128) were Hispanic or Latinx. A total of 56% (n=141) of the participants underwent phlebotomy procedures, while 17.6% (n=44) and 26% (n=65) were recruited from the radiology and infusion departments, respectively. There were no statistically significant differences in these demographic variables between the SOC and VR groups (all P >.05; Table 1 ). Preprocedural pain FPS-R, anxiety VAS scores, and anxiety sensitivity CASI scores did not differ between groups ( Table 1 ).

a P values from the chi-square test for categorical variables and the Mann-Whitney U test for continuous variables.

b SOC: Standard of care.

c VR: virtual reality.

d FPS-R: Faces Pain Scale-Revised.

e VAS: visual analog scale.

f CASI: Childhood Anxiety Sensitivity Index.

Conceptual moderation and mediation models for both postprocedural pain and anxiety are shown in Figure 1 . Among the 238 participants in the models, the CASI levels were determined using the 25th, 50th, and 75th percentile of our sample, low CASI (score=24), medium CASI (score=28), and high CASI (score=32). Significant moderation was found between the level of immersion and anxiety sensitivity when predicting postprocedural anxiety ( P =.01). At a low level of anxiety sensitivity (CASI=24; our sample’s 25th percentile), the no immersion group (SOC) had the highest adjusted mean postprocedural anxiety (2.4, 95% CI 2-2.8), but was not significantly different from the low, medium, or high immersion groups after adjusting for multiple comparisons. At the median level of anxiety sensitivity (CASI=28), significant decreases in postprocedural anxiety were found between high and medium immersion >groups versus no immersion (all P <.05; Table 2 ). These differences were sustained when anxiety sensitivity was high (75th percentile; CASI=32). No significant differences were found when comparing the low, medium, and high levels of immersion to each other ( Table 2 , Figure 2 ).

a CASI: Childhood Anxiety Sensitivity Index.

b Resulting from a moderation model controlling for preprocedural anxiety held at the mean of 2.1.

c CASI levels were determined using the 25th, 50th, and 75th percentile of our sample.

d SOC: standard of care.

e LS mean: least squared mean (adjusted mean).

f Adjusted pairwise least squared differences with simultaneous 95% CI values. P values are adjusted for multiple comparisons using the Bonferroni adjustment.

g N/A: not applicable.

h P <.05.

i P <.001.

Age, location of procedure, and gender of the patient were not significant moderators of immersion on postprocedural anxiety scores and no significant mediation models were found in our sample. No significant mediation or moderation effects of child characteristics (age, gender, procedure location, and anxiety sensitivity) with level of immersion were found for the outcome of postprocedural pain FPS-R (all P >.05).

Principal Results

The current findings suggest that a higher level of immersion during a VR intervention compared to no immersion decreases postprocedure anxiety among patients undergoing venipuncture procedures. For the purpose of this study, comparisons were made between each level of immersion. Overall, the immersion effect is stronger for patients who reported a higher anxiety sensitivity score, indicating that VR interventions work better to reduce anxiety in individuals who have a higher level of anxiety sensitivity. Patient characteristics such as age, gender, race or ethnicity, significant medical history, location of venipuncture, and type of procedure were not significant moderators of immersion on postprocedural scores, suggesting that VR intervention may be more universal in its application.

Comparisons With Prior Work

As previously discussed, various studies have concluded that VR reduces patient-reported pain and anxiety during pediatric venipuncture procedures more effectively when compared to the SOC [ 28 - 32 ]; however, this is the first study to analyze patient characteristics and immersion level as predictors of “who benefits” most optimally from the VR intervention.

The current findings support the idea that VR may lend itself to greater benefit for patients undergoing routine painful medical procedures, especially regarding anxiety management. This suggests that VEs need to be highly immersive, especially in the areas of (1) a sense of involvement, (2) perceived realism of the VR game, and (3) a sense of transportation into the experience as measured by the GRIP inventory. As the level of immersion is understood to be a critical element of the VR experience, this study begins to better understand which patients benefit most given their level of immersion. Distraction alone, as is often discussed, may not be the critical element in “why” or “how” VR works, but rather the level or degree of immersion. Previous research has alluded to the fact that deeper levels of engagement or immersion would contribute toward greater VR benefit; however, those ideas were mostly theoretical.

Future Directions

Over the years, it has been postulated that the greater the number of senses involved in the VR experience, the deeper the sense of immersion. In this study, the VR experience primarily harnessed the patient’s visual and auditory senses, and patients who were highly immersed benefited the most from the intervention. With this finding, it is important to consider the question “Would VR would be more effective if more senses were involved (eg, olfactory, tactile), and whether medium and low levels of immersion would reduce pain and/or anxiety for these patients?”

Future studies investigating unique patient characteristics that drive VR effectiveness may benefit from increasing the number of senses included in their VR experience. VEs designed to engage all 6 senses (vision, hearing, touch, taste, smell, and proprioception) may transform the VR experience beyond what we currently know. Research groups currently engaging in the kinesthetic aspect of VR, olfactory, and tactile vibration could significantly increase the immersion level for all participants, thus enhancing the effects of VR in line with the current findings.

Limitations

It is important to note that for the purpose of this study, only 1 VE was evaluated (ie, BearBlast), and the current findings are limited to the mentioned procedures and a single virtual experience. However, as the field moves toward a VR pharmacy that allows for choice and customization, the immersive nature may change and be more effective, given personal choice, selection, and preference. Additionally, the participants of this study were primarily Latinx individuals; therefore, it would be beneficial to replicate this study with patients of varying ethnic and racial backgrounds.

Conclusions

As previously mentioned, factors such as age, gender, and location of venipuncture were not significant. These findings are encouraging as they suggest that the impact of VR is more about the VR experience and less about specific patient and location characteristics. Thus, the use of VR can be implemented in various settings and should be more readily available and accessible for diverse groups of pediatric patients. It was previously discussed that children will remember past painful medical procedures [ 2 - 5 ], which may negatively inform their future perception of potentially painful procedures [ 7 - 9 ]. The use of VR with a high level of immersion can decrease future negative expectations, as well as fear and anxiety about medical procedures. Future studies and interventions should focus on VE and activities with high levels of immersion and interaction in order to best support patient care. Additionally, the use of VR interventions across patient age groups and medical settings may decrease or eliminate the need for pharmacological interventions, the associated negative side-effect profiles, and the negative impact of routine painful medical procedures on patients’ mental health, patients’ medical experiences, and ultimately reduce the fear and anxiety that may impact medical adherence in patients with the critical need for routine and complex chronic medical care.

Acknowledgments

This project was completed in partial fulfillment of the requirements of KMA’s and SA’s participation in the Psychology Postdoctoral Fellowship at Children’s Hospital Los Angeles, in the Leadership Education in Adolescent Health Training Program, and in the California Leadership Education in Neurodevelopmental Disabilities Interdisciplinary Training Program. This work was supported by laboratory funding from Beatrice and Paul Bennett (JIG), research funding from the Tower Cancer Foundation: Cancer Free Generation, and hardware and software donations from AppliedVR.

Conflicts of Interest

None declared.

• Nordgård R, Låg T. The effects of virtual reality on procedural pain and anxiety in pediatrics: a systematic review and meta-analysis. Front Virtual Real. Jul 15, 2021;2:1-33. [ FREE Full text ] [ CrossRef ]
• Noel M, Chambers CT, McGrath PJ, Klein RM, Stewart SH. The influence of children's pain memories on subsequent pain experience. Pain. Aug 2012;153(8):1563-1572. [ CrossRef ] [ Medline ]
• Noel M, Rabbitts JA, Fales J, Chorney J, Palermo TM. The influence of pain memories on children's and adolescents' post-surgical pain experience: a longitudinal dyadic analysis. Health Psychol. Oct 2017;36(10):987-995. [ FREE Full text ] [ CrossRef ] [ Medline ]
• von Baeyer CL, Marche TA, Rocha EM, Salmon K. Children's memory for pain: overview and implications for practice. J Pain. Jun 2004;5(5):241-249. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gedney JJ, Logan H. Pain related recall predicts future pain report. Pain. Mar 2006;121(1-2):69-76. [ CrossRef ] [ Medline ]
• Rocha EM, Marche TA, von Baeyer CL. Anxiety influences children's memory for procedural pain. Pain Res Manag. 2009;14(3):233-237. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chorney JM, Kain ZN. Behavioral analysis of children's response to induction of anesthesia. Anesth Analg. Nov 2009;109(5):1434-1440. [ FREE Full text ] [ CrossRef ] [ Medline ]
• El-Housseiny AA, Alamoudi NM, Farsi NM, El Derwi DA. Characteristics of dental fear among Arabic-speaking children: a descriptive study. BMC Oral Health. Sep 22, 2014;14:118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kain ZN, Mayes LC, Caldwell-Andrews AA, Karas DE, McClain BC. Preoperative anxiety, postoperative pain, and behavioral recovery in young children undergoing surgery. Pediatrics. Aug 2006;118(2):651-658. [ CrossRef ] [ Medline ]
• Birnie KA, Noel M, Parker JA, Chambers CT, Uman LS, Kisely SR, et al. Systematic review and meta-analysis of distraction and hypnosis for needle-related pain and distress in children and adolescents. J Pediatr Psychol. Sep 2014;39(8):783-808. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Trottier ED, Doré-Bergeron MJ, Chauvin-Kimoff L, Baerg K, Ali S. Managing pain and distress in children undergoing brief diagnostic and therapeutic procedures. Paediatr Child Health. Dec 2019;24(8):509-535. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Best practice guidelines for acute pain management in trauma patients. ACS Trauma Quality Programs. Nov 2020. URL: https://www.facs.org/media/exob3dwk/acute_pain_guidelines.pdf [accessed 2024-04-10]
• Shiferaw A, Mola S, Gashaw A, Sintayehu A. Evidence-based practical guideline for procedural pain management and sedation for burn pediatrics patients undergoing wound care procedures. Ann Med Surg (Lond). Nov 2022;83:104756. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stephens RJ, Dettmer MR, Roberts BW, Ablordeppey E, Fowler SA, Kollef MH, et al. Practice patterns and outcomes associated with early sedation depth in mechanically ventilated patients: a systematic review and meta-analysis. Crit Care Med. Mar 2018;46(3):471-479. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Faber AW, Patterson DR, Bremer M. Repeated use of immersive virtual reality therapy to control pain during wound dressing changes in pediatric and adult burn patients. J Burn Care Res. 2013;34(5):563-568. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hoffman HG, Patterson DR, Seibel E, Soltani M, Jewett-Leahy L, Sharar SR. Virtual reality pain control during burn wound debridement in the hydrotank. Clin J Pain. May 2008;24(4):299-304. [ CrossRef ] [ Medline ]
• Hoffman HG, Richards TL, Van Oostrom T, Coda BA, Jensen MP, Blough DK, et al. The analgesic effects of opioids and immersive virtual reality distraction: evidence from subjective and functional brain imaging assessments. Anesth Analg. Dec 2007;105(6):1776-83, table of contents. [ CrossRef ] [ Medline ]
• Shetty V, Suresh LR, Hegde AM. Effect of virtual reality distraction on pain and anxiety during dental treatment in 5 to 8 year old children. J Clin Pediatr Dent. 2019;43(2):97-102. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chow H, Hon J, Chua W, Chuan A. Effect of virtual reality therapy in reducing pain and anxiety for cancer-related medical procedures: a systematic narrative review. J Pain Symptom Manage. Feb 2021;61(2):384-394. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mohammad EB, Ahmad M. Virtual reality as a distraction technique for pain and anxiety among patients with breast cancer: a randomized control trial. Palliat Support Care. 2019;17(1):29-34. [ CrossRef ] [ Medline ]
• Khadra C, Ballard A, Déry J, Paquin D, Fortin JS, Perreault I, et al. Projector-based virtual reality dome environment for procedural pain and anxiety in young children with burn injuries: a pilot study. J Pain Res. 2018;11:343-353. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Humphrey GB, Boon CMJ, van Linden van den Heuvell GFEC, van de Wiel HBM. The occurrence of high levels of acute behavioral distress in children and adolescents undergoing routine venipunctures. Pediatrics. Jul 1992;90(1 Pt 1):87-91. [ Medline ]
• Schechter NL, Blankson V, Pachter LM, Sullivan CM, Costa L. The ouchless place: no pain, children's gain. Pediatrics. Jun 1997;99(6):890-894. [ CrossRef ] [ Medline ]
• Duff AJA. Incorporating psychological approaches into routine paediatric venepuncture. Arch Dis Child. Oct 2003;88(10):931-937. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cohen LL, MacLaren JE, Lim CS. Pain and pain management. In: Steele RG, Elkin TD, Roberts MC, editors. Handbook of Evidence-Based Therapies for Children and Adolescents: Bridging Science and Practice. Boston, MA. Springer US; Dec 7, 2008:281-295.
• Slifer KJ. A Clinician's Guide to Helping Children Cope and Cooperate with Medical Care: An Applied Behavioral Approach. Baltimore, MD. The Johns Hopkins University Press; Oct 30, 2013.
• McMurtry CM, Pillai Riddell R, Taddio A, Racine N, Asmundson GJG, Noel M, et al. Far from "just a poke": common painful needle procedures and the development of needle fear. Clin J Pain. Oct 2015;31(10 Suppl):S3-11. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arane K, Behboudi A, Goldman RD. Virtual reality for pain and anxiety management in children. Can Fam Physician. Dec 2017;63(12):932-934. [ FREE Full text ] [ Medline ]
• Gold JI, Mahrer NE. Is virtual reality ready for prime time in the medical space? A randomized control trial of pediatric virtual reality for acute procedural pain management. J Pediatr Psychol. Apr 01, 2018;43(3):266-275. [ CrossRef ] [ Medline ]
• Gold JI, SooHoo M, Laikin AM, Lane AS, Klein MJ. Effect of an immersive virtual reality intervention on pain and anxiety associated with peripheral intravenous catheter placement in the pediatric setting: a randomized clinical trial. JAMA Netw Open. Aug 02, 2021;4(8):e2122569. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wong CL, Lui MMW, Choi KC. Effects of immersive virtual reality intervention on pain and anxiety among pediatric patients undergoing venipuncture: a study protocol for a randomized controlled trial. Trials. Jun 20, 2019;20(1):369. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Özsoy F, Ulus B. Comparison of two different methods in reducing pain and fear due to dressing change in 7-10 years old children. J Pediatr Res. Mar 8, 2022;9(1):66-75. [ FREE Full text ] [ CrossRef ]
• Hundert AS, Birnie KA, Abla O, Positano K, Cassiani C, Lloyd S, et al. A pilot randomized controlled trial of virtual reality distraction to reduce procedural pain during subcutaneous port access in children and adolescents with cancer. Clin J Pain. Dec 30, 2021;38(3):189-196. [ FREE Full text ] [ CrossRef ] [ Medline ]
• İnangil D, Şendir M, Büyükyılmaz F. Efficacy of cartoon viewing devices during phlebotomy in children: a randomized controlled trial. J Perianesth Nurs. Aug 2020;35(4):407-412. [ CrossRef ] [ Medline ]
• Tennant M, Youssef GJ, McGillivray J, Clark T, McMillan L, McCarthy MC. Exploring the use of immersive virtual reality to enhance psychological well-being in pediatric oncology: a pilot randomized controlled trial. Eur J Oncol Nurs. Oct 2020;48:101804. [ CrossRef ] [ Medline ]
• Kumari S, Bahuguna R, Garg N, Yeluri R. Immersive and non-immersive virtual reality distraction on pain perception to intraoral injections. J Clin Pediatr Dent. Dec 01, 2021;45(6):389-394. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Miller S, Reid D. Doing play: competency, control, and expression. Cyberpsychol Behav. Dec 2003;6(6):623-632. [ CrossRef ] [ Medline ]
• Hoffman HG, Sharar SR, Coda B, Everett JJ, Ciol M, Richards T, et al. Manipulating presence influences the magnitude of virtual reality analgesia. Pain. Sep 2004;111(1-2):162-168. [ CrossRef ] [ Medline ]
• Gutierrez-Martinez O, Gutierrez-Maldonado J, Cabas-Hoyos K, Loreto D. The illusion of presence influences VR distraction: effects on cold-pressor pain. Stud Health Technol Inform. 2010;154:155-159. [ CrossRef ] [ Medline ]
• Piskorz J, Czub M. Effectiveness of a virtual reality intervention to minimize pediatric stress and pain intensity during venipuncture. J Spec Pediatr Nurs. Jan 04, 2018;23(1):e12201. [ CrossRef ] [ Medline ]
• Atzori B, Hoffman HG, Vagnoli L, Patterson DR, Alhalabi W, Messeri A, et al. Virtual reality analgesia during venipuncture in pediatric patients with onco-hematological diseases. Front Psychol. 2018;9:2508. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hicks CL, von Baeyer CL, Spafford PA, van Korlaar I, Goodenough B. The Faces Pain Scale-Revised: toward a common metric in pediatric pain measurement. Pain. Aug 2001;93(2):173-183. [ CrossRef ] [ Medline ]
• Klimek L, Bergmann KC, Biedermann T, Bousquet J, Hellings P, Jung K, et al. Visual analogue scales (VAS): measuring instruments for the documentation of symptoms and therapy monitoring in cases of allergic rhinitis in everyday health care. Allergo J Int. 2017;26(1):16-24. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Silverman WK, Fleisig W, Rabian B, Peterson RA. Childhood Anxiety Sensitivity Index. J Clin Child Psychol. Jun 1991;20(2):162-168. [ CrossRef ]
• Hayes AF. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach, Third Edition. New York, NY. The Guilford Press; 2022.

Abbreviations

Edited by T de Azevedo Cardoso, G Eysenbach; submitted 28.09.23; peer-reviewed by H Li, T Chang; comments to author 02.02.24; revised version received 07.03.24; accepted 20.03.24; published 01.07.24.

©Jeffrey I Gold, Krystal M Akbar, Sandra Avila, Nhat H Ngo, Margaret J Klein. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 01.07.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Arnold School of Public Health
• Location Location
• Contact Contact
• Colleges and Schools
• 2024 News Archive

I Am Public Health: Xiaowen Sun

July 1, 2024  | Erin Bluvas,  [email protected]

Xiaowen Sun discovered her love of biostatistics and public health during her master’s program. She had already studied mathematics for her bachelor's degree, and graduate school taught Sun new ways to apply what she had learned.

Originally from China, Sun grew up in Zibo – a city famous for its BBQ. The Zibo BBQ Association tallied more than 1,270 BBQ restaurants at last count, and the city hosts hundreds of thousands of hungry customers at its local food markets during seasonal festivals.

The move was not a big leap for Sun when she decided to attend Shandong University of Technology for her undergraduate studies, but her next step would take her across the world. At the University of Missouri in the United States, Sun enrolled in a master’s program focused on statistics.

I knew that USC's biostatistics program would provide me with the skills, knowledge and connections necessary to advance my career in clinical research and public health.

“My fascination with public health and biostatistics began during my master’s studies, where I was first introduced to statistical methods and their applications in real-world problems,” she says. “I was particularly drawn to survival analysis due to its critical role in medical research and public health.”

Her coursework and research projects led Sun to discover the potential of machine learning and deep learning to revolutionize data analysis. She was intrigued by the abilities of these methods to work with large-volume data sets that were high dimensional and non-linear. The application of these methods to help solve complex health care challenges cemented her commitment to the field.

Sun chose the Arnold School’s Ph.D. in Biostatistics program to elevate her analytical skills in clinical research. The curriculum was the perfect fit for her interests, and the Department of Epidemiology and Biostatistics offered numerous opportunities to be involved in varied projects led by enthusiastic faculty.

“The program’s focus on hands-on projects and real-world applications was highly appealing, and its reputation and the strong network of alumni also played a crucial role in my decision,” Sun says. “I knew that USC's biostatistics program would provide me with the skills, knowledge and connections necessary to advance my career in clinical research and public health.”

She found a mentor in her dissertation advisor, biostatistics professor Jiajia Zhang .

“Under her guidance, I have gained a deep understanding of advanced statistical methodologies and their applications in public health research,” Sun says. “She taught me how to approach complex data problems with a meticulous and analytical mindset, ensuring precision and accuracy in my work. Moreover, Dr. Zhang has provided invaluable career advice, helping me to set and achieve my professional goals.”

As a graduate research assistant with the South Carolina SmartState Center for Healthcare Quality , Sun amassed the research experience she was looking for by contributing to collaborative projects. She also spent a summer interning at Novartis with the pharmaceutical company’s immunology department.

Since last fall, Sun has been working at the MD Anderson Cancer Center at the University of Texas as a research biostatistician. She will wrap up her dissertation research over the next several months and plans to graduate later this year.

“My degree from USC has equipped me with a comprehensive understanding of biostatistics and its applications in clinical research,” Sun says. “The advanced coursework and hands-on projects have significantly enhanced my analytical skills, enabling me to tackle complex data challenges effectively.”

Study Biostatistics

Biostatisticians improve public health by unraveling the true story hidden within increasingly complex data. Our graduates hold positions in universities, research institutes, governmental agencies such as CDC, NIH, state health departments, and in private industry including pharmaceutical companies and even financial institutions.

Challenge the conventional. Create the exceptional. No Limits.

A bigura-based real time sentiment analysis of new media

Public opinion mining is an active research domain, especially the penetration of the internet and the adoption of smartphones lead to the enormous generation of data in new media. Thus generation of large amounts of data leads to the limitation of traditional machine learning techniques. Therefore, the obvious adoption of deep learning for the said data. A multilayer BiGura modal-based technique for real-time sentiment detection is proposed. The proposed system is analysed on different viral incidents such as Gaza’s invision. The exact case scenario is as follows “Taking Israel’s demand for millions of people from northern Gaza to migrate to the south”. In the experiment, the highest accuracy of the model in evaluating text content emotions and video content emotions reached 92.7% and 86.9%, respectively. Compared to Bayesian and K-nearest neighbour (KNN) classifiers, deep learning exhibits significant advantages in new media sentiment analysis. The classification accuracy has been improved by 3.88% and 4.33%, respectively. This research identified the fidelity of real-time emotion monitoring effectively capturing and understanding users’ emotional tendencies. It can also monitor changes in public opinion in real-time. This study provides new technical means for sentiment analysis and public opinion monitoring in new media. It helps to achieve more accurate and real-time monitoring of public opinion, which has important practical significance for social stability and public safety.

Introduction

In the digital media environment, the emergence of new media provides convenient ways for the public to obtain and express information, greatly promoting the diversification and personalization of information dissemination ( Hafzullah Tuncer, 2021 ; Jin et al., 2021 ). However, what comes with it is how to accurately understand and analyze the emotional and public opinion tendencies brought about by these massive amounts of information. This has significant implications for policymakers, marketers, and even the general public. An effective solution is to use deep learning technology for sentiment analysis and public opinion monitoring ( Su, 2021 ; Middya, Nag & Roy, 2022 ). Deep learning technology, with powerful nonlinear mapping and pattern recognition capabilities, has become a powerful tool for processing complex, high-dimensional, and unstructured data, especially demonstrating excellent performance in data analysis fields such as text, image, and speech ( Hafzullah Tuncer, 2021 ; Hui, 2021 ). Therefore, deep learning technology for emotional analysis and public opinion monitoring of new media content has high research and practical value. The research aims to explore emotional analysis and public opinion monitoring methods in the new media environment through deep learning technology. Firstly, deep learning technology is used for feature extraction and sentiment classification of new media content. Based on this, public opinion tendency is predicted.

Deep learning is continuously enhancing the abilities of automated systems for diverse data needs including sentiment detection, helps to timely identify potential risks, guide public opinion, and respond to crises. The contribution lies in providing a new and effective method for sentiment analysis and public opinion monitoring in new media.

Moreover, this method has important application value in understanding the public’s emotional attitudes towards specific topics and predicting and controlling public opinion risks. In addition, this also provides new research perspectives and methodological references for the relevant theories of new media sentiment analysis and public opinion monitoring ( Su, 2021 ). The research has four parts. The first part is an overview of new media sentiment analysis and public opinion monitoring technology based on deep learning. The second part is the research on new media sentiment analysis and public opinion monitoring technology based on deep learning. The third part is the experimental verification for the second part. The fourth part is a summary and points out the shortcomings.

Related Works

With the rapid development of new media, the application of deep learning in the emotional analysis of new media has become increasingly widespread, attracting wide attention from the academic community. Chen & Zhang (2023) applied edge computing and deep learning models to the emotion recognition model for non-profit organizations ( Middya, Nag & Roy, 2022 ; Hafzullah Tuncer, 2021 ; Hui, 2021 ; Chen & Zhang, 2023 ). The purpose is to understand the evolutionary mechanism of online public opinion in sudden public events. The research results reveal that non-profit organization text annotation based on emotional rules can achieve good recognition performance. The improved convolutional neural network has significantly better recognition performance than traditional support vector machines. This work provides a technical basis for non-profit organizations to scientifically handle sudden public events ( Chen & Zhang, 2023 ). Manohar & Logashanmugam (2022) proposed a new method for speech emotion recognition. The deep learning model is used to process preprocessed public speech emotion recognition datasets. When the learning rate is 85, the classification accuracy of the model is 3.15%, 5.37%, 4.25%, and 4.81%, higher than the particle swarm optimization algorithm, Grey Wolf Optimizer (GWO) algorithm, Whale Optimization Algorithm (WOA) algorithm, and the Dynamic Hybrid Optimization Algorithm (DHOA), respectively. It proves the superior performance of the model in speech emotion recognition. Zhang, Dai & Zhong (2022) proposed a deep learning computing method and an emotion recognition method. A public sentiment network communication model is established. The proposed Recurrent Neural Network-Convolutional Neural Network (RNN-CNN) structure can reduce the waiting time by about 20% compared to traditional models. The algorithm accuracy is improved by at least 3.1%. It can accurately reflect the emotional state of the public, providing a practical basis for the application of AI technology in online public opinion judgment. Schoneveld, Othmani & Abdelkawy (2021) proposed a deep learning-based audiovisual emotion recognition method to achieve an understanding of complex human behaviour. This method utilizes a model-level fusion strategy to fuse deep feature representations of audio and visual modalities. Recurrent neural network is used to capture temporal dynamics. The research results show that this method outperforms existing technologies in predicting the potency of the RECOLA dataset. It performs well on the AffectNet and Google facial expression comparison datasets. Wang, Luo & Song (2021) proposed a hybrid neural network model based on Recurrent Neural Network-Convolutional Neural Network (RNN-CNN) to solve the precise classification. The accuracy is 92.8%, the minimum loss rate is 0.2, and the trend is stable. The model can obtain more semantic information between texts. It can also better capture the dependency relationships.

Although many studies have explored the application of deep learning in public opinion monitoring, there are still many unresolved issues and areas for improvement in this field. El Barachi et al. (2021) proposed a novel framework. A complex bidirectional long short term memory (LSTM) classifier is used for real-time evaluation of the viewpoints of well-known public figures and their followers. The results show that the classifier has an accuracy of over 87% in identifying multiple emotions and viewpoints. The recognition accuracy of negative emotions is higher. Zheng & Xu (2021) proposed a deep learning-based facial detection and tracking framework. This framework integrates the SENResNet model based on a squeezing excitation network and residual neural network, as well as a face-tracking model based on a regression network. The SENResNet can accurately detect facial information and provide an initialization window for facial tracking. Numerous experimental results have shown that this framework outperforms existing technologies in accuracy and performance. Wang & Gao (2023) proposed a solution based on deep learning ( El Barachi et al., 2021 ). A system framework including text extraction, keyword extraction, and sentiment analysis modules is designed. An information extraction model is constructed using convolutional neural networks. By calculating the global Mutual Information (MI) values of text items and categories and inputting them into the model, information extraction results are obtained. The system has high extraction accuracy and fast extraction time. Keivanlou-Shahrestanaki, Kahani & Zarrinkalam (2022) proposed a deep-learning neural network architecture. This architecture explores the adaptive effects of different attention mechanisms, generating non-satirical posts with the same meaning as the original satirical posts. Numerous experimental results have demonstrated the effectiveness of this method in explaining satirical articles, especially when dealing with long posts.

In summary, deep learning has made significant contributions to new media sentiment analysis and public opinion monitoring, particularly demonstrating outstanding capabilities in processing large-scale, complex, and unstructured data. However, existing methods face challenges in dealing with complex, multi-semantic, and emotional new media content. Future research needs to develop more complex models to optimize the understanding and capture of emotional expression. In addition, empirical research is needed to verify the effectiveness and practicality of deep learning in the application of new media sentiment analysis and public opinion monitoring. Despite the challenges, the application prospects of deep learning-based new media sentiment analysis and public opinion monitoring remain broad ( Zheng & Xu, 2021 ). There is extensive application space in public opinion risk warning, public opinion analysis, and marketing decision-making.

Research Method

New media sentiment analysis and public opinion monitoring are key links in data mining. Deep learning technology has shown outstanding performance in processing large-scale data and extracting emotional information ( Zhang, Dai & Zhong, 2022 ). The attention convolutional neural network conditional random field word segmentation model combines attention mechanism and convolutional neural network to improve word segmentation accuracy and efficiency. The emotion analysis model based on aspect information utilizes aspect information to deepen the understanding and analysis of the emotional polarity of text ( De Martino & Netti, 2020 ). The combination of the two models can further improve the accuracy and depth of new media sentiment analysis and public opinion monitoring, providing accurate references for decision-making.

Construction of the attention convolutional neural network conditional random field word segmentation model

The attention convolutional neural network conditional random field word segmentation model is a deep learning model that combines the attention mechanism and convolutional neural network to improve accuracy and efficiency. The attention mechanism can focus on key vocabulary with important information. Convolutional neural networks can effectively capture the contextual relationships between words. Through this combination, the model can more accurately segment new media texts, providing more accurate input information for subsequent sentiment analysis and public opinion monitoring ( Li & Xu, 2020 ; Seokhoon, Jihea & Young-Sup, 2023 ). The overall framework diagram of the attention convolutional neural network conditional (ACNNC) model is displayed in Fig. 1 .

Figure 1: Overall framework diagram of the ACNNC model.

In Fig. 1 , the ACNNC model is composed of five levels. The order is an embedding layer, attention layer, CNN layer, fusion layer, and CRF layer. Firstly, the embedding layer generates 128-dimensional word vectors through training, which will serve as inputs for subsequent levels. Then, the word vector is input to the attention layer and CNN layer. The former is responsible for learning the overall features of the sequence. The latter learns the positional and local features of the sequence. Next, the fusion layer integrates the overall, local, and positional features obtained from the above learning. Finally, the integrated features are decoded at the CRF layer to complete the construction of the model ( Hou, Wang & Wang, 2023 ).

The embedding layer converts the i th word in a sentence into a word vector. Then, it is trained to obtain the word vector matrix. The size of the word vector matrix is determined by the effective length of the vocabulary in the corpus dataset and the dimension of the input word vector. Each word can be transformed into a corresponding word vector representation through a word vector matrix. The embedding layer is displayed in Eq. (1) . (1) e i = X × v i

In Eq. (1) , X is the word vector matrix. N is the effective length of the vocabulary in the corpus dataset. v i is a One-hot vector of size N . The sentence x isconverted to S , as shown in Eq. (2) . (2) S = e 1 , e 2 , … , e n ∈ R N × d .

In Eq. (2) , d stands for the dimension of the input word vector. The individual attention is shown in Eq. (3) . (3) h e a d i = A t t e n t i o n Q W i Q , K W i K , V W i V .

In Eq. (3) , Q ,  K ,  V stand for the same value. The attention layer adopts a multi-head self-attention mechanism to reduce the possible random errors caused by the single-head attention mechanism and further improve the accuracy. Unlike conventional attention mechanisms, the dependency of self-attention mechanisms lies in themselves. That is, the query, key, and value are all the same value. The multi-head self-attention mechanism is shown in Fig. 2 .

Figure 2: Multi-head self-attention mechanism.

In Fig. 2 , the query, key, and value are first linearly mapped, followed by eight individual attention calculations. Finally, the results are concatenated. The characteristic of this self-attention mechanism is that the queries, keys, and values are all the same value. In this way, the multi-head self-attention mechanism can improve model accuracy. In the CNN layer of the model, only the convolutional layer and output layer are included. Input sentences processed by the embedding layer are fed into the convolutional layer for convolutional operations ( Fang et al., 2022 ; Yang & Song, 2022 ). The convolution operation is shown in Eq. (4) . (4) x i k = σ ∑ X i : i + k ∘ H k + b .

In Eq. (4) , σ stands for the activation function. ∘ is a point multiplication operation. X i : i + k is a sequence of word vectors. H k is a convolutional kernel of size k . The fusion layer integrates the features of the attention layer and the CNN layer, generating feature parameters containing overall, local, and positional information and then inputting them into the CRF layer. This model utilizes a vector concatenation strategy for feature fusion. CRF is a discriminative undirected graph model that can consider changes in data content and labels. It is widely used in tasks such as Chinese word segmentation, named entity recognition, and part of speech tagging. The processed sentences in the model are output through the fusion layer and then input into the CRF layer to obtain the results of sequence annotation. After indexing and standardizing the scores of sequence annotations, the final probability value is obtained. The calculation of S t is shown in Eq. (5) . (5) S y x = ∑ i , k 1 θ k 1 t k 1 y i − 1 , y i , x , i + ∑ i , k 2 μ k 2 s k 2 y i , x , i .

In Eq. (5) , O t represents the character after passing through the fusion layer. t k 1 y i − 1 , y i , x , i is the transfer feature function. s k 2 y i , x , i is the state feature function. θ k 1 and μ k 1 are the model parameters to be estimated, respectively. The probability value is shown in Eq. (6) . (6) P y ˆ x = 1 Z x θ k 1 t k 1 y i − 1 , y i , x , i + ∑ i , k 2 μ k 2 s k 2 y i , x , i .

In Eq. (6) , P y ˆ x is the probability value. The normalization factor is the sum of all possible marker sequences, as shown in Eq. (7) . (7) Z x = ∑ y exp ∑ i , k 1 θ k 1 t k 1 + ∑ i , k 2 μ k 2 s k 2 y i , x , i .

Equation (7) Z is the normalization factor. The task of word segmentation sequence annotation is to select the most likely marking sequence for each character in a given sentence. The markers include the Begin (B), Middle (M), End (E), and Single (S). The sequence annotation model is used for training and prediction, calculating conditional probabilities. Then, the sequence with the highest probability is output to obtain the segmentation result. The process diagram of the word segmentation sequence is shown in Fig. 3 .

Figure 3: Process diagram of word segmentation sequence.

In Fig. 3 , the word segmentation process includes four stages: preprocessing, model training, prediction, and post-processing. During the preprocessing stage, special symbols, punctuation, and stop words are removed to reduce model complexity. During the model training phase, a large amount of annotation data is used to train sequence annotation models, such as HMM, BiLSTM, etc ., to generate the most likely label sequence for each character or sub-word. In the prediction stage, a trained model is used to segment new sentences and output a tag sequence representing the most likely markers for each character or sub-word. The post-processing stage converts the tag sequence into the actual segmentation result. The specific methods for each stage depend on task requirements and resource constraints.

Construction of an emotional analysis model based on aspect information

On the basis of the ACNNC model, this study further explores the construction of an emotion analysis model based on aspect information. Emotional analysis faces complex contextual and subjective challenges in extracting emotional tendencies from texts. To obtain emotional information more accurately, aspect information is introduced for more in-depth text fine-grained emotional analysis. The BiGRUA model flowchart is shown in Fig. 4 .

Figure 4: BiGRUA model flowchart.

In Fig. 4 , the construction of the model includes five stages, preprocessing, input layer, Bidirectional Gated Recurrent Unit (BiGRU) layer, attention layer, and classification layer. Firstly, in the preprocessing stage, the original text data is cleaned and standardized to remove irrelevant information and extract effective features. Next, the input layer trains the data through the Word2vec model to obtain a word vector matrix. It can transform the original text data into a sequence of word vectors with certain semantic information. Next, the training set and corresponding training labels are fed into the BiGRU layer. At this stage, the model learns contextual information in the text through forward and backward GRU and updates model parameters to better capture semantic information in the text. The output of the BiGRU layer is fed into the attention layer. The attention mechanism focuses on the most important parts of the text, thereby extracting more representative features. Finally, the task of the classification layer is to classify emotions based on the output of the attention layer. Through this process, the model can extract useful features from the original text data and perform accurate emotional analysis. The BiGRUA model framework diagram is shown in Fig. 5 and its clever setting is compared with other models.

Figure 5: BiGRUA model framework diagram ( Han et al., 2023 ).

In Fig. 5 , BiGRUA is proposed for sentiment analysis of text containing multiple words, which integrates aspect information. It uses GRU instead of LSTM for feature learning. The attention mechanism is used to allocate aspect information weights. Finally, sentiment analysis is achieved through a softmax classifier. The BiGRU layer solves the gradient vanishing and long-distance semantic capture in recurrent neural networks by using GRU units in update and reset gates. It combines positive and negative implicit states, which can fully learn reverse semantics and obtain more complete contextual information. The BiGRU layer structure is shown in Fig. 6 .

Figure 6: BiGRU layer structure.

In Fig. 6 , the unit structure of GRU plays a key role in the sentiment analysis model based on aspect information. Based on reset and update gates, the model can handle long-distance semantic dependencies and adapt to complex sentiment analysis tasks. The reset gate determines the old information that the model should retain or discard before further processing new information. The update gate is responsible for how to integrate new and old information to generate the latest status. This mechanism enables GRU to effectively alleviate the gradient disappearance while capturing long-term dependencies. In the emotion analysis model based on aspect information, the unit structure of GRU has crucial impacts on the accuracy of the model due to the special design and critical role. Based on the clever setting of reset and update gates, this model can handle the complex task of finding semantic associations in long-distance contexts, thus adapting to the complexity of sentiment analysis. This clever setting may be described from its connectivity with self-attention and Bayes condition probabilities. For a clear understanding, an the architecture of BiGRU is used for text emotion detection and visual emotion detection is given in the Fig. 5 . The purpose of resetting the gate is to determine which old internal states should be retained or discarded before further processing new information. This is a key mechanism. GRU can flexibly adjust the internal state based on new input and context, thereby effectively capturing and understanding long-distance dependencies in text. The update gate is responsible for fusing new and old information to generate the latest status. This mechanism is implemented through a carefully designed gating mechanism. It can fuse and update new and old information appropriately based on the characteristics and context of the input text. This design enables GRU to effectively alleviate the gradient vanishing while capturing long-term dependencies, thereby improving the stability and accuracy of the model.

The output of the BiGRU layer at a time t is displayed in Eq. (8) . (8) h t ′ = h → t , h ← t .

In Eq. (8) , h t ′ is the output layer. The attention layer is to distinguish the importance of different parts in a sentence, as the influence of target words and viewpoint words on the emotional orientation in the text is different. The attention mechanism is applied to assign different weights to different parts, highlighting the contribution of different aspect words to emotional tendencies. The input is the output of the BiGRU layer, which is transformed into a new hidden vector through a multi-layer perceptron. Then it is calculated with the context vector to obtain the weight value. This context vector is a high-dimensional vector used to determine the importance of words in a sentence. The attention layer is displayed in Eq. (9) . (9) u t ′ = tanh W w h t ′ + b w α t = exp u t ′ T u v ′ ∑ t exp u t ′ T u v ′ .

Equation (9) , u t ′ is the new hidden vector. α t is the weight value. u v ′ is a high-dimensional vector used to determine the importance of words in a sentence. The average target word vector is shown in Eq. (10) . (10) V a t = 1 m ∑ i = 1 m e i .

Equation (10) m is the target word in the sentence. V at G is the target word vector. The fusion vector is shown in Eq. (11) . (11) P i = h t ′ + V a t .

In Eq. (11) , h t ′ is the hidden vector of the output. P i is the fusion vector. The emotional features of attention weights that integrate aspect information are shown in Eq. (12) . (12) C ′ = ∑ t α t h t ′ .

In Eq. (12) , C ′ integrates the emotional features of aspect information attention weights. The classification layer inputs the features captured by the attention layer into the softmax classifier for classification. The activation function adopts the sigmod function, as shown in Eq. (13) . (13) y ˜ = s o f t max W C ′ C ′ + b C ′ .

In Eq. (13) , W C ′ is the weight matrix. b C ′ is the offset value. y ˜ is the classification result. The cross entropy loss function is shown in Eq. (14) . (14) L = − 1 N ∑ r = 1 m ∑ q = 1 C y r q log y ˜ r q + λ θ 2 .

In Eq. (14) , y and y ˜ stands for the actual label values and predicted label values. C is the number of label categories. λ is the L 2 regularization coefficient.

New media sentiment analysis and public opinion monitoring technology analysis based on deep learning

The rapid development and widespread application of new media have enabled the rapid acquisition and dissemination of large-scale public sentiment tendencies and public opinion dynamic information. Deep learning, as a complex and powerful machine learning technology, has been widely applied in sentiment analysis and public opinion monitoring. The application effect of deep learning in new media sentiment analysis is discussed in detail. The performance of new media public opinion monitoring technology is compared, providing theoretical reference and practical guidance for research in related fields.

The application effect of deep learning in new media sentiment analysis

In the new media sentiment analysis, the application effect of deep learning has attracted much attention. By constructing precise models and algorithms, deep learning can accurately identify and understand public emotions, providing valuable emotional feedback for enterprises and organizations. Table 1 displays the parameters.

To better demonstrate the practical application effect of deep learning in emotional analysis, a study collected new media content from a certain self-media platform as the parent dataset and divided the dataset into two sub-datasets: text content and video content. Based on these two sub-datasets, experiments were conducted on binary and ternary classification, as shown in Fig. 7 .

Figure 7: Experiment on two and three classifications of new media content.

During the experiment, the study conducted multiple iterations based on different parameter settings and recorded the accuracy of the model after each iteration. All parameter settings and corresponding accuracy are shown in Table 1 . In Fig. 7 , in the binary classification experiment, the model achieved the highest recognition accuracy of 92.7% in text-content emotional expression and 86.9% in video-content emotional expression evaluation. The accuracy of recognizing emotions in text content gradually increased from 82.9% in the first iteration to the highest in the 11th iteration and fluctuated after that. The accuracy of video content emotion recognition fluctuates and increases, reaching its highest after 15 iterations. When analyzing new media content datasets, the highest accuracy can reach 91.4%. In the three classification experiments, the results were similar to those of the two classifications. The highest accuracy of text content emotion expression recognition is 90.8%, and the highest accuracy of the parent dataset is 89.0%. The accuracy of emotional expression evaluation in video content is relatively low. The highest accuracy of the ternary classification is lower than that of the binary classification. The reason may be that the increase in classification types led to a decrease in accuracy. Deep learning achieved high accuracy in emotional analysis of text content and video content, whether it is for binary or ternary classification. These experimental results fully demonstrated the superior performance of deep learning in new media sentiment analysis. It further confirmed the effectiveness of the parameter settings in Table 1 . The accuracy of deep learning in new media sentiment analysis is shown in Fig. 8 .

Figure 8: Training classification accuracy based on NB and KNN classifiers.

In Fig. 8 , deep learning demonstrates significant advantages in the analysis of new media emotions. The classification accuracy is improved by 3.88% and 4.33% compared to Bayesian and KNN classifiers, respectively. After optimization, the accuracy is further improved to 6.74% and 6.97%. However, the performance of Bayesian models deteriorates when labelled data accounts for 40%. KNN performs excellently when labelled data accounts for 20%. Deep learning has improved the accuracy of sentiment classification and assisted in real-time public opinion monitoring, which is of great value for policy formulation and marketing.

Experimental results and analysis of deep learning in public opinion monitoring

The sentiments expressed via the Sina Weibo API center around the notion of Israel urging millions of individuals from northern Gaza to relocate southward. Following data preprocessing, the information is structured in JSON format. Using this scenario as a case in point, an examination is conducted on the comprehensive metrics encompassing topic popularity, sentiment ratio within comments, and the intensity of emotions expressed. In light of Israel’s relocation request, there has been a notable escalation in the intensity of public discourse, underscoring the necessity for vigilant public opinion monitoring, particularly in gauging both the fervor surrounding an incident and the prevailing emotional inclinations among the populace. The intensity of discussion, as reflected in the heat score pertaining to the topic “Israel requires millions of people from northern Gaza to migrate to the south,” is illustrated in Fig. 9 .

Figure 9: Score chart of the popularity of the topic “Israel demands millions of people from northern Gaza to migrate to the south”.

Figure 9 depicts the emotional landscape surrounding the topic of “Israel requiring millions of people from northern Gaza to migrate to the south.” Initially, following Israel’s relocation proposal, there was a surge in public sentiment and the intensity of discussion. However, over time, the frequency of public discourse decreased, and emotions gradually subsided, returning to a state of calm. This underscores the significance of robust public opinion monitoring models, which necessitate keen observation of both the momentum of events and the prevailing sentiments among the public. The emotional orientation map pertaining to the topic “Israel demands millions of people from northern Gaza to migrate to the south” is illustrated in Fig. 10 .

Figure 10: Emotional tendency map of the topic “Israel demands millions of people from the north of Gaza to move to the south”.

Figure 10 presents the emotional orientation map concerning “Israel’s demand for millions of people from northern Gaza to migrate to the south.” Initially, as the incident unfolded, the public expressed their perspectives with an expectation of Israel’s handling being reasonable, hence marking a low point in emotional inclination around the fifth timestamp. However, with Israel’s relocation request, public sentiment surged dramatically. Yet, by the 10th timestamp, due to various complex factors, emotions experienced a rebound. The evolution of the comprehensive score reflecting the intensity of emotional responses to the topic is depicted in Fig. 11 .

Figure 11: Evolution chart of comprehensive score of topic emotional intensity.

In Fig. 11 , initially, there is a noticeable level of heat, but no comments are recorded, resulting in a score of 0. As time progresses, attention towards the events escalates, leading to a surge in comments and an overall increase in the score reflecting emotional intensity. By the third timestamp, when Israel makes the relocation request, the score reaches its peak value, indicating heightened emotional engagement. However, as the event unfolds further, the score gradually diminishes, suggesting a gradual decrease in emotional intensity over time.

Emotional analysis and public opinion monitoring in the contemporary media landscape pose formidable challenges. The aim is to precisely discern public sentiments and trends in public opinion, thereby furnishing a foundation for informed decision-making. To address this, a novel public opinion monitoring model leveraging deep learning technology has been devised. Results from three classification experiments closely mirror those of the two-classification scenario. The pinnacle of accuracy in emotional analysis of textual content stands at 90.8%, while the highest accuracy achieved with the parent dataset reaches 89.0%. Notably, emotional recognition in video content initially registers the lowest accuracy. However, through model optimization efforts, this accuracy is notably enhanced, reaching 6.74% and 6.97%, respectively.

Meanwhile, deep learning models offer the capability to track and analyze the evolving emotional intensity of events in real-time, thereby facilitating the monitoring of shifts in public opinion dynamics. Using Israel’s migration request as an illustrative example, the model adeptly captures the progressive increase in the comprehensive score reflecting emotional intensity over time, peaking at the third timestamp. The significance of this research lies in the introduction of a novel media sentiment analysis and public opinion monitoring model grounded in deep learning principles. This model enables real-time monitoring of public opinion shifts while upholding a high level of accuracy, thereby holding substantial implications for social governance and decision-making processes. Nevertheless, owing to constraints inherent in the training dataset, the model may encounter occasional misjudgments when confronted with emotionally nuanced or complex scenarios. Consequently, future research endeavors will focus on refining the model, enhancing its generalizability and accuracy, thereby enabling its broader applicability across diverse scenarios.

Supplemental Information

Computer code in python, download article, report a problem.

Common use cases Typos, corrections needed, missing information, abuse, etc

Our promise PeerJ promises to address all issues as quickly and professionally as possible. We thank you in advance for your patience and understanding.

Typo Missing or incorrect metadata Quality: PDF, figure, table, or data quality Download issues Abusive behavior Research misconduct Other issue not listed above

Follow this publication for updates

You can also choose to receive updates via daily or weekly email digests. If you are following multiple publications then we will send you no more than one email per day or week based on your preferences.

Note: You are now also subscribed to the subject areas of this publication and will receive updates in the daily or weekly email digests if turned on. You can add specific subject areas through your profile settings.

Change notification settings or unfollow

Loading ...

Usage since published - updated daily

Share this publication

Articles citing this paper.

Physical Review Research

• Collections
• Editorial Team
• Open Access

High harmonic generation from an atom in a squeezed-vacuum environment

Shijun wang, shaogang yu, xuanyang lai, and xiaojun liu, phys. rev. research 6 , 033010 – published 1 july 2024.

• No Citing Articles
• INTRODUCTION
• THEORETICAL METHOD
• RESULTS AND DISCUSSION
• CONCLUSIONS AND OUTLOOK
• ACKNOWLEDGMENTS

We investigate high harmonic generation (HHG) of an atom in the presence of squeezed vacuum of a single harmonic mode. Based on a fully quantum time-dependent Schrödinger equation, we derive an analytical formula for the harmonic amplitude. Our simulations of the HHG spectrum with this formula show that the harmonic amplitude of the corresponding squeezed mode can undergo significant changes with different parameters of the squeezed vacuum. Using the time-frequency analysis method, the physics underlying the effects of the vacuum quantum fluctuation (VQF) on the harmonic generation is revealed, which is found to be consistent with the explanation of Fermi's golden rule. Our work establishes a profound connection between harmonic generation and VQF, and may provide an unconventional approach to manipulating harmonic emission.

• Received 12 June 2023
• Revised 25 March 2024
• Accepted 5 June 2024

DOI: https://doi.org/10.1103/PhysRevResearch.6.033010

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Physics Subject Headings (PhySH)

• Research Areas

Authors & Affiliations

• 1 State Key Laboratory of Magnetic Resonance and Atomic and Molecular Physics, Wuhan Institute of Physics and Mathematics , Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences, Wuhan 430071, China
• 2 School of Physical Sciences, University of Chinese Academy of Sciences , Beijing 100049, China
• 3 Wuhan Institute of Quantum Technology, Wuhan 430206, China
• * Contact author: [email protected]
• † Contact author: [email protected]

Article Text

Vol. 6, Iss. 3 — July - September 2024

Subject Areas

Authorization Required

Other options.

• Buy Article »
• Find an Institution with the Article »

Download & Share

Schematic view of HHG from intense laser-atom interaction in a squeezed vacuum of a harmonic mode. The motion of an electron in a strong driving laser field (red curve) is depicted by the green curve [ 25, 26 ], and the emission of harmonics resulting from the electron's transition back to the atomic core is represented by the blue curve. Due to the influence of the quantum fluctuation of the squeezed vacuum (gray shadow) on the electron transition probability, the harmonic amplitude of the corresponding squeezed mode emitted at time t r is weak but becomes strong at t r ′ .

(a), (b) TDSE simulations of the HHG spectra of an atom in a squeezed vacuum of the first-order harmonic mode with the squeezing angle θ k = 0.9 π and the ninth-order harmonic mode with θ k = 0.27 π , respectively, as a function of the squeezing parameter r k . Considering a realistic scenario, the wavelengths of the driving light are 800 nm and 3600 nm, respectively. For more details, see the text. In (b), to better illustrate the change of the amplitude of the ninth-order harmonic, only a part of the harmonic orders are shown. (c) and (d) The amplitude of the first- and ninth-order harmonics in the corresponding single-mode squeezed vacuum state with different r k as a function of θ k . In our simulation, the driving light has a peak intensity of I = 1.15 × 10 14 W / cm 2 with a trapezoidal profile (up- and down-ramped over two cycles, constant over six cycles).

The wavelet time-frequency profile of the harmonic of the corresponding squeezed mode with r k = 0 (solid black curve) and r k = 0.5 (dashed red curve). The gray dashed curves with respect to the right axis denote the | μ k ( t ) | of the squeezed mode. (a) and (b) present the squeezed vacuum of the first-order harmonic mode with θ k = 0.9 π and 1.9 π , respectively. (c) and (d) present the squeezed vacuum of the ninth-order harmonic mode with θ k = 0.27 π and 1.27 π , respectively.

Sign up to receive regular email alerts from Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

• Forgot your username/password?
• Create an account

Article Lookup

Paste a citation or doi, enter a citation.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Textual Analysis | Guide, 3 Approaches & Examples

Textual Analysis | Guide, 3 Approaches & Examples

Published on November 8, 2019 by Jack Caulfield . Revised on June 22, 2023.

Textual analysis is a broad term for various research methods used to describe, interpret and understand texts. All kinds of information can be gleaned from a text – from its literal meaning to the subtext, symbolism, assumptions, and values it reveals.

The methods used to conduct textual analysis depend on the field and the aims of the research. It often aims to connect the text to a broader social, political, cultural, or artistic context. Relatedly, it’s good to be careful of confirmation bias when conducting these sorts of analyses, grounding your observations in clear and plausible ways.

Table of contents

What is a text, textual analysis in cultural and media studies, textual analysis in the social sciences, textual analysis in literary studies, other interesting articles.

The term “text” is broader than it seems. A text can be a piece of writing, such as a book, an email, or a transcribed conversation. But in this context, a text can also be any object whose meaning and significance you want to interpret in depth: a film, an image, an artifact, even a place.

The methods you use to analyze a text will vary according to the type of object and the purpose of your analysis:

• Analysis of a short story might focus on the imagery, narrative perspective and structure of the text.
• To analyze a film, not only the dialogue but also the cinematography and use of sound could be relevant to the analysis.
• A building might be analyzed in terms of its architectural features and how it is navigated by visitors.
• You could analyze the rules of a game and what kind of behaviour they are designed to encourage in players.

While textual analysis is most commonly applied to written language, bear in mind how broad the term “text” is and how varied the methods involved can be.

Prevent plagiarism. Run a free check.

In the fields of cultural studies and media studies, textual analysis is a key component of research. Researchers in these fields take media and cultural objects – for example, music videos, social media content, billboard advertising – and treat them as texts to be analyzed.

Usually working within a particular theoretical framework (for example, using postcolonial theory, media theory, or semiotics), researchers seek to connect elements of their texts with issues in contemporary politics and culture. They might analyze many different aspects of the text:

• Word choice
• Design elements
• Location of the text
• Target audience
• Relationship with other texts

Textual analysis in this context is usually creative and qualitative in its approach. Researchers seek to illuminate something about the underlying politics or social context of the cultural object they’re investigating.

In the social sciences, textual analysis is often applied to texts such as interview transcripts and surveys , as well as to various types of media. Social scientists use textual data to draw empirical conclusions about social relations.

Textual analysis in the social sciences sometimes takes a more quantitative approach , where the features of texts are measured numerically. For example, a researcher might investigate how often certain words are repeated in social media posts, or which colors appear most prominently in advertisements for products targeted at different demographics.

Some common methods of analyzing texts in the social sciences include content analysis , thematic analysis , and discourse analysis .

Textual analysis is the most important method in literary studies. Almost all work in this field involves in-depth analysis of texts – in this context, usually novels, poems, stories or plays.

Because it deals with literary writing, this type of textual analysis places greater emphasis on the deliberately constructed elements of a text: for example, rhyme and meter in a poem, or narrative perspective in a novel. Researchers aim to understand and explain how these elements contribute to the text’s meaning.

However, literary analysis doesn’t just involve discovering the author’s intended meaning. It often also explores potentially unintended connections between different texts, asks what a text reveals about the context in which it was written, or seeks to analyze a classic text in a new and unexpected way.

Some well-known examples of literary analysis show the variety of approaches that can be taken:

• Eve Kosofky Sedgwick’s book Between Men analyzes Victorian literature in light of more contemporary perspectives on gender and sexuality.
• Roland Barthes’ S/Z provides an in-depth structural analysis of a short story by Balzac.
• Harold Bloom’s The Anxiety of Influence applies his own “influence theory” to an analysis of various classic poets.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Thematic analysis
• Cohort study
• Peer review
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Caulfield, J. (2023, June 22). Textual Analysis | Guide, 3 Approaches & Examples. Scribbr. Retrieved June 24, 2024, from https://www.scribbr.com/methodology/textual-analysis/

Is this article helpful?

Jack Caulfield

Other students also liked, what is qualitative research | methods & examples, critical discourse analysis | definition, guide & examples, how to do thematic analysis | step-by-step guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”