reasons for reading research articles

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Reading Research Effectively
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Reading a Scholarly Article or Research Paper

Identifying a research problem to investigate usually requires a preliminary search for and critical review of the literature in order to gain an understanding about how scholars have examined a topic. Scholars rarely structure research studies in a way that can be followed like a story; they are complex and detail-intensive and often written in a descriptive and conclusive narrative form. However, in the social and behavioral sciences, journal articles and stand-alone research reports are generally organized in a consistent format that makes it easier to compare and contrast studies and to interpret their contents.

General Reading Strategies

W hen you first read an article or research paper, focus on asking specific questions about each section. This strategy can help with overall comprehension and with understanding how the content relates [or does not relate] to the problem you want to investigate. As you review more and more studies, the process of understanding and critically evaluating the research will become easier because the content of what you review will begin to coalescence around common themes and patterns of analysis. Below are recommendations on how to read each section of a research paper effectively. Note that the sections to read are out of order from how you will find them organized in a journal article or research paper.

1.  Abstract

The abstract summarizes the background, methods, results, discussion, and conclusions of a scholarly article or research paper. Use the abstract to filter out sources that may have appeared useful when you began searching for information but, in reality, are not relevant. Questions to consider when reading the abstract are:

• Is this study related to my question or area of research?
• What is this study about and why is it being done ?
• What is the working hypothesis or underlying thesis?
• What is the primary finding of the study?
• Are there words or terminology that I can use to either narrow or broaden the parameters of my search for more information?

2.  Introduction

If, after reading the abstract, you believe the paper may be useful, focus on examining the research problem and identifying the questions the author is trying to address. This information is usually located within the first few paragraphs of the introduction or in the concluding paragraph. Look for information about how and in what way this relates to what you are investigating. In addition to the research problem, the introduction should provide the main argument and theoretical framework of the study and, in the last paragraphs of the introduction, describe what the author(s) intend to accomplish. Questions to consider when reading the introduction include:

• What is this study trying to prove or disprove?
• What is the author(s) trying to test or demonstrate?
• What do we already know about this topic and what gaps does this study try to fill or contribute a new understanding to the research problem?
• Why should I care about what is being investigated?
• Will this study tell me anything new related to the research problem I am investigating?

3.  Literature Review

The literature review describes and critically evaluates what is already known about a topic. Read the literature review to obtain a big picture perspective about how the topic has been studied and to begin the process of seeing where your potential study fits within the domain of prior research. Questions to consider when reading the literature review include:

• W hat other research has been conducted about this topic and what are the main themes that have emerged?
• What does prior research reveal about what is already known about the topic and what remains to be discovered?
• What have been the most important past findings about the research problem?
• How has prior research led the author(s) to conduct this particular study?
• Is there any prior research that is unique or groundbreaking?
• Are there any studies I could use as a model for designing and organizing my own study?

4.  Discussion/Conclusion

The discussion and conclusion are usually the last two sections of text in a scholarly article or research report. They reveal how the author(s) interpreted the findings of their research and presented recommendations or courses of action based on those findings. Often in the conclusion, the author(s) highlight recommendations for further research that can be used to develop your own study. Questions to consider when reading the discussion and conclusion sections include:

• What is the overall meaning of the study and why is this important? [i.e., how have the author(s) addressed the " So What? " question].
• What do you find to be the most important ways that the findings have been interpreted?
• What are the weaknesses in their argument?
• Do you believe conclusions about the significance of the study and its findings are valid?
• What limitations of the study do the author(s) describe and how might this help formulate my own research?
• Does the conclusion contain any recommendations for future research?

5.  Methods/Methodology

The methods section describes the materials, techniques, and procedures for gathering information used to examine the research problem. If what you have read so far closely supports your understanding of the topic, then move on to examining how the author(s) gathered information during the research process. Questions to consider when reading the methods section include:

• Did the study use qualitative [based on interviews, observations, content analysis], quantitative [based on statistical analysis], or a mixed-methods approach to examining the research problem?
• What was the type of information or data used?
• Could this method of analysis be repeated and can I adopt the same approach?
• Is enough information available to repeat the study or should new data be found to expand or improve understanding of the research problem?

6.  Results

After reading the above sections, you should have a clear understanding of the general findings of the study. Therefore, read the results section to identify how key findings were discussed in relation to the research problem. If any non-textual elements [e.g., graphs, charts, tables, etc.] are confusing, focus on the explanations about them in the text. Questions to consider when reading the results section include:

• W hat did the author(s) find and how did they find it?
• Does the author(s) highlight any findings as most significant?
• Are the results presented in a factual and unbiased way?
• Does the analysis of results in the discussion section agree with how the results are presented?
• Is all the data present and did the author(s) adequately address gaps?
• What conclusions do you formulate from this data and does it match with the author's conclusions?

7.  References

The references list the sources used by the author(s) to document what prior research and information was used when conducting the study. After reviewing the article or research paper, use the references to identify additional sources of information on the topic and to examine critically how these sources supported the overall research agenda. Questions to consider when reading the references include:

• Do the sources cited by the author(s) reflect a diversity of disciplinary viewpoints, i.e., are the sources all from a particular field of study or do the sources reflect multiple areas of study?
• Are there any unique or interesting sources that could be incorporated into my study?
• What other authors are respected in this field, i.e., who has multiple works cited or is cited most often by others?
• What other research should I review to clarify any remaining issues or that I need more information about?

NOTE :  A final strategy in reviewing research is to copy and paste the title of the source [journal article, book, research report] into Google Scholar . If it appears, look for a "cited by" followed by a hyperlinked number [e.g., Cited by 45]. This number indicates how many times the study has been subsequently cited in other, more recently published works. This strategy, known as citation tracking, can be an effective means of expanding your review of pertinent literature based on a study you have found useful and how scholars have cited it. The same strategies described above can be applied to reading articles you find in the list of cited by references.

Reading Tip

Specific Reading Strategies

Effectively reading scholarly research is an acquired skill that involves attention to detail and an ability to comprehend complex ideas, data, and theoretical concepts in a way that applies logically to the research problem you are investigating. Here are some specific reading strategies to consider.

As You are Reading

• Focus on information that is most relevant to the research problem; skim over the other parts.
• As noted above, read content out of order! This isn't a novel; you want to start with the spoiler to quickly assess the relevance of the study.
• Think critically about what you read and seek to build your own arguments; not everything may be entirely valid, examined effectively, or thoroughly investigated.
• Look up the definitions of unfamiliar words, concepts, or terminology. A good scholarly source is Credo Reference .

Taking notes as you read will save time when you go back to examine your sources. Here are some suggestions:

• Mark or highlight important text as you read [e.g., you can use the highlight text  feature in a PDF document]
• Take notes in the margins [e.g., Adobe Reader offers pop-up sticky notes].
• Highlight important quotations; consider using different colors to differentiate between quotes and other types of important text.
• Summarize key points about the study at the end of the paper. To save time, these can be in the form of a concise bulleted list of statements [e.g., intro has provides historical background; lit review has important sources; good conclusions].

Write down thoughts that come to mind that may help clarify your understanding of the research problem. Here are some examples of questions to ask yourself:

• Do I understand all of the terminology and key concepts?
• Do I understand the parts of this study most relevant to my topic?
• What specific problem does the research address and why is it important?
• Are there any issues or perspectives the author(s) did not consider?
• Do I have any reason to question the validity or reliability of this research?
• How do the findings relate to my research interests and to other works which I have read?

Adapted from text originally created by Holly Burt, Behavioral Sciences Librarian, USC Libraries, April 2018.

Another Reading Tip

When is it Important to Read the Entire Article or Research Paper

Laubepin argues, "Very few articles in a field are so important that every word needs to be read carefully." However, this implies that some studies are worth reading carefully. As painful and time-consuming as it may seem, there are valid reasons for reading a study in its entirety from beginning to end. Here are some examples:

• Studies Published Very Recently .  The author(s) of a recent, well written study will provide a survey of the most important or impactful prior research in the literature review section. This can establish an understanding of how scholars in the past addressed the research problem. In addition, the most recently published sources will highlight what is currently known and what gaps in understanding currently exist about a topic, usually in the form of the need for further research in the conclusion .
• Surveys of the Research Problem .  Some papers provide a comprehensive analytical overview of the research problem. Reading this type of study can help you understand underlying issues and discover why scholars have chosen to investigate the topic. This is particularly important if the study was published very recently because the author(s) should cite all or most of the key prior research on the topic. Note that, if it is a long-standing problem, there may be studies that specifically review the literature to identify gaps that remain. These studies often include the word review in their title [e.g., Hügel, Stephan, and Anna R. Davies. "Public Participation, Engagement, and Climate Change Adaptation: A Review of the Research Literature." Wiley Interdisciplinary Reviews: Climate Change 11 (July-August 2020): https://doi.org/10.1002/ wcc.645].
• Highly Cited .  If you keep coming across the same citation to a study while you are reviewing the literature, this implies it was foundational in establishing an understanding of the research problem or the study had a significant impact within the literature [positive or negative]. Carefully reading a highly cited source can help you understand how the topic emerged and motivated scholars to further investigate the problem. It also could be a study you need to cite as foundational in your own paper to demonstrate to the reader that you understand the roots of the problem.
• Historical Overview .  Knowing the historical background of a research problem may not be the focus of your analysis. Nevertheless, carefully reading a study that provides a thorough description and analysis of the history behind an event, issue, or phenomenon can add important context to understanding the topic and what aspect of the problem you may want to examine further.
• Innovative Methodological Design .  Some studies are significant and worth reading in their entirety because the author(s) designed a unique or innovative approach to researching the problem. This may justify reading the entire study because it can motivate you to think creatively about pursuing an alternative or non-traditional approach to examining your topic of interest. These types of studies are generally easy to identify because they are often cited in others works because of their unique approach to studying the research problem.
• Cross-disciplinary Approach .  R eviewing studies produced outside of your discipline is an essential component of investigating research problems in the social and behavioral sciences. Consider reading a study that was conducted by author(s) based in a different discipline [e.g., an anthropologist studying political cultures; a study of hiring practices in companies published in a sociology journal]. This approach can generate a new understanding or a unique perspective about the topic . If you are not sure how to search for studies published in a discipline outside of your major or of the course you are taking, contact a librarian for assistance.

Laubepin, Frederique. How to Read (and Understand) a Social Science Journal Article . Inter-University Consortium for Political and Social Research (ISPSR), 2013; Shon, Phillip Chong Ho. How to Read Journal Articles in the Social Sciences: A Very Practical Guide for Students . 2nd edition. Thousand Oaks, CA: Sage, 2015; Lockhart, Tara, and Mary Soliday. "The Critical Place of Reading in Writing Transfer (and Beyond): A Report of Student Experiences." Pedagogy 16 (2016): 23-37; Maguire, Moira, Ann Everitt Reynolds, and Brid Delahunt. "Reading to Be: The Role of Academic Reading in Emergent Academic and Professional Student Identities." Journal of University Teaching and Learning Practice 17 (2020): 5-12.

• << Previous: 1. Choosing a Research Problem
• Next: Narrowing a Topic Idea >>
• Last Updated: Apr 22, 2024 9:12 AM
• URL: https://libguides.usc.edu/writingguide

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

How, and why, science and health researchers read scientific (IMRAD) papers

Roles Conceptualization, Formal analysis, Funding acquisition, Methodology, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Trials Research and Methodologies Unit, HRB Clinical Research Facility, University College Cork, Cork, Ireland, School of Public Health, University College Cork, Cork, Ireland

Roles Data curation, Formal analysis, Investigation, Writing – original draft, Writing – review & editing

Roles Formal analysis, Methodology, Software, Validation, Writing – original draft, Writing – review & editing

Affiliation School of Public Health, University College Cork, Cork, Ireland

• Frances Shiely, 
• Kerrie Gallagher, 
• Seán R. Millar

• Published: January 22, 2024
• https://doi.org/10.1371/journal.pone.0297034
• Reader Comments

The purpose of our study was to determine the order in which science and health researchers read scientific papers, their reasons for doing so and the perceived difficulty and perceived importance of each section.

Study design and setting

An online survey open to science and health academics and researchers distributed via existing research networks, X (formerly Twitter), and LinkedIn.

Almost 90% of respondents self-declared to be experienced in reading research papers. 98.6% of the sample read the abstract first because it provides an overview of the paper and facilitates a decision on continuing to read on or not. Seventy-five percent perceived it to be the easiest to read and 62.4% perceived it to be very important (highest rank on a 5-point Likert scale). The majority of respondents did not read a paper in the IMRAD (Introduction, Methods, Results And Discussion) format. Perceived difficulty and perceived importance influenced reading order.

Science and health researchers do not typically read scientific and health research papers in IMRAD format. The more important a respondent perceives a section to be, the more likely they are to read it. The easier a section is perceived, the more likely it will be read. We present recommendations to those teaching the skill of writing scientific papers and reports.

Citation: Shiely F, Gallagher K, Millar SR (2024) How, and why, science and health researchers read scientific (IMRAD) papers. PLoS ONE 19(1): e0297034. https://doi.org/10.1371/journal.pone.0297034

Editor: Bogdan Nadolu, West University of Timisoara: Universitatea de Vest din Timisoara, ROMANIA

Received: February 21, 2023; Accepted: December 26, 2023; Published: January 22, 2024

Copyright: © 2024 Shiely et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The datasets generated and/or analysed during the current study are available https://osf.io/up4ny/ .

Funding: This study was funded by Teaching and Learning Enhancement Fund at University College Cork for a summer student scholarship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors declared that no competing interests exist.

Introduction

Reporting in the form of a peer-reviewed research paper, also known as a journal publication or research manuscript, is essential to the healthcare and science professions. The skill of writing a peer reviewed paper is highly specialized and challenging. It is also a challenge to teach this skill, yet it is essential to do so, as students are often required to engage with complex academic texts as well as write scientific reports [ 1 – 3 ]. Other cited reasons are: (i) increasing scientific literacy; (ii) staying informed of progress in a particular field of study; (iii) understanding the causation, clinical features, and natural history of a disease; (iv) evaluating the effectiveness of diagnostic tests and clinical therapies; and (v) determining whether there is support for or opposition to a particular argument [ 4 – 6 ]. Additionally, it is imperative that the reader is able to identify robustly designed research in order to make informed recommendations regarding policy or patient care [ 7 ].

Currently, most healthcare research papers are presented in the IMRAD format: I ntroduction (why the authors decided to do the research), M ethods (how they did it and how they chose to analyse their results), R esults (what they found), A nd D iscussion (what they believe the results to mean) with a preceding abstract. However, there is no evidence-based research determining the suitability of this approach. Nevertheless, it provides a means for scientific communities to organise and structure their work effectively [ 8 ]. Within the scientific community, the approach is based on the notion that having a clear structure and procedures can help scientists produce better quality work. In addition, it is thought to reduce the risk of mistakes and oversights and ensure compliance with best practices in research (10). We know that the amount of time students spend reading academic material is estimated to be between seven and fourteen hours per week, which represents an important component of the academic schedule [ 3 , 9 , 10 ]. Therefore, research on the suitability of the IMRAD approach is important.

Professor Trisha Greenhalgh, author of the seminal text “How to read a paper: the basics of evidence-based medicine and healthcare” suggests that if you are deciding whether a paper is worthy of study, you should do so based on the design of the methods section [ 11 ]. This is largely opinion based and is not predicated on clear evidence. Anecdotal evidence suggests people choose to read and examine research papers in different ways, but the literature is scant on the topic. One UK study attempts to address the strategies used by researchers and students when reading primary research [ 12 ]. The authors report that individuals at different career stages value different sections of scientific papers, with novice readers finding the methods and results sections to be particularly challenging to decipher [ 12 ]. Similarly, a study conducted in the US examined and compared how faculty members and students in an undergraduate science course engaged with a primary research article [ 13 ]. Faculty and students were able to demonstrate understanding of the research design at some point during the reading process, however, the faculty displayed this ability almost four times as often as students [ 13 ]. Both of these studies are limited in their capacity and generalisability as they are restricted to students and researchers in the biological sciences.

From a teaching and learning perspective, we are interested in knowing more about how science and health researchers read IMRAD research papers and the importance they place on each section. Our primary aim is to establish the order in which these researchers read an IMRAD formatted paper and why. By establishing this, educators can better craft their teaching to ensure that students understand the importance of each section, have the knowledge and skills necessary to write an effective scientific paper or report, and the ability to critically appraise the work of others.

Materials and methods

The survey ( S1 File ) was created on Google Forms by two members of the research team (KG and FS) and independently reviewed by two reviewers (ST and EM). The survey had three parts. Part 1 was concerned with written informed consent. When participants clicked the link, they were brought the informed consent page which provided details of the study, what was required from them, knowledge of the voluntary nature of the participation and right to withdraw at any stage, and the contact details of the principal investigator. To proceed, participants had to select either I consent to participate, which brought the participants to Part 2 of the survey, or I do not consent to participate, which meant the participants exited the survey. Part 2 of the survey collected data concerning the demographic characteristics of the respondents. Part 3 focused on questions pertaining to the order in which researchers read a primary research paper, how easy it is to read each of the sections (based on a 7-point scale) and how important each section of a primary research paper is for its understanding. The survey also assessed when a reader stops reading and why. The style of questions was mixed and included Likert scale ratings and closed and open-ended questions. Ethical approval was granted by the Social Research Ethics Committee (SREC), University College Cork (Log 2021–165). Participants provided written informed consent.

Recruitment

This was an online survey and recruitment was online. Inclusion criteria were: academics, health professionals, and patients and members of the public, involved in science or health research and/or teaching. Exclusion criterion was: under 18 years of age. Our recruitment strategy was to target academics, researchers and patient and public involvement members of our existing networks, all who work within or are affiliated with Universities in the UK, Ireland and Canada. The lead author, FS, is primarily associated with clinical trial networks. An email of invitation outlining the aim of the study and survey link was sent electronically via the Health Research Trial Methodology Research Network (HRB TMRN), Ireland (~3000 subscribers), Medial Research Council-National Institutes of Health and Care Research-Trial Methodology Research Partnership (MRC-NIHR-TMRP), UK, locally at University, University College Cork (all academic and research staff—~2500 people), via X, formerly known as Twitter (@FrancesShiely; @hrbtmrn) which was forwarded and liked and via LinkedIn (FS account). FS also distributed the link to her academic research partners in Ireland, UK, Hungary, Czech Republic, France, and Canada and asked them to forward to their respective Universities and contacts.

Statistical analysis

We obtained 152 responses to the survey, 139 of which completed answers to the order in which they read the research paper. These were included in the analyses. Descriptive characteristics were examined for the full sample. Likert scale answers to reading order, perceived difficulty and perceived importance questions for each research paper section are shown as percentages. Reading order, perceived difficulty and importance ranking were also examined according to career stage. Observations were independent, with no individuals belonging to more than one career stage group. Relationships between perceived difficulty ranking, perceived importance ranking and research paper reading order were also examined using Spearman’s rank-order correlation. Data analysis was conducted using Stata SE Version 13 (Stata Corporation, College Station, TX, USA) for Windows. For all analyses, a p value (two-tailed) of less than .05 was considered to indicate statistical significance. Qualitative variables were summarised according to the most frequent occurrence to provide a picture on the reasons participants chose to read a paper in their chosen format.

Table 1 shows descriptive characteristics of the study respondents. The majority of subjects were female (61.2%), 90.7% were under 60 years of age and 94.7% reside in Europe. Study respondents included MSc and PhD students (n = 17), early-career researchers (n = 39), mid-career researchers (n = 36) and established/leading researchers and research managers (n = 39). A majority (88.5%) worked in academic research at a university or college, with 61.9% indicating both research and teaching responsibilities. Almost 90% of respondents stated that they were experienced in reading a research paper.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0297034.t001

Reading order, perceived difficulty and perceived importance for IRMAD sections

Fig 1 shows research paper reading order according to each section. A majority of respondents (98.6%) indicated that they read the abstract section first when reading a scientific paper, with 36.0% (Introduction), 29.5% (Methods), 36.0% (Results–text), 31.7% (Results–figures & tables), 43.9% (Discussion) and 36.7% (Conclusion) of subjects stating that they read these sections second, third, fourth, fifth, sixth, and last, respectively. Noticeably, a majority of respondents indicated that they did not read a paper in the IMRAD order; for instance, while over one-third of respondents stated that they read the Introduction section second, 64.0% did not, with just over one-fifth (20.9%) indicating that they read this section last.

https://doi.org/10.1371/journal.pone.0297034.g001

We asked respondents why they read in their preferred order. For the 98.6% (149/152) who selected the abstract first, the reasons can be summarised as the fact the abstract gives an overview or summary of the paper and it allows one to see if the paper is worth continuing reading (“the abstract gives the summary and informs as to whether I will read the whole paper”, “abstract gives a feel for quality of the paper”, “fast and easy”, “the abstract has the main points and is usually freely available”). We were interested in what respondents read after the abstract. For those who read the introduction second, i.e., the IMRAD format (only two read the introduction first), the dominant reason was because it’s logical to read in the order it’s presented (“it’s the logical order”, “I usually read papers in the order it is written”). For those who chose the methods second, 21.2% (31/146) the reasons can be themed as to ensure robustness or quality of the study (“methods to understand whether it was well conducted”, “checking the methods to ensure it is relevant to me”, “understand how the methods led to such results”, “is this something I can trust”). Only 15 people read the results section second, regardless of whether it was the results-text or results-figures & tables. The key reasons for reading the results second can be summarised as establishing the findings (“get right to the results”, “results to understand the main findings”, “the results are arguably the most important part of the document”, “do the results show what they are saying?”). Those choosing to read the discussion second, 8.1% (12/149), did so to establish the key findings (“the discussion and conclusion is the essence of what the study found”, “discussion and conclusion are most interesting”, “discussion to see if anything interesting came out of the results”). For those who read the conclusion second, 17% (25/146), the reasons are summarised as establishing the overall view of the paper and if the research is of value “see if a paper of value”, “know if it’s useful to me”, “final result/outcome”, “overall view”, “clarifies what the author perceives to have been achieved”, “I read the conclusions to build on the summary conclusions of the abstract”.

To explore perceived difficulty when reading a research paper, participants were asked to rank a series of questions according to reading difficulty on a 7-point scale, 1 being the easiest and 7 being the most difficult ( Fig 2 ). Similar to reading order, a majority (75.0%) of respondents stated that they found the Abstract section to be easiest to read (rank 1). The Introduction and Conclusion sections were perceived as next easiest, respectively. Taking the blue and orange together (ranks 1 and 2) the same trend applies. On the opposite end of the scale (rank 7-most difficult, the dark navy colour), the Results-figures & tables section, was perceived to be most difficult (26.3%), followed by the Methods (25.6%) and Results-text (17.3%).

https://doi.org/10.1371/journal.pone.0297034.g002

Perceived importance for each section was assessed using a 5-point Likert scale ( Fig 3 ). The Abstract and Methods sections were perceived as very important by 62.4% and 58.8% of respondents, respectively. Although few respondents perceived any section as unimportant or very unimportant, only 29.5%, 31.8% and 32.6% of subjects believed that the Introduction, Discussion and Conclusion sections were very important.

https://doi.org/10.1371/journal.pone.0297034.g003

Reading order and career stage

Fig 4 shows the different sections of the research paper in reading order according to career stage. Differences in reading order were noted, with the greatest differences in reading order observed in the Results-text, Results-figures & tables and Discussion sections. Notably, 46.2% of established/leading researchers or research managers read the Results-text section fourth (in IMRAD order), compared to 29.4% of MSc by research/PhD students and 28.2% of mid-career researchers who did so. Similarly, a higher percentage of established/leading researchers or research managers indicated reading the Results-figures & tables and Discussion sections according to IMRAD reading order when compared to other career stages.

https://doi.org/10.1371/journal.pone.0297034.g004

Perceived difficulty and importance for each section according to career stage

Perceived difficulty ranking and perceived importance ranking according to career stage, for each IMRAD section and the abstract, are shown in Figs 5 and 6 . Consistent with results observed among all subjects, regardless of career stage, the Results-text and Results-figures & tables sections and Discussion sections were perceived as most difficult to read. Differences were found to be greatest for the Conclusion section, with MSc by research or PhD students being more likely to rank this section as difficult to read. With regard to the Introduction section, mid-career researchers were more likely to rank this section as important. Interestingly, MSc by research/PhD students were more likely to rank the Methods section as being important when compared to other career stages.

https://doi.org/10.1371/journal.pone.0297034.g005

https://doi.org/10.1371/journal.pone.0297034.g006

Correlations between perceived difficulty, importance and reading order

Spearman correlation coefficients between the ranking of perceived difficulty, perceived importance and reading order, according to each section, are shown in Table 2 . Significant correlations between perceived difficulty ranking and reading order were observed for the Methods (rho = 0.450, p < .001), Results-figures & tables (rho = 0.333, p < .001), Discussion (rho = 0.204, p = .018) and Conclusion (rho = 0.334, p < .001) sections, indicating that the easier a respondent perceived that section to read, the more likely they were to read it at an earlier stage. Significant correlations between perceived importance ranking and reading order were observed for the Introduction (rho = 0.467, p < .001), Methods (rho = 0.426, p < .001), Results (text) (rho = 0.250, p = .003), Results (figures & tables) (rho = 0.173, p = .048), Discussion (rho = 0.214, p = 0.14) and Conclusion (rho = 0.302, p < .001) sections, suggesting that the more important a respondent perceived that section to be, the more likely they were to read it at an earlier stage.

https://doi.org/10.1371/journal.pone.0297034.t002

When and why respondents stop reading a paper

We asked respondents why they stopped reading a research paper. The main sections were (in no particular order) results, introduction, methods and abstract. Of the 98.6% that read the abstract first, 28% (42/149) then stopped reading at this stage. The reason given in all cases is lack of relevance (“not relevant to my interests”, “will have identified if it is of relevance”). The main reasons for those that stop reading at the introduction is the writing style (“poorly written”, if the writing style is overly complex”, it’s too dense and not interesting”, not relevant, poorly conducted”. For those who stop reading at the results section, the main reasons given are the results are too complex or poorly explained (“it gets too difficult to understand”, “paper is not relevant”, “too complicated”, “it’s no longer relevant if results are not clear”). For those who stop reading at the methods section, the main reason is they are unclear or too difficult or there is a perception that the methods are not needed (“becomes technical and I don’t’ need more details”, the methods might not be interesting to what I am trying to learn from reading the paper”, generally for my work methods are not important”, not interested in it, I Know already by the methods if I ‘like’ the paper”, “most often the methodology is not clear enough”).

We know most research papers are published in IMRAD format, preceded by an abstract. We sought to establish if researchers, and at different career stages, typically read a paper in this way. We found that even though most researchers consider themselves experienced readers of primary research papers, respondents did not typically engage with the literature in IMRAD format. Reading strategies varied depending on perceived difficulty and perceived importance of the paper sections. The more important a respondent perceived the section to be, the more likely they were to read it at an earlier stage. Almost all science and health researchers read the abstract first, and a significant proportion stop reading there. The primary reason for stopping is lack of relevance.

We can see very clearly that the abstract is read first by most researchers, regardless of career stage, perceived to be the most important, and also perceived to be the easiest to read. While there isn’t prior research to compare this finding to, we surmise that it is because it is a summary of the overall paper, and a logical place to begin. It’s also possible that it is because it is presented first in all research papers, or for pay per view journals/papers it is usually available when the rest of a paper is beyond a paywall. It could be that researchers know the abstract is used by journal editors to decide if it is worth continuing to read a paper to decide if it should be peer-reviewed or it could be that the abstract is often used in systematic reviews when screening (typically title and abstract) and thus habitually researchers read it first. However, we don’t know any of this for sure and would need a qualitative study to confirm or refute these. We do know the reasons that a significant number of people stop reading the abstract and that is relevance, or the lack of it. Another consideration might be the reading level at which the abstract is pitched. We have conducted a previous study on the readability of trial lay summaries, which are written for a lay audience [ 14 ]. We found that no lay summary met the recommended reading age for health care information of 11–12 years. None of them were considered "easy" to read, in fact over 85% were considered "difficult" to read [ 14 ]. By extension we can assume the scientific abstract, which we have considered here, will be no better, and likely worse, and thus challenging for a novice researcher. We recommend that when teaching the skill of writing a scientific paper, or report, in the science or health research disciplines, teachers emphasise the importance of the abstract and give consideration to the target audience when formulating their approach. Online readability tools are readily available to assist the process, but we do not recommend relying on them solely [ 14 ].

Further evidence of the complexity of reading different parts of scientific papers is the response to the methods section. The evidence that does exist on how to read a paper suggests that if you are deciding whether a paper is worthy of study, you should do so based on the design of the methods section. As mentioned previously, this was opinion based rather than evidence based [ 3 ]. Our respondents typically considered the methods section to be of low importance on the 7-point Likert Scale and some stopped reading there due to the unclear language or technical nature of the section. This result was unexpected, as there is a general consensus in the scientific community that the methods section is considered one of the most important sections of any research paper, given that it provides essential insight into the conduct of the study and its integrity, the conclusions derived from them, and the reproducibility of the work [ 3 ]. Indeed, across several well recognized and validated critical appraisal tools, including the Critical Appraisal Skills Program (CASP) for Randomized Control Trials [ 15 ], the ROB 2.0 Risk of Bias Tool [ 16 ] and the ROBINS-I Risk of Bias for non-randomized (observational) studies [ 17 ], focus is directed towards systematically examining the methods section in order for the reader to determine the strength of the evidence presented, its reliability and relevance to clinical practice.

Strengths and weaknesses

We had a reasonable response (n = 139) but we are unable to calculate a response rate due to the mode of distribution, a significant weakness to the study. Our study likely demonstrates selection bias, given the known research networks through which the survey was distributed are all funded through academic grants and the majority of our respondents were academic researchers working in a University/College. The research would be enhanced if we had a larger response from non-profit organisations. There were only six respondents outside of Europe and this is a weakness in terms of generalisability. However, on the positive side, non-response bias was not evident, and we had a full dataset for 139 respondents.

Recommendations

The lessons for future practice are:

• Ensure your abstract gives enough detail to ensure relevance and pique interest because if you don’t, science and health researchers will lose interest and stop reading;
• Ensure the introduction is well written because if it is poorly written, you will lose the reader (we suggest using the freely available online readability scales, e.g., Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simplified Measure of Gobbledegook (SMOG), Gunning Fog (GF), Coleman-Liau Index (CLI), and Automated Readability Index (ARI) readability scales and paying attention to Plain Language guidelines [ 18 ], e.g., in the UK the Plain English UK guidelines are most relevant;
• Don’t make the methods section too technical. Find the balance between overcomplicating the methods and giving enough detail so the study can be replicated;
• Keep the results simple and explain them well.

Conclusions

This study provides an insight into the order in which IMRAD papers are read and the reasons for doing so. Existing evidence says that to determine if a paper is worthy of reading, you should read the methods section to decide. Our results refute this and show the methods section to be one of the sections perceived most difficult to read and also the least important. Our results show the importance of the abstract to the scientific and health research community, and we recommend when teaching the skill of scientific writing a particular focus is given to the abstract. Future research on this topic is welcome in a more diverse and larger sample.

Supporting information

S1 file. undergraduate student survey..

https://doi.org/10.1371/journal.pone.0297034.s001

Acknowledgments

We would like to thank those who took the time to fill in our questionnaire.

• View Article
• PubMed/NCBI
• Google Scholar

Beyond the Passive Absorption of Information: Engaging Students in the Critical Reading of Scientific Articles

• Open access
• Published: 07 March 2024

Cite this article

You have full access to this open access article

• Pablo Antonio Archila   ORCID: orcid.org/0000-0003-0225-4701 1 ,
• Brigithe Tatiana Ortiz 2 &
• Anne-Marie Truscott de Mejía 3  

617 Accesses

2 Altmetric

Explore all metrics

There is a consensus within the science education community that primary scientific literature is a legitimate and desirable educational resource. Moreover, critical reading of scientific articles is widely recognized as a key aspect of scientific literacy. However, university science courses rarely provide students with explicit opportunities to cultivate their critical reading skills. Much of the reason for this is that instructors tend to hold a passive learning view of reading in which students are expected to absorb information from scientific articles. The purpose of this study was to provide research evidence that an active learning scenario (ALS) combining (1) argumentation, (2) peer critique (also referred to as peer assessment), and (3) the Task-Oriented Reading Instruction framework (Ritchey & List, College Teaching, 70 (3), 280–295, 2022 ) could be a concrete and realistic possibility for engaging students in the critical reading of scientific papers. The data analyzed in this study were the written critiques of scientific research articles and written peer feedback produced by sixty-one university students (38 females and 23 males, 19–25 years old). The results indicate that the ALS effectively offered students explicit opportunities to become more active and more critical readers of scientific articles, producing arguments, anticipating counterarguments, and constructing rebuttals. Implications related to critical reading instruction in science education and supporting students’ development of critical reading skills are discussed.

Similar content being viewed by others

Online learning in higher education: exploring advantages and disadvantages for engagement

Amber D. Dumford & Angie L. Miller

Using technology to make learning fun: technology use is best made fun and challenging to optimize intrinsic motivation and engagement

Loukia David & Netta Weinstein

Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process

Alex Barrett & Austin Pack

Avoid common mistakes on your manuscript.

1 Introduction

The preparation of critical readers is a crucial but neglected aspect of higher education institutions. This is a common reflection made by the authors of the twelve chapters of Reading Across the Disciplines (Manarin, 2022 ). Critical reading is an umbrella term which means different things to different people and in different contexts. In the case of academic settings, becoming a critical reader implies developing skills about the identification of patterns of textual elements, the distinction between main and subordinate ideas, the evaluation of credibility, judgement about the argumentation of a text, and production of relevant inferences (Manarin et al., 2015 ). This definition gives us an idea of how challenging and complex the goal of preparing critical readers is. Unfortunately, there is a lack of training among faculty about how to nurture critical reading skills (Sutherland & Incera, 2021 ). One consequence of this is that instructors limit their didactic action to (1) giving reading assignments and (2) expecting students to read independently rather than acting as reading mentors (Hubbard, 2021 ; Hubbard et al., 2022 ). Hence, it is exceedingly difficult, if not impossible, for students to enrich their critical reading skills when they do not receive real guidance and support.

In the field of science education, there is a notable consensus that primary scientific literature—reports of novel observations, theories, or opinions, communicated in writing for peers in the scientific community (Van Lacum et al., 2014 )—is a legitimate and desirable educational resource that contributes to authentic scientific literacy. Griffiths and Davila ( 2022 ) reiterate that “critical reading of scientific texts is a key practice in science, as the primary literature constructs knowledge and communicates research in the field” (p. 143). Moreover, these scholars emphasize the need to create research-based teaching and learning strategies that help university students to become engaged in the critical reading of scientific articles (also referred to as scientific research articles, or scientific papers). Naturally, such creation entails moving away from the passive learning view of reading in which students are expected to absorb information towards more active learning practices that provide them with explicit opportunities to go beyond a superficial reading of scientific articles to critically engage with these readings.

It may be obvious to point out that critical reading of scientific papers needs argumentative reasoning. Nevertheless, “university science courses give undergraduates limited opportunities to strengthen their argumentation skills” (Archila et al., 2023a , p. 635). Previous attempts to cultivate critical reading of scientific research articles (e.g., Lammers et al., 2019 ; Van Lacum et al., 2014 , 2016 ; Wijayanti & Adi, 2022 ) have focused on giving students explicit opportunities to identify components of the authors’ argumentation (e.g., supports, counterarguments, refutations). In the current study, we explore an under-researched possibility: to engage undergraduates in the critical reading of scientific articles through the construction of arguments, counterarguments, and rebuttals. Additionally, we maintain that this possibility is a rational and reasonable scenario to involve students in peer assessment (also known as peer critique)—peer grading and providing useful feedback (Winstone & Carless, 2020 )—while critiquing the argumentation of other equal-status students.

Recently, Ritchey and List ( 2022 ) proposed a theoretically based framework called Task-Oriented Reading Instruction (TORI). This framework was the bedrock of our study. Much of the reason for this is that TORI was created to explicitly provide goals for reading, to specify strategies to help students meet these goals, and to align assessment with assigned reading goals. Thus, this study was aimed at providing evidence for the claim that an active learning scenario (ALS) combining (1) argumentation, (2) peer critique, and (3) the TORI could be a concrete and realistic possibility for engaging students in the critical reading of scientific research articles. With this in mind, we created an ALS. The central question addressed in this article is as follows: To what extent does the ALS engage science (Biology and Microbiology) students and engineering (Biomedical engineering, Chemical Engineering, and Food Engineering) students in the critical reading of scientific papers assigned in a Food Microbiology course?

2 Literature Review

Scientific reading is a fundamental component in university science education. Nonetheless, many students do not complete course readings. In a study conducted in the California State University (Gorzycki et al., 2019 ), 206 undergraduate open-ended comments about student priorities and beliefs about academic reading were analyzed. One key result was that although undergraduates tended to declare that reading is important, they do not read. Another outcome was that this situation seems to be exacerbated by deficient educational practices such as the lack of explicit opportunities for undergraduate students to cultivate their discrete academic reading skills (e.g., identifying implications, synthesizing sources, and critiquing assertions). Also, in California State University, Desa et al. ( 2020 ) analyzed the written comments about academic reading in the undergraduate experience of thirty-three instructors. Interestingly, these researchers found that even though faculty seemed to be convinced that academic reading is crucial for undergraduates’ success, they seldom integrated academic reading pedagogies into their practices. Moreover, these scholars concluded that there were contextual factors that reinforced this issue, namely, (1) faculty epistemologies, (2) faculty assumptions about learning in higher education (instructors tend to teach as they were taught), and (3) the institution’s cultural leanings. These factors undoubtedly evoke Kampourakis’s ( 2017 ) reflection that one fundamental cause of the issues in university science courses is the way “science teachers themselves were taught science in their undergraduate studies” (p. 202). Furthermore, he insists that nothing can change until higher education institutions become aware that instructors’ publications in prestigious journals is important but not their principal mission, which is teaching and learning.

In a recent survey study ( N  = 249), Hairston-Dotson and Incera ( 2022 ) discovered a paradox: although undergraduate students acknowledged that complex critical reading skills (e.g., applying) are more useful than simpler reading skills (e.g., skimming), they reported practicing skimming more often than applying when working on their assignments. These academics recommend that instructors, instead of merely asking students “to read” a text, should create time and space within the classroom so students have concrete and realistic opportunities to practice and develop complex critical reading skills while actively engaging with the text. It is noteworthy that in another study ( N  = 128), Sutherland and Incera ( 2021 ) sought to investigate the opinions about critical reading of faculty at Eastern Kentucky University. One outcome was that instructors considered more complex critical reading skills (e.g., applying) as more useful, while they considered simpler skills (e.g., skimming) less useful. Additionally, participants appeared to be more willing to invest more time of their classes to teach more complex critical reading skills than simpler ones. Arguably, it is this willingness that science educators can use as leverage to encourage and help science teachers take advantage of educational resources to promote critical reading in their courses.

There is empirical research that supports the claim that primary scientific literature is one legitimate and desirable educational resource than can be exploited in multiple ways, such as the use of this type of literature to introduce students to the nature of science. For example, Wenk and Tronsky ( 2011 ) maintain that research articles make explicit aspects of nature of science (e.g., collection, analysis, and interpretation of data) that many textbooks tend to ignore. Specifically, these scholars provide research evidence for the claim that introductory science classrooms can be transformed into educational spaces in which first year students find opportunities to strengthen their skills to read primary scientific literature and that doing so enriches their understanding of the process of science. In similar fashion, Schmid et al. ( 2021 ) explored the usefulness of an approach combining early exposure to this type of literature and interactions with scientists. The soundness of this approach was evidenced in the United States with 12 science undergraduate students from a research-intensive university who were enrolled in a seminar-style introduction to biological literature course. An interesting result of this study was that although explicit nature of science instruction was not part of this approach, these academics found that participants developed certain understandings about the nature of science. Of course, they acknowledged that explicit instruction would have benefited their approach.

Argumentation is intimately related to the nature of science since this skill is essential in the process of doing science as well as the communication in science (Lemke, 1990 ). Hence, Khishfe ( 2014 , 2023a , b ) highlights the relevance of improving science education practices by integrating argumentation skills and nature of science ideas. In this sense, the results of intervention studies conducted by Khishfe ( 2021 ( N  = 36), 2023b ( N  = 42)) with grade 10 students suggest that explicit instruction of argumentation connected with explicit instruction of the nature of science contributes to the promotion of students’ argumentation as well as the development of conceptions about the nature of science. As these scholars comment, one explanation for this is that argumentation and the nature of science are closely linked.

Additionally, it is important to recognize that in 2007, Hoskins and her colleagues developed the Consider, Read, Elucidate the hypotheses, Analyze and interpret the data, and Think of the next Experiment (CREATE) method. This is an educational tool for teaching science and introducing undergraduates to issues regarding the nature of science through primary scientific literature. They maintain that CREATE works as a supplement and complement of traditional lecture-based science teaching and inquiry laboratory classes. Likewise, they report that this method not only increased understanding of scientific research among undergraduate students but also their interest in this type of research. Also, improvements in the undergraduates’ ability to critically read and interpret data were found. CREATE was originally designed and implemented within the context of genetics and cell biology and has been applied or modified in various university courses such as introductory biology courses (Chatzikyriakidou et al., 2021 ) and ecological courses (Carter & Wiles, 2017 ; Smith & Paradise, 2022 ). Despite the benefits of CREATE, as Lennox et al. ( 2020 ) point out, some of its disadvantages are that this method was designed for students who have achieved a certain mastery of disciplinary knowledge and the application of this pedagogical tool requires a considerable amount of work by instructors. In addition, Segura-Totten and Dalman ( 2013 ) question the effectiveness of this method to lead to higher learning gains in comparison with more traditional ways of reading primary scientific literature.

Recently, Chatzikyriakidou et al. ( 2022 ) demonstrated that increasing student engagement with primary scientific literature contributed to the promotion of research skills. They used the five core concepts of biology as a framework for student engagement with this type of literature. The analyses included the responses of 19 undergraduates to a questionnaire before and after an educational intervention in a discussion-based introductory biology course. One of the major findings of this study was that asking participants to complete a five core concept matrix table, connecting the biological content from a piece of primary scientific literature to at least three corresponding boxes of the table, encouraged students to produce notes and/or summaries of the content contained within research articles in their own words. This is a clear example of how to move beyond the passive absorption of information. Furthermore, the results of another study (Chatzikyriakidou et al., 2021 ) reaffirmed that involving students in purposeful activities such as the completion of a five core concepts of biology matrix table as they read a selected scientific paper and prompted them to begin to think of scientific facts contained within that paper as part of larger biological concepts. Importantly, one of the conclusions of a study conducted by Lee et al. ( 2022 ) was that simply reading a PDF® does little to enrich reading strategies in students learning to read primary scientific literature.

Instructors should not assume that undergraduate students have the skills and strategies to critically engage with primary research literature. This was one of the conclusions of a recent study carried out in a UK university (Hubbard et al., 2022 ). Data were collected through thirty-three interviews which were conducted with second year undergraduates ( n  = 9), final year undergraduates ( n  = 7), PhD students ( n  = 5), postdoctoral researchers ( n  = 6) and academics ( n  = 6). Furthermore, Bjorn et al. ( 2022 ) observe that even doctoral students struggle to engage critically with primary literature. Thus, instructors should explicitly create opportunities for students to actively engage with research literature. To illustrate this, consider the positive results reported by Heiss and Liu ( 2022 ) in analytical chemistry courses where students were asked to read scientific articles which were then debated in a discussion led by a team of students. According to the authors, this contributed to the reinforcement of the importance of topics covered in these courses. Another example related to this link between active learning and primary scientific literature is work by Palavalli-Nettimi et al. ( 2022 ) who provided biology undergraduates with opportunities to (1) read a piece of primary research literature, (2) develop annotations to dig deeper into the research communicated in the article, and (3) produce a podcast episode to share the research results with a general audience.

Even though peer assessment is widely accepted as a strategic ally of active learning practices (Amo & Jareño, 2011 ; de-Armas-González et al., 2023 ; Delgado Rodríguez, 2017 ; Reuse-Durham, 2005 ), the use of peer critique to promote critical reading is an under-researched possibility in science education. Indeed, there is not much evidence about the use of this type of assessment to foster critical reading of primary research literature in undergraduate science education. The novel results reported by Deng and his colleagues ( 2019 ) provide encouragement to embrace peer assessment. These academics tested an educational strategy with twenty-two undergraduate chemistry students at a Chinese university. The outcomes indicate that the integration of reading, anonymous peer critique, and discussion helped participants to engage with scientific writing practice. Besides, evidence led the researchers to conclude that peer evaluation was an effective means to provide opportunities for undergraduate students to (1) share their scientific writing productions, (2) compare each other’s production, and (3) engage in self-reflection. It should be pointed out that the participants in this Chinese study were asked to read four entire organic synthesis papers.

At this point, it is important to bear in mind that asking students to read an entire article is not the only option to introduce primary scientific literature in university science classrooms. Asking students to read specific sections (e.g., the introduction) is another acceptable option (Bogucka & Wood, 2009 ). Hunter and Kovarik ( 2022 ), for instance, have explored the use of excerpts and data from primary literature to help students focus on applying what they have learned in class. This exploration was carried out using active learning in analytical chemistry courses offered by The College of New Jersey and Trinity College. One of the reasons mentioned by these academics against recommending that instructors ask the students to read one full article is that “reading the entire paper makes the assignments more laborious and likely too time-intensive for the frequent practice that is desired” (p. 1242). In our case, undergraduates were required to read two entire scientific articles outside class, while specific parts of these papers were addressed during class time. In the next section, this and other features of our ALS will be discussed in detail.

3 Conceptual Framing

In this section, we discuss the conceptual framework on which the pillars (TORI, argumentation, and peer critique) of our ALS were based. To begin with, it is important to define learning, active learning, and ALS. Archila et al. ( 2022a ) “define learning as constructing meaning and reflecting critically on this meaning” (p. 3). Therefore, we assume that critical reading is a prime contributor to students’ learning achievement since critical readers are able to actively construct meaning from textual (and visual) information as they think critically about this information, going beyond a surface reading to read between the lines (Griffiths & Davila, 2022 ). While numerous definitions exist, active learning in tertiary education can be described broadly as abandoning lecture-based instruction (passive learning), engaging students in activities where they do and/or produce something, cultivate their higher-order thinking skills (e.g., argumentation), and critically reflect on what they are doing and/or producing (Mizokami, 2018 ).

Archila et al. ( 2022a ) stress that active learning is a valuable option to counter the centuries old instructor-centered lecture format that has dominated science teaching and learning in higher education institutions around the globe, since active learning focuses on higher-order thinking development rather than the transmission, memorization, and recall of information. They further point out that it is more likely that science instructors will embrace active learning practices if these are presented as a sequence of concrete and realistic research-based actions instead of abstract and idealized discourses. From this perspective, in this article, we coin the term ALS to refer to a series of activities purposeful planned and implemented in a classroom to involve students in the process of learning.

To reiterate, the TORI is a vital pillar of our ALS. It is worth mentioning that Ritchey and List ( 2022 ) proposed this framework, drawing on decade-long research advances in education as well as in psychology. Importantly, the TORI framework highlights that tertiary education-level reading has three characteristics. The first characteristic results from the fact that tertiary education-level reading is domain specific . Texts are informed by the epistemic aims and practices of particular domains (e.g., business, chemistry, and history). Clearly, the content communicated in a text of a particular domain (e.g., business) differs from one from another particular domain (e.g., chemistry). This means that in the process of making sense of this content, students should make use of domain-specific strategies. Scientific research articles, for instance, are constructed on the basis of scientific argumentation processes. It is therefore reasonable to point out that this type of text follows an argumentative structure (Van Lacum et al., 2014 , 2016 ) which should be considered in the creation of strategies to help students critically engage with scientific articles.

The second characteristic is variability. Students in courses at tertiary level may be asked to read different types of texts (e.g., chapters from handbooks, news articles, scientific articles, and textbooks). The importance of this characteristic is that students are likely to be more familiarized with some type of texts (e.g., news articles) than others (e.g., scientific articles) which are not necessarily written with a student audience in mind. This leads to the claim that students should be specially assisted in the reading process of expert-level texts. And the third characteristic of tertiary education-level reading is autonomy. Tertiary education students are regularly expected to demonstrate a high level of autonomy during the completion of course reading assignments. It should be pointed out that “autonomy” does not mean that instructors can assume that students have the skills to be able to critically engage with disciplinary texts. Research evidence, for example, shows that students seldom use critical reading skills. Instead, skimming and skipping sections and information when working on reading assignments are very common actions among students (Hairston-Dotson & Incera, 2022 ; Lennox et al., 2020 ). In this regard, Hubbard ( 2021 ) claims that giving students support is essential in order to cultivate their critical reading skills.

Ritchey and List ( 2022 ) maintain that an attractive strength of the TORI is that this can be applied in a relatively simple way and in any course and discipline. An important reason for this is that the instructional framework consists of only five pragmatic steps all related to the instructor. The first step involves selecting the text that will be assigned to the students. This step tends to be familiar to instructors since text selection is an integral part of their course design process (Collins-Dogrul & Saldaña, 2019 ). It is important to stress that this selection should be guided by the goal(s) for reading as some text types (e.g., scientific article) may be more linked to specific types of reading goals (e.g., evaluating the coherence of an experimental design). The second step concerns the identification of the specific goal(s) that is expected students should achieve. It is recommended that this goal should be included in a short statement in declarative style, using a verb (e.g., judge) at the beginning. This helps students to have more clarity about the process of reasoning entailed in the reading goal. Of course, the accomplishment of such a goal would contribute to the strengthening of the students’ reading routines.

The third step addresses the definition of the means by which students’ reading will be assessed. Both format (e.g., open questions) and cognitive demand (e.g., argumentative questions) are aspects that can vary. Thus, the challenge for instructors in this step is to ensure that the assessment(s) is aligned with the assigned reading goal(s). Likewise, faculty should make clear to students how the assessment is influenced by the reading goal. The fourth step consists of identifying effective strategies to help students meet the goal of the assigned reading and prepare for assessment. This step is a good opportunity for faculty to not only present students with effective and pragmatic reading strategies to use to tackle expectations regarding reading but also to help them become aware of the importance of using strategies during the reading process. A prime reason for promoting the use of reading strategies among students is that these strategies can help them to better explore the text (e.g., preview the headings, highlight topic sentences, and write down main ideas) and reflect on the ideas of this text (e.g., think about what the highlighted sentences have in common). By the same token, these strategies should be acknowledged as concrete actions carried out to engage productively with the text. Accordingly, the major purpose of the fifth step of the TORI is devoted to the explicit communication of the information from steps 1 to 4 to the students. Instructors can use this step to show students the articulation and coherence between each of the four previous steps. Likewise, this step serves to help students recognize how their reading process is being supported. We will explain later how these steps were adopted in our ALS.

Another major feature of the TORI is that this is guided by the premise that presenting students with explicit, specific goals for reading is a condition sine qua non for the promotion of productive task-oriented reading behaviors. This premise makes sense if we acknowledge that students educated in passive science learning classrooms tend to read with the sole purpose of retaining information that they feel will be useful to do well in exams, rather than reading to analyze, understand, interpret, and criticize scientific texts (Hubbard & Dunbar, 2017 ). As Hubbard et al. ( 2022 ) observe, students’ passive and exam-oriented view of reading can be countered if instructors start to become aware that scientific texts are a powerful educational resource. In addition, they maintain that faculty should exploit the educational potential of scientific research literature by providing students with explicit opportunities to become more active and more critical readers of this type of literature.

Argumentation—defined as “the justification of claims with reasons and/or evidence” (Erduran et al., 2022 , p. 655)—is another pillar of our educational scenario. This cognitive-linguistic skill is intimately connected and vital to critically read scientific articles. One reason for this is that, as Van Lacum et al. ( 2014 ) point out, various elements of scientific argumentation (e.g., argument, counterargument, and rebuttal) are communicated in scientific papers. In fact, the production of research articles is a prime example of why scientific argumentation is considered not only an intellectual but also a communicative activity. Given that argumentation contributes to the production of any reason-based and/or evidence-based scientific construct (e.g., explanations, models, and theories), it is no exaggeration to say this skill plays a vital role in the progress of science. Finocchiaro ( 2021 ), for instance, reminds us that Galileo, the father of modern science, was an authentic practitioner of argumentation. Another example of the substantial value of argumentation in the scientific enterprise is documented in the recent book The Pandemic of Argumentation (Oswald et al., 2022 ). A common claim in its eighteen chapters is that scientific argumentation is a powerful skill to fight disinformation and/or misinformation about COVID-19. By the same token, it is an undeniable fact that argumentation is beneficial to the promotion of COVID-19 literacy—“the functional understanding of Covid-19 as well as making informed decisions based upon this understanding” (Archila et al., 2021a ).

The production of arguments, counterarguments, and rebuttals is the specific argumentation skill targeted in our study. The reason for this is that these skills are crucial elements of critical argumentation—the questioning of argumentation in order to develop a balanced (critical) point of view (Walton, 2006 ). Here, we assume that the production of counterarguments is a rational way to question arguments, while questioning counterarguments requires the formulation of rebuttals. It should be pointed out that critical argumentation is in line with the idea that argumentation benefits from critical thinking and vice versa (Andrews, 2015 ). Osborne ( 2010 ) maintains that scientists routinely engage in critical argumentation. Also, he notes that the construction of counterarguments and rebuttals is an example of the critical side of argumentation. The importance of critical argumentation is that it allows students to avoid the development of biased argumentation (Archila et al., 2023a , 2023b ). Therefore, it does not sound surprising that critical argumentation has become a key ally of critical reading which is widely acknowledged as a branch of critical thinking (Archila et al., 2019 ; Lin, 2014 ; Ritchey & List, 2022 ; Wilson, 2016 ). Wallace and Wray ( 2021 ) emphasize the importance of always reading academic journal articles with a critical eye, adopting a reasonable skeptical stance towards the authors’ claims. They explain that reasonable skepticism entails being open-minded and willing to recognize the robustness of the authors’ claims, but only if they can adequately support their claims.

Peer critique is the third pillar of our ALS. The reason why we separate peer critique from argumentation (the second pillar of the ALS) is that it is expected that first of all, students will produce arguments, counterarguments, and rebuttals, and then, they will become engaged in peer critique of the argumentation produced by others. Peer assessment is one of the various sources of assessment for learning (also referred to as formative assessment) which is defined as “a process in which teachers and students recognize and respond to student learning, during that learning” (Cowie, 2012 , p. 679). We agree with Slater ( 2020 ) that successful implementation of active learning in university science courses requires instructors to become aware that lecture-based instruction is the perpetuation of passive learning and that they should provide students with more meaningful learning experiences such as formative assessment. In this sense, peer assessment emerges as a powerful means to make such experiences a reality. Harris and Brown ( 2013 ) remind us that fundamental to successful implementation of peer assessment is the construction of awareness of the feature that this type of assessment entails, that students are free from instructor dependence, since peer assessment is essentially a student-led processes. Hence, students need to be thoroughly instructed on how to make useful judgements (useful feedback) about the work of others. It is true that simply instructing students is insufficient. Instructors should provide students with genuine opportunities to criticize the work of their partners in order for them to cultivate peer assessment skills. In our case, instruction and opportunities were even more explicit because the study was conducted in a formative assessment environment.

According to Topping ( 2018 ), there is a wide spectrum of products that can be used to engage students in peer assessment practices. Some examples include oral presentations, portfolios, test performance, and writing. Furthermore, he underlines the versatility of peer assessment. For instance, peer critique can be (i) one-way (e.g., an older class assessing a younger class), reciprocal (e.g. same-ability pairs in one class), or mutual within a group; (ii) voluntary or compulsory; (iii) anonymous or not; (iv) on single or multiple pieces of work; (v) on the same or different kind of products; (vi) guided or not by rubrics or structured formats for feedback; (vii) expected or not to lead to opportunities to rework the product in the light of feedback; (viii) organized by matching of students in deliberate and selective ways or random or accidental ways; (ix) carried out in class or outside class; and (x) supported or not by information technology. Specifically, the structure of peer assessment in our ALS was (i) reciprocal, (ii) voluntary, (iii) anonymous, (iv) carried out on multiple pieces of work, (v) carried out on the same kind of products, (vi) guided by rubrics, (vii) articulated with opportunities to make revisions after receiving feedback, (viii) organized by matching students in random ways, (ix) carried out in class, and (x) supported by information technology. We will describe more fully later the way in which these specificities were exploited in our scenario.

4 The Active Learning Scenario

Bearing in mind the idea that it is more likely that instructors will become receptive to implementing an ALS if, at the beginning, this is planned for a limited number of classes, we designed our scenario as two 75-min class sessions and followed a step-by-step structure (three steps in Session 1 and four steps in Session 2) (Fig.  1 ). It is important to clarify that before these two sessions, undergraduates were given 3 weeks to read two assigned scientific articles (Article 1 (Iacumin et al., 2022 ); Article 2 (Ayeni et al., 2011 )). The instructor explained to the students that Articles 1 and 2 would be the backbone of two in-class activities and that the goal of these readings was to carry out an initial exploration of these papers, at their own pace. Also, during these 3 weeks, they were free to ask the instructor any question about any feature (e.g., scientific content) of the articles. Furthermore, before students started the activity of the first session, they received instruction about how to read and critique scientific articles (Yeong, 2014 ). As recommended by Rybarczyk ( 2006 ) and Van Lacum et al. ( 2014 ), emphasis was placed on the role of each element of the conventional structure of research articles (e.g., abstract, introduction, method, results, and discussion).

Major elements of the active learning scenario

Articles 1 and 2 were chosen for four reasons. The first is that these papers present various of the concepts (e.g., food spoilage microorganisms, molecular biology) discussed during lectures beforehand, thus helping “students [not to] get frustrated with unfamiliar terminology” (Rybarczyk, 2006 , p. 166). Second, these manuscripts had standard scientific research article structures (Article 1 (Abstract, Introduction, Materials and methods, Results and discussion, and Conclusion); Article 2 (Abstract, Introduction, Material and Methods, Results, and Discussion)). Third, both articles followed a single-anonymized peer review process before being accepted for publication (Tomkins et al., 2017 ). And the fourth reason is that these research articles were published in specialized scientific journals.

The ALS revolved around two sections of the standard scientific research article structures, namely, the Introduction section and the Material and Methods section. Much of the reason for this is that we were particularly interested in the valuable opportunities that these sections could give the students to get a summarized idea of the articles (Introduction section) and go deeper into the way the authors conducted the studies (Material and Methods section). These two sections were addressed gradually throughout the two sessions of the ALS to avoid students becoming saturated with information. The first session of the ALS dealt with the Introduction section of Article 1. Students were told that the goal of this session was to read the information communicated in the Introduction section from a critical point of view. In the first step, each student was asked to answer the thought-provoking (and ambiguous) question, “You consider that the quality of the Introduction section of Iacumin et al.’s ( 2022 ) article is…”. The five options were Excellent, Very Good, Good, Fair, and Poor. In order to engage students in critical reading, they were required to produce (1) at least two valid arguments—based on reasons and/or evidence (Erduran et al., 2022 )—and coherent—effectively supporting her/his claim (Archila et al., 2022b )—; (2) at least two valid and coherent counterarguments; and (3) at least one valid and coherent rebuttal while answering the question, “Why did you make that decision?”. This question is expressly recommended by Archila et al., ( 2019 , 2021b ) to prompt students to read critically. We would like to stress that the instructor made it very clear that there was no one right answer (e.g., very good, fair). In other words, whatever students decided, what was most important was the argumentation they constructed.

In the second step, participants were engaged in reciprocal peer critique (Topping, 2018 ). To this end, students were given access to a Google Sheet® in which each graded and provided anonymous useful feedback on the argumentation of two anonymous peers. This means that each student played the role of critics (giving feedback) as well as being criticized (receive feedback). Following Archila et al. ( 2022b ), the instructor encouraged undergraduates to focus their criticisms on the quality of the argumentation of their peers and to avoid commenting “on trivial problems and errors (e.g., spelling)” (p. 2290). The importance of offering feedback respectfully, avoiding harmful comments was explained to them.

In the third step, students were provided with a final opportunity to read the Introduction section critically. Specifically, each undergraduate was asked to reflect upon the comments written by two peers in the previous step about her/his argumentation and to develop a (final) standpoint about the thought-provoking question mentioned in Step 1. It is worth adding here that students were free to put (or not) the feedback comments they received into practice, thus making Step 3 more about engaging in evaluative judgement practice and less about passive consumption of feedback.

With respect to the second session of our ALS, this addressed the Material and Methods section of Article 2. The instructor made it clear to the undergraduates that the goal of this session was to critically judge the information communicated in this section. In the first step, each student answered the thought-provoking question, “You consider that the quality of the Material and Methods section of Ayeni et al.’s ( 2011 ) article is…”; they could choose one of the following options: excellent, very good, good, fair, and poor. Moreover, students were asked to answer the question, “Why did you make that decision?”. They were required to construct at least two arguments, two counterarguments, and one rebuttal for their answer.

In the second step, students discussed in groups of three or four the decision made by each undergraduate in Step 1 with a view to making a group decision. In response to Aikin and Casey’s ( 2022 ) invitation to move from the disagreement-dependent argumentation (disagreement-only and disagreement-always) format to (dis)agreement-centered argumentation, the instructor encouraged students to interact argumentatively not only in possible dissensus situations but also in situations of agreement, since “one can give reasons, for example, to reinforce, maintain, or intensify someone’s agreement on a mutually shared proposition” (p. 3). In addition, each group was asked to co-construct at least two arguments, two counterarguments, and one rebuttal for their decision. Current research suggests that student–student argumentative interaction seems to be one possibility to foster productive co-construction of (counter) arguments (Archila et al., 2022c ). Thus, this step offered an opportunity for students to experience an important feature of argumentation, namely, the fact that it can be co-constructed by different individuals.

In the third step, students were involved in reciprocal peer assessment practice. In other words, the argumentation of each group was criticized by two other groups, and members had to discuss and interchange impressions. Each group completed a Google Sheet®, grading, and giving anonymous useful feedback on the argumentation of two anonymous groups. This step was intended to enable students to practice their small peer assessment-group discussion skills (e.g., to articulate and evaluate their ideas). Finally, in the fourth step, undergraduates had a final opportunity to critically read the Material and Methods section. Each student was required to think about the written feedback produced by the two small groups in Step 3 about the argumentation of her/his group and to individually construct a (final) view about the thought-provoking question they answered in Step 1. Each student was encouraged to adopt an attitude based on evaluative judgement, while reflecting on the feedback comments her/his group had received earlier. It should be pointed out that the inclusion of a “Group decision” step (Step 2 in Fig.  1 ) in the second session of our ALS explains why this is a 4-step session while the first session is a 3-step session.

5 Research Design and Method

To demonstrate the extent to which the ALS engaged participants in the critical reading of scientific papers, the present study used a mixed methods approach (Mertens, 2023 ), collecting and interpreting data by means of quantitative and qualitative measures. As such, we expected to provide empirical evidence to support the claim that the ALS, which combines argumentation, peer critique, and the TORI (Ritchey & List, 2022 ), can be a rational and reasonable possibility for engaging students in the critical reading of scientific papers. Specifically, quantitative analyses were based on frequency counts, mean, standard deviation, and effect size calculations. On the other hand, the qualitative analyses involved a coding process of the participants’ responses to open-ended questions.

5.1 Context and Participants

The ALS reported in this article was implemented in a university bilingual (Spanish–English) science course—a type of course in which two languages are used and treated as valuable resources for science teaching and learning (Archila & Truscott de Mejía, 2020a )—called Food Microbiology. A common translanguaging practice (Mazak & Herbas-Donoso, 2015 ) in this course is to assign readings in English and discuss them in Spanish in order to cultivate student bilingual scientific literacy—scientific literacy in two languages (Airey & Linder, 2011 ). Moreover, one educational aim of the Food Microbiology course is that students would be able to critically read primary scientific literature. It is important to clarify that this course was chosen through convenience sampling—“a sample that is selected because of its availability to the researcher” (Clark et al., 2021 , p. 606), as the second author was one of the two course instructors. The course is offered every semester by the Department of Chemical and Food Engineering of a major university in Bogotá, Colombia. This small-enrollment course (20–40 students per semester) serves undergraduate students from diverse academic programs such as, Biology, Biomedical engineering, Chemical Engineering, Food Engineering, and Microbiology. Usually, students enrolled in this course have completed about 60% of their undergraduate studies.

The study was designed with due consideration to the ethical guidelines of the American Psychological Association (APA 2017 ), and approval was granted from the University’s Research Ethics Committee before starting. Among the 65 eligible students, 63 (96.9%) consented to participate. However, two students did not fully participate in at least one of the two sessions of the ALS. Therefore, they were excluded from the analysis, resulting in a total sample size of 61 undergraduates (38 females and 23 males), aged 19 to 25 years old ( M age  = 20.9, SD age  = 1.55). The first language of the students was Spanish. All participants were fully informed about the purpose and the voluntary nature of the study. It was made clear to them that participation was completely anonymous and that they were free to withdraw at any time. Written informed consent was obtained from all participants. In addition, all information collected was treated confidentially, and students were assigned codes, for example, 1U28 means Class 1, undergraduate number 28.

These 61 participants were grouped into two classes with the ALS carried out in the following order:

Class 1: Undergraduates taking Food Microbiology during the spring semester (average age 21.1 years), 33 students (21 females and 12 males). Class 2: Undergraduates taking Food Microbiology during the fall semester (average age 20.6 years), 28 students (17 females and 11 males).

It is important to clarify that in Class 1 as well as in Class 2, science students and engineering students studied together. We decided to implement the ALS in two classes to have a larger sample size, but it is beyond the scope of this paper to examine the similarities and differences between these classes.

5.2 The Role of the Instructors

Instructors play an essential role in making active learning practices a reality rather than mere rhetoric. Of course, as Wenzel et al. ( 2022 ) emphasize, this implies that instructors should relinquish their outmoded role as transmitters of disciplinary knowledge and be convinced of the potential benefits of active learning (e.g., productive student learning experience). In our case, the two instructors who were responsible for the Food Microbiology course participated in the creation of the ALS and were wholly committed to implementing the scenario. Although the instructors were not experts on how to foster critical reading, they were aware of the importance of promoting this skill and spontaneously used to ask students not only to identify key features of scientific articles but also to read between the lines. The constructive participation of these instructors in the creation of the scenario helped us to propose a concrete, pragmatic, and realistic ALS. In the implementation of the scenario, they played the role of facilitators, giving room for undergraduates to assume responsibility for their learning. The sole function of these instructors was to encourage the undergraduates and engage them in the critical reading of the two scientific articles. While doing this, they assumed a neutral attitude without giving their impressions as much as possible and taking care not to use their authority to influence students’ decisions. “Some students may lack the confidence to critique papers […] because they are unsure that they understand the work presented” (Muench, 2000 , p. 256). With this in mind, the instructors created a symmetric instructor-student classroom atmosphere to help students feel free to ask what was confusing for them.

5.3 Data Collection

As already mentioned, the ALS consists of two sessions (Fig.  1 ). The data corpus of this study was derived from the written responses of the 61 participants. To be precise, students answered one questionnaire per session (Questionnaire 1, Session 1 in Appendix 1 ; Questionnaire 2, Session 2 in Appendix 2 ). These questionnaires were distributed at the beginning of each session. Questionnaire 1 contained four questions presented in two parts. The students answered the same questionnaire individually in both parts. A small-group decision section was included in Questionnaire 2 (Part two in Appendix 2 ). Each student had a copy of Articles 1 and 2 which s/he could refer to during each session. Article 1 had a length of ∼ 8,000 words while Article 2 was about 6000 words long. The Flesch Reading Ease Score (Flesch, 1948 )—a score that ranges from zero (very difficult to read) to 100 (very easy to read)—for the Introduction section (∼ 900 words) of Article 1 and the Material and Methods section (∼ 1500 words) of Article 2 were 20.8 and 17.1, respectively. This means that these sections were “very difficult” to read. Values between zero and 30 are typical of specialized texts such as scientific articles. The Flesch Reading Ease Score was calculated via the Microsoft Word® software (Stockmeyer, 2009 ).

Additionally, a Google Sheet® was used to collect the anonymous grading and the written peer critique produced by the students individually (Session 1, Step 2 in Fig.  1 ) and in small groups (Session 2, Step 3 in Fig.  1 ) about the argumentation (arguments, counterarguments, and rebuttals) of peers. Clearly, anonymity facilitates a willingness in students to communicate their critiques. The instructors made sure students understood how to use the assessment criteria (Table  1 ) and had as much time as they needed. Importantly, they asked students to assume a respectful attitude, avoiding comments that could be harmful. Likewise, online peer assessment offered us a major advantage, namely, the possibility of automatically recording data (Lu & Law, 2012 , cited in Topping, 2018 ). It is worth noting here that in the Food Microbiology course, students are regularly (1) provided with opportunities to produce (counter) arguments and rebuttals and (2) involved in peer assessment practices as part of the creation process of argument maps (for detailed examples from this course, see Archila et al., 2022b , 2022d ) and scientific argument podcasts (for detailed examples from this course, see Archila et al., 2023c ). Therefore, participants were familiar with argumentation and peer assessment practices. Another point to clarify here is that students received written feedback from the instructors after Session 2 of the ALS; however, this feedback was communicated to the students after the data collection period to avoid any influence of instructor feedback on the results of our study.

Finally, as in previous studies with undergraduates (Bennett & Taubman, 2013 ( N  = 20); Oliver, 2022 ( N  = 34)), students were asked to give us feedback for future improvements since we considered that our ALS was an unfinished and open educational resource that should be continually refined. Thus, at the end of each session, we administered a voluntary and anonymous online survey (Session 1, Survey 1 in Appendix 3 ; Session 2, Survey 2 in Appendix 4 ). The items were adapted from intervention feedback surveys developed by Archila et al., ( 2021a , 2021b , 2021c ), Bennett and Taubman ( 2013 ), and Oliver ( 2022 ). This adaptation did not change the purpose of the items as this was solely nominal. For example, the question, “Was the small-group debate useful for you to make a decision?” (Archila et al., 2021b , p. 288), was adapted to “Were the opportunities of giving written feedback to two peers useful for you to make a decision?” (Question 3 in Appendix 3 ). Fifty-six out of the 61 participants (32/33 in class 1 and 24/28 in class 2) answered Survey 1, while 47 students (26/33 in class 1 and 21/28 in class 2) answered the second survey.

5.4 Data Analysis

In response to our research question, “To what extent does the ALS engage science (Biology and Microbiology) students and engineering (Biomedical engineering, Chemical Engineering, and Food Engineering) students in the critical reading of scientific papers assigned in a Food Microbiology course?” the responses to closed-ended questions of Questionnaires 1 (Questions 1 and 3 in Appendix 1 ) and 2  (Questions 1, 3, and 5 in Appendix 2 ) were placed on a rating scale range of frequency: “poor (1),” “fair (2),” “good (3),” “very good (4),” or “excellent (5).” Note that this scale was presented to the students in both questionnaires. Our analysis continued by coding (scoring) the participants’ responses to open-ended questions in the questionnaires (Questions 2 and 4 in Appendix 1 ; Questions 2, 4, and 6 in Appendix 2 ). This was carried out according to four criteria as described below in Table  1 . For example, the following is the argumentation produced by 1U28 as response to Question 2 of the Questionnaire 2 and coded with the maximum score: six.

[Claim]: I consider that the Material and Methods section is good. [Argument 1]: A valuable feature of this section is that the authors mention the primers (Y1–Y2) they used in the partial sequencing of the ribosomal subunit 16 (16S rRNA) to identify the microorganisms. This contributes to the replicability of the experiments. [Counterargument 1]: However, in this section the sequence of the primers is not reported. These primers are crucial for the identification of the microorganisms. Even though the primers are reported in external sources, the sequence used must be detailed to avoid confusion or factory changes of the primers. Moreover, providing this information would benefit the replicability of the experiments.

[Argument 2]: In addition, I find it very useful that the authors indicate the commercial kits they used to determine the minimal inhibitory concentration. In this case, they used VetMicTM Lact-I microdilution tests to determine the MIC [referring to minimal inhibitory concentration] to some aminoglycosides, lincosamides, tetracyclines, and amphenicols. [Argument 3]: Also, it is very well described how the authors used hand-made plates to determine the MIC to other antibiotics such as ampicillin, nitrofurantoin, T-S [referring to trimethoprim–sulfamethoxazole], and fosfomycin. [Counterargument 2]: Yet, the use of hand-made plates can result in a technical difficulty to ensure the replicability of the experiments. This is not a problem for the methods that involved standard identification kits. The use of hand-made plates can influence the results of the study since the quality of this type of plates may be affected by several conditions (human errors, measurement errors, reactants of poor quality, etc.). [Rebuttal]: Despite this, I feel that the hand-made plates protocol is well explained throughout the subsection section, “Antibiotic resistance pattern”. This allows other researchers to replicate the protocol used by the authors (1U28).

In our coding process, we assumed that a (counter) argument or rebuttal is valid when it is supported by reason and/or evidence (Erduran et al., 2022 ) while its coherence depends on the fact that it effectively defends a claim (argument), opposes an argument (counterargument), or rebuts a counterargument (rebuttal) (Archila et al., 2022b ). The process of coding was carried out in a collaborative way (Saldana, 2021 ). The first and second authors worked independently to code the students’ responses and they then got together to compare their results. The Kappa statistic was used to evaluate the reliability of this process. Cohen’s kappa coefficient (Cohen, 1960 ) was 0.95 for Question 2 of Questionnaire 1, 0.99 for Question 4 of Questionnaire 1, 0.90 for Question 2 of Questionnaire 2, 0.95 for Question 4 of Questionnaire 2, and 0.96 for Question 6 of Questionnaire 2. According to Bryman ( 2016 ), “a coefficient of 0.75 or above is considered very good” (p. 276) inter-coder agreement. We used the Statistical Package for the Social Sciences (SPSS®, v28) for all statistical calculations. Any disagreements were discussed, and a consensus score was reached for all participants. Moreover, Cohen’s d effect size was calculated using students’ argumentation of the “Initial decision” (Question 2 in Appendixes 1  and 2 ) as the “control” condition and argumentation of the “Final decision” (Questions 4 and 6 in Appendixes 1  and 2 , respectively) was treated as the “experimental” condition. We adopted the benchmarks for interpreting effect size proposed by Cohen ( 1988 , pp. 25–26): small ( d  = 0.2), medium ( d  = 0.5), and large ( d  = 0.8). It should be pointed out that this control-experimental assumption is a legitimate option in cases in which a control group is not available to researchers (Creswell & Creswell, 2018 ; Ferron et al., 2023 ; Tanious & Onghena, 2021 ). Thus, effect size can be calculated using both the responses of the students at the beginning of the intervention (control condition) and after this (experimental condition) (Archila et al., 2021b , 2023a , 2023b ; Meli et al., 2022 ).

Additionally, the first and the second author coded the participants’ responses to Questions 4 and 6 of Questionnaires 1 and 2, respectively. This coding process was based on two criteria presented in Table  2 . Wu and Schunn ( 2023 ) maintain that engagement with peer feedback is a key element of authentic assessment of learning practices. Thus, the purpose of this coding was to get an idea of the type of engagement (passive or active) that occurred in our ALS. To be clear, “the MSWord Compare Documents function was utilized to identify revisions” (Wu & Schunn, 2023 , p. 6). The revisions identified then followed the coding process which was guided by a single passive-active coding method. Cohen’s kappa coefficient calculated was 0.87 for Question 4 of Questionnaire 1 and 0.84 for Question 6 of Questionnaire 2. Where there were differences, it was deemed necessary to re-examine the participants’ responses and then to discuss these until a consensus was reached.

In order to get an idea of the quality of the feedback given by the students, feedback in writing produced individually (Session 1, Step 2 in Fig.  1 ) and in small groups (Session 2, Step 3 in Fig.  1 ) was coded. Based on the coding rubric (Table  3 ) proposed by Archila et al., ( 2022b , p. 2294), the first and the second author carried out this coding process independently and attained a Cohen’s kappa coefficient of 0.87 and 0.91 for individual written peer feedback and small-group written peer feedback, respectively. Where there were disagreements, these were resolved by discussion. Finally, we used frequency counts to analyze participants’ responses to the intervention feedback surveys. Some answers to open-ended questions (Questions 3 and 4 in Appendix 3 ; Questions 2 and 3 in Appendix 4 ) are commented on in the next section.

We report the findings of the implementation of the ALS in the following three sections; the first section is about students’ judgements of the quality of specific parts of Articles 1 and 2; the second section focuses on participants’ production of arguments, counterarguments, and rebuttals; and the third section addresses peer feedback. Throughout these three sections, we present the results of the intervention feedback surveys to offer a deeper understanding of the contribution of our study.

6.1 Participants’ Judgements of the Quality of Specific Sections of Articles 1 and 2

In our ALS, the participating students were asked to judge the quality of the Introduction section of Article 1 as well as the Material and Methods section of Article 2. Table 4 shows the students’ average judgements (scores) along with the standard deviations. The maximum and the minimum possible average scores for each stage of judgement (e.g., initial individual judgement) were 5 (excellent) and 1 (poor), respectively. In general, the results indicate that students considered that the quality of these sections was very good/good. Although these outcomes provide a valuable overview of the students’ judgements, one might ask, were undergraduates effectively engaged in critical reading? Clearly, to make a judgement does not necessarily mean to become engaged in critical reading when this judgement is not accompanied by sound argumentation.

6.2 Participants’ Argumentation

Results suggest that students effectively became engaged in critical reading during our ALS. As demonstrated by Table  5 , students accompanied their judgement by valid and coherent arguments, counterarguments, and rebuttals. The fact that all the participants’ average scores were higher than 4.90/6 can be seen as a positive result that should be interpreted in the light of five aspects. First, as part of the Food Microbiology course, students received instruction about how to read and critique scientific articles. Second, a large fraction of the undergraduates (27/32 in class 1; 22/24 in class 2) who answered Survey 1 had received instruction in critical reading before taking this course (Question 1 in Appendix 3 ). Third, students were provided with strategies to help them meet the reading goals (Fig.  1 ). Fourth, before Sessions 1 and 2, undergraduates were given 3 weeks to read the two assigned articles. It is important to note that the great majority of the students considered that they had had sufficient time for reading Articles 1 (32/32 in class 1; 24/24 in class 2; Question 2 in Appendix 3 ) and 2 (24/26 in class 1; 21/21 in class 2; Question 1 in Appendix 4 ). And fifth, in the Food Microbiology course, undergraduates were regularly given explicit opportunities to produce (counter) arguments and rebuttals while creating argument maps and scientific argument podcasts.

These aspects largely explain why the participants’ average scores were higher than 4.90 even in the initial individual judgement. Given this situation it makes sense to have found only small ( d  = 0.32) and medium ( d  = 0.50) effect sizes (Table  5 ) when initial individual judgements were compared with final individual judgements. In other words, the students had not only had training in the production of (counter) arguments and rebuttals before they started Session 1 but also training in critical reading. Additionally, they had sufficient time for reading. Clearly, the contribution of our study is that it is the first in which the production of (counter) arguments and rebuttals is used as a platform to engage students in the critical reading of scientific articles. By the same token, Table  5 indicates that students went beyond the passive absorption of information; they actively engaged with Articles 1 and 2.

To illustrate active engagement in critical reading, consider the following argumentations produced by 2U7 and 2U20 as response to Question 4 of the Questionnaire 1:

[Claim]: I think that this introduction is good, [Argument 1]: as initially there is a brief explanation of the mixtures involved in the preparation of ice cream, as well as how these are used. This is helpful in order to be able to understand in more detail the reasons why this food is one of the most affected by the action of altering microorganisms. [Counterargument 1]: However, one of the shortcomings regarding this is that there is no adequate conceptualization of the Zygosaccharomyces rouxii microorganism because this is only seen as one of the most common osmophilic yeasts which can be found in food with high concentrations of sugar. [Rebuttal]: In spite of this, the introduction refers to the most important techniques and strategies used to control and slow the growth of osmophilic yeast, and this is very relevant to be able to understand the rest of the article.

[Argument 2]: On the other hand, the introduction is quite coherent, showing that the aim is clear in relation to the existing problem and the methods which are proposed to solve it. [Counterargument 2]: Nevertheless, these clarifications are short and only found at the end of the introduction. From my point of view, this is not sufficient, as the reader can lose interest in these important details (2U7).

[Claim]: I consider that the introduction is good [Argument 1]: because it contextualizes the topic to be developed and its importance, as well as clarifying the issue of interest. First of all, there is a definition of MBICs at the beginning of the introduction and a focus on important aspects such as the fact that these are “intermediates for the production of artisanal or industrial ice creams and desserts” (Iacumin et al., 2022 ), which helps to identify their role in the frozen food industry. [Counterargument 1]: However, in the contextualization section, there is no mention of information or numbers that help to better establish the importance of this topic in industry, or to position information at national or international level in order to improve the understanding of this issue.

[Argument 2]: Besides, [in the introduction section] the issue of interest is clarified by explaining that “the high number of organic compounds contained in these preparations favors contamination and, in some cases, even the development of a rich yeast population” (Iacumin et al., 2022 ). Moreover, there is an indication of the conditions which favor their growth and the type of yeasts which can develop at the base of the ice creams. These are osmotolerant and osmophilic yeasts. In addition, there is a connection with the yeast of interest – the Zygosaccharomyces rouxii . [Counterargument 2]: Nevertheless, there is no mention of the effect of the yeast at the base of ice cream and which is very important to understand the seriousness of the problem. [Rebuttal]: In any case, there is some information about the effect of yeasts in general in this type of substratum, which helps to give an idea (2U20).

It is encouraging to see that the arguments [Arguments 1 and 2] produced by 2U7 and 2U20 demonstrate that they read the introduction section of Iacumin et al.’s ( 2022 ) article with a purpose in mind: to produce arguments to support their decision that the introduction was good. In particular, it is interesting to note that 2U20 used citations in both arguments to make clearer what s/he was referring to. This indicates her/his valuable commitment to navigate the ideas of this section. Likewise, the counterarguments communicated by these students suggest that they strived to take into consideration alternative views. The importance of this result is that this shows that 2U7 and 2U20 were able to object their own arguments. The relevance of this is even more evident once we recognize that the “task [of anticipating counterarguments] is generally difficult” (Mercier, 2016 , p. 691). Active engagement in critical reading is also evidenced in the fact that 2U7 and 2U20 went further in the production of arguments and the anticipation of counterarguments; each generated one valid and coherent rebuttal as opposition to one of the counterarguments. One aspect to comment on the rebuttals produced by these students is that both focused on specific and desirable features of an introduction section (e.g., “the most important techniques and strategies […] relevant to be able to understand the rest of the article” (2U7) and “some information […] to give an idea” (2U20)). This indicates that they seem to have understood what it is to be expected from the introduction section of a research article. In short, the critical argumentation (argument, counterargument, and rebuttal) produced by these participants illustrates a concrete and realistic possibility for going beyond a passive and surface reading to read in a more active and more critical way.

6.3 Quality of the Written Peer Feedback

Peer critique is one of the pillars of our ALS (Fig.  1 ). The findings suggest that a relevant number of participants offered useful written peer feedback to their peers (Table  6 ). Topping ( 2018 ) invites us to embrace the idea that peer assessment is a valuable source of formative assessment. He argues that students demonstrate that they have developed an authentic understanding of the product subject of judgement when they are able to assume the role of peer assessors and provide useful peer feedback. Furthermore, we found that almost all the participants who answered Surveys 1 (29/32 in class 1; 20/24 in class 2; Question 3 in Appendix 3 ) and 2 (24/26 in class 1; 19/21 in class 2; Question 2 in Appendix 4 ), acknowledged that giving feedback to peers was useful for them to develop a judgement. Comments made in these surveys included: “I could compare my own text while reading the text of a peer, and thus I could realize my own weak points,” “being an assessor forced me to become more aware of what the other peer wrote. This helped me to dig deeper into my argumentation,” and “I could develop a broader judgement and not just focus on what I believed.” Furthermore, more than half of the respondents felt that receiving peer feedback was useful for them in making a decision ((Survey 1: 25/32 in class 1; 19/24 in class 2; Question 4 in Appendix 3 ) (Survey 2: (19/26 in class 1; 19/21 in class 2; Question 3 in Appendix 4 )). Some of their reasons include the following: “It was good to receive peer feedback as this helped me to revise my judgements,” “the comments that I received led me to identify what I had to improve,” and “this feedback helped me to analyze my decision and clarify my argumentation in more detail.”

Finally, the number of participants who were actively engaged with peer feedback is higher than those who were passively engaged (Table  7 ). As Wu and Schunn ( 2023 ) point out, providing students with opportunities to (1) offer, (2) receive, and (3) reflect on peer feedback is an important but (still in the twenty-first century) neglected element of constructive learning experiences. Therefore, the inclusion of peer critique in our ALS is a legitimate and desirable contribution if we take into account that the results of the surveys revealed that many of the respondents never or infrequently had the opportunity to give written peer individual feedback (26/32 in class 1; 21/24 in class 2; Question 5 in Appendix 3 ) or written peer group feedback (22/26 in class 1; 18/21 in class 2; Question 4 in Appendix 4 ) in other university courses.

7 Discussion

In 1983 , Beverley Bell insisted that promoting student active engagement with scientific texts should be an imperative within science education. The problem is that 40 years later, instructor-centered lecture formats dominate science teaching and learning in too many universities around the globe. Recently, Idsardi et al. ( 2023 ) stressed that institutions of higher education should invest more efforts and resources in switching from instructor-centered learning (passive learning) to student-centered learning (active learning). In this regard, Hubbard ( 2021 ) and Hubbard et al. ( 2022 ) observe that instructors commonly ask students to read assigned scientific articles, but rarely engage students in the critical reading of these articles. In addition, students tend to hold the limited view that reading is necessary solely to earn good grades (Marchant, 2002 ; Theriault, 2022 ), overlooking its contributions to several scientific domains (e.g., authentic inquiry) (Xiang, 2022 ). The TORI proposed by Ritchey and List ( 2022 ) is in essence an invitation for instructors to go beyond the mere role of reading assigners. Argumentation skills are intimately connected and fundamental to critically read scientific articles. Unfortunately, one consequence of the hegemony of lecture-based instructional practices is that “as we begin the third decade of the twenty-first century, argument and debate are not habitual practices of university science education” (Archila et al., 2022c , p. 236). Another consequence is that lecture-based instruction does not encourage peer assessment (Winstone & Carless, 2020 ). For all the reasons just mentioned, here, we provide evidence for the claim that an ALS combining the TORI, argumentation, peer assessment can be a concrete and realistic possibility for engaging students in the critical reading of scientific articles. In what follows, we discuss the results in terms in which they answer our research question, “To what extent does the ALS engage science (Biology and Microbiology) students and engineering (Biomedical engineering, Chemical Engineering, and Food Engineering) students in the critical reading of scientific papers assigned in a Food Microbiology course?”.

The great majority of the participating students became engaged in the critical reading of the Introduction section of Article 1 as well as the Material and Methods section of Article 2. To be clear, our results seem to indicate that most students produced valid and coherent argumentation while judging the quality of these sections (Table  5 ). As previously mentioned, students were asked to make a decision about the quality of each section (excellent, very good, good, fair, and poor). This is a thought-provoking (and ambiguous) task. Much of the reason for this is that there is no one right answer unless we have provided students with quite specific criteria to, for example, distinguish between “very good” and “good.” And we did not do so. We left it purposely ambiguous. This ambiguity encouraged students to prioritize the production of valid and coherent (counter) arguments and rebuttals over making a “right” decision. In other words, students were challenged to accompany their judgement by sound argumentation. This is in line with the claim that presenting students with thought-provoking and ambiguous questions is a legitimate means to engage them in genuine argumentation practices (Archila, 2015 ; Archila et al., 2021c , 2022e ).

The ALS effectively provided students with explicit opportunities to produce arguments, counterarguments, and rebuttals. This is particularly valuable since “arguments containing rebuttals are thought to be of the highest quality, as they require the ability to compare, contrast and distinguish different lines of reasoning” (Osborne, 2010 , p. 464). Recently, Erduran et al. ( 2022 ) reported that students are seldom encouraged to anticipate counterarguments, considering alternatives to their own arguments. Using the production of arguments, counterarguments, and rebuttals as a platform to engage undergraduates in the critical reading of scientific journal articles is an under-researched possibility. Earlier studies have focused on giving students explicit opportunities to identify components of the authors’ argumentation (e.g., supports, counterarguments, refutations) (e.g., Lammers et al., 2019 ; Van Lacum et al., 2014 , 2016 ); Wijayanti & Adi, 2022 ).

Also, our findings appear to reinforce the idea that the TORI can be applied in a relatively simple way and in any course and discipline (Ritchey & List, 2022 ). This idea is important due to the fact that nowadays the use of primary scientific literature is gaining attention in undergraduate science courses (Verkade & Lim, 2016 ). Essentially, this theoretically based framework helped us to give shape to our ALS. It may be obvious to point out that (1) stating explicitly the goals of assigned readings and (2) providing students with strategies to help them meet these goals are key conditions for productive reading. Nonetheless, as Hubbard ( 2021 ) and Hubbard et al. ( 2022 ) note, this rarely happens at the tertiary education level. One reason for this is that instructors commonly (prefer to) assume that students have the skills to be able to critically engage with the assigned readings. As Kerr and Frese ( 2017 ) suggest, instructors should guide students to understand the rational for reading (e.g., communicate the goals of reading) and how to capitalize effectively on reading assignments.

In our ALS, given that we followed the TORI framework, the two instructors not only made clear to the students what the goals of reading were but also equipped them with practical and functional strategies (e.g., Before reading, identify what prime characteristics it is expected to find in the Introduction section) (Fig.  1 ). Likewise, in Sessions 1 and 2 of the ALS, these instructors integrated “empathy” (Hubbard, 2021 , p. 60) in their practices, creating a symmetric instructor-student classroom atmosphere which allowed them to “act[ing] as reading mentors rather than expect[ing] students to read independently” (Hubbard, 2021 , p. 60). Arguably, the fact that we adopted an active learning approach instead of an instructor-centered lecture format facilitated this type of classroom atmosphere.

Additionally, the TORI highlights the importance of linking reading goals to assessment practices (Ritchey & List, 2022 ). In the ALS, we included peer assessment. The outcomes indicate that our scenario provided students with concrete opportunities to (1) give, (2) receive, and (3) reflect on written peer feedback (Table  6 and 7 ). Therefore, it is plausible to claim that our ALS contributes to the construction of responses to the call of Wu and Schunn ( 2023 ) for the creation of activities in which students have opportunities to receive comments from their peers about their work, provide constructive comments while reviewing and criticizing the work of others, and think about peer feedback. Furthermore, our results seem to corroborate Topping’s ( 2018 ) view that there is a wide battery of products that can be used to engage students in peer feedback assessment practices. To the best of our knowledge, this study is the first to combine critical reading of scientific articles, argumentation, and peer critique in the same scenario. It is worth adding here that the majority of the participants confirmed that they had never or had infrequently had the opportunity to offer written peer individual feedback or written peer group feedback in other university courses. This is why we agree with Campbell and Batista ( 2023 ) and Noroozi et al. ( 2023 ) that much work remains to be done to consolidate peer feedback as a recurrent educational practice in institutions of higher education.

Finally, taken together, the findings appear to suggest that our ALS is one legitimate and desirable possibility to move away from passive (and information absorption-based) learning views of reading to active learning practices of critical reading. The ALS is an original contribution as this integrates argumentation, peer critique, and the TORI framework (Ritchey & List, 2022 ). Thus, our results expand the literature on the use of primary scientific literature as an educational resource (Chatzikyriakidou & McCartney, 2022 ; Chatzikyriakidou et al., 2021 , 2022 ; Griffiths & Davila, 2022 ; Hoskins et al., 2007 ; Hunter & Kovarik, 2022 ; Lee et al., 2022 ; Muench, 2000 ; Palavalli-Nettimi et al., 2022 ; Smith & Paradise, 2022 ) by adding evidence supporting the idea that university students should be provided with a variety of opportunities to cultivate their critical reading skills. Likewise, this study sheds light on the importance of taking advantage of active learning principles to involve students in activities that require the application of discrete academic reading skills while engaging with scientific journal articles (Bennett & Taubman, 2013 ; Bogucka & Wood, 2009 ; Heiss & Liu, 2022 ). In our case, active learning occurred in the form of argumentation and peer critique practices. As Mizokami ( 2018 ) reminds us, active learning becomes a reality when students are engaged in activities where they do (e.g., critique the work of peers) and/or produce something (e.g., arguments, counterarguments, rebuttals), as well as cultivate their higher-order thinking skills (e.g., argumentation), and critically reflect on what they are doing and/or producing.

8 Conclusions

Students should be provided with opportunities to not only understand scientific articles but also to read them critically (Raimondi et al., 2020 ). Accordingly, the research question of our study revolved around the extent to which the ALS engaged undergraduate students in the critical reading of scientific journal articles. In the light of the above findings and discussion, one overarching conclusion that emerges from our research is that the ALS seems to be an original educational tool that offers promising potential for cultivating critical reading of scientific articles since the merits of this scenario are supported by the fact that the majority of the participating students became engaged in the critical reading of this type of articles. Another conclusion is that the integration of argumentation, peer critique, and the TORI framework (Ritchey & List, 2022 ) is effectively a concrete, realistic, and innovative possibility for providing students with explicit opportunities to practice critical reading skills.

9 Limitations and Future Directions

Despite the utility of the results, there are also various limitations which should be considered when interpreting these findings. First, one noticeable shortcoming of our study is the small number of participating students. This seriously limits the quality of our mixed method procedure since it does not allow us to make evidence-based generalizations of the eventual benefits of the ALS. Undeniably, had the number of students been larger, we would have been able to provide more robust research evidence relating to the benefits of our scenario. A second limitation is that this study was carried out in just one university science course with students from different science and engineering majors (e.g., food engineering, microbiology). This shows bias in our research design since we did not implement the ALS with a wider diversity of majors (e.g., law, psychology). Besides, many of the participating students, apart from the Food Microbiology course, had previously received instruction about how to read and critique scientific articles. Additionally, argumentation and peer assessment are common educational practices of this course. It must be admitted that these contextual factors could have largely influenced the promising results of our ALS. It would therefore be interesting to implement our scenario in other university courses in order to corroborate the findings reported in this article. Likewise, we did not collect data related to participants’ prior experience with other critical reading instructional initiatives, academic level, and prior content knowledge, which may be factors that influence our outcomes. Another main limitation of this study is that although undergraduates were given three weeks to read Articles 1 and 2, the two sessions of the ALS focused only on two specific sections of these articles, namely, the Introduction and the Material and Methods sections. Arguably, this limitation could be seen as a strength, if we acknowledge the positive experiences documented by Bogucka and Wood ( 2009 ), Hunter and Kovarik ( 2022 ), Spiegelberg ( 2014 ), and Vroom ( 2022 ), while implementing reading activities that required students to be focused just on quite specific parts of scientific journal articles.

It is our hope that the results presented here can contribute to enrich the global discussion of how to engage students in the critical reading of research articles. This is a discussion that is particularly motivated by the complex issue that students are often not reading the assigned texts (Gorzycki et al., 2019 ; Oliver, 2022 ; Sutherland & Incera, 2021 ) and instructors seldom integrate academic reading pedagogies into their courses (Desa et al., 2020 ). The situation is even more complex in countries where English is the second or foreign language since reading in this language can often constitute an extra challenge for some students (Archila & Truscott de Mejía, 2020b ). Of course, whatever means used to promote critical reading will be unproductive if students do not understand or complete their assigned texts. With this in mind, it should be pointed out that although our ALS seems to offer valuable benefits for both students (e.g., explicit opportunities to go beyond a superficial reading of research papers to critically engage with these readings) and instructors (e.g., pragmatic ways to move away from the passive learning view of reading towards more active learning practices), this does not mean that this scenario is finished. Put simply, we created this scenario as an unfinished and open possibility for instructors interested in cultivating students’ critical reading skills. Hence, further work on how to integrate this ALS with other active learning-based initiatives could be a particularly relevant research direction. For example, Gomez-Marin ( 2023 ) nowadays maintains that podcasts are becoming increasingly a strategic ally of scientific communication. It would therefore be interesting to explore the articulation of the ALS with the idea of involving undergraduates in the creation of podcast episodes in which they communicate the research outcomes of a piece of primary research literature with a general audience (Palavalli-Nettimi et al., 2022 ). Such articulation would result in students developing and producing scientific argument podcasts in which they communicate their judgements (arguments, counterarguments, rebuttals) of the quality of a scientific article.

An essential issue in the future will be to compare the ALS to other instructional initiatives (e.g., CREATE (Hoskins et al., 2007 )) and/or articulate our scenario with those initiatives. This could provide more clarity about the impact of the ALS as well as the ways how this scenario can benefit from the utilities of these initiatives and vice versa. Moreover, it would be relevant to study how to use the ALS as one possibility to prepare students to critically engage with multiple reading activities such as the search and selection of original research articles (Russo & Jankowski, 2023 ), the reading of annotated primary scientific literature (Kararo & McCartney, 2019 ), and participation in academic reading seminars (Afdal et al., 2023 ) and virtual journal clubs (Stengel et al., 2021 ). Furthermore, future research could involve developing specific active learning-based strategies that foster student skills to critically read fraudulent academic articles (Pflugfelder, 2022 ) as well as news articles that present people with false and/or inaccurate scientific information (Archila et al., 2019 , 2021b ). Clearly, there is much work to be done and the challenge and the need for future studies is increased by the fact that “the development of students’ critical reading skills is an important and urgent issue facing institutions of higher education today” (Sutherland & Incera, 2021 , p. 267).

Afdal, H. W., Spernes, K., & Hoff-Jenssen, R. (2023). Academic reading as a social practice in higher education. Higher Education, 85 (6), 1337–1355.

Article   Google Scholar  

Aikin, S. F., & Casey, J. (2022). Argumentation and the problem of agreement. Synthese, 200 (2), 1–23.

Airey, J., & Linder, C. (2011). Bilingual scientific literacy. In C. Linder, L. Östman, D. A. Roberts, P.-O. Wickman, G. Erickson, & A. MacKinnon (Eds.), Exploring the landscape of scientific literacy (pp. 106–124). Routledge.

Google Scholar  

American Psychological Association. (2017). Ethical principles of psychologists and code of conduct (2002, amended effective June 1, 2010, and January 1, 2017). https://www.apa.org/ethics/code/index.aspx

Amo, E., & Jareño, F. (2011). Self, peer and teacher assessment as active learning methods. Research Journal of International Studies, 18 , 41–47.

Andrews, R. (2015). Critical thinking and/or argumentation in higher education. In M. Davies & R. Barnett (Eds.), The Palgrave handbook of critical thinking in higher education (pp. 729–780). Palgrave Macmillan.

Archila, P. A. (2015). Using history and philosophy of science to promote students’ argumentation A teaching–learning sequence based on the discovery of oxygen. Science & Education, 24 (9), 1201–1226.

Article   ADS   Google Scholar  

Archila, P. A., Barbosa, V., Gravier, G., Levy, L., Ortiz, B. T., Wilches, L., Truscott de Mejía, A.-M., & Restrepo, S. (2022b). Integrating peer feedback and instructor feedback to support the construction of bilingual scientific argument maps. International Journal of Science Education, 44 (15), 2283–2305.

Archila, P. A., Danies, G., Molina, J., Truscott de Mejía, A.-M., & Restrepo, S. (2021a). Towards Covid-19 literacy. Science & Education, 30 (4), 785–808.

Archila, P. A., Forero, C., Truscott de Mejía, A.-M., & Restrepo, S. (2023c). Podcasting as a powerful tool for the promotion of bilingual scientific argumentation in a formative assessment environment. International Journal of Science Education . https://doi.org/10.1080/09500693.2023.2248669

Archila, P. A., Gravier, G., Levy, L., Ortiz, B. T., Rodríguez, A., Wilches, L., Truscott de Mejía, A., & Restrepo, S. (2022d). Using formal and informal formative assessment to support bilingual argument mapping in university bilingual science courses. International Journal of Bilingual Education and Bilingualism, 25 (9), 3420–3435.

Archila, P. A., Molina, J., Danies, G., Truscott de Mejía, A.-M., & Restrepo, S. (2021b). Providing undergraduates with opportunities to explicitly reflect on how news articles promote the public (mis)understanding of science. Science & Education, 30 (2), 267–291.

Archila, P. A., Molina, J., Danies, G., Truscott de Mejía, A.-M., & Restrepo, S. (2022e). Using the controversy over human race to introduce students to the identification and the evaluation of arguments. Science & Education, 31 (4), 861–892.

Archila, P. A., Molina, J., & Truscott de Mejía, A.-M. (2019). Promoting undergraduates’ awareness of the importance of thinking critically about false or inaccurate scientific information presented in news articles. Revista Eureka Sobre Enseñanza y Divulgación De Las Ciencias, 16 (13), 1–27.

Archila, P. A., Molina, J., & Truscott de Mejía, A.-M. (2021c). Using a controversy about health, biology, and indigenous knowledge to promote undergraduates’ awareness of the importance of respecting the traditions and beliefs of indigenous communities: The case of paragonimiasis in Colombia. Cultural Studies of Science Education, 16 (1), 141–171.

Archila, P. A., Restrepo, S., Truscott de Mejía, A.-M., & Bloch, N. I. (2023b). Drama as a powerful tool to enrich socio-scientific argumentation. International Journal of Science and Mathematics Education, 21 (5), 1661–1683.

Article   ADS   PubMed   Google Scholar  

Archila, P. A., Restrepo, S., Truscott de Mejía, A.-M., Rueda-Esteban, R., & Bloch, N. I. (2022a). Fostering instructor-student argumentative interaction in online lecturing to large groups: A study amidst the Covid-19 pandemic. Revista Eureka Sobre Enseñanza y Divulgación De Las Ciencias, 19 (1), 1–16.

Archila, P. A., & Truscott de Mejía, A.-M. (2020a). Bilingual university science courses: A questionnaire on professors’ practices and espoused beliefs. International Journal of Bilingual Education and Bilingualism, 23 (2), 132–152.

Archila, P. A., & Truscott de Mejía, A.-M. (2020b). Bilingual teaching practices in university science courses: How do biology and microbiology students perceive them? Journal of Language, Identity & Education, 19 (3), 163–178.

Archila, P. A., Truscott de Mejía, A., & Restrepo, S. (2022c). Online Verbal Argumentative Interaction (OVAI) in an online science class during the COVID-19 pandemic. Online Learning, 26 (4), 236–258.

Archila, P. A., Truscott de Mejía, A.-M., & Restrepo, S. (2023a). Using drama to enrich students’ argumentation about genetically modified foods. Science & Education, 32 (3), 635–668.

Ayeni, F. A., Sánchez, B., Adeniyi, B. A., de los Reyes-Gavilán, C. G., Margolles, A., & Ruas-Madiedo, P. (2011). Evaluation of the functional potential of Weissella and Lactobacillus isolates obtained from Nigerian traditional fermented foods and cow’s intestine. International Journal of Food Microbiology, 147 (2), 97–104.

Article   PubMed   Google Scholar  

Bell, B. (1983). Reading and the learner of science. Research in Science Education, 13 , 83–94.

Bennett, N. S., & Taubman, B. F. (2013). Reading journal articles for comprehension using key sentences: An exercise for the novice research student. Journal of Chemical Education, 90 (6), 741–744.

Article   ADS   CAS   Google Scholar  

Bjorn, G. A., Quaynor, L., & Burgasser, A. J. (2022). Reading research for writing: Co-constructing core skills using primary literature. Impacting Education: Journal on Transforming Professional Practice, 7 (1), 47–58.

Bogucka, R., & Wood, E. (2009). How to read scientific research articles: A hands-on classroom exercise. Issues in Science and Technology Librarianship . https://doi.org/10.29173/istl2504

Bryman, A. (2016). Social research methods (5th Ed.). Oxford University Press.

Campbell, C. W., & Batista, B. (2023). To peer or not to peer: A controlled peer-editing intervention measuring writing self-efficacy in South Korean higher education. International Journal of Educational Research Open, 4 , 100218.

Carter, B. E., & Wiles, J. R. (2017). A qualitative study examining the exclusive use of primary literature in a special topics biology course: Improving conceptions about the nature of science and boosting confidence in approaching original scientific research. International Journal of Environmental & Science Education, 12 (3), 523–338.

Chatzikyriakidou, K., Janelle Tacloban, M., Concepcion, K., & McCartney, M. (2022). The five core concepts of biology as a framework for promoting expert-like behaviors in undergraduates learning how to read primary scientific literature. Journal of Microbiology and Biology Education, 23 (3), 1–13.

Chatzikyriakidou, K., Manrique, C., Janelle Tacloban, M., & McCartney, M. (2021). Exploring primary scientific literature through the lens of the 5 core concepts of biology. Coursesource, 8 , 1–11.

Chatzikyriakidou, K., & McCartney, M. (2022). Motivation in reading primary scientific literature: A questionnaire to assess student purpose and efficacy in reading disciplinary literature. International Journal of Science Education, 44 (8), 1230–1250.

Clark, T., Foster, L., Sloan, L., & Bryman, A. (2021). Bryman’s social research methods (6th ed.). Oxford University Press.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20 (1), 37–46.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge.

Collins-Dogrul, J., & Saldaña, K. (2019). Text selection and course design: Faculty perspectives on critical reading and critical thinking. Journal of the Scholarship of Teaching and Learning, 19 (2), 88–101.

Cowie, B. (2012). Focusing on the classroom: Assessment for learning. In B. J. Fraser, K. G. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 679–690). Springer.

Chapter   Google Scholar  

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications.

de-Armas-González, P., Perdomo-Díaz, J., & Sosa-Martín, D. (2023). Peer assessment processes in a problem-solving activity with future teachers. Eurasia Journal of Mathematics, Science and Technology Education, 19 (4), em2245.

Delgado Rodríguez, M. J. (2017). La evaluación por compañeros como actividad de aprendizaje activo en la asignatura de Hacienda Pública. e-pública, Revista electrónica sobre la enseñanza de la Economía Pública, 20 (1), 1–18.

MathSciNet   Google Scholar  

Deng, Y., Kelly, G. J., & Deng, S. (2019). The influences of integrating reading, peer evaluation, and discussion on undergraduate students’ scientific writing. International Journal of Science Education, 41 (10), 1408–1433.

Desa, G., Howard, P. J., Gorzycki, M., & Allen, D. D. (2020). Essential but invisible: Collegiate academic reading explored from the faculty perspective. College Teaching, 68 (3), 126–137.

Erduran, S., Guilfoyle, L., & Park, W. (2022). Science and religious education teachers’ views of argumentation and its teaching. Research in Science Education, 52 (2), 655–673.

Ferron, J. M., Kirby, M., & Lipien, L. (2023). Single-case experimental design. In H. Cooper, M. N. Coutanche, L. M. McMullen, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology: Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 747–762). American Psychological Association.

Finocchiaro, M. A. (2021). Science, method, and argument in Galileo . Springer.

Book   Google Scholar  

Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32 (3), 221–233.

Article   CAS   PubMed   Google Scholar  

Gomez-Marin, A. (2023). Science in the age of podcasts. Science, 379 (6630), 338.

Gorzycki, M., Desa, G., Howard, P. J., & Allen, D. D. (2019). “Reading is important”, but “I don’t read”: Undergraduates’ experiences with academic reading. Journal of Adolescent & Adult Literacy, 63 (5), 499–508.

Griffiths, N., & Davila, Y. C. (2022). Embedding scaffolded reading practices into the first-year university science curriculum. In K. Manarin (Ed.), Reading across the disciplines (pp. 143–165). Indiana University Press.

Hairston-Dotson, K., & Incera, S. (2022). Critical reading: What do students actually do? Journal of College Reading and Learning, 52 (2), 113–129.

Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers’ implementation. Teaching and Teacher Education, 36 , 101–111.

Heiss, E. M., & Liu, Y. (2022). Use of primary literature in the classroom: Quantitative and instrumental analysis topics. In T. J. Wenzel, M. L. Kovarik, & J. K. Robinson (Eds.), Active learning in the analytical chemistry curriculum (pp. 107–120). American Chemical Society.

Hoskins, S. G., Stevens, L. M., & Nehm, R. H. (2007). Selective use of the primary literature transforms the classroom into a virtual laboratory. Genetics, 176 (3), 1381–1389.

Article   PubMed   PubMed Central   Google Scholar  

Hubbard, K. (2021). Disciplinary literacies in STEM: What do undergraduates read, how do they read it, and can we teach scientific reading more effectively? Higher Education Pedagogies, 6 (1), 41–65.

Hubbard, K. E., & Dunbar, S. D. (2017). Perceptions of scientific research literature and strategies for reading papers depend on academic career stage. PLoS ONE, 12 (12), e0189753.

Hubbard, K. E., Dunbar, S. D., Peasland, E. L., Poon, J., & Solly, J. E. (2022). How do readers at different career stages approach reading a scientific research paper? A case study in the biological sciences. International Journal of Science Education, Part B, 12 (4), 328–344.

Hunter, R. A., & Kovarik, M. L. (2022). Leveraging the analytical chemistry primary literature for authentic, integrated content knowledge and process skill development. Journal of Chemical Education, 99 (3), 1238–1245.

Iacumin, L., Colautti, A., & Comi, G. (2022). Zygosaccharomyces rouxii is the predominant species responsible for the spoilage of the mix base for ice cream and ethanol is the best inhibitor tested. Food Microbiology, 102 , 103929.

Idsardi, R. C., Luft, J. A., Wingfield, J. L., Whitt, B., Barriga, P. A., & Lang, J. D. (2023). Relationships between undergraduate instructors’ conceptions of how students learn and their instructional practices. Journal of Research in Science Teaching, 60 (9), 2076–2110.

Kampourakis, K. (2017). Science teaching in university science departments. Science & Education, 26 (3–4), 201–203.

Kararo, M., & McCartney, M. (2019). Annotated primary scientific literature: A pedagogical tool for undergraduate courses. PLoS Biology, 17 (1), e3000103.

Kerr, M. M., & Frese, K. M. (2017). Reading to learn or learning to read? Engaging college students in course readings. College Teaching, 65 (1), 28–31.

Khishfe, R. (2014). Explicit nature of science and argumentation instruction in the context of socioscientific issues: An effect on student learning and transfer. International Journal of Science Education, 36 (6), 974–1016.

Khishfe, R. (2021). Explicit instruction and student learning of argumentation and nature of science. Journal of Science Teacher Education, 32 (3), 325–349.

Khishfe, R. (2023a). Relationship between nature of science and argumentation: A follow-up study. International Journal of Science and Mathematics Education, 21 (4), 1081–1102.

Khishfe, R. (2023b). Connected learning: An approach for teaching nature of science aspects and argumentation components. Journal of Research in Science Teaching, 60 (10), 2180–2215.

Lammers, A., Goedhart, M. J., & Avraamidou, L. (2019). Reading and synthesising science texts using a scientific argumentation model by undergraduate biology students. International Journal of Science Education, 41 (16), 2323–2346.

Lee, S., Zhong, M., Foster, C., Segura-Totten, M., & McCartney, M. (2022). From novice to expert: An assessment to measure strategies students implement while learning to read primary scientific literature. Journal of Microbiology and Biology Education, 23 (3), 1–8.

Lemke, J. (1990). Talking science: Language, learning, and values . Ablex.

Lennox, R., Hepburn, K., Leaman, E., & van Houten, N. (2020). ‘I’m probably just gonna skim’: An assessment of undergraduate students’ primary scientific literature reading approaches. International Journal of Science Education, 42 (9), 1409–1429.

Lin, S.-S. (2014). Science and non-science undergraduate students’ critical thinking and argumentation performance in reading a science news report. International Journal of Science and Mathematics Education, 12 (5), 1023–1046.

Lu, J., & Law, N. (2012). Online peer assessment: Effects of cognitive and affective feedback. Instructional Science, 40 (2), 257–275.

Manarin, K. (Ed.). (2022). Reading across the disciplines . Indiana University Press.

Manarin, K., Carey, M., Rathburn, M., & Ryland, G. (2015). Critical reading in higher education . Indiana University Press.

Marchant, G. J. (2002). Student reading of assigned articles: Will this be on the test? Teaching of Psychology, 29 (1), 49–51.

Mazak, C. M., & Herbas-Donoso, C. (2015). Translanguaging practices at a bilingual university: A case study of a science classroom. International Journal of Bilingual Education and Bilingualism, 18 (6), 698–714.

Meli, K., Koliopoulos, D., & Konstantinos, L. (2022). A model-based constructivist approach for bridging qualitative and quantitative aspects in teaching and learning the first law of thermodynamics. Science & Education, 31 (2), 451–485.

Mercier, H. (2016). The Argumentative theory: Predictions and empirical evidence. Trends in Cognitive Sciences, 20 (9), 689–700.

Mertens, D. M. (2023). Mixed methods research . Bloomsbury.

Mizokami, S. (2018). Deep active learning from the perspective of active learning theory. In K. Matsushita (Ed.), Deep active learning (pp. 79–91). Springer.

Muench, S. B. (2000). Choosing primary literature in biology to achieve specific educational goals. Journal of College Science Teaching, 29 (4), 255–260.

Noroozi, O., Banihashem, S. K., Biemans, H. J. A., Smits, M., Vervoort, M. T. W., & Verbaan, C.-L. (2023). Design, implementation, and evaluation of an online supported peer feedback module to enhance students’ argumentative essay quality. Education and Information Technologies, 28 (10), 12757–12784.

Oliver, D. (2022). Pedagogical approaches for improving reading compliance and discussion in higher education classrooms. College Student Journal, 56 (2), 151–167.

Osborne, J. (2010). Arguing to learn in science: The role of collaborative, critical discourse. Science, 328 (5977), 463–466.

Article   ADS   CAS   PubMed   Google Scholar  

Oswald, S., Lewinski, M., Greco, S., & Villata, S. (Eds.). (2022). The pandemic of argumentation . Springer.

Palavalli-Nettimi, R., Fieldsend, T. W., Acosta, A., Catenazzi, A., & McCartney, M. (2022). Papers to podcasts: Curriculum for developing scientific practices in undergraduates through annotating primary scientific literature & creating podcasts. The American Biology Teacher, 84 (7), 428–437.

Pflugfelder, E. H. (2022). Evidence engines: Common rhetorical features of fraudulent academic articles. Written Communication, 39 (2), 303–331.

Raimondi, S. L., Marsh, T. L., & Guenther, M. F. (2020). Does repetition matter? analysis of biology majors’ ability to comprehend journal articles across a major. Journal of Microbiology & Biology Education, 21 (1), 1–14.

Reuse-Durham, N. (2005). Peer evaluation as an active learning technique. Journal of Instructional Psychology, 32 (4), 338–345.

Ritchey, K. A., & List, A. (2022). Task-oriented reading: A framework for improving college students’ reading compliance and comprehension. College Teaching, 70 (3), 280–295.

Russo, A., & Jankowski, A. (2023). It’s harder than it looks: How students identify original research articles in an undergraduate biology course. Public Services Quarterly, 19 (1), 1–25.

Rybarczyk, B. (2006). Incorporating primary literature into science learning. In J. J. Mintzes & W. H. Leonard (Eds.), Handbook of college science teaching (pp. 159–166). National Science Teachers Association.

Saldana, J. (2021). The coding manual for qualitative researchers (4th ed.). Sage.

Schmid, K. M., Dunk, R. D. P., & Wiles, J. R. (2021). Early exposure to primary literature and interactions with scientists influences novice students’ views on the nature of science. Journal of College Science Teaching, 50 (6), 40–47.

Segura-Totten, M., & Dalman, N. E. (2013). The CREATE method does not result in greater gains in critical thinking than a more traditional method of analyzing the primary literature. Journal of Microbiology & Biology Education, 14 (2), 166–175.

Slater, T. F. (2020). Active learning in astronomy: Learning goals, assessment, and class time. In J. J. Mintzes & E. M. Walter (Eds.), Active Learning in college science (pp. 803–819). Springer.

Smith, K. G., & Paradise, C. J. (2022). Teaching the process of science with primary literature: Using the CREATE pedagogy in ecological courses. Ecology and Evolution, 12 (12), e9644.

Spiegelberg, B. D. (2014). A focused assignment encouraging deep reading in undergraduate biochemistry. Biochemistry and Molecular Biology Education, 42 (1), 1–5.

Article   MathSciNet   CAS   PubMed   Google Scholar  

Stengel, A., Stanke, K. M., McPherson, M. R., & Drijber, R. A. (2021). Virtually engaging students through collaborative investigation of scientific literature, a case study. Natural Sciences Education, 50 (1), e20051.

Stockmeyer, N. O. (2009). Using Microsoft Word’s readability program. Michigan Bar Journal, 88 (1), 46–47.

Sutherland, A., & Incera, S. (2021). Critical reading: What do faculty think students should do? Journal of College Reading and Learning, 51 (4), 267–290.

Tanious, R., & Onghena, P. (2021). A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis. Behavior Research Methods, 53 (4), 1371–1384.

Theriault, J. C. (2022). College students’ conceptualizations of academic reading: What metaphors suggest about the important role of purpose in college reading and learning. Journal of Adolescent & Adult Literacy, 66 (1), 15–22.

Tomkins, A., Zhang, M., & Heavlin, W. D. (2017). Reviewer bias in single- versus double-blind peer review. Proceedings of the National Academy of Sciences of the United States of America, 114 (48), 12708–12713.

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Topping, K. J. (2018). Using peer assessment to inspire reflection and learning . Routledge.

Van Lacum, E., Koeneman, M., Ossevoort, M., & Goedhart, M. (2016). Scientific argumentation model (SAM): A heuristic for reading research articles by science students. In N. Papadouris, A. Hadjigeorgiou, & C. P. Constantinou (Eds.), Insights from research in science teaching and learning (pp. 169–183). Springer.

Van Lacum, E. B., Ossevoort, M. A., & Goedhart, M. J. (2014). A teaching strategy with a focus on argumentation to improve undergraduate students’ ability to read research articles. CBE-Life Sciences Education, 13 (2), 253–264.

Verkade, H., & Lim, S. H. (2016). Undergraduate science students’ attitudes toward and approaches to scientific reading and writing. Journal of College Science Teaching, 45 (4), 83–89.

Vroom, J. (2022). Research article introductions as hero narratives: A reading strategy for undergraduate students. Discourse and Writing/rédactologie, 32 , 48–58.

Wallace, M., & Wray, A. (2021). Critical reading and writing for postgraduates (4th) . Sage.

Walton, D. N. (2006). Fundamentals of critical argumentation . Cambridge University Press.

Wenk, L., & Tronsky, L. (2011). First-year students benefit from reading primary research articles. Journal of College Science Teaching, 40 (4), 60–67.

Wenzel, T. J., Kovarik, M. L., & Robinson, J. K. (2022). Preface. In T. J. Wenzel, M. L. Kovarik, & J. K. Robinson (Eds.), Active learning in the analytical chemistry curriculum (pp. ix–xi). American Chemical Society.

Wijayanti, E., & Adi, W. C. (2022). The ability to read research articles: Profile of students in the biology education programs’ of Uin Walisongo Semarang. Jurnal Pendidikan Matematika Dan IPA, 13 (2), 181–189.

Wilson, K. (2016). Critical reading, critical thinking: Delicate scaffolding in English for Academic Purposes (EAP). Thinking Skills and Creativity, 22 , 256–265.

Winstone, N., & Carless, D. (2020). Designing effective feedback processes in higher education . Routledge.

Wu, Y., & Schunn, C. D. (2023). Passive, active, and constructive engagement with peer feedback: A revised model of learning from peer feedback. Contemporary Educational Psychology, 73 , 1–14.

Xiang, L. (2022). Undergraduates’ & faculty members’ views on scientific reading & communication in authentic inquiry. The American Biology Teacher, 84 (3), 143–151.

Yeong, F. M. (2014). How to read and critique a scientific research article . World Scientific Publishing Co.

Download references

Acknowledgements

Our sincere gratitude to all the participating students for their time and feedback about the ALS reported in this article. This project was supported by funding from the Vice-Presidency of Research and Creation, Universidad de los Andes, Bogotá, Colombia.

Open Access funding provided by Colombia Consortium

Author information

Authors and affiliations.

Vice-Presidency of Research and Creation, Universidad de los Andes, Cra 1 Nº 18A-12, 111711, Bogotá, Colombia

Pablo Antonio Archila

Department of Chemical and Food Engineering, Universidad de los Andes, Cra 1 Nº 18A-12, 111711, Bogotá, Colombia

Brigithe Tatiana Ortiz

School of Education, Universidad de los Andes, Cra 1 Nº 18A-12, 111711, Bogotá, Colombia

Anne-Marie Truscott de Mejía

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Pablo Antonio Archila .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1. Questionnaire 1

1.1 part one: initial decision.

You consider that the quality of the Introduction section of Iacumin et al.’s ( 2022 ) article is…

Why did you make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

1.2 Part Two: Final Decision

Having reflected upon the written feedback provided by two peers, you consider that the quality of the Introduction section of Iacumin et al.’s ( 2022 ) article is…

Appendix 2. Questionnaire 2

2.1 part one: initial decision.

You consider that the quality of the Material and Methods section of Ayeni et al.’s ( 2011 ) article is…

2.2 Part Two: Small-group Decision

Your group consider that the quality of the Material and Methods section of Ayeni et al.’s ( 2011 ) article is…

Why did your group make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

2.3 Part Three: Final Decision

Having reflected upon the written feedback provided by two small groups, you consider that the quality of the Material and Methods section of Ayeni et al.’s ( 2011 ) article is…

Appendix 3. Survey 1

Apart from the Food Microbiology course, have you ever received instruction about how to read and critique scientific articles?

Did you have sufficient time to read Article 1?

Were the opportunities of giving written feedback to two peers useful for you to make a decision? Explain why or why not.

Were the written feedback provided by peers useful for you to make a decision? Explain why or why not.

How often do you have the opportunity to give written peer feedback in other university courses?

Very frequently.

Fairly frequently.

Infrequently.

Appendix 4. Survey 2

Did you have sufficient time to read Article 2?

Were the opportunities of providing written group feedback to two small groups useful for you to make a decision? Explain why or why not.

Was the written feedback given by small groups useful for you to make a decision? Explain why or why not.

How often do you have the opportunity to provide written peer group feedback in other university courses?

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Archila, P.A., Ortiz, B.T. & Truscott de Mejía, AM. Beyond the Passive Absorption of Information: Engaging Students in the Critical Reading of Scientific Articles. Sci & Educ (2024). https://doi.org/10.1007/s11191-024-00507-1

Download citation

Accepted : 06 February 2024

Published : 07 March 2024

DOI : https://doi.org/10.1007/s11191-024-00507-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

info This is a space for the teal alert bar.

notifications This is a space for the yellow alert bar.

Research Process

• Brainstorming
• Explore Google This link opens in a new window
• Explore Web Resources
• Explore Background Information
• Explore Books
• Explore Scholarly Articles
• Narrowing a Topic
• Primary and Secondary Resources
• Academic, Popular & Trade Publications
• Scholarly and Peer-Reviewed Journals
• Grey Literature
• Clinical Trials
• Evidence Based Treatment
• Scholarly Research
• Database Research Log
• Search Limits
• Keyword Searching
• Boolean Operators
• Phrase Searching
• Truncation & Wildcard Symbols
• Proximity Searching
• Field Codes
• Subject Terms and Database Thesauri

Reading a Scientific Article

• Website Evaluation
• Article Keywords and Subject Terms
• Cited References
• Citing Articles
• Related Results
• Search Within Publication
• Database Alerts & RSS Feeds
• Personal Database Accounts
• Persistent URLs
• Literature Gap and Future Research
• Web of Knowledge
• Annual Reviews
• Systematic Reviews & Meta-Analyses
• Finding Seminal Works
• Exhausting the Literature
• Finding Dissertations
• Researching Theoretical Frameworks
• Research Methodology & Design
• Tests and Measurements
• Organizing Research & Citations This link opens in a new window
• Scholarly Publication
• Learn the Library This link opens in a new window

Library Tutorial

• Reading a Scholarly Article Tutorial This interactive tutorial provides practice reading a scholarly or scientific article.

Additional Resources

• Anatomy of a Scholarly Article
• How to Read (and Understand) a Social Science Journal Article
• How to Read a Scientific Paper
• How to Read a Scientific Paper Interactive Tutorial
• How to Read Scientific Literature (YouTube Video)

General Dictionaries

• The American Heritage Dictionary of the English Language
• The American Heritage Student Science Dictionary
• The Chambers Dictionary
• Dictionary.com
• The Free Dictionary
• Merriam-Webster's Collegiate Dictionary
• Merriam-Webster Online
• The Penguin English Dictionary
• The Science Dictionary

Attempting to read a scientific or scholarly research article for the first time may seem overwhelming and confusing. This guide details how to read a scientific article step-by-step. First, you should not approach a scientific article like a textbook— reading from beginning to end of the chapter or book without pause for reflection or criticism. Additionally, it is highly recommended that you highlight and take notes as you move through the article. Taking notes will keep you focused on the task at hand and help you work towards comprehension of the entire article.

• Skim the article. This should only take you a few minutes. You are not trying to comprehend the entire article at this point, but just get a basic overview. You don’t have to read in order; the discussion/conclusions will help you to determine if the article is relevant to your research. You might then continue on to the Introduction. Pay attention to the structure of the article, headings, and figures.  
• Grasp the vocabulary. Begin to go through the article and highlight words and phrases you do not understand. Some words or phrases you may be able to get an understanding from the context in which it is used, but for others you may need the assistance of a medical or scientific dictionary. Subject-specific dictionaries available through our Library databases and online are listed below.  
• The abstract gives a quick overview of the article. It will usually contain four pieces of information: purpose or rationale of study (why they did it); methodology (how they did it); results (what they found); conclusion (what it means). Begin by reading the abstract to make sure this is what you are looking for and that it will be worth your time and effort.   
• The introduction gives background information about the topic and sets out specific questions to be addressed by the authors. You can skim through the introduction if you are already familiar with the paper’s topic.  
• The methods section gives technical details of how the experiments were carried out and serves as a “how-to” manual if you wanted to replicate the same experiments as the authors. This is another section you may want to only skim unless you wish to identify the methods used by the researchers or if you intend to replicate the research yourself.  
• The results are the meat of the scientific article and contain all of the data from the experiments. You should spend time looking at all the graphs, pictures, and tables as these figures will contain most of the data.  
• Lastly, the discussion is the authors’ opportunity to give their opinions. Keep in mind that the discussions are the authors’ interpretations and not necessarily facts. It is still a good place for you to get ideas about what kind of research questions are still unanswered in the field and what types of questions you might want your own research project to tackle. (See the Future Research Section of the Research Process for more information).  
•   Read the bibliography/references section. Reading the references or works cited may lead you to other useful resources. You might also get a better understanding of the basic terminology, main concepts, major researchers, and basic terminology in the area you are researching.  
• Have I taken time to understand all the terminology?
• Am I spending too much time on the less important parts of this article?
• Do I have any reason to question the credibility of this research?
• What specific problem does the research address and why is it important?
• How do these results relate to my research interests or to other works which I have read?  
• Read the article a second time in chronological order. Reading the article a second time will reinforce your overall understanding. You may even start to make connections to other articles that you have read on this topic.

Reading a Scholarly Article Workshop

This workshop presents effective techniques for reading and understanding a scholarly article, as well as locating definitions related to your research topic.

Subject-Specific Dictionaries

• Health Sciences
• Marriage & Family Science
• Research Methods
• Social Work

Was this resource helpful?

• << Previous: Subject Terms and Database Thesauri
• Next: Evaluating Information >>
• Last Updated: Apr 14, 2024 12:14 PM
• URL: https://resources.nu.edu/researchprocess

© Copyright 2024 National University. All Rights Reserved.

Privacy Policy | Consumer Information

Jump to navigation

• Learning Commons

College of DuPage Library

• Chat loading... Chat With Us -->

Go back to the Library's homepage

Catalog --> Catalog

Use the Catalog to find books, videos, e-books, and other media

Search for online journal and newspaper articles, e-books, and streaming video

Guides for finding and citing sources in many different subject areas

Learn about the Library's spaces and services

The COD Library and campus are closed Friday (Feb. 9) and Saturday (Feb. 10) due to weather conditions.

Coronavirus Updates & Closings

For the safety of the COD community, the Library will be closed from March 16 through April 19 . However, we are committed to supporting your learning and information needs through remote access to Library services and electronic collections . We are also compiling useful COVID-19 information sources to help keep you informed. You can keep up with COD's response to the coronavirus outbreak through the COD Coronavirus Information page . Last updated: March 15, 5:00 pm

How to Read a Research Article

Struggling to read your scientific scholarly article, even though it looks like it might be a perfect fit for your topic?

Try using the info below as a guidepost to help you understand the article. To begin, figure out if you're reading a Research Article or a Review Article.

Reading Research Articles

Start by looking for the distinctive markers of a scholarly article: are the authors' degrees or university affiliations listed? Do you see an abstract? How about charts, tables, graphs?

If you are using a scientific research article, you'll see the following distinctive sections:

• Abstract: a paragraph summary of the research question and findings
• Introduction: the research question: what did the scientists set out to know? Also provides context to the study: what did we know about the topic? Who answered the most important questions so far? Will include many citations.
• Method: the experiment design
• Results: The data gathered by the experiment
• Discussion: analyzes the results. What do we understand about the topic after the experiment has been conducted?
• Conclusion: lists further questions to be studied
• References or Works Cited: functions just as yours will. What research has been referenced throughout the paper?

Some of these sections may be merged with other sections, have slightly different names, be combined together (results and discussion often share a single section) or may not be labeled, but all should be present in one way or another.

Confused? Take a look at page one of a scholarly research article below:

• The authors list a university affiliation
• The abstract is right in the center of the page
• The (unmarked) introduction

Want to take a closer look? Cladophora (Chlorophyta) spp. Harbor Human Bacterial Pathogens in Nearshore Water of Lake Michigan is a research article found on PubMedCentral, the government-sponsored free article database. You can use this as a model scholarly research article.

• Remember to start with your abstract. The summary will tell you where the authors are heading and help you to fight through confusing sections.
• Try reading your article out of order! (No one said we have to follow the rules all the time, right?) Start with the abstract, and skim through the Introduction and the Conclusion (Don't see one? Read the Discussion instead.) Note the hypothesis and article findings. Then read the whole article, remembering that the Materials and Methods sections are often long and full of complex concepts.
• Be careful to be very conscious of whatever section you're reading, because that will tell you the types of info that you're reading: are you in Methods? If so, you're looking at experimental design. Are you looking through Results? If so, you're looking at the data that was gathered, etc., etc.
• Check out this handy book that discusses reading and critiquing scholarly articles.
• This article, "To understand a scientific paper, delve into its parts" by Bethany Brookshire (a working scientist) also does a good job of breaking down scientific articles. The second article, Four tips for reading a scientific paper , also offers great advice on how to deal with dense language, as well as important questions to ask about any article you read.
• Remember that you can use reference databases to explain words or concepts that you're unfamiliar with. Try searching Credo or Gale to start.
• E-mail page
• Send to phone

The Science of Reading Research

Understanding scientific evidence, what is scientific evidence, qualitative and quantitative research, evaluating research, appropriate methodologies, peer review, converging evidence, practical application, what scientific research says about reading, how does reading develop, how can we prevent reading failure, a systemwide response to reading failure.

Classroom observations under the best of circumstances (systematic and reliable observers) do not even permit generalization to other classrooms. (2004, p. 54)

The front line of defense for teachers against incorrect information in education is the existence of peer- reviewed journals in education, psychology, and other related social sciences. (Stanovich & Stanovich, 2003, p. 7)

Adams, M. J. (1990). Beginning to read: Thinking and learning about print . Cambridge: MIT Press.

Fletcher, J. M., & Lyon, G. R. (1998). Reading: A research-based approach. In W. Evers (Ed.), What's gone wrong in America's classrooms (pp. 49–90). Stanford, CA: Hoover Institute Press.

Hart, B., & Risley, T. R. (2003). The early catastrophe: The 30 million word gap. American Educator, 27 (1), 4–9.

Hersch, E. D. (2001). Overcoming the language gap. American Educator, 25 (2), 4–7.

Kennedy, M. M. (1997). The connection between research and practice. Educational Researcher, 26 , 4–12.

Liberman, A. M. (1997). Why is speech so much easier than reading? In H. C. Joshi & R. M. Joshi (Eds.), Reading and spelling: Development and disorders . Mahwah, NJ: Erlbaum.

Lonigan, C. J. (2003). Development and promotion of emergent literacy skills in children at risk of reading difficulties. In B. R. Foorman (Ed.), Preventing and remediating reading difficulties: Bringing science to scale (pp. 23–50). Baltimore: York Press.

Lyon, G. R. (1998, March). Why reading is not a natural process. Educational Leadership , 14–18.

Lyon, G. R. (2002). Reading development, reading difficulties, and reading instruction: Educational and public health issues. Journal of School Psychology, 40 , 3–6.

Lyon, G. R., Fletcher, J. M., Shaywitz, S. E., Shaywitz, B. A., Torgesen, J. K., Wood, F. B., Shulte, A., & Olson, R. (2001). Rethinking learning disabilities. In C. E. Finn, R. A. J. Rotherham, & C. R. Hokanson (Eds.), Rethinking special education for a new century (pp. 259–287). Washington, DC: Thomas B. Fordham Foundation & Progressive Policy Institute.

McCardle, P., & Chhabra, V. (2004). The voice of evidence in reading research . Baltimore: Brookes.

Moats, L. C. (1995). The missing foundation in teacher preparation. American Educator, 19 (9), 43–51.

Moats, L. C. (1999). Teaching reading is rocket science . Washington, DC: American Federation of Teachers.

National Center for Education Statistics. (2003). National assessment of educational progress: The nation's report card . Washington, DC: U.S. Department of Education.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Reports of the subgroups . Washington, DC: National Institute of Child Health and Human Development.

Partnership for Reading. (2003). Put reading first: The research building blocks for teaching children to read. Kindergarten through grade 3 . Washington, DC: Author.

Ravid, R. (1994). Practical statistics for educators . Lanham, MD: University Press of America.

Rayner, K., Foorman, B. R., Perfetti, C. A., Pesetsky, D., & Seidenberg, M. S. (2001). How psychological science informs the teaching of reading. Psychological Science in the Public Interest, 2 (2), 31–74.

Reyna, V. (2004). Why scientific research? The importance of evidence in changing educational practice. In P. McCardle & V. Chhabra (Eds.), The voice of evidence in reading research . Baltimore: Brookes.

Shavelson, R. J., & Towne, L. (2002). Scientific research in education . Washington, DC: National Academies Press.

Shaywitz, S. E. (2003). Overcoming dyslexia . New York: Knopf.

Snow, C., Burns, S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children . Washington, DC: National Academy Press.

Stanovich, P., & Stanovich, K. (2003). Using research and reason in education . Washington, DC: Partnership for Reading.

Torgesen, J. K. (2002a). The prevention of reading difficulties. Journal of School Psychology, 40 (1), 7–26.

Torgesen, J. K. (2002b). Lessons learned from intervention research in reading: A way to go before we rest. In R. Stainthorpe (Ed.), Literacy: Learning and teaching . London: British Psychological Association.

Whitehurst, G. (2001). Cognitive development during the preschool years . Paper presented at the White House Summit on Early Childhood Cognitive Development. Washington, DC: U.S. Department of Education.

Wiersma, W. (2000). Research methods in education: An introduction (7th ed.). Boston: Allyn and Bacon.

Author bio coming soon

ASCD is a community dedicated to educators' professional growth and well-being.

Let us help you put your vision into action., related articles.

STEM Doesn’t Have to Be Rocket Science

Reimagining Mathematics to Save the World

The Hidden Rigors of Data Science

Picture Books Aren’t Just for the Youngest Students

Transforming STEM by Focusing on Justice

From our issue.

To process a transaction with a Purchase Order please send to [email protected]

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

4 Reasons Why Managers Fail

• Swagatam Basu,
• Atrijit Das,
• Vitorio Bretas,
• Jonah Shepp

Nearly half of all managers report buckling under the stress of their role and struggling to deliver.

Gartner research has found that managers today are accountable for 51% more responsibilities than they can effectively manage — and they’re starting to buckle under the pressure: 54% are suffering from work-induced stress and fatigue, and 44% are struggling to provide personalized support to their direct reports. Ultimately, one in five managers said they would prefer not being people managers given a choice. Further analysis found that 48% of managers are at risk of failure based on two criteria: 1) inconsistency in current performance and 2) lack of confidence in the manager’s ability to lead the team to future success. This article offers four predictors of manager failure and offers suggestions for organizations on how to address them.

The job of the manager has become unmanageable. Organizations are becoming flatter every year. The average manager’s number of direct reports has increased by 2.8 times over the last six years, according to Gartner research. In the past few years alone, many managers have had to make a series of pivots — from moving to remote work to overseeing hybrid teams to implementing return-to-office mandates.

• Swagatam Basu is senior director of research in the Gartner HR practice and has spent nearly a decade researching leader and manager effectiveness. His work spans additional HR topics including learning and development, employee experience and recruiting. Swagatam specializes in research involving extensive quantitative analysis, structured and unstructured data mining and predictive modeling.
• Atrijit Das is a senior specialist, quantitative analytics and data science, in the Gartner HR practice. He drives data-based research that produces actionable insights on core HR topics including performance management, learning and development, and change management.
• Vitorio Bretas is a director in the Gartner HR practice, supporting HR executives in the execution of their most critical business strategies. He focuses primarily on leader and manager effectiveness and recruiting. Vitorio helps organizations get the most from their talent acquisition and leader effectiveness initiatives.
• Jonah Shepp is a senior principal, research in the Gartner HR practice. He edits the Gartner  HR Leaders Monthly  journal, covering HR best practices on topics ranging from talent acquisition and leadership to total rewards and the future of work. An accomplished writer and editor, his work has appeared in numerous publications, including  New York   Magazine ,  Politico   Magazine ,  GQ , and  Slate .

Partner Center

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Almost 1 in 5 stay-at-home parents in the U.S. are dads

Pew Research Center conducted this analysis to see how many mothers and fathers in the United States are not employed for pay and how stay-at-home parents are different from working parents. The analysis uses the  Annual Social and Economic Supplement (ASEC) of the Current Population Survey (CPS), which is conducted in March of every year.

Administered jointly by the U.S. Census Bureau and the Bureau of Labor Statistics, the CPS is a monthly survey of approximately 70,000 households . It is the source of the nation’s official statistics on unemployment. The ASEC survey typically features an expanded sample of about 95,000 households.

Parents are people ages 18 to 69 who live with at least one of their own children (biological, step or adopted) younger than 18. Stay-at-home parents are those who were not employed for pay at all in the calendar year prior to the survey.

The population of stay-at-home fathers who are caring for home or family is relatively small, so we combined the 2020, 2021 and 2022 ASEC files to analyze their characteristics.

The share of parents in the United States who are not employed for pay has been fairly stable over the last five years. In 2021, 18% of parents didn’t work for pay, which was unchanged from 2016, according to a new Pew Research Center analysis of U.S. Census Bureau data. The share who are stay-at-home parents differs between men and women: 26% of mothers and 7% of fathers.

Over the past 30 years, the share of stay-at-home parents has fluctuated, rising during periods of higher unemployment.

How stay-at-home dads and moms compare

Between 1989 and 2021, the share of mothers who were not employed for pay decreased slightly, from 28% to 26%. Over the same span, the share of fathers who were not working increased from 4% to 7%.

Due to these diverging trends, dads now represent 18% of stay-at-home parents, up from 11% in 1989.

The reasons mothers and fathers give for not working for pay differ significantly. In 2021, the vast majority of stay-at-home moms (79%) said they took care of the home or family. About one-in-ten (9%) said they were at home because they were ill or disabled, and smaller shares said they didn’t work because they were students, unable to find work or retired.

Stay-at-home dads cite more varied reasons for not working for pay. In 2021, 23% stayed home to care for the home or family. That is up from only 4% in 1989 but still well below the share of stay-at-home moms who said the same.

About one-third of stay-at-home dads (34%) were not working due to illness or disability, down from 56% in 1989. Some 13% were retired, 13% said they could not find work and 8% were going to school.

How stay-at-home dads are different from dads who work

Stay-at-home dads differ demographically from dads who work for pay.

• Education: Stay-at-home dads are less likely than dads working for pay to have completed at least a bachelor’s degree. Some 22% of stay-at-home dads have this level of education, compared with 42% of dads who work for pay.
• Poverty: The families of stay-at-home dads tend to be less economically well-off than the families of dads who work for pay. Some 40% of stay-at-home dads live in poverty, compared with 5% of dads who work for pay.
• Age: Stay-at-home dads tend to be older than dads working for pay. Some 46% of stay-at-home dads are age 45 or older, compared with 35% of dads working for pay.
• Race and ethnicity: Stay-at-home dads are a more racially and ethnically diverse group than working dads. Half of dads who don’t work for pay are non-Hispanic White. This compares with 60% of dads working for pay. Non-Hispanic Black fathers are a larger share of stay-at-home dads (18%) than they are of working dads (9%). Hispanic fathers are 21% of both stay-at-home and working fathers, and non-Hispanic Asian fathers are 7% of stay-at-home fathers and 8% of working fathers.
• Marriage: Some 68% of stay-at-home dads are married, as are 85% of dads who work for pay.

In addition, stay-at-home dads who are taking care of the home or family differ in some ways from those who stay home for other reasons.

• Education: 27% of dads who stay home to care for family have a four-year college degree, while 21% of dads who stay home for other reasons do.
• Age: A third of dads staying home to take care of family are age 45 or older, compared with half of those who are home for other reasons.
• Marriage: 73%of dads who stay home to care for family are married, as are 66% of dads who are home for other reasons.

Note: This is an update of a post originally published Sept. 24, 2018, written by former Senior Researcher Gretchen Livingston.

• Family Caregiving
• Household Structure & Family Roles
• Motherhood & Fatherhood

Parents, Young Adult Children and the Transition to Adulthood

More than half of americans in their 40s are ‘sandwiched’ between an aging parent and their own children, financial issues top the list of reasons u.s. adults live in multigenerational homes, how the political typology groups compare, more than one-in-ten u.s. parents are also caring for an adult, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

• International edition
• Australia edition
• Europe edition

Chinese students in US tell of ‘chilling’ interrogations and deportations

As tensions with China rise, scientists at America’s leading universities complain of stalled research after crackdown at airports

Stopped at the border, interrogated on national security grounds, laptops and mobile phones checked, held for several hours, plans for future research shattered.

Many western scholars are nervous about travelling to China in the current political climate. But lately it is Chinese researchers working at US universities who are increasingly reporting interrogations – and in several cases deportations – at US airports, despite holding valid work or study visas for scientific research.

Earlier this month the Chinese embassy in Washington said more than 70 students “with legal and valid materials” had been deported from the US since July 2021, with more than 10 cases since November 2023. The embassy said it had complained to the US authorities about each case.

The exact number of incidents is difficult to verify, as the US Customs and Border Protection (CBP) agency does not provide detailed statistics about refusals at airports. A spokesperson said that “all international travellers attempting to enter the United States, including all US citizens, are subject to examination”.

But testimonies have circulated on Chinese social media, and academics are becoming increasingly outspoken about what they say is the unfair treatment of their colleagues and students.

“The impact is huge,” says Qin Yan, a professor of pathology at Yale School of Medicine in Connecticut, who says that he is aware of more than a dozen Chinese students from Yale and other universities who have been rejected by the US in recent months, despite holding valid visas. Experiments have stalled, and there is a “chilling effect” for the next generation of Chinese scientists.

The number of people affected is a tiny fraction of the total number of Chinese students in the US. The State Department issued nearly 300,000 visas to Chinese students in the year to September 2023. But the personal accounts speak to a broader concern that people-to-people exchanges between the world’s two biggest economies and scientific leaders are straining.

The refusals appear to be linked to a 2020 US rule that barred Chinese postgraduate students with links to China’s “military-civil fusion strategy”, which aims to leverage civilian infrastructure to support military development. The Australian Strategic Policy Institute thinktank estimates that 95 civilian universities in China have links to the defence sector.

Nearly 2,000 visas applications were rejected on that basis in 2021 . But now people who pass the security checks necessary to be granted a visa by the State Department are being turned away at the border by CBP, a different branch of government.

“It is very hard for a CBP officer to really evaluate the risk of espionage,” said Dan Berger, an immigration lawyer in Massachusetts, who represents a graduate student at Yale who, midway through her PhD, was sent back from Washington’s Dulles airport in December, and banned from re-entering the US for five years.

“It is sudden,” Berger said. “She has an apartment in the US. Thankfully, she doesn’t have a cat. But there are experiments that were in progress.”

Academics say that scrutiny has widened to different fields – particularly medical sciences – with the reasons for the refusals not made clear.

X Edward Guo, a professor of biomedical engineering at Columbia University, said that part of the problem is that, unlike in the US, military research does sometimes take place on university campuses. “It’s not black and white … there are medical universities that also do military. But 99% of those professors are doing biomedical research and have nothing to do with the military.”

But “if you want to come to the US to study AI, forget it,” Guo said.

after newsletter promotion

One scientist who studies the use of artificial intelligence to model the impact of vaccines said he was rejected at Boston Logan International airport. He was arriving to take up a place at Harvard Medical School as a postdoctoral researcher. “I never thought I would be humiliated like this,” he wrote on the Xiaohongshu app, where he recounted being quizzed about his masters’ studies in China and asked if he could guarantee that his teachers in China had not passed on any of his research to the military.

He did not respond to an interview request from the Observer . Harvard Medical School declined to confirm or comment on the specifics of individual cases, but said that “decisions regarding entry into the United States are under the purview of the federal government and outside of the school’s and the university’s jurisdiction.”

The increased scrutiny comes as Beijing and Washington are struggling to come to an agreement about the US-China Science and Technology Agreement , a landmark treaty signed in 1979 that governs scientific cooperation between the two countries. Normally renewed every five years, since August it has been sputtering through six-month extensions.

But following years of scrutiny from the Department of Justice investigation into funding links to China, and a rise in anti-Asian sentiment during the pandemic, ethnically Chinese scientists say the atmosphere is becoming increasingly hostile.

“Before 2016, I felt like I’m just an American,” said Guo, who became a naturalised US citizen in the late 1990s. “This is really the first time I’ve thought, OK, you’re an American but you’re not exactly an American.”

Additional research by Chi Hui Lin

• US national security
• The Observer
• Asia Pacific
• US politics
• US justice system
• US immigration
• US foreign policy

Most viewed

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Perceptions of scientific research literature and strategies for reading papers depend on academic career stage

Katharine e. hubbard.

Department of Plant Sciences, University of Cambridge, Cambridge, United Kingdom

Sonja D. Dunbar

Associated data.

All relevant data are within the paper and its Supporting Information files.

Reading primary research literature is an essential skill for all scientists and students on science degree programmes, however little is known about how researchers at different career stages interact with and interpret scientific papers. To explore this, we conducted a survey of 260 undergraduate students and researchers in Biological Sciences at a research intensive UK university. Responses to Likert scale questions demonstrated increases in confidence and skill with reading the literature between individuals at each career stage, including between postdoctoral researchers and faculty academics. The survey indicated that individuals at different career stages valued different sections of scientific papers, and skill in reading the results section develops slowly over the course of an academic career. Inexperienced readers found the methods and results sections of research papers the most difficult to read, and undervalued the importance of the results section and critical interpretation of data. These data highlight a need for structured support with reading scientific literature at multiple career stages, and for senior academics to be aware that junior colleagues may prioritise their reading differently. We propose a model for the development of literature processing skills, and consider the need for training strategies to help inexperienced readers engage with primary literature, and therefore develop important skills that underpin scientific careers. We also encourage researchers to be mindful of language used when writing papers, and to be more inclusive of diverse audiences when disseminating their work.

Introduction

Engaging with the scientific literature is a key skill for researchers and students on scientific degree programmes; it has been estimated that scientists spend 23% of total work time reading [ 1 , 2 ]. The number of papers an individual scientist reads annually increased from 188 to 280 between 1993 and 2005, while total time spent only increased marginally [ 3 ]. Scientific writing is characterised by highly specialist vocabulary, concise and precise use of language, often accompanied by complex grammatical structures [ 4 , 5 ]. Making sense of scientific papers can be therefore cognitively challenging, particularly for readers who may be unfamiliar with the terminology of the field [ 6 ]. This challenge is faced by undergraduate students and early career scientists, but may also be encountered by experienced researchers exploring the literature in another discipline. We currently have a relatively poor understanding of how skills relating to the processing of scientific text develop through academic careers, the potential barriers to engaging with technical documentation and subsequent impact on the development of disciplinary or interdisciplinary research activities.

Most university level science courses require undergraduate students to engage with primary research literature, which is not generally used in school level education. In Biological Sciences, research literature is typically introduced early on or part way through the programme of study, with an expectation that students will be highly engaged with the literature in their final year of study. For example, the Quality Assurance Agency for UK degrees states that Biosciences graduates should have “ the ability to read and use appropriate literature with a full and critical understanding , while addressing such questions as content , context , aims , objectives , quality of information , and its interpretation and application .” [ 7 ]. Postgraduate students are generally expected to be fully engaged in primary literature throughout their graduate programmes; some postgraduate training programmes also include support for reading and interpreting literature, but many institutions assume this is a skill developed at undergraduate level. Some disciplines (e.g. theoretical physics) rely less heavily on primary research papers at undergraduate level due to their complexity, but all early career researchers will encounter scientific literature at some point in their training.

There have been many strategies and supporting resources developed to help inexperienced readers engage with the primary literature [ 6 , 8 – 14 ]. Undergraduates using these approaches have significantly increased performance in critical thinking tests, and report increased interest in primary research when interviewed after being taught specific reading strategies [ 11 , 15 ]. Small group journal clubs, with dedicated academic mentors, were shown to increase both scientific literacy and confidence in communicating scientifically with academic colleagues, ultimately facilitating transition to postgraduate study [ 16 ]. It is suggested that such strategies could help alleviate disengagement with science and prevent students dropping out of STEM subjects, particularly those from underrepresented backgrounds [ 11 ].

Many of these ‘strategy’ papers rely on anecdotal evidence that undergraduates adopt superficial reading strategies and lack analytical skills [ 17 ], and make a tacit assumption that more experienced researchers read the literature ‘correctly’. Faculty members are assumed to be ‘experts’ possessing both good content knowledge and science processing skills [ 18 ]. The skills that distinguish experts from novices stem from having a conceptual framework that allows experts to organise content effectively, combine it with relevant knowledge easily and recognise meaningful patterns within the information presented [ 19 ]. While the distinction between novice and expert is clear, the career stage at which the transition to expert is completed is broadly unknown.

Here we present results of a survey of researchers and students within Biological Sciences at a single research-intensive institution in the UK. This study aimed to begin an exploration of attitudes towards reading the literature, and start to uncover how readers at different career stages approached scientific papers. To explore this with participants who are likely to read a large number of scientific papers, the survey was administered at the University of Cambridge (UK), which has high academic entry standards for both undergraduate and postgraduate students. Survey participants include undergraduate students, PhD students, postdoctoral researchers and academic researchers, representing a wide range of career stages.

Survey design, implementation and ethics

A pilot survey of both students and researchers was designed to answer the following lines of enquiry for both students and researchers:

• How frequently did participants read scientific papers?
• How did participants feel about reading scientific papers?
• How easy to read did participants find different sections of papers?
• How important did participants think different sections of papers were?
• [Students only] How much training had they had in reading papers?
• [Researchers only] What advice would they give to someone reading a paper for the first time.

As a result of the pilot study, a full survey was designed to follow similar lines of inquiry. Where appropriate, this survey asked about primary research papers and review papers separately. This survey also included questions about the search engines used to find papers as an additional line of enquiry. Two versions of the full survey were designed, one for undergraduate students and one for postgraduate researchers. Questions were either direct replicas, designed to be mirrored where appropriate, or were unique to one of the two groups (see S1 and S2 Files for the text of the questionnaires). An optional ‘teaching’ section was added to the researchers survey to explore approaches to training students to read the literature, but the results of this section are outside the scope of the current study and are not included here. The results presented here are collected from the full survey, and do not include the results of the initial pilot questionnaire.

The undergraduate survey was administered at the end of the academic year (i.e. after examinations) to 2nd year and 3rd (final) year undergraduates studying biological sciences courses on the Natural Sciences programme. These courses included Biochemistry and Molecular Biology, Cell and Developmental Biology, Ecology, Genetics, Neuroscience, Pathology, Physiology, Plant Sciences, Psychology and Zoology. The researcher survey was administered at the same time to all researchers in biological sciences departments, which included Biochemistry, Genetics, Pathology, Physiology Development and Neuroscience, Plant Sciences, Psychology and Zoology. All participants were invited to participate via a weblink taking them to the online survey, which was hosted by www.surveymonkey.com .

No appropriate institutional ethics committee for educational research was in place at the time of the study being completed, however the study was designed in accordance with BERA (British Educational Research Association) ethical guidelines. At the start of the questionnaire, participants were presented with a short description of the study and the way in which their data would be stored and used. At the end of the survey participants were asked for their informed consent to their answers being used in publications resulting from the work, and to provide their email address if they were willing to participate in a follow-up interviews. Email addresses were removed from spreadsheets prior to data analysis, and responses remained anonymous throughout.

Quantitative analysis

Responses to Likert Scale questions were converted to numbers from 1–6 for statistical analysis, with 1 representing the most negative response to the question (e.g. Strongly disagree) and 6 the most positive (e.g. Strongly agree). The ordinal nature of the questionnaire data meant that parametric statistics were inappropriate for analysis, so non-parametric tests are used throughout with alpha = 0.95. Observations were independent, with no individuals belonging to more than one career group. For comparisons of multiple groups Kruskal-Wallis H tests were used, followed by pairwise Mann-Whitney U posthoc tests between particular groups where appropriate. Actual P values are reported throughout, except where P < 0.001. Statistical significance is indicated by asterisks throughout the manuscript; ** indicates P<0.01, * indicates P<0.05. For analysis of differences in responses between disciplines participants were grouped as ‘Molecular’, ‘Ecological’ or ‘Other’ on the basis of their reported subject or research area.

Qualitative analysis

For the question If you could only give one piece of advice to someone reading a scientific paper for the first time, what would it be?” responses were collected as free text. Responses were copied directly into NVivo and then assigned nodes to represent initial coding. Nodes were then aggregated into major themes, and the number of mentions of each theme determined. Themes or subthemes with fewer than 5 mentions out of the 88 responses are not presented for clarity.

Word clouds were generated using the ‘wordcloud’ package in R. Free text responses to the question were compiled into a.txt file, converted to lowercase and punctuation marks removed. Commonly used words in English were removed, along with the words ‘read’, ‘reading’ and ‘papers’. Words being mentioned fewer than 5 times are not presented for clarity, and colours used do not represent any formal analysis.

We obtained 300 responses to the survey, 260 of which completed responses to all questions. Of those who answered at least one question in the survey, participants included undergraduates in their second ( n = 81) and third (final) year of study ( n = 66), PhD students ( n = 53), PostDoctoral researchers ( n = 68) and Academics ( n = 49). Participants came from a range of disciplines within biological sciences, including biochemistry, physiology, ecology and mathematical biology (see S4 File for detailed breakdown). For the undergraduate survey, response rates were 47% for second years and 42% for third years. We were unable to determine response rates for the research survey as we did not have access to the total number of researchers employed by the university.

Perceptions of the research literature changes throughout academic careers

To explore the relationship that participants had with the literature, we asked them to what extent they agreed with a series of statements on a 6 point Likert scale ( Fig 1 ). The statement with the highest level agreement with all career stages was ‘Reading research papers is important for my general scientific training’, where only 10 respondents had any level of disagreement with the statement. The statement that was most disagreed with was the negatively worded question ‘Reading research papers is frustrating’, where 112 of the 300 participants disagreed (37%). Interestingly, 42% of academics agreed that they found reading papers frustrating, indicating that even experienced readers encountered difficulties in engaging with the primary research literature.

Responses to the 6 point Likert scale questions for 2nd year undergraduate n = 79; 3rd year undergraduate n = 42; PhD student n = 44; PostDoctoral researcher n = 55; Academic n = 43. Statistics alongside straight lines indicate the results of Kruskal-Wallis H tests (Kruskal-Wallis H and P values presented, degrees of freedom = 4 in all cases). Square brackets indicate significant Mann-Whitney tests for differences between groups, ** indicates P<0.01, * indicates P<0.05.

For all 8 statements there was a significant difference in the level agreement with the statement across the five career stages, with attitudes towards the literature generally becoming more positive with each subsequent stage. For the statement ‘I enjoy reading research papers’, attitudes became more positive a function of experience; 72% (57 out of 79) of 2nd year students agreed with the statement to some extent, whereas 95% of academics (41 out of 43) agreed that they enjoyed reading research papers. While this difference across all five stages was significant (Kruskal-Wallis H = 28.1, d.f. = 4, P < 0.001**), there were no significant differences between subsequent stages (e.g. PhD students were not significantly more likely to agree with the statement than 3rd year undergraduates), suggesting increasing enjoyment of the literature was not associated with particular career transitions. Undergraduates generally agreed that reading research papers was important for understanding material covered in lectures. 90% of 3 rd year undergraduates agreed that reading research papers was important for success in their end of year exams, but only 45% of 2 nd years agreed with this statement. In contrast, there was no difference in responses to these questions as a function of biological discipline (e.g. ‘I enjoy reading papers’ H = 0.68, d.f. = 2, P = 0.71; ‘I am confident in reading research papers without guidance’ H = 0.561. d.f. = 2, P = 0.561), indicating that ease with the literature not significantly influenced by disciplinary conventions.

The level of agreement with the statements increased significantly between sequential career stages, indicating that some or all career transitions were important in developing a relationship with the literature, not just at the undergraduate to postgraduate stage. For example, for the statement ‘I know how to identify papers that are of critical importance to the area I am investigating’, there was a significant increase in the level of agreement with the statement at every career stage, including between PostDoctoral Researchers and Academics (Mann-Whitney U = 866.5, P = 0.013*). A similar pattern was present for the statement ‘I know how to read research papers to extract information efficiently’, where PostDocs and Academics also had significantly different levels of agreement (U = 758.5, P = 0.001**), although for this statement there were no differences between 3rd year undergraduates and PhD students (U = 859.0, P = 0.557).

Perceptions of the literature may be shaped by the level of familiarity participants had with scientific papers. We therefore determined how many papers participants at different career stages were reading. All researchers and 3rd year undergraduates reported reading primary research papers, and only 7% of 2nd years said that they had not read any research papers, meaning that the vast majority of participants were engaging with primary literature from at least one source. More senior researchers were more likely to report reading multiple papers per week, with many academics reporting reading multiple research papers per day.

Undergraduates and early career researchers find methods and results sections the most difficult to understand

To explore where the challenges of engaging with the literature were in more detail, we asked survey participants “How easy do you usually find it to understand the following aspects of a research paper?”, with responses being recorded on a 6 point Likert scale. The figures and tables of the results section were considered in a separate question to the text-based description of the results to allow these aspects to be considered independently.

Nearly all participants considered the Abstract and Introduction sections to be relatively easy to read, with around 90% of all participants of all career stages describing these sections as easy to read (sum of responses to ‘Somewhat easy’, ‘Easy’ or ‘Very Easy’) ( Fig 2A ). However, there were highly significant differences in the frequencies of academics at different career stages describing the Methods, Results and Discussion sections as easy to read (Methods χ 2 (4) = 59.6, P < 0.001**; Figures χ 2 (4) = 23.6, P <0.001**; Results χ 2 (4) = 16.0, P = 0.003**; Discussion χ 2 (4) = 9.9, P = 0.041*). Fewer than half of undergraduates described the Materials and Methods section as being easy to read (2nd years 29%; 3rd years 24%), in contrast to PhD students and senior researchers (PhD students 81%; PostDocs 76%; Academics 83%). There was a very large shift in responses between 3rd year undergraduates and PhD students, but no significant difference in response was seen for any other career transition. For the results section (both the text and the figures/tables) there were increases in the proportion considering the section easy to read across all career stages. This was particularly striking for Figures and Tables, where 61% of academics described these as being ‘Very Easy’ to understand, compared with only 9% of PostDoctoral researchers.

A: The proportion of participants considering a section easy to read (presented as ‘Somewhat easy’, ‘easy’ ‘very easy’ combined) as a function of career stage. Results of Chi-square tests are indicated on the left hand side. B: The mean importance rank of sections as a function of career stage. Error bars are omitted from individual points for clarity, with the sole error bar in grey representing the largest 95% confidence interval for any of the data points. Asterisks above data points indicate significant differences in response compared with the previous career stage as determined by Mann-Whitney post-hoc tests.

Senior researchers place higher value on results and methods sections than those starting their academic careers

To explore the value that researchers placed on different sections of papers, we asked participants to rank the 6 sections in order of how important they thought that section was for understanding the paper, thereby forcing participants to make a relative value judgement ( Fig 2B ). There were significant differences in the rank given as a function of career stage for all sections with the exception of the introduction section, which was consistently regarded as relatively unimportant. The Abstract was seen as the most important section by undergraduates, whereas the Figures and Tables were ranked as most important to postgraduate researchers. The relative importance rank of the abstract decreased significantly as the level of seniority increased (H = 24.36, d.f. = 4, P <0.001**), with a similar pattern for the discussion (H = 31.75, d.f. = 4, P <0.001*). In contrast, the methods section was rated as the least important by undergraduates, but this section significantly increased in relative importance as a function of career stage (H = 43.77, d.f. = 4, P <0.001**), with academics considering this the 3rd most important section on average. The relative importance of both the text and figures/tables of the results also increased across the career span to become the two most highly ranked sections, with the Figures and Tables being ranked as more important than the text of the Results by all groups. There was therefore a major shift in the relative importance placed on different aspects of papers whereby undergraduate students ranked the abstract and discussion as being the most important sections, but senior researchers saw these as relatively unimportant for understanding the paper. In contrast, experienced researchers saw the Figures and Tables and Results as being the most important sections, but these were relatively undervalued by undergraduates ( Fig 2B ).

Researchers at different career stages read papers for different purposes

Scientific researchers read papers for many different reasons, which may shape the relative importance placed on different sections of a manuscript. To explore this, we asked participants to what extent they agreed with a series of 5 statements about the purpose of reading scientific papers, with undergraduates being presented with a further 2 statements relating to the use of papers in relation to their studies. For all questions there were statistically significant differences in the levels of agreement with the statement between respondents at different career stages ( Fig 3 ). The most significant responses were to the statements ‘[I read primary research papers to] critically evaluate the data’ and ‘understand the research methods used’, where undergraduates were significantly less likely to agree with the statement than researchers (Critically evaluate H = 134.36, d.f. = 4, P <0.001*; Research methods H = 127.61, d.f. = 4, P < 0.001*). Interestingly there was also a significant difference in the extent to which PostDocs and Academics agreed with the statement about critical evaluation (U = 715.0, P < 0.001**). 72% of 2nd year undergraduates agreed to some extent with the statement ‘[I read primary research papers to] find examples to include in exam answers’, but only 28% of this group agreed with the statement about critical evaluation. For final year undergraduates this increased to 92% agreement about examples for exams, and 76% for critical evaluation. There were also differences between the sources of papers being read frequently by the participants at different career stages; 16% of 2nd year undergraduates reported reading papers they had found themselves more than once a week, compared with 41% of PhD students and 63% of academics.

Responses to the 6 point Likert scale questions for 2nd year undergraduate n = 79; 3rd year undergraduate n = 42; PhD student n = 44; PostDoctoral researcher n = 55; Academic n = 43. Statistics alongside straight lines indicate the results of Kruskal-Wallis tests (H and P values presented, degrees of freedom = 4 in all cases). Square brackets indicate significant Mann-Whitney tests for differences between groups, ** indicates P<0.01, * indicates P<0.05.

Experienced researchers recommend a selective approach to scientific reading

At the end of the survey, we asked the researchers “If you could only give one piece of advice to someone reading a scientific paper for the first time, what would it be?”, and obtained 88 responses from the 142 participants. To gain an initial impression of the responses, the most commonly used words in the text of the advice given were determined ( Fig 4A ). Removing the words ‘paper’,‘read’ and ‘reading’, the most commonly used words in the responses were results (n = 28), figures (n = 26) and abstract (n = 25), reflecting the emphasis that senior researchers placed on their own reading of papers ( Fig 4A ).

A: Wordcloud extracted from advice given by researchers to someone reading a scientific paper for the first time. The size of the word is proportional to the frequency with which the word was mentioned. Words mentioned fewer than 5 times, and the words ‘read’ ‘reading’ and ‘papers’ are excluded for clarity. Colours used do not represent any aspect of analysis. B: Model of development of scientific literature processing skills. Researchers move acclimation through competency to proficiency through their careers, although these should be viewed as a continuum rather than defined phases. This development of skill is associated with increased familiarity with scientific papers, terminology and background knowledge, and with a change from extrinsic to intrinsic motivations for reading.

A more detailed qualitative analysis of the responses identified four major themes of the advice; reading selectively within a paper, practical suggestions for reading the paper, reading critically and reading with a specific purpose or question in mind ( Table 1 ). 60 of the 88 participants recommended selective reading, with 28 researchers advising reading the paper in a specific order. There was no clear consensus within the responses of which order the sections of a paper should be prioritised in, although starting with the abstract (n = 16) and figures (n = 6) were the most common suggestions. Some researchers recommended reading the abstract, then looking at the figures then reading the text, while others recommended starting with the abstract, then reading the introduction and discussion before looking at the figures. Some recommended prioritising particular sections of the paper (e.g. figures, n = 7) without giving specific instructions on which order the paper should be read in. Others did not make reference to particular sections, but gave more general advice such as identifying the key questions or hypotheses of the paper (n = 8). Some survey participants recommended ignoring the methods section of the paper altogether (n = 3).

88 respondents gave responses as free text, which were subsequently coded by theme. Main themes (bold) and sub-themes (italics) are presented below, with those being mentioned fewer than 5 times not presented for clarity. Specific sections of papers are mentioned where appropriate. Some responses included multiple pieces of advice.

Other advice related to practical strategies for reading papers, the most common of which was to read the paper multiple times (n = 18), with several researchers advising to read in more detail with each read of the manuscript. 5 researchers highlighted the importance of using the abstract and/or discussion to determine whether a paper was worth reading at all before reading in detail. A related theme was to read with a specific purpose in mind, with 6 researchers advising to be constantly thinking about why the paper is being read, and to read with specific questions in mind. The remaining major theme was to read critically, which was mentioned by 30 of the 88 respondents. Common pieces of advice were to assess whether the conclusions matched the data (n = 18), and to interpret the data for yourself (n = 5).

In this study, we identify significant differences in both opinions of papers and reading strategies between researchers at all career stages within the institution surveyed, indicating that the way researchers engage with scientific literature varies throughout an academic career. This study was originally conceived to investigate differences between undergraduate students and researchers, however subsequent analysis revealed significant differences in responses between PhD students, PostDoctoral researchers and senior academics, which are unlikely to be attributed to further formal training. All participants agreed that reading papers was important for their scientific training. Undergraduates at this research-intensive institution typically read scientific papers to broaden their knowledge and to find examples for exam answers, prioritised reading the abstract and discussion sections, and were most likely to report low confidence in their ability to process the literature. Experienced researchers read papers for a variety of reasons, valued the results and methods sections and were confident in their reading abilities. PhD students and PostDoctoral researchers had intermediate levels of confidence, and PhD students in particular prioritised their reading differently to both undergraduates and experienced researchers. Postgraduates reported a similar ease of reading for both the text and the figures/tables of results sections, whereas undergraduates considered figures and tables more difficult to engage with than the accompanying text, perhaps reflecting the specialist skills needed to interpret graphs and other visual representations of data [ 20 ].

Although this study is based on single institution which may not be representative, our data suggest both a change in the motivation for reading through an academic career and an increase in confidence when processing scientific text. 3 rd year undergraduates primarily reported reading papers to find examples to find examples for examination answers, which we define as ‘extrinisic’ motivation. This contrasts with experienced researchers who were more likely to report intrinsic motivations for reading. This is reminiscent of goal orientation theories of motivation, where individuals may be either seeking ‘mastery’ of a topic for self-improvement, or demonstration of enhanced ‘performance’ of abilities relative to others [ 21 , 22 ]. Seeking mastery is associated with deep learning approaches and increased self-regulation, which reinforce each other in a positive feedback loop [ 22 ]. Conversely, students who are performance motivated may lack the incentive to engage deeply with a topic. Undergraduates motivated primarily by exam performance may therefore adopt a relatively superficial approach to the literature; for students with multiple time pressures this may be highly efficient, but provide poor training for later study or subsequent research careers. Assessment strategies should therefore be considered carefully if the aim is to encourage deep and critical engagement with the literature. Instructors should also be mindful of the low self-efficacy many undergraduate and graduate students have when encountering complex literature, and adopt strategies that increase the confidence of inexperienced readers. It should be noted that undergraduates at this institution are assessed predominantly through terminal written examinations that reward the ability to include examples from scientific literature, but have limited scope for detailed critical analysis of research papers. As such, the nature of the institution and assessment strategy may have influenced engagement with the literature, and participants in our survey may not be representative of those on other courses with alternative assessment models; the relationship between assessment and engagement with literature warrants further research.

Conceptual models describing the development of scientific processing skills have been described previously. One study of literature use amongst Masters level students proposed a scheme based on Models of Domain Learning [ 23 – 25 ] whereby there is a continual acquisition of scientific processing skills, with individuals moving from an initial ‘acclimation’ phase, through to ‘competency’ and to eventual ‘proficiency’ in use of the literature [ 23 – 25 ]. Our data support and extend these models, with the development of literature processing skills being a function of (i) the familiarity with both scientific language and the wider research context and (ii) the degree of intrinsic motivation for reading ( Fig 4B ). Our data also suggest that the transition from acclimation to proficiency takes place over an extended period of time spanning multiple career stages. Undergraduate students have previously been shown to define conclusions of scientific papers differently to expert readers, and to overlook important grounds for particular conclusions, i.e. were more likely to rely on author statements in the discussion than to interpret primary results for themselves [ 14 ]. Inexperienced readers find the language of scientific papers challenging [ 23 ], and relative low levels of prior knowledge acts as a barrier to comprehension [ 26 , 27 ]. The importance of background knowledge was specifically commented on by one participant, who added the following disclaimer to their advice for someone reading a paper for the first time (the last question in the survey):

[Participant 83—Academic] “ This is quite a different question than [earlier ones in the survey] , which depend entirely on how familiar I am with the field covered by the paper . I answered [the earlier questions] as if the paper were from a field with which I had a lot of familiarity; almost the reverse would be true for a paper outside of my immediate experience ”

This suggests that even experienced researchers adopt more superficial reading strategies when encountering literature outside their area of expertise. This suggests that considerable experience of reading papers cannot substitute for a lack of prior knowledge, and the barrier to deep engagement with literature is the lack of an appropriate framework rather than reading experience per se . Our survey did not ask researchers to consider papers of different levels of familiarity, however expert readers adjusting their reading strategy on the basis of prior knowledge is consistent with previous studies [ 28 ]. Experienced readers reading within their field therefore benefit from having developed a ‘purpose-laden schema’ which allows the new information to be processed within their existing framework of knowledge [ 29 ], as well as a personal research interest to motivate their reading. This contrasts significantly with undergraduate students, who lack a personal research context and prior knowledge to relate their reading to; in our opinion it is therefore unrealistic to expect students to engage with the literature in the same way as professional researchers. Instructors should therefore be mindful of the lack of subject competency that inexperienced readers have to contextualise their reading, and ensure readers are introduced to relevant background concepts in more accessible formats before tackling highly technical literature.

Given the challenges of scientific language, lack of prior knowledge and low self-confidence, there is a need to support the development of academic reading and scientific processing skills within degree programmes. A variety of teaching strategies for introducing papers undergraduate students have been described, many of which have been demonstrated to increase student confidence and skill in the use of research literature [ 6 , 11 , 15 , 16 , 30 – 32 ]. A commonly used model is the CREATE method, which suggests structuring reading as C onsider, then R ead, E lucidate the hypothesis, A nalyse and interpret data and then to T hink of the next E xperiment [ 11 ]. In doing so this approach models the research process, therefore may provide undergraduates with sufficient personal research context to adopt a reading strategy closer to that typically used by experienced researchers. Our data also suggest that PhD students and PostDoctoral researchers may benefit from structured reading support, as individuals are unlikely have mastered this skill by the end of an undergraduate programme. Reading to support personal research may also require graduate students to adapt strategies used when they were undergraduates, therefore specific guidance in using the literature would be appropriate to support this transition. Focussed training on appropriate use of research papers should therefore be incorporated in postgraduate programmes, and activities promoting critical engagement with the literature should be embedded in the day-to-day activities of research groups. Research supervisors should also be mindful that saying “go and read a paper” may be interpreted differently by individuals at different career stages, so should be clear in their instructions and expectations.

A broader issue for the research community is highlighted in our data, in that even experienced researchers can find the scientific literature challenging. 42% of academics surveyed found reading papers frustrating, despite seeing it as a valuable use of their time. The frustrations researchers have with reading primary papers are sometimes discussed in an ironic or humorous manner [ 33 – 35 ], reflecting the widespread difficulties that many have with this form of scientific communication. Although we did not explore the specific causes of these frustrations in greater detail, previous studies and our interactions with both students and colleagues suggest that background knowledge, terminology, generic attributes of scientific papers and unfamiliar techniques are the most challenging aspects of papers for inexperienced readers [ 23 ]. These difficulties may be particularly acute for those reading scientific literature in a second language, and for those with Specific Learning Differences such as dyslexia [ 36 ]. Those from disadvantaged backgrounds may also have lower levels of academic language proficiency and confidence with complex text [ 37 ]. Individuals with lower self confidence or self-efficacy are less likely to adopt deep learning approaches and evaluate information critically [ 38 ], so may also need structured support to engage with the literature fully.

It should be noted that this study is based on participants at a research intensive university, most of whom who have chosen to embark on a scientific career. They therefore represent a self-selecting sample who may have been more confident with reading technical literature than peers who left research, therefore raising the possibility that research literature is itself a barrier to scientific career choices. In addition, our model of a constantly developing engagement with scientific literature is based on participants at a single institution. We only received responses from ~45% of the undergraduate population and were unable to determine the response rate from researchers, so our results may not be representative of all individuals at the institution and should therefore be treated with caution. It will be important to investigate whether these initial findings hold true for researchers from different career stages at other institutions in the UK and beyond; we hope that this study will provoke others to consider this largely unexplored difficulty with written scientific communication. In a world of open access publishing scientific papers are reaching wider audiences [ 39 ]; readers may include researchers from other fields unfamiliar with discipline specific vocabulary, as well as the general public who may struggle with the complexities of scientific text. As science becomes an increasingly data rich environment, and interdisciplinary approaches to research questions expand, the skills to efficiently process new scientific literature are only going to become more valuable. We therefore propose that the scientific community considers the language they are using to communicate their findings, and how this could be made more accessible and inclusive in order to benefit non-scientists and scientists across all career stages.

Supporting information

This survey was administered online via www.surveymonkey.com , so the layout presented does not match the online version. The text and order of questions is identical, with the exception of where institutional specific terms have been amended for a broader audience [indicated with italic text].

This survey was administered online via www.surveymonkey.com , so the layout presented does not match the online version, however the text and order of questions is identical.

For A: 2nd year undergraduates and B: 3rd year undergraduates, participants were asked which courses they were taking a proxy for disciplinary background. For C: Researcher, participants were asked ‘Which of the following best describes your area of research?’ and were able to select multiple options. Response rates cannot be determined for researchers as surveys were distributed via departmental bulletins or email lists for which the total number of individuals contacted is unknown.

Acknowledgments

We thank Jacqui Poon and Jeremy Solly for their contributions to the development of research questions, and Domino Joyce, Dominic Henri and Julian Hibberd for critical reading of the manuscript prior to submission. We also thank Bob Burwell for thought provoking discussions over the barriers to engaging with the literature.

Funding Statement

The author(s) received no specific funding for this work.

Data Availability