example of quantitative research about medicine and its contribution

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Quantitative medicine: Tracing the transition from holistic to reductionist approaches. A new “quantitative holism” is possible?

Silvano tagliagambe.

• Author information
• Article notes
• Copyright and License information

Luca Saba, University of Cagliari, SS 554 Monserrato, Cagliari 09124, Italy. Email: [email protected]

Received 2023 May 20; Accepted 2023 May 25; Collection date 2023 Apr.

This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License ( https://creativecommons.org/licenses/by-nc/4.0/ ) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages ( https://us.sagepub.com/en-us/nam/open-access-at-sage ).

The practice of medicine has evolved significantly over time, from a more holistic to a reductionist or mechanistic approach. This paper briefly traces the history of medicine and the transition to quantitative medicine, which has enabled more personalized and targeted treatments, and improved understanding of the underlying biological mechanisms of disease. However, this shift has also presented some challenges and criticisms, including the danger of losing sight of the patient as a unique, whole individual. This paper explores the underlying principles and key contributions of quantitative medicine, as well as the context for its rise, including the development of new technologies and the influence of reductionist philosophies. The challenges and criticisms of this approach, and the need to balance reductionist and holistic approaches in order to achieve a comprehensive understanding of human health will be discussed. Ultimately, by integrating insights from philosophy, physics, and other fields, we may be able to develop new and innovative approaches that bridge the gap between reductionism and holism and improve patient outcomes with the new “quantitative holism.”

Keywords: Medicine, holism, quantitative medicine, evolution, philosophy

Introduction

Medicine has a rich and complex history, shaped by a variety of factors including cultural, social, and scientific developments. From the earliest forms of healing practiced by ancient civilizations to the cutting-edge medical technologies of today, medicine has undergone significant changes over time. One of the most profound shifts in the history of medicine has been the move from a more holistic approach to a reductionist or mechanistic approach. 1

In its early days, medicine was often practiced by spiritual or religious leaders who saw illness as a punishment from the gods or a result of spiritual imbalance. 2 These healers sought to restore balance to the body, mind, and spirit of their patients through a variety of methods including prayer, meditation, and herbal remedies. 2 Over time, these approaches gave way to more empirical and scientific methods, as doctors began to develop a better understanding of the human body and its functioning. 3

During the Renaissance, for example, anatomists began to dissect cadavers in order to gain a deeper understanding of the human body’s structure and function. 4 This led to the development of new surgical techniques and treatments, as well as a greater emphasis on empirical observation and experimentation. 5 In the 19th and early 20th centuries, medicine became increasingly specialized, with doctors focusing on specific areas of the body or specific diseases. 6 , 7

However, the reductionist approach to medicine really came into its own in the mid-20th century, with the rise of the biomedical model. 8 This model sees the body as a machine, with individual parts that can be studied and manipulated independently. The goal of medicine became focused on identifying and treating specific diseases or conditions, often through the use of drugs or surgery. While this approach has led to significant advances in medical knowledge and treatment, it has also been criticized for the risk of ignoring the complex interactions between different systems in the body and for reducing patients to a collection of symptoms and diagnoses. 9

The shift toward a more reductionist approach to medicine was influenced by a variety of scientific and philosophical trends, as well as by the development of new technologies that allowed for more precise and targeted interventions. One of the key philosophical influences on the reductionist approach to medicine was the rise of Cartesian dualism in the 17th century. 10 , 11 This philosophical perspective sees the mind and body as separate entities, with the body being a machine that can be studied and understood independently of the mind. This perspective influenced many early anatomists and physiologists, who saw the body as a collection of discrete parts that could be studied in isolation. 12

Another influential philosophical trend was logical positivism, which emerged in the early 20th century. 13 This approach emphasized the importance of empirical observation and experimentation in the development of scientific knowledge. In medicine, this led to a greater emphasis on quantitative data and measurable outcomes, as well as on the use of randomized controlled trials to test the efficacy of different treatment. 14

One key figure in the development of the reductionist approach to medicine was French physiologist Claude Bernard. In the mid-19th century, Bernard argued that the body could be understood as a collection of independent physiological systems, each of which could be studied and understood in isolation. 15 He also emphasized the importance of experimentation and measurement in the development of medical knowledge. Another important figure in the history of quantitative medicine was English physician and epidemiologist Austin Bradford Hill. Hill’s work in the mid-20th century helped to establish the importance of randomized controlled trials in the testing of new treatments. 16 He also emphasized the need for careful observation and data collection, and argued that medicine should be based on empirical evidence rather than intuition or tradition. 16

At the same time, advances in technology were also contributing to the shift toward a more reductionist approach to medicine. The development of new tools like microscopes and X-rays allowed doctors to see inside the body and study its structure and function in greater detail. The invention of new drugs and surgical techniques also allowed for more precise and targeted interventions. In recent years, the development of new technologies like genomics and proteomics has further fueled the rise of quantitative medicine. These fields allow researchers to study the complex interactions between genes, proteins, and other biological molecules, and to develop more personalized and targeted treatments based on this knowledge. 17 At the same time, the growing availability of digital health data has opened up new opportunities for using big data and machine learning to analyze patterns in health and disease. 18 , 19

Despite these advances, however, there are still many challenges and limitations to the reductionist approach to medicine. Some critics argue that it ignores the complexity and interconnectedness of the human body, and that it can lead to a focus on treating individual symptoms rather than addressing underlying causes. Others argue that it can be dehumanizing, reducing patients to a collection of data points and diagnoses.

The rise of quantitative medicine

Quantitative medicine is a paradigm shift in the practice of medicine that emphasizes the use of quantitative data and mathematical models to understand and treat disease. 20 This approach is based on the idea that the human body can be studied as a complex system, with many interconnected parts that can be modeled and simulated using mathematical and computational tools. At its core, quantitative medicine is based on four key principles: precision, personalization, prediction, and prevention.

Precision Medicine : Precision medicine is a key tenet of quantitative medicine, and refers to the use of molecular and genetic information to guide the development of targeted treatments for individual patients. This approach recognizes that different patients may respond differently to the same treatment, and that a more personalized approach may be necessary to achieve optimal outcomes.

One example of precision medicine is the use of genomic sequencing to identify specific mutations or genetic markers that are associated with certain types of cancer. By identifying these markers, doctors can develop treatments that are tailored to the patient’s specific genetic profile, potentially improving outcomes and reducing side effects.

Personalized Medicine : Personalized medicine builds on the principles of precision medicine, but takes a broader view of the patient as a whole person. This approach recognizes that patients may have different needs and preferences, and that a one-size-fits-all approach to treatment may not be effective.

Personalized medicine emphasizes the importance of patient-centered care, and involves working closely with patients to develop treatment plans that take into account their individual circumstances, including their medical history, lifestyle, and preferences.

Predictive Medicine : Predictive medicine involves using mathematical and computational models to predict the risk of disease and to identify patients who may be at higher risk. This approach can be used to develop more targeted screening and prevention strategies, potentially reducing the overall burden of disease.

One example of predictive medicine is the use of risk prediction models to identify patients who are at higher risk of developing cardiovascular disease. By identifying these patients early, doctors can develop targeted interventions, such as lifestyle changes or medication, to prevent or delay the onset of disease.

Preventive Medicine : Preventive medicine involves identifying and addressing risk factors before they lead to disease. This approach emphasizes the importance of healthy lifestyle choices, such as exercise, diet, and smoking cessation, as well as targeted interventions, such as vaccination and screening, to prevent or reduce the risk of disease.

The principles of quantitative medicine are rooted in the use of quantitative data and mathematical models to understand and predict the behavior of complex systems. This approach has been enabled by advances in computing power and data analytics, as well as by the development of new tools and technologies, such as genomics and proteomics.

One key contributor to the rise of quantitative medicine has been the field of systems biology, which seeks to understand complex biological systems as networks of interacting components. 21 Systems biology approaches often involve the use of computational models to simulate the behavior of these systems, and can be used to identify new drug targets and to develop more personalized treatment strategies. 22 Another key contributor has been the field of digital health, which involves the use of technology, such as wearables and mobile apps, to collect and analyze health data. Digital health data can be used to develop predictive models of disease risk, to monitor patient outcomes in real-time, and to develop more personalized treatment plans.

The rise of quantitative medicine has also been driven by advances in genomic and proteomic technologies, which allow researchers to study the underlying biological mechanisms of disease in unprecedented detail. 23 By analyzing the expression patterns of genes and proteins, researchers can identify new drug targets and develop more targeted therapies.

The rise of quantitative medicine has been driven in part by advances in new technologies, such as imaging techniques and high-throughput sequencing, which enable the measurement of vast amounts of data on individual patients. However, these technologies alone cannot fully explain the shift toward a more quantitative approach. Rather, the rise of quantitative medicine can also be seen as a reflection of broader trends in science and philosophy toward reductionism and mechanistic thinking.

In this sense, the emergence of quantitative medicine has parallels with the emergence of theoretical physics in the early 20th century. 24 Just as theoretical physics sought to understand the underlying mechanisms of the physical world through the use of mathematical models and simulations, quantitative medicine seeks to understand the underlying biological mechanisms of disease through the use of complex data analysis and computational models.

One example of this connection can be seen in the use of network analysis to study biological systems. 25 In physics, network analysis has been used to study the properties of complex systems such as the internet or social networks. Similarly, in quantitative medicine, network analysis can be used to study the interactions between genes or proteins, and to identify key drivers of disease. Furthermore, the use of mathematical models and simulations in quantitative medicine is also reminiscent of the approach taken in theoretical physics. For example, mathematical models of the spread of infectious diseases have been used to predict the effectiveness of different interventions, such as vaccination campaigns or social distancing measures.

However, as with any reductionist approach, there are also limitations to the use of mathematical models and simulations in medicine. One potential danger is that these models may oversimplify complex biological systems, leading to inaccurate predictions and treatments. Additionally, as discussed earlier, there is a risk of losing sight of the patient as a unique, whole individual.

Challenges, criticism, and its impact

One challenge is the sheer complexity of biological systems, which can make it difficult to develop accurate models and simulations. Another challenge is the need for large, high-quality datasets to train these models, which can be difficult to obtain in some cases. Critics of quantitative medicine have also argued that this approach can be reductionist, focusing too narrowly on individual components of biological systems and neglecting the broader context in which these systems operate. This criticism is rooted in the philosophical debate between reductionism and holism, which we discussed in the previous section.

Despite these challenges, the rise of quantitative medicine has transformed the practice of medicine, enabling more personalized and targeted treatments, and improving our understanding of the underlying biological mechanisms of disease. As we move forward, it will be important to continue to balance the benefits and limitations of this approach, and to ensure that our models and simulations are grounded in a holistic understanding of the complex systems that underlie human health.

While the rise of quantitative medicine has undoubtedly led to significant advances in our understanding of human health and disease, it is important to recognize that the practice of medicine involves much more than simply identifying and targeting isolated biological processes. At its core, medicine is a humanistic endeavor that involves treating patients as unique individuals with complex physical, emotional, and social needs.

One of the challenges of the quantitative approach to medicine is that it can lead to a fragmented view of the patient, focusing solely on isolated biological processes and neglecting the broader context in which these processes occur (the human being). This approach can be particularly problematic when it comes to treating chronic diseases or complex conditions that involve multiple biological systems and psychological and social factors.

Moreover, the super-specialization of medical practitioners can exacerbate the problem of fragmentation, as physicians become increasingly focused on specific sub-fields or areas of expertise, and lose sight of the broader context in which their patients exist. This can lead to a situation in which physicians view their patients as a collection of isolated parts, rather than as unique, interconnected individuals.

This fragmentation of the medical approach runs counter to the holistic view of medicine, which emphasizes the interconnectedness of biological systems and the importance of treating patients as whole individuals, rather than simply as the sum of their individual parts. This view recognizes the importance of the social, cultural, and psychological factors that contribute to human health and disease, and emphasizes the importance of patient-centered care that takes into account the unique needs and circumstances of each individual patient.

The challenge of treating patients as unique individuals requires a shift in the culture and mindset of the medical profession. This shift involves recognizing the limitations of a reductionist approach to medicine and embracing a more holistic view that recognizes the interconnectedness of biological systems and the importance of patient-centered care. By doing so, we can develop more effective treatments and interventions that improve the health and wellbeing of individuals and communities around the world.

A pressing requirement: from physic to medicine

The convergence and reconciliation of reductionism and holism is an unavoidable need in scientific research in general, and in medicine, in particular. It can correctly be presented as the necessary middle ground between macroscopic and microscopic description. If we take, for example, a biological system, which we can consider as the prototype of a complex system, its macroscopic description can be very varied and require a language with a very rich vocabulary: the multiplicity and diversity of these descriptions can be taken as an indicator of complexity and cannot be neglected, so that a traditional reductionist approach would be ineffective.

Equally true, however, is that a global perspective, in which the nature of interactions between constituents is neglected, also seems sterile, as this characteristic is crucial in determining overall behavior. A fundamental property of complex systems is therefore the possibility, indeed the necessity, to be described both at the microscopic level and at a higher level where different categories and concepts must be used, implementing what has been called an “intermediate point of view,” which concretely realizes the continuous crossing, the coming and going between the two mentioned levels.

A concrete example of this need is the current situation in brain studies. Molecular neurobiology has been extraordinarily successful and has gathered very detailed and fully satisfactory information on the functioning of individual neurons. However, this knowledge does not allow us to directly understand how a billion neurons can behave like a mammalian brain. At the opposite extreme we have psychology, for which the properties of individual neurons (and more generally the chemical-physical properties of the brain) are completely irrelevant. This science has laboriously forged its own conceptual categories to describe human thought. The meeting of these two extremes seems arduous, but fortunately even here intermediate approaches are emerging, such as cognitive psychology, which is dedicated to the detailed study of the mechanisms and processes through which human beings perceive the world and organize their knowledge and activities.

An effective example by Giorgio Parisi, who was awarded the 2021 Nobel Prize in Physics precisely for his studies on complex systems, illustrates why and how the macroscopic and the microscopic, the global and the local interact in explaining the reality of these systems in such a way that they cannot be treated separately. A study, the result of an international collaboration, 26 including, among others, Pierfrancesco Urbani and Francesco Zamponi Parisi, dealt with a process, that of vitrification, that is, the transition from the liquid state, which occurs at high temperature, to the solid state, as it cools down, the molecular details of which had hitherto escaped notice. The authors managed to give a complete physical description of this by showing, surprisingly, that the set of different configurations assumed by the glass particles when solidification occurs has a fractal structure. Recall that, in mathematical terms, a fractal is a geometric object endowed with scale invariance: in practice, it appears to have the same structure at whatever dimensional scale one considers it. Fractal structures are often found in nature, and they unite incredibly diverse objects, such as a romanesco broccoli, a stretch of coastline and the edge of a leaf.

Phase transitions are processes that occur daily before our eyes: for example, when water reaches a temperature of zero degrees Celsius, it solidifies and becomes ice; a glass or wax sample in its liquid state, as it cools, becomes solid. However, these are two very different phase transitions because, in the case of water, solidification is sudden, whereas in the case of glass or wax, the process is gradual: as the liquid cools, it acquires greater and greater viscosity until it becomes a solid in its own right. To explain this difference, one cannot help but consider what happens at the microscopic level: when water cools to zero degrees, the initially disordered molecules arrange themselves neatly in a crystalline lattice, whereas in glass and wax, the atoms arranged in a completely disordered manner remain equally disordered even when the solid state is reached. From a physical point of view, the question is: why is there such a conspicuous macroscopic effect, the solidification of wax, even though it changes the microscopic arrangement of the molecules very little?

The answer can be illustrated through a metaphor proposed by Parisi. Let’s think of an underground carriage at rush hour, where the travelers inside are very compressed. Usually, however, there are small gaps that allow one person to change position, because another person perhaps moves a little, pushing another, and momentarily vacates the space. Under these conditions it would only take four or five more people for any movement to be blocked, but seen from the outside, these two situations do not appear very different.

In the case of glass or wax, something similar happens. As the temperature goes down, the molecules decrease their vibrational motions and become more and more stuck in their position because neighboring molecules are stuck, and so on. The traditional idea was that there was only one way for molecules to get stuck. Instead, Parisi and colleagues were able to show that the phase transition occurs with different configurations of the molecules. Returning to the people compressed in the underground, there may be many similar situations, but they are slightly different. A person, for example, can lift an arm, or be able to turn 90 degrees: as they say in physics, different configurations are possible. The same happens with the molecules of a glass that is cooling: for a given value of pressure, the molecules have a certain freedom of movement; as the pressure increases and the temperature decreases, the space available for movement becomes smaller and smaller and is fragmented into smaller spaces, which are no longer in communication with each other. In the set of possible configurations and spaces available for movement, the scale invariance typical of fractals manifests itself.

It is interesting to understand how they arrived at this result. Rather than dealing with real-world materials, which are extremely complex and diverse, and for which many different theories have been proposed that make extensive use of approximations, to the point that it often becomes quite difficult to establish whether the statements made by the theory are true even within the logical structure of the theory itself, they have elaborated a simple, solvable mathematical model that is valid for everything that can be classified, generically, as “glass.” The explanatory hypotheses gradually proposed, being limited to this mathematical model, could be checked directly, establishing, in a well-defined mathematical sense, their correctness. The advantage of this approach is that the model shows an interesting capacity for expansion and unification , allowing it to be applied not only to the phenomena for which it was originally developed, but also to others that were thought to be somehow distinct from it.

The interesting aspect of this approach is that it allowed it to work on two parallel but distinct levels, the mathematical model and the real world. The model’s statements do not concern concrete objects belonging to the real world but specific abstract mathematical objects; its structure is deductive, consisting of a few postulates concerning its objects and a method for deriving a potentially infinite number of consequences. The model is evolved mathematically and at the conclusion of this process its applicability to the real world is verified through a series of “correspondence rules” between the abstract objects of the theory and those of reality, the object of study. This allows the theory to be extended, using the deductive method and introducing new rules of correspondence, to deal with situations that were not a priori included in the initial objectives for which it was initially developed.

This is what concretely consists of what we can call the back-and-forth between the abstract model and reality, by virtue of which, to return to the metaphor of the underground, one can grasp both the macroscopic similarities and the microscopic differences between the situations in which people squeezed into a car find themselves. The two levels, initially distinct and parallel, thus converge, interacting in a concrete and productive way: this is what the only apparently counterintuitive evolutionary approach into a holistic one consists of.

If we want to arrive at a medicine that can be quantitative, precision, personalized, predictive and preventive, this lesson and its enrichments due to the recent evolution from the classical concept of model to that of digital twin must be taken into account. The model is, by definition, an artificial and simplified representation of reality. The relationship between a territory and all its possible maps, physical, political, geomorphological, hydrographic, nautical, economic, demographic etc. exemplifies this nature well, as does the fact that the choice between one or the other depends on the specific problem to be addressed. If, for example, one wants to visualize the distribution of different climatic types and make a reliable weather forecast, one will use a weather map that focuses on these aspects, neglecting all others. This is what simplification consists of, as mentioned above, acting as a perceptive and cognitive filter that responds to the need to conveniently frame and resolve the issue under study. However, if the problem posed concerns not the individual and specific properties of the territory, but the relationships between its various aspects and their interactions, such a model is not functional and effective.

The digital twin compensates for this limitation in that it mimics not a single distinctive feature, but the entire structure and nature of a phenomenon or process and even its context through sets of virtual information constructs dynamically updated thanks to data derived from its physical twin, with which it is constantly connected, throughout its entire life cycle and thanks to informed decisions that generate value. The characterizing element of the digital twin is the two-way, continuous dialog with the physical entity represented: on the one hand, the digital twin provides information to actively monitor and control the physical twin, on the other hand, the information generated by the real twin feeds the simulation algorithms of the digital twin.

The concept of a “twin strategy” was generated from NASA’s Apollo program, which build two real identical space vehicles. One was launched onto the air space, the other stayed on Earth to mirror the conditions of the launched one. The first mention of the term “digital twin” can be traced back to the year 2003 when Grieves mentioned it in the context of manufacturing. 27 Initially, the space industry was primarily concerned with the topic of Digital Twin (DT). In 2012, the NASA and the U.S. Air Force jointly published a paper about the DT, which stated the DT was the key technology for future vehicles. After that, the number of research studies on DT in aerospace has increased and the DT was introduced into more fields such as automotive, oil and gas as well as health care and medicine. Examples are online operation monitoring of process plants, traffic and logistics management, dynamic data assimilation enabled weather forecasting, real-time monitoring systems to detect leakages in oil and water pipelines, and remote control and maintenance of satellites or space-stations. For instance, Singapore is developing a digital copy of the entire city to monitor and improve utilities.

Grieves originally defined the DT in three dimensions 27 : a physical entity, a digital counterpart and a connection that ties the two parts together. In most definitions, the DT is considered as a virtual representation that interacts with the physical object throughout its lifecycle and provides intelligence for evaluation, optimization, prediction.

Taking into account all that has been said, those who fear this virtual duplication of the material world could be answered by pointing out that only if we want to force the situation do we open the door to many different realities, opposed to our real world: in essence, we are representing the latter, in all its folds and with all its extraordinary difficulties, accurately and with predictive potential, going beyond the present and simulating the future, thanks to the availability of Big Data, mathematical models, and AI algorithms.

The application to the medicine

At this point of our analysis come examples could be presented to better focus the ongoing process and impact. Multiple Sclerosis is a chronic autoimmune, degenerative and lifelong disease of the central nervous system (CNS) and the most common cause of neurological disability in young adults. At a pathological level, the infiltration of immune cells into the CNS manifests as localized demyelinating lesions in the white and gray matters of the brain and spinal cord, observed in pathological specimens as well as in magnetic resonance imaging (MRI) sequences. In addition, the disease leads to a progressive destruction of myelin layers (demyelination) and progressive axonal injury, loss and neurodegeneration, impairing the function of the CNS in several ways. MS has different clinical disease courses that have been classically described: beyond this raw classification of disease courses, each MS patient presents with a very individual course of his MS. Therefore, hen quantifying MS, it is necessary to distinguish between different dimensions and perspectives.

An emerging approach toward personalized treatment is precision medicine that takes into account individual variability in genes, environment, and lifestyle for each person. Precision medicine covers diagnosis, treatment and management to achieve better patient outcomes, through precision medicine and twin strategy it is possible to break down the complexity of the disease. The patterns and inter-individual variability can be better understood.

Concrete implementations of digital twins can already be found for organs such as the heart, Recently, a research group from Sofia University in Bulgaria performed a first exercise of simulation of DTs. Petrova-Antonova et al. 28 developed a web-based DT platform for MS diagnosis and rehabilitation that consists of two components: a transactional application that automates tests for MS diagnosis and rehabilitation, and an analytic application that provides data aggregation, enrichment, analysis, and visualization that can be used in any instance of the transactional application to generate new knowledge and support decision making. the analytical application is currently undeveloped and subject to further research.

Concrete implementations of digital twins can already be found for organs such as the heart, for example. Horizon 2020 project, iHEART, proposed and implemented by the team of the polytechnic of Milan led by Prof. Alfio Quarteroni, is developing a virtual heart, made up of mathematical equations that describe the complex interaction of physical phenomena that underpin the heartbeat itself. 29 The aim is to construct a “digital twin” of the patient – a virtual replica of an individual’s heart, based on their biometric data and diagnostic tests. This would prove to be a fundamental tool for heart surgeons and cardiologists, who could use it to explore different treatment options or surgical strategies before treating the actual patient, thus optimizing and personalizing their care based on the individual characteristics specific to them. The goal of iHEART is to construct a mathematical model of the human heart, that is, a virtual replica of the organ that allows us to study and predict its behavior by means of computer simulations. In order to build a model of this kind, Quarteroni and his team are seeking out mathematical equations capable of faithfully representing the behavior of the heart, from the scale of the cells all the way up to that of the atria and ventricles. This results in a system of equations, all of which are paired together according to a dense network of interactions. They use specific, computer-implemented algorithms for these highly complex equations, allowing us to find an approximate—yet nonetheless very precise—solution to the issue at hand. The model of the heart that we are developing could 1 day become a tool in the hands of cardiologists and heart surgeons. By using the biometric data and diagnostic tests of a specific patient, the virtual heart developed in the iHEART project could be personalized, effectively creating a “digital twin” of the patient’s heart. The doctor could then use this virtual replica to explore different treatment options or surgical strategies, tailored specifically to the individual patient, simply by interacting with the computer – and all before treating the actual patient. Secondly, the model could assist doctors in interpreting the results of diagnostic tests, giving them the opportunity to replace invasive methods of measurement with indirect, less invasive ones.

The model could also be used for medical research purposes. Indeed, it makes it possible to run scenario analyses, study the interactions between the different components of the organ, or simulate the effects of diseases or innovative treatments by applying them—entirely virtually—to the digital heart.

Potential evolution and new synthesis: A new holism?

Is it possible for the reductionist model we have experienced in the last century to transform into a new quantitative holistic approach, through digital and algorithmic evolution? Although it may appear to be a contradictory idea, could the evolution of the reductionist model aid in this transition?

The reductionist model has traditionally focused on breaking complex systems down into smaller, more manageable parts, in order to understand how they function. One possible evolution of the reductionist model could involve incorporating a quantitative holistic approach, which considers the entire system as a whole, while still utilizing quantitative methods to measure and analyze its components. This would require a shift in thinking, moving away from the reductionist perspective that sees the system as merely the sum of its parts, toward a more integrated approach that recognizes the importance of emergent properties and systemic interactions.

While it may seem counterintuitive for a reductionist approach to evolve into a holistic one, evolution is not always a straightforward process, and can involve unexpected transformations and adaptations. By incorporating new ideas and methodologies, the reductionist model could potentially evolve into a more comprehensive and accurate understanding of complex systems.

The concept of returning to a more holistic approach to healthcare through quantitative medicine could be considered as an example of a circular pattern in science, where ideas and approaches evolve over time, only to return to a starting point but at a new level of understanding. This circular pattern could be considered as the “spiral of science” or the “circle of knowledge.”

Philosopher and historian of science Thomas Kuhn proposed the concept of scientific paradigms, which are frameworks of understanding that guide scientific research and discovery. According to Kuhn, science undergoes periodic revolutions, in which existing paradigms are replaced by new ones, leading to a significant shift in scientific thinking and practice. However, Kuhn also noted that scientific progress is not always linear, and that there are often periods of stagnation or even regression before a new paradigm emerges. This circular pattern of progress and regression is sometimes referred to as the “Kuhnian circle.”

In the case of quantitative medicine, the use of mathematical and statistical methods together with the potentiality of artificial intelligence represents a new paradigm in healthcare that has the potential to revolutionize the way we understand and treat disease. However, this approach is not entirely new; holistic approaches to medicine have existed for thousands of years, and the concept of the body as a complex system has been recognized by many ancient medical traditions. This return could be achieved through the creation of a theoretical model of quantitative analysis that is so advanced and complex that it can “theoretically” simulate the human body as a digital twin.

However, the application of digital twin technology in medicine raises a number of unique challenges and considerations. One of the primary challenges is the need for large, high-quality datasets to accurately model complex biological systems. This requires the integration of data from a variety of sources, including medical imaging, genomics, and other forms of biological data. Additionally, the accuracy of digital twin models depends on the accuracy and completeness of the data used to create them.

Another challenge is the ethical and privacy considerations related to the use and storage of sensitive medical data. This is particularly important given the increasing prevalence of data breaches and cyber-attacks targeting medical records and other sensitive information. Researchers must ensure that appropriate measures are in place to protect the privacy and security of patient data.

Moreover, the use of digital twin technology in medicine also raises important philosophical and ethical considerations related to the true nature of human identity and the relationship between the physical body and the digital realm and the interconnection between these two domains. Philosophers and physicists have long debated the nature of reality and the relationship between physical and digital representations of the world and they also raise questions about the potential consequences of creating digital replicas of biological systems, and the extent to which these replicas can truly capture the complexity and richness of living systems.

Despite these challenges and considerations, the use of digital twins in medicine holds significant promise for advancing our understanding of human health and disease, and developing more personalized and targeted treatments. As researchers continue to refine and develop digital twin technology, it will be important to balance the benefits and limitations of this approach, and to ensure that ethical and privacy considerations are carefully considered and addressed.

In light of these challenges and criticisms, is it possible to reacquire a more holistic view of medicine? While the development of quantitative methods and technologies has undoubtedly led to significant advances in our understanding of human health and disease, it is important to recognize the limitations of this approach and to explore new modalities for understanding the complexity and richness of biological systems.

One possible way forward is to integrate the principles of holistic medicine with the advances of quantitative medicine, creating a new paradigm that acknowledges the interconnectedness of biological systems while still leveraging the power of quantitative methods and technologies. This approach would recognize the importance of individualized, patient-centered care, while also incorporating the latest advances in genomics, medical imaging, and other forms of biological data. Another way forward is to explore new technologies and approaches that can capture the complexity and richness of biological systems in a more comprehensive and nuanced way. This could involve the development of new simulation techniques that incorporate a wider range of biological processes and interactions, or the use of advanced imaging technologies that allow for a more detailed and precise understanding of the human body and this process surprisingly could create a new model that we could define “ quantitative holism ” that implies the potentiality to regenerate a unifying model through the inputs of the quantitative data analyzed by the advanced AI model.

Ultimately, the evolution of medicine is an ongoing process, shaped by a wide range of social, cultural, and technological factors. While the transition from a holistic to a quantitative approach to medicine has been an important step forward in our understanding of human health and disease, it is important to continue exploring new modalities and approaches that can capture the full complexity and richness of biological systems. By doing so, we can develop more effective treatments and interventions that improve the health and wellbeing of individuals and communities around the world.

In conclusion, the rise of quantitative medicine has marked a significant transition in the history of medicine, from a more holistic approach to a reductionist or mechanistic approach. While this has brought many benefits, including more personalized and targeted treatments, and improved understanding of the underlying biological mechanisms of disease, it has also presented some challenges and criticisms. In particular, the danger of losing sight of the patient as a whole, unique individual, and the need to ensure that our models and simulations are grounded in a holistic understanding of the complex systems that underlie human health.

Furthermore, the rise of quantitative medicine has not occurred in isolation, but rather reflects broader philosophical and scientific trends. The development of new technologies and the influence of reductionist philosophies have been key drivers of this shift. However, it is important to recognize that reductionism has its limitations and that a truly comprehensive understanding of human health will require a synthesis of reductionist and holistic approaches.

As we move forward, it will be important to continue to critically evaluate the benefits and limitations of quantitative medicine, and to ensure that we maintain a balanced approach that recognizes the unique complexity of each individual patient. By integrating insights from philosophy, physics, and other fields, we may be able to develop new and innovative approaches that bridge the gap between reductionism and holism in a model that we could define “ quantitative holism,” and ultimately improve patient outcomes.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.

• 1. Yawar A. Medicine and the human story. Lancet 2010; 375: 546–547. [ DOI ] [ PubMed ] [ Google Scholar ]
• 2. Puchalski CM. The role of spirituality in health care. Proc 2001; 14: 352–357. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 3. Masic I, Miokovic M, Muhamedagic B. Evidence based medicine – New Approaches and challenges. Acta Inform Med 2008; 16: 219–225. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 4. Ghosh SK. Human cadaveric dissection: a historical account from ancient Greece to the modern era. Anat Cell Biol 2015; 48: 153. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 5. Standring S. A brief history of topographical anatomy. J Anat 2016; 229: 32–62. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 6. Rosenberg CE. The tyranny of diagnosis: specific entities and individual experience. Milbank Q 2002; 80: 237–260. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 7. Leinster S. Training medical practitioners: which comes first, the generalist or the specialist? J R Soc Med 2014; 107: 99–102. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 8. Ahn AC, Tewari M, Poon CS, et al. The limits of reductionism in medicine: could systems biology offer an alternative? PLoS Med 2006; 3: 0709–0713. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 9. Rocca E, Anjum RL. Complexity, reductionism and the biomedical model. In: Anjum RL, Copeland S, Rocca E. (eds) Rethinking causality, complexity and evidence for the unique patient. Cham: Springer, 2020, pp.75–94. [ Google Scholar ]
• 10. O’Leary D. Medicine’s metaphysical morass: how confusion about dualism threatens public health. Synthese 2021; 199: 1977–2005. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 11. Mehta N. Mind-body dualism: A critique from a health perspectiveFNx08. Mens Sana Monogr 2011; 9: 202. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 12. Zampieri F, ElMaghawry M, Zanatta A, et al. Andreas Vesalius: Celebrating 500 years of dissecting nature. Glob Cardiol Sci Pract 2015; 2015: 66. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 13. Blumberg AE, Feigl H. Logical Positivism. J Philos 1931; 28: 281. [ Google Scholar ]
• 14. Delgado A, Guddati AK. Clinical endpoints in oncology - a primer. Am J Cancer Res 2021; 11: 1121–1131. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 15. Bernard C. The “Milieu Intérieur”, and Regulatory Physiology. Hist Philos Life Sci 1986; 8(1): 3–25. [ PubMed ] [ Google Scholar ]
• 16. Hill AB. The environment and disease: association or causation? Proc R Soc Med 1965; 58: 295–300. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 17. Hood LE, Omenn GS, Moritz RL, et al. New and improved proteomics technologies for understanding complex biological systems: addressing a grand challenge in the life sciences. Proteomics 2012; 12: 2773–2783. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 18. Zhang A, Xing L, Zou J, et al. Shifting machine learning for healthcare from development to deployment and from models to data. Nat Biomed Eng 2022; 6(12): 1330–1345. [ DOI ] [ PubMed ] [ Google Scholar ]
• 19. Dash S, Shakyawar SK, Sharma M, et al. Big data in healthcare: management, analysis and future prospects. J Big Data 2019; 6: 1–25. [ Google Scholar ]
• 20. Bollet AJ. Pierre Louis: the numerical method and the foundation of quantitative medicine. Am J Med Sci 1973; 266: 92–101. [ DOI ] [ PubMed ] [ Google Scholar ]
• 21. Auffray C, Noble D, Nottale L, et al. Progress in integrative systems biology, physiology and medicine: towards a scale-relative biology. Eur Phys J A 2020; 56: 1–24. [ Google Scholar ]
• 22. Finley SD, Chu LH, Popel AS. Computational systems biology approaches to anti-angiogenic cancer therapeutics. Drug Discov Today 2015; 20: 187–197. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 23. Hossain MU, Ferdous N, Reza MN, et al. Pathogen-driven gene expression patterns lead to a novel approach to the identification of common therapeutic targets. Sci Rep 2022; 12: 1–16. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 24. Weisskopf VF. Physics in the twentieth century selected essays. Cambridge: The MIT Press, 1974. [ Google Scholar ]
• 25. Ghiassian SD, Menche J, Chasman DI, et al. Endophenotype network models: Common core of complex diseases. Sci Rep 2016; 6: 27414. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 26. Parisi G, Urbani P, Zamponi F. Theory of simple glasses. Cambridge: Cambridge University Press, 2020. [ Google Scholar ]
• 27. Grieves M. Digital Twin: manufacturing excellence through virtual factory replication. This paper introduces the concept of a A Whitepaper by Dr. Michael Grieves. White Pap, 2015. [ Google Scholar ]
• 28. Petrova-Antonova D, Spasov I, Krasteva I, et al. A Digital Twin platform for diagnostics and rehabilitation of multiple sclerosis. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 12249 LNCS:503–518. 2020. DOI: 10.1007/978-3-030-58799-4_37/FIGURES/6 [ DOI ] [ Google Scholar ]
• 29. Quarteroni A. Un modello matematico del cuore umano. G Ital Cardiol 2020; 21: 907–914. [ DOI ] [ PubMed ] [ Google Scholar ]
• View on publisher site
• PDF (429.2 KB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

An official website of the United States government

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Publications
• Account settings
• Advanced Search
• Journal List

Research and Medical Students: Some Notable Contributions Made in History

Pravakar dawadi, sabina khadka.

• Author information
• Article notes
• Copyright and License information

Correspondence: Pravakar Dawadi, Nepalese Army Institute of Health Sciences, Sanobharyang, Kathmandu, Nepal. Email: [email protected] , Phone: +977-9841215580.

Issue date 2021 Jan.

This is an Open-Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/4.0 ) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Upon the commencement of the practice of modern medicine, the establishment of evidence based practice has played a crucial role in its advancement. Whether it be an expert medical practitioner or some beginner medical student who is in the early phase of pursuing their career, each of them can contribute in our own way. This article highlights some of the many contributions made in modern medicine by those early buds in the past.

Keywords: evidence based practice , medical students , research

The practice of medicine has started since the existence of humankind on the earth. But we cannot trace back the commencement of scientific and evidence-based medicine so far. The insight we are getting from current medical science is the output of people's laborious effort and contributions before us. The statement stating ‘we learn from our past’ is perfectly resonant with the practice of clinical medicine.

Hippocrates, who is commonly known to be the ‘Father of Modern Medicine,’ initiated the rational and logical medicine practice based on evidence. He formed a diagnostic system by integrating knowledge from the past based on clinical observation and logical reasoning. The rejection of the long-standing concept that some divine force caused illness was a revolutionary beginning of the modern medicine era. Hippocrates instead attributed the diseases to natural cause and insisted treatment should be based on observation, reasoning, and past experiences. 1

Those past experiences or findings which are scientifically cataloged are crucial in the advancement of modern medicine. The countless researches conducted in various aspects of medical science prove to be strong evidence of our past achievements. Many people's contribution to such research activities in the medical field has taken us this far in modern medicine. In the pool of such people, medical students who are in the early phase of pursuing their career in medicine have also contributed a lot to medical research time and again. 2

Heparin, a natural anticoagulant, has revolutionized the management and treatment of thromboembolic disorders and surgical procedures. Its discovery can be attributed to Jay Mclean (1890-1957), who identified a phosphatide anticoagulant in canine liver tissue while he was a second-year medical student at Johns Hopkins University in 1916. 3 After the introduction of heparin in the 1940s in clinical practice, it has had a significant impact on the clinical course of surgery.

RAYNAUD'S DISEASE/PHENOMENON

Raynaud's disease is a common vascular phenomenon that is recognized as a manifestation of a wide spectrum of clinical disorders. 4 Its discovery can be traced back as early as 1862 by Auguste-Maurice Raynaud (1834-1881). In his medical doctorate thesis entitled Local Asphyxia and Symmetrical Gangrene of the Extremities, he described this phenomenon. 5 Even after he completed his medical school, he continuously worked further expanding and updating his observations for the next 30 years. 6

ISLETS OF LANGERHANS

The study of the pancreas had begun in the 16 th century, but only secretory acini and ductal system were known so far. 7 Paul Langerhans (1847-1888) studied medicine in Berlin. During his time in medical school, he made two important discoveries. He described the dendritic cells in the skin, which is now commonly known as Langerhans cells. Another discovery was the pancreatic islets. Langerhans resumed his research on the pancreas’ microscopic anatomy in 1868, which he started briefly in 1867. They completed in 6 months. 8 , 9 Langerhans identified small clusters of ‘irregularly polygonal’ cells with clear cytoplasm scattered throughout the gland using pancreatic tissue of salamander, rabbit, and human. 10 , 11 Twenty four years later, French histopathologist Edouard Laguesse (1861-1927) identified the islets of Langerhans’ as the site of internal secretion of the pancreas and later named it insulin.

Charles Herbert Best (1899-1978) and Frederick Grant Banting (1891-1941) discovered insulin in 1921. After his study in the liberal arts program at the University of Toronto was interrupted due to World War I, Best switched course to physiology and biochemistry for preparation of medical degree after 1919. Banting was a 28 years old medical practitioner. John JR Macleod, a physiology teacher of Best, introduced him to Banting. They used Macleod's laboratory for their research project.

They were convinced that the extract of islets of Langerhans from a dog was crucial in preventing diabetes mellitus. They tested the pancreatic extract in a diabetic dog in 1921 and confirmed its efficacy in treating diabetes. Later, professor Macleod hired a biochemist, James Collip, to help purify the extract's active component. They were also successful in treating a diabetic patient with the extract. 12

The patient was a fourteen-year-old boy Leonard Thompson expected to live only a few more weeks. But, after they treated the boy with the extract, he lived for another 13 years. He died of complications from a road accident. In 1923, Banting and Macleod were awarded the Nobel Prize in physiology and medicine for the discovery of insulin. They later shared the award with Best and Collip, respectively. 13

SINOATRIAL NODE

Martin William Flack (1882-1931), a medical student, was spending a holiday with his lecturer by studying the hearts of moles, mice, and frogs. His anatomy lecturer, Dr. Arthur Keith (1866-1955), returned to the holiday cottage from a cycle ride and was amused by his student's finding. Flack showed him the structures he discovered in the right auricle of the heart of a mole. 14

Upon seeing the structure, Keith suspected it might be the atrioventricular node. It was identified by a Japanese anatomist, Sunao Tawara, a year before, which was confirmed by Flack and Keith themselves in an earlier study. 15 Then they studied the heart of other mammals as well. It allowed them to establish the presence of a ‘sinoatrial node’ in other mammals too. While doing so, they also found the origin of the ‘dominating rhythm of the heart. 16

Upon graduating in 1908, Flack further researched the rejuvenating effects of oxygen in athletes as a physiology lecturer in London Hospital. 17 He was appointed as the first Director of the Medical Research Council for the Royal Air Force in 1919. There he continued to investigate cardiopulmonary physiology and established techniques for assessing the physical fitness of pilot candidates.

KLUMPKE PARALYSIS

Augusta Klumpke (1859-1927) was initially destined to become a teacher until she read an article in a fashion magazine regarding a recently graduated woman in medicine in Paris. After her family moved to Paris in 1877, she admitted herself to the Faculty of Medicine. 18 As she was fluent in French, German, and English, she did not have any hard time understanding contemporary literature.

Meanwhile, she diagnosed a brachial plexus palsy associated with Horner's syndrome while working at the Hôtel-Dieu. She presented the lesions of the inferior roots of the brachial plexus as her thesis. 19 She won an Academy of Medicine prize for the research. Due to the male medical hierarchy, the award was not enough to get a hold of the internship opportunity. After repeated failed attempts and continuous lobbying, she was allowed to compete for an internship in 1886. She then became the first-ever female full intern in a Persian hospital.

SPHINCTER OF ODDI

Guggero Oddi (1864-1913) was a fourth-year medical student at the University of Perugia. He studied the actions of the sphincter present at the distal end of the common bile duct. 20 , 21 He established the relation between the sphincter and the bile's controlled flow from the liver to the duodenum. He also recommended that sphincter dysfunction might be related to biliary tract disease. He shifted to the University of Bologna from the University of Perugia as it could not award him a medical degree. At Bologna, he studied the pressure changes across the sphincter. 22

Ernest Duchesne (1874-1912) was a 23-year-old medical student at l'École du Service de Santé Militaire in Lyon, France. His graduation thesis, ‘Contribution to the study of vital competition between microorganisms: antagonism between molds and microbes,’ demonstrated the capability of the fungus Penicillium glaucum to treat harmful bacterial infections caused by Escherichia coli or Salmonella typhi. 23

He did in vivo experiments using guinea pigs. Duchesne proposed the action could be due to a toxin (antibiotic) released by the fungus. He also emphasized the therapeutic potential of the fungus. But, since Duchesne could not continue his research, his work was led to completion by Chain, Florey, and Jennings in 1942.

A medical student named William E. Clarke was the first to use ether anesthesia for surgery, as Lyman claimed. 24 This event happened in Rochester, New York, in January 1842, inside a dental clinic. It enabled the dentist to extract a tooth painlessly. But the recognition for the use of ether anesthesia is often given to others, including Crawford Williamson Long (1815-1878). He was a young doctor in Georgia. In March 1842, he used ether upon a young man to excise a small cyst from his neck painlessly. 25

SPERMATOZOA

Johan Ham (1651-1723) was a medical student from Leiden who discovered spermatozoa for the first time in 1677. He took a specimen containing the urethral discharge of a man suffering from gonorrhea to Lewenhoeck (1632-1723). In the specimen, Ham found out tiny living ‘animalcules’ with tails. 26 Leeuwenhoek, although being poorly educated, because of his passion for making lenses and studying biological tissues along with microorganisms made himself the father of microscopy. 27 , 28

Ham was often credited for the identification of spermatozoa in the semen of rooster. 26 He also pointed out the absence of spermatozoa in the semen of sterile men. 26 He also proposed that those spermatozoa cannot survive beyond 24 hours. 29 Lewenhoeck studied his own semen obtained by conjugal coitus and verified the presence of a small motile animalcule. A month later, Lewenhoeck published these findings in a letter to the Royal Society in London where he credited the discovery to Johan Ham. 29 , 30

CONCLUSIONS

These are a few notable achievements of medical science made by people while still in medical school. Lots of contributions have been made by the medical fraternity for the advancement and better understanding of modern medicine. We can now understand that students’ responsibility in medical school does not only start after they graduate. Instead, we can hope to contribute the medical science from the day we have begun our medical classes.

Whether it be a small task like assisting our professors in their fieldwork or be it some big task working on some ongoing research project, we would be contributing to the advancement of modern medicine while still studying for the same. These examples from our history keep us motivated and consistent on our path to greatness.

Conflict of Interest

• 1. Marketos SG, Skiadas PK. The modern hippocratic tradition. Some messages for contemporary medicine. Spine (Phila Pa 1976) 1999;24(11):1159–63. doi: 10.1097/00007632-199906010-00019. [ DOI ] [ PubMed ] [ Google Scholar ]
• 2. Stringer MD, Ahmadi O. Famous discoveries by medical students. ANZ J Surg. 2009;79(12):901–8. doi: 10.1111/j.1445-2197.2009.05142.x. [ DOI ] [ PubMed ] [ Google Scholar ]
• 3. McLEAN J. THE THROMBOPLASTIC ACTION OF CEPHALIN. Am J Physiol. 1916;41:250. doi: 10.1152/ajplegacy.1916.41.2.250. [ DOI ] [ Google Scholar ]
• 4. Wigley FM, Flavahan NA. Raynaud's Phenomenon. N Engl J Med. 2016;375(6):556–65. doi: 10.1056/NEJMra1507638. [ DOI ] [ PubMed ] [ Google Scholar ]
• 5. Raynaud M. De l'asphyxie locale et de la gangrène des extrémités. These, Paris: 1862. [ Google Scholar ]
• 6. Andrews J. Maurice R Raynaud and his protean disease. J Med Biogr. 1997;5(1):46–50. doi: 10.1177/096777209700500110. [ DOI ] [ PubMed ] [ Google Scholar ]
• 7. Morrison H. Contributions to the microscopic anatomy of the pancreas by Paul Langerhans (Berlin, 1869) Bull Hist Med. 1937;5:259–67. [ Google Scholar ]
• 8. Hausen BM. Paul Langerhans — Life and Work: Part I. Childhood, Early Education, and College Education. Am J Dermatopathol. 1987;9(2):151–6. doi: 10.1097/00000372-198704000-00012. [ DOI ] [ PubMed ] [ Google Scholar ]
• 9. Hausen BM. Paul Langerhans — Life and Work: Part II. Postgraduate Studies, Travels, First Signs of Disease, Madeira. Am J Dermatopathol. 1987;9(2):157–62. doi: 10.1097/00000372-198704000-00013. [ DOI ] [ PubMed ] [ Google Scholar ]
• 10. Hausen BM. Paul Langerhans — Life and Work: Part III: Scientific Research, Marriage, and Death. Am J Dermatopathol. 1987;9(3):264–9. doi: 10.1097/00000372-198706000-00015. [ DOI ] [ PubMed ] [ Google Scholar ]
• 11. Hausen BM. Paul Langerhans — Life and Work: Part IV: Publications. Am J Dermatopathol. 1987;9(3):270–5. doi: 10.1097/00000372-198706000-00016. [ DOI ] [ PubMed ] [ Google Scholar ]
• 12. Banting FG, Best CH. The internal secretion of the pancreas. Indian J Med Res. 2007;125(3):251–66. [ PubMed ] [ Google Scholar ]
• 13. Rosenfeld L. Charles Best. JAMA. 2004;291(15):1903–4. doi: 10.1001/jama.291.15.1903. [ DOI ] [ Google Scholar ]
• 14. Silverman ME, Hollman A. Discovery of the sinus node by Keith and Flack: on the centennial of their 1907 publication. Heart. 2007;93(10):1184–7. doi: 10.1136/hrt.2006.105049. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 15. Keith A, Flack M. The auriculo-ventricular bundle of the human heart. Lancet. 1906;168(4328):359–64. doi: 10.1016/S0140-6736(01)32375-9. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 16. Keith A, Flack M. The form and nature of the muscular connections between the primary divisions of the vertebrate heart. J Anat Physiol. 1907;41(Pt 3):172–89. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 17. GROUP CAPTAIN MARTIN FLACK C.B.E., M.A. Oxon., M.B., B.Ch.Oxon. Br Med J. 1931;2(3686):402–4. doi: 10.1136/bmj.2.3686.402. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ]
• 18. Satran R. Augusta Dejerine-Klumpke. First woman intern in Paris hospitals. Ann Intern Med. 1974;80(2):260–4. doi: 10.7326/0003-4819-80-2-260. [ DOI ] [ PubMed ] [ Google Scholar ]
• 19. Dejerine-Klumpke A. Des polynévrites en général et des paralysies et atrophies saturnines en particulier: étude clinique et anatomo-pathologique. Alcan. 1889 [ Google Scholar ]
• 20. Oddi R. Di una speciale disposizione a sfintere allo sbocco del coledoco. Arch ital Biol. 1886;7 doi: 10.1007/BF02471459. [ DOI ] [ PubMed ] [ Google Scholar ]
• 21. Oddi R. D'une disposition a sphincter speciale de l'ouverture du canal choledoque. Arch Ital Biol. 1887;8:317–22. [ Google Scholar ]
• 22. Oddi R. Sulla tonicita dello sfintere del coledoco. Arch Sci Med. 1888;12:333–9. [ Google Scholar ]
• 23. Duckett S. Ernest Duchesne and the concept of fungal antibiotic therapy. Lancet. 1999;354(9195):2068–71. doi: 10.1016/S0140-6736(99)03162-1. [ DOI ] [ PubMed ] [ Google Scholar ]
• 24. Lyman HM. Artificial anaesthesia and anaesthetics: W. Wood. 1881.
• 25. Rutkow IM. Crawford Williamson Long. Arch Surg. 1999;134(5):578. doi: 10.1001/archsurg.134.5.578. [ DOI ] [ PubMed ] [ Google Scholar ]
• 26. Ruestow EG. Images and ideas: Leeuwenhoek's perception of the spermatozoa. J Hist Biol. 1983;16(2):185–224. doi: 10.1007/BF00124698. [ DOI ] [ PubMed ] [ Google Scholar ]
• 27. Howards SS. Antoine van Leeuwenhoek and the discovery of sperm. Fertil Steril. 1997;67(1):16–7. doi: 10.1016/S0015-0282(97)81848-1. [ DOI ] [ PubMed ] [ Google Scholar ]
• 28. Tan S. Antoni van Leeuwenhoek (1632-1723): father of microscopy. Singapore Med J. 2003;44(11):557–8. [ PubMed ] [ Google Scholar ]
• 29. Lewenhoeck DA. Observationes D. Anthonii Lewenhoeck, De Natis E Semine Genitali Animalculis. Philosophical Transactions (1665-1678) 1677;12:1040–6. doi: 10.1098/rstl.1677.0068. [ DOI ] [ Google Scholar ]
• 30. Houtzager H. Antonie van Leeuwenhoek. Eur J Obstet Gynecol Reprod Biol. 1983;15(3):199–203. doi: 10.1016/0028-2243(83)90138-7. [ DOI ] [ PubMed ] [ Google Scholar ]
• View on publisher site
• PDF (223.1 KB)
• Collections

Similar articles

Cited by other articles, links to ncbi databases.

• Download .nbib .nbib
• Format: AMA APA MLA NLM

Add to Collections

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Recent quantitative research on determinants of health in high income countries: A scoping review

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Centre for Health Economics Research and Modelling Infectious Diseases, Vaccine and Infectious Disease Institute, University of Antwerp, Antwerp, Belgium

Roles Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing

• Vladimira Varbanova, 
• Philippe Beutels

• Published: September 17, 2020
• https://doi.org/10.1371/journal.pone.0239031
• Peer Review
• Reader Comments

Identifying determinants of health and understanding their role in health production constitutes an important research theme. We aimed to document the state of recent multi-country research on this theme in the literature.

We followed the PRISMA-ScR guidelines to systematically identify, triage and review literature (January 2013—July 2019). We searched for studies that performed cross-national statistical analyses aiming to evaluate the impact of one or more aggregate level determinants on one or more general population health outcomes in high-income countries. To assess in which combinations and to what extent individual (or thematically linked) determinants had been studied together, we performed multidimensional scaling and cluster analysis.

Sixty studies were selected, out of an original yield of 3686. Life-expectancy and overall mortality were the most widely used population health indicators, while determinants came from the areas of healthcare, culture, politics, socio-economics, environment, labor, fertility, demographics, life-style, and psychology. The family of regression models was the predominant statistical approach. Results from our multidimensional scaling showed that a relatively tight core of determinants have received much attention, as main covariates of interest or controls, whereas the majority of other determinants were studied in very limited contexts. We consider findings from these studies regarding the importance of any given health determinant inconclusive at present. Across a multitude of model specifications, different country samples, and varying time periods, effects fluctuated between statistically significant and not significant, and between beneficial and detrimental to health.

Conclusions

We conclude that efforts to understand the underlying mechanisms of population health are far from settled, and the present state of research on the topic leaves much to be desired. It is essential that future research considers multiple factors simultaneously and takes advantage of more sophisticated methodology with regards to quantifying health as well as analyzing determinants’ influence.

Citation: Varbanova V, Beutels P (2020) Recent quantitative research on determinants of health in high income countries: A scoping review. PLoS ONE 15(9): e0239031. https://doi.org/10.1371/journal.pone.0239031

Editor: Amir Radfar, University of Central Florida, UNITED STATES

Received: November 14, 2019; Accepted: August 28, 2020; Published: September 17, 2020

Copyright: © 2020 Varbanova, Beutels. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: This study (and VV) is funded by the Research Foundation Flanders ( https://www.fwo.be/en/ ), FWO project number G0D5917N, award obtained by PB. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Identifying the key drivers of population health is a core subject in public health and health economics research. Between-country comparative research on the topic is challenging. In order to be relevant for policy, it requires disentangling different interrelated drivers of “good health”, each having different degrees of importance in different contexts.

“Good health”–physical and psychological, subjective and objective–can be defined and measured using a variety of approaches, depending on which aspect of health is the focus. A major distinction can be made between health measurements at the individual level or some aggregate level, such as a neighborhood, a region or a country. In view of this, a great diversity of specific research topics exists on the drivers of what constitutes individual or aggregate “good health”, including those focusing on health inequalities, the gender gap in longevity, and regional mortality and longevity differences.

The current scoping review focuses on determinants of population health. Stated as such, this topic is quite broad. Indeed, we are interested in the very general question of what methods have been used to make the most of increasingly available region or country-specific databases to understand the drivers of population health through inter-country comparisons. Existing reviews indicate that researchers thus far tend to adopt a narrower focus. Usually, attention is given to only one health outcome at a time, with further geographical and/or population [ 1 , 2 ] restrictions. In some cases, the impact of one or more interventions is at the core of the review [ 3 – 7 ], while in others it is the relationship between health and just one particular predictor, e.g., income inequality, access to healthcare, government mechanisms [ 8 – 13 ]. Some relatively recent reviews on the subject of social determinants of health [ 4 – 6 , 14 – 17 ] have considered a number of indicators potentially influencing health as opposed to a single one. One review defines “social determinants” as “the social, economic, and political conditions that influence the health of individuals and populations” [ 17 ] while another refers even more broadly to “the factors apart from medical care” [ 15 ].

In the present work, we aimed to be more inclusive, setting no limitations on the nature of possible health correlates, as well as making use of a multitude of commonly accepted measures of general population health. The goal of this scoping review was to document the state of the art in the recent published literature on determinants of population health, with a particular focus on the types of determinants selected and the methodology used. In doing so, we also report the main characteristics of the results these studies found. The materials collected in this review are intended to inform our (and potentially other researchers’) future analyses on this topic. Since the production of health is subject to the law of diminishing marginal returns, we focused our review on those studies that included countries where a high standard of wealth has been achieved for some time, i.e., high-income countries belonging to the Organisation for Economic Co-operation and Development (OECD) or Europe. Adding similar reviews for other country income groups is of limited interest to the research we plan to do in this area.

In view of its focus on data and methods, rather than results, a formal protocol was not registered prior to undertaking this review, but the procedure followed the guidelines of the PRISMA statement for scoping reviews [ 18 ].

We focused on multi-country studies investigating the potential associations between any aggregate level (region/city/country) determinant and general measures of population health (e.g., life expectancy, mortality rate).

Within the query itself, we listed well-established population health indicators as well as the six world regions, as defined by the World Health Organization (WHO). We searched only in the publications’ titles in order to keep the number of hits manageable, and the ratio of broadly relevant abstracts over all abstracts in the order of magnitude of 10% (based on a series of time-focused trial runs). The search strategy was developed iteratively between the two authors and is presented in S1 Appendix . The search was performed by VV in PubMed and Web of Science on the 16 th of July, 2019, without any language restrictions, and with a start date set to the 1 st of January, 2013, as we were interested in the latest developments in this area of research.

Eligibility criteria

Records obtained via the search methods described above were screened independently by the two authors. Consistency between inclusion/exclusion decisions was approximately 90% and the 43 instances where uncertainty existed were judged through discussion. Articles were included subject to meeting the following requirements: (a) the paper was a full published report of an original empirical study investigating the impact of at least one aggregate level (city/region/country) factor on at least one health indicator (or self-reported health) of the general population (the only admissible “sub-populations” were those based on gender and/or age); (b) the study employed statistical techniques (calculating correlations, at the very least) and was not purely descriptive or theoretical in nature; (c) the analysis involved at least two countries or at least two regions or cities (or another aggregate level) in at least two different countries; (d) the health outcome was not differentiated according to some socio-economic factor and thus studied in terms of inequality (with the exception of gender and age differentiations); (e) mortality, in case it was one of the health indicators under investigation, was strictly “total” or “all-cause” (no cause-specific or determinant-attributable mortality).

Data extraction

The following pieces of information were extracted in an Excel table from the full text of each eligible study (primarily by VV, consulting with PB in case of doubt): health outcome(s), determinants, statistical methodology, level of analysis, results, type of data, data sources, time period, countries. The evidence is synthesized according to these extracted data (often directly reflected in the section headings), using a narrative form accompanied by a “summary-of-findings” table and a graph.

Search and selection

The initial yield contained 4583 records, reduced to 3686 after removal of duplicates ( Fig 1 ). Based on title and abstract screening, 3271 records were excluded because they focused on specific medical condition(s) or specific populations (based on morbidity or some other factor), dealt with intervention effectiveness, with theoretical or non-health related issues, or with animals or plants. Of the remaining 415 papers, roughly half were disqualified upon full-text consideration, mostly due to using an outcome not of interest to us (e.g., health inequality), measuring and analyzing determinants and outcomes exclusively at the individual level, performing analyses one country at a time, employing indices that are a mixture of both health indicators and health determinants, or not utilizing potential health determinants at all. After this second stage of the screening process, 202 papers were deemed eligible for inclusion. This group was further dichotomized according to level of economic development of the countries or regions under study, using membership of the OECD or Europe as a reference “cut-off” point. Sixty papers were judged to include high-income countries, and the remaining 142 included either low- or middle-income countries or a mix of both these levels of development. The rest of this report outlines findings in relation to high-income countries only, reflecting our own primary research interests. Nonetheless, we chose to report our search yield for the other income groups for two reasons. First, to gauge the relative interest in applied published research for these different income levels; and second, to enable other researchers with a focus on determinants of health in other countries to use the extraction we made here.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0239031.g001

Health outcomes

The most frequent population health indicator, life expectancy (LE), was present in 24 of the 60 studies. Apart from “life expectancy at birth” (representing the average life-span a newborn is expected to have if current mortality rates remain constant), also called “period LE” by some [ 19 , 20 ], we encountered as well LE at 40 years of age [ 21 ], at 60 [ 22 ], and at 65 [ 21 , 23 , 24 ]. In two papers, the age-specificity of life expectancy (be it at birth or another age) was not stated [ 25 , 26 ].

Some studies considered male and female LE separately [ 21 , 24 , 25 , 27 – 33 ]. This consideration was also often observed with the second most commonly used health index [ 28 – 30 , 34 – 38 ]–termed “total”, or “overall”, or “all-cause”, mortality rate (MR)–included in 22 of the 60 studies. In addition to gender, this index was also sometimes broken down according to age group [ 30 , 39 , 40 ], as well as gender-age group [ 38 ].

While the majority of studies under review here focused on a single health indicator, 23 out of the 60 studies made use of multiple outcomes, although these outcomes were always considered one at a time, and sometimes not all of them fell within the scope of our review. An easily discernable group of indices that typically went together [ 25 , 37 , 41 ] was that of neonatal (deaths occurring within 28 days postpartum), perinatal (fetal or early neonatal / first-7-days deaths), and post-neonatal (deaths between the 29 th day and completion of one year of life) mortality. More often than not, these indices were also accompanied by “stand-alone” indicators, such as infant mortality (deaths within the first year of life; our third most common index found in 16 of the 60 studies), maternal mortality (deaths during pregnancy or within 42 days of termination of pregnancy), and child mortality rates. Child mortality has conventionally been defined as mortality within the first 5 years of life, thus often also called “under-5 mortality”. Nonetheless, Pritchard & Wallace used the term “child mortality” to denote deaths of children younger than 14 years [ 42 ].

As previously stated, inclusion criteria did allow for self-reported health status to be used as a general measure of population health. Within our final selection of studies, seven utilized some form of subjective health as an outcome variable [ 25 , 43 – 48 ]. Additionally, the Health Human Development Index [ 49 ], healthy life expectancy [ 50 ], old-age survival [ 51 ], potential years of life lost [ 52 ], and disability-adjusted life expectancy [ 25 ] were also used.

We note that while in most cases the indicators mentioned above (and/or the covariates considered, see below) were taken in their absolute or logarithmic form, as a—typically annual—number, sometimes they were used in the form of differences, change rates, averages over a given time period, or even z-scores of rankings [ 19 , 22 , 40 , 42 , 44 , 53 – 57 ].

Regions, countries, and populations

Despite our decision to confine this review to high-income countries, some variation in the countries and regions studied was still present. Selection seemed to be most often conditioned on the European Union, or the European continent more generally, and the Organisation of Economic Co-operation and Development (OECD), though, typically, not all member nations–based on the instances where these were also explicitly listed—were included in a given study. Some of the stated reasons for omitting certain nations included data unavailability [ 30 , 45 , 54 ] or inconsistency [ 20 , 58 ], Gross Domestic Product (GDP) too low [ 40 ], differences in economic development and political stability with the rest of the sampled countries [ 59 ], and national population too small [ 24 , 40 ]. On the other hand, the rationales for selecting a group of countries included having similar above-average infant mortality [ 60 ], similar healthcare systems [ 23 ], and being randomly drawn from a social spending category [ 61 ]. Some researchers were interested explicitly in a specific geographical region, such as Eastern Europe [ 50 ], Central and Eastern Europe [ 48 , 60 ], the Visegrad (V4) group [ 62 ], or the Asia/Pacific area [ 32 ]. In certain instances, national regions or cities, rather than countries, constituted the units of investigation instead [ 31 , 51 , 56 , 62 – 66 ]. In two particular cases, a mix of countries and cities was used [ 35 , 57 ]. In another two [ 28 , 29 ], due to the long time periods under study, some of the included countries no longer exist. Finally, besides “European” and “OECD”, the terms “developed”, “Western”, and “industrialized” were also used to describe the group of selected nations [ 30 , 42 , 52 , 53 , 67 ].

As stated above, it was the health status of the general population that we were interested in, and during screening we made a concerted effort to exclude research using data based on a more narrowly defined group of individuals. All studies included in this review adhere to this general rule, albeit with two caveats. First, as cities (even neighborhoods) were the unit of analysis in three of the studies that made the selection [ 56 , 64 , 65 ], the populations under investigation there can be more accurately described as general urban , instead of just general. Second, oftentimes health indicators were stratified based on gender and/or age, therefore we also admitted one study that, due to its specific research question, focused on men and women of early retirement age [ 35 ] and another that considered adult males only [ 68 ].

Data types and sources

A great diversity of sources was utilized for data collection purposes. The accessible reference databases of the OECD ( https://www.oecd.org/ ), WHO ( https://www.who.int/ ), World Bank ( https://www.worldbank.org/ ), United Nations ( https://www.un.org/en/ ), and Eurostat ( https://ec.europa.eu/eurostat ) were among the top choices. The other international databases included Human Mortality [ 30 , 39 , 50 ], Transparency International [ 40 , 48 , 50 ], Quality of Government [ 28 , 69 ], World Income Inequality [ 30 ], International Labor Organization [ 41 ], International Monetary Fund [ 70 ]. A number of national databases were referred to as well, for example the US Bureau of Statistics [ 42 , 53 ], Korean Statistical Information Services [ 67 ], Statistics Canada [ 67 ], Australian Bureau of Statistics [ 67 ], and Health New Zealand Tobacco control and Health New Zealand Food and Nutrition [ 19 ]. Well-known surveys, such as the World Values Survey [ 25 , 55 ], the European Social Survey [ 25 , 39 , 44 ], the Eurobarometer [ 46 , 56 ], the European Value Survey [ 25 ], and the European Statistics of Income and Living Condition Survey [ 43 , 47 , 70 ] were used as data sources, too. Finally, in some cases [ 25 , 28 , 29 , 35 , 36 , 41 , 69 ], built-for-purpose datasets from previous studies were re-used.

In most of the studies, the level of the data (and analysis) was national. The exceptions were six papers that dealt with Nomenclature of Territorial Units of Statistics (NUTS2) regions [ 31 , 62 , 63 , 66 ], otherwise defined areas [ 51 ] or cities [ 56 ], and seven others that were multilevel designs and utilized both country- and region-level data [ 57 ], individual- and city- or country-level [ 35 ], individual- and country-level [ 44 , 45 , 48 ], individual- and neighborhood-level [ 64 ], and city-region- (NUTS3) and country-level data [ 65 ]. Parallel to that, the data type was predominantly longitudinal, with only a few studies using purely cross-sectional data [ 25 , 33 , 43 , 45 – 48 , 50 , 62 , 67 , 68 , 71 , 72 ], albeit in four of those [ 43 , 48 , 68 , 72 ] two separate points in time were taken (thus resulting in a kind of “double cross-section”), while in another the averages across survey waves were used [ 56 ].

In studies using longitudinal data, the length of the covered time periods varied greatly. Although this was almost always less than 40 years, in one study it covered the entire 20 th century [ 29 ]. Longitudinal data, typically in the form of annual records, was sometimes transformed before usage. For example, some researchers considered data points at 5- [ 34 , 36 , 49 ] or 10-year [ 27 , 29 , 35 ] intervals instead of the traditional 1, or took averages over 3-year periods [ 42 , 53 , 73 ]. In one study concerned with the effect of the Great Recession all data were in a “recession minus expansion change in trends”-form [ 57 ]. Furthermore, there were a few instances where two different time periods were compared to each other [ 42 , 53 ] or when data was divided into 2 to 4 (possibly overlapping) periods which were then analyzed separately [ 24 , 26 , 28 , 29 , 31 , 65 ]. Lastly, owing to data availability issues, discrepancies between the time points or periods of data on the different variables were occasionally observed [ 22 , 35 , 42 , 53 – 55 , 63 ].

Health determinants

Together with other essential details, Table 1 lists the health correlates considered in the selected studies. Several general categories for these correlates can be discerned, including health care, political stability, socio-economics, demographics, psychology, environment, fertility, life-style, culture, labor. All of these, directly or implicitly, have been recognized as holding importance for population health by existing theoretical models of (social) determinants of health [ 74 – 77 ].

https://doi.org/10.1371/journal.pone.0239031.t001

It is worth noting that in a few studies there was just a single aggregate-level covariate investigated in relation to a health outcome of interest to us. In one instance, this was life satisfaction [ 44 ], in another–welfare system typology [ 45 ], but also gender inequality [ 33 ], austerity level [ 70 , 78 ], and deprivation [ 51 ]. Most often though, attention went exclusively to GDP [ 27 , 29 , 46 , 57 , 65 , 71 ]. It was often the case that research had a more particular focus. Among others, minimum wages [ 79 ], hospital payment schemes [ 23 ], cigarette prices [ 63 ], social expenditure [ 20 ], residents’ dissatisfaction [ 56 ], income inequality [ 30 , 69 ], and work leave [ 41 , 58 ] took center stage. Whenever variables outside of these specific areas were also included, they were usually identified as confounders or controls, moderators or mediators.

We visualized the combinations in which the different determinants have been studied in Fig 2 , which was obtained via multidimensional scaling and a subsequent cluster analysis (details outlined in S2 Appendix ). It depicts the spatial positioning of each determinant relative to all others, based on the number of times the effects of each pair of determinants have been studied simultaneously. When interpreting Fig 2 , one should keep in mind that determinants marked with an asterisk represent, in fact, collectives of variables.

Groups of determinants are marked by asterisks (see S1 Table in S1 Appendix ). Diminishing color intensity reflects a decrease in the total number of “connections” for a given determinant. Noteworthy pairwise “connections” are emphasized via lines (solid-dashed-dotted indicates decreasing frequency). Grey contour lines encircle groups of variables that were identified via cluster analysis. Abbreviations: age = population age distribution, associations = membership in associations, AT-index = atherogenic-thrombogenic index, BR = birth rate, CAPB = Cyclically Adjusted Primary Balance, civilian-labor = civilian labor force, C-section = Cesarean delivery rate, credit-info = depth of credit information, dissatisf = residents’ dissatisfaction, distrib.orient = distributional orientation, EDU = education, eHealth = eHealth index at GP-level, exch.rate = exchange rate, fat = fat consumption, GDP = gross domestic product, GFCF = Gross Fixed Capital Formation/Creation, GH-gas = greenhouse gas, GII = gender inequality index, gov = governance index, gov.revenue = government revenues, HC-coverage = healthcare coverage, HE = health(care) expenditure, HHconsump = household consumption, hosp.beds = hospital beds, hosp.payment = hospital payment scheme, hosp.stay = length of hospital stay, IDI = ICT development index, inc.ineq = income inequality, industry-labor = industrial labor force, infant-sex = infant sex ratio, labor-product = labor production, LBW = low birth weight, leave = work leave, life-satisf = life satisfaction, M-age = maternal age, marginal-tax = marginal tax rate, MDs = physicians, mult.preg = multiple pregnancy, NHS = Nation Health System, NO = nitrous oxide emissions, PM10 = particulate matter (PM10) emissions, pop = population size, pop.density = population density, pre-term = pre-term birth rate, prison = prison population, researchE = research&development expenditure, school.ref = compulsory schooling reform, smoke-free = smoke-free places, SO = sulfur oxide emissions, soc.E = social expenditure, soc.workers = social workers, sugar = sugar consumption, terror = terrorism, union = union density, UR = unemployment rate, urban = urbanization, veg-fr = vegetable-and-fruit consumption, welfare = welfare regime, Wwater = wastewater treatment.

https://doi.org/10.1371/journal.pone.0239031.g002

Distances between determinants in Fig 2 are indicative of determinants’ “connectedness” with each other. While the statistical procedure called for higher dimensionality of the model, for demonstration purposes we show here a two-dimensional solution. This simplification unfortunately comes with a caveat. To use the factor smoking as an example, it would appear it stands at a much greater distance from GDP than it does from alcohol. In reality however, smoking was considered together with alcohol consumption [ 21 , 25 , 26 , 52 , 68 ] in just as many studies as it was with GDP [ 21 , 25 , 26 , 52 , 59 ], five. To aid with respect to this apparent shortcoming, we have emphasized the strongest pairwise links. Solid lines connect GDP with health expenditure (HE), unemployment rate (UR), and education (EDU), indicating that the effect of GDP on health, taking into account the effects of the other three determinants as well, was evaluated in between 12 to 16 studies of the 60 included in this review. Tracing the dashed lines, we can also tell that GDP appeared jointly with income inequality, and HE together with either EDU or UR, in anywhere between 8 to 10 of our selected studies. Finally, some weaker but still worth-mentioning “connections” between variables are displayed as well via the dotted lines.

The fact that all notable pairwise “connections” are concentrated within a relatively small region of the plot may be interpreted as low overall “connectedness” among the health indicators studied. GDP is the most widely investigated determinant in relation to general population health. Its total number of “connections” is disproportionately high (159) compared to its runner-up–HE (with 113 “connections”), and then subsequently EDU (with 90) and UR (with 86). In fact, all of these determinants could be thought of as outliers, given that none of the remaining factors have a total count of pairings above 52. This decrease in individual determinants’ overall “connectedness” can be tracked on the graph via the change of color intensity as we move outwards from the symbolic center of GDP and its closest “co-determinants”, to finally reach the other extreme of the ten indicators (welfare regime, household consumption, compulsory school reform, life satisfaction, government revenues, literacy, research expenditure, multiple pregnancy, Cyclically Adjusted Primary Balance, and residents’ dissatisfaction; in white) the effects on health of which were only studied in isolation.

Lastly, we point to the few small but stable clusters of covariates encircled by the grey bubbles on Fig 2 . These groups of determinants were identified as “close” by both statistical procedures used for the production of the graph (see details in S2 Appendix ).

Statistical methodology

There was great variation in the level of statistical detail reported. Some authors provided too vague a description of their analytical approach, necessitating some inference in this section.

The issue of missing data is a challenging reality in this field of research, but few of the studies under review (12/60) explain how they dealt with it. Among the ones that do, three general approaches to handling missingness can be identified, listed in increasing level of sophistication: case-wise deletion, i.e., removal of countries from the sample [ 20 , 45 , 48 , 58 , 59 ], (linear) interpolation [ 28 , 30 , 34 , 58 , 59 , 63 ], and multiple imputation [ 26 , 41 , 52 ].

Correlations, Pearson, Spearman, or unspecified, were the only technique applied with respect to the health outcomes of interest in eight analyses [ 33 , 42 – 44 , 46 , 53 , 57 , 61 ]. Among the more advanced statistical methods, the family of regression models proved to be, by and large, predominant. Before examining this closer, we note the techniques that were, in a way, “unique” within this selection of studies: meta-analyses were performed (random and fixed effects, respectively) on the reduced form and 2-sample two stage least squares (2SLS) estimations done within countries [ 39 ]; difference-in-difference (DiD) analysis was applied in one case [ 23 ]; dynamic time-series methods, among which co-integration, impulse-response function (IRF), and panel vector autoregressive (VAR) modeling, were utilized in one study [ 80 ]; longitudinal generalized estimating equation (GEE) models were developed on two occasions [ 70 , 78 ]; hierarchical Bayesian spatial models [ 51 ] and special autoregressive regression [ 62 ] were also implemented.

Purely cross-sectional data analyses were performed in eight studies [ 25 , 45 , 47 , 50 , 55 , 56 , 67 , 71 ]. These consisted of linear regression (assumed ordinary least squares (OLS)), generalized least squares (GLS) regression, and multilevel analyses. However, six other studies that used longitudinal data in fact had a cross-sectional design, through which they applied regression at multiple time-points separately [ 27 , 29 , 36 , 48 , 68 , 72 ].

Apart from these “multi-point cross-sectional studies”, some other simplistic approaches to longitudinal data analysis were found, involving calculating and regressing 3-year averages of both the response and the predictor variables [ 54 ], taking the average of a few data-points (i.e., survey waves) [ 56 ] or using difference scores over 10-year [ 19 , 29 ] or unspecified time intervals [ 40 , 55 ].

Moving further in the direction of more sensible longitudinal data usage, we turn to the methods widely known among (health) economists as “panel data analysis” or “panel regression”. Most often seen were models with fixed effects for country/region and sometimes also time-point (occasionally including a country-specific trend as well), with robust standard errors for the parameter estimates to take into account correlations among clustered observations [ 20 , 21 , 24 , 28 , 30 , 32 , 34 , 37 , 38 , 41 , 52 , 59 , 60 , 63 , 66 , 69 , 73 , 79 , 81 , 82 ]. The Hausman test [ 83 ] was sometimes mentioned as the tool used to decide between fixed and random effects [ 26 , 49 , 63 , 66 , 73 , 82 ]. A few studies considered the latter more appropriate for their particular analyses, with some further specifying that (feasible) GLS estimation was employed [ 26 , 34 , 49 , 58 , 60 , 73 ]. Apart from these two types of models, the first differences method was encountered once as well [ 31 ]. Across all, the error terms were sometimes assumed to come from a first-order autoregressive process (AR(1)), i.e., they were allowed to be serially correlated [ 20 , 30 , 38 , 58 – 60 , 73 ], and lags of (typically) predictor variables were included in the model specification, too [ 20 , 21 , 37 , 38 , 48 , 69 , 81 ]. Lastly, a somewhat different approach to longitudinal data analysis was undertaken in four studies [ 22 , 35 , 48 , 65 ] in which multilevel–linear or Poisson–models were developed.

Regardless of the exact techniques used, most studies included in this review presented multiple model applications within their main analysis. None attempted to formally compare models in order to identify the “best”, even if goodness-of-fit statistics were occasionally reported. As indicated above, many studies investigated women’s and men’s health separately [ 19 , 21 , 22 , 27 – 29 , 31 , 33 , 35 , 36 , 38 , 39 , 45 , 50 , 51 , 64 , 65 , 69 , 82 ], and covariates were often tested one at a time, including other covariates only incrementally [ 20 , 25 , 28 , 36 , 40 , 50 , 55 , 67 , 73 ]. Furthermore, there were a few instances where analyses within countries were performed as well [ 32 , 39 , 51 ] or where the full time period of interest was divided into a few sub-periods [ 24 , 26 , 28 , 31 ]. There were also cases where different statistical techniques were applied in parallel [ 29 , 55 , 60 , 66 , 69 , 73 , 82 ], sometimes as a form of sensitivity analysis [ 24 , 26 , 30 , 58 , 73 ]. However, the most common approach to sensitivity analysis was to re-run models with somewhat different samples [ 39 , 50 , 59 , 67 , 69 , 80 , 82 ]. Other strategies included different categorization of variables or adding (more/other) controls [ 21 , 23 , 25 , 28 , 37 , 50 , 63 , 69 ], using an alternative main covariate measure [ 59 , 82 ], including lags for predictors or outcomes [ 28 , 30 , 58 , 63 , 65 , 79 ], using weights [ 24 , 67 ] or alternative data sources [ 37 , 69 ], or using non-imputed data [ 41 ].

As the methods and not the findings are the main focus of the current review, and because generic checklists cannot discern the underlying quality in this application field (see also below), we opted to pool all reported findings together, regardless of individual study characteristics or particular outcome(s) used, and speak generally of positive and negative effects on health. For this summary we have adopted the 0.05-significance level and only considered results from multivariate analyses. Strictly birth-related factors are omitted since these potentially only relate to the group of infant mortality indicators and not to any of the other general population health measures.

Starting with the determinants most often studied, higher GDP levels [ 21 , 26 , 27 , 29 , 30 , 32 , 43 , 48 , 52 , 58 , 60 , 66 , 67 , 73 , 79 , 81 , 82 ], higher health [ 21 , 37 , 47 , 49 , 52 , 58 , 59 , 68 , 72 , 82 ] and social [ 20 , 21 , 26 , 38 , 79 ] expenditures, higher education [ 26 , 39 , 52 , 62 , 72 , 73 ], lower unemployment [ 60 , 61 , 66 ], and lower income inequality [ 30 , 42 , 53 , 55 , 73 ] were found to be significantly associated with better population health on a number of occasions. In addition to that, there was also some evidence that democracy [ 36 ] and freedom [ 50 ], higher work compensation [ 43 , 79 ], distributional orientation [ 54 ], cigarette prices [ 63 ], gross national income [ 22 , 72 ], labor productivity [ 26 ], exchange rates [ 32 ], marginal tax rates [ 79 ], vaccination rates [ 52 ], total fertility [ 59 , 66 ], fruit and vegetable [ 68 ], fat [ 52 ] and sugar consumption [ 52 ], as well as bigger depth of credit information [ 22 ] and percentage of civilian labor force [ 79 ], longer work leaves [ 41 , 58 ], more physicians [ 37 , 52 , 72 ], nurses [ 72 ], and hospital beds [ 79 , 82 ], and also membership in associations, perceived corruption and societal trust [ 48 ] were beneficial to health. Higher nitrous oxide (NO) levels [ 52 ], longer average hospital stay [ 48 ], deprivation [ 51 ], dissatisfaction with healthcare and the social environment [ 56 ], corruption [ 40 , 50 ], smoking [ 19 , 26 , 52 , 68 ], alcohol consumption [ 26 , 52 , 68 ] and illegal drug use [ 68 ], poverty [ 64 ], higher percentage of industrial workers [ 26 ], Gross Fixed Capital creation [ 66 ] and older population [ 38 , 66 , 79 ], gender inequality [ 22 ], and fertility [ 26 , 66 ] were detrimental.

It is important to point out that the above-mentioned effects could not be considered stable either across or within studies. Very often, statistical significance of a given covariate fluctuated between the different model specifications tried out within the same study [ 20 , 49 , 59 , 66 , 68 , 69 , 73 , 80 , 82 ], testifying to the importance of control variables and multivariate research (i.e., analyzing multiple independent variables simultaneously) in general. Furthermore, conflicting results were observed even with regards to the “core” determinants given special attention, so to speak, throughout this text. Thus, some studies reported negative effects of health expenditure [ 32 , 82 ], social expenditure [ 58 ], GDP [ 49 , 66 ], and education [ 82 ], and positive effects of income inequality [ 82 ] and unemployment [ 24 , 31 , 32 , 52 , 66 , 68 ]. Interestingly, one study [ 34 ] differentiated between temporary and long-term effects of GDP and unemployment, alluding to possibly much greater complexity of the association with health. It is also worth noting that some gender differences were found, with determinants being more influential for males than for females, or only having statistically significant effects for male health [ 19 , 21 , 28 , 34 , 36 , 37 , 39 , 64 , 65 , 69 ].

The purpose of this scoping review was to examine recent quantitative work on the topic of multi-country analyses of determinants of population health in high-income countries.

Measuring population health via relatively simple mortality-based indicators still seems to be the state of the art. What is more, these indicators are routinely considered one at a time, instead of, for example, employing existing statistical procedures to devise a more general, composite, index of population health, or using some of the established indices, such as disability-adjusted life expectancy (DALE) or quality-adjusted life expectancy (QALE). Although strong arguments for their wider use were already voiced decades ago [ 84 ], such summary measures surface only rarely in this research field.

On a related note, the greater data availability and accessibility that we enjoy today does not automatically equate to data quality. Nonetheless, this is routinely assumed in aggregate level studies. We almost never encountered a discussion on the topic. The non-mundane issue of data missingness, too, goes largely underappreciated. With all recent methodological advancements in this area [ 85 – 88 ], there is no excuse for ignorance; and still, too few of the reviewed studies tackled the matter in any adequate fashion.

Much optimism can be gained considering the abundance of different determinants that have attracted researchers’ attention in relation to population health. We took on a visual approach with regards to these determinants and presented a graph that links spatial distances between determinants with frequencies of being studies together. To facilitate interpretation, we grouped some variables, which resulted in some loss of finer detail. Nevertheless, the graph is helpful in exemplifying how many effects continue to be studied in a very limited context, if any. Since in reality no factor acts in isolation, this oversimplification practice threatens to render the whole exercise meaningless from the outset. The importance of multivariate analysis cannot be stressed enough. While there is no “best method” to be recommended and appropriate techniques vary according to the specifics of the research question and the characteristics of the data at hand [ 89 – 93 ], in the future, in addition to abandoning simplistic univariate approaches, we hope to see a shift from the currently dominating fixed effects to the more flexible random/mixed effects models [ 94 ], as well as wider application of more sophisticated methods, such as principle component regression, partial least squares, covariance structure models (e.g., structural equations), canonical correlations, time-series, and generalized estimating equations.

Finally, there are some limitations of the current scoping review. We searched the two main databases for published research in medical and non-medical sciences (PubMed and Web of Science) since 2013, thus potentially excluding publications and reports that are not indexed in these databases, as well as older indexed publications. These choices were guided by our interest in the most recent (i.e., the current state-of-the-art) and arguably the highest-quality research (i.e., peer-reviewed articles, primarily in indexed non-predatory journals). Furthermore, despite holding a critical stance with regards to some aspects of how determinants-of-health research is currently conducted, we opted out of formally assessing the quality of the individual studies included. The reason for that is two-fold. On the one hand, we are unaware of the existence of a formal and standard tool for quality assessment of ecological designs. And on the other, we consider trying to score the quality of these diverse studies (in terms of regional setting, specific topic, outcome indices, and methodology) undesirable and misleading, particularly since we would sometimes have been rating the quality of only a (small) part of the original studies—the part that was relevant to our review’s goal.

Our aim was to investigate the current state of research on the very broad and general topic of population health, specifically, the way it has been examined in a multi-country context. We learned that data treatment and analytical approach were, in the majority of these recent studies, ill-equipped or insufficiently transparent to provide clarity regarding the underlying mechanisms of population health in high-income countries. Whether due to methodological shortcomings or the inherent complexity of the topic, research so far fails to provide any definitive answers. It is our sincere belief that with the application of more advanced analytical techniques this continuous quest could come to fruition sooner.

Supporting information

S1 checklist. preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (prisma-scr) checklist..

https://doi.org/10.1371/journal.pone.0239031.s001

S1 Appendix.

https://doi.org/10.1371/journal.pone.0239031.s002

S2 Appendix.

https://doi.org/10.1371/journal.pone.0239031.s003

• View Article
• Google Scholar
• PubMed/NCBI
• 75. Dahlgren G, Whitehead M. Policies and Strategies to Promote Equity in Health. Stockholm, Sweden: Institute for Future Studies; 1991.
• 76. Brunner E, Marmot M. Social Organization, Stress, and Health. In: Marmot M, Wilkinson RG, editors. Social Determinants of Health. Oxford, England: Oxford University Press; 1999.
• 77. Najman JM. A General Model of the Social Origins of Health and Well-being. In: Eckersley R, Dixon J, Douglas B, editors. The Social Origins of Health and Well-being. Cambridge, England: Cambridge University Press; 2001.
• 85. Carpenter JR, Kenward MG. Multiple Imputation and its Application. New York: John Wiley & Sons; 2013.
• 86. Molenberghs G, Fitzmaurice G, Kenward MG, Verbeke G, Tsiatis AA. Handbook of Missing Data Methodology. Boca Raton: Chapman & Hall/CRC; 2014.
• 87. van Buuren S. Flexible Imputation of Missing Data. 2nd ed. Boca Raton: Chapman & Hall/CRC; 2018.
• 88. Enders CK. Applied Missing Data Analysis. New York: Guilford; 2010.
• 89. Shayle R. Searle GC, Charles E. McCulloch. Variance Components: John Wiley & Sons, Inc.; 1992.
• 90. Agresti A. Foundations of Linear and Generalized Linear Models. Hoboken, New Jersey: John Wiley & Sons Inc.; 2015.
• 91. Leyland A. H. (Editor) HGE. Multilevel Modelling of Health Statistics: John Wiley & Sons Inc; 2001.
• 92. Garrett Fitzmaurice MD, Geert Verbeke, Geert Molenberghs. Longitudinal Data Analysis. New York: Chapman and Hall/CRC; 2008.
• 93. Wolfgang Karl Härdle LS. Applied Multivariate Statistical Analysis. Berlin, Heidelberg: Springer; 2015.

Quantitative Research Methods in Medical Education

Affiliation.

• 1 From the Division of Hospital Internal Medicine (J.T.R.) Division of General Internal Medicine (A.P.S., T.J.B.), Mayo Clinic College of Medicine and Science, Department of Medicine, Mayo Clinic, Rochester, Minnesota.
• PMID: 31045900
• DOI: 10.1097/ALN.0000000000002727

There has been a dramatic growth of scholarly articles in medical education in recent years. Evaluating medical education research requires specific orientation to issues related to format and content. Our goal is to review the quantitative aspects of research in medical education so that clinicians may understand these articles with respect to framing the study, recognizing methodologic issues, and utilizing instruments for evaluating the quality of medical education research. This review can be used both as a tool when appraising medical education research articles and as a primer for clinicians interested in pursuing scholarship in medical education.

Publication types

• Research Support, Non-U.S. Gov't
• Biomedical Research / methods*
• Education, Medical / methods*
• Evaluation Studies as Topic*
• Research Design*

• Open access
• Published: 01 April 2022

Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice

• Katherine M. Wright   ORCID: orcid.org/0000-0001-5967-8156 1 ,
• Larry D. Gruppen   ORCID: orcid.org/0000-0002-2107-0126 2 ,
• Kevin W. Kuo 3 ,
• Andrew Muzyk   ORCID: orcid.org/0000-0002-6904-2466 4 ,
• Jeffry Nahmias 5 ,
• Darcy A. Reed 6 ,
• Gurjit Sandhu 7 ,
• Anita V. Shelgikar   ORCID: orcid.org/0000-0002-3629-0084 8 ,
• Jennifer N. Stojan 9 ,
• Toshiko L. Uchida   ORCID: orcid.org/0000-0002-3251-5872 10 ,
• Rebecca Wallihan 11 &
• Larry Hurtubise   ORCID: orcid.org/0000-0002-2150-8933 12  

BMC Medical Education volume  22 , Article number:  227 ( 2022 ) Cite this article

2792 Accesses

9 Altmetric

Metrics details

As a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators.

A bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes.

There were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies.

Conclusions

Overall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research.

Peer Review reports

Health professions education is a landscape of practice made up of multiple Communities of Practice (CoP) [ 1 , 2 , 3 , 4 ]. CoP are groups of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis. Barab, Barnett, and Squire stress that CoPs are persistent and develop mutual professional values and shared history [ 5 ].

Published research literature is clearly a critical component of an academic CoP. The scholarly literature reflects the three components of a CoP [ 2 ]. First, the published literature reflects the domain of a CoP. The domain is the common ground of relevant problems, topics of interest, knowledge, and practice that define the contributions and participation of members of the community. The domain has boundaries that help define the community as well as ‘leading edges’ for expanding or redirecting the domain.

Secondly, the literature reflects the community and social fabric of the CoP. As a vehicle for communication, the literature enables shared ideas, knowledge, and priorities. It also reflects the social networks within the community through collaborations and citations [ 6 ]. Thirdly, scholarly publications serve as a repository and resource of community practice. The literature is particularly important for identifying new techniques and methods, theoretical perspectives, findings, and language for the community.

CoPs change over time as new members enter into the core of the community and older members leave. They change as the domain of the community shifts and grows or shrinks (becomes more specialized). Changes in practice also changes the CoP. Many changes in the medical education CoP can be identified: the recent emphasis on competency-based education [ 7 , 8 ], newer models of faculty development [ 9 ], the comings and goings of different curricular models (systems-based, problem-based, team-based), the shift in a predominantly male community in the 1970s to an increasingly gender diverse community in the early twenty-first century, and the movement from a preponderance of quantitative research methods to a breadth of quantitative, qualitative, and mixed methods.

Although change is inevitable in a CoP and the associated scholarly literature that is part of it, we know little about the dynamics of those changes. Of particular interest in the present paper is how and how quickly the characteristics of the scholarly literature change over time. Changes in the scholarly literature may be both the result of change as well as the agent of change in the CoP. Understanding the dynamics of change in the research literature informs appropriate selection and design of interventions to improve that communication stream within the medical education CoP.

Our research question for this study is “How much and what kinds of change take place in the quality of research literature for medical education over a (11 year) period of time?” The question of change in a CoP can be challenging. One must identify a specific outcome to evaluate over some period of time but neither outcome nor time period are obvious. Gathering outcomes data over a period of time is also difficult, given the paucity of databases that preserve these kinds of data. Literature databases (e.g., MEDLINE) often serve as the data source for such studies, either through an analysis of outcomes that can be assessed over a period of time, such as the academic disciplines represented in research topics [ 10 ], or a longitudinal examination of specific topics or themes, like clinical reasoning [ 11 ].

Another methodological approach is to identify an historic study and seek to replicate it sometime later. By comparing results before and after some intervening period, investigators can make observations about changes and their potential implications. One example of this approach examined eight units of medical education research, comparing individual reports in a special issue of Academic Medicine with new interviews of the original unit directors 14 years later [ 12 ]. The investigators analyzed transitions in community characteristics such as research productivity, community membership, and goals of the community.

For the present study, we have elected to follow a similar method to this last example. We identified a major study by Reed et al. [ 13 ], which examined the methodological strengths and weaknesses of the concurrent medical education literature by analyzing studies published in 13 medical and medical education journals between September 2002 and December 2003.

Since this initial work, there has been continued growth in the numbers of medical education research journals and conferences, the number of advanced degree programs in medical education scholarship) [ 14 ], as well as the number of individuals engaged in medical education research. Regulatory agencies increasingly mandate more rigor in educational assessment and innovation [ 15 , 16 ], and the research and publication environment has become more competitive. However, it is unknown how medical education research quality has changed in tandem with these changes in the CoP.

We sought to investigate the nature and magnitude of potential changes in medical education research quality by replicating Reed, et al.’s study 11 years after the original analysis. We explored the question of whether the quality of medical education research studies would have increased, decreased or remained constant when reassessed after a period of time, using the same measures of study quality and the same journals to gauge how changes in the scholarly literature may relate to evolution of the medical education CoP.

Literature search and retrieval

An informationist with expertise in conducting literature searches guided the development of the search strategy with the goal of replicating Reed, et al. [13] using the same 13 peer-reviewed journals included in the initial study. These journals represent broad multidisciplinary medical research (JAMA, New England Journal of Medicine), seven core medical specialties (Academic Emergency Medicine, American Journal of Obstetrics and Gynecology, American Journal of Surgery, Annals of Internal Medicine, Family Medicine, Journal of General Internal Medicine, Pediatrics), as well as medical education-specific journals (Academic Medicine, Medical Education, Medical Teacher, Teaching and Learning in Medicine). The search was conducted on MEDLINE for research studies published from 9/01/2013 to 12/31/2014 to match the timeframe of the original study and included the keywords medical education and medical education research; MeSH term: Education, Medical (see appendix for full search syntax). The interval between the first and subsequent sample of the literature (11 years) reflects the time period in which the authors established their collaboration and began the time-consuming work of literature screening and abstraction and then data analysis, writing and publication. While this is not intended primarily as an indicator of current literature quality, it does provide insight into the evolution of communities of practice in medical education.

Eligibility screening

Consistent with the previous study, medical education research was operationally defined as “any original research study pertaining to medical students, residents, fellows, faculty development, or continuing medical education for physicians” [ 13 ]. Studies focusing on patient education and/or non-physician clinicians were excluded. As in the original study, additional exclusion criteria were: qualitative studies (because the MERSQI does not assess the quality indicators of qualitative studies), meta-analyses and systematic reviews, clinical reviews, letters, editorials, and reports of educational interventions without any evaluation or outcomes.

Eleven of the authors participated in the screening and review process. As an initial calibration exercise, the research team reviewed articles outside the review sample for inclusion-exclusion decision agreement. Each of the 9286 articles in the review sample was then screened by arbitrary pairs of reviewers for inclusion-exclusion decisions. Disagreements between raters were arbitrated through group discussion until consensus was achieved. A kappa coefficient was calculated to estimate rater agreement in selection screening using a sub-sample of 10% (928 papers) and demonstrated moderate agreement between raters (Cohen kappa = 0.43).

After the title and abstract screening, the full-text of all articles meeting inclusion criteria were retrieved. The same inclusion and exclusion criteria as the title/abstract screen were then applied to these full-text articles. The full-text articles that met inclusion-exclusion criteria were abstracted for the study variables.

Data abstraction

We used the Medical Education Research Study Quality Instrument (MERSQI) [ 13 ] to measure the methodological quality of medical education research studies. The MERSQI was designed to measure methodologic quality rather than the quality of reporting (but it is still dependent on the information provided in the written manuscript [ 17 ]). This instrument includes 10 items grouped into 6 domains of study quality including: study design (with options of single group cross-sectional or single group post-test only; single group pre and post-test; non-randomized, 2 group; and randomized controlled experiment), sampling (number of institutions (1, 2, or more) and response rate (< 50%, 50–74%; ≥ 75%), type of data (assessment by study subject; or objective measurement), validity evidence (internal structure, content, and relationships to other variables), data analysis (appropriateness and complexity), and outcomes (satisfaction, attitudes, perceptions, opinions, general facts; knowledge, skills; behaviors; patient/health care outcome). Each MERSQI domain has a maximum possible score of 3. Prior work documents an intraclass correlation coefficient for interrater reliability ranging from 0.72 to 0.98 for scoring the 6 domains [ 13 ].

The MERSQI has excellent inter- and intra-rater reliability in addition to strong validity evidence related to construct, content, and internal structure. The original MERSQI report has been widely cited in the medical education literature (86 citations in PubMed as of 19 January 2021). It is frequently used as a quality measure in systematic and other reviews in a wide range of medical fields [ 18 , 19 , 20 ] Validity evidence for assessing methodological and research characteristics has been reported [ 17 , 21 ].

Descriptive statistics were calculated to explore indicators of study quality. Current data were compared to the Reed, et al., [ 13 ] results using chi-square tests for relative frequency data and t-tests for comparison of mean scores. The primary outcomes were the six mean MERSQI scores for the individual categories of study quality. These were calculated by standardizing the percentage of total achievable points after accounting for “not applicable” responses. A total score was not computed for the MERSQI, following recommendations of the original authors [ 17 ] .For all analyses, a two-tailed alpha level of 0.05 was used to determine statistical significance. Effect sizes are reported for all comparisons; Cohen’s d for t-tests and h for tests of two proportions [ 22 ] (Table  1 ). Data were analyzed using SPSS version 24 for Mac (IBM Corp., Armonk, New York) and R version 3.4.3 for Mac (R Foundation for Statistical Computing, Vienna, Austria).

The Northwestern University Institutional Review Board deemed this study exempt from review (STU00205046).

Identification of studies

A total of 9286 articles were initially identified by the search. After inclusion and exclusion screening, 877 (9.4%) articles remained. Full text articles were retrieved for these 877 articles and screened again, using the same inclusion and exclusion criteria. This resulted in 482 (55.0%) articles that went on to be coded for quality using the MERSQI tool. A summary of the eligibility screening process is presented in Fig.  1 . Overall, 482 (5.2%) articles met eligibility criteria from the 2013–14 sample, compared to 210 articles (2.5%) out of 8505 total publications in the original Reed et al. study from 2002 to 03.

PRISMA flow diagram [ 23 ] showing the process for identifying and screening articles for inclusion in the study. Data were obtained from Ovid MEDLINE and included citation data for research studies published in 13 medical education journals between 2013 and 2014

Comparisons of study quality measures between 2002 and 03 vs. 2013–14

Consistent with the prior study, the highest mean domain quality score in the replication review was for the data analysis domain (mean = 2.6, SD 2.6, Table 1 ). The overall MERSQI score increased from 9.9 (SD 2.3) to 10.7 (SD 2.6) between 2002 and 03 and 2013–14 ( p  < 0.001). Of the six domains of study quality measured by the MERSQI, there were statistically significant improvements in four measures: study design, type of data, validity and outcomes. Scores that did not change significantly in the time between the two analyses were in the domains of data analysis and sampling.

The mean score on the study design domain improved from 1.3 to 1.4 ( p  < 0.01), but there were no statistically significant changes for any specific type of study design. The majority (64.1%) of designs continued to be single group cross-sectional or post-test only. Randomized control designs were still infrequent, although their relative proportion among published studies increased almost four-fold over this time period, from 2.9 to 11.0% of included studies.

For the sampling domain, the proportion of studies that were multi-institutional was stable over this period. Despite calls for more collaborative, multi-institutional research, there was little change over the intervening decade, with the majority of papers (62.2%) continuing to be single-institution studies.

The 2013–14 set of studies had a significantly greater use of objective measurements than the prior cohort of studies (45.7% of articles in 2002–03 and 54.4% of articles in 2013–14, p  < 0.001).

The reporting of validity evidence for medical education research studies was a frequent deficiency in the literature from 2002 to 03 and, although there was a statistically significant improvement (from 0.69 in 2002–03 to 1.06 in 2013–14, p  < 0.001), this was still the lowest scoring domain among all of the MERSQI dimensions (mean = 1.06 out of a possible maximum score of 3.00). The 2013–2014 analysis showed increased reporting of all three forms of validity evidence (internal structure, content, relationship to other variables) analyzed using the MERSQI, compared to the 2002–03 analysis.

The highest scores were in the domain of data analysis (2.6 and 2.6 out of 3.0, in 2002–03 and 2013–14, respectively), and these did not change significantly over the study time period ( p  = 0.22). The large majority of studies in both time periods were considered to have appropriate data analysis procedures for the data reported and most went beyond simple descriptive statistics to reflect the number and relationship among variables in the study.

Study outcome scores showed a small but statistically significant increase from the 2002–03 sample to the 2013–14 sample (means = 1.4 vs. 1.6, p  = 0.01). The most common outcomes in both samples were attitudes, satisfaction, perceptions, opinions and general facts (48.6% of articles in 2002–03 and 41.3% of articles in 2013–14, p  < .0.01)). Patient and health care outcomes were reported almost four times more frequently in the 2013–14 sample compared to the previous sample (2.4% of articles in 2002–03 and 9.1% of articles in 2013–14), yet these important outcomes were still reported in only a small fraction of medical education research studies.

The larger CoP for medical education research has changed over the past couple of decades in ways that these results may reflect. There has been an increase in the number of medical education research journals. This may have acted to decrease the number of submissions to the journals included in this study by spreading potential publications across a greater number of outlets. On the other hand, the percentage of articles meeting study inclusion criteria more than doubled from 2.5% in 2002–03 to 5.2% in 2013–14, which may indicate that these high-impact journals are attracting more high-quality submissions while less rigorous work has other outlets.

Similarly, the proliferation of professional societies and academic conferences related to medical education globally has grown significantly, which suggests that there are many more investigators producing research articles. This increased demand for journal space may have driven the increase in the number of journals, but the causal relationship is not clear.

These findings suggest that the methodological quality of quantitative medical education research improved from 2002 to 03 to 2013–14. This is encouraging, given the established need for increased methodological rigor, efforts to increase faculty skills in education research, and the recognized importance of a robust evidence base in medical education [ 24 ]. The improvement in methodologic quality reflects growth in both the domain of the CoP as well as the practice of medical education research itself.

Some of the most challenging components of study quality within medical education had notable gains between the two time periods. In particular, the inclusion of and attention to validity evidence for the measures used in the studies increased significantly from 2002 to 03 to 2013–14. The medical education research community has called for an emphasis on validity evidence for more than 20 years [ 25 , 26 , 27 , 28 ]. This, therefore, is a welcome improvement in medical education research quality as defined by the accuracy and relevance of the measurement methods used to acquire data. Reporting of patient and healthcare outcomes also increased nearly four-fold. Although only 9.1% of studies assessed patient outcomes in the 2013–14 cohort, this is an important step towards the ultimate goal of medical education—to improve health. At the same time, there was a comparable decrease in reliance on learner self-reported data such as satisfaction, opinions and self-assessments as primary outcome measures.

Our analysis also reveals that randomized controlled trials (RCTs) were being used more frequently in medical education in our analysis compared to the 2002–03, although RCTs still comprised only 11% of education studies. While RCTs are viewed as the gold standard in the clinical world, that is not necessarily the case in education. RCTs can be costly and time consuming to conduct and, in medical education, they may violate ethical principles related to withholding a potentially beneficial educational intervention from the learners who are randomized to the control arm. A well-designed quasi-experiment may generate more meaningful evidence than a poorly designed RCT. Methodological and ethical limitations unique to medical education warrant ongoing discussion around best practices in research design.

In 2013–14, nearly two thirds of education research studies still used single group designs. Single group studies are more convenient to conduct and often reflect the natural environment of education, which tends to provide curricular and teaching innovations for the entire learner group rather than segregate them into comparison conditions. Nonetheless, reliance on single-group designs hinders interpretation of the effects of the studied educational interventions.

Similarly, almost two-thirds of the studies in both samples were conducted at single institutions. This limits the generalizability of these studies to other settings, learners, and contexts. The lack of growth in multi-institutional studies over the period of this study is a concern and may partially reflect limited funding for medical education research. Indeed, in the 2002–03 cohort there was a much greater proportion of multi-institutional studies among studies with higher levels of funding, as multi-institutional collaboration facilitates rigorous, generalizable research but requires additional resources [ 29 ].

This study has several limitations. While the follow-up time period of 2013–14 is not current, the goal of this study was to examine the change in methodologic quality of medical education research and how the CoP is evolving, not to give a current snapshot of the medical education literature.

We also note that the MERSQI assesses aspects of study design, not study hypotheses or research questions. Study design needs to match the research question and single group, post study assessment may be a perfectly appropriate design for some research questions. In other words, our analyses implicitly assume that the content, focus, and questions are more or less consistent from the initial to the comparison time period. If that is not the case, changes in study design quality become more difficult to interpret. Another limitation of MERSQI is its lack of assessment of quality indicators of qualitative studies. The evolving interest in use of qualitative studies in medical education research demonstrates a shift in the CoP’s priorities, as qualitative studies have become foundational in medical education and other health professions education research.

Reviewers were not blind to the study authors or journals. We attempted to mitigate this issue by asking reviewers to recuse themselves from the review if a potential conflict of interest was noted. Additionally, inter-rater agreement on the screening decisions was only moderate (Cohen kappa = 0.43), which attenuates the ability to make statistically significant distinctions between our results and those of Reed et al. [ 13 ]. We acknowledge that our quality ratings were derived from published reports only, and publication requirements and practices (e.g., electronic appendices and other supplemental information) may limit the data that are included in publications, thereby impacting MERSQI scores. However, this was necessary to provide comparable data to Reed et al. [ 13 ].

In addition, in order to compare our data to Reed et al., our study focused solely on the journals that were included in the 2002–03 cohort. In contrast, an examination of all published education studies (across a wider array of journals) would provide useful data on the full body of medical education research. There has been a proliferation of journals that accept or are devoted to medical education research, but these new journals were excluded from this analysis to maintain consistency with the original study.

It is also very important to note that the original study and this replication only examined quantitative research. Any changes to the number and rigor of qualitative studies was not addressed in this study. To the extent that qualitative studies emphasize exploratory investigations and deeper understanding of mechanisms and phenomena, it may be that the inclusion of qualitative studies would increase the preponderance of outcomes in the attitudes, perceptions and opinions category over patient and health care outcomes.

Despite these limitations, our study may serve as a data point to chart the evolution of medical education research quality and its impact on the medical education research CoP. We found that quality improved from 2002 to 03 to 2013–14 as measured by the MERSQI. By 2013–14, a greater proportion of studies reported validity evidence and used patient-centered endpoints and more rigorous study designs. With continued attention to these areas, medical education research quality could continue to rise in coming years. Medical education research quality is positively associated with research funding [ 30 ] and this characteristic of the CoP may drive increases in resources dedicated to medical education resources. Engagement of the medical education research CoP with professional organizations, governmental and non-governmental groups may further support development of a high quality evidence base to guide medical education practice and further improve patient outcomes.

In terms of the larger question of how the research literature serves as a means of communication for the medical education CoP, these results may be a glass half full or half empty. Indeed, some characteristics of the literature show improvement over an 11-year period, yet others do not. The pace of change might also be disappointing to some who hope to see a more rapid transformation of the CoP toward an evidence base in education that supports adoption of new models of medical care, greater access to care, and a responsive educational system. Although the interpretation of these findings are open to discussion, we believe it does provide some encouragement for efforts to map the changes in the CoP with changes in one of its primary means of communicating information, values, and perspectives.

Availability of data and materials

Data were obtained from Ovid MEDLINE.

Abbreviations

Community of Practice

Medical Education Research Study Quality Instrument

Wenger EC, Snyder WM. Communities of practice: the organizational frontier. Harv Bus Rev. 2000;78:139–45. https://doi.org/10.1177/0170840603024003909 http://www.stevens-tech.edu/cce/NEW/PDFs/commprac.pdf.

Article   Google Scholar  

Wenger E, McDermott RA, Snyder W. Cultivating communities of practice: a guide to managing knowledge. Boston: Harvard Business Review Press; 2002.

Google Scholar  

van der Vleuten CPM. Medical education research: a vibrant community of research and education practice. Med Educ. 2014;48(8):761–7. https://doi.org/10.1111/medu.12508 Accessed 8 Oct 2014. http://www.ncbi.nlm.nih.gov/pubmed/25039732 .

Wenger E, Trayner-Wenger B. Communities of practice: a brief introduction. :1–8. 2015. https://doi.org/10.2277/0521663636 Accessed 2018 Dec 8. http://wenger-trayner.com/wp-content/uploads/2015/04/07-Brief-introduction-to-communities-of-practice.pdf .

Barab SA, Barnett M, Squire K. Developing an empirical account of a community of practice: characterizing the essential tensions. J Learn Sci. 2002;11(4):489–542. https://doi.org/10.1207/S15327809JLS1104_3 .

Newman MEJ. The structure of scientific collaboration networks. PNAS. 2001;98(2):404–9.

Frank JR, Snell L, Englander R, Holmboe ES. Implementing competency-based medical education: moving forward. Med Teach. 2017;39(6):568–73. https://doi.org/10.1080/0142159X.2017.1315069 https://www.tandfonline.com/doi/full/10.1080/0142159X.2017.1315069 .

Frank JR, Snell LS, Cate O ten, Holmboe ES, Carraccio C, Swing SR, Harris P, Glasgow NJ, Campbell C, Dath D, et al. 2010. Competency-based medical education: theory to practice. Med Teach 32(8):638–645. http://informahealthcare.com/doi/abs/10.3109/0142159X.2010.501190 .

Steinert Y. Faculty development in the new millennium: key challenges and future directions. Med Teach. 2000;22(1):44–50.

Norman G. Fifty years of medical education research: waves of migration. Med Educ. 2011;45(8):785–91. https://doi.org/10.1111/j.1365-2923.2010.03921.x .

Norman G. Research in clinical reasoning: past history and current trends. Med Educ. 2005;39:418–27.

Humphrey-Murto S, O’Brien B, Irby DM, van der Vleuten C, ten Cate O, Durning S, et al. 14 years later: a follow-up case-study analysis of 8 health professions education scholarship units. Acad Med. 2019;95(4):629–36. https://doi.org/10.1097/acm.0000000000003095 .

Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298:1002–9.

Tekian A, Harris I. Preparing health professions education leaders worldwide: a description of masters-level programs. Med Teach. 2012;34:52–8.

Swanson DB, Roberts TE. Trends in national licensing examinations in medicine. Med Educ. 2016;50(1):101–14. https://doi.org/10.1111/medu.12810 .

Edgar L, Roberts S, Holmboe E. Milestones 2 . 0 : A Step Forward. J Grad Med Educ. 2018:367–9.

Cook DA, Reed DA. Appraising the quality of medical education research methods: the medical education research study quality instrument and the Newcastle-Ottawa scale-education. Acad Med. 2015;90(8):1067–76. https://doi.org/10.1097/ACM.0000000000000786 .

Wasson LT, Cusmano A, Meli L, Louh I, Falzon L, Hampsey M, et al. Association between learning environment interventions and medical student well-being a systematic review. JAMA. 2016;316(21):2237–52. https://doi.org/10.1001/jama.2016.17573 .

Timberlake MD, Mayo HG, Scott L, Weis J, Gardner AK. What do we know about intraoperative teaching? Ann Surg. 2017;266(2):251–9. https://doi.org/10.1097/SLA.0000000000002131 .

Geerts JM, Goodall AH, Agius S. Evidence-based leadership development for physicians: a systematic literature review. Soc Sci Med. 2020;246:112709. https://doi.org/10.1016/j.socscimed.2019.112709 .

Sawatsky AP, Beckman TJ, Edakkanambeth Varayil J, Mandrekar JN, Reed DA, Wang AT. Association between study quality and publication rates of medical education abstracts presented at the Society of General Internal Medicine annual meeting. J Gen Intern Med. 2015;30(8):1172–7. https://doi.org/10.1007/s11606-015-3269-7 .

Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates, Inc; 1988.

Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7). https://doi.org/10.1371/journal.pmed.1000097 .

Gruppen LD, Yoder E, Frye A, Perkowski LC, Mavis B. Supporting medical education research quality: the Association of American Medical Colleges’ medical education research certificate program. Acad Med. 2011;86(1):122–6. https://doi.org/10.1097/ACM.0b013e3181ffaf84 .

Chen FM, Burstin H, Huntington J. The importance of clinical outcomes in medical education research. Med Educ. 2005;39(4):350–1. https://doi.org/10.1111/j.1365-2929.2005.02117.x .

Kalet AL, Gillespie CC, Schwartz MD, Holmboe ES, Ark TK, Jay M, et al. New measures to establish the evidence base for medical education: identifying educationally sensitive patient outcomes. Acad Med. 2010;85(5):844–51. https://doi.org/10.1097/ACM.0b013e3181d734a5 http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001888-201005000-00030 .

Chen C, Petterson S, Phillips R, Bazemore A, Mullan F. Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries. JAMA. 2014;312(22):2385–93. https://doi.org/10.1001/jama.2014.15973 http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2014.15973.

Chahine S, K Mahan K, Wright S, Monteiro S, LEM G, Barber C, et al. A call to investigate the relationship between education and health outcomes using big data. Acad Med. 2018;93(6):829–32. https://doi.org/10.1097/ACM.0000000000002217 http://insights.ovid.com/crossref?an=00001888-900000000-97954 .

Gruppen LD, Durning SJ. Needles and haystacks: finding funding for medical education research. Acad Med. 2016;91(4):480–4. https://doi.org/10.1097/ACM.0000000000000983 http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001888-900000000-98646 .

Reed DA, Kern DE, Levine RB, Wright SM. Costs and funding for published medical education research. JAMA. 2005;294:1052–7.

Download references

Acknowledgements

The authors wish to thank Emily C. Ginier, MLIS for her assistance retrieving articles.

This project was partially supported by a Central Group on Educational Affairs (CGEA) mini-grant.

Author information

Authors and affiliations.

Department of Family & Community Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

Katherine M. Wright

Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, USA

Larry D. Gruppen

Stanford University, Palo Alto, CA, USA

Kevin W. Kuo

Department of Medical Education, Duke University, Durham, NC, USA

Andrew Muzyk

University of California, Irvine, Orange, CA, USA

Jeffry Nahmias

Mayo Clinic College of Medicine and Science, Rochester, MN, USA

Darcy A. Reed

Department of Surgery, University of Michigan, Ann Arbor, MI, USA

Gurjit Sandhu

Department of Neurology, University of Michigan, Ann Arbor, MI, USA

Anita V. Shelgikar

Departments of Internal Medicine and Pediatrics, University of Michigan, Ann Arbor, MI, USA

Jennifer N. Stojan

Northwestern University Feinberg School of Medicine, Chicago, IL, USA

Toshiko L. Uchida

General Pediatrics Residency, Nationwide Children’s Hospital & The Ohio State College of Medicine, Columbus, OH, USA

Rebecca Wallihan

The Michael V. Drake Institute for Teaching and Learning, The Ohio State University, Columbus, OH, USA

Larry Hurtubise

You can also search for this author in PubMed   Google Scholar

Contributions

All authors wrote and edited the main manuscript text. K.W. analyzed data and prepared the figures. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Katherine M. Wright .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors have nothing to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wright, K.M., Gruppen, L.D., Kuo, K.W. et al. Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice. BMC Med Educ 22 , 227 (2022). https://doi.org/10.1186/s12909-022-03301-1

Download citation

Received : 15 November 2021

Accepted : 28 March 2022

Published : 01 April 2022

DOI : https://doi.org/10.1186/s12909-022-03301-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical education research
• Research quality
• Research methods
• Methodological quality

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]