The qualitative orientation in medical education research

Abstract

Qualitative research is very important in educational research as it addresses the “how” and “why” research questions and enables deeper understanding of experiences, phenomena and context. Qualitative research allows you to ask questions that cannot be easily put into numbers to understand human experience. Getting at the everyday realities of some social phenomenon and studying important questions as they are really practiced helps extend knowledge and understanding. To do so, you need to understand the philosophical stance of qualitative research and work from this to develop the research question, study design, data collection methods and data analysis. In this article, I provide an overview of the assumptions underlying qualitative research and the role of the researcher in the qualitative process. I then go on to discuss the type of research objectives which are common in qualitative research, then introduce the main qualitative designs, data collection tools, and finally the basics of qualitative analysis. I introduce the criteria by which you can judge the quality of qualitative research. Many classic references are cited in this article, and I urge you to seek out some of these further reading to inform your qualitative research program.

Introduction

When we speak of “quantitative” or “qualitative” methodologies, we are in the final analysis speaking about an interrelated set of assumptions about the social world which are philosophical, ideological, and epistemological. They encompass more than just data collection methodologies [1].

It is easy to assume that the differences between quantitative and qualitative research are solely about how data is collected—the randomized controlled trial versus ethnographic fieldwork, the cohort study versus the semi-structured interview. However, quantitative and qualitative approaches make different assumptions about the world [2], about how science should be conducted, and about what constitutes legitimate problems, solutions and criteria of “proof” [3].

Why is it important to understand differences in assumptions, or philosophies, of research? Why not just go ahead and do a survey or carry out some interviews? First, the assumptions behind the research tools you choose provide guidance for conducting your research. They indicate whether you should be an objective observer or whether you have a contributory role in the research process. They guide whether or not you must slavishly ask each person in a study the same questions or whether your questions can evolve as the study progresses. Second, you may wish to submit your work as a dissertation or as a research paper to be considered for publication in a journal. If so, the chances are that examiners, editors, and reviewers might have knowledge of different research philosophies from yours and may be unwilling to accept the legitimacy of your approach unless you can make its assumptions clear. Third, each research paradigm has its own norms and standards, its accepted ways of doing things. You need to “do things right”. Finally, understanding the theoretical assumptions of the research approach helps you recognize what the data collection and analysis methods you are working with do well and what they do less well, and lets you design your research to take full advantage of their strengths and compensate for their weaknesses.

In this short article, I will introduce the assumptions of qualitative research and their implications for research questions, study design, methods and tools, and analysis and interpretation. Readers who wish a comparison between qualitative and quantitative approaches may find Cleland [4] useful.

Ontology and epistemology

We start with a consideration of the ontology (assumptions about the nature of reality) and epistemology (assumptions about the nature of knowledge) of qualitative research.

Qualitative research approaches are used to understand everyday human experience in all its complexity and in all its natural settings [5]. To do this, qualitative research conforms to notions that reality is socially constructed and that inquiry is unavoidably value-laden [6]. The first of these, reality is socially constructed, means reality cannot be measured directly—it exists as perceived by people and by the observer. In other words, reality is relative and multiple, perceived through socially constructed and subjective interpretations [7]. For example, what I see as an exciting event may be seen as a threat by other people. What is considered a cultural ritual in my country may be thought of as quite bizarre elsewhere. Qualitative research is concerned with how the social world is interpreted, understood, experienced, or constructed. Mann and MacLeod [8] provide a very good overview of social constructivism which is a excellent starting point for understanding this.

The idea of people seeing things in diverse ways also holds true in research process, hence inquiry being valued-laden. Different people have different views of the same thing depending on their upbringing and other experiences, their training, and professional background. Someone who has been trained as a social scientist may “see” things differently from someone who has been medically trained. A woman may see things differently to a man. A more experienced researcher will see things differently from a novice. A qualitative researcher will have very different views of the nature of “evidence” than a quantitative researcher. All these viewpoints are valid. Moreover, different researchers can study the same topic and try to find solutions to the same challenges using different study designs—and hence come up with different interpretations and different recommendations. For example, if your position is that learning is about individual, cognitive, and acquisitive processes, then you are likely to research the use of simulation training in surgery in terms of the effectiveness and efficacy of training related to mastery of technical skills [9,10]. However, if your stance is that learning is inherently a social activity, one which involves interactions between people or groups of people, then you will look to see how the relationships between faculty members, participants and activities during a simulation, and the wider social and cultural context, influence learning [11,12].

Whether researchers are explicit about it or not, ontological and epistemological assumptions will underpin how they study aspects of teaching and learning. Differences in these assumptions shape not only study design, but also what emerges as data, how this data can be analysed and even the conclusions that can be drawn and recommendations that can be made from the study. This is referred to as worldview, defined by Creswell [13] as “a general orientation about the world and the nature of research that a researcher holds.” McMillan [14] gives a very good explanation of the importance of this phenomenon in relation to medical education research. There is increasing expectation that researchers make their worldview explicit in research papers.

The research objective

Given the underlying premise that reality is socially constructed, qualitative research focuses on answering “how” and “why” questions, of understanding a phenomena or a context. For example, “Our study aimed to answer the research question: why do assessors fail to report underperformance in medical students? [15]”, “The aim of this work was to investigate how widening participation policy is translated and interpreted for implementation at the level of the individual medical school [4].”

Common verbs in qualitative research questions are identify, explore, describe, understand, and explain. If your research question includes words like test or measure or compare in your objectives, these are more appropriate for quantitative methods, as they are better suited to these types of aims. Bezuidenhout and van Schalkwyk [16] provide a good guide to developing and refining your research question. Lingard [17]’s notion of joining the conversation and the problem-gap-hook heuristic are also very useful in terms of thinking about your question and setting it out in the introduction to a paper in such a way as to interest journal editors and readers.

Do not think formulating a research question is easy. Maxwell [18] gives a good overview of some of the potential issues including being too general, making assumptions about the nature of the issue/problem and using questions which focus the study on difference rather than process. Developing relevant, focused, answerable research questions takes time and generating good questions requires that you pay attention not just to the questions themselves but to their connections with all the other components of the study (the conceptual lens/theory, the methods) [18].

Theory

Theory can be applied to qualitative studies at different times during the research process, from the selection of the research phenomenon to the write-up of the results. The application of theory at different points can be described as follows [19,20,21]: (1) Theory frames the study questions, develops the philosophical underpinnings of the study, and makes assumptions to justify or rationalize the methodological approach. (2) Qualitative investigations relate the target phenomenon to the theory. (3) Theory provides a comparative context or framework for data analysis and interpretation. (4) Theory provides triangulation of study findings.

Schwartz-Barcott et al. [20] characterized those processes as theoretical selectivity (the linking of selected concepts with existing theories), theoretical integration (the incorporation and testing of selected concepts within a particular theoretical perspective), and theory creation (the generation of relational statements and the development of a new theory). Thus, theory can be the outcome of the research project as well as the starting point [22].

However, the emerging qualitative researcher may wish a little more direction on how to use theory in practice. I direct you to two papers: Reeves et al. [23] and Bordage [24]. These authors clearly explain the utility of theory, or conceptual frameworks, in qualitative research, how theory can give researchers different “lenses” through which to look at complicated problems and social issues, focusing their attention on different aspects of the data and providing a framework within which to conduct their analysis. Bordage [24] states that “conceptual frameworks represent ways of thinking about a problem or a study, or ways of representing how complex things work the way they do. Different frameworks will emphasise different variables and outcomes.” He presents an example in his paper and illustrates how different lens highlight or emphasise different aspects of the data. Other authors suggest that two theories are potentially better than one in exploring complex social issues [25]. There is an example of this in one of my papers, where we used the theories of Bourdieu [26] and Engestrom [27,28] nested within an overarching framework of complexity theory [29] to help us understand learning at a surgical bootcamp. However, I suggest that for focused studies and emerging educational researchers, one theoretical framework or lens is probably sufficient.

So how to identify an appropriate theory, and when to use it? It is crucially important to read widely, to explore lots of theories, from disciplines such as (but not only) education, psychology, sociology, and economics, to see what theory is available and what may be suitable for your study. Carefully consider any theory, check its assumptions [30] are congruent with your approach, question, and context before final selection [31] before deciding which theory to use. The time you spend exploring theory will be time well spent in terms not just of interpreting a specific data set but also to broadening your knowledge. The second question, when to use it, depends on the nature of the study, but generally the use of theory in qualitative research tends to be inductive; that is, building explanations from the ground up, based on what is discovered. This typically means that theory is brought in at the analysis stage, as a lens to interpret data.

1. Design

In the qualitative approach, the activities of collecting and analyzing data, developing and modifying theory, and elaborating or refocusing the research questions, are usually going on more or less simultaneously, each influencing all of the others for a useful model of qualitative research design [18]. The researcher may need to reconsider or modify any design decision during the study in response to new developments. In this way, qualitative research design is less linear than quantitative research, which is much more step-wise and fixed.

This is not the same as no structure or plan. Most qualitative projects are pre-structured at least in terms of the equivalent of a research protocol, setting out what you are doing (aims and objectives), why (why is this important), and how (theoretical underpinning, design, methods, and analysis). I have provided a brief overview of common approaches to qualitative research design below and direct you to the numerous excellent textbooks which go into this in more detail [32,33,34,35].

There are five basic categories of qualitative research design: ethnography, narrative, phenomenological, grounded theory, and case study [13,32].

2. Ethnography

In ethnography, you immerse yourself in the target participants’ environment to understand the goals, cultures, challenges, motivations, and themes that emerge. Ethnography has its roots in cultural anthropology where researchers immerse themselves within a culture, often for years. Through multiple data collection approaches—observations, interviews and documentary data, ethnographic research offers a qualitative approach with the potential to yield detailed and comprehensive accounts of different social phenomenon (actions, behavior, interactions, and beliefs). Rather than relying on interviews or surveys, you experience the environment first hand, and sometimes as a “participant observer” which gives opportunity to gather empirical insights into social practices which are normally “hidden” from the public gaze. Reeves et al. [36] give an excellent guide to ethnography in medical education which is essential reading if you are interested in using this approach.

3. Narrative

The narrative approach weaves together a sequence of events, usually from just one or two individuals to form a cohesive story. You conduct in-depth interviews, read documents, and look for themes; in other words, how does an individual story illustrate the larger life influences that created it. Often interviews are conducted over weeks, months, or even years, but the final narrative does not need to be in chronological order. Rather it can be presented as a story (or narrative) with themes, and can reconcile conflicting stories and highlight tensions and challenges which can be opportunities for innovation.

4. Phenomenology

Phenomenology is concerned with the study of experience from the perspective of the individual, “bracketing” taken-for-granted assumptions and usual ways of perceiving. Phenomenological approaches emphasise the importance of personal perspective and interpretation. As such they are powerful for understanding subjective experience, gaining insights into people’s motivations and actions, and cutting through the clutter of taken-for-granted assumptions and conventional wisdom.

Phenomenological approaches can be applied to single cases or to selected samples. A variety of methods can be used in phenomenologically-based research, including interviews, conversations, participant observation, action research, focus meetings, and analysis of personal texts. Beware though—phenomenological research generates a large quantity data for analysis.

The phenomenological approach is used in medical education research and there are some good articles which will familiarise you with this approach [37,38].

5. Grounded theory

Whereas a phenomenological study looks to describe the essence of an activity or event, grounded theory looks to provide an explanation or theory behind the events. Its main thrust is to generate theories regarding social phenomena: that is, to develop higher level understanding that is “grounded” in, or derived from, a systematic analysis of data [39]. Grounded theory is appropriate when the study of social interactions or experiences aims to explain a process, not to test or verify an existing theory. Rather, the theory emerges through a close and careful analysis of the data.

The key features of grounded theory are its iterative study design, theoretical (purposive) sampling, and cycles of simultaneous data collection and analysis, where analysis informs the next cycle of data collection. In keeping with this iterative design, the sample is not set at the outset but is selected purposefully as the analysis progresses; participants are chosen for their ability to confirm or challenge an emerging theory. As issues of interest are noted in the data, they are compared with other examples for similarities and differences.

Grounded theory was first proposed by Glaser and Strauss [40] in 1967 but since then there have been many interpretations of this approach, each with their own processes and norms [41,42,43].

Beware—grounded theory is often done very badly, and numerous studies are rejected by journals because they claim to use grounded theory but do not actually do so, or do so badly.

6. Case study

Researcher Yin [44] defines the case study research method as an empirical inquiry that investigates a contemporary phenomenon within its real-life context; when the boundaries between phenomenon and context are not clearly evident; and in which multiple sources of evidence are used. The case study method enables a researcher to closely examine the data within a specific context—for example, in a small geographical area or a very limited number of individuals as the subjects of study. Case studies explore and investigate contemporary real-life phenomenon through detailed contextual analysis of a limited number of events or conditions, and their relationships. A case study involves a deep understanding through multiple types of data sources. For example, we used case study methodology recently to explore the nature of the clinical learning environment in a general surgical unit, and used both documents and interviews as data sources. Case studies can be explanatory, exploratory, or describing an event [44] and case study design can be very open or more structured [45]. Case studies are a useful approach where the focus is to explain the complexities of real life situations.

While the five methods generally use similar data collection techniques (observation, interviews, and reviewing text—see below), the purpose of the study differentiates them.

Data collection methods

The qualitative methods most commonly used for research purposes can be classified in three broad categories: (1) interviews (individual or group), (2) observation methods, and (3) document review.

The qualitative research interview seeks to describe and gain understanding of certain themes in the life world of the subjects. Interviews can be organised one-to-one or group (focus groups) depending on the topic under study, the cultural context, and the aims of the project. Observational data collection in qualitative research involves the detailed observation of people and events to learn about behaviors and interactions in natural settings [46]. Such study designs are useful when the study goal is to understand cultural aspects of a setting or phenomenon [47], when the situation of interest is hidden, (tacit), or when subjects in the setting appear to have notably different views to other groups. Written materials or documents such as institutional records, personal diaries, and historical public documents may also serve as a valuable source of secondary data, providing insight into the lives and experiences of the group under study. For example, in one of my recent studies we used document analysis to uncover the thinking behind the design of a new medical school, then carried out interviews with “users” of the new building to explore how the intentions of the planners played out in reality. However, this is only one way of incorporating document analysis into a study: see Bowen [48] for an excellent introduction to the purpose and practicalities of document review within qualitative research.

See Dicicco-Bloom and Crabtree [49] for a useful summary of the content and process of the qualitative research interview, Creswell [50] for further discussion of the many different approaches in qualitative research and their common characteristics.

1. Data management

Qualitative research may use some form of quantification, but statistical forms of analysis are not central [51]. Instead, qualitative data analysis aims to uncover emerging themes, patterns, concepts, insights, and understandings [52]. The data are allowed to “speak for themselves” by the emergence of conceptual categories and descriptive themes. Trying to squeeze narratives into boxes (like “0” and “1”) would result in the loss of contextualisation and narrative layering. The researcher must immerse themselves in the data in order to be able to see meaningful patterns and themes, making notes as they go through the processes of data collection and analysis, and then using these notes to guide the analysis strategy.

Qualitative data has to be managed before it can be analysed—you can generate a lot of data from just a few interviews or observations! You may want to use a specialist qualitative database to facilitate data management and analysis. NVivo is a well-known qualitative data analysis software package (note that qualitative software packages enable you to make and store notes, and explanations of your codes, so you do not need to juggle bits of paper and electronic data files). These and similar databases are available commercially (i.e., at a cost) and are used widely by universities. The choice of database may be dictated by the resources of your institution, your personal preference, and/or what technical support is available locally. However, if you do not have access to qualitative data management software, then use paper and pencil: read and re-read transcripts, take notes on specifics and the bigger patterns, and label different themes with different coloured pen. You do all this in a software package anyway, as data management software does not describe or analyse your data for you. See Cleland et al. [53] for comprehensive guidance on how to use qualitative databases in education research.

Data analysis

While bearing in mind that qualitative data collection and analysis are iterative rather than linear (see earlier), Miles and Huberman [54] explain the process of qualitative data analysis as (1) data reduction (extracting the essence), (2) data display (organizing for meaning), and (3) drawing conclusions (explaining the findings).

Data analysis usually follows an inductive approach where the data are allowed to “speak for themselves” by the emergence of conceptual categories and descriptive themes. The researcher must be open to multiple possibilities or ways to think about a problem, engaging in “mental excursions” using multiple stimuli, “side-tracking” or “zigzagging,” changing patterns of thinking, making linkages between the “seemingly unconnected,” and “playing at it,” all with the intention of “opening the world to us in some way” [52]. The researcher must immerse themselves in the data in order to be able to see meaningful patterns and themes, making notes as they go through the processes of data collection and analysis, and then using these notes to guide the analysis strategy and the development of a coding framework.

In this way, good qualitative research has a logical chain of reasoning, multiple sources of converging evidence to support an explanation, and rules out rival hypotheses with convincing arguments and solid data. The wider literature and theory are used to derive analytical frameworks as the process of analysis develops and different interpretations of the data are likely to be considered before the final argument is built. For example, one of our own studies aimed to explore how widening access policy is translated and implemented at the level of individual medical schools [4]. Data was collected via individual interviews with key personnel. We initially conducted a primary level thematic analysis to determine themes. After the themes emerged, and following further team discussion, we explored the literature, identified and considered various theories, in some depth, before identifying the most appropriate theory or conceptual lens for a secondary, theory-driven analysis.

There are some excellent text books which discuss qualitative data analysis in detail [35,55].

Judging the quality of research

There are various criteria by which you can judge the quality of qualitative research. These link to efforts by the research team to consider their findings. The most common ways of doing so are triangulation, respondent validation, reflexivity, detail and process, and fair dealing [56] (but see also Varpio et al. [57] for a detailed discussion of the limitations of some of these methods).

Triangulation compares the results from either two or more different methods of data collection (for example, interviews and observation) or, more simply, two or more data sources (for example, interviews with different people). The researcher looks for patterns of convergence to develop or corroborate an overall interpretation. This is as a way of ensuring comprehensiveness. Respondent validation, or “member checking,” includes techniques in which the investigator’s account is compared with those of the research subjects to establish the level of correspondence between the two sets. Study participants’ reactions to the analyses are then incorporated into the study findings. Providing a clear account of the process of data collection and analysis is important. By the end of the study, it should be possible to provide a clear account of how early, simple coding evolved into more sophisticated coding structures and thence into clearly defined concepts and explanations for the data collected. Reflexivity is discussed earlier but in terms of analysis reflexivity means sensitivity to the ways in which the researcher and the research process have shaped the collected data, including the role of prior assumptions and experience. These two points address credibility, whether the study has been conducted well and the findings seem reasonable. It is important to pay attention to “negative cases,” data that contradict, or seem to contradict, the emerging explanation of the phenomena under study. These can be a very useful source of information in terms of refining the analysis and thinking beyond the obvious. The final technique is to ensure that the research design explicitly incorporates a wide range of different perspectives. In practice this can mean presenting data from a wide range of diverse participants. A very practical point is worth mentioning here—any reviewer will want to see quotes labelled in some way; for example, P11FFG2 would be participant 11, female, focus group 2). This helps the reader see that your data does not just represent the view of one or two people, but that there is indeed some sort of pattern or commonality to report.

Guba and Lincoln [58] provide the following criteria for judging qualitative research: credibility, transferability, dependability, and confirmability. I direct you to the original resource and to a very good explanation of these criteria in Mann and MacLeod [8].

Conclusion

Qualitative research is very important in educational research as it addresses the “how” and “why” research questions and enables deeper understanding of experiences, phenomena, and context. Qualitative research allows you to ask questions that cannot be easily put into numbers to understand human experience. Getting at the everyday realities of some social phenomenon and studying important questions as they are really practiced helps answer big questions. To do so, you need to understand the philosophical stance of qualitative research and work from this to develop the research question, study design, data collection methods, and data analysis.