A practical guide for conducting qualitative research in medical education: Part 2—Coding and thematic analysis

Abstract

An organized and rigorous approach to analyzing qualitative data can yield important insights into the intricacies of human behavior and can answer “how” and “why” questions. One process of coding interview responses from study subjects using grounded theory with a constructivist approach is outlined and applied to an example study. The emergent themes can provide insight to medical educators to create interventions that optimize the learning environment. For researchers, they may generate hypotheses to study by quantitative analysis.

INTRODUCTION

A well‐executed qualitative study that is conducted using a systematic approach to study design, data analysis, and interpretation can shed light on topics that are of interest to researchers¹ and allows for an in‐depth understanding of human behavior.² In medical education, this may prompt a quantitative study to test a hypothesis whose goal is to advance educational methods that could lead to improved clinical outcomes. There are multiple qualitative paradigms that align with various study questions (e.g., grounded theory, ethnography, phenomenology)³ depending on the conceptual framework outlined by the research team. Once they select the appropriate study design and collect the data, a well‐constructed, organized approach to analysis and interpretation is key.³, ⁴, ⁵ Our objective was to describe one detailed, step‐by‐step approach to analyzing qualitative data, a process often referred to as “coding.” A code is a short representation that captures the salient features of a main idea put forth by subjects through their responses in a qualitative study.⁶

To illustrate the qualitative data analysis process, we present a sample, fictional study that we have coded and analyzed. We have chosen a thematic analysis of one‐to‐one telephone interviews that received IRB approval from the study site. This process also can be applied to in‐person interviews, focus groups, or free‐response written survey data. A flow chart of the process is provided in Figure 1.

FIGURE 1.

Qualitative analysis flow chart

EXAMPLE SCENARIO

As the vice‐chair of education, you are responsible for overseeing the off‐service rotators in the emergency department (ED) and providing formative and summative feedback. You have noticed that few formal evaluations are submitted on behalf of these residents and that some complain of being treated like “second‐class citizens.” You believe their presence is valuable in fostering collaboration with EM residents. Exposure to the “EM way of thinking” may facilitate consultations and admission decisions in the future. You determine that you don't have enough information to make critical decisions to improve the experience for these rotating residents nor to meet your obligations for feedback to the residents and their departments.

Research question

How do off‐service residents rotating in the ED perceive their educational experience?

Study methods

Although several different qualitative methods could be applied in this study, a logical choice is grounded theory.⁷ The researchers' points of view (epistemology) should be decided before choosing an analytical approach. A positivist approach views the data as absolute and is the most common approach in hypothesis‐testing research. The postpositivist approach acknowledges that there is room for interpretation of absolute data, whereas the constructivist approach relies on the notion that reality is a social construct that arises through different points of view of shared experiences and that no absolute data exist.³, ⁸ Thus, it is up to the researcher(s) to identify emerging theories that arise from the data and develop a meaningful way to analyze them. In the example scenario, we apply grounded theory with a constructivist approach.

Data collection

Data collection strategies include free‐response surveys (paper or electronic), interviews (phone, videoconferencing, or in‐person), or focus groups (videoconferencing or in‐person). For this example, the research team chose to interview each of the 12 off‐service rotators. One researcher conducted individual recorded (with consent) telephone interviews that were transcribed by an assistant. The researcher read the transcripts and made corrections, including clarifying/correcting data collected (e.g., names of people, institutions, or locations mentioned) and filling in missing or unintelligible (to the transcriptionist) portions of the interview. The amended transcripts were deidentified and are ready for analysis (Appendix S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10645/full). The details of this process are outlined in part 1 of this series.⁹

DATA ANALYSIS

Objective data

Objective data (e.g., age, sex, year of training, specialty, etc.) can be extracted and collated by one individual.

Subjective data

Subjective data (i.e., the opinions shared by the interviewees) can be analyzed through the process of coding. A “code” is a short representation that captures the salient features of a main idea expressed by subjects in their responses to questions in a qualitative interview study. In the early stage, “open coding,” or identifying main ideas rather than granular details, is prudent. It is helpful to create a guide, or “codebook,” that indicates what each code represents, what ideas should be included in it, and examples from the raw data. Some codebooks include references to existing literature on the topic. It serves as a structured way to operationalize a coding scheme and can improve consistency among analysts when multiple researchers are performing the coding.¹⁰

Preliminary coding

Researchers perform an initial analysis to develop a preliminary coding scheme. To prevent reviewer fatigue, each can start with a different transcript and follow a predetermined analysis order. Some teams find it helpful to choose one particular transcript to analyze and hold a meeting to discuss their overall approach to coding that will be applied to the rest of the transcripts. While this may prevent the need to recode all of the transcripts to have an aligned approach, well‐established teams often skip this step. Additionally, software may assist teams in the coding process and is described in part 3 of this series.¹¹

Researchers can employ several different ways to keep track of their initial codes on paper or electronically. Some options include: (1) highlight relevant text with a unique color to represent each code; (2) highlight/select/underline/circle the relevant text and create a comment box in the margin that lists the applicable code word; or (3) create a table for each code and copy/paste text and line number for reference. For our sample study, Figure 2 represents our initial coding process. The most important factor is for each analyst to process the initial data in a personally meaningful way to facilitate discussion at the coding meeting.

FIGURE 2.

Sample initial coding strategies

After each analyst has independently considered each response and applied their constructivist view to the data, a meeting with all the analysts occurs. At this meeting, each explains their organizational coding strategy and consensus is reached on how to label each discrete entity and how to categorize the type of comments that belong within each broad category (Table 1).

TABLE 1.

Coding scheme example

How do you perceive your role as a team member in the ED?
Grand unifying theme: in‐group/out‐group self‐identification
Codes	Ideas to include
Unclear expectations	Educational objective absent Learner unaware of objectives
Specialty identity is …
Advantage	Knowledge and skills unique
Disadvantage	Limited knowledge scope Lack expected skills Unfairly categorized
Bias
Perceived external bias	Supervisors discount value
Internal bias	Feel inadequate
Group status
Behaviors that transcend “out‐group” status	Inclusion behaviors (how to cross over to in‐group status) Demonstrating interest
Behaviors that reinforce “out‐group” status	Avoidance Lack of self‐advocacy

A coding scheme was developed at the coding meeting where each individual coder shared preliminary coding ideas and the group synthesized all ideas into categories to be applied in the formal coding round by all coders. For convenience, the “ideas to include” column provides guidance to coders to aid in assigning the correct codes from the “Codes” column.

This is a good time to assess for saturation, which means that the ideas being expressed in each of the transcripts are fairly repetitive and not much new information is being gleaned with each new transcript.⁷ This is a safe time to stop collecting new data (i.e., no new interviews are needed). Often, the interviewer notices an absence of new ideas during the process of interviewing. The research team needs to make an informed decision as to whether to cancel already scheduled interviews or whether to proceed with the predetermined sample to ensure diverse stakeholder representation.⁹ Alternatively, if each new transcript reveals unique ideas and there is little overlap to allow the coding process to advance, more interviews are needed until no new ideas are being introduced by the subjects.

Formal coding

Once the coding template has been finalized, each of the coders rereads each transcript and assigns the agreed‐upon codes to the responses. It is helpful to perform this task on a fresh copy of the transcript to minimize bias and avoid confusing the formal codes with the initial codes. A given response may fit into more than one code, and it is generally advisable to note this and mark it as a point for discussion with the other coder(s). After the researchers have formally coded each transcript, some teams choose to have an independent researcher who is agnostic to the development of the coding scheme to apply it as an additional coder for the transcripts. This lends an extra degree of reliability to the process. Although it is beyond the scope of this paper, this is a good time to consider additional validity methods to ensure trustworthiness of the data, such as member checking during the interview phase⁹ or after data analysis¹²; triangulation using other investigational methods¹³; or by creating an external audit trail.¹⁴

Resolution of disputes

The coders meet again to review their scheme‐based coding of each of the transcripts. In many cases, comments fit neatly into a specific code and that each coder has assigned it as such. This percentage of agreement can be calculated and reported and indicates a well‐thought‐out coding scheme and careful review process. However, there are times when the coders disagree or have failed to assign a code to a data chunk. In these cases, a simple discussion allows each person to explain their thinking and arrive at a mutually acceptable decision.

Identification of themes and subthemes

After successfully coding the transcripts, the researchers can look for ideas that are similar and designate these as discrete themes. Within these broad areas there may exist subthemes. In our sample research question, several rotating residents claimed that they were treated differently than the EM residents (specialty identity theme) but some of the residents (e.g., resident 4) viewed this as a problem (subtheme = disadvantage) while others (e.g., resident 6) viewed it as a benefit (subtheme = advantage). Compare these example quotes: “Whenever there is a really cool case, they go look for one of their own interns, even if I am next up for a patient or am the only one hanging around” (Resident 4); versus “I feel really sorry for the EM interns who have to see a lot of patients and do a lot of procedures” (resident 6). Table 2 provides our final coding for the example question.

TABLE 2.

Final coding of “How do you perceive your role as a team member in the ED?”

Respondent number	Comment	Code
1	I feel like a hot potato. Nobody ever wants to hear my presentations and when someone finally gets stuck with me, they just say 'OK' and then go see the patient themselves	Perceived external bias
	They write all the orders and bond with the patient	Reinforce out‐group status
	I feel like I am not really part of the care team	Internal bias
2	Even though I am an R2, I feel like I know less than the EM interns	Internal bias
	I don't go see any challenging patients	Reinforce out‐group status
	Since I am an internal medicine resident	Specialty: Advantage
	I try to avoid all the surgical and pediatric cases as much as possible	Reinforce out‐group status
	I feel that this helps the team since I know more than they do about medicine problems and can supervise their medical students on these cases	Specialty: Advantage
3	All they ever want me to do is go see patients with skin conditions	Specialty: Disadvantage Perceived external bias
	I mean, I know I am going to be a dermatologist someday, but really, I wish they would let me see some patients who have other chief complaints. I already see plenty of derm cases in my clinic!	Perceived external bias Lack of self‐advocacy
4	I am sad that they favor their own interns over me. I always have to work in the ED during their educational conferences so they can go, but I would be really interested in going too! Whenever there is a really cool case, they go look for one of their own interns, even if I am next up for a patient or am the only one hanging around	Perceived external bias Lack of self‐advocacy
5	I love my experience in the ED! It lets me see all the cases before they're diagnosed and it really will help me on the wards to understand how all these diseases present. I keep asking the attending to let me know when exciting cases come in and sometimes I even go with the senior resident as a helper in really critical cases. They always let me know when they think there's something that might interest me	Transcend out‐group status
6	I feel overwhelmed in the ED, but the senior resident and the attending let me go at my own pace	Specialty: advantage
	They watch over everything I do and I feel this helps me learn	Specialty: advantage
	They even let me watch the resuscitations as long as I don't get in the way and they're so happy that I can be there to talk to the family of the victim while they take care of them. I really feel sorry for the EM interns who have to see a lot of patients and do a lot of procedures. It was such a great month for me!	Specialty: advantage Reinforce out‐group status Unclear expectations
7	I spend a long time each shift trying to figure out who everyone is and whether the attending on duty will let me do stuff or wants to assign me cases that they think I can handle	Unclear expectations
	It seems like every shift I am starting over. I'd really like to learn more about different presentations, but I have to prove myself every single shift before I can take on this responsibility	Perceived external bias
8	I love how the EM residents all go out after their shift together and that they include me. This makes it so much easier for me to feel like 'one of the guys' while on shift	Transcend out‐group status
	I am really going to miss this experience when I go back to the OR	Transcend out‐group status
9	I have no idea what cases to see or what's expected of me	Unclear expectations Reinforce out‐group status
10	Every single person goes out of their way to teach me. Sometimes I wish I had chosen EM as my residency! I love the variety and it seems like no case is off limits	Transcend out‐group status
11	I am afraid of adult patients	Specialty: disadvantage
	I really try to see as many kids and adolescents as I can	Specialty: advantage
	I hate to admit it, but when there are old sick people up next, I just used to go to the bathroom and wait until someone else had picked them up	Reinforce out‐group status
	It didn't really seem like anyone was keeping track of what I was doing, so this strategy worked well for me as long as they knew I showed up for my shift	Perceived external bias
12	I feel like it's us versus them. The other rotators at every level get treated one way while the EM residents seem to be treated like gods. I mean, they could hide their favoritism a little bit, couldn't they?	Perceived external bias

This table demonstrates the application of the final coding scheme (Table 1) to the question. For some statements, more than one code applies. This can be resolved during the thematic analysis.

If this had been an actual study, it is likely that there would have been additional questions as part of the interview. Each question (item) would have been coded and interpreted in a similar fashion. Depending on the questions and responses, the same codes and themes may have been relevant to additional items, but most certainly, new codes and themes would have emerged based on the intent and content of each subsequent item.

Interpreting themes to engender change

After organizing the data into themes and subthemes, the researchers must interpret the meaning within the context of the research question. In our example, the coders believed that the answers were all related to an overarching theme of belonging (in‐group) or not belonging (out‐group) as a member of the EM team during their rotation. The emergent themes centered around unclear expectations, one's specialty identity, bias, and behaviors that portended in‐ or out‐group status. In an article, the researchers would next apply the standards for reporting on qualitative research to report and interpret their results.⁵

The researcher can now use this organized information to inform the next steps to improve the educational program for rotating residents to meet the desired educational objectives. In this example, perhaps the modifications would include: (1) a formal curriculum for the rotating residents; (2) a faculty development session to provide tools to enhance supervision of off‐service residents; and (3) an orientation session for the residents to outline expectations and provide advice about successful behaviors in the ED. The researcher can then decide whether to implement changes based on this observation alone or to design a hypothesis testing study to gauge the effectiveness of an intervention. This underscores the notion that the insight gleaned from a qualitative analysis can inform understanding of teacher or learner behavior, which can lead to the development of theoretical models to explain learning events and help create best practices for instruction.¹⁵

CONCLUSION

An organized and rigorous approach to analyzing qualitative data can yield important insights into the intricacies of human behavior and can answer “how” and “why” questions to lend guidance to educators and researchers to optimize the learning environment and to generate ideas for topics of study by quantitative analysis.

Supporting information

Data Supplement S1. Supplemental material.

Coates WC, Jordan J, Clarke SO. A practical guide for conducting qualitative research in medical education: Part 2—Coding and thematic analysis. AEM Educ Train. 2021;5:e10645. 10.1002/aet2.10645