Better understanding the clinical reasoning skills of 4th-year medical students through think aloud interviews: implications for theory and practice

Abstract

Clinical reasoning skills develop through increased knowledge acquisition, greater clinical experience, and continued practice over time. Yet, across undergraduate and graduate medical education, it is inconsistently taught. As progressive clinical reasoning curricula emerge, research is needed to help inform the content and activities appropriate for different learner levels. While much is understood about the clinical reasoning skills of novices and experts, less has been theorized about students in between those two extremes. Our study explores the clinical reasoning skills of medical students in their final year of medical school, informed by clinical reasoning models and information processing theories. We conducted think-aloud interviews with 18 4th-year medical students tasked with completing a novel web-based assessment. Students reviewed simulated patient charts, answered clinically relevant questions, and justified their thinking and responses. Using a qualitative data collection and analysis framework, we coded interviews for clinical reasoning elements and emergent themes. Our findings present an initial framework for understanding the clinical reasoning skills of 4th-year medical students. The framework includes four high-level skills that we defined as interpreting, framing, generating, and justifying. These skills reflect elements of nonanalytic and analytic thinking in that students used semantic qualifiers, partially activated illness scripts, and engaged in aspects of hypothetical-deductive reasoning. Our framework can help shape how best to structure clinical reasoning instruction in medical education across the novice-to-expert continuum, as well as aid in the development of clinical reasoning theories that incorporate a range of learner levels.

Background

Touted as a keystone of medical practice, clinical reasoning is a “foundational science” that underscores all aspects of patient care (Rencic et al., 2015). Formulations of clinical reasoning typically contain a set of skills related to identifying diagnoses and treating patients. These skills often include information gathering, hypothesis generation, problem representation, differential diagnosis development, leading diagnosis identification, diagnostic justification, and management and treatment (Audétat et al., 2012; Bowen, 2006; Daniel et al., 2019; Kassirer et al., 2010). Together, they roughly outline a progression from data collection to care plan, but physicians enact different combinations of clinical reasoning skills in different sequences based on how a patient case unfolds. Research suggests that strong clinical reasoning skills may help reduce adverse patient outcomes, especially those stemming from misdiagnoses and/or mismanagement due to lapses in physician judgement and decision-making (Auerbach et al., 2024; Balogh et al., 2015; Graber et al., 2005; Kohn et al., 2000; Norman & Eva, 2010; Raffel et al., 2020; Singh et al., 2013; Trowbridge & Graber, 2015).

Expert clinical reasoning skills develop over time. This advancement is largely informed by exposure to diverse medical issues and patient populations and hands-on practice interacting with patients and their caregivers in clinical settings (Rencic et al., 2015). As such, individuals training to become physicians historically have learned and practiced clinical reasoning skills in the context of the clinical learning environment. This has occurred during clinical clerkships in the later years of medical school and across the duration of specialty-specific residency training programs. More recently, scholars and educators have argued that clinical reasoning instruction should be explicitly thread throughout the entirety of medical education and training, starting in the early years of medical school (Rencic et al., 2015, 2017; Singh et al., 2021). This call largely stems from a lack of instruction during the pre-clerkship medical school years, which focus on establishing foundational medical knowledge, and an overreliance on informal approaches during medical school clinical clerkships (Cooper et al., 2021; Rencic et al., 2015, 2017), as well as a lack of intentional long-term, longitudinal curricular designs (Cooper et al., 2021; Singh et al., 2021). In response, inquiry into current clinical reasoning instruction practices in medical school as well as the development of explicit curricular content and pedagogical approaches for use in medical school have followed suit (Connor et al., 2021; Cooper et al., 2021; Duca & Glod, 2019; Fonseca et al., 2024; Gay et al., 2013; Levin et al., 2016; Moghadami et al., 2021; Rencic et al., 2015; Singh et al., 2021; Weinstein & Pinto-Powell, 2016). For example, results of a survey of internal medicine clerkship directors at US medical schools revealed that while most directors supported the inclusion of clinical reasoning content into all phases of medical education, most schools did not include teaching sessions devoted to clinical reasoning knowledge, skills, and abilities (Rencic et al., 2017). Similarly, in the UK, survey results focused on longitudinal clinical reasoning curricula indicated that most respondents stressed the need for such curricula, yet few indicated the existence of one at their institution (Kononowicz et al., 2020).

A standard, comprehensive clinical reasoning curriculum for use in medical school would need to be tailored to the developmental progression of learners, with instruction starting in year one and building incrementally through year four. Different clinical reasoning skills overall as well as their specific elements may be appropriate for the early years of medical school prior to clinical clerkships compared to the teaching and learning that occur later in medical school during clinical clerkships. For example, it seems sensible to introduce and explore simple clinical reasoning concepts in the context of typical presentations of common conditions at the start of medical school and to increase the complexity of the concepts, presentations, and conditions across all medical school years (Rencic et al., 2015; Singh et al., 2021). This progression can help build pattern recognition skills which would allow for the development of increasingly more detailed understandings of specific illnesses, which would help establish a solid base from which to enter residency (Rencic et al., 2015).

The recent development and implementation of a longitudinal, spiral clinical reasoning curriculum in a single UK medical school provides a promising example of how clinical reasoning content can be incorporated into existing medical school curriculum (Singh, et al., 2021). The development and implementation of the curriculum stems from a systematic, evidence-based approach grounded in (a) clinical reasoning frameworks, (b) teaching, learning, and assessment theories, (c) curriculum development best practices and processes, and (d) faculty development initiatives. The integration of the curriculum reflects a reconfiguration of existing structures and a reframing of existing teaching and assessment materials, offering an approach for embedding clinical reasoning instruction and assessment throughout various stages of undergraduate medical education.

When developing progressive curricula, it is important to align content with conceptual models of clinical reasoning that shed light on how clinical reasoning processes unfold and how clinical reasoning skills develop (Rencic et al., 2015; Singh et al., 2021). Such illuminations can lend credibility to decisions made about the appropriate aspects of clinical reasoning and the complexity of clinical content to stress at different learner levels over the course of medical school. They also can help to identify the associated expected behaviors of medical students over time. Moreover, empirical investigations of the use of clinical reasoning skills at different learner levels can be evaluated against existing theoretical frameworks as well as contribute to their continued refinement.

Fourth-year medical students represent a unique group of learners in that they are not complete clinical reasoning novices, yet they are far from clinical reasoning experts. This liminal expertise aligns more with diagnostic reasoning (i.e., reasoning skills related to the formulation and defense of a differential diagnosis) than to management reasoning (i.e., reasoning skills related to short- and long-term treatment of patients and their health concerns), as this latter aspect of clinical reasoning requires greater experience than typically afforded 4th-year medical students. Having gained some clinical experience, 4-th year medical students will transition to residency training where they will receive greater hands-on involvement with diverse patients presenting with varied symptoms and be expected to safely and effectively apply what they learned in medical school. Older studies have examined the diagnostic justification abilities of 4th-year medical students, defined as how students explained moving from their initial differential diagnosis to their final diagnosis (Williams & Klamen, 2012; Williams et al., 2014). Yet, little is known about the nuances of the clinical reasoning skills of contemporary 4th-year medical students; their clinical reasoning skills relative to their position along the novice-to-expert continuum have been understudied and undertheorized.

This paper investigates how 4th-year medical students employ the diagnostic components of clinical reasoning when asked to review simulated patient charts, answer clinically relevant questions, and justify their thinking and responses. We examine the extent to which conceptual models of clinical reasoning align with how students in their final year of medical school think through clinical cases. Below we describe the conceptual models that best aligned with students’ thought processes. Our analysis can shed light on how 4th-year students translate what they learn into how they think, with implications for medical school curricula and for the advancement of theories of clinical reasoning appropriate for different learner levels more broadly.

Conceptual models

Developed over decades, conceptual models of clinical reasoning draw elements from various theories of cognition. Different cognition models may be useful for understanding distinct aspects of clinical reasoning at various learner levels. We focus on information processing theories, where dual process theory has been perhaps the most overarching and impactful to date. Dual process theory asserts that two primary processes guide clinical reasoning: System 1 or nonanalytic thinking and System 2 or analytic thinking (Croskerry, 2009; Eva, 2005; Kahneman, 2011; Pelaccia et al., 2011; Marcum, 2012; Norman & Eva, 2010; Ratcliffe & Durning, 2015; Rencic et al., 2015; Stanovich & West, 2000). In this framework, physicians are theorized to use each process to varying degrees and in varying sequences depending on the clinical task(s) at hand and the clinical context(s) in which the task(s) needs to be accomplished.

The first process type, System 1 or nonanalytic thinking, refers to fast, intuitive, and automatic information processing (Kahneman, 2011). Nonanalytic thinking in the context of clinical reasoning relies on the breadth and depth of past clinical experiences, the ability to detect meaningful patterns in patient data, and the quick, subconscious linking of the two (Ratcliffe & Durning, 2015). Physicians use various interrelated mechanisms to create these links. Tacit knowledge, heuristics, and illness scripts can be viewed as three important ones.

Tacit knowledge refers to understanding through the automatic interpretation of implicit cues in the context of caring for patients and does not require careful consideration of how to handle a particular patient’s case, as thinking and doing occur concurrently (Marcum, 2012; Thornton, 2006). Heuristics refer to specific mental shortcuts that physicians enact subconsciously to focus their understanding and evaluation of a patient’s condition(s) (Ratcliffe & Durning, 2015). Semantic qualifiers can be thought of as a type of heuristic commonly utilized in medical practice and reflect broad categorizations that help physicians interpret, organize, and classify patient information (Bowen, 2006; Ratcliffe & Durning, 2015; Shea & Chan, 2023). Illness scripts represent a list-like schematic used to organize and interpret information and are specific to the discipline of medicine (Bowen, 2006; Custers, 2015; Moghadami et al., 2021; Ratcliffe & Durning, 2015). They refer to shared mental models of illnesses – individually held knowledge structures or representations of a particular disease or condition shared across a group of healthcare providers (Floren et al., 2018; McComb & Simpson, 2014).

Nonanalytic thinking largely relies on pattern recognition and involves a holistic approach to patient care based on overall perceptions of patients and their symptoms (Marcum, 2012; Ratcliffe & Durning, 2015). Tacit knowledge, heuristics, and illness scripts represent three important, interrelated aspects of nonanalytic thinking (Marcum, 2012; Ratcliffe & Durning, 2015), but they do not constitute an exhaustive list. Moreover, they do not necessarily occur in isolation from each other nor in the absence of analytic thinking.

The second process type outlined in dual process theory, System 2 or analytic thinking, refers to slow, deliberate, and reflective information processing (Croskerry, 2009; Eva, 2005; Kahneman, 2011; Pelaccia et al., 2011; Marcum, 2012; Norman & Eva, 2010; Ratcliffe & Durning, 2015; Rencic et al., 2015; Stanovich & West, 2000). Analytic thinking in the context of clinical reasoning highlights the ability to formulate problems and generate hypotheses and collect and use patient data to evaluate them. Based on those evaluations, subsequent refining of problem representations (concise summaries that highlight the key features of a patient case) (Bowen, 2006; Daniel et al., 2019) and hypotheses can occur as physicians learn new information about a patient from, for example, test results or the development and discovery of new symptoms. The hypothetical-deductive model, causal reasoning, and Bayesian reasoning represent three central methods by which physicians have been theorized to enact analytic thinking when caring for patients.

Hypothetico-deductive reasoning involves answering a question or solving a problem through a logical sequence of generating theoretical possibilities and supporting or refuting them with observed data (Marcum, 2012; Ledford & Nixon, 2015; Ratcliffe & Durning, 2015). In causal reasoning, physicians connect existing clinical knowledge with clinical observations in a linear, cause-and-effect chain to structure their reasoning and support their decisions (Kuipers & Kassirer, 1984; Ledford & Nixon, 2015; Rizzi, 1994; Shin, 2019). Physicians evoke Bayesian reasoning when they base clinical decisions on the probability that a patient has a medical condition, which is informed by existing patient data and new patient data they obtain (Ratcliffe & Durning, 2015; Rencic et al., 2015; Rottman et al., 2016).

Analytic thinking, in general, involves a deliberate, measured approach to patient care based on a synthesis and evaluation of patient data relative to medical knowledge and clinical experience (Ratcliffe & Durning, 2015). Hypothetico-deductive reasoning, causal reasoning, and Bayesian reasoning represent three important types of analytic thinking, but other types also exist. Within a dual process theory, seasoned physicians are thought to move back and forth between analytic and nonanalytic thinking effortlessly, whereas junior practitioners are believed to rely more on analytic thinking.

Methods

With the above discussion of clinical reasoning and information processing models in mind, our study explores the skills 4th-year medical students employed when completing an assessment designed to measure diagnostic reasoning. To better understand medical students’ diagnostic reasoning processes, we conducted think-aloud interviews with 18 4th-year medical students from 13 Liaison Committee on Medical Education (LCME)-accredited US medical schools between November 2021 and March 2022. During a think-aloud interview, participants are asked to verbalize their thoughts as they perform a task and/or solve a problem (Ericsson & Simon, 1993; Johnson et al., 2023). Underlying cognitive processes are then inferred from participants’ verbalized thoughts, which can then be analyzed using qualitative frameworks (Johnson et al., 2023). For our study, we asked medical students to report everything that went through their minds as they completed an assessment designed to measure clinical reasoning skills at the level of a 4th-year medical student, especially a student’s ability to justify their thinking and decision-making. This study was reviewed and approved by the American Institutes for Research Institutional Review Board (Project # EX00564).

Participant recruitment

We invited 4th-year medical students from 218 LCME-accredited US/Canadian medical schools to participate in the study via email. The email invitation specified that to be eligible, students needed to have (a) completed United States Medical Licensing Examination (USMLE)^® Step 1, and (b) completed USMLE Step 2 Clinical Knowledge (CK) no more than six months prior to the study or signed up to take USMLE Step 2 CK by the end of 2021. Six hundred and eighty-three students responded to the email invitation and met the participation criteria. Based on a convenience sampling approach (Miles & Huberman, 2019), we invited a subset of these students to participate in the study. We did this in a rolling manner such that we continued to receive student responses to the original invitation after we began scheduling interviews. Moreover, participant selection and data analysis occurred concurrently until no new themes relevant to our research question emerged from the data, indicating that we reached theoretical saturation (Glaser & Strauss, 1967; Miles & Huberman, 2019; Saunderset al., 2018). Students were offered a $100 Amazon gift card for completing the interview.

Data collection

The assessment that students completed during the interviews included 12 Short Answer Rational Provision (SHARP) items. Developed by NBME, SHARP items ask students to make clinical decisions about a patient’s care and then to provide evidence in support of those decisions. Specifically, students are presented with a patient chart and asked an open-ended question querying some element of the patient’s case (e.g., What is the most likely diagnosis? or What test is most likely to confirm the diagnosis?). Students must enter their responses using free text in a blank text box. After a student enters their response, they are presented with the same patient chart and asked to highlight the elements of the chart that best support the answer they just provided. Figure 1 provides an example of the second step of the SHARP item format including supporting evidence highlighted in gray. See Runyon et al. (2024) for a detailed description of the development and use of the SHARP item format.

Fig. 1.

SHort Answer Rationale Provision (SHARP) item format justification task example

Participants were provided with information about the study and instructions for completing the SHARP items at the start of their interview. The instructions asked participants to treat the assessment task as if they were in a high-stakes test setting. To gain familiarity with the item format and the think-aloud activity, participants first completed two practice items including content unrelated to medicine. Participants then completed the SHARP item assessment, verbalizing their thoughts as they did so. The SHARP items included in the assessment represented a range of chief medical concerns, patient acuities, and clinical contexts. The assessment was presented via the online survey platform Qualtrics, and the interviews took place over Zoom. Each student answered the same 12 items, although in different sequences. Interviews were not timed and lasted, on average, 83 min (range of 40 to 114 min). Interviews were audio/video recorded and transcribed.

Data analysis

Working within a qualitative research framework, we used both deductive and inductive approaches to analyze the think-aloud data (Braun & Clarke, 2006, 2022; Miles & Huberman, 2019; Saldaña, 2013). We began with a deductive tactic where we created an initial codebook based on existing conceptual models of clinical reasoning (e.g., Bowen, 2006; Daniel et al., 2019) and input from physician educators with expertise teaching and/or researching clinical reasoning. The initial codebook included 7 major deductive codes: information gathering, hypothesis generation, problem formation, differential diagnosis, leading or working diagnosis, diagnostic justification, and management/treatment. The early use of this predefined set of codes allowed for interpretation of linkages between students’ verbalizations of what they thought and vetted understandings of clinical reasoning processes from the literature.

To prepare for coding, one author (MMC) parsed each transcribed interview into meaningful, codable segments based on observed substantive breaks in thinking and linguistic pauses in speech patterns as suggested by students’ voices (Someren et al., 1994; Yang, 2003). Subsequent deductive and inductive coding occurred at the segment level.

Two authors (MMC and UALD) used the initial codebook to code two interviews using the deductive codes, assigning multiple codes to a single segment of text when appropriate. Based on discussions of their coding processes and resulting codes, we simplified some codes and collapsed others in an inductive manner to include emergent themes and better fit the data. For example, it became clear that the apriori code of management/treatment was largely irrelevant for our purposes since the SHARP item format focuses on diagnostic reasoning and as such students did not consider management reasoning to any meaningful degree when navigating them. However, students did engage in several justification strategies to support their thinking when completing the items that were not explicitly part of our initial set of codes, but that we considered important to capture.

Next, seven authors (CR, JL, MMC, PH, SI, SS, UALD) independently coded the same excerpts from three new interviews. Again, through discussion they derived new codes and further clarified existing ones. Lastly, three physician educators with expertise in clinical reasoning independently offered input on the codebook and its use in analyzing the interviews. This resulted in the final codebook, which included both major and minor codes, as well as descriptions of their relationships (Saldaña, 2013).

Using the final codebook, seven authors (CR, JL, MMC, SI, SS, RS, UALD) worked in pairs to fully code 18 interviews. Each pair was assigned a distinct interview such that two authors independently coded each interview. No two authors worked together exclusively. Pairs met to reconcile coding discrepancies through discussion and finalize codes. Regular discussion sessions took place with the entire team of authors, allowing them to voice any feedback, inquiries, or issues. Coding and data analysis occurred simultaneously. Analysis included elements of the family of methods known as thematic analysis in its focus on deductive coding, inductive coding, and reflexivity broadly defined as the critical reflection of researcher perspectives and experiences and how they inform the collection, analysis, and interpretation of qualitative data (Braun & Clarke, 2019).

Throughout the coding and analysis process, we utilized analytic memos to refine codes and categorize them into themes which we then examined for recurring connections (Birks et al., 2008; Saldaña, 2013). More specifically, we wrote memos to make sense of the codes we generated while developing the codebook, as well as to understand how codes clustered and related across the interviews. For example, we labeled an early code “Recognizing Normal/Abnormal Findings” and through memoing reconfigured it conceptually to reflect the broader process of information interpretation including evaluations of patient information along multiple dimensions, not only normal/abnormal test results. One memo by one author (MMC) stated,

“Recognizing Normal/Abnormal Findings” refers to when a student interprets a piece of information from the chart. Often, but not always, the student is interpreting a lab result or chart/image. Here interpretation means making a judgement or providing an evaluation. It doesn’t have to be explicitly about something being within or outside normal range. Key words include normal, abnormal, high, low, old, young, weird, etc. While this [sub]code is currently under Framing the Problem it probably should be under Interpreting Information.

Ultimately, we recategorized students’ evaluations of patient information as indications of an aspect of the patient’s condition (i.e., interpreting information), rather than as demonstrative of the patient’s case (i.e., framing the problem). In this way, memoing helped us draw distinctions between how students decipher specific pieces of patient information and how they understand the patient and their condition(s) as a whole.

We illustrate themes with direct quotations, accompanied by participant numbers in parentheses. When needed, we refined quotations to correct grammatical errors and reduce verbal fillers (e.g., um, ah) to enhance readability (Lingard, 2019). These modifications were made without changing the original meaning of the quotations.

Reflexivity

We considered the role of reflexivity in two central ways. First, after coding an interview, authors wrote a reflective statement that documented how certain factors informed their coding process: (a) interpersonal reactions to a student participant, (b) assumptions about and understandings of clinical reasoning and/or medical students broadly, and (c) educational, professional, and personal backgrounds (Olmos-Vega et al., 2022). When reconciling their codes, coding pairs shared and discussed their reflective statements, using them to shape their conversations and better understand discrepancies when they emerged. Second, we held recurring, discussion-based meetings with all coders which allowed them to share their experiences, delineate commonalities and discrepancies, and increase understanding of the codebook and emerging themes over the course of the study (Olmos-Vega et al., 2022). Often, insights drawn from memos were incorporated into the discussion. We acknowledge that as employees of NBME we hold perceived power over the medical students who participated in our study and that that power may have informed how students responded (Olmos-Vega et al., 2022).

Findings

Sample

The sample included 18 4th-year medical students from 13 US LCME-accredited medical schools. At the time of their interviews, all students had completed Step 1 and either completed Step 2 CK in the last six months or scheduled to take Step 2 CK in the remainder of 2021. This helped to ensure that students were generally comparable in terms of learner level. All students identified English as their primary language, suggesting that language comprehension issues were minimal.

Diagnostic reasoning skills framework

Our thematic analysis revealed that students in our study demonstrated four primary clinical reasoning skills when asked to think aloud while completing an assessment of diagnostic reasoning. We developed a framework to describe these four skills: interpreting, framing, generating, and justifying. Figure 2 provides a visual representation of the full framework.

Fig. 2.

Fourth-year medical student diagnostic reasoning skills framework

As shown, while a logical flow from interpreting to framing to generating to justifying exits, all skills reinforced each other, and students engaged in them dynamically.

Interpreting

Interpreting refers to when students make clinical sense of the information provided in the patient chart. Students in our sample interpreted patient information in three main ways: they filtered it, sought it, and evaluated it.

When filtering information, students identified pieces of patient data that they considered important in the context of the patient’s case, as well as pieces of data that they considered irrelevant to understanding the clinical problem at hand. When reviewing the history of a 37-year-old woman in a medical office who frequently gets winded, one student noted:

Had an intermittent, nonproductive cough and wheezing, especially when she has colds during the winter. Father is alive with emphysema, diagnosed in his early 40s. Now that’s an important thing, just because emphysema was diagnosed so early on. (10)

Another student disregarded a piece of information and highlighted another when responding to a 62-year-old male in urgent care having chest pain:

He has smoked one pack of cigarettes daily for 40 years and drinks three cocktails on weekends. I’m not too concerned about the alcohol use in this case. It doesn’t seem that relevant to what he’s presenting with. I’m more concerned with his cigarette use, which gives him increased risk for COPD. (05)

In dismissing and underscoring information, this student prioritized the patient data they found most useful for understanding the case. Filtering goes beyond simply reading select pieces of information in the order they are presented in the patient chart; it involves students’ verbal specification of a piece of information as useful or not useful to understanding the case and completing the item.

When seeking information, students deliberately looked for specific pieces of information to clarify and/or confirm a thought process, hypothesis, and/or a question answer. When presented with a 62-year-old female in a medical office who complains of being tired and cold and having a cough that sometimes produces blood, one student considered her presentation in this way:

She has rheumatoid arthritis too. I do want to know what medication she’s on because she’s probably on immunosuppressing medications that can [make it easier to get] infections. Medications. Looks like she’s on insulin and stuff for blood pressure and cholesterol. Here we go, methotrexate and prednisone, so we already have someone who has a pre-disposition to be immunocompromised from being on an immunosuppressant for RA. (06)

Once this student saw in the patient’s history that they suffered from rheumatoid arthritis, they directed their inquiry and navigated to the medications section of the chart to gather the answer to a specific question. Other students realized the chart did not include information they deemed important and indicated that knowing that information would benefit their evaluation of the patient. In reaction to a 72-year-old male in the hospital who complains that it is hard to breath whenever he does anything, one student opined, “I would love to have more information here. I’d love to have a BMP. I’d love to have this patient’s [electro]lytes. I’d love to see the chest X-ray for myself and not have the interpretation.” (07) When seeking information, students generally navigated to the section of the chart in which the information they sought was most likely to be listed, reflecting a familiarity with the typical structure of patient charts.

When evaluating information, students reviewed clinical findings in the chart and either used clinical language to interpret them or classified them dichotomously (e.g., normal/abnormal, low/high). One student assessed a very weak 47-year-old man in the emergency department like this:

On his exam, he’s tachycardic, he’s febrile, he’s not saturating very well. And then as time goes on, he becomes more tachycardic, he becomes hypotensive and I’m very worried that he’s in distributive shock. Let’s see. So, signs of him being in shock. His creatine is up. His hemoglobin is down, but that might just be chronic. He has some elevated neutrophil count. But then lymphocytes, monocytes, EOS, EOS are a little bit high too. (13)

This student uses medical vocabulary to establish meaning and draw inferences from findings (e.g., tachycardic, febrile, not saturating), as well as specific findings as higher or lower than expected or outside normal ranges.

Framing

Framing refers to when students characterize the key clinical issues represented in a case. Students in our sample used two central techniques to frame a patient case: they classified the clinical issues, and they summarized the case.

When classifying clinical issues, students used existing broad categories and/or frameworks (e.g., organ system, chronic/acute, infectious/non-infectious, viral/bacterial) to help narrow their focus. Often students classified clinical issues early in their review of the patient chart based on basic information related to the patient’s chief medical concern and medical history. This allowed for a more targeted review of subsequent information presented later in the chart. Faced with an older male in the emergency department, one student commented:

Probably some sort of acute concern. It looks like his legs are all swollen, and he can’t breathe. So, right off the bat, we have a 70-year-old male. Swollen legs. Can’t breathe. We’re thinking about heart disease, right off the bat. We are thinking about cardiac etiologies, primarily. We could also be thinking about lung pathology. We could be thinking about liver etiologies as well. (11)

Here, the student starts with situating the case within the acute realm and then further narrows their focus using broad system-based categories. Classifying clinical issues does not include the articulation of hypotheses related to specific diagnoses. Rather, as demonstrated by this student, it involves a broad assessment of the patient’s presentation from which a student can build. It is within classifying clinical issues that students tended to use semantic qualifiers to evaluate and compartmentalize data.

When summarizing a case, students provided a brief overview of the patient presentation and key clinical issues. To do this, they tended to stitch together different pieces of information from various places in the chart, deliberately pausing to articulate a curated collection of patient data. Often, simple case summaries evolved into more complex problem statements, but not always. One student summarized the case of the 47-year-old man in the emergency room in this way:

So, I’m just going to go back over and synthesize what I have. 47-year-old. Very weak. Fevers and chills. Three-hour history of weakness. So, it’s a very acute shortness weakness, but he’s also had fevers and chills and nasal congestion achiness, and stiffness. (10).

While this student provides a fair amount of information, students varied in the level of detail they provided when summarizing a case. Moreover, they tended to summarize cases at various times throughout their reasoning processes, often dependent on the content of the case.

Generating

Generating refers to when students produce ideas about what is happening with the patient clinically. Students in our sample generated ideas using three central approaches. They proposed hypotheses, managed discordant information, and determined final diagnoses.

When proposing hypotheses, students typically identified a collection of possible diagnoses. One student offered two initial options in response to learning about a 32-year-old male in a medical office setting complaining of heel pain, “You know he’s an avid runner. So, I think about running complications like planter fascia, tarsal tunnel syndrome, et cetera.” (08) Proposing hypotheses is a more specific activity than when students classified clinical issues to frame a clinical problem. Often students classified clinical issues first, and then proposed hypotheses, moving from a broad understanding of the patient and their concerns to a narrower one. As students read increasingly more of a patient chart, this sequence was often repeated in an iterative way. In addition to proposing viable hypotheses as part of their reasoning processes, students also ruled them out. Regarding the 70-year-old male having trouble breathing in the emergency department, one student reasoned: “No shortness of breath lying down, helps rule out heart failure.” (01) Proposing hypotheses includes the identification or rejection of possible conditions and/or diagnoses and often was intertwined with justifying, but not always.

When managing discordant information, students identified and sometimes reconciled information that countered a working hypothesis. In relation to a 20-year-old male in a medical office who is having stomach pains, one student reasoned:

We also see here [that] he has some shortness of breath, which is not quite in line with a fully GI-based symptomatology. So now we’re kind of thinking about maybe adding in a differential of a VIPoma, which would give us the diarrhea as well as some of the lung symptoms. So, we’re kind of broadening that side of the differential. (11)

Here, the student uses the discordant information to reframe their thinking and advance their understanding of the case by broadening their list of possible diagnoses. In contrast, other students simply identified discordant information and did not attempt to account for it nor integrate it into their reasoning. Reviewing the same case, another student noted:

A 20-year-old male in the office, so less acute, having stomach problems. Three-month history of moderate epigastric pain, nausea, decreased appetite, watery, non-bloody diarrhea, shortness of breath, headache. I feel like in a 20-year-old, it’s kind of weird to have this three-month history of symptoms, especially intermittent diarrhea and nausea. (15)

This student recognizes a possible disconnect between the patient’s age and their symptoms but stops there. They do not elaborate further on what the disconnect could mean or how it could inform their thinking.

When determining a final diagnosis, students relied on different amounts of patient data and spent different amounts of time finalizing their response. The following two students arrived at the same final diagnosis for a 28-year-old female who fainted, but in quite diverse ways. The first student carefully considered multiple possibilities and analyzed patient data to support or refute them. They arrived at their final diagnosis in the following manner:

Said she did not feel well. She had this prodromal syndrome, again sounds like she had a vagal episode. And then she regained consciousness immediately. So, I don’t think she had a seizure. She’s having no other episodes like that. Let’s see. And then nothing else [is] going on. Initially when she comes in, she’s normotensive, she’s not tachycardic, she’s saturating well, she’s like looking well, no abnormalities. So then looking at her orthostatic test I guess what I’m looking for is whether systolic dropped by 20, diastolic by 10 and then the pulse increased by 10. It doesn’t look like she fits the criteria [for] orthostatic hypotension. So, I don’t think she’s volume down. So, then we look at her blood work and [I’m] very interested to see if she’s anemic and she’s not. Otherwise, things look fine. So, I think she had a vagal episode. (12)

In contrast, the second student arrived at their final diagnosis through an incomplete review of the patient chart, mostly referencing data that confirmed their thinking. They arrived at their final diagnosis like this:

A 28-year-old female who fainted. Patient’s a medical student. Vasovagal syncope. Lost consciousness during class teaches phlebotomy technique. I really don’t even think I need to read the rest of the question at this point, but I’ll scan it all just in case. Scanning everything down here, blah, blah, blah, pulse, a little bit different while standing versus sitting, but not anything, not orthostatic hypertension. Again, with the blood pressure. Okay. Yeah. I don’t need any of this, so let’s see, EKG, nothing, good. It rules out a cardiac cause, and blood hemoglobin’s fine. Coming down here, everything looks basically fine. (04)

While the above examples are somewhat extreme, they provide useful contrasting illustrations of variation in how students finalized their diagnoses. As might be expected, students often used more patient data and took more testing time with complex cases and used fewer patient data and took less testing time with more straightforward ones.

Justifying

Justifying refers to when students articulate rationales for their proposed hypotheses. Students in our sample used four primary strategies to support their reasoning. They identified associations, explained associations, thought probabilistically, and prioritized evidence.

When identifying associations, students connect patient information with conditions and/or diagnoses to substantiate their thinking. Patient information can include the presence or absence of a single piece of patient information, as well as the presence or absence of a collection of patient data. The argument typically unfolds where given a specific finding, a certain condition and/or diagnosis is thought to also be present. Considering an 8-year-old female in the emergency department with a cough and fever similar to her last visit to the ER, one student commented:

So, young female coming to the emergency department with this recurrent cough, fever picture; productive of sputum. So maybe like some sort of bacterial pneumonia, viral pneumonia, those are things I’m initially thinking about. (12)

Linking the recurring symptoms of fever and productive cough to the condition of pneumonia, this student offers a reason for their hypothesis of pneumonia grounded in known associations between patient concerns and a diagnosis. Similarly, one student assessed a 57-year-old male in the emergency department who is extremely winded. They noted, “He has pleural effusion and edema. I think it’s also helpful that it’s [been happening for] two months. It’s shortness of breath on exertion, which is like classic for CHF.” (15) Both students demonstrated elements of pattern recognition, typical of how students tended to identify associations. At times, students referenced these relationships as causal. In reaction to the 20-year-old male in the medical office having stomach issues, one student surmised: “I think he has a parasitic infection where he now has these parasites that are in his GI system causing the diarrhea, causing the shortness of breath, and then causing headache.” (05) In this example, the student hypothesizes a parasitic infection and links it back to three symptoms that they think such an infection can cause. In all instances of identifying associations, students focused on a few key features suggestive of a condition and/or diagnosis, without providing much explanation about the mechanisms underlying the identified associations.

When explaining associations, students include portrayals of the biological processes and/or complex clinical relationships underlying an association they identified. Upon learning that the 70-year-old man presenting with breathing difficulties and swollen legs in the emergency room was recently diagnosed with lung cancer and has an elevated jugular venous pressure (JVP), one student posited, “I still think that’s related to the lung cancer, because if it’s all backing [up] from vena circulation then that includes the vena circulation from the head and so if that’s backed up, that can certainly cause JVP as well.” (06) Here the student does not merely connect lung cancer and JVP. Rather, they offer the connection in the context of how the two conditions are biologically linked through venous circulation. To assess the extremely winded 57-year-old man in the emergency department, one student used their basic science understanding of the mechanisms underlying two types of heart failure to select and ultimately justify a final diagnosis:

There’s systolic and diastolic heart failure. Systolic being like there’s structural things and you’re not squeezing enough to get the volume out. And then diastolic is like your heart is kind of stiff. He’s pretty young and he has this history of hypertension. So, I’m thinking this is probably more of a systolic failure because he’s like pumping against a more high-pressure system. So, it’s harder to pump. Systolic could happen if you have a heart attack or something like that. He hasn’t had one. Diastolic you can also get if you’re pumping against a high-pressure system, because then your heart is going to hypertrophy [which] is not good. What’s his ejection fraction? I’m trying to think pathophysiologically what would that look like? Is his heart not squeezing enough or is it stiff because of the hypertrophy? I think maybe if it’s uncontrolled hypertension, it might actually be diastolic because he doesn’t have a history of any sort of heart attack that would make his muscle weaker. (15)

This student references biological processes related to heart muscle strength and the ability to pump blood when linking hypertension and heart failure. By providing additional descriptive clinical information and context for an identified association, students demonstrated a slower, more deliberate process compared with pattern recognition.

When thinking probabilistically, students relied on their knowledge of risk factors, likelihoods, epidemiology, and/or statistics to substantiate their reasoning. In these instances, students often utilized a probability lens to generate and justify hypotheses. When considering pulmonary embolism (PE) as a diagnosis in the case of the 62-year-old male in urgent care, one student based their decision on a general assessment of risk factors, weighing evident risk factors against absent ones, “I think [this] changes things a little because on the physical exam you don’t really have diminished breath sounds for PE and he doesn’t really have any crazy risk factors for PE besides the fact that he smoked cigarettes.” (15) Another student evoked broad knowledge of population health statistics to support their diagnosis of cystic fibrosis for the 8-year-old girl in the emergency room, “It’s associated with a lot of failure to thrive, and I think it’s actually one of the most common genetic conditions in the white patient population.” (18) When determining a diagnosis for the winded 57-year-old man in the emergency department, one student referenced epidemiology as the reason for their selection, although they did not expand on the specifics:

Am I concerned that he has bacterial meningitis? Because he has achy, stiff … doesn’t say anything about his neck; doesn’t specifically say he got confused. I don’t think so. I think more due to him coming in and being hypotensive in septic shock, I think he has Strep pneumo. I think he could also have Neisseria meningitis, but I think Strep pneumo [is] most likely, just based on epidemiology. (13)

When prioritizing evidence, students identified, and sometimes ranked, pieces of patient information included in the patient chart that best substantiated their reasoning. Not surprisingly, for each case, students most often prioritized evidence during the second part of the items included in the assessment when they were asked to select the patient data that best support their free-text response. When selecting supporting evidence, one student noted the following in relation to the 72-year-old man in the hospital having trouble breathing:

I would probably choose his three-week history first, and then I would probably choose the weight gain. Well, but the weight could be suggestive of multiple things. I think the fact that he has edema of the lower extremities is maybe more specific to the diagnosis. (11)

This student ranks two pieces of patient information and chooses them both. Upon reflection, they opt for the one that is more clearly related to the patient’s presentation, reflecting an evaluation of the strength of clinical relationships as part of their prioritization process. Another student, along with a few others, prioritized patient data by implicitly or explicitly valuing “objective” data over “subjective” data. As one student contended when justifying their thinking about the 32-year-old male with heel pain, “I think it’s important that you have some physical exam data to support your diagnosis, so I’m going to put one of those things.” (06) Here, the student privileges empirical physical exam data over other types of potentially subjective information, such as information that the patient might have verbalized during the history. Many students selected information in the chart to support their reasoning and responses without much additional commentary other than that the selected information was helpful. The bulk of when students verbalized their reasoning tended to happen when students reviewed the chart and responded to an open-ended question, rather than when they reviewed the chart to select supporting evidence.

Fourth-year medical student diagnostic reasoning skill patterns

Fourth-year medical students demonstrated each of the four skills in diverse ways such that students utilized multiple strategies and demonstrated multiple behaviors within each skill. Across the clinical cases, all students engaged in all skills, although to varying degrees and in different sequences. Different sets of skills were also used across students based on the case. A typical pattern included the enactment of interpreting and framing concurrently, where each informed the other, followed by generating, given synthesis of the outcomes of interpreting and framing, all in recurrent loops over the course of the review of a patient chart. Most often justifying occurred in tandem with generating, but to varying degrees and in varying ways across students and cases. Figure 3 illustrates examples of the types of patterns students demonstrated when evoking each skill. It presents six students and two clinical cases to showcase how students iterated among skills and how skills clustered.

Fig. 3.

Diagnostic reasoning skill patterns by medical student and clinical case

Patterns suggest that students often engaged in interpreting and framing at the start of a case and that the two skills often clustered together. The ways in which students 1 and 3 navigated SHARP item 1 are examples. Generating and justifying tended to occur after interpreting and framing and often hung together. The ways in which student 6 navigated SHARP item 1 and the ways in which student 5 navigated SHARP item 5 provide illustrations. Iterative loops are evident where the rough sequence of interpreting, framing, generating, and justifying occurs multiple times over the course of a case. Student 4’s performance on SHARP item 2 illustrates this process. While some students used similar approaches across cases (e.g., Student 6), others employed significantly different thought processes for different cases (e.g., Student 4, who moved quickly through the first patient chart, suggesting the use of non-analytic reasoning, but adopted a more complex strategy in the second, suggesting the use of a more analytic strategy).

Discussion

Clinical reasoning is a multifaceted construct comprised of complex, intertwined processes and skills that underpin medical practice. As such, clinical reasoning instruction needs to be deliberately and explicitly incorporated into both undergraduate and graduate medical education in a progressive manner. Better understanding medical students’ reasoning processes may help to inform the development and implementation of clinical reasoning curricula across learners’ medical education journeys. Such understanding may also serve to refine existing theories of clinical reasoning, especially regarding the development of skills in the spaces between novice and expert.

Conceptual insights

Through a qualitative analysis of think-aloud interviews, we developed an initial framework for better understanding the clinical reasoning skills of 4th-year medical students. This framework includes four high-level skills that students in our sample exhibited while navigating patient charts to answer clinical questions in the context of a novel assessment format. Based on a combination of existing models and emerging themes, we defined these skills as interpreting, framing, generating, and justifying.

Interpreting

The first major component of our framework, interpreting, involves students making clinical sense of information provided in a patient chart and aligns most closely with the information gathering and hypothesis generation elements of clinical reasoning as documented in the literature. Although students in our sample could not actively engage with patients to obtain information about their concerns, students filtered information, classifying it as important or irrelevant to a case, akin to how they might evaluate patient information obtained through a history and physical examination in an actual clinical encounter. In addition, as students reasoned through a case, they actively sought patient information they considered useful, but did not see in the chart, based on a progressive understanding of the patient and their health concerns. This speaks, at least somewhat, to a student’s ability to know the types of information to collect to better inform their identification and justification of an appropriate diagnosis, consistent with a hypothetical-deductive approach.

Students also evaluated findings and test results - often one by one - to ascertain if they indicated a problem. This aspect of interpreting is not always explicitly highlighted in definitions of clinical reasoning, although often it is assumed to occur throughout the reasoning process. It is possible that it represents an aspect of clinical reasoning deliberately utilized by learners with some, but not extensive, clinical experience. Fourth-year medical students may carefully review findings and test results, evaluating each as normal or abnormal to help them understand patients’ concerns in more measured ways, reflecting analytic thinking, than seasoned physicians who might enact those same processes in more holistic ways, reflecting nonanalytic thinking.

Framing

A second major component of our framework, framing, involves students characterizing the primary clinical issues represented in a case and aligns most closely with problem representation. Students in our sample classified a case in broad strokes using pre-existing sets of categories, like semantic qualifiers, to help narrow their focus, often early in their review of a patient chart. Classifying a patient’s condition(s) as acute, subacute, or chronic based on information gleaned from taking a history and performing a physical examination allows a physician to narrow the scope of clinical possibilities and weigh them against other emergent patient information. The inclusion of semantic qualifiers in learners’ reasoning processes can contribute to their development of robust illness scripts, another way in which physicians use nonanalytic thinking to care for patients (Ratcliffe & Durning, 2015; Rencic et al., 2015).

Illness scripts typically include three components: enabling conditions, fault(s), and consequences (Feltovich & Barrows, 1984). Enabling conditions refer to patient and contextual factors that influence the likelihood of a disease, fault(s) refer to the biomedical misfires that contribute to a disease, and consequences refer to the manifestations of those misfires including concerns, signs, and symptoms (Feltovich & Barrows, 1984). Physicians are thought to develop a growing collection of increasingly comprehensive illness scripts with increased exposure to illnesses and greater clinical experiences with diverse patient populations over the course of their education and practice (Ledford & Nixon, 2015). Illness script theory posits that physicians utilize nonanalytic thinking to activate illness scripts based on verbal and nonverbal cues from clinical presentations and interactions with patients (Bowen, 2006; Custers, 2015; Moghadami et al., 2021; Shea & Chan, 2023). This activation is theorized to occur through comparing pre-existing networks of information organized into categories with real-life encounters (Bowen, 2006; Custers, 2015; Moghadami et al., 2021; Shea & Chan, 2023). Multiple illness scripts can be activated at the same time wherein analytic thinking is used to compare them in light of known and emerging patient information to arrive at a final diagnosis.

Framing, especially the use of semantic qualifiers, and to a lesser extent the accessing of illness scripts, primarily reflects the heuristics element of nonanalytic thinking within a dual process theory of cognition. Students quickly mapped their interpretation of a patient case to existing categorization schemes familiar to them to help streamline their focus and ultimately their reasoning. While students used heuristics, it is unclear whether they relied on tacit knowledge, or knowledge rooted in sensory and motor experiences, another element of nonanalytic thinking. This is not surprising given the stage of their education and training and the assessment task they were asked to complete.

As physicians gain more medical knowledge, amass more clinical experience, and interact with a greater range of patients, their use of tacit knowledge is thought to increase (Marcum, 2012). Here, clinical expertise becomes embodied and manifested in habitual ways – like muscle memory, where the body infused with learnings from past experiences automatically knows and remembers how to act in specific medical situations (Marcum, 2012). Our findings suggest that 4th-year medical students may employ nonanalytic thinking in terms of heuristics learned through instruction, but perhaps not in terms of embodied pattern recognition learned through experience.

To a lesser extent, students provided succinct clinical summaries of cases that included an account of key findings within the realm of framing. A few students deliberately paused to review and synthesize what they had learned, verbally acknowledging that they were taking stock, which is more indicative of analytic thinking. While most students demonstrated some elements of problem representation, in general, comprehensive accounts of patients’ backgrounds and conditions were rare. This might indicate a lack of clear focus in this area in current medical school curricula and/or highlight a developmental area that takes longer to build than other aspects of clinical reasoning.

Generating

The third major component of our framework, generating, involves students producing ideas about what is happening with the patient clinically and aligns most closely with hypothesis generation and differential diagnosis development. Students proposed possible hypotheses similarly to how hypothesis generation is described in existing models of clinical reasoning.

Historically, hypothetic-deductive reasoning has been used across diverse scientific disciplines and is generally proposed to involve multiple phases. In medicine, an early phase entails observing cues and collecting and interpreting initial information about a patient and their chief medical concern. Physicians use those data to generate one or more initial, non-specific hypotheses pertaining to the causes of the patient’s issues and/or their diagnosis. In later phases, physicians test, refine, and rank their hypotheses based on thoughtful analysis and synthesis of additional relevant patient information in an iterative fashion. This involves repeated assessment of the patient’s condition to confirm or refute working diagnoses. Ultimately, physicians arrive at the most likely diagnosis and the most appropriate treatment plan that they can best support with available clinical evidence.

In the context of generating, students adjusted their thinking as they read, interpreted, and framed additional information from the patient chart mostly in line with hypothetico-deductive reasoning principles. Students reasoned in this way to varying degrees of complexity and comprehensiveness, and to varying degrees of success. Students generally did not consistently nor fully develop differential diagnoses when reviewing a patient chart to determine a final diagnosis and answer a clinically relevant question. More often, they provided one or two viable diagnoses early and upon further review of the patient chart, sometimes incorporated other possibilities.

When faced with clearly contradictory data, students managed the discordance primarily through analytic thinking. Students slowed down, carefully considered how best to make sense of observed discrepancies and either arrived at what they considered a reasonable explanation or abandoned the endeavor altogether if they failed to reconcile the conflict. Here students tended to rely on their prior knowledge, as well as their working understanding of the patient’s case to inform their thinking. Students’ frequent use of hypothetico-deductive reasoning may reflect lack of clinical expertise, as less experienced physicians are thought to leverage hypothetico-deductive reasoning approaches more than experts who can quickly and effectively rely on pattern recognition and a stockpile of diverse illness scripts. Students’ use of hypothetico-deductive reasoning may also stem from current medical school instruction and/or the justification task associated with the diagnostic reasoning assessment.

Justifying

The last major component of our framework, justifying, involves students articulating rationales for their proposed hypotheses and most closely aligns with diagnostic justification. Overall, students justified their thinking and responses similarly to how the literature defines diagnostic justification – the act of evaluating clinical evidence collected from a patient encounter to identify the most likely diagnosis and to substantiate the determination of that diagnosis as definitive.

Specific diagnostic justification strategies are not well delineated in the literature, particularly in relation to progressive learner levels. We offer four strategies utilized by 4th-year medical students that reflect aspects of both nonanalytic and analytic thinking – identifying associations, explaining associations, thinking probabilistically, and prioritizing evidence. When justifying, students mostly focused on concerns, signs, and symptoms of disease when they identified associations to support their responses. Here they tended to demonstrate rapid pattern recognition, leveraging their existing knowledge and past training experiences to outline and articulate a rationale. Students leveraged the aspect of an illness script that stresses the underlying biological mechanisms responsible for observed clinical associations, in addition to the other aspects of illness scripts, when explaining associations. In doing so, they tended to employ a slower, more deliberate approach, consistent with the tenets of analytic thinking.

Some students used causal language when discussing clinical associations and/or supporting their answers but often did not fully apply causal reasoning; the instances of application tended to occur when students described the pathophysiological mechanisms they deemed responsible for the patient’s presentation. While a priori understanding of causal chains allows physicians to predict future subsequent outcomes (Rizzi, 1994; Shin, 2019), causal reasoning can be limited in its use for exploring multifactorial causes of diseases and elevated levels of uncertainty around clinical diagnoses.

Although some students considered patient cases through a probability lens to substantiate their claims, students generally did not employ Bayesian reasoning. In medicine, Bayesian reasoning typically involves three main measures: the prevalence of the condition in the population relevant for the individual, the outcomes of a history, physical examination, and clinical tests, and the quality of those outcomes, often in terms of both sensitivity and specificity (Rottman et al., 2016). Bayesian reasoning provides physicians with a means to manage uncertain, difficult cases requiring high-stakes clinical decisions (Ratcliffe & Durning, 2015). It can help inform transitions from test batteries to treatment plans. More generically, Bayesian reasoning involves the mathematical quantification of continuously updating existing thoughts based on the introduction of new material. The technique relies on correlational patterns but does not necessarily imply causality. While Bayesian reasoning is often taught in medical school, understanding its useful application in the clinical environment remains murky (Kinnear et al., 2019). This may have translated to uncertainty around how best to utilize it in the context of the diagnostic reasoning assessment task. These findings may reflect the clinical reasoning knowledge and skills of 4th-year medical students and/or highlight appropriate/inappropriate clinical reasoning content and activities for medical students in the later years of medical school.

Educational implications

Our framework has implications for clinical reasoning instruction and assessment in medical school. It could be used as a tool to confirm that what is being taught is what students are learning. For example, if a medical school’s formal 4th-year curriculum stresses diagnostic justification, particularly the ability to rely on illness scripts to make sense of a patient’s case, our model suggests that this curricular focus may align with the reasoning skills of 4th-year medical students and as such, the curricular content appears to be at the appropriate learner level. Alternatively, if that same medical school emphasizes probabilistic thinking and expects 4th-year medical students to use epidemiology and statistics frequently and easily to understand a patient’s concerns and conditions, our model suggests that this may not be an area in which 4th-year medical students currently gravitate towards when asked to review a patient chart and arrive at a diagnosis. This could then, in turn, spark the advancement and implementation of formative assessments of the application of probability theory to patient care that allow students to practice and develop their reasoning skills over time. In general, the framework affords the opportunity to evaluate the extent to which curricular domains and learning objectives are represented by one model of how 4th-year medical students think.

In addition, our framework could be used as a starting point from which to work backwards to develop a longitudinal curriculum. If the skills represented in the framework resonate as appropriate for 4th-year medical students to be able to demonstrate, efforts could be made to articulate what students in prior years need to know and be able to do in order to reach the development level displayed in the framework by the time they leave medical school. For example, if students at a medical school are intended to be able to manage discordant information while working with a patient by their final year of medical school, it could be useful to ensure that students understand foundational concepts such as conformation bias (seeking and favoring patient information that supports an existing hypotheses and ignoring evidence to the contrary) and pivot points (crucial features of a case which shift the reasoning process) prior to starting their clinical clerkships, and that during the first year of those clerkships they have some exposure to patients with common conditions, but with particular aspects of their medical history that counter those conditions.

In general, though, regarding the practical utility of the framework as is, it is important to emphasize that its content represents an early step in documenting the clinical reasoning skills of 4th-year medical students. The framework will need to be evaluated, tested, and likely enhanced over time. As an exploratory qualitative study, our findings are not necessarily generalizable, but they can be used to begin the process of mapping the clinical reasoning knowledge and skills indicative of learners at distinct stages in the novice-to-expert continuum and developing associated instruction and assessment materials.

Limitations and future research

We note the limitations of this work, several of which are consistent with known drawbacks of the think-aloud data collection method broadly. When participating in the think-aloud interviews, students’ reasoning processes may have been influenced by the expectation that they voice them and the inclination to articulate responses considered desirable. Students also may vary in the degree to which they can verbalize their thoughts, both quantitatively and qualitatively, which could have impacted their interviews. Moreover, our understanding of how students interpret patient information is limited by the fact that SHARP items do not allow students to actively collect information from patients iteratively over time; SHARP items provide a patient chart with a fixed amount of patient information. Relatedly, students’ clinical reasoning processes may differ outside of an assessment setting in a clinical environment, especially given insights gleaned from both information processing theories and social cognitive theories (Parsons et al., 2024) about how physicians learn to think and how that thinking is enacted over time in practice.

Social cognitive theories such as situated cognition (Durning & Artino, 2011; Durning et al., 2011; Ratcliffe & Durning, 2015; Rencic et al., 2020) and distributed cognition (Boyle et al., 2023; Ratcliffe & Durning, 2015), provide a more complete description of the processes that physicians engage in when reasoning in a clinical setting. Our think-aloud study did not allow us to capture information about these important aspects of cognition. Future study designs may be able to simulate some of these conditions by conducting interviews in groups or in a simulated clinical setting.

With these limitations in mind, our study provides an initial framework for considering the clinical reasoning skills of 4th-year medical students. We view our framework as an early foray into better understanding how learners with a moderate amount of clinical experience demonstrate clinical reasoning processes. In its current form, our framework can best be understood through the application of clinical reasoning models and dual processing theory. Future research could empirically test our framework with a generalizable sample of 4th-year medical students, with the aim of refining it based on broader input. Future research also could examine the model in relation to 1st-, 2nd-, and 3rd-year medical students, noting whether instruction at those learner levels includes the four primary skills included in our framework and the extent to which different learner levels draw on them when reasoning clinically. It is possible that similar primary diagnostic reasoning skills are appropriate across all levels of medical education but that the specific ways in which students enact them differ.

To varying extents, the students in our study toggled between nonanalytic and analytic thinking as they reviewed patient charts, answered clinically relevant questions, and substantiated their responses. Because this group of students had a moderate amount of exposure to patient care in clinical settings, they represent neither novices nor experts. In future research, our framework can be compared with existing models to identify curricular areas in medical school in need of greater attention as well as serve as a starting point for future work devoted to the advancement of clinical reasoning theories across the clinical reasoning novice-to-expert continuum.

Author contributions

MMC and CR contributed to the conception and design of the study, as well as the collection, analysis, and interpretation of data. MMC drafted an initial version of the paper and MMC, CR, and SI reviewed and revised it. All authors contributed to the analysis and interpretation of data. All authors approved the final manuscript prior to submission.

Funding

No funding was received to support conducting this study and preparing this manuscript.

Data availability

The data used for this study are not openly available due to participant confidentiality and consent reasons.

Declarations

Ethical approval

This study was reviewed and approved by the American Institutes for Research Institutional Review Board (Project # EX00564).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Disclosures

We have no interests that are directly or indirectly related to our work that we need to declare.

Competing interests

The authors declare no competing interests.