Methods and Outcomes for the Remediation of Clinical Reasoning

ABSTRACT

INTRODUCTION

There is no widely accepted structured, evidence based strategy for the remediation of clinical reasoning skills.

AIM

To assess the effectiveness of a standardized clinical reasoning remediation plan for medical learners at various stages of training.

SETTING

Learners enrolled in the University of Colorado School of Medicine Remediation Program.

PROGRAM DESCRIPTION

From 2006 to 2012, the learner remediation program received 151 referrals. Referrals were made by medical student clerkship directors, residency and fellowship program directors, and through self-referrals. Each learner’s deficiencies were identified using a standardized assessment process; 53 were noted to have clinical reasoning deficits. The authors developed and implemented a ten-step clinical reasoning remediation plan for each of these individuals, whose subsequent performance was independently assessed by unbiased faculty and senior trainees. Participant demographics, faculty time invested, and learner outcomes were tracked.

PROGRAM EVALUATION

Prevalence of clinical reasoning deficits did not differ by level of training of the remediating individual (p = 0.49). Overall, the mean amount of faculty time required for remediation was 29.6 h (SD = 29.3), with a median of 18 h (IQR 5–39) and a range of 2–100 h. Fifty-one of the 53 (96 %) passed the post remediation reassessment. Thirty-eight (72 %) learners either graduated from their original program or continue to practice in good standing. Four (8 %) additional residents who were placed on probation and five (9 %) who transferred to another program have since graduated.

DISCUSSION

The ten-step remediation plan proved to be successful for the majority of learners struggling with clinical reasoning based on reassessment and limited subsequent educational outcomes. Next steps include implementing the program at other institutions to assess generalizability and tracking long-term outcomes on clinical care.

INTRODUCTION

Seven to twenty-eight percent of medical students and residents require remedial teaching to achieve competence in all of the Accreditation Council for Graduate Medical Education (ACGME) competencies.^1–5 Most learners have difficulty with cognitive domains, including the application of medical knowledge in the form of clinical reasoning.^3–9

There is a lack of evidence to guide best practices in remediation and no widely accepted framework for the remediation of clinical reasoning. When focus groups were conducted with clinical educators, the participants stated they did not have a structured or systematic approach for remediation, and expressed skepticism about the impact of remediation on clinical reasoning.¹⁰

While reasoning errors can co-exist with medical knowledge problems, there exists a misconception that clinical reasoning errors are driven by a lack of medical knowledge rather an inability to apply that knowledge in clinical practice.¹¹ If both deficits exist, knowledge must be remediated first, to provide the foundation to develop clinical reasoning skills.^12,13

In the clinical reasoning process, clinicians routinely adopt three methods to assist with diagnosis: hypothetico-deductive reasoning, which involves formulating and testing hypotheses; pattern recognition, which involves comparing cases to prior or textbook cases; and schema group diagnostics, which is algorithmic, dividing possible diagnoses up into clinical and pathophysiologic categories.^14–16

Approaches to remediating clinical reasoning would have to take into account these methods as well as the following relevant information: the mental processing that begins through observation early in the patient encounter; the initial list of possible diagnoses including the most common and the most lethal, and the gathered historical and physical exam information.^17–19 The process includes narrowing the list of diagnoses, considering the supporting and refuting evidence and invoking probabilities, which rely heavily on how knowledge is organized into memory.^20–22 The ability to create and recall illness scripts, or mental models of disease, and compare and contrast these scripts results in better clinical reasoning.^23,24

Semantic qualifiers provide abstract, qualitative descriptions of a case’s signs and symptoms that allow for generation of problem representation, and build in oppositions that allow for comparing and contrasting relevant diagnoses. An example of a sematic qualifier is as follows: for a 55-year-old African American male with an acute, episodic, monoarticular arthritis, the terms African American, male, acute, episodic and monoarticular all serve to categorize the presentation and are semantic qualifiers. Bordage et al. demonstrated a positive association between learners who use more semantic qualifiers in their assessment statement of a case and their reasoning, with the assumption that knowledge structures are strengthened and they are better able to compare and contrast diagnoses.^22,25–28

Remedial teaching involves deliberate practice as a method of improving performance through intentionally practicing tasks beyond one’s level of comfort and competence under the guidance of a devoted coach. The role of the coach is to provide feedback and teach self-reflection.²⁹

Based on these guiding principles, a ten-step remedial teaching plan was created for learners with clinical reasoning deficits. We then performed a prospective observational study on the impact of this clinical reasoning remediation plan on medical students, residents and fellows who were identified as having this deficit.

PROGRAM DESCRIPTION

Context

The remediation program at the University of Colorado School of Medicine received referrals for assessment and remediation from learners through self-referral, medical student clerkship directors, and residency and fellowship program directors. Referrals were made because of negative comments on rotation evaluations, danger of failing, or having failed a course or rotation.

Each referred learner was interviewed by a remediation specialist using a semi-structured interview exploring the learner’s level of performance in each of the ACGME competencies, as well as mental well-being.¹² For the purposes of this program, the competency of patient care is subdivided into clinical skills, clinical reasoning, and time management and organization. Mental well-being includes psychiatric diagnoses, as well as substance abuse, learning disabilities and psychosocial stressors. Remediation specialists were MDs who had received additional training in education, remediation and expertise development.¹³

A summary of the interview and learner’s academic record, plus available reports from neuropsychiatric evaluations and testing, direct observations, script concordance testing, and chart review, were reviewed by a remediation team. The team was comprised of the learner and remediation specialist, with or without referring faculty, a psychiatrist or mental health professional, and the student affairs dean or program director. The team then identified the areas of deficiency by group consensus. If the team was unable to come to a consensus, additional assessments were conducted.

Clinical reasoning was defined as thinking through aspects of patient’s care to arrive at a reasonable decision regarding the prevention, diagnosis, and/or treatment of a clinical problem,³⁰ and deficits were identified through script concordance testing, direct observation, and chart review. The group identified learners with a clinical reasoning deficit if he was unable to perform the following skills based on level of training and specialty: identify pertinent information in the patient’s history; identify pertinent physical exam maneuvers; create a reasonable differential diagnosis; justify the differential diagnosis; and weight appropriately between diagnostic and treatment options.

Study Design

Between July 2006 to June 2012, of 151 learners referred to the remediation program, 53 were identified as having clinical reasoning deficits. The remaining 98 learners entered a different arm of the remediation program.¹² One of the remediation specialists kept confidential records of the learners enrolled, their level of training, their identified deficits, and academic outcomes. All remediation was provided by a remediation specialist through one-on-one instruction and case review, which included both face-to-face time and e-mail correspondence related to implementing the remediation plan. One-on-one time was recorded in hours. Faculty were asked to record time reading and replying to emails from individual learners that directly related to providing feedback and asking reflective questions that corresponded with the various steps in the remediation plan. Faculty members were asked to record the time in 10 min increments on a spreadsheet.

The remediation team implemented a standardized remediation plan based on the current understanding of the clinical reasoning process and deliberate practice,²⁹ using the following steps:

• Step 1:

  Teach the learner how to develop a framework for creating a differential diagnosis using anatomical location (e.g., right upper quadrant pain), pathophysiology (e.g., anemia) and an organ system-based approach (e.g., shortness of breath).

• Step 2:

  The learner practices creating differentials based on four semantic qualifiers: age, gender, race/ethnicity, and chief complaint. Cases were chosen based on the most common presentations for the learner’s specialty or level of training, as determined by the core curriculum. The learner was asked to create three lists: list A is a comprehensive list of diagnoses, list B consists of the most likely four to five diagnoses based on prevalence, and list C is the top two to three diagnoses that a clinician wouldn’t want to miss. For feedback, the learner checks references, such as First Consult’s³¹ differential list and discusses the lists with senior faculty or supervisors in the authentic clinical environment or the remediation specialist, as appropriate to the remediation plan.

• Step 3:

  The learner writes out history questions to ask in order to rule-in or rule-out each of the diagnoses on lists B and C. The learner seeks feedback on his questions as per Step 2. As the learner corrects his history questions, he reflects on key questions missed and extraneous questions. The learner continues to progressively complete Table 1.

• Step 4:

  The learner writes down the physical exam maneuvers in Table 1 that he would perform and the signs he is looking for to rule-in or rule-out each of the diagnoses in lists B and C. The learner obtains feedback as per step 2, and corrects his list of physical exam maneuvers and signs. Reflection on any discrepancies is preferred over memorization.

Table 1.

Clinical Reasoning Grid. Learners Were Asked to Create Similar Grids Based on the Most Common Patient Presentation for Their Clinical Rotation or Specialty, to Help Compare and Contrast Information Between Diagnoses with the Same Chief Complaint

Differential diagnosis	Symptoms and historical information	Signs	Diagnostic work-up	Treatment
Gastroesophageal reflux disease	Subacute, epigastric, burning, supine, relief with antacids	Tenderness to palpation of the epigastrium	History alone, Trial of H2 blocker or PPI, Abnormal EGD	Raise head of bed, change diet, avoid tobacco and alcohol, weight loss, H2 blocker or PPI, f/u in 12 weeks
Muscular strain	Pain in muscle, worse with use of muscle, acute injury or repetitive use	Tenderness to palpation of muscle, +/− mild swelling	History and physical alone	Rest, ice, NSAIDs, f/u in 4–8 weeks for PT referral for strengthening and mechanics
Costochondritis	Female, sharp, worse with deep inspiration	Tenderness to palpation of costochondral junction	+/− erosions on Xray if chronic	Rest, ice, NSAIDs, f/u if doesn’t resolve in 8 weeks
Angina	Male, advanced age, pressure with radiation to arm or jaw, exertional, +/− SOB, nausea, improves with rest, DM, HTN, HLD, tobacco, + FmHx	May have murmur, lateral PMI, gallop, paradox split S2, or normal	Abnormal EKG, Dynamic EKG, Stress test	Modify risk factors such as… weight reduction, DM control, HTN control, smoking cessation NTG, plus tx for CAD: ASA, statin, ACE-I, B-blocker May need to hospitalize
Myocardial infarction	Male, advanced age, pressure with radiation to arm or jaw, exertional, progressive +/− SOB, nausea, diaphoresis, DM, HTN, HLD, tobacco, cocaine, + FmHx	Hypotension, rales, S3/S4 gallop, elevated JVP.	Arrythmias, LBBB, or S-T, T wave changes on EKG, +troponin, + CKMB, ECHO, Cath	Aspirin, nitrates, heparin, beta-blockers, in STEMI, then lytics or stent Hospitalize
Pulmonary Embolism	Female, pleuritic pain, dyspnea, presyncope, palpitations, fever, leg swelling or pain, inactivity, surgery, hypercoag state	Tachycardia, +/− tachypnea, elevated JVP, right-sided S3, lower extremity cord, warmth, tenderness or edema	CT angio or V/Q scan, consider lower extremity Doppler ultrasound, hypercoag work-up	Stop offending agents, i.e., OCPs, consider thrombolysis, start Heparin gtt, or LMWH +/− Warfarin If Warfarin goal INR 2-3 for 24 h, check daily.

PPI proton pump inhibitor, EGD esophagogastroduodenoscopy, f/u follow-up, NSAIDs non-steroidal anti-inflammatory drugs, PT physical therapy, SOB shortness of breath, DM diabetes mellitus, HTN hypertension, HLD hyperlipidemia, FmHx family history, EKG electrocardiogram, PMI post of maximal impulse, NTG nitroglycerin, CAD coronary artery disease, ASA aspirin, ACE-I angiotensin-converting-enzyme inhibitor, JVP jugular venous pressure, LBBB left bundle branch block, CKMB creatine kinase-MB, ECHO echocardiogram, Cath cardiac catheterization, STEMI S-T elevation myocardial infarction, Lytics thrombolytics, CT angio CT angiography, V/Q scan ventilation/perfusion scan, OCPs oral contraceptive pills, LMWH low molecular weight heparin, INR international normalized ratio

*Learners were instructed to create grids, as demonstrated, for each of the most common clinical presentations. For a given presentation, learners entered their differential diagnosis in the first column. As they proceeded though the steps of the remediation plan, the grid was populated with information until complete

**Additional columns may also be added for pathophysiology, especially for medical students who rely more heavily on basic science principles, or complications for procedure-based specialities

Steps 1–4 are repeated with multiple case scenarios until the remediation specialist determines this skill is mastered or all the main presentations for that specialty have been considered, usually 10–20 cycles.

• Step 5:

  The learner is provided with several printed copies of the same complete history and physical, without labs or imaging. The learner creates a differential diagnosis as per Step 1, and then writes each of the diagnoses from lists B and C individually at the top of each copy of the history and physical. The learner highlights the information that supports that diagnosis in green, the information that does not support the diagnosis in red, and crosses out distracting information. This is repeated with multiple cases, and the learner obtains feedback from senior faculty, supervisors, or the remediation specialist and reflects on the feedback.

• Step 6:

  Using the same case as in step 5, the learner creates a table of symptoms and signs, rating their relevance for each diagnosis (Table 2). This chart is used to help the learner understand gradations of supporting and refuting data with arrows either up for supporting or down for refuting, and the number of arrows to indicate degree of influence on the diagnostic decision.³² An equal sign may be used to indicate data that neither increases nor decreases the probability of the disease.

Table 2.

Weighing Data in the Clinical Reasoning Process*

Differential diagnosis	Female	Left-sided	Pleuritic pain	Shortness of breath	Pregnancy	Hypotension	Elevated JVP
Gastroesoph-ageal reflux disease	=	↓	↓↓	↓↓	↑↑	↓↓	↓↓↓
Muscular strain	=	↑	↑	↑	=	↓↓	↓↓↓
Costochond-ritis	↑	↑	↑	↑	=	↓↓	↓↓↓
Angina	↓	↑	↓	↑↑	=	=	=
Myocardial infarction	↓	↑	↓	↑↑	=	↑↑	↑↑
Pulmonary Embolism	↑	↑	↑↑	↑	↑↑↑	↑↑	↑↑

*After reading a case history and physical examination, a learner is asked to populate the columns with the problem list and then rate supporting and refuting data with arrows: up for supporting, down for refuting. The number of arrows indicates the degree of influence on the diagnostic decision. An equal sign indicates that the data neither increases nor decreases the probability of the disease

Steps 5 and 6 are also repeated with multiple case scenarios until the remediation specialist determines that this skill is mastered.

• Step 7:
  The learner summarizes these cases using as many semantic qualifiers in both written and oral format:

  • Pt Characteristics – young, middle aged, elderly, race, gender
  • Onset – slow, sudden, acute, sub-acute, chronic
  • Site – bilateral, unilateral, central, peripheral
  • Course – constant, intermittent, episodic, progressive
  • Severity – mild, moderate, severe
  • Context – rest, activity, lying, seated

  The supervisor gives feedback on his ability to represent the patient’s problem(s) and use of semantic qualifiers. The learner is given several opportunities to re-summarize the case based on feedback.

• Step 8:

  The learner creates a diagnostic work-up for each of the diagnoses, considering the following categories: monitor the patient, order lab(s), order test(s), and/or prescribe a therapeutic trial of medication. The learner writes down the diagnostic work-up necessary for the case, in order of priority, circling the work-up that is required immediately. The learner checks references, including practice guidelines and algorithms for feedback.

• Step 9:

  The learner reflects on the data collected and asks himself, “have I read about a similar case before? How is this case similar and how is it different?” The learner considers how his patient may be the same or different from the patients used in algorithms. This process teaches the learner how to frame specific and productive questions for faculty and peers, and how to discuss the differences between his judgment and the references so that he can adapt his future practice.

• Step 10:

  The learner creates a treatment plan for each of the leading diagnoses, considering the following categories: monitor the patient, prescribe prescription and non-prescription treatment, provide education, place referrals, and review follow-up needs. The learner prioritizes the treatment plan, circling the treatment that is required most urgently. Again, feedback and reflection are incorporated.

While the steps were standardized for all learners, learners progressed through the steps at their own rate. The support of the remediation specialist was removed when all ten steps were completed. Reassessment was performed post-remediation by an independent faculty member or team, as part of a routine clinical assessment based on the level of the learner; assessors were unaware that the learner was being remediated. Reassessment methods included observed structured clinical examinations with standardized patient encounters and script concordance testing,³³ repeating part or all of a rotation, mini-clinical evaluation examinations, chart stimulated recall, and attending and direct team rotation evaluations. The results of the reassessments were shared with the rotation or program director, who determined whether remediation was a success or failure. If the learner’s skills met minimum competence for his/her level of training at this assessment, the learner was permitted to continue his training. Otherwise, remediation continued and the learner was subsequently reassessed. If comorbid deficits had not resolved through remediation of clinical reasoning, the learner entered an arm of the remediation program focused on the remaining deficit(s).

This study was approved by the Colorado Multiple Institutional Review Board.

Data Analysis

Descriptive statistics were performed. Mean, median and range were determined for faculty time to remediate learners with clinical reasoning deficits overall and based on level of learner. Logistic regression was used to determine if faculty time predicted negative academic outcomes, including restricted practice, probationary status, transfer to another program with and without subsequent graduation, withdrawal or dismissal.

Program Evaluation

Of the 151 learners referred to the remediation program, 29 medical students, 19 residents and five fellows were identified as having clinical reasoning deficits. Only 12 of the 53 learners presented with a clinical reasoning deficit alone. Comorbid deficits are shown in Fig. 1. More males (29, 60 %) than females (24, 40 %) were found to have deficits in clinical reasoning, but the difference was not significant (p = 0.43). All learners participated in the program, and the two learners who did not complete the program have been included in the data set in an intention-to-treat analysis.

Figure 1.

Comorbid deficits based on level of learner. Most learners had more than one deficit, based on the semi-structured interview, direct observation and academic records. Twelve of the 53 learners presented with only a clinical reasoning deficit.

Overall, the mean amount of faculty time required for remediation was 29.6 h (SD = 29.3), with a median of 18 h and a range of 2–100 h (IQR 5–39), with an average of 20.4 h was spent face-to-face with the learner and an average of 9.2 h spent via e-mail communications. The time required for medical students, residents and fellows were not statistically different (p = 0.89) (Table 3).

Table 3.

One-on-One Faculty Time in Hours Required for Remediation of Clinical Reasoning*

Level of learner	Mean	Median	SD	Interquartile range
Medical students	24.9	38	24.9	4.5–30
Residents	39.8	12	39.8	8–60
Fellows	18.0	16	12.0	8–29
All learners	29.6	18	29.3	5–39

*Reported time does not include time for planning, assessment, or preparation

Fifty-one of the 53 (96 %) learners with clinical reasoning deficits passed their post remediation plan reassessment (see Fig. 2). Thirty-eight (72 %) learners either graduated from their original program in good standing or continue to practice in good standing. Two (4 %) medical students were placed on probation and remain in medical school. Four (8 %) of the residents were temporarily placed on probation and have since graduated, six (11 %) transferred to either a different program in the same specialty or to a different specialty, one (2 % of the 53) of the six did not graduate. Two (4 %) residents choose to pursue non-patient care careers in the field of medical science, one of who did not complete the remediation plan. The one fellow learner (2 %) who also did not complete the plan, withdrew and chose a career in the specialty of their residency training in a less acute clinical environment that was better suited to their skill set. No learners were dismissed from their educational programs.

Figure 2.

Flowchart of program participant outcomes. Learner outcomes were collected from program directors, the Office of Student Affairs and the Office of Graduate Medical Education. At the completion of the study, no learners were practicing with restricted privileges and none had been terminated.

The type or presence of comorbid deficits did not correlate with outcome, or faculty time spent in remediation. Those learners who were placed on probation or ultimately had a negative outcome (restricted practice, probationary status, transfer to another program with failure to graduate, withdrawal and termination) required significantly more faculty time (P < 0.01; P < 0.01).

DISCUSSION

The ten-step remediation plan for clinical reasoning was time consuming for faculty but successful for the participants, with the majority (96 %) passing the reassessment and 91 % deemed competent for graduation and continued practice. The remediation plan was intentionally created using our understanding of the clinical reasoning process and prior work on the development of expertise.^{14–29,34,35} The steps provide a concrete way of teaching discrete skills needed to problem solve in medicine.

Face-to-face remediation with faculty was preferred, but email communications were substituted when learners were unavailable to meet in person, and were used to review their lists, tables, summary statements and plans, provide feedback and to answer and ask reflective questions based on the remediation plan.

Two learners chose not to complete the program. The authors chose to include all 53 learners referred, including the two that did not complete the program, to accurately represent real life outcomes. While most of the learners had multiple deficits and required on average 20–40 h of individualized faculty teaching, not including significant data gathering and data analysis, all 51 who completed the plan passed their unbiased reassessment.

Prior to the initiation of a formalized and structured remediation program, only 44 of 82 (54 %) residents and fellows with deficiencies in either medical knowledge or patient care graduated. The remaining learners resigned, did not have their contracts renewed, or were dismissed.³⁶

A Canadian study found that remediation of residents across all deficit types took an average of 6 months. While we measured time in faculty hours rather than months, our remediation periods lasted approximately 4–6 weeks rather than months, yet we obtained similar overall outcomes. In the Canadian study, 78 of 95 (82 %) residents completed their training programs or remained in good academic standing; we had 43 (81 %) learners of all levels either graduate, remain in good standing, or be placed on probation but allowed to continue training with graduation anticipated.³⁷

While the success rates are similar, we believe this description more clearly outlines the techniques used to remediate this deficit. We also moved the remediation process along more rapidly with similar outcomes, potentially decreasing the time the learner requires direct oversight, reducing the number of schedule disruptions and exposing fewer patients to the learner while he is underperforming.

This paper describes the feasibility of creating and implementing a clinical reasoning remediation program. Although with early outcomes, this is not an efficacy study. Additional limitations include that this is a single institution study. There is no standard definition of what constitutes a deficit in clinical reasoning. We tried to overcome this by using a group decision by individuals with expertise in remediation.³⁸ The authors do not have quantitative data on which specific aspects of the program were most successful in teaching clinical reasoning; however, the remediation faculty noticed the most advancement with use of Table 1 and Step 7. It is unclear if it was the curriculum or faculty attention that improved skills. While reassessment was multi-faceted, including observed structured clinical examinations with standardized patient encounters and script concordance testing, repeating part or all of a rotation, mini-clinical evaluation examinations, chart stimulated recall, and attending and direct team rotation evaluations, it may or may not have rigorously assessed clinical reasoning in a standardized way.

We created and described a program that is well grounded in clinical reasoning and learning theory, is time consuming but feasible, and allows learners to achieve competence in clinical reasoning. Next steps are to study this method at different institutions for continuous improvement of better outcomes and efficiency, and to study patient care outcomes.