ABSTRACT
BACKGROUND
Research on the outcomes of pre-clinical curricula for clinical skills development is needed to assess their influence on medical student performance in clerkships.
OBJECTIVE
To better understand the impact of a clinical-skills curriculum in the pre-clinical setting on student performance.
DESIGN
We conducted a non-randomized, retrospective, pre-post review of student performance evaluations from 3rd-year clerkships, before and after implementation of a clinical-skills curriculum, the Colleges (2001–2007).
MAIN RESULTS
Comparisons of clerkship performance data revealed statistically significant differences favoring the post-Colleges group in the Internal Medicine clerkship for 9 of 12 clinical-skills domains, including Technical Communication Skills (p < 0.023, effect size 0.16), Procedural Skills (p < 0.031, effect size 0.17), Communication Skills (p < 0.003, effect size 0.21), Patient Relationships (p < 0.003, effect size 0.21), Professional Relationships (p < 0.021, effect size 0.17), Educational Attitudes (p < 0.001, effect size 0.24), Initiative and Interest (p < 0.032, effect size 0.15), Attendance and Participation (p < 0.007, effect size 0.19), and Dependability (p < 0.008, effect size 0.19). Statistically significant differences were identified favoring the post-Colleges group in technical communication skills for three of six basic clerkships (Internal Medicine, Surgery, and Pediatrics).
CONCLUSIONS
Implementation of a pre-clinical fundamental skills curriculum appears to be associated with improved clerkship performance in the 3rd year of medical school, particularly in the Internal Medicine clerkship. Similar curricula, focused on teaching clinical skills in small groups at the bedside with personalized mentoring from faculty members, may improve student performance. Continued efforts are needed to understand how to best prepare students for clinical clerkships and how to evaluate outcomes of similar pre-clinical skills programs.
KEY WORDS: medical education, clinical skills, medical students, pre-clinical, curriculum
INTRODUCTION
Teaching fundamental clinical skills should be a priority in medical school curricula, in keeping with the importance of mastering clinical skills for becoming a physician.1–8 However, there is a paucity of curricula devoted to formal clinical skills instruction in US medical schools.9–19 As academic faculty increasingly focus on their own clinical and research efforts, declining resources support bedside teaching and clinical skills development for medical students.20–25
In response, the University of Washington School of Medicine (UWSOM) in 2001 implemented a competency-based curriculum (the Colleges program) designed to teach fundamental clinical skills at the bedside to 2nd-year medical students; this program has been described previously.26 Published outcomes of similar teaching models are scarce. A previous study of this UWSOM curricular model suggests that early introduction of clinical skills increases student comfort on the wards at the start of the 3rd year;27 several other studies of similar models have shown improved interpersonal and communication skills.28–29 Despite the contemporary focus in medical education research on outcomes, few meaningful outcome measures assess the influence of specific teaching innovations on students’ clinical performance.
As a part of ongoing efforts to investigate the effectiveness and impact of the Colleges program, this study explored whether the Colleges curriculum improved 3rd-year clerkship performance as assessed by performance evaluations. We conducted a non-randomized, retrospective, pre-post review of 3rd-year clerkship student performance evaluations from 2001 to 2007. We hypothesized that the Colleges curriculum would result in improved student performance in basic clinical skills on which the Colleges focus most strongly (communication, history-taking, physical exam, oral case presentations, and preparing write-ups), compared with ratings prior to inception of the Colleges.
METHODS
Educational Intervention
The UW Colleges curriculum, implemented in 2001, aims to teach fundamental clinical skills and to provide longitudinal mentoring for medical students. Thirty-six College faculty mentors meet their small groups of six students weekly throughout the students’ second year. The first several sessions are conducted in a conventional curricular setting using observation, practice with peers, and standardized patients to teach history and physical exam skills. The mentors then take students to the hospital wards for weekly bedside teaching sessions interspersed with monthly organ-system-specific advanced-exam teaching sessions.
At each bedside teaching session, two students interview and examine hospitalized patients under their mentor’s supervision and then present their patients at the bedside. The other four students observe the bedside presentation and provide feedback. Through the year, each student independently interviews, examines, and performs bedside presentations on six hospitalized patients and observes their peers’ presentations on 30 additional patients. Faculty mentors provide additional teaching and skills modeling for the group at the bedside. The emphasis on focused teaching of clinical skills at the bedside with direct observation and feedback by faculty physicians enables role modeling, careful monitoring of skills development, and personalized feedback. Specific skills targeted by the curriculum include patient communication, history-taking, physical exam, oral case presentations, write-ups, professional values, and introduction to clinical reasoning.
Data Approach
The UWSOM Department of Medical Education and Biomedical Informatics, which maintains longitudinal student performance evaluation data from basic 3rd-year clerkships, prepared de-identified end-of-clerkship evaluations for the purposes of aggregate analyses. This research was approved by Human Subjects at the University of Washington prior to data analysis.
Subjects
The Colleges program was implemented for medical students matriculating in fall 2001 and was fully phased in for students in fall 2002. We identified 3 years of performance evaluations prior to the Colleges (E-98, E-99, E-00) and 3 years after the Colleges implementation (E-02, E-03, E-04). We excluded the E-01 class, as the Colleges curriculum underwent substantial formative development and revision during that period; we also excluded students who did not proceed directly to 3rd-year clerkships within 3 years of matriculation (i.e., MD/PhD students, students in an expanded curriculum).
Instrument
For each basic 3rd-year clerkship at the University of Washington School of Medicine (Family Medicine, Internal Medicine, Obstetrics-Gynecology, Pediatrics, Psychiatry, and Surgery), students receive end-of-clerkship evaluations with the purpose of providing feedback on performance to students and to the Dean’s Office. Similar global ratings are frequently used in medical education to assess performance.30–32 Faculty ratings of students’ clinical competence have been predictive of performance on medical licensing examinations and clinical competence ratings in residency.33
Evaluations at the UWSOM include a final numeric rating on a Likert scale (1–7) for each of 12 clinical skills domains (Table 1). Clinical clerkships at the UWSOM are conducted at clinical sites throughout five states—Washington, Wyoming, Alaska, Montana, and Idaho. Evaluations by residents and faculty at individual sites are merged by site coordinators and clerkship directors to create final numeric scores. Common descriptive anchors on each clerkship evaluation provide stability in assessment. We analyzed six separate performance evaluation data sets from each of the basic clerkships; variables include matriculation year, clerkship dates, and numeric values for each clinical-skill domain.
Table 1.
Clinical-skills Domains in Student Performance Evaluations in 3rd-year Clerkships at the University of Washington School of Medicine
| Knowledge in subject area | Knowledge of medical principles and pathophysiology related to patient’s problems |
|---|---|
| Problem-solving ability | Ability to identify patient’s major/ minor problems. Utilization of time, laboratory, other services in solving problems |
| Data gathering skills | Basic skills of history-taking and physical examination appropriate to clerkship level |
| Technical communication skills | Completeness, logic, and accuracy of communications regarding patient’s problems |
| Procedural skills | Performance of tests or procedures. Regard for patient’s comfort and dignity during procedure |
| Communication skills | Communications with patients, families, and colleagues |
| Relationships with patients | Courtesy, empathy, and respect afforded all patients |
| Professional relationships | Ability to work collaboratively with other members of the team; courteous and cooperative attitude |
| Educational attitudes | Participation, eagerness to learn and responsiveness to evaluation stimulation of the learning process |
| Initiative and interest | Motivation and interest in the subject and skills of the clerkship |
| Attendance/ preparation | Promptness and preparation for scheduled activities |
| Dependability | Willingness to undertake and complete responsibilities for patient care |
On students’ final evaluation, a final grade was also reported. To understand how a final grade is calculated for each clerkship, we interviewed the six clerkship directors. Each director reported a distinct system, communicated to students in writing at the beginning of the clerkship, for determining the final grade. All clerkships reported at least one structural change in the prior 7 years that may have impacted how grades were determined or that could have influenced the average final grade. The system for assigning final clerkship grades does not provide an anchor system to give stability from year to year. Because of the variability between clerkships and the lack of an anchor system, we elected not to include final grades in analyses.
Data Analysis
STATA 10, a statistical software package, was used for statistical analyses of student performance evaluations.34 Analyses consisted of t-tests comparing the pre and post groups and assumed unequal variance. Standard alpha (0.05) was used to identify statistical significance. Effect sizes were calculated using Cohen’s d for comparisons in which a statistically significant difference was found pre- and post-Colleges. Pre-Colleges and post-Colleges cohorts were compared within each clerkship for statistically significant differences, but not across clerkships (i.e., clerkships were not compared to one another).
RESULTS
A total of 5,240 performance evaluations were analyzed (2,634 pre-Colleges and 2,606 post-Colleges implementation). Total numbers of evaluations were similar for each clerkship: Internal Medicine (871), Family Medicine (908), Obstetrics—Gynecology (819), Pediatrics (875), Psychiatry (934), and Surgery (833). Each clerkship had similar total numbers of evaluations in the pre- and post-College groups.
Comparisons of Internal Medicine clerkship performance data revealed statistically significant increases in the post-Colleges group in 9 of 12 clinical-skills domains (Table 2). Calculation of Cohen’s d effect sizes for comparisons found to be statistically significant in the Internal Medicine clerkship showed moderate effects; the largest effect sizes were for educational attitudes (0.24), patient relationships (0.21), and communication skills (0.21).
Table 2.
Clerkship Performance Evaluations Before and After Inception of the Colleges Program: Internal Medicine
| Pre-Colleges | Post-Colleges | 95% CI | Effect size | ||
|---|---|---|---|---|---|
| Knowledge of subject area | 5.88 | 5.86 | (−0.73, 0.1) | p < 0.755 | |
| Problem solving | 5.97 | 5.90 | (−0.28, 0.16) | p < 0.166 | |
| Data gathering | 6.19 | 6.26 | (−0.02, 0.15) | p < 0.112 | |
| Technical comm. skills | 5.97 | 6.08 | (0.02, 0.2) | p < 0.023* | 0.16 |
| Procedural skills | 6.17 | 6.28 | (0.01, 0.2) | p < 0.031* | 0.17 |
| Communication skills | 6.29 | 6.41 | (0.04, 0.21) | p < 0.003* | 0.21 |
| Patient relationships | 6.59 | 6.7 | (0.04, 0.18) | p < 0.003* | 0.21 |
| Professional relationships | 6.57 | 6.66 | (0.01, 0.17) | p < 0.021* | 0.17 |
| Educational attitudes | 6.43 | 6.58 | (0.06, 0.23) | p < 0.001* | 0.24 |
| Initiative and interest | 6.44 | 6.54 | (0.01, 0.19) | p < 0.032* | 0.15 |
| Attendance and participation | 6.53 | 6.64 | (0.03, 0.19) | p < 0.007* | 0.19 |
| Dependability | 6.55 | 6.66 | (0.03, 0.19) | p < 0.008* | 0.19 |
Analyses in other clerkships revealed few individual statistically significant differences pre- and post-Colleges, and no clear trends (Table 3). Across clerkships, there was a trend towards improvement post-Colleges in technical communication skills, with statistically significant improvements in Internal Medicine, Pediatrics, and Surgery.
Table 3.
Statistically Significant Changes in Clerkship Performance Evaluations Before and After Inception of the Colleges Program: All Clerkships 3rd Year Basic Clerkships
| Performance evaluation domains | Internal Medicine | Family Medicine | Obstetrics − Gynecology | Pediatrics | Psychiatry | Surgery |
|---|---|---|---|---|---|---|
| Knowledge of subject area | − | − | − | − | − | − |
| Problem solving | − | − | − | − | − | − |
| Data gathering | − | − | − | − | − | ↑ |
| Technical comm. skills | ↑ | − | − | ↑ | − | ↑ |
| Procedural skills | ↑ | − | − | − | − | − |
| Communication skills | ↑ | − | − | − | − | ↑ |
| Patient relationships | ↑ | ↑ | − | − | − | − |
| Professional relationships | ↑ | − | − | − | − | − |
| Educational attitudes | ↑ | ↓ | − | − | ↓ | − |
| Initiative and interest | ↑ | ↓ | − | − | − | − |
| Attendance and participation | ↑ | − | − | − | − | − |
| Dependability | ↑ | − | − | − | − | − |
↑Represents a statistically significant difference favoring the post-College evaluations (as compared with pre-College); ↓ represents a statistically significant difference favoring the pre-College evaluations
DISCUSSION
Our study examined the influence of a structured, bedside clinical-skills curriculum for 2nd-year medical students, the Colleges program, on students’ performances in basic 3rd-year clerkships. Analyses identified improvement in student performance ratings on a substantial set of clinical skills domains within the Internal Medicine clerkship. The improvements were primarily in relational, communications, or attitudinal domains. These findings confirm the value of a preclinical skills curriculum focused on bedside teaching under the guidance of faculty mentors for advancing students’ internal medicine skills beyond the 2nd year.
The positive findings specifically for the Internal Medicine clerkship may be due to several factors. The specialty of internal medicine has historically been described as “the mother of all clinical divisions,” the foundation on which the instruction of the art of medicine is built.35,36 This may account for why internal medicine faculty are heavily involved in teaching the Introduction to Clinical Medicine courses nationwide.9 Faculty in the Colleges represent a diverse range of specialties; approximately one-third are internists. Further, the skills taught in the Colleges program are the foundational internal medicine skills of history-taking, physical examination, communication, oral case presentation, and write-ups. The patients with whom students work in the Colleges program are predominantly adult inpatients, often on general medicine wards. This increases the preclinical skills curriculum’s internal medicine orientation and enhances students’ future comfort and confidence with adult inpatients. Students’ skills in other areas are developed later and differentiated into other clerkships and skill sets, but the foundational approach to the history and physical lies within the specialty of internal medicine. In addition, students spend more time in the internal medicine clerkship than in other basic 3rd-year clerkships—12 weeks compared with 4–8 weeks. Thus, performance in internal medicine should provide a more in-depth portrait of a student’s performance, enhancing the likelihood that positive (or negative) results may be demonstrated.
It is also likely that the statistically significant improvements seen in the domain of technical communication in Internal Medicine, Surgery, and Pediatrics are related to the Colleges curriculum. Technical communication refers to skills in oral case presentation and write-ups, described on performance evaluation forms as “completeness, logic, and accuracy of communications (with the medical team) regarding patient’s problems.” This curricular element has been particularly emphasized by the Colleges curriculum and was minimally covered for pre-clinical students prior to the Colleges.
While we expected to demonstrate improvement in performance evaluations in each clerkship, we did not find consistent improvements. Isolated declines in performance for individual areas within a few clerkships were identified, but it is difficult to assign meaning. Students may not have been as confident working with a more diverse patient panel in these clerkships (including pediatric, psychiatric, surgical, or obstetrics patients). Skills specific to each of these clerkships are more specialized than the general skills taught by the Colleges.
LIMITATIONS
Interpretation of retrospective pre-post data analyses is subject to flaws. Though the initiation of the Colleges represents a significant change in the curriculum, we cannot determine a causal relationship; however, this was the only substantive change to the curriculum affecting preclinical students at UWSOM in the prior several years. Structural changes in individual clerkships may have contributed to changes in average grades and could have contributed to trends in clinical skills domains over time, despite constant anchors on evaluation forms. Data obtained for clerkship performance evaluations do not represent evaluations for every class member; the Department of Medical Education and Biomedical Informatics does not routinely receive all evaluations for its school-wide analyses, but for most years and clerkships, the data represent well over 90% of the class.
Clerkship performance evaluations do not necessarily measure the entire range of performance. However, a prior study evaluating the predictive validity of student evaluations in basic clerkships on performance for USMLE Step 2 and 3 scores and supervisors’ ratings in residency found the strongest association in evaluations from the internal medicine clerkship.33 This affirms the use of clerkship performance evaluations across a range of skills and to assess change over time. Finally, our analyses are limited by an inability to test for other potential predictor variables, such as clerkship site (location of clerkship) or setting (inpatient vs. outpatient), due to restrictions on data use by Human Subjects considerations.
CONCLUSIONS
Implementation of the Colleges pre-clinical curriculum appears to be associated with increased confidence27 and improved student performance in 3rd-year clerkships, especially in the Internal Medicine clerkship. Continued efforts to elucidate how to best prepare students for clerkships and how to evaluate outcomes of clinical skills training programs should be encouraged. The Colleges clinical skills curriculum, featuring guided bedside teaching on the wards with a dedicated faculty mentor during the 2nd year of medical school, appears to be an effective method for improving some aspects of student performance in basic clerkships.
ACKNOWLEDGEMENTS
Funding support for Dr. Jackson for her work in medical education research and program evaluation comes from the Office of the CEO and Dean of the School of Medicine at the University of Washington. Funding support for Dr. Keen for his work as a UW College faculty member is provided by UW School of Medicine.
Conflicts of Interest None disclosed.
REFERENCES
- 1.Kern DC, Parrino TA, Korst DR. The lasting value of clinical skills. JAMA. 1985;254:70–6. [DOI] [PubMed]
- 2.Cooke M, Irby DM, Sullivan W, Ludmerer KM. American Medical Education 100 years after the Flexner Report. N Engl J Med. 2006;355:1339–44. [DOI] [PubMed]
- 3.Emanuel EJ. Changing premed requirements and the medical curriculum. JAMA. 2006;296(9):1128–31. [DOI] [PubMed]
- 4.Reilly BM. Physical examination in the care of medical inpatients: an observational study. Lancet. 2003;362:1100–5. [DOI] [PubMed]
- 5.Novack DH, Volk G, Drossman DA, Lipkin M. Medical interviewing and interpersonal skills teaching in the U.S. medical schools. Progress, problems, and promise. JAMA. 1993;269:2101–5. [DOI] [PubMed]
- 6.Holmboe ES. Faculty and the observation of trainees’ clinical skills: Problems and opportunities. Acad Med. 2004;79:16–22. [DOI] [PubMed]
- 7.Nutter D, Whitcomb M. The AAMC Project on the Clinical Education of Medical Students. Washington, DC: Association of American Medical Colleges; 2001.
- 8.Wilkerson L, Lee M. Assessing physical examination skills of senior medical students: knowing how versus knowing when. Acad Med. 2003;78:S30–2. [DOI] [PubMed]
- 9.Omori D, Wong R, Antonelli M, Hemmer P. Introduction to clinical medicine: A time for consensus and integration. Am J Med. 2005;118(2):189–94. [DOI] [PubMed]
- 10.Mechaber AJ, et al. Clinical skills training: Time for a national dialogue and consensus. Am J Med. 2005;118(8):931–2. [DOI] [PubMed]
- 11.Pfeiffer C, Madray H, Ardolino A, Williams J. The rise and fall of students’ skill in obtaining a medical history. Med Educ. 1998;32:283–8. [DOI] [PubMed]
- 12.Ramsey PG, Curtis JR, Paauw DS, Carline JD, Wenrich MD. History-taking and preventive medicine skills among primary care physicians: an assessment using standardized patients. Am J Med. 1998 Feb;104(2):152–8. [DOI] [PubMed]
- 13.Wilson BE. Performance-based assessment of internal medicine interns: evaluation of baseline clinical and communication skills. Acad Med. 2002;77:1158. [DOI] [PubMed]
- 14.Ortiz-Neu C, Walters CA, Tenenbaum J, Colliver JA, Schmidt HJ. Error patterns of 3rd-year medical students on the cardiovascular physical examination. Teach Learn Med. 2001;13:161–6. [DOI] [PubMed]
- 15.Engum SA. Do you know your students’ basic clinical skills exposure? Am J Surg. 2003;186:175–81. [DOI] [PubMed]
- 16.Freedman KB, Bernstein J. The adequacy of medical school education in musculoskeletal medicine. J Bone Joint Surg Am. 1998;80:1421–7. [DOI] [PubMed]
- 17.Mangione S. Cardiac auscultatory skills of physicians-in-training: a comparison of three English-speaking countries. Am J Med. 2001;110:210–6. [DOI] [PubMed]
- 18.Mangione S, Nieman LZ. Cardiac auscultatory skills of internal medicine and family practice trainees. A comparison of diagnostic proficiency. JAMA. 1997;278:717–22. [DOI] [PubMed]
- 19.Fred HL. Hyposkillia. Deficiency of clinical skills. Tex Heart Inst. 2005;J32:255–6. [PMC free article] [PubMed]
- 20.Ludmerer K. Time to Heal: American Medical Education from the Turn of the Century to the Era of Managed Care. New York: Oxford University Press; 1999.
- 21.Whitcomb ME, Anderson MB. Transformation of medical students’ education: Work in progress and continuing challenges. Acad Med. 1999;74(10):1076–9. [DOI] [PubMed]
- 22.Ozuah PO. Undergraduate medical education: thoughts on future challengesBMC Med Educ. 2002; 2: 8. Jul 30. [DOI] [PMC free article] [PubMed]
- 23.Stevens DP, Leach DC, Warden GL, Cherniack NS. A strategy for coping with change: an affiliation between a medical school and a managed care health system. Acad Med. 1996;71(2):133–7. [DOI] [PubMed]
- 24.Reilly BM. Inconvenient truths about effective clinical teaching. Lancet. 2007;370(9588):705–11. [DOI] [PubMed]
- 25.Feddock CA. The Lost Art of Clinical Skills. APM Perspectives. Am J Med. 2007;120(4):374–8. [DOI] [PubMed]
- 26.Goldstein EA, MacLaren CF, Smith S, et al. Promoting fundamental clinical skills: a competency-based college approach at the University of Washington. Acad Med. 2005;80:423–33. [DOI] [PubMed]
- 27.Whipple ME, Barlow CB, Smith S, Goldstein EA. Early introduction of clinical skills improves medical student comfort at the start of third year clerkships. Acad Med. 2006;81:S40–3. [DOI] [PubMed]
- 28.Kossoff EH, Hubbard TW, Gowen CW. Early clinical experience enhances third-year pediatrics clerkship performance. Acad Med. 1999;74(11):1238–41. [DOI] [PubMed]
- 29.Hook KM, Pfeiffer CA. Impact of a new curriculum on medical students’ interpersonal and interviewing skills. Med Educ. 2007;41(2):154–9. [DOI] [PubMed]
- 30.Pulito AR, Donnelly MB, Plymale M. Factors in faculty evaluation of medical students’ performance. Med Educ. 2007;41:667–75. [DOI] [PubMed]
- 31.Carline JD, Paauw DS, Thiede KW, Ramsey PG. Factors affecting the reliability of ratings of students’ clinical skills in a medicine clerkship. J Gen Int Med. 1992;7:506–10. [DOI] [PubMed]
- 32.Whitfield CF, Mauger EA, Zwicker J, Lehman EB. Differences Between Students in Problem-Based and Lecture-Based Curricula Measured by Clerkship Performance Ratings at the Beginning of the Third Year. Teach Learn Med. 2002;14(4):211–7. [DOI] [PubMed]
- 33.Callahan CA, et al. Validity of Faculty Ratings of Students’ Clinical Competence in Core Clerkships in Relation to Scores on Licensing Examinations and Supervisors’ Ratings in Residency. Acad Med. 2000;75:S71–3. [DOI] [PubMed]
- 34.StataCorp. Stata Statistical Software: Release 10. College Station. TX: StataCorp LP; 2007.
- 35.Lexis W. Das Unterrichtswesen im deutschen Reich. Vol. I. Berlin, 1904.
- 36.Flexner A. Medical education in the United States and Canada. Carnegie Foundation for the Advancement of Teaching, 1910. [PMC free article] [PubMed]