A Randomized Controlled Trial of the Impact of a Teaching Procedure Service on the Training of Internal Medicine Residents

Abstract

Introduction

Academic medical centers must provide safe inpatient procedures while balancing resident autonomy and education. We performed a randomized, controlled trial to evaluate the effect of a 2-week hospitalist procedure service (HPS) rotation on interns' self-perceived procedure ability, knowledge, and autonomy versus the standard curriculum.

Methods

We randomly selected 16 of 57 internal medicine interns (28%) to participate in the intervention group rotation, with 29 interns in the control group. All interns were surveyed before the start of residency and at the end of the postgraduate year-1 (PGY-1) and PGY-2 years to evaluate self-reported knowledge and ability to (1) safely perform procedures, (2) supervise procedures, and (3) use bedside ultrasound.

Results

Ninety-four percent of HPS interns (15/16) and 71% of control interns (29/41) completed all surveys. Baseline knowledge and experience did not differ significantly between the groups. The intervention group performed significantly more paracentesis (9 versus 4; P < .001), thoracentesis (6 versus 2; P < .001), and lumbar puncture (4 versus 3; P < .001) procedures than did the control group. After their first year, residents who completed the HPS rotation rated their ability to safely perform and supervise all of the assessed procedures as higher (P < .05 for all procedures) and were more likely to rate self-perceived knowledge as very good or excellent in all surveyed aspects of procedure performance (P < .05).

Discussion

A 2-week hospitalist-supervised procedure service rotation substantially improved residents' experience, confidence, and knowledge in performing bedside procedures early in their training, with this effect sustained through the PGY-2 year. Standardized procedure service rotations are a viable solution for programs seeking to improve their procedure-based education.

What was known

Residents early in their training are expected to perform and supervise bedside procedures and have indicated a lack of confidence and desire for additional formal training.

What is new

A randomized trial of a 2-week procedural training rotation for interns with intensive didactics and added experience found higher self-reported confidence in the intervention group in performing and supervising procedures and use of ultrasound.

Limitations

Single-site, single-specialty study, with self-reported, mostly subjective data (some data on complications were collected).

Bottom line

A hospitalist-supervised procedure service rotation substantially improved interns' experience, confidence, and knowledge in performing and supervising bedside procedures, with this benefit sustained well into the PGY-2 year.

Introduction

A 1989 study¹ reported that internal medicine residents failed to master routine procedures and made recommendations for increased training, supervision, and competency assessment. Since that time, residents have continued to report discomfort with procedural skills, as well as a desire for increased training.^2–5 Certification requirements currently exclude performance and volume-based metrics, requiring only that residents, “recognize indications, contraindications and manage complications” of selected bedside procedures.⁶ The literature suggests that residents routinely perform procedures,^5,7,8 and frequently do this with uncomfortable or underqualified peer supervisors.^5,9

With increased attention on patient safety and resident work hours, faculty-staffed procedure services have been developed to handle common bedside procedures, such as paracentesis, thoracentesis, lumbar puncture, arthrocentesis, and central line insertion.^7,10 Such a service can simultaneously provide residents with procedural training and offer a possible solution to balance quality procedural performance with resident education. Most programs to date have been for postgraduate year-2 (PGY-2) or PGY-3 residents,^2,3,5 despite procedural expectations for residents earlier in training.^5,9

To better address procedural training early in residency, we created a comprehensive hospitalist procedure service (HPS) with a 2-week rotation for internal medicine interns. We performed a randomized, controlled trial to evaluate the effect of participation in the HPS rotation on interns' longitudinal, self-perceived procedure ability and knowledge as compared with the standard curriculum.

Methods

Site and Participants

Participants included interns at a university-based, academic medical center in the 2008–2009 academic year. The intervention group comprised 16 of 57 randomly selected internal medicine interns (28%) who participated in a 2-week procedure service rotation at the 560-bed university hospital setting (1 of 3 hospital sites in the program). Interns in both the intervention group and control group who did not complete all of the surveys were excluded from the study (n  =  13; 23%). The study was reviewed and approved by our Institutional Review Board, The Committee on Human Research.

Procedure Service

The HPS began offering procedure services for paracentesis, thoracentesis, and lumbar puncture in November 2008. Patients were voluntarily referred to this service. Interns rotating on the HPS performed referred procedures during a 2-week rotation; all procedures were supervised by an attending hospitalist with additional ultrasound and procedural training. Procedures not referred to the HPS were independently performed by residents on the primary team or referred to interventional radiology. Additionally, residents and interns continued to perform procedures independently at the 2 other hospital sites. A detailed description of the structure of the service and faculty training has been described previously.¹¹

Resident Education

We employed a best-practices technique to teach interns, which included repetitive practice to develop proper technique, real-time ultrasonographic guidance, direct supervision by expert faculty, and a team-based approach. All interns received the standard procedure curricular instruction: a half-day simulation course dedicated to providing hands-on instruction with ultrasound technique, central venous catheter insertion, thoracentesis, and lumbar puncture performance. In addition, the intervention group participated in 10 didactic sessions that included procedure safety, informed consent, infection control and documentation practices, ultrasound basics, and in-depth, procedure-based lectures reviewing indications and contraindications, procedure preparation, anatomy, technique, troubleshooting, management of complications, and follow-up studies and their interpretation. These sessions were supplemented with independent readings and 3 evidence-based, interactive computer modules reviewing best practices for each procedure. Each intern was required to demonstrate procedure technique using step-by-step hands-on simulation with the procedure kit before performance in a clinical setting.

Survey Development and Distribution

The 3 survey instruments for this study were developed by the HPS faculty in conjunction with divisional research leadership and the program director and were reviewed by a committee of local survey-design experts. The survey tools were pilot-tested by a group of attending physicians, residents, and the investigative team but were not formally validated.

The initial PGY-1 survey captured incoming interns' experience, knowledge, and confidence in performing common, invasive, bedside procedures listed in the American Board of Internal Medicine's Policies and Procedures for Certification.⁶ The survey also asked about baseline experience and confidence in observing, performing, and supervising procedures as well as interns' confidence in using ultrasound for procedure guidance. Interns specifically rated their confidence to (1) safely perform specific bedside procedures, and (2) safely supervise others in conducting procedures. The surveys at the end of their PGY-1 and PGY-2 year captured changes in experience and knowledge over time and self-perceived procedure knowledge on a range of topics, including indications and contraindications, landmarks, use of procedure-specific ultrasound, evidence-based procedure technique, troubleshooting, and management of complications. Questions regarding central-line performance served as a check item because that procedure was not taught or performed by the HPS.

Surveys were distributed via e-mail in July 2008 and 2009 and in June 2010. Up to 3 reminder e-mails were sent to nonrespondents.

Analysis

We analyzed baseline and yearly differences between the intervention and nonintervention group using χ² tests for categoric and nominal variables. Differences in Likert scale ratings for “confidence” to perform and supervise procedures were compared with Student t tests. Residents were considered to have self-perceived knowledge of a subject area if they rated their knowledge as 4 (very good) or 5 (excellent). Knowledge was compared between intervention and control groups using Fisher exact tests. Multivariate logistic regression was used to analyze the contribution of age, sex, future subspecialty, and procedure experience.

Reported complications and those found in patients' medical charts were classified into major and minor based on reported definitions in the literature.^12–15 Major complications included bleeding requiring transfusion, pneumothorax requiring a chest tube, respiratory failure, bowel perforation, cerebral herniation or shock, cerebrospinal fluid leak requiring intervention, and transfer to a higher level of care. For patients receiving thoracentesis, the patient's medical record was reviewed to check for a follow-up chest x-ray, the presence of a pneumothorax, or clinical evidence for re-expansion pulmonary edema.

Results

Demographics of Respondents

Ninety-four percent (15/16) of HPS interns and 71% of controls (29/41) completed all 3 surveys. Baseline self-reported experience and confidence was low and not significantly different between the intervention and the control groups (table 1).

TABLE 1.

Baseline Demographics of Study Participants

Procedure Volume

In its first 8 months, the 16 interns on the procedure service performed 265 supervised procedures (122 paracenteses, 77 thoracenteses, and 66 lumbar punctures). Interns performed an average of 16 procedures per intern during the 2-week period. During the 8 months, the intervention group performed significantly more paracenteses (median, 9 versus 4; P < .001), thoracenteses (median, 6 versus 2; P < .001), and lumbar punctures (median, 4 versus 3; P < .001) than did the control group. The number of central lines performed did not differ between the 2 groups (median, 2 versus 3; P  =  .83), and the volume of procedures performed by the control group was similar to previously published data.⁹

Procedure Performance

Interns who completed the rotation rated their ability to safely perform each of the assessed procedures higher than did the controls (P < .036 for paracentesis, thoracentesis, lumbar puncture; table 2). For central lines (not taught or performed by the HPS), there was no difference between the intervention and control groups. The HPS interns were also more confident in their use of ultrasound (P < .001) and were more likely to use bedside ultrasound for paracentesis and thoracentesis, and many of those gains persisted through the end of the PGY-2 year (P < .041; table 3). The largest and most sustained gains in confidence between the intervention and control groups were seen in thoracentesis confidence. In a multivariate analysis of procedure confidence adjusting for age, sex, future invasive subspecialty, and procedure volume at the end of the PGY-1, the number of procedures performed was a significant predictor for confidence performing thoracentesis, lumbar puncture, and central line insertion.

TABLE 2.

Resident Confidence in Performing Bedside, Invasive Procedures and Use of Ultrasound

TABLE 3.

Resident Report of Likelihood of Bedside Ultrasound Use

Procedure Supervision

Interns rotating on the HPS reported significantly higher levels of confidence in safely supervising bedside invasive procedures offered by the service (P < .023; table 4), and these gains persisted until the end of the PGY-2 year. In a multivariate analysis of procedure confidence adjusting for age, sex, future invasive subspecialty, and procedure volume at the end of the PGY-1 year, the number of procedures performed was a significant predictor for confidence supervising thoracentesis, lumbar puncture, and central line insertion.

TABLE 4.

Resident Confidence in Supervising Bedside, Invasive Procedures

Self-Rated Knowledge

The HPS interns were more likely to rate self-perceived knowledge as very good or excellent in all surveyed aspects of procedure performance, including indications and contraindications, landmarks, procedure-specific ultrasound use, evidence-based procedure technique, troubleshooting, and management of complications (P < .022 for all). Significant relative gains in all knowledge-based areas persisted until at least the end of PGY-2 year (P < .043 for all).

Complications

Of the 265 procedures performed by the interns during the initial year of the service, 4 patients (1.5%) had a major complication. All were patients who received a paracentesis and required transfusion by their primary team on the same day as the procedure; none required further intervention. Of the 77 patients that had a thoracentesis performed, all (100%) had a follow-up chest x-ray, and none (0%) suffered an iatrogenic pneumothorax or re-expansion pulmonary edema.

Feasibility

A 0.72 full-time equivalent faculty member was devoted to the training, education, and supervision of interns on the procedure service. In addition, the medical center purchased an ultrasound for the use of the service and provided procedure kits and training supplies, which were a significant expense. Training for faculty was provided by colleagues in interventional radiology and the emergency department at no cost. At the end of our first year, the service averaged almost 3 procedures per day, which represents a third of procedures needed for a self-supporting service. We estimate that we would need to perform at least 4 therapeutic thoracentesis or, at most, 9 lumbar punctures per day to be self-supporting. If a mix of procedures were performed, the service would need to perform more than 6 procedures per day.

Discussion

A 2-week rotation on the HPS provided interns with significantly more procedure experience and confidence to safely perform and supervise bedside, invasive procedures versus the standard curriculum. Benefits included a low rate of complications but sustained resident confidence, which persisted until the end of the academic year and, for many, until the completion of their PGY-2 year. The gains in confidence from the structure of the procedure service could not be separated from those resulting from the increased experience interns gained by participating in the HPS. These gains were sustained despite the relatively brief exposure to the HPS. The experience allows for repetitive performance in a well-supervised setting with dedicated feedback after each experience, which has been recommended as the “gold standard” for procedural education.^17,18 Simulation has seen similar gains in procedure-related knowledge and comfort, although most have not demonstrated such a sustained response^19,20 nor has increased confidence to supervise been assessed after simulation. Our emphasis on the modeling of supervision might also explain the sustained confidence in supervising, despite similar numbers of procedures performed and supervised during the interns' PGY-2 year. Although the intervention group may have overestimated their abilities because of having completed an intensive procedure-training program, the lack of confidence to perform or supervise procedures demonstrated by the control group remains troubling and likely represents residents' accurate assessment of their procedural deficits.

Residents routinely perform bedside procedures, often without adequate supervision, particularly during nights and weekends. Patient safety demands that residents be properly trained to perform and supervise bedside procedures. Residency programs and teaching hospitals should consider implementing programs like the HPS, which can provide supervised, dedicated procedural training that positively affects residents' comfort with performing and supervising procedures. The most notable gap between the 2 groups was in experience and confidence to perform and supervise thoracenteses.

Our study has several limitations. First, it was a single-site intervention, which limits generalizability. Second, the cost of our model could be an obstacle to wide adoption. Finally, data were self-reported, and the measures for procedure confidence and self-rated knowledge were subjective and collected using a nonvalidated survey instrument.

Conclusions

A 2-week hospitalist-supervised procedure service rotation substantially improved residents' experience, confidence, and knowledge in performing invasive bedside procedures. In addition, many residents who had completed the rotation also felt qualified to supervise these procedures throughout their training.