Procedural Experience and Comfort Level in Internal Medicine Trainees

Abstract

BACKGROUND

The American Board of Internal Medicine (ABIM) has recommended a specific number of procedures be done as a minimum standard for ensuring competence in various medical procedures. These minimum standards were determined by consensus of an expert panel and may not reflect actual procedural comfort or competence.

OBJECTIVE

To estimate the minimum number of selected procedures at which a majority of internal medicine trainees become comfortable performing that procedure.

DESIGN

Cross-sectional, self-administered survey.

SETTING

A military-based, a community-based, and 2 university-based programs.

PARTICIPANTS

Two hundred thirty-two internal medicine residents.

MEASUREMENTS

Survey questions included number of specific procedures performed, comfort level with performing specific procedures, and whether respondents desired further training in specific procedures. The comfort threshold for a given procedure was defined as the number of procedures at which two thirds or more of the respondents reported being comfortable or very comfortable performing that procedure.

RESULTS

For three of seven procedures selected, residents were comfortable performing the procedure at or below the number recommended by the ABIM as a minimum requirement. However, residents needed more procedures than recommended by the ABIM to feel comfortable with central venous line placement, knee joint aspiration, lumbar puncture, and thoracentesis. Using multivariate logistic regression analysis, variables independently associated with greater comfort performing selected procedures included increased number performed, more years of training, male gender, career goals, and for skin biopsy, training in the community-based program. Except for skin biopsy, comfort level was independent of training site. A significant number of advanced-year house officers in some programs had little experience in performing selected common ambulatory procedures.

CONCLUSION

Minimum standards for certifying internal medicine residents may need to be reexamined in light of house officer comfort level performing selected procedures.

Competence in performing selected ambulatory and hospital-based procedures is integral to the practice of high-quality internal medicine. Surveys have found that general internists commonly perform outpatient procedures such as pelvic examinations and joint aspirations, as well as more invasive procedures such as lumbar puncture, thoracentesis, and central line placement.¹ Although a consensus may be reached about which procedures internal medicine residents need before completion of training,² not all trainees master these procedures.^3,4 Surveys of residency directors have identified areas in which internal medicine residents may be deficient,⁵ and surveys of residency graduates have identified procedures in which residents are either “overprepared” or “underprepared.”^6,7 Consequently, educational programs to teach and evaluate primary care procedural techniques have been developed.^8,9

In 1991, the American Board of Internal Medicine (ABIM) responded to a need for procedural credentialing standards among residents in training by requiring documented experience in seven core procedures.¹⁰ The ABIM now requires that programs document a minimum number of each of these procedures that must be performed before qualifying to take the Internal Medicine Certifying Examination.¹¹ However, in the absence of data, these standards were developed by expert consensus, and for several common procedures, no standards have been set. Subspecialty organizations have also defined a minimum standard for some procedures commonly done by general internists. For example, while the ABIM has no requirement, clinical observations have led to the recommendation that 10 to 30 flexible sigmoidoscopy procedures are required for competence.^1,12,13

To our knowledge, there is little data correlating procedural experience with competence in internal medicine trainees, and measuring competence in procedural skills is a difficult task.¹⁴ A first step toward defining procedural requirements for residents during their training is to correlate procedural experience with self-reported comfort in performing a specific procedure. The basis for this approach assumes that residents with a low comfort level performing a given procedure are also unlikely to be competent in that procedure. Therefore, the purpose of this study was to determine, among commonly performed procedures by internists, the minimum number at which a trainee achieves comfort, and to compare these estimates with ABIM guidelines.

METHODS

Five internal medicine training programs were approached, and all agreed to participate. All internal medicine house officers in a military-based program (Madigan Army Medical Center, Tacoma, Wash, n = 25), a community-based program (Saint Joseph's Hospital, Denver, Colo, n = 34), and two university-based programs (University of Nebraska, Omaha, Neb, n = 33 and University of Colorado, Denver, Colo, n = 140) were asked to complete a brief, voluntary, self-administered survey delineating the number (0, 1–2, 3–5, 6–10, 11–20, or 20+) of 12 selected procedures they had performed. These programs were selected because they represented a wide spectrum of training sites and were willing to participate. We included all seven procedures required by the ABIM (arterial blood gas [ABG], central venous line, knee aspiration, lumbar puncture, paracentesis, pelvic exam/Pap smear, and thoracentesis) and also five other procedures that were not required by the ABIM but that are frequently performed by internists (advanced cardiac life support [ACLS], endotracheal intubation, flexible sigmoidoscopy, joint splint application, and skin biopsy). House officers were asked to note their comfort level with performing each of these procedures using a Likert scale (1 = very uncomfortable, 2 = uncomfortable, 3 = neutral, 4 = comfortable, and 5 = very comfortable). They were also asked if they desired further training in each procedure (yes or no). Additional information obtained included gender, year of training, and career goal. The survey was conducted between December 1998 and May 1999.

The primary objective of the study was to estimate the minimum number of procedures necessary to achieve comfort for a given procedure. To accomplish this, we defined a “comfort threshold” as the number of procedures at which two thirds or more of the house staff, for a given strata of procedure number, reported being comfortable or very comfortable performing that procedure. For the ABIM procedures, we compared the number required for residents to reach the comfort threshold with the number required by the ABIM. Statistical significance was assessed for the association between procedure number and comfort level using the Cochran-Mantel-Haenszel (CMH) trend test (P < .05). To internally validate these “comfort thresholds,” we compared the reported desire for further training in a given procedure with the comfort threshold. We evaluated the independence of association between procedure number and comfort level using multivariate logistic regression, and adjusted for house staff gender, level of training, residency training site (university-based vs other), residency program type (primary care vs other), and career goals (practice vs other). To estimate how much variation in comfort level was related to procedure number, we compared the c-statistic for each multivariate model with and without the presence of covariates in the model. All statistical tests were performed using the SAS statistical application program (release 6.14) (SAS Institute Inc., Cary, NC). Results are presented as adjusted odds ratios (OR) with 95 percent confidence intervals (95% CI). This study was reviewed and approved by the Colorado Multiple Institutional Review Board.

RESULTS

The overall response rate was 83%, ranging from 61% to 91% among all four sites. The demographics of respondents, stratified by site, are shown in Table 1. For this analysis, the university-based programs were combined since they did not differ significantly. Differences between sites include a higher number of males at the military-based site and a higher proportion of trainees at the first-year level at the community hospital site.

Table 1.

Demographics of Respondents

	Study Sites
Variable	Military, n (%)	Community, n (%)	University, n (%)	Total, N (%)
Respondents	21/25 (84)	31/34 (91)	140/173 (81)	192/232 (83)
Level (P = .675)
R1	6 (29)	15 (48)	52 (37)	73 (38)
R2	7 (33)	7 (23)	42 (30)	56 (29)
R3	8 (38)	9 (29)	46 (33)	63 (33)
Male (P = .24)	15 (71)	15 (48)	84 (60)	114 (59)
Program (P = .001)
Primary care	0 (0)	0 (0)	33 (24)	34 (18)
Categorical	21 (100)	23 (74)	98 (70)	141 (73)
Transitional	0 (0)	8 (26)	9 (6)	17 (9)
Career goals (P = .52)
General practice	13 (62)	19 (61)	77 (55)	109 (57)
Subspecialty practice	6 (29)	12 (39)	53 (38)	71 (37)
Research	2 (10)	0 (0)	10 (7)	12 (6)

Figure 1 shows the number of procedures required to meet and/or exceed our comfort threshold for all respondents for four selected procedures (ACLS, central venous line placement, knee joint aspiration, and pelvic exam/Pap smear). These four procedures were selected because they reflect both inpatient and outpatient procedures and a wide range of “comfort thresholds.” As shown in Figure 1, the proportion of respondents feeling comfortable or very comfortable increases as the number of procedures performed increased. For each procedure, the association of number performed with comfort level was statistically significant (CMH trend test, P < .05 for all). Meeting or exceeding the comfort threshold for each procedure was associated with not desiring further training in that procedure (P ≤ .005) for all procedures. The minimum number of procedures performed to achieve either a comfortable or very comfortable level by two thirds of respondents from all sites for each procedure is shown in Figure 2. Current ABIM recommendations for the number of procedures to be performed are also depicted in Figure 2. To achieve a comfortable level, internal medicine trainees needed more ex periences than recommended by the ABIM for central venous line placement, knee joint aspiration, lumbar puncture, and thoracentesis. Although no specified number of intubations or flexible sigmoidoscopies have been required by the ABIM, trainees required consistently high numbers for comfort. At most, six procedures were necessary for comfort with ACLS, joint splint application, and skin biopsy, which are also procedures without ABIM requirements. Residents needed as many abdominal paracenteses as recommended by the ABIM to achieve comfort. Fewer experiences than needed for published minimum requirements occurred consistently for ABG and pelvic exam/Pap smear.

FIGURE 1.

Relationship between number of procedures and comfort level.

FIGURE 2.

Comfort threshold (range of the number of procedures at which two thirds of respondents were either comfortable or very comfortable, bracketed lines) and ABIM requirement (solid bars) for selected procedures.

We performed multivariate logistic regression analyses to determine if any variables other than number of procedures performed were independently associated with comfort level (Table 2). Not surprisingly, for each procedure, procedure number accounted for the majority of variation in comfort level for each procedure, as assessed by comparing the c-statistic for each model with and without covariates present in the model. In addition, we also found that gender, residency level, training site, and career goals were independently associated with comfort for a single procedure. Specifically, independent of number of procedures done, women were less comfortable than men in the performance of endotracheal intubation (OR, 0.38; 95% CI, 0.16 to 0.91). Higher residency level was associated with greater comfort (OR, 2.6; 95% CI, 1.4 to 4.7) in performance of ACLS but not other procedures. Training in the university-based program was associated with less comfort for skin biopsy compared to nonuniversity programs. For the remainder of the procedures, comfort was independent of training site. Private practice career goal was associated with lower comfort for central venous line placement and skin biopsy but not for other procedures. Being a transitional versus categorical resident was associated with greater comfort for endotracheal intubation (OR, 9.15; 95% CI, 1.44 to 58.07), knee joint aspiration (OR, 7.79; 95% CI, 1.16 to 52.3), and skin biopsy (OR, 7.20; 95% CI, 1.50 to 34.52), but not for any other procedures.

Table 2.

Independent Association Between Selected Variables and Comfort with Procedures, Odds Ratio(95% Confidence Interval)

							c-Statistic:Covariates#
Procedures	ProcedureNumber	FemaleGender†	ResidentTraining Level‡	Univ-BasedResidencyProgram§	Primary CareProgram∥	Private PracticeCareer Goals¶	(−)	(+)
Advanced cardiac life support	3.2 (2.1 to 4.9)*	0.80 (0.34 to 1.9)	2.6 (1.4 to 4.6)*	0.6 (0.24 to 1.7)	0.98 (0.30 to 3.2)	0.46 (0.18 to 1.1)	0.88	0.90
Abdominal paracentesis	8.6 (3.7 to 19.8)*	0.97 (0.32 to 3.0)	0.63 (0.24 to 1.6)	2.1 (0.61 to 7.4)	0.69 (0.13 to 3.7)	0.35 (0.10 to 1.2)	0.91	0.94
Arterial blood gas	3.2 (1.7 to 6.1)*	0.75 (0.22 to 2.5)	1.7 (0.61 to 4.6)	1.2 (0.30 to 4.9)	0.25 (0.38 to 16.6)	0.26 (0.7 to 1.0)	0.87	0.90
Central venous line	5.4 (3.1 to 9.5)*	0.62 (0.28 to 1.4)	0.60 (0.32 to 1.1)	0.60 (0.23 to 1.6)	0.66 (0.23 to 1.9)	0.41 (0.17 to 0.98)*	0.85	0.88
Endotracheal intubation	5.9 (3.4 to 10.5)*	0.38 (0.16 to 0.91)*	0.58 (0.32 to 1.0)	0.62 (0.23 to 1.6)	2.7 (0.83 to 8.8)	0.61 (0.24 to 1.5)	0.87	0.90
Flexible sigmoidoscopy	3.8 (2.1 to 7.1)*	0.71 (0.12 to 4.2)	1.5 (0.60 to 3.9)	1.4 (0.29 to 6.6)	0.31 (0.04 to 2.8)	2.8 (0.56 to 13.5)	0.96	0.96
Joint splint	6.5 (3.4 to 12.6)*	1.2 (0.54 to 2.7)	0.69 (0.38 to 1.2)	0.55 (0.22 to 1.4)	1.4 (0.46 to 4.2)	0.60 (0.25 to 1.4)	0.83	0.86
Knee joint aspiration	4.5 (2.4 to 8.1)*	1.2 (0.39 to 3.6)	0.69 (0.35 to 1.4)	0.65 (0.19 to 2.2)	2.3 (0.52 to 9.9)	0.43 (0.14 to 1.3)	0.83	0.86
Lumbar puncture	4.8 (2.8 to 8.0)	1.5 (0.66 to 3.4)	0.58 (0.33 to 1.0)	0.40 (0.14 to 1.2)	0.95 (0.31 to 2.9)	0.79 (0.33 to 1.88)	0.83	0.86
Pelvic exam/ Pap smear	2.8 (1.9 to 4.5)*	1.7 (0.48 to 6.0)	0.94 (0.46 to 1.9)	1.2 (0.34 to 3.9)	1.3 (0.13 to 12.8)	0.88 (0.26 to 2.9)	0.86	0.84
Skin biopsy	4.4 (2.8 to 6.8)*	0.89 (0.35 to 2.2)	0.64 (0.37 to 1.1)	0.39 (0.11 to 0.76)*	1.2 (0.28 to 5.2)	0.58 (0.13 to 1.0)*	0.86	0.90
Thoracentesis	8.1 (4.0 to 16.5)*	1.7 (0.64 to 4.5)	0.59 (0.29 to 1.2)	0.83 (0.22 to 3.1)	0.63 (0.17 to 2.4)	1.0 (0.37 to 2.8)	0.91	0.92

^*Statistically significant.

^†Odds ratio for females compared with males as a reference.

^‡Odds ratio for increased resident level (R2, R3) compared with R1 level as a reference.

^§Odds ratio for university-based residency programs compared with community- and military-based residency programs as a reference.

^∥Odds ratio for primary care residency program type compared with categorical and transitional program types as a reference.

^¶Odds ratio for primary practice career goals compared to subspecialty and research career goals as a reference.

^#The c-statistic estimates the variation in comfort level due to procedure number, without (−) and with (+) covariates for each multivariate logistic regression analysis.

We also assessed the level of procedural experience received by trainees prior to program completion. This analysis was confined to trainees in their third year of training. Table 3 presents the percentage of trainees that either met or exceeded ABIM requirements or other published requirements.^1,12,13 Procedures for which there are no current ABIM requirements, with the exception of flexible sigmoidoscopy,^12,13 are not included in the table. For most of the procedures, postgraduate year 3 (PGY 3) residents met current recommendations. For some procedures (flexible sigmoidoscopy, joint splint application, and skin biopsy), a significant percentage of PGY 3 residents had had no experience. Specifically, 75% of military, 44% of community, and 43% of university seniors had reported no joint splint applications. Twenty-two percent of community and 61% of university PGY 3 trainees had never performed a flexible sigmoidoscopy. Twenty-two percent of community and 33% of PGY 3 trainees had never performed a skin biopsy. All military residents had reported performing at least one of each of these procedures. Seven percent of university PGY 3 residents had never performed a knee joint aspiration, and 4% had never performed an endotracheal intubation.

Table 3.

Proportion of Senior Residents by Training Site who Met or Exceeded ABIM or Published Guidelines and with Comfortable or Very Comfortable Attitudes Toward Performing the Procedure^*

	Military (n = 8)		Community (n = 9)		University (n = 46)
Procedure	Met orExceeded	Comfortableor VeryComfortable	Met orExceeded	Comfortableor VeryComfortable	Met orExceeded	Comfortableor VeryComfortable	P Value
Paracentesis	100	75	100	67	100	67	.91
Arterial blood gas	100	100	78	67	85	100	.001
Central venous line	100	88	89	67	100	72	.584
Flexible sigmoidoscopy	50	38	22	44	2	17	.136
Knee aspiration	68	13	89	22	74	11	.646
Lumbar puncture	88	88	89	89	76	78	.668
Pelvic exam/Pap smear	75	100	89	100	98	94	.559
Thoracentesis	100	100	100	100	83	86	.144

^*All statistical comparisons performed using the χ² test.

DISCUSSION

The present study provides data on internal medicine trainee comfort level with regard to performance of selected procedures. In this study, the number of procedures trainees needed to perform to feel comfortable for central venous line placement, knee joint aspiration, lumbar puncture, and thoracentesis exceeded the number currently required by the ABIM. For arterial puncture for blood gases and pelvic exam/Pap smear, trainee comfort occurred at numbers less than those recommended by the ABIM. Also, four procedures for which the ABIM currently has not made recommendations (ACLS, endotracheal intubation, joint splint application, and skin biopsy), required five or more procedures to establish comfort. Residents required more than 20 flexible sigmoidoscopies to achieve comfort, correlating with published recommendations for certification.^12,13

Interestingly, among procedures without specific ABIM requirements, the residents' comfort threshold surpassed prior published recommendations for attaining competence among practicing internists.¹ Of practicing general internists, a minimum (25th percentile) estimate of the number required to attain competence for flexible sigmoidoscopy was 10, with 8 and 2 for endotracheal intubation and skin biopsy, respectively.¹

Although comfort level was most strongly related to procedure number, other factors sometimes related to trainee comfort level independent of the number of procedures performed. For example, women were less comfortable than men in doing endotracheal intubation regardless of the number done. While the reason for these differences may be complex, published data suggest that men may feel more self-confident with technically invasive or aggressive procedures in the medical environment.^15,16 Because first-year residents receive formal ACLS training at the end of internship in these hospitals, it is not surprising that senior residents feel more comfortable performing ACLS independent of number done. The greater comfort of community-based residents over university-trained residents for skin biopsy may reflect the fact that the community-based program either trained a higher proportion of first-year preliminary house officers transitioning into dermatology in their second year or had more skin biopsy experience in their training program.

While most trainees in their final year of training met or exceeded requirements for the ABIM required procedures, for some procedures a significant minority had not. For example, for knee joint aspiration, a relatively low fraction of university-trained third-year trainees (74%) had performed at least three, and 7% had performed none. Experience was less for procedures not required by the ABIM. For example, 48% of the third-year residents had never performed a flexible sigmoidoscopy. However, exposure to this procedure depended on site of training; all of the military-trained residents had done at least 6 flexible sigmoidoscopies, and 50% had done more than 20. By contrast, 22% and 61% of the community-based and university-based trainees, respectively, had done none. Many internists have no interest in performing flexible sigmoidoscopy, so universal training requirements may not be necessary. For skin biopsies, all of the military-trained residents had done at least 1, while 22% and 33% of the community-based and university-trained residents, respectively, had performed none. Differences in procedural experiences may reflect more opportunity among military residents to do procedures, differences in emphasis on trainee procedural competence at various sites, or differences in reliance upon specialists in dermatology and gastroenterology in the various programs.

Several limitations should be considered in interpreting the results of this study. First, competence in performing a procedure is the primary goal for trainees and training programs alike; comfort is likely distinct from competence in performing a procedure. Although the comfort threshold was arbitrarily defined (the number of procedures at which two thirds of respondents report comfort or high comfort levels), the strong associations between comfort threshold and desire for further training in that procedure support our definition. Comfort is only a proxy, and perhaps a weak one, for competence. It could be argued that our arbitrarily defined comfort level is not sufficiently stringent since one third of residents still do not feel comfortable with the procedure. Second, the survey required trainees to remember the number of procedures they had performed and choose from a listed range of experiences. Not only could memories be inaccurate, they could also be subject to recall bias. For example, residents who feel uncomfortable with a procedure may report having performed fewer than they actually performed, while residents who are comfortable with a procedure may report having performed more. Third, because our data are based on group averages, they may not be applicable at the individual level. For example we defined comfort as the level at which two thirds felt comfortable. The other one third may have required a much higher number of procedures to achieve comfort. Fourth, our results reflect the experiences of selected programs, and may not be generalizable to internal medicine programs across the country or to other non-internal medicine specialties. Fifth, because the survey was administered 6 to 9 months into the final training year for third-year trainees, the results do not reflect the PGY 3 senior residents' experiences in their last months prior to graduation. Sixth, our study is cross-sectional, and causal relationships between associated variables have not been demonstrated. Seventh, although our study had a high response rate, nonrespondents may have differed from respondents in ways that were unmeasured and resulted in unrecognized selection bias. Finally, our analyses cannot account for intrinsic differences in residents with regard to their individual interests, skills, and desires in performing procedures. For example, it is possible that there is a subset of “procedure-loving” residents with lower thresholds for doing procedures and with greater interest and comfort in doing procedures.

In summary, the results of our study suggest that trainees require more experiences to achieve comfort in several procedures commonly performed by practicing internists than are currently recommended by the ABIM, regardless of the type of training program. Some procedures are not commonly performed by a high percentage of trainees. Greater emphasis on the performance of such procedures appears to be indicated in some of the internal medicine training programs studied. Future research should focus on correlating comfort level with achieving demonstrable procedural competence.