Does operative experience during residency correlate with reported competency of recent general surgery graduates?

Abstract

Background

Identification of attributes of residency training that predict competency would improve surgical education. We hypothesized that case experience during residency would correlate with self-reported competency of recent graduates.

Methods

Aggregate case log data of residents enrolled in 2 general surgery programs were collected over a 12-month period and stratified into Surgical Council on Resident Education (SCORE) categories. We surveyed recent (< 5 yr) residency graduates on procedural competency. Resident case volumes were correlated with survey responses by SCORE category.

Results

In all, 75 residents performed 11 715 operations, which were distributed by SCORE category as follows: essential-common (EC) 9935 (84.8%), essential-uncommon (EU) 889 (7.6%) and complex 891 (7.6%). Alimentary tract procedures were the most commonly performed EC (2386, 24%) and EU (504, 56.7%) procedures. The least common EC procedure was plastic surgery (4, 0.04%), and the least common EU procedure was abdomen–spleen (1, 0.1%). The questionnaire response rate was 45%. For EC procedures, self-reported competency was highest in skin and soft tissue, thoracic and head and neck (each 100%) and lowest in vascular–venous (54%), whereas for EU procedures it was highest in abdomen–general (100%) and lowest in vascular–arterial (62%). The correlation between case volume and self-reported competency was poor (R = 0.2 for EC procedures).

Conclusion

Self-reported competency correlates poorly with operative case experience during residency. Other curriculum factors, including specific rotations and timing, balance between inpatient and outpatient surgical experience and competition for cases, may contribute to procedural competency acquisition during residency.

Surgical residency must prepare future general surgeons for a variety of professional experiences, ranging from subspecialty practice in urban centres to general surgical practice in rural communities in which the full complement of surgical specialty coverage may be lacking. Canadian data on the distribution of general surgeons by practice type include the 2011 Canadian Association of General Surgeons (CAGS) survey in which 19% of 335 respondents identified their practices as “community” in type.¹ The 2009 CAGS survey identified 14% of respondents as practising in a community with a population of 50 000 or fewer, whereas 47% of respondents worked in centres with a population of 500 000 or more.²

Depending on the specific needs of a surgical practice and the operative expertise acquired during residency, newly trained surgeons may feel variably prepared for their professional roles. Kaminsky and colleagues³ reported that most Canadian residents in their final year of training expressed confidence with basic general surgical procedures, including laparoscopic cholecystectomy, appendectomy, mesh hernia repair, simple mastectomy and hemor-rhoidectomy, yet expressed a lack of confidence in their ability to perform procedures that are often required in rural jurisdictions, including gynecological, genitourinary and orthopedic surgery. The same appears to be true for advanced laparoscopic procedures, with less than 25% of Canadian-trained residents reporting feeling comfortable performing laparoscopic splenectomy, Heller myotomy, fundoplication and adrenalectomy.⁴ Hence, Canadian general surgical training programs face a challenge in producing graduates whose acquired skills during residency result in self-perceived competence during early professional practice. Despite formalization of surgical curricula and the establishment of CanMEDS training competencies,⁵ it seems likely that gaps exists between competencies achieved during residency and competencies required across the spectrum of Canadian general surgical practice.

In recognition of the diversity of expertise required by Canadian general surgeons and of the critical need for general surgeons in community-based practice,⁶ the present study sought to compare the educational and technical skills acquired during residency with the skill set requirements perceived by recent general surgical graduates of 2 western Canadian general surgical training programs.

Methods

We conducted a prospective, institutional research board–approved study involving general surgery residents at the University of Alberta (UofA) and University of British Columbia (UBC), and surgeon graduates of both programs who were within their first 5 years of practice. Both residency programs lead to eligibility for certification by the Royal College of Physicians and Surgeons of Canada and the American Board of Surgery. Both universities have other postgraduate surgical training programs that have some overlap with general surgery: UofA has fellowship programs in hepatobiliary and transplant, minimally invasive/bariatric, thoracic and acute care surgery, whereas UBC offers fellowship training in minimally invasive, pediatric, thoracic and vascular surgery.

General surgery resident case experience

Case volumes for selected surgical and endoscopic procedures performed by general surgery residents at the 2 programs who commenced their training on or before July 1, 2009, were included. Residents whose current year of training was interrupted by planned (e.g., research) or unplanned absences were excluded. Both programs use T-RES (Resilience Software Inc.) case-counting software, and only those cases in which the resident logged his or her role as “primary surgeon” or “teaching assistant” were counted.

Aggregate case numbers were combined between the 2 programs. To allow for differences in program length (5 yr at UofA v. 6 at UBC), aggregate operative case experience was subcategorized according to postgraduate year (PGY) as either junior (PGY 1–2) or senior (PGY 3–6).

Operative case categories

The aggregate surgical case experiences of residents from the UofA and UBC programs were then categorized according to the Surgical Council on Resident Education (SCORE) case classification system, which was developed by the U.S. Association of Program Directors in Surgery.⁷ This operative case classification tool stratifies resident operative experiences into 3 categories, with the following definitions.

1. Essential-common (EC) comprises frequently performed operations in general surgery; specific procedure competency is required by the end of training (and should be attainable primarily by case volume).

2. Essential-uncommon (EU) includes rare, often urgent, operations seen in general surgery practice and not typically performed in high volume by trainees; specific procedure competency is required by the end of training (but cannot be attained by case volume alone).

3. Complex describes procedures not consistently performed by general surgeons in training and not typically performed in general surgery practice. Generic experience in complex procedures in residency is required, but competence in individual complex procedures is not required. Some residency programs may provide sufficient experience for competence in some specific procedures.

The UBC program directors reviewed the SCORE case categorization in detail and amended it slightly to more accurately reflect Canadian general surgery residency training expectations.

Self-reported competencies of recent general surgery program graduates

Surgeons who had graduated from the UofA or UBC general surgery residency programs within the last 5 years were contacted by letter and/or email and invited to participate in this study. Those who consented were sent a link to a questionnaire. After 1 week, everyone was sent a second message, which served as a “thank you” for those who had already responded and as a friendly reminder for those who had not. Three weeks later, a follow-up email with a questionnaire link was sent to nonresponders only. Two weeks after this, nonresponders’ offices were contacted by phone to verify postal/email addresses before sending out a final invitation to participate.

The questionnaire was designed to collect information on practice demographics; subspecialty training (if any); general surgery call (among subspecialists); and the benefit of additional, specific residency training. In addition, using the modified SCORE-categorized operative case lists, each surgeon was asked to self-report his or her perceived procedural competency using a 3-point scoring system, with 3 indicating fully competent, 2 somewhat competent and 1 not competent). An aggregate competency score for each procedure was determined from the actual score divided by the maximum possible score and expressed as a percentage. For example, if 5 surgeons reported full competence (5 × 3 = 15) and 5 surgeons reported partial competence (5 × 2 = 10), the aggregate competency score would be 25 ÷ 30 × 100 = 83%.

Statistical analysis

We calculated the procedure-specific correlation between the aggregate case volume of general surgery residents and the aggregate competency score of recent general surgery graduates for EC and EU categories using the Pearson product moment correlation coefficient.

Results

Residents

A total of 15 764 logged cases were abstracted from 75 residents with a PGY distribution as follows: PGY-1 (n = 15), PGY-2 (n = 16), PGY-3 (n = 13), PGY-4 (n = 14), PGY-5 (n = 9) and PGY-6 (n = 8). Cases for which the resident was not the primary surgeon or teaching assistant were excluded, leaving a total of 11 715 (74.3%) cases for analysis. The case numbers per modified SCORE category were 9935 EC, 889 EU and 891 complex. Alimentary tract procedures were most commonly performed in the EC (2386, 24%) and EU (504, 56.7%) categories. The least commonly performed EC procedures were plastic surgery (4, 0.04%), abdomen–spleen (40, 0.4%) and thoracic (49, 0.5%), whereas the least commonly performed EU procedures were abdomen–spleen (1, 0.1%), pancreas and endocrine (each 12, 1.3%), and vascular–arterial (15, 1.7%).

General surgical graduate competencies

In all, 9 of 25 (36%) and 10 of 17 (58%) identified surgical graduates completed the UofA and UBC surveys, respectively, for an overall response rate of 19 of 42 (45%). Of the 19 respondents, 8 had completed 1 or 2 years of additional subspecialty training after their general surgery residency. Among subspecialty-trained surgeons, 6 of 8 took general surgery call. The practice demographics and details of additional training of respondents are shown in Table 1. Self-reported competency for EC cases was highest for skin and soft tissue, thoracic and head and neck (each 100%) and lowest for vascular–venous (54%), plastic surgery (55%) and endoscopy (84%). For EU cases, self-reported competency was highest for abdomen–general (100%), head and neck (96%), and skin and soft tissue and abdomen–hernia (each 93%) and lowest for vascular–arterial (62%), endocrine (63%) and alimentary tract (80%). Of the 19 respondents, 9 (47%) indicated that they could have benefited from additional training during residency, with rotations in plastic surgery and surgical oncology being the areas most frequently cited (Table 2).

Table 1.

Attributes of recent graduates of general surgery residency programs

Attribute	No. (%)
Sex
Male	15 (78.9)
Female	4 (21.1)
Population of the community served
50 000–100 000	5 (21.1)
More than 100 000	14 (78.9)
Hospital type
24/7 coverage for all surgical subspecialties	8 (42.1)
24/7 coverage for all surgical subspecialties not available	11 (57.9)
Subspecialty training	8 (42.1)
Pediatric surgery	1 (12.5)
Thoracic surgery	1 (12.5)
Hepatobilliary	1 (12.5)
Colorectal	1 (12.5)
Minimally invasive surgery	1 (12.5)
Trauma, intensive care unit	3 (37.5)
Subspecialty trained but cover general surgery calls	6 (75.0)

Table 2.

Aggregate self-reported competencies and areas of need for additional training among recent graduates of general surgery residency programs

Essential-common	% competency	Essential-uncommon	% competency	Additional training needed	No.
Abdomen–general	96	Abdomen–general	100	Surgical oncology	2
Abdomen–hernia	93	Abdomen–hernia	93	Minimally invasive surgery	1
Abdomen–liver and biliary	98	Abdomen–liver and biliary	81	Plastic surgery	3
Abdomen–spleen	91	Pancreas	82	Thoracic surgery	2
Alimentary tract	96	Abdomen–spleen	88	Head and neck surgery	2
Pediatric surgery	91	Alimentary tract	80	Orthopedic	2
Plastic surgery	55	Skin and soft tissue	93	Vascular surgery	2
Endoscopy	84	Trauma	87	Urology	1
Breast	99	Vascular–arterial disease	62	Pediatric surgery	1
Endocrine	91	Thoracic surgery	81	Neurosurgery	1
Skin and soft tissue	100	Pediatric surgery	83	Trauma	1
Surgical critical care	96	Genitourinary	88	Obstetrics/gynecology	1
Vascular–venous	54	Head and neck	96
Thoracic surgery	100	Endocrine	63
Head and neck	100

Residents at UofA complete 1 month of plastic surgery rotation in their junior years, whereas UBC residents do not have this rotation at all. There is no specific surgical oncology rotation at UofA, whereas UBC offers a month of surgical oncology rotation as a junior and 3 months as a senior. About 75% of junior and 50% of senior residents are assigned to surgical oncology at UBC. The type and duration of surgical rotations in each program are summarized in Table 3.

Table 3.

Type and duration of rotations by training program

Rotation	Duration of residency, mo.
	UBC		UofA
	Junior (1, 2)	Senior (3–6)	Junior (1, 2)	Senior (3–5)
General surgery	6	7–16 *	12	20
Vascular surgery	2	0	0	2
Endocrine	GS	GS	GS	GS
Colorectal	GS	GS	GS	GS
Thoracic surgery	2	0	1	3
Community surgery	0	3–12†	0	3
Acute surgery	4	3	GS	GS
Surgical oncology	1‡	3‡	GS	GS
Trauma	2	2	GS	GS
Endoscopy	0	2	0	3
Pediatric surgery	2	0	1	2
Plastic surgery	0	0	1	0
HPB–transplant	0	3	0	GS

GS = no dedicated rotation — experience attained during general surgery rotation; HPB = hepatopancreatobiliary; UBC = University of British Columbia; UofA = University of Alberta.

^*Minimum of 7 months. Includes a 3-month rotation on service, which provides a combined experience of colorectal with thyroid and parathyroid surgery, all residents rotate on service.

^†Minimum of 3 months.

^‡Includes thyroid and parathyroid experience, only a subgroup (75% as junior, 50% as senior) is assigned to this rotation.

Pearson correlation analyses (Figs. 1–3) established a weak association between resident aggregate case volumes and surgical graduates’ perceived competencies for EC cases only (R = 0.2). There was essentially no correlation between case volume and reported competency for EU cases. Despite the weakness of overall correlation for EC cases, there was an association trend between decreased competency reported by general surgeons and low case numbers performed by residents. For instance, lowest perceived competency among graduates (vascular–venous and plastic surgery) corresponded with very low resident case volumes for these procedural categories (Fig. 1).

Fig. 1.

Residents’ aggregate essential-common case volumes and surgical graduates’ perceived competencies.

Fig. 3.

Resident aggregate complex case volumes by residency level. Junior = postgraduate year (PGY) 1–3; senior = PGY 4–6.

Discussion

Training general surgeons to be competent assumes that operative skills and knowledge acquired by the end of residency will match what is required in surgical practice. For surgeons in rural communities, the scope of surgical skills required often goes beyond that which is taught in a traditional general surgery residency program. A report from a recent graduate general surgeon in British Columbia expressed concern about residency preparation for community practice in multiple specialty areas, including pediatric surgery, endoscopy and hand surgery.⁸ A survey comparing urban and rural surgeons in the United States found that a perceived need for additional training was greater in 7 of 8 specialty areas (gynecology, cesarean sections, urology, thoracic surgery, endoscopy, orthopedics, plastic surgery and hand surgery) among surgeons in rural practice.⁹ This opinion is supported by practice data, which demonstrate that rural surgeons spend 27% of their operative time performing endoscopic, gynecologic, orthopedic, urologic and otolaryngology procedures. This is in contrast to the 5% of operative time that urban surgeons devote to operations that most training programs would consider to be outside of the classic realm of general surgery.¹⁰

An awareness of the unique surgical skill set required of surgeons who are going to practice in rural jurisdictions has resulted in the development of broadly based training programs in the United States. One in particular, the Mary Imogene Bassett Hospital in central New York State, has for 50 years employed a rural hospital–based curriculum that has resulted in its graduates performing more cases in the areas of genitourinary, plastics, hand, gynecology, nervous system and orthopedics procedures than the national averages of all U.S. programs.^9,11 Another study estimated that about 10% of U.S. programs offer training that is somewhat adapted to the needs of rural surgeons based on any 1 the following 3 attributes: rural location of training program, rural-focused curriculum and self-identified interest in rural training. However, these attributes alone do not necessarily identify programs that are more likely to place graduates into rural practice settings.¹²

The situation in Canada is less well studied, but almost certainly comparable. From a perspective of geography (land area) and population density, the need for broadly trained surgeons may be proportionately greater in Canada than in the United States. Unlike the situation in the United States, Canadian general surgery training programs have remained rooted in urban centres by the nature of their affiliation with large universities. However, many have modified the traditional urban teaching hospital–based curriculum to include rotations at more rural locations (for example, the UBC general surgery program has had rural surgery rotations since 1991, and the UofA general surgery program has a 3-block rural rotation in PGY-4). Nonetheless, based on recent Canadian reports citing deficiencies in preparedness for rural surgical practice,^6,8 we conducted the present study to explore more thoroughly the association between resident case logs and the perception of competence of recently graduated general surgeons.

Our survey identified specific clinical areas where surgeons perceived insufficient competence. For EC procedures, these areas included plastic surgery, vascular venous and, to a lesser extent, endoscopy, whereas for EU procedures, areas included vascular–arterial and endocrine. Essential common procedures, by SCORE definition, are procedures for which “competency is primarily attainable by sufficient case volume,” yet we found that the overall correlation between resident case volumes and surgeon’s self-reported competency for both EC and EU cases was poor, although resident case volumes were extremely low (5 cases or fewer) for some procedures associated with insufficient competence (e.g., plastic surgery).

If resident surgical case volumes are a poor proxy for a surgical graduate’s competence, then there are likely to be other curriculum factors that contribute to the development of competence. An obvious factor is whether the curriculum includes a formal rotation in a specific discipline. For plastic surgery, for example, it is likely that the lack of a formal rotation at UBC and a short rotation at UofA results both in a low number of performed procedures (technical competence) and a lack of clinical expertise, which together likely contribute to self-perceived competence. However, simply offering an inpatient rotation in a focused discipline does not guarantee that residents will receive the educational exposure necessary for competence acquisition. Whereas most general surgery training programs require a rotation in vascular surgery, which exposes residents to inpatient vascular surgery, there is likely to be limited exposure to outpatient varicose vein surgery. Another curriculum determinant of reduced competence may be a lack of clarity or logistical barriers to achieving subspecialty-specific training objectives within a general surgery program. A recent study by Sidhu and colleagues¹³ suggested that varicose vein surgery, leg amputation and femoral embolectomy are procedures for which competency should be achieved by Canadian general surgery residents during the course of training, yet the extent to which that is achievable is unknown.

Another curriculum variable that likely contributes to the development of competence is the timing of a rotation within the 5- or 6-year residency. This may be relevant to the acquisition of competence in pediatric surgery, which tends to be a junior resident rotation, as data from the 2 programs surveyed indicate. High competence in EC procedures in this category (91%) despite limited operative experience could have been attained by performing similar procedures (e.g., appendectomy, hernia repairs) on adults. On the other hand, most EU procedures were exclusive to pediatric surgery (e.g., excision of thyroglossal duct cyst), and were performed in limited numbers. As a result, our data showed a pattern of insufficient competence (83%) in general surgery graduates. The same appears to be true for endoscopy (84%), in which junior residents performed most reported cases and senior residents performed fewer but more advanced endoscopies (e.g., endoscopic retrograde cholangiopancreatography).

Competence in endocrine and vascular arterial surgery may have been diminished owing to competition for operative case experience between general surgery residents and subspecialty fellows. Such a phenomenon in endocrine surgery among general surgical graduates has been well documented in the United States and has been variably attributed to the evolution of minimally invasive adrenal surgery (for which general surgery residents are likely to compete with minimally invasive surgery [MIS] fellows) and to competition with otolaryngology training programs for thyroid and parathyroid surgery.¹⁴ The centralization of subspecialty general surgery (e.g., colorectal, endocrine, surgical oncology, advanced laparoscopy) into “centres of excellence” may decrease resident experience owing to competition with an affiliated fellowship program. One fellowship that has become especially popular is MIS. Kothari and colleagues¹⁵ demonstrated that despite the coexistence of an MIS fellowship, general surgery residents did not experience a reduction in the total number of basic and nonbariatric advanced laparoscopic cases, nor did they perform fewer operations during the chief year.

Although hepatopancreaticobiliary (HPB) surgery has traditionally been in the domain of general surgeons, our data and that of others suggest that despite exposure to advanced HPB surgery and liver transplantation, the experience obtained is unlikely to be sufficient to produce competency in liver and pancreas resections,¹⁶ which themselves are procedures that should not be performed outside of specialty centres. Therefore, reduced competence in HPB surgery is less relevant than, for example, competence in varicose vein surgery for recent graduates practising outside of a specialty centre.

Our study has a number of important findings, including the identification of several practice areas (particularly plastic, vascular, and endocrine surgery and surgical oncology), in which recent program graduates self-report insufficient competence. Our data suggest that case volume alone does not determine competence, but that there are other curricular factors involved, such as the specific rotations offered; the balance between inpatient and outpatient experience; the timing of rotations in junior versus senior years; and the effects of other programmatic influences, including subspecialty centralization and the competition for procedures between residency and fellowship programs.

Limitations

Our study has some limitations. The most important limitation is that our data are not longitudinal, as the case log experience is not that of the surveyed graduates. Another limitation is the small sample size, both from the perspective of the number of surgical programs surveyed (2) and the survey response rate among recent program graduates (45%). The small number of respondents (19) introduces the possibility of a type-II error, which may mean that a stronger correlation between EC case volume and self-reported competence of graduates could actually exist (as is predicted for SCORE EC cases). These limitations prevent us from making assumptions about the generalizability of our findings.

Conclusion

Our data offer a contemporary insight into the challenges of creating a balanced educational curriculum in general surgery that will meet the needs of future program graduates.

Fig. 2.

Residents’ aggregate essential-uncommon case volumes and surgical graduates’ perceived competencies.