The ACGME case log: General surgery resident experience in pediatric surgery

Abstract

Background

General surgery (GS) residents in ACGME programs log cases performed during their residency. We reviewed designated pediatric surgery (PS) cases to assess for changes in performed cases over time.

Methods

The ACGME case logs for graduating GS residents were reviewed from academic year (AY) 1989–1990 to 2010–2011 for designated pediatric cases. Overall and designated PS cases were analyzed. Data were combined into five blocks: Period I (AY1989–90 to AY1993–94), Period II (AY1994–95 to AY1998–99), Period III (AY1999–00 to AY2002–03), Period IV (AY2003–04 to AY2006–07), and Period V (AY2007–08 to AY2010–11). Periods IV and V were delineated by implementation of duty hour restrictions. Student t-tests compared averages among the time periods with significance at P < .05.

Results

Overall GS case load remained relatively stable. Of total cases, PS cases accounted for 5.4% in Period I and 3.7% in Period V. Designated pediatric cases declined for each period from an average of 47.7 in Period I to 33.8 in Period V. These changes are due to a decline in hernia repairs, which account for half of cases. All other cases contributed only minimally to the pediatric cases. The only laparoscopic cases in the database were anti-reflux procedures, which increased over time.

Conclusions

GS residents perform a diminishing number of designated PS cases. This decline occurred before the onset of work-hour restrictions. These changes have implications on the capabilities of the current graduating workforce. However, the case log does not reflect all cases trainees may be exposed to, so revision of this list is recommended.

General surgery training is based on a model of experiential education with a broad range of cases promoting a well-rounded surgeon. Further, the transition from novice to expert is predicated upon maximizing the number of casesperformed. In most training programs, trainees will gain their operative experiences in childhood diseases on rotations devoted to pediatric surgery (PS). However, this experience may occur on any rotation in which children are cared for. Because the education of surgical residents has evolved over the last two decades, we examined a national database of operative case numbers to gain insight into the experiential training that general surgery residents undergo specifically in PS. While of interest for surgical educators, these results may also have implications on how well the surgical workforce is able to provide for the surgical needs of children.

1. Methods

1.1. Description of the data source

A request to the Accreditation Council of Graduate Medical Education (ACMGE) was made for all available case log data for general surgery residents. Each resident's case log represents the sum total of all cases performed during that trainee's entire residency. The ACGME aggregates all of the individual reports and then provides a review that consists of data for the entire graduating resident cohort for that specific academic year (AY). The ACGME provided reports on US residents graduating in AY 1989–90 through AY 2010–11. The data were provided in the form of Portable Document Files (PDFs) and was converted to an Excel (Microsoft, Redmond, WA) database to facilitate statistical analysis. We excluded procedures added over the review time period to eliminate the additive effect of new procedures. Only operative experience was included; endoscopy cases were excluded. Annual case-log data for each cohort of graduating surgery residents were combined into approximately five-year time periods to facilitate comparisons over time: Period I (AY1989–90 to AY1993–94), Period II (AY1994–95 to AY1998–99), Period III (AY1999–00 to AY2002–03), Period IV (AY2003–04 to AY2007–08), and Period V (AY2008–09 to AY2010–11).The latter three periods were delineated by the implementation of duty hour restrictions. Period III represents the last four classes of graduating residents who were trained fully in the non-restricted era. Period IV represents those classes who trained partially in the unrestricted era and partially in the era of work-hours restrictions. Period V represents the first four classes of graduating surgery residents who were trained fully under the new ACGME work hour rules (the “80-h work week”).

1.2. Analytic methods

Yearly averages and standard deviations were combined into period averages and period standard deviations; this was done for the Total Major Cases (TMC) category, the ACGME-designated pediatric cases, the five most frequent PS cases, and the “Other Pediatric” category. To evaluate changes in mean number of cases performed during each period, a one-way analysis of variance (ANOVA) was performed for periods I, II, III, IV and V utilizing the “aovsum” command (STATA version 12, StataCorp, College Station, TX), which can generate a statistical model from summary statistics (mean, SD, n); we chose the conservative Bonferroni adjustment for multiple comparisons, and significance was set at α = 0.05.

The relative contribution of pediatric cases to surgical residents' overall operative experience was calculated by dividing designated pediatric cases by the overall cases and evaluating this over the twenty years of data provided by the ACGME. Specific cases within the designated pediatrics category were examined in a similar fashion with overall and average cases for that academic year and period.

Because summary statistics such as means and standard deviations can obscure actual distributions of data, especially non-normally distributed data, we used percentiles, means, and standard deviations published by the ACGME to generate a model of the actual distribution of cases for an entire cohort of graduating residents, and this was done for three years (AY1989–90, AY2000–01, and AY2009–10) to demonstrate how the overall distribution of pediatrics cases was changing over time. The ACGME provides the 10th, 30th, 50th (median), 70th, and 90th percentiles for each graduating year. The maximum number of cases performed by a single resident is also recorded, so the right-skewed tail can be modeled (using each year's mean and standard deviation to shape the acuity of the tail between the 90th percentile to the maximum).

The ACGME began collecting data on minimally invasive PS procedures in AY1998–99. Currently, only laparoscopic anti-reflux procedures are captured by the case logs. The average number of cases per resident per year was graphed, and the relationship between year of graduation and average operative volume was assessed via simple linear regression, which was used to assess whether a significant trend was discernible from the data and the overall rate of change from year to year. Because open anti-reflux procedures were noted to begin declining shortly after coding was initiated for laparoscopic anti-reflux procedures, we also evaluated the decline in open operative experience using simple linear regression.

2. Results

The operative experience of the entire resident cohort is presented by procedure type and stratified by five time periods in Table 1. The overall average of major cases remained relatively steady over the entire periods analyzed, except for a significant drop in Period IV (P < .001), which represents the transition period immediately following initiation of ACGME-instituted work-hours restrictions; however, as Table 1 demonstrates, total operative volume, on average for graduating residents, recovered to the prework-hours restriction era by Period V. ACGME-designated pediatric cases showed a different pattern: the average number of cases per residents consistently dropped over each time period. As a result, the contribution of pediatric cases relative to the overall cases performed declined with each time period. Hernia repairs, both inguinal and umbilical, were the most common type of pediatric case listed in each time period and accounted for over half of all pediatric cases performed (Fig. 1). ANOVA revealed a significant decrease in hernia repairs between each period (P < .001 between each period).These declines in hernia repairs, in addition to the declines in the “Other Pediatric” category, were the major drivers of the overall decline in ACGME-designated pediatric cases. Among the other four most commonly performed procedures (repair of pyloric stenosis, operations for malrotation or intussusception, repair of branchial cleft or thyroglossal duct, and orchiopexy) there were variations between time periods (Table 1) but due to the relatively small percentage, these procedures did not lead to notable changes to the overall pediatric cases. All other operations accounted for less than 5% of the overall operative experience. For the first ten years of the data, the “Other Major Pediatrics” category represented the second-largest category of cases, but the number of cases categorized as “other” markedly fell in the latter half of the dataset (P < .001).Currently, pediatric appendectomy (age < 13) falls within this “other” category (personal communication), but it is not clear whether this classification has been consistent over the two decades of the data.

Table 1.

Changes in specific designated pediatric cases over different time periods.

Procedure Type	Period I AY1989–90 to AY1993–94	Period II AY1994–95 to AY1998–99	Period III AY1999–00 to AY2002–03	Period IV AY2003–04 to AY2006–07	Period V AY2007–08 to AY2010–11
Total Major Cases	904.1 (210)	950.6 (178.9)*	949.2 (198)	912.7 (177.2)*	945.8 (162.1)*
All Designated Pediatric Cases	47.7 (27.3)	42.7 (23.2)*	40.6 (19.5)	37.6 (17.9)*	33.8 (17.6)*
% of total major cases that were  designated pediatric (%)	5.4% (0.2)	4.7 % (0.3)*	4.4% (0.2)	4.3% (0.1)	3.7% (0.2)*
Hernia Repair  (Inguinal & Umbilical)	26.7 (17.2)	24.2 (15)*	22.1 (12.3)*	20.7 (10.9)*	18.5 (10.3)*
Repair Pyloric Stenosis	3.7 (3.5)	3.7 (3.0)	4.0 (3.0)*	3.7 (3.6)	3.8 (3.8)
Operations for Malrotation  or Intussusception	1.4 (1.9)	1.3 (1.8)	1.4 (1.8)	1.5 (1.8)	1.5 (2.0)
Repair of Branchial Cleft  or Thyroglossal Duct	1.4 (2.1)	1.3 (2.0)	1.2 (1.0)*	1.0 (1.2)*	0.8 (1.0)*
Orchiopexy	2.4 (3.2)	2.0 (2.8) *	1.8 (2.0)*	1.5 (2.0)*	1.4 (2.0)*
Other Pediatric Cases	5.7 (10.6)	3.8 (8.5)*	2.7 (5.9)*	1.3 (1.9)*	1.3 (2.0)

Standard deviations for the period means are reported in parentheses.

^*Significant change from immediate earlier time period (p < 0.05).

Fig. 1.

Types of cases and relative contribution to total.

In the model of the case distributions for AY1989–90, AY2000–01, and AY2009–10 (Fig. 2), the curves are skewed to the right (the median is smaller than the mean); in unimodal distributions such as these curves, rightward skew indicates that most residents performed fewer cases than the mean; that is, the mean is pulled above the median by a relatively small group of residents who perform a very large number of cases. Notably, the shape of the distribution of cases has remained quite stable over time even as overall resident experience in PS can be seen to be diminishing.

Fig. 2.

Distribution of resident experience in pediatric surgery (as Percentiles).

In terms of pediatric minimally invasive experience, this only began to exist as a coding category in AY1998– 99, and only anti-reflux procedures are subcategorized in this way (Fig. 3). A simple linear regression of the average number of laparoscopic anti-reflux procedures against graduating year reveals a significant increase per year (P < .001, R² = 0.81).While the annual rate of increase is small at 0.08 case/resident/year (95% CI 0.05–0.10), the data suggest that minimally invasive anti-reflux procedures have been performed more frequently with each passing year since AY1998-1999. Initially, when the ACGME began recording laparoscopic anti-reflux procedures, open procedures were also increasing in frequency; however, beginning in 2004–2005, average resident experience in open anti-reflux procedures began decreasing, and this trend has continued until the present (P < .001, R² = 0.96).

Fig. 3.

Anti-reflux procedures—with total number performed by graduating general surgery residents for each academic years divided between open and laparoscopic approaches.

3. Discussion

General surgery residents receive some of their most important training by learning to do procedures [1]. This experience is influential in creating a surgeon competent in key procedures and may attract some into further subspecialty training. However, there is concern that general surgery trainees are receiving insufficient exposure to the breadth of procedures that they might encounter in a general surgery practice [2,3]. Specifically in terms of PS, there is concern that current surgical trainees may not feel competent to perform pediatric surgical cases [4] and will more likely refer cases once entering practice [4,5]. This review of the ACGME case logs demonstrates a decline in PS cases from an average of 47.7 cases in the first time period to 33.8 in the most recent time period. The decline in pediatric cases can be attributed largely to a decline in herniorrhaphies, which make up over half of the designated PS cases. While there have been other changes, the contribution made by these types of cases to overall PS experience was not large enough to affect overall averages. These declines are concerning, as PS training for surgical residents is considered an important factor in the development of technical skills and surgical judgment [4] and has consistently been considered a valuable educational experience [6,7].

One of the most high profile changes to the structure of surgical residency was implementation of the 80-h work-week. As of July 1, 2003, new ACGME duty hour rules came into effect, which lead to new restrictions on the amount of time a resident could be in the hospital. There was concern by surgical educators and trainees that such restrictions would lead to a reduction in surgical experience [8-10]. In the analysis of the total major cases, there is a drop with the onset of the duty hour rules (Period IV). However that decline was reversed by Period V, and graduating residents again logged as many cases as their predecessors from the pre-duty hour restriction era. The average number of ACGME-designated pediatric cases declined between Periods III to IV and continued to decline from Periods IV to V. However, it is important to note that the decline in pediatric cases began prior to Period IV and the implementation of duty hours restrictions. Therefore, duty hour restrictions alone cannot explain the decline in pediatric experience. Nonetheless, the most recent set of changes implemented on July 1, 2011 may yet lead to further reduction in clinical experience and will need to be carefully monitored[11].

Over the last two decades, there has been a rise in the number of residents pursuing fellowship training [2,12,13], and PS is no exception. Currently there are approximately 250 surgery residencies and 40 PS fellowship programs in the US. Since general surgery residents and PS fellows co-exist and there are a finite number of cases to perform, it stands to reason that some cases may be funneling towards fellowship trainees [14]. However, since the ACGME case log data are not sortable based on institutions with and without PS fellowship training programs, it is difficult to determine to what extent this effect has made a difference. Nonetheless, with the rise in PS fellowships [15], this effect may become more substantial in the future.

Over the last two decades, there has also been a rise in the use of minimally invasive surgery overall, and pediatric surgeons have similarly embraced its adoption. However, MIS requires a trainee to have sufficient skills to be able to perform those cases. While learning these skills is not beyond early learners, the smaller workspace in children presents special challenges. Therefore, such cases may be offered only to those that have mastered MIS skills [16]. One study found that most general surgical residents are not learning how to perform pyloromyotomies in part because of the wide adoption of laparoscopic approach to this surgery [5]. And not only are pyloromyotomies being performed by MIS, so are several other designated pediatric cases such as surgery for intussusception [17], pull-through procedures [18,19], diaphragmatic hernia repairs [20], and repair of tracheo-esophageal fistula and esophageal atresia [21]. Therefore, while prior surgical residents may have performed the open version of a case, current trainees may not be offered the comparable MIS case due to insufficient skills.

Regardless of why these declines have occurred, the current general surgery graduate completes training with a different skill set than two decades ago, which raises questions as to how well the future surgical workforce will be prepared to offer surgical services to children. Broadly speaking, children may have their operations performed either by fellowship-trained pediatric general surgeons or general surgeons. While the number of fellowship-trained pediatric surgeons has been rising [15], this statistic is deceiving. Two decades ago, the rise in trainees specializing in PS was thought to be adequate to meet the needs of the community [22,23]. However, current forecasts show a strong demand for more pediatric surgeons [24-27]. A recent analysis showed that of the 3107 counties in the US, only 216 (7%) have coverage by a pediatric general surgeon [28]. Many children must travel long distances to receive care from a pediatric surgeon [29]. General surgeons often care for children who need surgical care; however, the workforce issues facing general surgeons are even more daunting. There is currently a major shortage across the country [30,31], especially in urban areas [32]. Furthermore, data on the average ages of general surgeons across the country suggest that this shortage will be magnified as practicing general surgeons increasingly retire [33]. Given this anticipated shortage of fellowship-trained pediatric surgeons and experienced general surgeons in the community, the data contained in our study come into focus: if new graduates are less equipped to provide pediatric surgical services, who will be there to care for children with surgical needs in the future?

Certainly, accurate and adequate data are required to help surgical educators (especially those interested in PS) develop appropriate polices for improving the pediatric surgical training of general surgery residents. Current tools such as the ACGME case log are not optimized to this task; furthermore, we do not have a good understanding of the PS cases that are being performed in the community outside of tertiary pediatric centers. An analysis of cases being performed in the community would be helpful to guide surgical educators in maximizing the time spent on PS rotations: ensuring that rotations focus on procedures that trainees are more likely to see in a general surgery practice makes time spent on the rotation more beneficial. Further, rethinking how surgical trainees are selected may be needed to help serve underserved areas [31]. Those destined to practice in communities not served by pediatric surgeons may benefit from focused training in PS. Augmenting PS rotations or adding additional time spent in PS training may be an effective mechanism for improving the pediatric competence level for those future general surgeons who will essentially act as PS “extenders.”

While the ACGME case log provides insight into surgical trainee experiences, there are some important limitations of this study. The current iteration of the case log system is an online system that went live on July 1, 2001. The most recent case logs would have been entered in nearly real-time by surgical trainees, thereby creating logs that should closely match their operative experiences. However, prior to this, trainees accumulated their operative data and stored them on their own, to be recorded and submitted at the end of residency. Trainees were instructed to submit this information on a diskette to the ACGME. Such a system may have been more open to inaccuracies than the web-based version, with cases added or lost due to faulty memory, and cases potentially added to fulfill mandatory requirements. Nevertheless, the data between the two systems do appear comparable, without abrupt changes after AY 2001.

A second issue is the type of pediatric cases included in the designated list. While these data provide a measure as to the number of index cases being performed by a training program, the list of cases is not optimized to provide the most useful data for the surgical illnesses that trainees should be expected to care for in the community. The list does include some commonly performed procedures (e.g. inguinal hernias, pyloromyotomies), but it is difficult to identify other common procedures in children because several are listed in other categories factored into adult case numbers (e.g. appendicitis, soft tissue procedures, management of empyema, trauma, thoracic procedures). Since these cases likely represent commonly performed procedures, we are likely underestimating the real pediatric surgical experience. To obtain an accurate view of pediatric surgical education, we would recommend that all codable cases performed on patients less than 13 years old be included as a “pediatrics case” and identified as such in the ACGME case logs. Currently, the ACGME web-based system contains a check box to identify patients below age 13, but these data are reflected neither in the case logs nor in the national reports. This information would provide a better reflection of what is truly being taught and provide insight into PS proficiency, because cases performed on children often require a separate skill set from that needed for adult procedures.

In conclusion, the past two decades have seen a decline in designated pediatrics cases by general surgery trainees. Possible causes include ACGME work hour restrictions, rise in fellowship training, and rise in minimally invasive techniques; notably, however, the data suggest that the decline started well before implementation of work hours restrictions. We recommend broadening the ACGME report for pediatrics cases so that a better assessment can be made of trainee experience in PS. Most importantly, as we consider our future surgical workforce, the data presented here suggest that our capacity to offer adequate services may be diminishing. This suggests that surgical residency programs may need to consider changing the selection and training of general surgery residents, especially those who plan to practice in communities not served by tertiary centers or fellowship-trained pediatric surgeons.