Abstract
Background There is limited information about how residents in surgical specialties view program strengths and opportunities for improvement (OFIs).
Objective This study aggregated surgical residents' perspectives on program strengths and OFIs to determine whether there was agreement in perspectives among residents in 5 surgical specialties.
Methods Resident consensus lists of program strengths and areas for improvement were aggregated from site visits reports during 2012 and 2013 for obstetrics and gynecology, orthopaedic surgery, otolaryngology, plastic surgery, and surgery programs. Four trained individuals coded each strength or OFI in 1 of 3 categories: (1) factors common to all specialties; (2) program or institutional resources; and (3) factors unique to surgical specialties. Themes were classified as most frequent when listed by residents in more than 20% of the programs and less frequent when listed by residents in less than 20% of the programs.
Results This study included a total of 359 programs, representing 27% to 49% of the Accreditation Council for Graduate Medical Education accredited programs in the 5 specialties. The most frequent strengths were progressive autonomy, collegiality, program leadership, and operative volume. Improving research and didactics, increasing faculty teaching and attendance at educational sessions, and increasing the number of nurse practitioners and physician assistants were common OFIs.
Conclusions Factors identified as important by surgical residents related to their learning environment, their educational program, and program and institutional support. Across programs in the study, similar attributes were listed as both program strengths and OFIs.
What was known and gap
Identifying program attributes important to residents may be useful in efforts to improve the residency experience.
What is new
A study that analyzed common strengths and opportunities for improvement identified by residents in 5 surgical specialties.
Limitations
Data collection as part of accreditation raises the potential for response and social desirability bias.
Bottom line
Program attributes identified by residents included aspects of their educational program, learning environment, and clinical support, which is useful for ongoing program evaluation and improvement.
Introduction
In the new accreditation system (NAS) of the Accreditation Council for Graduate Medical Education (ACGME), programs are accountable for continuous improvement. During the annual program evaluation and the NAS self-study, it is expected that programs will examine their achievements since their last accreditation review and develop action plans for future improvements.1 Soliciting residents' views about their program's strengths and opportunities for improvement (OFIs) and incorporating this information into program improvement activities will be critical to the successful implementation of this process. To our knowledge, no studies to date have explored how residents in surgical specialties judge the strengths and OFIs of their programs, either those related to the ACGME accreditation standards or those related to other program attributes important to trainees.
Since January 2012, site visit reports filed by ACGME field representatives included a consensus list of residents' views of their program's strengths and OFIs. Prior to ACGME site visits, field representatives request that residents develop a consensus list of their program's strengths and OFIs without the involvement of the program director and faculty. This list is e-mailed to the field representative and is discussed during the site visitor's interview with the residents. The list of strengths and OFIs that results from this discussion is included in the site visit report without editorial comments by the site visitor.
The goals of this descriptive study were to (1) determine factors of importance to residents in 5 surgical specialties that influence their identification as a program strength or as an OFI, and (2) assess the degree to which program aspects described as strengths and OFIs were common among the 5 surgical specialties.
Methods
The study population consisted of all accredited programs having site visits during the calendar years 2012 and 2013 in 5 surgical specialties: obstetrics and gynecology, orthopaedic surgery, otolaryngology, plastic surgery, and surgery. The review committee for each specialty scheduled a site visit if a program was due for its periodic site visit and accreditation review. Programs applying for ACGME accreditation were excluded from the study.
Site visit reports were grouped by specialty, and a unique identification number was assigned to each report. The site visit date, number of residents in the program, and the residents' lists of program strengths and OFIs were extracted from each report. The first author trained 3 individuals (a physician in academic medicine, a chemist, and a lawyer) to review the lists and to code each theme as a strength or area for improvement. A criterion of 75% agreement among the coders was required for placement of a strength or an area for improvement into 1 of 3 categories: (1) factors common to all specialties; (2) program or institutional resources; and (3) factors unique to surgical disciplines. Themes were classified in 1 of 2 groups: most frequent, when listed by residents in more than 20% of programs, and less frequent, when listed by residents in less than 20% of programs.
The study was approved by the Institutional Review Board of the American Institutes for Research.
Results
This study included 359 programs in 5 surgical specialties, representing nearly half of the ACGME-accredited programs in obstetrics and gynecology, otolaryngology, and surgery, and one-quarter of the programs in orthopaedic surgery and plastic surgery (table 1). A total of 7863 residents were enrolled in the participating programs.
table 1.
Distribution of the 359 Programs in the Study Expressed as a Percentage of Accreditation Council for Graduate Medical Education (ACGME)–Accredited Programs by Specialty and Number of Residents in Each Specialty Group
Strengths
Table 2 depicts the most frequent themes that residents identified as strengths of their programs. The common program theme of progressive autonomy was the only strength listed by residents in more than half of the programs in all 5 specialties. Residents in the majority of 4 of the 5 specialties (obstetrics and gynecology, orthopaedic surgery, plastic surgery, and surgery) rated collegiality as the next most frequent attribute. Leadership of the program director and having resident input matter in the program were listed as a strength by residents in one-quarter to more than one-third of all specialties, with the exception of surgery, where they were mentioned less often. Residents in one-third of the orthopaedic surgery programs rated didactic conferences as a strength, in contrast to a lower percentage in the other specialties.
table 2.
Most Frequent Themes Identified as Strengths by Residents in More Than 20% of Programs Expressed as Percentage of Programs by Specialty
The surgery-specific themes of operative case volume and breadth were described as a strength by residents in the majority of programs in orthopaedic surgery, plastic surgery, and surgery, and to a lesser degree in obstetrics and gynecology and otolaryngology. Residents in nearly one-quarter of programs in all specialties except otolaryngology listed senior residents teaching junior residents in the operating room as a strength.
Academic benefits, including funding for books, attendance at professional meetings and board review courses, and stipends for books and equipment, were listed as a strength by residents in more than one-quarter of otolaryngology and plastic surgery programs. They were mentioned less frequently in the other 3 specialties.
Opportunities for Improvement
Residents in nearly one-quarter to one-third of programs in obstetrics and gynecology, orthopaedic surgery, plastic surgery, and surgery noted common program themes of improving the overall quality of didactic conferences, increasing faculty teaching of didactics, and increasing faculty attendance at conferences as OFIs (table 3). Wanting more input was rated by residents in approximately one-quarter of programs in the same 4 specialties. Improving research was listed by residents in more than half of the programs in obstetrics and gynecology, more than one-third in orthopaedic surgery, and one-quarter in plastic surgery and surgery. In describing opportunities to improve research in their programs, residents most often cited lack of structure, insufficient faculty mentorship, lack of clear expectations, inadequate administrative and statistical support, and insufficient dedicated time for project completion.
table 3.
Most Frequent Themes Identified as Opportunities for Improvement by Residents in More Than 20% of Programs Expressed as the Percentage of Programs by Specialty
Residents in all specialties indicated that an increase in at least 1 specific type of operative procedure was an OFI. Within each discipline the type of operative case tended to be similar, as follows: obstetrics and gynecology—complex pelvic surgery; orthopaedic surgery—foot, ankle, and spine surgery; otolaryngology—facial plastics; plastic surgery—cosmetic and craniofacial procedures; and surgery—liver and pancreas.
The availability of nurse practitioners (NPs) and physician assistants (PAs) was listed by residents as an OFI in one-third of obstetrics and gynecology, otolaryngology, and surgery programs, and by a smaller percentage of residents in programs in orthopaedic surgery and plastic surgery. Residents in one-quarter of programs in orthopaedic surgery and otolaryngology indicated that increasing their academic benefits was an OFI.
Less Frequent Themes
Across programs, 11 characteristics were described either as a strength or an OFI (table 4), with 10 themes listed by residents in all 5 specialties. Residents in plastic surgery and surgery noted that fellows in the program were either a strength or an OFI. The same theme, such as feedback from faculty across different programs, was described both as a program strength (eg, real time feedback in the operating room and after clinical decision making) or an OFI (eg, as inadequate real time feedback).
table 4.
Less Frequent Themes Listed as Strengths and Opportunities for Improvement by Residents in Less Than 20% of Programs
Discussion
A significant finding of this study was the importance surgical residents placed on the ability of their program to foster progressive autonomy, a collegial environment, and an ample volume and breadth of operative cases. A surprising common finding was the percentage of programs where residents identified the quality of their program's educational conferences as an area for improvement.
Our results support a previous report in which residents across all specialties were noted to share common perceptions about strengths and OFIs in their learning environments.2 A strength of this study is that the themes for strengths and OFIs, derived from site visit reports, represent a consensus view from the residents and likely the issues residents consider most critical. Although there are no published data that describe residents' understanding of the accreditation process, it is assumed that they grasped the potential of their comments to have positive or adverse consequences for their program.
By rating progressive autonomy and collegiality as the most frequent common program strengths, the perspective of the residents in surgical programs in this study was similar to the results of a multiyear survey that evaluated the priorities of applicants to residencies at 2 highly selective institutions.3 In that study, programs' ability to prepare residents for their first position or fellowship and resident morale were the top characteristics sought by applicants. It is likely that the degree to which residents experienced progressive autonomy is linked to their perception of the ability of their program to prepare them for the next phase of education or practice. This strength and camaraderie among program stakeholders may be core values conveyed by residents to medical students applying to surgical programs.
This study did not distinguish between resident perceptions of progressive autonomy in the operating room and in other clinical settings. This is significant, given a recent survey of fellowship directors in which surgery graduates were found to be deficient in the domains of independent practice ability, patient responsibility, and operative skills.4 Residents in 4 of 5 specialties also listed mechanisms for senior residents to teach junior residents in the operating room as a strength, reinforcing progressive autonomy as a vital program attribute. One way to validate progressive autonomy as a key element of competence in practice could be assessment of the performance of residency graduates as judged by colleagues, supervisors, and/or fellowship directors.
Another finding relates to the importance residents placed on program leadership and responsiveness to resident input into the program. In the site visit reports, mention of this frequently was accompanied by specific examples, such as how a problem was addressed by combined efforts of the program director and the residents. When lack of responsiveness to resident input was indicated as an area for improvement, residents identified areas in which their input did not receive follow-up by program leadership.
Several themes were described across all specialties as a strength or OFI, confirming that residents' perceptions of their learning environment are similar across different surgical specialties. This was particularly evident for the educational milieu surrounding educational conferences, including their organization, degree of faculty engagement, and inclusion of preparatory sessions for the board examinations. Residents also viewed the number of NPs and PAs as an area for improvement. Recent data from a survey of academic medical centers in the University HealthSystem Consortium found that 81% reported using NPs and PAs to reduce resident clinical obligations under ACGME duty hour limits.5 The resident lists described a variety of roles that would be filled by having additional NPs and PAs available to reduce clinical workload, including covering the inpatient service while residents are in the operating room and attending conferences, expediting patient discharges, and improving continuity of care.
This study has several limitations, including a deidentified sample size without knowledge of programs' accreditation status, citations, and compliance with the common and specialty-specific requirements. There also is the potential for sample bias due to the residents who participated in the site visit interviews and whose views supplemented the consensus list of program strengths and OFIs submitted prior to the visit. In addition, each specialty was represented by less than half of their accredited core residencies, reducing generalizability. Finally, the absence of a strength or an area for improvement from the residents' consensus list did not necessarily imply that this factor was not operational within the program.
The longitudinal nature of the NAS self-study process will permit future research that evaluates how surgical programs use resident input in achieving desired improvement outcomes, both at the individual program level and within the larger context of their specialty. Differences between surgical resident and faculty perceptions of program strengths and OFIs is another area for research and is discussed in a separate article.6
Conclusion
During site visit meetings with residents representing one-quarter to one-half of programs in 5 surgical specialties, progressive autonomy, collegiality, program leadership, and operative volume and breadth were the most frequently mentioned strengths. Residents identified several common OFIs across programs, including improvements in the educational conferences, research opportunities, and additional resources, such as more NPs and PAs. Across programs in the study, similar attributes were listed as both program strengths and OFIs.
References
- 1. Philibert I, Lieh-Lai M. A practical guide to the ACGME self-study. J Grad Med Educ. 2014; 6 3: 612– 614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Philibert I. Satisfiers and hygiene factors: residents' perceptions of strengths and limitations of their learning environment. J Grad Med Educ. 2012; 4 1: 122– 127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Phitayakorn R, Macklin EA, Goldsmith J, Weinstein DF. Applicants' self-reported priorities in selecting a residency program. J Grad Med Educ. 2015; 7 1: 21– 26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Mattar SG, Alseidi AA, Jones DB, Jeyarajah DR, Swanstrom LL, Aye RW, et al. General surgery residency inadequately prepares trainees for fellowship. Ann Surg. 2013; 258 3: 64– 73. [DOI] [PubMed] [Google Scholar]
- 5. Moote M, Krsek C, Kleinpell R, Todd B. Physician assistant and nurse practitioner utilization in academic medical centers. Am J Med Qual. 2100; 26 6: 452– 460. [DOI] [PubMed] [Google Scholar]
- 6. Caniano DA, Yamazaki K, Yaghmour N, Philibert I, Hamstra SJ. Resident and faculty perceptions of program strengths and opportunities for improvement: comparison of site visit reports and ACGME resident survey data in 5 surgical specialties. J Grad Med Educ. 2016; 8 2: 291– 296. [DOI] [PMC free article] [PubMed] [Google Scholar]