Perspective of ophthalmology residents in the United States about residency programs and competency in relation to the International Council of Ophthalmology guidelines

Abstract

Purpose

To evaluate the perspective of ophthalmology residents in the US about their residency programs and compare the competency of residency programs to international competency levels set by the International Council of Ophthalmology (ICO).

Methods

A cross-sectional web-based survey extracted from the ICO published competency standards was sent to program directors of ophthalmology residency programs in the US to forward it to current PGY-3, 4 residents, and residency graduates from 2011 to 2014.

Results

Eighty-seven responses were received, comprising 61 residents and 26 graduates. Most respondents were highly satisfied with their programs (93.6%). Clinic-based training was rated satisfactorily. Insufficient exposure to low-vision rehabilitation (38.5%), refraction and contact lenses prescription (38.5%), and vitreo-retinal surgeries (38.5%) was reported. Respondents were satisfied with their overall surgical experiences, with the vast majority (>83%) rating case volume, complexity, and variety as satisfactory or better. A significant group stated they had insufficient exposure to extra-capsular cataract extraction (26.3%), refractive surgery (19.7%), and orbital surgery (64.5%). All graduates surveyed passed their Ophthalmic Knowledge Assessment Program (OKAP) examinations, and 72% felt their residency programs adequately prepared them for the examinations. All respondents reported insufficient training in certain nonclinical areas, such as practice management, staffing, and administration skills.

Conclusions

Ophthalmology residents in the US express high levels of satisfaction with their residency training programs. While most programs adequately address most ICO core objectives, certain curriculum modifications should be considered.

Introduction

The International Council of Ophthalmology (ICO) published a series of rules and regulations that set the competencies and skills every resident in ophthalmology worldwide should achieve before graduating from their programs.¹ Before these regulations, residency programs provided residents with variable skill levels in each ophthalmic subspecialty. In the United States, the Accreditation Council for Graduate Medical Education (ACGME) sets the guidelines and follows up the performance of each residency and fellowship program to warrant their compliance with the ACGME guidelines.² However, US programs were never studied in projection next to international standards.

While some of ICO standards are considered by US standards to be beyond the scope of most comprehensive ophthalmologists, it is more likely that the postgraduate education system will face future demands in the form of technological advances, diversification of procedures, and essential integration of skills that will push all residency programs to continually reassess and expand their curricula beyond the current scope to make sure their graduates are armed with sufficient skills and knowledge to tackle the rapidly evolving field.3, 4, 5

While ophthalmology residents in the US give annual assessments to the ACGME regarding their programs, no measures are taken to make sure these evaluations are protected from programs' influence. This article is the first to report residents' comprehensive, anonymous, and independent evaluation of their own residency programs for evaluation outside of their program or for purposes aside from the ACGME.

Methods

A previously compiled survey derived from the ICO guidelines was used in this study.⁶ These guidelines outlined the curriculum for education that included didactic knowledge as well as the clinical and surgical skills that every ophthalmologist-in-training should acquire during each of their residency years of training. In each ophthalmic subspecialty, the guidelines list the necessary scientific and medical concepts, communication skills, surgical techniques, and managerial skills an ophthalmology specialist should be armed with ahead of starting their practice. ICO guidelines were used as a reference to the goals and objectives outlined in the survey. In these guidelines, the basic level of skills corresponded to the U.S. post-graduate year (PGY)-2, the standard level corresponded to the U.S. PGY-3, and the advanced level corresponded to the U.S. PGY-4.¹ The guidelines were listed as educational points in areas that every ophthalmology specialist should be able to master. The survey was designed to illustrate these points and give a general idea about the competency level of each resident and graduate presumed to be exposed to these educational areas. For questions related to satisfaction or quality of teaching, we used a five-point Likert scale (very satisfied, satisfied, neutral, unsatisfied, and very unsatisfied). For questions related to competency or whether the respondent felt comfortable performing a certain procedure, we used a three-point scale (yes, no, and unsure). A copy of the survey is available as a supplemental file to this article.

In the US, there are 116 ACGME-accredited ophthalmology residency programs. Contact information for these programs was accessed through the American Medical Association database and Fellowship and Residency Electronic Interactive Database (FREIDA). The survey was emailed to all programs in the winter of 2014, with a grace period of 8 weeks for receiving responses. Emails were directed to both program directors and chairmen of all programs, with the request to forward the survey to their PGY-3 and 4 residents and 2011–2014 graduates.

In the survey, a clear statement was made to all participants that the data was being kept confidential, and the residency programs would not be able to see the individual respondents' answers. Nevertheless, information about geographic location and the program name for every respondent was collected. Email reminders were sent on a bi-weekly basis to programs that did not have any residents or graduates participating in the survey. An incentive was provided for all programs in the form of a comprehensive evaluation report of the points of strength and weakness in their programs in addition to tailored recommendations derived from every program's respondents.

The study was executed in concordance with tenets of the Declaration of Helsinki and was certified exempt by the Institutional Review Board of the David Geffen School of Medicine, University of California Los Angeles.

Results

Demographic information

Responses were received from 87 participants (n = 61 residents and 26 graduates). Residents in PGY-3 comprised 29.9% of the cohorts (n = 26), with residents in PGY-4 representing 40.2% (n = 35). Graduates were mostly within one year of graduation (n = 15), with fewer graduates within two years (n = 7), within three years (n = 3), and within four years of graduation (n = 1). Most graduates were currently working in a private practice setting (53.9%). Those having current academic appointments were almost 16% of all graduates. The remainder were current fellows undergoing their fellowships in the US, with one respondent undergoing an international fellowship (Table 1).

Table 1.

Demographic information of survey respondents.

Characteristic	n (%)
Level of training
Residents PGY-3	26 (29.9)
Residents PGY-4	35 (40.2)
Graduates within 1 year	15 (17.2)
Graduates within 2 years	7 (8)
Graduates within 3 years	3 (3.4)
Graduates within 4 years	1 (1.1)
Geographic Area of Training
US Northeast	6 (27.3)
US Midwest	8 (36.4)
US West	2 (9.1)
US South	6 (27.3)
Current Work Status for Graduates
Fellow in the US	7 (27)
Fellow internationally	1 (3.4)
Pursuing fellowship in the US	0 (0)
Pursuing fellowship internationally	0 (0)
Working in a community practice	14 (53.9)
Working in an academic practice	4 (15.4)

PGY = post-graduate year.

General satisfaction

Most respondents reported satisfaction with their programs, with 48.7% rating as “very satisfied” and 44.8% rating as “satisfied”. On sub-group analysis, satisfaction levels were found to be similar regardless of the respondent's level of training (92.3% for PGY-3, 94.3% for PGY-4, and 92.3% for graduates, rating as satisfactory or better). Most respondents felt very satisfied with case volume, complexity, and variety within their programs (83.5%, 87.3%, and 86.1% being “satisfied” or “very satisfied”). On sub-group analysis, residents in PGY-3 and PGY-4 showed lower satisfaction rates with case volume and complexity (77.8% and 77.8%, respectively).

Quality of teaching

Most respondents rated the satisfaction with the teaching curriculum as “satisfied” or “very satisfied”. Elements of the curriculum included didactic courses (77.2%), operating room (88.6%), outpatient clinic (84.8%), grand rounds (84.8%), wet labs (77.2%), conferences (83.5%), and journal clubs (73.4). The only elements that received lower satisfaction levels were hospital rounds (49.4%) and journal clubs (68.8%) among residents in PGY-3.

Surgical volumes

Of all participants, and by the time of participating in the survey, 43% had performed less than 50 cataract phacoemulsification surgeries, 77% performed less than 10 trabeculectomies or tube shunts, and 58% performed less than 20 strabismus surgeries. Of graduates, and during their entire residency, 38% performed between 150 and 200 cataract surgeries.

Of PGY-4 residents, 43% performed between 50 and 99 cataract surgeries, between 1 and 10 trabeculectomies or tube shunt surgeries, and between 10 and 20 strabismus surgeries.

All PGY-3 residents performed less than 50 cataract surgeries, none of them performed any trabeculectomies or tube shunt surgeries, and 41% had performed between 10 and 20 strabismus surgeries.

Of graduates, 27% performed between 10 and 20 trabeculectomy surgeries during their residencies and between 10 and 20 strabismus surgeries (Table 2).

Table 2.

Level of satisfaction and surgical volumes achieved by respondents.

Questions		All respondents (n = 87)	PGY-3 (n = 26)	PGY-4 (n = 35)	Graduates (n = 26)
		“Very satisfied” or “satisfied”, %
General satisfaction		93.6	92.3	94.3	92.3
Operative experience	Case volume	83.5	77.8	77.8	96.2
Case complexity	87.3	85.2	85.2	92.3
Case variation	86.1	85.2	84.9	88.5
Quality of teaching	Didactic	77.2	81.5	71.4	76.9
O.R.	88.6	85.2	82.9	96.2
Clinic	84.8	81.5	82.9	92.1
Hospital rounds	49.4	51.9	45.7	57.7
Grand rounds	84.8	81.5	85.7	84.6
Wet lab	77.2	74.1	74.3	80.8
Conferences	83.5	88.9	85.7	76.9
Journal club	73.4	70.4	68.8	77.1
		Percent of respondents
Surgical volume	Cataract
0–49	43.0	100	28.57	0
50–99	15.19	0	42.86	0
100–149	10.13	0	17.14	15.38
150–199	13.9	0	5.71	38.46
>199	17.78	0	5.71	46.16
Trabeculectomy
0	36.71	85.2	31.43	0
<10	40.51	14.8	57.14	42.31
10–20	18.99	0	11.43	46.15
21–30	2.53	0	0	7.69
>30	1.27	0	0	3.85
Squint
0	6.33	18.5	0	0
<10	17.72	37.04	11.43	0
10–20	34.18	40.74	42.86	26.92
21–30	10.13	3.70	20	3.85
31–40	13.92	0	20	19.23
41–50	8.86	0	5.7	23.08
>50	8.86	0	0	23.08

PGY = post-graduate year; O.R. = operating room.

Ophthalmology skills

All respondents expressed variable levels of comfort mastering general skills such as prescribing glasses, contact lenses, phacoemulsification, extracapsular cataract surgery, toric intraocular lenses (IOL) implantation, refractive surgery, and cornea surgery. The largest percent felt comfortable prescribing glasses (100% of PGY-4), phacoemulsification (100% of graduates), and toric IOL implantation (96% of graduates). However, the lowest percent felt comfortable performing refractive surgery (0% of PGY-3), extracapsular cataract extraction (18% of PGY-4), and contact lens prescription (23.5% of PGY-4) (Table 3).

Table 3.

Levels of comfort mastering ophthalmology skills reported by respondents.

Questions		All respondents (n = 87)	PGY-3 (n = 26)	PGY-4 (n = 35)	Graduates (n = 26)
		“Comfortable”, %
Anterior segment skills	Glasses prescription	74.4	96	100	96.2
Contact lenses Prescription	27.6	16	23.5	38.5
Phacoemulsification	71.1	16	91.2	100
ECCE	26.3	4	17.7	57.7
Toric IOL	64.5	4	88.2	96.2
Refractive surgery	19.7	0	17.7	34.6
Cornea surgery	27	8.3	27.3	42.3
Glaucoma	Glaucoma ALT	82.9	56	94.1	96.2
Glaucoma complications	64.5	32	67.7	84.6
Retina	Posterior segment management	82.7	68	82.4	100
Vitreoretinal surgery	18.4	4	8.8	38.5
Oculoplastic	Lid Trauma management	94.7	92	100	92.3
Orbital trauma management	64.5	72	64.7	61.5
Lid surgery	84.2	72	97.1	84.6
Lacrimal surgery	39.5	44	44.1	30.8
Enucleation	61.8	40	76.1	65.4
Conjunctival tumors	60.5	48	61.8	76.9
Pediatrics	Clinical pediatrics	81.6	68	85.3	92.3
Squint management	56.6	52	64.7	53.9
ROP	29.3	25	23.5	38.5
Pediatric surgery	69.7	52	79.4	76.9
Low vision rehabilitation	27.6	16	23.5	38.5
		“Yes”, %
Managerial skills taught?	Professionalism	81.6	84	76.5	92.3
Practice management	42.1	28	44.1	53.9
Administrative skills	23.7	12	29.4	34.6
Ethics	82.9	76	88.2	88.5
Practice settings	54	40	52.9	65.4
Information technology	64.5	64	58.8	73.1
Feedback received?	Clinic setting	81.1	80	85	84
O.R. setting	89.0	84	94	92
Educational progress	93	96	94	88
OKAP preparation	64.1	52	70	72

ECCE = extracapsular cataract extraction; IOL = intraocular lens; ALT = argon laser trabeculoplasty; ROP = retinopathy of prematurity; OKAP = ophthalmic knowledge assessment program; PGY = post-graduate year; OR = operating room.

More than 90% of PGY-4 residents reported comfort in performing glaucoma procedures, such as Argon laser trabeculoplasty (ALT) and Selective Laser Trabeculoplasty (SLT), and 68% of them reported comfort in managing glaucoma complications, with a higher percent expressed by graduates (85%). Most PGY-4 residents also reported comfort in the management of posterior segment diseases (82.3%), though only 8% reported comfort in performing vitreoretinal surgical procedures. More than 60% of participants felt comfortable performing oculoplastic procedures, such as managing lid trauma, orbital trauma, lid surgery, enucleation, and conjunctival tumors. However, only 44% of PGY-4 residents and 31% of graduates felt comfortable performing lacrimal surgeries.

Participants expressed variable levels of comfort in mastering pediatric ophthalmology skills. Most respondents felt comfortable in clinical pediatric encounters (81.6%), but less than 65% of PGY-4 and 54% of graduates felt comfortable managing strabismus in children. Only 38% of participants expressed comfort in managing retinopathy of prematurity and low vision rehabilitation.

Managerial and academic performance

Most respondents commented favorably on managerial areas such as professionalism and ethics (81.5% and 82.8%, respectively), while few confirmed that they were exposed to practice management (42.1%) or administrative skills (23.6%). Most respondents stated that they were given adequate and timely feedback about their academic performance, such as their clinic and outpatient performance (81%) and operating room performance (89%), and the majority had time and opportunities to discuss progress with the program director (93%). The majority of respondents also commented favorably that their programs adequately prepared them for their ophthalmic knowledge assessment program (OKAP) examinations (72% of graduates and 70% of PGY-4).

Discussion

This study examines the international standards set by the ICO in relation to the skills and curricula provided by ophthalmology residency programs in the US, using the most important beneficiaries of these programs: residents and graduates. The ICO residency curriculum was based on an analysis of curricula for the training of ophthalmologists from more than 30 countries, collected and reviewed by the task force.¹ These standards while itemized, each of them was not quantified. However, they were tested both qualitatively and quantitatively in several countries, including Canada, Jordan, and India.4, 5, 6 In our study, results reveal that the U.S. residents express high levels of satisfaction with their programs, with perceptions of efficiency reported in most subscales of the guidelines. However, some areas were found to be common sources of inadequacy according to participating residents.

Most residents commented favorably on the usefulness of wet labs and surgical simulators offered by their programs (77.2% satisfaction rate). These simulators provide an excellent opportunity to train on models before training on real patient eyes in the operating room, thereby providing an opportunity to develop and enhance fine motor skills and the experience of managing potential complications. It was previously reported that simulators can actually help improve surgical outcomes for residents.7, 8, 9 Slightly over half of the respondents commented that they felt their experience with patient rounding was inadequate. This might be attributed to either the lack of an in-patient facility at their respective institutions, or the lack of the educational setting that allows faculty members to supervise residents during their patient encounters. Also, the availability of journal clubs was a potential area of deficiency and may represent a lack of such an element in some didactic programs.

With regard to surgical volumes, it is not surprising that most respondents felt highly satisfied with the number of cataract surgery procedures they performed. However, some residents have never performed some surgeries such as trabeculectomy or tube shunts and strabismus surgeries. Refractive surgery was also one area in which only 19.7% of the participating residents felt they were exposed to enough. These inadequate training complaints are similar to those reported previously by McDonnell et al.¹⁰ It is not surprising that this area is lacking enough exposure since most refractive surgery procedures take place in private practice settings while most residency programs are university-based. Another inadequate exposure was training related to low vision rehabilitation, although patients access this service only by referral from community ophthalmologists to university centers.¹¹ Moreover, and though still crucial as the only option in complicated cataract, most residents did not feel competent performing extracapsular cataract surgeries (82%). Surgical volumes reported by U.S. residents were found to be – on average – lower than the numbers reported in the United Kingdom and Canada; however, the length of training programs is different in each country.4, 12, 13

Some surgeries, such as vitreoretinal surgeries, were reported to have a very low competency level (8%), which raises concerns about the management of routine surgery complications that might require basic competency in handling the posterior segment inside the operating room (e.g. anterior vitrectomy). Although vitreoretinal skills are thought to be something acquired during a fellowship in the US, the ICO specifies a few surgical skills that should be learned before graduation, namely cryotherapy, pan-retinal photocoagulation, scleral buckling, and supervised pars plana vitrectomy. This also raises considerations about the surgical outcomes of simulators presented specifically for the sake of enhancing vitreoretinal surgery skills.14, 15, 16

Teaching in non-clinical areas was rated satisfactory in regard to professionalism and ethics. However, only 42% and 24% of respondents reported having received any teaching on practice management and administrative skills, respectively. This is not surprising because most residency programs are university-based, and practice management skills are often not part of the physician's everyday activities. This findings was also previously reported.¹⁷ However, residents in Canada reported much higher numbers when surveyed about these same skills.⁶

In a previous study,⁴ many suggestions were given to address the areas of low competency or lack of comfort among residents. Examples of such suggestions include networking of universities with private practice groups to allow residents to perform refractive surgery and learn the basics of practice management and administrative skills. Also, networking with organizations specializing in low-vision support and establishing international affiliations to allow residents to travel for overseas electives may be useful to support their training on complicated cases and to perform extra-capsular cataract surgeries.

A potential for selection bias exists as it is possible that residents and graduates who were satisfied with their training were more confident about their competency and would have been more likely to respond to the survey. For this reason, we compared our study data to the ACGME annual surveys data. Though only information about surgical volumes was accessible, we found that the required minimum number of procedures for graduating residents echoed the surgical volumes reported by our study respondents. For instance, ACGME requires residents to perform at least 86 cataract surgeries before graduation, and nearly half of our PGY-4 respondents reported performing 50 to 99 cataract surgeries while the other half performed more than 100 surgeries. Also, while ACGME requires a minimum of 5 glaucoma surgeries for graduation, half of our PGY-4 respondents performed between 1 and 10 surgeries and the other half performed more than 10 surgeries. Lastly, according to ACGME, the required minimum number of strabismus operations to be performed by the last year resident is 10, and 43% of PGY-4 residents in our survey had done between 10 and 20 surgeries with the rest performing more than 20 surgeries. This shows that respondents of this survey study – despite their small number – may validly be representative of the population of ophthalmology residents in the U.S. as a whole.

Our study has several limitations, one of which is the response rate which might be a potential source of ascertainment bias, although such response rates are considered high for online surveys. Because of the way the study was carried out, and the unavailability of the data collected by the ACGME, we were not able to compare our results to theirs. Also, part of the data represents the graduates' perspectives about the quality of education they received in the past, which might include a recall bias, but this would not apply to the residents' responses which we analyzed separately for this purpose. Aside from that, each resident and graduate will most likely change their perspective over time during their residency training in regard to each component of the survey. The lower response from graduates is also a limitation, but this can be attributed to a loss of contact between residency programs and their alumni.

One of the strengths of this study is the anonymous nature of the survey. Because the survey was confidential, there was much less chance that its answers were influenced by concerns of disclosure to residency program directors or other department faculty members.

In summary, U.S. ophthalmology residents express high levels of satisfaction with their residency training programs. Although most programs appear to adequately address most ICO core objectives, certain curriculum modifications should be studied in a larger-scaled study in order to improve the residents' perspective of their programs.

Appendix A. Supplementary data

The following is the supplementary data related to this article: