Education Research: Neurology training reassessed

Abstract

Objective:

To assess the strengths and weaknesses of neurology resident education using survey methodology.

Methods:

A 27-question survey was sent to all neurology residents completing residency training in the United States in 2011.

Results:

Of eligible respondents, 49.8% of residents returned the survey. Most residents believed previously instituted duty hour restrictions had a positive impact on resident quality of life without impacting patient care. Most residents rated their faculty and clinical didactics favorably. However, many residents reported suboptimal preparation in basic neuroscience and practice management issues. Most residents (71%) noted that the Residency In-service Training Examination (RITE) assisted in self-study. A minority of residents (14%) reported that the RITE scores were used for reasons other than self-study. The vast majority (86%) of residents will enter fellowship training following residency and were satisfied with the fellowship offers they received.

Conclusions:

Graduating residents had largely favorable neurology training experiences. Several common deficiencies include education in basic neuroscience and clinical practice management. Importantly, prior changes to duty hours did not negatively affect the resident perception of neurology residency training.

There have been dramatic changes in neurology residency training.¹ The Accreditation Council for Graduate Medical Education (ACGME) instituted duty hours, restricting residents to 80 hours/week in 2003 with at least 1 day off per week and 12 hours in between shifts.² These duty hours were further refined in 2011 with the restriction of postgraduate year–1 shift lengths to less than 16 hours and graduated supervision of middle and senior residents.³ Residents are also challenged by increased clinical productivity demands. Despite the restricted timeline for training, residents must develop the necessary skills to become proficient in neurology.

The Graduate Education Subcommittee (GES) has been charged by the Workforce Task Force of the American Academy of Neurology (AAN) to evaluate the training residents receive by using a survey every 3 years.⁴ This process allows the AAN to receive feedback regarding the quality of the training process and identify deficiencies. The 2008 AAN Resident Survey represented one of the largest efforts to date to assess neurology resident education.⁵ Residents assessed the impact of duty hour restrictions, faculty and curriculum quality, and attitudes regarding fellowship training and made specific recommendations based on those data. The current survey assesses the effect of those recommendations and the quality of neurology resident education as perceived by trainees.

METHODS

The chair of the Consortium of Neurology Residents and Fellows (CNRF) revised the original 2008 survey created by members of the CNRF and GES. The member research subcommittee (MRS) reviewed the draft prior to distribution.

The target audience was all US adult and child neurology residents who completed training in 2011 (n = 742). Excluded from survey distribution were members not in their final year of training, residents who had received 3 surveys in the past 3 years, officers of the CNRF, and members of the GES and MRS. The survey was distributed in May 2011 via postal mail with a cover letter or e-mail with a link to the online version. Second and third reminders were distributed by postal mail and e-mail. Data collection closed in July 2011. Survey analysis combined adult and child neurology residents. Longitudinal differences between survey responses were tested for significance using Fisher exact test.

RESULTS

The survey response rate was 49.8% (308/619). The margin of error for all respondents was 5.4%, 95% confidence level. The average age of the respondents was 33 years and 57.9% of the respondents were male. Differences in age and gender between the survey respondents and nonrespondents were not significant.

Residents were generally satisfied with their training (appendix e-1 on the Neurology® Web site at www.neurology.org). Fifty-nine percent rated their neurology faculty as excellent. The majority of residents believed their programs adequately prepared them in the areas of patient management and differential diagnosis. Both clinical skills training and grand rounds conferences rated very well or well (90% clinical skills training, 83% grand rounds conferences). However, resident endorsement of basic science education was less robust; only 54% rated preparation in basic science as very well or well. The majority of residents were satisfied with research opportunities provided during residency. A minority of residents (28%) reported somewhat well or not well training in subspecialty areas they desired. A review of their comments indicates nerve conduction studies (NCS)/EMG was the most commonly deficient subspecialty area.

The majority of residents did not believe their residency prepared them adequately for practice issues (billing, contracts, malpractice, coding, and office management), with the exception of electronic health records. The majority of residents (67%) supported residency training as the appropriate time to learn this information.

In comparison with the previous survey, all residents who responded to this survey worked under the 2003 duty hours restrictions and prior to the institution of the 2011 duty hour restrictions. The majority of residents responded that patient care and resident education have either been positively impacted or not affected by the changes, while endorsing a positive impact on resident's quality of life.

A total of 86% of residents intend to pursue fellowship training following residency, split evenly between their current institution and another institution. A total of 81% of these respondents will be pursuing a clinical fellowship. When evaluating how residents chose their fellowship, the 3 top reasons were patient contact, academic environment, and quality of life. Financial reasons and location were less important in residents' decisions. Most fellowships will be 1 year in duration. Residents were most often guided in their decision by a mentor at their institution. Following fellowship training, 37% plan to enter academic practice, 23% private practice, 7% clinical or basic science research, and 29% are undecided.

The majority of residents (71%) agreed the Residency In-service Training Examination (RITE) assisted them with self-study. A total of 14% of respondents indicated the scores were used as one component of fellowship selection criteria, promotion to the next year of residency training, selection for honors, or for comparison to other residents. The RITE is intended as an educational tool within residency. Other use may qualify as misuse of this examination, specifically use as a certifying or qualifying examination, such as selection of candidates for fellowship positions.⁶

Most foreign medical graduates (79%) intend to stay in the United States following completion of their residency training.

One objective measure of neurology training might be considered ABPN board pass rates. In 2008, adult and child neurology applicants took the same examination, while in 2011 the examinations were separate (table e-1). Overall, neurology residents had a statistically significant decrease in their first and overall pass rates despite similar examination format (ABPN staff, personal communication, 2011).

Similar items between the 2 surveys were analyzed for change. No responses were statistically different except for an increase in the number of residents entering fellowship (p < 0.05) (table 1). Specifically, residents remained satisfied with their clinical training and continued to identify deficiencies in practice issues and basic sciences. A similar percentage of respondents endorsed the RITE as helpful for self-study and reported their scores used for purposes other than self-improvement. There was no change in resident career choices following fellowship training.

Table 1.

Comparison of the 2008 and 2011 neurology resident surveys

Abbreviation: RITE = Residency In-service Training Examination.

^aSum of top 2 response options; either excellent and good, very well and well, or strongly agree and agree.

^bAveraged sum response of the excellent and good options across billing, contracts, malpractice, coding, and office management. Note: a question about electronic health records was asked on the 2011 survey but not on the 2008 survey and is not included in this analysis to allow for comparisons.

DISCUSSION

This survey is a longitudinal effort to capture the quality of neurology residency training, assessed by the graduating trainees, and is the largest of its kind to date.^4,5,7,8 Overall, neurology residents are generally satisfied with their faculty and their training. This positive response is consistent over time. Residents feel duty hour restrictions have had a positive impact on quality of life. The 2011 resident survey was completed after implementation of the 2003 restrictions and prior to implementation of the most recent work hour restrictions.³ The impact of the new work hour restrictions on resident quality of life and clinical care is unclear.

Consistent across surveys is a perceived deficiency in basic neuroscience education. The recent removal of a time-based basic science education requirement from residency program requirements may further diminish basic science education.² The previous authors noted the addition of a basic science curriculum at the AAN annual meeting to address this deficiency.⁵ This course is well attended, but has not impacted residents' perceptions of their basic science preparation. There are ongoing efforts to make this curriculum available in a Web-based format for national use. Additionally, ACGME core competencies may need to clarify the emphasis of basic science education in a neurology residency.

Practice management education is a deficiency in neurology residency, despite residents' belief that residency is the appropriate time and venue. Though the AAN offers practice management courses and a series of Webinars, these offerings do not provide the foundational education in practice management appropriate for residents. The authors note several ongoing efforts by the AAN to address this deficiency by either developing content within the AAN or contracting with an outside entity.

A minority of residents did not receive the clinical subspecialty training they desired, most commonly EMG, EEG, critical care, and neuro-oncology. Similar concerns were not noted in the 2008 study.⁵ The ACGME's program requirements for neurology mandate certified faculty in all neurology subspecialties be available to neurology residents during their training.² Although growth in emerging fields such as neurocritical care may improve educational access to these areas in some programs, deficiencies in exposure to common technical disciplines, such as EEG and NCS/EMG, likely represent programmatic issues within specific residencies. Compliance with these ACGME directives has taken on heightened importance, and the development of formalized reciprocation programs between institutions may become necessary.^5,9

A significantly higher fraction of trainees are pursuing subspecialty fellowship training, resulting in a fundamental education shift with fellowship becoming an extension of residency. Internal AAN member data indicate 25% of junior members are fellows. This is notable given the relative lack of attention this group receives by regulatory and specialty organizations. Aside from patient contact and the educational environment of the fellowship, quality of life is the third most common factor in deciding which fellowship to pursue; 50% of respondents indicate quality of life was a very important factor in their fellowship decision. This may influence traditionally underselected subspecialties to tailor their fellowships to become more competitive. Following fellowship training, more residents express a preference for an academic practice setting than private practice, although 28% were undecided. This preference differs from AAN membership demographics, which report 21% of US neurologists practice in a university setting,¹⁰ and suggests a change in practice preference during fellowship training.

This survey assesses the quality of residency training as perceived by the trainee at the time of completion. There is a paucity of objective measures of residency training quality. One objective measure, neurology certification first-time pass rates are significantly lower in the 2011 group (table e-1) (ABPN staff, personal communication, 2011). This dataset was restricted to the years residents were surveyed, is of unclear significance, and does not necessarily constitute a trend toward lower pass rates. Other potential measures include trainee perception 10 years after completion of residency, which may further inform the perception of strengths and weaknesses. Beyond resident perception, future studies may obtain the perspective of employers and hospital systems with regard to specific core competency or milestone outcomes.

One potential limitation of this study is the relatively low response rate on both the 2008 and 2011 surveys. This rate may potentially bias the results toward a subset of residents and may not represent the entire cohort. A second limitation is the combined analysis of adult and child neurology residents. While this provides a global perspective, future surveys ought to include subanalysis of both residencies. Finally, this is an anonymous survey which limits the authors' ability to provide clarification of respondents' answers. It is possible that respondents may interpret questions differently. However, questions provided in 2008 and 2011 were answered similarly, which suggests similar interpretations between different cohorts of residents.

Neurology residents appear satisfied with the structure and quality of their training program. Consistent limitations of training programs include the availability of all subspecialties, education in practice management, and basic neuroscience. Since these same areas were cited as problem areas in the 2008 survey, an innovative approach to addressing these issues will be required. Given the AAN infrastructure already in place, we anticipate transitioning AAN educational programs in basic neuroscience and practice management to an online format as a potential response to these concerns.

GLOSSARY

AAN

American Academy of Neurology

ACGME

Accreditation Council for Graduate Medical Education

CNRF

Consortium of Neurology Residents and Fellows

GES

Graduate Education Subcommittee

MRS

member research subcommittee

NCS

nerve conduction studies

RITE

Residency In-service Training Examination

AUTHOR CONTRIBUTIONS

Nicholas E. Johnson: drafting manuscript, design of project. Matthew Maas: revision of manuscript, analysis of data. Mary Coleman: design of project, revision of manuscript. Ralph Jozefowicz: design of project, revision of manuscript. John Engstrom: revision of manuscript, analysis of data.

DISCLOSURE

The authors report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.