Remediation in the Context of the Competencies: A Survey of Pediatrics Residency Program Directors

Abstract

Background

The 6 competencies defined by the Accreditation Council for Graduate Medical Education provide the framework of assessment for trainees in the US graduate medical education system, but few studies have investigated their impact on remediation.

Methods

We obtained data via an anonymous online survey of pediatrics residency program directors. For the purposes of the survey, remediation was defined as “any form of additional training, supervision, or assistance above that required for a typical resident.” Respondents were asked to quantify 3 groups of residents: (1) residents requiring remediation; (2) residents whose training was extended for remediation purposes; and (3) residents whose training was terminated owing to issues related to remediation. For each group, the proportion of residents with deficiencies in each of the 6 competencies was calculated.

Results

In all 3 groups, deficiencies in medical knowledge and patient care were most common; deficiencies in professionalism and communication were moderately common; and deficiencies in systems-based practice and practice-based learning and improvement were least common. Residents whose training was terminated were more likely to have deficiencies in multiple competencies.

Conclusion

Although medical knowledge and patient care are reported most frequently, deficiencies in any of the 6 competencies can lead to the need for remediation in pediatrics residents. Residents who are terminated are more likely to have deficits in multiple competencies. It will be critical to develop and refine tools to measure achievement in all 6 competencies as the graduate medical education community may be moving further toward individualized training schedules and competency-based, rather than time-based, training.

What was known

For the past decade, the 6 ACGME (Accreditation Council for Graduate Medical Education) competencies have been used in resident assessment.

What is new

A survey of pediatrics program directors about deficiencies requiring remediation showed that residents in remediation most often have deficiencies in medical knowledge and patient care, with deficits in professionalism and communication less common, and deficiencies in systems-based practice and practice-based learning and improvement being the least common.

Limitations

Nonvalidated survey; a 54% response rate may introduce respondent bias.

Bottom line

Medical knowledge appears to be the most commonly remediated and most “remediable” competency. When residents require remediation of several competencies, this is more likely to be unsuccessful, leading to termination.

Introduction

In 2002, the Accreditation Council for Graduate Medical Education (ACGME) formally introduced the 6 competencies (medical knowledge [MK], patient care, professionalism, communication, practice-based learning and improvement [PBLI], and systems-based practice [SBP]) as the framework of assessment for all residents and fellows.¹ This framework was intended to shift the educational paradigm from one based on fulfillment of specified processes to one based on the achievement of specified outcomes.² During the past 10 years, much work has been done to improve the understanding and assessment of the competencies, with more than 200 articles published on the topic. However, few studies have addressed the impact of the competencies on remediation, the process of correcting knowledge deficits or other deficiencies in trainees with the ultimate goal of producing well-trained and competent attending physicians.^3,4

Remediation was recently described as occurring in 3 steps: (1) the identification of deficiencies in an individual's performance; (2) an attempt to provide remedial education to that individual; and (3) reassessment to determine the impact of the education.⁵ This conceptual model of remediation was used to develop a survey of pediatrics residency program directors in order to study remediation in pediatrics training. The goal of this article is to describe the degree to which each of the competencies contributed to deficiencies requiring remediation.

Methods

Data were obtained via an anonymous online survey of pediatrics residency program directors. The survey was based on the 3-step approach to remediation described above and was developed in consultation with medical education and survey design experts at the authors' institution. The survey included items to characterize the programs, determine the frequency of remediation, identify the methods used for each stage of remediation, and describe the outcomes of remediation. For the purposes of the survey, respondents were instructed to define remediation as “any form of additional training, supervision, or assistance above that required for a typical resident.” Respondents were asked to quantify the residents requiring remediation owing to problems in each of the competencies. Respondents were also asked to quantify the residents with problems in each of the competencies whose training was extended or terminated owing to issues related to remediation. For each of the 3 domains, respondents were instructed to include residents who had deficiencies across multiple competencies in the counts for each individual competency.

After pilot testing with several members of the graduate medical education community at the authors' institution to ensure clarity and content validity, a link to the survey was distributed to program directors of core pediatrics residency programs by using the Association of Pediatric Program Directors (APPD) listserv. The FREIDA Online database maintained by the American Medical Association⁶ was used to obtain contact information for programs for which the program director was not listed in the APPD membership. Two reminders were sent to the APPD listserv during the fielding of the survey.

For the purposes of this study, all data were aggregated before analysis. All analyses were performed by using the STATA version 10.1 statistical software package (2009; STATA Corp, College Station, TX). The frequency of deficiencies in each of the 6 competencies was calculated for residents requiring remediation, residents whose training was extended for remediation purposes, and residents whose training was terminated owing to issues related to remediation. Chi-square analysis was used to compare residents whose training was terminated to those who successfully completed remediation.

The University of Michigan Medical School Institutional Review Board determined this study as exempt.

Results

The survey was completed by 105 of 196 (54%) program directors. Respondents represented self-identified academic, community, and combined programs of ranging sizes (table). Overall, there were approximately 4500 residents enrolled in the programs of respondents at the time of the survey. Respondents reported that 545 residents required remediation during the 5 years leading up to the survey. Of those, 145 residents required extension of training, and 118 residents experienced termination of training owing to issues related to remediation.

TABLE.

Program Characteristics

Program directors reported the highest number of deficiencies in MK and patient care for residents requiring remediation. Deficits in professionalism and communication were moderately common, while those in SBP and PBLI were reported least often. The same pattern of competency-based deficiencies was reported for residents whose training was extended or terminated owing to issues related to remediation (figure).

FIGURE.

Deficiencies in Core Competencies Among Residents Requiring Remediation

Frequency of deficiencies in each of the 6 core competencies among residents requiring remediation. Residents whose training was extended and those whose training was terminated are subsets of residents requiring remediation. Residents whose training was terminated were more likely to have deficiencies in patient care, professionalism, communication, practice-based learning and improvement (PBLI), and systems-based practice (SBP) than those who successfully completed remediation (all P < .05).

From the absolute number of deficiencies reported for residents requiring remediation, at least 26% (142 of 545) of the residents requiring remediation had deficiencies in multiple competencies. In comparison, with the same mathematical approach, at least 42% (50 of 118) of those residents whose training was terminated had deficiencies in multiple competencies.

Residents whose training was terminated were equally likely to have deficiencies in medical knowledge as compared to residents who successfully completed remediation. However, residents who were terminated were significantly more likely to have deficiencies in patient care (77% [91 of 118] versus 43% [184 of 427], P < .001), professionalism (45% [53 of 118] versus 35% [150 of 427], P < .05), communication and interpersonal skills (45% [53 of 118] versus 29% [124 of 427], P < .01), PBLI (36% [43 of 118] versus 11% [47 of 427], P < .001), and SBP (19% [23 of 118] versus 7% [30 of 427], P < .001) than residents who successfully completed remediation.

Discussion

The most important finding of this study is that a deficiency in any competency may result in remediation of pediatrics residents. However, those residents who are unable to successfully complete remediation and ultimately undergo termination of training are more likely to have deficits in multiple competencies. This finding may be due to increased difficulty performing remediation for multiple deficiencies because residents with multiple deficiencies may not be capable of becoming competent physicians, or may be due to increased assessment of residents who continue to struggle. It will be essential to study each of the 3 steps of remediation for residents with specific patterns of deficiencies in order to inform improved remediation for this higher-risk group.

Overall, the most common deficits reported were in MK and patient care. This is somewhat different from a prior study of residents needing remediation in neurology residencies, in which MK deficiencies were reported less often.³ The difference in the frequency of MK deficits may be due to underlying differences in the population of neurology and pediatrics residents, or it may be due to differences in the requirements for competence or assessment of MK in neurology and pediatrics residencies. With the exception of MK and SBP, which was not measured in the neurology study, the patterns seen in the study of neurology residents needing remediation were similar to those found in this study of pediatrics residents requiring remediation.

The relative frequencies of deficits in a given competency for residents who successfully completed remediation versus those whose training was terminated can be used as a proxy for the ability to remediate that competency. When using this approach, MK appears to be the most “remediable” competency. The reasons for this finding cannot be ascertained from this study. It is possible that additional reading and study is an established process for programs to assign, and residents to complete, contrasted with deficiencies that require change in complex behaviors and attitudes related to skills in communication or professionalism.

We hypothesize that the lower frequency of deficits and poor success in remediating deficiencies in PBLI and SBP may be related to the fact that these are newer constructs with a limited collection of measurement tools. This is consistent with a prior study of remediation practices in general surgery residencies, suggesting that programs struggled least with remediation of MK and patient care, while many had no approach for remediation of PBLI and SBP.⁴ It will be critical to develop and refine tools to measure achievement in all 6 competencies as the graduate medical education community moves further toward more individualized training schedules and competency-based, rather than time-based, curricula. The Milestones Project of the ACGME^7,8 and the Program Directors' Guide to Professionalism of the American Board of Pediatrics⁹ represent 2 examples of these efforts.

This study has several limitations. The survey was developed for this study and specific items in it have not been validated. The respondents may not be reflective of the overall population and a nonresponse bias may be present. Although survey responses were anonymous, all data were retrospective and self-reported, such that social desirability and recall biases may also be present. The data have been aggregated, so it is impossible to draw any conclusions about specific combinations of deficiencies. In addition, the year in which deficiencies became apparent was not addressed, and we therefore cannot comment on the use of any particular deficiencies as an early warning system. Finally, the survey asked respondents to reflect on a relatively long duration (5 years). It is likely that some respondents had not been the program director for the entire reported period.

Conclusion

This study sheds important light on remediation in the context of the competencies. It will be critical for future studies to investigate the impact of the competencies across multiple specialties, addressing questions such as which methods of remediation are most efficacious for which competencies.