Implementing the Promotion in Place Model of Competency-Based Time-Variable Graduate Medical Education in Pathology

ABSTRACT

Background Promotion in Place (PIP) is a competency-based time-variable graduate medical education innovation model developed at Mass General Brigham and not previously tested for feasibility, acceptability, or outcomes.

Objective To assess early PIP outcomes in the Massachusetts General Hospital (MGH) pathology residency program.

Methods Approved by the American Board of Pathology, PIP was launched in 2021. The Clinical Competency Committee developed and implemented explicit graduation criteria. Trainees meeting criteria who chose early graduation became credentialed, board-eligible junior attendings at MGH in a period of “sheltered independence” and were assessed using the Focused Professional Practice Evaluation. PIP sheltered independence (PIP-SI) participation rates, time, and activities were followed. We also assessed board pass rates, Milestones, and patient safety reports for all residents. We measured additional participant time and resources.

Results Over 4 years, 24 of 30 (80%) residents qualified for PIP-SI, 17 of 24 (71%) accepted, and 7 of 24 (29%) declined. Eleven of 17 (65%) had a period of sheltered independence as junior attendings (median 9 weeks [range 5-24 weeks]) in various anatomic and clinical pathology roles. Of 6 eligible residents not participating, 4 took a leave of absence, and 2 experienced licensing or visa delays. All residents passed their board examinations on the initial attempt and none had concerns identified through patient safety reports. Time and resource requirements were acceptable to stakeholders.

Conclusions Most residents met criteria for PIP-SI and accepted, demonstrating high acceptability. PIP-SI was feasible with implementation into standard workflows over 4 years. PIP-SI attendings had no adverse outcomes identified.

Introduction

Competency-based medical education is increasingly used as a framework for graduate medical education (GME).¹ Competency-based time-variable (CBTV) GME, where each trainee completes training when they achieve competency,² may result in shorter, standard, or longer training periods. The time-fixed approach to GME results in wide variability in assessment, learning outcomes, and gaps in readiness for unsupervised practice,^3-5 including in pathology residency training.⁶ Individualized training allows for greater attention to the transition to practice,⁵ and there has been a call for greater autonomy in pathology residency training.⁷ CBTV GME models have been developed across specialties including pilots in pediatrics,^8,9 internal medicine,¹⁰ anesthesia,¹¹ family medicine,¹² plastic surgery,¹³ and orthopaedics.¹⁴ Early outcomes from US CBTV GME programs appear favorable,^10,15-17 though more studies are needed to establish acceptability, feasibility, and to demonstrate additional outcomes.

Funded by the American Medical Association (AMA) Reimagining Residency initiative,¹⁸ Promotion in Place (PIP) is a specific model of CBTV GME¹⁹ in which trainees fully graduate when deemed competent—at the standard time, earlier, or later. Those who voluntarily graduate early are “promoted in place” to a period of “sheltered independence,” as fully credentialed “junior” attendings in their training institution until their original standard graduation date. This fully credentialed attending in PIP sheltered independence (PIP-SI) is board eligible, has billing privileges, and practices in settings and experiences they have rotated through as a resident¹⁹ with planning and approval from their former program director (PD) and the service chief. The general PIP model has been described, and customizable templates and guidance for adoption across specialties and institutions are available.¹⁹ We assessed acceptability, feasibility, and early performance outcomes from PIP-SI as implemented by a residency program in pathology, the first specialty to launch PIP.¹⁹

KEY POINTS

What Is Known

There is growing interest in competency-based time-variable (CBTV) training models in graduate medical education, in which training is completed when competencies are achieved, and a paucity of literature describing the feasibility of such programs.

What Is New

The Massachusetts General Hospital pathology residency studied the Promotion in Place (PIP) model, in which qualified residents are offered early graduation followed by a period of “sheltered independence” as credentialed, board-eligible attendings, after rigorous assessment.

Bottom Line

Most of the final-year residents qualified for and accepted early graduation and sheltered independence. Early performance outcomes are reassuring. The program was considered feasible in terms of current resources.

Methods

Setting and Participants

The PIP model¹⁹ was developed at Mass General Brigham (MGB), which includes Massachusetts General Hospital (MGH). In academic year 2020-2021, PIP¹⁹ was implemented in the MGH Pathology program, which has 34 to 38 total residents per year on 4 tracks: straight anatomic pathology or clinical pathology at 3 years, and combined anatomic and clinical pathology or combined anatomic and neuropathology at 4 years. A PIP-SI participant is defined as a junior attending who voluntarily graduates early from residency after demonstrating competency and practices as a fully credentialed, board eligible attending until the originally planned graduation date.

Regulatory Considerations

The American Board of Pathology (ABPath) approved the PIP pilot in the MGH Pathology program, based on review of the program’s competency-based assessment system²⁰ and strong historic board pass rate. Exemptions from “time in program” requirements were granted, allowing individual residents to graduate and advance to PIP-SI¹⁹ up to 1 year prior to their original graduation date. The MGB PIP proposal and model were reviewed through the Accreditation Council for Graduate Medical Education (ACGME) Advancing Innovation in Residency Education (AIRE) process, but the ACGME determined that, with ABPath approval, the AIRE process and approval was not required and that the Pathology Review Committee would be informed and provide program oversight.

Interventions

PIP Planning and Process:

PDs, trainees, and faculty provided key input into the planning process. Faculty and PDs participated in professional development initiatives.¹⁹ The process for implementing PIP is outlined in the Pathology PIP Process Map (online supplementary data 1). All residents were eligible for PIP-SI consideration in their final year, and those who did not qualify continued in the standard program²¹ (Box 1). A PIP Frequently Asked Questions (online supplementary data 2) was iteratively developed during the pilot.

Box 1 Promotion in Place in Pathology: Elements for Success

Preexisting Program Assets

Pathology residency tracks offered:

• Anatomic pathology

• Clinical pathology

• Anatomic and clinical pathology

• Anatomic pathology and neuropathology

Flexible senior year with up to 12 months of elective rotations

Competency-based granular assessment tool (The MGH Passport System mapped to ACGME Milestones)

Graduated responsibility experiences:

• Intraoperative (“frozen section”) senior call, with oversight supervision

• Transfusion medicine senior rotation

Post-residency experiences in pathology:

• ACGME-accredited fellowship (most 12 months duration, some with board certification)

• “Sign-out” fellowship model (nonaccredited) consisting of 1-year experience with attending level responsibilities

• Postdoctoral research training

• Attending physician position (academic or otherwise)

Implemented During PIP Development

Clinical Competency Committee (CCC):

• Increased meeting frequency to 4 times/year with spring and fall dates focused on assessment of qualification for early promotion of rising seniors

• Revised membership to include more faculty with frequent one-on-one contact with trainees

• Enhanced discussion and programmatic recommendations for non-PIP qualified residents to recognize gaps and ensure achievement of competency

• Added 2 CCC “simulation” meetings (spring and fall of 2020) to practice making competency-based promotion decisions

Resident steering committee provided input and feedback on PIP implementation

Developed MGH Pathology PIP explicit criteria for competency-based advancement, graduation, and promotion to “sheltered independence”:

• Residency In-Service Examination score in at least 30th percentile (highly predictive of board pass rate)

• Milestones achievement with no identifiable gaps or deficits (eg, Milestones at or above level 3)

• CCC vote and consensus agreement

• No concerns in the areas of professionalism, teamwork, interpersonal or communication skills

Abbreviations: MGH, Massachusetts General Hospital; ACGME, Accreditation Council for Graduate Medical Education; PIP, Promotion in Place.

Criteria for Competency-Based Advancement:

Program leadership and Clinical Competency Committee (CCC) members developed explicit criteria for competency-based advancement, graduation, and promotion to PIP-SI (Box 1). Several years prior to launching PIP, the program developed and implemented the MGH Passport System of assessment,²⁰ which incorporated granular, level-specific attainments for each rotation, mapped to the ACGME Pathology Milestones.²² In this system, residents self-assess during each rotation, followed by faculty review and agreement or modification of assessment. Milestones were assessed during residency only. Residents are also evaluated by other health professionals. After milestones achievement at or above level 3,²² a Resident In-Service Examination score in at least the 30th percentile, and CCC vote with consensus agreement, program leadership discussed PIP-SI with qualified trainees approximately 6 months in advance of their potential eligibility (Box 1).

Strengthening the CCC:

Augmenting the structure, role, and process of the CCC (Box 1) was essential preparation for CBTV GME and PIP.^19,23 Beginning in fall 2020, the CCC implemented a standard assessment process for all residents (Box 1). Two CCC meetings were held to simulate¹⁹ advancement decisions based on demonstrated competency. All trainees received feedback, and a PD explored interest in PIP-SI participation with qualified trainees (online supplementary data 3). Interested trainees, with PDs and service chiefs, further defined anticipated PIP-SI activities (online supplementary data 4). A glossary of terms was developed (online supplementary data 5).

Implementation

Hospital and payor credentialing processes began 6 months prior to the anticipated period of sheltered independence. PIP-SI participants applied for a full medical license 6 to 9 months in advance of anticipated graduation. Subspecialty clinical service chiefs were consulted to ensure that the timing and anticipated volume of clinical cases would be sufficient to engage the PIP-SI participant. Participants received attending-level malpractice insurance from the MGH Department of Pathology.

In consultation with the clinical service chief, PIP-SI participants chose the timing of PIP-SI (Figure 1). PIP-SI junior attendings underwent the standard new faculty onboarding process and assumed a subspecialty role tailored to their interests and career plans. Participants were expected to maintain some of their prior responsibilities (eg, senior call) during PIP-SI to avoid disrupting clinical care and minimize effects on residents. As attendings, PIP-SI participants did not require supervision, were monitored using Focused Professional Practice Evaluation as required by The Joint Commission,²⁴ and were never paired with former peers in a supervisory role.

Figure 1.

Examples of Pathology Promotion in Place Trajectories Demonstrating Variable Periods of Sheltered Independence as Junior Attendings

Abbreviations: PIP-SI, Promotion in Place with sheltered independence; ACGME, Accreditation Council for Graduate Medical Education.

Note: Three example trajectories (A, B, C) demonstrate options for PIP-SI time relative to the standard graduation date (June 30, thick vertical line).

A: A resident plans to do a “sign-out” surgical pathology fellowship, nonaccredited, which entails independent practice as a pathologist in a specific area. With early promotion, they experience sheltered independence for 3 months transitioning into their fellowship early and becoming able to assume a faculty position earlier than the standard graduation date.

B: A resident who completed an intercalated fellowship before their senior year is expected to graduate and stay as faculty in the department. With PIP, they initiate sheltered independent responsibilities as a laboratory medical director in their focus area 4 months early, easing the transition into a full-time faculty position.

C: A resident plans to do an ACGME-accredited fellowship with a 12-month board requirement (cytopathology, hematopathology, etc). With PIP, they start their fellowship early and utilize 3 months post-fellowship for sheltered independence in which they sign out in their subspecialty area plus/minus other areas based on their projected job requirements.

Feasibility

Stakeholder time and resources associated with planning and implementing PIP are outlined in Box 2. Clinical services needed to have sufficient volume to accommodate PIP-SI participants in their attending roles. During this pilot phase, salary and benefits for those in PIP-SI did not change, maintaining parity with nonparticipants as recommended by trainees involved in developing the model. Also, newly graduated PIP-SI participants were typically given reduced case volume compared to experienced attendings. The AMA grant reimbursed the cost of the full Massachusetts medical licensure (required of all trainees for board eligibility) and the initial ABPath certification examination fee for PIP-SI participants. PDs and 2 staff received a small stipend annually.

Box 2 Promotion in Place in Pathology: Feasibility Components

Time Investments

Program directors (PDs)

• Additional CCC meetings—overall time during meetings increased up to 1.5 times (frequency increased to 4 meetings per year, length per meeting decreased by streamlining processes)

• Meetings with trainees post-CCC for feedback and planning—approximately 1 to 1.5 hours per trainee

• PD ACGME Developing Faculty Competencies in Assessment course (20 CME credits, free during COVID-19 pandemic, one-time investment per PD)

• Resident steering committee meetings—1 hour monthly during program development

• Advocacy and planning with regulators (ABPath, ACGME) and local leadership

• PIP team meetings—meetings with grant team (1 hour weekly, years 1 and 2; then 1 hour every 2 to 4 weeks, years 3 to 5)

Program manager/coordinators

• Additional CCC meetings (see above)

• PIP participant credentialing for hospital privileges—approximately 3 hours per trainee

Clinical service directors and other faculty

• Additional meetings for CCC members (see above)

• Meetings with PIP participants for planning of sheltered independence—approximately 2 hours per participant

• Observation and assessment for FPPE—intermittent observation and availability for consultation over 2 weeks per trainee (balanced by PIP participant performing attending-level responsibilities)

• Faculty development—2-hour seminar (other informational sessions were included in regular faculty meeting times)

Trainees

• Resident steering committee meetings—1-hour monthly meetings during program development

• Earlier full license application

• Credentialing packet completion

• Time in FPPE (observation and assessment by service chiefs over 2 weeks)

Grant team

• Dedicated planning, implementation, evaluation

• PIP team meetings: 1 hour weekly with program directors and program managers for 2 years, then every 2 to 4 weeks years 3 to 5

Monetary Investments

From AMA grant

• Small stipend toward program directors and program managers

• PIP participant reimbursement of full license and board certification application fees (all pathology trainees need a full license to obtain board certification, usually paid by each resident at MGH)

From MGH Department of Pathology

• Participants’ malpractice insurance

• PIP participant salary and benefits—no cost differential during the pilot

Abbreviations: CCC, Clinical Competency Committee; ACGME, Accreditation Council for Graduate Medical Education; CME, continuing medical education; ABPath, American Board of Pathology; PIP, Promotion in Place; FPPE, Focused Professional Practice Evaluation; AMA, American Medical Association; MGH, Massachusetts General Hospital.

Outcomes and Analysis

Acceptability was measured by the number and proportion of trainees who accepted PIP-SI. We measured the time (in weeks) and activities in PIP-SI. For all trainees we examined milestones ratings,²² board pass rates, and patient safety reports. A qualitative stakeholder analysis program evaluation of perceptions of the value of PIP is reported elsewhere.²⁵

The MGB Institutional Review Board determined the evaluation of the PIP pilot met criteria for exemption (45 CFR 46.104(d)(#)).

Results

PIP launched in academic year 2020-2021. In 4 graduating classes, 24 of 30 residents (80%) qualified for PIP-SI (Figure 2) and 17 of 24 (71%) accepted, while 7 of 24 (29%) declined due to specific career plans or a desire to continue the standard program. Of the 17 who accepted, 11 (65%) entered PIP-SI. Of the 6 who did not participate after qualifying for PIP-SI, 4 trainees took a leave of absence from the program and graduated on the standard date, consistent with ACGME guidance during the COVID-19 pandemic,²⁶ which was active at that time. Two were unable to participate due to local board licensing delays or visa-related requirements. No trainees required an extension of training.

Figure 2.

Promotion in Place Participation Rate

Abbreviations: AP, anatomic pathology; CP, clinical pathology; LOA, leave of absence

The time in PIP-SI ranged from 5 to 24 weeks, with a median of 9 weeks. Participants in the last 2 years had a median of 12 weeks (range 7 to 24 weeks) of PIP-SI (online supplementary data 6). Due to credentialing delays in the first 2 years, participants had oversight supervision without independent billing. PIP-SI junior attendings in the latter 2 years were fully credentialed and able to bill as planned. Participant PIP-SI activities included sign-out responsibilities in specific areas of anatomic pathology and laboratory director roles in clinical pathology, the same roles as those of attendings in these areas, but with workload adjusted for their junior status. As shown in Figure 1, some PIP-SI participants decided to use sheltered independence time immediately after graduation and before starting regular faculty positions or fellowships (3 of 11, 27%). Most PIP-SI participants used their sheltered independence time in their subspecialty area after completing an early-commenced fellowship (8 of 11, 73%).

As a group, PIP-SI qualified trainees had higher Milestone ratings in all core competencies compared to nonqualified residents, particularly in Patient Care, Professionalism, and Interpersonal and Communication Skills (Figure 3). All trainees, regardless of qualifying for PIP-SI, passed their board certification examination on the first attempt, and none had a patient safety report filed. Focused Professional Practice Evaluation activities were monitored as part of standard faculty practice,²⁴ and PIP-SI participants progressed as expected.

Figure 3.

Differences in Mean Milestones Scores Between PIP-Qualified (N=20) and Non-Qualified (N=4) Residents

Abbreviations: PIP, Promotion in Place; PGY, postgraduate year; PC, Patient Care; MK, Medical Knowledge; SBP, Systems-Based Practice; PBLI, Practice-Based Learning and Improvement; PROF, Professionalism; ICS, Interpersonal and Communication Skills; SD, pooled standard deviation; ACGME, Accreditation Council for Graduate Medical Education.

Note: Waterfall plot shows differences in ACGME Milestone levels averaged over individuals and competency from the first 3 cohorts. Four time points are displayed for each competency, since the cohort includes residents in 3-year and 4-year programs: spring of PGY-1, spring of PGY-2, spring of the year prior to graduation (FINAL-1), and spring of the year of graduation (FINAL). Positive values (above the 0.00 line) represent higher scores in the PIP-qualified trainees. At the end of training, PIP-qualified trainees demonstrated higher scores in all competencies, most notably in Patient Care, Professionalism, and Interpersonal and Communication Skills.

Discussion

In this first PIP pilot,¹⁹ in a large US pathology residency program over 4 years, most residents met criteria for PIP, and most voluntarily accepted early graduation and a period of PIP-SI as board-eligible, credentialed junior attendings. There were similar assessment outcomes for PIP-SI participants and nonparticipants, and the program is continuing with support and oversight from departmental leaders, the ABPath, and the ACGME Review Committee for Pathology.

Successful implementation of the intervention required explicit competency-based graduation criteria to determine resident qualification for PIP-SI (Box 1), new CCC processes,^19,23 development of customizable and adaptable templates and guides, engagement with national and local stakeholders, and planning with multiple stakeholders (Box 2). The pilot took time to establish processes (eg, credentialing and payors), to advance residents to PIP-SI, and to gain acceptance by residents and faculty. The varied interests and focus areas in a richly subspecialized discipline and department allowed space for PIP-SI without substantial disruptions to department operations (eg, faculty/resident schedules).

When grant funding ends, the program will continue with residents resuming responsibility for specialty-specific license and board fees. Given the feasibility and high acceptability of this initiative, the PIP-SI model may fit other programs and specialties. The enhanced CCC processes, including more frequent meetings with increased actionable feedback for all residents, may facilitate early recognition of individual needs and higher overall competency achievement. These potential benefits are appealing amid evidence that the transition from GME to independent practice needs improvement in many specialties.^3,4,6 Programs seeking to implement the PIP model¹⁹ will need to obtain relevant exemptions from their specialty boards, pursue relevant local regulatory approvals,¹⁹ and achieve stakeholder buy-in.

The pilot took place in a large, well-resourced pathology program, which limits generalizing the findings to other specialties and settings. In addition, this pilot occurred in part during the COVID-19 pandemic, which may have had unknown effects on acceptability and feasibility. The outcomes are preliminary and limited by the small comparator group—those not qualified or not participating in PIP-SI; an ideal comparison group may be junior faculty who graduated from programs without the option of early graduation. Additional performance outcomes, such as practice metrics after completing PIP-SI, would be of interest.

Next steps may include examining obstacles and enablers for implementing PIP in other specialties and settings, including those with key workforce shortages. As the MGH pathology residency PIP program is continuing, additional outcomes with new cohorts should provide new evidence about this innovative approach.

Conclusions

With considerable planning and stakeholder involvement, PIP was implemented in a large US pathology residency program, and the majority of residents qualified for and accepted PIP-SI. Assessment of patient safety reports and board certification rates over 4 years of PIP were the same for PIP-SI participants and nonparticipants. The program was found to be feasible with current resources and is continuing.

Editor’s Note

The online supplementary data contains further data and guides from the study.

Author Notes

* Denotes co-first authors.

** Denotes co-last authors.

Funding: The Promotion in Place project is supported by an American Medical Association Reimagining Residency ChangeMedEd Grant.

Conflict of interest: The authors declare they have no competing interests.

Disclaimer: The content of this manuscript reflects the views of the authors and does not purport to reflect the views of the American Medical Association or any member of the Accelerating Change in Medical Education Consortium.