Redefining the Acting Internship in the Era of Entrustment: One Institution’s Approach to Reforming the Acting Internship

Abstract

Creating a Core Entrustable Professional Activities (Core EPA) curriculum requires a longitudinal approach. Current curricular efforts have focused primarily on the pre-clerkship and clerkship phases of training; however, the role of the Acting Internship (AI) has not been explored. The AI experience offers opportunities for students to have enhanced clinical responsibility, demonstrate proficiency, and allows for assessment of Core EPAs that are beyond the focus of clerkships. We share our experience developing an interdepartmental AI experience designed to assess designated Core EPAs and highlight tensions that should be considered when incorporating an AI experience into a longitudinal Core EPA-oriented curriculum.

Background

Medical schools are moving beyond traditional Flexner time-based education models to competency-based medical education (CBME) to ensure students possess the knowledge, skills, and attitudes to practice medicine effectively on their first day as an intern [1, 2]. The Core Entrustable Professional Activities for Entering Residency (Core EPAs), a set of 13 activities deemed necessary for starting interns to be able to perform regardless of medical specialty, were developed in collaboration with the Association of American Medical Colleges (AAMC) to close the gap between medical school and residency [3, 4]. The optimal method for developing a Core EPA-oriented curriculum requires a longitudinal and systems-based approach, beginning with the pre-clinical curriculum and continuing to graduation [5, 6].

While in continual evolution, the current structure of most medical schools involves a 1–2 year pre-clinical phase followed by a 2–3 year clinical phase. The core clinical component typically includes a 12-month period of required clinical clerkships (e.g., Internal Medicine, Surgery, and Pediatrics), followed by a subsequent year of required Acting/Sub-Internships (AI) and elective opportunities [7, 8].

To date, much of the literature regarding the development of a Core EPA-oriented curriculum has involved the clerkship phase of training [9, 10], and to a lesser extent, the pre-clinical curriculum [11–13]. In contrast, the role of the fourth year and the AI experience in particular has not been well described. In this commentary, we discuss how the AI may fit into the larger concept of a longitudinal Core EPA-oriented curriculum and how it must be reconfigured to offer a core component of a learner’s longitudinal progression toward entrustment of each Core EPA. As an illustrative example, we share our experience developing an interdepartmental AI experience designed to assess several Core EPAs and highlight some of the tensions that can arise when considering how the AI experience fits into the larger picture of formulating entrustment decisions.

The Acting Internship in the Context of a Longitudinal Core EPA-Oriented Curriculum

In a longitudinal, Core EPA-oriented curriculum, each phase of the curriculum should contribute to the training and assessment data required to formulate summative entrustment decisions [14]. While the pre-clinical phase has been primarily focused on medical knowledge acquisition, it may also serve as an optimal time to incorporate foundational concepts underlying each Core EPA (e.g., principles of screening for disease provides a foundation for recommending diagnostic tests) while simultaneously providing an opportunity to assess learners’ trustworthiness [12]. Completion of the pre-clinical phase may be guided by satisfactory information gathering, communication skills, and identification of key resources required to make medical decisions [11].

The clerkship phase is often students’ first introduction to assuming responsibility in clinical medicine. Thus, it serves as a substantial transition period as students become acclimated to their new responsibilities in this environment [15]. This phase therefore provides a natural opportunity for students to practice early clinical skills associated with the Core EPAs, primarily related to history and physical examination [9, 10]. Ideally, the clinically-based third-year should conclude with a gateway summative assessment for these lower complexity EPAs. As others have suggested, it may allow for rendering an entrustment decision that students can be trusted to perform EPAs such as history and physical with indirect supervision [16].

The final year of medical school, and the AI in particular, provides an opportunity for students to refine the skills they have learned in the clerkship phase, practice new and more complex skills, and begin preparation for their chosen career [7, 16–18]. Several authors have suggested that assessment of more complex skills and specific EPAs (e.g., EPA #8—Give or receive a patient handover to transition care responsibility) may be outside the scope of the clerkship phase [9, 10]. Therefore, the AI rotation offers an opportunity to develop and assess more complex EPAs, such as patient handovers and recognizing “sick” patients, while simultaneously offering confirmation of students’ abilities to perform lower complexity EPAs in a variety of settings [19, 20].

Restructuring the AI Experience

The AI was originally designed for practical purposes during World War II in an effort to counter shortages of physicians and improve the transition from clinical clerk to intern [21]. The same principle holds true today; graduating students view the AI as the most valuable experience of their fourth year as they prepare for residency [22–24]. Educational leaders in Pediatrics, Surgery, and Internal Medicine have identified national objectives for the AI experience [19, 25–27]. However, there has been limited discussion of how the AIs may incorporate into the CBME models, entrustment decisions, and how to structure unifying curricula and assessment methods [28, 29].

In order to reform the AI rotations to discern, develop, and assess key clinical functions relevant to the Core EPAs, it is important to standardize the curricular approach and expectations across an institution and all medical specialties. Some authors have suggested a specialty-specific approach to curricular reform at the AI level [19]. For example, in internal medicine, educational leaders have proposed specific recommendations for developing a fourth year, EPA-oriented curriculum for medical students who plan to enter into internal medicine training programs. While such a departmental approach may be valuable in ultimately transitioning from an undifferentiated student to a post-graduate trainee, the recognition that specific EPAs may be beyond the scope of all clerkships suggest that an interdepartmental approach may avoid redundancy and may be more in line with the notion that the Core EPAs are intended to be agnostic to specialty choice.

An Interdepartmental Acting Internship Experience: One Institution’s Experience

Prior to 2016, the Virginia Commonwealth University School of Medicine (VCU-SOM) required all students complete two departmentally-based AIs. The curriculum for each respective AI was left to the discretion of the departmental leadership team. In recognition of the longitudinal nature of EPAs and highly variable experience between AI rotations, we sought to develop an interdepartmental AI experience. The purpose of this experience was to centralize the administration, direction, and core curriculum for all AIs while maintaining the specialty-specific focus of the clinical experience among the existing AIs.

To begin developing this experience, we first centralized the administration and direction of the AI to the Dean’s office rather than departmentally-based rotations. In addition to centralizing the curriculum, we recruited two individuals from different specialties to serve as interdepartmental AI directors. The co-directors created uniform objectives (organized by Core EPAs (see Table 1)) and assessment methods applicable across all clinical venues. This was instrumental in setting the foundation to standardize the AI experience.

Table 1.

VCU School of Medicine Universal Acting Internship Objectives

By the end of the Acting Internship rotation, fourth-year medical students will be able to:
Relevant Core EPA	Corresponding AI Objectives
EPA 1: Gather a history and perform a physical examination	1. Demonstrate ability to independently and efficiently perform both a comprehensive and focused history and physical 2. Display proficiency in performing physical examinations and identifying abnormal findings
EPA 2: Prioritize a differential diagnosis following a clinical encounter	1. Independently develop and prioritize differential diagnoses 2. Show ability to explain rationale behind differential 3. Provide evidence-based support for the differential
EPA 3: Recommend and interpret common diagnostic and screening tests	1. Identify appropriate diagnostic and screening tests to substantiate diagnoses 2. Incorporate results of diagnostic tests to the clinical presentation and management plan
EPA 4: Enter and discuss orders and prescriptions	1. Identify and complete core admission orders within the electronic medical record (EMR) or propose and/or enter orders alongside the supervising physician as allowed by the facility in which they are training 2. Prioritize and complete daily patient-care orders via EMR
EPA 5: Document a clinical encounter in the patient record	1. Write comprehensive admission history and physicals with documented decision-making 2. Compose succinct daily progress notes (including transfer, significant event, and procedure notes) with independently formulated plan 3. Explain medical decision-making thought process/rationale within the medical record 4. Comprehend essential components of a discharge summary and demonstrate proficiency completing discharge summaries independently
EPA 6: Provide an oral presentation of a clinical encounter	1. Perform accurate, concise, hypothesis-driven clinical presentations on rounds 2. Communicate effectively with patients and family members in an understandable manner, avoiding medical jargon
EPA 7: Form clinical questions and retrieve evidence to advance patient care	1. Utilize the literature to help guide decision-making process and diagnosis 2. Recognize the utility and limitations of protocols and/or clinical guidelines
EPA 8: Give or receive a patient handover to transition care responsibility	1. Describe essential components to providing effective and safe written patient handoffs 2. Perform verbal and written patient handoffs utilizing I-PASS as a template 3. Directly facilitate longitudinal transitions of care (inpatient to outpatient/facility and outpatient to inpatient)
EPA 9: Collaborate as a member of an interprofessional team	1. Communicate directly and effectively with nurses, patients and healthcare team members including social workers and care coordination 2. Recognize roles of each team member within an interprofessional team 3. Explain when consults are needed and demonstrate how to effectively call consults
EPA 10: Recognize a patient requiring urgent or emergent care and initiate evaluation and management	1. Recognize patients with unstable vital signs 2. Identify patients that require immediate attention by a supervising physician (upper level resident and/or attending physician) 3. Assist with implementing initial steps to manage a patient with an urgent condition or change in clinical status 4. Recognize when and how to activate a rapid response and code blue 5. Appropriately triage patients on cross-cover and resolve common cross-cover issues 6. Realize self-limitations and when to seek additional assistance and escalate care
EPA 11: Obtain informed consent for tests and/or procedures EPA 12: Perform general procedures of a physician	1. Identify indications for common procedures 2. Recite risks, benefits, and indications of these procedures to patients and families in an easily understood fashion in order to obtain informed consent under supervision of the upper level resident and/or attending 3. Assist and perform appropriate procedures under supervision
EPA 13: Identify system failures and contribute to a culture of safety and improvement	1. Recognize situations that could lead to adverse-events in patient care 2. Articulate steps to correct potential patient safety issues 3. Demonstrate ability and rationale for reporting a patient safety issue or event
*Identify potential complications of care or disease-related complications	1. Anticipate potential pitfalls of patient care decisions and its impact on patient outcomes 2. Recall advantages and disadvantages of high-risk treatment options and incorporate this into management plan decisions 3. Identify strategies to minimize risk of complications
*Demonstrate professionalism and maturity in all aspects of patient care	1. Display integrity, compassion, respect, altruism, and empathy when interacting with all members of the health care team including patients and their families 2. Show respect for patient confidentiality 3. Incorporate cultural considerations into patient care plans

*Additional EPAs (beyond those articulated by the AAMC) identified by curriculum development team at VCU

In order to provide additional training in more advanced EPAs, the VCU-SOM AI co-directors implemented a structured curriculum across all AI rotations. The co-directors also conducted an AI orientation session on the first day of each rotation separate from any departmental AI orientations, which are limited in number. The school-wide orientation curriculum addressed certain Core EPAs, including identifying and tailoring admission orders to common patient presentations (EPA #4—Enter and discuss orders and prescriptions), how to give and receive a patient handover (EPA #8—Give or receive a patient handover to transition care responsibility), effectively call a consult (EPA #6—Provide an oral presentation of a clinical encounter and EPA #9—Collaborate as a member of an interprofessional team), and tips to providing cross-coverage (EPA #10—Recognize a patient requiring urgent or emergent care and initiate evaluation and management) [30]. Students were expected to practice these skills during the AI rotation in the clinical setting, during which time they were directly observed and assessed performing these activities. These required direct observations assess the ability of the students to perform patient handoffs, call consults, and place orders in the electronic medical record.

We moved to a criterion-based summative assessment for the AI (see Table 2) (adapted from the Clerkship Directors in Medicine (CDIM) Sub-Internship Fourth-Year Medical Student Performance Criteria) [31] with a curriculum that focuses on developing competencies mapped to those within the Core EPAs and ACGME milestones. This summative assessment is utilized across all AIs and grades are assigned by the VCU-SOM AI co-directors based on these assessments and direct feedback from faculty. These changes helped provide assessment data, which could ultimately be utilized for entrustment decisions without significant additional demand from faculty.

Table 2.

Criterion-based Acting Internship Summative Evaluation (Please CIRCLE the descriptor that best matches the student’s current ability)

Medical knowledge	Demonstrates only basic knowledge of pathophysiology and common treatments plans	Possesses acceptable knowledge of pathophysiology and major treatments. Occasional difficulty applying knowledge to patient care	Demonstrates in-depth knowledge of core pathophysiology and ability to readily apply this to patient care
Procedural knowledge and communication (ICU/surgery)	Unable to consistently demonstrate the knowledge of indications, benefits, or contraindications of procedures	Demonstrates partial knowledge of the indications, contraindications, risks, and benefits of common procedures	Demonstrates understanding of the indications, contraindications, risks, and benefits of common procedures
Clinical reasoning and decision-making	Able to perform complete history and physical exam with appropriate differential diagnoses but has difficulty assimilating new data	Able to perform complete history and physical exam. Emerging skills at selecting and interpreting relevant data. Offers independent suggestions for management	Incorporates relevant physical exam and history into assessment and treatment plan. Reassesses patient and changes care plan based on changes in clinical status
Documentation	Notes are often vague with inability to document rationale and formulate differential and daily plan	Notes contain all required elements with appropriate detail but often not concise. Conveys basic understanding of the assessment and plan	Notes are well-written, explaining the medical-decision making and plan. Easily and reliably assimilates new data into documentation
Ability to recognize acute changes in clinical status	Inconsistently recognizes changes in clinical status and acutely ill patients. At times, able to triage the patient but with limited knowledge of when to escalate care	Able to frequently recognize changes in clinical status and acutely ill patients. Developing skills to determine when to escalate care	Able to consistently recognize changes in clinical status and acutely ill patients. Knows limitations of the clinical scenario and repeatedly demonstrates ability to escalate care when needed
Patient handoffs (verbal and written)	Performs handoffs that are often not organized or succinct. Written handoff often not updated and may exclude essential information	Able to perform written and verbal handoffs that conveys most essential information with few lapses. Often too verbose or brief	Provides succinct, organized verbal and written handoffs; includes anticipatory guidance, follow-up with provisions, and emphasizes patients that are unstable
Professionalism	Demonstrates intermittent issues with respect, compassion, and integrity when dealing with patients and peers. Often does not follow through own responsibilities	Demonstrates basic respect, compassion, and responsibility. Completes all assigned tasks with direction	Demonstrates respect, compassion, and integrity. Demonstrates accountability and dependability in independently carrying out responsibilities
Limitations and initiative	Often does not acknowledge own limitations or take initiative	Able to self-identify some knowledge gaps with prompting. Does not necessarily demonstrate self-initiative in helping team and patients	Takes initiative to identify and address needs of patient and team. Able to identify own knowledge gaps and makes effort to improve gaps and patient care
Response to feedback	Does not frequently seek feedback and often is unaccepting of feedback	Appears to be accepting of feedback and sometimes utilizes it for improvement	Accepting of feedback. Demonstrates improvement reflective of feedback
Interpersonal and communication skills	Demonstrates difficulty effectively explaining medical condition to patient	Usually communicates to patients and family members effectively, though at times uses medical jargon	Communicates with the patients and family members utilizing appropriate terms. Effectively communicates with all members of the health care team
Respect of the patient and their personal/cultural preferences	Frequently unable to identify patient values that may impact patient care	Consistently demonstrates respect for patients, emerging ability to perform patient-centered care	Demonstrates appreciation for the cultural sensitivities of the patient and incorporates this into interactions and health care decision-making
Practice-based learning and improvement/life-long learning	Does not independently seek to advance knowledge on new topics and patient care issues	Seeks information and literature to advance knowledge and decision-making. Intermittent difficulty applying new knowledge or requires prompting	Uses references to access evidence-based medicine to solve clinical problems and advance medical knowledge
System-based practice	Demonstrates basic knowledge of the resources available in the hospital and community that contribute to effective patient care	Possesses basic knowledge of resources available and recognizes value, but has some difficulty utilizing on a regular basis	Utilizes resources available in the hospital and the community to advance patient care
The following questions are only required for Acting Interns on surgical services (including OB-GYN, Gen Surgery, Trauma, and Ortho)
Surgical anatomy	Lacks understanding of relevant surgical anatomy	Possesses basic understanding of surgical anatomy relevant to patient disease	Demonstrates in-depth understanding of surgical anatomy and operative procedures. Able to discuss effect on clinical disease
Principles of perioperative management	Demonstrates difficulty articulating basic principles of perioperative management	Able to identify pertinent medical history that could impact surgical outcomes. Needs assistance identifying patients appropriate for operative intervention	Identifies pertinent medical history and factors that may impact surgical outcomes. Offers independent suggestions for risk modification; identifies patients too high risk for operative intervention
Surgical decision making	Articulates indications for surgery but unable to consistently explain risks and alternatives	Able to discuss indications and alternatives for surgery with minor omissions	Comprehends and independently explains risks, alternatives, and benefits of procedures
Surgical dexterity and wound closure	Often unable to tie basic surgical knot or effectively close basic surgical incision	Demonstrates proficiency in basic knot tying and wound closure with occasional need for assistance	Effectively closes surgical wounds. Independently assists and anticipates surgical steps within procedures

Length of time with AI: (drop down menu of choices: 3 days or less, 4 days–1 week, 1–2 weeks, > 2 weeks)

Evaluator level of training (intern, resident, fellow, attending) —drop down menu of choices

In the following sections, please provide specific feedback to the degree possible:

Formative comments (areas for improvement): free text box

Summative comments (overall impression/assessment of student and their performance): free text box

The AI experience at VCU fits into the bigger picture of our Core EPA-oriented curriculum. Beginning with the Class of 2019, all clerkships introduced mobile friendly, workplace-based assessments (WBA) designed to promote direct observation for the Core EPAs as previously described by ten Cate and colleagues [32]. The specific EPAs selected were identified by clerkship and AI leadership teams based on the opportunities they felt students would encounter in each respective clerkship. As shown in Table 3, the AI experience was the only clinical rotation identified in which students would be afforded the opportunity to receive feedback on direct observation for Core EPAs 8, 10, and 12. At our institution, the AI experience plays a critical role in the curricular strategy for developing an EPA-oriented curriculum.

Table 3.

Core EPAs across Clerkships and Acting Internships

Core EPAsa	Ambulatory	Family Medicine	Internal Medicine	Neurology	Obstetrics/Gynecology	Pediatrics	Psychiatry	Surgery	AIs
1	X	X	X	X	X	X	X	X
2	X	X		X	X	X	X	X
3	X	X	X		X	X		X
4				X			X		X
5	X					X	X	X
6	X	X	X	X	X	X	X	X	X
7	X		X
8									X
9	X		X	X		X		X	X
10									X
11	X						X
12									X
13	X

^aCore EPA #1—Gather a history and perform a physical examination; EPA #2—Prioritize a differential diagnosis following a clinical encounter; EPA #3—Recommend and interpret common diagnostic and screening tests; EPA #4—Enter and discuss orders and prescriptions; EPA #5—Document a clinical encounter in the patient record; EPA #6—Provide an oral presentation of a clinical encounter; EPA #7—Form clinical questions and retrieve evidence to advance patient care; EPA #8—Give or receive a patient handover to transition care responsibility; EPA #9—Collaborate as a member of an interprofessional team; EPA #10—Recognize a patient requiring urgent or emergent care and initiate evaluation and management; EPA #11—Obtain informed consent for tests and/or procedures; EPA #12—Perform general procedures of a physician; EPA #13—Identify system failures and contribute to a culture of safety and improvement

Tensions

Moving AIs to an EPA-oriented rotation with the goal of providing ad hoc entrustment decisions and ultimately data that can be used for summative entrustment decisions may create tensions which need to be addressed. These tensions may be summarized as: the balance between CBME and grades, the ramifications of learners who are deemed not yet entrustable, and availability of assessment instruments.

Balance Between CBME and Grades

CBME is focused on the ability of the student in relationship to a criterion of competency or degree of entrustment and without comparison to other students. It is based on direct observations of the students performing authentic tasks. These competency-based assessments can be meaningful to the students as formative feedback for improvement but also for other stakeholders. As medical schools work to ensure that all students are entrustable to perform the Core EPAs, EPA-based assessments help to measure that goal. Similarly, for residency programs, if EPA judgments are effective, they should have confidence in the ability of the intern. Thus, AI assessments must provide information for entrustment and the degree of intern supervision required. While the workplace-based assessments are providing data regarding student clinical ability, there remains the need to validate these assessments before this data can contribute to high-stakes entrustment decisions.

The importance placed on grades is in direct conflict with competency assessment. Students aim for achieving a certain grade rather than aiming for competency or expert performance. Similarly, residency programs may view grades as a gauge of the student’s overall ability and many still utilize grades as a factor not only in selecting which student applicants to interview but also in ranking students for the Residency Match process [33]. As Pereira et al. recently pointed out, now is the ideal time to move past grades and examinations to direct observation assessments of student ability and competency achievement, with the goal of being able to predict future clinical performance [34].

While the AI experience is still graded, as previously mentioned, we utilize a criterion-based rubric (see Table 2) and also rely on evaluator narrative to assign grades. Ultimately, we hope to move toward a pass-fail system for the AIs based on the RIME model [35].

Ramifications for Learners Deemed “Not yet Entrustable”

The second tension is that medical schools need to consider the ramifications of learners who are not able to demonstrate competence in the Core EPAs. In CBME, ultimately, there must be a hard stop for those graduates who are not competent. Some schools may determine which students should not start an AI, but instead, must remediate the identified EPAs and achieve competency prior to entering an AI. Other schools may instead focus on remediating students during the AI. Either way, it provides an opportunity to coach underperforming students and allow for deliberate practice to address deficiencies and demonstrate proficiency.

Moreover, the fourth year will allow students to pursue additional AI rotations in order to achieve desired competencies if not initially successful. Thus, additional rotations serve as remediation until competency is achieved. While the fourth year allows the opportunity to remediate, it may require a frameshift from the current fourth year structure that includes vacation, travel, and interview blocks. Instead, the focus changes to working with students to become the best interns they can be. All of these challenges require resources for students, faculty and administrative time, faculty development, as well as a robust learning management system.

At our institution, we have developed WBA and coaching structures beginning in the first year of medical school. These initiatives will hopefully identify students in need of further coaching or remediation earlier on than the AI. In addition, our fourth-year policy has changed, and students are now required to have more clinical rotations, which should add further rigor and opportunity for skill practice and refinement prior to graduation.

Assessment Instruments

The final challenge is developing a robust EPA-driven AI assessment system. We need to work to ensure that data collected from direct observations and assessments have validity evidence. It is important to keep in mind the volume of students and that multiple observations are required in different contexts. Another step in demonstrating validity would be to confirm with program directors whether graduated students met these EPA performance levels as interns. Finally, there should be an entrustment committee to provide summative assessment using programmatic assessment.

Next Steps

The AI serves as one of the final checkpoints between undergraduate (UME) and graduate (GME) medical education training programs. As such, there is an opportunity for this experience to serve as a gateway to GME, ideally, in a time-independent, competency-based manner. A competency-based AI would focus on ad hoc entrustment decisions based on clinical responsibilities of the Acting Intern and also providing observational and assessment data for the final summative Core EPA entrustment decisions. The AI would then serve as one of the final steps in providing data to entrustment committees [14] to make decisions regarding a student’s entrustment for each of the Core EPAs. With the reformed AI assessments and direct observations, entrustment committees will have the essential information needed to make entrustment decisions and provide individualized recommendations for continued skill and competency achievement to residency training programs. In doing so, UME programs will be providing a more comprehensive educational handover to ensure the successful transition to the GME setting [36–39].

Conclusion

As we aim to ensure that graduating students are ready for internship through the Core EPAs, the AI provides a robust and needed context for curriculum and assessment. In the process of improving the AI, the changes involve stakeholders at many levels, but these reforms would strengthen the fourth year of medical school and namely the hallmark course of the year, the AI. In doing so, students as well as patients will benefit. The feedback generated from multi-modal direct assessments will help foster student development, competency achievement, and ultimately entrustment to perform core clinical activities with indirect supervision. This is the current goal of our medical school graduates and the ideal standard to provide effective and safe care on the first day of intern year.

Funding Information

Dr. Santen has support from the AMA Accelerating Change in Medical Education Grant.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

For this type of study/manuscript, formal consent is not required.