Development and Evaluation of a Direct Care Hospitalist Service Internal Medicine Sub-internship Rotation

Abstract

Background

Direct Care Hospitalist Services (DCHS) can increase internal medicine (IM) sub-internship rotation availability while providing hospitalists additional teaching opportunities.

Aim

Implement and evaluate a DCHS sub-internship.

Setting

Urban Academic Medical School.

Participants

IM sub-interns, hospitalists.

Program Description

One to two sub-interns were paired with three hospitalists on 3 weeks of day service and five nights in an apprenticeship model. Sub-interns admitted and followed patients on days and cross-covered and admitted on nights.

Program Evaluation

DCHS sub-intern rotation satisfaction and skills preparedness were surveyed over 2 years. Sub-interns rotating on resident-covered service (RCS) were surveyed in year 2, and results compared to DCHS. Hospitalists were surveyed year 1 to rate satisfaction. Year 2 DCHS cross-cover paging data was tabulated to evaluate clinical content. DCHS and RCS sub-interns rated satisfaction and preparedness similarly. DCHS sub-interns rated time management (3.86 vs 4.33, p = 0.19) and calling consults (4.4 vs 4.8, p = 0.56) lower, but cross-cover higher (4.14 to 3.67, p = 0.34) than RCS. DCHS sub-interns averaged 39.4 (SD 4.1) nightly cross-cover pages with most related to acute symptoms (46%). Hospitalists were highly satisfied with their rotation experience.

Discussion

Sub-interns were highly satisfied with DCHS sub-internship. Future work will target gaps in preparedness for urgent patient care issues.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11606-024-08878-5.

INTRODUCTION

Internal medicine (IM) sub-internships, or acting internships, transition students from learning to practicing roles.¹ Sub-internships arose during World War II intern shortages,² and are integral to training.^3–5 Internship represents increased responsibility and workload amplified by medical school training gaps.^6,7 The sub-internship remains the core rotation to master entrustable professional activities⁸ and transition from undergraduate medical education (UME) to graduate medical education (GME).^9,10 Sub-internships can gauge a residency applicant’s ability to perform intern year. Pass/fail standardized testing^11,12 has increased the importance of sub-internship grades and narrative evaluations, particularly in residency applications.¹³

Direct Care Hospitalist Services (DCHS) teaching services, pairing learners with hospitalists, are increasingly used for UME rotations, but descriptions are limited.¹⁴ DCHS may offer educational advantages, including one-on-one time with hospitalists, enhanced direct observation, and increased exposure to hospital operations and systems of care.¹⁴ Hospital medicine (HM) is a rapidly growing field with many hospitalists interested in medical education,¹⁵ but time available to attend on resident-covered services (RCS) has remained static.¹⁶ DCHS teaching services provide unique educational opportunities for hospitalists.

Nationally, increasing student class size strains clerkship resources.¹⁷ Historically, University of Chicago Pritzker School of Medicine (PSOM) IM sub-internships occurred on ward RCS, but like other institutions, early fourth-year sub-internship demand outstripped capacity. In response, we created a DCHS sub-internship. We describe the development, implementation, and evaluation of the first 2 years of this sub-internship.

SETTINGS AND PARTICIPANTS

PSOM has approximately 88 students per class rotating at either the University of Chicago Urban Medical Center or an affiliated community hospital. PSOM requires month-long sub-internships. Excluding visiting students, 25–40 students rotate on IM sub-internships annually. The University of Chicago Section of HM has 74 physicians and eight advanced practice providers staffing fifteen DCHS, two RCS, and seven off-hours admitting and night services at the urban medical center.

PROGRAM DESCRIPTION

The DCHS sub-internship pilot was implemented in 2021–2022 collaboratively with UME leaders, HM clinical directors, and IM residency leadership using Clerkship Directors of Internal Medicine (CDIM) IM sub-internship curricular objectives.⁴ Example rotations were solicited from the CDIM listserv. A multi-disciplinary working group discussed priorities in designing an optimal rotation for all stakeholders.

We prioritized exposure to cross-cover and admissions with standard day-to-day ward care. The month-long rotation incorporated two blocks: (1) day service for 3 weeks; and (2) four to five night shifts. Each rotation was staffed by three hospitalists and hosted one to two sub-interns. Hospitalists rotated on 7-day blocks for day service and one to two nights. The daily team census of 11 patients was divided between sub-interns and hospitalists. Hospitalists and sub-interns worked side-by-side throughout shifts. A comparison of DCHS to RCS structure is provided in Appendix 1.

Sub-interns received new day-service patients in two ways: (1) sub-intern admissions from 10 a.m. to 4 p.m.; and (2) off-hours provider admissions distributed to sub-interns mimicking GME “short-call” or drip system models. On nights, each sub-intern admitted two patients and cross-covered approximately 16 patients.

The DCHS sub-internship entered the PSOM lottery system in the academic year 2021–2022. Students ranked sub-internship rotation and month preferences prior to random assignment. Hospitalists were recruited and scheduled based on interest, teaching experience, and performance.

Prior to the rotation, sub-interns and hospitalists received orientation emails. The course director (E.M.H.) oriented sub-interns to day and night blocks prior to clinical duties. Feedback was solicited from sub-interns and hospitalists at multiple points.

During year 1, six sub-interns rotated on DCHS sub-internships. In year 2, nine sub-interns rotated on DCHS and ten on RCS. Demographic data for representative sub-interns are presented in Appendix 2.

PROGRAM EVALUATION

A multi-phase evaluation utilized anonymous surveys (Appendix 3) rating satisfaction and preparedness with options for free-text comments, and audits of clinical exposure (Fig. 1). During year 1, we surveyed hospitalists and sub-interns at the conclusion of the rotation. Both rated satisfaction on daily patient census, number of admissions, and cross-cover on a 3-point scale. Sub-interns used 5-point Likert-type scales to self-report readiness for clinical skills of internship, satisfaction with teaching and supervision, and comfort requesting a letter of recommendation (LOR). RCS sub-interns were not surveyed in year 1 due to ongoing curriculum development. In year 2, we administered the post-rotation survey to DCHS and traditional RCS sub-interns and compared responses. One author (S.K.M.) with no role in supervising sub-interns sent survey emails with two reminders.

Figure 1.

Timeline of Direct Care Hospitalist Service (DCHS) sub-internship rotation evaluation, University of Chicago Medicine, 2020–2022.

Knowing that students rotating on DCHS sub-internships rather than RCS could impact senior IM resident rotation experience, we concurrently surveyed postgraduate year (PGY) 2 and PGY 3 IM residents in December 2021, midway through the pilot roll-out. Residents were asked to rate DCHS sub-internships as favorable, unfavorable, or neutral, and rated their interest and confidence teaching sub-interns on 5-point Likert-type scales. Attendings and residents were only surveyed in year 1 as part of pilot development assessing initial reaction, feasibility, and unintended consequences of the rotation on resident experience.

In response to year 1 feedback highlighting night shift experiences, we retrospectively reviewed logged cross-cover DCHS sub-intern pages in year 2. Pages were manually reviewed and tabulated into four categories: acute symptoms/vital sign changes, lab abnormalities, routine intern tasks, and communication requests. RCS pages were unavailable and not reviewable through the paging log.

Fisher’s exact test was used to compare proportions for categorical variables, and Likert-scale median ratings were compared using the Wilcoxon signed rank sum test via STATA SE 18 (College Station, TX) with p < 0.05 used for statistical significance.

The University of Chicago Institutional Review Board (IRB) granted an educational exemption for survey evaluation. Review of cross-cover pages received a quality improvement determination and exemption from IRB review.

Year 1 Evaluation

Sub-intern Rotation Satisfaction and Preparedness for Duties of Intern Year

All six year 1 sub-interns completed the post-rotation survey (Table 1). All sub-interns judged the daily patient census and the number of admissions as “Just Right.” One student rated the number of cross-cover patients exposed to as “Too Few” (1/6, 16.7%). Sub-interns felt well-prepared for most clinical skills of internship (Table 1); the lowest-rated skills were cross-cover and recognizing patients requiring emergent care. They were highly satisfied with teaching, supervision, feedback, role modeling, and team camaraderie (Table 1). Written feedback highlighted their preparation for internship: “It was a huge opportunity to work so closely with attendings and get their moment-by-moment teaching. I feel significantly more prepared for intern year now as I also received a lot of independence to put in orders, come up with plans, and write discharge summaries.” Students appreciated the unique relationships cultivated with attendings: “I can’t emphasize enough how special it was to work so closely with attendings every day, observe their strengths, and try to emulate them.”

Table 1.

Comparison of Sub-intern Reported Experiences Between DHCS and RCS Team Sub-internship Students, University of Chicago, Academic Year 2022–2023

	2021–2022		2022–2023
	DHCS team (n = 6)		DHCS team (n = 7)		RCS team (n = 6)		p value
Perception of no. of patients, n (%)	“Just right”	“Too few”	“Just right”	“Too few”	“Just right”	“Too few”	DHCS vs RCS1
Personally admitted	6 (100)	0 (0)	6 (85.7)	1 (14.3)	6 (100)	0 (100)	1.0
Off-hours admission	6 (100)	0 (0)	7 (100)	0 (0)	5 (83.3)	1 (16.7)	0.46
Cared for on a daily basis	6 (100)	0 (0)	7 (100)	0 (0)	5 (83.3)	1 (16.7)	0.46
Cared for on cross-cover	5 (83.3)	1 (16.7)	7 (100)	0 (0)	5 (83.3)	1 (16.7)	0.46
How prepared do you feel for the following aspects of intern year?2							DHCS vs RCS3
Mean (standard deviation [SD]), median (range)
Select, interpret diagnostic tests	4.00 (0.0), 4 (4-4)		4.43 (0.53), 4 (4-5)		4.5 (0.55), 4.5 (4-5)		0.80
Propose management plan	4.17 (0.37), 4 (4-5)		4.43 (0.53), 4 (4-5)		4.33 (0.52), 4 (4-5)		0.74
Recognize a patient requiring emergent care	3.50 (0.5), 3.5 (3-4)		4.14 (0.69), 4 (3-5)		4.17 (0.75), 4 (3-5)		0.94
Know when to ask for help	4.33 (0.75), 4.5 (3-5)		4.71 (0.49), 5 (4-5)		4.67 (0.52), 5 (4-5)		0.86
Perform cross-cover	3.50 (0.96), 3.5 (2-5)		4.14 (0.38), 4 (4-5)		3.67 (1.03), 4 (2-5)		0.34
Respond to pages	4.00 (0.59), 4 (3-5)		4.43 (0.53), 4 (4-5)		4.33 (0.52), 4 (4-5)		0.74
Update patients on plan of care	4.17 (0.37), 4 (4-5)		4.71 (0.49), 5 (4-5)		4.83 (0.41), 5 (4-5)		0.63
Enter orders	4.50 (0.50), 4.5 (4-5)		4.57 (0.53), 5 (4-5)		4.33 (0.52), 4 (4-5)		0.41
Call a consult	4.50 (0.50), 4.5 (2-5)		4.43 (1.13), 5 (2-5)		4.83 (0.41), 5, (4-5)		0.56
Receive an off-hours admission	4.43 (1.13), 5 (2-5)		4.43 (1.13), 5 (2-5)		4.67 (0.52), 5 (4-5)		1.00
Time management	4.33 (0.47), 4 (4-5)		3.86 (0.69), 4 (3-5)		4.33 (0.52), 4 (4-5)		0.19
Write a discharge summary	4.50 (0.50), 4.5 (4-5)		4.57 (0.53), 5 (4-5)		4.67 (0.52), 5 (4-5)		0.74
Complete a written signout	4.17 (0.69), 4 (3-5)		4.71 (0.49), 5 (4-5)		4.5 (0.55), 4.5 (4-5)		0.45
Hand off service to covering provider	4.17 (0.69), 4 (3-5)		4.43 (0.53), 4 (4-5)		4.33 (0.52), 4 (4-5)		0.74
How satisfied were you with the following elements of your sub-internship? 2							DHCS vs RCS3
Mean (SD), median (range)
Teaching quality by attendings?	4.33 (0.47), 4 (4-5)		4.86 (0.38), 5 (4-5)		4.67 (0.82), 5 (3-5)		0.82
Quality of teaching by residents?	-		-		4.67 (0.82), 5 (3-5)		-
Supervision by attendings?	4.33 (0.47), 4 (4-5)		4.63 (0.38), 5 (4-5)		4.83 (0.41), 5 (4-5)		0.91
Supervision by residents?	-		-		4.83 (0.41), 5 (4-5)		-
Feedback from attendings?	4.33 (0.47), 4 (4-5)		5 (0), 5 (5-5)		4.83 (0.41), 5 (4-5)		0.28
Feedback from residents?	-		-		4.83 (0.41), 5 (4-5)		-
Role-modeling by attendings?	4.33 (0.47), 4 (4-5)		4.71 (0.76), 5 (3-5)		4.83 (0.41), 5 (4-5)		1.00
Role-modeling by residents?	-		-		4.67 (0.82), 5 (3-5)		-
Team camaraderie amongst you and attendings?	-		4.86 (0.38), 5 (4-5)		4.83 (0.41), 5 (4-5)		0.91
Team camaraderie amongst you and residents?			-		4.83 (0.41), 5 (4-5)		-
Ability to request a letter of recommendation for residency (if applicable)?			4.80 (0.45), 4 (4-5) (n=5)		4.83 (0.41), 5 (4-5)		0.89

¹Fisher’s exact test

²Responses were on a 5-point Likert scale

³Wilcoxon rank sum

Hospitalist Rotation Satisfaction

Eighty percent (12/15) of hospitalists completed post-rotation surveys. Most perceived sub-intern patient exposure as “Just Right” (11/12, 91.7%), with 2/12 (16.7%) noting “Too Few” cross-cover patients. Hospitalists rated satisfaction as high, including ability to teach, contributions to career development, and integration of sub-interns into shift workflows (Appendix 3).

Impact of DCHS Sub-internship on Resident Perspectives on Working with Sub-interns

Three-quarters (49/64) of senior IM residents responded to the survey; 24 (49%) were PGY2 and 25 (51%) PGY3 (Appendix 4). Most (35/49, 71%) had experience supervising sub-interns. Regarding DCHS sub-internships, 14/35 (40%) viewed them favorably, 7/35 (20%) unfavorably, and 14/35 (40%) were neutral or unsure. In explanatory comments, residents emphasized the primacy of student choice in rotation. Residents felt DCHS sub-internships could open career opportunities and strengthen LORs. Most felt positive about teaching opportunities but emphasized challenges with team workload: “I love working with students, I just wish we had a service structure that would allow for more teaching.” Others highlighted the benefits of exposure to residency tasks on RCS sub-internships: “I think sub-interns benefit from seeing what their role as a resident will look like.”

Year 2 Evaluation

Comparison of DCHS and RCS Sub-intern Post-rotation Surveys

In year 2, both DCHS and RCS sub-interns were surveyed, and responses compared (Table 1). Overall response rate was 13/19 (68%), 7/9 (77.8%) from DCHS and 6/10 (60%) from RCS. Statistical analysis did not yield significant differences in satisfaction and preparedness ratings; however, trends were seen (Table 1). DCHS sub-interns felt less well-prepared to manage time and organizational skills and less prepared to call consults than RCS sub-interns, while DCHS sub-interns felt more prepared for cross-cover (Table 1).

Evaluation of Cross-cover Clinical Exposure

In reviewing 315 logged DCHS sub-intern cross-cover pages from May to October 2022, DCHS sub-interns received 39.4 (SD 4.1) mean pages per sub-intern (Appendix 5). Acute symptoms or vital sign changes (46%) were most common, followed by routine intern tasks (26%), communication requests (18%), and lab abnormalities (12%). Half of the sub-interns were paged to evaluate critically ill rapid response patients.

DISCUSSION

We describe the successful development and implementation of a DCHS IM sub-internship with high satisfaction and preparedness for clinical practice similar to RCS rotations. While not statistically significant with small sample sizes, differences in perceived time management and organizational skills were observed, potentially stemming from inherent DCHS and RCS workflow differences or perceptions while under direct supervision.

Incorporating dedicated shifts for nights and cross-cover was key for the DCHS sub-internship. Following duty hours regulations, student night experiences waned as documented by annual CDIM surveys: 40.7% of sub-internships included nights in 2014 and only 12–14% of school required overnight call or night float in 2017.^18,19 Paging data from DCHS night shifts showed most involved acute symptoms and changes in vital signs, requiring critical thinking and decision-making. This represented strong exposure to the overnight intern role and real-world experience for sub-interns, addressing CDIM sub-internship curricular goals of recognizing sick patients and knowing when to seek help.¹

Despite this experience, DCHS sub-interns felt least prepared for cross-cover and calling consultations, highlighting the need for practical curricular experiences. The original number of DCHS cross-cover patients (6) was deemed insufficient, requiring an increase to 10–16 patients. A simulation curriculum is in development, including triaging multiple simultaneous pages and case-based approaches to emergencies UME consultation communication training is known to be variable.²⁰ Observed differences in consult preparation in our pilot may be due to RCS sub-interns rotating with residents who have undergone a standardized institution-wide consultation training prior, which we are incorporating into future DCHS sub-intern orientation.²¹

Resident perspectives highlighted tradeoffs, noting they may miss out on unique teaching experiences with the development of this rotation. While sub-interns rated satisfaction with residents and hospitalists highly, there are substantial differences in supervisor selection between DCHS and RCS. DCHS hospitalists are intentionally scheduled and receive rotation-based faculty development, whereas RCS sub-interns are placed on prespecified teams with attending and resident variability.^22,23 This can translate into differing sub-intern experiences. For example, in RCS rotation, residents often complete orders and answer pages for efficiency. Sub-interns value autonomy²⁴, and DCHS rotation feedback highlighted the independence hospitalists fostered in such tasks.

We encountered challenges in implementing the DCHS sub-internship. We increased off-hours admissions from a cycled call day to daily to maintain sub-intern census. Sub-interns rotate monthly and hospitalists weekly requiring adjustment of hospitalist start dates for continuity. Hospitalists developed a skillset of dividing time between supervising sub-interns and caring for their direct care patients. Most felt this was manageable and used these patients for teaching. The night requirement was prohibitive for some hospitalists due to schedule preferences, and a dedicated group of teaching nocturnists is being developed. Systems issues arose including pager assignments, optimizing patient distribution to maintain continuity, and obtaining student electronic health record shared list access and permissions to reconcile medications and admission navigators.

Our rotation evaluation is limited to one institution with small sample sizes. Survey data is self-reported preparedness for clinical skills and does not contain intern-level performance outcomes. Surveys were completed at rotation end and preparedness or recall may decay over time.

Despite these limitations, the DCHS sub-internship has been successful, and its impact has grown rapidly. The improvement in satisfaction from year 1 to year 2 suggests modifications made from lessons learned enhanced the rotation, and iterative improvement continues based on year 2 results and ongoing feedback. In 2023–2024, 18 of 40 PSOM students choosing IM sub-internships selected the DCHS rotation, the most of available sub-internships. Its success has led to additional local opportunities to utilize DCHS for clinical student education, including additional DCHS sub-internship teams, developing a fourth-year HM elective, and incorporating DCHS into the core IM clerkship. The next steps include evaluating the DCHS sub-internship via objective or evaluative performance data, including data from internship performance.

DCHS sub-internships are a promising solution to address the need for meaningful sub-internship experiences to meet demand. Our experience added capacity for 24 IM sub-interns with one team. Scaling to other institutions will vary depending on local DCHS structures and preceptor availability. Exploration of hybrid models including DCHS hosting residents and sub-interns may further increase capacity. Our model was strictly in IM but may apply to other specialties utilizing direct care providers to expand sub-internship capacity.

Our work adds to the literature describing DCHS rotations. We developed our model using informal advice and lived experience from other institutions to fit our needs, focusing on the preferred day and night balance to provide sufficient clinical experience and continuity. We recognize institutions will differ substantially in many features including patient census, scheduling needs, and admitting structures. Comprehensive work reviewing and developing best practices for these rotations is needed to guide DCHS development and evaluation to best meet student needs.

Supplementary Information

Below is the link to the electronic supplementary material.

Author Contribution:

All authors listed have contributed sufficiently to the project to be included as authors, and all those who are qualified to be authors are listed in the author byline.

Declarations:

Conflict of Interest:

The authors declare that they do not have a conflict of interest.