Expanding Telehealth Competencies in Primary Care: A Longitudinal Interdisciplinary Simulation to Train Internal Medicine Residents in Complex Patient Care

Abstract

Background

Despite increasing use of telehealth, there are limited published curricula training primary care providers in utilizing telehealth to deliver complex interdisciplinary care.

Objective

To describe and evaluate a telehealth curriculum with a longitudinal objective structured clinical examination (OSCE) to improve internal medicine residents' confidence and skills in coordinating complex interdisciplinary primary care via televisits, electronic consultation, and teleconferencing.

Methods

In 2019, 56 first- and third-year residents participated in a 3-part, 5-week OSCE training them to use telehealth to manage complex primary care. Learners conducted a standardized patient (SP) televisit in session 1, coordinated care via inter-visit e-messaging, and led a simulated interdisciplinary teleconference in session 2. Surveys measured confidence before session 1 (pre), post-session 1 (post-1), and post-session 2 (post-2). SP televisit checklists and investigators' assessment of e-messages evaluated residents' telehealth skills.

Results

Response rates were pre 100%, post-1 95% (53 of 56), and post-2 100%. Post-intervention, more residents were “confident/very confident” in adjusting their camera (33%, 95% CI 20–45 vs 85%, 95% CI 75–95, P < .0001), e-messaging (pre 36%, 95% CI 24–49 vs post-2 80%, 95% CI 70–91, P < .0001), and coordinating interdisciplinary care (pre 35%, 95% CI 22–47 vs post-2 84%, 95% CI 74–94, P < .0001). More residents were “likely/very likely” to use telemedicine in the future (pre 56%, 95% CI 43–69, vs post-2 79%, 95% CI 68–89, P = .001).

Conclusions

A longitudinal, interdisciplinary telehealth simulation is feasible and can improve residents' confidence in using telemedicine to provide complex patient care.

Introduction

Telehealth utilization has increased in the United States, with over 60% of health care systems incorporating telemedicine technology,¹ and this is likely to expand post-COVID-19. Telehealth is multifaceted and requires both synchronous and asynchronous skills in using technology to provide effective care.¹ As telehealth usage expands, the American Medical Association has encouraged training in telemedicine for students and residents.² However, curricula do not address how longitudinal and complex primary care can be delivered utilizing telehealth and electronic technologies.

In the literature on telehealth education, over a quarter of medical schools delivered preclinical telemedicine curricula, and almost half provided clinical exposure through routine practice or telehealth electives.³ Other specialties, such as dermatology,^4,5 psychiatry,⁶ and neurology,^7,8 have reported telemedicine training for residents and fellows. There are a few reports of primary care telehealth training, with limited uses of telehealth technologies. Family medicine program directors report limited and infrequent use of telehealth for e-visits and store-and-forward services among residents.⁹ A 3-year internal medicine telehealth curriculum provided online didactic content and experience in remote patient monitoring.¹⁰ Lee et al also highlighted the importance of incorporating electronic consultation in graduate medical education to improve value-based access to specialty care.¹¹ Although objective structured clinical examinations (OSCEs) have been used in telehealth training, most interventions are limited to single encounter e-visits at the undergraduate medical level of training.^12–14 Only one study reported that an OSCE with nurse practitioners and occupational/physical therapy students utilized an interdisciplinary teaching model.¹⁵

Primary care can be challenging, and for patients with complex psychosocial needs, we should use interdisciplinary telehealth modalities to manage patients. The Interprofessional Education Collaborative highlights communication technologies to facilitate health professional engagement in shared patient-centered problem solving.¹⁶

The study goal was to describe and evaluate a telehealth curriculum with a longitudinal OSCE to improve internal medicine (IM) residents' confidence and skills in coordinating complex, interdisciplinary primary care via televisits, electronic consultation, and teleconferencing.

Methods

Setting and Participants

In 2019, 56 first- and third-year IM residents participated in mandatory training sessions during their ambulatory block at a university-based residency program in Stony Brook, New York. The residency follows a 4+1 block schedule, and each session had 10 to 12 participants. Due to scheduling conflicts, second-year residents did not participate.

What was known and gap

There are limited curricula for resident telehealth training that focuses on longitudinal, interdisciplinary primary care.

What is new

We report a longitudinal simulation to teach interdisciplinary telemedicine and e-communication skills.

Limitations

Single center study and resource-intensive intervention limit generalizability.

Bottom line

Interdisciplinary telehealth training is feasible and can improve residents' confidence in using telemedicine and e-communication to provide complex patient care.

Intervention

Residents attended two 120-minute sessions that were 5 weeks apart. Each session was repeated for 5 consecutive weeks in order to capture all ambulatory groups. Session 1 was conducted in the clinical simulation center (CSC) and required 4 standardized patients (SPs) per session for televisit OSCEs. The inter-visit simulation occurred during the 4 weeks between sessions 1 and 2, and electronic messaging was done in a training domain of Cerner PowerChart. The session 2 teleconference was held virtually using Skype for Business (Figure). Two IM faculty members delivered lectures and facilitated sessions. Faculty from social work, pain management, psychiatry/addiction medicine, and nursing participated via Skype for the session 2 teleconference.

Figure.

Components of Telemedicine Simulation Activity

Note: Icons made by Freepik from www.flaticon.com.

Abbreviations: OSCE, objective structured clinical examination; EHR, electronic health record.

The intervention was funded by an internal department grant, which covered the cost of a TytoCare medical exam kit ($2,750) and CSC use ($4,000). No protected time was allotted for IM faculty investigators, who facilitated 10 sessions. The CSC technicians set up Skype for Business for OSCE televisits. One IM investigator initiated teleconferences and was present to troubleshoot technological issues. Psychiatry, pain management, social work, and nursing faculty volunteered five 1-hour blocks to participate in virtual teleconferences. Residents had 2 half-day clinic sessions blocked to participate in the simulation. Electronic messages and responses were sent by a study investigator weekly to resident Cerner PowerChart electronic inboxes, and a reminder email was sent 1 week after session 1 to respond to messages.

Training Description

Session 1: 

Residents received an orientation with didactic on screening for social determinants of health and OSCE instructions to address psychosocial factors, then rotated through three 30-minute stations: (1) televisit OSCE, (2) e-consultation and e-messaging, and (3) virtual examination tools.

Inter-visit Simulation: 

Over 4 weeks between sessions, the simulation continued via e-messaging. A nursing e-message was sent to participants regarding patient decompensation, prompting a series of e-consultations between the participant and specialty providers. The exercise required the learner to address the nursing message and to communicate with the social worker, pain management physician, and psychiatry providers using e-messages (activity provided as online supplemental material).

Session 2: 

This session included (1) a 20-minute didactic on interdisciplinary care and teleconferencing; (2) a 40-minute group preparation for the teleconference; (3) a 45-minute interdisciplinary teleconference; and (4) a 15-minute debriefing (provided as online supplemental material). Please email the corresponding author for simulation materials.

Evaluation

Residents voluntarily completed anonymous Qualtrics surveys before session 1 (pre) and after session 1 (post-1) and session 2 (post-2). The surveys evaluated residents' confidence with telehealth and interdisciplinary care using 5-point Likert scales (1, strongly disagree, to 5, strongly agree, and 1, very unconfident, to 5, very confident). Due to an error in survey format, data on knowledge and attitudes toward telehealth (presurvey questions 8–17) were not included in this report. SP checklists to evaluate televisit skills were based on locally developed templated checklists from the CSC library, with additional case-specific tasks in assessing psychosocial factors¹⁷ and pain using the Pain, Enjoyment of life, and General activity (PEG) scale (provided as online supplemental material).¹⁸ E-messaging evaluations were based on a consultation model from Podolsky et al¹⁹ and were scored independently by 2 IM faculty investigators for internal validity. Each e-message was scored as unsatisfactory, satisfactory, or superior (evaluation provided as online supplemental material).

Twenty survey questions were based on prior studies on the impact of telemedicine simulation training,¹³ telehealth usability,²⁰ and working in multidisciplinary teams.²¹ Additional investigator-developed questions were not tested prior to use: 6 on demographics, 3 on satisfaction and likelihood of using telehealth in the future, and 8 on learner confidence in interdisciplinary care coordination using e-messaging and teleconferences (surveys provided as online supplemental material).

Statistical analysis was conducted in SAS 9.4 with chi-square test of independence for categorical variables and Kruskal–Wallis for continuous variables. The study was approved by the Institutional Review Board, and residents reviewed a consent form prior to participation.

Results

Survey response rates were pre 100% (56 of 56), post-1 95% (53 of 56), and post-2 100% (56 of 56). Demographics are described in Table 1.

Table 1.

Study Population Demographics (N = 56)

Characteristic	Pre (n = 56), n (%)	Post-1 (n = 53), n (%)	Post-2 (n = 56), n (%)	P Value
Age (mean, STD)	29.3 (2.6)	29.6 (2.8)	29.3 (2.4)	.94
Gender				.75
Male	33 (59)	35 (66)	35 (62)
Female	23 (41)	18 (33)	21 (37)
Racea				.99
Caucasian	19 (34)	17 (32)	19 (35)
Asian/Pacific Islander	28 (50)	26 (49)	23 (41)
Black/African American	1 (1)	1 (1)	1 (1)
Hispanic/Latino	2 (4)	3 (6)	2 (4)
Other	2 (4)	2 (4)	3 (6)
Prefer not to answer	4 (7)	4 (8)	7 (13)
PGY levelb				.99
1	28 (50)	26 (49)	28 (50)
3	28 (50)	27 (51)	28 (50)
Medical schoolc				.90
Allopathic medical school (MD)	45 (82)	43 (81)	45 (80)
Osteopathic medical school (DO)	10 (18)	9 (17)	10 (18)
International	0 (0)	1 (2)	1 (2)

^a1 missing response in post-2.

^b3 missing responses in post-1.

^c1 missing response in pre.

After the OSCE in session 1, more residents reported being “confident/very confident” in adjusting their camera (33%, 95% CI 20–45 vs 85%, 95% CI 75–95, P < .0001), troubleshooting videoconferences (18%, 95% CI 8–28 vs 58%, 95% CI 45–72, P < .0001), and speaking in front of a camera (42%, 95% CI 29–55 vs 76%, 95% CI 68–90, P = .001). They also reported improved confidence in explaining a televisit (29%, 95% CI 17–41 vs 74%, 95% CI 62–85, P = .002), establishing rapport (38%, 95% CI 25–51 vs 91%, 95% CI 83–98, P < .0001), and communicating effectively (33%, 95% CI 20–45% vs 79%, 95% CI 68-90%, P < .0001; Table 2).

Table 2.

Internal Medicine Resident Confidence in Televisit Skills^a

Skill		Very Unconfident and Unconfident, n (%), 95% CI	Neutral, n (%), 95% CI	Very Confident and Confident, n (%), 95% CI	P Value
Adjusting a telemedicine camera to maximize my positioning and visibility	Pre	20 (36), 24–49	17 (31), 19–43	18 (33), 20–45	< .0001
	Post-1	1 (2), 0–6	7 (13), 4–22	45 (85), 75–95
Communicating effectively with a patient via telemedicine	Pre	14 (25), 14–37	23 (42), 29–55	18 (33), 20–45	< .0001
	Post-1	1 (2), 0–6	10 (19), 8–28	42 (79), 68–90
Troubleshooting poor performance with videoconferencing during your patient encounter	Pre	26 (47), 34–60	19 (35), 22–47	10 (18), 8–28	< .0001
	Post-1	2 (4), 0–9	20 (38), 24–49	31 (58), 45–72
Taking a patient history via telemedicine	Pre	6 (11), 3–19	21 (38), 25–51	28 (51), 38–64	.0002
	Post-1	0 (0)	6 (11), 3–19	47 (89), 80–97
At the start of the visit, explaining to the patient what the visit will entail and what to expect	Pre	13 (24), 12–35	26 (47), 34–60	16 (29), 17–41	.0017
	Post-1	0 (0)	14 (26), 14–37	39 (74), 62–85
Providing counseling to the patient for treatment and follow-up via telemedicine	Pre	11 (20), 9–31	22 (40), 27–53	22 (40), 27–53	< .0001
	Post-1	1 (2), 0–6	7 (13), 4–22	45 (85), 75–95
Establishing rapport with a patient via telemedicine	Pre	12 (22), 11–33	22 (40), 27–53	21 (38), 25–51	< .0001
	Post-1	0 (0)	5 (9), 1–17	48 (91), 83–98
Feeling comfortable speaking in front of a camera	Pre	9 (16), 7–26	23 (42), 29–55	23 (42), 29–55	.0008
	Post-1	2 (4), 0–9	11 (20), 9–31	42 (76), 68–90

^a1 response missing in presurvey (n = 55).

Regarding resident skills during the televisit, SPs reported that most residents introduced their role (98%, 55 of 56), explained televisit expectations (96%, 54 of 56), and redirected the patient for optimal camera viewing (80%, 45 of 56). Most residents addressed SP needs (95%, 53 of 56), portrayed empathy (100%), and 98% of SPs reported that they would return to their trainee for care. There was no significant difference in mean OSCE grades between third-year and first-year learners (68 of 100, SD 10.7 vs 65.4 of 100, SD 12.1). Despite orientation instructions to address psychosocial factors, residents rarely asked about making ends meet (16%, 9 of 56), food security (11%, 6 of 56), housing stability (5%, 3 of 56), and transportation (20%, 11 of 56), with no significant difference between resident levels.

For most evaluations of resident e-consultations, there was no significant difference in scores between 2 independent IM faculty evaluators. Resident response rate decreased with consecutive messages to providers from pain management (82%, 46 of 56), social work (66%, 37 of 56), and psychiatry (34%, 19 of 56). Residents struggled with several tasks, where the e-messaging skills most frequently scored as unsatisfactory: indicating urgency of consultation (50%–93%), documenting plan for follow-up (49%–57%), and communicating plan of care (46%–68%; Table 3).

Table 3.

Faculty Evaluation of Residents' E-Consultation/E-Messaging Skills^a

Evaluation by Specialty	Mean Faculty Score of Evaluation, %			P Valueb
	Unsatisfactory	Satisfactory	Superior
Pain management (N = 46)
Provided a succinct case summary	36	53	11	.43
Asked a clear, specific question	13	78	9	.81
Indicated urgency of consult	73	23	4	.33
Documented plan for follow-up	49	47	4	< .0001
Advocated for chosen plan of care	25	68	7	.0017
Discussed risk/benefits of care options	57	38	5	.13
Effectively communicated plan of care	46	48	7	.12
Social work (N = 37)
Provided a succinct case summary	26	58	16	.0260
Asked a clear, specific question	28	53	19	.69
Indicated urgency of consult	93	4	3	.24
Documented plan for follow-up	57	41	1	.10
Advocated for chosen plan of care	51	46	3	.0004
Discussed risk/benefits of care options	66	34	0	.0028
Effectively communicated plan of care	68	22	11	.55
Psychiatry (N = 19)
Provided a succinct case summary	8	76	16	.37
Asked a clear, specific question	34	55	11	.41
Indicated urgency of consult	50	47	3	.75
Documented plan for follow-up	50	47	3	.52
Advocated for chosen plan of care	39	53	8	.47
Discussed risk/benefits of care options	29	68	3	.73
Effectively communicated plan of care	47	50	3	.33

^aMessages were scored independently by 2 general internal medicine faculty evaluations.

^bP values > .05 indicate no significant difference between faculty scores.

After the simulated interdisciplinary teleconference in session 2, there was an increase in residents who were “confident/very confident” in using e-messaging to communicate with specialists (36%, 95% CI 24–49 vs 80%, 95% CI 70–91, P < .0001) and coordinating interdisciplinary care (35%, 95% CI 22–47 vs 84%, 95% CI 74–94, P < .0001). Confidence increased in presenting to another provider using telemedicine (35%, 95% CI 22–47 vs 79%, 95% CI 68–89, P < .0001), facilitating an interdisciplinary teleconference (25%, 95% CI 14–37 vs 88%, 95% CI 79–96, P < .0001), and formulating an interdisciplinary care plan (42%, 95% CI 29–55 vs 91%, 95% CI 84–99, P < .0001; Table 4).

Table 4.

Resident Confidence in Using Telemedicine for Interdisciplinary Care^a

Skill		Very Unconfident and Unconfident, n (%), 95% CI	Neutral, n (%), 95% CI	Very Confident and Confident, n (%), 95% CI	P Value
Presenting a patient to another provider via telemedicine	Pre	12 (22), 11–33	24 (44), 30–57	19 (35), 22–47	< .0001
	Post-2	4 (7), 0–14	8 (14), 5–23	44 (79), 68–89
Using the electronic health record messaging to communicate with specialists in the ambulatory setting	Pre	9 (16), 7–26	26 (47), 34–60	20 (36), 24–49	< .0001
	Post-2	1 (2), 0–5	9 (16), 6–26	45 (80), 70–91
Coordinating outpatient interdisciplinary care using electronic health record messaging	Pre	15 (27), 16–39	21 (38), 25–51	19 (35), 22–47	< .0001
	Post-2	1 (2), 0–5	8 (14), 5–23	47 (84), 74–94
Facilitating an interdisciplinary teleboard to coordinate patient care	Pre	19 (35), 22–47	22 (40), 27–53	14 (25), 14–37	< .0001
	Post-2	2 (4), 0–8	4 (7), 0–14	49 (88), 79–96
My ability to address the needs of a patient with pain in an interdisciplinary team setting	Pre	12 (22), 11–33	27 (49), 36–62	16 (29), 17–41	< .0001
	Post-2	2 (4), 0–8	8 (14), 5–23	46 (82), 72–92
My ability to communicate with professionals in other health fields regarding the management of pain for my patients	Pre	8 (15), 5–24	24 (44), 30–57	23 (42), 29–55	< .0001
	Post-2	2 (4), 0–8	8 (14), 8–24	46 (82), 72–92
My ability to work with an interdisciplinary team to formulate a care plan for my patients with pain	Pre	7 (13), 4–22	25 (45), 32–59	23 (42), 29–55	< .0001
	Post-2	2 (4), 0–8	3 (5), 0–11	51 (91), 84–99
Feeling comfortable speaking in front of a camera	Pre	9 (16), 7–26	23 (44), 29–55	23 (42), 29–55	.0006
	Post-2	2 (4), 0–8	11 (20), 9–30	43 (77), 66–88

^a1 response missing in presurvey (n = 55).

Overall, residents rated sessions as “good” or “excellent” (post-1 96%, post-2 95%) and wanted to participate in additional telemedicine training (post-1 92%, post-2 84%). More residents reported that they were “likely/very likely” to use telemedicine in the future (pre 56%, 95% CI 43–69 vs post-2 79%, 95% CI 68–89, P = .001).

Discussion

A longitudinal interdisciplinary simulation improved IM residents' self-reported confidence in using telemedicine, e-messaging, and teleconferences, and increased their reported likelihood of using telemedicine in the future. For patients with complex needs, provider skills in team management and synchronous and asynchronous communication are important. Residents were least skilled in screening for psychosocial barriers to care and with documenting urgency of consults, plan of care, and follow-up on e-consultation.

As expertise becomes more specialized and care becomes more decentralized, residents will require training in telemedicine and remote expert consultation. Although we conducted the OSCE in a simulation center, the case was virtual and only required access to software such as Skype for Business. For future telemedicine training in the post-COVID-19 era, interactive computer-based virtual cases,²² virtual SP, and interdisciplinary cases could be used to increase the reach of simulation centers, expand access in settings without simulation centers, or to accommodate social distancing requirements. E-consultation and teleconferences, especially in complex disease, provides an opportunity to evaluate and expand the boundaries of traditional Accreditation Council for Graduate Medical Education and Interprofessional Education Collaborative competencies. While we did not evaluate the change in e-consultation skills before and after intervention, it was interesting that there were specific areas of e-consultation that residents struggled with. We hypothesize that there were deficiencies in these skills because our residents do not routinely write e-messages to consultants between ambulatory blocks. This may be an opportunity for future educational interventions and evaluation. Although interdisciplinary teleconferences in primary care may seem futuristic, the concept is familiar in complex care of cancer through tumor boards and hospice interdisciplinary meetings. By educating residents in leading interdisciplinary teleconferences, we aim to develop leadership skills and create demand for better practices in caring for complex patients.

Our study had several limitations. The intervention was delivered to only one program, and this simulation may not be generalizable to other specialties or programs. We did not measure the duration of training effect or assess patient-level outcomes. Due to funding and time limitations, we were unable to assess resident teleconference skills with an OSCE or change in e-messaging skills after intervention. Resident participation in the e-messaging exercise declined with each sequential message and future interventions should limit the number of required tasks to avoid learner disengagement.

Nevertheless, we learned many valuable lessons. We found that the success of the televisits and teleconferences depended on technicians or faculty to troubleshoot videoconferencing issues. Anecdotally, the residents commented that the SPs faced many of the challenges their continuity patients experience, and as the case progressed we found opportunity to teach different skills in telemedicine and e-messaging. To increase feasibility of reproducing this simulation, educators may consider deconstructing components of the simulation. Individual modules could focus on conducting televisits, e-messaging, and interdisciplinary teleconferencing. This curriculum could also be used with different patient cases and subspecialties. As educators, we should consider how future care will be delivered using telehealth technology and expand curricula to mimic real-world interdisciplinary care.

Conclusions

A longitudinal interdisciplinary telehealth simulation is feasible and can improve residents' confidence in using telemedicine and e-messaging tools to provide complex patient care.