Abstract
Background and Objectives
Telemedicine has become a highly-utilized form of primary care, requiring medical schools and residency programs to develop standardized telemedicine training to meet learners’ educational needs. This study highlights specific areas of clinical teaching and faculty development regarded as highly valuable in a family medicine (FM) residency program.
Methods
We developed a needs assessment survey instrument based on Accreditation Council for Graduate Medical Education (ACGME) milestones and circulated it to faculty and residents at a suburban FM residency program in August 2020. We mapped each survey question to ACGME core competencies to identify common themes. We performed two-sample t tests to compare perceived self-confidence in faculty assessment and resident performance of key telemedicine clinical skills.
Results
A total of 29 respondents (15 faculty, 14 residents) completed the survey. Both residents and faculty expressed comfort with obtaining a focused history, ruling out red flag symptoms, formulating a differential diagnosis, and planning follow-up care. Faculty reported confidence in their ability to provide feedback about medical knowledge and clinical decision making, but also identified a need for better feedback tools. Both faculty and residents identified a need for better teaching of physical exam skills during video visits. There were no statistically significant differences for perceived self-confidence in evaluating and performing key telemedicine skills between faculty and residents, respectively.
Conclusion
Development of effective telemedicine educational experiences should emphasize teaching virtual communication and physical exam skills, and developing new approaches to learner evaluation.
Introduction
Telemedicine is quickly emerging as a highly-utilized form of medical care. The COVID-19 pandemic prompted a notable surge in telemedicine visits1,2—synchronous telephone and video visits between providers and patients—presenting unique challenges to medical educators including how to effectively precept learners’ telemedicine visits.3 Prior to 2020, telemedicine training in medical school, residency, and faculty development was limited,3,4 and most family medicine residencies were not utilizing video visits.5 A growing number of residency programs and medical schools have sought to develop standardized telemedicine training before and throughout the pandemic.6–8 In 2020, the Society of Teachers of Family Medicine (STFM) created a telemedicine task force entrusted with developing a national telemedicine curriculum for students and residents.9
Even prior to the pandemic, telemedicine was known to be associated with high patient satisfaction and improved clinician-patient communication.10–12 Some studies have outlined general topics in telemedicine curricula based on process mapping and expert consultation.8,13 However, there is a paucity of research examining specific needs of residency faculty and trainees related to telemedicine.3 The unique aim of this study was to identify specific areas of telemedicine clinical skills training for which FM residents and faculty felt a need for greater education and faculty development was needed at a six-residents-per-year, community-based FM residency program.
Methods
We developed needs assessment surveys based on Accreditation Council for Graduate Medical Education (ACGME) milestones and through collaboration with members of the STFM Telemedicine Task Force. In August 2020, a 16-item survey for FM program faculty and a 25-item survey for residents was distributed over a 2-week period.14 Questions were presented using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), and mean scores were calculated for each response. Residents were asked additional questions to measure prior experience and to gauge interest in telemedicine. Due to the residency program size, we pooled resident data for all 3 years (PGY 1–3). All residents had at least 2 months of telemedicine exposure at the time of the survey.
We employed a descriptive statistics approach where authors S.V. and P.S. independently mapped every faculty and resident survey question to two ACGME core competencies, to mitigate bias and to identify common themes for program improvement. For similar items between surveys, a 2-sample t test was performed to compare perceived self-confidence in faculty assessment and resident performance of key telemedicine clinical skills, including taking histories, performing focused physical exams, formulating differential diagnoses, making treatment plans, and arranging appropriate follow-up care. The Kaiser Permanente Southern California Institutional Review Board granted this project exemption.
Results
A total of 14 family medicine residents (14/18; 78% response rate) and 15 program faculty (15/15; 100% response rate) completed the survey. Survey responses mapped to two ACGME competency domains by two study authors (Table 1) had interrater reliability of 87% (data not shown). Among the resident respondents, four were PGY 1, five were PGY 2, and five were PGY 3. Due to level of training, prior experience in telemedicine was variable, with three residents (3/14; 21%) having previously completed fewer than 10 telemedicine visits, one having completed between 10 and 50 visits (1/14; 7%) and 10 of 14 (71%) having completed more than 50 telemedicine visits.
Table 1.
Summative Evaluation of Faculty and Resident Perceived Self-confidence in Telemedicine Clinical Skills, With Responses Mapped to ACGME Core Competency Domains
| ACGME Competency Domain | Faculty (N=15) | Residents (N=14) | ||||
|---|---|---|---|---|---|---|
| %SA/A | %N | %SD/D | %SA/A | %N | %SD/D | |
| Practice-Based Learning and Improvement | 68.9 | 20 | 11.1 | 76.1 | 14.4 | 9.5 |
| Professionalism | 57.7 | 31.2 | 11.1 | 63.1 | 14.3 | 22.6 |
| Medical Knowledge | 86.7 | 4.4 | 8.9 | 69.7 | 19.6 | 10.7 |
| Systems-Based Processes | 80 | 13.3 | 6.7 | 52.6 | 19.5 | 27.9 |
| Patient Care and Procedural Skills | 73.3 | 10 | 16.7 | 70.2 | 15.6 | 14.2 |
| Interpersonal and Communication Skills | 56.2 | 32.3 | 11.5 | 46.4 | 23.8 | 29.8 |
Percentage of positive (strongly agree [SA], agree [A]), neutral [N]), and negative (disagree [D]/strongly disagree[SD]) responses for each group.
Faculty reported the highest confidence in evaluating a resident’s ability to take appropriate histories, formulate differential diagnoses, and plan appropriate follow-up care (Table 2) consistent with the highest scores mapped to the Medical Knowledge, Systems-Based Practice, and Patient Care domains (Table 1). Areas of need for faculty development included learning to better evaluate a resident’s communication skills and having better assessment tools for telemedicine precepting (Table 2).
Table 2.
Faculty Reported Self-confidence in Assessment of Residents for Key Telemedicine Skills
| Skill | Number of Faculty Responses (N=15) | Mean (SD) | Median | ||
|---|---|---|---|---|---|
| Strongly Agree or Agree | Neutral | Disagree or Strongly Disagree | |||
| Taking a focused history | 13 | 1 | 1 | 4.07 (1.03) | 4 |
| Performing a focused physical exam | 4 | 8 | 3 | 3.07 (0.96) | 3 |
| Formulating a differential diagnosis | 13 | 0 | 2 | 4.07 (1.16) | 4 |
| Making a diagnosis and treatment plan | 11 | 3 | 1 | 3.73 (0.96) | 4 |
| Making an appropriate follow-up plan | 12 | 2 | 1 | 4.07 (1.10) | 4 |
| Communication skills during visit | 7 | 6 | 2 | 3.4 (1.06) | 3 |
| Having appropriate evaluation tools | 6 | 6 | 3 | 3.33 (1.18) | 3 |
Scale: 1 = strongly disagree, 2= disagree, 3 = neutral, 4 = agree, 5 = strongly agree. Faculty grouped according to positive (strongly agree/agree), neutral, and negative (disagree/strongly disagree) responses.
Resident responses revealed high self-confidence for taking focused histories, including ruling out red flag symptoms (Table 3). Need for a telemedicine-focused elective and additional training on efficiency among residents was also revealed. Both faculty and residents identified a need for better teaching of telemedicine physical exams (Table 4). Residents also identified curricular needs in conducting visits with a language interpreter, prescribing controlled medications, arranging transportation for patients requiring a higher level of care, and conducting a visit involving multiple chief complaints (Table 3).
Table 3.
Resident-Reported Self-confidence in Performance of Key Telemedicine Skills
| Skill | Number of Resident Responses (N=14) | Mean (SD) | Median | ||
|---|---|---|---|---|---|
| Strongly Agree/Agree | Neutral | Disagree/Strongly Disagree | |||
| Taking a focused history | 12 | 1 | 1 | 4.42 (0.93) | 5 |
| Guiding patient through physical exam | 3 | 6 | 5 | 2.78 (0.89) | 3 |
| Ruling out red flag symptoms | 11 | 2 | 1 | 4.21 (0.97) | 4 |
| Managing multiple chief complaints | 6 | 4 | 4 | 3.21 (0.97) | 3 |
| Establishing a diagnosis | 8 | 4 | 2 | 3.5 (0.85) | 4 |
| Creating a follow-up plan | 12 | 1 | 1 | 4.14 (0.86) | 4 |
| Deciding on appropriate transportation | 6 | 2 | 6 | 3.14 (1.16) | 3 |
| Need for more efficiency training | 13 | 0 | 1 | 4.43 (1.09) | 5 |
| Interest in a telehealth elective | 12 | 0 | 2 | 4.07 (0.99) | 4 |
| Conducting visit in another language | 7 | 1 | 6 | 3.07 (1.21) | 3 |
| Prescribing controlled medications | 2 | 3 | 9 | 2.35 (1.15) | 2 |
Scale: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly Agree.
Residents grouped by positive (strongly agree/agree), neutral, and negative (disagree/strongly disagree) responses.
Table 4.
Faculty and Resident Scores on 5-point Likert Scale for Matched Questions on Needs Assessment Survey.
| Key Telemedicine Skills | Faculty | Resident | P Value | ||
|---|---|---|---|---|---|
| Mean (SD) | Median | Mean (SD) | Median | ||
| Taking a focused history | 4.07 (1.03) | 4 | 4.42 (0.94) | 5 | .17 |
| Formulating a differential diagnosis | 3.73 (0.96) | 4 | 3.5 (0.85) | 4 | .25 |
| Planning appropriate follow-up care | 4.07 (1.10) | 4 | 4.14 (0.86) | 4 | .83 |
| Making an appropriate treatment plan | 3.73 (0.96) | 4 | 3.5 (0.85) | 4 | .49 |
| Performing a focused telemedicine physical exam | 3.07 (0.96) | 3 | 2.79 (0.89) | 3 | .21 |
There were no statistically significant differences at P=.05 in perceived self-confidence of faculty (evaluating visits) and residents (performing visits) for the key clinical skills of taking histories, performing focused physical exams, formulating differential diagnoses, making treatment plans, and arranging appropriate follow-up care (Table 4). Mapping of survey responses demonstrated that faculty provided more positive responses for the domains of Medical Knowledge and Systems-Based Practice compared to residents (Table 1).
Discussion
This study highlighted several aspects of telemedicine precepting and care delivery for FM faculty and residents. Faculty and resident feedback of telemedicine as a care modality was positive overall. Faculty felt confident in their ability to provide feedback about residents’ medical knowledge and clinical decision making during telemedicine visits. The identified faculty development needs included utilizing standardized learner evaluation tools for telemedicine visits and greater instruction on teaching telemedicine physical exam skills to residents. Residents proposed the development of a telemedicine rotation as a potential solution for meeting their educational needs.
A major limitation of this study was the small sample size, which makes it difficult to nuance small differences between faculty and residents. Also, faculty and residents had prior limited exposure to video visits at their institution at the time of the survey. These factors may limit generalizability to certain residency programs. However, this study led to important developments including the development of a 1-week telemedicine rotation and creation of succinct evaluation forms for faculty precepting.15,16
This single-institution study highlighted components of telemedicine instruction for which FM residents felt a greater educational need exists, including coaching patients through telemedicine physical exams, conducting visits in a language other than English, and dealing with more complex chief complaints over the phone. Future curricula may include additional training resources for the telemedicine physical exam17,18 and assessment using University of Minnesota’s Entrustable Professional Activities, which suggest appropriate levels of supervision based on a learner’s experience level.19 Telemedicine is now an integral part of primary care; thus, teaching best practices within telemedicine will remain an essential component of graduate medical education for years to come.
Acknowledgments
Presentations: This study was presented at the STFM 2021 Annual Spring Conference (virtual), May 2021.
References
- 1. Hincapié MA, Gallego JC, Gempeler A, Piñeros JA, Nasner D, Escobar MF. Implementation and usefulness of telemedicine during the COVID-19 pandemic: a scoping review. J Prim Care Community Health. 2020;11 doi: 10.1177/2150132720980612. 2150132720980612. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Doraiswamy S, Abraham A, Mamtani R, Cheema S. Use of telehealth during the COVID-19 pandemic: scoping review. J Med Internet Res. 2020;22(12):e24087. doi: 10.2196/24087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Stovel RG, Gabarin N, Cavalcanti RB, Abrams H. Curricular needs for training telemedicine physicians: a scoping review. Med Teach. 2020;42(11):1234–1242. doi: 10.1080/0142159X.2020.1799959. [DOI] [PubMed] [Google Scholar]
- 4. Pourmand A, Ghassemi M, Sumon K, Amini SB, Hood C, Sikka N. Lack of telemedicine training in academic medicine: are we preparing the next generation? Telemed J E Health. 2021;27(1):62–67. doi: 10.1089/tmj.2019.0287. [DOI] [PubMed] [Google Scholar]
- 5. Moore MA, Jetty A, Coffman M. Over half of family medicine residency program directors report use of telehealth services. Telemed J E Health. 2019;25(10):933–939. doi: 10.1089/tmj.2018.0134. [DOI] [PubMed] [Google Scholar]
- 6. Jumreornvong O, Yang E, Race J, Appel J. Telemedicine and medical education in the age of COVID-19. Acad Med. 2020;95(12):1838–1843. doi: 10.1097/ACM.0000000000003711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Jenkins J, Oyama O. Telemedicine: the art of innovative technology in family medicine. Int J Psychiatry Med. 2020;55(5):341–348. doi: 10.1177/0091217420951038. [DOI] [PubMed] [Google Scholar]
- 8. Ha E, Zwicky K, Yu G, Schechtman A. Developing a telemedicine curriculum for a family medicine residency. PRiMER Peer-Rev Rep Med Educ Res. 2020;4:21. doi: 10.22454/PRiMER.2020.126466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Theobald M, Brazelton T. STFM forms task force to develop a national telemedicine curriculum. Ann Fam Med. 2020;18(3):285–286. doi: 10.1370/afm.2549. [DOI] [Google Scholar]
- 10. Kruse CS, Krowski N, Rodriguez B, Tran L, Vela J, Brooks M. Telehealth and patient satisfaction: a systematic review and narrative analysis. BMJ Open. 2017;7(8):e016242. doi: 10.1136/bmjopen-2017-016242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Polinski JM, Barker T, Gagliano N, Sussman A, Brennan TA, Shrank WH. Patients’ satisfaction with and preference for telehealth visits. J Gen Intern Med. 2016;31(3):269–275. doi: 10.1007/s11606-015-3489-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Orlando JF, Beard M, Kumar S. Systematic review of patient and caregivers’ satisfaction with telehealth videoconferencing as a mode of service delivery in managing patients’ health. PLoS One. 2019;14(8):e0221848. doi: 10.1371/journal.pone.0221848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Costich M, Robbins-Milne L, Bracho-Sanchez E, Lane M, Friedman S. Design and implementation of an interactive, competency-based pilot pediatric telemedicine curriculum. Med Educ Online. 2021;26(1):1911019. doi: 10.1080/10872981.2021.1911019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Venditti S, Fuchs L, Snarskis C, Sazegar P.Telemedicine needs assessment survey for family medicine faculty and residents. STFM Resource Library. Jan 24, 2022. [Accessed February 1, 2022]. http://connect.stfm.org/resourcelibrary/ourlibrary/viewdocument?DocumentKey=fd6e6246-9b6f-4db0-9135-28badf80c9dc .
- 15.Venditti S, Fuchs L, Snarskis C, Sazegar P. Telemedicine Evaluation Form – KP San Diego FMR. STFM Resource Library; Jan 24, 2022. [Accessed February 1, 2022]. http://connect.stfm.org/resourcelibrary/ourlibrary/viewdocument?DocumentKey=7a8e77a3-1957-4fd3-9fea-9d2b838fbb25 . [Google Scholar]
- 16.Sazegar P, Venditti S, Fuchs L, Snarskis C.Telemedicine field note – KP San Diego FMR. STFM Resource Library. Jan 24, 2022. [Accessed February 1, 2022]. http://connect.stfm.org/resourcelibrary/ourlibrary/viewdocument?DocumentKey=bebdf1b9-c625-48e8-b1cf-8c45dd32a13c .
- 17. Benziger CP, Huffman MD, Sweis RN, Stone NJ. The telehealth ten: a guide for a patient-assisted virtual physical examination. Am J Med. 2021;134(1):48–51. doi: 10.1016/j.amjmed.2020.06.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Yedlinsky NT, Peebles RL. Telemedicine management of musculoskeletal issues. Am Fam Physician. 2021;103(3):147–154. [PubMed] [Google Scholar]
- 19.Englander R, Benson B, Rosenberg M, Thompson J, Maier A. Provide telemedicine services: an entrustable professional activity. Association of American Medical Colleges; Apr 30, 2020. [Accessed: February 1, 2022]. https://www.aamc.org/resource-library/clinical-teaching-and-learning-experiences/provide-telemedicine-services-entrustable-professional-activity . [Google Scholar]