Abstract
Introduction:
Teleneurology has become widely adopted during severe acute respiratory syndrome coronavirus 2 pandemic. However, provider impressions about the teleneurology experience are not well described.
Methods:
A novel questionnaire was developed to collect provider impressions about video teleneurology encounters. All providers in the University of Pennsylvania Health System (UPHS) Neurology Department (N = 162) were asked to complete a questionnaire after each video teleneurology patient encounter between April and August 2020. Individual patient and encounter-level data were extracted from the electronic medical record.
Results:
One thousand six hundred three surveys were completed by 55 providers (response rate of 10.12%). The history obtained and the ability to connect with the patient were considered the same or better than an in-person visit in almost all encounters. The quality of the physician–patient relationship was good or excellent in 93%, while the overall experience was the same as an in-person visit in 73% of visits and better in 12%. Sixty-eight percent of respondents reported that none of the elements of the neurological examination if performed in person would have changed the assessment and plan. Assessment of the visit as the same or better increased from 83% in April to 89% in July and 95% in August. Headache (91%), multiple sclerosis and neuroimmunology (96%), and movement disorder (89%) providers had the highest proportion of ratings of same or better overall experience and neuromuscular providers the lowest (60%).
Conclusions:
Provider impressions about the teleneurology history, examination, and provider–patient relationship are favorable in the majority of responses. Important differences emerge between provider specialty and visit characteristics groups.
Keywords: teleneurology, provider, quality improvement, telemedicine
Introduction
Teleneurology is neurological care delivered through telephone or videoconference electronic communication.1 Before 2020, the proportion of neurological outpatient visits that were performed virtually was small in comparison with in-person visits,2 and most were conducted to overcome lack of access to specialized care due to geographical or physical barriers.
Telestroke, the most successful implementation of teleneurology,3,4 has improved acute stroke management, and patients treated through telestroke have outcomes similar to in-person neurological care.5–7 Teleneurology models have been proposed in neurocritical care,8 epilepsy,9 movement disorders,10 headache,11 multiple sclerosis,12 traumatic brain injury,13 and dementia.14 However, none have been widely implemented in part due to the cost of implementation, challenges in reimbursement, and regulatory barriers.15 In addition, few studies have reported the provider impressions of teleneurology visits.
The severe acute respiratory syndrome coronavirus 2 (SARS CoV-2) pandemic led to a dramatic increase in teleneurology out of necessity to ensure continuity of care and the safety of patients and providers. Among neurology providers in the University of Pennsylvania Health System (UPHS), three teleneurology visits were conducted during the first 2 weeks of March 2020 before the SARS CoV-2 outbreak in Philadelphia. In contrast, an average of 1,079 visits per week were conducted between April and July 2020.16 This sudden rise in teleneurology provided an opportunity to capture information from neurology providers about the teleneurology experience in a hypothesis-generating, quality-improvement approach. Here, we present provider impressions of teleneurology at UPHS during the SARS CoV-2 pandemic from individual encounters by videoconference teleneurology.
Methods
Survey Design and User Testing
A novel teleneurology provider questionnaire was developed by all authors. Key concepts included adequacy of technology, assessment of the history and physical examination, and overall impressions from individual teleneurology encounters comparing teleneurology in-person visits. An initial draft of the questionnaire was written (L.W.) and edited by all coauthors. REDCap was used to administer the questionnaire. It was designed to be completed rapidly and self-administered by each provider after each individual teleneurology patient-encounter. Six providers (C.P., D.J., K.Y., M.S., T.F.T., and Christina Blum, MD) conducted a pilot over the course of 3 days including 48 patient encounters and provided feedback on survey usability. Suggestions were incorporated, and a final questionnaire was created. Pilot-phase data are excluded from the final dataset as the wording of questions differed between the initial and final questionnaires.
Individual patient and encounter-level identifiers were collected allowing a direct link between the questionnaire response and the patient encounter for future research. The questions and responses are shown in Table 1.
Table 1.
Teleneurology Provider Questionnaire
| QUESTION | POSSIBLE RESPONSES |
|---|---|
| Videoconference software | Bluejeans, FaceTime, Vidyo, Zoom, Other (optional free text) |
| Patient device (optional) | Phone, Tablet, Desktop, Laptop |
| Compared with an in-person visit, the history obtained in this encounter was: | Worse, Same, Better, Comment (optional free text) |
| In evaluating this patient, what elements of the examination might have changed your assessment and plan if performed in person? | None, Mental status, Cranial nerves, Motor, Sensory, Cerebellar, Reflexes, Gait, Other, Comment (optional free text) Please specify what other elements of the examination would have been more helpful if performed in person (optional free text) |
| Please rate the quality of the physician–patient relationship during this telemedicine encounter: | Poor, Fair, Good, Excellent, Comment (optional free text) |
| Compared with an in-person examination, the ability to personally connect with the patient by telemedicine was: | Worse, Same, Better, Comment (optional free text) |
| Was the video quality adequate? | Yes, No, Comment (optional free text) |
| Was the audio quality adequate? | Yes, No, Comment (optional free text) |
| Compared with an in-person visit, the time required for this telemedicine visit was: | Less, Same, More, Comment (optional free text) |
| Compared with an in-person visit, the overall telemedicine experience was: | Worse, Same, Better, Comment (optional free text) |
Provider name, survey timestamp, and date of visit were also recorded. Responses were required unless otherwise noted.
Survey Distribution and Data Collection
Data were collected in real time and managed using REDCap electronic data capture tools hosted at UPenn.17 One hundred sixty-two adult neurology providers including physicians, resident physicians, fellows, nurse practitioners, and genetic counselors across the University of Pennsylvania Health System (UPHS) were asked to complete a questionnaire for each completed teleneurology videoconference patient-encounter. Resident physicians completed surveys for patients in their continuity clinic; attending physicians did not complete surveys for these encounters. Telephone-only encounters and attempted encounters with a technology failure were excluded to focus on videoconference encounters. Throughout the study period, patients and providers were given the flexibility to determine the need for in-person visits based on patient and provider comfort level or lack of access to technology necessary for teleneurology visits. Because of the COVID-19 pandemic providers were encouraged to use telemedicine as the default modality. As a result, 59.14% (15,839 of 26,784 visits) of all completed outpatient visits in the neurology department were conducted through videoconference, and an additional 10.82% (2,898 visits) were conducted through telephone. Each week an e-mail reminder was sent to all providers to complete the questionnaire; a hyperlink and QR code to the REDCap survey were also distributed. Responses recorded >14 days after the visit were removed from the analysis as this was considered to be too long to recall details necessary to reliably answer survey questions. Questionnaires were requested between April 25 and August 6, 2020.
Statistical Analysis
Data were analyzed, and summary statistics were determined using Stata (StataCorp. 2015. Stata Statistical Software: Release 14; StataCorp LP, College Station, TX, USA). Figures were created in Prism (GraphPad Software, San Diego, CA; www.graphpad.com). Omnibus Pearson χ2 analyses were conducted in exploratory studies to compare the frequency of providers reporting the same or better history, ability to personally connect, or overall experience, or good or excellent provider–patient relationship during the teleneurology encounter did not differ between providers' years of experience, month of visit, visit type, or division groups. Frequency of providers reporting that no elements of the examination might have changed the assessment and plan if performed in person was also compared across the above groups. Observed frequencies were compared with predicted frequencies assuming equivalence between groups. Alpha was set at 0.05 without correction for multiple testing in these exploratory studies. Standardized (Pearson) residuals were observed post hoc for any statistically significant results.18
Standard Protocol Approvals, Registrations, and Patient Consents
This project was reviewed and determined to qualify as quality improvement by the University of Pennsylvania's Institutional Review Board. Therefore, informed consent was not obtained.
Results
Survey Completion Summary
Surveys were completed between the period of April 25 and August 6, 2020. During this time period, 15,839 teleneurology video visits were conducted throughout the neurology department, and 1,697 surveys were completed. Figure 1A shows the weekly number of total outpatient and teleneurology outpatient visits completed during March to August 2020. Nonvideo visits were comprised of in-person and telephone-only visits. Survey records were removed from this analysis for the following reasons: no shows (N = 2, 0.12%), telephone-only entries (N = 20, 1.78%), incomplete entries (N = 1, 0.06%), duplicate entries (N = 29, 1.71%, the first completed survey was retained), responses >2 weeks after the visit (N = 16, 0.94%), and inpatient encounters (N = 26, 1.53%). Ninety-four entries (5.54% of total entries) were removed leaving 1,603 surveys of completed, outpatient, videoconference encounters, representing 10.12% of the total teleneurology video visits.
Fig. 1.
Teleneurology visits and provider impressions across the University of Pennsylvania Hospital System. (A) Completed outpatient visits per week in the University of Pennsylvania Health System (UPHS) Department of Neurology between March and August 2020. Black line represents the initiation of the provider survey. New and established patient visits are included. (B) Summary responses from all completed video visits (N = 1,620). (C) Provider impressions about the teleneurology examination. Responses are number of affirmative responses (% of total responses). Multiple selections were possible.
Provider and Encounter Summary
Fifty-five providers completed the survey at least once representing 33.95% of the providers requested to complete the survey. Among providers, 31 (56.36%) were male, and 24 (43.63%) were female. Four Certified Registered Nurse Practitioners (7.27% of providers who responded to the survey), 1 genetic counselor (1.81%), 8 resident physicians (14.54%), and 42 attending physicians (76.36%) completed the survey. The median years of provider experience (calculated as the time in years since terminal degree graduation) was 11 years (interquartile range [IQR] 7–24.5 years). Providers are represented from 10 different departmental divisions, including MS (N = 6, 10.91% of providers who responded to the survey), Headache (N = 1, 1.82%), Vascular (N = 10, 18.18%), Cognitive (N = 2, 3.64%), Movement Disorders (N = 8, 14.55%), General Neurology (N = 12, 21.82%), Neuromuscular (N = 2, 3.64%), Neuro-ophthalmology (N = 1, 1.81%), Epilepsy (N = 5, 9.09%), and Traumatic Brain Injury (N = 1, 1.81%), and 7 (12.72%) are neurology residents. Five providers completed >100 surveys, each accounting for 43.61% of the total data; 27 providers completed <10 surveys each. The median number of survey responses per provider was 9 (IQR 4–34.5, range 1–173). The top five responders were all providers with no prior telemedicine experience.
Seventy-four percent of entries were completed on the same day as the encounter, and 98% were completed within 1 week. The longest difference between visit date and data entry date was 12 days. Three hundred thirty-nine (21.21%) of the encounters were new patient or consultation visits, 1,259 (78.79%) were established patient visits, and 4 (0.25%) had missing billing or no charges associated. One hundred (6.23%) survey responses were completed for patients who were seen more than once during the survey collection period; one patient was seen four times and 48 patients were seen twice during the study period. Providers from the MS division completed 266 surveys (16.59% of all surveys recorded), 94 from Headache (5.86%), 160 from Vascular (9.98%), 106 from Cognitive (6.61%), 404 from Movement Disorders (25.20%), 358 from General Neurology (22.33%), 120 from Neuromuscular (7.49%), 1 from Neuro-ophthalmology (0.06%), 37 from Epilepsy (2.31%), 8 from Traumatic Brain Injury (0.50%), and 49 from neurology residents (3.06%).
Patient Characteristics
One thousand six hundred three survey responses were recorded from 1,548 unique patients. Eight hundred sixty-nine survey responses (56.14%) were from encounters with female patients (four were missing sex information). Patients had a mean age of 53.22 (± 17.93), with a range of 18–95 years. Patients self-reported their race as follows: 1,112 (71.83%) White, 218 (14.08%) Black or African American, 38 (2.45%) Asian, 4 (0.26%) American Indian or Alaskan Native, and 176 (11.37%) other or not reported. Forty (2.58%) patients self-reported Hispanic Latino ethnicity. The majority of the visits (79%) were with established patients known to the providers. Fewer examinations in new patients were rated as “none” for the elements of the examination that might have changed the assessment and plan if performed in person. New and established visits were rated similarly when combining good/excellent or same/better responses, but excellent and better ratings favored established patients.
Technical Aspects Summary
Bluejeans, the UPHS preferred platform during the pandemic, was the videoconference platform used most frequently (1,534 encounters, 95.70% of survey responses). Use of FaceTime (33, 2.06%), Doximity (33, 2.06%), Vidyo (1, 0.06%), and Zoom (1, 0.06%) was also reported. Platform was missing for one survey. Patient device was reported for 1,144 survey responses. Smart phones (654 encounters, 57.17% of survey responses) and laptops (319, 27.88%) were the most commonly used devices; tablet computer (110, 9.62%) and desktop (61, 5.33%) were reported less frequently. The video quality was adequate for 1,461 (91.14%) of encounters, while audio quality was adequate for 1,482 (92.45%) of encounters. The time required to complete each visit compared with an in-person examination was reported to be less in 32.56% (522 responses), the same in 61.63% (988 responses), and more in 5.80% (93 responses). Incomplete encounters, encounters converted from videoconference to telephone, and cancelled appointments due to technical problems were not captured in this survey.
Provider Impression Survey
Providers' overall impressions are summarized in Figure 1B, and impressions about the teleneurology examination are summarized in Figure 1C. The quality of the provider–patient relationship during the teleneurology encounter was reported to be excellent in 522 (32.56%), good in 960 (59.89%), fair in 106 (6.61%), and poor in 15 (0.94%) of encounters. Compared with an in-person visit, the overall telemedicine experience was rated as better in 187 (11.67%), same in 1,173 (73.18%), and worse in 243 (15.16%) encounters. Responses based on patient and provider characteristics are shown in Tables 2–5. The frequency of “None” in response to “What elements of the examination might have changed your assessment and plan if performed in person?” differed based on providers' years of experience [χ2(3, 1603) = 33.58, p = 0.000], visit type [χ2(1, 1598) = 14.80, p = 0.004], and division [χ2 (9, 1603) = 90.34, p = 0.000]. Post hoc analyses by observing standardized (Pearson) residuals demonstrate lower “None” responses (standardized residual <−2) among providers with 20–29, and ≥30 years of experience, for new patient visits, and among General and Neuromuscular division visits. Higher “None” responses (standardized residual >2) were identified for providers with 10–19 years of experience, and among MS, Headache, Movement Disorders, and Cognitive division visits (Table 2). The frequency of “Same” or “Better” overall experience with telemedicine compared with in-person visits differed based on providers' years of experience [χ2 (3, 1603) = 10.42, p = 0.015], with lower “Same” or “Better” responses in the 20–29 years of experience group (Table 5). No additional significant effects of provider years of experience, month of visit, visit type, or division groups on the history, ability to personally connect, provider–patient relationship, or overall experience were detected.
Table 2.
Provider Impressions About the History and Examination During Teleneurology Encounters
| NO. OF RESPONSES | HISTORY |
EXAMINATION |
|
|||
|---|---|---|---|---|---|---|
| WORSE, n (%) | SAME, n (%) | BETTER, n (%) | NONE,a n (%) | |||
| Whole cohort | 1,603 | 54 (3) | 1,502 (94) | 47 (3) | 1,095 (68) | |
| Provider experience, years | ||||||
| <10 | 646 | 15 (2) | 613 (95) | 18 (3) | 460 (71) | * |
| 10–19 | 438 | 6 (1) | 419 (96) | 13 (3) | 362 (83) | |
| 20–29 | 272 | 14 (5) | 255 (94) | 3 (1) | 150 (55) | |
| ≥30 | 247 | 19 (8) | 215 (87) | 13 (5) | 123 (50) | |
| Month of encounter | ||||||
| April | 126 | 4 (3) | 116 (92) | 6 (5) | 90 (71) | |
| May | 707 | 29 (4) | 655 (93) | 23 (3) | 490 (69) | |
| June | 619 | 20 (3) | 585 (95) | 14 (2) | 406 (66) | |
| July | 131 | 1 (1) | 126 (96) | 4 (3) | 93 (71) | |
| August | 20 | 0 (0) | 20 (100) | 0 (0) | 16 (80) | |
| Visit typeb | ||||||
| New/consult | 339 | 13 (4) | 317 (94) | 9 (3) | 180 (53) | * |
| Established | 1,259 | 41 (3) | 1,180 (94) | 38 (3) | 913 (73) | |
| Division | ||||||
| MS | 266 | 8 (3) | 251 (94) | 7 (3) | 217 (82) | * |
| Headache | 94 | 2 (2) | 88 (94) | 4 (4) | 86 (91) | |
| Vascular | 160 | 11 (7) | 148 (93) | 1 (1) | 92 (58) | |
| Movement | 404 | 7 (2) | 382 (95) | 15 (4) | 345 (85) | |
| Cognitive | 106 | 2 (2) | 104 (98) | 0 (0) | 90 (85) | |
| General | 358 | 18 (5) | 326 (91) | 14 (4) | 168 (47) | |
| NM | 120 | 1 (1) | 119 (99) | 0 (0) | 36 (30) | |
| Epilepsy | 37 | 4 (11) | 32 (86) | 1 (3) | 19 (51) | |
| Residents | 49 | 1 (2) | 44 (90) | 4 (8) | 35 (71) | |
| Otherc | 9 | 0 (0) | 8 (89) | 1 (11) | 7 (78) | |
Numbers represent number of responses (percentage of group).
None refers to N (%) of those responding “None” to the question “In evaluating this patient, what elements of the examination might have changed your assessment and plan if performed in person?” Established includes nonbilled visits.
N = 1,598 due to missing billing code data.
Other includes neuro-ophthalmology and traumatic brain injury due to low response rate among these divisions.
Indicates p < 0.05 on omnibus χ2 test.
MS, multiple sclerosis and neuroimmunology; NM, neuromuscular.
Table 3.
Provider Impressions About the Ability to Personally Connect During Teleneurology Encounters
| ABILITY TO PERSONALLY CONNECT |
|||
|---|---|---|---|
| WORSE, n (%) | SAME, n (%) | BETTER, n (%) | |
| Whole cohort | 148 (9) | 1,382 (86) | 73 (5) |
| Provider experience, years | |||
| <10 | 39 (6) | 580 (90) | 27 (4) |
| 10–19 | 27 (6) | 388 (89) | 23 (5) |
| 20–29 | 39 (14) | 229 (84) | 4 (1) |
| ≥30 | 43 (17) | 185 (75) | 19 (8) |
| Month of encounter | |||
| April | 17 (13) | 98 (78) | 11 (9) |
| May | 70 (10) | 596 (84) | 41 (6) |
| June | 52 (8) | 557 (90) | 10 (2) |
| July | 8 (6) | 112 (86) | 11 (8) |
| August | 1 (5) | 19 (95) | 0 (0) |
| Visit typea | |||
| New/consult | 32 (9) | 297 (88) | 10 (3) |
| Established | 115 (9) | 1,082 (86) | 62 (5) |
| Division | |||
| MS | 9 (3) | 247 (93) | 10 (4) |
| Headache | 5 (5) | 86 (91) | 3 (3) |
| Vascular | 21 (13) | 137 (86) | 2 (1) |
| Movement | 34 (8) | 343 (85) | 27 (7) |
| Cognitive | 12 (11) | 94 (89) | 0 (0) |
| General | 33 (9) | 303 (85) | 22 (6) |
| NM | 24 (20) | 96 (80) | 0 (0) |
| Epilepsy | 6 (16) | 30 (81) | 1 (3) |
| Residents | 4 (8) | 41 (84) | 4 (8) |
| Otherb | 0 (0) | 5 (56) | 4 (44) |
Numbers represent number of responses (percentage of group).
N = 1,598 due to missing billing code data.
Other includes neuro-ophthalmology and traumatic brain injury combined due to low response rate among these divisions. Established includes nonbilled visits. No. of responses per group is found in Table 3. No significant differences in the frequencies of those reporting the same or better were detected within groups for all provider or patient characteristics.
Table 4.
Provider Impressions About the Quality of the Provider–Patient Relationship During Teleneurology Encounters
| QUALITY OF THE PROVIDER–PATIENT RELATIONSHIP |
||||
|---|---|---|---|---|
| POOR, n (%) | FAIR, n (%) | GOOD, n (%) | EXCELLENT, n (%) | |
| Whole cohort | 15 (1) | 106 (7) | 960 (60) | 522 (33) |
| Provider experience, years | ||||
| <10 | 4 (1) | 39 (6) | 333 (52) | 270 (42) |
| 10–19 | 2 (<1) | 28 (6) | 349 (80) | 59 (13) |
| 20–29 | 3 (1) | 23 (8) | 165 (61) | 81 (30) |
| ≥30 | 6 (2) | 16 (6) | 113 (46) | 112 (45) |
| Month of encounter | ||||
| April | 1 (1) | 13 (10) | 66 (52) | 46 (37) |
| May | 9 (1) | 50 (7) | 418 (59) | 230 (33) |
| June | 3 (<1) | 38 (6) | 389 (63) | 189 (31) |
| July | 2 (2) | 4 (3) | 72 (55) | 53 (40) |
| August | 0 (0) | 1 (3) | 15 (42) | 4 (11) |
| Visit typea | ||||
| New/consult | 4 (1) | 20 (6) | 228 (67) | 87 (26) |
| Established | 11 (1) | 85 (7) | 729 (58) | 434 (34) |
| Division | ||||
| MS | 2 (1) | 5 (2) | 88 (33) | 171 (64) |
| Headache | 0 (0) | 5 (5) | 78 (83) | 11 (12) |
| Vascular | 3 (2) | 23 (14) | 80 (50) | 54 (34) |
| Movement | 1 (<1) | 20 (5) | 291 (72) | 92 (23%) |
| Cognitive | 2 (2) | 17 (16) | 82 (77) | 5 (5) |
| General | 4 (1) | 15 (4) | 205 (57) | 134 (37) |
| NM | 3 (3) | 15 (13) | 98 (82) | 4 (3) |
| Epilepsy | 0 (0) | 5 (14) | 21 (57) | 11 (30) |
| Residents | 0 (0) | 1 (2) | 16 (33) | 32 (65) |
| Otherb | 0 (0) | 0 (0) | 1 (11) | 8 (89) |
Numbers represent number of responses (percentage of group).
N = 1,598 due to missing billing code data.
Other includes neuro-ophthalmology and traumatic brain injury combined due to low response rate among these divisions. Established includes nonbilled visits. No. of responses per group is found in Table 3. No significant differences in the frequencies of those reporting good or excellent were detected within groups for all provider or patient characteristics.
Table 5.
Provider Impressions About the Overall Teleneurology Experience
| OVERALL EXPERIENCE |
||||
|---|---|---|---|---|
| WORSE, n (%) | SAME, n (%) | BETTER, n (%) | ||
| Whole cohort | 243 (15) | 1,173 (73) | 187 (12) | |
| Provider experience, years | ||||
| <10 | 67 (10) | 503 (78) | 76 (12) | * |
| 10–19 | 42 (10) | 338 (77) | 58 (13) | |
| 20–29 | 75 (28) | 190 (70) | 7 (3) | |
| ≥30 | 59 (24) | 142 (57) | 46 (19) | |
| Month of encounter | ||||
| April | 22 (17) | 87 (69) | 17 (13) | |
| May | 118 (17) | 492 (70) | 97 (14) | |
| June | 88 (14) | 476 (77) | 55 (9) | |
| July | 14 (11) | 103 (79) | 14 (11) | |
| August | 1 (5) | 15 (75) | 4 (20) | |
| Visit typea | ||||
| New/consult | 59 (17) | 256 (76) | 24 (7) | |
| Established | 187 (15) | 914 (73) | 162 (13) | |
| Division | ||||
| MS | 11 (4) | 225 (85) | 30 (11) | |
| Headache | 8 (9) | 77 (82) | 9 (10) | |
| Vascular | 30 (19) | 114 (71) | 16 (10) | |
| Movement | 44 (11) | 312 (77) | 48 (12) | |
| Cognitive | 22 (21) | 65 (61) | 19 (18) | |
| General | 66 (18) | 245 (68) | 47 (13) | |
| NM | 48 (40) | 71 (59) | 1 (1) | |
| Epilepsy | 8 (22) | 27 (73) | 2 (5) | |
| Residents | 6 (12) | 32 (65) | 11 (22) | |
| Otherb | 0 (0) | 5 (56) | 4 (44) | |
Numbers represent number of responses (percentage of group).
N = 1,598 due to missing billing code data.
Other includes neuro-ophthalmology and traumatic brain injury combined due to low response rate among these divisions. Established includes nonbilled visits. No. of responses per group is found in Table 3. No significant differences in the frequencies of those reporting the same or better were detected within groups for all provider or patient characteristics.
Indicates p < 0.05 on omnibus χ2 test.
Discussion
We surveyed the experience of providers performing teleneurology visits during the SARS CoV-2 pandemic when most patient interactions in our department were performed by telemedicine. In addition to overall impressions and technical adequacy, we asked providers to compare the ability to connect with patients and the quality of the relationship with the patient to in-person visits. We also asked whether any limitations of the neurological examination by telemedicine might have altered the impression or plan. We had a wide representation of providers across the neurology department encompassing multiple subspecialties. Surveys were completed by physicians, nurse practitioners, residents, and genetic counselors.
Several previous surveys of teleneurology encounters focused on patient experiences or provider satisfaction.19–22 We were unable to find any surveys that asked about elements of the neurological examination or focused on the patient–provider interaction. Many surveys assessed global impressions of teleneurology in a one-time questionnaire in contrast to our survey that was completed after each individual encounter.23,24 A recent study of 2,589 telehealth encounters including 2,093 audio/video visits in a pediatric neurology population assessed provider and patient satisfaction using questions embedded in an electronic medical record (EMR) template. Telemedicine was considered satisfactory by providers in 93% of encounters, and an in-person assessment was thought to be required in 5%.25
The inability to perform a complete examination and to connect with patients is often reason physicians resist using telemedicine. Our survey sought to examine these issues in a wide spectrum of neurological conditions. Since almost all patients regardless of diagnosis were seen by telemedicine during this time interval, patients were not selected for telemedicine based on their appropriateness for a remote visit.
Several interesting trends emerged from the survey results. First, 68% of responses stated that none of the elements of the neurological examination if performed in person would have changed the assessment and plan. This suggests that in the majority of cases, the neurological examination by telemedicine is adequate for diagnosis and management decisions. The motor (15%) and reflex (18%) components of the examination were most frequently considered suboptimal. Utility of the teleneurology examination can be influenced by patient, provider, or visit characteristics and may differ between subspecialties. Indeed, these results demonstrate that during new patient encounters, General or Neuromuscular division visits, or visits with more experienced providers, providers more frequently reported that an in-person examination may have influenced the plan. Less experienced providers and during MS, Headache, Movement Disorders, or Cognitive division visits, providers were more likely to report that an in-person examination would not have influenced the plan. Reflexes are not typically performed by telemedicine, and it is difficult to quantify the motor examination. Although the examination was considered to be adequate in roughly two-thirds of encounters, the remaining one-third were still considered insufficient. For those cases, the teleneurology examination could serve as a screen with close in-person follow-up. In addition, simple maneuvers could improve the teleneurology examination. For instance, lifting household objects as weights has been suggested to remotely grade strength.26 Although the examination was considered inadequate in one-third of teleneurology encounters, further analysis could identify patients better served by an in-person examination and may lead to innovative approaches to teleneurology. The results of this survey suggest the motor and sensory examinations as areas where novel techniques for improving remote examinations might be focused.
Second, the quality of the provider–patient interaction and ability to connect with the patient were considered the same or better than an in-person examination in almost all encounters. This finding contradicts the impression that telemedicine encounters are impersonal, and lack empathy and compassion. Telemedicine visits at home may provide insights into patient activities and personalities not available during office visits. Patients were noted to demonstrate artwork or play music for their physician, which would be unlikely to occur in person. Third, the time for teleneurology visits was assessed to be the same or less than an in-person visit in 94% of encounters, dispelling the impression that the technology adds complexity and reduces efficiency. Fourth, established patient visits were rated better than in-person or excellent at a higher frequency than new patient visits across most questions asked, suggesting that established visits might be a more effective use of teleneurology.
Finally, the overall experience was the same as an in-person visit in 73% of visits and better in 12%. Assessment of the visit as the same or better than an in-person visit increased from 83% in April to 89% in July, suggesting that satisfaction with teleneurology visits increased with experience. Teleneurology may not be equally applicable across neurology subspecialties; Headache (91%), MS (96%), and Movement Disorder (89%) providers had the most ratings of same or better overall experience and Neuromuscular providers the lowest (60%), although there was not a significant group effect of division on the omnibus statistical tests.
There are several limitations to our survey. The survey instrument was written by the investigators and has not been validated. Although 55 providers completed surveys, 43% were submitted by the top 5 providers. There may be a sampling bias with providers with favorable views of teleneurology being more likely to complete the survey. In addition, the response rate was 10%, suggesting that these results may not be representative of the entire neurology department. The responses represent the opinion of the provider, and there were no objective measures of visit time or audio/video quality. The survey was completed only for successful audiovisual telemedicine visits and not audio-only or cancelled encounters due to technology failures. In addition, our center redeployed research staff to conduct previsit technology checks to improve compliance with the teleneurology platform. These may have influenced the responses to technical aspects of these visits. Responses were not collected for incomplete encounters, including those that were unsuccessful due to technical challenges. Surveys were not completed for all teleneurology visits during this interval, and the opinions of those not completing surveys may differ from those submitted. Finally, these data were collected during the SARS CoV-2 pandemic when the alternatives to teleneurology were delaying visits, cancelling visits, or completing visits in person with the added risk of infectious transmission. These factors may have influenced providers' opinions about teleneurology visits.
Our survey represents one of the largest samples of provider impressions of teleneurology across a multispecialty adult neurology department during a period when almost all outpatient encounters were performed by telemedicine. The results indicate substantial overall satisfaction and technical adequacy. In most cases the ability to connect with patients was at least the same as in-person visits. The most frequent components of the neurological examination that might have changed the impression or plan if performed in person were the motor and reflex examinations; however in 68% of visits, no examination elements would have altered the conclusions. Further analysis of differences between subspecialties, provider experience, and diagnosis should help define the optimal utilization of telemedicine when both in-person and remote options are equally available.
Conclusion
Provider impressions about the teleneurology history, examination, and provider–patient relationship are generally favorable in this quality-improvement study. These findings are important for the widespread use of teleneurology during the SARS CoV-2 pandemic and for the development of teleneurology applications in the future. Differences between provider responses based on specialty and visit characteristics suggest areas for future research to improve the teleneurology experience.
Acknowledgments
We thank our patients and all of the neurology providers in the Department of Neurology for their generous time in completing questions, and Christina Blum, MD for her assistance in creating the teleneurology provider survey.
Authors' Contributions
All authors were involved in the design and conceptualization of the study. Drs Tropea, Fuentes, Roberts, and Wechsler were involved in data analysis. Dr. Tropea drafted the article. All authors reviewed and edited the article. Dr Wechsler finalized the article.
Data Availability Statement
Data will be made available by requesting access from the corresponding author.
Disclosure Statement
No competing financial interests exist.
Funding Information
Funding was provided by the Department of Neurology, Perelman School of Medicine, University of Pennsylvania. Dr Tropea is additionally funded by the NINDS (K23-NS114167-01A1).
References
- 1. Hatcher-Martin JM, Adams JL, Anderson ER, et al. Telemedicine in neurology: Telemedicine Work Group of the American Academy of Neurology update. Neurology 2020;94:30–38. [DOI] [PubMed] [Google Scholar]
- 2. George BP, Reminick JI, Scoglio NJ, et al. Telemedicine in leading US Neurology Departments. Neurohospitalist 2012;2:123–128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Wechsler LR, Demaerschalk BM, Schwamm LH, et al. Telemedicine quality and outcomes in stroke: A scientific statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2017;48:e3–e25. [DOI] [PubMed] [Google Scholar]
- 4. Schwamm LH, Rosenthal ES, Hirshberg A, et al. Virtual TeleStroke support for the emergency department evaluation of acute stroke. Acad Emerg Med 2004;11:1193–1197. [DOI] [PubMed] [Google Scholar]
- 5. Amorim E, Shih MM, Koehler SA, et al. Impact of telemedicine implementation in thrombolytic use for acute ischemic stroke: The university of pittsburgh medical center telestroke network experience. J Stroke Cerebrovasc Dis 2013;22:527–531. [DOI] [PubMed] [Google Scholar]
- 6. Müller-Barna P, Hubert GJ, Boy S, et al. Telestroke units serving as a model of care in rural areas: 10-year experience of the telemedical project for integrative stroke care. Stroke 2014;45:2739–2744. [DOI] [PubMed] [Google Scholar]
- 7. Zaidi SF, Jumma MA, Urra XN, et al. Telestroke-guided intravenous tissue-type plasminogen activator treatment achieves a similar clinical outcome as thrombolysis at a comprehensive stroke center. Stroke 2011;42:3291–3293. [DOI] [PubMed] [Google Scholar]
- 8. Wilcox ME, Adhikari NKJ. The effect of telemedicine in critically ill patients: Systematic review and meta-analysis. Crit Care 2012;16:R127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Bahrani K, Singh MB, Bhatia R, et al. Telephonic review for outpatients with epilepsy—A prospective randomized, parallel group study. Seizure 2017;53:55–61. [DOI] [PubMed] [Google Scholar]
- 10. Dorsey ER, Deuel LM, Voss TS, et al. Increasing access to specialty care: A pilot, randomized controlled trial of telemedicine for Parkinson's disease. Mov Disord 2010;25:1652–1659. [DOI] [PubMed] [Google Scholar]
- 11. Müller KI, Alstadhaug KB, Bekkelund SI. Acceptability, feasibility, and cost of telemedicine for nonacute headaches: A randomized study comparing video and traditional consultations. J Med Internet Res 2016;18:e140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Kane RL, Bever CT, Ehrmantraut M, Forte A, Culpepper WJ, Wallin MT. Teleneurology in patients with multiple sclerosis: EDSS ratings derived remotely and from hands-on examination. J Telemed Telecare 2008;14:190–194. [DOI] [PubMed] [Google Scholar]
- 13. Marshall KR, Holland SL, Meyer KS, Martin EM, Wilmore M, Grimes JB. Mild traumatic brain injury screening, diagnosis, and treatment. Mil Med 2012;177(8 Suppl):67–75. [DOI] [PubMed] [Google Scholar]
- 14. Kim H, Jhoo JH, Jang JW. The effect of telemedicine on cognitive decline in patients with dementia. J Telemed Telecare 2017;23:149–154. [DOI] [PubMed] [Google Scholar]
- 15. Wechsler LR. Advantages and limitations of teleneurology. JAMA Neurol 2015;72:349–354. [DOI] [PubMed] [Google Scholar]
- 16. Spindler M, Jacobs D, Yuan K, et al. A department approach to teleneurology. Telemed Ehealth 2020;27:1078–1084. [DOI] [PubMed] [Google Scholar]
- 17. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Sharpe D. Your chi-square test is statistically significant: Now what? Pract Assess Res Eval 2015;20. [Google Scholar]
- 19. Hanson RE, Truesdell M, Stebbins GT, Weathers AL, Goetz CG. Telemedicine vs office visits in a movement disorders clinic: Comparative satisfaction of physicians and patients. Mov Disord Clin Pract 2019;6:65–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Davis LE, Harnar J, Lachey-Barbee LA, Pirio Richardson S, Fraser A, King MK. Using teleneurology to deliver chronic neurologic care to rural veterans: Analysis of the first 1,100 patient visits. Telemed J E Health 2019;25:274–278. [DOI] [PubMed] [Google Scholar]
- 21. Mammen JR, Elson MJ, Java JJ, et al. Patient and physician perceptions of virtual visits for Parkinson's disease: A qualitative study. Telemed J E Health 2018;24:255–267. [DOI] [PubMed] [Google Scholar]
- 22. Craig J, Russell C, Patterson V, Wootton R. User satisfaction with realtime teleneurology. J Telemed Telecare 1999;5:237–241. [DOI] [PubMed] [Google Scholar]
- 23. Kennedy C, Johnston K, Taylor P, Murdoch IE. Determining clinician satisfaction with telemedicine. J Telemed Telecare 2003;9(Suppl 1):S62–S64. [DOI] [PubMed] [Google Scholar]
- 24. Hopp F, Whitten P, Subramanian U, Woodbridge P, Mackert M, Lowery J. Perspectives from the Veterans Health Administration about opportunities and barriers in telemedicine. J Telemed Telecare 2006;12:404–409. [DOI] [PubMed] [Google Scholar]
- 25. Rametta SC, Fridinger SE, Gonzalez AK, et al. Analyzing 2,589 child neurology telehealth encounters necessitated by the COVID-19 pandemic. Neurology 2020;95:e1257–e1266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Grossman SN, Han SC, Balcer LJ, et al. Rapid implementation of virtual neurology in response to the COVID-19 pandemic. Neurology 2020;94:1077–1087. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data will be made available by requesting access from the corresponding author.