Analyzing 2,589 child neurology telehealth encounters necessitated by the COVID-19 pandemic

Abstract

Objective

To assess the rapid implementation of child neurology telehealth outpatient care with the onset of the coronavirus disease 2019 (COVID-19) pandemic in March 2020.

Methods

This was a cohort study with retrospective comparison of 14,780 in-person encounters and 2,589 telehealth encounters, including 2,093 audio-video telemedicine and 496 scheduled telephone encounters, between October 1, 2019 and April 24, 2020. We compared in-person and telehealth encounters for patient demographics and diagnoses. For audio-video telemedicine encounters, we analyzed questionnaire responses addressing provider experience, follow-up plans, technical quality, need for in-person assessment, and parent/caregiver satisfaction. We performed manual reviews of encounters flagged as concerning by providers.

Results

There were no differences in patient age and major ICD-10 codes before and after transition. Clinicians considered telemedicine satisfactory in 93% (1,200 of 1,286) of encounters and suggested telemedicine as a component for follow-up care in 89% (1,144 of 1,286) of encounters. Technical challenges were reported in 40% (519 of 1,314) of encounters. In-person assessment was considered warranted after 5% (65 of 1,285) of encounters. Patients/caregivers indicated interest in telemedicine for future care in 86% (187 of 217) of encounters. Participation in telemedicine encounters compared to telephone encounters was less frequent among patients in racial or ethnic minority groups.

Conclusions

We effectively converted most of our outpatient care to telehealth encounters, including mostly audio-video telemedicine encounters. Providers rated the vast majority of telemedicine encounters to be satisfactory, and only a small proportion of encounters required short-term in-person follow-up. These findings suggest that telemedicine is feasible and effective for a large proportion of child neurology care. Additional strategies are needed to ensure equitable telemedicine use.

In response to the coronavirus disease 2019 (COVID-19) pandemic, there was a rapid and unprecedented conversion of outpatient clinical care delivery from in-person to remote telehealth services,¹ including audio-visual telemedicine encounters and scheduled telephone encounters. Telehealth is typically defined as the use of a broad range of telecommunications technologies to support long-distance clinical health care,² whereas telemedicine refers more narrowly to remote health care services that include audio and video equipment.^3,4 Telemedicine in neurology care has been reported primarily in adult stroke care, rural health systems, and specific disease populations such as epilepsy and headache.^5–11 Reports of pediatric neurology telemedicine delivery have been limited to programs for underserved populations.^12,13

Here, we analyzed the implementation of telehealth services in our large pediatric neurology care network, including scheduled telephone encounters and newly implemented audio-video telemedicine encounters. Because audio-video telemedicine encounters represented a new modality of patient care, we performed a quality improvement study to determine the effectiveness of telemedicine encounters as assessed by the clinicians, the utility of telemedicine for future care, the need for short-term in-person follow-up, and patient/caregiver satisfaction with the telemedicine experience. We expected that these data would allow us to make quick adjustments during the crisis and to determine how to incorporate telehealth services, including audio-video telemedicine encounters, into postpandemic pediatric neurology care.

Methods

Standard protocol approvals, registrations, and patient consents

This was a quality improvement initiative that did not require review by an Internal Review Board. We applied Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting standards.¹⁴

Setting

This study was performed by the Division of Neurology at Children's Hospital of Philadelphia, a pediatric specialty care network composed of an urban quaternary care hospital, an ambulatory center, and an additional 8 satellite locations offering child neurology care. The practice includes 55 child neurologists, 7 advanced practice providers, and 21 graduate medical trainees. The program provides 32,000 outpatient encounters per year across general child neurology and specialty programs in epilepsy, neuromuscular, neuroimmune disorders, brain protection, stroke, neurocardiac care, fetal and neonatal neurology, headache, cognitive/behavioral neurology, leukodystrophy, and Friedreich ataxia. All clinical care is documented with the Epic (Verona, WI) electronic medical record (EMR) system. Before the pandemic, our practice offered no outpatient telehealth services. All providers in the practice were licensed in the state of Pennsylvania. During the pandemic, all providers received emergency licenses for the care of children in New Jersey through telehealth services.

Implementation of telehealth, including audio-video telemedicine and scheduled telephone encounters

On March 9, 2020, our institution suspended all nonurgent in-person neurology office visits due to regional adoption of social distancing measures at the onset of the COVID-19 pandemic. New patient referrals were screened by a physician, and a small number were seen in person in a dedicated urgent clinic. Most new and all established patients were scheduled for audio-video telemedicine encounters. Established patients who lacked access to a smartphone or computer application required to enable telemedicine encounters were scheduled for structured (audio-only) telephone encounters. Unscheduled telephone calls by a provider in response to a patient message were not considered telephone encounters, and they were not included in our analyses. Providers used the telemedicine software embedded in the Epic EMR to conduct telemedicine encounters. Telephone encounters were conducted with a phone.

Coordinators instructed patients or caregivers regarding the download of the patient portal smartphone application required for telemedicine encounters when appointments were scheduled. One day before the encounter, a nursing assistant called the family or caregivers to provide additional instructions about the encounter procedure to increase the likelihood of successfully accessing the patient portal. Families were instructed to initiate the encounter 15 minutes before the scheduled time to sign consents and to verify allergies, medications, and the active problem list. If patients did not successfully connect to the encounter within 5 minutes of the scheduled encounter time, then staff reached out to patients to troubleshoot. Neurology providers designated as EMR superusers trained all providers to perform telemedicine and telephone encounters through a narrated slide presentation or a one-on-one tutorial that included a simulated encounter. Providers reviewed resources on conducting video telemedicine neurologic examinations from the American Academy of Neurology.¹⁵

At the time of the encounter, providers accessed the system through a smartphone (Haiku) or tablet (Canto) application and completed documentation by accessing Epic on a computer with a remote desktop connection (Citrix Systems). Providers documented telemedicine and telephone encounters within the EMR using structured documentation templates that contained key medical data fields from typical in-person note templates. In addition, new fields were added, including the need for the encounter, identification of participants within the encounter, and appointment duration. Audio-video telemedicine encounters contained a template for a physical examination, which was not included in templates for telephone encounters.

Design of provider questionnaires embedded in telemedicine notes

To assess the effectiveness of telemedicine encounters, we embedded 5 multiple choice questions in the note templates. Because these questions were aimed at evaluating the quality of the newly established telemedicine encounters, these questions were not included in the telephone encounter template. The questions assessed (question 1) provider satisfaction, (question 2) follow-up plans using telemedicine, (question 3) presence of technical issues, (question 4) presence of concerns requiring sooner in-patient assessment, and (question 5) caregiver evaluation of the telemedicine encounter assessed by the provider. The question regarding caregiver evaluation was added to the survey ≈4 weeks after the initial survey deployment. Providers were asked to query families about their satisfaction with telemedicine at the conclusion of the encounter.

Data abstraction

Data from the Epic EMR were accessed via the Clarity database (Epic). We developed a dedicated EMR data extraction protocol that identified all outpatient encounters including the 3 categories of telehealth encounters (telemedicine new, telemedicine follow-up, and telephone follow-up). We extracted demographic variables, including age, sex, race, and ethnicity, for all encounters. We mapped patient zip codes to median household income (MHI) using 2018 US Census data.¹⁶ We retrieved the primary ICD-10 code for each encounter and used higher-level grouping of primary ICD-10 codes for analysis (e.g., G40 instead of G40.10). For the small minority of encounters with primary diagnoses mapping on 2 ICD-10 codes, we used the more common ICD-10 code for the analyses to allow grouping of common codes. We performed data extraction and analysis within an institutional Health Insurance Portability and Accountability Act–compliant framework. We developed a Natural Language Processing pipeline within Oracle SQL that detected the quality improvement question text within the full text and parsed the semistructured answers, including free-text options. We used the R analysis framework (R Foundation for Statistical Computing, Vienna, Austria) for data analysis and visualization.¹⁷ For our study, the unit of analysis was a patient encounter (not patient). We present data as descriptive statistics or comparisons using Fisher exact tests and Wilcoxon rank sum tests.

Analyses

We assessed all in-person encounters conducted from October 1, 2019 until March 15, 2020 and all telehealth encounters, including telemedicine encounters and telephone encounters, from March 16, 2020 until April 24, 2020. We used the term telemedicine to refer exclusively to patient encounters performed through the audio and video software embedded in the Epic EMR, while telehealth includes both telemedicine encounters and telephone encounters. The rationale for including both encounter types was that the combination of both encounter types reflects the overall care provided by our care network after cessation of in-person encounters and that telephone encounters included all components of a telemedicine encounter except visualization of the patient and remote physical examination. However, telemedicine encounters and telephone encounters are often considered conceptually different.^2,3 Therefore, we also compared in-person encounters with telemedicine encounters, excluding telephone encounters. Because telephone encounters included only follow-up encounters for established patients, we performed a separate analysis of follow-up encounters.

First, we compared in-person encounters with telehealth encounters to determine whether practice had changed. Second, within the telehealth cohort for return encounters, we compared patient and diagnosis variables between telemedicine and telephone encounters. Third, we compared telemedicine encounters for which providers did or did not complete the quality improvement questionnaires to determine whether the assessed cohort was representative of the overall population. Fourth, we assessed responses to the quality improvement questionnaire. For the analysis of provider satisfaction (question 1), the options “very satisfied” and “somewhat satisfied” were collapsed. For the analysis of follow-up plans (question 2), the options “Yes, I only need to see them in person if there is a new symptom or major change” and “Yes, but as a mix of telemedicine and in-person encounters” were collapsed. For the analysis of technical issues (question 3), answers to any option were analyzed jointly and separately. A binary choice (yes/no) was given to providers for questions regarding concerning features requiring in-person evaluation sooner than if the encounter had occurred in person (visits of concern, question 4) and provider-assessed caregiver reception of telemedicine encounters (question 5). Finally, a board-certified child neurologist manually assessed all encounters flagged as concerning by providers in the embedded questionnaire, including the reason for concern, primary diagnosis, documented follow-up plan, placement of orders, and disposition. The primary encounter diagnosis was used to compare frequency of diagnostic subgroups in the cohort with concerns to the full telemedicine cohort.

Data availability

Data in a deidentified format will be made available by request to the corresponding author.

Results

Recovery of patient volume by telehealth during the COVID-19 pandemic

Patient volume initially decreased but then recovered. Compared to baseline outpatient in-person clinical volume (average 610.75 encounters per week), patient volume during the initial 2 weeks of telehealth care decreased by 41% (average 247.5 encounters per week). However, for the next 2 weeks, the mean volume increased to 3% over baseline (average 626.5 encounters per week). Telemedicine encounters accounted for 80% of all telehealth encounters. Only 21 of 2,459 patient encounters (1%) were performed in person after transition to telehealth.

Patient demographics before and after telehealth transition

We compared patient demographics and spectrum of diagnoses between 14,780 in-person encounters and 2,589 telehealth encounters (including 2,093 telemedicine and 496 telephone encounters) between October 1, 2019 and April 24, 2020 (table 1 and figure 1). The median age was 11.6 and 11.4 years in the in-person and telehealth cohorts, respectively. The age distribution in both cohorts was virtually identical. Self-reported ethnicity and race were not different between the cohorts except for a small increase in the number of individuals self-reporting as multiple races in the telehealth cohort. MHI was identical between the cohorts. There were no differences in age, self-reported ethnicity, race, and MHI when the in-person cohort was compared to only the telemedicine component of the telehealth cohort (excluding telephone encounters). The ratio of new to established patients was higher in the telemedicine cohort compared to the broader telehealth cohort but still lower compared to the in-person cohort (645 of 2,093 [31%] vs 5,103 of 14,780 [35%], odds ratio [OR] 1.2, 95% confidence interval [CI] 1.1–1.3).

Table 1.

Demographic data of child neurology outpatient encounters before and after transition to telehealth in response to the COVID-19 pandemic

Figure 1. Transition of child neurology outpatient encounters in response to the COVID-19 pandemic.

COVID-19 = coronavirus disease 2019.

Diagnostic spectrum before and after telehealth

The most common primary diagnoses in the in-person and telehealth cohorts were epilepsy (G40) in 30% and migraine (G43) in 20% (figure 2). Epilepsy diagnoses were slightly more prevalent in the telehealth cohort than in the in-person cohort (telehealth 30% vs in-person 27%, OR 1.2, 95% CI 1.1–1.3). The proportion of patients with migraine was 20% in both cohorts. In an assessment of all 376 primary diagnoses seen in the overall cohort, 16 diagnoses were significantly different between the in-person and telehealth cohort. Diagnoses that were overrepresented in the telehealth cohort included metabolic disorders (E75, OR 3.3, 95% CI 1.4–7.2), while underrepresented diagnoses in the telehealth cohort included back pain (M54, OR 0, 95% CI 0–0.7), malaise and fatigue (R53, OR 0.15, 95% CI 0–0.9), and syncope (R55, OR 0.45, 95% CI 0.2–1.0). When the in-person cohort was compared to the telemedicine cohort (excluding telephone encounters), there were no differences in the frequency of epilepsy or migraine diagnoses. Metabolic disorders remained overrepresented in the telemedicine vs in-person cohort (E75, OR 4.1, 95% CI 1.8–8.9), while back pain was underrepresented (M54, OR 0, 95% CI 0–0.9).

Figure 2. Spectrum of diagnoses before and after transition to telehealth in response to the COVID-19 pandemic.

COVID-19 = coronavirus disease 2019; ICD-10 = International Classification of Diseases, 10th revision.

Provider and parent/caregiver assessments of telemedicine encounters

The provider replied to at least 1 question in the provider questionnaire in 63% (1,314 of 2,093) of telemedicine encounters. Questions were not included for telephone encounters. The patient encounters with questionnaire answers were representative of the overall telemedicine cohort (table 2). Providers indicated overall satisfaction with the telemedicine encounters in 93% (1,200 of 1,286) of encounters, including a subset of 60% (767 of 1,286) of encounters for which the providers were very satisfied with the encounters. Providers indicated that they would use telemedicine for at least a component of the follow-up plan for the patient in 89% (1,144 of 1,286) of encounters, including 38% (484 of 1,286) of patient encounters for which telemedicine could be used exclusively unless the patient had new symptoms or a clinical change. In 40% (519 of 1,314) of encounters, the technical quality was impaired, and the most frequent single causes affecting quality were poor audio (19%), poor video (13%), and interruption of the encounter (9%). In 5% of encounters, providers documented additional technical quality problems in free-text notes. After a patient/caregiver questionnaire was implemented, responses were provided for 217 of 559 encounters. Caregivers indicated an interest in telemedicine as part of future care for 86% (187 of 217) of encounters.

Table 2.

Responses to provider questionnaires

Evaluation of visits of concern

In 5% (65 of 1,285) of telemedicine encounters, the provider flagged the clinical scenario as concerning enough to necessitate in-person evaluation. These 65 encounters were evaluated further by chart review (table 3). Patient age in visits of concerns was not significantly different from that in visits without concern. Epilepsy (G40) was the most common primary diagnosis in visits of concern, but the frequency was not significantly different from the overall telemedicine or telehealth cohort. Migraine (G43) was significantly underrepresented in the visits of concern, while metabolic disorders (E75), facial nerve disorders (G51), sensory disturbance (R20), neuromuscular disorders (G71), and abnormal movements (R25) were significantly overrepresented. In all visits of concern, manual review revealed an adequate plan documented in the provider notes.

Table 3.

Manual chart review of encounters flagged to necessitate in-person evaluation

Comparison of telemedicine and telephone follow-up encounters

As a result of parent/caregiver preference or feasibility, 496 follow-up encounters were conducted as telephone encounters instead of telemedicine encounters. Given that telephone encounters were used only for follow-up of established patients, we compared demographics and patient diagnoses of the 496 telephone and 1,448 telemedicine follow-up encounters.

Patients with telephone encounters were more likely to be male compared to follow-up telemedicine encounters (56% male telephone encounter, 48% follow-up telemedicine encounter, OR 1.4, 95% CI 1.1–1.7). Age did not differ between the cohorts. Patients in racial or ethnic minority groups were evaluated by telephone encounters instead of telemedicine encounters more often than patients self-identified as white. Patients self-identified as black made up 21% of the group opting for telephone encounters compared to 11% of the group using telemedicine encounters (OR 2.2, 95% CI 1.7–3.0). Hispanic/Latino patients made up of 14% of telephone encounters compared to 9% of telemedicine encounters (OR 1.7, 95% CI 1.2–2.3). MHI was lower in patients evaluated by telephone encounters compared to telemedicine encounters ($72,373 MHI telephone vs $79,997 MHI telemedicine encounters, difference $10,656, 95% CI 7,509–13,723). Epilepsy (G40) was overrepresented in telephone encounters compared to telemedicine encounters (42% of telephone vs 35% of telemedicine encounters, OR 1.3, 95% 1.1–1.7). The frequency of patients with a primary diagnosis of migraine (G43) was 19% in both cohorts.

Given the differences in both cohorts, we also compared telephone encounters with the 9,677 in-person follow-up encounters, which showed similar results. Again, we did not observe age differences, but male (OR 1.3, 95% CI 1.1–1.6), black (OR 1.63, 95% CI 1.3–2.1), and Hispanic/Latino (OR 1.4, 95% CI 1.1–1.8) patients were overrepresented in telephone encounters compared to in-person follow-up encounters. In addition, we saw similar differences in MHI ($72,373 MHI telephone vs $78,540 MHI in-person follow-up encounters, difference $7,915, 95% CI 5,162–10,695) and epilepsy diagnoses (42% of telephone vs 33% of follow-up in-person encounters, OR 1.4, 95% CI 1.2–1.7). These results suggest that the differences between telephone encounters and telemedicine follow-up encounters can be attributed to changes in the patient population evaluated by telephone encounters rather than changes in the patient group evaluated by telemedicine.

Discussion

In this quality improvement study after rapid implementation of telehealth services for outpatient child neurology care, we made 5 key observations. First, conversion of outpatient care to telehealth encounters occurred across our patients with a similar distribution of demographic and clinical characteristics compared to prepandemic in-person encounters. Second, when using dedicated telemedicine services, providers reported that telemedicine was satisfactory for almost all encounters and that they would opt for ongoing use of telemedicine for most patients. Providers reported a high level of satisfaction despite technical issues in about one-third of encounters, suggesting that these issues did not substantially interfere with care delivery. Third, in a single basic measure, most parents/caregivers reported satisfaction with the telemedicine encounter. Fourth, urgent in-person evaluation was needed in a small percentage of patients. Fifth, access to telemedicine encounters compared to telephone encounters was lower in racial and ethnic minority groups, highlighting an inequity that must be addressed.

In the context of the COVID-19 pandemic, telemedicine as a subset of remote health care services that include audio and video equipment aligns with the 6 health care quality domains described by the Institute of Medicine.¹⁸ By providing a way to receive care without increasing the risk of pathogen exposure, telemedicine facilitates safe care. By allowing information exchange with patients and caregivers to inform medical decision-making, providers can offer effective care. Accessibility and convenience while maintaining high patient and family satisfaction allow patient-centered care. Telemedicine provides timely care by avoiding suspension or delays in care during a pandemic that requires social distancing, efficient care by reducing the burden on providers or families related to travel and time required for an in-person encounter, and equitable care by ensuring that we continue to meet the needs of our diverse patient population.

High levels of satisfaction with the telemedicine process in a practice where few providers had prior telemedicine experience suggest that this method of health care delivery is sustainable during and after the current pandemic. High levels of satisfaction despite frequent technical issues may have been influenced by lack of alternative methods in the setting of the COVID-19 pandemic, and infrastructural improvements to rapidly address technical issues are needed. These improvements may include software updates and bandwidth expansion given the massive increase in data traffic across the hospital's networks.

The vast majority of our providers indicated that they would continue to perform telemedicine encounters beyond the current pandemic if given the opportunity. This finding demonstrates that remote history taking and virtual examinations are effective for providing most child neurology care. In some instances, telemedicine may be able to remove barriers to care that result from in-person encounters. This benefit may be especially true for underserved patients whose caregivers cannot afford to miss work or travel to the clinic in person, who live far from our facilities, or who have complex transportation needs. However, this study uncovered disparities in the delivery of telemedicine care to patients in racial and ethnic minority groups, who received care in the same proportion as in-person encounters but were less likely to have access to the potentially more robust care that telemedicine encounters can provide compared to telephone encounters.

The need for rapid implementation of quality assurance measures alongside rapid implementation of telemedicine led to some notable limitations in our study. First, the provider questionnaire was completed in only 63% of telemedicine encounters. While the cohort with completed questionnaires was representative of the overall population of encounters, it is possible that providers who did not routinely complete the surveys in their telemedicine encounters were less technically savvy and therefore may have had differing opinions on the utility of telemedicine. Second, the survey questions used in the telemedicine encounters were not validated assessments of care effectiveness but rather offered targeted insights that will be used to implement future systems changes. Third, while our data suggest that most providers perceive telemedicine to be at least equivalent to in-person care for a variety of neurologic conditions, we cannot conclude that outcomes from telemedicine encounters are comparable to those of in-person encounters. Prospective studies of process measures and patient-centered outcome measures are needed to evaluate the effectiveness of child neurology telemedicine more robustly. Fourth, a measurement of patient satisfaction was carried out with only a single question asked by the clinician providing care, and this approach may influence answers toward a positive reply. Using electronic survey technology to gather anonymous satisfaction assessments may provide a less biased view in future studies.

While telemedicine encounters are often strictly defined as encounters with both audio and video components, we chose to include both audio-video telemedicine encounters and audio-only telephone encounters in our analyses. We opted for the inclusion of telephone encounters for 2 reasons. First, the combination of both telemedicine encounters and telephone encounters represented the total scheduled patient encounters provided by our care network. Therefore, the combination of both encounter types reflected the outpatient services provided more adequately than telemedicine encounters alone. Second, scheduled telephone encounters were structured and mirrored audio-video telemedicine encounters in all ways except remote physical examination and nonverbal communication. The rapid implementation of both encounter types occurred simultaneously, and we deemed both types of care as important within our care delivery model. Given the disparities among racial or ethnic groups uncovered by our analyses, further work is ongoing to address these differences.

Priority questions for future studies include determining whether previously documented benefits of telemedicine such as reduction in no-show burden,^12,19 in costly emergency department visits,⁶ or in miles traveled for patients^20,21 occur when telemedicine is implemented more broadly in child neurology practice. While many of our patients travel far for subspecialty care, those who live in our primary catchment region may also benefit from telemedicine child neurology services. In addition, evaluation of the impact of remote monitoring technologies, including seizure detection devices, long-term remote EEG monitoring, electronic pill boxes, and actigraphy might further bolster the future use of telehealth services beyond provider consultation.

We describe the successful implementation of telehealth services across all the subspecialties of a pediatric neurology program with a detailed evaluation of audio-video telemedicine encounters. We expect that further research into optimizing these technologies will show telemedicine to be even more valuable than demonstrated in our study. We recognize that implementation during the COVID-19 crisis was possible because legislators and payers quickly accommodated this approach, and we hope evidence of effectiveness and benefit across all 6 Institute of Medicine health care quality domains¹⁸ will help ensure that children with neurologic conditions have continued access to telemedicine care.

Glossary

CI

confidence interval

COVID-19

coronavirus disease 2019

EMR

electronic medical record

ICD-10

International Classification of Diseases, 10th revision

MHI

median household income

OR

odds ratio

STROBE

Strengthening the Reporting of Observational Studies in Epidemiology

Appendix. Authors

Study funding

No targeted funding reported.

Disclosure

The authors report no disclosures relevant to this manuscript. Go to Neurology.org/N for full disclosures.