Telehealth Use and Access to Neurology Outpatient Clinical Services for Children: An Observational Cohort Study

Abstract

Background

Access to neurology services is important for children’s well-being. We sought to evaluate the effects of telehealth on pedi­atric neurology appointment outcomes for children.

Methods

Using electronic health record (EHR) data, information about children with pediatric neurology appointments in a tertiary care children’s hospital in North Carolina was collected. Appointment outcomes (completion, cancellation, and no-show rates) were calculated for children who had a neurology appointment scheduled both in the pre-pandemic (March 10, 2019, to March 9, 2020) and pandemic (March 10, 2020, to March 9, 2021) periods. “Telehealth user” was a child who had at least one telehealth appointment scheduled between March 10, 2020, and March 9, 2021. In bivariate and multivariate analyses, we compared appointment outcomes for telehealth users and non-users.

Limitations

EHR data were limited to a single institution and could have missed children’s clinical encounters elsewhere. Since this study used secondary data that were not collected for research purposes, it is possible that there were unmeasured confounders. Because of the observational nature of the study, the association between telehealth use and appointment outcomes does not indicate causation.

Results

A total of 2110 children had 11,194 pediatric neurology appointments scheduled. Telehealth users compared with non-users were more likely to be White, non-Hispanic, have private insurance, and live farther from the children’s hospital. There was a statistically sig­nificant decrease in completion (66% versus 57%) and increase in cancellation (27% versus 33%) and no-show (7% versus 10%) rates in the pandemic when compared to the pre-pandemic period. In the pre-pandemic period, telehealth user and non-user groups had similar cancellation and no-show rates. In the pandemic period, there was a statistically significant decrease in cancellation and no-show rates in the telehealth user group when compared to the non-user group.

Conclusions

Pediatric neurology outpatient clinic visits decreased substantially during the pandemic. Telehealth mitigated these problems. Children belonging to a minority race/ethnicity are less likely to use telehealth, increasing their risk of poor access to neurology services.

Access to subspecialty services, including pediatric neurology services, is important for the health and well-being of children. However, pediatric specialty services are often not easily available.¹ Access to neurology care is particularly problematic for children of vulnerable social groups, or those living farther from tertiary care children’s hospitals.² These problems in access to specialist ser­vices worsened during the COVID-19 pandemic, threatening the health of children.³

Even before the pandemic, telehealth has been consid­ered as a solution to reduce problems in access to neurology services for children.^2,4 A systematic review of random­ized controlled trials showed telehealth was equal to usual care in clinical effectiveness and patient satisfaction and quality of life.⁵ Children living in rural areas who used pediatric neurology telehealth visits have lower emergency room (ER) visits and hospitalizations compared to those who received care through in-person visits.⁶ A recent study showed that pediatric neurology was one of the subspecial­ties with high telehealth use.³ Another study showed that telehealth use was higher for children compared to adults attending neurology clinics.⁷ However, there is a concern that telehealth services may not be available for children of vulnerable groups.⁸

At the beginning of the COVID-19 public health emer­gency, like many states in the country, North Carolina enacted policies to allow reimbursement of clinicians for telehealth services.⁹ This exponentially increased adop­tion of telehealth in North Carolina¹⁰ and provided an unprecedented opportunity to test whether use of telehealth had an effect on children’s access to neurology services. Therefore, the objective of our study is to evaluate whether children who used telehealth services had greater access to outpatient pediatric neurology services compared to those who did not use telehealth services. We used appointment outcome—completion, cancellation, and no-show rates—as a proxy for access to neurology services.

Methods

Setting

This study was conducted at Brenner Children’s Hospital (BCH), a tertiary care children’s hospital within the Atrium Health Wake Forest Baptist (AHWFB) system in Winston-Salem, North Carolina. The Institutional Review Board of Wake Forest University School of Medicine approved the study. We employed an observational cohort study design and used existing demographic and clinical encounter data for children available in the institution’s electronic health record (EHR). We followed the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) check­list for cohort studies to report our study.¹¹

Telehealth at BCH

Prior to March 2020, telehealth was not an option for children attending neurology clinics at BCH. In March 2020, in response to the COVID-19 pandemic, structures and pro­cesses to deliver telehealth services were created within the AHWFB system. The pediatric neurology department cre­ated a policy about encounters/conditions for which tele­health visits were appropriate for new patients. For return patient visits, appropriateness of telehealth was at the discretion of the neurology provider. Video visits were con­ducted using Doximity or through WebEx within the institu­tion’s EHR, Epic (Epic Systems, Verona, WI).

Study Cohort

We used a clinical administrative database to identify children who had at least one pediatric neurology outpa­tient appointment scheduled between March 10, 2019, and March 9, 2020, at BCH. Using children’s medical record numbers, we queried the EHR to identify children eligible for the study. Inclusion criteria were: age < 18 years as of March 10, 2019, residence in North Carolina, and at least one pedi­atric neurology appointment scheduled between March 10, 2020, and March 9, 2021.

Pediatric Neurology Outpatient Encounters

Outpatient encounters were initially identified using search terms for pediatric neurology services within BCH’s pediatrics and neurology departments. Encounters were then limited to the 37 types of outpatient encounters rel­evant to pediatric neurology and provided by one of the 18 pediatric neurology providers (nine attending physicians, three advanced practice providers, and six resident phy­sicians) practicing at BCH between March 10, 2019, and March 9, 2021.

Child-Level Data

Child-level data abstracted from the EHR included date of birth, race, ethnicity, health insurance type, and address. Encounter-level data included date of encounter, type of visit (in-person versus telehealth), and health insurance and address at the time of the encounter.

Encounter Data

An outpatient encounter was coded as a telehealth visit if any one of the 13 video or audio appointment codes was applied to the encounter. Data about outcome of appoint­ments (completed, cancelled, or no-show) were abstracted. Duplicate encounters (i.e., same type of encounter in a day) were excluded. When there was more than one type of encounter on the same day (e.g., completed and cancelled), we used the following inclusion rule: completed appointment was considered superior to cancelled appointment, and can­celled appointment superior to no-show appointment.

Variables

Age, race, ethnicity, health insurance type, rural/urban residence, and distance from BCH were the demographic variables. Race was categorized as White or Caucasian, Black or African American, and Other. The Other category included American Indian, Asian, Latin American, Native Hawaiian, and other. Ethnicity was categorized as Hispanic and non-Hispanic. Health insurance was categorized into public and private. Public insurance included Medicaid, Medicare, Tricare, and other government programs. Private insurance included commercial insurance and managed care.

Geocoding. We used child’s address for residence (rural versus urban) and to calculate distance to BCH. We used data from a database developed from the American Community Survey based on 2015 geocoding at census-tract level. Rurality score for each address (0 to 1, with 0 being most rural and 1 being most urban) was calculated. A rural­ity score of > 0.5 was defined as urban, and a score of ≤ 0.5 as rural. Distance from BCH was calculated using coordi­nates for BCH and the child’s address, using a custom-built function within the local database developed by the Wake Forest Center for Biomedical Informatics that uses variables from the American Community Survey.¹²

Telehealth use was the independent variable. “Telehealth user” was defined as a child who had at least one telehealth appointment scheduled between March 10, 2020, and March 9, 2021. Completion, cancellation, and no-show rates were the outcome variables and were calculated as the pro­portion of scheduled appointments that were completed, cancelled, and no-show, respectively. “Time period” was categorized into pre-pandemic (March 10, 2019, to March 9, 2020) and pandemic (March 10, 2020, to March 9, 2021).

Data Analysis

Race, ethnicity, health insurance, and rural/urban resi­dence were categorical variables. Age and household dis­tance from BCH were continuous variables. For bivariate analysis, we used demographic characteristics of children at the time of their first scheduled appointment and compared these characteristics between telehealth-user and non-user groups. We used chi-square tests for categorical variables and t-tests for continuous variables.

To compare completion, cancellation, and no-show rates between telehealth user and telehealth non-user, and between pre-pandemic and pandemic periods, we used paired t-tests. For multivariate analysis, we tested two models: one comparing cancellation versus completion and another comparing no-show versus completion rates. Both models included demographic variables, the inde­pendent variable (telehealth user versus non-user), and the time-period variable (pre-pandemic versus pandemic). Demographic variables in the model were at the encoun­ter level (time-dependent covariates). An interaction term between the time-period and telehealth-user variables was included in the models. In multivariate models, data for age, rural/urban status, distance from BCH, and health insurance at each encounter were used. Odds ratios for cancellation versus completion and no-show versus completion rates, adjusted for all other variables in the model, were calcu­lated. Stratified analysis was performed for the interaction term (time-period and telehealth-user). A p-value of < .05 was considered statistically significant. For multivariate models, Generalized Estimating Equations using R package geepack¹³ was used.

Results

Of the 5097 children who had a neurology appointment scheduled in the pre-pandemic period (March 10, 2019, to March 9, 2020), 2110 (41%) had an appointment sched­uled in the pandemic period (March 10, 2020, to March 9, 2021) as well and were included in the study (Figure 1). Demographic characteristics of children included in the study were as follows: mean age 9.8 years; 65% public insur­ance; 64% White; 14% Hispanic; 65% lived in an urban area; mean distance to the hospital 30 miles. Of these children, 1251 (59%) had at least one telehealth visit (telehealth user) and 859 (41%) had no telehealth visit (telehealth non-user) scheduled. These constituted 11,194 scheduled encounters.

Figure 1. Flow Diagram^a.

^aNumbers in parentheses indicate neurology encounters

Characteristics of Telehealth User versus Telehealth Non-User

Demographic characteristics of telehealth user and non-user are compared in Table 1. There was no difference between telehealth user and non-user in age, distance from the children’s hospital, and rural versus urban residence. Telehealth users were more likely to be White and non-His­panic, and to have private health insurance.

Table 1. Demographic Characteristics of Children Based on Telehealth Use.

^aTelehealth user is a child who had at least one telehealth appointment (video or audio) scheduled during the pandemic (3/10/20 to 3/9/21)

^bChi-square tests for categorical variables and t-tests for continuous variables comparing telehealth user and telehealth non-user groups

Appointment Outcomes in the Pre-Pandemic and Pandemic Periods

Overall, outcomes of appointments were as follows: 57% completed, 35% cancelled, and 8% no-showed. In the pan­demic compared to the pre-pandemic period, there was a statistically significant decrease in completion rate (66% versus 57%, p <.0001), and increase in cancellation rate (27% versus 33%, p < .0001) and no-show rate (7% versus 10%, p < .0001).

Completion rates were significantly higher during the pre-pandemic compared with the pandemic period for both groups: 67% versus 63% (p < .01) for telehealth users, respectively, and 66% versus 48% (p < .0001) for non-users, respectively (Table 2). Cancellation rates between the pre-pandemic and pandemic periods were not different for the telehealth user group (27% versus 28%, p = .6). However, the cancellation rate was significantly higher in the pan­demic period for the non-user group: pre-pandemic versus pandemic (26% versus 41%, p < .0001). The no-show rate increased significantly in the pandemic period compared to the pre-pandemic period for both groups: pre-pandemic ver­sus pandemic for the telehealth user group was 6% versus 10% (p < .0001); and for the non-user groups was 9% versus 11% (p = .02)

Table 2. Pediatric Neurology Appointment Outcomes for Telehealth Users and Non-Users in the Pre-Pandemic and Pandemic Periods.

^aTelehalth user is a child who had at least one telehealth appointment (video or audio) scheduled during the pandemic (3/10/20 to 3/9/21)

^bbased on t-tests

When comparing the telehealth users and non-users in the pre-pandemic period, there was no statistically signifi­cant difference between the groups in completion and can­cellation rates (Table 3). However, in the pandemic period, there was a statistically significant increase in completion rate and decrease in cancellation and no-show rates in the telehealth user group compared to the non-user group (Table 3).

Table 3. Multivariate Models Comparing Completed versus Cancelled and No-Show Neurology Visits for Children.

^aReference category

^bSignificant interaction term: Odds Ratios from stratified analysis for cancellation rates for telehealth user vs. telehealth non-user in the pre-pandemic period was 1.07 (p = 0.26) and in the pandemic period was 0.67 (p < .0001)

Multivariate Analysis

In the statistical model comparing cancellation ver­sus completion rates, age and time period were significant variables, after adjusting for other variables (Table 4). Increasing age was associated with decreased odds of can­cellation. The odds of cancellation were three times higher in the pandemic period compared with the pre-pandemic period. In the no-show versus completion rates model, age, race, health insurance, and distance from BCH were signifi­cant variables. With increasing age and distance from BCH, no-show rates decreased. Children of Black race compared with those of White race and those with public insurance compared to private insurance were twice as likely to have no-shows rather than completion. Ethnicity and rural/urban residence were not significant variables in either model.

Stratified analysis. The interaction term for telehealth-use and time-period variables was statistically significant for the cancellation versus completion rate model, but not for the no-show versus completion rate model. In Figure 2 we show the results of cancellation and no-show rates for telehealth users and non-users stratified by time period. Cancellation and no-show rates were not significantly different between telehealth users and non-users in the pre-pandemic period. However, in the pandemic period, cancellation rate was lower for telehealth user compared to non-user (34.7% versus 43.5%, p < .0001); no-show rate was lower for telehealth user compared to non-user (7.9% versus 10.6%, p =.01).

Figure 2. Cancellation and No-Show Rates of Telehealth User versus Non-User Stratified by Time Period (Pandemic versus Pre-Pandemic).

Discussion

Our study shows that children of minority race and ethnic­ity and those with public insurance were less likely to be tele­health users compared to their counterparts. Children who lived farther from the tertiary care children’s hospital were more likely to be telehealth users than those who lived closer. Completion rates for neurology appointments decreased sig­nificantly during the pandemic. We found telehealth users to have lower cancellation and no-show rates during the pan­demic compared to non-users. This suggests the potential usefulness of telehealth in mitigating access problems for pediatric neurology services resulting from the pandemic.

In our study, telehealth users were more likely to be White and non-Hispanic, have private insurance, and live farther from the children’s hospital. Our findings of lower telehealth use in children of minority race and ethnicity are consistent with prior studies.^3,14,15 Differences in tele­health use based on children’s race/ethnicity and insurance status are concerning, since they potentially widen the dis­parities in access to neurology services for these vulnerable groups. One possible reason for this is problems in access to the technology needed for telehealth, including broadband internet access.¹⁶ Disparities in telehealth use warrant further evaluation. Joshi and coauthors noted that telemedi­cine use was substantially lower for children living in rural communities.¹⁷ Interestingly, we did not find differences in telehealth use based on rural versus urban residence. On the other hand, children who lived farther from the children’s hospital were more likely to use telehealth in our study. Telehealth is an alternative to in-person visits when pediat­ric neurology services may be limited, such as for children living farther from a tertiary care children’s hospital.⁶

Our study showed that there was a significant drop in the number of children scheduled for a pediatric neurol­ogy visit during the pandemic compared to the year prior. Completion rates for pediatric neurology appointments also decreased substantially during the pandemic. Telehealth use helped mitigate these appointment problems. In North Carolina, telehealth use during the pandemic was associ­ated with less need for follow-up clinic visits, supporting the benefit of telehealth visits to mitigate health care access problems.¹⁰ Previous studies have also reported lower ER visits, hospitalizations, and mortality in children using tele­health for pediatric neurology outpatient services.^6,17 Additionally, parents and providers alike are satisfied with telehealth for pediatric neurology clinic visits.^14,18 As health insurance companies are considering scaling back provision of telehealth offered during the pandemic, policy­makers should consider the positive effect of telehealth in improving pediatric specialty access and health outcomes of children. It is encouraging that NC Medicaid has made tele­health flexibilities permanent.¹⁹ The effect of this policy change in improving access to specialty care for children should be evaluated in the future.

Cancellation and no-show rates increased significantly during the pandemic even after controlling for other fac­tors, including telehealth use. In a prior study, no-show rate increased in subspecialty clinics that adopted telehealth.³ In our study, while telehealth use had a marked effect in reducing cancellation rate during the pandemic, it was not as pronounced for reducing no-show rates. Age was an impor­tant factor in pediatric neurology appointment completion. As child’s age increased, both cancellation and no-show rates decreased, suggesting parents of younger children may have been fearful of bringing their children for clinic visits during the pandemic. While race and insurance were not factors in the cancellation versus completion rate multivariate model, Black race and public insurance were significant factors in the no-show versus completion rate model. Prior studies have noted the higher number of missed clinic appointments for vulnerable groups including minority race/ethnicity and those using public insurance.^20–22 Economic and logis­tic factors, mistrust in the health care system, and systemic racism have all been attributed as barriers to disadvan­taged populations attempting to access health care services.^8,15,21,23 The barriers to accessing pediatric neurology services faced by children in these vulnerable sociodemo­graphic groups need to be investigated further.

There are certain limitations to our study. EHR data were limited to a single institution and could have missed children’s clinical encounters elsewhere. The definition we used for telehealth user has not been validated. Since this study used secondary data, we did not have other sociode­mographic variables that could have affected appointment outcomes, such as transportation or parents’ preferred language. Since data were not collected for research pur­poses, it is possible that there were unmeasured confound­ers. Although race and ethnicity variables were associated with telehealth use and appointment outcomes, there are challenges in using race/ethnicity variables in EHR data as described before.²⁴ Finally, since this is an observational study, the association between telehealth use and appoint­ment outcomes do not indicate causation.

Conclusion

Completion rates for pediatric neurology appointments decreased substantially during the pandemic. However, telehealth use mitigated the problems with access to neu­rology services for children. Since children of minority race and ethnicity and those with public insurance are less likely to use telehealth, this puts them at increased risk for wors­ening access to neurology services when in-person visits are not available.

Disclosure of interests

No interests were disclosed.

Funding Statement

The project described was supported by the Wake Forest Clinical and Translational Science Institute (WF CTSI) Pilot Grant (PI: Nageswaran) as part of the CTSI Pilot Grant Program and the Wake Forest Center for Biomedical Informatics (WFBMI) funded by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through Grant Award Number UL1TR001420. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.