Skip to main content
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2022 May 17;52(12):5099–5113. doi: 10.1007/s10803-022-05576-1

Provider and Caregiver Satisfaction with Telehealth Evaluation of Autism Spectrum Disorder in Young Children During the COVID-19 Pandemic

Debra L Reisinger 1,, Elesia Hines 1, Christine Raches 1, Qing Tang 2, Cristina James 1, Rebecca McNally Keehn 1
PMCID: PMC9112250  PMID: 35579789

Abstract

The present study examines provider and caregiver satisfaction with telehealth evaluation of autism spectrum disorder (ASD) in young children during the coronavirus SARS-CoV-2 (COVID-19) pandemic. A telehealth model of ASD evaluation was implemented with 308 children ages 14 to 78 months between May 2020 to June 2021. Data were gathered from electronic health records, autism-specific telehealth diagnostic tools, and post-evaluation surveys. Overall, the majority of providers and caregivers were satisfied with telehealth ASD evaluation. Multiple variables were associated with ratings of satisfaction, differing by providers and caregivers. Findings have important implications for the feasibility and acceptability of telehealth ASD evaluations, in addition to key factors to consider in optimizing and sustaining telehealth evaluations beyond the COVID-19 pandemic.

Keywords: Autism spectrum disorder, Telehealth, Diagnostic evaluation, Young children, Satisfaction


Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by impairments in social communication and social interaction as well as restricted, repetitive patterns of behavior, interests, or activities (American Psychiatric Association, 2013). The Centers for Disease Control (CDC) estimates that about 1 in 54 children are identified with ASD (Maenner et al., 2020). Despite the increase in prevalence, families of children with ASD continue to encounter substantial difficulties in accessing diagnostic evaluations, often waiting months or years between first developmental concern, and receiving a diagnosis (Maenner et al., 2020). Although the early ASD phenotype becomes stable around 14 months of age (Pierce et al., 2019), the average age of diagnosis in the United States remains around 4 years of age (Maenner et al., 2020). This lag in diagnosis can be explained by a number of factors, including a shortage of trained specialists (Malik-Soni et al., 2021), families living in a rural or under-resourced area (Antezana, 2017), and long wait lists (Gordon-Lipkin et al., 2016). However, the coronavirus SARS-CoV-2 (COVID-19) pandemic added an unprecedented disruption in addition to the already known barriers with access to a diagnostic evaluation.

The onset of the pandemic necessitated a rapid shift to telehealth for health care systems given stay-at-home orders and social distancing requirements. Providers had to quickly weigh options for telehealth diagnostic evaluations and implement novel assessment practices in order to meet the needs of patients and their families. Many providers value and utilize technology to manage work tasks and consult with other professionals (Iacono et al., 2016). However, some have been hesitant to adopt telehealth models in their practice despite physicians reporting that they could treat patients well with telehealth and being satisfied with the services provided long before COVID-19 (Becevic et al., 2015).

Prior to the COVID-19 pandemic, researchers and providers explored the feasibility and acceptability of using telehealth evaluations to address barriers to accessing an ASD diagnosis (Juárez et al., 2018; Kanne et al., 2018; Reese et al., 2015; Stainbrook et al., 2019). Three primary methods of remote assessment of ASD symptoms have been developed and studied in recent years. One method includes asynchronous or “store-and-forward” telehealth, in which caregivers record in-home videos of their child’s behavior and upload the videos for providers to view as part of the diagnostic assessment (Nazneen et al., 2015; Kanne et al., 2018; Sutantio et al., 2021). Another method utilizes audiovisual technology allowing a remote provider (i.e., a clinician with expertise in ASD) to interview caregivers directly and observe a trained on-site provider administer a standardized ASD screening tool (e.g., Screening Tool for Autism in Toddlers and Young Children [STAT; Stone et al., 2008]). For example, Juárez et al., (2018) used telehealth to observe the administration of the STAT and found that trained providers correctly identified 80% of children who later received a diagnosis of ASD via a comprehensive in-person evaluation. The third method of telehealth assessment of ASD symptoms focuses on real-time, remote administration of caregiver mediated ASD assessment tools in which the provider coaches caregivers to complete interactive activities with their child as part of a standardized assessment protocol. For example, the TELE-ASD-PEDS (Corona et al., 2020) is a clinical assessment tool that was developed pre-pandemic for caregiver-mediated telehealth evaluations of ASD and has been a valuable tool during the COVID-19 pandemic (Wagner et al., 2020, 2021).

Findings from the aforementioned studies provide preliminary support for the feasibility, utility, and accuracy of telehealth ASD diagnostic evaluations for young children. A recent review examining various telehealth approaches used in ASD diagnostic evaluations suggests the use of telehealth for ASD diagnosis was acceptable to both families and providers (Alfuraydan et al., 2020). However, the acceptance of telehealth diagnostic evaluations for ASD and the development of robust ASD telehealth diagnostic tools remains limited. This is likely due to a myriad of reasons, including legal and ethical considerations (Quigley et al., 2019), the difficulties associated with making a diagnosis in a highly heterogenous population, and the necessity of observation and interaction with an experienced provider (McNally Keehn et al., 2021). Nonetheless, research conducted since the onset of the COVID-19 pandemic continues to support the feasibility and acceptability of telehealth ASD screening and evaluations in young children (Talbott et al., 2020; Wagner et al., 2020).

Studies conducted during the pandemic examining provider satisfaction with neurodevelopmental telehealth evaluation in children have revealed that most providers are satisfied and comfortable with the process (McNally Keehn et al., 2021; Wagner et al., 2021). Specifically, providers have reported feeling comfortable with completing telehealth assessments, providing ASD diagnoses based on telehealth evaluation results, and giving feedback and recommendations to families via telehealth (Wagner et al., 2020, 2021). The use of telehealth during the pandemic has also resulted in greater appreciation for the value of evaluating a child’s behavior in their home environment (Jang et al., 2021) and increasing caregiver involvement in the evaluation process (Wagner et al., 2021).

Utilization of telehealth for ASD evaluations has the potential to provide significant advantages to families compared to in-person evaluations. For example, previous work has reported a potential cost savings of over thirty-five dollars for families related to travel alone, not considering additional expenses such as meals, hotels, childcare, or potential lost wages for families in rural or underserved areas (Reese et al., 2015). More recently, a provider survey indicated telehealth increases access to ASD diagnostic evaluations for children and families who may have had difficulty attending in-person appointments prior to COVID-19 due to transportation and other barriers such as taking time off work and finding childcare for siblings (Wagner et al., 2021). Notably, parents shared that if telehealth were not an option, they would have waited to travel to a diagnostic center, which was likely to delay the diagnosis, subsequently impacting entry into early intervention services (Juárez et al., 2018).

Caregivers have also reported high levels of satisfaction with ASD and neurodevelopmental telehealth diagnostic evaluations Corona et al., 2021; Juárez et al., 2018; McNally Keehn et al., in press; Reese et al., 2013; Talbott et al., 2020). Prior to the pandemic, Juárez et al., (2018) examined caregiver satisfaction related to receiving a diagnosis of ASD via telehealth and found parents reported high levels of satisfaction with the telehealth evaluation method. In a recent study utilizing the TELE-ASD-PEDS and TELE-STAT, caregivers found the telehealth screening process to be acceptable and comfortable (Corona et al., 2021). The majority of parents provided positive feedback about their involvement in the assessment and indicated that their child may have felt more comfortable playing with a parent compared to an unfamiliar adult. Further, most parents reported the provider’s instructions were easy to follow, they understood what they were asked to do, and they felt comfortable being a part of the evaluation. Other factors associated with caregiver satisfaction include convenience, privacy, and reduced travel burden (Juárez et al., 2018; McNally Keehn et al., 2021). These findings contrast with some previous literature on caregiver satisfaction with receiving an ASD diagnosis via traditional, in-person evaluations (Crane et al., 2016; Howlin & Moore, 1997). Specifically, caregiver dissatisfaction with the traditional process has been linked to factors such as the wait time for receiving a diagnosis, the way in which providers deliver an ASD diagnosis, and perceived lack of support during and after a diagnosis is provided (Crane et al., 2016). Although ASD telehealth diagnostic evaluations may improve the wait time for receiving a diagnosis, additional work is needed to explore the impact on satisfaction with providers and perceived support families obtain post-diagnosis following a telehealth diagnostic evaluation.

It is important to note that studies reporting high levels of satisfaction with ASD telehealth diagnostic evaluations have also highlighted potential drawbacks of the telehealth experience. Specifically, caregivers and providers have cited technology barriers with telehealth, including audio/video distortions and internet connection issues (Corona et al., 2021; Wagner et al., 2020). Further, some caregivers have indicated concern about the small size of their device screen and their child’s interest in touching the camera and/or screen (Corona et al., 2021). In addition, some caregivers encounter difficulty operating their devices during the evaluation or keeping their child on the screen while engaging in assessment activities (Wagner et al., 2021). Providers have also reported challenges related to the family’s home environment, including lack of play materials and distractions. While providers regularly experience barriers related to technology and family challenges, a recent provider survey indicated that these problems were rarely rated as disruptive to the evaluation process (McNally Keehn et al., 2021).

At the onset of the pandemic, our large outpatient clinic at Children’s Hospital quickly shifted to providing telehealth diagnostic ASD evaluations in order to continue serving patients and maintain access to ASD diagnostic services. The present study seeks to expand the current literature on provider and caregiver satisfaction with telehealth ASD diagnostic evaluations completed with young children and their families during the COVID-19 pandemic. Specifically, we examine caregiver and provider satisfaction, their relationship, and separately by provider (e.g., psychologist/pediatrician). Based on previous work, we hypothesized that both caregivers and providers would be satisfied with telehealth ASD evaluations, satisfaction ratings between caregivers and providers to be positively correlated, and caregiver satisfaction to relate similarly regardless of provider type. Further, we explored associations between caregiver and provider satisfaction and child demographics, diagnostic outcomes, and telehealth evaluation factors. Although the present literature on satisfaction with ASD telehealth evaluations is limited, we hypothesized that caregiver satisfaction would be related to travel time saved, diagnostic outcome, and technical or setup difficulties. Similarly, we hypothesized that provider satisfaction would be related to child age, diagnostic outcome, and technical and setup difficulties. This study expands on the growing literature using a large sample across both caregivers and providers, including both psychologists and pediatricians, in addition to exploring a broader range of factors related to satisfaction to build on the feasibility, utility, and acceptability of telehealth methodology for diagnostic ASD evaluations in young children during an unprecedented global pandemic and in the future.

Methods

Participant and Provider Characteristics

A total of 308 children between the ages of 14 and 78 months (M = 37.9, SD = 10.9) referred by their primary care provider (PCP) or specialist within the Riley Hosptial for Children completed a telehealth evaluation. Data were drawn from a larger study offering telehealth neurodevelopmental evaluations between May 2020 and June 2021 at the Riley Hospital for Children outpatient neurodevelopmental evaluation clinic. Children and their families that required a language interpreter were excluded. See Table 1 for child and family demographics.

Table 1.

Child characteristics and family demographics

N = 308
Age in months, mean ± SD (range) 37.9 ± 10.9 (14–78)
Sex, n (%)
Male 225 (73.05%)
Female 83 (26.95%)
Race, n (%)
White/Caucasian 95 (68.35%)
Black/African American 20 (14.39%)
More than one race 15 (10.79%)
Other 9 (6.47%)
Ethnicity, n (%)
Not Hispanic or Latino 118 (86.76%)
Hispanic or Latino 18 (13.24%)
Household Income, n (%)
<$40,000 61 (48.03%)
$40,000 - $74,999 35 (27.56%)
>$75,000 31 (24.41%)
Household Size, mean ± SD 4.0 ± 1.1
Insurance, n (%)
Medicaid 177 (58.03%)
Private Insurance 109 (35.74%)
Medicaid and Private Insurance 19 (6.23%)
ASD Diagnostic Outcomes, n (%)
ASD Present 170 (55.19%)
ASD Absent 94 (30.52%)
ASD Unsure 44 (14.29%)
Vineland-3, mean ± SD (range)
Adaptive Behavior Composite 71.9 ± 11.7 (44–115)
Communication 66.7 ± 17.3 (27–124)
Daily Living Skills 75.2 ± 14.6 (33–124)
Socialization 74.1 ± 14.7 (26–128)
Motor Skills 81.0 ± 14.1 (42–124)

Note. ASD = autism spectrum disorder; ASD Present = an ASD diagnosis was given; ASD Absent = an ASD diagnosis was not given; ASD Unsure = the clinician was not sure about whether the child had ASD or not at the time of the evaluation; Vineland-3 scores are reported as standard scores (SS)

Providers included a total of seven licensed psychologists and four pediatricians with neurodevelopmental training who completed the neurodevelopmental telehealth evaluations. All providers had significant experience in the evaluation of children with neurodevelopmental disabilities (mean years of experience = 13; range = 2–35 years). Prior to COVID-19, none of the providers had experience in the use of telehealth for neurodevelopmental evaluation; however, all were trained in the TELE-ASD-PEDS prior to the start of the study. Specifically, all providers participated in the online TELE-ASD-PEDS training webinars offered through Vanderbilt University (https://vkc.vumc.org/vkc/triad/tele-asd-peds). Further, providers participated in regularly scheduled peer supervision and consultation meetings led by a clinical psychologist (and Certified Autism Diagnostic Observation Schedule™, Second Edition [ADOS-2; Lord et al., 2012] Independent Trainer) who engaged in additional, ongoing trainings and consultation with the TELE-ASD-PEDS developers.

Measures

Provider Telehealth Evaluation Survey. A survey to capture provider experiences with providing telehealth neurodevelopmental evaluations was developed for this study. Survey items included information about evaluation components (including use of assessment tools), whether any diagnosis was communicated to the family (e.g., ASD, developmental delay, language/communication delay [e.g., mixed receptive-expressive language disorder], unspecified behavioral/emotional disorder, etc.), ASD diagnostic outcomes and provider certainty, evaluation next steps, technology (e.g., audio distortions/limitations, video distortions/freezing) and family/home setup barriers (e.g., child not compliant, distracting family members/siblings/pets/home environment, placement of the video camera, inadequate toys/materials) encountered, and ratings of satisfaction with the telehealth evaluation. Two provider satisfaction items were rated on a dichotomous yes/no scale and two items were rated on a five-point Likert scale (e.g., Not satisfied [1] – Very satisfied [5]; see Table 2 for the list of satisfaction questions asked). These two scores were averaged to create an overall provider satisfaction score. Some survey items were adapted from Juárez et al., (2018). Due to small sample size in some cells, provider ratings of diagnostic certainty (completely certain, somewhat certain, somewhat uncertain, or completely uncertain) were dichotomized (certain or uncertain) for analysis.

Table 2.

Provider and caregiver satisfaction item summary scores

Mean
(SD)
Not satisfied
n (%)
Somewhat satisfied
n (%)
Neutral
n (%)
Satisfied
n (%)
Very satisfied
n (%)
1Clinician Satisfaction Ratings

Please rate your satisfaction with the information obtained

during the telehealth visit.

4.2 (0.7) 1 (0.35%) 10 (3.48%) 7 (2.44%) 174 (60.63%) 95 (33.10%)

Please rate your overall satisfaction with the remote telehealth

service you provided for this patient.

4.2 (0.7) 2 (0.70%) 9 (3.14%) 6 (2.09%) 181 (63.07%) 89 (31.01%)
2Provider Satisfaction Ratings
Mean (SD)

Strongly disagree

n (%)

Disagree

n (%)

Neutral

n (%)

Agree

n (%)

Strongly Agree

n (%)

I felt the telehealth provider was engaged and part of the visit. 4.9 (0.4) 1 (0.71%) 18 (12.77%) 122 (86.52%)
I was able to communicate my concerns to the telehealth provider during the visit. 4.9 (0.4) 1 (0.71%) 17 (12.06%) 123 (87.23%)
I felt the telehealth provider was able to collect important information about my child. 4.6 (0.7) 3 (2.13%) 8 (5.67%) 34 (24.11%) 96 (68.09%)
The information I received through the telehealth appointment will help me make decisions for next steps in my child’s support plan. 4.7 (0.6) 3 (2.13%) 3 (2.13%) 32 (22.70%) 103 (73.05%)
I felt the equipment used during the telehealth visit was not distracting and did not take away from the effectiveness of the visit. 4.4 (0.9) 1 (0.71%) 6 (4.26%) 17 (12.06%) 30 (21.28%) 87 (61.70%)
My telehealth visit was just as private as an in-person evaluation. 4.7 (0.7) 2 (1.42%) 3 (2.13%) 3 (2.13%) 20 (14.18%) 113 (80.14%)
Telehealth made it easier and more convenient for me to visit with a provider. 4.4 (1.0) 4 (2.84%) 5 (3.55%) 16 (11.35%) 28 (19.86%) 88 (62.41%)
I am likely to recommend telehealth to others. 4.3 (1.0) 1 (0.71%) 9 (6.38%) 19 (13.48%) 33 (23.40%) 79 (56.03%)
I am interested in participating in future telehealth visits at this clinic. 4.3 (0.9) 2 (1.42%) 2 (1.42%) 25 (17.73%) 32 (22.70%) 80 (56.74%)
Overall, I am satisfied with my telehealth experience. 4.6 (0.8) 1 (0.71%) 4 (2.84%) 11 (7.80%) 25 (17.73%) 100 (70.92%)

Note. 1 N=287; Items rated on Likert scale (1 = Not satisfied; 5 = Very satisfied); 2 N=141; Items rated on Likert scale (1 = Strongly Disagree; 5 = Strongly Agree

Caregiver Telehealth Satisfaction Survey. A survey measuring caregiver experience and satisfaction with their child’s telehealth evaluation was adapted from Juárez et al., (2018) for the present study. Ten satisfaction items were rated on a five-point Likert scale [e.g., Strongly Disagree (1) - Strongly Agree (5)]). A summary score was calculated by averaging the ratings from all ten items (range 0–5) to represent overall satisfaction (See Table 2 for the list of satisfaction questions asked). Further, to examine differences in satisfaction with the provider versus the method of evaluation, two subscales were created from the ten caregiver satisfaction items: overall provider satisfaction (mean of four items) and overall satisfaction with the telehealth evaluation provider (mean of five items). Additional items measured caregiver report of travel time saved, preferred method of evaluation (e.g., in-person or telehealth), and child/family demographic information. Due to small sample size in some cells, caregiver ratings of their preference for their visit (in-person with another specialist in my region, in-person with a specialist at Children’s Hospital, telehealth visit with specialist doctors, or telehealth visit with a specialist at my primary care office) were dichotomized (in-person or telehealth) for analysis.

ASD Diagnostic Assessment. The TELE-ASD-PEDS (Corona et al., 2020) is a newly developed remote assessment tool designed to aid in the screening and diagnosis of ASD symptoms and behaviors in young children ages 14 to 36 months (clinicaltrials.gov, NCT03847337). The TELE-ASD-PEDS takes approximately 15–20 min to administer and is led by a trained provider in conjunction with a comprehensive diagnostic interview (e.g., developmental history and ASD symptom-specific interview). The trained provider leads the child’s caregiver through a series of eight discrete activities or social bids, including opportunities for interactive play, physical play routines, and requests using toys readily available in the child’s home. Following administration, providers complete a rating form on seven different behaviors (e.g., socially directed speech and sounds, eye contact, unusual vocalizations, unusual or repetitive play or body movements, unusual sensory exploration) using both dichotomous (yes/no) and Likert (3 = behaviors characteristic of ASD clearly present; 2 = behaviors characteristic of ASD present at subclinical levels; 1 = behaviors characteristic of ASD not present) scoring procedures. A total score is calculated by summing the seven Likert scale item scores. This total score was utilized in the present study to quantify ASD symptom severity in the participants, with higher scores being indicative of greater ASD symptoms.

Adaptive Behavior. The Vineland Adaptive Behavior Scales, Third Edition - Comprehensive Parent/Caregiver Form (Vineland-3) was used to assess overall adaptive functioning of child participants (Sparrow et al., 2016). The Vineland-3 was administered via remote link emailed to caregivers prior to their evaluation to obtain the child’s overall adaptive functioning from four subdomains (communication, daily living skills, socialization, and motor skills). The Vineland-3 provides an adaptive behavior composite score (mean of 100 and standard deviation of 15) that was used in the current study in addition to the subdomain standard scores. A subset of caregivers completed the questionnaire due to adding the measure several months into the start of the study.

Procedures

Due to COVID-19, children previously scheduled or on the waitlist for an in-person ASD evaluation who were less than 48 months of age were contacted by the clinic scheduling team to offer the telehealth evaluation. For the present study, we also included children outside of the TELE-ASD-PEDS age range (up to age 6:11) who presented with language deficits ranging from no functional word use to occasional simple 2-word phrases. If a child was between 4 and 6 years of age, a language screening was conducted by a speech-language pathologist to determine if the child met the language inclusion criteria. A verbal script (similar to the examples provided in the TELE-ASD-PEDS training: https://vkc.vumc.org/vkc/triad/tele-asd-peds) was developed for the scheduling team in order to ensure standardized delivery of information regarding telehealth evaluation appointments. Caregivers who consented to telehealth were provided verbal and written instructions on how to prepare for their evaluation, including accessing the virtual Zoom Health platform, and were scheduled for a 90- to 120-minute telehealth diagnostic evaluation led by a psychologist or a pediatrician. The necessary speed and bandwidth of internet access needed was not explicitly outlined to caregivers; however, if the internet connection was found to be insufficient during the evaluation, a follow-up in-personevaluation was recommended. Caregivers who declined telehealth were informed their child would remain on the waitlist for in-person evaluation and would be contacted when scheduling resumed. A specific wait time for families who declined a telehealth evaluation was not provided given the unknown timeline regarding safe return to in-person evaluations and associated hospital policies and guidelines. Data on the differences between wait times for in-person and telehealth evaluations was not available.

Members of the research team gathered data from the Electronic Health Record (EHR) on child age, sex, home zip code, insurance type, and primary referral question (e.g., categorized into ASD, developmental delay/intellectual disability/language delay, and other neurodevelopmental disability [e.g., ADHD, learning disability, behavioral or emotional problem]) and entered it into a secure research database. Prior to the day of the evaluation, caregivers were emailed the link to complete the Vineland-3 via remote administration. On the day of evaluations, providers conducted a comprehensive diagnostic interview, administered the TELE-ASD-PEDS via Zoom Health with the child and their family, completed the TELE-ASD-PEDS rating form, and provided recommendations and next steps to the caregivers. A diagnosis was communicated to a family if the provider was able to determine the appropriate diagnosis during the telehealth evaluation; however, a diagnosis was not always provided for various reasons (e.g., technical difficulties, diagnostic uncertainty, etc.). Providers were also sent a secure link via email to complete an individual survey for each evaluation. On the day following their child’s evaluation, caregivers received a secure link via email (unless they requested another method) to complete a satisfaction survey. Caregivers were also provided remote follow-up care navigation from one of two social workers staffed in our clinic approximately three to four weeks after their telehealth evaluation. Providers completing surveys for > 85% of scheduled evaluations were provided a $25 gift card for each month of participation. Caregivers were provided a $25 gift card for completed surveys.

Statistical Analyses

Descriptive statistics were used to analyze demographics, diagnostic outcomes, and Vineland-3 and TELE-ASD-PEDS scores. Means (standard deviation) were reported for continuous variables and counts (percentages) were reported for categorical variables.

Next, generalized mixed effect models with random effects were utilized to examine the association between provider satisfaction and caregiver overall satisfaction, caregiver satisfaction with the provider, and caregiver satisfaction with the telehealth evaluation. Psychologist and pediatrician satisfaction scores were analyzed separately in relation to caregiver overall satisfaction, caregiver satisfaction with the telehealth evaluation, and caregiver satisfaction with the provider. Similarly, generalized mixed effect models with random effects were utilized to examine caregiver overall satisfaction in relation to child demographics, diagnostic outcomes, and telehealth evaluation factors. Lastly, generalized mixed effect models with random effects were utilized to examine provider satisfaction in relation to child demographics, diagnostic outcomes, and telehealth evaluation factors. Generalized mixed effect models with random effects were utilized to account for correlations within the same provider. Data analysis was performed with SAS version 9.4 (SAS Institute, Inc., Cary, NC). p values < 0.05 were considered statistically significant. Multiple comparisons were not controlled for in the analyses since we focused on a few specific a-priori hypotheses and reported the individual p-values for each; therefore, it was deemed unnecessary to adjust the p-values. Missing data were considered to be random due to caregivers or providers not completing the questionnaires or the information not being available in the child’s medical record. Using the demographic data collected on all participants, we examined group differences between caregivers that completed the survey and those who did not complete the survey. No significant group differences were found across gender, type of insurance, ASD diagnostic outcome, or ASD symptom severity. The average age between the two groups of respondents was significantly different (p = 0.013), with the children of caregivers who completed the survey being younger (M = 36.25, SD = 10.27) than those who did not complete the survey (M = 39.32, SD = 11.22).

Results

Of the 308 evaluations that were conducted, 287 provider satisfaction surveys (93.19%) and 141 caregiver satisfaction surveys (45.78%) were collected. Of the 287 provider satisfaction surveys completed, 235 were completed by psychologists and 52 were completed by pediatricians. Table 2 details the mean, SD, and proportions for each of the satisfaction questions that were asked to both providers and caregivers. Overall, the majority of caregivers and providers indicated they were satisfied (4) or very satisfied (5) with the telehealth evaluation.

Relationship Between Provider and Caregiver Satisfaction

Overall, both providers (M = 4.2, SD = 0.7) and caregivers (M = 4.6, SD = 0.6) reported on average being satisfied with the telehealth ASD evaluations. Provider satisfaction was not significantly associated with caregiver overall satisfaction (t = 0.69, p = 0.494), caregiver satisfaction with the provider (t = 1.70, p = 0.091), or caregiver satisfaction with the telehealth appointment (t = 0.11, p = 0.914).

Both psychologists (M = 4.1, SD = 0.7) and pediatricians (M = 4.6, SD = 0.6) indicated they were on average satisfied with the telehealth ASD evaluations overall. Psychologist satisfaction was not significantly related to caregiver overall satisfaction (t = 0.77, p = 0.444) or caregiver satisfaction with the telehealth appointment (t = 0.06, p = 0.951); however, it was significantly related to caregiver satisfaction with the provider (t = 2.06, p = 0.042). Pediatrician satisfaction was not significantly related to caregiver overall satisfaction (t=-0.30, p = 0.771), caregiver satisfaction with the provider (t=-0.76, p = 0.458), or caregiver satisfaction with the telehealth appointment (t = 0.08, p = 0.936).

Caregiver Satisfaction in Relation to Demographics, Diagnostic, and Telehealth Evaluation Factors

Caregiver overall satisfaction ratings were examined in relation to child demographics, diagnostic outcomes, and telehealth evaluation factors (Table 3). Caregiver overall satisfaction was significantly related to child adaptive functioning (Vineland-3 ABC, Communication skills, Daily Living Skills, and Socialization Standard Scores). Specifically, results suggest caregivers of children with lower adaptive skills were more satisfied with the telehealth evaluation (ps < 0.024). The relationship between caregiver satisfaction and child sex approached significance (p = 0.052), with caregivers of female children (M = 4.7, SD = 0.6) reporting slightly higher satisfaction than caregivers of male children (M = 4.5, SD = 0.6). Caregiver satisfaction was also significantly related to ASD diagnostic outcomes. Specifically, caregiver satisfaction was significantly related to the child’s TELE-ASD-PEDS Total score (p = 0.014); caregivers of children with greater ASD symptom severity were more satisfied with their telehealth evaluation. Further, caregivers of children diagnosed with ASD reported higher satisfaction ratings (M = 4.7, SD = 0.5) in comparison to those who did not receive an ASD diagnosis (M = 4.0, SD = 0.7) or the provider was unsure (e.g., could not rule in or out) of the diagnosis (M = 4.3, SD = 0.5; p = 0.017). Caregiver satisfaction was also significantly related to provider diagnostic certainty (p = 0.02). Specifically, caregivers were more satisfied with providers who were certain about their ASD diagnosis (M = 4.6, SD = 0.6) in comparison to those who were uncertain (M = 4.1, SD = 0.8). Caregiver satisfaction was also significantly related to their visit preference (p < 0.001), with caregivers preferring telehealth visits (M = 4.8, SD = 0.3) being more satisfied than those who preferred an in-person visit (M = 4.3, SD = 0.7).

Table 3.

Caregiver satisfaction in relation to child demographics, diagnostic outcomes, and telehealth evaluation factors

Variable n Mean (SD) F β t p
Age 0.01 1.64 0.103
Sex 3.87 0.052
Male 100 4.5 (0.6)
Female 41 4.7 (0.5)
Race 4.70 0.392
Black/African American 20 4.7 (0.5)
White/Caucasian 95 4.5 (0.6)
More than one race 15 4.6 (0.5)
Other 9 4.8 (0.4)
Ethnicity 0.63 0.431
Hispanic or Latino 18 4.7 (0.4)
Not Hispanic or Latino 118 4.6 (0.6)
Vineland-3
Adaptive Behavior Composite -0.01 -2.77 0.008*
Communication -0.01 -3.01 0.004*
Daily Living Skills -0.01 -2.33 0.023*
Socialization -0.01 -2.33 0.024*
Motor Skills < 0.01 -0.48 0.634
Household Income 0.57 0.568
<$40,000 61 4.6 (0.6)
$40,000 - $74,999 35 4.5 (0.6)
>$75,000 31 4.5 (0.6)
Household Size -0.05 -1.22 0.223
Insurance 1.12 0.329
Medicaid 80 4.6 (0.6)
Private Insurance 51 4.5 (0.6)
Medicaid and Private Insurance 80 4.6 (0.6)
Travel Time Saved 1.05 0.371
Less than one hour 34 4.6 (0.5)
One to two hours 51 4.4 (0.7)
Three to four hours 43 4.6 (0.5)
Five or more hours 13 4.7 (0.6)
Diagnosis Provided 1.12 0.293
Yes 130 4.5 (0.6)
No 4 4.2 (0.8)
Provider Diagnostic Certainty 5.23 0.024*
Certain 131 4.6 (0.6)
Uncertain 10 4.1 (0.8)
Provider Preference for In-Person 0.11 0.737
Yes 87 4.5 (0.6)
No 47 4.6 (0.6)
Caregiver Preference for Visit 42.53 < 0.001*
Telehealth 72 4.8 (0.3)
In-Person 66 4.3 (0.7)
ASD Diagnostic Outcomes 4.20 0.017*
ASD Present 78 4.7 (0.5)
ASD Absent 44 4.4 (0.7)
ASD Unsure 19 4.3 (0.5)
TELE-ASD-PEDS Total Score 0.03 2.49 0.014*
Technical Difficulties 0.258
Yes 22 4.7 (0.6)
No 112 4.5 (0.6)
Setup Difficulties 0.699
Yes 32 4.5 (0.7)
No 102 4.5 (0.6)

Note. *p < 0.05; ASD = autism spectrum disorder; SD = standard deviation

Caregiver satisfaction was not related to child race, child age, ethnicity, household income, household size, type of insurance, travel time saved, whether or not the provider could provide any type of diagnosis, provider preference for in-person or telehealth visit, or technical and setup difficulties during the appointment (ps > 0.05).

Provider Satisfaction in Relation to Child Demographics, Diagnostic Outcomes, and Telehealth Evaluation Factors

Provider overall satisfaction ratings were examined in relation to child demographics, diagnostic outcomes, and telehealth evaluation factors (see Table 4). Provider overall satisfaction was significantly related to whether or not the provider could make a diagnostic determination in the telehealth evaluation (p < 0.001). Specifically, providers who were able to provide any diagnosis during the evaluation (M = 4.2, SD = 0.6) were more satisfied than those who were not (M = 3.5, SD = 1.0). Similarly, providers who were able to provide a diagnosis of ASD (M = 4.4, SD = 0.5) in the telehealth evaluation were more satisfied than those who were unsure of the diagnosis (M = 3.6, SD = 0.9) or the diagnosis was ruled out (M = 4.2, SD = 0.6; p < 0.001). Similarly, providers who were certain about their diagnosis (M = 4.3, SD = 0.6) were more satisfied in comparison to those who were uncertain (M = 3.4, SD = 1.1; p < 0.001). Provider satisfaction was also significantly related to the provider’s preference for an in-person visit, (p < 0.001). Specifically, providers who preferred an in-person visit (M = 3.9, SD = 0.5) were less satisfied than those who did not (M = 4.8, SD = 0.5). Lastly, technical difficulties (p = 0.007) and setup difficulties (p < 0.001) were significantly related to provider satisfaction, with providers who did not report any technical or setup difficulties having higher satisfaction ratings. Lastly, provider satisfaction was significantly related to child TELE-ASD-PEDS Total scores (p < 0.001), with providers rating higher satisfaction for evaluation of children with greater ASD symptom severity.

Table 4.

Provider satisfaction in relation to child demographics, socioeconomic variables, diagnostic outcomes, and telehealth visit factors

Variable n Mean (SD) F β t P
Age < 0.01 -0.34 0.737
Sex 0.02 0.890
Male 210 4.2 (0.6)
Female 77 4.2 (0.8)
Race 1.70 0.172
Black/African American 18 3.9 (1.0)
White/Caucasian 91 4.2 (0.7)
More than one race 15 4.4 (0.5)
Other 8 4.3 (0.6)
Ethnicity 0.00 0.997
Hispanic or Latino 18 4.3 (0.6)
Not Hispanic or Latino 111 4.2 (0.7)
Vineland-3
Adaptive Behavior Composite < 0.01 -1.23 0.221
Communication < 0.01 -1.51 0.134
Daily Living Skills < 0.01 -0.71 0.476
Socialization < 0.01 -1.45 0.150
Motor Skills < 0.01 -0.24 0.811
Household Income 0.32 0.726
<$40,000 57 4.2 (0.8)
$40,000 - $74,999 35 4.3 (0.5)
>$75,000 28 4.3 (0.7)
Household Size < 0.01 0.10 0.942
Insurance 0.29 0.748
Medicaid 160 4.2 (0.7)
Private Insurance 107 4.3 (0.6)
Medicaid and Private Insurance 18 4.1 (0.8)
Travel Time Saved 0.20 0.897
Less than one hour 34 4.1 (0.7)
One to two hours 50 4.3 (0.6)
Three to four hours 39 4.1 (0.8)
Five or more hours 11 4.3 (0.7)
Diagnosis Provided 22.28 < 0.001*
Yes 275 4.2 (0.6)
No 12 3.5 (1.0)
Provider Diagnostic Certainty 56.80 < 0.001*
Certain 264 4.3 (0.6)
Uncertain 23 3.4 (1.1)
Provider Preference for In-Person 159.72 < 0.001*
Yes 176 3.9 (0.5)
No 111 4.8 (0.5)
Caregiver Preference for Visit 0.07 0.796
Telehealth 67 4.2 (0.7)
In-Person 64 4.2 (0.7)
ASD Diagnostic Outcomes 29.13 < 0.001*
ASD Present 156 4.4 (0.5)
ASD Absent 88 4.2 (0.6)
ASD Unsure 43 3.6 (0.9)
TELE-ASD-PEDS Total Score 0.03 3.79 < 0.001*
Technical Difficulties 7.38 0.007*
Yes 45 4.0 (0.8)
No 242 4.3 (0.6)
Setup Difficulties 25.43 < 0.001*
Yes 45 4.0 (0.8)
No 228 4.3 (0.6)

Note. *p < 0.05; ASD = autism spectrum disorder; SD = standard deviation

Provider satisfaction was not significantly related to child age, adaptive skills, race, ethnicity, gender, household income, household size, type of insurance, travel time saved, or caregiver preference for an in-person or telehealth visit.

Discussion

The present study builds on the current literature showcasing the feasibility, utility, and satisfaction with ASD telehealth evaluations during the unprecedented COVID-19 pandemic (Alfuraydan et al., 2020; Corona et al., 2021; Juárez et al., 2018; Wagner et al., 2020, 2021). Given that the current literature on ASD telehealth evaluations is limited, additional research is needed to further our understanding of the variables related to satisfaction with telehealth evaluations, especially with the increased emphasis on telehealth utilization since COVID-19. Utilizing a recently developed telehealth assessment tool specifically designed for remote ASD diagnostic evaluations, our results indicate both caregivers and providers were highly satisfied with the telehealth evaluation overall. Further, provider and caregiver satisfaction were related to different factors.

Approximately 88% of caregivers who completed the survey in the present study reported they were satisfied with the telehealth evaluations (e.g., agreed or strongly agreed they were satisfied). In addition to caregivers being satisfied with the telehealth evaluations overall, there were a few specific factors related to their high satisfaction. Regarding child characteristics, caregivers of children with lower adaptive skills, including communication, daily living skills, and socialization skills, and higher autism symptomology scores reported higher rates of satisfaction. This is important with regard to the utility of ASD telehealth evaluations. Specifically, for children who are more impaired developmentally and who have higher ASD symptoms, a telehealth evaluation likely sped up the wait time for accessing an evaluation and diagnosis, in addition to opening doors for these children to access early interventions and supports. Furthermore, these findings may demonstrate increased caregiver confidence in the use of telehealth ASD evaluations for children who present with this type of developmental profile and higher ASD symptom severity. Additional research into caregiver decision making when choosing to have their child evaluated via telehealth could help further delineate how child characteristics are related to caregiver satisfaction. Moreover, children referred with these characteristics may be an ideal targeted population who could benefit most from the option of a telehealth evaluation. It is important to note that child demographics such as race, age, and ethnicity were not related to caregiver satisfaction in the present study; however, this study did exclude non-English speaking families and the majority of our sample included non-Hispanic participants.

With regard to ASD diagnostic outcomes, caregiver satisfaction was increased if their child received an ASD diagnosis at the end of the telehealth evaluation and if the provider was certain about their diagnosis. As noted above, there are several barriers that can occur during a telehealth evaluation including technical difficulties and issues with the home setup/environment (Corona et al., 2021; Wagner et al., 2020). Aside from these barriers potentially occurring, proper and extensive training of providers in conducting ASD telehealth evaluations is critical. This will likely increase their confidence to provide a diagnosis with certainty, ultimately increasing caregiver satisfaction, and likely bolstering acceptability of ASD telehealth evaluations in both caregivers and providers beyond the unexpected circumstances of the COVID-19 pandemic.

Although telehealth evaluations can increase access to care for some families, it may not be the preferred evaluation method by all families that would likely benefit (e.g., rural or underserved populations), especially for families who may have limited access to adequate audio/video technology, access to internet, or experience with technology. Therefore, it is important to always consider the caregivers’ preference when offering a telehealth or in-person evaluation. Specifics regarding the necessary speed and bandwidth of the caregiver’s internet was not explicitly outlined during the time of scheduling in order to obtain an appointment; however, poor internet connection can play a critical role in the ability to observe the child’s behaviors and provide an accurate diagnosis during a telehealth evaluation. Approximately 20% of caregivers in our sample reported technical difficulties; however, there were no differences found in their ratings of satisfaction between those who experienced technical difficulties and those who did not. Unsurprisingly, we found that caregivers who preferred the telehealth evaluation were more satisfied than those who would have opted for an in-person visit, underscoring the importance of considering caregiver preference. Similarly, Corona and colleagues (2021) also found that some parents appreciated a telehealth appointment as an initial step in the diagnostic procedure but also wanted to be seen in-person during the diagnostic process. Together, these findings suggest a hybrid model may be appropriate and ideal for some families and providers, particularly when a diagnosis is not clear through a telehealth evaluation.

Unexpectedly, caregiver satisfaction was not related to travel time saved. This is contrary to our hypotheses and previous findings showing families appreciated telehealth services due to time and money saved related to travel (Corona et al., 2021; Reese et al., 2015). These findings have implications for increased acceptance and utility of telehealth evaluations for children and their families regardless of their distance from the provider. For some caregivers who may be within close proximity to the provider’s location, they may still prefer and be highly satisfied with a telehealth evaluation. This inconsistent finding may also be closely related to the COVID-19 pandemic. Specifically, caregivers may have been more focused on and were satisfied with the ability to still obtain access to services during these unprecedented times rather than their concern with the amount of travel time saved. Further, these findings could be related to general feelings of safety during the pandemic regardless of travel time saved. Additional work is needed to further clarify and understand the relationship between access to care and telehealth ASD evaluations.

Importantly, socioeconomic factors such as household income or the type of insurance (Medicaid versus private insurance) a child had at the time of the evaluation did not impact caregiver satisfaction. This may provide preliminary evidence that individuals at all economic levels who can benefit from ASD telehealth evaluations are reporting similar satisfaction levels. However, given that we excluded non-English speaking families and our reduced survey completion rate by caregivers (approximately 46% completion rate), those who are non-English speaking and who did not complete the post-survey may be more vulnerable to these socioeconomic impacts, potentially affecting caregiver satisfaction with the telehealth diagnostic model. Therefore, additional research is needed to examine if telehealth evaluations may be an optimal option for leveling access to high quality ASD evaluations for all children regardless of their socioeconomic status. Lastly, caregiver satisfaction was not related to provider reported technical or setup difficulties. This could potentially be due to caregivers being unaware that there were any difficulties with the audio/video or their current home setup for the evaluation, especially since providers reported technical or setup difficulties in less than 20% of all visits. Further, provider skill in managing technical difficulties or setup issues could also have played an important role. As noted above, it is imperative for providers to receive extensive training in telehealth ASD evaluations that includes not only how to administer and interpret the diagnostic assessment tools reliably and accurately, but also to manage and troubleshoot technical and/or setup/home environment difficulties.

Similar to caregivers, providers also reported high rates of satisfaction overall with the telehealth evaluations, with 94% of all providers being satisfied or very satisfied with the evaluations conducted. Providers in the study included both psychologists with an expertise in ASD evaluation and pediatricians trained in ASD assessment and evaluation. Incorporation of data from both psychologists and pediatricians is consistent with previous literature focused on ASD diagnostic telehealth evaluations (Corona et al., 2021); notably, the present study was novel with the majority of providers being psychologists. When collapsing all providers into one group, provider satisfaction was not significantly related to caregiver satisfaction; however, differences between disciplines (psychologists versus pediatricians) were found. Specifically, caregiver satisfaction with the provider was positively related to psychologist satisfaction, but this relationship was not found with pediatricians. Of note, our sample of evaluations conducted by pediatricians was much smaller than those conducted by psychologists, and this could be one reason for this finding. Overall, both pediatricians and psychologists reported high rates of satisfaction, providing preliminary evidence for the feasibly of telehealth evaluations using the TELE-ASD-PEDS in both provider types. Future studies could examine differences in caregiver satisfaction with telehealth ASD evaluations across a variety of different providers (e.g., pediatricians, psychologists, developmental pediatricians).

Provider satisfaction increased when they were able to provide any diagnosis (e.g., ASD, developmental delay, speech/language delay, behavior, etc.) as a result of telehealth evaluation. Similar to caregivers, provider satisfaction was also higher if they were able to provide an ASD diagnosis and they were certain about their diagnosis. As noted above, investing in proper training, education, and supervision to empower providers with the confidence to make a diagnostic determination is important to increasing acceptance of telehealth evaluations for ASD. Further, best practices in the evaluation of ASD include the combination of caregiver interview and observational assessment (Hyman et al., 2020). Having well developed, evidence-based diagnostic and screening observation tools, like the TELE-ASD-PEDS (Wagner et al., 2020, 2021) is necessary to increase the ability of providers to offer effective telehealth evaluations, ultimately leading to an appropriate diagnosis.

Not surprisingly, provider satisfaction was directly related to the provider’s preferred method of evaluation; those who preferred telehealth reported higher levels of satisfaction. Similar to caregivers, provider preference for evaluation is also important to consider. Although this was not analyzed more extensively in the present study, it is likely that diagnostic uncertainty played a role in provider preference for an in-person visit. While telehealth may work well for identifying and diagnosing some children with ASD, it may not be ideal for all. Our findings contrast with those reported by Becevic et al., (2015) where providers reported that they could adequately deliver services via telehealth, yet t preferred in-person visits. It is unclear from their findings as to why this preference was indicated given their overall high ratings of satisfaction with using telehealth. One potential reason for differential findings when compared to the present study is that they collected data regarding general preferences for telehealth versus in-person evaluations, while our study assessed preferences for evaluation modality for each telehealth assessment conducted. As noted above, offering a hybrid model for some families that allows both telehealth and an in-person option, if the telehealth evaluation does not lead to a clear diagnostic determination, may be helpful for providers in the future.

Consistent with previous findings (Wagner et al., 2020, 2021), providers who reported experiencing no technological or setup difficulties during the telehealth appointment reported higher levels of satisfaction. Technology is unfortunately fallible; however, as highlighted above, technical and setup difficulties occurred in less than 20% of the telehealth evaluations, adding to the feasibly of utilizing telehealth to diagnosis ASD. Notably, despite these technical and setup difficulties being reported by the providers, caregiver satisfaction was unrelated and not impacted. This suggests telehealth ASD evaluations can still be a useful tool for providers despite the limitations to come with using technology. Further exploration is needed into how the different technical and setup difficulties may impact caregiver and provider satisfaction.

From a clinical utility perspective, several key considerations from these data emerge with regards to how other clinics or hospitals may consider provider and clinician satisfaction in the utilization of telehealth ASD evaluations moving forward and beyond the current pandemic. More specifically, many of the variables examined in the present study could be screened for at the time of referral to better prioritize triaging children into a telehealth ASD evaluation. For example, clinics could consider prioritizing children who are more impaired developmentally and with high ASD symptoms severity by administering a questionnaire asked via phone by triage staff or from commonly used ASD and developmental milestone screeners in the primary care community often included in referrals (e.g., Modified Checklist for Autism in Toddlers, Revised with Follow-up [M-CHAT-R/F; Robins et al., 2009]; Ages and Stages Questionnaire, Third Edition [ASQ-3; Squires et al., 2009]).

Furthermore, assessing access to the necessary technology needed for a telehealth visit at the time of triage, including the details of what to expect with a telehealth visit, what materials/toys are needed, and how the appropriate technology setup (e.g., access to reliable internet, a device with a camera and audio) and ideal home environment are important to a successful telehealth visit. It would also be important to gauge caregiver preference for an in-person or telehealth appointment. As previously highlighted, the utility of a hybrid model including both telehealth and in-person evaluation may be ideal. For example, the telehealth evaluation could be the first step in the diagnostic process followed by an in-person evaluation; however, if the telehealth evaluation is conclusive, it may not be necessary to bring the patient in for an evaluation which could minimize the delay in getting access to the appropriate diagnosis and subsequent treatment.

Lastly, clinics and hospitals should consider investing in appropriate and extensive training for their providers in telehealth evaluation and the use evidenced-based ASD telehealth diagnostic tools. As demonstrated by our results, provider confidence and certainty to make an accurate diagnosis, including ruling in or out ASD, is important to provider and caregiver satisfaction. Given the unprecedented circumstances, the need for telehealth has shifted and continues to remain at the forefront for many providers and caregivers, but the utility, feasibility, and benefits that telehealth evaluations has to offer suggests it is an important tool we should continue utilizing into the future.

Limitations and Future Directions

Although the present study expands and builds on the current literature with provider and caregiver satisfaction of telehealth ASD evaluation, there are limitations that need to be considered. This study did not collect data from families who were unsuccessful in accessing their telehealth visit (did not attend the scheduled appointment), from families who declined a telehealth visit when offered, or the differences in wait times for in-person and telehealth visits. Gathering this information could increase our understanding of the families who felt comfortable choosing telehealth and those who did not, which may help providers and families in the decision-making process. This may also help with triaging patients appropriately to an in-person or virtual evaluation at the time of referral and better understand accessibility of an ASD evaluation based on wait times for either a telehealth or in-person evaluation.

Our results are also limited and should interpreted with caution for patients who are Hispanic/Latinx due to low sample size for this population, as well as exclusion of non-English speaking families. Additionally, caregiver response to the follow-up survey was 46%. This return rate exceeded those documented in a recent systematic review of patient health care and professional survey return rates (Meyer et al., 2020). However, it is important to note that our findings may not be representative of all families who engage in telehealth ASD evaluation and should be considered when interpreting our findings. A notable strength of our study is the larger sample size as compared to previous telehealth ASD studies (Corona et al., 2021; Juárez et al., 2018). Additional work using large samples is critical for moving the field of telehealth evaluations forward in addition to exploring other methods of increasing caregiver participation in post-evaluation survey collection. Further, important questions remain surrounding the longitudinal impact of telehealth ASD evaluations on children and their families, including the ability to access services and interventions. Lastly, future work should include the exploration of mediating factors, such as demographic variables or ASD symptom severity, on caregiver and provider satisfaction with ASD telehealth evaluations.

Our sample of providers was also unbalanced with there being substantially more psychologist-led telehealth evaluations in comparison to pediatricians; however, this was a novelty in comparison to previous work (Corona et al., 2021; Becevic et al., 2015). Further, we did not control for multiple comparisons due to the focused nature of our analyses and individual reporting of p-values; however, our findings between psychology providers and caregiver satisfaction may not have survived error correction prompting caution when interpreting these results. Lastly, additional research is needed to better understand the technology and setup barriers that providers encounter during telehealth evaluations. For example, are there certain barriers that are more impactful to provider satisfaction and if so, can these be identified at the triaging process to exclude patients from being scheduled for telehealth evaluations and route them directly into an in-person only visit.

At the time of the COVID-19 pandemic, access to diagnostic evaluations for children referred for concerns of ASD took a significant hit. There were limited tools developed for telehealth ASD evaluations, limited use of telehealth diagnostic evaluations in this population, and the evidence-based literature supporting the feasibility and utility is still in its infancy. Families were given the option of transitioning to virtual appointments or waiting for in-person assessments to resume, with no clear indication of when that may occur. Overall, our results build on the feasibility and utility of telehealth ASD evaluations with the large majority of providers and caregivers being satisfied and their satisfaction being related to different child characteristics, diagnostic outcomes, and aspects of the telehealth evaluation. Specifically, caregivers reported higher rates of satisfaction for children with lower adaptive skills (communication skills, daily living skills, and socialization skills) and greater ASD symptom severity, if they had a preference for telehealth evaluation, and if the provider was certain about their child’s diagnosis. For providers, satisfaction was higher for children presenting with greater ASD symptom severity, there were no technical or setup difficulties, if they provided an ASD diagnosis, and if they were certain about their diagnosis. The feedback collected by providers and caregivers highlight both acceptability of the telehealth model in addition to key factors to be considered for optimizing the use of telehealth in this population. Future work is needed to validate our results and continue the exploration of these, and other variables related to provider and caregiver satisfaction with telehealth evaluations for ASD.

Acknowledgements

We are most grateful to the children and families who participated in both the clinical care and research portions of this study. We are also very grateful to members of our team that supported this project, including our director (Angela Tomlin, PhD.) and our social work team (Carrie Fitterling, LCSW & Margo Ramaker, LCSW). Additionally, we are grateful to the many providers and LEND (Leadership Education in Neurodevelopmental Disabilities) trainees in the Division of Developmental Medicine, including Angela Paxton, Leah Bode, Allison Etling, Brett Enneking, Maura Rouse, Tybytha Ryan, and Alyssa Jones for their contributions to study development and data collection and management.

Author Contributions

DLR, EH, and CR contributed to the manuscript preparation and revisions. DLR contributed to study aims, planned analyses, and interpreted the data. QT contributed to study analyses and interpretation. CJ contributed to data extraction and preparation, and to manuscript revisions. RMK conceptualized and designed the study and contributed to data collection and manuscript revisions. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Declarations

This research was supported by the Robert & Helen Haddad Family Foundation and Riley Children’s Foundation (award to RMK) and T73MC00015-27-01 (HRSA; PI: Tomlin). The authors have no relevant financial or non-financial interests to disclose.

Footnotes

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. Alfuraydan M, Croxall J, Hurt L, Kerr M, Brophy S. Use of telehealth for facilitating the diagnostic assessment of autism spectrum disorder (ASD): A scoping review. PloS one. 2020;15(7):e0236415. doi: 10.1371/journal.pone.0236415. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. American Psychiatric Association . Diagnostic and statistical manual of mental disorders. 5. Arlington, VA: Author; 2013. [Google Scholar]
  3. Antezana., Scarpa A, Valdespino A, Albright J, Richey J. Rural trends in diagnosis and services for autism spectrum disorder. Frontiers of Psychology. 2017;8:590. doi: 10.3389/fpsyg.2017.00590. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Becevic M, Boren S, Mutrux R, Shah Z, Banerjee S. User satisfaction with telehealth: Study of patients, providers, and coordinators. The Health Care Manager. 2015;34(4):337–349. doi: 10.1097/HCM.0000000000000081. [DOI] [PubMed] [Google Scholar]
  5. Corona, L., Hine, J., Nicholson, A., Stone, C., Swanson, A., Wade, J., et al. (2020). TELE-ASD-PEDS: A Telemedicine-based ASD Evaluation Tool for Toddlers and Young Children. Vanderbilt University Medical Center. https://vkc.vumc.org/vkc/triad/tele-asd-peds
  6. Corona LL, Weitlauf AS, Hine J, Berman A, Miceli A, Nicholson A, Warren Z. Parent perceptions of caregiver-mediated telemedicine tools for assessing autism risk in toddlers. Journal of Autism and Developmental Disorders. 2021;51(2):476–486. doi: 10.1007/s10803-020-04554-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Crane L, Chester JW, Goddard L, Henry LA, Hill E. Experiences of autism diagnosis: A survey of over 1000 parents in the United Kingdom. Autism. 2016;20(2):153–162. doi: 10.1177/1362361315573636. [DOI] [PubMed] [Google Scholar]
  8. Gordon-Lipkin E, Foster J, Peacock G. Whittling down the wait time: Exploring models to minimize the delay from initial concern to diagnosis and treatment of autism spectrum disorder. Pediatric Clinics of North America. 2016;63(5):851–859. doi: 10.1016/j.pcl.2016.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Howlin P, Moore A. Diagnosis in Autism: A survey of over 1200 patients in the UK. Autism. 1997;1(2):135–162. doi: 10.1177/1362361397012003. [DOI] [Google Scholar]
  10. Hyman, S. L., Levy, S. E., Myers, S. M., Kuo, D. Z., Apkon, S., Davidson, L. F. … Bridgemohan, C. (2020). Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics, 145(1). Iacono, T., Dissanayake, C., Trembath, D., Hudry, K., Erickson, S., & Spong, J. (2016). Family and practitioner perspectives on telehealth for services to young children with autism. Studies in Health Technology and Informatics, 231, 63–73. doi: 10.3233/978-1-61499-712-2-63 [PubMed]
  11. Jang J, White SP, Esler AN, Kim SH, Klaiman C, Megerian JT, Kanne SM. Diagnostic evaluations of autism spectrum disorder during the COVID-19 pandemic. Journal of Autism and Developmental Disorders. 2021 doi: 10.1007/s10803-021-04960-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Juárez AP, Weitlauf AS, Nicholson A, Pasternak A, Broderick N, Hine J, Warren Z. Early identification of ASD through telemedicine: Potential value for underserved populations. Journal of Autism and Developmental Disorders. 2018;48(8):2601–2610. doi: 10.1007/s10803-018-3524-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Kanne SM, Carpenter LA, Warren Z. Screening in toddlers and preschoolers at risk for autism spectrum disorder: Evaluating a novel mobile-health screening tool. Autism Research. 2018;11:1038–1049. doi: 10.1002/aur.1959. [DOI] [PubMed] [Google Scholar]
  14. Lord C, Rutter M, DiLavorne PC, Risi S, Gotham K, Bishop SL. Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) Torrance, CA: Western Psychological Services; 2012. [Google Scholar]
  15. Maenner MJ, Shaw KA, Baio J, Washington A, Patrick M, DiRienzo M, Dietz PM. Prevalence of autism spectrum disorder among children aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016. Morbidity and Mortality Weekly Report Surveillance Summaries. 2020;69(SS-4):1–12. doi: 10.15585/mmwr.ss6904a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Malik-Soni N, Shaker A, Luck H, Mullin AE, Wiley RE, Lewis MES, Frazier TW. Tackling healthcare access barriers for individuals with autism from diagnosis to adulthood. Pediatric Research. Advance online publication. 2021 doi: 10.1038/s41390-021-01465-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. McNally Keehn R, Enneking B, James C, Tang Q, Rouse M, Hines E, Etling A. Telehealth evaluation of pediatric neurodevelopmental disabilities during the COVID-19 pandemic: Clinician and caregiver perspectives. Journal of Developmental & Behavioral Pediatrics. 2021 doi: 10.1097/dbp.0000000000001043. [DOI] [PubMed] [Google Scholar]
  18. Meyer, V. M., Benjamens, S., Moumni, E., Lange, M., J. F., & Pol, R. (2020). Global overview of response rates in patient and health care professional surveys in surgery: a systematic review.Annals of Surgery [DOI] [PMC free article] [PubMed]
  19. Nazneen N, Rozga A, Smith CJ, Oberleitner R, Abowd GD, Arriaga RI. A novel system for supporting autism diagnosis using home videos: Iterative development and evaluation of system design. JMIR mHealth and uHealth. 2015;3(2):e68. doi: 10.2196/mhealth.4393. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Quigley SP, Blevins PR, Cox DJ, Brodhead MT, Kim SY. An evaluation of explicit ethical statements in telehealth research with individuals with autism spectrum disorder. Behavior Analysis: Research and Practice. 2019;19(2):123–135. doi: 10.1037/bar0000094. [DOI] [Google Scholar]
  21. Pierce K, Gazestani VH, Bacon E, Barnes CC, Cha D, Nalabolu S, Courchesne E. Evaluation of the Diagnostic Stability of the Early Autism Spectrum Disorder Phenotype in the General Population Starting at 12 Months. JAMA Pediatrics. 2019;173(6):578–587. doi: 10.1001/jamapediatrics.2019.0624. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Robins D, Fein D, Barton M. Modified Checklist for Autism in Toddlers, revised with follow-up (M-CHAT-R/F) Storrs: Self-published; 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Reese RM, Braun MJ, Hoffmeier S, Stickle L, Rinner L, Smith C, Hadorn M. Preliminary evidence for the integrated systems using telemedicine. Telemedicine Journal and e-Health. 2015;23(7):581–587. doi: 10.1089/tmj.2014.0124. [DOI] [PubMed] [Google Scholar]
  24. Reese RM, Jamison R, Wendland M, Fleming K, Braun MJ, Schuttler JO, Turek J. Evaluating interactive videoconferencing for assessing symptoms of autism. Telemedicine Journal and e-Health. 2013;19(9):671–677. doi: 10.1089/tmj.2012.0312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Sparrow, S. S., Cicchetti, D., & Saulnier, C. (2016). Vineland adaptive behavior scales–third edition (Vineland-3). San Antonio, TX: Pearson
  26. Squires J, Bricker DD, Twombly E. Ages & stages questionnaires (pp. 257 – 182) Baltimore, MD, USA:: Paul H. Brookes; 2009. [Google Scholar]
  27. Stainbrook JA, Weitlauf AS, Juárez AP, Taylor JL, Hine J, Broderick N, Warren Z. Measuring the service system impact of a novel telediagnostic service program for young children with autism spectrum disorder. Autism. 2019;23(4):1051–1056. doi: 10.1177/1362361318787797. [DOI] [PubMed] [Google Scholar]
  28. Stone, W., & Ousley, O. Y. (2008). Screening tool for autism in toddlers and young children (STAT). Vanderbilt University
  29. Sutantio JD, Pusponegoro HD, Sekartini R. Validity of telemedicine for diagnosing autism spectrum disorder: Protocol-guided video recording evaluation. Telemedicine Journal and e-Health. 2021;27(4):427–431. doi: 10.1089/tmj.2020.0035. [DOI] [PubMed] [Google Scholar]
  30. Talbott MR, Dufek S, Zwaigenbaum L, Bryson S, Brian J, Smith IM. Brief report: Preliminary feasibility of the TEDI: A novel parent-administered telehealth assessment for autism spectrum disorder symptoms in the first year of life. Journal of Autism and Developmental Disorders. 2020;50:3432–3439. doi: 10.1007/s10803-019-04314-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Wagner L, Corona LL, Weitlauf AS, Marsh KL, Berman AF, Broderick NA, Warren Z. Use of the TELE-ASD-PEDS for autism evaluations in response to COVID-19: Preliminary outcomes and clinician acceptability. Journal of Autism and Developmental Disorders. 2020;51:3063–3072. doi: 10.1007/s10803-020-04767-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Wagner L, Weitlauf AS, Hine J, Corona LL, Berman AF, Nicholson A, Warren Z. Transitioning to telemedicine during COVID-19: Impact on perceptions and use of telemedicine procedures for the diagnosis of autism in toddlers. Journal of Autism and Developmental Disorders, Jun. 2021;4:1–11. doi: 10.1007/s10803-021-05112-7.. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Autism and Developmental Disorders are provided here courtesy of Nature Publishing Group

RESOURCES