Video-based screening for children with suspected autism spectrum disorder – experience during the COVID-19 pandemic in India

Abstract

Background

Assessments for children with autism spectrum disorder (ASD) must adapt to the current COVID-19 pandemic through innovation in screening and assessment strategies using technology. To our knowledge there are no such studies reported from India. We aimed to study the predictive ability of video-based screening tool with definitive diagnosis in children with ASD.

Method

Thirty-nine children were screened independently by two examiners with a video-based screening tool to start intervention followed by an in-person evaluation by clinical DSM-5 diagnosis three months later.

Result

Similar to studies from developed countries, videos assessments showed a 94.87% correlation with the final diagnosis. Interobserver video agreement had a kappa correlation of 0.803, which was classified as substantial agreement.

Conclusion

Video-based evaluations may be used as an interim assessment to initiate early intervention in children with ASD in resource-limited setups in the current pandemic situation. Large, well-designed prospective studies are required to confirm our results.

1. Introduction

India has so far reported the second-highest number of Covid-19 cases in the world. The enormous burden of the pandemic on the already stretched healthcare services and extended lockdowns have severely restricted the delivery of non-emergency health care facilities, especially in the field of developmental pediatrics.

Autism spectrum disorder (ASD) is defined as significant impairment in social communication in addition to the presence of repetitive behaviors and restricted interests (American Psychiatric Association, 2013)(American Psychiatry Association, 2013). ASD is one of the most common developmental disabilities. Approximately 1/100 children are diagnosed with autism spectrum disorder around the world (Zeidan et al., 2022). With increased societal awareness, children with ASD are identified earlier by parents and caregivers. Early diagnosis and intervention are known to optimize the developmental outcomes in ASD (Dawson et al., 2010).

Healthcare services for children with ASD are maximally available in tertiary care hospitals in capital or metro cities in India. Travel restrictions due to the COVID 19 pandemic have severely affected the access to tertiary care developmental centers for assessment and interventions for these children. Parents seeking the first consultation for developmental concerns about their children face enormous stress, further aggravated while waiting for the travel restrictions to be lifted or accessing in-person observations with the specialty clinics. Inevitably, precious time is lost for diagnosis and early intervention. As a result, ASD assessment must adapt to this changing landscape through innovation in screening and assessment strategies (Dahiya et al., 2021).

The systematic review undertaken by Dahiya et al. (2021) included sixteen studies, fourteen from high-income countries which used some form of technology-based ASD screening or assessment in children till 12 years of age. Overall, the results of this systematic review support the effectiveness of technology-based tools in optimizing service delivery for ASD during situations such as the pandemic.

The value of diagnosing ASD through telemedicine has been investigated, using either live videoconferencing in prepared facilities or a store-and-forward approach that facilitates sharing behavioral examples via video recordings. Wagner et al. (2021) used the TELE-ASD-PEDS, a tool specifically developed for caregiver-mediated telemedicine evaluation of ASD. They could identify ASD in 71 % of children, which helped early intervention.

Limited data is available on telemedicine and online assessments for ASD from low-middle-income countries (LMIC). A recent study from LMIC by Sutantio et al. (2021) from Indonesia used a store and forward approach to evaluate the validity of protocol-guided video recording compared with direct assessment for diagnosing ASD. Parents were instructed to video record specific scenarios. The in-person assessment was done using DSM-5 criteria by a separate examiner. The diagnostic agreement between the two methods was 82.5 %.

It is important to note that the studies that utilized in-vivo observation took place in a clinical setting with standard materials. As such, this may not be feasible during a pandemic lockdown, as many standardized materials often need to be sent to parents. To our knowledge there are no similar studies reported from India. The current tools used in India for diagnosis of ASD are the Indian Scale for Assessment of Autism (ISAA) (Chakraborty et al., 2015) and the INCLEN Diagnostic Tool for Autism Spectrum Disorder (INDT-ASD). (Gulati et al., 2019) Both have not been standardised for video-based observations. In addition, the ISAA needs simultaneous observation and an assessment kit that needs to be transported. Considering the need of the hour, we developed an informal video-based interim assessment with toys from the child’s home environment to screen children with suspected ASD. We hypothesized that such an assessment of children will provide an effective screening tool for later confirmed diagnosis of ASD. Therefore, the aim of the study was to study the predictive ability of video-based screening tool with definitive diagnosis in children with ASD.

2. Methods

2.1. Design and setting

This prospective cohort study was conducted in a tertiary teaching hospital in India between May 2021 and Oct 21.

2.2. Ethics

Approval from the institutional ethics committee was obtained before commencing the study.

2.3. Participants

2.3.1. Inclusion criteria

(1) Parents seeking first consult for concerns of speech delay and social responsiveness in their child. (2) Children aged 18 months to 5 years (3) No intervention commenced. (4) Parents unable to travel immediately for an in-person evaluation (5) Written consent for a video and an in-person evaluation at a later date.

2.3.2. Exclusion criteria

(1) Children who already have a diagnosis of ASD. (2) Children previously diagnosed with vision or hearing impairment. (3) Parents refusing consent for video consultation.

2.3.3. Sample size calculation

An estimated total sample size of 39 children was considered adequate using kappa statistics with 80% correlation between 2 observers, a confidence level of 95 %, and loss to follow up of 10 %.

2.4. Materials and design and procedures

2.4.1. Provisional diagnosis

A written informed consent for the video consult was taken from all the parents. The consent form mentioned that the video assessment was an interim assessment to start intervention if necessary. The final diagnosis would be confirmed by a trained in-person evaluation conducted three months later. The store-and-forward approach was used wherein parents recorded videos during the day-to-day activities, including natural expressions of the child's behavior. These home recordings could be carried out over the course of several days. The parents received a telephone call prior; to explain how to send a summary of their concerns and the video recording (~1–2 min) of eight discrete caregiver-led activities or social bids, including opportunities for interactive play, imitation, directing attention, and requesting. The videos were shot using an android device, preferably the parent's personal mobile phone, which the child was used to. The back camera was used to avoid distraction for the child. The parents were also sent the Modified Checklist for Autism in Toddlers Revised with Follow up (M-CHAT-R/F) for children under 30 months of age and the Ages and Stages Questionnaire (ASQ-3). The consent form, questionnaires, and videos were sent to the nodal person's mobile number two days before the appointment.

The video recordings were sent to two independent clinicians. The clinician (AK) who designed the format of the videos is an ADOS and ADI-R trained developmental pediatrician with 16 years of clinical experience. The other clinician (IS) is an occupational therapist with ten years of clinical experience in the intervention of children with ASD. Both decided on a consensus rating for each behaviour variable. Both were blinded to each other's findings and provided ratings on ten behaviours. These included response to name, hearing, age-appropriate word use, age-appropriate gesture use, unusual speech, eye contact, sensory concerns, behaviour for need, body to communicate, and age-appropriate play skills using dichotomous (Yes/No) procedures. ( Table 1). Each clinician was asked to classify children as ASD yes or no; guided by Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) checklist for ASD (Carpenter, 2013).

Table 1.

Ten behaviours assessed in the videos.

Serial no	Variables	Yes	No
1	Response to name: When child is involved with a toy; call him or her by name
2	Hearing: If no response to name, play favourite song or rhyme or toy
3	Eye contact: check during play when needs help, response to name, sharing the play or tasks, during rhyme time
4	Age-appropriate gestures: check during play, need, rhyme time, picture book
5	Age-appropriate word use: check during doll play, for need, rhyme time, hide and seek or ball play time
6	Unusual speech: echolalia, self-talk, prosody, jargon, neologisms, scripted speech
7	Body to communicate: check when need arises, help during tasks, rhyme time, picture book task
8	Age-appropriate play: check with doll play- functional, imitation, imaginational representational, multi scheme play. Compare with puzzle play as well
9	Behaviour for need: check for tantrums for need instead of gesture, word or eye contact.
10	Sensory concerns: flicking items, lining, throwing, mouthing, rocking, hand flapping

The same developmental pediatrician (AK) also did an online live evaluation through the WhatsApp video call facility. During the online session, a detailed history was obtained. In case of suboptimal video quality, the parents were asked to demonstrate some tasks in the live session with guidance by the clinician if required, and the ratings on the ten behaviours noted in the videos were reconfirmed. Two weeks later, a repeat online session was organized to counsel the parents and initiate intervention. Each child was given an individualized stimulation program and commenced on online therapy applied behaviour analysis (ABA), Speech therapy, and Occupational therapy as per clinical discretion.

2.4.2. Definitive diagnosis

The same clinician evaluated each child after three months by an in- person evaluation. During the in-person evaluation, current concerns, family history, birth history, and current school or child rearing practices of the child were verified. Parents were also asked about reported improvements observed and current therapy modalities initiated after the video diagnosis. A brief developmental screen to check for vision, hearing, neurological evaluation and a developmental screening tool; the TDSC (Trivandrum Developmental Screening charts 0–3years and 3–6 years)(Nair et al., 2013) was administered as per the age of the child. The children were diagnosed as ASD or No ASD; using DSM-5 criteria for ASD. Children were referred for Brain Stem Evoked Response Audiometry if clinically indicated. The children were also referred for further evaluations like speech, cognitive, sensory profile evaluations as per clinical discretion. Parents of children who were given an alternative diagnosis or needed a modification of therapy plans were counselled about the same.

Socioeconomic status was assessed by the updated Kuppuswamy's socioeconomic classification meant for urban population of India (Saleem, 2019). For every year, a new income range was calculated based on the All-India consumer price index for industrial workers. We used income ranges relevant to the study period. According to this classification, socioeconomic class is classified as Upper (26−29), Upper middle (16−25), Lower middle (11−15), Lower upper (5−10) and Lower (<5) class.

The findings of the other clinician who saw only the videos were revealed only after the final in-person diagnosis.

2.5. Analysis

The correlation between the video finding of both clinicians was assessed by using Cohen's Kappa with mean and range of kappa to be reported. Kappa is the ratio of observed agreement in classification beyond that what would be expected by chance. Scores range between − 1 and 1, in which 0 indicates that the observed agreement can be attributed completely to chance, 1 indicates perfect agreement and − 1 indicates no agreement at all. Cohen suggested the Kappa result be interpreted as follows: values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement. Examiner 1(AK) also did the in- person assessment using DSM − 5 criteria and since there was a chance of bias, we decided to correlate the examiner 2(IS) findings with the definitive diagnosis to avoid bias.

Continuous variables were expressed as mean ± standard deviation (SD) and categorical variables were presented as frequencies and percentages. The baseline data was expressed as mean and standard deviation in normal distribution and median and interquartile range (IQR) in case of skewed distribution.

3. Results

We enrolled 44 children who fulfilled the inclusion criteria, of which one did not follow up after the video evaluation, and four were unable to attend the in-person appointment due to travel restrictions. Thus, a total 39 children (27 boys, 12 girls) were included in the final analysis. Their mean (SD) age was 37.05 (10.738) months. Twenty children were from upper class, 18 from upper middle class and one from lower middle socioeconomic class.( Table 2).

Table 2.

Demographic characteristics of children enrolled in the study.

Gender Group N (%)	Boys	27 (69.23 %)
	Girls	12 (30.77 %)
Socioeconomic status N (%)	Upper class	20 (51.29 %)
	Upper middle	18 (46.15 %)
	Lower middle	1 (2.56 %)

Of the 39 children whose videos were assessed by both examiners (AK- rater 1) and (IS, rater 2); there was agreement between their diagnoses allocated in 37(94.87 %) children. They agreed that 32 children (82.05 %) children had ASD based on the video findings. They also agreed that 5 children (12.82 %) had no ASD based on their videos. They disagreed on their diagnoses in 2(5.13 %) children.

Rater 2 classified 33 children as having ASD and 6 as having no ASD based on video findings. Rater 1 who did the in-person assessment after 3 months classified 35 children as ASD and 4 children as no ASD. Thus, there was disparity in the diagnoses between rater 2 video findings and in-person diagnosis in 2 children. The agreement was in 94.87% children and disagreement in 5.13% children.( Table 3A, Table 3B).

Table 3B.

Comparison of diagnosis of video findings of rater 2 and in-person diagnosis.

	Definitive ASD: Yes	Definitive ASD: No
Rater 2 ASD: Yes	33 (84.62 %)	0 (0 %)	33 (84.62 %)
Rater 2 ASD: No	2 (5.12 %)	4 (10.26 %)	6 (15.38 %)
	35(89.74 %)	4 (10.26 %)	39 (100.00 %)

Kappa agreement between two raters’: kappa value: 0.772 p-value: < 0.001

Sensitivity = 33/35 = 94.29 % (95 % CI 80.84 – 99.30 %)

Specificity = 4/4 = 100.0 % (95 % CI 39.76 – 100 %)

Positive Predictive Value = 33/33 = 100 % (95 % CI 100–100 %)

Negative Predictive Value = 4/6 = 66.67 % (95 % CI 34.24 – 88.48 %)

Accuracy = 37/39 = 94.87 % = (82.68–99.37 %)

Table 3A.

Correlation between video diagnoses of both raters.

	Rater 1 ASD: Yes	Rater 1 ASD: No
Rater 2 ASD: Yes	32(82.05 %)	1(2.56 %)	33(84.62 %)
Rater 2 ASD: No	1(2.56 %)	5 (12.82 %)	6 (15.38 %)
	33(84.62 %)	6(15.38 %)	39 (100.00 %)

Kappa agreement between two raters’: kappa value: 0.803 p-value: < 0.001

We classified the children as per their age into two groups $\le$ 30 months and 30–60 months of age. We had 13 children $\le$ 30 months and 26 children were older than 30 months. The correlation between both the raters on video findings in children $\le$ 30 months was 100 %. On comparison of video diagnosis of rater 2 with in person diagnosis, the correlation was in 12 (92.31 %) children. Eleven children whom rater 2 thought had ASD were confirmed as ASD on the in-person diagnosis. Of the two children who rater 2 thought had no ASD, there was disparity in the diagnosis in one child. In all these children, the M-CHAT-R/F was administered. There was correlation in 12 (92.31 %) children. The same child in whom there was disparity between the video rater 2 and in person diagnosis had low risk on MCHAT-R-F too. We had 26 children in the age range of more than 30–60 months. There was correlation between both the raters in 24 (92.31 %) children. The correlation between rater 2 diagnosis on video and the in-person assessment was in 25 (96.15 %) children.

On ASQ-3 scoring; we found delay in the area of personal social skills domain had significant correlation with the in- person diagnosis of ASD. The other four areas; gross motor, fine motor, communication and problem solving did not have statistically significant correlation with the in-person diagnosis of ASD.

Early intervention was initiated after the video evaluation. Fifteen children were given ABA of which five had improvements in use of gesture for need and compliance to task. Sensory integration given to 25 children helped settle sensory issues, feeding issues in 3 children, while speech therapy helped foster need-based communication in 4 children out of 17 for whom it was commenced. These improvements were noted at the three month in-person appointment.

A Logistic Regression was carried out with in-person diagnosis of ASD by examiner 1 as the dependent variable to study the predictive ability of the ten video assessment variables by examiner 2 as independent (predictor) variables. None of the ten variables individually had a statistically significant p value of < 0.05, either in univariate or multivariate logistic regression analysis.

A model was yielded after putting all ten independent variables in multiple logistic regression analysis. The overall model was significant with a chi square value of 23.021 with a p-value of 0.011. The R-square value in this model was 0.446 (Cox and Snell R²) and 0.921 (Nagelkerke R²). Therefore, the explained variation in the dependent variable based on our model ranges from 44.6% to 92.1% depending on whether we reference it to the Cox and Snell R² or Nagelkerke R² methods respectively. The Hosmer-Lemeshow chi square test statistic (which tests the null hypothesis that predictions made by model fit perfectly with observed group memberships) yielded a non-significant p value of 1, indicating that the data fit the model well. With all independent variables added to the model, it correctly classified 97.4% (38 out of 39) cases, sensitivity of 97.1% (34/35) and specificity of 100% (4/4).( Table 4).

Table 4.

Behavioural variable and correlation with final diagnosis.

No	Variables	Unstandardized regression weight	SE	Significance
1	Response to name	-37.622	12499.523	.998
2	Hearing	20.128	14430.469	.999
3	Eye contact	18.715	10151.632	.999
4	Age- appropriate gestures	16.125	20641.511	.999
5	Age- appropriate word use	18.618	9155.523	.998
6	Unusual speech	-0.960	10067.908	1.000
7	Body to communicate	-1.095	17663.127	1.000
8	Age-appropriate play	-37.011	24294.472	.999
9	Behaviour for need	-36.514	13679.336	.998
10	Sensory concerns	.733	21308.560	1.000
	Constant	38.883	15662.809	.998

4. Discussion

Most studies on telediagnosis and video assessments for screening and diagnosis of ASD are from high-income countries rather than LMICs. To our knowledge there are no similar studies reported from India. The current tools used in India for diagnosis of ASD are the (Indian Scale for Assessment of Autism (ISAA) and the INCLEN Diagnostic Tool for Autism Spectrum Disorder (INDT-ASD). Given the limited resources in settings like ours and the travel restrictions due to the pandemic, wherein standardized material could not be transported to parents; precious time is lost for diagnosis. The significance of this issue cannot be overemphasized, considering the high prevalence of ASD and the need for early intervention for ASD in this setup. Keeping this in mind, we aimed to study the predictive ability of a video-based screening tool with toys from the child’s home environment (store and forward approach with a live video observation) for ASD with a definitive diagnosis by an in-person assessment later, to optimize early intervention during the pandemic. This study was conducted during the second wave of the pandemic in a tertiary center of a city in India with the highest number of Covid cases in the state then.

Videos assessments by two examiners blinded to each other's findings showed 94.87 % correlation with the final diagnosis with a kappa correlation of 0.803, which was classified as substantial agreement. Also, correlation between rater 2 video diagnosis and in person diagnosis was also 94.87 %. The sensitivity was 94.29 %, specificity 100 %, positive predictive value 100 % and negative predictive value was 66.67 %.

These findings are similar to those reported by studies from developed countries (Smith et al., 2017), included in the systematic review by Dahiya et al. (2021). Smith et al. used the scenario-based protocol (Naturalistic Observation Diagnostic Assessment [NODA] as a method of video observation and ADOS and ADI-R for in-person assessments. Smith et al. reported 84.9 % sensitivity and 94.4 % specificity. They too reported high PPV (96.5 %), but low NPV (54.4 %). Furthermore, out of the sixteen studies in the systematic review; ten studies included in the systematic review focused on screening whereas the remaining six involved diagnosis of ASD. Twelve studies are from high income countries, two from families of lower socioeconomic class of high-income countries and the remaining two are from LMIC. The studies that utilized in-vivo observation took place in a clinical setting with standard materials. As such, this may not be feasible during a pandemic lockdown, as many standardized materials often need to be sent to parents. Of the studies that used live video evaluations and video observations as a screening tool and compared the results with an in- person ADOS evaluation later, they found relatively high levels of specificity. Interobserver agreement was in the range of 75 % (lowest) to 88.2 % (highest). Overall, the systematic review supported the effectiveness of technology-based tools in optimising service delivery for ASD during situations such as the pandemic.

It is important to note that the studies included in the systematic review by Dahiya et al. (2021) are from the pre- COVID Era wherein a complete ADOS evaluation was feasible. In contrast, we used video-based screening tool with toys from the child’s home environment and a clinical DSM-5 diagnosis as an in-person evaluation.

In a recent study from the COVID Era, Wagner et al. (2021) used a telemedicine-based model of ASD assessment with an initial focus on children under 36 months of age. They used the TELE-ASD-PEDS (Corona et al., 2020a) a tool specifically developed for caregiver-mediated telemedicine evaluation of ASD for providers with expertise in the early diagnosis of ASD, facilitating the remote observation of ASD based upon caregiver-mediated interactions, using readily available toys and materials found in most families' homes.The clinicians used TELE ASD PEDs behavioural observations with information (clinical interview, adaptive information, spontaneous child behaviours observed throughout the visit) to form a diagnostic impression (ASD vs. no ASD). The TELE-ASD-PEDS, includes eight discrete, caregiver-led activities or social bids, including opportunities for interactive play, physical play routines. Seven behaviours were scored to decide on the final diagnosis based on dichotomous and Likert scoring procedure. A total of 204 children (Mean age: 27.54 years, Boys: 157, Girls: 47) were seen by 9 providers via tele consult. Of these, 71 % received diagnoses of ASD and 7 % had ASD Suspected. The diagnosis was uncertain in 11 %, and another 11 % had no ASD. All children with uncertain diagnosis were referred for an in-person evaluation. A total of 6 % of those with ASD diagnosis and 48 % of those with NO ASD diagnosis were referred for in person evaluation In our study where all the children were seen by an in -person evaluation. However, we had only two clinicians trained for the assessments.

The TELE-ASD-PEDS is designed to elicit symptoms of ASD and is standardized for children till 36 months of age. We had children till age 5 years who were on our waiting list then. Hence, we designed our own tool rather than use an already existing tool like TELE- ASD -PEDS.

Recently, Sutantio et al. (2021) from Indonesia used a store and forward approach to compare the validity of protocol-guided video recording with direct assessment for diagnosing ASD. Forty children aged 18–30 months with main complaints of delayed speech or social indifference and (M-CHAT-R) score > 2 were included. Parents were instructed to video record specific scenarios by utilizing the (NODA). In-person assessment was done using DSM-5 criteria by a separate examiner. To ensure similar competence levels, before the study, kappa statistical analysis had suggested overall agreement of diagnoses was 87.2 %, with 0.74 Cohen's Kappa, indicating substantial agreement. The diagnostic agreement between the video assessment and physical assessment was 82.5 %, similar to ours. The method used was similar to ours, but we supplemented our video recordings with live observations by one clinician in all our children to overcome poor quality video recordings or situations where the parent may not comprehend video recording instructions. They performed physical assessments within two weeks of the video assessment and revealed the findings of video observations only after the physical diagnosis. Due to the lockdown, our physical assessments were delayed to three months later, hence we used our video diagnosis to initiate intervention. Their sensitivity for diagnosing ASD was 91.3 %, similar to ours while the specificity was 70.6 %, lower than ours. The positive predictive value was 80.7 % similar to ours, while the negative predictive value was 85.7 %; higher than ours. They had a high false-positive number due to fewer video recordings and higher discrepancies in criteria.

A Logistic Regression was carried out with in-person diagnosis of ASD as the dependent variable to study the predictive ability of the ten video assessment variables by examiner 2 as independent (predictor) variables. None of the ten variables individually had a statistically significant p value of < 0.05, either in univariate or multivariate logistic regression analysis.

The strengths and limitations of our study need to be discussed. Both examiners in our study were blinded to each other's findings with substantial agreement (k- values 0.803). Examiner 2's assessment had a very good correlation (94.87%) with the final diagnosis by DSM-5. This suggests that video-based assessment could be used in the current pandemic situation to help diagnose ASD early and start early intervention. The main limitation of our study was that we had the same clinician (examiner 1) as part of the video and the in-person assessment. In India, there is paucity of resources for trained personnel. We have only one research reliable clinician trained on the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al., 2012) and ADI-R actively engaged in clinical practice related to ASD diagnosis with sixteen years’ experience (examiner 1) in our city. The examiner 2 is a certified occupational therapist with ten years of active clinical experience in intervention with children with ASD. Keeping this limitation in mind, we had the developmental pediatrician design the tool, brief examiner 2 about the behaviour variables to be observed and see the child again instead of examiner 2. We compared the blinded behavior variables on video findings of examiner 2 with the in-person evaluation by examiner 1. The unusual circumstances of the pandemic made it necessary for us to come up with this methodology to reduce waiting time for our children on the wait list to see the developmental pediatrician and provide best possible services at the earliest. We compared the blinded behavior variables on video findings of examiner 2 with the in-person evaluation by examiner 1. The small sample size of our study is another limitation. There were more children who had a diagnosis of ASD in the group. This may be due to the higher referrals of children with concerns of social communication being referred to our tertiary centre.

In summary, the results from our study suggests that video-based evaluations may be used as an interim assessment to initiate early intervention in children with ASD in resource-limited setups in the current pandemic situation. Large, well-designed prospective studies are required to confirm our results.

CRediT authorship contribution statement

Archana Kadam: Conceptualization, Formal analysis, Investigation, Writing – original draft preparation. Isha Godiwala Soni: Investigation. Sandeep Kadam: Methodology, Formal analysis. Anand Pandit: Supervision. Sanjay Patole: Supervision, Writing – review & editing.