Digital phenotyping could help detect autism

Abstract

Researchers have developed a screening tool for autism that uses computer vision and machine learning to analyze autism-related behaviors – but greater reliability and robust validation will be needed if such tools are to be used in primary care settings.

Objective, reliable and scalable digital tools could improve the ability of primary care practitioners to detect autism spectrum disorder (ASD) in young children. But before such tools can be used clinically, they must be reliable at an individual level (that is, a child must receive the same score consistently within a short period of time)¹ and their value must be established across populations – including those historically facing disparities in timely ASD diagnosis.

In this issue of Nature Medicine, Perochon et al.² present an innovative, multi-task digital phenotyping tool that was able to detect the likelihood of ASD in toddlers, with good sensitivity and specificity, in a large sample of predominantly white, neurotypical children. The tool, called SensetoKnow, uses computer vision technology and machine learning and has many impressive features. Children’s eye mvements, head movements and other features are measured as the child watches carefully designed, brief videos for 10 minutes on a digital device during a well-child visit. The tool builds on well-established tasks rooted in research and was able to successfully identify nearly 88% of the autistic 17- to 36-month-old children who were assessed.

Yet there are continued issues, some of which have recurrently plagued other attempts at early detection of ASD³, and others that are specific to this study. Most striking is that although the sensitivity and specificity of the tool were good, the positive predictive values – that is, the proportion of children who screened positive with the tool who were true positives – ranged from less than 13% for the youngest toddlers to 40% for 24- to 36-month-old children. This means that more than twice as many children who did not have ASD were identified as positive than children who actually did have ASD. It is striking that another digital phenotyping measure of ASD, a tool known as GeoPref (which takes just one minute), found opposite results, with very high specificity but poor sensitivity⁴. The implications for a primary care physician who must interpret and deliver such results are daunting. Not only are there concerns about how this affects families whose children are misidentified as having ASD, but it also seems likely that such experiences would affect physicians’ use of such devices. Similar issues have arisen with other ASD-screening strategies; when potential ASD was indicated in early screening (at the 1-year well-child visit), many physicians did not follow up on this information even when referral services were offered⁵.

A second major concern – again, not unique to SensetoKnow, but a general issue with digital phenotyping – is that the accuracy of the device across race, gender and age is not clear⁶. Although the full sample is large, Perochon et al.² included only 49 children who were identified as having ASD, and so was not powered to address age, race and quality of data differences. There were no statistically significant effects of race or sex or age, but when the data are perused closely, girls, Black children and young toddlers were overidentified as potentially having ASD at high rates. The children with ASD were, on average, significantly developmentally delayed, particularly in receptive language, and even more delayed than the non-autistic delayed group. Thus, it is possible that what was identified (when SensetoKnow worked accurately) was general delay or even limited receptive language.

A strength of SensetoKnow is the range of tasks and the engineering that enables integration of multiple types of data including eye-tracking and other information – particularly when taking into account the vagaries of showing videos to a toddler on a small device. The study from Perochon et al.² contains hints about these issues – the quality of the score (that is, the rating of data acquisition) was reported and appeared to have an effect on the specificity of the results (again resulting in overdiagnoses in the small number of lower-quality results). More information will be needed to guide healthcare providers in what this means. Another strength of the study is that data are presented about the increased sensitivity and specificity (but not about positive predictive values) of combining the SensetoKnow results with information from M-CHAT, a parent report measure⁷.

Recognition of early ASD symptoms and appropriate evaluation and diagnosis are often significantly delayed by stigma and socioeconomic, racial and ethnic disparities. Another cause of delays is lack of access to higher-level practice providers, such as child neurologists, child psychiatrists and psychologists, and developmental behavioral pediatricians, who can confirm a diagnosis and work with families to determine appropriate intervention strategies. Many autistic children have issues with attention and cognition, motor skills and language. Given that there is no single treatment for all autistic children, even with accurate early identification of ASD, there will always be a need to support families in the long term as they seek personalized intervention programs that may vary from intense multi-disciplinary interventions to supported placement in ordinary preschool programs and other specific therapies, based on individual needs⁸.

Historically, ASD diagnoses based on comprehensive evaluations were considered one of the most reliable diagnoses in child psychiatry – although recently, concerns about over-diagnosing have been raised, particularly on the basis of more restricted information⁹. ASD diagnoses are intended to be based on observations and on information from caregivers about a set of behaviors relating to social communication and restricted, repetitive or sensory behaviors¹⁰. Thus, core elements of diagnosis often depend on parents’ awareness of typical and atypical development¹¹. A contribution of SensetoKnow to clinical practice, once reliability is established, would be to provide primary care practitioners a way to bring potential issues to caregivers’ attention to help increase awareness, if this could be done without causing undo alarm.

The creation of devices and strategies that enable objective measurement of child development is important to the degree that it gives providers accurate information that can be used to increase parent and professional awareness and determine next steps. Researchers have focused on early detection of ASD, but future applications of digital phenotyping tools could focus on identifying and monitoring factors that contribute to ASD, such as limited social interactions, or those that co-occur with ASD, such as motor difficulties or general developmental delays¹². Practitioners may benefit from this information, which might give an accurate representation of observations at the individual level while also reflecting the broad heterogeneity of developmental disabilities. Implementation studies of how providers use information – particularly probabilistic information that could change a family’s perception of their child – will be critical in building on the impressive digital technology that the SensetoKnow tool represents.