Abstract
Using robots in therapy for children on the autism spectrum is a promising avenue for child-robot interaction, and one that has garnered significant interest from the research community. After preliminary interviews with stakeholders and evaluating music selections, twelve typically developing (TD) children and three children with Autism Spectrum Disorder (ASD) participated in an experiment where they played the dance freeze game to four songs in partnership with either a NAO robot or a human partner. Overall, there were significant differences between TD children and children with ASD (e.g., mimicry, dance quality, & game play). There were mixed results for TD children, but they tended to show greater engagement with the researcher. However, objective results for children with ASD showed greater attention and engagement while dancing with the robot. There was little difference in game performance between partners or songs for either group. However, upbeat music did encourage greater movement than calm music. Using a robot in a musical dance game for children with ASD appears to show the advantages and potential just as in previous research efforts. Implications and future research are discussed with the results.
INTRODUCTION
The emerging field of socially assistive robotics involves robots that assist people primarily via social interaction. Elderly users, individuals with physical impairments, individuals in rehabilitation facilities, individuals with cognitive disorders, and students could receive support for tasks, such as tutoring, physical therapy, self-care, and emotional expression (Feil-Seifer & Matarić, 2011). As robots become fixtures of our homes, hospitals, business, and schools, they will inevitably encounter children and, at least thus far, are frequently poorly equipped to handle these unpredictable little humans. At the same time, there is significant potential for robots to enhance children’s lives acting as tutors, companions, and assistants.
Beyond the motivation for children to interact with robots in general, there is a more specific goal of utilizing robots to assist children with special needs in a variety of ways. One idea that has garnered significant interest is using social robots to help children with Autism Spectrum Disorder (ASD). In this vein, researchers have been studying innovative ways to take advantage of the ability of robots to capture the attention of children with ASD in order to enhance children’s therapeutic gains.
This paper examines the interactions between children and robots in the context of a musical game, Dance Freeze, where players dance while music plays and then freeze when the music stops unexpectedly. It contrasts these interactions with the same game played with a human female partner. The responses of children with ASD are compared with their TD peers in the hopes of beginning to define how such interactions may be different and can be used in future robotics research for these populations. The current paper contributes to (1) designing a new type of game scenario that can be used in clinical sessions for children with ASD, (2) delineating the exemplary experimental environment and procedure for child-robot interaction so other researchers can follow, and (3) systematically investigating the distinctions of the two populations in interactions with the social robot.
A Note Regarding Terminology
There is considerable debate among people involved with the autistic community regarding how to appropriately refer to people with Autism Spectrum Disorder. Person-first language (e.g., a girl with ASD) is favored by some and disability-first language (e.g., an autistic man) by others. The debate is deeply nuanced and often passionate, but in broad strokes, person-first language, common in the medical community, is intended to prioritize the individuals with a disability over the condition. Meanwhile, disability-first language, closely associated with the disability rights movement, is intended to encompass a disability as an integral part of the individual’s identity. The former is often favored by professionals and researchers, while the latter has much support from autistic adults (Kapp, Gillespie-Lynch, Sherman, & Hutman, 2013; Kenny et al., 2016).
As this debate is ongoing within the community, this paper uses both forms. This decision was made out of respect for the proponents of both schemes, to avoid unnecessarily signaling an ideological stance beyond the scope of the present work, and to provide some linguistic variety since such phrases will inevitably occur frequently in these pages.
AUTISM SPECTRUM DISORDER
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by socio-communication impairments and repetitive behaviors and interests. The current definition of ASD encompasses what had previously been referred to as autistic disorder, Asperger’s disorder, and pervasive developmental disorder not otherwise specified in the previous fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (American Psychiatric Association, 2000, 2013). The symptoms of autism always appear in early childhood, although they may not be recognized as related to the condition until much later. The manifestations of ASD vary widely from individual to individual, hence the term spectrum. ASD frequently co-occurs with other conditions including anxiety, depression, ADHD, and intellectual disability (American Psychiatric Association, 2013). It is suspected that ASD is caused by a combination of genetics and environmental factors, but the clear causes of ASD are currently unknown.
ASD can have wide ranging effects. In school, socio-communication impairments mean that children with ASD have trouble making friends, find it difficult to maintain relationships with teachers, and can have problems with bullying. There are also common issues with organization, attention, anxiety, adaptability, emotion regulation, and understanding humor. Parents report being more worried about social challenges than academic challenges if their children transition to college (Auger, 2013).
The Centers for Disease Control and Prevention estimate that 1 in 54 children in the US aged 8 years old in 2016 had ASD (Maenner, Shaw, Baio et al., 2020). The prevalence has been increasing, at least partially due to better awareness of the condition, but potentially also due to the methodology of the study (Durkin, Bilder, Pettygrove, & Zahorodny, 2015; Mandell & Lecavalier, 2014). The condition is presently four times more likely to be diagnosed in males than females. It is also more commonly diagnosed in white children than in Hispanic or black children (Centers for Disease Control and Prevention (U.S.), 2014). It is unclear how much of the difference is attributable to under diagnosis among females and minorities (American Psychiatric Association, 2013).
There is no cure for ASD. Some proponents of neurodiversity disagree with the idea of a cure at all. They argue that autism is a natural variation in humanity and that treatment should not attempt to eliminate autistic characteristics or make someone with ASD behave like a TD person. In practice, there is much overlap between treatments acceptable to proponents of both the medical model of autism and the neurodiversity model (Ariel Cascio, 2012; Kapp et al., 2013). A wide variety of treatments are used including pharmaceutical, behavioral, and alternative and complementary medicine. Often multiple treatments are used with each child. Pharmaceutical and biomedical interventions focus on treating symptoms of ASD and comorbid conditions rather than ASD itself. Drugs for ADHD, seizures, irritability, aggression, anxiety, depression, repetitive behaviors, and sleep disturbances can be used on and off-label for children with ASD (Anagnostou et al., 2014). Applied Behavior Analysis (ABA) is perhaps the best-known treatment. ABA is a behavioral intervention that focuses on improving skills using learning principles. The focus is usually on increasing desired behavior, decreasing undesired behavior, and teaching new skills. However, there can be wide variance in style, duration, and intensity of ABA interventions, and it is still unsettled what combinations work best for which patients. ABA is particularly commonly used with young children (Anagnostou et al., 2014; Myers & Johnson, 2007). Treatment and Education of Autistic and Related Communication Handicapped Children (TEACCH) is another behavioral intervention. It emphasizes structure and predictability in learning, but tends to focus both on the child adapting and on adapting the environment to the child (Myers & Johnson, 2007). Socio-communicative deficits are often addressed by a speech pathologist. The Picture Exchange Communication System (PECS) or another alternative/augmented communication strategy may be employed. Other social skills, such as joint attention and symbolic play skills, are also addressed by a variety of behavioral interventions, particularly those incorporated into daily activities by parents. Occupational therapists often train self-care and academic skills. Sensory integration therapy, intended to address difficulties processing sensory input is also common. Complementary and alternative medicine therapies are commonly used, but understudied. Some, including music therapy, have shown some promise, but others, like chelation, have shown some harm (Myers & Johnson, 2007). All of the therapies discussed in the following sections fall under the complementary and alternative medicine category at present.
THERAPEUTIC APPROACHES FOR ASD
Robotic Therapy
Researchers have proposed involving robots in interventions for autistic individuals (e.g., Dautenhahn, 1999; Feil-Seifer & Matarić, 2009; Park, Jeon, & Howard, 2015; Scassellati, 2007; Zhang, Barnes, Ryan, Jeon, Park, & Howard, 2016). The motivation for this suggestion is based on the observation that many individuals with autism are more object-focused than people-focused and therefore, may be more willing to engage with a robot than with a human therapist. Additionally, robots bring a number of potential advantages to clinical interactions. For instance, they may allow a therapist to better observe interactions or participate in other capacities. Also, robots often bring a variety of sensors and recording devices, which could be used for quantitative analysis of interactions. It is also possible that, as robot costs decline over time, a robot could be used in environments and at durations not currently feasible with human interventions.
Thus far, experiments regarding robot interventions for children with ASD can be divided into four major categories: exploratory studies investigating responses to robots; attempts to elicit behavior using robots; using robots to develop skills; and using robots to provide feedback. Attempts to elicit behavior can be further divided into attempts to elicit diagnostically relevant behaviors and attempts to elicit prosocial behaviors (Diehl, Schmitt, Villano, & Crowell, 2012). The present study is exploratory in nature even though there are some expected behaviors from children in the game situation. There is some debate over how to measure interactions between children with ASD and robots, but common metrics include eye gaze, utterances, touch, and performance in an explicit task (Begum et al., 2015).
Many of the foundational experiments in this domain have been exploratory studies investigating children’s responses to robots. However, given the wide variety of possible scenarios and advancing technology, exploratory studies continue to remain both useful and common. Dautenhahn’s AURORA project, started in 1998, used a non-biomimetic mobile robot to encourage children with ASD to initiate interactions (Dautenhahn, 1999). A doll-like robot was also used to the same end (Dautenhahn & Billard, 2002). Subsequent work used KASPAR, a humanoid robot with simplified facial expressions and the form factor of a toddler (Robins, Amirabdollahian, & Dautenhahn, 2013). Feil-Seifer and Matarić examined how using a robot might differ from using a non-interactive toy. Using a robot that blew bubbles and spun around, children would interact with the robot in two conditions: one where the robot blew bubbles and spun around when a button was pressed and one where it did so randomly. The children interacted more with the robot and with present adults under the interactive condition (Feil-Seifer & Matarić, 2009). Social interactions of children with ASD were compared when they interacted with adults, Pleo (a dinosaur robot), and a tablet game. It was found that the children spoke more while interacting with the robot than in either of the other conditions and the utterances were often directed at the adult in the room rather than at the robot itself (Kim et al., 2013).
Attempts to use robots to elicit behavior in a diagnostic capacity are attractive because there is no definitive test for ASD and diagnosis is therefore subjective and dependent on the examiner’s skill and individual style. Involving a robot in diagnosis could increase the reliability and repeatability of diagnosis across sites. Scassellati has created a framework for using robots in autism screening/diagnosis and has attempted to do so using gaze direction and vocal prosody (Scassellati, 2005, 2007).
Attempting to elicit prosocial behaviors is more common and has met with some success using a wide variety of robots. The Keepon robot is a minimally socially interactive robot. Its form factor is a very simple snowman shape with eyes and a mouth. The robot is made of a deformable material that allows for squash-and-stretch style animation to express emotion and attention. In an unusually long-term study, children with ASD at a preschool were able to interact with Keepon over the course of three years. Keepon was able to elicit positive social interactions including triadic interactions with therapists and caretaking behavior toward the robot (Kozima, Michalowski, & Nakagawa, 2009). Shybot is reminiscent of the Keepon project, a very visually simple robot that has some human-like mannerisms. The robot can locate people’s faces, identify whether someone is a stranger, friend, or master, and react accordingly. A bit like a shy dog, it runs from strangers, stays near friends, and reacts excitedly to its owner. Shybot is capable of interacting to more than one person at a time and can serve as a socially engaged focus of joint attention (Lee, Kim, Breazeal, & Picard, 2008). In another long-term study, Scassellati et al. (2018) showed improvements in joint attention skills with adults during a 1-month, home-based intervention for increasing social communication skills of 12 children with ASD between 6 and 12 years old using an autonomous social robot.
Experiments explicitly designed to train a specific skill with just-in-time feedback are presently rare. One study that incorporates both used a NAO robot to teach simple greetings. The robot praised correct responses and modeled or guided appropriate behavior when a correct response was not given. The results were mixed and showed that the traditional metrics used to evaluate interest in exploratory studies did not necessarily translate to meaningful performance in a learning context (Begum et al., 2015).
Music Therapy
Empirical studies show that many autistic children possess musical potential that can and should be developed even though they do not meet savant criteria (Heaton, 2009). Music therapy is a broad category that encompasses a variety of techniques that are frequently mixed and combined including receptive, recreative, compositional, improvisational, and musical activity therapies (Accordino, Comer, & Heller, 2007). While this variety can make coming to a general conclusion about the effectiveness of music therapy difficult, meta-analysis of existing studies has shown positive effects of some music therapy (Accordino et al., 2007; Geretsegger, Elefant, Mössler, & Gold, 2014).
At the University of Iowa, typically developing children with normal hearing, children with ASD, and children with cochlear implants were asked to recognize the emotions and movement cues conveyed by novel violin music pieces. The study looked at five basic emotions (happiness, sadness, anger, disgust, and fear) and four movements (walk, run, skip, and climb). When compared to a control group comprised of typically developing children with normal hearing, there was no significant difference in the emotion recognition of children with ASD and the control. However, there were significant differences in emotion recognition between the children with cochlear implants and both the control children and children with ASD. For movement cue recognition, children with ASD performed similarly to the control group and children with cochlear implants performed significantly worse than both of the other groups (Whipple et al., 2015). This work suggests that music therapy is a viable option for children with ASD insofar as music is apparently conveying the intended messages despite the inherent socio-communicative impairments of ASD.
Some work focuses on a particular branch of music therapy called Neurological Music Therapy (NMT) that uses music to address sensory and motor problems with neurological bases. While traditional NMT uses physical instruments arranged in different configurations, the BendableSound device replicates some of the physical aspects to NMT while avoiding any reliance on musical skill, which makes it particularly suitable for young children. BendableSound is a flexible fabric canvas children with ASD touch and press to create sounds in interactive therapeutic games intended to develop proprioception (Vazquez, Cardenas, Cibrian, & Tentori, 2016). Other work has attempted to integrate humanoid robots into existing dance/movement therapy structures with some success (Beer, Boren, & Liles, 2016).
Dance Therapy
Dance therapy, called dance/movement therapy, has also been used with children with ASD, though it is less common. While not part of the diagnostic criteria for ASD, motor and proprioceptive impairments are commonly observed in individuals. In theory, dance therapy can address these deficits directly, but also indirectly can help children develop social skills by creating a bond between the child and the therapist and by giving children a non-verbal means of self-expression (Scharoun, Reinders, Bryden, & Fletcher, 2014). Martin (2014) suggested that dance/movement therapy can be used as an early intervention tool for children with ASD through mirroring, body awareness, and rhythm. Amos (2013) also emphasized the importance of rhythm and timing in learning to dance for children with ASD given that the process of timing our own actions to accord with the actions of others is vital to our experience of emotion. In addition, the success or failure of mutual timing can profoundly influence our relationships with and feelings about others. Nonverbal communication can also connect the body to expression of emotions. Through this dance and movement therapy, “attunement” can happen, which is a “feeling of connection and unification with another” (Kossak, 2009) or “the adapting of another person’s rhythms” (Kestenberg, Lowman, Lewis, & Sossin, 1999). Therefore, a creative dance session, such as the dance freeze discussed in this study, is seen as a good format for assistive robotic interventions. Indeed, researchers have adopted the dance freeze game in a number of robotic interventions for children with ASD and other related disabilities. Ferrari, Robins, and Dautenhahn (2009) used “Dance with me” as one of the ten scenarios in therapeutic and educational robot-assisted play. Amanatiadis, Kaburlasos, Dardani, and Chatzichristofis (2017) also included dance freeze in one of their robot-assisted special education programs. Preliminary results suggest that robot-assisted dance and movement therapy can improve children’s behavior. Halperin et al. (2013) also used the dance freeze game in their intervention with preschool children with ADHD and showed significant improvements in ADHD severity from pre- to post-treatment.
Play Therapy
Play therapy is another option for children with emotional and behavioral problems, though it is somewhat uncommon in literature for children with ASD. Young children usually lack the cognitive and verbal skills needed to participate fully in traditional talk therapy. This lack can be exacerbated in children with ASD. Play therapy allows children to express themselves through actions and communicate more effectively with therapists. Additionally, play therapy has been adapted to allow parents, teachers, and other non-therapists to provide treatment. Meta-analysis found play therapy effective for treating social maladjustment (Bratton, Ray, Rhine, & Jones, 2005). A case study with a 6 ½ year old autistic boy found that the child formed an attachment relationship with his therapist over the course of five months of undirected play therapy. He also showed increased initiative and autonomy, better eye contact, more joint attention, direct interaction with the therapist, and greater empathy including at home with his siblings (Josefi & Ryan, 2004). Another study used play therapy with siblings to address ritualistic behaviors in children with ASD. Themes of ritualistic behaviors were incorporated into games that were then played with siblings, e.g., a child fixated on movies would be given a movie-based bingo to play with their siblings. This provided children with a positive interaction since children with ASD had the opportunity to display competence related to their ritualistic behaviors (Baker, 2000). Attempts have also been made to develop a general purpose robot for robot assisted play with a variety of children with special needs (Robins, Ferrari, & Dautenhahn, 2008).
Relevance to Present Research
The dance freeze experiment presented here is an exploratory study investigating responses to robots. Specifically, it compares the responses to a robot and a human interaction partner given by TD children and autistic children in the context of a musical game. Even though eliciting specific behavior is not a primary goal, the analysis also looks at how children play the game and how they may or may not mimic their interaction partner. The dance freeze scenario was chosen for the experiment as it incorporates elements of music therapy, dance therapy, and play therapy. Dance freeze has a number of pragmatic advantages in our particular context of robotic therapy for children with ASD. It is within the physical capabilities of both small children and many robots. The rules of the game are simple to both explain and observe, which we considered an advantage as autistic children frequently have difficulties with language. Additionally, by pre-planning the movements, music, and timings, we were able to make the experimental conditions relatively repeatable. We hope that this exploratory research can shed light on the design of further activity sessions with children with ASD and clinicians that incorporates sound, movement, and robotics.
METHODOLOGY
Dance freeze, also known as freeze dance and a variety of other names, is a children’s game in which children dance and freeze in place while music plays and pauses randomly. Children dance while the music plays and have to remain still when it stops, then resume dancing when the music resumes. If used in a therapeutic context, the nature of the game automatically combines elements of music, dance, and play therapy. Therefore, we selected the dance freeze game as an ideal test case for comparing responses to interacting with a robot or a human in such a context. Since the game is flexible enough to work with essentially any music, it also allows for comparison of different types of music as well.
Within the context of the dance freeze game, we sought to answer three primary questions in this study. First, is there a difference between TD children and autistic children while playing a physical musical game? Second, is there a difference between how children interact with a humanoid robot and an adult human partner during the game? Finally, how do different music choices influence the interaction of children and robots or children and an adult human during the game?
Before we conduct an experimental study with our robot, we conducted formative research (stakeholder interviews) to learn more about our target population and to inform the experimental design.
STAKEHOLDER INTERVIEWS
To answer the research questions associated with this study and design the experimental protocol, interviews were conducted with a variety of stakeholders to build connections with the community and get input on the experimental design.
Participants
Four professionals (M = 44; SD = 7.07; 4 female; 4 white) who work with autistic children in the local community participated in semi-structured interviews and/or focus groups. One was a special education teacher, one a physical therapist, and two were speech language pathologists. Two worked in an elementary school and two worked at an outpatient clinic. They had a combined 57 years of experience with autistic children (M = 14.25; SD = 5.85).
Methods
The professionals met individually or in small groups with a researcher or researchers for semi-structured interviews that lasted roughly 30 minutes to an hour. Questions centered on the professionals’ experience working with autistic children and their views and suggestions for the proposed research. Participants also filled out a basic demographic questionnaire.
Results
The therapists’ first priority was to determine what motivates each individual child. Interests and motivation differ considerably amongst children with ASD, so initial data are gathered by talking with parents and going through a list of items that have been sufficiently motivating for other children. From there, it is a process of trial and error. The therapists looked for signs of interest particularly eye gaze, visually tracking moving objects, smiling, hand flapping, and toe walking (tip toeing). They also looked for signs that something is upsetting including gaze aversion, avoidance, high pitched squealing, fleeing and dashing (running and hiding under things). Sometimes, perhaps even most times, it took months to find enough motivators to fill an entire therapy session.
The therapists suspected that using robots may attract children with ASD, especially if the robots perform a repetitive behavior. This could also be helpful for therapists. Thousands of repetitions are required to teach new behaviors and that much repetition is tiring for most adults.
They suggested using a robot to teach very simple and basic conversations and greetings, rote social niceties, etc. Simplicity of a robot’s face may be easier for children with ASD to look at than a human face. Therapists hypothesized that part of the reason the children like Thomas the Tank Engine, a very popular toy amongst the children in their practice, is because it is simple and only has one expression at a time.
The robot could motivate children by clapping, praise, and other celebratory rewards or by allowing children to discuss a topic in which they are intensely interested. The therapists repeatedly insisted on having a back and forth interaction between the robot and the child. If the robot has the skill to imitate body motion, the children may enjoy being imitated and it could provide good feedback for those with poor body awareness.
For communication, a robot may be better in some ways than the tablet apps currently in use. Systems now involve a child pushing a button to have a sentence voiced by the device. There is no relationship between pushing a button and speaking, which is the eventual goal. A robot might be able to voice a sentence based on something closer to speaking than pressing a button, i.e., translate a child saying “mm” into “mom”. However, there does not seem to be a reason this functionality could not be implemented in a tablet.
It was also suggested that children may be willing to teach the robot or at least show the robot what they know. It can be difficult to motivate children to prove to a parent or teacher they have learned something, since children assume that the people who taught them are aware of what they have learned and therefore, do not see the point of demonstrating their knowledge.
On a practical level, it is imperative that it is possible to adjust the volume for each child as many have sensitivities to sound. It was also suggested that a robot might be a better motivator if access to it is limited to keep it “special”. This contradicts ideas about making robots a constant companion.
The novelty effect might have different ramifications for children with ASD than for TD children. Once an interest is identified, it commonly persists for years at a time without discernable decreases in interest or motivation. An example was given of a child who liked one particular song – and only that particular song – for well over a year. At the same time, new things are often frightening rather than intriguing. The novelty effect may in some cases be reversed for children with ASD, so that they dislike novel things and have a persistent preference for some familiar objects, sounds, games, etc.
Interestingly, the therapists agree that children with ASD understand body language very well in practice. For instance, patients can usually tell when they can get away with mischief and when they cannot based on the therapist’s body language. In their opinion, the problem seems to be processing rather than understanding. Children with ASD may not be able to sort and analyze body language in a timely fashion if there is too much other input.
It can be difficult to get children to express what they know to teachers. The coordinators theorized that children do not understand the need to demonstrate their knowledge as they assume the teachers already know that they know what is being tested. Children with ASD, particularly children who are highly verbal or have traits of the former Asperger’s Syndrome diagnosis, may talk non-stop about a topic with no reference to the interest or lack thereof of the listener. Topics of special interest were a recurring theme. Teachers used these interests to motivate the children to complete tasks. In the experience of the intermediate school district staff, many of the children with ASD in their district are “very tech savvy” and likely to be interested in robotics. From this basis, the coordinators suggested that robots may become a special interest of some children with ASD and could be used as a motivation tool. It may also be possible to connect the robot to an existing special interest, for instance, by having the robot play Disney songs or discuss train facts for a child interested in those. Interestingly, the idea of having the children demonstrate their knowledge by teaching the robot also came up. Since the robot would presumably not be present when the information is initially imparted to the student, the problem of students assuming their teachers already know what is being discussed may not arise.
Also, of interest, the teacher described a learning process that is very long term and involves little useful feedback to the teacher. Students often give no indication they are learning what is being taught until they suddenly begin applying it several months later. One example provided was of a young girl who was being taught the sounds of the alphabet. The child did not respond to the instruction at all until she began to quickly demonstrate mastery of the concept five months after the instruction began. Also, of interest, was that the tone of voice matters significantly for children with ASD and that a soft, calm voice with little inflection is often more effective than a loud voice or one with significant affect. Overall, no explicit music therapy was being conducted in the schools. None of the individuals interviewed is using robots either, though other technologies, particularly iPads, were common.
Discussion
Based on the interviewees’ comments, there were several roles a social robot could play for an autistic child. First, the robot could be a motivating activity, like an iPod game or favorite toy, that the child is given access to as a reward for cooperation. Second, the robot could either be a special interest itself or give a child an outlet for a special interest. To address these two points and in order to keep the experience “special”, access to the robot was limited in our experiment. The consent procedure and explanations about the study was conducted in a separate room from the robot. For the robot to perform a repetitive behavior more naturally, we chose a dance game. Dance innately includes many repetitions. As an attempt to connect the robot to an existing special interest, we selected half of the songs among famous children’s songs in our study. Next, a robot could be used to teach new skills, perhaps by demonstrating a social story, for instance. Finally, and somewhat unexpectedly, the robot could be a student and allow the child to demonstrate their knowledge. As the therapists mentioned, if the robot has the learning skill to imitate children’s body motion, it would be a good mirroring intervention. Our research team was also interested in this topic and this has been explored in a separate study (Burns, Jeon, & Park, 2018). Multiple interviewees stressed that children with ASD respond much better to visual information than auditory information. This could bode well for the use of robots. Since robots are embodied, they can better convey different visual information than a screen-based device and have a certain advantage in 3D space, which could be useful for navigation, for instance.
Much of these discussions are related to broader, more ambitious research directions suitable for later studies. However, some themes and specific details were relevant to the design of the narrower, exploratory study we needed to start with. First, the general agreement among interviewees that some children with autism were likely to be interested in robotics and find them motivating reinforced our research direction. Second, the confirmation that eye gaze is used as a proxy for interest and engagement in the field validated one of the primary metrics under consideration. Additionally, the interviewees provided practical, specific information about potential comfort issues that could arise within the experiment. This helped inform the experimental procedure to mitigate these risks and prepared the researchers to better recognize non-verbal expressions of happiness or discomfort frequently observed in autistic children.
The discussions informed a number of our hypotheses (see Hypotheses below). As expected from the literature, it was confirmed that autistic children may be interested in both robots and music but that there would likely be substantial individual variability. The conversations around simplified emotional expression suggested that children with ASD would be more easily engaged by a robot than a person and a possible mechanism for it. It was also posited that the novelty effect may not apply in the usual manner for autistic children contributing to a hypothesis that children who engaged first with the robot would continue to engage with the researcher.
MUSIC VALIDATION
In order to select appropriate music for the dance freeze game, a small music validation experiment was conducted by having participants evaluate a set of potential songs. All potential songs selected were in the public domain and the recordings were either Creative Commons licensed for an applicable use or used with permission. Classical and traditional music presented more options in this regard than many other types of music primarily because the age of the canon puts many more classical songs into the public domain.
Calm and upbeat categories were selected for the music type in the hopes of eliciting different styles of dance from the participants. Based on this process, we selected 12 songs for evaluation. Six of the songs selected were traditional children’s songs: “A Tisket A Tasket”, “Farmer In The Dell”, “Freres Jacques”, “London Bridge”, “This Old Man”, and “Twinkle Twinkle Little Star”. The other six songs selected were classical pieces: Brahms’s “Ballade Op. 10 No. 1”; Chopin’s “Berceuse Op. 57” and “Nocturne for Piano in C Minor, B108”; Debussy’s “La Plus Que Lente”; Haydn’s “Ravel Menuet”; and Joplin’s “Harmony Club Waltz”.
Methods
Five college students were asked to listen to the 12 songs and rated how positive, negative, upbeat, calm, and pleasant each was on a scale of 0–4. Participants were also asked if there was anything in the songs that seemed offensive or inappropriate for a five year old child.
Results
To limit confounding variables, all songs were required to be positive and pleasant as measured by participants. The songs also had to be rated as age appropriate by the participants. The calmest and most upbeat songs from each genre were then selected for use in the dance freeze experiment. To avoid having positive-negative valence as a confounding variable, only songs that were rated more positive than negative were used. As expected, no song was rated as having inappropriate or offensive content, so that was not a concern while downselecting songs. From the results of the experiment, four songs were chosen for the dance freeze game: an upbeat traditional children’s song, a calm traditional children’s song, an upbeat classical piece, and a calm classical piece.
The children’s song with the highest average upbeat rating was “A Tisket A Tasket” (M = 2.8; SD = 1.64). The highest average calm rated for a children’s song was “Twinkle Twinkle Little Star” (M = 3.2; SD = 1.10). For classical songs, the calmest was Chopin’s “Berceuse” (M = 2.2; SD = 1.64) and the most upbeat was Joplin’s “Harmony Club Waltz” (M = 1.8; SD = 1.30).
DANCE FREEZE GAME EXPERIMENT
Given inputs from the qualitative study and the music validation experiment, we designed the experimental protocol for the dance freeze game experiment. The experiment dealt with two populations of children – TD and autistic – as well as four music samples – an upbeat instrumental classical piece, a calm instrumental classical piece, an upbeat children’s folk song, and a calm children’s folk song. Parents were also asked to observe the experiment and provide feedback based on their perceptions of their child(ren)’s interactions.
Hypotheses
Hypothesis 1: Neurotypical children would be more engaged in all conditions than children with ASD.
- Hypothesis 2:
- Neurotypical children would be more engaged interacting with the robot than with the adult human.
- Children with ASD would be more engaged interacting with the robot than with the adult human.
- Hypothesis 3:
- Order of interaction would not have a significant effect on TD children’s level of engagement relative to the interaction partner.
- Children with ASD who interact with the robot first would be more engaged when interacting with the adult human than children with ASD who interact with the adult human first.
- Hypothesis 4:
- Preference for genre of music would vary on an individual basis.
- Children would be more engaged when dancing to music they prefer.
Hypothesis 5: Both groups of children would dance more energetically to the upbeat songs than to the calm songs.
Hypothesis 6: Both groups of children would mimic the movements of the robot or human during the game despite not being instructed to do so.
Hypothesis 1
Children with ASD generally have more difficulty with social interaction and engagement than TD children, so it is expected that they will overall be less engaged in the experiment.
Hypothesis 2
Hypothesis 2a
Robots, particularly social robots as sophisticated as the one used in this experiment, are still relatively uncommon. However, adult Caucasian women, such as the researcher who participated as the adult human in our study, are commonly encountered by children; demographic data associated with therapists (https://datausa.io/profile/soc/291123/#demographics) suggests Caucasian women make up the largest majority of this workforce population. Therefore, a significant novelty effect was anticipated for the robot interaction condition, but not for the adult human condition. It was expected that this novelty effect would translate to greater enthusiasm and engagement rather than fear or reluctance to participate for TD children.
Hypothesis 2b
Children with ASD may respond to novel objects and interactions with fear or with interest, so the reasoning used for Hypothesis 2a regarding TD children was not as clearly applicable to autistic children. However, in previous studies, children with ASD have consistently shown greater interest in technology and robots than in new people, so it was expected that children with ASD would also be more engaged with the robot than with the human.
Hypothesis 3
Hypothesis 3a
Related to the anticipated novelty effect explained above, it was expected that TD children would be more engaged when interacting with the robot than the human, regardless of which they played the game with first. Essentially, it was expected that the novelty effect of working with the robot would outweigh any fatigue or boredom caused by playing the game through with a human first.
Hypothesis 3b
Since one of the primary goals of therapeutic interventions for children with ASD is often improved social skills, it is critical to demonstrate that robots can facilitate that improvement either by acting as a catalyst for social interaction with other people or by teaching skills that later are transferred to interactions with people. This hypothesis attempted to capture both possibilities. If children with ASD who played the game with the robot first focused more on the human during the second game or if they otherwise interacted more with the human present, both would be counted as confirmation of this.
Hypothesis 4
Hypothesis 4a
Preference for different genres of music varies considerably among different people as a matter of course and it was expected to vary for both groups of children. However, it was also conceivable that children with similar cultural backgrounds and ages would develop similar musical taste. Given the diversity exhibited by autistic individuals and common auditory sensitivities, this group could vary even more than TD children.
Hypothesis 4b
It was expected that children would dance in a manner that suited the music, i.e., slow movements for slow songs. However, that was not included in the instructions and it was possible children would disregard the music and dance however they pleased.
Hypothesis 5
It was expected that children will dance more energetically to upbeat songs than calm songs in keeping with common habits of dance.
Hypothesis 6
Children were not instructed to mimic the movements of the human or robot dancing with them, but mimicry is common among children and at least some mimicry was expected. Examining the point at which most mimicry occurred was also of interest.
Participants
Twelve TD children (M = 7.25; SD = 1.54) participated. There were 5 males and 7 females. Two of the children were Asian and the remaining 10 were white. Three children with ASD (M = 8.33; SD = 4.04) also participated. All were white males.
Demographic information about the children was provided by their parents; demographic information about the parent participants was not collected. Twelve parents participated in total. Fewer parents than children participated because of sibling participants associated with the same parent.
Methods
The experimental protocol and supporting documents (including the consent and assent forms, the parental permission letter, and all questionnaires) were reviewed and approved by the Michigan Technological University Human Subjects Committee Institutional Review Board. The researcher introduced herself to the participant(s) and their parent/guardian (hereafter referred to as their parent for simplicity) and showed them the rooms being used for the experiment. The parent was given a parental permission form for their child’s participation and consent form for their own participation. After the parent had signed, the children were read the assent and given time to discuss with their parent. Assent was given verbally and documented by the researcher. Refusal to participate did not result in any consequences for the participant.
The experiment took place in a suite of three rooms designed for usability testing. The rooms were located side-by-side and one-way mirrors separated the center room from each of the end rooms as shown in Figures 1 and 2. This allowed a person in the center room to observe both of the end rooms, but prevented anyone in either end room from seeing clearly into either of the other rooms. Parents observed the experiment from the center room while one (leftmost) of the end rooms was used for the experiment. Children who were not participating in the experiment at any given time waited in the final room (rightmost). This setup better allowed families with multiple children to participate in the experiment since parents were able to monitor all of their children at the same time, but children were able to participate one at a time without observing each other during the experiment. The function of a one-way mirror was explained and demonstrated to the children so that they were aware that while they were not able to see their parent clearly, their parent was able to see them.
Figure 1.
Simplified layout of experimental setup
Figure 2.
Detail of main experiment area; C1 and C2 are cameras
The recording setup was also explained and demonstrated to both children and parents. Two color cameras were used to record the participant, robot, and adult human. A Microsoft Kinect was used to record skeleton position information as well. Children were able to see the visual representation of the skeleton information on a laptop screen during the experiment, but the video was generally not easily visible.
The experiment consisted of a simple game where music was played and paused at intervals that appeared random to participants. The rules of the game as explained to the children were to dance while the music played and to freeze in place when the music stopped, then to resume dancing when the music started again. Children were not given instructions regarding whether or not to mimic their interaction partner’s movements. If children asked about it, they were told they could choose whether or not to match the dance.
All children interacted with both the robot NAO, a 58 cm tall humanoid robot pictured in Figure 3 and the adult human, who was present in the room. Order of interaction was counter-balanced so that 7 children interacted with the NAO robot dancing first, with the adult human present and 8 children interacted with the adult human dancing first, with the robot present. The same four songs with the same added pauses were used in each interaction in random order. Two were traditional English language children’s songs, “Twinkle Twinkle Little Star” and “A Tisket A Tasket”. The other two songs were classical instrumental songs, Frederic Chopin’s “Berceuse in D Flat Major Op. 57” and Scott Joplin’s “Harmony Club Waltz”. The songs were selected from music vetted in the experiment described in the Music Validation section. Songs were edited for length and to introduce the silences necessary for the game, resulting in approximately 45 seconds of “Twinkle Twinkle Little Star”, 66 seconds of “A Tisket A Tasket”, 53 seconds of “Harmony Club Waltz”, and 60 seconds of “Berceuse”. The robot used the same dances with all participants. The human scripted their dance movements to mimic the robot’s dances and tried to be consistent across participants.
Figure 3.
Sample images of NAO dancing
After the conclusion of the dance freeze game interactions, parents filled out a questionnaire gathering their impressions from the session, demographic information, and background information about their children’s previous interest in robotics, music, exposure to robots, experience with the dance freeze, etc. Children were verbally given a similar questionnaire. The adult human, parent, and (to the extent possible) the child also discussed the interactions in an unstructured interview.
Metrics
Audiovisual recordings of each trial were coded for qualitative metrics. The videos for dance freeze game errors (e.g., dancing when the music stopped or not dancing while it was playing) and discrete social actions (e.g., laughing or speaking) were coded. To be more specific, the qualitative metrics included mimicry of partner, quality of the dance, quality of gameplay, interaction level with the robot, and interaction level with the adult human. Ratings were on a 1–5 scale with 5 being the maximum. Mimicry ranged from no mimicry to perfect mimicry; dance quality from dance not correlated with the music to highly appropriate to the music, could have been choregraphed; gameplay from not following the rules (except accidentally) and not engaged in the game to following the rules perfectly and focused on playing. Interaction with the robot and researcher were roughly estimated percentages from not paying attention, 0% focused to 100% focused. Totals for the two could exceed 100% in cases where the child was simultaneously focused on both, such as when talking with the researcher about the robot. The average of the two independent raters’ scores was used for all subsequent calculations.
We also calculated the percentage of time the child’s face was oriented toward the researcher/skeleton display of the Kinect device, the robot, or the mirror. Otherwise unspecified orientations were counted as other. Facing the robot is fairly self-explanatory. Recordings only covered periods where the children were playing the game; time before, after, and in between songs was not included in the analysis.
Results
Typically-Developing Child Questionnaire Results
Four of the twelve TD children had previous experience with robots; eight did not. However, only one had previous experience with the NAO robot. Most of the children (9) had played the dance freeze game before and five had previously taken dance classes. All children rated the frequency with which they listened to instrumental classical (M = 1.83; SD = 1.03) and children’s music (M = 2.25; SD = 1.71) on numerical scales from 0 (“Never”) to 4 (“Almost every day”). The frequencies with which children listened to instrumental classical music and children’s music were comparable. A paired samples t-test of the scores did not reach statistical significance. The children liked to dance (M = 2.67; SD = 1.50) on a scale from 0 (“Not at all”) – 4 (“Very much”). They rated the difficulty of learning the rules (M = 1.42; SD = 0.90) on a scale from 1 (“Very easy”) – 5 (“Very hard”).
In forced choice questions, nine of the children reported that interacting with the robot was more interesting and more enjoyable than interacting with the adult human. Numerical scales used to rate the children’s comfort with the robot and adult human did not reveal a difference in comfort. A paired samples t-test of the scores for the robot (M = 3.70; SD = 1.34) and adult human (M = 2.82; SD = 1.40) did not have statistical significance.
Typically-Developing Parents’ Questionnaire
Parents reported that seven of the children preferred children’s music to instrumental classical and five preferred the reverse. Eight of the children were considered by their parents to have been more engaged with the robot as a dance partner than with the researcher. However, when asked which their child enjoyed dancing with more, the results were equally split. Finally, when asked under which condition the children danced better, seven reported better dances with the researcher, nearly inverting the engagement scores. A paired samples t-test of the scores given for comfort with the robot (M = 2.92; SD = 1.31) and researcher (M = 3.58; SD = 0.51) did not reach statistical significance.
Typically-Developing Children’s Quantitative Metrics
By far the most common events across all TD children were dance errors (111), when the child moved during silences. For comparison, freeze errors, when the child did not move while music was playing, occurred only 30 times. Other events included utterances (45), laughing (32), and social gestures (2) (see Table 1). Qualitative scores of TD participants are in Table 2.
Table 1.
Event counts of TD participants, S# denotes subject number
| Subject | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Behavior | S1 | S2 | S3 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S15 |
| Dance Error | 9 | 22 | 4 | 14 | 8 | 9 | 7 | 10 | 6 | 10 | 12 |
| Freeze Error | 18 | 0 | 0 | 2 | 0 | 0 | 1 | 0 | 0 | 0 | 9 |
| Utterance | 17 | 0 | 0 | 0 | 2 | 0 | 1 | 0 | 1 | 0 | 24 |
| Laughing | 15 | 0 | 0 | 0 | 10 | 0 | 0 | 0 | 4 | 0 | 3 |
| Gesture | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Table 2.
Qualitative scores of TD participants
| Mean | StdDv | |
|---|---|---|
| Mimicry | 1.91 | 1.22 |
| Dance Quality | 2.88 | 0.99 |
| Gameplay | 3.56 | 0.92 |
| Robot Interaction | 2.03 | 1.27 |
| Human Interaction | 2.84 | 1.17 |
Typically-Developing Children’s Qualitative Metrics
Analysis by Song Genre and Tempo
Paired samples t-tests of each child’s average qualitative metrics scores by song genre (children’s music vs. instrumental classical) and song tempo (calm vs. upbeat) all did not reach statistical significance.
Typically-Developing Face Orientation
The mirror referenced is the two-way mirror behind which the parents watched. It was difficult to see anything other than the child’s reflection in the mirror unless very close to the glass, but the children knew their parents were there. Some children appeared to be watching themselves in the mirror and some appeared to be acting for their parent’s benefit. The researcher was seated or stood in front of the laptop running the Kinect and displaying a stick figure image of the skeleton being tracked. It was not always clear from the video whether children were paying attention to the human or the Kinect visualization, so both are counted in a single category. Children’s interest in the Kinect display was not anticipated in the original experiment design, but appeared to be quite strong for some individuals. Therefore, it is not safe to assume that children looking toward the adult human were actually looking at the adult human.
Participants spent more time facing the human-Kinect than any of the other targets (Table 3). This result was found significant in a one-way ANOVA (F(3,90) = 75.70, p = 0.00). Further analysis using paired samples t-tests showed participants spent significantly more time facing the human-Kinect than the robot (t(90) = −4.43, p = 0.00) over all conditions. Splitting the results by song, one-way ANOVAs of face orientation toward human-Kinect and robot, neither reached statistical significance.
Table 3.
Descriptive statistics of TD children’s face orientations as a percentage of total time per trial
| Robot | Human-Kinect | Mirror | Other | |
|---|---|---|---|---|
| Mean | 27% | 58% | 5% | 9% |
| StdDv | 35% | 34% | 10% | 17% |
Kinect Data Analysis
The children’s skeletons were tracked using Kinect SDK 2.0 and joint positions were logged for offline analysis. Technical difficulties required the exclusion of data for seven dances. All data for autistic children S5 and S13 was excluded due to tracking problems, largely attributable to the children wandering out of the optimal tracking area or assuming positions that obscured most of their bodies.
Data were processed before analysis using a median filter to remove noise spikes and a central moving average filter to smooth the data. For each child’s eight dances, the average velocity of each joint was calculated in meters per second and the total displacement of each joint in meters. Kinect data are not continuous, so all calculations were made from one frame to the next and then averaged or summed appropriately.
For the TD children’s data, a significant difference was found between movement during upbeat songs and calm songs indicating that children moved faster and a greater distance during upbeat songs. For each child’s joints, a paired samples t-test was conducted comparing the average displacement (t(299)= 7.23; p = 0.00) and velocity (t(299) = ; p = 0.00) of the joint across calm songs and upbeat songs; both results were statistically significant.
Autistic Children’s Results
Given the wide variation between autistic children and the small sample size provided in this study, each child’s results are discussed individually. In addition to similar analyses to that provided for TD children, one sample t-tests were used to compare each autistic child’s mean results to average TD children’s mean results (Figure 4).
Figure 4.
Mean qualitative scores of children with ASD and TD children; error bars indicate the standard error of the mean
Subject 5
Subject 5 (S5) was a nine years old white male. He was diagnosed with ASD at 2.5 years old as he was nonverbal and had poor social skills. During the experiment, he was somewhat verbal and intelligible, but did not speak much. He had previous experience with robotics, NAO, and dance freeze, but had not taken a dance class. His favorite form of music was children’s music. By his parent’s estimation, he enjoyed the interaction with the robot more than the adult. The parent also rated their child one point more comfortable with the robot (2) than with the adult (1). This is supported by the face orientation data. During all three of the robot songs this child experienced (the fourth was not possible due to technical problems), the child looked exclusively at the robot. In contrast, during the human conditions, the child only looked at the adult human 13% of the time in a single song and not at all in the other songs. In none of the conditions did the child actually play the game, resulting in high error rates, low gameplay scores, low dance quality, and low mimicry. One-sample t-tests comparing S5’s average qualitative scores to the qualitative scores of the TD children were all statistically significant indicating that his results did not belong to that population (Table 4).
Table 4.
One-sample t-test results comparing average qualitative scores of S5 to TD children’s average scores
| t(11) | p | |
|---|---|---|
| Mimicry | 2.72 | 0.02 |
| Dance Quality | 7.73 | 0.00 |
| Gameplay | 11.49 | 0.00 |
| Interaction Robot | −7.35 | 0.00 |
| Interaction Human | 11.14 | 0.00 |
Face orientation data (Table 5) show that the child ignored everything except the robot during the robot condition, but did not pay attention during the human condition, spending most of his time looking at the mirror or elsewhere. Music genre and tempo had very little effect.
Table 5.
Face orientation of S5 as a percentage of total time
| Robot | Human-Kinect | Mirror | Other | |
|---|---|---|---|---|
| All | 46% | 2% | 30% | 22% |
| Robot | 100% | 0% | 0% | 0% |
| Human | 6% | 3% | 53% | 38% |
Despite his reported preference for children’s music, the effects of song genre are not clear for S5 (Table 6). There was no difference in mean mimicry (1), dance quality (1), or gameplay scores (1). However, there were differences for the interaction scores. Interaction with the robot was slightly higher and interaction with the adult human slightly lower during the children’s songs (3.17 and 1.00, respectively) as opposed to the instrumental classical pieces (3.00 and 1.13).
Table 6.
Average qualitative scores of S5 by music type
| Mimicry | Dance Quality | Gameplay | Interaction Robot | Interaction Researcher | |
|---|---|---|---|---|---|
| Children’s | 1.00 | 1.00 | 1.00 | 3.17 | 1.00 |
| Classical | 1.00 | 1.00 | 1.00 | 3.00 | 1.13 |
| Calm | 1.00 | 1.00 | 1.00 | 3.63 | 1.00 |
| Upbeat | 1.00 | 1.00 | 1.00 | 2.33 | 1.17 |
Subject 13
Another white male, Subject 13 (S13) was four years old. According to the parental questionnaire, he was diagnosed with ASD first by his pediatrician, and then mental health professionals who administered the ADOS test, and finally the local intermediate school district. S13 was vocal with 41 utterances, but largely was not intelligible. The word “robot” was often repeated, though. His parent reported that he did not have previous robotics experience, but had interacted with the NAO robot. He also had played the dance freeze game before and had dance classes. He listened to children’s music and instrumental classical music with about the same frequency, but preferred children’s music. Contrary to the other children with ASD, S13 was reported to be more engaged interacting with the adult human and appeared to enjoy it more than dancing with the robot. However, he was rated as equally comfortable with the robot and the adult human (4). Also, the face data towards the robot in the robot condition was numerically higher than the data towards the adult human in the human condition (Table 7).
Table 7.
Face orientation of S13 as a percentage of total time
| Robot | Human-Kinect | Mirror | Other | |
|---|---|---|---|---|
| All | 33% | 39% | 11% | 19% |
| Robot | 55% | 33% | 10% | 0% |
| Human | 8% | 44% | 12% | 38% |
Face orientation data and video data suggest that S13 was less distracted (looking at the mirror or other) in the robot condition than the adult human condition. He was more focused during the robot condition, but since he interacted with the adult human second, simple boredom could have caused the discrepancy.
One sample t-tests comparing S13’s qualitative score means to TD children had significant results for all but the interaction human score (Table 8). Music genre and tempo had very little effect (Table 9).
Table 8.
One-sample t-test results comparing average qualitative scores of S13 to TD children’s average scores
| t(11) | p | |
|---|---|---|
| Mimicry | 2.72 | 0.02 |
| Dance Quality | 7.47 | 0.00 |
| Gameplay | 10.36 | 0.00 |
| Interaction Robot | −2.45 | 0.03 |
| Interaction Human | 1.34 | 0.21 |
Table 9.
Average qualitative scores of S13 by music type
| Mimicry | Dance Quality | Gameplay | Interaction Robot | Interaction Researcher | |
|---|---|---|---|---|---|
| Children’s | 1.00 | 1.13 | 1.36 | 2.38 | 2.75 |
| Classical | 1.00 | 1.00 | 1.13 | 2.38 | 2.50 |
| Calm | 1.00 | 1.13 | 1.38 | 2.25 | 2.38 |
| Upbeat | 1.00 | 1.00 | 1.13 | 2.50 | 2.88 |
Subject 14
At 12 years old, Subject 14 (S14) was the oldest of the participants with ASD. He also had a cerebral palsy diagnosis. S14 held fluent conversations with the adult human. While the parent reported that he did not have previous robotics experience, they also reported that he had previous experience with NAO. What exactly this experience was and why it was not considered previous robotics experience was not specified. However, based on conversation with the parent, it appears this was in reference to meeting the NAO robot during a demonstration at a local fair. As expected, he had played the dance freeze game before. S14 was not a big music listener, but did listen to children’s music slightly more than classical. However, he was said to prefer classical music. His favorite music overall was rock and roll. He liked to dance. He was judged to enjoy dancing with the robot more than the adult human and to be more engaged and better dancing during the robot condition as well. He was considered one point more comfortable with the robot (4) than with the adult human (3).
Unlike S5 and S13, he was able to complete a children’s questionnaire. The child agreed with the parent’s questionnaire on most points. Children’s music and classical music were reversed for listening frequency: children’s music 0 and classical music 1. He also rated himself a point higher in comfort with the adult human than his parent did. He seemed to understand the rules and rated them easy to figure out, but expressed them oddly when asked as, “To freeze when you were not expecting it”.
Face orientation data show that he looked at the robot in the robot condition more than the adult human during the human condition (Table 10). As with S13, he looked at the mirror and other more in the human condition than the robot condition.
Table 10.
Face orientation of S14 as a percentage of total time
| Robot | Human-Kinect | Mirror | Other | |
|---|---|---|---|---|
| All | 59% | 25% | 6% | 9% |
| Robot | 93% | 7% | 0% | 0% |
| Human | 26% | 44% | 13% | 18% |
One sample t-tests comparing S14’s qualitative score means to TD children had significant results for all but the gameplay score (Table 11). Music genre and tempo had very little effect (Table 12).
Table 11.
One-sample t-test results comparing average qualitative scores of S14 to TD children’s average scores
| t(11) | p | |
|---|---|---|
| Mimicry | 2.72 | 0.02 |
| Dance Quality | 2.26 | 0.05 |
| Gameplay | 1.04 | 0.32 |
| Interaction Robot | −7.29 | 0.00 |
| Interaction Human | 5.68 | 0.00 |
Table 12.
Average qualitative scores of S14 by music type
| Mimicry | Dance Quality | Gameplay | Interaction Robot | Interaction Human | |
|---|---|---|---|---|---|
| Children’s | 1.0 | 2.25 | 3.13 | 2.88 | 2.13 |
| Classical | 1.0 | 2.38 | 3.50 | 3.25 | 1.75 |
| Calm | 1.0 | 2.38 | 3.38 | 3.13 | 1.63 |
| Upbeat | 1.0 | 2.25 | 3.25 | 3.00 | 2.25 |
S14’s Kinect data showed significantly more movement and greater speed during upbeat songs than during calm songs. Paired samples t-tests of mean displacement (t(24) = 31.36; p = 0.00) and velocity (t(24) = 37.61; p = 0.00) measures for upbeat and calm songs were statistically significant. These results are consistent with the results found for TD children.
DISCUSSION
Typically-developing and autistic children played the dance freeze game with both the NAO robot and the adult human to a variety of music while being recorded and observed. The combination of subjective measures from children and parents along with more objective measures provided by Kinect skeleton tracking and video data allowed for a variety of analysis methods. Results showed statistically significant differences in overall qualitative scores between TD children and autistic children. Unlike their parents’ subjective report that TD children looked more engaged with the robot, their face orientation data showed that they actually spent significantly more time facing the adult human than the robot. The parents also stated that the dance performance was better with the adult human than with the robot. In contrast, results showed that children with ASD tended to face the robot more than the adult human. Moreover, all three children with ASD tended to show a higher score than TD children in the interaction score with the robot (Figure 4). All three children with ASD also tended to show a lower score than TD children in the interaction score with the human. Upbeat music corresponded with more energetic dancing than calm music, but music genre did not seem to have a large effect.
Hypothesis 1
Typically-developing children would be more engaged in all conditions than children with ASD.
This hypothesis was mostly supported. One sample t-tests of the qualitative scores showed that children with ASD differed from the TD sample on gameplay, interaction with the robot, and interaction with the adult human in all but two cases. S13 did not differ from TD children in human interaction and S14 did not differ from TD children in gameplay.
Face orientation data could also be considered to support this conclusion. However, it is somewhat dubious to conclude that face orientation – or even eye gaze – translates directly to attention and engagement for children with ASD. Eye contact is known to be difficult for many autistic individuals, so it is entirely possible that they may avoid eye contact even when fully engaged in an activity. With that caveat, the face orientation data for the “Other” category could be considered a rough proxy for distraction in the TD group. The “Mirror” category is harder to interpret. Sometimes, it was children looking at the reflection which may or may not be considered distraction and sometimes, them hamming it up for their parents on the other side of the glass. Across all the TD children, 9% of experiment time was spent looking at other and 5% looking at the mirror. For autistic children, it varied substantially by child. S5 differed most from the TD children with 22% other and 30% mirror. S13 had 19% other and 11% mirror. Finally, S14, who was the highest functioning of the autistic children, was very close to TD children with 9% other and 6% mirror, differing by just 1%.
Hypothesis 2
Hypothesis 2a
Typically-developing children would be more engaged interacting with the robot than with the adult human.
This hypothesis had mixed support. Engagement varied by child and, at least to an extent, age, but the trend seemed to be more engagement with the adult human. Eight of twelve children were rated by the parents as more engaged with the robot. However, the quantitative enjoyment results were split 50/50 and dance results were better with the human for seven. Overall, paired samples t-test showed significant differences in favor of human interaction over robot interaction and the average face orientation was 58% toward human/Kinect and only 27% toward the robot. Children who were interested in the robot often engaged with the adult human while they were interacting with the robot to ask questions or to draw the adult’s attention to the robot. This is encouraging for efforts to use robots as a point of joint attention during therapy, but makes it difficult to compare engagement levels. It may have been a truer comparison to have the adult in another room or somehow similarly unresponsive during the robot condition as the robot was during the human condition. This result might imply that we may need different approaches for different populations. For example, teaching a specific subject or task might be better with a human teacher for TD children, whereas developing the triadic relationship for social interactions might be better with the social robot for autistic children.
Hypothesis 2b
Children with ASD would be more engaged interacting with the robot than with the adult human.
Two of the three children with ASD were considered by their parents to be more engaged, enjoy the interaction more, and dance better with the robot than the adult human. The questionnaire from a child with ASD supports this. Face orientation data is supportive of this hypothesis with children spending overall 46% of time looking toward the robot and 23% looking toward the human or Kinect. For S13, interaction with the robot score and interaction with the adult human score seem to be equivalent. However, all three autistic children’s face orientation data in the robot condition are higher than the adult human which differs from TD children’s data (see Table 13). Therefore, overall the trend of autistic children’s data favored engagement with the robot over the adult human.
Table 13.
Average face orientation of children with ASD as a percentage of time
| Human-Kinect | Robot | ||
|---|---|---|---|
| CID | Partner | ||
| S13 | Adult Human | 44% | 8% |
| Robot | 33% | 57% | |
| S14 | Adult-Human | 44% | 26% |
| Robot | 7% | 93% | |
| S5 | Adult-Human | 3% | 6% |
| Robot | 0% | 100% |
Hypothesis 3
Hypothesis 3a
Order of interaction would not have a significant effect on TD children’s level of engagement relative to the interaction partner.
No differences were found based on order of interaction. Paired samples t-tests of qualitative scores based on interaction order did not reach statistical significance.
Hypothesis 3b
Children with ASD who interact with the robot first would be more engaged when interacting with the adult human than children with ASD who interact with the adult human first.
There was not sufficient data available to make a good conclusion one way or another about order of interaction. Based on what we have, however, this hypothesis is not supported. The only participant with ASD who interacted with the adult human first was the only one who seemed to be more engaged with the adult.
Hypothesis 4
Hypothesis 4a
Preference for genre of music would vary on an individual basis.
Based on self- and parent-reported music preference among TD children, music taste varied by individual. There was no overall trend in the answers to the favorite type of music question. Two children preferred rock and roll and two children did not know what their favorite type of music was. Otherwise, there were no common preference. The full list was rock and roll, indie pop, children’s, pop, country, Disney, Broadway, hymns, all kinds, and unknown. For the question forcing a choice between the two types of music in the experiment, seven children preferred children’s music and five instrumental classical.
Among children with ASD, two of three preferred children’s music over instrumental classical. Each had a different overall music preference: children’s, upbeat, and rock and roll.
Hypothesis 4b
Children would be more engaged when dancing to music they prefer.
This was not supported. There was no significant difference found in independent t-tests in the gameplay, interaction with robot, and interaction with human based on their music preference. There could be a lot of reasons for this. First, music might just not matter during musical activities with robots, though that intuitively seems quite extreme. A more limited interpretation is that the music used in this activity was insufficient either in emotional significance, impact, variation, duration, or some other factor to elicit differing responses. The songs used were not extreme examples of emotional range or stylistic differences and tempo was limited by the robot’s animation speed, which is significantly slower than people can dance.
Hypothesis 5
Both groups of children would dance more energetically to the upbeat songs than to the calm songs.
This hypothesis was supported. Kinect data for TD children showed significantly greater distance and speed of movement during upbeat songs compared to during calm songs. Paired samples t-tests of average displacement (t(299) = 7.23; p = 0.00) and velocity (t(299) = 5.92; p = 0.00) of the children’s joints during calm and upbeat songs were significantly different. Due to technical problems, similar data are not available for two of the three children with ASD. Since neither S5 nor S13 danced much at all, it seems unlikely there would be a difference. S14, for whom there was data, however, did dance more energetically to the upbeat songs.
Hypothesis 6
Both groups of children will mimic the movements of the robot or human during the game despite not being instructed to do so.
Mimicry did occur spontaneously with some TD children, but the average (M = 1.91; SD = 1.22) was little mimicry. While the average mimicry score interacting with the human (M = 2.02; SD = 1.16) was numerically higher than average mimicry while dancing with the robot (M = 1.80; SD = 1.22), the difference was not significant. None of the children with ASD showed mimicry (M = 1.0; SD = 0.0).
GENERAL DISCUSSION
At the outset of the experiment, we sought to address three general research questions: 1) Is there a difference between TD children and autistic children while playing a physical musical game?; 2) Is there a difference between how children interact with a humanoid robot and an adult human partner during the game?; and 3) How do different music choices influence the interaction of children and robots or children and an adult human during the game? By and large, TD children were more engaged than children with ASD (hypothesis 1). They also mimicked their partners more than autistic children in this study (hypothesis 6). This indicates that there are differences in how these populations play a physical musical game. The differences between interactions in the robot and human conditions also varied by population. The trend was for TD children to be more engaged with the adult human (hypothesis 2a) and for autistic children to engage more with the robot (hypothesis 2b). Music choices did influence the interactions in so far as children danced more energetically to upbeat songs (hypothesis 5) but genre did not have an effect (hypothesis 4b).
One of the significant challenges of this research is keeping the robot and human scenarios similar enough to be comparable while maintaining the distinct advantages of each. There is limited benefit to proving a robot is preferable to a clinician mimicking robotic behavior, for example. With that in mind, we elected to keep both interactions based on the same game, but otherwise as naturalistic as possible. For instance, the robot played music during the robot condition, but music was played from a laptop during the human condition.
The dance freeze game, particularly when played with just a partner instead of in a large group, appeared anecdotally to be more enjoyable for and more suited to extraverted children. Particularly for older children, dancing alone while being watched by their parent and the adult human appeared to make them self-conscious and uncomfortable. It would be wise to look for more solitary games in future studies of this kind.
The lack of significant results for music genre and style is disappointing. Intuitively, music ought to change how someone dances. It is difficult to point to one reason or another this did not occur in the study. Perhaps, there was not enough difference between the songs used. Perhaps, the nature of dance freeze or the experimental context minimized variation. Further research is needed to understand this and how music might be used more efficaciously in the final system.
Overall, this study has suggested that a robot is somewhat engaging for children regardless of whether they are autistic or not, but the effect seems more mixed than anticipated, particularly for TD children. Again, it is difficult to tease out what led to this result. It may be that the depth of interaction with the adult was so much greater than the depth of interaction with the robot that children warmed up more to the adult than the robot. The reason may be even simpler than that. There are differences between how a robot dances and how a human does. Some of the performance differences could be based in distraction watching the robot dance and having more difficulty emulating it than the adult. More research is required to clearly unpack this speculation.
What is hopeful, however, is the prevalence of triadic interactions between the participants, the robot, and the adult human. Dance freeze is not a game that typically involves talking, but both TD and autistic children engaged with the adult human to discuss the robot. Furthermore, most if not all of the families who participated in the experiment stayed after it was finished to see a demonstration of the robot and continue talking about it.
FUTURE WORK AND CONCLUSION
The results of this research will need to be validated with a larger sample of children with ASD or better with more carefully defined subsets of the autistic population to control for differing abilities. It would also be worthwhile to determine if the effects accrue over time or persist beyond a single session in a longer-term study that involves multiple interactions. Stakeholders interviewed repeatedly stressed that consistency and repetition were very important for children with ASD learning new knowledge and skills, so it is reasonable to expect a series of similar sessions may have better results. Additionally, dance freeze presented some unexpected challenges. A comparison of different activities that incorporate music and movement (Weisblatt et al., 2019), or other types of games (Roper, Millen Dutka, Cobb, & Patel, 2019; Sturm, Kholodovsky, Arab, Smith, Asanov, & Gillespie-Lynch, 2019) might yield a more universally enjoyable activity that could have better engagement.
There were very few effects based on music tempo or genre in this experiment. Input from the ASD professionals suggests that, for some children with ASD, music is a powerful but very specific motivator, with some children fixating on a single song for months. Further research using songs of the children’s selection might show greater impact. On the other hand, using more extreme examples of each genre or varying the emotion of the pieces may also work.
It would be beneficial to record the full session between the children and the adult human. Considerably rich interaction took place before, after, and between the songs and was missed with the current experiment’s recording scheme. Of course, the downside of this is that such interactions were highly variable, difficult to compare, and currently impossible to replicate naturally with a robot.
Finally, a number of interesting suggestions came up during the stakeholder interviews that may provide useful avenues of research. The idea of having children teach the robot to demonstrate knowledge presents opportunities for new scenario designs and motivations for using robots. Social stories could be the basis for interactions that teach specific functional skills, such as greeting an adult or verbally asking for a desired item. While this work focused on having children imitate the robot, future work could invert that dynamic and have the robot imitating the children as suggested by a physical therapist to improve body awareness.
The combination of robot therapy, dance therapy, and play therapy has potential for children both with and without ASD, but still needs more research to find effective combinations and interaction scenarios. Robots interest parents, educators, therapists, and – most importantly – children. They facilitated interactions between the researcher and the participants even when the structure of the experiment discouraged conversation. While the expected effects of music style were largely absent, a more thorough study that includes more popular music may have more success. There was significant individual variance on all fronts which is encouraging for the larger scheme of physical and musical robot interactions for children with ASD since one of the strengths of both music and robot interactions are their potential for individualization.
ACKNOWLEDGMENTS
This project is supported by the National Institutes of Health under grant No. 1 R01 HD082914-01.
AUTHOR BIOGRAPHIES
Jaclyn is a PhD candidate in Computer Science at Michigan Technological University. Her research interests include assistive technology, human computer interaction, human robot interaction, and computer security.
Chung Hyuk Park is an Associate Professor in Biomedical Engineering at The George Washington University. He received his PhD from Georgia Tech in 2012. His Assistive Robotics and Tele-Medicine Lab. conducts research on HRI and machine learning with special focuses on assistive robotics, telemedical robotics, haptics, computer vision, and multi-modal communications.
Ayanna Howard, Ph.D. is Professor and Chair of the School of Interactive Computing at Georgia Tech. Her work, which encompasses advancements in AI, assistive technologies, and robotics, has resulted in over 250 peer-reviewed publications. Prior to Georgia Tech, Dr. Howard was a senior robotics researcher at NASA’s JPL.
Myounghoon Jeon is an Associate Professor in Industrial and Systems Engineering at Virginia Tech. He received his PhD from Georgia Tech in 2012. His Mind Music Machine Lab conducts research on HCI and HRI with a focus on auditory displays, affective computing, assistive technologies, aesthetic computing, and automotive user interfaces.
References
- Accordino R, Comer R, & Heller WB (2007). Searching for music’s potential: A critical examination of research on music therapy with individuals with autism. Research in Autism Spectrum Disorders, 1(1), 101–115. 10.1016/j.rasd.2006.08.002 [DOI] [Google Scholar]
- Amanatiadis A, Kaburlasos VG, Dardani C, & Chatzichristofis SA (2017, September). Interactive social robots in special education. In 2017 IEEE 7th International Conference on Consumer Electronics-Berlin (ICCE-Berlin)(pp. 126–129). IEEE. [Google Scholar]
- American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, D.C.: American Psychiatric Association. [Google Scholar]
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, D.C. [Google Scholar]
- Amos P (2013). Rhythm and timing in autism: learning to dance. Frontiers in integrative neuroscience, 7, 27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Anagnostou E, Zwaigenbaum L, Szatmari P, Fombonne E, Fernandez BA, Woodbury-Smith M, … Scherer SW (2014). Autism spectrum disorder: advances in evidence-based practice. Canadian Medical Association Journal, 186(7), 509–519. 10.1503/cmaj.121756 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ariel Cascio M (2012). Neurodiversity: Autism pride among mothers of children with autism spectrum disorders. Intellectual and Developmental Disabilities, 50(3), 273–283. 10.1352/1934-9556-50.3.273 [DOI] [PubMed] [Google Scholar]
- Auger RW (2013). Autism Spectrum Disorders: A Research Review for School Counselors. Professional School Counseling, 16(4), 256–268. Retrieved from 10.5330/PSC.n.2013-16.256 [DOI] [Google Scholar]
- Baker MJ (2000). Incorporating the Thematic Ritualistic Behaviors of Children with Autism into Games: Increasing Social Play Interactions with Siblings. Journal of Positive Behavior Interventions, 2(2), 66–84. 10.1177/109830070000200201 [DOI] [Google Scholar]
- Baron-Cohen S (1991). Do People with Autism Understand What Causes Emotion? Child Development, 62(2), 385–395. [PubMed] [Google Scholar]
- Beer JM, Boren M, & Liles KR (2016). Robot Assisted Music Therapy: A Case Study with Children Diagnosed with Autism. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction (pp. 419–420). Piscataway, NJ, USA: IEEE Press. Retrieved from http://dl.acm.org/citation.cfm?id=2906831.2906906 [Google Scholar]
- Begum M, Serna RW, Konktak D, Allspaw J, Kuczynski J, & Yanco HA (2015). Measuring the Efficacy of Robots in Autism Therapy: How Informative are Standard HRI Metrics? Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, 335–342. 10.1145/2696454.2696480 [DOI] [Google Scholar]
- Bratton SC, Ray D, Rhine T, & Jones L (2005). The Efficacy of Play Therapy With Children: A Meta-Analytic Review of Treatment Outcomes. Professional Psychology: Research and Practice, 36(4), 376–390. 10.1037/0735-7028.36.4.376 [DOI] [Google Scholar]
- Burns R, Jeon M, & Park CH (2018). Robotic motion learning framework to promote social engagement. Applied Sciences, 8(2), 241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dautenhahn K (1999). Robots As Social Actors: Aurora and the Case of Autism. In Proc. CT99, The Third International Cognitive Technology Conference, 359(3), 374. [Google Scholar]
- Dautenhahn K, & Billard A (2002). Games children with autism can play with Robota, a humanoid robotic doll. Universal Access and Assistive Technology, 179–190. 10.1007/978-1-4471-3719-1_18 [DOI] [Google Scholar]
- Diehl JJ, Schmitt LM, Villano M, & Crowell CR (2012). The clinical use of robots for individuals with Autism Spectrum Disorders: A critical review. Research in Autism Spectrum Disorders, 6(1), 249–262. 10.1016/j.rasd.2011.05.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Durkin MS, Bilder D. a., Pettygrove S, & Zahorodny W (2015). The validity and usefulness of public health surveillance of autism spectrum disorder. Autism, 19(1), 118–119. 10.1177/1362361314548732 [DOI] [PubMed] [Google Scholar]
- Feil-Seifer D, & Matarić MJ (2009). Toward Socially Assistive Robotics for Augmenting Interventions for Children with Autism Spectrum Disorders. Springer Tracts in Advanced Robotics, 54, 201–210. 10.1007/978-3-642-00196-3_24 [DOI] [Google Scholar]
- Feil-Seifer D, & Matarić MJ (2011). Socially assistive robotics. Robotics & Automation Magazine, IEEE, 18(1), 24–31. 10.1109/ICORR.2005.1501143 [DOI] [Google Scholar]
- Ferrari E, Robins B, & Dautenhahn K (2009, September). Therapeutic and educational objectives in robot assisted play for children with autism. In RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication (pp. 108–114). IEEE. [Google Scholar]
- Geretsegger M, Elefant C, Mössler KA, & Gold C (2014). Music therapy for people with autism spectrum disorder. Cochrane Database of Systematic Reviews, (6). 10.1002/14651858.CD004381.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Halperin JM, Marks DJ, Bedard ACV, Chacko A, Curchack JT, Yoon CA, & Healey DM (2013). Training executive, attention, and motor skills: a proof-of-concept study in preschool children with ADHD. Journal of attention disorders, 17(8), 711–721. [DOI] [PubMed] [Google Scholar]
- Heaton P (2009). Assessing musical skills in autistic children who are not savants. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1522), 1443–1447. https://datausa.io/profile/soc/291123/#demographics accessed on November 9th 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Josefi O, & Ryan V (2004). Non-Directive Play Therapy for Young Children with Autism: A Case Study. Clinical Child Psychology and Psychiatry, 9(4), 533–551. 10.1177/1359104504046158 [DOI] [Google Scholar]
- Kapp SK, Gillespie-Lynch K, Sherman LE, & Hutman T (2013). Deficit, difference, or both? Autism and neurodiversity. Developmental Psychology, 49(1), 59–71. 10.1037/a0028353 [DOI] [PubMed] [Google Scholar]
- Kenny L, Hattersley C, Molins B, Buckley C, Povey C, & Pellicano E (2016). Which terms should be used to describe autism? Perspectives from the UK autism community. Autism, 20(4), 442–462. 10.1177/1362361315588200 [DOI] [PubMed] [Google Scholar]
- Kestenberg Amighi J, Loman S, Lewis P, & Sossin KM (1999). The meaning of movement: Developmental and clinical perspectives of the Kestenberg Movement Profile. Brunner-Routledge, New York, NY. [Google Scholar]
- Kim ES, Berkovits LD, Bernier EP, Leyzberg D, Shic F, Paul R, & Scassellati B (2013). Social robots as embedded reinforcers of social behavior in children with autism. Journal of Autism and Developmental Disorders, 43(5), 1038–1049. 10.1007/s10803-012-1645-2 [DOI] [PubMed] [Google Scholar]
- Kossak M (2009). Therapeutic attunement: A transpersonal view of expressive arts therapy. The Arts in Psychotherapy, 36, 13–18. [Google Scholar]
- Kozima H, Michalowski MP, & Nakagawa C (2009). Keepon: A playful robot for research, therapy, and entertainment. International Journal of Social Robotics, 1(1), 3–18. 10.1007/s12369-008-0009-8 [DOI] [Google Scholar]
- Lee C-HJ, Kim K, Breazeal C, & Picard R (2008). Shybot: Friend-stranger Interaction for Children Living with Autism. In CHI ‘08 Extended Abstracts on Human Factors in Computing Systems (pp. 3375–3380). New York, NY, USA: ACM. 10.1145/1358628.1358860 [DOI] [Google Scholar]
- Maenner MJ, Shaw KA, Baio J, et al. Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016. MMWR Surveill Summ 2020, 69 (No. SS-4): 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mandell D, & Lecavalier L (2014). Should we believe the Centers for Disease Control and Prevention’s autism spectrum disorder prevalence estimates? Autism, 18(5), 482–484. 10.1177/1362361314538131 [DOI] [PubMed] [Google Scholar]
- Martin M (2014). Moving on the spectrum: Dance/movement therapy as a potential early intervention tool for children with Autism Spectrum Disorders. The Arts in Psychotherapy, 41(5), 545–553. [Google Scholar]
- Myers SM, & Johnson CP (2007). Management of Children With Autism Spectrum Disorders. Pediatrics, 120(5), 1162–1182. 10.1542/peds.2007-2362 [DOI] [PubMed] [Google Scholar]
- Park CH, Jeon M, & Howard AM (2015, August). Robotic framework with multi-modal perception for physio-musical interactive therapy for children with autism. In 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) (pp. 150–151). [Google Scholar]
- Robins B, Amirabdollahian F, & Dautenhahn K (2013). Investigating Child-Robot Tactile Interactions: A taxonomical classification of tactile behaviour of children with autism towards a humanoid robot. International Conference on Advances in Computer-Human Interactions, (c), 89–94. [Google Scholar]
- Robins B, Ferrari E, & Dautenhahn K (2008). Developing scenarios for robot assisted play. Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN, 180–186. 10.1109/ROMAN.2008.4600663 [DOI] [Google Scholar]
- Roper T, Millen Dutka L, Cobb S, & Patel H (2019). Collaborative virtual environment to facilitate game design evaluation with children with ASC. International Journal of Human–Computer Interaction, 35(8), 692–705. [Google Scholar]
- Scassellati B (2005). Quantitative metrics of social response for autism diagnosis. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication, 2005, 585–590. 10.1109/ROMAN.2005.1513843 [DOI] [Google Scholar]
- Scassellati B (2007). How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism. Robotics Research, 552–563. 10.1007/978-3-540-48113-3_47 [DOI] [Google Scholar]
- Scassellati B, Boccanfuso L, Huang CM, Mademtzi M, Qin M, Salomons N, … & Shic F (2018). Improving social skills in children with ASD using a long-term, in-home social robot. Science Robotics, 3(21), eaat7544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scharoun SM, Reinders NJ, Bryden PJ, & Fletcher PC (2014). Dance/Movement Therapy as an Intervention for Children with Autism Spectrum Disorders. American Journal of Dance Therapy, 36(2), 209–228. [Google Scholar]
- Sturm D, Kholodovsky M, Arab R, Smith DS, Asanov P, & Gillespie-Lynch K (2019). Participatory Design of a Hybrid Kinect Game to Promote Collaboration between Autistic Players and Their Peers. International Journal of Human–Computer Interaction, 35(8), 706–723. [Google Scholar]
- Vazquez V, Cardenas C, Cibrian FL, & Tentori M (2016). Designing a Musical Fabric-Based Surface to Encourage Children with Autism to Practice Motor Movements. Proceedings of the 6th Mexican Conference on Human-Computer Interaction - MexIHC’16, 1–4. 10.1145/2967175.2967384 [DOI] [Google Scholar]
- Weisblatt EJ, Langensiepen CS, Cook B, Dias C, Plaisted Grant K, Dhariwal M, … & Rybicki A (2019). A tablet computer-assisted motor and language skills training program to promote communication development in children with autism: development and pilot study. International Journal of Human–Computer Interaction, 35(8), 643–665. [Google Scholar]
- Whipple CM, Gfeller K, Driscoll V, Oleson J, McGregor, & Karla. (2015). Do Communication Disorders Extend to Musical Messages? An Answer from Children with Hearing Loss or Autism Spectrum Disorders. Journal of Music Therapy, 52(1), 78–116. Retrieved from https://search.proquest.com/docview/1721982825?accountid=28041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhang R, Barnes J, Ryan J, Jeon M, Park CH, & Howard A (2016). Musical robots for children with ASD using a client-server architecture. Proceedings of the 22nd International Conference on Auditory Display, Canberra, Australia, July 2016. [Google Scholar]