A trial of an iPad^™ intervention targeting social communication skills in children with autism

Abstract

This study evaluated a technology-based early intervention for social communication skills in pre-schoolers in a randomised controlled trial. Participants were 54 children aged under 6 years with a diagnosis of autism, assigned to either intervention or control conditions. The app engaged children, who played consistently, regardless of developmental level, and was rated highly by parents. There were no significant group differences in parent-report measures post-intervention, nor in a measure of parent–child play at follow-up. Therefore, this intervention did not have an observable impact on real-world social communication skills and caution is recommended about the potential usefulness of iPad^™ apps for amelioration of difficulties in interaction. However, positive attitudes among participants, lack of harms and the potential of apps to deliver therapeutic content at low economic cost suggest this approach is worth pursuing further, perhaps targeting other skill domains.

Technology-based interventions have proven successful in teaching new skills to children with autism spectrum disorder (ASD) (Grynszpan et al., 2013; Pennington, 2010; Ramdoss et al., 2011a, 2011b). Social skills are often considered a priority focus for autism intervention (White et al., 2007), and technology-based training, despite being considered a ‘non-social’ medium, has also been successfully applied to this target (Beaumont and Sofronoff, 2008; Ramdoss et al., 2012). However, there is still a dearth of high-quality trials of technological interventions for autism (Grynszpan et al., 2013). Moreover, until now technological interventions have not been provided for very young children, probably because the skills required to access the traditional computer interfaces of keyboard and mouse are beyond most pre-schoolers’ abilities (Clark et al., 2014). Even when working with older children, computer-based learning studies have often had to incorporate an extended period of teaching basic computer skills before the intervention could be accessed (Hopkins et al., 2011; Williams et al., 2002).

The advent of touchscreen tablets (such as the iPad^™) means that we can now explore the delivery of therapeutic and educational content to very young children using technology (Díez-Juan et al., 2014; Kagohara et al., 2013; Murdock et al., 2013). This approach brings together the belief in early intervention as the best opportunity for children with ASD (Wallace and Rogers, 2010) with the specific advantages of technology. These include the use of a highly motivating medium with integrated rewards, close control of learning rate and content and personalisation to the child’s own preferences (Fletcher-Watson, 2014). For example, if children with ASD have specific difficulties using social cues as feedback for learning (Bedford et al., 2013), non-social rewards embedded in computer games may be more effective. In addition, the intervention has the potential to be immediately and widely shared with an international community of beneficiaries. Given the popularity of touchscreen technologies among families with children with autism, combined with a lack of good evidence, there is a pressing need for well-designed studies in this area (Clark et al., 2014).

What is an appropriate target for a technology-based intervention for pre-schoolers with ASD? One possibility is joint attention (Charman, 2003) which has been established in research as an important skill in both typical and atypical development (Moore and Dunham, 1995). Joint attention abilities have been linked to later language and to social interaction profiles in autism (Mundy and Newell, 2007). Moreover, the consistent presence of joint attention impairments among children with ASD means that this is a key item in early screening and diagnostic instruments (e.g. Lord et al., 2012; Wetherby and Prizant, 2002). Joint attention and related social communication skills are often the explicit focus of early intervention strategies (Kaale et al., 2012; Kasari et al., 2006; Wong and Kwan, 2010), which are usually therapist-led. Such interventions have been modestly successful in improving joint attention abilities in young children with ASD, but overall findings are mixed with a lack of evidence for generalisation to novel settings and maintenance in long-term follow-up in particular (Fletcher-Watson et al., 2014). In addition, therapist-led interventions such as these involve a very high financial investment which could be reduced by the use of technology-based support.

Joint attention at its most complex involves co-ordinated, triadic attention between the child, another person and an object, cued by eye-gaze, gestures such as pointing, and verbal cues (e.g. ‘look at that!’). This is challenging to re-create in a technological learning environment. However, joint attention also builds on important developmental precursors including looking at other people, and following their cues – looking where they are looking or pointing (Striano and Reid, 2006). It is established that these sub-skills are also frequently absent among young children with autism (Dawson et al., 1998; Leekam et al., 1998). Targeting intervention at these basic components of social attention has the theoretical potential to trigger a cascade of social communication development. Such cascading effects, from a proximal intervention target to distal social and communication behaviours, or from a specific skill to the global array of features of autism, have been emphasised in recent intervention trial reports (e.g.(Green et al., 2010)) and reviews of the literature (Yoder et al., 2013).

This study evaluates the impact of a specially-developed iPad application (hereafter ‘app’) called FindMe on these two basic social communication skills in young children with ASD. The development of the app included user-centred design and pilot testing to create an engaging app with a foundation in theoretical models of ASD development (Fletcher-Watson et al., 2013). FindMe aimed to enhance the real-world social communication skills of the children through motivating, daily rehearsal of very basic sub-skills. On occasion, screen-based technologies have been successful in teaching skills of relevance to face-to-face social interactions (for a review, see Ramdoss et al., 2012) but it is not clear whether this will be the case for this specific app and population. Nevertheless, for the intervention to have genuine therapeutic value, we believe it must show an impact on social communication behaviours in interaction with others, and not just improvement within the game, and thus efficacy is evaluated in this context. Thus, we hypothesise that children in the intervention group will show changes in social communication (e.g. eye contact, social response, directed vocalisation) observed in interaction with their caregiver. Treatment effects on the same social communication behaviours, and on language, measured using parent report, will also be evaluated.

Method

Design

The study used a randomised controlled design in which participants were allocated to either an Intervention (n = 27) or waitlist Control (n = 27) condition following baseline assessment. The Intervention group received 2 months of app access at the same time as all usual treatments, while the waitlist group received only treatment as usual (TAU). TAU in the region in which the study took place consists largely of one-to-one support in nursery or primary school, including specialist units and integrated mainstream classes; some children receive low levels of speech and language therapy (52% of the sample reported <30 min per month) and occasionally occupational therapy. At Baseline, mean hours of additional support of any kind, per week, was 11.5 h, for the whole sample. Measures were taken immediately following the intervention period (or equivalent time point) and again at 6-month follow-up, measured from the Baseline appointment. Figure 1 illustrates the trial process including all recruitment figures.

Figure 1.

CONSORT diagram illustrating participant experience and sample sizes.

The study was approved by the local UK National Health Service ethics committee (reference number: 10/S0501/66) and registered with the United Kingdom Clinical Research Network (ID 10428) and on clinicaltrials.gov (ref. NCT01493609). Parents gave informed consent on behalf of their children, and consent of the parent and assent of the participating child were reaffirmed at each meeting.

Participants

Children were recruited via two main routes. Most participants in the study were approached via the Communication Clinic at the Royal Hospital for Sick Children, Edinburgh, a local hub for autism diagnosis and intervention services. Other participants came forward having heard about the study through word of mouth or online. On making an expression of interest via either route, participants were sent further information, contacted in person to ascertain basic eligibility and then invited to the assessment site to give informed consent and for baseline assessment.

Inclusion criteria for the study were that the participant was (a) aged under 6 years at the time of first assessment (in the country of recruitment, all children aged 6 or older are in school); (b) had a clinical diagnosis of ASD or was on the waiting list for diagnosis; (c) met the Autism cut-off on the Autism Diagnostic Observation Schedule (ADOS) social-communication algorithm; (d) did not have a neurological disorder such as epilepsy; and (e) had English-speaking parents (sufficient to give informed consent). Table 1 summarises descriptive information about the two groups. Two children (both assigned to the control group) had Down syndrome as well as an autism diagnosis.

Table 1.

Child and family characteristics at Baseline by group.

		Intervention (n = 27)	Control (n = 27)
Age in months	Mean (SD)	49.30 (10.9)	49.96 (13.2)
Gender	Male	21 (78%)	22 (81%)
	Female	6 (22%)	5 (19%)
Ethnicity	White	23 (85%)	24 (89%)
	Other	4 (15%)	3 (11%)
SESa	Bottom 20% in Scotland	4 (15%)	5 (19%)
	Middle 60% in Scotland	12 (46%)	14 (52%)
	Top 20% in Scotland	10 (39%)	8 (29%)
Maternal education	College or below	15 (55%)	16 (60%)
	Undergraduate or above	11 (41%)	11 (40%)
	Don’t know	1 (4%)	0
Hours of additional support per weekb	Median (IQR)	10.5 (4–20.5)	7.5 (2–13.5)
Hours of computer gaming per weekb	Median (IQR)	5.0 (1.5–10.5)	5.0 (0–7)
Days from Baseline to Outcome	Mean (SD)	72 (7)	73 (7)
Days from Baseline to Follow-Up	Mean (SD)	181 (28)	178 (21)
Dropout	During intervention	1 (4%)	0
	Before Follow-Up	2 (7%)	2 (7%)

SES: socio-economic status; SD: standard deviation; IQR: inter-quartile range.

There is no evidence of significant differences between groups on any Baseline variables.

^aBased on Scottish Index of Multiple Deprivation (SIMD). One family in the intervention group enrolled from England and so they are not included in this calculation.

^bAs these variables were positively skewed, medians and IQR are reported.

Randomisation

Randomisation was stratified by ADOS social-communication algorithm score (12–17, versus ⩾ 18) to ensure that both groups had equal representation of children with more or less severe autism symptom profiles. Two randomisation lists were produced using block randomisation with varying and randomly-ordered block sizes. An independent researcher subsequently produced numbered opaque envelopes containing the random allocation.

Procedure

Before the Baseline appointment, children were provided with a short visual ‘social story’ to introduce them in advance to the researcher, location and activities. In this 2 hr appointment, assessments were conducted in a flexible order depending on the child’s mood. For example, selecting a parent–child play activity to start the session for very anxious children. As a break from other activities, the child was given an opportunity to interact with an iPad with a series of commercially available apps for pre-schoolers. It was apparent that all the children were able to interact with the iPad and quickly learn to use the touchscreen interface. After scoring of assessment measures, stratified random allocation was completed and parents notified by phone.

For the intervention group, iPads were sent out to each child’s home using an overnight secure delivery service. Functions aside from the intervention app were shut down. Parents were provided with a brief instruction document which dealt with the basics of working and charging the iPad and offering advice on troubleshooting. The document suggested that parents aimed for game play of about five minutes per day, or ten minutes every other day.

At the end of the intervention period, or equivalent timing for the waitlist group, immediate outcome data were collected. For the intervention group, outcome data were collected at a home visit during which the iPad was also taken back. For the control group, data collection was by phone and post. The Follow-Up appointment at the assessment site was approximately 6 months after Baseline, when most of the baseline assessments were repeated. There were no differences between groups in the timing of data collection (see Table 1).

Measures: baseline characterisation

Mullen Scales of Early Learning (MSEL)

The visual reception and only fine motor sub-scales were administered as these evaluate skills relevant to the intervention delivery system – that is, those needed to use an iPad touchscreen (Mullen, 1995). About a third of children (n = 20) did not achieve baseline Mullen scores due to their lack of engagement with the tasks (e.g. wandering, refusal to co-operate), but were retained in the study. Following randomisation, we confirmed that these children had been equally distributed between the two groups (n = 10 per group).

Individual background form

Parents completed a bespoke background questionnaire which gathered information about family circumstances and both parents’ education and employment status. This information was collected to check for group matching on these variables, as they may influence access to TAU (Salomone et al., 2015) or response to intervention.

Measures: outcome

ADOS

All children were administered an ADOS module one or two (n = 43, module one; n = 9, module two) at Baseline to confirm diagnostic status, and at Follow-Up (Lord et al., 1999, 2012). For enrolment in the study, children were required to score above the algorithm cut-off for Autism. Subsequently, these assessments were re-scored using the new ADOS-2 criteria and these scores are provided in Table 2.

Table 2.

Means and standard deviations for all Baseline, Outcome and Follow-Up measures by group.

		Baseline		Outcome		Follow-Up
		Intervention (n = 27)	Control (n = 27)	Intervention (n = 26)	Control (n = 27)	Intervention (n = 24)	Control (n = 25)
MSELa	Fine motor raw score	28.9 (9.9)	25.4 (10.8)
	Fine motor age equivalent (months)b	27 (22–36)	21 (16–25)
	Visual reception raw score	29.4 (10.8)	28.9 (14.7)
	Visual reception age equivalent (months)	30 (14.5)	31 (21.9)
ADOS-2	Social affect total	13.8 (3.9)	13.2 (4.4)			12.0 (5.1)	10.9 (5.4)
	Restricted repetitive behaviour total	3.0 (2.3)	2.7 (2.1)			3.1 (2.6)	2.7 (2.2)
	Overall total	16.7 (5.8)	15.9 (5.7)			15.1 (7.0)	13.6 (6.9)
	Comparison score	7.7 (1.4)	7.3 (1.3)			7.1 (1.7)	6.1 (1.9)
BOSCC	Overall total	32.1 (11.1)	33.4 (12.1)			32.0 (11.7)	31.2 (12.1)
	Social communication total	24.6 (8.5)	25.5 (8.3)			24.7 (8.0)	23.9 (8.4)
MCDI	Words understood total	207 (135)	203 (129)	223 (140)	221 (132)	226 (138)	247 (141)
	Words produced total	142 (156)	123 (138)	162 (163)	148 (155)	167 (165)	178 (165)
	Gestures total	30 (15.1)	33 (14.4)	31 (16.4)	32 (15.2)	36 (17.4)	35 (17.5)
CSBS-DP	Social communication total	23.8 (6.2)	24.5 (6.1)	24.9 (6.0)	24.9 (5.2)	25.3 (5.8)	27.3 (5.9)
	Gestures total	6.6 (2.5)	7.1 (2.1)	6.9 (2.3)	7.0 (2.2)	7.1 (2.7)	7.7 (2.4)

MSEL: Mullen Scales of Early Learning; ADOS: Autism Diagnostic Observation Schedule; BOSCC: Brief observation of social communication change; MCDI: MacArthur Communicative Development Inventory; CSBS-DP: Communication and Symbolic Behaviour Scale – Developmental Profile.

For the ADOS and BOSCC, lower numbers indicate improvement; for the MCDI and CSBS, improvement is indicated by a higher score.

^aSome children were not able to complete the Mullen assessment and so these data report on a sub-set of the sample.

^bThese data are skewed and so we report medians and inter-quartile ranges.

Brief observation of social communication change (BOSCC)

The primary outcome measure was a preliminary version of the brief observation of social communication change (BOSCC), an in-development way of scoring parent–child interaction to provide a change-sensitive measure of key autistic behaviours (Grzadzinski et al., submitted). The measure was collected at Baseline and Follow-Up, though not immediately post-intervention. These time points were selected to reduce participant burden, but also to assess the developing effect of the intervention on complex social communication behaviours founded on the basic skills rehearsed within the app.

Parents and children were filmed for 10 min playing with a standard set of toys with no specific instructions given. These episodes were then coded using a 16-item manifest to score behaviours familiar from the ADOS system, but in the context of a naturalistic parent–child interaction. Items provide a comprehensive account of the key features associated with autism, such as Unusual Eye Contact, Quality of Social Overtures and Unusually Repetitive Interests or Stereotyped Behaviours. Each item is scored on a scale from 0 to 5, with higher scores indicating more atypical behaviour. The BOSCC provides a total score (using all core items, numbered 1–13) and a social communication sub-score (using the first eight items only). In addition, there are three items (14–16) which do not contribute to the total score, as these are designed instead to capture the child’s mood at the assessment (e.g. anxiety, activity level). Scale internal consistency was good with Cronbach’s alpha of 0.864 for items 1–8 and 0.856 for items 1–13 (using Baseline data).

To minimise risk of bias and reduce variability, all BOSCC codes from both Baseline and Follow-Up appointments were scored by a single, independent researcher (J.S.B.) who was trained on the BOSCC and blind to participant group. Using a random sampling procedure stratified across sessions, we selected one-third (n = 36) of BOSCC recordings for inter-rater reliability evaluations. The second rater for inter-rater agreement was the first author (S.F.-W.), who was blind to the first rater’s results but not blind to group or time point. Mean whole scale agreement (ratings within one point scored as agreements) was high at 83% between raters.

MacArthur Communicative Development Inventory – words and gestures (MCDI)

This checklist of vocabulary was completed by all parents at Baseline, Outcome and Follow-Up assessments. The checklist provides individual scores for words understood, words used and gestures (Fenson et al., 1993).

Scripted interview including Communication and Symbolic Behaviour Scales – Developmental Profile and Gaming Experience

The social sub-scale of the caregiver questionnaire was administered to parents as part of a scripted interview at Baseline, Outcome and Follow-Up (Communication and Symbolic Behaviour Scale (CSBS); Wetherby and Prizant, 2002). CSBS items assess the presence of social communicative behaviours in everyday life. Sample items of particular relevance to this intervention include If you look and point to a toy out of your child’s reach, does your child look at the toy? Does your child greet you when you come into the room or leave? In the same interview, information was gathered about the child’s current nursery, school or childcare provision; current TAU; and time spent using electronic media. This last variable was captured by asking parents to provide an estimate of the amount of time their child spent watching TV per week, and the amount of time their child spent playing computer games per week. To increase precision, parents were prompted to consider weekday average screen time in each category, across settings and devices, and how this might vary at the weekend. The purpose of this variable was to permit exploration of possible interactions between response to intervention and experience with screen media more generally.

Reporting on game play

In the intervention group, we collected parent impressions of the app via a short, loosely scripted interview at Outcome. In addition, game play data were downloaded from the iPad, providing information about the number of completed scenes, the total number of minutes of play, the maximum level reached and average session length (defined as a period of play without breaks).

Blinding

Baseline assessments were administered and scored by S.F.-W. before group allocation. It was not possible to blind participants or parents to intervention allocation. However, participants and parents were blind to hypotheses regarding the skills being targeted by the intervention. Assessments at immediate Outcome were all parent-reported measures and were not blinded. Follow-Up assessments were administered and scored by an independent researcher (A.P.) who was blind to group. As noted above, a different blind researcher (J.S.-B.) scored the assessment used for primary outcome (BOSCC) at both Baseline and Follow-Up.

Intervention

More information about the evidence-based design of the FindMe app can be found elsewhere (Fletcher-Watson et al., 2013). The app was designed to give children an opportunity to rehearse two key social communication skills: attending to people and following social cues. In Part 1, the correct response was to touch the single person shown on the screen (see Figure 2). In Part 2, the character was depicted in a shop and looked and pointed to a desired item in one of six locations around the screen. The correct response was to touch the item being pointed and looked at; this was the only motor skill required to successfully play the app. Correct responses resulted in the child being awarded a token which appeared in the bottom left corner of the screen. Collection of five tokens resulted in presentation of a short animation sequence designed to provide a positive reinforcement. As the child progressed, levels increased in complexity with more distractors appearing on screen, and in Part 2, the character moved from pointing and looking, to just looking at a target.

Figure 2.

Screenshots from the FindMe app.

Statistical methods

Sample size was determined with reference to a previous evaluation of an intervention targeting social and communication skills, involving 28 pre-school children with ASDs and producing a large (d = 0.8) effect size for the ADOS social-communication algorithm following a 6-month intervention (Aldred et al., 2008). These figures suggested that a sample of 26 participants per group is required to arrive at an alpha level of 0.05 with 80% power.

Five children did not complete all of the study appointments (see Figure 1). One provided no data at Outcome, one provided partial data at Outcome (MCDI only), and three provided full Outcome data. None of these five provided Follow-Up data and reasons for dropout are given in Figure 1. In addition, at Outcome and Follow-Up, some participants did not complete the MCDI measure of vocabulary when asked to complete and return by post. There were missing MCDI data at Outcome for four participants in the intervention group and five participants in the control group. Only one participant (intervention group) had missing MCDI data at Follow-Up. We adopted an intention-to-treat model for analysis, in which missing values were replaced using the conservative method of carrying forward the last recorded observation for that measure.

All outcomes were calculated as change scores by calculating the difference between Baseline and Outcome or Follow-Up score for each individual, to create variables where a positive value indicates a positive change. These were compared across groups using t-tests or, if not normally distributed, Mann–Whitney U. We used Pearson’s r with Bonferroni corrections (alpha level for statistical significance of 0.05/number of variables entered) to explore the possibility of relationships between participant characteristics and size of change observed. Where individual data points were outlying according to Grubb’s test, (p = 0.05), correlational analyses were checked excluding these outliers.

In addition, we used a measure of reliable change to identify individual participants who may have responded to the intervention. Reliable change was defined on the BOSCC according to the formula developed by Jacobson and Truax (1991) and used in other similar intervention studies (Frankel et al., 2010; Goods et al., 2013). This formula defines whether each individual participant’s change score significantly exceeds what might be expected from normal variability across multiple evaluations. As a proxy for test–retest reliability of this pre-publication measure, we used correlation between BOSCC total scores at Baseline and Follow-Up for the control group only (r = 0.87, p < 0.001).

Results

Game play

Children in the intervention group had access to the FindMe app for a period of 72 days on average (95% CI = 70–75 days). The mean number of days on which children actually played the app was 28 (95% CI = 20–36 days). This play was distributed over a mean number of 1.7 unbroken ‘sessions’ of play per day (95% CI = 1.5–1.9 sessions), each one lasting on average 11 min (mean = 10.78 min, 95% CI = 9.2–13.0 min).

The total number of minutes played per child was skewed. Median length of game play was 339 min (inter-quartile range = 206–1074) over the intervention period (minimum 15 min, maximum 3522 min). This median equates to game play of approximately 6 min per session. In other words, to the extent that there was an ‘average’ child in the intervention group, that child played the game about every second or third day for 10 weeks, normally playing for 2 sessions per day, with each session lasting 5–10 min.

Only four children in the Intervention group failed to reach the most complex level of the game. Of the 23 children reaching the highest level, 22 carried on to repeat the game cycle after achieving that top level. There was no evidence of statistically significant correlation between minutes of game play and baseline variables including age, socio-economic status, parental educational level, ADOS-2 scores or level of language (all p > 0.15). In particular, there was no relationship between game play and either Mullen visual reception (n = 19, r = −0.341, p = 0.152) or fine motor (n = 17, r = −0.334, p = 0.190) raw scores.

When one outlying data point in minutes of game play was removed, some correlations were revealed. These were not statistically significant after Bonferroni corrections for multiple correlations (alpha level for significance = 0.003) but may be worth noting due to the small sample size involved. There was a positive correlation between maternal age of leaving school and total game play (n = 22, r = 0.547, 95% CI: 0.16–0.79, p = 0.008). There were negative correlations between Mullen visual reception and fine motor raw scores and total game play (respectively, n = 18, r = −0.502, 95% CI: −0.78–−0.05, p = 0.034 and n = 16, r = −0.530, 95% CI: −0.81–−0.05, p = 0.035). These indicate that children with more highly educated mothers and lower developmental level may have played the game for longer.

Qualitative data

Parents were asked to give their verbal comments on the app and what they perceived to be their child’s response to it. Replies were categorised as Positive, Mixed or Negative. Questions with percentage of parents giving affirmative responses were What was your overall experience with the app? (60% positive); Did your child play the app? (92% positive); Did your child like the app? (96% positive); Did you like the app? (92% positive); Was it easy to use? (100% positive). In addition, only 7% of parents reported experiencing any technical faults.

The two children who dropped out of the study citing lack of enjoyment of the app as their reason did play the app less than most others (105 and 159 min, respectively). These children were among the most experienced in computer gaming and we therefore hypothesise that for them, our app, which followed a simple, repetitive structure, was not exciting enough to hold their attention.

Background variables

Additional support and access to computers (both measured in hours per week) were recorded by parent report at Baseline, Outcome and Follow-Up. There were no significant differences between groups in the amount of additional support received at any time point (all p > 0.10) nor in the amount of time spent playing computer games (all p > 0.40). In addition, differences in scores reflecting how additional support and computer game play changed over time showed no differences between groups (all p > 0.50).

Primary and secondary outcomes: group comparisons

Change scores for the BOSCC total and social communication sub-score revealed no significant differences between intervention and control groups (both p > 0.28). Furthermore, there were no significant correlations between baseline variables and degree of change from Baseline to Follow-Up, including child age at Baseline; maternal age; maternal age of leaving school; hours of additional support received by child per week (both at Baseline and a measure of change during the intervention period); and hours of non-intervention computer game play (both at Baseline and a measure of change during the intervention period). In addition, for the Intervention group only, we examined the relation between BOSCC total change and social-communication change score and the total number of minutes played, recorded by the app itself (Figure 3). These correlations were also non-significant (both p > 0.16) and remained so when re-examined, excluding two participants whose minutes of game play exceeded two standard deviations above the mean.

Figure 3.

The relationship between game play (total minutes) and change on the BOSCC primary outcome measure. This figure includes only observed data and not scores derived from imputed values.

Change scores between Baseline and Outcome or Baseline and Follow-Up on the MCDI, Communication and Symbolic Behaviour Scale – Developmental Profile (CSBS-DP) caregiver questionnaire and ADOS-2 were calculated for each participant. No statistically significant differences were found (all p > 0.06, see Table 3).

Table 3.

Means and standard deviations for all Change scores by group, with mean difference, 95% confidence intervals and p values for group comparisons.

		Baseline to Outcome		Difference in means, 95% CI and p value for group comparison	Baseline to Follow-Up		Difference in means, 95% CI and p value for group comparison
		Intervention (n = 27)	Control (n = 27)		Intervention (n = 27)	Control (n = 27)
BOSCC	Overall total				0.13 (8.44)	2.17 (5.08)	−2.04 (−5.84–1.77), p = 0.288
	Social communication total				0.91 (5.75)	1.69 (3.82)	−0.78 (−3.44–1.89), p = 0.561
ADOS-2	Communication total				0.67 (1.44)	0.63 (1.42)	0.04 (−0.74–0.82), p = 0.925
	Reciprocal social interaction total				1.11 (1.72)	1.67 (2.92)	−0.56 (−1.86–0.75), p = 0.399
	Social affect total				1.78 (2.26)	2.30 (3.46)	−0.52 (−2.11–1.08), p = 0.518
	Restricted repetitive behaviour total				−0.15 (1.17)	−0.04 (2.01)	−0.11 (−1.01–0.79), p = 0.805
	Overall total				1.63 (2.75)	2.26 (3.28)	−0.63 (−2.28–1.02), p = 0.448
	Comparison score				0.67 (1.39)	1.15 (1.83)	−0.48 (−1.37–0.41), p = 0.281
MCDI	Words understood total	15 (31)	18 (43)	−3.4 (−23.9–17.2) p = 0.743	19 (42)	44 (72)	−25 (−57.4–7.2), p = 0.124
	Words produced total	20 (31)	25 (44)	−5.5 (−26.2–15.2) p = 0.598	25 (38)	55 (81)	−30 (−64.6–5.1), p = 0.092
	Gestures total	1.0 (6.9)	−0.8 (6.1)	1.78 (−1.76–5.32) p = 0.318	6.1 (8.3)	2.0 (8.0)	4.15 (−0.28–8.58), p = 0.066
CSBS-DP	Social communication total	1.07 (3.80)	0.33 (5.12)	0.74 (−1.72–3.20) p = 0.549	1.52 (3.78)	2.80 (5.16)	−1.26 (−3.73–1.21), p = 0.311
	Gestures total	0.26 (1.63)	−0.19 (1.39)	0.44 (−0.38–1.27) p = 0.286	0.52 (1.76)	0.56 (1.45)	−0.04 (−0.92–0.84), p = 0.933

ADOS: Autism Diagnostic Observation Schedule; MCDI: MacArthur Communicative Development Inventory; CSBS-DP: Communication and Symbolic Behaviour Scale – Developmental Profile.

Mean change scores have been calculated such that in each case, improvement is indicated by a positive score.

Exploratory analysis of meaningful change

There was variability in the degree of change exhibited by participants in both groups (Figure 3). Therefore, we explored whether we could identify a sub-set of participants showing evidence of intervention effects. For each participant having full BOSCC data at both Baseline and Follow-Up (n = 49), we calculated whether they exhibited reliable change at the 5% level. No participant showed a significant negative change. Participants in each group showing reliable change were intervention group reliable change (n = 2 (8%)) and control group reliable change (n = 1 (4%)). Using the same calculation with BOSCC scores for the social communication score only (items 1–8), participants in each group showing reliable change were intervention group reliable change (n = 3 (12%)) and control group reliable change (n = 2 (8%)).

Discussion

We report on a randomised controlled trial of a therapeutic iPad app targeting basic social communication skills. The trial design set a high bar for measuring benefit by selecting as the outcome measure a parent–child play-based observational measure taken at the Follow-Up appointment and not immediately following the intervention. This decision was made because the theoretical model for the study design was that successful intervention influencing the developmental precursors of joint attention (attention to people, following social cues) could trigger a cascade of skill development which would be most apparent some time after the intervention period itself (Yoder et al., 2013).

Thus, we were judging the ability of the app to produce a generalised and sustained change in symptom-relevant real-world behaviour – something which few technology-based interventions have achieved (Golan et al., 2009; Swettenham, 1996), but which is essential if technology-based interventions are to have clinically significant impact. Analyses did not reveal statistically significant effects of intervention on the primary outcome, nor on other measures, including parent-report measures taken immediately following the intervention period. This does not mean that no gains were made – in contrast, we recorded substantial improvements in both groups across the majority of measures recorded.

Taken together, results indicate that the intervention did not have an immediate effect on the behaviours targeted by the app (e.g. parent report of following others’ points in the CSBS-DP) nor were there later downstream effects on social behaviours observed in naturalistic parent–child interaction. We infer that the app failed to trigger the hypothesised learning cascade and attribute this to the challenge for the child to generalise a skill learnt within a two-dimensional animated game to a real-world scenario. Difficulty with generalisation is a component of some theoretical models of autism (Happé and Frith, 2006), and other early social skills interventions have also uncovered limited generalisation of interventions in this population (Fletcher-Watson et al., 2014).

Limitations

This study used a new measure as the primary outcome, which was administered at 6-month follow-up (when cascading intervention effects were predicted to be seen) but not immediately post-intervention. The authors had access to a pre-publication version of the measure including full definitions of items, specific ratings and decision trees designed to facilitate item-level inter-rater agreement. Nevertheless, while we demonstrated good inter-rater agreement, it is unclear how these findings relate to other intervention studies given the reliance on a measure which had not yet been established or validated in the literature (Grzadzinski et al., submitted). In particular, the lack of intervention effect is potentially attributable to a lack of sensitivity in measurement, though we also note that this absence of group-level effects was observed across all the measures employed, at both Outcome and Follow-Up. Identification of outcome measures which can be blind-rated and are sensitive to change on relevant dimensions is a pressing problem in early autism intervention research (Fletcher-Watson and McConachie, 2015; McConachie et al., 2015).

Can apps provide beneficial effects?

The results of this study should be treated with caution. They do not imply that other apps for the iPad (or other devices) would not be beneficial, any more than a null effect in a therapist-led intervention could be used to discredit all therapist-led interventions. On the contrary, we believe that this study provides several positive findings which lead us to conclude that touchscreen mobile devices should continue to be investigated as a way to support the development of children with ASD.

This report demonstrates the feasibility of conducting a randomised controlled trial using iPads. While use of the app was variable, this mode of intervention delivery permitted detailed adherence data to be collected and incorporated into analyses. The relaxed instructions to parents mimicked real-life application of this kind of therapeutic support, increasing external validity of the findings (Jonsson et al., 2015). Concerns at the outset about the security of this approach were not upheld. Faults with the iPad and the app were rare and usually straightforward to fix. There were no problems with access of inappropriate content via the iPad and no parent reported concerns about the child becoming obsessive. Parent perceptions of the intervention were largely positive and it is probable that one of the main advantages of this kind of approach is in the potential beneficial impact on family life and reduction of burden. Some parents reported that their child was able to play and concentrate for longer using the FindMe app than with other toys, even at the end of the 2-month access period. Other parents enjoyed the iPad as a way to sit with their child while mutually focussing on a rewarding activity.

Our null finding emphasises the role of rigorously designed trials in the evaluation of supports for children with autism. Apps have proven popular (Clark et al., 2014) and case-series designs have revealed promising uses of these technologies (e.g. Díez-Juan et al., 2014). Our study does not refute these findings but does indicate that the impact of an app on behaviour may have its limits, and caution should be applied when recommending technology-based supports for this population. In particular, the claims made by commercial operators based on anecdotal reports would not necessarily stand up to scientific scrutiny. New research formats are required to keep pace with technological innovation and provide evidence of value to the community (Fletcher-Watson, 2015).

Future directions

Other technology-based intervention strategies teaching social skills to people with ASD have similarly found limited effects on real-world behaviours (Golan et al., 2009; Hopkins et al., 2011; Swettenham, 1996; Williams et al., 2012). One way to expand the impact of such approaches may be to combine technology with other training (Beaumont and Sofronoff, 2008; Golan and Baron-Cohen, 2006) to deliver a ‘daily dose’ of learning to top up time with a therapist which is necessarily restricted. We are also keen to explore how a child’s individual learning needs could be matched to specific apps and monitored over time, in much the same way that experienced practitioners provide eclectic interpersonal therapies or medication regimes.

During this study, we did not evaluate iPad skills in themselves, though we were struck by how confident children were with the iPad even when their traditional play skills were limited. It is notable that there was no lower non-verbal ability limit for access to our app in this sample. In future, we hope that the skills required to access and manipulate technology will become more highly valued, just as reading is valued as a skill, regardless of the material being read. This is an area where children with ASD may be likely to shine. Cultivation of human–computer interaction can open up independence, social communication and learning in groups otherwise excluded from mainstream activities (Durkin, 2010).

Conclusion

This rigorously designed randomised controlled trial revealed no significant benefits of playing the FindMe app on real-world social communication behaviours. However, there are enough indications of positive impact, considering the broader context, to encourage us to explore this intervention delivery route further. It is possible that targeting social development via technology requires more support to bridge the gap from learning within a game environment to real-world interaction. Technologies could be more effectively applied to learning non-social skills (e.g. Wass et al., 2011) In future, we hope to be able to begin to link specific technologies to personal profiles and to combine technology more effectively with interpersonal therapeutic approaches.