Reliability of Parent Recall of ASD Symptom Onset and Timing

Abstract

Past events are often reported as occurring more recently than they actually took place, an error called forward telescoping. This study examined whether forward telescoping was evident in parent reports of ASD symptom emergence and onset classification. Parents were interviewed when their child was 2–3 years old (Time 1) and approximately 6 years old (Time 2). Significant forward telescoping was found in both age of social regression and age when language milestones were achieved, but not age of language regression. The correspondence between Time 1 and Time 2 onset report was low (kappa=.38). Approximately one-quarter of the sample changed onset categories, most often due to parents not recalling a regression at Time 2 that they had reported at Time 1. These results challenge the use of retrospective methods in determining onset patterns.

The onset of behavioral signs of autism spectrum disorder (ASD) is usually conceptualized as occurring in one of two ways: an early onset pattern, in which children demonstrate social-communication delays early in life, and a regressive pattern, in which children develop typically for some period and then lose previously developed skills. The most common procedure for collecting information about the timing of symptom onset is parent report. It has long been documented that retrospective reports are subject to problems of memory and interpretation and several factors can influence report validity, including awareness of the child’s eventual diagnosis and knowledge of developmental milestones.

One particular type of recall error is forward telescoping (Loftus & Marburger, 1983), in which people report events as having occurred more recently (closer to the time of recollection) than they actually took place. When this error occurs during developmental history-taking, it can result in caregivers reporting that milestones were achieved later than they actually were; if parents are interviewed multiple times, the age at acquisition can become later and later as the child grows older. Multiple explanations have been advanced for the phenomenon of forward telescoping (Pickles et al., 1996): distant events being recalled with less accuracy over time, change in the informant’s conceptualization of the behavior as the child ages, or the nature of the behavior (whether it is a discrete event with clear definitions, like first independent steps, or a gradually unfolding phenomenon like communication). It has also been suggested that forward telescoping results from a drive to achieve consistency between previous and current functioning (Hus et al., 2011). For example, if a child is manifesting clear developmental delays at the time of the interview, there may be a tendency to recall her/him as having always been delayed, resulting in informants systematically (but unconsciously) moving milestone achievement forward in time.

Multiple studies have documented this compression of timescale in parent recall of the timing of developmental milestone acquisition, illnesses, and immunization history (e.g., Majnemer & Rosenblatt, 1994; Pless & Pless, 1995; Suarez et al., 1997), including studies of the early development of children with ASD. Hus et al. (2011) interviewed parents longitudinally when their children were 2, 3, 5, and 9 years of age, using the Autism Diagnostic Interview-Revised (ADI-R; LeCouteur et al., 2003). Telescoping was found in ages of first single words and phrase speech, both of which were reported as occurring significantly later when parents were re-interviewed as the child aged. For example, mean age at first words was reported as 14.9 months when parents were first interviewed when their child was 2 years old, but had advanced to mean ages of 20.2 months at the 5-year interview and 30.2 months when the child was 9. This resulted in significantly more children being identified as speech-delayed from the data collected at ages 5 and 9 than those completed at ages 2 and 3. In contrast, no telescoping was found in age at first walking. In general, children with lower IQ had greater parent report discrepancies (more telescoping) – that is, their reports over time became more consistent with the child’s current function (e.g., delayed development).

One study has also suggested that this phenomenon applies to the reporting of timing of ASD onset; thus, forward telescoping appears to happen not only in the reporting of developmentally appropriate behaviors (e.g., milestone achievement) but also in the reporting of developmentally atypical behaviors (e.g., symptom onset). Lord et al. (2004), in a longitudinal study of language regression, noted that 3 of 18 parents (16.7%) reported word loss when the child was 2–3 years of age, but not when re-interviewed when the child was age 5. These 3 children were verbally fluent by age 5 and had short duration of word loss before stable words reappeared. This suggests that parents of higher functioning children may be less likely to “recall” loss, perhaps because it is inconsistent with current good verbal function. Among the parents who continued to recall word loss, there was significant forward telescoping, with language regression reported at a mean age of 14.9 months at age 2, 17.8 months when re-interviewed when the child was 3, and 19.8 months by age 5. Consistent with this are the findings of a meta-analysis of 85 studies of ASD onset (Barger et al., 2013), in which parents of older children reported later ages of regression onset. Tuchman and Rapin (1997) found that the likelihood of regression being reported was significantly influenced by age of the child at evaluation, with parents of younger children more likely to report regression than parents of older children who were interviewed farther from the time of the event.

The current study conducted longitudinal interviews with parents about onset of ASD symptoms to examine whether and how forward telescoping affects onset classification by parents. It sought to replicate and extend the longitudinal study of Lord et al. (2004), which examined only word loss, and the studies meta-analyzed by Barger et al. (2013), which used cross-sectional data.

METHODS

Initial enrollment at Time 1 (T1) took place when the children were between 2 and 3.9 years of age (M T1 age 35.2 months, SD=5.5 months). All participants entered the study with a community diagnosis; diagnostic confirmation was conducted at the first visit using the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2002) and the ADI-R (LeCouteur et al., 2003). All participants met ADI-R, ADOS, and DSM-IV criteria for Autistic Disorder. Participants were native English speakers, ambulatory, and had no suspected vision or hearing problems, known genetic disorders, or other neurological conditions. The study was approved by the university’s IRB and informed consent was obtained from the parents of each participant.

Participants for the current study included all children who returned for a Time 2 (T2) visit and whose parents completed a second interview about onset status (n=73; 12 females; M age 71.7 months, SD=13.2). The average length between interviews was 36.3 months (SD=12.7 months). Table 1 contains demographic characteristics of the sample.

Table 1.

Demographic characteristics of the sample

	Time 1	Time 2
Age at ADI-R interview in months (M, SD)	35.2 (5.5)	71.7 (13.2)
Verbal DQ/IQ (M, SD)	60.2 (26.1)	78.8 (31.4)
Nonverbal DQ/IQ (M, SD)	75.0 (20.2)	86.4 (26.1)
Overall DQ/IQ (M, SD)	67.6 (21.8)	83.0 (29.7)
ADOS Total (SA + RRB) (M, SD)	18.0 (5.0)	15.2 (7.0)
Father SRS raw score (M, SD)	36.3 (26.9)	–
Mother SRS raw score (M, SD)	28.2 (17.7)	–
Family income (% $50,000 or higher)	69.0%	–
Family affectedness (% multiplex)	9.6%	–
Maternal education (% 4-yr college degree or higher)	45.2%	–

Note: n=73 for all measures except family income (n=42) and maternal education (n=62)

The ADI-R onset questions were used to collect information about symptom onset at both T1 and T2. At T2, interviewers were unaware of T1 responses. At both time points, Question 11 (“Were you ever concerned that XX might have lost language skills during the first years of her/his life?”) was used to define language regression (0=no loss, 1=loss). Question 25 (“Has there ever been a period when XX seemed to get markedly worse or dropped further behind in her/his social engagement, responsiveness, social relatedness, interest or involvement?”) was used to define social regression (0=no loss, 1 or 2=loss). The mother was the ADI-R informant, either by herself or with the father, at both T1 and T2 in 67 of 72 cases (one child had missing informant data). The father was the sole informant at T1 for the other 5 ADI-Rs; in four of these cases, he was also a respondent at T2, either alone or with the mother. Thus, the ADI-R informants were the same at T1 and T2 in all but one instance.

At T1, participants were administered the Mullen Scales of Early Learning (MSEL; Mullen, 1995), which provides T-scores on four subtests (Visual Reception, Fine Motor, Receptive Language, Expressive Language). Updated cognitive assessments were conducted at T2 using the Differential Ability Scales-2^nd edition, which provides Verbal and Nonverbal standard scores and an overall General Conceptual Ability score. In order to create scores that were scaled comparably and permitted comparison across time points, developmental quotients (DQs) were calculated for the MSEL by averaging the relevant subtest age equivalent scores, dividing by chronological age, and multiplying by 100. The ADOS was collected at both T1 and T2. Parents completed the Social Responsiveness Scale (Constantino, 2002) on each other as a measure of parent social-communication ability.

RESULTS

At T1, 41 children (56.2%) had regression reported (4 lost language skills alone, 25 lost social developmental milestones alone, 12 lost both). At T2, the number of parents reporting regression dropped to 32 (43.8). The correspondence between onset classification from T1 to T2 was low, kappa = 0.38, 95% confidence interval 0.23 to 0.52 (see Table 2; Cohen, 1960). Nearly one-quarter of the sample (n=17; 23.2%) changed classification, with the majority (n=13) going from parent reporting of a loss at T1 to not reporting a loss at T2. Thus, parents were less likely to report a loss over time, consistent with cross-sectional data from previous studies (Barger et al., 2013; Tuchman & Rapin, 1997).

Table 2.

Parent report of regression from the ADI-R at Times 1 and 2

Regression at Time 2
Regression at Time 1	Language Only	Social Only	Language + Social	No Loss	Total
Language Only	0	1	2	1	4
Social Only	1	7	6	11	25
Language + Social	1	2	8	1	12
No Loss	1	1	2	28	32
Total	3	11	18	41	73

Note: kappa = 0.38, 95% CI: 0.23 to 0.52

The consistency of reporting was examined next as a function of type of loss. It was more common for a T1 report of social regression to change to “no loss” at T2 (11 of 25 cases, 44%) than for a report of language regression at T1 to be followed by a report of “no loss” at T2 (only 2 of 16 cases or 12.5%). This suggests higher reliability of the report of language than social loss.

Forward telescoping in the reporting of age of milestone acquisitions and losses was also documented (see Table 3). There was significant telescoping in age at loss of social developmental milestones, with social loss reported to begin at 16.6 months in the T1 interview and at 19.1 months by T2 report, t(29)=2.1, p<.05. Telescoping was of even greater magnitude in the reporting of milestone achievements, with mean age of first single words reported as 18.5 months at T1 and 24.7 months at T2, t(71)=3.8, p<.001, and age of first phrases as 28.3 months at T1 and 37.3 months at T2, t(60)=5.7, p<.001. There was no telescoping of age at word loss.

Table 3.

Characteristics of regression in children with language and/or social losses

	Time 1 (M, SD)	Time 2 (M, SD)
Reported age at language loss, in months (T1 n=16; T2 n=22)	19.9 (5.2)	19.5 (6.1)
Reported age at social loss, in months (T1 n=40; T2 n=30)	16.6 (5.5)	19.1 (6.6)
Reported duration of language loss, in months (T1 n=16; T2 n=22)	11.3 (3.9)	14.4 (9.4)
Reported duration of social loss, in months (T1 n=40; T2 n=30)	16.2 (6.0)	20.8 (12.7)
Reported age at first words, in months (T1 n=62; T2 n=72)	18.5 (7.7)	24.7 (13.9)
Reported age at first phrases, in months (T1 n=38; T2 n=61)	28.3 (6.2)	37.3 (12.3)

Note: No significant differences

We examined whether there were differences in child or family variables between those who changed onset classification (loss/no loss, in either direction) from T1 to T2 (n=17) and those who did not (n=56). There were no statistically significant differences in age at time of evaluation, age at loss, duration of loss, length of time between T1 and T2 interviews, verbal, nonverbal, or overall DQ/IQ at either T1 or T2, change in DQ/IQ from T1 to T2, ADOS total score at T1 or T2, change in ADOS score from T1 to T2, simplex/multiplex family status, mother or father SRS score, or maternal level of education (all p’s > .10, uncorrected; see Table 3). We also examined whether there were any differences in these same variables between the children whose parents reported regression at both time periods (n=28) and those for whom regression was reported only at T1 (n=13); there were again no statistically significant differences between the groups (all p’s > .10, uncorrected).

DISCUSSION

This study found a substantial rate of change in onset classification from T1 report, when children were 2–3 years of age, to T2, when participants were approximately 6 years old (mean length of follow-up 3.0 years). The correspondence in onset classification from T1 to T2 was modest (kappa=.38). This is a low reliability statistic for a variable that is often used in neurobiological and genetic analyses. Many studies have examined whether onset types are associated with potential etiologic factors and biological correlates, such as brain growth and function, immunizations, gastrointestinal problems, immune deficits, and genetic variations (Downs et al., 2014; Goin-Kochel et al., 2016; Molloy et al., 2006; Nordahl et al., 2011; Parr et al., 2011; Richler et al., 2006; Valvo et al., 2016; Webb et al., 2007). So far, none of these factors has been reliably associated with onset types. This may be due to the errors that are likely to have occurred in the classifications of onset done in these studies. Clearly, examining the biological correlates of an imprecise measure is problematic and potentially misleading.

We found that change in onset classification was most often due to parents not recalling a regression at Time 2 that they had reported when interviewed earlier at Time 1. This is consistent with the findings of previous cross-sectional studies that parents were more likely to report regression when interviewed when their child was younger and closer to the time of symptom onset (Barger et al., 2013; Tuchman & Rapin, 1997). We did not find any differences in child or family variables between groups defined by change in onset classification over time. Thus, we did not replicate that parents of children with higher verbal or cognitive skills are more likely to “un-endorse” communication or other losses over time or to make their report of early development more consistent with the child’s current functioning (Hus et al., 2011; Lord et al., 2004).

This study also documented forward telescoping in the reporting of age at social regression, as well as age at acquisition of language milestones, replicating earlier work (Hus et al., 2011). Previous studies (Barger et al., 2013; Tuchman & Rapin, 1997) have suggested that under-reporting of regression is most likely in studies done when children are older and farther from age of symptom onset. However, recent investigations using more objective documentation methods (e.g. prospective monitoring or home video analysis) indicate that under-reporting of regression is common at all ages. In one investigation (Ozonoff et al., 2011), classification of onset based on objective coding of home video was compared with onset type as recalled by parents on an ADI-R interview. Less than half of children whose home video displayed clear evidence of a major decline in social-communication behavior were reported to have had a regression by parents. Similarly, only 40% of participants with evidence of early delays in social-communication and little evidence of skill decline on video were reported as having an early onset pattern by parents. There was no difference in the child’s age at time of interview between those for whom parent report was (M=39.4 months) and was not validated by home video (M=38.2 months). Data from prospective studies also suggest that regression is under-reported or under-recognized by parents. Ozonoff et al. (2010) compared onset type based on prospectively observed emergence of symptoms in serial standardized assessments to onset type as recalled retrospectively by parents, when the child was 36 months of age, using the ADI-R. By parental recall, 17% of the children were classified as having regressive onset. By prospective observational data, 86% of the same sample was documented to show significant declines in social-communication behavior over time. These data, collectively, suggest that reporting issues are broader than simple recall difficulties and may reveal a fundamental interpretive difference in how onset phenomena are judged by parents. Questions about the validity of onset report exist at all ages, whether the information is collected when the child is younger (e.g., closer to the timing of the events) or older.

Under-reporting or forward telescoping of regression is consistent with the growing recognition that what constitutes skill loss is broader than initially appreciated. Recent investigations have demonstrated that the phenomenon of regression encompasses more than dramatic, sudden losses of skills and includes smaller, incremental declines in development (Ozonoff et al., 2010; Thurm et al., 2014) that may be challenging for parents to identify. It has been suggested (Pickles et al., 2009) that losses in the social realm, in particular, may be more subtle or less salient to parents than language losses and, thus, less consistently reported over time. The current data support this, with more reliable reports of language regression over time than social regression: only 12% of parents changed their reports of word loss at T1 to no loss at T2, compared to 44% of parents who changed onset classifications from social regression at T1 to no regression at T2. In addition, no telescoping of age at word loss was found, whereas there was significant telescoping in age at social loss. Reporting may be more consistent for language loss because first words are an eagerly awaited milestone with a clear definition and discrete onset, whereas social development unfolds more gradually over time and may not raise the same degree of parental anticipation and excitement. Thus, if a child loses words, it may be far more noticeable to parents than if a child loses social interest.

Several strategies have been proposed to improve reporting (Ayhan & Isiksal, 2005). To minimize comprehension or interpretation problems, it is recommended that further specific information about the behavior in question be provided. ASD screening instruments have begun to incorporate video to improve accuracy (e.g., Marrus et al., 2015; Smith et al., in press) and this strategy could be adapted to improve reporting of onset patterns. For example, longitudinal video of a child experiencing skill loss could be shown to parents to illustrate the kinds of changes in behavior that define regression. To minimize recall problems, the simplest approach, and the one shown to have the best validity, is to ask respondents to consult relevant records prior to completing the interview (Ayhan & Isiksal, 2005). Parents could, for example, review entries in baby books or journals or watch home video of the child, prior to the interview. Another approach is to link reporting to key events in the respondent’s life by creating a detailed timeline and context that assist recall of specific details (Loftus & Marberger, 1983). This method has already been used by Werner et al. (2005) to improve recall of early development in ASD and it could be further adapted for reporting about onset patterns. Whether these methods will enhance the validity of parent report of onset remains to be seen and would be a fruitful area of future study.

In conclusion, the findings of this study call into question the reliability of parent report of onset patterns, particularly regressive patterns or declining trajectories of development. More generally, the data challenge the use of retrospective interview methods to determine symptom onset patterns in children with ASD (see also Barbaresi, 2016; Dawson, 2011). Forward telescoping creates a significant limitation to the interpretation of parent report and suggests caution in using parent report data when the timing of an event is important, such as in research on genetic and non-genetic etiologies of ASD.