Abstract
Background
Adverse childhood experiences (ACEs; e.g., abuse, neglect, parental loss, etc.) have been associated with increased risk for later-life disease and dysfunction using adults’ retrospective self-reports of ACEs. Research should test whether associations between ACEs and health outcomes are the same for prospective and retrospective ACE measures.
Methods
We estimated agreement between ACEs prospectively-recorded throughout childhood (by Study staff at Study member ages 3, 5, 7, 9, 11, 13, and 15) and retrospectively-recalled in adulthood (by Study members when they reached age 38), in the population-representative Dunedin cohort (N=1,037). We related both retrospective and prospective ACE measures to physical, mental, cognitive, and social health at midlife measured through both objective (e.g., biomarkers and neuropsychological tests) and subjective (e.g., self-reported) means.
Results
Dunedin and CDC ACE distributions were similar. Retrospective and prospective measures of adversity showed moderate agreement (r=.47, p<.001; weighted Kappa = .31, 95% CI: .27–.35). Both associated with all midlife outcomes. As compared to prospective ACEs, retrospective ACEs showed stronger associations with life outcomes that were subjectively assessed, and weaker associations with life outcomes that were objectively assessed. Recalled ACEs and poor subjective outcomes were correlated regardless of whether prospectively-recorded ACEs were evident. Individuals who recalled more ACEs than had been prospectively recorded were more neurotic than average, and individuals who recalled fewer ACEs than recorded were more agreeable.
Conclusions
Prospective ACE records confirm associations between childhood adversity and negative life outcomes found previously using retrospective ACE reports. However, more agreeable and neurotic dispositions may respectively bias retrospective ACE measures toward underestimating the impact of adversity on objectively-measured life outcomes and overestimating the impact of adversity on self-reported outcomes. Associations between personality factors and the propensity to recall adversity were extremely modest and warrant further investigation. Risk predictions based on retrospective ACE reports should utilize objective outcome measures. Where objective outcome measurements are difficult to obtain, correction factors may be warranted.
Keywords: Adverse childhood experiences, physical health, mental health, cognitive health, epidemiology
Introduction
In the quest to predict and prevent the development of hard-to-treat and costly later-life diseases, childhood has emerged as a key window of risk determination (Weintraub et al., 2011). In particular, childhood exposures to adverse conditions, including abuse, neglect, and family dysfunction, have been linked to numerous physical diseases and psychological problems (Felitti et al., 1998; Anda et al., 2006; Wilson et al., 2006; Afifi et al., 2008; Benjet, Borges, & Medina-Mora, 2010; Green et al., 2010; Scott et al., 2011; Varese et al., 2012; Solís et al., 2015). These associations are hypothesized to result from alterations in health-risk behaviors (e.g., increased drug use to cope with distress) and/or physiological reactions to chronic stress (Felitti, 2009; Danese & McEwen, 2012). They follow a dose-response relationship; exposure to more adversities forecasts poorer health (Felitti et al., 1998). Public health advocates have called childhood adversity a “hidden health crisis” with “far reaching consequences” (Center for Youth Wellness, 2014, p.1). Such is the concern over the consequences of childhood adversity that many states in the U.S. now monitor for childhood adversity among adults through an Adverse Childhood Experience (ACE) module, provided by the U.S. Center for Disease Control (CDC) in Behavioral Risk Factor Surveillance System surveys (Austin & Herrick, 2014 May).
To date, evidence linking ACEs with adult health comes primarily from studies that measure adults’ recollections of childhood adversity. The validity of such retrospective reports has been questioned because of possible misclassification and bias. On the one hand, adult participants may not be able to retrieve episodic memory from their early years (so called infantile amnesia; Pillemer & White, 1989; Usher & Neisser, 1993) and may fail to correctly retrieve episodic memory from their distant past, particularly at older ages (Hänninen & Soininen, 2012). On the other hand, adult participants may be more or less likely to report childhood adversity based on individual features. For example, they may choose not to divulge intimate information to avoid distress or embarrassment (Hardt & Rutter, 2004). Alternatively, the presence of disease, psychopathology or certain personality styles may unconsciously increase an individual’s propensity to recall childhood adversity –artificially linking childhood experience and adult disease outcomes (Matt, Vazquez, & Campbell, 1992; Henry, Moffitt, Caspi, Langley, & Silva, 1994; McFarland & Buehler, 1998; Prescott et al., 2000; Susser & Widom, 2012; Colman et al., 2016). Although prospective measures of childhood adversity are less sensitive to bias linked to individual features, their validity may nevertheless be limited because of other sources of misclassification including under-reporting by caregivers or under-detection by agencies (Hardt & Rutter, 2004).
The goal of our study was to compare retrospective and prospective measures of ACEs in the prediction of later-life disease and dysfunction. Poor adult health and social outcomes have been associated with prospectively measured ACEs (Solís et al., 2015). However, to our knowledge only one previous comparison of outcome predictions from retrospective and prospective measures of ACEs in the same sample has been undertaken (Patten et al., 2015). Past comparisons of retrospective and prospective reports of child maltreatment raised concern that these two forms of measurement do not match and, further, do not predict outcomes equally (e.g., Widom & Engel, 1996; Widom & Morris, 1997; Horwitz, Widom, McLaughlin, & White, 2001), although at least one study has reported low agreement between measures but similar prediction of mental-health outcomes (Scott, McLaughlin, Smith, & Ellis, 2012). Furthermore, evidence suggests that retrospective reports of ACEs are inconsistent over time, depending on psychological distress at the time of recall (Colman et al., 2016). Here we compare the associations among retrospective and prospective measures of ACEs and physical, cognitive, mental and social health outcomes. Based on previous literature (e.g., Hardt & Rutter, 2004), we predicted that our prospective and retrospective ACE measures would show moderate agreement and that both would associate with later-life outcomes. We conducted our comparison in the Dunedin Study, a population-representative longitudinal birth-cohort born in the early 1970s and followed to early midlife. Prospective ACE counts were generated from dossiers that we compiled for each Study member, which contained information drawn from Study staff assessments and observations, parent and teacher reports, and evidence of social service contacts collected at Study member ages 3, 5, 7, 9, 11, 13, and 15). Retrospective ACE counts were generated from Study member recollections of childhood adversity reported in adulthood.
In addition to potential discrepancies between retrospectively-recalled and prospectively-recorded ACEs, we anticipated differences in associations of ACE measures with outcomes that are objectively measured as compared to those that are measured subjectively (i.e., through self-report). Health-psychology research has documented that self-reports tend to be suffused with biases stemming from reporters’ personality styles, like neuroticism, while objective measures are not (Watson & Pennebaker, 1989). We therefore tested both objective and subjective outcomes measurements.
Finally, because personality styles may also influence recall of ACEs, we tested if reporters’ personality styles were associated with discrepant retrospectively-recalled and prospectively-recorded ACE exposures.
Methods
Sample
Participants are members of the Dunedin Study, a longitudinal investigation of health and behavior in a representative birth cohort. Study members (N=1,037; 91% of eligible births; 52% male) were all of the individuals born between April 1972 and March 1973 in Dunedin, New Zealand (NZ), who were eligible based on residence in the province and who participated in the first assessment at age 3. The cohort represented the full range of socioeconomic status (SES) in the general population of New Zealand’s South Island. On adult health, the cohort matches the New Zealand National Health and Nutrition Survey on key health indicators (e.g. body mass index, smoking, visits to the doctor; Poulton et al. 2015). The cohort is primarily white; fewer than 7% self-identify as having non-Caucasian ancestry, matching the demographics of the South Island (Poulton et al., 2015). Assessments were carried out at birth and ages 3, 5, 7, 9, 11, 13, 15, 18, 21, 26, 32, and, most recently, 38 years, when 95% of the 1,007 study members still alive took part. In the interest of reproducibility, the analysis plan for this paper was posted in advance (http://www.moffittcaspi.com; Trinh & Sun, 2013). Study member informed consent was obtained, with study protocol approval by the institutional ethical review boards of the participating universities.
Measures
Adverse childhood experiences (ACEs)
The U.S. Centers for Disease Control & Prevention (CDC) have articulated a leading approach to conceptualizing ACEs (Felitti et al., 1998). Our measure of ACEs corresponds to the 10 categories of childhood adversity introduced by the CDC Adverse Childhood Experiences Study (Felitti et al., 1998; http://www.cdc.gov/violenceprevention/acestudy/prevalence.html): Five types of child harm (physical abuse, emotional abuse, physical neglect, emotional neglect and sexual abuse) and five types of household dysfunction (incarceration of a family member, household substance abuse, household mental illness, loss of a parent, and household partner violence). Because the Dunedin Study began in the early-1970s and the awareness of ACEs in the health sciences dates to the mid-1990s, Dunedin Study operational definitions of retrospective and prospective ACEs were necessarily somewhat different.
Retrospective ACE counts
The ACE Study collects retrospectively recalled ACEs via a self-report questionnaire (http://www.cdc.gov/violenceprevention/acestudy/prevalence.html). Our retrospective ACEs measure draws on structured interviews conducted when Dunedin Study participants were adults. Like the CDC ACE Study, we administered the Childhood Trauma Questionnaire (CTQ) (Bernstein & Fink, 1998), which ascertains physical, sexual and emotional abuse, physical neglect, and emotional neglect; the CTQ was administered at age 38. Following the CTQ manual a specific category of harm was present if the Study member had a moderate to severe score. Study members were also interviewed about memories of exposure to family substance abuse, mental illness, and incarceration during childhood via the Family History Screen (Milne et al., 2009). Exposure to partner violence was assessed by asking Study participants, “Did you ever see or hear about your mother/father being hit or hurt by your father/mother/stepfather/stepmother?” We also interviewed participants about parental loss (due to separation, divorce, death, or removal from home).
Prospective ACE counts
Prospective ACE counts were generated from archival Dunedin Study records gathered during 7 biennial assessments carried out from ages 3 to 15 years. The records include: social service contacts; structured notes from assessment staff who interviewed Study children and their parents; structured notes from pediatricians and psychometricians who observed mother-child interactions at the research unit; structured notes from nurses who recorded conditions witnessed at home visits; and notes of concern from teachers who were surveyed about the Study children’s behavior and performance. Separately, parental criminality was surveyed via postal questionnaire to the parents. Attrition analysis found no significant difference in exposure to ACEs between those individuals who completed the Study assessment at age 38 and those who did not (X2 (4, N = 1034) = 7.36, p =.12). Prospective ACEs data were missing for only 3 of the 1037 cohort members.
Archival Study data were reviewed in 2015 by four independent raters who were trained on the CDC definitions of ACEs. Individual ACEs were agreed upon by at least three of the four raters 80% of time. The sole exception was emotional neglect where half the cases were identified by only two raters. Agreement across the full ACE count between the four raters ranged from kappa = .76 to .82, with an average inter-rater agreement kappa of .79.
The completeness of archival Dunedin Study records of adversity varied by the type of ACE considered. Some ACEs (notably childhood sexual abuse) will have been under-detected to the extent that these experiences were not actively queried, reflecting assumptions in the 1970’s that sexual abuse was exceedingly rare (Jenny, 2008). To ensure that potential under-detection in any ACE category did not bias the results of our analyses, we repeated the full suite of tests used in this study with each type of ACE iteratively removed from the total ACE count. As presented in the Results section, these “leave-one-out” analyses produced no significant changes to the results.
Prevalence of retrospective and prospective ACEs in the Dunedin cohort
Each ACE type was coded as present (=1) or not, with a theoretical maximum of 10 ACEs. (Following the CDC ACE Study, scores were coded 0, 1, 2, 3, or 4+ for all analyses.) Figure 1A shows a similar, zero-inflated, distribution of ACEs in the Dunedin study whether ACEs were gathered retrospectively or prospectively. The Figure also documents that the Dunedin ACE distribution resembled that of the CDC ACE Study. According to retrospective reports our cohort experienced more ACEs than they did according to our prospective records. Figure 1B shows that prospective rates were lower than retrospective rates on many, though not all, types of adversity.
Adult health and social outcomes
We assessed four outcome domains: physical, cognitive, mental, and social health. In every domain, where possible, each outcome was measured both objectively and subjectively. Table 1 describes the outcome measures, which have been previously published in the Dunedin study.
Table 1.
Subjective | Objective | |
---|---|---|
Domains | ||
Physical Health | Self-rated poor health | Biomarker-indexed poor health |
Self-rated poor health was measured at age 38 by a 5-point scale in response to the question: “In general, would you say your health is?” Response options were “poor,” “fair,” “good,” “very good” or “excellent”. (Idler & Benyamini, 1997). | Biomarker-indexed poor health is an objective measure of physical health taken by summing nine indicators of physical health measured at age 38 including metabolic abnormalities (waist circumference, high-density lipoprotein level, triglyceride level, blood pressure, and glycated hemoglobin), cardiorespiratory fitness, pulmonary function, periodontal disease, and systemic inflammation. Details are provided in Israel et al. (2014). | |
Cognitive Health | Complaints of cognitive impairment | Working memory performance on the WAIS-IV |
Study members reported at age 38 how often in the past year (never, sometimes, or often) they experienced problems with, e.g., keeping track of appointments, remembering why they went to a store, repeating the same story to someone, multi-tasking, thinking when the TV or radio is on, word-finding difficulty, among other items based on symptom criteria for DSM-IV Mild Neurocognitive Disorder. Scores on the 19 questions were summed (score range = 0 to 31; mean (SD) = 9.1(5.3); internal consistency reliability = 0.83). The complaints score was converted to a Z-score, mean = 0, SD = 1 (Moffitt et al., 2015). | Working memory performance was assessed at age 38 through the Working Memory Index of the Wechsler Adult Intelligence Scale –IV (WAIS-IV) (Wechsler, 2008). | |
Mental Health | p-factor of General Psychopathology | |
Our measure of poor mental health is a general factor of psychopathology, the p-factor, derived from confirmatory factor analysis of symptom-level psychopathology data collected in the Study population between ages 18 and 38. Every 2 to 6 years, Study members were interviewed about past-year symptoms of DSM-defined disorders by trained non-lay interviewers. Details on the p-factor are provided in Caspi et al. (2014). | Our study includes no objective measures of mental health as there are no lab tests for mental disorders. | |
Social Health | Poor Partner Relationship Quality | |
Poor partner relationship quality was assessed at age 38 through a 28-item survey about shared activities and interests, balance of power, respect and fairness, emotional intimacy and trust, and open communication (α = .93) (Cerdá et al., 2016). | Our study includes no objective measure of partner relationship quality as this is generally measured only through self or partner/informant reports. Given the late onset of marriage and high rates of de-facto relationships, divorce is not a useful objective indicator of poor relationship. |
Potentially biasing Big Five personality factors
The Big Five were assessed at age 38 via informants (Israel et al., 2014). Study members nominated someone who knew them well; most were best friends, partners, or other family members, with a 97% response rate. These “informants” were mailed questionnaires asking them to describe the Study member using a brief version of the Big Five Inventory (Benet-Martínez & John, 1998), which assesses individual differences in: Extraversion (α = 0.79), Agreeableness (α = 0.75), Neuroticism (α = 0.83), Conscientiousness (α = 0.81), and Openness to Experience (α = 0.85).
Results
Do retrospective and prospective ACE measures agree?
Table 2 presents the correlation and agreement (Cohen’s Kappa) coefficients between ACE scores measured retrospectively and prospectively. At the item level, agreement between retrospectively-recalled and prospectively-recorded adversities ranged from excellent (loss of parent) to poor (emotional abuse). At the scale level of total ACE count, the correlation between retrospective and prospective ACE scores was r=.47, p<.001, a moderate effect size. Precise agreement between the number of adverse experiences retrospectively-recalled and prospectively-recorded was fair (weighted Kappa = .31, 95% CI: .27 – .35). Supplementary Table S1 shows the percentage of prospectively-measured ACEs that were retrospectively recalled and the percentage of retrospective ACEs that were prospectively-recorded. Further analyses showed the overall level of agreement between retrospective and prospective reports was dependent, in part, on the high level of agreement about parental loss. Agreement between retrospective and prospective reports was lower when parental loss was not a part of the ACE measure (Supplementary Table S2).
Table 2.
ACE | Correlation | Agreement |
---|---|---|
| ||
Pearson’s r | Weighted Kappa | |
Child Harm | ||
Physical abuse | .07* | .07 |
Emotional abuse | .03 | .02 |
Physical neglect | .14*** | .13 |
Emotional neglect | .07* | .07 |
Sexual abuse | .10** | .03 |
Household Dysfunction | ||
Family member incarceration | .16*** | .14 |
Household substance abuse | .22*** | .16 |
Household mental illness | .16*** | .15 |
Loss of parent | .83*** | .83 |
Household partner violence | .11** | .10 |
Total ACE count | .47*** | .31 |
Notes.
p < .05,
p < .01,
p < .001.
Do retrospective and prospective ACE measures predict later-life health and social outcomes?
Despite only moderate agreement between retrospective and prospective ACE measures, both were associated with later-life outcomes (Table 3). Effect-sizes for associations between prospective ACEs and adult outcomes were small and relatively uniform (from r=.11 to r=.23, Table 3 Column One). In contrast, effect sizes for associations between retrospective ACEs and outcomes were more variable (Table 3 Column Three). Retrospective ACE associations with objectively-measured outcomes had smaller effect-sizes as compared to associations with subjectively-measured outcomes (e.g., r=.07 for biomarker-indexed poor physical health versus r=.18 for self-reported poor physical health). The largest effect-sizes were for associations between retrospective ACEs and outcomes of a more psychological nature (mental and social health; e.g., r=.40 for psychopathology).
Table 3. Associations of prospectively-recorded and retrospectively-reported counts of adverse childhood experiences (ACEs) with physical, cognitive, mental, and social health outcomes by age 38.
|
|
||||
---|---|---|---|---|---|
Prospective ACE counts | Retrospective ACE counts | ||||
Unadjusted | Adjusted for Retrospective ACE Counts | Unadjusted | Adjusted for Prospective ACE Counts | ||
| |||||
N | Effect size (Pearson’s r) | Effect size (Pearson’s r) | |||
Self-reported Outcomes | |||||
Physical Health | |||||
Self-reported Poor Health | 950 | .13*** (.07, .20) | .05 (−.02, .12) | .18*** (.12, .24) | .16*** (.09, .23) |
Cognitive Health | |||||
Self-reported Poor Memory | 946 | .11** (.05, .17) | .06 (−.01, .13) | .13*** (.06, .19) | .10** (.03, .17) |
Mental Health | |||||
Self-reported Psychopathology (clinical interview) | 950 | .23*** (.16, .28) | .05 (−.02, .12) | .40*** (.34, .46) | .38*** (.31, .44) |
Social Health | |||||
Self-reported Poor Relationship Quality1 | 844 | .11** (.04, .18) | .01 (−.07, .09) | .21*** (.14, .27) | .20*** (.13, .28) |
Objectively-Measured Outcomes | |||||
Physical Health | |||||
Biomarker-derived Poor Health | 920 | .11** (.04, .17) | .09* (.02, .16) | .08* (.02, .15) | .04 (−.03, .11) |
Cognitive Health | |||||
Poor Tested Working Memory | 938 | .15*** (.09, .22) | .15*** (.08, .23) | .07* (.01, .13) | .00 (−.07, .07) |
Notes.
p < .05,
p < .01,
p < .001.
All analyses were conducted controlling for sex. 95% confidence intervals for the effects are reported in parentheses. Effect sizes may be compared at http://quantpsy.org/corrtest/corrtest2.htm.
Relationship quality was not obtained for 105 Study members who had no partner within the past year.
Do beliefs about childhood adversity predict outcomes regardless of what adversities were prospectively recorded?
We used multivariate linear regressions to test associations between retrospective ACE counts and adult outcomes while controlling for prospective ACE counts (Table 3). For subjectively-measured outcomes, retrospective ACEs remained a statistically-significant predictor even after accounting for prospective ACEs. In contrast, retrospective ACE associations with objectively-measured outcomes dropped to non-significance after adding controls for prospective ACEs (Table 3, Column 4). This pattern of results suggested that, regardless of what prospective childhood records indicated, individuals’ beliefs that they experienced adversity appeared to be strongly related to their appraisals of their current life outcomes. The belief that one has experienced childhood adversity did not, however, necessarily relate to outcomes that were objectively measured once prospective adversities were taken into account.
Do prospectively-recorded adversities predict outcomes regardless of beliefs about childhood adversity?
We used multivariate linear regressions to test associations between prospective ACE counts and adult outcomes while controlling for retrospective ACE counts (Table 3). As column two in Table 3 shows, prospective ACE associations with all subjectively-measured outcomes dropped to non-significance after controlling for retrospective ACE counts. In contrast, prospective ACEs remained a statistically-significant predictor of objectively-measured outcomes after adding controls for retrospective counts. This pattern of results suggested two things. First, greater adversity in childhood was followed by poorer mid-life outcomes (e.g., poorer physical and cognitive health) regardless of whether or not the adversity was remembered. Second, individuals who did not recall their prospectively-recorded adversities when interviewed as adults tended not to make negative appraisals of their life outcomes.
Were findings biased by potentially missing prospective ACEs?
As noted earlier, sexual abuse may have been under-recorded in the prospective ACE data. Sexual abuse is thought to be especially harmful. To evaluate whether this under-detection could have biased associations between prospective ACEs and adult outcomes, we repeated our analyses with sexual abuse removed from the count of total retrospective and prospective ACEs. If false negatives for sexual abuse biased the prospective ACE associations with outcomes, the strength of outcome-associations for the two ACE measures should become more similar after removing sexual abuse from total ACE counts. We then also iteratively removed each additional ACE type from our total count in turn and re-ran the analyses. These leave-one-out tests did not change the results, suggesting that the overall results were unlikely to have been biased by misclassification in any of the ACEs components. Results of leave-one-out tests are reported in the Supplementary Figure S1.
Are personality factors linked to ACE reports?
Our analysis suggested that ACE associations with adult outcomes depended on what was remembered from childhood in the case of some outcomes but not others. For objectively-measured adult outcomes (e.g. health measured using biomarker indices), prospectively-recorded childhood adversity that was not recalled by participants in adulthood nevertheless predicted poor adult outcomes. In contrast, prospectively-recorded adversities that were not recalled in adulthood were unrelated to adult outcomes measured by subjective self-reports. This suggested that self-reports of adult outcomes could be biased by some individuals taking an overly positive view of their childhood and adulthood. Further, adversity that was recalled but not prospectively-recorded predicted self-reports of poor health and memory problems that were not confirmed by objective outcome tests. This suggested that self-reports of adult outcomes could also be colored by some individuals taking an overly negative view of their childhood and adulthood.
We next tested for the potential influence of personality factors on adult ACE recall and an individual’s potential discrepancy between prospective and retrospective ACE counts. To quantify this discrepancy, we subtracted each participant’s prospective ACE exposure score from their retrospective ACE score to create a measure of directional divergence ranging from −10 to +10. Results showed that three personality traits were significantly linked to the discrepancy between retrospective and prospective data: Neuroticism (r=.10, p=.004), Conscientiousness (r=−.07, p<.05), and Agreeableness (r=−.09, p<.01). As illustrated in Figure 2, individuals scoring high on Neuroticism or low on Conscientiousness were likely to have recalled more ACEs than were prospectively recorded and individuals scoring high on Agreeableness were likely to have recalled fewer ACEs than were prospectively recorded.
Discussion
Our longitudinal analysis of the association between ACEs and adult health outcomes revealed five findings. First, we replicated the association between retrospective ACEs and adult health outcomes reported in previous studies. In fact, when we compare our effect sizes in Table 3 to effect sizes reported in Felitti et al.’s (1998) Table 8 (p.254–255) we find that they are very similar. For example, in individuals who recalled four or more ACEs, Felitti et al. reported an increased risk of poor self-rated health on the order of d =.44 where we report d=.49. Second, consistent with observations from previous analyses of the agreement between prospective and retrospective measures of specific childhood adversities (Hardt & Rutter, 2004; Patten et al., 2015), we found that prospective and retrospective ACE measures in the Dunedin cohort agreed only modestly. Such modest agreement may raise eyebrows, but it should be interpreted in context. For example, it is common in the behavioral sciences to observe modest levels of agreement between different reporters (e.g., mother and teacher reports of child ADHD symptoms typically correlate around .30; Wolraich et al., 2004). Rather than suggesting that one reporter’s information is invalid, this finding is interpreted to reflect that different reporters have access to complementary sources of information. By extension, it is possible that retrospective and prospective ACE measures may share something in common and also contain unique information. Modest agreement here may thus suggest that retrospective ACE measures be viewed as complementary to prospective measures, rather than as potentially invalid. Third, notwithstanding low agreement with the retrospective measures, both retrospective and prospective ACEs predicted adult outcomes. Fourth, retrospective ACE measures, in comparison to prospective measures, more strongly predicted adult outcomes when outcomes were measured subjectively (through self-report). Fifth, prospective ACE measures, in comparison to retrospective measures, more strongly predicted adult outcomes when outcomes were measured objectively (through tests and biomarkers).
These results suggest that, relative to prospective ACE measures, retrospective ACE measures underestimate the influence of childhood adversity on “objective” adult outcomes and overestimate the influence of childhood adversity on “subjective” outcomes. This is consistent with a recent review of the limitations of retrospective recall of childhood adversity, which noted that adult psychopathology is more strongly associated with retrospective self-reports of childhood abuse than with official records of abuse (Susser & Widom, 2012). The authors concluded that, “the most plausible interpretation [for this phenomenon] is that people who have more problems in adulthood look back on childhood and report more problems” (p.674). Our data support this conclusion across domains as diverse as physical, cognitive, mental and social health but also suggest that the opposite is true: people who do not recall problems in childhood also do not see themselves as having problems in adulthood. In our study, adult individuals with documented childhood adversity that they did not recall had objectively poorer physical health and cognitive ability but surprisingly were unlikely to self-rate their health or memory as poor. Thus, strong associations between recalled adversity and poor self-rated health may result both from individuals who over-recall ACEs and under-rate health and from individuals who under-report ACEs and over-rate health. This suggests caution for interpretation; for domains where self-reports are the only means of measurement and no “objective” diagnostic tests are available, a downward correction factor may be desirable for interpreting health risks based on adult recollections of childhood adversity.
There is precedent for this finding. Past studies have found that negative mood or existing psychological dysfunction at the time of recall results in negative recollection bias (Dalgleish & Watts, 1990; Brewin, Andrews, & Gotlib, 1993; Brewin, Reynolds, & Tata, 1999). But “healthy” individuals may create biases of their own. In a comparison of clinical records with adult recollection of adversity, Robins et al. (1985) found that healthy adults were “more likely to deny adverse information about early home life” than those with mental illness (p.31). The authors suggested that “those who are without current problems tend to forget and forgive.” In a review of similar findings, Hardt and Rutter (2004) concluded that recall bias can sometimes show “a tendency for people with good functioning in adult life to forget early parental negativity” (p.267). Colman et al. (2016) asked 7,466 young adults to report on childhood adversity on two occasions separated by a twelve-year delay. They found that most participants forgot some episodes of adversity that they had originally recalled –but those with the most psychological distress at the time of second testing “forgot” the least. Perhaps “forgetting and forgiving” can be both willful and adaptive (Anderson & Levy, 2009; Nørby, 2015).
Our data suggest that trait-level personality may influence the process of “forgetting and forgiving.” For example, we found that more neurotic individuals tend to recall more ACEs than their prospective records indicate and that more agreeable individuals tend to recall fewer ACEs than their prospective records indicate. Further, we found that an individual’s recollection of their youth relates to their self-appraisals but not necessarily to their actual performance on objective tests once prospective ACE records are taken into account. Taken together these findings suggest that “grey” and “sunny” dispositions could be biasing both memories of childhood and self-appraisals of adulthood in a manner that exaggerates the predictive capacity of retrospective ACE measures. Further research on this phenomenon is warranted, however, as our analysis on this point was only suggestive. First, the effect sizes for personality associations with divergent ACE counts in our study were very small. Second, our design, which measured Study member personality factors after potential exposure to ACEs, did not allow for the evaluation of cause-and-effect in the relationship between childhood adversity and adult personality (e.g., we are unable to disentangle the extent to which negative emotionality in adulthood is caused by childhood adversity or merely relates to the recall of adversity).
An additional contribution of our study is the finding that the imprint of childhood adversity maintains even when adversity is not recalled. We found that prospectively-recorded adversity that an individual does not recall still results in demonstrable detriments to their physical and cognitive health. Critically, these detriments were only detected through “objective” tests that do not rely on self-evaluation of abilities or self-awareness of poor functioning.
We acknowledge limitations. First, we only examined ACEs as defined in the original ACE study (Felitti et al., 1998), and not “Expanded ACEs” (Cronholm et al., 2015; Wade et al., 2015). Second, our aggregation of ACE events across the first 15 years of life precluded testing the influence of developmental timing or duration of adversity. Third, our findings may not be representative of the recall reliability of elderly populations, who often evidence poor memory (Hänninen & Soininen, 2012) and positive-event recall bias (Ros & Latorre, 2010). Finally, it is possible that our prospective ACE measures under-detected events of child harm, particularly sexual abuse. Our comparison of prospective and retrospective ACEs was not, therefore, a perfect one, because the mapping of the retrospective and prospective assessment was not exact. Yet, as far as we are aware, it was as complete as any other yet reported. Importantly, evidence suggests that most prospective records of childhood sexual abuse, including official court records, tend to under-detect events (Hardt & Rutter, 2004) and could benefit from the addition of retrospective measures (Shaffer, Huston, & Egeland, 2008).
Conclusion
Notwithstanding its limitations, our study holds several implications. Experienced adversity, whether it is recalled or not, increases risk for poor physical and cognitive health at midlife. Psychological resilience to adversity thus may not confer physiological resilience (Seery, 2011; Whitson et al., 2016), and primary care clinics using retrospective ACE reports to screen for high-risk patients may overlook individuals at-risk from ACEs that they experienced but do not remember. Meanwhile, the strong association between recalled adversity and self-rated health and social outcomes suggests a bias of both under- and over-reporting. Future studies on the influence of recalled childhood adversity on later-life outcomes should take pains to include outcome measures that are obtained objectively (e.g., Juonala et al., 2016). Sometimes objective measures are not available. This is especially true in the case of mental health. We would not advise clinicians to respond to reports of childhood adversity or perceived current malaise with suspicion –or for researchers to dismiss such valuable data. But neither would we advocate for uncritical acceptance of recollections of childhood as having purely causal implications for health.
The conceptualization of childhood adversity as a risk factor for later disability represents a powerful tool for identifying individuals at risk and, potentially, for developing targeted interventions (Garg & Dworkin, 2016). Our study confirms that the maltreatment and deprivation of the child holds implications for the adult. However, an individual’s perception of the past and present, accurate or not, plays a role as well, potentially influencing both prophesies of risk and their fulfillment.
Supplementary Material
Key points.
What’s known
Adverse childhood experiences (ACEs) have been associated with increased risk for later-life disease/dysfunction using adults’ retrospective self-reports of ACEs.
What’s new
We replicate past findings using both retrospective and prospective measures of ACEs to predict poor adult outcomes in a longitudinal birth-cohort.
In comparison to prospective ACE measures, retrospective ACEs showed stronger associations with outcomes that were subjectively assessed (i.e., through self-report) and weaker associations with outcomes that were objectively measured.
Beliefs about childhood adversity predicted subjective outcomes regardless of whether adversities were prospectively recorded.
“Forgotten” adversities still predicted objective outcomes.
What’s clinically relevant
“Sunny” and “grey” dispositions may bias predictions from retrospective ACE measures toward underestimating ACE impacts on objectively-measured outcomes and overestimating impacts on subjectively-measured outcomes.
Predictions of health risk based on retrospective ACE reports should utilize objective outcome measures; when these are difficult to obtain, correction factors may be warranted.
Acknowledgments
We thank the Dunedin Study members, their parents, teachers, and peer informants, Unit research staff, and Study founder Phil Silva. The Dunedin Multidisciplinary Health and Development Research Unit is supported by the New Zealand Health Research Council and New Zealand Ministry of Business, Innovation and Employment (MBIE). This research received support from US-National Institute of Aging grants R01AG032282, R01AG049789, R01AG048895, the U.K. Medical Research Council grant MR/K00381X, the Economic and Social Research Council grant ES/M010309/1, and the Jacobs Foundation.
References
- Afifi TO, Enns MW, Cox BJ, Asmundson GJ, Stein MB, Sareen J. Population attributable fractions of psychiatric disorders and suicide ideation and attempts associated with adverse childhood experiences. American Journal of Public Health. 2008;98:946–952. doi: 10.2105/AJPH.2007.120253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Anda RF, Felitti VJ, Bremner JD, Walker JD, Whitfield C, Perry BD, … Giles WH. The enduring effects of abuse and related adverse experiences in childhood. A convergence of evidence from neurobiology and epidemiology. European Archives of Psychiatry and Clinical Neuroscience. 2006;256(3):174–186. doi: 10.1007/s00406-005-0624-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Anderson MC, Levy BJ. Suppressing unwanted memories. Current Directions in Psychological Science. 2009;18(4):189–194. doi: 10.1177/0963721417689881. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Austin AE, Herrick HWB. North Carolina State Center for Health Statistics Study No. 167. 2014. May, The effect of Adverse Childhood Experiences on adult health: 2012 North Carolina Behavioral Risk Factor Surveillance System Survey; pp. 1–15. [Google Scholar]
- Benet-Martínez V, John OP. Los Cinco Grandes across cultures and ethnic groups: Multitrait-multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology. 1998;75:729–750. doi: 10.1037//0022-3514.75.3.729. [DOI] [PubMed] [Google Scholar]
- Benjet C, Borges G, Medina-Mora ME. Chronic childhood adversity and onset of psychopathology during three life stages: Childhood, adolescence and adulthood. Journal of Psychiatric Research. 2010;44(11):732–740. doi: 10.1016/j.jpsychires.2010.01.004. [DOI] [PubMed] [Google Scholar]
- Bernstein DP, Fink L, Handelsman L, Foote J, Lovejoy M, Wenzel K, Sapareto E, Ruggiero J. Initial reliability and validity of a new retrospective measure of child abuse and neglect. American Journal of Psychiatry. 1994;151:1132–1136. doi: 10.1176/ajp.151.8.1132. [DOI] [PubMed] [Google Scholar]
- Bernstein DP, Fink L. Childhood Trauma Questionnaire: A retrospective self-report. San Antonio, TX: The Psychological Corporation; 1998. [Google Scholar]
- Brewin CR, Andrews B, Gotlib IH. Psychopathology and early experience: a reappraisal of retrospective reports. Psychological Bulletin. 1993;113:82–98. doi: 10.1037/0033-2909.113.1.82. [DOI] [PubMed] [Google Scholar]
- Brewin CR, Reynolds M, Tata P. Autobiographical memory processes and the course of depression. Journal of Abnormal Psychology. 1999;108:511–517. doi: 10.1037//0021-843x.108.3.511. [DOI] [PubMed] [Google Scholar]
- Caspi A, Houts RM, Belsky DW, Goldman-Mellor SJ, Harrington H, Israel S, … Moffitt TE. The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science. 2014;2(2):119–137. doi: 10.1177/2167702613497473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Center for Youth Wellness. Data Report. San Francisco, CA: The Center for Youth Wellness; 2014. A hidden crisis: Findings on Adverse Childhood Experiences in California. [Google Scholar]
- Cerdá M, Moffat TE, Meier MH, Harrington H, Houts R, Ramrakha S, Hogan S, Poulton R, Caspi A. Persistent cannabis dependence and alcohol dependence represent risks for midlife economic and social problems: A longitudinal cohort study. Clinical Psychological Science. 2016:E1–19. doi: 10.1177/2167702616630958. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Colman I, Kingsbury M, Garad Y, Zeng Y, Naicker K, Patten S, … Thompson AH. Consistency in adult reporting of adverse childhood experiences. Psychological Medicine. 2016;46(3):543–549. doi: 10.1017/S0033291715002032. [DOI] [PubMed] [Google Scholar]
- Cronholm PF, Forke CM, Wade R, Bair-Merritt MH, Davis M, Harkins-Schwarz M, … Fein JA. Adverse Childhood Experiences: Expanding the concept of adversity. American Journal of Preventive Medicine. 2015;49(3):354–361. doi: 10.1016/j.amepre.2015.02.001. [DOI] [PubMed] [Google Scholar]
- Dalgleish T, Watts FN. Biases of attention and memory in disorders of anxiety and depression. Clinical Psychology Review. 1990;10:589–604. [Google Scholar]
- Danese A, McEwen BS. Adverse childhood experiences, allostasis, allostatic load, and age-related disease. Physiology & Behavior. 2012;106(1):29–39. doi: 10.1016/j.physbeh.2011.08.019. [DOI] [PubMed] [Google Scholar]
- Felitti VJ, Anda RF, Nordenberg D, Williamson DF, Spitz AM, Valerie E, Koss MP, Marks JS. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The Adverse Childhood Experiences (ACE) Study. American Journal of Preventative Medicine. 1998;14(4):245–258. doi: 10.1016/s0749-3797(98)00017-8. [DOI] [PubMed] [Google Scholar]
- Felitti VJ. Adverse Childhood Experiences and adult health. Academic Pediatrics. 2009;9(3):131–132. doi: 10.1016/j.acap.2009.03.001. [DOI] [PubMed] [Google Scholar]
- Garg A, Dworkin PH. Surveillance and screening for social determinants of health: The medical home and beyond. JAMA Pediatrics. 2016;170(3):189–190. doi: 10.1001/jamapediatrics.2015.3269. [DOI] [PubMed] [Google Scholar]
- Green J, McLaughlin KA, Berglund PA, Gruber MJ, Sampson NA, Zaslavsky AM, Kessler RC. Childhood adversities and adult psychiatric disorders in the national comorbidity survey replication: Associations with first onset of DSM-IV disorders. Archives of General Psychiatry. 2010;67(2):113–123. doi: 10.1001/archgenpsychiatry.2009.186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hänninen DT, Soininen H. Age-associated memory impairment. Drugs & Aging. 2012;11(6):480–489. doi: 10.2165/00002512-199711060-00007. [DOI] [PubMed] [Google Scholar]
- Hardt J, Rutter M. Validity of adult retrospective reports of adverse childhood experiences: Review of the evidence. Journal of Child Psychology and Psychiatry. 2004;45(2):260–273. doi: 10.1111/j.1469-7610.2004.00218.x. [DOI] [PubMed] [Google Scholar]
- Henry B, Moffitt TE, Caspi A, Langley J, Silva PA. On the “remembrance of things past”: A longitudinal evaluation of the retrospective method. Psychological Assessment. 1994;6(2):92–101. http://doi.org/10.1037/1040-3590.6.2.92. [Google Scholar]
- Horwitz AV, Widom CS, McLaughlin J, White HR. The impact of childhood abuse and neglect on adult mental health: A prospective study. Journal of Health and Social Behavior. 2001;42:184–201. [PubMed] [Google Scholar]
- Idler EL, Benyamini Y. Self-rated health and mortality: A review of twenty-seven community studies. Journal of Health and Social Behavior. 1997;38(1):21–37. http://doi.org/10.2307/2955359. [PubMed] [Google Scholar]
- Israel S, Moffitt TE, Belsky DW, Hancox RJ, Poulton R, Roberts B, … Caspi A. Translating personality psychology to help personalize preventive medicine for young adult patients. Journal of Personality and Social Psychology. 2014;106(3):484–498. doi: 10.1037/a0035687. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jenny C. Medicine discovers child abuse. Journal of the American Medical Association. 2008;300(23):2796–2797. doi: 10.1001/jama.2008.842. [DOI] [PubMed] [Google Scholar]
- Juonala M, Pulkki-Råback L, Elovainio M, Hakulinen C, Magnussen CG, Sabin MA, … Raitakari OT. Childhood psychosocial factors and coronary artery calcification in adulthood: The Cardiovascular Risk in Young Finns Study. JAMA Pediatrics. 2016:E1–7. doi: 10.1001/jamapediatrics.2015.4121. [DOI] [PubMed] [Google Scholar]
- Matt GE, Vazquez C, Campbell WK. Mood congruent recall of affectively toned stimuli: A meta-analytic review. Clinical Psychology Review. 1992;12:227–255. [Google Scholar]
- McFarland C, Buehler R. The impact of negative affect on autobiographical memory: The role of self-focused attention to moods. Journal of Personality and Social Psychology. 1998;75:1424–1440. doi: 10.1037//0022-3514.75.6.1424. [DOI] [PubMed] [Google Scholar]
- Milne BJ, Caspi A, Crump R, Poulton R, Rutter M, Sears MR, Moffitt TE. The validity of the family history screen for assessing family history of mental disorders. American Journal of Medical Genetics. Part B, Neuropsychiatric Genetics: The Official Publication of the International Society of Psychiatric Genetics. 2009;0(1):41–49. doi: 10.1002/ajmg.b.30764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moffitt TE, Caspi A, Taylor A, Kokaua J, Milne BJ, Polanczyk G, Poulton R. How common are common mental disorders? Evidence that lifetime prevalence rates are doubled by prospective versus retrospective ascertainment. Psychological Medicine. 2010;40:899–909. doi: 10.1017/S0033291709991036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moffitt TE, Houts R, Asherson P, Belsky DW, Corcoran DL, Hammerle M, … Caspi A. Is adult ADHD a childhood-onset neurodevelopmental disorder? Evidence from a four-decade longitudinal cohort study. The American Journal of Psychiatry. 2015;172(10):967–977. doi: 10.1176/appi.ajp.2015.14101266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nørby S. Why forget? On the adaptive value of memory loss. Perspectives on Psychological Science. 2015;10(5):551–578. doi: 10.1177/1745691615596787. [DOI] [PubMed] [Google Scholar]
- Patten SB, Wilkes TCR, Williams JVA, Lavorato DH, El-Guebaly N, Schopflocher D, … Bulloch AGM. Retrospective and prospectively assessed childhood adversity in association with major depression, alcohol consumption and painful conditions. Epidemiology and Psychiatric Sciences. 2015;24(2):158–165. doi: 10.1017/S2045796014000018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pillemer DB, White SH. Childhood events recalled by children and adults. In: Reese HW, editor. Advances in child development and behavior. Vol. 21. Orlando, FL: Academic Press; 1989. pp. 297–340. [DOI] [PubMed] [Google Scholar]
- Poulton R, Moffitt TE, Silva PA. The Dunedin Multidisciplinary Health and Development Study: Overview of the first 40 years, with an eye to the future. Social Psychiatry and Psychiatric Epidemiology. 2015;50(5):679–693. doi: 10.1007/s00127-015-1048-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Prescott A, Bank L, Reid JB, Knutson JF, Burraston BO, Eddy JM. The veridicality of punitive childhood experiences reported by adolescents and young adults. Child Abuse & Neglect. 2000;24(3):411–423. doi: 10.1016/s0145-2134(99)00153-2. [DOI] [PubMed] [Google Scholar]
- Robins LN, Schoenberg SP, Holmes SJ, Ratcliff KS, Benham A, Works J. Early home environment and retrospective recall: A test of concordance between siblings with and without psychiatric disorders. American Journal of Orthopsychiatry. 1985;55:27–41. doi: 10.1111/j.1939-0025.1985.tb03419.x. [DOI] [PubMed] [Google Scholar]
- Ros L, Latorre JM. Gender and age differences in the recall of affective autobiographical memories using the autobiographical memory test. Personality and Individual Differences. 2010;49:950–954. [Google Scholar]
- Scott KM, Von Korff M, Angermeyer MC, Benjet C, Bruffaerts R, de Girolamo G, … Kessler RC. The association of childhood adversities and early-onset mental disorders with adult-onset chronic physical conditions. Archives of General Psychiatry. 2011;68(8):838–844. doi: 10.1001/archgenpsychiatry.2011.77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scott KM, McLaughlin KA, Smith DAR, Ellis PM. Childhood maltreatment and DSM-IV adult mental disorders: comparison of prospective and retrospective findings. The British Journal of Psychiatry. 2012;200(6):469–475. doi: 10.1192/bjp.bp.111.103267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Seery MD. Resilience a silver lining to experiencing adverse life events? Current Directions in Psychological Science. 2011;20(6):390–394. [Google Scholar]
- Shaffer A, Huston L, Egeland B. Identification of child maltreatment using prospective and self-report methodologies: A comparison of maltreatment incidence and relation to later psychopathology. Child Abuse & Neglect. 2008;32(7):682–692. doi: 10.1016/j.chiabu.2007.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Solís CB, Kelly-Irving M, Fantin R, Darnaudéry M, Torrisani J, Lang T, Delpierre C. Adverse childhood experiences and physiological wear-and-tear in midlife: Findings from the 1958 British birth cohort. Proceedings of the National Academy of Sciences. 2015;112(7):E738–E746. doi: 10.1073/pnas.1417325112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Susser E, Widom CS. Still Searching for lost Truths about the bitter sorrows of childhood. Schizophrenia Bulletin. 2012;38(4):672–675. doi: 10.1093/schbul/sbs074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Trinh Q, Sun M. Statistical analysis plans in observational research. Journal of the American Medical Association. 2013;309(1):32–32. doi: 10.1001/jama.2012.56907. [DOI] [PubMed] [Google Scholar]
- Usher JA, Neisser U. Childhood amnesia and the beginnings of memory for four early life events. Journal of Experimental Psychology. 1993;122:155–165. doi: 10.1037/0096-3445.122.2.155. [DOI] [PubMed] [Google Scholar]
- Varese F, Smeets F, Drukker M, Lieverse R, Lataster T, Viechtbauer W, Read J, van Os J, Bentall RP. Childhood adversities increase the risk of psychosis: A meta-analysis of patient control, prospective- and cross-sectional cohort studies. Schizophrenia Bulletin. 2012;38(4):661–671. doi: 10.1093/schbul/sbs050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wade R, Cronholm PF, Fein JA, Forke CM, Davis MB, Harkins-Schwarz M, … Bair-Merritt MH. Household and community-level Adverse Childhood Experiences and adult health outcomes in a diverse urban population. Child Abuse & Neglect. 2015;52:135–145. doi: 10.1016/j.chiabu.2015.11.021. [DOI] [PubMed] [Google Scholar]
- Watson D, Pennebaker JW. Health complaints, stress, and distress: exploring the central role of negative affectivity. Psychological Review. 1989;96(2):234–254. doi: 10.1037/0033-295x.96.2.234. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Wechsler Adult Intelligence Scale. 4. San Antonio, TX: Pearson Assessment; 2008. [Google Scholar]
- Weintraub WS, Daniels SR, Burke LE, Franklin BA, Goff DC, Hayman LL, … Whitsel LP. Value of primordial and primary prevention for cardiovascular disease: A policy statement from the American Heart Association. Circulation. 2011 doi: 10.1161/CIR.0b013e3182285a81. CIR.0b013e3182285a81. [DOI] [PubMed] [Google Scholar]
- Widom CS, Engel SR. Accuracy of adult recollections of childhood victimization, part 1: Childhood physical abuse. Psychological Assessment. 1996;8(4):412–421. [Google Scholar]
- Widom CS, Morris S. Accuracy of adult recollections of childhood victimization, part 2: Childhood sexual abuse. Psychological Assessment. 1997;9(1):34–46. [Google Scholar]
- Wilson RS, Krueger KR, Arnold SE, Barnes LL, de Leon CFM, Bienias JL, Bennett DA. Childhood adversity and psychosocial adjustment in old age. The American Journal of Geriatric Psychiatry. 2006;14(4):307–315. doi: 10.1097/01.JGP.0000196637.95869.d9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Whitson HE, Duan-Porter W, Schmader KE, Morey MC, Cohen HJ, Colón-Emeric CS. Physical resilience in older adults: Systematic review and development of an emerging construct. The Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 2016;71(4):489–495. doi: 10.1093/gerona/glv202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wolraich ML, Lambert EW, Bickman L, Simmons T, Doffing MA, Worley KA. Assessing the impact of parent and teacher agreement on diagnosing attention-deficit hyperactivity disorder. Journal of Developmental and Behavioral Pediatrics. 2004;25:41–47. doi: 10.1097/00004703-200402000-00007. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.