Still Searching for Lost Truths About the Bitter Sorrows of Childhood

A large proportion of children across the globe suffer abuse or neglect. ¹ , ² Therefore, studies that shed light on the health consequences of adverse childhood experiences and examine the potential for resilience are of great importance. The meta-analysis by Varese et al. ³ in this volume is a welcome addition to these studies.

Their meta-analysis addresses the question of whether childhood adversities are associated with risk of psychosis. Although a large number of studies have reported such an association, they have used a variety of research designs and a multitude of measures of childhood adversity and of psychosis. The meta-analysis brings together the many disparate strands of evidence that bear on this association. As the authors note, previous attempts to synthesize this literature have been “narrative reviews with inconsistent conclusions.” By contrast, their meta-analysis adopts a systematic and comprehensive approach. They took great care to collect results from as many sources as possible and to check for publication bias. Its results demonstrate a remarkable strength and consistency of evidence for some association “writ broad.” Thus, it demonstrates that we would be foolish to overlook the potential contribution of childhood adversity when we study the causes of psychoses.

Nonetheless, we believe that the authors’ conclusion—that there is a strong association between adverse childhood experiences prior to age 18 and risk of psychosis—is too strong. The susceptibility of these studies to bias, discussed below, poses a challenge to such a straightforward interpretation of the meta-analysis. The heterogeneity of the studies introduces a further challenge. These challenges have to be directly confronted in order to make further progress in this field.

Many readers might assume that the conclusion of a meta-analysis should be given much more weight than that of a narrative review. The uncertainties raised by particular studies appear to be overridden by the large numbers of individuals and of designs that can be included in a meta-analysis. However, this is not always true, ⁴ and the association between childhood adversity and psychosis provides an instructive example.

The central problem in this field is that in almost all studies, the measure of childhood adversity is weak and is susceptible to bias that could create an artifactual association with psychosis. This is not a problem that can be solved by meta-analysis. Pulling together many studies that share a similar bias will produce a biased result. No matter how the studies are grouped, and no matter how large their number, this fundamental problem cannot be overcome by aggregating their data. The similarity of results across studies, therefore, provides limited assurance of validity.

Figure 2 of the meta-analysis separates the studies into three kinds: “case-control,” “epidemiological [cross-sectional],” and “cohort study [prospective].” The choice of terms is unfortunate and needs to be addressed to understand what has actually been done. The term “cohort study” is not synonymous with “prospective.” There are many kinds of cohort studies. ⁵ , ⁶ In a classic “prospective” cohort study, the exposure status is determined at the outset, and the exposed and unexposed groups are then followed over time. Almost none of the studies in the meta-analysis were “prospective” cohort studies in this sense. The majority were cohorts in which childhood adversity was measured retrospectively, during the follow-up of the cohort but prior to the assessment of the psychosis outcome. Thus, the studies grouped as “cohort study [prospective]” generally relied on retrospective recall to measure childhood adversity. Also, the term “epidemiological” is not in any way synonymous with “cross-sectional.” The classic designs in epidemiology are cohort, case-control, and other designs that provide a stronger platform for causal inference. ⁵ , ⁶ For readers who are unfamiliar with epidemiology, it is important not to equate epidemiological with cross-sectional studies. This does not mean, however, that their grouping is meaningless. The cohort studies do have the advantage that the measure of the exposure preceded the outcome of psychosis. This would be expected to reduce, though not eliminate, the problems that arise from the use of retrospective recall discussed below.

Since almost all the reported associations between childhood adversity and psychosis are based on retrospective recall of childhood adversity, the key question is whether we can rely on this kind of recall to provide unbiased estimates of association. We are compelled to pose this question because over a long period, an extensive literature has cast doubt on the validity of retrospective reports about childrearing, family conflicts, and psychological states in childhood.^7–11 Questions about how events are experienced by children and how memories are formed have been explored by novelists and historians over an even longer time, and the title of this article draws from the great novels by George Eliot ¹² and Proust ¹³ [also used in the citation by Henry et al. ⁹ ]. We now also know that during recall, new information is incorporated into established memories at the level of the biological substrate of memory itself, as illustrated by recent work on PKMzeta, a molecule maintaining memory through long-term potentiation. ¹⁴

Asking people to report retrospectively about their childhood experiences is limited by how good their memory is, how they evaluated the experience when looking back on it [ie, their cognitive appraisal of the event], and whether they choose to disclose these experiences. Information we remember from childhood may be heavily dependent on information told to us in childhood or later and/or constructed by a parent. For establishing associations, “recall bias” is more problematic than inaccurate recall per se. The presence or absence of the health outcome being studied may influence the accuracy of retrospective reports of childhood experiences. Generally, the net effect of recall bias is to inflate measures of association. For example, people with an illness may be more likely to report adverse experiences due to a search for potential causes of their illness. Recall bias may also be produced by antecedents of the actual onset of the illness being studied, such as poor mental health, or by factors associated with it.

Although these problems pertain to all studies based on retrospective recall of childhood experience, they are accentuated in studies of childhood adversities and mental health outcomes. Child maltreatment has been associated with amnesias and other types of forgetting. ¹⁵ , ¹⁶ In a birth cohort, reports of child abuse were found to vary substantially even over the short period of time from 18 to 21 years. ¹⁰ Some studies have found that reports are influenced by the current emotional state of the participant. ¹¹ Thus, it may be particularly difficult to determine whether a person is recalling the objective details of an adverse childhood experience or reconstructing details of what occurred based on other knowledge. A thorough review of the evidence on retrospective recall of adverse childhood experiences suggested that substantial measurement error was pervasive but that valid conclusions might sometimes be drawn when asking about “major adversities of an easily defined kind.” ¹⁷

Only a few studies have directly compared retrospective reports with fully documented abuse in childhood. The largest such study was led by one of us,¹⁸ and we use the results to illustrate how difficult it can be to give meaningful interpretations to retrospective reports. This prospective cohort study followed an “exposed” group of children [ages 0–11 with documented court cases of abuse and neglect] and a matched “unexposed” group [no known history of abuse or neglect]. The exposure was documented in 1967–1971, and numerous follow-ups were conducted including one during 2000–2002 when the mean age of the cohort was 40. Participants were not told that the purpose of the study was to document long-term consequences of child abuse and neglect.

In the 1989–1995 follow-up, participants were assessed for drug abuse and/or dependence and also were asked to retrospectively self-report childhood abuse.¹⁹ Results obtained based on documented exposure status in childhood could then be compared with results obtained based on retrospective self-report. These two different sets of results presented a vastly different picture of the relationship between childhood maltreatment and drug abuse. In the prospective cohort study, there was no increase in risk associated with child maltreatment: children with documented cases of abuse and neglect were equally likely to meet the criteria for a drug abuse diagnosis [35%] compared with a matched control group of individuals without such histories [34%]. By contrast, based on retrospective self-report, child maltreatment was strongly associated with a drug abuse diagnosis.

Using data from a subsequent follow-up during 2000–2002, Widom and Czaja ²⁰ examined the extent to which a person’s self-reports of childhood sexual abuse and physical abuse are accurate and consistent over time. They used as the anchor for this analysis whether the person had a documented [official] case of physical or sexual abuse during the years 1967–1971 and self-reports during interviews that were conducted during 1989–1995 ²¹ , ²² and again in 2000–2002. The format of the self-report questions varied somewhat across the two follow-ups, but each follow-up used multiple questions to retrospectively assess child abuse via self-report. Among persons with a documented history of childhood sexual abuse, 29% did not report it at both follow-ups, 49% reported it at both follow-ups, and 22% were inconsistent across the two follow-ups. For physical abuse, results were better, but even here, 8% reported none and 20% were inconsistent across the two follow-ups. This empirical evidence of underreporting and inconsistent reporting, especially for sexual abuse, is consistent with what has been found in other studies.

This study also examined the potential for recall bias in self-reports from the 2000–2002 follow-up. ²⁰ The authors compared groups who had a history of abuse by self-report only, by official record only, by both, or by neither. The results differed from their predictions, indicating more potential for bias than they had anticipated. People who self-reported abuse but had no official record of abuse had the highest rates of psychopathology across all the disorders assessed [PTSD, alcohol or drug abuse and/or dependence, major depressive disorder, dysthymic disorder, generalized anxiety disorder, and antisocial personality disorder]. People who had an official record but did not self-report abuse had the lowest rates of psychopathology, with the one exception of antisocial personality disorder. These findings suggest that retrospective self-reports of childhood victimization are stronger “drivers” of rates of psychopathology than official reports. The most plausible interpretation is that people who have more problems in adulthood look back on childhood and report more problems.

These results should not be interpreted as showing that adverse childhood experience is not associated with adult mental health. Rather, they suggest that associations can differ substantially according to the measurement used for adverse childhood experience. They also suggest that for a wide range of mental disorders, associations based on retrospective recall are susceptible to recall bias. Although psychoses were not included in this study, it would be surprising if the same general pattern did not also pertain to psychoses. The published data on recall bias specifically for people with psychoses are too sparse to draw firm conclusions, though sometimes reassuring,²³ and do not suggest that recall bias is more problematic for psychoses than other mental health outcomes.

Based on general principles, one might suggest several approaches to reduce recall bias. For example, it should be preferable to measure childhood adversity before the outcome of interest in the study and to use the more developed and extensive measures among those available. Yet study design and method of measurement did not seem to have an appreciable impact on the magnitude of associations reported in the meta-analysis of adverse childhood experience and psychosis. ³ We have no ready explanation for this. Possibly, the aforementioned results provide a clue. If there is an overall tendency for people with mental health problems in adulthood to report more adverse childhood experiences, such bias might pertain across a variety of approaches to retrospective measurement and obscure the differences among those approaches.

In addition, the heterogeneity of studies in the meta-analysis might make it difficult to observe systematic differences related to the type of retrospective measurement. With respect to exposure, for example, the meta-analysis groups together several kinds of exposures from birth up to age 18. The resulting heterogeneity of the kind and timing of exposures in the various studies complicates interpretation. Different kinds of childhood victimization may have different consequences, and childhood victimization at one age may have different consequences than at other ages. ^24–26

Finally, it should be noted that a few of the studies in the meta-analysis did not depend on recall of events that happened a long time ago. In the study by Cutajar et al., ²⁷ exposure was based on officially recorded cases of suspected child sexual abuse before age 16, a community sample was used as a comparison group, and the outcome of psychosis was identified by use of a psychiatric case register. Penetrative abuse, especially after age 12, was associated with increased risk of psychosis. In the study by Arsenault and his coworkers, mothers in a longitudinal twin study were interviewed about maltreatment of their child by an adult starting at age 5, ²⁸ , ²⁹ and mother-reported maltreatment was associated with child-reported psychotic symptoms at age 12. Perhaps, these studies merit separate consideration. This would require a narrative review, however, since even these two studies were very different from one another.

This comment has noted a number of cautions that are required in the interpretation of the meta-analysis. We do think that its conclusions are too strong, given the kind of data available. We also think that in this field, a meta-analysis cannot substitute for an in-depth review that considers the different strengths and weaknesses of the research and identifies the most compelling findings. Nonetheless, as we emphasized earlier, the article represents an important contribution and with appropriate interpretation will help advance this field of research.