Assessment of adverse childhood experiences in women: a study of reliability and predictive validity of self-reporting versus medical records abstraction in an American population

Abstract

Objective

To study the reliability and validity of adverse childhood experiences (ACE) scores measured using a questionnaire versus abstraction of medical records, and to test whether the scores vary by history of bilateral oophorectomy, or by age and presence of anxiety or depressive symptoms at the time of questionnaire administration.

Design

The study involved a reliability component and a predictive validity component.

Setting

A population-based sample in Olmsted County, Minnesota, was derived from the Mayo Clinic Cohort Study of Oophorectomy and Aging-2.

Participants

We included 198 women who underwent premenopausal bilateral oophorectomy for a non-malignant indication between 1988 and 2007 and 174 referent women of the same age randomly sampled from the general population (total of 372 women). At a later time (median of 22.7 years later), the women were contacted and invited to self-administer the ACE questionnaire during an inperson visit. Independent of the visit, their medical records were abstracted for ACE by a physician.

Interventions

Questionnaire and abstraction-based ACE scores.

Main outcome measures

Agreement between the two ACE scores (reliability; weighted kappa statistics) and comparison of incidence of multimorbidity in women with ACE scores ≥1 vs 0 (predictive validity; survival analyses). Data were analysed in March and April 2024.

Results

The 372 women in the study had a median age of 65 years at the time of ACE questionnaire self-administration (IQR, 62–69). Questionnaire-based ACE scores showed moderate agreement with abstracted ACE scores (weighted kappa 0.44 (95% CI 0.34 to 0.54)). The cut-off score of ACE ≥1 showed an overall fair agreement between the two scores (kappa 0.33 (95% CI 0.24 to 0.43)). A comparison of the cumulative incidence of multimorbidity in women with an ACE score ≥1 versus women with ACE score of 0 yielded a HR of 1.13 (95% CI 1.00 to 1.27) for abstracted ACE and 1.13 (95% CI 1.01 to 1.27) for questionnaire-based ACE. The best predictive validity was for ACE measured by both methods combined (either or) with a HR of 1.29 (95% CI 1.13 to 1.46).

Conclusions and relevance

Questionnaire-based ACE scores have moderate agreement with medical records abstracted ACE scores. However, both sets of ACE scores are predictive of the accumulation of multimorbidity at older age and should be considered complementary.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• The study provides measures of both reliability and predictive validity for two assessment methods of adverse childhood experiences (ACE).

• The study can be considered population-based because the sample was derived from a geographically defined population.

• The medical records-linkage system of the Rochester Epidemiology Project provided a unique research infrastructure, including data collected historically over more than half a century.

• The study may have underestimated the frequency of ACE in women, and the statistical power was limited for some stratified analyses.

• In analyses using the ACE score, all adverse events were assumed to have the same weight. In addition, we did not consider the overall family environment or the presence of surrogate caring figures.

Introduction

At 25 years after the publication of the pioneering study by Felitti and colleagues, there is a sizeable and rapidly expanding body of evidence linking adverse childhood experiences (ACE) with a variety of diseases and health conditions later in life.^{1 2} However, almost all of the studies have relied on the retrospective assessment of ACE obtained at direct interview or via self-administered questionnaire at only one point in life. Only a few studies have addressed the validity of this retrospective assessment. Unfortunately, the presence of a late-life adverse outcome may motivate a selective recall and the willingness to report ACE, thus causing a recall bias in case–control studies.³,⁶

A 2019 systematic review included 16 studies of the agreement between prospective and retrospective assessment of ACE.⁶ However, only two studies included the full range of children’s adversities, and in most studies, the retrospective measure was made at a young age. Only in a study in New Zealand, the ACE documented at age 0–18 were compared with self-report at age 38 years.⁵ Therefore, the evidence remains limited when the lag time between the ACE and the self-report involves four or more decades.

We expanded the existing knowledge by studying the reliability and validity of the traditional ACE score (10 items) in a sample of women from a US population. The ACE score includes a broad range of experiences, including both child harm and family dysfunction.^{7 8} We compared the scores obtained via self-administered questionnaire later in life with the scores obtained at manual review of the complete medical records in a population-based records-linkage system. Our method of abstraction of ACE scores from medical records is novel, and its reliability and validity have not been addressed. We also investigated the modifying effect of bilateral oophorectomy, age at self-report, and presence of anxiety or depression on the self-report of ACE.

Methods

Sample

We used the infrastructure of the Rochester Epidemiology Project (REP) medical records-linkage system and the established Mayo Clinic Cohort Study of Oophorectomy and Aging-2 (MOA-2) to identify the study sample. The MOA-2 study passively identified all women who underwent premenopausal bilateral oophorectomy with or without concurrent or preceding hysterectomy for a non-malignant indication between 1988 and 2007 and age-matched referent women who had not undergone premenopausal bilateral oophorectomy before the index date (date of bilateral oophorectomy in the matched woman).⁹ Details about the Olmsted County population and the REP have been reported elsewhere.¹⁰,¹²

At a later time (from 2019 to 2022; median of 22.7 years later), a randomly selected subsample of the women included in the MOA-2 study underwent an inperson visit as part of an active follow-up study conducted by the Mayo Clinic Specialized Center of Research Excellence in Women’s Health (SCORE study). During the inperson visit, the women were asked to complete an ACE questionnaire.¹³ Even though the sampling for this study was determined by the intersection of the original MOA-2 study and the more recent SCORE study, the assessment of the ACE score using two independent methods was conducted specifically for this study.

This study follows the Standard for Reporting Diagnostic Accuracy Studies guidelines. Women were excluded from the study if they did not authorise the use of their medical records for research (2.5% of women in the overall population).¹⁰ In addition, written informed consent was obtained from all women who completed the ACE questionnaire. All research activities were approved by the Mayo Clinic Institutional Review Board (approval number 1715-99) and the Olmsted Medical Center Institutional Review Board (approval number 016-OMC-96). Race and ethnicity were abstracted from medical records.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of our research.

Definition of ACE

We used the 10 categories of childhood adversity occurring from birth throughout the age of 18 years introduced by the Center for Disease Control and Prevention.^{1 7 8} We considered five types of child harm (physical abuse, verbal or emotional abuse, sexual abuse, emotional neglect and physical neglect) and five types of household dysfunction (parental separation, domestic violence, substance abuse, mental illness and incarceration of a family member). Any abuse included physical abuse, verbal or emotional abuse, and sexual abuse. The items were also grouped as an overall ACE score, assigning the same weight to each item (0–10 points).

ACE scores from the questionnaire

Women were invited to complete the ACE questionnaire along with several other questionnaires as part of their inperson visit for the SCORE study conducted at the Mayo Clinic in Rochester, Minnesota (online supplemental table S1).¹³ Anxiety and depressive symptoms at the time of visit were assessed using the Beck Anxiety Inventory (BAI) and the Beck Depression Inventory (BDI).^{14 15} Education, marital status, body mass index (BMI) and cognitive performance on the Modified Mini-Mental State Examination (MMSE)¹⁶ were also collected at the time of the inperson visit.

ACE scores from medical records abstraction

Information on ACE was obtained using the medical records-linkage system of the REP, which includes complete records (inpatient and outpatient) from all major medical care providers in Olmsted County, Minnesota, as described extensively elsewhere.¹⁰,¹² Demographic and clinical characteristics, as well as the individual ACE were collected via abstraction of medical records.^{9 17} The complete narrative notes were manually abstracted by a physician (LGR) to identify any reports of the adverse experiences included in the ACE questionnaire. The abstractor searched for events that occurred many decades apart and that were documented in records spread across paediatric, psychiatric, reproductive, preventive medicine or gynaecological specialties. The median length of medical records in the REP at the time of inperson visit was 49.2 years. In addition, for 148 women in the study (39.8%), the REP included medical records dating back to age 15 years or earlier; therefore, there were at least 4 years of concurrent capture of childhood experiences (age 15–18 years). For 82 women in the study (22.0%), the medical records dated back to birth.

Examples of the statements found in the medical records that correspond to the 10 items of the ACE questionnaire are reported in online supplemental table S1.¹⁷ Women without medical record documentation of ACE were considered free of adverse experiences. The abstractor was kept unaware of the ACE self-reported by the women (the abstraction preceded the visit for the majority of women). As part of two previous studies, we reported an intrarater agreement of 90% with a kappa of 80% (95% CI 54% to 100%) and an inter-rater agreement of 94% with a kappa of 83% (95% CI 76% to 92%) for the medical records abstraction.^{17 18}

Assessment of multimorbidity

The validity of ACE derived by questionnaire and by medical records abstraction was assessed by comparing their ability to predict the accumulation of multimorbidity between the time of recruitment in the original MOA-2 study (index date) and 31 December 2021 (predictive validity). We defined multimorbidity using the 20 chronic conditions and the corresponding International Classification of Diseases codes recommended by the Department of Health and Human Services.^{19 20} However, we excluded non-melanoma skin cancer codes from the definition of cancer. The diagnostic codes were extracted electronically from the REP records-linkage system. A person was required to have received two or more codes separated by more than 30 days from among the list of codes defining a condition to be counted as having the condition.²¹

Statistical analyses

The agreement of the two methods to measure ACE scores was assessed by graphic display of the scores and by calculating the weighted kappa statistic of the ordinal ACE scores. ACE items with missing information at medical abstraction or questionnaire were considered negative (absence of that specific adverse experience). Analyses were performed for all women combined (overall analyses) and separately within strata by history of bilateral oophorectomy, by median age at the time of inperson visit (≤65 years vs >65 years), and by presence of anxiety symptoms (BAI score 0–7 vs 8–63; minimal versus mild, moderate and severe) or depressive symptoms (BDI score 0–13 vs 14–63; minimal versus mild, moderate and severe) at the time of inperson visit. The weighted kappa statistics were compared across strata using χ² tests. In addition, we computed overall agreement, positive agreement, negative agreement and the kappa statistic (with 95% CIs) for each of the items separately and for several cut-points of the score (dichotomous variables).²²

The validity of the two methods was assessed by comparing their ability to predict multimorbidity as an outcome (predictive validity). The ACE scores were dichotomised (≥1 vs 0) or treated as a continuous variable. For dichotomous analyses, the accumulation of multimorbidity was represented graphically using Aalen-Johansen curves (a multistate generalisation of cumulative incidence curves; unadjusted curves considering all 20 chronic conditions equally). For both the dichotomous and the continuous analyses, we computed HRs using Andersen-Gill regression models with age as the time scale.²³,²⁵ None of the models violated the proportional hazards assumption. The starting point for the cumulative incidence curves was the time of enrolment in the original MOA-2 study (index date). We also conducted analyses including three variables considered as possible mediation variables, namely smoking (ever versus never), obesity (BMI ≥30 kg/m²) and suicidal ideation.

All tests of statistical significance were conducted at the two-tailed alpha level of 0.05. Analyses were performed using R V.4.2.2 (R Foundation for Statistical Computing, Vienna, Austria) and SAS V.9.4 (SAS Institute, Cary, NC).

Results

Characteristics of the sample

Online supplemental figure S1 shows a detailed flow chart for the sample. Of the original 3241 women included in the MOA-2 study, 2095 underwent medical record abstraction for ACE. Of these 2095 women, 1126 were also randomly selected for an inperson visit for the SCORE study, and 372 provided a self-administered ACE questionnaire, whereas 754 women refused participation. Therefore, we studied 372 women who underwent an inperson visit at a median age of 65 years (IQR 62–69). The MOA-2 index date preceded the time of questionnaire administration by a median of 22.7 years (IQR 19.1–27.0). However, there were no sizeable correlations of the lag time between index date and questionnaire administration with ACE score (Spearman rho -0.143) or with number of comorbidities (Spearman rho 0.012). As shown in online supplemental table S2, a few women had missing data at medical abstraction or questionnaire.

In online supplemental table S3, we compared the ACE scores obtained from medical records abstraction for women who did participate (n=372) or did not participate (n=754) in the inperson visit for the SCORE study. The frequencies of ACE scores were not significantly different, except for the score ≥4, which was more common among non-participants.

The characteristics of the women included in the study overall and separately by ACE score of 0 or ≥1, either at medical record abstraction or self-report, are shown in table 1. The women in the sample were cognitively unimpaired (median Modified MMSE score 97; IQR 94–99). For ACE scores obtained by medical record abstraction, women with an abstracted ACE score ≥1 were more likely to have undergone bilateral oophorectomy. In addition, women with ACE ≥1 had higher BMI and were more likely to have a history of alcohol abuse, drug abuse, cigarette smoking, eating disorders and suicidal ideation or attempt. Finally, women with an ACE score ≥1 had higher scores on an anxiety inventory and a depression inventory. Many of these same differences were also observed for ACE scores obtained by questionnaire. Additionally, women with a questionnaire ACE score ≥1 were younger at the inperson visit than women with a score of 0.

Table 1. Characteristics of women overall and in strata by ACE score from record abstraction and self-reported questionnaire.

Characteristic N (%)*	Overall (n=372)	Abstracted ACE score			Questionnaire ACE score
		0 (n=167)	≥1 (n=205)	P value	0 (n=181)	≥1 (n=191)	P value
Age at index date, median (IQR)	43 (40–45)	44 (41–45)	43 (40–45)	0.02	43 (40–45)	43 (40–45)	0.71
Years from index date to visit, median (IQR)	22.7 (19.1–27.0)	NA	NA	NA	23.4 (20.1–27.3)	21.6 (18.3–26.1)	0.02
Age at visit, median (IQR), years	65 (62–69)	NA	NA	NA	66 (62–70)	64 (61–68)	0.02
Age at visit >65 years	173 (46.5)	NA	NA	NA	94 (51.9)	79 (41.4)	0.04
Depth of record at visit, median (IQR), years	49.2 (40.3–61.1)	NA	NA	NA	48.4 (41.1–61.2)	49.6 (39.4–61.1)	0.76
Bilateral oophorectomy status				<0.001			0.63
Premenopausal bilateral oophorectomy	198 (53.2)	61 (36.5)	137 (66.8)		94 (51.9)	104 (54.5)
Referent	174 (46.8)	106 (63.5)	68 (33.2)		87 (48.1)	87 (45.5)
Years of education				0.59			0.90
≤12	61 (16.4)	30 (18.0)	31 (15.1)		31 (17.1)	30 (15.7)
13–16	253 (68.0)	109 (65.3)	144 (70.2)		123 (68.0)	130 (68.1)
>16	58 (15.6)	28 (16.8)	30 (14.6)		27 (14.9)	31 (16.2)
Race				NA			NA
American Indian or Alaskan native	2 (0.5)	0 (0.0)	2 (1.0)		0 (0.0)	2 (1.0)
Asian	5 (1.3)	4 (2.4)	1 (0.5)		3 (1.7)	2 (1.0)
Black or African American	0 (0.0)	0 (0.0)	0 (0.0)		0 (0.0)	0 (0.0)
Native Hawaiian or Pacific Islander	1 (0.3)	1 (0.6)	0 (0.0)		1 (0.6)	0 (0.0)
White	364 (97.8)	162 (97.0)	202 (98.5)		177 (97.8)	187 (97.9)
Hispanic or Latino ethnicity	1 (0.3)	0 (0.0)	1 (0.5)	NA	0 (0.0)	1 (0.5)	NA
Marital status				0.82			0.79
Married or living with a partner	275 (73.9)	121 (72.5)	154 (75.1)		138 (76.2)	137 (71.7)
Separated or divorced	62 (16.7)	28 (16.8)	34 (16.6)		27 (14.9)	35 (18.3)
Widowed	22 (5.9)	12 (7.2)	10 (4.9)		10 (5.5)	12 (6.3)
Never married	13 (3.5)	6 (3.6)	7 (3.4)		6 (3.3)	7 (3.7)
Body mass index ≥30 kg/m2	148 (39.8)	61 (36.5)	87 (42.4)	0.25	64 (35.4)	84 (44.0)	0.09
Alcohol abuse	27 (7.3)	4 (2.4)	23 (11.2)	0.001	10 (5.5)	17 (8.9)	0.21
Drug abuse	21 (5.6)	3 (1.8)	18 (8.8)	0.004	6 (3.3)	15 (7.9)	0.06
Cigarette smoking	155 (41.7)	56 (33.5)	99 (48.3)	0.004	63 (34.8)	92 (48.2)	0.009
Pack years				0.004			0.009
0	217 (58.6)	111 (67.3)	106 (51.7)		118 (65.9)	99 (51.8)
0.1–14.9	76 (20.5)	31 (18.8)	45 (22.0)		26 (14.5)	50 (26.2)
≥15.0	77 (20.8)	23 (13.9)	54 (26.3)		35 (19.6)	42 (22.0)
Unknown	2	2	0		2	0
Parity				0.75			0.14
Nulliparous	65 (17.5)	28 (16.8)	37 (18.0)		37 (20.4)	28 (14.7)
≥1	307 (82.5)	139 (83.2)	168 (82.0)		144 (79.6)	163 (85.3)
Any pregnancy loss	140 (37.6)	58 (34.7)	82 (40.0)	0.30	68 (37.6)	72 (37.7)	0.98
Eating disorder	24 (6.5)	5 (3.0)	19 (9.3)	0.01	7 (3.9)	17 (8.9)	0.048
Suicidal ideation or attempt	23 (6.2)	2 (1.2)	21 (10.2)	<0.001	5 (2.8)	18 (9.4)	0.008
Beck Anxiety Inventory				0.03			0.005
Minimal (0–7)	319 (86.0)	151 (90.4)	168 (82.4)		165 (91.2)	154 (81.1)
Mild, moderate or severe (8–63)	52 (14.0)	16 (9.6)	36 (17.6)		16 (8.8)	36 (18.9)
Unknown	1	0	1		0	1
Beck Depression Inventory				0.04			0.004
Minimal (0–13)	331 (89.2)	155 (92.8)	176 (86.3)		170 (93.9)	161 (84.7)
Mild, moderate or severe (14–63)	40 (10.8)	12 (7.2)	28 (13.7)		11 (6.1)	29 (15.3)
Unknown	1	0	1		0	1
Modified MMSE score, median (IQR)	97 (94–99)	97 (94–99)	97 (94–99)	0.23	97 (94–99)	97 (94–99)	0.76
Number of chronic conditions, median (IQR)
At index date (0–20)	1 (0–1)	0 (0–1)	1 (0–2)	<0.001	0 (0–1)	1 (0–2)	0.006
At visit date (0–20)	5 (3–7)	4 (3–6)	6 (4–8)	<0.001	5 (3–7)	6 (4–8)	0.003

^*Data are presented as n (%) or as median (IQR) where indicated.

ACE, adverse childhood experiences; MMSE, mini-mental state examination; NA, not applicable.

Reliability of the ACE scores

The results of our analyses for reliability (or agreement of the two measures) are shown in table 2. For most of the ACE, the frequency of positive responses was higher in the questionnaire than in the medical records abstraction. However, substance abuse and mental illness in the family were more frequently documented in the medical records. The overall per cent agreement for the 10 items ranged from 73.4% for mental illness in the family to 96.5% for physical neglect and incarceration of a family member. However, the values of kappa ranged from 0.03 for emotional neglect to 0.51 for substance abuse. Because of the small numbers, some kappa could not be estimated. Considering the ACE score as dichotomous (0 vs ≥1), the overall per cent agreement was 66.7% and the kappa was 0.33 (95% CI 0.24 to 0.43). Online supplemental table S4 shows the characteristics associated with concordance or discordance for the two methods, considering the ACE score as dichotomous.

Table 2. Agreement of ACE values based on medical record abstraction versus self-reported questionnaire.

ACE	N (%) (n=372)		Agreement (%)
	A	Q	Overall	Positive*	Negative†	Kappa (95% CI)	P value‡
Q1—verbal or emotional abuse	40 (10.8)	83 (22.3)	76.6	29.3	86.0	0.17 (0.06, 0.28)	<0.001
Q2—physical abuse	19 (5.1)	47 (12.6)	86.6	24.2	92.6	0.18 (0.04, 0.32)	<0.001
Q3—sexual abuse	28 (7.5)	52 (14.0)	87.6	42.5	93.1	0.36 (0.22, 0.51)	<0.001
Q4—emotional neglect	19 (5.1)	44 (11.8)	84.7	9.5	91.6	0.03 (-0.08, 0.13)	<0.001
Q5—physical neglect	5 (1.3)	8 (2.2)	96.5	0.0	98.2	NA	0.41
Q6—parental separation	28 (7.5)	41 (11.0)	86.8	29.0	92.7	0.22 (0.07, 0.37)	0.06
Q7—witness domestic violence	7 (1.9)	20 (5.4)	93.8	14.8	96.8	0.12 (−0.06, 0.31)	0.007
Q8—substance abuse	106 (28.5)	83 (22.3)	81.5	63.5	87.6	0.51 (0.41, 0.61)	0.006
Q9—mental illness	103 (27.7)	64 (17.2)	73.4	40.7	82.8	0.25 (0.14, 0.36)	<0.001
Q10—incarceration	1 (0.3)	12 (3.2)	96.5	0.0	98.2	NA	0.002
Any abuse§	60 (16.1)	121 (32.5)	71.8	42.0	81.3	0.26 (0.16, 0.36)	<0.001
Any child harm¶	73 (19.6)	130 (34.9)	68.0	41.4	78.0	0.22 (0.12, 0.32)	<0.001
Any household dysfunction**	173 (46.5)	135 (36.3)	67.7	61.0	72.5	0.34 (0.25, 0.44)	<0.001
ACE score ≥1	205 (55.1)	191 (51.3)	66.7	68.7	64.4	0.33 (0.24, 0.43)	0.21
ACE score ≥2	95 (25.5)	118 (31.7)	68.5	45.1	78.0	0.23 (0.13, 0.34)	0.03
ACE score ≥3	35 (9.4)	68 (18.3)	82.0	35.0	89.5	0.26 (0.13, 0.38)	<0.001
ACE score ≥4	14 (3.8)	37 (9.9)	90.6	31.4	94.9	0.27 (0.11, 0.44)	<0.001

^*Calculated as 2×(concordant positive)/(positive from abstraction plus positive from questionnaire).²²

^†Calculated as 2×(concordant negative)/(negative from abstraction plus negative from questionnaire).²²

^‡P values obtained from McNemar’s test.

^§Includes verbal or emotional abuse, physical abuse and/or sexual abuse.

^¶Includes verbal or emotional abuse, physical abuse, sexual abuse, emotional neglect and/or physical neglect.

^**Includes parental separation, witness domestic violence, substance abuse, mental illness and incarceration.

A, abstraction; ACE, adverse childhood experiences; NA, not applicable; Q, questionnaire.

The correlation between ACE scores obtained via medical record abstraction and ACE scores obtained via questionnaire overall and in strata is shown in figure 1. The weighted kappa was somewhat higher in women with a history of oophorectomy (kappa 0.48 (95% CI 0.36 to 0.61) vs kappa 0.34 (95% CI 0.22 to 0.46)) and in women who had higher scores on the depression inventory at the time of inperson visit (kappa 0.54 (95% CI 0.33 to 0.76) vs kappa 0.40 (95% CI 0.29 to 0.52)); however, the differences were not statistically significant.

Figure 1. Scatter plots of the ACE scores assessed using medical records abstraction and self-reported questionnaire. (A) All women overall. (B) Strata by history of bilateral oophorectomy. (C) Strata by age at inperson visit (age ≤65 vs >65 years). (D) Strata by the Beck Anxiety Inventory at inperson visit (minimal versus mild, moderate or severe anxiety; score 0–7 vs 8–63). (E) Strata by the Beck Depression Inventory at inperson visit (minimal versus mild, moderate or severe depression; score 0–13 vs 14–63). ACE, adverse childhood experiences; PBO, premenopausal bilateral oophorectomy.

Predictive validity of the ACE scores

The results for the validity of the ACE score to predict multimorbidity accumulation later in life in models using an ACE score cut-off and in models using ACE scores as continuous variables are shown in table 3. The median follow-up from the index date was 23.4 years (IQR 19.8–27.6) in the overall sample. A comparison of the cumulative incidence of multimorbidity in women with an ACE score ≥1 versus women with an ACE score of 0 yielded a HR of 1.13 (95% CI 1.00 to 1.27) for abstracted ACE and 1.13 (95% CI 1.01 to 1.27) for questionnaire-based ACE. The best predictive validity was for ACE measured by both methods combined (either or), HR 1.29 (95% CI 1.13 to 1.46).

Table 3. Accumulation of multimorbidity^* by strata of ACE scores.

ACE definition	N at risk	Total effect†		Direct effect‡		Indirect effect§
		HR (95% CI)	P value	HR (95% CI)	P value	HR (95% CI)	P value	%
Medical record abstraction
ACE score ≥1	205	1.13 (1.00–1.27)	0.04	1.08 (0.96–1.22)	0.22	1.05 (1.01–1.09)	0.006	37.50
ACE score 0	167	1 (reference)		1 (reference)		1 (reference)
Self-reported questionnaire
ACE score ≥1	191	1.13 (1.01–1.27)	0.03	1.09 (0.97–1.22)	0.17	1.04 (1.01–1.09)	0.006	34.30
ACE score 0	181	1 (reference)		1 (reference)		1 (reference)
Composite ACE score
Abs ≥1 or Ques ≥1	260	1.29 (1.13–1.46)	<0.001	1.23 (1.08–1.40)	0.002	1.05 (1.01–1.09)	0.002	19.00
Abs 0 and Ques score 0	112	1 (reference)		1 (reference)		1 (reference)
Concordance of ACE score
Abs 0 and Ques score 0	112	1 (reference)		1 (reference)		1 (reference)
Abs 0 and Ques score ≥1	55	1.38 (1.15–1.65)	<0.001	1.33 (1.12–1.59)	0.002	1.04 (1.02–1.12)	0.002	11.00
Abs ≥1 and Ques score 0	69	1.32 (1.12–1.56)	0.001	1.27 (1.08–1.50)	0.005	1.04 (1.00–1.09)	0.04	13.60
Abs ≥1 and Ques score ≥1	136	1.23 (1.07–1.42)	0.005	1.16 (0.99–1.34)	0.06	1.07 (0.99–1.09)	0.12	30.90
ACE score (continuous)¶
Medical record abstraction	372	1.05 (1.01–1.10)	0.02	1.04 (0.99–1.09)	0.12	1.01 (0.99–1.03)	0.09	27.80
Self-reported questionnaire	372	1.03 (0.99–1.07)	0.07	1.02 (0.98–1.05)	0.36	1.01 (1.00–1.03)	0.02	47.20

^*Multimorbidity was defined using 20 chronic conditions, with each condition considered equally.

^†The total effect HRs and p values were calculated using Andersen-Gill regression models with age as the time scale. Follow-up for these analyses was started at index date.

^‡The direct effect HRs and p values were calculated using Andersen-Gill regression models with age as the time scale, and including mediation variables for obesity (body mass index <30 kg/m2 vs ≥30 kg/m2), cigarette smoking (ever versus never), and suicidal ideation or attempt (any versus none). The HRs represent the direct effects of the ACE scores that were not mediated by the included characteristics. Follow-up for these analyses was started at index date.

^§The indirect effect was calculated by subtracting the direct effect beta coefficient from the total effect beta coefficient, and was expressed as an HR with bootstrapped 95% CI. We also computed a per cent of mediation (indirect effect beta coefficient divided by the total effect beta coefficient).

^¶HRs for the continuous ACE scores are reported for each 1-unit increase in the score, and assume a linear association.

Abs, abstraction; ACE, adverse childhood experiences; Ques, questionnaire.

Cumulative incidence curves for multimorbidity are shown in figure 2. Two curves for women with either questionnaire or abstracted ACE score ≥1 (either or) and for women with both questionnaire and abstracted scores of 0 (neither nor) are shown in figure 2A. Four curves for four combinations of scores are shown in figure 2B. The best predictive value was observed for women with an ACE score ≥1 from either method (treating the two methods as complementary; figure 2A).

Figure 2. Accumulation of multimorbidity in women with or without ACE. The accumulation of multimorbidity was represented graphically using Aalen-Johansen curves. (A) Two curves for women with an ACE score ≥1 from either questionnaire or abstraction (either or) and for women with a score of 0 from both questionnaire or abstraction (neither nor). (B) Four curves for women with an ACE score ≥1 from both questionnaire and abstraction (both and), for women with a score of ≥1 from abstraction but of 0 from questionnaire, for women with a score of 0 from abstraction but of ≥1 from questionnaire, and for women with a score of 0 from both questionnaire and abstraction (neither nor). The dashed lines in (A) and (B) are the same. Abs, abstraction; ACE, adverse childhood experiences; Ques, questionnaire.

The effect of three possible mediation variables occurring between the experience of ACE and the accumulation of morbidity later in life is also shown in table 3. When smoking, obesity and suicidal ideation were entered into the models, they explained an important per cent of the total effect of ACE (per cent of indirect effect due to mediation).

Discussion

Principal findings

In this population-based sample of women, questionnaire-based ACE scores had a moderate agreement with ACE scores abstracted from medical records. However, both sets of ACE scores were predictive of the accumulation of multimorbidity at older age and should be considered complementary.

Historical comparison with other studies

A 2019 systematic review summarised the results of studies of the agreement between prospective and retrospective measures of childhood maltreatment.⁶ Of the 16 studies identified, only two included a broader range of childhood events comparable to the 10 items of the ACE score.^{4 5} In addition, the retrospective measure was made early in life in most studies (before the age of 40 years). The UK National Child Development Study reported a comparison of a neglect score measured at age 7 years from parents and teachers with a self-reported questionnaire at age 45 years.²⁶ However, the study was restricted to neglect. Only the study by Reuben and colleagues is more directly comparable with our study.⁵

Reuben and colleagues conducted a large-scale study of the reliability and validity of ACE scores in men and women combined within the Dunedin, New Zealand, cohort study.⁵ They reported a modest agreement between self-reported ACE scores compared with documented ACE scores (modest reliability); however, both types of measures were predictive of adverse medical outcomes (good predictive validity). In addition, individuals who self-reported more ACE than documented were described by an informant as more neurotic than average, whereas individuals who self-reported fewer ACE than documented were described as more agreeable on a personality inventory.⁵ Unfortunately, the Dunedin study did not report findings separately for men and women, the lag time between the occurrence of ACE (age 0–18 years) and the interview was relatively short (age 38 years at self-report), and the operational definition of the individual ACE items was somewhat different in the two methods.

Our results from a US population of women are consistent with the results from the Dunedin study. In particular, we confirm that both self-reported and medical records documented ACE have a sizeable impact on the health of women later in life. We also confirm that the mental state of the women at the time of self-report may influence the ability to recall and the willingness to report adverse events as part of a research study.

Interpretation

Our findings are important because they confirm that self-reported ACE are often in agreement with documented ACE even when the self-report occurs at older ages (age >65 years). Because documented ACE are only available in selected populations, the ability to use an interview to obtain an ACE score is strategic for future research. Unfortunately, we cannot use the medical records documenting ACE as a gold standard because some experiences may have occurred but were never documented. In addition, the same adverse event (at the same age and with the same intensity) may have different effects depending on the response of the woman to the event and her ability to process the event and to build resilience. More research is needed to understand the mechanisms by which ACE may impact the trajectory of health and disease over the full life course.

Strengths and limitations

This study has several strengths. In particular, the sample can be considered representative of a defined population of the upper Midwest (population-based study).²⁷ However, women who had undergone bilateral oophorectomy were oversampled because they were originally recruited for a different study. We did not report detailed results of sensitivity analyses restricted to referent women who had not undergone bilateral oophorectomy because the numbers were too small for a meaningful interpretation. However, the overall agreement between the two methods of assessment of ACE scores was similar in the strata with or without bilateral oophorectomy (figure 2B).

Second, the REP records-linkage system was unique in providing access to historical medical data across multiple healthcare institutions and over more than half a century. Even though most recordings of adverse experiences involved some retrospective component, the multiplicity of opportunities and contexts for reporting the experiences over the life course is a strength. During the time in which women were in contact with the records-linkage system for routine medical care, they had multiple opportunities for sharing their childhood experiences with a care provider.

This study has some limitations. First, our detection of ACE was imperfect, and we may have missed some ACE both with the questionnaire assessment and with the abstraction assessment. We have likely underestimated the frequency of adverse experiences similar to what has been observed in other studies.^{1 3} Whether data are collected through an interview, a mail survey or through medical record abstraction, the risk of underestimation is inherent to the sensitivity of the information. Interestingly, our stratified analyses suggest that the ability to self-report ACE did not vary noticeably by age at the time of questionnaire administration (figure 2C).

Second, the abstraction of ACE from medical records is a time-consuming process and involves some level of judgement. We developed specific guidelines to obtain and interpret the data, as detailed elsewhere.¹⁷ In addition, both intrarater and inter-rater reliability were excellent in two previous studies.^{17 18}

Third, consistent with the original use of the ACE score, each one of the 10 items was given the same weight when we used cut-offs.^{1 7 8} However, some ACE may have been more emotionally traumatic than others. Therefore, we also conducted analyses for each experience separately. Finally, the same experience may have had a greater or lesser impact depending on other factors such as the entire family environment (eg, the presence of grandparents or other surrogate caring figures).

Fourth, the median lag time between recruitment in the MOA-2 study and administration of the questionnaire was longer than 20 years. Therefore, to measure the cumulative incidence of multimorbidity, we assumed that the responses to the questionnaire remained stable over time and that they were not affected by the accumulation of multimorbidity. In support of this assumption, there were no sizeable correlations between the lag time and the ACE score or between the lag time and the number of chronic conditions present at the time of questionnaire administration.

Fifth, the population of Olmsted County is primarily white of European ancestry, and our findings may not be generalisable to other populations with a different racial or ethnic composition.²⁷ Finally, although our study had adequate power for overall analyses of reliability and validity, some of the stratified analyses or the analyses for individual items of the score were based on small numbers (limited power).

Conclusions

In this population-based sample of women, questionnaire-based ACE scores had moderate agreement with abstracted ACE scores. However, both sets of ACE scores were predictive of the accumulation of multimorbidity at older age and should be considered complementary.

Data availability statement

Data are available upon reasonable request.