Concordance between youth and caregiver report of eating disorder psychopathology: Development and psychometric properties of the Eating Disorder-15 for Parents/Caregivers (ED-15-P)

Abstract

Objective:

Outcome measurement in youth with eating disorders relies heavily on self-report, which is problematic in a population that may deny or minimize symptoms. Caregiver-report measures are lacking. This study examined the psychometric and clinical properties of a newly-developed measure designed to assess short-term change in eating disorder symptomatology as observed by caregivers—the Eating Disorder-15 for Parents/Caregivers (ED-15-P).

Method:

At initial presentation to an outpatient eating disorders assessment, 206 families (primary caregiver and their child up to 18 years old) completed psychological measures.

Results:

The ED-15-P demonstrated excellent reliability, preliminary evidence of discriminant and concurrent validity, and good sensitivity to change early in treatment (first eight weeks). Agreement between primary caregivers and children on eating disorder psychopathology was moderate, while agreement between pairs of caregivers was relatively strong. Level of agreement varied across behaviors and informants, with poorer caregiver-child agreement on dieting and driven exercise.

Conclusions:

Agreement was stronger within caregiver pairs than caregiver-child pairs, indicating that caregivers and their children have related but unique perspectives on eating disorder symptoms, highlighting the importance of multi-informant assessment. The ED-15-P complements a parallel youth report measure to more comprehensively assess eating disorder psychopathology, with high utility for measuring change over time.

Introduction

While routine outcome monitoring throughout treatment enhances clinical benefits (Bickman et al., 2011), it is not widely used in treatment of youth with eating disorders (EDs), where outcomes are modest (Lock, 2015). Development of measures for weekly assessment of eating pathology among youth has been limited by relatively complex language. For example, the brief Eating Disorders Examination–Questionnaire (EDE-QS; Gideon, Hawkes, Mond, Saunders, Tchanturia, & Serpell, 2016) addresses symptoms weekly, but requires advanced reading skills (10^th grade level with Flesch-Kincaid score of 9.9). Another adapted version of the EDE-Q uses more appropriate language but has a two-week timeframe (Carter, Stewart, & Fairburn, 2001).

Developed more recently, the ED-15-Y (Accurso & Waller, 2021), based on the ED-15 (Tatham et al., 2015), allows weekly outcome monitoring while using appropriate language for children and adolescents (3^rd grade level). ED psychopathology assessment is limited by the paucity of developmentally appropriate measures for youth, especially brief measures, as well as the lack of parallel measures for caregiver informants. However, assessment is also limited by historical reliance on self-report (Lock, 2015) even though youth may minimize or deny symptoms due to shame and/or the ego-syntonic nature of their disorder. Poor concordance between youth and caregivers (Couturier, Lock, Forsberg, Vanderheyden, & Lee, 2007) indicates the need for multiple informants. Therefore, a caregiver-report measure of weekly ED psychopathology is needed.

This study examines the psychometric and clinical properties of a new caregiver version of the ED-15-Y—the ED-15 for Parents/Caregivers (ED-15-P). It addresses the convergence of youth and caregiver perspectives of ED symptoms, comparing the utility of the ED-15-Y and ED-15-P with the Eating Disorders Examination–Questionnaire (EDE-Q; Fairburn & Beglin, 2008) and Parent EDE-Q (P-EDE-Q; Parter, Loeb, Hail, McGrath, & Fairburn, 2015).

Methods

All procedures were approved by the Institutional Review Board at the University of California, San Francisco.

Participants and Procedures

Participants were youth (≤18 years) and their caregivers evaluated at a specialty ED program (11/2015–04/2020). When two caregivers consented, the one who scheduled the appointment was designated as primary caregiver. Demographic and clinical information was collected during the intake interview. ED diagnosis was made by a clinical psychologist or supervised psychology intern using the Eating Disorder Assessment for DSM-5 (EDA-5; Sysko et al., 2015).

Measures

Eating Disorder-15 for Youth (ED-15-Y).

The ED-15-Y is a brief measure of ED psychopathology over the past week for youth ≥8 years (Accurso & Waller, 2021). The measure was adapted from the adult ED-15 (Tatham et al., 2015) by simplifying language for each item, including 10 attitudinal items (measured on a 1–5 scale) and five behavioral items (frequency of dieting, binge eating, vomiting, laxative use, driven exercise). The attitudinal items were also rated on modified five-point scale (1–5) rather than the original seven-point scale (0–6). It has demonstrated excellent reliability, validity, and sensitivity to change over time in youth with EDs (Accurso & Waller, 2021). At assessment, youth ≥8 years completed the ED-15-Y.

Parent/Caregiver ED-15 (ED-15-P).

A parallel caregiver-report version (ED-15-P, see Supporting Information) was developed by changing first- (e.g., I) to third-person referents (e.g., my child). Caregivers completed the ED-15-P at assessment and at each session through session eight if their child engaged in treatment (10/2015–10/2018) for an ED involving weight and shape concerns. The version for this study used binary gender pronouns (e.g., he/she), which have been updated to be more gender-inclusive.

Eating Disorder Examination Questionnaire (EDE-Q) and Parent Version of the Eating Disorder Examination Questionnaire (P-EDE-Q).

The EDE-Q global scale has demonstrated good reliability in adolescents with adolescents with EDs (Fairburn & Beglin, 2008; Jennings & Phillips, 2017). The P-EDE-Q measures ED psychopathology from the perspective of parents/caregivers, exhibiting excellent reliability and validity in parents of adolescents seeking EDs treatment (Parter et al., 2015).

Symptoms and Functioning Severity Scale (SFSS).

Caregivers completed a measure of global child psychopathology (SFSS; Bickman et al., 2010) that has demonstrated excellent reliability and validity in caregivers of youth receiving mental health services (Athay, Riemer, & Bickman, 2012).

Data analyses

Given interdependence of data, analyses utilized data from the primary caregiver only unless otherwise noted. IBM SPSS Statistics v.27 was used. Significance level was set at < 0.01 given the number of planned analyses. Internal consistency of the ED-15-P and split-half reliability (first five versus last five items) were examined using omega (McDonald, 1999), Cronbach’s alpha, and Spearman-Brown coefficients. The factor structure for the ten attitudinal items was examined with principal axis factoring (unrotated, no predetermined number of factors) to establish whether the items load on one latent factor. Convergent validity between the ED-15-P and the ED-15-Y and between the P-EDE-Q and EDE-Q, and discriminant validity between the ED-15-P and SFSS were measured using Spearman’s rho. Inter-rater agreement between caregivers (ED-15-P) and youth (ED-15-Y) on behavioral items was tested using percentage of exact agreement (PEA) (i.e., percentage of caregiver-youth pairs who agreed on the presence or absence of a behavior), prevalence and bias adjusted kappa (PABAK), and kappa. Kappa coefficients >.40 were considered moderate, >.60 substantial, and >.80 almost perfect. Convergence between “primary” and “secondary” caregivers was also examined (PEA, PABAK, k; paired t-test). Convergent validity was established through Spearman’s rho correlation between the ED-15-P and P-EDE-Q. Chi-squared tests examined the association of ED-15-P behavioral items and parallel P-EDE-Q items. Concurrent validity was examined by comparing ED-15-P scores by ED diagnoses. To assess sensitivity to change over time, multilevel mixed-effects models were used to examine weekly change in ED-15-P total scores from treatment sessions 1 to 8, adjusting for baseline ED-15-P score and session. A Reliable Change Index was calculated (Jacobson & Truax, 1991) from ED-15-P scores (SD and alpha) to determine what proportion of patients made reliable changes (95% criterion) in caregiver-reported symptoms.

Results

Of 301 patients approached, 86.7% (n = 261) agreed to participate. ED-15-P data were missing for 55 (21.1%) primary caregivers, resulting in a final sample of 206 families (i.e., primary caregiver and youth) (see Supplementary Figure 1 for participant flowchart). Primary caregivers were on average 48.90 years old (SD = 7.66) and included 175 mothers, 30 fathers, and one grandmother. Secondary caregivers (n = 108) included 86 fathers, 20 mothers, one adoptive father, and one stepmother. Youth (n = 206) were on average 14.81 years old (SD = 2.32, range: [6,18]) with a mean duration of illness of 2.07 years (SD = 2.96); fifty-one spent time in multiple households (see Supplementary Table 1 for participant characteristics).

Reliability and Factor Structure

The ED-15-P demonstrated excellent internal consistency (omega = .917, alpha = .913) and split-half reliability (Spearman-Brown = .876). Omega and alpha for other measures were also high (ED-15-Y: .949, .946; EDE-Q: .976, .973; P-EDE-Q: .960, .956; SFSS: .921, .918, respectively). Factor analysis of the ED-15-P revealed one factor (eigenvalue = 5.725) consisting of all ten items (loadings 0.504–0.902), indicating a single latent factor.

Do caregivers agree with their child on ED symptoms?

ED-15-Y data were available for 89.3% (n = 184) of youth. Table 1 provides descriptive statistics and correlation table for the ED-15-P and other measures. For the 184 youth-caregiver pairs, ED-15-Y and ED-15-P scores demonstrated moderate convergence (r = .55), as found between the EDE-Q and P-EDE-Q (r = .64) and the ED-15-P and the EDE-Q (r = .58).

Table 1.

Scores on youth and caregiver report questionnaires and their correlations.

	M (SD)	[min, max]		ED-15-P	ED-15-Y	P-EDE-Q	EDE-Q	SFSS
ED-15-P total	2.96 (1.04)	[1.00, 5.00]		---
ED-15-Y total	3.08 (1.26)	[1.00, 5.00]		.55**	---
P-EDE-Q global	2.34 (1.63)	[0.00, 5.39]		.60**	.55**	---
EDE-Q global	2.41 (1.85)	[0.00, 5.75]		.58**	.90**	.64**	---
SFSS total	48.68 (8.11)	[30.23, 72.21]		.44**	.19*	.33**	.18*	---

Abbreviations: Eating Disorder-15 for Parents/Caregivers (ED-15-P), Eating Disorder-15 for Youth (ED-15-Y), Parent Version of the Eating Disorders Examination-Questionnaire (P-EDE-Q), Eating Disorders Examination-Questionnaire (EDE-Q), Symptoms and Functioning Severity Scale (SFSS).

Note:

^*p < .05,

^**p < .001.

Table 2 shows strong convergence across all five behaviors (dieting: χ² = 36.268, binge eating: χ² = 40.735, vomiting: χ² = 88.784, laxative misuse: χ² = 59.319, driven exercise: χ² = 28.920, ps < .001). Interrater agreement between youth and their primary caregiver on the ED-15-Y/ED-15-P reached >80% exact agreement with moderate to substantial agreement for binge eating (PEA = 80.6%, PABAK = .611, k = .471), vomiting (PEA = 93.9%, PABAK = .877, k = .698), and laxative misuse (PEA = 97.8%, PABAK = .956, k = .492). However, there was more moderate agreement for dieting (PEA = 71.5%, PABAK = .430, k = .438) and driven exercise (PEA = 73.3%, PABAK = .467, k = .400). Caregivers reported greater levels of dieting than their child, whereas youth endorsed the presence of other behaviors more often than caregivers. When behaviors were endorsed, correlations of the frequency of behaviors were moderate-strong for dieting (r = .45), binge eating (r = .45), vomiting (r = .73), laxative misuse (r = .58), and compensatory exercise (r = .45) (ps ≤ .001).

Table 2.

Convergence on eating disorder behaviors (presence vs. absence) between youth (ED-15-Y) and their primary caregiver (PED-15).

			Primary Caregiver (ED-15-P)			Total Responders Endorsing Behaviors (Youth and/or Caregiver) (n, %)
			No (n, %)	Yes (n, %)
Dieting	Youth (ED-15-Y)	No	64 (35.8%)	36 (20.1%)
		Yes	15 (8.4%)	64 (35.8%)		115 (64.2%)
Binge Eating	Youth (ED-15-Y)	No	119 (66.1%)	13 (7.2%)
		Yes	22 (12.2%)	26 (14.4%)		61 (33.9%)
Vomiting	Youth (ED-15-Y)	No	153 (85.5%)	3 (1.7%)
		Yes	8 (4.5%)	15 (8.4%)		26 (14.5%)
Laxative Misuse	Youth (ED-15-Y)	No	176 (97.7%)	0 (0.0%)
		Yes	4 (1.2%)	2 (0.6%)		6 (3.3%)
Driven Exercise	Youth (ED-15-Y)	No	96 (53.3%)	22 (12.2%)
		Yes	26 (14.4%)	36 (20.0%)		84 (46.7%)

Note: Cells with italics indicate youth-caregiver agreement on the presence (yes/yes) or absence (no/no) of a behavior.

Do caregivers agree with each other?

ED-15-P (r = .73, p < .001; n = 108) and P-EDE-Q scores (r = .75, p < .001) converged strongly within caregiver pairs. There was no significant ED-15-P score difference between caregivers (M difference = 0.016, SE = 0.073, t = 2.205, p = .030). Reported frequency of behaviors by caregivers on the ED-15-P converged well (dieting: χ² = 29.913, binge eating: χ² = 21.266, vomiting: χ² = 12.527, laxative misuse: χ² = 27.329, driven exercise: χ² = 35.999, ps < .001) (see Supplementary Table 2). Agreement on behaviors as indicated by the PABAK was almost perfect for laxative misuse (PEA = 98.3, PABAK = .965, k = .391); moderate for binge eating (PEA = 82.4%, PABAK = .647, k = .330), vomiting (PEA = 89.4%, PABAK = .788, k = .256), and driven exercise (PEA = 78.0%, PABAK = .560, k = .444); and fair for dieting (PEA = 67.3%, PABAK = .345, k = .366). Primary caregivers were more likely than secondary caregivers to state that their child engaged in dieting and driven exercise. When caregivers agreed on the presence of dieting or driven exercise, their reports of behavioral frequency were moderately correlated (dieting: r = .49, p < .001; driven exercise: r = .64, p < .001). Other behaviors were too rarely identified for meaningful comparisons between caregivers.

Are caregivers consistent in their reports across measures?

ED-15-P scores converged well with P-EDE-Q scores (r = .85, p < .001) and ED behaviors (dieting: χ² = 70.926, binge eating: χ² = 70.144, vomiting: χ² = 89.773, laxative misuse: χ² = 35.256, driven exercise: χ² = 82.930, ps < .001) (see Supplementary Table 3). Caregiver report suggests that between ≈5–20% of ED behaviors occurred in the past month but not the past week, but (as expected) this pattern was not reflected the other way round. Correlations of ED-15-P and P-EDE-Q behavior frequencies were relatively strong: restraint (r = .60), objective binge eating (r = .46); vomiting (r = .59); and driven exercise (r = .68) (ps < .001). No correlation was calculated for laxative misuse (n = 2).

Differentiation of diagnostic groups

ED diagnosis was significantly associated with ED-15-P scores (F = 25.443, p < .001; no ED: n = 15, M = 2.01, SD = 0.90; ARFID: n = 27, M = 1.84, SD = 0.69; AN: n = 61, M = 3.36, SD = 0.83; Atypical AN: n = 50, M = 3.44, SD = 0.83; BN/atypical BN: n = 16, M = 3.31, SD = 0.92]. Post-hoc comparisons indicated that scores for the first two groups were significantly lower than scores in the other three groups (EDs involving weight and shape concerns), with no difference between individuals with ARFID and those with no eating disorder.

Do caregivers report reductions in eating pathology early in treatment?

For the subset of youth with EDs involving weight and shape concerns (n = 60), there was a significant effect of session on ED-15-P scores, after adjusting for baseline ED-15-P score (B = 0.593, SE = 0.121, F = 24.118, t = 4.911, p < .001). There was as an overall decrease of 0.744 points (SE = 0.017, F = 39.062, t = −6.250, p < .001) in the early phase of treatment. The Reliable Change Index for the ED-15-P (SD = 1.04; alpha = .913) was 0.85. Of the 29 patients with at least one ED-15-P score between sessions 6–8, 51.7% (n = 15) achieved reliable improvement and none showed reliable deterioration.

Discussion

Caregiver perspectives on ED recovery have been neglected despite their clinical significance (Accurso, Sim, Mulheim, & Lebow, 2020). The ED-15-P is the first caregiver report measure of ED psychopathology that enables tracking of session-to-session change. It demonstrated strong convergent validity, moderate discriminant validity, excellent concurrent validity with ED diagnosis, and reliable sensitivity to change early in therapy. Caregivers agreed with each other in their assessment of eating problems, but less so with youth, particularly around less obvious behaviors (restriction and driven exercise), as shown elsewhere (e.g., Mariano, Watson, Leach, McCormack, & Forbes, 2013), supporting the importance of obtaining multiple perspectives on eating pathology in youth. Measures of early improvement have mostly focused on weight and binge/purge behavior. While such changes are critical determinants of outcome at the end of treatment and/or follow-up (Matheson et al., 2020), they ignore key changes in psychological symptoms.

Limitations include a relatively small sample size, missing data for secondary caregivers, and the absence of data on community or non-treatment-sample samples. Further research is needed to examine the clinical applications of this measure in different therapies and cultures, and to determine whether measurement feedback systems using the ED-15-P can improve patient outcomes. Evidence-based assessment that integrates multiple perspectives might help to augment current modest outcomes of evidence-based treatment for youth with EDs. Given its brevity and the availability of a parallel youth-report measure, the ED-15-P has potential to track progress for children or adolescents across treatment. Future directions include integrating ED-15-Y and ED-15-P scores and evaluating the clinical utility of routine outcome monitoring in youth with EDs.