Evaluating Psychometric Properties of the Emotional Eating Scale Adapted for Children and Adolescents (EES-C) in a Clinical Sample of Children Seeking Treatment for Obesity: A Case for the Unidimensional Model

Abstract

Background:

The Emotional Eating Scale - Adapted for Children and Adolescents (EES-C) assesses children’s urge to eat in response to experiences of negative affect. Prior psychometric studies have demonstrated the high reliability, concurrent validity, and test-retest reliability of theoretically defined subconstructs among non-clinical samples of children and adolescents who were primarily healthy weight; however, no psychometric studies exist investigating the EES-C among clinical samples of children with overweight/obesity (OW/OB). Furthermore, studies conducted in different contexts have suggested a discordant number of subconstructs of emotions related to eating. The purpose of this study was to evaluate the validity of the EES-C in a clinical sample of children seeking weight-loss treatment.

Method:

Using a hierarchical bi-factor approach, we evaluated the validity of the EES-C to measure a single general construct, a set of two separate correlated subconstructs, or a hierarchical arrangement of two constructs, and determined reliability in a clinical sample of treatment-seeking children with OW/OB aged 8 to 12 years (N=147, mean age= 10.4 yrs.; mean BMI z=2.0; female=66%;Hispanic=32%, White and other= 68%).

Results:

Comparison of factor-extraction methods suggested a single primary construct underlying EES-C in this clinical sample. The bi-factor indices provided clear evidence that most of the reliable variance in the total score (90.8 for bi-factor model with three grouping factors and 95.2 for bi-factor model with five grouping factors) was attributed to the general construct. After adjusting for relationships with the primary construct, remaining correlations among sets of items did not suggest additional reliable constructs.

Conclusion:

Results suggest that the primary interpretive emphasis of the EES-C among treatment-seeking children with overweight or obesity should be placed on a single general construct, not on the 3- or 5- subconstructs as was previously suggested.

Introduction

The Emotional Eating Scale (EES) for adults was designed to assess eating behaviors in response to a range of affective states.¹ In an effort to measure how different dimensions of emotional eating relate to weight-related outcomes this scale is typically divided into several subscales including anger/frustration, anxiety, and depression. The EES was later adapted for use with children and adolescents to determine whether similar behaviors occur at this younger age.²

Although theoretically defined subscales of the EES-C have proven useful in investigating the relationship between selected subscales and outcomes such as loss of control eating status and increased caloric intake in children, there are discrepancies in the proposed number of unique subscales for assessment among children. For instance, in the original validation study comprised of a non-clinical group of children aged 8 to 18 years with primarily a healthy weight status, Tanofsky-Kraff et al.² proposed three subscales (‘anger, anxiety, frustration’, ‘depression’, and ‘unsettledness’). However, when using the Spanish version of the EES-C, Perpina et al.³ suggested five subscales (‘anger’, ‘anxiety’, ‘depression’, ‘restlessness’, and ‘helplessness’). Of note, this study was conducted among a mixed sample of children aged 9 to 16 years representing clinical groups seeking weight loss treatment and nonclinical groups with both healthy weight and overweight status. More recently, Vannucci et al.⁴ suggested a single dimension of emotional eating in response to negative mood, which was captured using a total score (summing all items other than eating in response to feeling “happy”). This study was conducted among healthy-weight children between the ages of 8 and 18, and scores were highly correlated with observed energy intake. Different analytic approaches, levels of emotional eating, populations of interest, and developmental aspects of emotional development may have influenced the number of unique scales that could reliably be formed across these studies.^2–5

As children participating in the previous studies evaluating the EES-C were predominantly healthy weight, it is of value to evaluate the psychometrics of the EES-C and its true dimensionality in a treatment seeking sample of children with OW/OB. This information would allow a more accurate assessment of the dimensions of emotional eating in children and could prove useful for clinicians when tailoring interventions to address the specific aspects of emotional eating.

To date, psychometric evaluations of the EES-C have not provided consistent support for conceptually identified sub-domains. To the best of our knowledge, no prior psychometric study of the EES-C has examined whether a subscale of emotional eating has greater utility beyond that of the single, overarching emotional eating construct. The hierarchical bi-factor model, which concurrently describes the common general emotional eating trait and the set of subscales (e.g., eating in response to anger, depression, etc.), may supplement empirical evidence that prior psychometrics studies were unable to contribute.^6–9 By adopting a higher-order factor analysis, we can begin to partition whether responses to items were more likely to arise from smaller correlated subconstructs (e.g., eating in response to anger, depression, etc.) or if item responses were reflective of a single general construct. Thus, this study aims to evaluate the validity of the EES-C in a clinical sample of children seeking treatment for overweight or obesity by assessing a single general construct, a set of two separate sub-constructs, and a hierarchical arrangement of the two using a bi-factor approach.

Materials and Methods:

The Family, Responsibility, Education, Support and Health (FRESH) study was a randomized clinical non-inferiority trial, conducted between July 2011 and July 2015 in San Diego, California (Clinical Trial: ), and evaluated two 6-month treatments for childhood obesity.¹⁰ A non-inferiority trial seeks to determine whether a treatment is no worse than a reference treatment. Detailed recruitment methods are described elsewhere.^{10, 11} Briefly, eligibility criteria included children aged 8 to 12 years, child body mass index (kg/m², BMI) from 85^th to 99.9^th percentile, a parent in the household with a BMI of at least 25 kg/m² who could read English at a minimum of a fifth-grade level, and availability to participate in the study on designated evenings. Exclusion criteria included children with extreme levels of obesity (higher than 99.9^th percentile), child diagnosis of a current physical disease for which physician supervision of diet and exercise is required, and psychiatric comorbidities that could interfere with participation in the treatment (e.g., autism, severe depression).¹⁰ In total, 150 children who meet the inclusion criteria and their parents were recruited through local advertisement, school listservs, and local pediatric clinics. The current study uses measures completed by these children at baseline with complete data, prior to starting any treatment (N=147). The institutional review boards of the University of California San Diego and Rady Children’s Hospital, San Diego, California approved the study. Written consent and assent were obtained from parents and children, respectively.

Emotional Eating Scale- Adapted to Use in Children and Adolescents (EES-C):

The EES-C is a 26-item questionnaire that assesses eating when confronted with 25 negative emotions (e.g., resentful, discouraged, etc.) and 1 positive emotion (“happy”) on a 5-point Likert scale (from “no desire” to “very strong desire to eat”).² Summing the individual EES-C items generates an EES-C total score. Of note, the “happy” item was an exploratory item of the EES-C, and has been excluded in validation studies.^2–4 Thus, we excluded the emotion “happy” item for all analyses. Alternative factor models derived from prior studies^{2, 3} in non-clinical samples have been replicated to provide context and are described in the analysis section.

Child Eating Disorder Examination (ChEDE):

The ChEDE is a semi-structured clinical interview that assesses eating disorder features in children.¹² The binge eating section was administered to evaluate the number of objective bulimic episodes (i.e., objectively large amount of food with loss of control over eating) and subjective bulimic episodes (i.e., smaller amount of food, still viewed as excess to participant, with loss of control over eating) in the past 3 months. To test the convergent validity, we dichotomized children into two groups, ‘any experience of loss of control eating’ (LOC) or ‘no experience of loss of control eating’ (no-LOC), respectively.¹³

Child Behavior Checklist (CBCL):

The CBCL is a parent-report questionnaire that assesses children’s behavioral problems.¹⁴ The CBCL yields standardized T scores and age-adjusted scores on internalizing, externalizing, and total behavioral difficulties, which were used to test the discriminant validity of the EES-C. The CBCL has been evaluated in clinical and community populations with good inter-rater and intra-rater reliability.¹⁵ Low correlations between the EES-C and CBCL scales would indicate that the EES-C measures emotional eating behaviors which are distinct from general mood or behavioral issues that the CBCL measures.

Statistical Analysis

Analyses were conducted using the R statistical programing language (version 3.4)¹⁶ and SPSS (version 23, IBM).¹⁷ Polychoric correlations were used where appropriate¹⁸ because these correlations are called for when ordered categorical responses, such as the Likert type scale items found in the EES-C, are used. Prior to the bi-factor analysis, we replicated the methods used in prior studies to help define multiple EES sub-constructs for the present clinical sample. In brief, these methods used the Kaiser-one criterion for class enumeration and principal component or exploratory factor analysis with varimax rotation. The lack of agreement on the number of subscales within the EES-C from exploratory models in published studies using non-clinical samples (e.g. ‘excited/uneasy/resentful’, ‘loneliness’, ‘depression’ for the three-factor model; ‘anxiety’, ‘agitated’, ‘guilty’, ‘upset’, and ‘loneliness’ for the five-factor model) highlighted the need to examine both unidimensional and multidimensional models. For the current study, we focused on the hierarchical bifactor model which simultaneously evaluated a primary single construct underlying either 3- or 5-factor configurations for individual subscales.

Construct validity

The optimal solution for the number of factors to be retained was determined by the Kaiser-one criterion.¹⁹ The following procedures were also tested: 1) Velicer’s minimum average partial (MAP) criteria;²⁰ 2) Horn’s parallel analysis (PA);²¹ 3) the optimal coordinates (OC);²² 4) the acceleration factor (AF);²² 5) the Very Simple Structure (VSS);²³ and 6) Ruscio and Roche’s Comparison Data (CD).¹⁸ Summing the factored items generated the scores for each EES-C subscale.

Convergent and Discriminant Validity

To assess convergent validity, comparisons between groups (LOC vs no-LOC) were performed using nonparametric statistical tests, and p-values <0.05 were considered significant. To assess discriminant validity, Spearman’s correlations were used to determine whether the total and subscale scores for the EES-C were significantly related to internalizing, externalizing and total behavior problems as assessed by the CBCL.

Bi-factor model indices

Hierarchical bifactor models were examined to simultaneously evaluate the strength of support for a primary single factor underlying the responses and the degree to which additional group factors suggested the multidimensionality of the remaining variability among items after adjustment was made for relationships with the primary construct.^{6, 9}

Explained common variance (ECV):

ECV was used to estimate the degree to which a general construct and correlated subconstructs could be used to explain and organize item responses.^{6, 7, 24}

Percent of uncontaminated correlations (PUC):

PUC, a bifactor-specific index, presents information on the percentage of correlation that is not contaminated by multidimensionality.²⁵

Reliability coefficients:

Cronbach’s coefficient alpha (α) was used to estimate the internal scale reliability coefficient.²⁶ McDonald’s coefficient omega (ω) was used to compliment the alpha coefficient, which estimates the proportion of variance in the unit-weighted total score attributable to all sources of common variance.²⁷ Omega hierarchical (ωH) and Omega hierarchical subscale (ωHS) were used to estimate the variance that is attributable to a single general construct and/or correlated subconstructs.^28–30

Scalability (Coefficient H):

Coefficient H was used to evaluate how well a set of items’ scalability represent the latent variable.²⁴

Results

The mean age of child participants was 10.4 years, and 66.0% (n=97) were males. Almost one-third of the subjects were Hispanic, and the rest were White and other See Table 1 for participant demographics and characteristics.

Table 1.

Sample characteristics

Variable names	Total (N=147)
Total EES-C score	14.77 (15.51)
Sex (male)	50 (34.01)
Hispanic	47 (31.97)
Non-Hispanic Other	30 (20.41)
Non-Hispanic White	70 (47.62)
BMI z-score	2.00 (0.34)
CBCL (percentile)
Internalizing	42.54 (29.19)
Externalizing	34.80 (27.53)
Total	40.68 (28.64)

Mean (SD) or N (%) were reported

Convergent and Discriminant Validity of the EES-C

An examination of convergent validity between the EES-C total score and LOC eating behavior was statistically significant, such that children who endorsed LOC eating had significantly higher EES-C total scores than children who did not endorse LOC eating (mean 18.14 (SD 13.28) vs. mean 13.52 (SD 16.38); p<0.05).

Table 2 presents an examination of discriminant validity of the EES-C total and subscale scores from both three or five grouping factors. The correlation coefficients between age-adjusted percentiles of internalizing, externalizing, and total behavior problems on the CBCL with the EES-C total score or subscales were all small (range = −0.08 to 0.08). No statistically significant differences were noted, suggesting the EES-C reliably assessed a construct of emotional eating that was distinct from general emotional or behavior problems. The validity results did not differ when we used means adjusted by Hispanic ethnicity and sex (p’s>0.5).

Table 2.

Correlation coefficient between percentile of internalizing, externalizing, and total behavior problems and sum of total EES-C score and subscale scores from either three or five grouping factors

	Internalizing	Externalizing	Total
EES-C total score	−.03	.02	.00
Three-factor model
EES-C TFS-F1	−.03	.02	.02
EES-C TFS-F2	−.05	.03	−.02
EES-C TFS-F3	−.08	−.06	−.08
Five-factor model
EES-C FFS-F1	−.04	.05	−.01
EES-C FFS-F2	−.06	.04	.01
EES-C FFS-F3	−.02	.04	.04
EES-C FFS-F4	−.08	−.06	−.08
EES-C FFS-F5	−.01	.06	.06

No bivariate Spearman correlations were significant at the .05 level

Exploring Construct Validity

Figure 1 presents a scree plot of indices for determining the number of factors to be retained. While the Kaiser-one approach suggested that five factors be retained, Velicer’s MAP criteria provided minimum squared average partial correlations of 0.02 for the first and second steps, suggesting one or two factors. The remaining four methods (three are displayed in figure 1) suggested that one factor be retained.

Figure 1.

Scree plot of indices for the optimal number of factors to be retained

Applying the Bi-factor Model

Table 3 presents summary results of standardized factor loadings and bi-factor reliability indices of the three-grouping factor. The single general factor loadings ranged from .57 to .79 across all items and most were within the DeVellis common criteria for an acceptable range.³¹ All subscales’ item-loadings for correlated factors were poor with the exception of emotional eating in response to feeling ‘furious’ (.79). Across all factor extractions, the single general factor of the bi-factor model accounted for 90% of reliable variance with 10% of the residual variance spread across subscales. After accounting for the variance due to the general factor, the subscales for the correlated factors accounted for a small proportion of the total variance (ωHS = .17, .11, .37). The remaining 3% of the ω total is estimated to be due to random error. With a coefficient H of .94, the general factor presents near perfect construct replicability. None of the indices of the three grouping factors show strong construct replicability.

Table 3.

Standardized bi-factor loadings and indices from three-factor solution (TFS)

		GF	TFS-F1	TFS-F2	TFS-F3	u2
1	Resentful	0.70				0.44
2	Discouraged	0.66	0.31			0.46
3	Shaky	0.74		0.24		0.39
4	Worn out	0.57				0.63
5	Not doing enough	0.70	0.24			0.40
6	Excited	0.72		0.35		0.36
7	Disobedient	0.75				0.39
8	Down	0.64	0.36			0.46
9	Stressed out	0.70				0.47
10	Sad	0.66	0.30			0.47
11	Uneasy	0.73		0.28		0.38
12	Irritated	0.74		0.28		0.32
13	Jealous	0.71				0.45
14	Worried	0.65	0.36			0.42
15	Frustrated	0.77				0.35
16	Lonely	0.61	0.34			0.50
17	Furious	0.64			0.79	0.05
18	On edge	0.76	0.23			0.36
19	Confused	0.71	0.25			0.42
20	Nervous	0.67		0.21		0.47
21	Angry	0.79			0.28	0.26
22	Guilty	0.58	0.42			0.45
23	Bored	0.62		0.20		0.58
24	Helpless	0.73	0.25			0.39
25	Upset	0.72	0.38			0.32
Indices
Eigenvalue		11.98	1.31	0.61	1.01
Coefficient α		0.96
Coefficient ω total		0.97	0.94	0.91	0.89
ω hierarchical and subscale		0.88	0.16	0.11	0.33
Reliable variance from ω		90.82	17.55	11.75	37.30
Explained common variance		0.80
Percent uncontaminated corr		0.58
Scalability (H)		0.94	0.55	0.31	0.63

GF= general factor; TFS-F1: Depression; TFS-F2: Anxiety; TFS-F3: Angry

Table 4 presents the summary results of standardized factor loadings and indices of a bi-factor model with five grouping factors. The single general factor loadings remained strong and ranged from .61 to .80 across all items. Within the bi-factor model with five grouping factors, item loadings were all less than 0.50 with the exception of the item ‘furious’ (.81). The single general factor accounted for 95% of the reliable variance, implying only 5% of the residual variance is distributed across subscales. After accounting for the variance due to the general factor, the subscale grouping factors accounted for a small proportion of the total variance (ωHS = .11, .14, .18, .33, .14). The coefficient H of .95 suggests strong construct replicability of the general factor, whereas none of the indices of the five grouping factors show strong construct reliability. The fourth grouping factor in this model (FFS-F4; table 4) had an H index of .66 which meets the recommended cutoff for favored construct replicability but had only two items (‘furious’ and ‘angry’), suggesting a set of closely related items strongly defined by eating in response to feeling furious. Results suggesting the single general factor instead of subscales remained the same when the analyses were stratified by sex.

Table 4.

Standardized bifactor loadings and indices from five-factor solution (FFS)

		GF	FFS-F1	FFS-F2	FFS-F3	FFS-F4	FFS-F5	u2
1	Resentful	.69	.20					0.43
2	Discouraged	.67					.22	0.46
3	Shaky	.73	.27					0.38
4	Worn out	.59		.24				0.58
5	Not doing enough	.70					.23	0.36
6	Excited	.71	.37					0.36
7	Disobedient	.76						0.38
8	Down	.64			.44			0.24
9	Stressed out	.70			.40			0.44
10	Sad	.67			.31			0.44
11	Uneasy	.71	.32					0.35
12	Irritated	.75	.28					0.30
13	Jealous	.70						0.42
14	Worried	.69		.32				0.37
15	Frustrated	.77			.25			0.30
16	Lonely	.61					.48	0.37
17	Furious	.62				.81		0.05
18	On edge	.78		.25				0.32
19	Confused	.73		.26				0.38
20	Nervous	.67	.22					0.43
21	Angry	.80				.25		0.19
22	Guilty	.62		.46				0.32
23	Bored	.61	.22					0.57
24	Helpless	.73					.32	0.31
25	Upset	.74			.37			0.26
Indices
Eigenvalue		12.17	.70	.68	.76	1.05	.74
Coefficient α		.96
Coefficient ω total		.97	.92	.87	.91	1.04	.84
ω hierarchical and subscale		.92	.11	.14	.18	.33	.14
Reliable variance from ω		95.25	12.10	16.66	19.78	31.82	17.50
Explained common variance		.80
Percent uncontaminated corr		.79
Scalability (H)		.95	.36	.36	.43	.66	.34

GF= general factor; FFS-F1: Anxiety; FFS-F2: Guilty; FFS-F3: Down; FFS-F4: Angry; FFS-F5: Loneliness

Discussion

This study evaluated the construct validity and psychometric properties of the EES-C using a hierarchical bi-factor approach among children with overweight or obesity seeking weight-loss treatment. Nearly all of the reliable variance of the EES-C was captured by a single general construct underlying the responses, with multiple bi-factor indices supporting the general factor’s unidimensionality. Results suggested the single general factor of emotional eating directly influenced responses on each of the subscales from the correlated factors rather than simply reflecting the accumulation or indirect influence of separately assessed constructs. Scores from the general factor demonstrated good convergent validity with a measure of LOC eating behavior, and good discriminant validity with no evidence of significant relationships with competing measures of general emotional or behavioral problems from the CBCL.

Developmentally, it may be useful to use a single general construct for emotional eating in children with overweight or obesity, rather than focusing assessments on discrete emotions in relationship to eating behaviors. Children between the ages of 8 and 12 years old are still developing the cognitive and emotional awareness needed to distinguish between different affective states that are represented in the EES-C.³² For example, children in this age range may just be crystallizing the ability to distinguish overall levels of arousal (e.g., furious vs. calm) or general valence of affect (e.g., positive vs. negative) in addition to discrete emotions (e.g., lonely).³³ Therefore assessing different dimensions of emotional eating may be too developmentally complex for younger children, and outcomes would be misattributed to specific emotional dimensions that are not actually recognized by the children if the multi-structure construct was used.

In terms of applied methodology, our study utilized several newer approaches that move the previous psychometric work conducted on the EES-C forward. One of the greatest challenges in factor analysis is choosing the correct number of factors to retain. The traditional Kaiser one approach suggested that five factors exist in the EES-C. Of the six alternative factor extraction methods tested (OC, AF, PA, CD, VSS, and MAP), five suggested that one factor be retained and the sixth (VSS) suggested that one or two factors should be retained. This implies that, while multiple sources of variability in item responses within the EES-C could be scored separately, the identification of items or relative importance of extracted subscales may not be stable or replicable across studies. Rather, multiple indices suggest a more stable and parsimonious solution may be to organize all items using the single primary construct.

Another stabilizing methodological approach addresses which test of correlation best reflects the ordered categorical response process for these items.^{2, 3, 5} The EES-C, which used a five-point Likert type scale, has a strong skewedness and kurtosis, and using Pearson correlations may produce factors that are based solely on item distribution similarity and can cause items to appear as multidimensional when, in fact, they are not.³⁴ In the present study, we have implemented the polychoric correlation approach, which leads to more robust estimations of dimensionality than factor analyses using Pearson correlations.

Furthermore, our study utilized several modern coefficients to evaluate internal consistency. Prior psychometrics studies of the EES-C have extensively used coefficient alpha (α), which demonstrated strong internal consistency; however, high α values from previous studies may be partly attributable to the many redundant items within the scale, which inflate correlations within the group factor. The reliance on α alone has been criticized as an exclusive indicator of scale reliability because it underestimates true reliability and is not sensitive to violations of assumptions of the unidimensional nature of the scale.^{35, 36} By implementing a bi-factor approach, we have partitioned single general and correlated group factor variance to better understand the strength of a single primary factor underlying the EES-C. Upon evaluating the percent of the total score variance attributable to a single general factor, ωH provided clear evidence that most of the reliable variance in the total score was attributable to the general factor, not to the subscales. We also provided a coefficient H, which is interpreted as a replicability coefficient. Only the general factor passed the threshold of coefficient H (.7); none of the subscales met this criterion. The low coefficient H of all the subscales suggests poor construct reliability, as they are likely to differ from one study to another and in different contexts. The total score, however, had loadings greater than .90, indicating high construct reliability between studies.

One major strength of this study is its use of newer empirical approaches that have been absent from previous validation studies. These methods provide a more robust evaluation of the psychometric properties of the EES-C and a more complete picture of scale performance. Furthermore, this was the first study to examine the psychometric properties using treatment seeking children with OW/OB. Several limitations, however, must be considered. As this was a randomized control clinical trial with a population of children seeking to lose weight, social desirability and self-report bias may have possibly influenced our participants’ responses with regards to their emotional eating behaviors. For instance, the median score of the EES-C of our clinical sample was nominally lower (8–12 years; median 9.5) compared to the original validation study with 151 youths (8–18 years; median 13).⁴ This result may imply that children at this young age might not yet be aware of their tendency to eat in response to emotions. Furthermore, given that children with significant psychiatric comorbidity were excluded, the range of eating in response to negative affective states may have been limited. Additionally, generalizability to other children with overweight/obesity or to healthy weight children warrants further investigation. Lastly, as children reach adolescence and develop greater insight into the influence of emotions on behavior, we may observe a greater range of scores. Future studies should test the reliability of this scale in older pediatric populations while using a similar bi-factor approach.

In summary, these results suggest that in a clinical sample of children with overweight or obesity, the EES-C should be implemented with a unidimensional scale and demonstrated good construct validity, paralleling findings from non-treatment seeking children using a total score.⁴ Thus, recommendations to use a single total score should be applied to both treatment-seeking and non-treatment seeking children. Future studies are needed to determine whether the single general factor, as manifested in the total score, is clinically relevant and can predict excessive weight gain trajectories or success in pediatric obesity treatment.