A Factor Analysis and Test of Longitudinal Measurement Invariance of the Children’s Depression Inventory (CDI) Across Adolescence

Abstract

Depression increases dramatically during adolescence. This finding has been demonstrated using multiple measures, including the Children’s Depression Inventory (CDI). The CDI is one of the most commonly used measures to assess depression in youth. However, there is little agreement on its factor structure, and it is possible that its factor structure changes over time. Yet, no study to date has investigated whether this structure is longitudinally invariant from early- to mid-adolescence. The present study examined the factor structure of the CDI in a sample of 227 adolescents aged approximately 13 at baseline and 16 at follow-up. The analyses revealed that a one-factor structure was a good fit to the data at each assessment. Moreover, tests of measurement invariance supported configural, metric, and scalar invariance across time. These findings suggest that changes in depressive symptoms during adolescence are due to true developmental changes, rather than changes in measurement properties.

One of the most commonly used measures to assess depressive symptoms in youth is the Children’s Depression Inventory (CDI; Kovacs, 1985). The CDI was designed as a measure of depression symptom severity that assesses multiple dimensions of depression. The CDI has demonstrated good reliability, discriminant validity, distinguishing children with general distress from other healthy children, and convergent validity based on associations with psychiatrists’ ratings of children’s symptoms and a self-report measure of attributional style (Saylor, et al., 1984; Masip et al., 2010). The CDI is validated for use in youth from ages 7 to 17 (Sitarenios & Kovacs, 1999). However, these norms presuppose that the structure and item properties of the measure are consistent over time. These assumptions remain unexamined, despite their important implications. If the measurement structure and/or functioning of items changes over time, it is not possible to discern whether observed changes in depression are due to “true” changes in depression, or to changes in the measurement of the construct over time.

The CDI is an extrapolation of the Beck Depression Inventory, modified with language for youth (Kovacs, 1992). It was designed to assess five factors (anhedonia, negative mood, negative self-esteem, ineffectiveness, and interpersonal problems) contributing to a total depression score representing symptom severity (Kovacs, 1985). Indeed, these factors represent many of the domains necessary for a diagnosis of Major Depressive Disorder, including depressed or irritable mood, markedly diminished pleasure in all or most activities (including socializing), cognitive impairment (difficulty with decision making, concentration or slowed thinking), and feelings of worthlessness (American Psychological Association, 2013). However, studies using exploratory factor analysis (EFA) have failed to find consistent factor structures for this measure. For example, one study found that a two-factor model, reflecting enjoyment/interpersonal relations and affective behavioral aspects of depression, fit the data (Spence & Milne, 1987), and another found that both two- and three-factor models fit the data well (Carey et al., 1987). In this study, the two-factor model had factors of depressive affect and oppositional behavior, and the three-factor model had the same two factors plus a personal adjustment factor. Hodges et al. (1983) conducted an EFA and concluded that a four-factor model reflecting cognitive, motivational, social integration, and somatic components of depression fit the data best in a clinical sample of children, and a two-factor model comprised of one “general” factor and a second non-compliant behavior factor fit the data in a non-clinic school sample. A fourth study found support for eight and seven factors in samples of healthy and clinic-referred children, respectively (Saylor et al., 1984). A fifth study implementing only a confirmatory factor analysis (CFA) found evidence for the original five-factor structure in both European-American and African-American children (Steele, Little, Ilardi, Forehand, Brody, & Hunter, 2006). Finally, one study obtained a unidimensional factor structure of the CDI in a sample of American Indian and Native Alaskan youth in grades 4–12 (Scott, Clapp, Mileviciute, & Mousseau, 2016).

The differences in factor structures are especially apparent when comparing samples of children and adolescents. For instance, after conducting an EFA and CFA in a large community sample of youth from 8 to 17 years, Craighead et al. (1998) obtained five factors for children (externalizing, dysphoria, self-deprication, school problems, social problems) and the same five factors plus an additional sixth factor (biological dysregulation) in adolescents. Subsequent CFAs found support for the same 5- and 6-factor structures, as well as two higher order factors (internalizing and externalizing).

Weiss et al. (1991) found evidence for five similar factors for children and adolescents using EFA. A subsequent CFA supported this five-factor model and also found evidence for a single higher order “depression” factor; however, the factor loadings from the depression factor to observed indicators differed across children and adolescents. Importantly, this study reported that a preliminary scree test indicated the best fitting model for both children and adolescents is a single factor.

Some studies, however, have concluded that the factor structure of the CDI is the same for children and adolescents. Garcia, Aluja, and Barrio (2008) conducted both an EFA and CFA in a large sample of Spanish children, guided by findings from previous studies. They found that a model with five first-order factors and one higher order factor fit the data well for both children (aged 7–12) and adolescents (aged 13–16).

In sum, there is limited agreement on the factor structure of the CDI in the literature, and differences in factor structure may exist across developmental stages. To our knowledge, only two studies to date have directly compared the factor structure of the CDI across development (Weiss et al., 1991; Garcia et al., 2008). However, the samples used were broad and heterogeneous in age. Both studies defined youth aged 7- or 8- to 12-years old as children and youth aged 13–16 years old as adolescents. There are substantial developmental differences between a 7- and 12-year old, as well as between a 13- and 16-year old. More work is needed to understand whether there are differences in the structure of the CDI across each stage of development, especially from early to later adolescence.

The ages of 13 and 15–16 are particularly relevant to the study of depression. Many studies have found that increases in depression begin around age 13, and the preponderance of depression in females is evident by age 15–16 (Hankin & Abramson, 2001; Hankin, Abramson, Moffitt, Silva, & McGee, 1998). These findings have been replicated using measures of depressive symptoms, like the Symptom Checklist-90 (SCL-90), the Children’s Depression Inventory (CDI), the Kandel Depression Scale, the Child Behavior Checklist (CBCL) and the Youth Self-Report (YSR) (Ge, Conger, & Elder, 2001; Cole, Martin, Peeke, Seroczynski & Fier, 1999; Petersen et al., 1991; Wichstrom, 1999; Compas et al., 1997) and diagnoses, like the Diagnostic Interview Schedule (DIS) and the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS) (Hankin et al., 1998; Lewinsohn, 1993; Cohen, et al., 1993). However, little work has investigated the stability of the measurement structure across adolescence in many of the most commonly used instruments for assessing depressive symptoms and diagnoses, like the CDI. There are significant biological, cognitive, emotional and social differences in youth of these ages (Steinberg, 2010; Yurelun-Todd, 2007; Brook-Gunn & Warren, 1989), all of which could influence their report of depressive symptoms and structure of responses on the CDI across time. Differences in the structure of depression across development would challenge our understanding of mean level changes in symptoms across time. Thus, in order to have confidence that the developmental changes observed are attributable to actual changes in individuals across time and not measurement error, formal tests for longitudinal measurement invariance in depressive measures are needed. The current study first conducted an EFA to determine the best fitting factor structure for the CDI in a sample diverse in gender, race, and socioeconomic status; we then tested for measurement invariance in the CDI between individuals aged 12–13 and aged 15–16 in this sample.

Methods

Participants

The study sample was drawn from the Adolescent Cognition and Depression (ACE) Project, an ongoing prospective, longitudinal study of risk factors for the development of depression during adolescence. Recruitment was conducted through either school mailings and follow-up phone calls inviting the mother/primary caregiver and her child to participate or through advertisements in Philadelphia-area newspapers. Eligibility required that the adolescent was 12 or 13 years old, identified as Caucasian/White, African American/Black, or Biracial, and the mother/primary caregiver also agreed to participate. Individuals who identified as other ethnicities were excluded because an aim of the study was to examine differences in cognitive vulnerabilities to depression between Black and White adolescents. Exclusion criteria were the absence of a mother/primary caregiver, either the child or the mother’s English was insufficient to complete study assessments, and either the adolescent or mother was psychotic, intellectually disabled, or had a pervasive developmental disorder or learning disability.

The current study sample consisted of 227 early adolescents (M_age=12.88, SD=.65 at baseline) with complete data at baseline (T1) and follow-up (T2). At T2, participants were approximately 16 years old (M_age=16.03, SD=.71). In the current sample, 51.5% of the adolescents identified as female, 55.9% identified as African-American, and 42.7% were eligible for subsidized school lunch, which takes into account the number of dependents supported on the family’s income and was a proxy for socioeconomic status (SES). SES and race were not significantly correlated with depressive symptoms at T1 or T2 (SES: r=.09, p=.17, r=.12, p=.08, respectively; race: r=−.02, p=.77, r=.07, p=.31, respectively). Females reported significantly higher depressive symptoms at both T1 and T2 (r=−.13, p=.04, r=.17, p=.01, respectively).

The current sample did not differ from the larger study sample on sex (t(639)=1.56, p=.12), race (t(639)=.99, p=.32), or CDI at T1 (t(631)=1.18, p=.24), but they were less likely to be eligible for subsidized school lunch (a proxy for low SES; t(610)=.21, p=.04).

Procedures

Participants in the present study were assessed at two timepoints. The goal of the ongoing ACE Project is to interview participants every six months. At each follow-up, participants complete a battery of self-report questionnaires, interviews, and behavioral tasks. The CDI is given at every follow-up. Data for the current project were drawn from participants’ baseline assessment (T1) and a routine assessment three years later (M=3.15 years, SD=.27).

Measures

Depressive Symptoms.

The Children’s Depression Inventory (CDI; Kovacs, 1985) was used to assess depressive symptoms. This self-report measure consists of 27 items, each scored on a scale of 0 to 2. Participants choose the statement that best describes how they have been feeling for the past two weeks. An example item is “I am sad once in a while/I am sad many times/I am sad all the time.” For some items on the CDI, a higher score indicates a more severe symptom, whereas for other items, a lower score indicates a more severe symptom. Prior to running analyses, we reverse scored items where lower scores indicated more severe symptoms; so for all items, a higher score indicated more severe symptoms. This allowed us to interpret a positive correlation with a factor to mean that the item was significantly positively associated with that factor without referring to the content of that item. Thus, all items were expected to have positive factor loadings. On some items, there was limited to no endorsement of response options that indicated elevated symptoms. Three items at T1 (2, 3, and 9) and six items at T2 (1, 3, 4, 9, 22 and 27) had the options indicating the most severe depressive symptom endorsed one or zero times. Thus, data were recoded to be dichotomous (0/1, not endorsed/endorsed). The CDI has been demonstrated to have good construct validity (Doerfler, Felner, Rowlison, Raley, & Evans, 1988), to predict depressive disorders, and to differentiate depressive disorders from anxiety disorders and externalizing disorders among youth (Timbremont, Braet, & Dreessen, 2004). Internal consistency for this measure was adequate at T1 and T2 (α=.85 and α=.87, respectively).

Data Analysis

All analyses were completed in Mplus Version 7 (Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus User’s Guide. Seventh Edition. Los Angeles, CA: Muthén & Muthén.). As previous studies have reported discrepant factor structures for the CDI, we first conducted EFA to identify the structure in this sample and then fit CFA models to test for longitudinal invariance. We evaluated model fit based on the comparative fit index (CFI), Tucker-Lewis index (TLI) and root mean square error of approximation (RMSEA). Good model fit is indicated by a CFI ≥ .95, TLI > .95, and a RMSEA ≤ .05. Acceptable fit is indicated by a CFI > .90, TLI > .90, and a RMSEA < .08. We did not rely on the chi square value as an indicator of fit, as the chi square is known to be sensitive to large sample sizes (Schermelleh-Engel et al., 2003).

To test longitudinal measurement invariance, we implemented a CFA framework. This model specified individual depression factors at ages 12/13 and 15/16 and permitted these factors to be correlated. We followed three steps, each imposing an additional restriction on the measurement model. Model fit information was compared with that of the previous, less restricted model. Invariance was considered to be met if the fit of the more restrictive model was not significantly less than that of the less restricted model (Widaman, Ferrer, & Conger, 2010).

The first step of longitudinal measurement invariance is configural invariance, which imposes the same pattern of factor loadings at each timepoint, but does not make any equality constraints. We tested configural invariance by specifying the CDI factors at each timepoint and permitted correlations among factors across time. The second level of longitudinal measurement invariance is metric invariance, which tests whether factor loadings to each observed indicator are equal across timepoints. Thus, to test metric invariance, we constrained factor loadings to be equal at age 12/13 and age 15/16. The third level of longitudinal measurement invariance is scalar invariance, which tests whether the item thresholds are invariant across time. In other words, it tests whether the likelihood of endorsing each item is equivalent at each timepoint. Support for scalar invariance indicates that there are no inherent differences in the likelihood of endorsing items between each timepoint, and therefore, that changes in depression scores on the CDI reflect a true change in depression levels over time, not just the effect of age.

When testing measurement invariance, we used a guideline of a change in either the CFI, TFI, or RMSEA of ≥ .01 to be indicative of significant decrement in model fit (Chen, 2007). Per recommendations by Muthen, du Toit, and Spisic (2013), a weighted least square mean and variance (WLSMV) estimation method was used in Mplus (Muthen and Muthen, 2005) because observed variables were binary.

Results

Exploratory Factor Analyses of the CDI at Ages 13 and 16¹

An EFA using an oblique oblimin rotation was conducted to evaluate the best fitting model for each timepoint. At both age 12/13 and age 15/16, eigenvalues between a 1-factor solution and 2-factor solution had a ratio > 4:1 (T1: 1 factor = 11.843 & 2 factor = 1.993; T2: 1 factor = 9.728 & 2 factor = 2.335), which suggests that this measure captures a unidimensional measure of depression (Slocum-Gori & Zumbo, 2011). In addition, other model fit indices indicated that model fit was good (T1: CFI = 0.94, TLI = 0.94, RMSEA = .03; T2: CFI = 0.94, TLI = 0.94, RMSEA = .04). Model fit for models with 2, 3, and 4-factors, was also good, and in some cases, better than a 1-factor model. However, these factor structures had questionable interpretations. First, the items that significantly loaded onto each factor did not represent a unitary construct. Second, there were multiple cross-loadings for items, which further diminished the interpretations of the scales. Therefore, a one factor model was used to test measurement invariance. A CFA was conducted to test the fit of a 1-factor model for each timepoint. Model fit was good at both ages (Table 1) and factor loadings at each timepoint are presented in Table 1.

Table 1.

Factor loadings for a one-factor model of the Children’s Depression Inventory at Time 1 and Time 2.

	Time 1		Time 2
	b	SE	b	SE
CDI1	0.78***	0.08	0.74***	0.07
CDI2	0.58***	0.07	0.60***	0.08
CDI3	0.55***	0.09	0.65***	0.08
CDI4	0.50***	0.08	0.71***	0.06
CDI5	0.68***	0.09	0.67***	0.08
CDI6	0.45***	0.08	0.56***	0.08
CDI7	0.80***	0.07	0.85***	0.06
CDI8	0.59***	0.08	0.77***	0.07
CDI9	0.70***	0.07	0.72***	0.07
CDI10	0.76***	0.06	0.78***	0.06
CDI11	0.73***	0.05	0.74***	0.05
CDI12	0.53***	0.09	0.58***	0.08
CDI13	0.52***	0.08	0.64***	0.07
CDI14	0.56***	0.08	0.66***	0.07
CDI15	0.47***	0.08	0.57***	0.08
CDI16	0.67***	0.07	0.61***	0.07
CDI17	0.74***	0.06	0.64***	0.08
CDI18	0.30**	0.09	0.48***	0.09
CDI19	0.34***	0.09	0.52***	0.08
CDI20	0.83***	0.05	0.81***	0.05
CDI21	0.24***	0.09	0.61***	0.07
CDI22	0.30***	0.10	0.64***	0.08
CDI23	0.46***	0.09	0.63***	0.07
CDI24	0.59***	0.08	0.61***	0.07
CDI25	0.57***	0.09	0.81***	0.07
CDI26	0.56***	0.08	0.47***	0.08
CDI27	0.74***	0.09	0.73***	0.09

Note:

^***p<.001,

^**p<.01,

^*p<.05.

Configural Invariance

Given the good fit of the one-factor model for both ages, measurement invariance testing continued. In the configural models, we specified one factor at each time point and permitted correlations among factors across time. The CFI and RMSEA for this model were both good (Table 2), and all factor loadings to their respective latent variables were significant and in the expected direction (Table 1). This indicates that the same factor structure of the CDI fits the data at each timepoint.

Table 2.

Model fit indices for baseline models and measurement invariance models.

Model	χ2	df	pχ2	CFI	RMSEA	90% CI
T1, only	396.089	324	0.0038	0.941	0.031	.019–.041
T2, only	433.183	324	<.0001	0.943	0.039	.028–.048
Configural	1557.448	1376	0.0004	0.922	0.024	.017–.030
Metric	1592.343	1402	0.0007	0.924	0.024	.016–.030
Scalar	1630.556	1428	0.0003	0.918	0.024	.017–.030

Note: χ²= chi square; df = degrees of freedom; p_χ2 = significance of the chi square statistic; CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation; 90% CI = 90% confidence interval of the RMSEA.

Metric Invariance

To test metric invariance, items had loadings to the single factor, and factor loadings for corresponding items were constrained to be equal across time. Overall model fit was good (Table 2). Metric invariance was supported because the changes in RMSEA, CFI, and TLI were < .01 (Table 2). This implies that the factor loadings to items on the CDI do not differ between age 12/13 and age 15/16.

Scalar Invariance

To test scalar invariance, we imposed equality of thresholds across the two timepoints in the model, allowing us to test whether the likelihood of response endorsement can be attributed to the true level of depression of the participant as opposed to age. Overall model fit was good (Table 2). There was a negligible reduction in model fit relative to the metric invariance (i.e., changes in CFI, TLI, and RMSEA were < .01 of those for the metric invariance model). These results suggest that thresholds are equal across the ages.

Discussion

There is a large literature on the increase in depression during adolescence (Hankin et al., 1998). Many studies utilize the CDI to measure depressive symptoms among youth, but the factor structure of this measure has not been highly replicable, and no study has investigated the longitudinal measurement invariance of this measure from early- to mid-adolescence within the same sample of youth. In order to have confidence in longitudinal assessments of adolescent depression and that our findings are due to developmental differences and not changes in measurement properties, it is essential that we evaluate the structure of the measure and test longitudinal measurement invariance.

The current study found support for a one-factor model of depression using the CDI in this sample. Prior work has found a number of discrepant factor solutions, with models ranging from two to eight factors depending on the sample (Craighead et al. 1998; Weiss et al., 1991; Garcia et al., 2008; Spence & Milne, 1987; Carey et al., 1987; Hodges et al., 1987; Saylor et al., 1984). However, several of these studies also found evidence for a single higher order depression factor (Weiss et al., 1991; Garcia et al., 2008). Consistent with the present findings, Weiss et al. (1991) reported that a scree test suggested that a one-factor model may fit the data best, and Scott et al. (2016) found support for a unidimensional model of depression from the CDI. In practice, the CDI typically is treated like a unidimensional measure, with studies reporting total scores and not utilizing subscale scores. Thus, the current study suggests that this is an appropriate way to use this measure.

It is unclear why this is one of the only studies to conclude that a one-factor model fit the data for this measure, but one possibility is that we used different criteria for evaluating how many factors should be retained. This study relied on the ratio between the eigenvalues of the first and second factors, RMSEA, TLI, and CFI. Most previous work has retained any factor with an eigenvalue > 1.0. However, this approach can yield too many factors that lack clinically meaningful constructs (Boyle, 1985). When Scott and colleagues (2016) evaluated the conceptual basis of the two to four factor solutions for the CDI, they concluded that, despite having good model fit, these models represented a response-style artifact or that the factors were poorly defined, and a unidimensional model was more conceptually sound. Indeed, this also could explain the discrepancy in the appropriate number of factors for the CDI in the literature. Alternatively, this finding could be attributed to the fact that the present study changed response options from 3-point scales to binary ones. This led to restriction in range for the analyses and led to a simpler factor structure.

The current study also found support for measurement invariance of the CDI in a sample of youth aged approximately 13 at baseline and 16 at follow-up. Most work has found that age 13 is the time that depression prevalence rates start increasing among girls, and the rate of depression among girls is nearly double that of boys by ages 15–16 (Hankin & Abramson, 2001), making these ages particularly salient to the study of measurement invariance in our measures of depressive symptoms. Our results support the assumption that these findings are due to true developmental changes, and not differences in the measurement of depression across time. In particular, support for scalar invariance allows us to conclude that there are not inherent differences in the likelihood to endorse an item on the CDI between these age groups. Therefore, differences in scores between a 12/13-year old and a 15/16-year old can be attributed to differences in depressive symptoms, not age. Our findings complement existing work that has found that the CDI is invariant across race (Steele et al., 2006) and between children and adolescents (Garcia et al., 2008).

Most factor loadings were comparable from T1 to T2 (Table 1). However, the factor loadings on items 21 (“I never have fun at school/I have fun at school once in a while/I have fun at school many times”) and 22 (“I have plenty of friends/I have some friends but I wish I had more/I do not have any friends”) were quite different between these timepoints. As youth move from childhood to adolescence, they transition from their family as their primary social context to friends. Furthermore, feelings about school may be determined by how much emphasis they place on socializing with friends. It is plausible that most children have not yet made this transition at age 13, but they have by ages 15–16. This could explain why the magnitude of the loadings to these items differs from age 12/13 to age 15/16. However, this difference in the way children value school and social contexts should be considered in the measurement and assessment of depression as youth advance from childhood to adolescence.

This study should be interpreted in light of its limitations. Given that we studied a community sample of adolescents, some options on items were very rarely endorsed. The items in this measure were recoded for the purposes of these analyses to be dichotomous, rather than the original item scaling. Thus, whether these findings generalize to use of the questionnaire in the manner it was written or for more severe samples is unclear. Future work using a larger sample with more variability in symptoms should aim to replicate these findings.

Additionally, these analyses did not consider sex or pubertal development, which both independently and in interaction with each other influence the onset of adolescent depression (Angold, Costello, & Worthman, 1998). The current sample did not have sufficient variability in pubertal development and was not large enough to split by sex, so the question of measurement invariance by sex or stage of pubertal development could not be tested. However, future work should aim to test these questions, as it is essential to the longitudinal measurement of depression.

Finally, relative to the overall sample, those participants included in these analyses were less likely to be eligible for subsidized school lunch. Therefore, the generalizability to low-SES populations should be interpreted cautiously. However, roughly 43% of the current sample was eligible for subsidized school lunch, suggesting that these findings are likely to generalize across different levels of SES.

It also should be noted that the present study discusses and evaluates the CDI, although an updated version of this measure, the 28-item CDI-2 (Kovacs, 2004), is available. While most of the items remain unchanged, some edits were made. Items 6 and 26 were removed from the original CDI. The most symptomatic options of these items read “I believe bad things will happen to me,” and “I never do what I am told.” Items 5, 10, and 25 were edited (e.g., The most symptomatic option on item 10 used to say “I feel like crying everyday” but now says “I feel cranky all the time). Finally, three new items were added to assess hypersomnia, increased appetite, and cognitive difficulties. This new version includes two additional second-order subscales assessing Emotional and Functional Problems and has demonstrated excellent psychometric properties (Bae, 2012). This measure has been utilized and evaluated to a lesser extent than its predecessor, and there is still a need to understand the structure and measurement invariance of the original version. Further, the current study allows us to have confidence in current models of adolescent depression that have been derived from clinical research using the original version of the CDI and in work that may still come from datasets that include the original version. Nevertheless, the field should move towards using more updated instruments, and the present study should be replicated using the CDI-2.

The implications for clinical practice that can be derived from the current study are somewhat limited. There is evidence that the structure of depressive symptoms changes over the course of treatment (Fried et al., 2016), but the sample composition and time frame of the current study (e.g., community sample followed for two years) is quite different from that of a treatment-seeking sample followed over the course of an intervention. Thus, shorter term studies in treatment-seeking youth are needed to evaluate whether changes in CDI scores over the course of treatment reflect true changes in depressive symptoms.

In summary, the current study found evidence for a one-factor model of the CDI, supporting its use as a unidimensional measure in practice. The present findings also offer preliminary evidence for longitudinal measurement invariance of a one-factor model of the CDI from early- to mid-adolescence, adding to the existing literature demonstrating this measure’s sound psychometric properties. Therefore, past work showing an increase in depressive symptoms between ages 13 and 16 is likely reflective of true developmental change. However, more work is needed to test measurement invariance of the CDI by sex, by pubertal development, and in a sample with more variability in symptoms in order to have more confidence in the generalizability of these findings.