Psychometric Evaluation of the Behavioral Activation for Depression Scale-Short Form in Alzheimer’s Caregivers

Abstract

The present study evaluated the psychometric properties of scores from the Behavioral Activation for Depression Scale Short Form (BADS-SF) in a sample of older age, spousal, Alzheimer’s caregivers participating in an evaluation of Behavioral Activation (BA) therapy compared to an Information Support (IS) group. At baseline assessment, caregivers (N = 170) completed the BADS-SF, which is comprised of two subscales (Activation and Avoidance) that can be summed to produce a total score. Confirmatory factor analysis was used to evaluate structural validity. A two-factor solution fit the data adequately, however the first item on the scale did not load onto either factor. Internal consistency reliability for the total and subscales scores was poor as measured by Cronbach’s alpha. Construct validity was evidenced by significant expected relationships with depression. Pre- to post-intervention scores did not evidence sensitivity to change. These findings provide some support but raise important concerns about the validity and reliability of BADS-SF scores in a population of older adult caregivers.

The Behavioral Activation for Depression Scale (BADS) was developed to measure activation and avoidance that are the theorized mechanism of change in Behavioral Activation (BA) therapy (Kanter, Mulick, Busch, Berline, & Martell, 2007). According to Lewinsohn (1974), depression is a condition resulting from losses of, reductions in, or chronically low levels of response contingent positive reinforcement (RCPR). The aim of BA therapy is to promote behaviors that could lead to increases in RCPR (i.e., activation) and reduce behaviors that could maintain a lack of RCPR (i.e., avoidance; Kanter, Callaghan, Landes, Busch, & Brown, 2004). Although the goal of BA therapy is to treat depression and several measures already exist that assess depressive symptoms, the BADS was developed specifically to measure levels of activation and avoidance behaviors during BA therapy, and to be sensitive to changes in those levels (Kater et al., 2007). The short form version of the scale (BADS-SF) is a nine-item measure comprised of two subscales: a six-item subscale assessing activation and a three-item subscale assessing avoidance (Manos, Kanter, & Luo, 2011).

The original version of the BADS was a 25-item scale comprised of four subscales: activation, avoidance, work/school impairment, and social impairment. The BADS-SF was developed later in response to several limitations of the original scale identified by the scale developers (Manos et al., 2011). The activation subscale was less related to depression than the other subscales or the BADS total score. Item 6 performed somewhat poorly, with a factor loading of only .49 on the work/school impairment subscale in the initial development sample (Kanter et al., 2007). Additionally, the scale developers noted that the confirmatory model fit of the BADS in a community sample was adequate but had room for improvement (Kanter, Rusch, Busch, & Sedivy, 2009). Also, the four subscales of the BADS were derived empirically without a priori theoretical basis rather than designing subscales to assess key theoretical constructs. In addition, the scale developers believed it would be advantageous to design a shorter, lower burden scale that could be administered weekly over the course of BA therapy to monitor treatment progress (Manos et al., 2011).

The BADS-SF was developed and evaluated over the course of four studies that aimed to 1) identify psychometrically strong items from the original BADS that loaded onto the two theoretically derived subscales of activation and avoidance, 2) perform exploratory and confirmatory factor analyses to confirm a two-factor solution, 3) provide evidence for test-retest reliability, and 4) explore the scale’s sensitivity to change (Manos et al., 2011).

In study 1, a panel of experts in BA treatment evaluated the existing items of the BADS to determine which items best fit the theoretical model (i.e., a two-factor model including activation and avoidance) and eliminate items that did not fit. The panel reworded several items and added an additional item to the scale, eventually deciding to include 12 items. This 12-item scale was subjected to factor analysis both an undergraduate and a community sample. From these results, the developers decided to eliminate three items resulting in a nine-item scale. Internal consistency for this nine-item scale was acceptable for both the community sample (α = .776) and the undergraduate sample (α = .844). Additionally, this nine-item version correlated with the CES-D in both the community sample (r = −.583, p < .001) and in the undergraduate sample (r = −.612, p < .001).

In study 2, changes in wording recommended by the consultants were implemented and the new item was included. This ten-item scale was administered to a sample of 471 participants recruited through psychology classes at a local university. Factor analysis revealed that one item performed poorly and was dropped from the scale. CFA on the final, 9-item version of the scale resulted in good model fit with a two-factor solution. The total scale demonstrated good internal consistency (α = .819), and the total and subscale scores showed strong correlations with other measures of activation, avoidance, and depression as predicted; demonstrating good convergent validity (Manos et al., 2011). The final scale items and factor structure appear in Table 1.

Table 1.

Published items and factor structure of the BADS-SF

		Activation	Avoidance
1.	There were certain things I needed to do that I didn’t do.	R	-
2.	I am content with the amount and types of things I did.	X	-
3.	I engaged in many different activities.	X	-
4.	I made good decisions about what kinds of activities and/or situations I put myself in.	X	-
5.	I was an active person and accomplished the goals I set out to do.	X	-
6.	Most of what I did was to escape from or avoid something unpleasant.	-	R
7.	I spent a long time thinking over and over about my problems.	-	R
8.	I engaged in activities that would distract me from feeling bad.	-	R
9.	I did things that were enjoyable.	X	-

R indicates the item is reverse scored

Study 3 assessed the test-retest reliability and the predictive validity of the BADS-SF by examining the association between BADS-SF scores and scores on the Environmental Reward Observation Scale (EROS), a scale designed to measure changes in RCPR. Results indicated that higher BADS-SF total scores were positively and significantly associated with more time spent in high-reward activities (β = .590, t = 2.145, p = .045). Similarly, higher BADS-SF activation scores were positively and significantly associated with more time spent in high-reward activities (β = .525, t = 2.142, p = .045). BADS-SF avoidance scores were not associated with time spent in high-reward activities. Regarding test-retest reliability, BADS-SF total scores from Time 1 and Time 2 were significantly correlated (r = .451, p = .035), as were the activation (r = .608, p < .01) but not avoidance (r = .068, p = .762) scores.

The scale developers assessed sensitivity to change by gathering data on two clients receiving BA as outpatient treatment for depression. Each client completed the BAD-SF and a scale measuring depressive symptoms (Depression and Anxiety Stress Scale or Beck Depression Inventory II) prior to every treatment session and attended therapy for 15 weeks. Client 1 showed that change in activation per BADS-SF scores predicted change in depression both concurrently (r = −.92, p < .001) with a 1-week lag (r = −.67, p = .004). Client 2 showed that change in BADS-SF scores predicted changes in depression concurrently (r = −.87, p < .001), but did not show the time-lagged association.

The BADS-SF has been translated and validated in multiple languages including French, German, and Spanish. In general, these studies have found support for a two-factor solution comprised of activation and avoidance. However, certain items have performed poorly in factor analysis resulting in slightly different factor structures across studies. In the German version of the scale, item 8 had a factor loading of < .50 and was therefore eliminated from the final scale. Additionally, item 1 loaded on the avoidance factor in the German version rather than the activation factor as reported in the scale’s development (Fuhr, Hautzinger, Krisch, Berking, & Ebert, 2016). The French version of the scale did not find adequate factor loadings for items 1 or 9, and did not include them in their CFA (Wagener, Van der Linden, & Blairy, 2015). In the Spanish version of the scale, item 1 was specified as part of the avoidance rather than the activation scale (Santos, 2013). These issues are not limited to translations of the BADS-SF. A psychometric study of the BADS-SF in an adolescent sample also found that item 1 loads better onto the avoidance factor (Petts, Foster, Douleh, & Gaynor, 2016).

Despite these problems with factor structure and the limited evidence for sensitivity to change, the BADS-SF has been used extensively since its development in several populations including inpatient psychiatric patients (Folke et al., 2015; Reins et al., 2013), outpatient psychotherapy patients (Forland et al., 2018), adolescents (Petts et al., 2016), and pregnant women (Dimidjian et al., 2017). However, to date no studies have reported its usage in a sample of older adults. Recent adaptations of BA therapy for older adults prompts an evaluation of the BADS-SF in this population.

The purpose of the present study was to evaluate the psychometric properties (reliability, structural validity, construct validity, sensitivity to change) of BADS-SF scores in a sample of older age, spousal Alzheimer’s caregivers. Because the BADS-SF has yet to be evaluated in a sample of older adults, establishing structural invariance of scores for this age group is necessary to ensure that the scale is measuring the same construct regardless of age. The study aimed to: 1) evaluate the structural validity of BADS-SF scores, 2) assess the internal consistency of the total scale and subscales, 3) assess convergent validity by examining the association of BADS-SF total and subscale scores to depression, and 4) evaluate the sensitivity to change of BADS-SF scores following BA intervention.

Methods

Participants and Procedure

Spousal Alzheimer’s caregivers (N = 170) with at least mild depressive symptoms were recruited as part of a larger study examining the efficacy of a behavioral activation intervention for caregivers at reducing caregiver stress, depressive symptoms, and associated biomarkers of cardiovascular health. To be included in the study, participants had to be caregivers to a spouse with AD, 55 years or older at the time of enrollment, providing at least 20 hours of in-home care per week, and have screened positive for mild depressive symptoms. Caregivers were excluded if they had prior participation in a behavioral caregiver intervention, were diagnosed with a terminal illness, demonstrated cognitive impairment, had severe hypertension (>200/120 mm Hg), received treatment with anticoagulants, or had a history of myocardial infarction or stroke. The sponsoring universites’ Institutional Review Boards approved all study procedures and materials, and participants provided informed consent prior to participation.

Participants were recruited and randomly assigned to either the BA therapy intervention or a time-equivalent Information-Support condition (IS). All participants received a baseline assessment prior to random assignment followed by six, 60-minute sessions of their assigned treatment condition. All participants were assessed at baseline, upon completion of the six 60-minute sessions, and at 8 and 12-months post-baseline. Both the BADS-SF and the CES-D were administered at each assessment.

Measures

Demographics.

Participant demographic information was provided by self-report.

Behavioral Activation for Depression Scale-Short Form (BADS-SF).

The BADS-SF (Manos et al., 2011) is a nine-item scale that was designed to assess the proposed mechanisms of behavioral activation therapy: activation and avoidance. The scale is composed of two subscales assessing each of these mechanisms. Respondents rate the degree to which they agree to statements on a scale from 0 (‘Not at all’) to 6 (‘Completely’). Total scores range from 0 to 54 with higher scores indicating greater activation and less avoidance. Items are divided between the subscales unevenly, with six items in the activation factor and three in the avoidance factor. The total scale was demonstrated to have good internal consistency (α = .819) in its initial development; subscale alphas were not reported.

Center for Epidemiologic Studies Depression Scale (CES-D).

A short, self-report scale to assess depressive symptomatology, the CES-D (Radloff, 1977) has become one of the most widely used assessments of depressive symptoms. It contains 20 statements describing depressive thoughts or feelings respondents may have experienced. Respondents rate the frequency they felt each statement in the past week on a four-point scale ranging from “Rarely or none of the time (less than one day)” to “Most or all of the time (5–7 days).” The scale has demonstrated reliability across a wide range of racial, gender, and age groups, including older adults. In the current study, internal consistency reliability was moderate (α = .64).

Data Analysis

BADS-SF total and subscale scores were calculated for all participants at baseline and post-treatment. Confirmatory factor analysis (CFA) was used to assess the structural validity of the BADS-SF. A two-factor solution was expected. However, as previous investigations have shown discrepancies with the loading of item 1, three different models were tested: one each with item 1 loading onto the activation subscale, one with item 1 loading onto the avoidance subscale, and a third model with item 1 omitted.

Model fit was determined using multiple fit indices, including (a) the comparative fit index (CFI; Bentler, 1990), an absolute index of model fit; (b) the root mean square error of approximation (RMSEA; Steiger, 1990), a parsimony-adjusted index of model fit; and (c) the standardized root mean residual (SRMR; Hu & Bentler, 1999), an absolute index of model fit. For the CFI, values > 0.95 indicated good model fit, and values > 0.90 indicated acceptable model fit. For the RMSEA and SRMR, values < 0.08 indicated acceptable model fit, and values < 0.05 indicated good model fit. A model was determined to adequately fit the observed data if at least two of the three descriptive fit indices met acceptable model fit criteria. The likelihood ratio chi-square was also reported; however, it did not serve as the only indicator of model fit because it is highly influenced by sample size and does not demonstrate degree of fit (Gerbing & Anderson, 1992).

Internal consistency reliability was evaluated using Cronbach’s alpha on the baseline scores for the total scale and each of the subscales. The construct validity of BADS-SF total and subscale scores was evaluated by examining Pearson product–moment correlations with scores on the CES-D. Sensitivity to change was assess by conducting an independent samples t-test of the change in scores from pre- to post-treatment comparing the BA therapy group and the IS group.

The CFA was conducted using MPlus version 8.2 (Muthén & Muthén, 1998–2018). All other analyses were completed in SPSS version 25 (IBM, 2017).

Results

Descriptive Statistics

Descriptive statistics can be found in Table 2. The sample was primarily white (86.5%) and non-Hispanic (86.5%) with mean age of 73.72 (SD = 8.06). Overall, this was a highly educated sample with 14.36 average years of education.

Table 2.

Sample Characteristics

	N = 170
Agea	73.72 (8.06)
Sexb
Female	131 (77.5%)
Male	38 (22.5%)
Years of Educationa	14.36 (1.77)
Employment Statusb
Never Employed	4 (2.4%)
Retired	141 (82.9%)
Employed Part-time	13 (7.6%)
Employed Full-time	7 (4.1%)
Raceb
Black or African American	8 (4.7%)
American Indian or Alaskan	2 (1.2%)
Asian	5 (2.9%)
Native Hawaiian or Pacific Islander	4 (2.4%)
White	147 (86.5%)
Ethnicityb
Not Hispanic or Latino	147 (86.5%)
Mexican	9 (5.3%)
Other Hispanic	9 (5.3%)
Years Caregivinga	4.66 (3.44)
CES-Da	17.13 (8.4)

^aM (SD)

^bn (%)

Structural Validity

Skewness of the nine items ranged from −1.135 to 0.284 and kurtosis from −0.774 to 0.502, indicating no strong deviation from a normal distribution. For model 1, the activation factor included items 1 through 5 and item 9 while the avoidance factor contained items 6 through 8 consistent with the developers’ original assignment of items. In model 2, item 1 was included in the avoidance factor instead of the activation factor. In model 3, item 1 was omitted from the analysis and the CFA was instead performed on an eight-item version of the scale. In all models, an inter-factor correlation was specified between the two latent variables.

For model 1, the two-factor model did not fit well statistically (χ² [26, N = 170] = 52.407, p = .0016) but it did fit adequately descriptively (CFI = .886, RMSEA = .077, SRMR = .058). All standardized factor loadings were generally large and statistically significant for both the activation (values ranged from .509 to .715) and avoidance factors (values ranged from .410 to .722) except for item 1. Item 1 had a factor loading of .181 on the activation factor that was statistically significant. The inter-factor correlation was not statistically significant (r = −.079, p = .478).

The CFA of model 2 yielded a slightly worse model fit (χ² [26, N = 170] = 56.179, p = .0005; CFI = .869, RMSEA = .083, SRMR = .065). Factor loadings for the activation scale were statistically significant and ranged from .522 to .713. For the avoidance scale, factor loadings were significant and ranged from .418 to .709 except for item 1, which had a low factor loading of .065 that was not statistically significant (p = .517). The inter-factor correlation was not statistically significant (r = −.077, p = .497).

Model 3 yielded the optimal results. Although the model did not fit well statistically (χ² [19, N = 170] = 40.420, p = .0029,), it did fit adequately descriptively (CFI = .904, RMSEA = .081, SRMR = .054). All standardized factor loadings were generally large and statistically significant for both the activation (values ranged from .522 to .713) and avoidance factors (values ranged from .411 to .721). The inter-factor correlation was not statistically significant (r = −.084, p = .455).

Reliability

Table 3 shows the means, standard deviations, and internal consistency (Cronbach’s alpha) for each scale. Internal consistency reliability was poor for the BADS-SF total scale (α = .570) and for the two subscales (Activation: α = .691; Avoidance: α = .553). Notably, when item 1 was deleted, the internal consistency of the activation subscale increased substantially (α = .744).

Table 3.

Means, standard deviations, and Cronbach’s alphas for the BADS-SF and CES-D

		Total N = 170		Cronbach’s α
		M	SD
BADS-SF	Total	32.38	6.81	.57
	Activation	20.09	5.90	.69
	Avoidance	12.28	3.59	.55
CES-D		17.13	8.42	.64

Convergent Validity

There were moderate correlations between the BADS-SF total and avoidance subscale scores and scores on the CES-D (BADS-SF: r = −.35, p < .001; BADS-SF Avoidance: r = .39, p < .001). The correlation between the activation subscale score and CES-D scores was weak (BADS-SF Activation: r = −.18, p = .020)

Sensitivity to Change

Independent samples t-tests were performed comparing the pre- to post-intervention change in BADS-SF total and subscale scores for individuals in the behavioral activation group to individuals in the supportive care group. Differences between groups were non-significant (BADS-SF: t(117) = −.669, p = .505; BADS-SF Activation: t(117) = −.947, p = .346; BADS-SF Avoidance: t(117) = .456, p = .649).

Discussion

These results provide some support for the psychometric properties of BADS-SF scores in this sample of older adult, spousal Alzheimer’s caregivers, but also raise significant concerns. Results from the CFA revealed an adequate fit for the proposed two-factor model comprised of activation and avoidance. However, item 1 performed poorly as in previous studies and may need to be omitted or altered to improve the psychometric properties of the scale. Internal consistency reliability was poor for the total BADS-SF score and for each of the subscales. With regard to convergent validity, correlations between the BADS-SF total and subscale scores and the CES-D were in expected directions, but were substantially weaker than those reported in the scale’s development (BADS-SF: r = −.71; BADS-SF Activation: r = −.60; BADS-SF Avoidance: r = .58; Manos et al., 2011). Finally, and notably, the scale scores did not exhibit sensitivity to change from pre- to post-treatment. Given that the scale was specifically designed to measure change resulting from BA therapy, this is a significant potential shortcoming.

These results should be interpreted within the context of the study’s limitations. The sample size was somewhat small to perform confirmatory factor analysis, although results generally supported the expected two-factor solution (except for item 1). Additionally, most of the sample was non-Latino white, female, and highly educated, limiting generalizability of the findings. Despite these limitations, the results raise significant concerns about using the BADS-SF in a population of older adults. The psychometric properties of the scale scores were notably weaker for the current sample than for the sample used in the scale’s development. Item 1 continued to perform poorly as demonstrated in other evaluations of the scale’s psychometrics, and the reason for its poor performance remains unclear. The phrasing of the item could reasonably apply to either activation or avoidance, which may explain it loading onto either factor in different administrations. Within a sample of older age caregivers, the meaning of the item may be unclear. It is possible that some participants interpreted the item as referring to things they needed to get done regarding caregiving. Furthermore, most of our sample consisted of retirees, which is likely different than the samples in which the scale was developed and that may have further obscured the meaning of the item. The item’s poor performance suggests that this item should be omitted in future administrations of the BADS-SF, but that would preclude direct comparison to BADS-SF scores in extant studies. However, considering the issues with item 1, the relatively weak evidence for convergent validity, and the lack of evidence supporting sensitivity to change; the BADS-SF may not be the best tool to monitor changes related to BA therapy in elderly populations.

Possible alternative measures of RCPR include the Pleasant Events Schedule (PES), the Environmental Reward Observation Scale (EROS), and the Reward Probability Index (RPI); however, these scales have notable drawbacks. The PES consists of 320 items, making it considerably more burdensome for respondents and likely a poor way to monitor changes in activation and avoidance over the course of treatment. Both the EROS and the RPI were developed in samples consisting primarily of young, highly educated individuals and have yet to be validated across diverse age groups. The EROS is a unidimensional scale that measures only environmental reward and does not assess avoidance. The RPI consists of two factors, but assesses RCPR using a global, retrospective perspective with a time frame of ‘the past several months’. Therefore, without modification it may not be a good measure of changes in activation and avoidance over the course of therapy. Future studies should assess the psychometric properties of these scales across diverse age groups. However, it is possible that existing scales will not be sufficient for sensitive measurement of behavioral activation and avoidance in therapeutic settings, and that modifications to those scales, or even creation of a new scale, will be necessary.