Using a New Taxonomy to Combine the Uncombinable: Integrating Results Across Diverse Interventions

Abstract

Researchers have examined numerous psychosocial and behavioral interventions intended to alleviate distress among family caregivers of persons with dementia. Many of these interventions are complex, combining numerous treatment components. Although some multicomponent interventions have been successful in reducing caregiver distress, the impact of specific elements of these interventions on outcomes is not known. The article presents results of an analytic approach that allows researchers to describe the individual elements of multicomponent interventions and to examine the relationships between those components and outcomes. This approach is successfully applied to interventions being evaluated in the Resources for Enhancing Alzheimer’s Caregiver Health (REACH) program. The results indicate that actively targeting caregiver behavior is effective in achieving positive outcomes with respect to caregiver depression.

Providing care to a family member with Alzheimer’s disease and related disorders (ADRD) is a stressful experience that can result in adverse effects for caregivers. Family caregiving has been associated with increased levels of depression and anxiety, increased use of psychoactive medications, poorer self-reported physical health, compromised immune function, and increased mortality (Light, Niederehe, & Lebowitz, 1994; Schulz & Beach, 1999; Schulz, O’Brien, Bookwala, & Fleissner, 1995).

Over the last 20 years, researchers have examined numerous psychosocial interventions aimed at alleviating distress associated with dementia caregiving. Intervention programs have included intensive personalized counseling, supportive group counseling, providing knowledge about ADRD through educational programs, specific therapeutic skills training, enhancing problem-solving skills, and teaching behavior management techniques (see reviews by Bourgeois, Schulz, & Burgio, 1996; Kennet, Burgio, & Schulz, 2000; Knight, Lutzky, & Macofsky-Urban, 1993; Schulz et al., 2002; Toseland & Rossiter, 1989; Zarit & Teri, 1992). Because of the multiple challenges faced by caregivers, the large majority of intervention studies involve combining different treatment elements into a multicomponent treatment package. Multicomponent interventions are more efficacious than single-component trials for some outcomes (Kennet et al., 2000; Sörensen, Pinquart, Habil, & Duberstein, 2002), but the link between specific intervention components and successful outcomes is not well understood.

This issue is common to other types of psychosocial intervention research as well. For example, interventions to improve medication adherence among older people include restructuring medication information so that it is more readily understood (e.g., Morrell, Park, & Poon, 1990), providing external supports such as medication organizers (e.g., Park, Morrell, Frieske, & Kincaid, 1992), training (e.g. Feinberg, 1988), or a combination of these techniques. The typical analysis examines the effects of these multicomponent interventions in their entirety. For example, Leirer, Morrow, Pariante, and Doksum (1989) found that a teleminder system combined with verbal and postal reminders improved adherence for influenza vaccines among a sample of elderly people. However, what is not known is if the improvement in adherence was linked to the teleminder system, the verbal and postal reminders, or the combination. As a result, it is not known whether all program elements were required to provide benefit, or whether some portions of the program were unnecessary.

Interventions, whether single or multiple component, can be distinguished by the manner in which they are delivered (e.g., face-to-face vs. telephone; group setting vs. individual) and the amount of intervention delivered. Both of these issues impact on the intensity of the intervention that may ultimately influence treatment success (Sörensen et al., 2002). However, little is known about the relationship between the intensity of treatments for ADRD caregivers and outcome (Bourgeois et al., 1996). Furthermore, treatment characteristics may interact with caregiver or care-recipient (dyad members) characteristics. Therefore, examining individual treatment components and delivery methods in conjunction with dyad characteristics may help to identify the optimal therapy for a target population.

This article demonstrates a way to understand how the individual components of complex interventions are related to caregiver outcomes. The methodology also affords the opportunity to combine information from diverse interventions. We illustrate the technique using data from the Resources for Enhancing Alzheimer’s Caregiver Health (REACH) program, a cooperative agreement among six intervention sites, a coordinating center, and scientists at the National Institutes of Health. Unlike multicenter clinical trials testing the same intervention(s) at all of the sites, REACH was designed to examine simultaneously the feasibility and outcomes of several different intervention approaches for adult family caregivers of people with ADRD. Each of six sites implemented interventions designed by the investigators based on their own particular knowledge and expertise.

At the six REACH sites, nine different active interventions were evaluated and compared to one of two control conditions: usual care or a more intense minimal support condition (MSC) that provided caregivers with information and empathic listening. The active interventions were broad-based and consisted of psychosocial and educational services, behavioral interventions, environmental modifications, and technology interventions (Wisniewski et al., 2003).

Many elements of the study designs were standardized across the sites (e.g., data items, data-collection time points, inclusion and exclusion criteria), enabling a preplanned meta-analysis to combine information across sites about the effectiveness of the interventions. Despite the commonalities, having different interventions at each site created obstacles with respect to pooling data.

As presented in Gitlin et al. (2003), within-site analyses and meta-analyses were conducted to assess the relative impact of active versus control interventions on measures of caregiver burden and distress at each site and overall. The results of the meta-analyses indicated that active interventions were superior to control conditions with respect to caregiver burden, but there was not a significant difference between active interventions and control conditions on a measure of depression. These results are important and informative, but they do not take full advantage of the data available within REACH, and the opportunity to identify optimal intervention strategies. In that analysis, all active interventions were treated as if they were the same, as were both types of control conditions. This does not account for the diversity and complexity of the treatment strategies. One way to address this limitation is to conduct separate meta-analyses for subsets of similar active interventions (e.g., for the technological interventions used at the Miami and Boston sites). This strategy was used in a preplanned meta-analysis of the Frailty and Injuries: Cooperative Studies of Intervention Techniques (Province et al., 1995) trials, in which 13 of the trials were combined as exercise training and 8 others were pooled as resistance-training interventions. In REACH, however, when creating subsets of interventions based on similarities in the active treatments, the control conditions were often dissimilar. For example, of the two sites that implemented technological interventions, Miami used the MSC as its control condition, whereas Boston had a usual care control condition. Treating the two control conditions as if they were the same would not take into account that the MSC was devised to provide some support, hence some benefit, to caregivers (though hypothesized to be less than that provided by the active interventions). In summary, analyses that disregard differences among treatments termed active or treatments termed control are likely to be insensitive to important variations within these general categories.

To overcome this problem, we developed an alternative analytic approach to examine the relationship between outcomes and intervention components, rather than interventions in their entirety. As described in Czaja et al. (2003), the first step was to decompose the interventions using a task analysis and then to create measures for three dimensions: the primary entity targeted by the intervention (caregiver, care recipient, sociophysical environment); the functional domain targeted by the intervention (cognition knowledge, cognition skills, behavior, and affect); and the delivery characteristics (e.g., dose, mode of delivery, group vs. individual, adaptability/control). Here we extend the work of Czaja et al. by integrating measures of the three dimensions (i.e., the weights that characterize the relative emphasis that the interventions placed on targeted entity-domain attributes, and the intensity or dosage of the interventions), and examined the relationships between individual components of the interventions and the outcomes of burden and depression. We also examined the interactions between intervention components and dyad characteristics in an attempt to identify the most effective components of complex intervention strategies and the subgroups most likely to benefit from them.

The same outcome measures analyzed by Gitlin et al. (2003) were also examined here. These measures are commonly used in caregiving intervention research and at least one of the two was a primary goal of the interventions at all of the sites.

Method

Sample

The six REACH sites randomized 1,222 family caregivers of persons with dementia. Details about recruitment efforts (Nichols, Malone, Tarlow, & Lowenstein, 2000; Tarlow & Mahoney, 2000), inclusion/exclusion criteria, and a description of the entire cohort (Wisniewski et al., 2003) are reported elsewhere. A brief description of the sample is provided here.

To be eligible, consenting caregivers had to live with the care recipient and provide a minimum of 4 hr of supervision or direct care per day for at least 6 months prior to screening. Caregivers were excluded if they were involved in another caregiver intervention study or had an illness that would prevent them from participating for at least 6 months. Care recipients had to have been assessed with at least one limitation in basic activities of daily living (ADLs; Katz, Ford, Moskowitz, Jackson, & Jaffe, 1963) or with at least two dependencies in their instrumental activities of daily living (IADLs; Lawton & Brody, 1969). These criteria were selected to ensure that caregivers were involved in daily tasks and responsibilities that could be burdensome.

The caregivers averaged 62.3 years of age (SD = 13.6; range = 22–95 years). Reflecting the intent to recruit a sizable number of minority caregivers, the sample of caregivers was ethnically diverse; 24.2% were African American/Black and 19.0% were Hispanic/Latino. Over 80% of the caregivers were women with approximately three quarters of the caregivers being either wives (35.7%) or daughters or daughters-in-law (39.0%) of the care recipients.

The care recipients averaged 79.1 years of age (SD = 8.2; range = 44–101 years), 71.3% were at least 75 years old, and the majority was female (55.6%). Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975) scores ranged from 0 to 29, with an average of 12.6. Functional abilities were assessed by the number of ADLs and IADLs for which the care recipient required some assistance. On average, care recipients had 3.3 (of 6) ADL deficits and needed assistance with 7.3 (of 8) IADLs.

Procedure

The REACH program is described in detail in this issue by Wisniewski et al. (2003). Potential participants were initially screened for eligibility. After obtaining informed consent and baseline information from eligible caregivers, they were randomly assigned to either a control condition or an active intervention. Three of the sites (Birmingham, Boston, and Philadelphia) had a single active intervention, whereas the other three (Memphis, Miami, and Palo Alto) had two active interventions. Caregivers participated in follow-up interviews 6, 12, and 18 months after randomization using one of three standard batteries depending upon whether the care recipient (a) was living at home, (b) had been placed in an institution, or (c) had died. The 6-month follow-up coincided with the planned end of intervention at most of the sites.

Outcome Measures

The outcomes were assessed 6 months after randomization and included a measure of burden and a measure of depression. Caregiver burden was assessed using the Revised Memory and Behavior Problems Checklist (RMBPC; Teri et al., 1992). The RMBPC measures caregiver burden with behaviors exhibited by care recipients. Caregivers were asked at baseline and, if their care recipient was still living at home, 6 months following randomization which of 24 problem behaviors (7 memory, 9 depressive, and 8 disruptive) the care recipients exhibited during the past week. Using a 5-point ordinal response set ranging from 0 (not at all) to 4 (extremely), caregivers were asked how bothered or upset they were by each of the behaviors that they had identified. For behaviors not seen during the previous week, 0 (no bother) was assigned. The summary calculation created a Likert-type scale with a potential range from 0 to 96, with higher scores indicating greater burden.

Emotional distress was determined by the Center for Epidemiological Studies—Depression Scale (CES–D; Radloff, 1977), a global measure of depression. The CES–D was initially designed as a screen for community dwellers at risk of developing major depression. It has been used widely in studies with older adults and family caregivers. The CES–D was administered at follow-up to the caregivers regardless of whether the care recipients were alive or still living with them. For each of the 20 items, caregivers rated the frequency of occurrence during the past week along a 4-point scale from 0 (rarely or none of the time) to 3 (most or almost all of the time). Scores therefore have a potential range of 0 to 60, with a higher score indicating greater frequency of depressive symptoms.

Independent Variables—Design Variables

One set of variables included the baseline value of the outcome measure and dyad characteristics used in designing the studies (i.e., as part of the randomization scheme or in statistical power calculations). These so-called design variables consisted of indicator variables for caregiver sex (baseline: females), caregiver/care-recipient relationship (spouse vs. non-spouse [baseline]), and caregiver ethnic identity (White/Caucasian [baseline] vs. Hispanic/Latino vs. Black/African American).

Independent Variables—Intervention Measures

The intervention measures were created by multiplying the domain-entity weights by the total amount of time a caregiver spent in the intervention over the 6-month period, an indicator of intensity or dosage. Note that different caregivers received different amounts of intervention, even within sites, primarily due to practical issues such as caregiver availability or choice. As discussed by Czaja et al. (2003), a limitation of using the domain-entity weights alone as measures of the interventions is that they sum to 1 within an intervention. Thus, although the weights are useful for assessing the relative importance an intervention placed on a specific domain-entity attribute within an intervention, they are not particularly useful for cross-intervention comparisons.

As an example of how the intervention measures were calculated, the weight for caregiver knowledge in the Birmingham skills training intervention was .064 (Czaja et al., 2003). For a caregiver at that site who received, for example, 9 hr of intervention, the amount of time spent providing knowledge about the caregiver was computed as .064 × 9 = 0.58 hr. By combining the intervention weights and intervention duration measures in this way, we characterized the interventions for each caregiver as the amount of each caregiver’s total intervention time that was allocated to each of the 12 entity-domains.

Missing Data

The analyses included participants with complete data for outcome and all independent variables. There were few missing data due to attrition as 89% of the participants remained active in REACH at the 6-month visit. There were participants who missed the 6-month visit but for whom data were available at the 12- or 18-month visit. To reduce the number of participants with missing data, we used linear interpolation to impute 6-month values when baseline and 12-month, or baseline and 18-month data were available. Because the RMBPC was not administered to caregivers whose care recipient was placed in an institution or who had died, there were only 910 caregivers with RMBPC data at both baseline and 6 months, of whom 32 had their follow-up data imputed. Unlike the RMBPC, the CES–D was administered to bereaved caregivers and to those whose care recipient had been placed in an institution, so there were 1,087 participants with outcome data for the CES–D, 50 of whom had their 6-month follow-up data imputed.

There were no significant (p < .05) differences across sites between participants with complete data and those excluded from analyses with respect to caregiver or care-recipient age, caregiver or care-recipient sex, or caregiver/care-recipient relationship. Black/African Americans and Hispanic/Latinos were slightly more likely than White/Caucasians and other ethnic groups to have complete data for the RMBPC (although there was not a significant difference for the CES–D), whereas higher educated and higher-income caregivers were more likely to have complete data for the CES–D than less educated, lower-income caregivers. There was not a significant relationship between income or education and complete data on the RMBPC. Compared to caregivers who did not have follow-up RMBPC, caregivers with both baseline and follow-up data on the RMBPC tended to have lower baseline scores (i.e., less burden). The relationship was similar with CES–D, whereby caregivers who did not complete the follow-up CES–D tended to have more depressive symptomotology at baseline than those who provided CES–D data at both time points.

Multivariable Analysis

REACH was essentially six studies, although with many common design features (Wisniewski et al., 2003). Importantly, the same measures were used at all sites and there was common training and certification of all aspects of the study. However, there were differences among the sites with respect to recruitment strategies and interventions, so it is conceivable that participants within a site were more likely to be similar to each other than to participants at other sites. If so, the assumption of independent observations inherent to many statistical methods was violated, and using such methods would result in variance estimates that were too small. This would, in turn, lead to incorrect significance testing because the p values obtained by erroneously assuming independent observations would be lower than the true value.

The intraclass correlation coefficient (ICC) is a measure of the degree to which observations within a class are more similar to each other than to those in another class, that is, it is a measure of dependence or clustering. The effect of clustering on variance estimates is well known (e.g., Goldstein, 1995; Kreft & De Leeuw, 1998) and simulation studies demonstrate that even small values of intraclass correlation are associated with increased probability of a Type I error, with larger sample sizes within classes associated with larger increases in the actual significance level for a given nominal level (Barcikowski, 1981).

In addition to potentially correlated observations, REACH data had a hierarchical structure (i.e., participants were randomly assigned to interventions that were nested within sites), and effects may have differed across sites. To account for within-site correlated observations and for the potentially different effects of independent variables on outcome across sites, multilevel models (Goldstein, 1995; Kreft & De Leeuw, 1998; Snijders & Bosker, 1999) were used. These models allow the coefficients of regression models fit for the first level of the hierarchy (caregiver/care-recipient dyad) to vary across second-level units (sites) by considering each coefficient to consist of an average value (fixed effect) and variability around that value (random effect). As such, if the effect of an intervention component varied across the sites, then these models would provide an estimate of the average effect across the sites (i.e., the fixed effect) and an estimate of the variability of the effect among the sites (i.e., the random effect). Continuous independent variables were centered about their means. Quadratic terms were included in models to test for nonlinearity in the relationship. Models were fit separately for 6-month RMBPC and 6-month CES–D scores. For each outcome, after the main effects model was fit, the two-way interaction terms were tested and retained in the model if statistically significant. Then, each intervention component was examined individually to determine whether it was independently and significantly associated with the outcome, adjusting for the base model. We also created principal components of the individual intervention elements to use as independent variables because the individual intervention components were highly correlated with each other. Similar to the process with the individual intervention components, the principal components were examined individually, and as a group, to determine whether any were independently and significantly associated with the outcome, adjusting for the base model. Residuals of the final models were examined and model fits compared using Akaike’s information criterion.

Results

For descriptive purposes, average values of the integrated intervention measures were calculated and are presented for the RMBPC cohort in Table 1. The average total intervention times ranged from less than 1 hr for the three usual care conditions to more than 20 hr in the Palo Alto coping with caregiving intervention and the Miami family-based structural multisystems in-home intervention plus computer telephone integration system (FSMII + CTIS) intervention. All of the interventions were multicomponent, although the usual care conditions targeted only caregiver knowledge and care-recipient knowledge. At the other extreme, the Memphis enhanced intervention targeted all 12 attributes, and the Birmingham skills training and Miami FSMII + CTIS targeted 11 of the 12 attributes. Caregiver and care-recipient knowledge were targeted by all interventions, but the amount of time spent on these domain-entities during the first 6 months of intervention varied widely across interventions. The Boston usual care intervention spent less than 2 min, on average, targeting caregiver knowledge, compared to the Palo Alto enhanced intervention, which averaged more than 3 hr on this attribute. Care-recipient knowledge was targeted for an average of less than 1 min in the Philadelphia usual care condition compared to almost 3 hr in the Palo Alto enhanced intervention. Cells with no time for an intervention (e.g., environmental skills for the Birmingham skills training) represent domain-entities not targeted by that intervention. Note that the active intervention in Boston (telephone-linked computer, or TLC) and one of the active interventions at Memphis (behavior) provided less intervention, on average, than the control conditions at either Birmingham or Miami (MSC), and that the enhanced intervention group at Memphis had comparable intervention time to the MSC group at Birmingham. Although the cohort used for the depression outcome is somewhat different, the distributions of intervention measures were similar.

Table 1.

Entity-Domain Weights Multiplied by Average Intervention Time Between Baseline and 6-Month Follow-Up for RMBPC Cohort (n = 910)

Assignment	Average total intervention time (hr)	CGK	CGS	CGB	CGA	CRK	CRS	CRB	CRA	EVK	EVS	EVB	EVA
Birmingham
MSC	1.97	0.40	0	0	0.69	0.50	0.14	0	0	0.24	0	0	0
Skills training	8.72	0.56	1.45	0.82	0.37	0.97	2.17	1.21	0.33	0.31	0	0.41	0.13
Boston
Usual care	0.21	0.03	0	0	0	0.18	0	0	0	0	0	0	0
TLC	1.23	0.34	0.23	0	0	0.48	0	0	0.12	0.06	0	0	0
Memphis
Usual care	0.37	0.10	0	0	0	0.27	0	0	0	0	0	0	0
Behavior	1.66	0.15	0	0	0	0.47	0.30	0.18	0.11	0.21	0.13	0.07	0.04
Enhanced	1.98	0.26	0.20	0.12	0.23	0.36	0.22	0.14	0.10	0.16	0.08	0.05	0.05
Miami
MSC	1.55	0.27	0	0	0.61	0.50	0	0	0	0.18	0	0	0
FSMII	13.81	0.81	1.56	0.90	1.66	1.04	0	0	0	1.12	3.29	1.08	2.35
FSMII + CTIS	20.80	1.66	2.14	1.04	2.81	1.83	0	0.42	0.46	1.60	4.26	1.25	3.35
Palo Alto
MSC	3.14	0.65	0	0	1.28	0.85	0	0	0	0.37	0	0	0
Coping	20.93	2.22	5.02	3.14	6.68	1.55	0	0.96	0	0.69	0	0.67	0
Enhanced	18.30	3.31	2.23	0	8.89	2.85	0	0	0	0.99	0	0	0
Philadelphia
Usual care	0.06	0.05	0	0	0	0.01	0	0	0	0	0	0	0
ESP	8.21	0.53	1.26	0.62	0	0.50	0	0.85	0	0.89	1.76	1.80	0

Note. CG = caregiver; CR = care recipient; EV = environment; K = knowledge; B = behavior; S = skills; A = affect (e.g., CGK = caregiver knowledge; CRS = care-recipient skills); MSC = minimal support condition; TLC = telephone-linked computer; FSMII = family-based structural multisystems in-home intervention; CTIS = computer telephone integration system; ESP = environmental skill building program.

Cronbach’s alpha for the RMBPC at baseline was .87. At baseline, the 910 caregivers with data for RMBPC at both baseline and 6 months reported that the care recipients had an average of 10.1 (of 24) problem behaviors and reported an average Burden score of 16.1 (of 96; range = 0–68). This score is roughly equivalent to a caregiver responding extremely bothered to 4 behaviors, or moderately bothered to 8 behaviors. The ICC was .021 using an empty model (i.e., a model without any explanatory variables). In this model, the baseline RMBPC for each individual is expressed as an overall mean plus second-level (site) deviations from that mean and a random error associated with the individual first-level (dyad) observations (Snijders & Bosker, 1999). By not taking clustering into account when analyzing baseline RMBPC, the simulation results presented by Barcikowski (1981) demonstrate that this seemingly small value of ICC would be associated with a more than tripling of the nominal significance level of .05, given the 100 to 245 baseline observations at each site in the REACH study.

Cronbach’s alpha for the RMBPC at 6 months was .87, the same value found at baseline. The average RMBPC burden at 6 months was 13.7 (approximately the score obtained if responding extremely bothered by 3 behaviors and moderately bothered by 1 behavior, or moderately bothered by 7 behaviors), ranging from 12.1 in Boston to 14.3 in Birmingham. The two-level model for 6-month RMBPC with baseline RMBPC, caregiver sex, caregiver/care-recipient relationship, and caregiver ethnic identity at the first level included only a random effect for the second-level unit (site) because the procedure did not converge when including both fixed and random effects for site. This will occur when a model is overparameterized. Having only random, and not fixed, effects for site means that site effects were described by an overall mean and random variability around that mean, after adjusting for baseline RMBPC, caregiver sex, caregiver–care-recipient relationship, and caregiver ethnic identity. The site-specific random effect estimates in Table 2 were obtained by assuming that the six REACH sites were randomly sampled from a population of sites and that at each site the mean RMBPC at 6 months deviated from the overall mean by a normally distributed residual with a mean of zero and a site-specific variance. The estimates were obtained using residual (restricted) maximum likelihood (Brown & Prescott, 1999). As shown in Table 2, the variability around the mean 6-month RMBPC at each site was small compared to the overall intercept, and the residual (or unexplained) ICC calculated after fitting the model with explanatory variables was only .001. Burden at 6 months was positively associated with baseline burden (p <.0001), and spouse caregivers had higher 6-month RMBPC scores than did caregivers with a different relationship to the care recipient (typically children). Significant interaction terms indicated that Hispanic caregivers with higher baseline scores (p =.008) and husbands (p =.016) had lower burden scores than would be expected by summing the corresponding main effects.

Table 2.

Model for Revised Memory and Behavior Problems Checklist (RMBPC) Burden Scores at 6 Months (n = 909^a).

Variable	Parameter estimate	SE	p
Overall intercept	13.77	0.62
Birmingham	−0.14	0.33
Boston	0.01	0.33
Memphis	0.21	0.32
Miami	0.009	0.32
Palo Alto	−0.14	0.32
Philadelphia	0.04	0.32
Baseline RMBPC	0.58	0.03	<.0001
Male caregiver	1.19	1.39	.390
Black/African American	−1.33	0.82	.107
Hispanic/Latino	−0.76	0.88	.389
Spouse	1.54	0.75	.040
Baseline × Hispanic	−0.16	0.06	.008
Male × Spouse	−4.20	1.74	.016

^aComplete data were not available for 1 participant.

None of the intervention components was significantly related to 6-month RMBPC after adjusting for baseline RMBPC, caregiver sex, caregiver/care-recipient relationship, caregiver ethnic identity, and the interactions of baseline RMBPC with ethnic identity and caregiver sex with caregiver/care-recipient relationship (see Table 3).

Table 3.

Relationship of Each Intervention Component With RMBPC Burden Score at 6 Months, Adjusting for Baseline RMBPC, Caregiver Sex, Ethnic Identity, and Relationship to Care Recipient and Interactions Between Baseline RMBPC and Ethnic Identity and Caregiver Sex and Relationship (n = 909^a).

Entity and domain	Parameter estimate (per hr)b	SE	p
Care recipient
Behavior	−1.15	0.76	.132
Knowledge	−0.45	0.39	.240
Skills	−0.71	0.67	.292
Affect	−2.16	2.33	.354
Caregiver
Behavior	−0.37	0.36	.303
Knowledge	−0.37	0.32	.252
Skills	−0.31	0.22	.170
Affect	−0.11	0.11	.340
Environment
Behavior	−0.36	0.50	.475
Knowledge	−0.48	0.62	.439
Skills	−0.03	0.23	.885
Affect	−0.0009	0.32	.998

Note. RMBPC = Revised Memory and Behavior Problem checklist.

^aComplete data were not available for 1 participant.

^bA negative parameter estimate represents the amount an RMBPC score at 6 months is expected to be lowered for every 1 hr of intervention targeting the particular entity-domain after adjusting for baseline RMBPC, caregiver sex, ethnic identity, relationship to care recipient, Baseline Score × Hispanic and Male × Spouse interaction terms.

Due to high intercorrelations among the intervention components, we created principal components of the individual intervention measures. None of the principal components was significantly related to 6-month RMBPC after adjusting for baseline RMBPC, caregiver sex, caregiver/care-recipient relationship, and caregiver ethnic identity, and the interaction terms defined previously.

Cronbach’s alpha at baseline for the CES–D was .90. At baseline, the average CES–D among the 1,087 caregivers with CES–D at both baseline and 6 months was 15.1 (range = 0–54). A score of 16 or higher has been interpreted as being at risk for clinical depression (Radloff & Teri, 1986) and 39.7% of the REACH caregivers were in this category. The ICC from the empty model (no covariates) for baseline CES–D was .036, comparable to, and larger than, the value for baseline RMBPC. Thus, the effect of not accounting for clustering in analyses would be to increase the actual Type I error rate from the nominal level of .05 to between .17 and .43 (Barcikowski, 1981).

Cronbach’s alpha at 6 months was the same as for baseline (i.e., .90). The average value of the CES–D at 6 months was 14.6 (range = 12.8 in Memphis to 16.6 in Miami). As with the RMBPC, the two-level model for 6-month CES–D included baseline CES–D, caregiver sex, caregiver/care-recipient relationship, and caregiver ethnicity and only a random effect for site, as the model with both fixed and random effects for site was overparameterized. There were no significant interaction terms. The residual ICC estimated from this model was only .0008. After adjustment for baseline CES–D, caregiver sex, caregiver/care-recipient relationship, and caregiver ethnicity, the intervention measures were introduced one at a time as potential predictors. Care-recipient behavior, caregiver behavior, caregiver skills, and caregiver affect were all significantly associated with CES–D (see Table 4). The significance level associated with caregiver behavior was the smallest, so that intervention component was adjusted for, and each of the other intervention components was examined for a significant, independent relationship with CES–D (none was found; see Table 5).

Table 4.

Relationship of Each Intervention Component With CES–D Score at 6 Months, Adjusting for Baseline CES–D, Caregiver Sex, Ethnic Identity, and Relationship to Care Recipient (n = 1,086^a)

Entity and domain	Parameter estimate (per hr)b	SE	p
Care recipient
Behavior	−1.19	0.58	.038
Knowledge	−0.35	0.29	.233
Skills	−0.14	0.53	.795
Affect	−2.67	1.78	.134
Caregiver
Behavior	−0.89	0.27	.0008
Knowledge	−0.40	0.24	.099
Skills	−0.53	0.16	.001
Affect	−0.18	0.08	.033
Environment
Behavior	−0.38	0.40	.333
Knowledge	−0.44	0.48	.364
Skills	−0.13	0.19	.500
Affect	−0.27	0.26	.310

Note. CES–D = Center for Epidemiological Studies—Depression Scale.

^aComplete data were not available for 1 participant

^bA negative parameter estimate represents the amount a CES–D score at 6 months is expected to be lowered for every 1 hr of intervention targeting the caregiver behavior entity-domain after adjusting for baseline CES–D, caregiver sex, ethnic identity, and relationship to care recipient.

Table 5.

Relationship of Each Intervention Component With CES–D Score at 6 Months, Adjusting for Baseline CES–D, Caregiver Sex, Ethnic Identity, and Relationship to Care Recipient and Caregiver Behavior (n = 1,086^a)

Entity and domain	Parameter estimate (per hr)b	SE	p
Care recipient
Behavior	0.31	0.81	.705
Knowledge	−0.21	0.29	.466
Skills	−0.07	0.48	.891
Affect	−1.62	1.71	.344
Caregiver
Knowledge	−0.17	0.25	.503
Skills	−0.15	0.36	.677
Affect	−0.08	0.09	.385
Environment
Behavior	0.58	0.41	.164
Knowledge	0.12	0.49	.815
Skills	0.10	0.17	.574
Affect	−0.02	0.24	.925

Note. CES–D = Center for Epidemiological Center—Depression Scale.

^aComplete data were not available for 1 participant.

^bA negative parameter estimate represents the amount a CES–D score at 6 months is expected to be lowered for every 1 hr of intervention targeting each entity-domain after adjusting for baseline CES–D, caregiver sex, ethnic identity, and relationship to care recipient, and caregiver behavior.

The final model indicated that more baseline depressive symptomatology was associated with greater depressive symptomatology at 6 months (see Table 6). In addition, Black/African American caregivers had less depressive symptomatology at follow-up than either White/Caucasian or Hispanic caregivers. The coefficient for caregiver behavior indicated that, over a 6-month period, each additional hour of intervention targeting caregiver behavior would be expected to result in a 6-month CES–D score 0.89 points lower than would be expected without that additional hour (p =.0008). Another way of expressing this result is that a 3-point reduction in the 6-month CES–D was predicted for every 3 hr 22 min targeting caregiver behavior over a 6-month period. The slope of the linear relationship between caregiver behavior and 6-month depressive symptomatology did not vary across sites after adjusting for the baseline CES–D, caregiver sex, caregiver/care-recipient relationship, and caregiver ethnic identity, so caregiver behavior was a fixed effect in this model. There were no significant (p <.05) interactions between time spent targeting caregiver behavior and either caregiver sex, caregiver/care-recipient relationship, caregiver ethnic identity, or site. Also, there were no other significant cross-level interactions.

Table 6.

Model for Center for Epidemiological Studies—Depression Scale (CES–D) Scores at 6 Months (n = 1,086^a)

Variable	Parameter estimate	SE	p
Overall intercept	14.45	0.45
Birmingham	−0.05	0.22
Boston	−0.01	0.22
Memphis	0.01	0.22
Miami	−0.0008	0.22
Palo Alto	−0.09	0.22
Philadelphia	0.14	0.22
Baseline CES–D	0.68	0.02	<.0001
Male caregiver	0.05	0.63	.940
Black/African American	−1.19	0.61	.035
Hispanic/Latino	0.80	0.65	.223
Spouse	0.65	0.50	.196
Caregiver–behavior time (per hr)b	−0.89	0.27	.0008

^aComplete data were not available for 1 participant.

^bA negative parameter estimate of caregiver–behavior time represents the amount a CES–D score at 6 months is expected to be lowered for every 1 hr of intervention targeting the caregiver behavior entity-domain after adjusting for baseline score, caregiver sex, caregiver/care-recipient relationship, and caregiver ethnic identity.

It should be noted that the relationship between intervention time targeting caregiver behavior and CES–D score cannot be linear across the full range of potential time targeting caregiver behavior, and there must be a threshold as it is not possible to improve the CES–D score without bound. Also, it is conceivable that too much time spent in an intervention reduces time providing care and may actually increase depressive symptoms in caregivers. However, within the range of intervention times in this study there was no evidence of nonlinearity.

To assess the relationship between pairs of the (n = 12) intervention components, we calculated the n(n −1)/2 = 12(11)/2 = 66 Pearson correlation coefficients. The correlation coefficients were at least .50 for 22 of the 66 pairs of intervention components, with 6 pairs having values of at least .80, so principal components were examined as independent variables. Because the principal-components analysis was performed due to high intercorrelations among the intervention components not in order to reduce dimensionality, no rotation was performed and the original structure of the interventions was maintained. Table 7 presents the coefficients of the linear combinations of the intervention components creating the principal components (i.e., the eigenvector matrix). Correlations between the ith principal component and the jth intervention component were computed by multiplying the coefficient a_ij shown in Table 7 by the standard deviation of principal component i and dividing by the standard deviation of intervention component j. The first principal component accounted for nearly half of the variability in the intervention components and the correlation coefficients between this principal component and all intervention components except care-recipient skills and care-recipient affect were at least .50. The fifth principal component was the only one significantly associated with the 6-month CES–D (parameter estimate = 0.58, SE = 0.25, p =.022), but this principal component only accounted for 4% of the variability in the intervention components. It is noteworthy that the largest coefficient for this principal component belongs to caregiver behavior, the same intervention component identified as being significantly associated with 6-month CES–D in Table 4. However, the correlation between caregiver behavior and the principal component was only .39.

Table 7.

Proportion of Variance Explained by Each Principal Component of the Entity-Domain Intervention Components and Elements of the Eigenvector Matrix 6-Month CES–D Cohort (n = 1,087)

		Eigenvector elements
Principal component	Proportion of variance explaineda	CRK	CRS	CRB	CRA	CGK	CGS	CGB	CGA	EVK	EVS	EVB	EVA
1	.47	.34	.01	.22	.18	.33	.37	.30	.29	.39	.28	.28	.27
2	.21	−.27	.15	.12	.35	−.36	−.20	−.05	−.44	.10	.41	.33	.36
3	.16	−.11	.54	.57	.13	.08	.19	.30	−.07	−.20	−.28	.03	−.30
4	.11	.30	.48	−.16	.53	.17	−.18	−.36	.11	.02	−.05	−.38	.14
5	.04	.19	.19	.21	−.29	.13	−.25	−.55	−.005	.27	.04	.47	−.36
6	.02	.01	.61	−.25	−.63	−.09	.06	.13	.03	.03	.17	−.09	.33
7	.001	−.55	.06	.06	.03	.24	.31	−.41	.40	−.37	.17	.13	.16
8	.0006	−.38	−.06	.37	−.08	−.11	−.39	.04	.40	.49	−.09	−.33	.15
9	.0002	.47	−.19	.48	−.17	−.32	−.09	−.18	.15	−.44	.08	−.09	.34
10	.0002	−.07	−.05	.23	−.14	.72	−.35	.13	−.39	−.20	−.03	−.08	.25
11	<.0001	.03	.05	−.01	.04	.05	−.31	.25	.21	−.22	.71	−.16	−.46
12	<.0001	.02	.08	−.24	.12	−.03	−.47	.29	.40	−.25	−.31	.52	.14

Note. CES–D = Center for Epidemiological Studies—Depression Scale; CR = care recipient; CG = caregiver; EV = environment; K = knowledge; B = behavior; S = skills; A = affect (e.g., CGK = caregiver knowledge; CRS = care-recipient skills). Boldface coefficients represent correlations of at least .50 between the principal component and the entity-domain intervention component.

^aAll values are greater than zero.

Discussion

Like many other public health problems, caregiving poses challenges that can result in multiple problems including compromised physical and psychological well-being and disrupted family relations. Addressing the complex problem of caregiving typically requires testing complicated multidimensional interventions that might include elements of respite care, support, therapy, and information and referral delivered in a variety of modalities and contexts. Although multicomponent interventions may be systematically decomposed in subsequent studies, the cost of evaluating separately the efficacy of individual components of a complex intervention through individual studies is often prohibitive. An alternative strategy is to articulate clearly a detailed conceptual framework and obtain data from multiple studies. In REACH, this was facilitated by common design elements and a central coordinating center.

This analysis demonstrates a new methodology to understand how the various REACH interventions aided family caregivers of people with ADRD. The method allows information to be combined across different interventions by decomposing them to describe them according to a set of common measures. The REACH interventions were described by the relative emphasis placed on targeting particular entities (caregiver, care recipient, sociophysical environment) and functional domains (knowledge, skills, behavior, affect).

This resulted in some meaningful findings such as the entity-domain most heavily targeted by an intervention. However, when using only the weights, there are limitations with respect to comparing interventions. This is exemplified by the extreme example in which an intervention targets only a single entity-domain. By definition, that single entity-domain receives a weight of 1. Another intervention targeting the same entity-domain and one or more others would necessarily receive a weight of less than 1 for the common entity-domain. Thus, there may be differences in weights between interventions as a function of the number of attributes targeted.

To overcome this limitation, we integrated the entity-domain weights with the intervention intensity as measured by intervention time. There are several possible ways to achieve this integration. One is to include in a multilevel model the weights for each intervention component as intervention-level variables and the total intervention time as a dyad-level variable. Note that one must be careful using this approach because the weights, by definition, sum to 1. Hence, the set of 12 weights is collinear and cannot all be used in a model. A solution to this problem is to reparameterize the model, for example, by including 11 of the terms (rewriting the other term as 1 minus the sum of the other 11). Another way to integrate dose and emphasis is to combine the variables in some way. This was our approach as there were strong correlations between total intervention time and intervention weights in REACH. We created a composite variable by multiplying the total intervention time by the intervention component weights, yielding measures corresponding to the amount of intervention time allocated to each of the 12 entity-domain attributes. This was a caregiver/care-recipient dyad-level variable because the total intervention time varied across caregivers, even within a site. Note that collinearity occurs when all participants receive the same amount of intervention. Although this was not an issue with REACH, it could be when this methodology is applied to other intervention studies. If so, some other way to integrate dose and emphasis, or reparameterizing the model must be considered. In future caregiving intervention studies, it may actually be possible to measure the amount of time an intervention targets an individual entity-domain, but these data were not available in REACH. Future work should also consider ways to include delivery style and method in the measure of intensity as outcomes may be related to these aspects of intervention (Sörensen et al., 2002).

Although meta-analysis demonstrated that active interventions were significantly more effective than the control conditions with respect to burden at 6 months (Gitlin et al., 2003), no particular intervention component was significantly associated with burden independently of the dyad characteristics examined. This may be because the intervention effect, although statistically significant, was small. The lack of a finding may also be a result of not considering some important aspects of the intervention when measuring the components, for example, placebo effects, perception of help by participating, or simply having contact with someone on the research team. It is also possible that an important moderator or mediator was not accounted for in the analysis, although many of the moderators considered to be important in a recently published meta-analysis of caregiver interventions (Sörensen et al., 2002) were included, (i.e., caregiver age, caregiver/care-recipient relationship, caregiver sex, and the baseline value of the outcome). It is important to note that, even if mediators were properly accounted for, it is possible that the interventions did not adequately affect them. So, for example, even if social support was a mediator and we measured it properly, it is possible that the interventions did not alter social support sufficiently to have an impact on burden.

On the other hand, the results for the measure of depression enable us to go beyond the interpretation possible from the series of site-specific analyses and from the meta-analysis reported in Gitlin et al. (2003). The analyses reported here suggest that interventions should emphasize caregiver behavior in order to reduce caregiver depressive symptomatology. This finding may help explain why many caregiver interventions fail to achieve significant outcomes. Virtually all caregiver interventions emphasize enhancing caregiver knowledge about caregiving and dementia, and some attempt to teach the caregiver general problem-solving skills, but relatively few go so far as to validate the caregiver’s ability to enact effective behavioral strategies. These findings have implications for designing interventions to reduce depressive symptomatology.

Although attrition in REACH was small, those who dropped out by 6 months tended to be more distressed at baseline than those who remained in the study. It was also the case that, in general, caregivers with greater baseline distress (either burden or depressive symptomatology) received more intervention than those with lower levels of distress. Hence, it is possible that those who dropped out became overwhelmed by the need to spend time in the study, and had they received the full dose of intervention, would not have been helped as much as those who completed the interventions. If so, we would have overestimated the benefit of targeting care-recipient behavior on caregiver depression. On the other hand, it is also possible that those who dropped out would, by choice, have received less intervention than others. If those with higher CES–D scores at baseline had greater depressive symptomatology at follow-up, having received less intervention targeting caregiver behavior, then we would have underestimated the benefit.

This article describes the first application of a novel approach for decomposing multiple complex interventions to identify and measure individual intervention components associated with improved outcome. The approach yields measures of individual aspects of interventions that can be used in appropriate analyses to combine information from diverse interventions and to determine the relative contribution of each component, or groups of components, to the success of the intervention.

The appropriate analytical method is determined, in part, by the study design. For example, analysis of a single-center, randomized study might employ generalized least squares regression models because there is only one site and participants should be comparable in the different treatment groups. However, the REACH study was more complex than even multicenter clinical trials that test a single intervention because REACH was, in essence, six randomized intervention studies. Hence, an alternative analytical method was used. The REACH study design fit into a hierarchy with participants grouped within sites, so multilevel models were a natural choice for analysis. The amount of clustering was small due to some design features of REACH (i.e., the sites adopted standard methods such as inclusion/exclusion criteria, applied common definitions and measures, and there was central training and certification for interviewers). Also, although the interventions were diverse, they were all grounded in a common theory.

One caveat, if using this approach, is the minimum number of second-level units needed before using a multilevel analytical approach. With few second-level units, estimation procedures can lead to negative estimates for variance components; fewer than 5 second-level units may preclude fitting random effects at that level (Brown & Prescott, 1999). Even more conservatively, it has been suggested that with fewer than 30 second-level units, caution should be exercised when interpreting results of significance tests (Sullivan, Dukes, & Losina, 1999). Regardless of the minimum number of second-level units needed, it is important to recognize that caution is required when interpreting results from a multilevel model when there are few second-level observations, regardless of the number of first-level observations.

The approach taken to identify intervention components associated with outcomes in this study should be viewed as exploratory. The multiple comparisons performed will lead to an inflated Type I error rate, incorrect confidence intervals, and may produce a model unduly influenced by small differences among independent variables, particularly if those variables are highly correlated. However, exploratory analysis is a potentially useful approach when generating hypotheses, as was intended here. There were no a priori reasons to hypothesize that any particular intervention component had a stronger association with outcome than any other. In fact, because all of the interventions were theory-based and the interventions all targeted more than one of the components, with most targeting many different components, it would be difficult to agree upon which was most important. The results of the exploratory analysis should serve only as a foundation for designing and testing new interventions, a strategy currently being pursued by the REACH research group, which is testing an intervention designed, in part, on the results of these analyses.

The methodological approach used here is not intended to supplant traditional methods for analyzing intervention research. Instead, it is intended to provide one more option for characterizing and analyzing interventions in the hope of enhancing our ability to understand, replicate, and extend effective interventions better.

This article reports an important step in an analytic process. It needs to be emphasized, however, that the ability to perform these analyses relies on the ability to decompose interventions, which in turn requires extensive information concerning the content, structure, and delivery methods of interventions. In general, the level of detail needed to implement this methodology is not available in the published literature. Considering the importance of replication in science, and of widespread implementation of successful interventions, better descriptions of interventions would be desirable in future publications.