Measuring burden in caregivers of people with multiple sclerosis: psychometric properties of the CSI questionnaire

Abstract

Background

Understanding caregiver strain may be crucial to determine which interventions are most needed to mitigate the negative impact of caring for people with multiple sclerosis (MS). The Caregiver Strain Index (CSI) is a brief self-assessment tool for measuring the caregivers’ perceived level of burden. Limited information is available on the psychometric performance of the CSI in MS.

Objective

The objective of this study was to assess the factor structure and construct validity of the CSI in MS.

Methods

A multicenter, cross-sectional study in adults with relapsing-remitting and primary-progressive MS (McDonald 2010 criteria) was conducted. A non-parametric item response theory (IRT) procedure, Mokken analysis, was conducted to assess the dimensional structure of the CSI. A parametric IRT model for dichotomous responses, Rasch model, was conducted to assess item characteristics. Discriminative validity was assessed comparing the distribution of its overall score between people with mild and moderate-severe disability according to the Expanded Disability Status Scale.

Results

A total of 72 MS caregivers were studied. The prevalence of a high level of strain was 23.6% (n=17). Internal reliability was high (Cronbach’s alpha =0.91). According to Mokken analysis, CSI represented a unidimensional construct of caregiver burden although two of the total 13 items (#1 and #13) could not be assigned to any factor by an automatic item selection procedure. Without these items, the scalability moved from a weak (H_i =0.37) to a medium scale (H_i =0.44). However, the item characteristic curve of the Rasch model showed a range of appropriate difficulty and the item and person parameters showed good fit (Andersen likelihood ratio test =18.40, df =11; P-value =0.07; all item values for the infit). The CSI score showed a good discriminative validity between the levels of disability of the care recipient.

Conclusion

The CSI questionnaire shows appropriate psychometric characteristics being a useful instrument to assess different aspects of burden in MS caregivers in clinical practice.

Introduction

Multiple sclerosis (MS) is one of the most common causes of neurological disability in young adults and its prevalence is increasing throughout Europe.^1–3 Given that the initial symptoms of MS typically appear between 20 and 40 years of age, it is understandable that people with MS (pwMS) have major consequences in their employment and that the disease may cause them to experience difficulties in their capacity to actively maintain work.^1,2,4

Thirty percent of pwMS usually require caregiving due to disability and >80% of this care is provided by informal caregivers, mainly spouses or other relatives.^5–11 Fatigue and impaired mobility are the MS symptoms most often perceived as stressful by caregivers, followed by bladder dysfunction, cognitive impairment, and depression.^10–13 More hours per week spent providing assistance and greater restriction on the caregivers’ ability to perform daily activities are additional predictors of burden.⁹ Caregivers of people with higher disability levels are more likely to suffer moderate to severe impact on their own health.^11,14 Care partners of people with primary progressive MS reports greater perceived burden than partners caring for people with relapsing forms of MS.¹² Patient perception of stigma is associated with greater informal caregivers’ burden.¹⁵ Female caregivers have higher level of burden and stress as well as more medication use for anxiety and mood disorders.¹² The unpredictable natural history of MS may also exacerbate caregiver stress.⁶

Understanding caregiver burden may be crucial for carrying out specific intervention strategies to reduce burden and improve caregiver health.¹⁶ Several validated questionnaires for assessing this construct have been used in different neurological disorders, such as the Zarit Burden Interview, the Care Related Quality of Life, the Caregiver Strain Index (CSI), and the Burden Scale for Family Caregivers.^6,17 The CSI is a brief self-assessment tool for measuring the caregivers’ perceived level of burden and includes employment, financial, physical, social, and time domains.¹⁸ It was developed by Robinson studying a sample of spouses, family, friends, and neighbors who provided varying degrees of care to recently hospitalized hip surgery and cardiac elderly people. The CSI has shown high internal consistency, validity, and unidimensional structure and is one of the most frequently used questionnaires to measure caregiver burden.¹⁷ However, limited information is available on the psychometric performance of the CSI in pwMS.

Our aim in this study was to assess the dimensional structure and construct validity of the CSI in the management of MS.

Methods

A non-interventional, cross-sectional study in adult subjects with relapsing-remitting and primary-progressive MS (McDonald 2010 criteria) was conducted in 19 MS units throughout Spain (W-IMPACT study).^19,20 The aim of this study was to assess the psychometric properties of the Spanish version of the Multiple Sclerosis Working Difficulties Questionnaire. We performed a post hoc analysis using data from the aforementioned study in order to assess the psychometric performance of the CSI. Written informed consent was obtained from all subjects. This study was conducted in accordance with the provisions of the International Conference on Harmonization Guidelines for Good Clinical Practice and the Declaration of Helsinki. The study was approved by the institutional review board of the Fundación Jiménez Díaz (Madrid, Spain).

On clinical appointment, accompanying caregivers completed the CSI questionnaire. Each of its 13 items is a statement that can be responded to with either “no” (0) or “yes” (1). A total score of 7 or higher indicates a high level of caregiver burden.¹⁸ The Spanish validated version of the instrument was used.²¹

Statistical analyses

For continuous data, descriptive statistics were expressed as mean and SD, or median and interquartile range (IQR). For categorical data, descriptive statistics were expressed as frequencies and percentages.

A non-parametric item response theory (IRT) procedure, Mokken analysis, was conducted to assess the dimensional structure of the CSI.²² All items were required to have a scalability coefficient (H_i) ≥0.30 and the overall scale a scalability coefficient (H) ≥0.30. Mokken suggested the following thresholds for interpreting scalability coefficients: weak scale 0.3 ≤ H < 0.4; medium scale 0.4 ≤ H < 0.5; and strong scale H ≥ 0.5.²² Cronbach’s α reliability was estimated from the tetrachoric correlations among CSI items.

A parametric IRT model for dichotomous responses, Rasch model (RM), was conducted to assess item characteristics.²³ The RM accounts for both the difficulty of tasks and the abilities of subjects by modeling the relationship between the latent trait and the items used to measure it. Values of item location below 0 represent easy items; values around 0 represent average difficulty; and values above 0 represent more difficult items. Ideally, a questionnaire should have items covering the full range of difficulty, to ensure complete assessment of the latent dimension it is supposed to measure. The fit of the RM was assessed by the infit and outfit mean-square statistics and the Andersen likelihood ratio (LR) test. The infit and outfit mean-square statistics assess item and person fit. For both the measures, the expected value is 1, and as a rule of thumb, items showing values >2 distort or degrade the measurement system; items with values between 1.5 and 2 are unproductive for the construction of measurement but not degrading; items with values between 0.5 and 1.5 are items productive for measurement; and items with values <0.5 are items less productive for measurement but not degrading. The infit and outfit mean-square values provide some evidence of construct validity. The Andersen LR test evaluates the gain in likelihood based on separate estimation of item parameters in score groups, against single estimates based on the total sample.²⁴ The adequacy of the scale was also assessed with a person-item or Wright map. The person-item map displays the location (difficulty) of item parameters as well as the distribution of person parameters along the latent dimension on the same logit scale. These are useful to compare the range and position of the item measure distribution to the range and position of the person measure distribution. Items should ideally be located along the whole scale to meaningfully measure the ability of all persons.

Finally, we assessed the discriminative validity of the CSI comparing the distribution of its overall score between MS people with mild and moderate to severe disability according to the Expanded Disability Status Scale-EDSS (EDSS scores ≤4.5 and ≥5.0, respectively). We used the raw and standardized mean difference (SMD) as effect sizes, and the value for the area under the curve (AUC) from a receiver operating characteristic analysis. We considered SMDs of 0.20, 0.50, and >0.80 to reflect small, moderate, and large effect sizes, respectively.²⁵ The AUC value is interpreted as the probability to correctly discriminate between disability severity based on the total score of the CSI scale. A value of 0.5 for the AUC implies no discrimination beyond chance whereas a value of 1 implies a perfect discrimination.²⁶

We performed the statistical analyses with the program R v3.4.4 with the libraries “mokken,” “eRm,” “meta,” and “pROC.”^27–31

Results

A total of 72 caregivers from a sample of 201 pwMS were studied. The mean total CSI score was 3.9±3.4. The prevalence of a high level of strain was 23.6% (n=17). Table 1 shows the demographic and clinical characteristics of the pwMS.

Table 1.

Sociodemographic and clinical characteristics of the pwMS cared by the study participants (n=72)

Age years, mean (SD)	44.9 (11.7)
Gender female, n (%)	43 (59.7)
Type of MS, n (%)
Relapsing-remitting	54 (75)
Primary progressive	18 (25)
EDSS, median (IQR)	3.0 (3.5)
EDSS score, n (%)
≤4.5	53 (73.6)
Disease-modifying treatments for MS, n (%)
Yes	55 (76.4)

Abbreviations: EDSS, Expanded Disability Status Scale; IQR, interquartil range; MS, multiple sclerosis; pwMS, people with MS.

Internal reliability

Cronbach’s α reliability was 0.91.

Non-parametric (Mokken) IRT

Automated item selection procedures suggest most items represented by a single underlying trait of caregiver burden. Most items were well fitting for the unidimensional scale with the exception of items #1 (“sleep is disturbed”) and #13 (“feeling completely overwhelmed”). These items also had the lowest values for item-test correlation. Most items had scalability indices over 0.30, except for items #1 (H_i =0.18) and #13 (H_i =0.21). The scalability for the overall scale was 0.37 indicative of a weak scale according to Mokken’s criteria. This value increased to 0.44 (medium scale) when items #1 and #13 were excluded from the analysis. Table 2 shows the endorsement frequencies and Mokken scalability coefficients.

Table 2.

Endorsement frequencies, Mokken scability, and the fit of Rasch model for the 13 items of the Caregiver Strain Index

Item	Item endorsement (yes) N (%)	Item-test correlation	Hi	Item location	Infit MSQ	Outfit MSQ
1. Sleep disturbed	19 (26.4)	0.45	0.18	0.24	1.38	1.47
2. Inconvenient	28 (38.9)	0.72	0.39	−0.59	1.02	0.96
3. Physical strain	21 (29.2)	0.81	0.42	0.04	0.80	0.84
4. Confining	19 (26.4)	0.79	0.39	0.24	0.88	0.95
5. Family adjustments	21 (29.2)	0.72	0.36	0.04	0.96	0.79
6. Changes in personal plans	16 (22.2)	0.79	0.42	0.55	0.84	0.59
7. Other demands on time	12 (16.7)	0.72	0.40	1.02	0.93	0.84
8. Emotional adjustments	22 (30.6)	0.74	0.37	−0.05	0.95	0.85
9. Behavior upsetting	16 (22.2)	0.70	0.35	0.55	0.95	1.20
10. Care recipient change upsetting	33 (45.8)	0.88	0.58	−1.02	0.68	0.56
11. Work adjustments	22 (30.6)	0.72	0.36	−0.05	0.98	0.93
12. Financial strain	39 (54.2)	0.63	0.42	−1.53	1.16	1.24
13. Feeling completely overwhelmed	16 (22.2)	0.48	0.21	0.55	1.24	1.90

Abbreviations: H_i, Loevinger’s scalability coefficient; MSQ, mean square.

Rasch analysis

The item characteristic curve (ICC) of the RM for the CSI showed a range of appropriate difficulty (Figure 1). The item and person parameters showed a good fit (Andersen LR test=18.40, df =11, P-value =0.07). Table 2 provides a summary of the location and fit of each item and Figure 2 shows the item-person map. Figure 2 is a combined graph that shows in its lower part the location of item parameters (difficulty) along the latent dimension, as well as the distribution of person parameters (ability or attainment level) along the latent dimension (upper part of the graph). Most items cluster along mean difficulty, with three easy items (#2, #10, and #12) and one difficult item (#7). Patients tend to cluster at the mean and lower end of the scale, indicating that most caregivers in this sample would endorse the corresponding items. Figure 2 also shows that some items have the same location scores (#8 and #11, #13 and #15, #1 and #4, and #6, #9 and #13). These items do not add scale information and, in principle, they could be either reworded or removed from the scale. However, no item showed significant misfit (only item #13 had an outfit MSQ >1.5 but <2, and thus represents an unproductive but not degrading item), and the person-item map shows an appreciable match between item and person scores, with the spread of item locations through the latent dimension showing evidence of construct validity (Figure 2).

Figure 1.

Item characteristic curve (ICC) of the Rasch model for the Caregiver Strain Index.

Figure 2.

Caregiver Strain Index item-person map.

Discriminative validity

The overall CSI score showed a good discriminative validity between the levels of disability of the care recipient. The mean CSI score was greater in the 19 caregivers of MS patients who had more disability (6.58±3.44) than in the 53 caregivers of MS patients who had lower levels of disability (3.0±2.92). The mean difference was 3.58 points (95% CI =1.85–5.31; P-value <0.0001) and the standardized mean difference was 1.12 (95% CI =0.60–1.71), compatible with a medium to very large difference. The AUC was 0.79 with values for 95% CI =0.67–0.91, reflecting a moderate to good discrimination.

Discussion

Caregivers play an important role in supporting individuals with MS. However, caregiving can be a stressful experience that often impacts on work performance and ability to maintain paid employment, as well as physical and psychological integrity.^5,6 MS caregivers have even greater activity impairment, more emergency room visits, and more hospitalizations than Alzheimer’s disease caregivers.⁷ The International MultiPlE Sclerosis Study found that caregiver costs related to productivity losses were double the cost reported by pwMS (€31,653 and €16,318, respectively).² In addition, the negative impact of MS for caregivers is often not considered as a major problem by health authorities.¹⁶ Therefore, identifying caregiver burden may be relevant in the context of a greater patient engagement and a more comprehensive management of the disease.^16,32

This study shows that the CSI scale is a worthwhile and feasible tool to assess burden in MS caregivers. The Cronbach’s α reliability estimate is large and compatible with values reported in other studies.¹⁷ The CSI scale fits a unidimensional latent trait of caregiver burden according to Mokken criteria, and shows appropriate ICCs and person-item map according to RM criteria. Also, as theoretically predicted, the CSI scale is able to discriminate burden among caregivers according to different levels of disability of care recipients.

Our study has several limitations. First, the study population included mostly a sample of caregivers of clinically stable pwMS with low physical disability. The results may thus not be generalizable to caregivers of less stable subjects or with more severe levels of disability. Second, a small sample size of 72 subjects may lead to imprecision in the psychometric estimates. Third, the lack of assessment of altogether socio demographic characteristics of the caregivers related to burden, such as gender.¹² Despite these limitations, the study also has several strengths. The sample was managed in 19 different MS units on a national level, which allows results to be generalized to community practice.

Conclusion

A better understanding of MS patient needs, including the patient and caregiver perspective, could have a great impact on quality of life and management of the disease. The CSI shows appropriate psychometric characteristics and fit a unidimensional latent trait of caregiver burden. It is therefore a useful instrument to assess different aspects of burden in caregivers of pwMS in clinical practice. Multidisciplinary teams managing MS may be able to utilize this questionnaire to determine which interventions are most needed and which dimension of caregiver burden to target.