Understanding health-related quality of life in caregivers of civilians and service members/veterans with traumatic brain injury: Establishing the reliability and validity of PROMIS Fatigue and Sleep Disturbance item banks

Abstract

Objective:

To examine the reliability and validity of Patient Reported Outcomes Measurement Information System (PROMIS) measures of sleep disturbance and fatigue in TBI caregivers and to determine the severity of fatigue and sleep disturbance in these caregivers.

Design:

Cross-sectional survey data collected through an online data capture platform.

Setting:

Four rehabilitation hospitals and Walter Reed National Military Medical Center.

Participants:

Caregivers (N=560) of civilians (n=344) and service member/veterans (n=216) with TBI.

Intervention:

Not Applicable

Main Outcome Measures:

PROMIS sleep and fatigue measures administered as both computerized adaptive tests (CATs) and 4-item short forms (SFs).

Results:

For both samples, floor and ceiling effects for the PROMIS measures were low (<11%), internal consistency was very good (all alphas ≥0.80), and test-retest reliability was acceptable (all r≥0.70 except for the fatigue CAT in the service member/veteran sample r=0.63). Convergent validity was supported by moderate correlations between the PROMIS and related measures. Discriminant validity was supported by low correlations between PROMIS measures and measures of dissimilar constructs. PROMIS scores indicated significantly worse sleep and fatigue for those caring for someone with high levels versus low levels of impairment.

Conclusions:

Findings support the reliability and validity of the PROMIS CAT and SF measures of sleep disturbance and fatigue in caregivers of civilians and service members/veterans with TBI.

Traumatic brain injury (TBI) is often associated with long-term health problems, including difficulties with physical, behavioral, and/or cognitive functioning, especially when injuries are moderate-to-severe in nature.^1–3 For persons with TBI that require support, this caregiving role can be demanding, time-consuming, emotionally draining, and can greatly impact quality of life.^4–18 While some research suggests that caregivers (in general) are at greater risk for health problems (compared to non-caregivers),^19,20 the nature of the health problems among caregivers of persons with TBI are less well-known.

Recent evidence in caregivers of civilians with TBI,^4,21,22 as well as in caregivers of service members/veterans (SMVs) with TBI,^23,24 suggests that sleep difficulties and fatigue are common. Approximately 60% of caregivers of people with TBI report changes in sleep pattern.^25,26 These findings are consistent with previous research suggesting that caregivers have worse sleep quality than non-caregivers.^27–30

The broader caregiver literature proposes a model for sleep that posits that caregivers’ health is negatively affected by not getting enough sleep (sleep deprivation) and fragmented sleep (sleep disruption).³¹ In this model, sleep deprivation and sleep disruption lead to sleep loss, which in turn results in worse physical, cognitive/behavioral (including fatigue), mental, and social health. While the precise cause of the sleep disturbance reported by caregivers of individuals with TBI is unknown, caregivers of individuals with dementia have been shown to have sleep problems secondary to the nocturnal problems among people with dementia.³² Since individuals with TBI have similar nocturnal problems, it is reasonable to speculate that similar issues are encountered by caregivers of people with TBI. Many individuals with TBI require round-the-clock care, which can deprive caregivers of sleep.³³ Sleep difficulties^25,26 and sleep disorders^34,35 are also much more common among individuals with TBI than the general population. These sleep problems (in the person with TBI), likely contribute to sleep disruption among their caregivers.

Caregivers of people with TBI may be especially vulnerable to problems with sleep and fatigue. Over one-third of caregivers of people with TBI have pre-existing (pre-injury) emotional distress^36,37 which may predispose them to sleep problems. Also, high levels of perceived stress,³⁸ fatigue, pain, anxiety, depression, onset of new health conditions, and increased rates of clinically significant emotional difficulties^{4,6–9,11–18,39–48} likely increase the risk that caregivers will experience sleep problems. In fact, rates of emotional distress in caregivers of people with TBI are significantly higher than in other clinical caregiving populations (e.g., caregivers of people with mobility impairments or developmental intellectual disabilities).^49,50

To better understand sleep and fatigue in caregivers of people with TBI, a well-validated measure is needed to provide a comprehensive assessment of the impact that sleep and fatigue have on the health-related quality of life (HRQOL) of these individuals. The purpose of this study was two-fold. First, we examined Patient-Reported Outcome Measurement Information System (PROMIS) Sleep Disturbance and PROMIS Fatigue in caregivers of civilians and SMVs with TBI to establish reliability and validity of these new measures. Second, we compared sample scores to population norms in order to determine the severity of fatigue and sleep disturbance in these caregivers.

Methods

Participants

We recruited 560 participants who were caregivers of civilians and SMVs (sample details provided in Carlozzi et al., this issue⁵¹). Eligible caregivers needed to be ≥18 years old and able to read and understand English. Caregivers were required to be providing some form of care (emotional support, physical assistance, or financial assistance) to an individual with a medically documented TBI who was >1 year post-injury (i.e., those caring for an individual still in the acute stage of recovery were not eligible^52–63). For the caregivers of civilians, the individual with TBI also had to meet TBI model system inclusion criteria for a complicated mild, moderate or severe TBI.⁶⁴ Data was collected in accordance with the local site institutional review boards and participants provided consent prior to participation. Approximately three weeks after the initial study visit, 145 caregivers completed a retest.

Measures

Two PROMIS measures were administered: Sleep Disturbance⁶⁵ (perceptions of sleep quality; perceived difficulties with getting to sleep or staying asleep; and adequacy of and satisfaction with sleep) and Fatigue⁶⁵ (a sense of exhaustion that decreases one's ability to carry out daily activities). PROMIS measures were administered as computer adaptive tests (CATs) plus standard 4-item Short Forms (SFs). Both the CAT and SFs of the PROMIS measures were scaled on a T-score metric (i.e., M=50, SD=10); a higher score represents worse physical health. The reliability and validity of both of these measures has been established in other clinical populations.^66–72 Clinical cut scores exist for patients with cancer^68,73 and rheumatic diseases.⁷⁴ Minimally important differences are available for patients undergoing anterior cervical spine surgery⁷⁰ or with lumbar degenerative disease.⁷¹ Responsiveness has been demonstrated for both measures for spinal surgery⁷¹ and cancer treatment,⁷⁵ and for acupuncture (Sleep Disturbance only).⁷⁶

Generic HRQOL was measured using the RAND-12 Health Status Inventory.⁷⁷ The RAND-12 is a 12-item self-report measure in the public domain that assesses physical health and mental health; mental health includes one fatigue/vitality item. Scores range from 0 (low health) to 100 (highest level of health); administration takes ~5 minutes. Previous studies of the RAND-12 have demonstrated satisfactory reliability and validity.^78–81

We measured caregiver burden using two scales: the modified Caregiver Appraisal Scale (CAS)⁸² and the Zarit Burden Interview (ZBI).⁸³ The 35-item CAS measures perceived burden, caregiver relationship satisfaction, caregiving ideology, and caregiving mastery,⁸⁴ with higher scores indicating better functioning. Reliability and validity of this measure have been established.^84–86 The ZBI⁸³ is a 22-item scale that measures caregiver burden. Total score ranges from 0 (low burden) to 88 (high burden). The ZBI offers literature-based evidence to support its reliability and validity;^86–89 cut-scores have been established for the ZBI for identifying caregivers at risk for depression.⁹⁰

The Mayo-Portland Adaptability Inventory-Fourth Edition (MPAI-4)⁹¹ is a 35-item measure used to assess caregiver impressions of the functional ability of the person with TBI. It is scored on a T-score metric (M=50, SD=10); higher scores indicate lower functioning and score >60 suggest severe limitations in functioning for significant other ratings.⁹² Administration time is ~5–10 minutes. Previous studies indicate that the MPAI-4 has good reliability and validity.^93–97

Data Analysis

Assessment of data skewness and kurtosis indicated that the data were normally distributed and appropriate for parametric analyses.

Reliability.

Two forms of reliability were calculated: internal consistency reliability and test-retest reliability. Cronbach’s alpha was calculated for all SFs and IRT-based internal consistency was calculated for all CATs to determine internal consistency reliability; minimal acceptable reliability was specified as ≥0.70.^98,99 Test-retest reliability was calculated for those with repeat testing; the minimum test-retest reliability criterion was ≥0.70.^98,99

Floor and Ceiling Effects.

Ceiling effects represent the percentage of participants who had the highest possible scores on a given scale or subscale. Similarly, floor effects represent the percentage of participants who had the lowest possible scores on a given scale or subscale. For the CATs, floor and ceiling effects were calculated by dividing the raw CAT scores by the number of items administered (a score of 1 was a “ceiling effect” and a score of “5” was a “floor effect” for CATs). Acceptable floor and ceiling rates were ≤20%.^100,101

Administration times.

To establish feasibility for the administration of these measures, we examined timing data for both CAT and SF versions of the PROMIS measures. Start and stop times for each item were recorded electronically.

Convergent and Discriminant Validity.

To evaluate convergent and discriminant validity of the PROMIS measures, we examined correlations between similar and dissimilar traits.¹⁰² Strong correlations (r >0.6) between scores from measures of the same trait were interpreted as being good evidence for convergent validity.¹⁰³ In contrast, weak correlations (r <0.3) between scores of different traits were interpreted as evidence for discriminant validity.¹⁰³

Known groups validity.

We divided the participants into two groups, based on their MPAI-4 derived assessment of the individual with TBI: T scores less than 60 were considered “high functioning” and T scores greater than or equal to 60 were considered “low functioning.” We chose this cutoff because T-scores above 60 suggest severe limitations in functioning.⁹² For each PROMIS measure, comparisons were then made between low and high functioning groups using independent sample t-tests. We hypothesized that caregivers of high functioning individuals would rate their own sleep/fatigue as being better than those who cared for low functioning individuals.

Impairment Rates.

We examined clinical impairment rates (participants whose scores were >1 SD worse than the PROMIS normative sample mean of 50) to determine if caregivers of individuals with TBI were at greater risk than the general population for physical health impairments. According to the normal curve, 16% of the scores are expected to fall 1 SD below the mean (i.e., representing being impaired); therefore, impairment rates exceeding 16% would be judged as having greater impairment than might be expected compared to demographically comparable peers.¹⁰⁴

Missing Data.

For participants who had a small amount of missing data (<10% of items), scores were imputed using expectation maximization.¹⁰⁵ This method was chosen because it has been shown to be relatively unbiased compared to other common methods, such as mean substitution and regression.¹⁰⁶ Data were imputed for n=39 participants on the MPAI-4 and n=15 on the CAS. Participants who had larger amounts of missing data, or had no data for the entire scale, were removed from the dataset (n=2).

Results

Participants

Detailed descriptive data for both the civilian- and SMV samples are provided in Carlozzi et al..⁵¹ Briefly, caregivers of civilians were older (M=51.6 years of age; SD=14.0) than caregivers of SMVs with TBI (M=37.2 years of age; SD=8.6), t(555.94)=15.04, p<.01, and they were more likely to be caring for someone that was older (M=42.3 years of age, SD=14.6), than the caregivers of the SMVs (M=37.1 years of age, SD=7.5; t(535.41)=5.56, p<.01). Caregivers of SMVs had a higher proportion of women (98.1%), than those of civilians (78%), χ²(1)=44.74, p<.001; they also were more likely to be married (93%) relative to caregivers of civilians (36.6%; χ²(1)=37.74, p<.001). The groups did not differ for time providing care, t(513.244)=.82, p=.41. With regard to TBI severity, 56.1% were classified as severe, 20.9% moderate, and 17.7% complicated mild (5.2% were of unknown severity) for the caregivers of individuals with civilian TBI. For those caregivers of SMVs with TBI, 17.3% were uncomplicated mild, 1.9% were complicated mild, 1.4% were moderate, 1.4% were severe, 14.9% were TBI equivocal mild TBI, 1.4% were unknown, and 1.4% were penetrating (severity data was not available for 61.3% of this sample). Since these individuals were recruited from the community, it can be reasonably estimated that much of this sample (i.e., >80%) would be classified as mild TBI (according to existing prevalence rates¹⁰⁷). Regardless of the reported TBI severity for each of these groups, caregivers of SMVs were also more likely to be caring for an individual that they perceived to be low functioning (53.0%), relative to their civilian caregiver counterparts (12.3%; χ²(1)=74.49, p<.001).

Reliability

All of the scales demonstrated good internal consistency reliability (i.e. they met or exceeded the suggested minimum level of Cronbach’s alpha or IRT-based reliability of 0.70, see Table 1). Test-retest reliability was generally acceptable for Fatigue in both the civilian sample (CAT: r=.78, SF: r=.76) and acceptable for the SF (r =.70) in the SMV sample; the reliability was low for the Fatigue CAT in the SMV sample (r = .63). Test-retest reliability was good for Sleep Disturbance in the civilian sample (CAT: r = .86, SF: r = .85) and the SMV sample (CAT: r=.80, SF: r=.79).

Table 1.

Descriptive Information and Reliability Data for Self-Report Physical Health Measures

	N	Mean Score (SD)	Internal Consistency°	3 Week Test- Retest Reliability	% of the sample with floor effects	% of the sample with ceiling effects	Mean Administration Time (sec)	Mean Administration Time/#items
Civilian sample
Fatigue CAT	335	51.83 (9.63)	.94	.78	0.0	1.2	32.45	7.43
Fatigue SF	337	51.33 (9.93)	.93	.76	0.9	10.7	23.81	5.95
Sleep Disturbance CAT	335	51.32 (9.44)	.92	.86	0.0	1.5	32.92	6.37
Sleep Disturbance SF	336	51.46 (8.63)	.84	.85	0.6	4.5	24.29	6.07
Military sample
Fatigue CAT	208	58.30 (9.22)	.94	.63	0.8	0.0	38.94	9.23
Fatigue SF	208	58.04 (9.21)	.94	.70	5.8	1.4	19.63	4.90
Sleep Disturbance CAT	208	58.16 (9.53)	.92--	.80	0.8	0.0	22.50	4.25
Sleep Disturbance SF	208	57.23 (8.11)	.81	.79	6.3	1.0	15.98	3.99

Note.

* = higher scores = better functioning;

° Cronbach ’s alphas are reported for all measures EXCEPT the computer adaptive tests which are reported as marginal (i.e., item response theory based) reliabilities.

PROMIS = Patient Reported Outcomes Measurement Information System;

CAT = Computer Adaptive Test;

SF = 4-item Short-Form;

CAS = Caregiver Appraisal Scale;

MPAI-4 = Mayo-Portland Adaptability Inventory-Fourth Edition;

ZBI = Zarit Burden Inventory;

retest samples were 56 civilians, 91 military

Floor and Ceiling Effects

Floor and ceiling effects for each PROMIS measure and administration format are shown in Table 1. In both samples, both floor effects and ceiling effects were lower for the CATs than the SFs for both fatigue and sleep disturbance. All were within acceptable limits.

Administration Times

Overall CAT and SF administration times were brief (Table 1), ranging from 24–33 seconds for the civilian sample, and 16–39 seconds for the SMV sample.

Convergent and Discriminant Validity

Correlations supporting convergent and discriminant validity are reported in Table 2. With regard to convergent validity, the two PROMIS Physical Health measures were moderately correlated with each other. The correlations between both PROMIS measures and the Rand-12 MHC (which includes an item about fatigue/vitality) were also moderate, providing further support for convergent validity. Correlations with physical health were small, supporting discriminant validity. Discriminant validity was also supported by negligible to small correlations between the two measures of Physical Health and the other three subscales of the CAS.

Table 2.

Convergent and discriminant validity of the PROMIS Physical Health CATs

				-----Discriminant validity-----
Measures (CATs)	Sleep Disturbance	RAND- 12 Mental Health	RAND-12 Physical Health	Caregiver Appraisal Scale				Zarit Burden Interview
				Burden	Satisfaction	Ideology	Mastery
Civilian sample
Fatigue	.65**	−.67**	−.29**	−.54**	−.19**	−.05	−.24**	.54**
Sleep Disturbance	--	−.49**	−.22**	−.38**	−.16**	−.10	−.22**	.41**
Military sample
Fatigue	.56**	−.51**	−.34**	−.55**	−.23**	.04	−.24**	.55**
Sleep Disturbance	--	−.41**	−.26**	−.36**	−.08	.09	−.16*	.31**

^*p < .05,

^**p < .01

Known Groups Validity

Known-groups analyses compared scores on Fatigue and Sleep Disturbance for each group identified by the MPAI-4 (low and high functioning; Table 3). Caregivers of high functioning individuals had significantly lower Fatigue and Sleep Disturbance scores, generally about ½ of a standard deviation lower, compared to caregivers of low-functioning individuals.

Table 3.

Known Groups Validity for PROMIS Physical Health CATs

	Caregiver of a High Functioning Individual		Caregiver of a Low Functioning Individual
PROMIS CATs	(MPAI-4 <60)		(MPAI-4>60)
	Mean (SD)	% Impaired*	Mean (SD)	% Impaired*	t	p
Civilian sample	N = 293		N = 42
Fatigue	51.20 (9.34)	17.1	56.27 (10.50)	40.5	3.24	.001
Sleep Disturbance	50.68 (9.35)	15.0	55.76 (8.92)	28.6	3.31	.001
Military sample	N = 104		N = 101
Fatigue	56.02 (8.98)	32.7	60.53 (8.68)	53.5	3.66	<.001
Sleep Disturbance	56.22 (9.35)	32.7	60.22 (9.45)	43.6	3.05	.003

Note.

PROMIS = Patient Reported Outcome Measurement Information System;

CATs = Computer Adaptive Tests;

MPAI-4 = Mayo-Portland Adaptability Inventory – Fourth Edition;

^{* =}impairment rates reflect individuals with T score > 60

Impairment Rates

Impairment rates for caregivers of low functioning civilians with TBI were elevated relative to the general population, as were impairment rates for caregivers of both high and low functioning SMVs with TBI (Table 3). Rates of Sleep Disturbance and Fatigue for caregivers of high functioning civilians with TBI were comparable to that of the general population (Table 3).

Discussion

For caregivers of people with TBI, well-validated patient reported outcomes (PRO) measures that assess self-reported sleep or fatigue have been sorely needed. This study provides support for the reliability and validity of the PROMIS measures of Fatigue and Sleep Disturbance in caregivers of people with TBI.

Cronbach’s alpha for PROMIS Sleep Disturbance and Fatigue were generally higher than or equivalent to other measures of physical function and HRQOL in caregivers of both civilians and SMVs with TBI. Furthermore, the CAT administration of the PROMIS measures was generally superior to the associated calibrated SF (as evidenced by fewer floor and ceiling effects and greater internal consistency reliability). In addition, the administration times for both CAT and SF administrations were generally 30 seconds or less per administration, highlighting the brevity of these new measures.

Convergent and discriminant validity of the PROMIS Sleep Disturbance and Fatigue measures were also supported by our findings. Specifically, convergent validity was supported by moderate associations between the PROMIS measures, as well as between the PROMIS measures and a composite score of mental health that included an item about vitality/fatigue. Furthermore, the moderate correlations between the sleep/fatigue measures and mental health are consistent with published findings supporting a well-established relationship between mental health and sleep/fatigue in other caregiver populations.^29,108–111 In addition, discriminant validity was supported by smaller associations between the PROMIS measures and other measures of caregiver burden and HRQOL. This pattern of findings was consistent with our proposed hypotheses and provides support for the validity of PROMIS in caregivers of civilians and SMVs with TBI.

As expected, we also found that caregivers of individuals that were low functioning had more problems with sleep disturbance and fatigue than those caregivers of individuals that were high-functioning TBI for both the civilian and SMV samples. Furthermore, clinical impairment rates were higher for caregivers of those that were low functioning, relative to those that were caring for individuals that were high functioning (for both groups). In addition, for caregivers of SMVs, impairment rates were elevated regardless of the level of functioning of the SMV. We hypothesize that this may be due to the high rates of other comorbid clinical conditions commonly reported for these SMVs¹¹² (not assessed as a part of this study). Elevated impairment rates are consistent with the existing literature, which suggests that approximately 60% percent of caregivers of persons with TBI report changes in sleep patterns.^25,26 These findings are consistent with qualitative work in caregivers of TBI that indicate these individuals experience significant difficulties with sleep.^4,21 Together, these findings support construct validity of these PROs in caregivers of civilians or SMVs with TBI and suggest that these measures may be especially relevant for caregivers of SMVs with TBI.

Study Limitations

This study relies solely on self-report data and did not include any objective assessments of sleep. Future research should focus on understanding how this PRO data relates to objective sleep measures. In addition, caregivers were primarily Caucasian, women, and spouses of the person with the TBI, thus generalizability is somewhat limited with regard to racial/ethnic minorities, male caregivers and caregivers that are parents or relations other than spouses. Future work could also focus on an examination of the overall intensity and duration of caregiver commitment, available social support for the caregiver, and other caregiver-specific factors (e.g., overall health status, occupation). Medical record documentation for injury severity was only available for 40% of the SMV sample, precluding our ability to examine the impact of TBI severity on analyses. Future research is needed to evaluate how TBI severity, as well as objective measures of functioning ability of the person with the TBI, affect caregiver HRQOL. The test-retest reliability time frame of 3 weeks was longer than what is typical of test-retest reliability, especially given that sleep and fatigue are likely to fluctuate over the course of a three-week time frame. Thus test-retest correlations are likely less robust that what might be expected given a shorter time frame. Future work is needed to establish responsiveness to change data in caregivers (including establishing minimal important differences for these measures). Also, more work is needed to understand the differences that were observed between caregivers of civilians and those of SMVs, including more consideration of TBI severity and functional capacity (mentioned above), as well as the influence that comorbid clinical conditions have on HRQOL for persons with TBI.

Conclusions

PROMIS Sleep Disturbance and Fatigue measures appear to provide brief, reliable, and valid assessments of physical health for caregivers of civilians and SMVs with TBI. Furthermore, these measures are able to differentiate between individuals that are caring for individuals with TBI that are low versus high functioning. Additional work is still needed to examine change over time and to determine the clinical utility of these measures. Ultimately, these measures fill a significant measurement gap for caregivers of people with TBI.

Highlights:

• The PROMIS Sleep and Fatigue measures are both reliable and valid

• The PROMIS Sleep and Fatigue measures are clinically relevant for caregivers

• Caregivers of persons with brain injury have problems with sleep and fatigue

List of abbreviations:

CAS

Caregiver Appraisal Scale

CAT

Computer Adaptive Test

HRQOL

Health-Related Quality of Life

MPAI-4

Mayo-Portland Adaptability Inventory-Fourth Edition

PRO

Patient-Reported Outcome

PROMIS

Patient-Reported Outcomes Measurement Information System

SF

Short Form

SMV

Service Member/Veteran

WRNMMC

Walter Reed National Military Medical Center

TBI

Traumatic Brain Injury

ZBI

Zarit Burden Inventory