Initial evidence for reliable and valid use of scores on the 8-item Econ-QOL short form to measure economic quality of life in caregivers of persons with traumatic brain injury

Abstract

Objective:

To provide reliability and validity data to support the clinical utility of Econ-QOL scores in caregivers of civilians and service members/veterans (SMVs) with TBI.

Design:

Cross-sectional survey study.

Setting:

Three academic medical centers and a VA treatment facility.

Participants:

376 caregivers of civilians (n=213) and SMVs (n=163) with TBI.

Interventions:

N/A.

Main Outcome Measures:

Econ-QOL and several patient-reported outcome measures (TBI-CareQOL Caregiver-Specific Anxiety and Caregiver Strain, PROMIS Sleep-related Impairment, Neuro-QoL Positive Affect and Well-being) and measures of financial status (self-reported income).

Results:

Internal consistency reliability of the Econ-QOL short form scores were excellent (all Cronbach’s alphas ≥0.92). There were no floor or ceiling effects for scores. There was evidence of convergent and discriminant validity, with the Econ-QOL scores having the strongest relationships with self-reported income (convergent validity evidence), and weak relationships with the other measures (discriminant validity evidence). Individuals with scores that were “below or possibly below” the poverty line (according to 2016 Federal Government poverty level thresholds) reported worse economic QOL relative to those individuals that were definitely above the poverty line, supporting known-groups validity.

Conclusions:

This report establishes the clinical utility of scores on the Econ-QOL Short Form in caregivers of persons with TBI and provides evidence that it is valid and appropriate to use such scores not only in a variety of different disability populations (e.g., spinal cord injury, stroke) but also in caregivers.

The financial impact of a traumatic brain injury (TBI) is substantial. In fact, the medical cost of TBI is $22,838 higher than non-TBI compared to age-matched controls.¹ Not surprisingly, TBI can profoundly impact the entire family. For those individuals sustaining more severe injuries, informal caregivers often provide long-term care. These informal caregivers commonly experience significant emotional distress, social difficulties, and overall diminished quality of life.^2–9 Caregivers also report financial challenges related to the caregiver role. Common complaints include: withdrawing (partially or totally) from the workforce (often at reduced wages), encountering numerous expenses related to direct care, and hardship created by the impulsive spending of the person with TBI.^10–17

Although caregivers report financial challenges related to caring for an individual with TBI, there are very limited empirical data examining the actual costs of informal family caregiving. Published data in caregivers of veterans with TBI indicate that 62.3% of caregivers of veterans have depleted assets and/or accumulated debt, and almost half of these caregivers (i.e., 41%) left the labor force due to their caregiver role.¹⁸ Importantly, the general literature would suggest that the costs of informal caregiving are substantial. Approximately $20,000 are lost in wages (for the caregiver) each year,¹⁹ with an estimate of $164,726 of lost wages over the caregiver’s lifetime.²⁰ Although we have estimates of lost wages and medical costs, we do not have empirical data on the impact that these financial costs have on the overall well-being of these informal care partners and their families. This is due, in part, to the lack of a consensus measure of economic hardship.

In our own research, we developed a measurement system for caregivers of persons with TBI, the TBI-CareQOL. The TBI-CareQOL provides several patient-reported outcome measures, each with evidence for valid use. While the TBI-CareQOL captures important components of caregiver-specific health-related quality of life (HRQOL),^21–32 this system did not initially include a measure of economic quality of life. In fact, most existing measures of HRQOL do not include content to evaluate economic or material challenges, despite the fact that such challenges can have a profound impact on day-to-day social participation.^15–17 Instead, most research has focused on objective reports of income, without consideration of the perceived effect on HRQOL. This perceived effect is important, as it considers the impact that finances can have on HRQOL. For example, individuals may have objectively high incomes, yet they may still experience insufficient economic resources, which can negatively affect their HRQOL.^{17, 33} Given that the availability of financial resources allows people to more readily have a healthy lifestyle³⁴ and engage in activities that promote well-being,³⁵ understanding peoples’ perceived economic quality of life is an important component of understanding the impact that finances and financial stability have on lived experience after TBI.

To address this oversight, the Economic Quality of Life (Econ-QOL) item bank was developed.¹⁶ This new measure examines the impact that economic factors have on HRQOL and community participation among people with disabilities who frequently have lower community participation³⁶ and increased economic burden as a result of their condition.^37–39 The Econ-QOL item bank includes 28 items that measure economic aspects of quality of life related to facilitators and barriers to community participation. Evidence for valid use of the Econ-QOL has been demonstrated in individuals with disabilities (including individuals with traumatic brain injury [TBI]).^{16, 40, 41} While the Econ-QOL helps to assess a critical measurement gap (i.e., perceived financial burden), data are needed from caregivers of persons with disabilities to establish its clinical utility and to justify its inclusion within the TBI-CareQOL measurement system.

The purpose of this analysis is to establish initial psychometric evidence for the reliable and valid use of Econ-QOL scores in caregivers of persons with TBI, in order to support the inclusion of this measure as a part of the TBI-CareQOL measurement system. We hypothesize that scores on the Econ-QOL will demonstrate strong evidence of psychometric reliability (i.e., minimal acceptable internal consistency and test-retest stability specified as ≥ 0.70^{42, 43}), as well as construct validity evidence (i.e., (a) strong correlations between Econ-QOL scores and other socioeconomic status measures, and weak correlations with non-socioeconomic constructs;⁴⁴ (b) the ability of Econ-QOL scores to differentiate between persons “above” versus “below or possibly below” the poverty line, with clinical impairment rates to exceed 16% of individuals in the “below or possibly below the poverty line” group).⁴⁵ Secondarily, we compared Econ-QOL sample scores to population norms in order to better understand the financial impact on caregivers and their families relative to the more general population of persons with disabilities. We hypothesized that caregivers of persons with TBI that were “below or possibly below” the poverty line would exhibit elevated financial problems relative to the general disability population, since such injuries may result in large per capita direct and indirect costs due to the fact that many injuries happen to those who are young and with severe disability.⁴⁶

Methods

Study Participants

We examined 376 caregivers of civilians (n=213) and SMVs (n=163) with TBI. Participants were recruited through caregiver-specific research registries, medical record data capture systems, ⁴⁷ and TBI model system databases. Medical record review involved identifying and contacting individuals with a medically-documented TBI who also met inclusion criteria (described below); the persons with TBI were then asked for contact information for their caregiver. Community-based recruitment targeted local support groups for caregivers of persons with TBI (e.g., Hearts of Valor). Three academic medical centers recruited caregivers of civilians. In addition, caregivers of SMVs were recruited through a U.S. Department of Veteran Affairs (VA) hospital; these efforts were supplemented by community outreach to SMV samples.

Inclusion criteria were as follows: Caregivers must be at least 18 years of age and able to read and understand English. Caregivers of civilians were required to be caring for an individual who was at least 16 years of age at the time of incurring a TBI, at least one year post injury, and have medical documentation that supported a complicated mild, moderate, or severe TBI according to TBI Model Systems criteria.⁴⁸ Caregivers of SMVs were required to be caring for an individual who was at least 16 years of age at the time of incurring a TBI, at least one year post injury, and could provide medical documentation of a TBI from a U.S. Department of Defense or U.S. Department of Veteran Affairs (DoD/VA) treatment facility. We expected that the majority of SMVs with TBI would have TBIs of mild severity, given existing prevalence rates within the military.⁴⁹ Because of this, we required that caregivers of SMVs answer a screening question confirming they were providing assistance to the SMV; a response of ≥1 to the following question was required: “On a scale of 0–10, where 0 is “no assistance” and 10 is “assistance with all activities,” how much assistance does the person you care for require from you to complete activities of daily living due to problems resulting from his/her TBI? Activities could consist of personal hygiene, dressing and undressing, housework, taking medications, managing money, running errands, shopping for groceries or clothing, transportation, meal preparation and cleanup, remembering things, etc.?”

Study Procedures

Since previous response data analyses support the existence of only minimal administrative mode effects,⁵⁰ surveys were completed either independently at home (using a personal computer, phone, or tablet with an internet connection) or in-person with study staff (using a study-specific computer or tablet through an online platform, i.e., Assessment Center^SM). Data were collected in accordance with local institutional review board approval at each site; all participants provided informed consent prior to participation in this study (or a waiver was secured given that this was a minimal risk study).

Study Measures

The Econ-QOL.

Caregivers completed an 8-item custom short form from the Economic-QOL item bank. Items administered were relevant to caregivers of persons with TBI (i.e., items specifically referencing the person with a disability were not selected); included items related to finances and affordability of living expenses. Likert-type scale responses for each item included: Never (1); Rarely (2); Sometimes (3); Usually (4); and Always (5). Resulting scores are on a T-score metric (M=50, SD=10); higher scores indicate better economic quality of life. Item content and response distributions are reported in the Appendix.

Convergent Validity Evidence.

Caregivers provided self-report data about income and number of children in the household. Categorical responses for annual income included: less than $10,000; $10,000 to $19,999; $20,000 to $29,999; $30,000 to $39,999; $40,000 to $49,999; $50,000 to $59,999; $60,000 to $69,999; $70,000 to $79,999; $80,000 to $89,999; $90,000 to $99,999; $100,000 to $109,999; more than $110,000; and “Don’t Know.” Poverty status was calculated by comparing reported household income to the number of children in the household plus two, the rationale being that the caregiver and the person with TBI were also dependent on the household’s income. Using 2016 Federal Government poverty-level thresholds,²⁸ participants were classified into the following mutually exclusive categories: 1) above the poverty line; 2) below the poverty line; or 3) possibly (“potentially”) below the poverty line (this income bracket included incomes below the poverty line for their household size).

Discriminant Validity Evidence.

Caregivers completed the TBI-CareQOL⁵¹ Caregiver-Specific Anxiety⁵² and Caregiver Strain item banks,⁵³ which measure (a) feelings of worry/anxiety related to the caregiver role and (b) feelings of burden related to the caregiver role, respectively. Caregivers also completed the PROMIS Sleep-related Impairment item bank, which evaluates perceived daytime problems related to inadequate sleep,⁵⁴ and the Neuro-QoL Positive Affect and Well-being item bank, which evaluates feelings of happiness and general positive outlook ^{55, 56}. All measures were administered as computer adaptive tests (CATs), an administration format which selects items to administer based on participants’ previous responses. These item banks are all scored on a T-score metric (M=50, SD=10); higher scores indicate more of the named construct (i.e., higher scores for negative health constructs indicate worse HRQOL; higher scores for positive health constructs indicate better HRQOL). The scores on these measures all have evidence supporting their reliability and validity in caregivers of both SMVs and civilians.²⁵

Data Analysis

Data were examined for normality, and parametric analyses (described below) were deemed appropriate.⁵⁷ All analyses were conducted separately for each caregiver group (i.e., civilian, SMV).

Evidence for Reliability.

Cronbach’s alpha was used to estimate the internal consistency reliability of Econ-QOL scores. A priori minimal acceptable reliability was ɑ≥0.70.^{42, 43}

Floor and Ceiling Effects.

Floor (% of participants with the lowest score possible) and ceiling effects (% of participants with the highest score possible) were also calculated for Econ-QOL scores. A priori acceptable floor and ceiling effects were specified as ≤20%.^{58, 59}

Evidence for Convergent and Discriminant Validity.

Correlations between Econ-QOL scores and socioeconomic status, as well as other measures of HRQOL, were used to provide evidence of convergent and discriminant validity.⁴⁴ Pearson correlations were examined for continuous measures, and polyserial correlations were examined for ordinal variables. We theorized there would be the most robust (i.e., moderate to large) correlations between scores on the Econ-QOL and self-reported income (i.e., “large”: r > 0.60; “moderate”: r between .40 and .60)⁴⁴ and less robust correlations between scores on the Econ-QOL and other measures of HRQOL (i.e., Caregiver Strain, Caregiver-Specific Anxiety, Positive Affect and Well-Being, and Sleep-related Impairment; i.e., “small”: r between 0.10 and 0.39; “negligible”: r < .10).⁴⁴

Evidence for Known-groups Validity.

Independent samples t tests were used to compare persons “above” versus “possibly below or below” the poverty line. Evidence for known-groups validity would be demonstrated by caregivers “below or possibly below” the poverty line having worse Econ-QOL mean scores than those “above” the poverty line.

Effect sizes.

Effect sizes for scores on the Econ-QOL were computed for the “above” the poverty line and “below/possibly below” the poverty line groups using Cohen’s d.⁴³ Values of d between 0.20 and 0.49 were considered “small,” values between 0.50 and 0.79 were considered “moderate,” and values greater than or equal to 0.80 were considered “large.” We expected large effect sizes for those with scores “below or possibly below” the poverty line versus “above” the poverty line.

Base Rates for Severe Problems with Economic QOL.

The rate of participants with poor economic QOL was evaluated to determine if caregivers of individuals with TBI are at greater risk than the United States general population. Specifically, we examined the percent of participants whose scores were ≥1 SD worse than the score mean of the normative sample on the Econ-QOL (N=675, M=50, SD=10). ⁶⁰. We expected that, based on the mean rate (i.e., 16% of the U.S. general population have poor economic QOL), rates greater than 16% would indicate poorer than expected economic QOL for caregivers of individuals with TBI.

All analyses were conducted using SAS 9.4.⁶¹

Results

Descriptive Data.

Table 1 provides a summary of the descriptive data for the study participants. Most participants were white (76%) and female (84%); average caregiver age was 48.0 years (SD=14.5), and most were above the poverty line (71%).

Table 1.

Sample Descriptive Data

Variable	Caregivers of Civilians (n=213)	Caregivers of Service Members/Veterans (n=163)	Combined Sample (N=376)
Age (years)
Mean (SD)	51.2 (14.5)	43.9 (13.4)	48.0 (14.5)
Sex, n (%)
Male	49 (23)	10 (6)	59 (16)
Female	164 (77)	153 (94)	317 (84)
Race, n (%)
White	148 (69)	139 (85)	287 (76)
Black or African American	48 (23)	9 (6)	57 (15)
American Indian or Alaska Native	1 (0)	2 (1)	3 (1)
Asian	2 (1)	2 (1)	4 (1)
Other Pacific Islander	1 (0)	1 (1)	2 (1)
Other	8 (4)	1 (1)	9 (2)
More than one race	0 (0)	3 (2)	3 (1)
Native Hawaiian	0 (0)	1 (1)	1 (0)
Missing	5 (2)	5 (3)	10 (3)
Ethnicity, n (%)
Hispanic	20 (9)	17 (10)	37 (10)
Non-Hispanic	193 (91)	145 (90)	338 (90)
Education, n (%)
Grades 1–8	0 (0)	1 (1)	1 (0)
Grades 9–12, no high school degree	12 (6)	7 (4)	19 (5)
High school or equivalent	42 (20)	14 (9)	56 (15)
Some college	62 (29)	55 (34)	117 (31)
Associates degree	27 (13)	22 (13)	49 (13)
Bachelor’s degree	44 (21)	40 (25)	84 (22)
Master’s degree	24 (11)	18 (11)	42 (11)
Professional school	1 (0)	3 (2)	4 (1)
Vocational degree/certification	1 (0)	3 (2)	4 (1)
Income, n (%)
Less than $10,000	21 (10)	3 (2)	24 (6)
$10,000 to $19,999	39 (19)	8 (5)	47 (13)
$20,000 to $29,999	7 (3)	17 (10)	24 (6)
$30,000 to $39,999	19 (9)	9 (6)	28 (8)
$40,000 to $49,999	9 (4)	14 (9)	23 (6)
$50,000 to $59,999	5 (2)	11 (7)	16 (4)
$60,000 to $69,999	15 (7)	21 (13)	36 (10)
$70,000 to $79,999	17 (8)	27 (17)	44 (12)
$80,000 to $89,999	8 (4)	12 (7)	20 (5)
$90,000 to $99,999	13 (6)	8 (5)	21 (6)
$100,000 to $109,999	12 (6)	9 (6)	21 (6)
More than $110,000	42 (20)	15 (9)	57 (15)
Don’t Know	3 (1)	9 (6)	12 (3)
Poverty, n (%)
Below poverty line	40 (19)	10 (6)	50 (13)
Maybe below poverty line	25 (12)	17 (10)	42 (11)
Above poverty line	140 (66)	127 (78)	267 (71)
Unknown	8 (4)	9 (6)	17 (5)
Marital status, n (%)
Never married	41 (19)	4 (2)	45 (12)
Cohabitating/common law	6 (3)	1 (1)	7 (2)
Engaged	2 (1)	1 (1)	3 (1)
Married	125 (59)	136 (83)	261 (69)
Separated	3 (1)		3 (1)
Divorced	26 (12)	12 (7)	38 (10)
Widowed	9 (4)	9 (6)	18 (5)
Other	1 (0)	0 (0)	1 (0)

Evidence for Reliability and Validity.

Evidence supported internal consistency reliability. Scores on the Econ-QOL displayed no important floor or ceiling effects (Table 2).

Table 2.

Descriptive Data for scores on the Econ-QOL SF

	N	Internal consistency reliability	Mean (SD)	% at Floor	% at Ceiling
Caregivers of Civilians	213	0.93	51.8 (11.1)	4.2	9.4
Caregivers of SMVs	163	0.92	49.3 (9.4)	1.8	4.3
Combined Sample	376	0.93	50.8 (10.4)	3.2	7.2

The pattern of correlations between Econ-QOL scores and associated criterion measures provided evidence for convergent and discriminant validity (Table 3). Specifically, there were moderate correlations between Econ-QOL scores and self-reported income, providing evidence for convergent validity, and there were small correlations between Econ-QOL scores and other measures of mental and physical HRQOL. An exception was the correlation between Econ-QOL scores and Positive Affect and Well-being scores (r=0.36; SMV caregiver group only): This did not meet our a priori cutoff for a “small or negligible” correlation (i.e., <0.40), supportive of evidence for discriminant validity. Note, however, that the correlation between Econ-QOL scores and Positive Affect and Well-being scores for the civilian caregiver and combined caregiver groups (r=0.33 and 0.35, respectively) did provide evidence for discriminant validity.

Table 3.

Evidence for Convergent and Discriminant Validity for Scores on the Econ-QOL SF

	Convergent Validity Evidence	Discriminant Validity Evidence
Econ-QOL SF	Income	Caregiver Strain	Caregiver Specific Anxiety	Positive Affect and Well-Being	Sleep-Related Impairment
Caregivers of Civilians	0.56	−0.23	−0.24	0.33	−0.26
Caregivers of SMVs	0.47	−0.23	−0.34	0.36	−0.35
Combined Sample	0.48	−0.26	−0.30	0.35	−0.31

All correlations significant with p<0.01

Evidence also supported the known-groups validity of Econ-QOL scores (Table 4). For all caregiver groups (i.e., civilian, SMV, combined), caregivers of persons who were either “below or possibly below” the poverty line had worse economic quality of life scores than those who were “above” the poverty line. There was an elevated risk for having poor economic quality of life status (i.e., Econ-QOL score ≥1 SD in the direction of worse quality of life) for caregivers who were “below or “possibly below” the poverty line (all caregiver groups). Conversely, there was not an elevated risk for having poor economic quality of life status (i.e., Econ-QOL score ≥1 SD in the direction of worse quality of life) for caregivers of persons who were high functioning (all caregiver groups) or “above” the poverty line (all caregiver groups).

Table 4.

Evidence for Known-groups Validity for scores on the Econ-QOL SF

Econ-QOL SF score sample	Below or Possibly Below the Poverty Line			Definitely Above the Poverty Line			t		P
	n	Mean (SD)	%	n	Mean (SD)	%		|d|
			Impaired a			Impaired a
Caregivers of Civilians	65	42.2 (8.9)	33.85	140	56.1 (9.2)	2.14	10.18	1.25	<0.001
Caregivers of SMVs	27	44.1 (6.2)	18.52	127	50.1 (9.7)	8.66	4.07	0.64	<0.001
Combined Sample	92	42.8 (8.2)	29.35	267	53.3 (9.9)	5.24	9.98	1.00	<0.001

Note.

^a= Econ-QOL score ≤ 40

Discussion

The analyses herein provide evidence to support the psychometric reliability and validity of scores on the Econ-QOL PRO short form measure. Specifically, internal consistency reliability was excellent for both caregivers of civilian individuals with TBI, for caregivers of SMV individuals with TBI, and for the combined sample (all alphas≥0.92). In addition, scores on the Econ-QOL did not show significant floor or ceiling effects for any of the groups. The identification of individuals with economic problems is more important than identifying participants that are financially comfortable, as those with economic problems are the individuals that may benefit from intervention or financial assistance. Thus, the very low rates for floor effects (ranging from 1.8 to 4.2% across the different subsamples) provide good evidence of clinical utility; the slightly higher rate for ceiling effects (i.e., ranging from 4.3 to 9.4%) is less problematic, given that these individuals are likely not the target of clinical interventions for economic factors.

Evidence also supported the clinical utility of Econ-QOL scores. As expected, for all caregiver groups there were moderate correlations between Econ-QOL scores and self-reported income, providing evidence for convergent validity. There were small correlations between Econ-QOL scores and Caregiver Strain, Caregiver-Specific Anxiety, and Sleep-related Impairment scores for all caregiver groups and between Econ-QOL scores and Positive Affect and Well-being scores for the civilian caregiver and combined caregiver groups, providing evidence for discriminant validity. For the SMV caregiver group, we observed a correlation of r=0.36 between Positive Affect and Well-being scores and Econ-QOL scores, which was a slightly stronger correlation than theorized. This correlation provides some initial evidence for an alternative theory: That one’s economic QOL status may relate with, perhaps influence, one’s status in other particular quality of life spheres. From a clinical standpoint, it is not surprising that economic well-being might be associated with emotional well-being; this warrants future investigation. Overall, scores derived from the Econ-QOL with our sample generated sufficient construct validity evidence to support its use with similar samples.⁶²

Evidence for known-groups validity also supported the clinical utility of scores on the Econ-QOL. For all groups, Econ-QOL scores were able to differentiate between those individuals who were “below or possibly below” the poverty line versus “above” the poverty line (i.e., those who were “below or possibly below” the poverty line had significantly worse perceived economic quality of life). As expected, Econ-QOL scores were worse for all caregivers who were “below or possibly below” the poverty line.

Study Limitations.

While this study exhibits several strengths, including its large sample size and the diversity of the caregiver groups (i.e., those caring for civilians versus those caring for SMVs), we also acknowledge several limitations. First, male caregivers were underrepresented in this sample. Thus, future work is needed to determine if results will be generalizable to male caregivers. Second, all caregivers in the present study had to be providing care for individuals who were at least 1-year post injury, a time frame where we expect most of the clinical gains to have occurred in the person with TBI and relative stability to have been achieved in terms of the caregiver role and its responsibilities. It is likely that an examination of economic quality of life during the immediate 12 months following injury is much more volatile and variable; thus, this “near-term” time period warrants future examination. Future work is also needed to examine the sensitivity of scores on the Econ-QOL to change over time. In addition, further examination is warranted for evidence of the valid use of this measure with racially and ethnically diverse groups.

Conclusions.

Reliable and valid measurement of economic quality of life for caregivers of persons with TBI is important, given what we know about the negative impact on overall well-being arising from financial strain associated with informal caregiving in other populations.^63–65 These findings provide strong evidence for the psychometric reliability and clinical utility of scores on the Econ-QOL in caregivers of civilians and SMVs with TBI, allowing researchers to screen for economic distress with confidence in caregivers of persons with TBI. In addition, this new measure can be used to explore potential financial disparities that might be present for people who are caring for a survivor of traumatic brain injury.

List of Abbreviations:

CAT

Computer Adaptive Test

ECON-QOL

The Economic Quality of Life Measure

HRQOL

Health-Related Quality of Life

PROMIS

Patient-Reported Outcomes Measurement Information System

Neuro-QoL

Neurological Quality of Life Measurement System

SMV

Service Member/Veteran

TBI

Traumatic Brain Injury

TBI-CareQOL

Traumatic Brain Injury Caregiver Quality of Life

VA

Veterans Affairs