Assessing vigilance in caregivers after traumatic brain injury: TBI-CareQOL Caregiver Vigilance

Abstract

Objective.

Caregivers of individuals with traumatic brain injury (TBI) frequently experience anxiety related to the caregiver role. Often this is due to a caregiver’s perceived need to avoid people and situations that might upset or “trigger” the care recipient. There are currently no self-report measures that capture these feelings; thus this paper describes the development and preliminary validation efforts for the TBI-CareQOL Caregiver Vigilance item bank.

Design.

A sample of 532 caregivers of civilians (n = 218) or service members/veterans (SMVs; n = 314) with TBI completed 32 caregiver vigilance items, other measures of health-related quality of life (Rand-12, PROMIS Depression, PROMIS Social Isolation, Caregiver Appraisal Scale), and the Mayo-Portland Adaptability Inventory-4.

Results.

The final item bank contains 18 items, as supported by exploratory and confirmatory factor analysis, IRT graded response modeling (GRM), and differential item functioning investigations. Expert review and GRM calibration data informed the selection of a 6-item short form and programming of a computer adaptive test (CAT). Internal consistency reliability for the different administration formats were excellent (reliability coefficients ≥ .90). Three-week test-retest stability was supported (i.e., r ≥ .78). Correlations between Vigilance and other self-report measures supported convergent and discriminant validity (0.01 ≤ r ≤ 0.69). Known-groups validity was also supported.

Conclusions.

The new TBI-CareQOL Caregiver Vigilance CAT and corresponding 6-item short form were developed using established rigorous measurement development standards, providing the first self-report measure to evaluate caregiver vigilance. This development work indicates that this measure exhibits strong psychometric properties.

Introduction

Over 1.4 million civilians experience TBI each year (Langlois, Rutland-Brown, & Thomas, 2004). Up to 20% of these individuals experience moderate to severe disability (Langlois et al., 2003), with persisting, devastating deficits in everyday functioning. For these individuals, family members and friends often have to assume a caregiver role and assist the person with TBI with physical, mental, financial, and leisure activities (Verhaeghe, Defloor, & Grypdonck, 2005). Traumatic brain injury (TBI) is also the most common combat-related injury among service members and veterans (SMVs; "TBI Numbers," 2010). Over 1/3 of SMVs with TBI require assistance from caregivers with daily tasks and management of medical issues (Griffin et al., 2012), and over one-third require direct or indirect supervision (Bailey et al., 2017).

Civilians with moderate to severe TBI commonly experience problems with behavior and mood, as well as changes in personality, and problems with cognition and physical ailments (Frank, Rosenthal, & Caplan, 2010). Behavioral problems can manifest as irritability, denial and unawareness of deficits, impulsivity, and sexual disturbances (Sherer & Madison, 2005). Anxiety and depression are common (Fann et al., 2009; Ponsford, Draper, & Schonberger, 2008; Rogers & Read, 2007), and up to 60% experience marked changes in personality (Diaz et al., 2014; Golden & Golden, 2003; Norup & Mortensen, 2015; Prigatano, 1992; Rao, Spiro, Handel, & Onyike, 2008; Warriner & Velikonja, 2006). Among all of the different areas of function that are affected in persons with moderate to severe TBI, the associated behavioral changes/problems have the most substantial impact on caregiver functioning and well-being (Marsh, Kersel, Havill, & Sleigh, 1998).

Caregivers of SMVs with TBI may be even more vulnerable to caregiver stress due to co-occurring mental health difficulties in the person with TBI. TBIs sustained in theater often occur under combat situations and exposure to intense psychological stress, such as firing weapons or being fired upon, seeing human remains, knowing someone to be killed or injured, being responsible for the death of a person, threat of abuse or execution if captured, and exposure to biological, chemical, or radiological weapons (Dausch & Saliman, 2009; French, Iverson, Lange, & Bryant, 2012; Hoge et al., 2004; Kennedy, Leal, Lewis, Cullen, & Amador, 2010; Sammons & Batten, 2008; Seal et al., 2008). Comorbid post-traumatic stress disorder (PTSD) is common (Hines, Sundin, Rona, Wessely, & Fear, 2014; Schneiderman, Braver, & Kang, 2008; Yurgil et al., 2014), affecting up to 44% of SMVs who have sustained a mild TBI and 42% of SMVs with a moderate, severe, or penetrating TBI (Hoge et al., 2008; Lange, French, Lippa, Ballie, & Brickell, Under review; Schneiderman et al., 2008). Co-morbid TBI and PTSD is associated with physical and psychological expressions of anger and hostility, re-experiencing and hyper-arousal symptoms, emotional numbness, and withdrawal and detachment(Arzi, Solomon, & Dekel, 2000; Beks, 2016; Mansfield, Schaper, Yanagida, & Rosen, 2014; Monson, Taft, & Fredman, 2009). Caregivers for these SMVs often experience high levels of perceived burden, depression, and anxiety that is secondary to the neurobehavioral problems that many of the SMVs experience.

While measures exist to capture many of the aforementioned constructs mentioned above (e.g., burden, depression, anxiety), many of these assessments are lengthy and/or borrowed from other clinical populations. For example, one of the most commonly used measures, the Zarit Burden Interview (Zarit, Orr, & Zarit, 1985) was developed specifically for use in caregivers of persons with dementia and has unknown validity in caregivers of those with TBI (Knight, Devereux, & Godfrey, 1997), focuses on only a sole aspect of HRQOL (i.e., burden), and is lengthy. Similarly, generic scales of HRQOL (e.g., the Medical Outcomes Study Short Form-36 (Ware, Kosinski, & Keller, 1994) or the Sickness Impact Profile(Bergner, Bobbitt, Carter, & Gilson, 1981) lack content coverage of TBI caregiver-specific issues. Recently, a new measurement system, the TBI-CareQOL, was developed to capture both the generic and TBI-caregiver-specific aspects of HRQOL for caregivers of both SMVs and civilians with TBI (Carlozzi, Kallen, et al., 2018). To guide this measure development, multi-institutional patient-report outcome (PRO) assessments delivered via focus group discussions were conducted with caregivers of both civilians and SMVs with TBI (Carlozzi et al., 2016b; Carlozzi et al., 2015a). Caregivers of SMVs with TBI frequently discussed concerns of hypervigilance or having to constantly monitor and control their own behavior, the behavior of other individuals, and the behavior of the person with TBI to minimize emotional upset or avoid physical violence (directed at the caregiver or at other individuals). This heightened state of vigilance seems likely due to both the unpredictable behaviors of the person with the TBI, as well as the high prevalence of comorbid PTSD among SMVs. Caregivers of civilians also discussed concerns of hypervigilance related to unpredictable behaviors of the person with TBI, although less frequently than caregivers of SMVs.

Given the emergence of hypervigilance as a concern of caregivers, and the fact that there are currently no measures that examine these feelings of vigilance that are specific to caregivers, a new PRO item bank of health-related quality of life (HRQOL) related to caregiver vigilance was developed and validated. The purpose of the current paper is to describe the development of this item bank and to present data on its reliability and validity.

Methods

Study Participants

A total of 532 caregivers of persons with TBI participated in this study (n = 218 caregivers of civilians and n = 314 caregivers of SMVs). Multi-site data collection occurred at the University of Michigan, Rehabilitation Institute of Michigan, TIRR Memorial Hermann, James A. Haley Veterans’ Hospital, and Walter Reed National Military Medical Center (WRNMMC). Caregivers of civilians with TBI were recruited using hospital and community-based recruitment, as well as site-specific research registries (of both caregivers and persons with TBI) and medical record data capture systems (Hanauer, Mei, Law, Khanna, & Zheng, 2015) at the University of Michigan, Rehabilitation Institute of Michigan and TIRR Memorial Hermann. Caregivers of civilians were also recruited through the existing TBI Model Systems databases at Rehabilitation Institute of Michigan and TIRR Memorial Hermann. Caregivers of SMVs were primarily recruited through community outreach (at the University of Michigan and WRNMMC), through hospital-based recruitment (James A. Haley Veterans’ Hospital and WRNMMC), and through the existing TBI Model Systems database (James A. Haley Veterans’ Hospital only).

To be eligible for the study, caregivers had to be able to read and understand English and indicate that they were providing physical assistance, financial assistance, and/or emotional support to an individual with TBI. The person with TBI had to be ≥ 16 years of age (18 years of age for SMVs) at the time of injury and ≥ 1-year post-injury. In addition, caregivers of civilians had to be caring for an individual with a medically documented complicated mild, moderate, or severe TBI (based on TBI Model Systems criteria; Corrigan et al., 2012), and caregivers of SMVs had to be caring for an individual with TBI that was medically documented by a U.S. Department of Defense (DoD) or U.S. Department of Veteran Affairs treatment facility. Medical record data were used to determine TBI severity (mild, moderate, severe, or penetrating) according to the U.S. Department of Veteran Affairs and U.S. Department of Defense criteria (The Management of Concussion/mTBI Working Group, 2009) for all caregivers of SMVs that were enrolled by WRNMMC and the James A. Haley Veterans’ Hospital (Note: Patients with loss of consciousness and posttraumatic amnesia in the mild range, but who have evidence of intracranial abnormality, were classified as complicated mild injuries). Similar DoD or DoVA criteria-establishing neuroimaging data were not available for SMV participants recruited by the University of Michigan (~30% of the sample). Professional, paid caregivers were not eligible. Study activities were conducted in accordance with local institutional review boards. Caregivers provided informed consent prior to the commencement of study activities (or a waiver was secured given the low-risk status of this study).

Measures

TBI-CareQOL Caregiver Vigilance Item Pool.

The initial Caregiver Vigilance item pool was developed using data from focus group discussions among caregivers of civilians and SMVs with TBI (Carlozzi et al., 2016a; Carlozzi et al., 2015b). Item development was iterative and included expert review, item reading-level assessment, and cognitive interviews with caregivers of both civilians and SMVs with TBI (see Figure 1).

Figure 1.

Iterative Process for Item Pool Development for Caregiver Vigilance

RAND-12.

The RAND-12 is a 12-item short form that measures generic health status (Hays, Sherbourn, & Mazel, 1995). Separate composite scores are calculated for Physical Health (PHC) and Mental Health (MHC). Scores are on a T-score metric (M = 50; SD = 10); higher scores indicate better health status.

Caregiver Appraisal Scale (CAS).

The CAS is a 47-item measure of positive and negative appraisals of the caregiver experience (Lawton, Kleban, Moss, Rovine, & Glicksman, 1989). For the purposes of this study, this measure was scored using Struchen and colleagues’ (2002) recommendations. Separate scores were generated for perceived burden, caregiving ideology, caregiving relationship satisfaction, and caregiving mastery; higher scores indicate more positive perceptions of caregiving.

Patient-Reported Outcomes Measurement Information System (PROMIS) Anxiety Item Bank (Cella et al., 2010).

PROMIS Anxiety is a computer adaptive test designed to measure self-reported fear, worry, nervousness, and hyperarousal. PROMIS Anxiety is scored on a T-score metric (M = 50; SD = 10); higher scores indicate more anxiety.

PROMIS Social Isolation.

PROMIS Social Isolation is a computer adaptive test designed to measure self-reported feelings of avoidance, exclusion, and detachment from other people (Carlozzi, Ianni, et al., 2018). PROMIS Social Isolation is scored on a T-score metric (M = 50; SD = 10); with higher scores indicating more social isolation.

Mayo-Portland Adaptability Inventory-4 (MPAI-4).

The MPAI-4 provides a measure of the caregivers’ perceptions of the overall functional disability of the person with TBI for whom they provide care (Malec, 2005). The MPAI-4 is scored on a T-score metric (M = 50; SD = 10); higher scores indicate more disability. Scores on the MPAI-4 were used to divide caregivers into two different groups according to established cutoff scores: caregivers of persons with TBI who are high functioning (i.e., T-scores < 60) and caregivers of persons who are low functioning (i.e., T scores ≥ 60; Malec, 2005). Please note that scores were unable to be generated for n= 29 caregivers of civilians and n = 11 caregivers of SMVs due to missing data (these caregivers skipped one or more items which precluded score calculation).

Data Collection

All self-report measures were completed using an online data collection platform (www.assessmentcenter.net). A subset of participants at the University of Michigan, Rehabilitation Institute of Michigan, and TIRR Memorial Hermann repeated these assessments within 2-3 weeks of the initial survey.

Statistical Analyses

Item Bank Development.

Item bank development followed existing measurement development standards, which includes both classical test theory and item response theory analytical approaches. The 32-item Caregiver Vigilance item pool was examined for essential unidimensionality. Specifically, an iterative process was used to select a unidimensional set of items. This process included exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), which both modeled categorical indicators and used the WLSMV estimation method, as well as clinical input (Cook, Kallen, & Amtmann, 2009; McDonald, 1999; Reise, Morizot, & Hays, 2007). EFA analyses used a geomin oblique rotation with an epsilon setting of 0.001, optimized for one to three extracted factors. With regard to EFA, essential unidimensionality would be supported if: 1) the ratio of eigenvalue 1 to eigenvalue 2 > 4; and 2) the proportion of variance accounted for by eigenvalue 1 >.40. Items with sparse cells (i.e., n < 10 respondents for a response category), low correlations for item-adjusted total scores (i.e., < 0.40), or that were non-monotonic (tested using Testgraf Software; Ramsay, Aug 1, 2000) were candidates for exclusion. CFA was also used to identify items for exclusion, including items with low factor loadings (i.e., lx < 0.50) and items demonstrating local dependence (i.e., those with residual correlations > 0.20 and/or a correlated error modification index ≥ 100; Cook et al., 2009; McDonald, 1999; Reise et al., 2007; Whittaker, 2012). When a set of items exhibited local independence, the content of each item contributing to the flagged residual correlation was reviewed. Expert review was used to identify the item that was most relevant to Caregiver Vigilance was retained. EFA and CFA analyses were conducted using Mplus version 7.4 (Muthén & Muthén, 2011).

Following EFA and CFA analyses, Samejima’s graded response model (GRM; Samejima, van der Liden, & Hambleton, 1996) was fit to the response data to further refine the pool, and items displaying significant misfit (S-χ², p < 0.01) were excluded. In addition, differential item functioning (DIF) was used to identify items that might exhibit bias for age (≤ 40 vs. > 40 years), education (high school graduate or less vs. > high school), and caregiver status (civilian vs. SMV). Items were excluded if Nagelkerke pseudo-R² change was ≥ 0.20 and if > 2% of DIF-corrected vs. uncorrected score differences exceeded uncorrected score standard errors. DIF analyses used a hybrid item response theory (IRT) ability score-ordinal logistic regression framework (Crane, Gibbons, Jolley, & van Belle, 2006); they were conducted using the R package LORDIF version 0.3-3 (Choi, Gibbons, & Crane, 2011; R Core Team, 2014). GRM analyses were conducted using IRTPRO version 3.0 (Cai, Thissen, & du Toit, 2015).

After EFA, CFA, and GRM analyses were completed, and the final item bank identified, CFA was again repeated as a final check of unidimensionality. A comparative fit index (CFI) ≥ 0.90, Tucker-Lewis index (TLI) ≥ 0.90 were specified as model fit indices supporting unidimensionality (Bentler, 1990; Hatcher, 1994; Hu & Bentler, 1999; Kline, 2005). While the literature has traditionally suggested root mean square error of approximation (RMSEA) be < 0.08 (excellent fit) or < 0.10 (acceptable fit), conflicting studies have indicated that these cutoffs may be too conservative for assessing essential unidimensionality, as number of items, latent score distribution, model specification, degrees of freedom, and sample size directly impact standard CFA fit criteria (Cook et al., 2009; Chen, Curran, Bollen, Kirby, & Paxton, 2008). As such, we specified RMSEA be < 0.15 for the purposes of the current analyses (Bentler, 1990; Chen, Curran, Bollen, Kirby, & Paxton, 2008; Hatcher, 1994; Hu & Bentler, 1999; Kline, 2005).

A 6-item short form (SF) was selected from the final item bank. SF items were selected to balance psychometric considerations (i.e., the selection of items with good psychometric properties according to item calibration and calibration-related statistics) and adequate concept coverage (according to clinical expert opinion). In addition, computer adaptive test (CAT) scores were simulated using Firestar version 1.3.2 software (Choi, 2009).

Assessment of data normality.

The distribution of the data was examined to ensure that parametric testing was appropriate. An examination of skewness and kurtosis indicated that the data were normally distributed and appropriate for parametric analyses.

Reliability and Validity Analyses.

Internal consistency reliability and test-retest stability were examined for both administration formats of Caregiver Vigilance (i.e., CAT and SF). With regard to internal consistency, an IRT-based internal consistency (i.e., calculated using average standard error-based reliability) was examined for the Caregiver Vigilance CAT, and Cronbach’s alpha was calculated for the SF. Reliability coefficients ≥ 0.70 would provide support for internal consistency (Cohen, 1988; DeVellis, 2017). Two-way mixed consistency intraclass correlations were calculated to examine test-retest stability for those participants with repeat testing.

Pearson correlations were also examined between the different administration formats (i.e., full item bank, simulated CAT, and SF). Floor and ceiling effects were examined for the Caregiver Vigilance administration formats, and percentages of participants with responses at the floor and ceiling were calculated for the SF. For the CATs, floor and ceiling effects were calculated by dividing the raw CAT scores by the number of items administered. Floor and ceiling rates were expected to be ≤ 20% (Andresen, 2000; Cramer & Howitt, 2004). Administration times were also examined for both administration formats of Caregiver Vigilance to support feasibility.

Pearson correlations were examined to evaluate convergent and discriminant validity of Caregiver Vigilance. We hypothesized that convergent validity would be supported by moderate to strong relationships (r > 0.4) between Caregiver Vigilance and the RAND-12 MHC, CAS Burden, PROMIS Anxiety, and PROMIS Social Isolation (Campbell & Fiske, 1959). We also hypothesized that discriminant validity would be supported by weak relationships (r < 0.3) between Caregiver Vigilance and the RAND-12 PHC, CAS Satisfaction, CAS Ideology, and CAS Mastery (Campbell & Fiske, 1959). Correlations ≥ .90 would suggest that the measures are redundant with one another (DeVellis, 2017).

Independent sample t-tests were used to examine known-groups validity for caregivers of low versus high functional ability. Specifically, we hypothesized that caregivers of persons with low functioning would report more Caregiver Vigilance than caregivers of persons with high functioning. Finally, to provide additional evidence of known-groups validity, we also examined clinical impairment rates for caregivers of persons who were low versus high functioning. According to the normal curve, 16.1% of scores are expected to fall > 1 SD below the mean (i.e., representing impairment). Therefore, impairment rates exceeding 16.1% would be classified as elevated relative to demographically-comparable peers (Heaton, Miller, Taylor, & Grant, 2004). Thus, we hypothesized that caregivers of persons with low functioning would also exhibit higher rates of clinical impairment on Caregiver Vigilance than caregivers of persons with high functioning. A Z-test for two proportions was used to identify significant differences in impairment rates across these groups.

Sample Size Requirements.

Sample size estimates were based on recommendations for item bank development using GRM-based estimation and sample sizes needed to examine DIF. Specifically, a minimum of 5-10 individuals per item (Bryant & Yarnold, 1995; Everitt, 1975; Gorsuch, 1983) and sample sizes ranging from 200 to 1000 have been proposed for GRM-based estimation (Samejima, 1969; Samejima et al., 1996). Furthermore, a minimum sample size of 200 participants per factor level within each DIF factor tested is recommended (Clauser & Hambleton, 1994).

Results

Participant Characteristics

There were a number of differences between the two caregiver subsamples (see Table 1). Caregivers of civilians were generally older, t (402.08) = 7.93, p < .01, more likely to be African American, χ² (1) = 31.22, p < .001, less likely to be female, χ² (1) = 34.81, p < .001, less educated, χ² (2) = 11.10, p < .01, less likely to be married, χ² (1) = 37.74, p < .001, and more likely to be parents or other family members, χ² (4) = 174.8, p < .001, than caregivers of SMVs. Caregivers of civilians were providing care for a person with TBI that was older, t (319.04) = 3.15, p < .01, than a person with TBI cared for by caregivers of SMVs.

Table 1.

Descriptive Information for Caregivers Completing the TBI-CareQOL Caregiver Vigilance Items

Characteristic	Caregivers of Civilians with TBI	Caregivers of Service Members/Veterans (SMVs) with TBI	Combined Caregiver Sample
	(n = 218)	(n = 314)	(n = 532)
Age (Years)*
M (SD)	51.3 (14.5)	42.2 (11.7)	45.9 (13.7)
Sex (%)*
Female	77.5	94.6	87.6
Male	22.5	5.4	12.4
Ethnicity (%)
Not Hispanic or Latino	90.8	90.4	90.6
Hispanic or Latino	9.2	9.6	9.4
Race (%)*
White	69.7	84.4	78.4
Black/African American	22.0	4.1	11.4
Other	8.3	11.5	10.2
Education (%)*
Less than High School	5.5	2.9	3.9
High School Graduate or Equivalent	19.7	5.1	11.1
More than High School	74.8	92.0	85.0
Marital Status (%)*
Single, Never Married	18.8	1.3	8.5
Married/Cohabitating	61.9	89.4	78.2
Separated/Divorced	13.3	4.5	8.1
Widowed	4.6	3.5	3.9
Other	1.4	1.3	1.3
Years in Caregiver Role
M (SD)	7.4 (5.4)	6.8 (3.2)	7.1 (4.2)
Relationship to Person with TBI (%)*
Spouse	26.6	77.7	56.8
Parent	38.5	15.0	24.6
Child/Other Family Member	24.3	4.8	12.8
Other (e.g. Friend)	10.6	2.5	5.8
Age of person with TBI *
M (SD)	43.1 (15.0)	38.9 (9.0)	40.6 (12.0)
Sex of Person with TBI (%)*
Male	72.5	97.8	87.4
Female	27.5	2.2	12.6
Time since injury (Years)
M(SD)	9.1 (6.9)	8.6 (3.4)	8.8 (5.4)
TBI Severity** (%)
Uncomplicated Mild	0.0	21.3	12.6
Complicated Mild	25.2	4.1	12.8
Equivocal	0.0	18.8	11.1
Moderate	17.4	4.1	9.6
Severe	57.3	15.6	32.7
Penetrating	0.0	6.4	3.7
Unknown	0.0	29.6***	17.6
Mechanism of Injury (%)
MVA	48.6	40.4	43.8
Falls	16.1	6.7	10.6
Struck by an object or thrown against an object	0.5	23.1	13.8
Gunshot or Assault	19.3	4.2	10.4
Other Accidents (e.g., Bicycle accident, pedestrian struck by motor vehicle)	11.0	3.2	6.4
Sports-related TBI	0.9	1.3	1.1
Other/Unknown	3.6	21.1***	14.0

Note. TBI = traumatic brain injury; entries in the table represent percentage of participants unless otherwise specified

^*indicates significant group differences: age, t (402.22) = 7.74, p < .0001, caregivers of civilians with TBI were older relative to caregivers of SMVs with TBI; sex (male, female), more caregivers of SMVs were female, relative to caregivers of civilians χ² (1) = 34.47, p < .0001; race (White, Black, Other), more caregivers of civilians were Black, relative to caregivers of SMVs, χ² (2) = 40.71, p < .0001; education (less than high school, high school graduate/equivalent, more than high school), caregivers of civilians were more likely to have educational attainment of high school or less relative to caregivers of SMVs, χ² (2) = 31.61, p < .0001; more caregivers of SMVs were spouses of the person they care for compared to caregivers of civilians χ² (3) = 140.76, p < .0001; marital status (married/cohabiting vs. all other groups), caregivers of civilians were more likely to be unmarried than caregivers of SMVs, χ²(7) = 78.31, p < .0001; gender of the person with injury was more likely to be male in the SMV group than in the civilian group χ² (1) = 74.52, p < .0001; and method of injury was more likely to be other/unknown in the SMV group than in the civilian group χ² (10) = 135.51, p < .0001.

^**Documentation of TBI Severity was different for the two groups; for the civilian sample, TBI severity for the civilian sample was determined according to TBI model system criteria (Corrigan et al., 2012), whereas for the SMV sample, TBI severity was determined according to U.S. Department of Veteran Affairs and U.S. Department of Defense criteria (The Management of Concussion/mTBI Working Group, 2009) with one exception, patients with evidence of intracranial abnormality and loss of consciousness and posttraumatic amnesia the mild range, as having a complicated mild TBI (rather than a moderate TBI).

^***Documentation of TBI severity was unavailable for the vast majority of the SMV sample collected through community outreach by the University of Michigan.

Item Bank Development.

Table 2 provides a summary of the EFA, CFA, and GRM-specific analyses that were used to establish a unidimensional set of Caregiver Vigilance items. Of note, seven items were deleted due to high residual correlations (criterion: > .20); five items were deleted due to high correlated error modification index values (criterion: ≥ 100) ; and two items displayed item misfit. The final CFA model indicated excellent model fit; marginal reliability and internal consistency of the final item bank was also excellent (Table 3). The final calibration statistics for the Caregiver Vigilance Item bank are provided in Table 4.

Table 2.

Unidimensional Modeling and Analyses for TBI-CareQOL Caregiver Vigilance Item Pool

	Unidimensional Modeling								Initial Item Performance		IRT Modeling
TBI- CareQOL Domain	Initial Item pool	EFA E1/E2 ratio (criterion >4)	Percent of variance for E1 (criterion >40)	1-factor CFA loading (criterion <.50)	1-factor CFA residual correlation (criterion >.20)	1-factor CFA modification index (criterion >100)	Item-adjusted total score correlations (criterion <.40)	Sparse cells (criterion<10)	Problems with monotonicity	IRT item misfit	DIF	Final item pool
Caregiver Vigilance Item Bank	32 items	6.99	55.8%	0 items	7 items	5 items	0 items	0 items	0 items	2 items	0 items	18 items

Note. CFA = Confirmatory Factor Analysis; EFA = Exploratory Factor Analysis; IRT = Item Response Theory

Table 3.

Final Item Parameters for TBI-CareQOL Caregiver Vigilance Item Bank

TBI-CareQOL Domain	Item Bank	CFI (criterion >.90)	TLI (criterion >.90)	CFA-based RMSEA (criterion < .15)	Cronbach’s Alpha Reliability (criterion > .80)	IRT-based RMSEA (criterion < .15)	IRT-based Marginal Reliability (criterion > .80)
Caregiver Vigilance	18 items	.96	.96	.14	.96	.09	.97

Note. CFI = Comparative Fit Index, TLI = Tucker-Lewis Index, CFA = Confirmatory factor Analysis; RMSEA: Root Mean Square Error of Approximation; IRT= Item response theory

Table 4.

Item Parameters for TBI-CareQOL Caregiver Vigilance Item Bank

Item	Slope	Threshold 1	Threshold 2	Threshold 3	Threshold 4
CAREGIVER VIGILANCE
I spend time controlling my environment in order to avoid triggers for the person I care for.	5.29	−0.64	−0.12	0.49	1.14
I have to monitor my environment to avoid triggers for the person I care for.	5.11	−0.69	−0.18	0.39	1.08
I try to control my environment to avoid triggers.	4.80	−0.71	−0.15	0.60	1.21
I spend time controlling the environment for the person I care for.	4.42	−0.78	−0.13	0.69	1.25
I feel like I am on the lookout for potential triggers for the person I care for.	4.29	−0.74	−0.21	0.40	1.06
I scan my environment when I am with the person I care for to identify things that may trigger them.	3.32	−1.17	−0.58	0.07	0.77
I am on the lookout for triggers that may anger the person I care for.	2.84	−1.26	−0.75	−0.16	0.60
I try to control other people's behavior in order to avoid triggers for the person I care for.	2.57	−0.61	0.25	1.08	1.86
I serve as a buffer between the person I care for and other people.	2.54	−1.35	−0.73	0.17	0.96
I avoid crowds when I am with the person I care for.	2.51	−0.89	−0.43	0.30	1.16
I avoid situations that are likely to cause the person I care for to have an outburst.	2.46	−1.38	−0.71	−0.01	1.10
I spend time trying to make sure I do not anger the person I care for.	2.44	−0.87	−0.17	0.72	1.47
I have to control the noise level in the room because it is a trigger for the person I care for.	1.80	−0.82	−0.20	0.73	1.62
I feel I am walking on eggshells around the person I care for.	1.77	−1.00	−0.03	1.00	1.87
I am on guard to make sure the person I care for doesn't hurt me emotionally.	1.72	−0.69	0.22	0.95	1.83
I agree with the person I care for to avoid arguments.	1.65	−1.45	−0.47	1.12	2.02
I need to protect the person I care for.	1.43	−1.79	−0.68	0.40	1.00
I am unable to predict what will trigger the person I care for.	1.03	−1.85	0.30	2.12	4.02

Note. Items that are indicated in bold were selected for inclusion on the 6-item Caregiver Vigilance short form

Table 5 provides a look-up table for converting total raw SF scores to T-scores. The correlation between the full item bank and CAT scores was excellent (r = 0.98). Information was good (i.e., ≥ 10) for scaled scores between 34 and 70 (see Figure 2). The CAT administered the maximum number of items (i.e., 12) at ≤ −1.3 SD units and ≥ +1.8 SD units, while it administered the minimum number of items (i.e., 4) between −1.1 to +1.4 SD units (see Figure 3).

Table 5.

Short-Form Total Raw Score to T-Score Conversion Table for TBI-CareQOL Caregiver Vigilance

	Caregiver Vigilance
Raw Score	T-score	SE *
6	30.81	5.01
7	35.23	3.73
8	37.44	3.41
9	39.32	3.11
10	40.97	2.88
11	42.47	2.72
12	43.84	2.60
13	45.12	2.53
14	46.34	2.50
15	47.55	2.49
16	48.76	2.50
17	49.97	2.50
18	51.18	2.51
19	52.40	2.52
20	53.64	2.53
21	54.91	2.53
22	56.19	2.52
23	57.48	2.53
24	58.82	2.55
25	60.23	2.61
26	61.75	2.72
27	63.44	2.89
28	65.38	3.17
29	67.71	3.54
30	71.78	4.70

^*SE = Standard error

Figure 2. Caregiver Vigilance Test Information Plot.

In general, we want total information to be ≥ 10.0 and the standard error to be ≤ 0.32 (this provides a reliability of 0.9). This figure shows excellent total information and standard error for Caregiver Vigilance scale scores between 34 and 70.

Figure 3. Caregiver Vigilance Number of CAT Items by CAT Theta.

This figure shows the number of CAT items used for different scale scores in standard deviation units: at approximately ≤ −1.3 SD units and ≥ +1.8 SD units the maximum of 12 items from the item bank were used by the CAT; from approximately −1.1 to +1.4 SD units the CAT tended to use the minimum of four items from the item bank.

Reliability and Validity Analyses.

Internal consistency reliability and test-retest stability were supported for both the CAT and SF administrations of Caregiver Vigilance for both the civilian and SMV subsamples as well as the combined sample (see Table 6). Correlations among the three administration formats (i.e., full item bank, CAT, and SF) were excellent (all r = 0.96). CAT and SF administration formats were also free of excessive floor and ceiling effects, and administration times were all less than 1 minute (Table 6).

Table 6.

Descriptive Data for Caregiver Vigilance Short Form and Computer Adaptive Test Administration Formats

TBI-CareQOL Caregiver Vigilance Administration Format	n	Internal Consistency°	Test-retest stability*	% of the sample with floor effects	% of the sample with ceiling effects	M (SD)	Median Administration time (sec)	Median Administration time per item (sec)
Combined Sample (n = 534)
Short Form	532	0.91	0.92	2.1	6.6	50.21 (9.21)	49.76	8.29
Computer Adaptive Test	532	0.95	0.91	2.1	6.6	50.05 (9.78)	40.41	7.47
Caregivers of Civilians (n = 218)
Short Form	218	0.90	0.86	0.9	12.9	46.14 (9.06)	52.65	8.78
Computer Adaptive Test	218	0.94	0.86	2.3	11.1	45.14 (9.60)	46.76	8.43
Caregivers of Service Members/Veterans (n = 314)
Short Form	314	0.90	0.88	2.9	2.2	53.03 (8.21)	43.13	7.19
Computer Adaptive Test	314	0.95	0.78	1.9	3.6	53.46 (8.38)	36.02	6.21

Note.

^°= internal consistency for static short form measures is reported as Cronbach’s alpha and for computer adaptive tests is reported as average standard error-based reliability (i.e., an item response theory-based reliability estimate)

^*Test-retest stability uses the intraclass correlation coefficient (ICC; criterion ≥ 0.70).

Findings for convergent and discriminant validity are reported in Table 7. Specifically, correlations did not indicate redundancy between the new measures and any existing measures. There were moderate to high correlations between Caregiver Vigilance and measures of mental HRQOL (i.e., Rand-12 MHC, CAS Burden, PROMIS Anxiety, and PROMIS Social Isolation) and negligible correlations with physical HRQOL (Rand-12 PHC) and positive aspects of caregiving (CAS Satisfaction, Ideology, and Mastery). Relationships that were inconsistent with proposed hypotheses was the moderate correlation between Caregiver Vigilance and Caregiving Mastery for caregivers of civilians only (SF = 0.46, computer adaptive test = 0.49).

Table 7.

Convergent and Discriminant Validity of TBI-CareQOL Caregiver Vigilance Item Bank

	Convergent				Discriminant
Caregiver Vigilance Administratio n Format	Rand-12 Mental	CAS Burden	PROMIS Anxiety	PROMIS Social Isolation	Rand-12 Physical	CAS Satisfacti on	CAS Ideology	CAS Mastery
Combined Sample (n = 532)
Short Form	0.50**	0.69**	0.57**	0.60**	0.22**	0.25**	0.17**	0.32**
Simulated computer adaptive test	0.46**	0.64**	0.51**	0.57**	0.20**	0.20**	0.22**	0.31**
Caregivers of Civilians (n = 218)
Short Form	0.41**	0.67**	0.55**	0.55**	0.19*	0.32**	0.29**	0.46**
Simulated computer adaptive test	0.37**	0.64**	0.51**	0.54**	0.16*	0.30**	0.32**	0.49**
Caregivers of Service Members/Veterans (n = 314)
Short Form	0.43**	0.60**	0.52**	0.55**	0.21*	0.28**	0.01	0.20*
Simulated Computer adaptive test	0.37**	0.51**	0.39**	0.47**	0.19*	0.20**	0.07	0.15*

Note: CAS = Caregiver Appraisal Scale; PROMIS = Patient Reported Outcome Measurement Information System; absolute values for correlations are presented to highlight magnitude

^*p < .05

^**p < .01

Caregivers of low-functioning individuals had significantly higher Caregiver Vigilance scores than those caring for high-functioning individuals, supporting known-groups validity across both administration formats (CAT, SF) for both caregiver groups (see Table 8). Impairment rates were elevated for those caring for low-functioning individuals in the SMV sample but not the civilian sample (Table 8).

Table 8.

Known Groups Validity Data for Caregivers of High and Low Functioning Individuals with Traumatic Brain Injury (TBI)

Caregiver Vigilance Sample and Administration Format	Caregiver of a High Functioning Individual (MPAI-4 <60)		Caregiver of a Low Functioning Individual (MPAI-4 ≥60)
	M (SD)	%	M (SD)	%	t	p
		Impaireda		Impaireda
Civilian sample	n = 163		n = 26
Short Form	44.85 (8.40)	3.1**	53.00 (7.49)	19.2	4.65	<.001
Simulated Computer	44.03 (8.92)	2.5**	52.17 (7.60)	11.5	4.40	<.001
Service Member/Veteran sample	n = 168		n = 135
Short Form	49.75 (7.39)	7.7**	57.34 (7.03)	34.8**	9.07	<.001
Simulated Computer Adaptive Test	50.34 (8.12)	7.7**	57.36 (7.23)	30.4**	7.85	<.001

^aEmotional Suppression T-score > 60

^*p < .05

^**p < .01.

Discussion

This paper described the development and preliminary validation of a new item bank, TBI-CareQOL Caregiver Vigilance, designed to capture caregiver feelings of anxiety, hyperarousal, and/or vigilance related to concerns about the emotional and behavioral status of persons following their experience with TBI. Results supported the development of a new item bank that could be administered as either a CAT or a 6-item short form. The CAT performed well (i.e., fewer than 10 items were administered) for individuals with Caregiver Vigilance scale scores between 37 and 68, and there are strong relationships between scores derived from different administration formats (i.e., between the full-item bank, CAT, and SF scoring formats).

In addition, the TBI-CareQOL Caregiver Vigilance item bank meets established psychometric standards: 1) items are homogeneous (i.e., unidimensional); 2) items are devoid of bias for age, education, and status (SMV vs. civilian); 3) internal consistency and test-retest reliabilities are excellent; 4) CAT and SF administrations are free of excessive floor or ceiling effects; 4) CAT and SF administrations times are brief (i.e., less than one minute); 5) convergent validity was supported; and 6) discriminant validity was supported. Impairment rates were elevated for those caring for low-functioning individuals in the SMV sample but not the civilian sample, which is consistent with literature suggesting that post-9/11 caregivers of SMVs exhibit worse outcomes than caregivers of civilians (Ramchand et al., 2014).

TBI-CareQOL Caregiver Vigilance scores conform to a T-score metric (M = 50, SD = 10), with higher scores indicating more caregiver vigilance; this increases the clinical utility of the measure since obtained scores immediately provide an estimation of an individual’s functioning relevant to a reference group (in this case, other caregivers of persons with TBI). As such, caregivers with Caregiver Vigilance scores of 60 (i.e., ≥ 1 SD) are functioning worse than 83.9% of caregivers of persons with TBI, whereas individuals with scores of ≥70 (i.e., ≥ 2 SDs) are exhibiting symptoms of hypervigilance, such scores are worse than 97.9% of their caregiving peers.

The TBI-CareQOL measurement system is the first measurement system to offer CAT test administration of concepts that are specific to caregivers of persons with TBI. This is important, given that CAT only requires the administration of the most relevant items, per person, in the bank. In this manner, CAT shortens administration time without sacrificing sensitivity/specificity, thereby decreasing overall test administration burden. This is an important factor when assessing caregivers who are often already overwhelmed by their caregiving responsibilities. While these findings provide preliminary psychometric support for the clinical utility of this new item bank in caregivers of both SMVs and civilians with TBI, it is important to acknowledge several study limitations. First, psychometric data for the CAT administration of this measure were based on simulation data (using the responses from individuals that completed all items within the item bank). Thus, prospective data are needed to confirm the validity of the Caregiver Vigilance CAT. In addition, the caregivers in this sample are predominantly female (88%), and, as such, the generalizability to male caregivers may be limited. In addition, documentation for TBI severity was missing for ~30% of the SMV sample due to the difficulties of securing medical record documentation for SMV participants at a civilian data collection site. While we expect that the majority of these participants (i.e., > 80%) were caring for an individual with a mild TBI (given existing prevalence rates for mild TBI in the military (DVBIC, 2015), we are unable to confirm this information. In addition, for the remaining 70% of the SMV sample, we were unable to determine severity for ~20%, due to insufficient or questionable information provided in the medical records. The two SMV data collection sites are regional centers that care for Service Members and Veterans initially treated at other medical facilities. As such, detailed injury severity care trauma data may not be available. While we expect that the majority of these participants (i.e., > 80%) were caring for an individual with a mild TBI, we were unable to confirm this information.

The TBI-CareQOL Caregiver Vigilance item bank provides a novel and psychometrically sound PRO measure that can be used to assess the health-related quality of life impact of the need to control the environment to reduce unpredictable behaviors in the person with TBI. Furthermore, this item bank, as well as the other measures that comprise the TBI-CareQOL measurement system, offer a battery of measures that offer a sensitive assessment of the aspects of HRQOL that are most important to these. This new Caregiver Vigilance measure is the first to assess caregiver feelings of vigilance related to providing care for an individual who exhibits unpredictable and occasionally volatile behavior. This type of measure would be an excellent candidate for inclusion in caregiver intervention research that is focused on training caregivers to manage the behavioral dysregulation problems characteristic of persons with TBI. Promising research in this area demonstrating that caregiver training for managing behavioral problems in individuals with TBI-polytrauma is associated with improvements in caregiver anxiety, depression, burden, and self-esteem, relative to caregivers who did not receive this training. Finally, although this measure was developed for caregivers of persons with TBI, it may also have clinical utility in other populations where the care-recipient exhibits behavioral problems and/or changes in personality (e.g., Alzheimer’s disease, Frontotemporal Dementia, and Huntington disease).

Impact and Implications Statement:

• Vigilance as a caregiver can be both fatiguing and emotionally overwhelming, given the supervision needs required for persons with traumatic brain injury.

• This new patient-reported outcome measure, TBI-CareQOL Vigilance, was developed to help clinicians assess for anxiety and hyperarousal, and/or vigilance, and its effect on health-related quality of life in caregivers of persons with traumatic brain injury.

• The TBI-CareQOL Vigilance measure is brief (low burden) and relevant to both research and clinical practice.

List of Abbreviations:

CAS

Caregiver Appraisal Scale

CAT

Computer Adaptive Test

CFA

Confirmatory Factor Analysis

CFI

Comparative fit index

DIF

Differential item functioning

DoD

Department of Defense

DVBIC

Defense and Veterans Brain Injury Center

EFA

Exploratory Factor Analysis

GRM

Graded Response Model

HRQOL

Health-Related Quality of Life

IRT

Item response theory

MHC

Mental Health Composite

MPAI-4

Mayo-Portland Adaptability Inventory-4

PHC

Physical Health Composite

PRO

Patient-Reported Outcome

PROMIS

Patient-Reported Outcomes Measurement Information System

RMSEA

Root mean square error of approximation

SF

Short Form

SMV

Service member/veteran

TBI

Traumatic Brain Injury

TBI-CareQOL

Traumatic Brain Injury Caregiver Quality of Life

TLI

Tucker-Lewis Index

WRNMMC

Walter Reed National Military Medical Center