The development of a new computer adaptive test to evaluate feelings of being trapped in caregivers of individuals with traumatic brain injury: TBI-CareQOL Feeling Trapped Item Bank

Abstract

Objective

To develop a new patient-reported outcome measure that captures feelings of being trapped that are commonly experienced by caregivers of individuals with traumatic brain injury (TBI).

Design

Cross-sectional

Setting

Three TBI Model Systems rehabilitation hospitals, an academic medical center, and a military medical treatment facility.

Participants

Caregivers of civilians with TBI (n=344) and caregivers of service members/veterans with TBI (n=216).

Interventions

Not applicable.

Outcome Measures

TBI-CareQOL Feeling Trapped Item Bank

Results

From an initial item pool of 28 items, exploratory and confirmatory factor analyses supported the retention of 16 items. After graded response model (GRM) and differential item functioning (DIF) analyses were conducted, 15 items were retained in the final measure. GRM calibration data, along with clinical expert input, was used to choose a 6-item, static short form, and the calibration data was utilized for programming of the TBI-CareQOL Feeling Trapped computer adaptive test (CAT). CAT simulation analyses produced an r = 0.99 correlation between CAT scores and the full item-bank. Three-week short form test-retest reliability was very good (r = 0.84).

Conclusions

The new TBI-CareQOL Feeling Trapped item bank was developed to provide a sensitive and efficient examination of the impact that feelings of being trapped, due to the caregiver role, have on health-related quality of life (HRQOL) for caregivers of individuals with TBI. Both the CAT and corresponding 6-item short form demonstrate excellent psychometric properties. Future work is needed to establish the responsiveness of this measure to clinical interventions for these caregivers.

Moderate to severe traumatic brain injury (TBI) can result in a variety of impairments that render the person with injury dependent on caregivers. About one-third to one-half of persons with moderate to severe TBI (per TBI model system criteria)¹ are not independent in everyday activities.^2,3 Physical impairments (e.g., motor spasticity and balance problems) can lead to the need for physical assistance with mobility and transportation.^4,5 Cognitive impairments can result in dependence in everyday tasks, including scheduling appointments, taking medication, and performing basic self-care.⁶ Behavioral impairments such as impulsivity can pose a safety risk.^7,8 Emotional distress leads to suicidal ideation for many people with TBI,^9,10 increasing the need for supervision. Given these difficulties, it is not surprising that about one-third of persons with moderate to severe TBI require supervision one year following injury,^11,12 and over one-fourth receive some form of supervision 2–9 years post-injury.¹³ Cognitive and behavioral impairments can also result in social isolation, rendering caregivers the primary source of social support for persons with an injury.¹⁴ Furthermore, unemployment is a major problem, resulting in financial dependence on family caregivers.^2,15,16

The need for partial or full-time supervision for persons with moderate to severe TBI can negatively impact caregivers. The presence of emotional distress^17–19 and perceived burden (i.e., caregiver’s perceptions that caring for the person with TBI negatively affects them) have been shown to have long-term effects.^20,21 The extent to which caregivers feel trapped or not in control can impact the level of distress and perceived burden they experience. The concept of entrapment or role captivity refers to the extent to which caregivers feel they are not free to live their own life or they are trapped by the illness or condition of the person for whom they provide care.²² The feeling of entrapment has been shown to be associated with depression in caregivers of persons with dementia²³ and a warning sign of caregiver burnout (i.e., physical, emotional, and/or mental exhaustion) for caregivers of persons with Amyotrophic Lateral Sclerosis.²⁴ The concept of entrapment has not been applied to caregivers of persons with TBI to date; however, qualitative research indicates that this feeling is present in this population. In early work on the impact of TBI on family members, Lezak (p. 593) noted that “Feeling trapped is an almost universal phenomenon,”²⁵ and that family members can feel trapped even if they are working and maintaining activity outside of the home. This feeling is related to fear of what will happen to the person with the injury while they are gone. Subsequent literature emphasized emotional distress and perceived burden in caregivers, but not the specific feeling of entrapment. The theme of feeling trapped emerged in recent focus groups conducted with caregivers of civilians with TBI, and with caregivers of service members/veterans (SMVs) with TBI.^26,27

As rehabilitation for individuals with TBI continues to evolve, many programs have recognized the need to treat not only the person with TBI, but also their care system. Although there is literature to support the reciprocal relationship of the well-being of the caregiver and of the person with TBI^28,29 there has been little done to develop ways of measuring symptoms or experiences unique to caregivers of persons with brain injury.

Given its potential contribution to emotional distress and to overall health for caregivers of civilians and SMVs with TBI, the concept of feeling trapped is important to quantify. Although measures of caregiver burden currently exist, we are unaware of measures that include the concept of feeling trapped. As such, an item bank was developed to assess this concept (i.e., TBI-CareQOL Feeling Trapped) in caregivers of civilians and SMVs with TBI in the hopes of being able to measure this common experience.

Methods

Study Participants

Five hundred sixty caregivers of civilians (n=344) or SMVs with TBI (n=216) participated in this study; a subset also completed a retest survey (n=56 civilian and n=89 military). A description of this sample is detailed elsewhere.³⁰ Briefly, participants were recruited through existing TBI-specific caregiver databases, medical record data capture systems,³¹ hospital-based recruitment and community outreach efforts at two TBI model system sites, an academic medical treatment center, and Walter Reed National Military Medical Center (WRNMMC). A caregiver was defined as an individual who provides physical assistance, financial assistance, or emotional support to an individual with a TBI. Caregivers of civilians and SMVs were ≥18 years old, able to read and understand English, and were caring for someone who was ≥1 year post-injury (given that the majority of recovery occurs within the first 12-months post-TBI, we excluded individuals who were caring for more acute injuries to maximize sample homogeneity^32–43). Caregivers of civilians with TBI were caring for an individual that was ≥16 years old at the time of the initial injury and who met TBI Model Systems criteria for a medically-documented complicated mild, moderate, or severe TBI.¹ Caregivers of SMVs with TBI were included if they were caring for an individual who was ≥16 years old at the time of the injury and that had a TBI that was medically documented by a Department of Defense or Department of Veteran Affairs treatment facility. All data were collected in accordance with each site’s local institutional review board. Participants (i.e. caregivers) provided consent prior to participation. Study participants completed study measures online (https://www.assessmentcenter.net) between 12/2014 and 04/2016. Surveys were completed using a designated research visit computer or on a personal or publically-available computer with internet access. All measures were completed within a 2-week time period. The retest only occurred at a single data collection site (72.5% of participants at this site completed the retest).

Feeling Trapped Item Pool

A detailed summary of the origination of the Feeling Trapped Item Pool can be found in Carlozzi et al.³⁰ Briefly, in accordance with established measurement development standards, ⁴⁴ an initial item pool of 75 items was generated from focus group discussion among caregivers of individuals with TBI (nine groups with caregivers of civilian-related TBI and nine groups with caregivers of SVMs with TBI)^26,27 and systematic literature reviews. Items were refined via an iterative process that included expert review, evaluation of item literacy level, and caregiver cognitive debriefing to ensure adequate content coverage and appropriate reading and comprehension levels. The final pool included 28 items designed to evaluate feelings of being trapped; all items used the same response options (never, rarely, sometimes, usually, always). These items served as the basis of this new patient-reported outcome measure (PRO), which assesses caregiver feelings of being unable to leave the person for whom they provide care.

Statistical Analyses

Sample size considerations are reported in Carlozzi et al., this issue.³⁰

Overview

In order to develop an item bank from the initial item pool, we followed established guidelines⁴⁴ incorporating classical test theory and item response theory (IRT) for measurement development. IRT is a mathematical modeling technique used to find a latent scale score from its component items, rather than using simple linear addition (as in classical test theory); it offers several advantages, including providing data about both the discriminability and difficulty of each item, and it can be used to program computer adaptive test (CAT) administration of a PRO measure.^45,46

First, we identified a unidimensional set of items. Second, we used IRT⁴⁷ analyses to estimate the bank’s item parameters (slopes and thresholds), thereby establishing the calibrations required for CAT and short form (SF) administrations of this final item bank. Initial analyses commenced by utilizing all 28 items in the item pool.

Exploratory Factor Analysis (EFA)/Confirmatory Factor Analysis (CFA), Initial Item Performance Assessments.

Full-sample EFA then CFA were used to iteratively examine the dimensionality of the item pool.^48–50 Analyses were conducted using MPlus (version 7.4).⁵¹ For EFA, unidimensionality was considered supported if the ratio of eigenvalue 1 to eigenvalue 2 was >4 and the proportion of variance accounted for by eigenvalue 1 was >.40. Items were deleted from the item pool if they had sparse cells (one or more response categories with n<10 respondents), low item-adjusted total score correlations (r <0.40), or were non-monotonicity (item response scores did not increase as measure response scores increased; this assumption was tested using Testgraf⁵² by examining item-rest plots and expected scores by latent trait plots obtained from a non-parametric IRT model). Next, CFA was used to identify (and delete) items with low factor loadings (lx<0.50), as well as items demonstrating local dependence (residual correlation>0.20; correlated error modification index≥100).^48–50

IRT Modeling, Final Item Performance Assessments, Differential Item Function (DIF) Studies, Final CFA Modeling.

Once a unidimensional item set was identified and refined, Samejima’s graded response model (GRM)⁵³ was used to model the data using IRTPRO (version 3.1).⁵⁴ Items were retained if they demonstrated good individual item fit (items displaying significant misfit, S-X2, p<0.01, were deleted from the item pool). Next, a hybrid IRT ability score-ordinal logistic regression framework⁵⁵ was used to flag items for DIF that were potentially biased for age, education, and caregiver civilian vs. SMV status; items that exhibited impactful DIF (Nagelkerke pseudo-R² change ≥0.20, plus >2% of DIF-corrected vs. uncorrected score differences exceeding uncorrected score standard errors) were deleted from the item pool (analyses were conducted using the R package LORDIF Version 0.3–2^56,57). A final CFA model was run to confirm unidimensionality and assess overall model fit. Standard fit criteria were employed: comparative fit index (CFI) ≥0.95, Tucker-Lewis index (TLI) ≥0.95, and root mean square error of approximation (RMSEA) <0.15.^58–61

CAT Simulation, Short Form Development

CAT scores on the new Feeling Trapped Item Bank were simulated using Firestar simulation software.⁶² In addition, a six-item short form was also constructed that balanced psychometric and clinical considerations to attain item bank representativeness.

Scoring of the Feeling Trapped Item bank

IRT-based theta scores for the Feeling Trapped CAT and SF administrations were estimated using the GRM; scores were then placed on a T-score metric (mean=50, SD=10); higher scores indicate more feelings of entrapment.

Results

Study Participants

Detailed descriptive data about the study sample can be found in Carlozzi et al.³⁰ Briefly, on average caregivers were 46.1 years old (SD=14.1) and in the caregiver role for 5.8 years (SD=5.4). The majority of caregivers were female (85.5%), Caucasian (77.2%), married (74.2%), caring for a spouse (58.2%) or parent (22.9%), and well educated (81.5% have some college or a college degree). The vast majority of TBIs for the SMV group were a result of military involvement (98%).

Unidimensional Modeling and Analyses

Table 1 provides a summary of the iterative process described below.

Table 1.

Unidimensional Modeling and Analyses

Domain	Unidimensional Modeling								Initial Item Performance		IRT Modeling
	item pool	EFA E1/E2ratio (craticriterion >4)	Proportion of variance for E1 (criterion >0.40)	1-factor CFA loading (criterion <.50	1-factor CFA residual correlation (criterion >.20)	1-factor CFA modification index (criterion >100)	Item-adjusted total score correlations (Criterion <.40)	Sparse cells (criterion<10)	Problems with monotonicity	IRT item misfit	DIF	Final item bank
Feeling Trapped	28 items	6.11	0 items	1 item	10 items	0 items	1 item	0 items	0 items	1 item	0 items	15 items

Note. CFA = Confirmatory Factor Analysis; EFA = Exploratory Factor Analysis, IRT = Item Response Theory

EFA/CFA, Initial Item Performance Assessments.

Field testing was conducted for the 28 items from the Feeling Trapped item pool. Initial EFA evidence suggested unidimensionality of this item pool. None of the items demonstrated problems with non-monotonicity, nor were there problems with sparse cells; one item was removed for low factor loadings and 11 items were excluded due to evidence of local dependence (ten items were deleted due to large residual correlations; one item was deleted due to large correlated error modification indices). The resultant item pool was 16 items.

IRT Modeling, Final Item Performance Assessments, DIF Studies, Final CFA Modeling

IRT modeling (GRM) identified an additional item with significant misfit (S-X2, p<0.01) which was excluded from the pool. Analyses did not identify any items with significant DIF for age, education, or civilian/SMV status.

Final Item Bank Criteria

A final CFA model indicated good overall model fit for the final 15 Feeling Trapped items; findings also supported response pattern (marginal) reliability (r = .93) and Cronbach’s alpha (internal consistency; r = .97); see Table 2.

Table 2.

Caregiver Strain Final Item Parameters

Domain	Item Bank	CFI (criterion ≥.95)	TLI (criterion ≥.95)	CFA-based RMSEA (criterion < .15)	Cronbach’s Alpha Reliability (criterion > .80)	IRT-based RMSEA (criterion < .15)	Response Pattern Reliability (criterion > .80)
Feeling Trapped	15 items	.98	.97	.14	.97	.05	.93

Note. CFI = Comparative Fit Index, TLI = Tucker-Lewis Index, RMSEA: Root Mean Square Error of Approximation

Estimates of final item bank item parameters are shown in Table 3. Score-level reliability was excellent from theta=−0.8 to +2.4, with expected score-level reliability ≥.90; score-level from theta =−1.2 to +2.4 was at least very good (≥.80), while score-level reliability from theta =−1.2 to +2.8 was at least good (≥.70). Feeling Trapped Test information is found in Figure 1.

Table 3.

TBI-CareQOL Item Parameters for Feeling Trapped

Item	Slope	Threshold 1	Threshold 2	Threshold 3	Threshold 4
I feel like I cannot leave home because of the responsibilities I have as a caregiver.	4.83	−0.21	0.35	1.09	1.5
I cannot run errands because I need to be with the person I care for.	4.20	0.01	0.57	1.39	1.81
I find it difficult to do things alone because of the care I give to the person with the injury.	4.12	−0.26	0.31	1.1	1.62
My freedom feels limited when it comes to providing care for the person with the injury.	3.91	−0.37	0.2	1.11	1.52
I feel like I need to rush home to be with the person I care for.	3.64	−0.4	0.07	0.78	1.25
I feel limited about the places I can go because of the care I give to the person with the injury.	3.58	−0.31	0.2	0.97	1.41
I feel as if I am unable to go to public places because I need to take care of the person with the injury.	3.55	0.08	0.7	1.5	1.91
I feel like I cannot leave the person with the injury, even if other people are providing care.	3.49	0.12	0.61	1.25	1.64
I come home early from social gatherings because I need to be with the person I care for.	3.40	−0.37	0.22	0.89	1.39
I feel like I cannot have a minute away from the person I care for without worrying about him/her.	3.30	−0.2	0.37	1	1.49
I feel sad because I lost the freedom I once had before the person I care for was injured.	2.84	−0.2	0.47	1.29	1.7
I feel I have to be physically near the person I care for.	2.63	−0.47	0.07	1.02	1.79
I am unable to leave the person I care for alone.	2.50	0	0.54	1.19	1.55
It is difficult for me to trust that someone else can care for the person with the injury as well as I do.	2.37	−0.39	0.18	0.81	1.31
I feel restricted by the disability of the person I care for.	2.24	−0.45	0.27	1.33	1.94

Note. Items that are indicated in bold were selected for inclusion on the 6-item short form

Figure 1.

Feeling Trapped Test Information Plot

In general, we want total information to be . 10.0 and the standard error to be . 0.32 (this provides a reliability of 0.9). This figure shows excellent total information and standard error for Feeling Trapped scale scores between −0.8 and +2.4.

CAT Simulation, Short Form Development

For Feeling Trapped estimated scores (full item bank vs. CAT administration), the correlation between the item bank and the CAT scores was excellent (r = .99). The most frequent occurring CAT lengths were (in descending order) 4 items (n=357, 64.2%), 12 items (n= 114, 20.5%), 5 items (n= 33, 5.9%), and 7 items (n= 19, 3.4%). CATs 6 or 8 to 11 items long occurred infrequently (n= 33, 5.9%). For 4-item CATs, observed thetas ranged from −0.54 to +1.98; theta ranges for 5 and 7-item CATs were similar to those observed for 4-item CATs, though slightly wider (five items: −0.59 to +1.86; seven items: - 1.34 to +1.95). Observed thetas for 12-item CATs were negative and ranged from −1.58 to - 0.87. Thus, 12-item CATs occurred when extreme low levels of Feeling Trapped were measured.

Figure 2 shows the number of CAT items used for different scale scores at standard deviation units: at −1 SD units, the CAT always used the maximum 12 items administered from the item bank; from +0 to +2 SD units, the CAT tended to use the minimum number of four items administered from the item bank; and at >+2 SD units the CAT used more than four items from the item bank. The average CAT administration time was 32 seconds.

Figure 2.

Feeling Trapped Number of CAT Items by CAT Theta

This figure shows the number of CAT items used for different scale scores in standard deviation units: at approximately . −1.0 SD units, the maximum of 12 items from the item bank were used by the CAT; from approximately −0.5 to +2.0 SD units, the CAT tended to use the minimum of four items from the item bank.

Clinical expert knowledge and psychometric considerations (i.e., item performance statistics) were used to select the best set of items for the Feeling Trapped six-item SF. The reliability for this SF was examined on a measurement continuum from theta −2.8 (T-score=22) to +2.8 (T-score=78). Score-level reliability was excellent (i.e. ≥.90) from theta −0.4 to +2.0. Score-level reliability from theta −0.8 to +2.0 was very good (≥.80), while score-level reliability from theta −0.8 to +2.4 was at least good (≥.70). Three-week test-retest reliability was very good (r = 0.84). Summed score to T-score conversions for the Feeling Trapped 6-item SF are shown in Table 4. The average 6-item SF administration time was 34 seconds.

Table 4.

Short-Form Summed Score to t Score Conversion Table for TBI-CareQOL Feeling Trapped Item Bank

Raw Score	T-score	SE *
6	37.00	5.39
7	42.90	3.07
8	44.97	2.73
9	46.70	2.38
10	48.06	2.11
11	49.29	2.11
12	50.41	2.06
13	51.49	2.04
14	52.55	2.04
15	53.60	2.06
16	54.65	2.07
17	55.69	2.08
18	56.72	2.08
19	57.74	2.08
20	58.75	2.07
21	59.76	2.06
22	60.76	2.04
23	61.76	2.02
24	62.76	2.02
25	63.80	2.05
26	64.89	2.12
27	66.09	2.22
28	67.55	2.48
29	69.16	2.68
30	73.16	4.10

^*SE = Standard error

Discussion

Given that feelings of entrapment or role captivity appear to be associated with emotional distress and burnout in caregivers,^22–24 a better understanding of these feelings and how they impact health-related quality of life (HRQOL) is important and may help to drive the development and implementation of clinical interventions. Despite this need, there is an understudied need for caregivers of individuals with TBI. To help foster research in this area, this study provided a comprehensive effort to develop a new assessment tool that assesses feelings of entrapment for caregivers of individuals with TBI. It also marks the first effort to develop CATs that are specific to caregiver needs.

The final Feeling Trapped item bank was comprised of a unidimensional set of 15 items that were free from bias (related to age, or education-level and status—civilian or SMV); we interpret the measure as reflecting the feelings of entrapment experienced by caregivers of individuals with TBI. Resulting scores are on a T-score metric, allowing for clinical interpretation of scores relative to other caregivers of individuals with TBI. Scores that are ≥60 indicate feelings of entrapment that are worse than 84.1% of their peers, whereas scores ≥70 indicate feelings that are worse than 97.7% of their peers. Both the TBI-CareQOL Feeling Trapped Item Bank CAT and SF administrations demonstrated excellent reliability and can be administered in under a minute.

The psychometric developmental strengths of incorporating the patient perspective, as well as the advantages afforded by the combination of classical test theory and IRT-based approaches to development, ^45,46 and the strong psychometric properties of this measure (internal consistency, test-retest reliability, convergent and discriminant validity, and known groups validity findings, reported elsewhere in this special issue⁶³) support the clinical utility of this measure. The brief administration time makes this measure an ideal candidate for inclusion within an outpatient visit that is focused on the support network of the person with the TBI. Ideally, this type of integration could help foster appropriate referrals for caregivers that are in need of additional services/support. Such services should also have downstream positive effects for the person with the TBI, given the established relationship between caregiver and care-recipient outcomes.^29,64–70

Study Limitations

Several study limitations must be acknowledged. While diverse, this sample was comprised largely of spousal caregivers; there were not enough parent caregivers to determine if items perform differently among parent caregivers. In addition, we were not able to control for caregiver health status, intensity of care provided, or TBI severity; medical record documentation for TBI severity was not available for a large portion of the military sample (i.e., those recruited outside of the WRNMMC). Also, caregivers of persons with acute injuries (<1 year post-injury) were excluded, limiting the generalizability of findings to these individuals. However, given the likely universal applicability of feelings of entrapment related to the caregiver role, these items may be appropriate for other caregiver subgroups. Future work could examine caregiver-specific factors like overall health status and occupation, as they might have an impact on the level of perceived stress.

Conclusion

This is the first time that a measure of feelings of entrapment has been developed, and the TBI-CareQOL measurement system is the first to be developed specifically for caregivers of individuals with TBI that includes CAT measures. Additionally, both CAT and SF administrations from the item bank generate standardized scores (T-scores), which has the advantage of aiding clinicians and researchers in score interpretations; scores systematically indicate how a caregiver is functioning relative to other caregivers of individuals with TBI. Ultimately, this measure can be used to help identify caregivers who are in need of respite care services and will foster assistance for caregivers who are seeking instrumental support in their daily lives.

Highlights:

• Caregivers of persons with traumatic brain injury often feel trapped

• TBI-CareQOL Feeling Trapped was developed to assess quality of life in caregivers

• This new measure addresses an important caregiver emotional concern

List of Abbreviations:

CAT

Computer Adaptive Test

CFA

Confirmatory Factor Analysis

CFI

Confirmatory Fit Index

DIF

Differential Item Functioning

EFA

Exploratory Factor Analysis

GRM

Graded Response Model

HRQOL

Health-Related Quality of Life

PRO

Patient-Reported Outcome

PROMIS

Patient-Reported Outcomes Measurement Information System

RMSD

Root Mean Square Deviation

RMSEA

Root Mean Squared Error of Approximation

SF

Short Form

SMV

Service Member/Veteran

TBI

Traumatic Brain Injury

TBI

CareQOL Traumatic Brain Injury Caregiver Quality of Life

TLI

Tucker Lewis Index