Psychometric Characteristics of the Insomnia Severity Index in Veterans with History of Traumatic Brain Injury

Abstract

Objective/Background

The Insomnia Severity Index (ISI) is a widely used self-report measure of insomnia symptoms, however to date this measure has not been validated or well-characterized in Veterans who have experienced traumatic brain injury (TBI). This study assessed the psychometric properties and convergent, divergent, construct, and discriminate validity of the ISI in Veterans with a history of TBI.

Participants

Eighty-three Veterans with history of TBI seen in the VA San Diego Healthcare System as part of a research protocol.

Methods

Measures included the ISI, Pittsburgh Sleep Quality Index (PSQI), Epworth Sleepiness Scale, Neurobehavioral Symptom Inventory, Beck Depression Inventory-II, Beck Anxiety Inventory, and PTSD Checklist–Military Version.

Results

The ISI demonstrated moderate to strong/excellent convergent and divergent validity. A principal component analysis indicated a single construct with excellent internal consistency (Chronbach’s alpha=0.92). In exploratory analyses, the ISI discriminated well between those with (73%) and without (27%) sleep disturbance based on the PSQI.

Conclusions

Results from this study indicate good validity of the ISI in assessing insomnia in Veterans with history of TBI and suggest a cut-off score not dissimilar from non-TBI populations. Findings from this study can help inform clinical applicability of the ISI, as well as future studies of insomnia in TBI.

INTRODUCTION

Insomnia symptoms are commonly reported by Veterans who have experienced traumatic brain injuries (TBIs) (Orff, Ayalon, & Drummond, 2009). Research suggests that upwards of 30–70% of patients with a history of TBI endorse chronic sleep problems, with over half reporting persistent insomnia (Castriotta & Murthy, 2011). Insomnia symptoms are known to be associated with and/or increase risk for psychiatric problems, suicidal ideation, and unhealthy lifestyles (e.g., alcohol/drug abuse), lead to poorer physical health, disruption in major social and occupational responsibilities, and decreased quality of life, and may contribute to the persistence of post-concussive symptoms beyond the expected period of recovery.

The identification of symptoms and diagnosis of insomnia is a multicomponent process requiring clinical judgement using information on lifestyle factors, comprehensive physical exams, polysomnography, actigraphy, sleep diaries, and most commonly patient report (Schutte-Rodin, Broch, Buysse, Dorsey, & Sateia, 2008). While all these factors are given differing degrees of weight when forming an insomnia diagnosis, there is no “gold standard” assessment for insomnia symptoms and physicians often have to rely on clinical judgement in reaching a diagnosis. As such, patient questionnaires on sleep disturbances and insomnia symptoms are critical tools to hone in on the type and severity of insomnia symptoms in Veterans with TBI, who commonly present with complex multi-symptom concerns (Orff et al., 2009).

Validation of self-reported measures of insomnia is important as these instruments may identify patients in greatest need for sleep treatments, and be useful for research in informing the allocation of participants to treatment groups. The Insomnia Severity Index (ISI; Bastien, Vallières, & Morin, 2001) is one of the most frequently used measures to assess the severity of and distress associated with insomnia symptoms in both general and clinical populations as well as in both civilian and military populations (Bryan, 2013; Jain, Mittal, Sharma, Sharma, & Gupta, 2014; Mollayeva et al., 2016; Spira, Lathan, Bleiberg, & Tsao, 2014; Sullivan, Berndt, Edmed, Smith, & Allan, 2016). Given the recognized importance of insomnia in TBI, the instrument has been increasingly used to study insomnia in Veterans who have sustained a TBI (Bryan, 2013; Spira et al., 2014). However, the ISI has yet to be validated for use in patients (and Veterans) with history of TBI. Validation of the ISI in Veterans with TBI will substantially help with research, assessment, and treatment of insomnia symptoms in this population. Additionally, while objective sleep measures can assess sleep-wake cycles and/or patterns, they cannot measure patient distress which can only be obtained through subjective report. As a number of self-report instruments have been validated for use in TBI patients (e.g., the Pittsburgh Sleep Quality Index) (Buysse, Reynolds, Monk, Berman, & Kupfer, 1989), validating the ISI against such instruments may yield useful insights about the ISI. The purpose of this study was to examine psychometric characteristics of the ISI in Veterans with a history of TBI, and to compare these properties to that of the PSQI.

METHODS

Participants

Study participants included 83 Operation Enduring Freedom/Operation Iraqi Freedom/Operation New Dawn (OEF/OIF/OND) Veterans with a history of TBI, as defined by VA/Department of Defense 2009 guidelines (Management of Concussion/mTBI Working Group, 2009). Veterans between 18–50 years old were recruited from the VA San Diego Healthcare System for participation in a parent research protocol. Exclusion criteria included non-English speakers, history of other neurological disorders and developmental disabilities, current (30 days) alcohol/substance dependence (or positive toxicology test at first visit), current/past history of bipolar, schizophrenia or other psychotic disorder, active suicidal ideation, and psychostimulant medication use. Participants provided written informed consent and the study protocol was approved by the VA Institutional Review Board. Table 1 shows demographic characteristics.

Table 1.

Demographic Characteristics (N=83).

Age, mean (SD); years	32.5 (6.8)
Education, mean (SD); years	14.2 (1.6)
Gender, n(%)
Male	72 (86.7)
Female	11 (13.3)
Months since TBI, mean (SD)	77.4 (48.5)
TBI Severity, n(%)
Mild	70 (84.3)
Moderate	11 (13.3)
Severe	2 (2.4)
TBI type, n(%)
Blast	18 (21.7)
Blunt	51 (61.4)
Blast only + secondary blunt	14 (16.9)
AOC%:LOC%	36.1%:63.9%
Posttraumatic amnesia, n(%)
No	33 (39.8)
Yes	49 (59.0)
Unsure	1 (1.2)

Note. TBI characteristics reported for most significant TBI. LOC=Loss of Consciousness; AOC=Alteration in Consciousness

Measures

Participants completed the ISI (Bastien et al., 2001), a seven-item questionnaire measuring insomnia symptoms and their impact on daytime functioning. Scores range from 0 to 28, and recommended interpretations include values of 0–7 as no insomnia, 8–14 as sub-threshold, 15–21 as moderate, and 22–28 as severe insomnia (Bastien et al., 2001). A prior psychometric study of the ISI by Morin et al. shows that scores >10 in community samples and scores >11 for those seen in generalized clinical settings reflect clinically significant insomnia (Morin, Belleville, Bélanger, & Ivers, 2011).

Convergent validity was measured with the global score of the PSQI (Buysse et al., 1989), a measure of sleep quality with scores ranging from 0–21 where higher scores indicate worse sleep quality. The PSQI was our primary convergent measure, as it has been validated in TBI populations based on a clinical diagnosis of insomnia (Fictenberg, Putnam, Mann, Zafonte, & Millard, 2001), and has been used as a validation measure for the ISI in generalized populations (Morin et al., 2011). Notably, the ISI measures patient “satisfaction” and “distress/worry” regarding insomnia symptoms, making a self-report measure of sleep disturbance and self-rated quality, rather than objective measures such as polysomnography, appropriate as a primary convergent measure. Convergent validity was also measured by correlating the ISI with the Epworth Sleepiness Scale total score (range 0 to 24; higher scores indicating greater sleepiness) (Johns, 1991), sleep item (#16: “changes in sleeping pattern” for past two weeks) score from the Beck Depression Inventory-II (range 0 to 3; higher scores indicating worse sleep) (BDI-II; Beck, Steer, Ball, & Ranieri, 1996), and the sleep item (#18: “Difficulty falling or staying asleep” since injury) score from the Neurobehavioral Symptom Inventory (range 0 to 4; higher scores indicating worse sleep) (NSI; King et al., 2012). In primarily descriptive analyses, we also assessed the ISI against the PSQI individual component scores (subjective sleep quality, sleep latency, sleep duration, sleep efficiency, sleep disturbance, sleep medication use, and daytime dysfunction).

Divergent validity measures included the Beck Anxiety Inventory (BAI; Leyfer, Ruberg, & Woodruff-Borden, 2006) total score, a measure of generalized anxiety (range: 0–63 with higher scores being worse anxiety), the modified PTSD Checklist-Military Version (PCL-M; Weathers, Huska, & Keane, 1991) (sum of items excluding sleep item #13), a measure of PTSD symptoms (range: 0 to 80 without sleep item, higher scores corresponding to worse PTSD symptoms), and the modified BDI-II (sum of items excluding sleep item #16), a measure of depressive symptoms (range: 0–60 without sleep item, higher scores being greater depression). Consistent with previous methodology (Bushnik, Englander, & Wright, 2008b; 2008a), modified versions were used in order to prevent overlap between constructs of sleep and psychiatric symptoms.

Analyses

Data were visually inspected, and Shapiro-Wilks tests conducted to assess data normality. The assumption of normality was found to be violated for several measures; thus, Spearman Rank Order Correlations were used. Based on a Bonferroni correction accounting for the four main convergent and three divergent measures (0.05/7), correlations were considered statistically significant at p<.007. Correlations between 0.1–0.3 were interpreted as small, 0.3–0.5 medium, and 0.5+ large effect sizes (Cohen, 1992). A principal component analysis (PCA) was conducted to assess ISI dimensions. Cronbach’s alpha was calculated to determine the internal consistency of ISI items. Finally, we conducted an exploratory analysis examining the Receiver Operating Characteristics (ROC) of the ISI. We evaluated the sensitivity and specificity of the ISI based on a cutoff of >8 on the PSQI. A PSQI cutoff score of >8 was found to be an appropriate cut-point of insomnia for individuals specifically with TBI, with sensitivity and specificity rates of 93 and 100%, respectively, based on clinician diagnosis as the gold standard (Fictenberg et al., 2001). While a PSQI cut-off of >5 is typically used, this score was derived from non-TBI samples (Buysse et al., 1989). Missing data were handled by listwise deletion, as <5% of data for any one variable were missing.

RESULTS

The mean ISI total score was 13.98 (SD=7.20). Based on the original (Bastien et al., 2001) ISI categorical scores, 17 (20.5%) participants were classified as having no insomnia, 24 (28.9%) as sub-threshold, 29 (34.9%) as moderate, and 13 (15.7%) as severe insomnia. Based on Morin’s cut-off point for clinical samples (>11), 67.5% were classified as having “clinical insomnia”.

The ISI total score was significantly associated with all convergent validity items (Table 2). The ISI was strongly correlated with NSI (ρ=.76) and BDI-II (ρ=.56) sleep items and the PSQI global score (ρ=.76), and moderately correlated with the ESS total score (ρ=.32). The ISI was moderately to strongly correlated (ρ’s >.32) with the PSQI individual component scores (except sleep medication use; ρ=.23), with a particularly strong association observed for subjective sleep quality (ρ=.79). Correlations between the ISI and divergent validity items were statistically significant, though slightly weaker (ρ’s = .45–.51).

Table 2.

Spearman correlations between ISI and measures indicating convergent and divergent validity.

	Mean (SD)	Range	ρ
ISI total score	13.98 (7.20)	0, 28	–
Convergent
NSI sleep item	2.23 (1.28)	0, 4	.764**
BDI-II sleep item	1.58 (0.93)	0, 3	.557**
ESS total	10.11 (5.14)	1, 24	.321*
PSQI global score	11.27 (4.23)	1, 19	.757**
Subjective quality	1.87 (0.83)	0, 3	.791**
Latency	2.01 (0.93)	0, 3	.491**
Duration	1.91 (1.06)	0, 3	.582**
Efficiency	1.08 (1.15)	0, 3	.324*
Disturbance	1.77 (0.71)	0, 3	.439**
Sleep medication	1.13 (1.27)	0, 3	.226
Daytime dysfunction	1.51 (0.92)	0, 3	.493**
Divergent
BAI total	12.99 (10.71)	0, 44	.450**
PCL-M total (sleep item excluded)	41.87 (16.85)	16, 76	.513**
BDI-II total (sleep item excluded)	19.65 (12.78)	0, 47	.476**

^**p≤0.001,

^*p≤0.007;

Higher scores for all measures represent worse symptomatology.

The PCA indicated that a 1-component model best explained total variance (Total Eigenvalues=4.83, percent variance explained=69.0). All ISI items loaded onto the single factor (factor loadings≥0.731) (Table 3). Chronbach’s alpha=0.92 indicating excellent internal consistency for the 7 items.

Table 3.

Principal component analysis for ISI items

ISI Item	Mean (SD)	Factor Loading
Difficulty falling asleep	1.57 (1.32)	0.752
Difficulty staying asleep	1.69 (1.18)	0.822
Problem waking up too early	1.61 (1.29)	0.731
Satisfied with current sleep pattern	2.78 (1.14)	0.865
Interfere with daily function	2.42 (1.24)	0.863
How noticeable impairment to others	1.92 (1.30)	0.894
How worried are you about sleep problem	1.99 (1.24)	0.873

Finally, in exploratory ROC analyses, we used the recommended PSQI cutoff score of >8 (Fictenberg et al., 2001) to indicate elevated insomnia symptoms (57 participants [73%] were classified as having elevated symptoms, whereas 21 [27%] did not). The area under the curve was 0.87 (95% CI=0.79, 0.95, p<0.001) (Supplemental Figure 1), suggesting a cut-off score of 11.5 (Sensitivity=81%, Specificity=71%).

DISCUSSION

In this study, the ISI was found to demonstrate good convergent and adequate divergent validity in a sample of Veterans with history of TBI. Findings also showed that the ISI is internally consistent and measures a univariate construct in our TBI sample.

As expected, the ISI was strongly correlated with sleep quality/disturbance as measured by the PSQI, as well as the changes in and severity of sleep problems as measured by the NSI and BDI sleep items. The ISI was only moderately associated with sleepiness as measured by the ESS and the correlation coefficient was within the same range as the correlations between the ISI and psychiatric (divergent) measures. This is consistent with previous research that supports a dissociation between sleepiness and sleep disturbance in TBI (Cantor et al., 2012). While psychiatric symptoms (anxiety, PTSD, depression) were associated with the ISI, these relationships were weaker than those of the sleep measures, suggesting that the ISI is related to, but does not simply measure psychiatric disturbance. Instead, the ISI appears to reflect a true “sleep disturbance” construct.

In our exploratory analysis, we found that a cutoff score of >11 may indicate elevated insomnia symptoms in Veterans with TBI. This cut-off is on par with the cut-off of >11 recommended for individuals seen in clinical settings (Morin et al., 2011), while incidentally, is lower than the >14 cut-off recommended in the published ISI guidelines (Bastien et al., 2001). Based on the former cut-off, our sample meets criteria for “clinical significant insomnia”, while with the latter, our TBI participants would be considered, on average, within the “subthreshold” level of insomnia. Based on our ROC results, our sample meets criteria for clinical insomnia and highlights the importance of establishing relevant cut-offs for specific populations. Moreover, the typical categorical cut-offs used to denote subthreshold, moderate, and severe insomnia will need to be examined in future studies to establish their utility in TBI.

Limitations of this study include generalizability to more severe TBI and non-Veteran populations. Veteran samples may represent a unique cohort, with differing comorbidities than civilian samples. For example, Veterans have particularly high prevalence of post-traumatic stress disorder and often experience chronic pain, commonly comorbid with each other (Lew et al., 2009) and with insomnia/sleep difficulties (Lew et al., 2010). Future research may seek to explore possible predictors, mediators, and moderators (e.g., psychiatric symptoms, age, and gender) of insomnia symptoms in individuals with TBI. Future research could also focus on the validation of the ISI in more severe TBI samples and those stratified by mild, moderate, and severe severity levels, as well as different TBI etiologies (e.g., blast and blunt trauma cases). Additionally, our sample size was somewhat small; however, our sample to item ratio was 11.9 participants per item. While inherent in neurotrauma research, TBI characteristics were obtained via retrospective self-report. Although we believe self-rated measures to establish convergent validity of the ISI is appropriate due to the self-rated quality of insomnia items inherent in the ISI, future studies in TBI populations may wish to explore the relationship between objective measures of sleep (e.g., polysomnography) and the ISI. In addition, future studies that compare various gold standard cut points and relevant cross-sectional and longitudinal outcomes in TBI samples may elucidate the utility of aforementioned ISI cut-off scores for “clinically meaningful insomnia”.

In sum, the results of this study provide preliminary validation of the ISI for assessing insomnia in Veterans with a history of TBI. Findings of this study may add considerable clinical utility in TBI-related insomnia symptom assessment and treatment as well as inform future research studies of insomnia in Veterans and TBI.