Psychometric Validation of the Insomnia Severity Index in Adults with Sickle Cell Disease

Abstract

Background

The Insomnia Severity Index (ISI) is an instrument to evaluate insomnia symptoms. The psychometric properties have not been established in adults (18 years of age or older) with sickle cell disease (SCD).

Objective

Evaluate the reliability and validity of the ISI among adults with SCD.

Methods

Analysis included psychometric evaluation with exploratory factor analysis.

Results

Our 263 participants had a mean age of 35.6 years and primarily were female (54.8%) with HbSS genotype (69.2%). Almost 41% were classified as clinical insomnia cases (ISI ≥14) using the traditional scoring approach. Two factors, Insomnia Symptoms and Insomnia Impact, emerged during factor analysis. Reliability of both factor-scales was good and each correlated with pain severity and depressive symptomatology (r = 0.38 to 0.66, p<.01).

Conclusion

The ISI demonstrated construct validity and reliability for evaluating insomnia symptomatology among adults with SCD and can be used in research and clinical practice.

Sickle cell disease (SCD) is a group of autosomal recessive hemolytic genetic disorders.¹ In the United States, approximately 100,000 Americans are estimated to have sickle cell disease.² Individuals with SCD experience a variety of disorder-related symptoms including sleep disturbances, where prevalence estimates of up to 70% have been reported.^3–5 Two common sleep disturbances are difficulty initiating sleep^6,7 and maintaining sleep,⁶ which are key symptoms of insomnia when they cause distress and/or affect daytime function.⁸ The etiology of sleep disturbances has been attributed to nocturnal hypoxemia secondary to obstructive sleep apnea,⁶ depression,^3,5 and pain.^3,5,9

Polysomnography (PSG) is the gold standard for objectively measuring sleep and diagnosing sleep disorders. However, PSGs are not currently indicated for routine diagnosis of insomnia in the absence of risk factors. Diagnosis of insomnia is routinely made through clinician interviews,¹⁰ yet interviews are time-consuming and require the clinician to have in-depth knowledge about insomnia.¹¹ Thus, self-report remains the primary approach to screening and quantifying insomnia severity.¹¹ In routine clinical practice settings, brief screening instruments that have been shown to be sensitive to interventions, such as the Insomnia Severity Index (ISI), can be extremely useful.

The ISI is a seven-item self-report instrument that measures perceived insomnia symptoms, concern or distress, and interference with daily functioning over the past two weeks.^11,12 Concurrent and convergent validity with other measures of insomnia symptoms including PSG parameters, sleep diaries, and clinician interviews have been established for this instrument.^11,12 The ISI, which was originally validated in a group of patients with insomnia (mean age of 65 years),¹² has since been validated in adults without sleep disturbances,^11,13 adolescents,¹⁴ and individuals with cancer.¹⁵ It has been translated into several languages.^14,16–19 Overall, the ISI has been widely used to evaluate insomnia symptom severity and treatment response to both sedative hypnotics and behavioral treatments for insomnia.¹²

After a review of the literature, to our knowledge, although widely used in research and practice, the ISI has only been validated in one chronic disease population (cancer patients)¹⁵ and has not been validated to assess subjective sleep complaints in individuals with SCD, a unique chronic pain population. Furthermore, both two-and three-factor solutions for the ISI have been proposed using factor analytic approaches, suggesting the dimensionality of this measure may vary from population to population. Thus, before recommending routine use of the ISI to evaluate insomnia symptoms in patients with SCD, the psychometric properties of the ISI among this patient population should be evaluated.

Methods

Participants and setting

Participants are a subsample from the Improving Patient Outcomes with Respect and Trust (IMPORT) study. The IMPORT study is a federally funded, multi-site longitudinal observational study designed to evaluate SCD patients’ experiences with respect and trust in routine health care environments and its effect on adherence to therapy, appropriateness of care, and health outcomes.

Participants were a convenience sample of individuals recruited from SCD clinics in the Mid-Atlantic area. A total of 291 individuals who were ≥16 years of age, had a SCD hemoglobinopathy (genotype verified by medical record review), and provided informed consent were included in the parent study. For this analysis, 263 participants who were ≥18 years of age at enrollment and provided complete data on the ISI during the baseline visit were included. The institutional review boards at the recruitment study sites approved all study procedures.

Measures

Insomnia Severity Index (ISI)

The ISI measures perceived insomnia symptomatology.¹² Each item is scored on a five-point likert scale from zero (“none” or “not at all”) to four (“extremely”). The traditional scoring approach is to sum scores across all items. Higher total scores (range zero to 28) indicate greater insomnia symptom severity.¹² Comparison between ISI scores, PSG parameters, and clinician interviews has helped to identify clinical cut-off scores for identifying cases of insomnia using the traditional scoring approach. Estimates vary across studies, but ISI scores of 10 or greater appear to be optimal for identifying clinically significant insomnia symptoms in non-sleep clinic samples,^11,13,15 while scores of 14 or greater identify clinical insomnia cases with a high degree of accuracy.^11,13,15 Overall, the ISI has demonstrated good reliability across population groups (Cronbach’s alpha 0.74–0.91).^{11–15,17–19}

Multiple studies have explored the factor structure of the ISI. The developers of the instrument reported a three-factor structure (i.e., severity of symptoms, satisfaction with sleep, and distress associated with sleep difficulty) among a sample of participants with insomnia.¹² This factor structure was replicated by the developers of the Spanish version of the ISI.¹⁹ In four additional studies, however, a two-factor structure was found (i.e., severity of symptoms and impact of sleep difficulties) among samples of adults with breast and prostate cancer,¹⁵ Persian sleep clinic patients,¹⁸ Chinese school-based adolescents,¹⁴ and older adults.¹⁷ Despite the aforementioned factor solutions, clinical evaluations and research and have predominately used the ISI as a unidimensional instrument.

Center for Epidemiology Studies in Depression (CESD-10)

The CESD-10 measures the degree of depressive symptomatology experienced over the previous week.²⁰ Each item is scored on a four-point likert scale from zero (“Rarely or None of the time”) to three (“Most or all of the time”). In general, higher total scores across items indicate a greater degree of depressive symptomatology.²¹ The CESD-10 is a valid and reliable measurement tool with good internal consistency in our sample (Cronbach’s alpha = 0.83).

Brief Pain Inventory (BPI)

The BPI measures clinical pain in patients with acute and chronic pain conditions.²² The BPI measures two distinct constructs of the pain experience: perceived pain severity (four items) and impact of pain on daily functioning (seven items). For our analysis, items from the pain severity subscale were used. The pain severity subscale measures “worst,” “least,” “average,” and “current” pain severity experienced within the past 24 hours on a numerical rating scale (range zero (“no pain”) to 10 (“as bad as you can imagine”)). A BPI severity composite score (mean severity score) was calculated, with higher scores indicating greater pain severity.²² The BPI has been psychometrically and linguistically validated and was a reliable measure of clinical pain in our sample (Cronbach’s alpha for the BPI pain severity subscale was 0.87).

Demographic characteristics

Demographic information was assessed at the baseline visit. Characteristics including age, gender, education, and employment status were assessed with a self-report demographic questionnaire.

Analysis

All statistical analyses were performed using Stata statistical software version 13.²³ Descriptive summary statistics were used to examine demographic and clinical characteristics of the sample. Insomnia symptom characteristics were also initially described using the traditional summary (one-factor) score in order to compare our sample’s scores with published scores from other non-SCD populations, which have used the traditional scoring approach.

We then used two common methods to assess the ability of the ISI to measure the construct insomnia symptomatology and underlying dimensions of this construct. We conducted a reliability assessment using measures of internal consistency. We also assessed factorial and construct validity using factor analysis and by examining the correlation between the ISI and theoretically-related constructs. As a general rule, when conducting factor analyses, at least 10 participants per item²⁴ or greater than 200 participants is recommended and ideal.²⁵ Our sample size of 263 participants provided a sufficiently large sample to perform the subsequent analyses.

To explore the factor structure of the ISI, we used exploratory factor analysis (EFA) with maximum likelihood estimation. We examined the determinant of the correlation matrix, Bartlett’s test of sphericity, and the Kaiser-Meyer Olkin (KMO) test to assess the adequacy of the correlation matrix for conducting factor analyses. Factors were extracted and refined using principal axis factoring with oblique promax rotation since we hypothesized that the factors would be correlated. The number of factors retained was determined by the eigenvalue > 1.0 guideline, examination of Akaike’s Information Criterion (AIC) values, and interpretability of the factors.

After identifying the factor structure, we created composite factor-scales for each identified factor by calculating mean scores across items that loaded >.40 on each factor.²⁶ Reliability and validity analyses were performed on the resulting factor-scales. To assess reliability, a Cronbach’s alpha was calculated for each factor-scale. Spearman’s correlation was used to examine construct validity between each subscale and theoretically-related constructs (i.e., clinical pain severity and depressive symptomatology).

Results

Descriptive and clinical characteristics of the sample are displayed in Table 1. Participants ranged from 18 to 70 years of age (mean age 35.6 years). Most of the sample was female (54.8%), had HbSS genotype (69.2%), had a high school education (61.2%), and were disabled (43.0%). There was good variability in responses for the CESD-10 and BPI severity scale with scores ranging from the lowest to the highest for each scale. Using the traditional scoring approach, approximately 40.7% of the sample would be diagnosed with clinical insomnia (ISI summary score of 14 or greater). This estimate is well above the national prevalence of chronic insomnia (10%),²⁷ but similar to the prevalence of general sleep complaints reported by individuals with SCD.^3,4

Table 1.

DESCRIPTIVE CHARACTERISTICS OF THE STUDY PARTICIPANTS (N=263)

Characteristic	n (%)
Age, years, Mean (SD)	35.6 (11.8)
Female	144 (54.8)
Education Level
High School or less	161 (61.2)
Some College	44 (16.7)
College Graduate	54 (20.5)
Refused to Answer	4 (1.5)
Employment Status
Working	94 (35.7)
Not Working	36 (13.7)
Disabled	113 (43.0)
Retired	9 (3.4)
Other	11 (4.2)
SCD Hemoglobinopathy
HbSS	182 (69.2)
HbSC	55 (20.9)
HbSBeta (0) Thalassemia	4 (1.5)
HbSBeta (+) Thalassemia	22 (8.4)
BPI Severity Score, median (range)	4 (0–10)
CESD-10 Score, median (range)	8 (0–29)

Examination of the ISI correlation matrix indicated that all items significantly correlated with other items in the matrix (range 0.44–0.75). In addition, the determinant of the correlation matrix was 0.013 and the Bartlett’s test of sphericity was significant (χ² = 1129.1, p <.05), which also supported the adequacy of the correlation matrix to conduct factor analyses. Finally, the KMO statistic was 0.908, which suggested a high degree of shared variance and adequacy for factor analysis.

Initially, we examined results for a one-factor model. The one-factor solution explained 91.2% of the variance among ISI items (eigenvalue = 4.2). We then tested a two-factor model using oblique promax rotation. This model identified two distinct factors we named Insomnia Symptoms and Insomnia Impact (see Table 2). Each factor accounted for approximately one half of the total variance (50.9% and 49.1% of variance, respectively). The factor Insomnia Symptoms was composed of the first three items of the ISI (i.e., Difficulty falling asleep, Difficulty staying asleep, and Problem waking up too early), while Insomnia Impact was composed of the last four items (i.e., Satisfaction with current sleep problem, Interference with daily functioning, Noticeability of sleeping problem, and Worry about current sleep problem). Examination of AIC values for the one and two-factor models suggested that the two-factor model was the best fit for the data (AIC = 34.9, df = 13). The mean composite score for the Insomnia Symptoms factor-scale was 1.50 (range zero to 4.0), and the mean score of the Insomnia Impact factor-scale was 1.66 (range zero to 4.0). Table 3 shows a significant correlation between the two factor-scales (r = 0.77, p <.001).

Table 2.

ISI ROTATED TWO-FACTOR PATTERN MATRIX: PRINCIPAL AXIS FACTORING WITH OBLIQUE PROMAX ROTATION

ISI Items by Factor	Factors Loadings
	I	II
I. Insomnia Symptoms
1. Difficulty falling asleep	0.53	0.33
2. Difficulty staying asleep	0.93	0.04
3. Problem waking up to early	0.67	0.06
II. Insomnia Impact
4. Satisfaction with current sleep pattern	0.29	0.47
5. Interference with daily functioning	0.07	0.81
6. Noticeability of sleep problem	0.01	0.79
7. Worry about current sleep problem	0.23	0.64

Table 3.

FACTOR CORRELATIONS AND FACTOR ALPHA COEFFICIENTS FOR THE COMPOSITE SUB-SCALES (N=263)

Factor	Mean Composite Score**	SD	Factors
			I	II
I. Insomnia Symptoms (3 items)	1.50	1.07	(0.85)
II. Insomnia Impact (4 items)	1.66	1.05	0.77*	(0.87)

SD = Standard Deviation

^*p <.01

^**Range: 0.00 to 4.00

Reliability estimates for the two factor-scales are presented in parentheses on the diagonal in Table 3. Reliability statistics demonstrated good internal consistency among items in each factor-scale (Cronbach’s alpha =0.85 and 0.87, respectively). Additionally, the correlation between ISI subscales and theoretically-related constructs was also examined. Results revealed Insomnia Symptoms significantly correlated with pain severity (r = 0.41, p<.01) and depressive symptomatology (r = 0.60, p<.01). Furthermore, Insomnia Impact significantly correlated with pain severity (r = 0.38, p<.01) and depressive symptomatology (r = 0.66, p<.01). Both results supported the construct validity of each factor-scale.

Discussion

Results of this analysis suggest the ISI has good internal consistency and construct validity among our sample of respondents with SCD. We suggest, then, that the ISI is a valid and reliable instrument for evaluating insomnia symptoms in research and clinical practice involving patients with SCD.

Using exploratory factor analysis, we identified two distinct factors, Insomnia Symptoms and Insomnia Impact, among the seven items of the ISI. Our results differ slightly from those of most studies with two-factor solutions for the ISI, notably Savard et al.,¹⁵ Sadeghniiat-Haghighi et al.,¹⁸ and Yu.¹⁷ Similar to Chung et al.,¹⁴ we found that the item “Satisfaction with current sleep problem” uniquely clustered with other items that constituted Insomnia Impact rather than Insomnia Symptoms. We believe that “Satisfaction with sleep pattern” conceptually and empirically fits better with Insomnia Impact as satisfaction with sleep is not an insomnia symptom (i.e., sleep initiation or sleep maintenance disturbance), but rather a degree of happiness or pleasure associated with one’s sleep.

Our analysis has potential limitations. First, while our sample of 263 participants is within the recommended sample size required for exploratory factor analyses, a larger sample size (more than 500 participants) would have enhanced the precision of our estimates.²⁵ Second, we investigated the reliability and validity of the ISI in a sample of adults with SCD for whom a confirmed diagnosis of insomnia had not been established. Our results cannot be used to determine the accuracy of the ISI for identifying individuals at risk for clinical insomnia or whether individuals with SCD and clinically diagnosed insomnia would respond differently than individuals with SCD and without clinically diagnosed insomnia. Third, the ISI was the only instrument used to evaluate insomnia symptoms therefore we were unable to establish concurrent validity between the ISI and other measures of insomnia symptomatology. Finally, the present analysis is a cross-sectional investigation of the validity and reliability of the ISI therefore, we are unable to determine the stability over time of the ISI to measure insomnia symptomatology in this population.

Despite these limitations, our results have important implications for clinicians and researchers working with adults with SCD. For clinicians, screening for insomnia symptoms is important. Many studies have noted a bi-directional relationship between sleep disturbances such as insomnia and pain,²⁸ which is a primary symptom in this population.²⁹ This is the first study we are aware of to validate psychometrically any sleep assessment tool in this population. Through our present analysis, we found that the ISI is at least one reliable and valid tool, which clinicians and researchers can use to assess insomnia symptoms in this population. Further, we found that the seven items of ISI can be divided into two unique subscales. While clinicians can administer the full seven-item scale, our analysis provides evidence that clinicians can use the first three items when assessing the presence and severity of insomnia symptoms in this population. This is an important finding, when considering the brevity of the patient-provider interaction, which exists in many patient care environments. Finally we suggest that researchers evaluate both ISI factor-scales as individual predictors or outcomes within the context of multivariate or multivariable regression analyses. For example, among patients with SCD and insomnia, research could examine if the Insomnia Impact is a predictor of health care utilization³⁰ or quality of life. Additionally, it might examine whether treatment of nocturnal hypoxemia secondary to obstructive sleep apnea or depression leads to improvements in Insomnia Symptoms, which could elucidate specific treatment options for patients with SCD and comorbid insomnia. To our knowledge these types of analyses have not been conducted to date, yet are important for evaluating potential risk factors and the effect interventions to reduce insomnia.