Validation of the Three-Item Insomnia Severity Index Short Form in Young Adult Cancer Survivors: Comparison with a Structured Diagnostic Interview

Abstract

Chronic insomnia affects ∼25% of young adult cancer survivors (YACS) but is often overlooked in routine follow-up. A recently introduced three-item version of the Insomnia Severity Index (ISI-3) was compared with a diagnostic interview (SCID-5) in 250 YACS (ages 18–40) to evaluate its validity in this population. The ISI-3 had good discrimination compared with the SCID-5 (area under the receiver operating characteristic curve = 0.88). Although no ISI-3 cutoff met study criteria for both sensitivity (≥0.85) and specificity (≥0.75), an ISI-3 cutoff of ≥4 had high sensitivity (94%) and moderate specificity (70%), and is recommended as the first step in a two-step screening procedure.

Introduction

After completion of therapy, young adult cancer survivors (YACS) are at risk for chronic insomnia, which limits their ability to resume premorbid activities and achieve critical developmental milestones.^1–4 Despite the importance of sleep to development, lack of guidance on appropriate insomnia screening measures contributes to underidentification and undertreatment in YACS.^5–8 One potential screening option is the Insomnia Severity Index (ISI), a well-respected measure of insomnia widely used in studies of cancer survivors.^5,9,10 Indeed, a prior study by our team supported validity of the ISI for identifying insomnia disorders in YACS specifically.¹⁰

Brief screening measures are critical tools for improving identification of insomnia, particularly in populations such as YACS, where numerous and complex medical needs may “crowd out” other concerns such as sleep. For that reason, a recent study supporting the validity of an abbreviated three-item version of the ISI (ISI-3) for identifying insomnia in older adults may have important implications for identifying insomnia in YACS.¹¹ To our knowledge, the ISI-3 has been validated only in studies comparing it with the standard ISI in nononcology samples of older adults.^11,12

However, if found to be valid in YACS, the newly introduced ISI-3 has the potential to provide a highly practical and efficient method of insomnia screening for use in clinical settings serving YACS. To address this question, this study set out to validate the ISI-3 in YACS by comparing it with a structured diagnostic interview. Based on prior results supporting validity of the standard ISI in YACS, and the ISI-3 in older adults, we expected to find the ISI-3 to be strongly associated with insomnia disorder in YACS. However, as cutoff scores can operate differently depending on the population, an important goal of the study was to identify ISI-3 cutoff scores with greatest utility for screening YACS.

Methods

Participants were 250 YACS in E-Quest Stress and Coping, a study on the validity of self-report measures of stress symptoms in YACS.¹⁰ YACS were recruited during scheduled oncology clinic visits at a single cancer center.

After consenting, participants completed demographic questionnaires and the ISI⁹ on paper forms; questionnaires were then reviewed by study staff and participants were asked to complete any missing items if possible. All participants were subsequently administered the insomnia disorder module of the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) (SCID-5) by a single trained clinical research coordinator who was blind to participant responses on all study measures.^13–15 All procedures were approved by the Dana-Farber Cancer Institute's Institutional Review Board; please refer to Michaud et al.¹⁰ for additional details on study procedures.

Measures

Insomnia Severity Index

The ISI is a seven-item self-report measure evaluating insomnia symptoms over the prior 2 weeks. It includes items on difficulty sleeping (e.g., problems falling/staying asleep), satisfaction with current sleep, worry about sleep, and interference of sleep problems with daily functioning. Items are rated on a Likert scale (0–4 points) and summed to obtain a total score, with higher scores indicating greater insomnia severity. Consistent with prior research,¹¹ ISI-3 scores were derived by summing ISI items no. 4 (sleep dissatisfaction), no. 6 (sleep worry), and no. 7 (insomnia interference). These ISI-3 items were originally selected by Thakral et al.¹¹ because they had the highest item-total correlations with the total ISI score. The ISI-3 was found to be a valid measure of insomnia in older adults in a primary care setting with a recommended screening cutoff of ≥7.

SCID-5 Insomnia Disorder Module

The SCID-5 is a widely used semistructured clinical interview based on DSM-5 criteria.^13–15 The Insomnia Disorder Module begins by asking respondents if they have been “dissatisfied” with their sleep in the prior 3 months. Respondents who report dissatisfaction are then administered six additional questions about difficulty falling asleep, difficulty staying asleep, early morning awakening, normal bed time and wake times, consequences of sleep problems, and frequency/duration of sleep problems. Responses to these questions were scored according to standard SCID-5 algorithms and participants were assigned a SCID-5 insomnia disorder diagnosis if they met all criteria.

Statistical analyses

Participants' demographic, medical, and insomnia characteristics were described, and corrected item-total correlations were used to quantify the relationship between ISI items and the standard ISI total score. Area under the receiver operating characteristic curve (AUC) quantified discrimination of the ISI-3 compared with SCID-5 criteria.

Potential ISI-3 cutoff scores were evaluated for correspondence with the SCID-5 by calculating three conditional probabilities: (1) sensitivity (i.e., true positive rate), reflecting likelihood of correctly identifying participants with insomnia disorder; (2) specificity (i.e., true negative rate), reflecting likelihood of correctly identifying participants without insomnia disorder; and (3) total percent correct, reflecting proportion of correct screening results overall. Consistent with screening recommendations and previous studies,^10,16 we specified a priori that a cutoff score on the ISI-3 with sensitivity ≥0.85 and specificity ≥0.75 would be acceptable.

Results

Participants were 125 males and 125 females aged 18–40 (mean [M] = 29.36, standard deviation [SD] = 6.16); 50% were diagnosed with cancer before age 21, and 80% were diagnosed before age 30. Participants' first cancer diagnoses included lymphoma (n = 106, 42%), leukemia (n = 49, 20%), breast cancer (n = 27, 11%), sarcoma (n = 27, 11%), and other solid tumors (n = 41, 16%). Treatment included chemotherapy (89%), radiation (53%), surgery (52%), and bone marrow or stem cell transplant (20%). Participants were an average of 9.63 years (SD = 7.67, range = 8 months to 35 years) from their first cancer diagnosis, and their time since end of treatment was classified as follows: 6–23 months (19%), 2–4 years (32%), 5–7 years (13%), 8–10 years (12%), or >10 years (25%). ISI-3 total scores ranged from 0 to 12 (M = 3.49, SD = 2.76) and 52 (20%) participants met SCID-5 criteria for insomnia disorder.

Using the standard ISI total score, corrected item total correlations were calculated for the seven standard ISI items. Consistent with Thakral et al.,¹¹ the standard ISI items no. 6 (sleep worry; r = 0.83), no. 7 (insomnia interference; r = 0.76), and no. 4 (sleep dissatisfaction; r = 0.76) were most highly correlated with the total ISI score and were used as the ISI-3 items in this study.

The ISI-3 demonstrated good overall discrimination compared with the SCID-5 (AUC = 0.88). In contrast to a prior study of the ISI-3 where a cutoff score of ≥7 was reported to maximize sensitivity in a sample of older adults in a primary care setting,¹¹ a ≥ 7 cutoff had unacceptably low sensitivity in this YACS sample (42%). Although no ISI-3 cutoff score met a priori criteria for sensitivity and specificity, a ≥ 4 cutoff came closest with sensitivity of 94% and specificity of 70% (Table 1). These results were similar to our previous report using the standard ISI with YACS where a cutoff score of ≥8 had a slightly lower sensitivity (85%) and slightly higher specificity (77%).¹⁰

Table 1.

Sensitivity and Specificity of the ISI-3 for Detecting Survivors with SCID-5 Insomnia Diagnoses

ISI-3 total scores	Insomnia diagnosis n = 52 (of 250)
Alternative cutoff values	Sensitivity (95% CI)	Specificity (95% CI)	% correct
≥2	0.98 (0.88–1.00)	0.39 (0.33–0.47)	51.6
≥3	0.98 (0.88–1.00)	0.58 (0.51–0.65)	66.4
≥4	0.94 (0.83–0.99)	0.70 (0.63–0.76)	74.8
≥5	0.77 (0.63–0.87)	0.81 (0.74–0.86)	80.0
≥6	0.64 (0.49–0.76)	0.88 (0.82–0.92)	82.8
≥7	0.42 (0.29–0.57)	0.93 (0.89–0.96)	82.8
≥8	0.35 (0.22–0.49)	0.96 (0.91–0.98)	82.8

CI, confidence interval; ISI-3, Insomnia Severity Index Short Form; SCID, Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorder, 5th Edition.

To demonstrate practical implications of using the ISI-3 with YACS, we applied the ≥4 cutoff to a hypothetical example of 100 survivors (20 with an insomnia diagnosis; Fig. 1). In this example, of the 20 YACS with a SCID-5 insomnia disorder 19 (94%) would be correctly classified as having an insomnia disorder and appropriately referred for services (i.e., “True Positives”); one survivor with insomnia disorder would be incorrectly classified and not referred (i.e., “False Negative”). Of the 80 YACS without insomnia disorder, 56 (70%) would be correctly classified and appropriately not referred for services (i.e., “True Negatives”), but 24 YACS without an insomnia diagnosis would be incorrectly classified as having an insomnia disorder and erroneously referred (i.e., “False Positives”).

FIG. 1.

Expected clinical decisions using the ISI-3 to screen for a diagnosis of insomnia (cutoff ≥4). ISI-3, Insomnia Severity Index Short Form.

Overall, of the 43 survivors referred for services (19 True Positives and 24 False Positives), only 44% would have a SCID-5 insomnia disorder diagnosis. For comparison, results are similar to those reported when the standard ISI (with cutoff score of ≥8) was applied to this same hypothetical example; of 20 YACS with insomnia disorder 17 were true positives and 3 false negatives, and of 80 YACS without insomnia disorder 62 were true negatives and 18 were false positives.¹⁰ Please refer to Michaud et al.¹⁰ for additional details.

Discussion

Approximately 20%–40% of YACS experience sleep disturbances,^17,18 but they are rarely screened for insomnia due to limited availability of accepted measures and expertise in sleep assessment.^5–8 Moreover, as YACS are at risk for a variety of late effects, insomnia can be overlooked by both patients and their providers who may need to evaluate multiple organ systems in a single survivorship visit.^17,19,20 Our results demonstrating the ISI-3 has good discrimination compared with a diagnostic interview measure of insomnia provide important new information supporting its use in this setting.

Specifically, findings indicate the cutoff (≥7) validated in an older adult sample is not appropriate for YACS, but that an alternative cutoff (≥4) can be recommended. Brief measures, particularly “ultrashort” measures (≤4 items)²¹ such as the ISI-3, may be critical in effective screening of YACS. If every screening measure is reduced to ≤4 items while maintaining validity, significant progress could be made in evaluating the more than 120 potential late effects^6,22 recommended for screening by clinical care guidelines.

Although a cutoff score meeting sensitivity and specificity criteria for a stand-alone screening measure was not identified, the high sensitivity and moderate specificity of the ≥4 cutoff on the ISI-3 makes it well suited as the first step in a two-step screening procedure. After using the ISI-3 to identify YACS in need of additional insomnia screening in Step-1, high risk individuals are administered a second screen with high specificity to rule out false positive cases and prevent unnecessary referrals. Limiting this second step to those at high risk for insomnia limits survivor burden and clinic resources.

Although the ISI might seem a reasonable choice for this second screening step, it offers only a modest increase in specificity over the ISI-3 (77% vs. 70%).¹⁰ For that reason, we recommend the insomnia disorder module of the SCID-5 or another brief clinical interview for this second step. The SCID-5 module is typically completed in <10 minutes and is designed to be administered by clinicians familiar with diagnostic criteria for insomnia disorder but does not require specialized training in sleep medicine.¹⁶ Brief clinical interviews such as the SCID-5 insomnia module that gather information on duration and frequency of sleep problems may be particularly helpful as this information is critical for diagnosis¹⁵ and not well captured by the ISI or ISI-3.

Results must be considered in the context of study limitations, including a convenience sample of YACS drawn from a single institution. In addition, ISI-3 items were not administered alone, but as part of the full ISI, which may have influenced results. Future research of the ISI-3 as a stand-alone screen in YACS with broader demographics will be important to evaluate generalizability of findings. Although the ISI and SCID-5 were administered on the same day, they have different reference periods (2 weeks and 3 months, respectively) which may reduce agreement between them.

Despite these limitations, this study provides new information supporting use of the ISI-3. Until now, the ISI-3 has been validated against the standard ISI,¹¹ but this approach can inflate validity estimates because of shared items; validation against a diagnostic interview reported in this study should reinforce confidence in use of the ISI-3. In addition, results provide new information critically important for using the ISI-3 to address the current gap in insomnia screening and treatment for YACS.⁷ Whereas the ISI-3 has only been validated in older adult samples to date, our results support its use in YACS using a cutoff score specifically validated in this population. As the ISI-3 can be administered without specialized sleep training, it may be particularly helpful in increasing provider confidence in assessing sleep and making appropriate referrals to sleep experts.

Authors' Contributions

Formal analysis, writing—original draft, writing—review and editing, and visualization by L.L.C. Formal analysis, data curation, writing—review and editing, and visualization by A.L.M. Conceptualization, methodology, writing—review and editing by E.S.Z. and G.C. Conceptualization, methodology, formal analysis, investigation, resources, writing—original draft, writing—review and editing, visualization, supervision, project administration, and funding acquisition by C.J.R.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This study was supported by the National Cancer Institute (1R21CA223832) and Swim Across America. Sponsors played no role in study design, implementation, data analysis, or writing and submitting of the article.