The Sleep Condition Indicator: a clinical screening tool to evaluate insomnia disorder

Abstract

Objective

Describe the development and psychometric validation of a brief scale (the Sleep Condition Indicator (SCI)) to evaluate insomnia disorder in everyday clinical practice.

Design

The SCI was evaluated across five study samples. Content validity, internal consistency and concurrent validity were investigated.

Participants

30 941 individuals (71% female) completed the SCI along with other descriptive demographic and clinical information.

Setting

Data acquired on dedicated websites.

Results

The eight-item SCI (concerns about getting to sleep, remaining asleep, sleep quality, daytime personal functioning, daytime performance, duration of sleep problem, nights per week having a sleep problem and extent troubled by poor sleep) had robust internal consistency (α≥0.86) and showed convergent validity with the Pittsburgh Sleep Quality Index and Insomnia Severity Index. A two-item short-form (SCI-02: nights per week having a sleep problem, extent troubled by poor sleep), derived using linear regression modelling, correlated strongly with the SCI total score (r=0.90).

Conclusions

The SCI has potential as a clinical screening tool for appraising insomnia symptoms against Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria.

Strengths and limitations of this study.

• Existing instruments used to evaluate insomnia lack specificity or do not permit assessment against the latest diagnostic criteria.

• This study describes the development and validation of a new instrument (the Sleep Condition Indicator (SCI)) for use in everyday clinical practice.

• The SCI is valid, reliable and sensitive to change in insomnia severity. Its brevity and appealing visual format permit rapid assessment and interpretation of poor sleep against contemporary clinical diagnostic criteria.

• While we have used large surveys and treatment evaluation to assess SCI properties, more work is required to assess predictive validity with reference to independent clinical evaluation of insomnia disorder.

Introduction

Although insomnia is the most common of all mental health problems,¹ it is seldom adequately assessed and treatment services are often poor.^{2 3} This perhaps reflects the perspective that insomnia is usually a symptom,⁴ coupled with minimal medical education on sleep and its disorders.⁵ However, there are three reasons why this perspective must now change. First, insomnia is not merely a symptom. It has for some time been proposed as a genuine diagnosis (see Harvey review),⁶ and recently the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) Work Group recognised that the previous dichotomy of primary versus secondary insomnia is not evidence based.^{7 8} Accordingly, DSM-5 now recommends that ‘insomnia disorder’ should be coded “whenever diagnostic criteria are met whether or not there is a co-existing physical, mental or sleep disorder”.⁹ Second, insomnia is not necessarily transient or benign. Once established, it is remarkably persistent,^{10 11} constituting a risk factor for the development of physical and mental health problems, notably depression,^12–14 as well as adverse effects on quality of life.^{15 16} Chronic insomnia is also associated with high societal cost¹⁷ and is, for example, a robust predictor of work disability.¹⁸ Third, insomnia is treatable. There is a very substantial level 1 evidence-base, evaluating pharmacological and cognitive–behavioural therapies (CBTs),^{19 20} although the latter are very seldom available.^{3 21}

Insomnia is ubiquitous,²² so it is important that clinicians, general practitioners in particular, have a reliable, valid and brief screening tool. A wide range of such instruments is a standard part of patient-centred care,²³ for example, for depression,²⁴ anxiety²⁵ and alcohol problems.²⁶ In the insomnia field, two scales in particular are widely used: the Pittsburgh Sleep Quality Index (PSQI)²⁷ and the Insomnia Severity Index (ISI).²⁸ The PSQI is an established research tool, which has a cut-off score indicative of sleep disturbance. However, it lacks specificity for insomnia. The ISI is very sound psychometrically, and is more specific and based on DSM-IV criteria for insomnia. It is used in the main to select people for clinical trials and as an outcome measure.

In this paper, we present a further measure (the Sleep Condition Indicator (SCI)) that may have some useful features. The SCI is informed by the development phase for DSM-5 insomnia disorder,^{7 8} coupled with published research diagnostic criteria²⁹ and recommended quantitative parameters for sleep disturbance.³⁰ In keeping with DSM-5, the SCI also evaluates associated daytime factors, which are important drivers of clinical symptoms²² and should be incorporated in insomnia measurement.^{15 16 31} In terms of utility, the SCI is brief but versatile. It yields (1) a dimensional perspective on sleep quality: on an intuitive, global scale where higher scores represent better sleep; (2) a visual profile of night-time and daytime symptoms that the clinician can use in consultations and (3) indicative cut-off points for clinically-significant insomnia. This paper summarises the development and evaluation of the SCI across several studies, and in doing so addresses two major questions. First, does the SCI have adequate psychometric properties? Second, is it possible to derive an even briefer, short form, SCI that has similar psychometric characteristics?

Methods

Sample characteristics

Data are reported from five validation studies (total n=30 941; 71% female) in which SCI items were administered. The Great British Sleep Survey (GBSS) was an open access, web-based survey completed by adults (18+years) with a UK postcode yielding data on 12 628 participants (72% female; mean age=38.7 years (SD=14.5)) between February 2010 and August 2011.³² The GBSS+ was a revision of the GBSS, extended to any valid zip code worldwide, from May 2011 to March 2012 (n=11 017; 68% female; 42.3 years (16.5)). The TV sample was obtained in response to a network programme (the Food Hospital, Channel 4) on the sleep benefits of tart cherry juice (n=6876; 76% female, 36.4 years (13.3)). Glasgow Science Centre data (n=256; 56% female, 40.3 years (14.9)) were collected in 2009–2010 during a study which assessed the relationship between salivary α-amylase, sleep pressure and diurnal preference.³³ Finally, a randomised controlled trial (RCT) sample comprised 164 participants (72% female, 48.9 years (13.7)) recruited into a placebo-controlled evaluation of CBT for insomnia.³⁴ Ethical agreement concerning the latter two studies is provided in the source manuscripts. For the open access web surveys, participation was covered by the site terms.

Across our combined largest samples (GBSS, GBSS+, TV), women were slightly younger than men (38.5 (14.9) vs 40.1 (14.7) years; t(27 638)=7.94, p<0.0001) and the great majority was in average or better physical health (86%) and mental health (82%; 5-point scale: 0 ‘very good’, 1 ‘good’, 2 ‘average’, 3 ‘poor’ and 4 ‘very poor’). Respondents who made some use of prescription sleeping pills (9.1%) were 7 years older than those who did not (46.2 (15.1) vs 39.1 (15.1) year, t(20 813)=19.6, p<.0001) and had a substantially poorer SCI score (8.56 (4.93) vs 15.6 (7.80), t(20 813)=38.8, p<0.0001). Of the total sample, 18.1% took over-the-counter sleep aids (OTCs), and more than one-third of those taking sleeping pills also used OTCs.

Design and measurement

This is a psychometric scale development study. Standard approaches to the appraisal of validity, reliability and sensitivity were applied, making appropriate selection among the available datasets. These methods and datasets will be introduced in an integrated way in subsequent sections, so that the research process can be more clearly expressed and understood.

Results

Development of the SCI

Content validity

An eight-item scale was developed (see appendix) based on DSM-5 workgroup draft criteria that were available at the time (in 2010).^{7 8} At that stage, a consultation process was underway and draft information was posted on the American Psychiatric Association website. Consequently, the SCI items generated comprised two quantitative items on sleep continuity (item 1, getting to sleep; item 2, remaining asleep), two qualitative items on sleep satisfaction/dissatisfaction (item 4, sleep quality; item 7, troubled or not), two quantitative items on severity (item 3, nights per week; item 8, duration of problem) and two qualitative items on attributed daytime consequences of poor sleep (item 5, effects on mood, energy or relationships (personal functioning); item 6, effects on concentration, productivity or ability to stay awake (daytime performance)).

Validated quantitative criteria for sleep disruption (eg, 31–45 min to fall asleep) served as responses for sleep continuity items 1 and 2.^{8 29 30} Items 5 and 6 on daytime effects were derived by principal components analysis (PCA; Varimax rotation) of six individual proposed DSM-5 impact areas, using combined datasets (GBSS, GBSS+, TV, RCT: valid n=29 650). PCA yielded satisfactory Kaiser-Meyer-Olkin (KMO=0.874) and Bartlett (p<0.0001) statistics. Iteration converged after three rotations. A two component model (derived from inspection of the scree plot and a criterion for associated variance ≥10%) explained 75.8% of total variance, with item loadings ≥0.60. Component 1 (eigenvalue=3.83; 63.8% of variance) comprised ‘mood’ (loading=0.812), ‘energy’ (0.651) and ‘relationships’ (0.859), and was subsequently named ‘personal functioning’. Component 2 (eigenvalue=0.72; 12.0% of variance) comprised ‘concentration’ (0.719) and ‘get through work’ (0.724), and ‘stay awake’ (0.875) may be regarded as ‘daytime performance’. PCA offered a relatively pure solution, although ‘energy’ also loaded significantly on component 2 (0.519).

We then investigated the inter-relationships of our eight SCI items using the same methodology, but applying this time a minimum item loading of 0.40, to permit incorporation of all eight items. PCA with Varimax Rotation (KMO=0.888; Bartlett, p<0.0001) yielded a two component solution (66.4% explained variance). Component 1 (eigenvalue=4.256, 53.2% variance), named ‘sleep pattern’, comprised items 1, 2, 3, 4 and 8 with factor loading ranging from 0.453 to 0.776. Component 2 (eigenvalue=1.06, 13.2% variance), named ‘sleep-related impact’, comprised item 5 (factor loading 0.886) and item 6 (0.911). Consistent with clinical presentation, concerns about sleep (item 7) loaded significantly and similarly on ‘sleep pattern’ (0.616) and ‘sleep-related impact’ components (0.576).

Response format

Each item was scored on a 5-point scale (0–4), with lower scores, in the 0–2 range, reflecting putative threshold criteria for insomnia disorder (shaded area: see appendix). The clinician can then see at a glance the profile of possible concerns. The possible total score ranges from 0 to 32, with higher values indicative of better sleep. However, scores can be readily transformed into a more intuitive 0–10 SCI range, either by dividing the total by 3.2, or by using an online version with automated scoring, which is available free of charge (http://www.sleepio.com/clinic/).

Concurrent validity and association with related domains

Data from the Science Centre sample demonstrated that the SCI correlates inversely with the PSQI (r=−0.734) and the ISI (r=−0.793), suggesting measurement properties consistent with these related measures. There was also a small but significant association of sleep condition with self-rated physical health (r=0.222), and an association also with mental health (r=0.335). Using more specific measures, in the Science Centre sample, correlation of the SCI with symptoms of depression (r=−0.426) and anxiety (r=−0.400) on the Hospital Anxiety and Depression Scale³⁵ was modest, and greater than we observed in our RCT sample (on the Depression Anxiety Stress Scale³⁶: depression (r=−0.267), anxiety (r=−0.236) and stress (r=−0.263)).

We have not at this stage tested the discriminant validity of the SCI against clinical diagnosis of insomnia disorder. As a first step, however, using our Science Centre sample, we tested the concurrent validity of SCI cut-offs (score ≤16), reflecting the minimum criteria for putative insomnia disorder (see appendix), against published, validated ISI cut-off scores. We first categorised our sample according to ISI ranges (ISI score=0–14, reflecting ‘no insomnia disorder’ (n=228) vs ‘probable insomnia disorder’ ISI score=15–28, n=27)) and conducted an independent t test on SCI total score. The mean SCI values for the ‘probable insomnia disorder’ category were 10.7 (SD=5.3) vs 22.9 (SD=6.2) for ‘no insomnia disorder’ (t=9.86, p<0.0001). Applying an SCI cut-off ≤16, 89% of the sample was correctly identified as having ‘probable insomnia disorder’ (ISI scores of ≥15), while an SCI score of >16 correctly identified 82% of those with ‘no insomnia disorder’. These findings provide further evidence of concurrent validity for the SCI and help to confirm that a score of ≤16 on the SCI seems reasonable to detect possible insomnia disorder.

Internal consistency

Cronbach's α for the GBSS sample was strong at 0.857 (range of α-if-item-deleted 0.822–0.860). Replication of these internal consistency data was obtained from the GBSS+ sample (α=0.865). The mean corrected item-total correlation was moderate (r=0.620), indicating a substantial unique variance per item (shared variance=38%).

Sensitivity to change

We have previously reported that the SCI is sensitive as a measure of treatment outcome.³⁴

Short-form version of the SCI

Although the SCI is brief, in clinical practice ultra short-form scales are often helpful (eg, generalised anxiety disorder (GAD) 2-items).³⁷ Accordingly, we conducted a stepwise linear regression analysis to determine which subset of items (independent variables) explained the greatest proportion of variance in the dependent variable, SCI total score. A two-item (SCI-02), comprising item 3 ‘how many nights’ (standardised β=0.515) and item 7 ‘troubled you in general’ (β=0.491), together predicted 82% of variance (adjusted R²=0.820) in the full scale SCI (R² change=0.672 + 0.148; F(227 637)=62 770, p<0.0001). As a check on the independence of residuals, we computed the Durbin-Watson statistic, which was found to be 1.80, suggesting no serial correlation. The SCI-02 also correlated strongly with the SCI score total (r=0.904).

Discussion

A prerequisite to improved insomnia care is the availability and regular use of reliable and valid insomnia assessment. Only then can a clinical problem be recognised as distinct from normal variation, and a persistent problem be differentiated from a transient one. We have reported here the development and preliminary validation of the SCI, a DSM-5 compliant, brief screening measure that may be fit for such purposes. Results indicate that the SCI is internally consistent, sensitive to change and correlates strongly with established screening instruments, known to be sensitive to clinical insomnia (PSQI and ISI). PCA revealed a two-component solution (66% of the variance), reflecting the underlying complaint of insomnia; that is, concerns about sleep pattern and concerns about the impact of poor sleep, both of which need to be addressed in clinical practice. The derived two-item short-form version, focusing on the severity of the presenting complaint coupled with frequency of the sleep problem, correlated strongly with total SCI score and we would suggest that these might be the lead questions for a clinician to use in the context of their consulting room practice.

Of course, further work is required, particularly real-world studies of how the SCI might be used in population screening and in the evaluation of outcome following an episode of care. While comparisons with ISI cut-offs provide evidence of concurrent validity and indicate that an SCI score ≤16 may help detect probable insomnia disorder, studies of predictive validity with reference to independent clinical evaluation of insomnia disorder (the gold standard) are essential before firm conclusions can be made.

It should be noted that over half of the respondents to our online surveys screened positive for possible insomnia disorder. These, of course, are not prevalence data as we did not adopt a formal population sampling approach. Nevertheless, the inevitable bias of these open access surveys towards those with sleep concerns does permit us (1) to profile many respondents against criteria; (2) to conduct powerful analyses of the properties of the SCI and (3) to make comparisons with sizeable cohorts of good sleepers. Importantly, for our Science Centre sample, approximately 10% scored in the probable insomnia disorder range (ISI score ≥15), consistent with prevalence data,² providing further support for our ISI–SCI concurrent validity analysis.

Furthermore, a limitation of the SCI is that it does not contain specific questions relating to early morning awakenings (EMA; premature awakening with inability to return to sleep)—a symptom which has recently been incorporated into DSM-5 criteria. While established insomnia questionnaires, including the ISI²⁸ and Athens Insomnia Scale,³⁷ probe perceived severity of EMA, quantitative values for EMA, to our knowledge, are yet to be defined. To some extent, SCI item 2 on wakefulness during the night may capture this symptom, complaint, but we recommend that the clinician follows up a ‘positive’ answer to locate the nature and temporal position of wakefulness during the sleep period. Moreover, other core DSM-5 criteria do not feature as SCI items (eg, the sleep difficulty occurs despite adequate opportunity for sleep; the insomnia is not better explained by and does not occur exclusively during the course of another sleep-wake disorder; and the insomnia is not attributable to the physiological effects of a substance). However, these items are not easy to probe unambiguously with a self-completed psychometric instrument and would require careful scrutiny by a treating clinician. It is, of course, possible that sleep disorders other than insomnia (eg, circadian rhythm sleep disorders, sleep-breathing disorders) may also lead to low scores on the SCI. Thus, the SCI should be viewed as an insomnia screening tool, consistent with features of DSM-5, but requiring careful follow-up in clinical practice to fully define the nature of sleep disturbance.

Appendix

Appendix: The sleep condition indicator

Scoring instructions:

Add the item scores to obtain the SCI total (minimum 0, maximum 32).

A higher score means better sleep.

Scores can be converted to 0–10 format (minimum 0, maximum 10) by dividing total by 3.2.

Item scores in grey area represent threshold criteria for Insomnia Disorder.