Development and Initial Validation of the Iowa Sleep Disturbances Inventory (ISDI)

Abstract

The Iowa Sleep Disturbances Inventory (ISDI) is a new measure of self-reported sleep difficulties, which was designed to help facilitate research on the overlap of sleep disturbances and psychopathology. This instrument was developed in 2 large student samples using principal factor analyses; the psychometric properties of the scales then were examined in 3 additional samples (students, psychiatric patients, sleep disorder patients). The ISDI consists of 11 specific scales (Nightmares, Initial Insomnia, Fatigue, Fragmented Sleep, Nonrestorative Sleep, Anxiety at Night, Light Sleep, Movement at Night, Sensations at Night, Excessive Sleep, Irregular Schedule) and 1 general scale (Daytime Disturbances). The structure of the ISDI generalizes across both patient and non-patient samples. In addition, the ISDI scales are internally consistent, show good retest reliability, demonstrate convergent and discriminant validity with widely used measures of sleep disturbances, and display criterion validity in relation to psychiatric patient status and specific symptoms of depression and anxiety.

A significant body of research has documented that self-reported sleep disturbances are associated with a variety of psychological disorders. One of the most robust findings in this literature is the association of insomnia and fatigue with depression and anxiety disorders (Benca, 2005; Mellman, 2006; Papadimitriou & Linkowski, 2005; Peterson & Benca, 2006; Stein & Mellman, 2005). Moving beyond these relatively common sleep disturbances, more unusual sleep experiences (e.g., narcolepsy symptoms, nightmares, complex behaviors at night) have been linked to dissociation, schizotypy, and posttraumatic stress disorder (PTSD) (Koffel & Watson, 2009b).

Although many studies have demonstrated general relations between sleep complaints and psychological disorders, very few have examined whether certain sleep disturbances show specificity to individual disorders. For example, there is some indication that nightmares are specific to non-sleep-related symptoms of PTSD compared to other psychological disorders (e.g., substance abuse, depression, panic disorder) and compared to other sleep complaints (e.g., insomnia) (Neylan, et al., 1998). Similarly, lassitude/fatigue appears to be specific to depression, whereas insomnia tends to show more moderate, nonspecific associations with both depression and anxiety (Koffel & Watson, 2009a). To continue mapping the specific relations between nighttime and daytime symptoms, it is necessary (1) to determine the structure of self-reported sleep disturbances, particularly those disturbances that coincide with mental disorders and (2) to develop comprehensive measures of the sleep disturbances that are based on this underlying structure.

A Review of Current Sleep Disturbance Measures

Overview

There currently are a large number of questionnaires that measure sleep disturbances. Some focus on specific sleep complaints, such as insomnia or fatigue, whereas other instruments have multiple scales assessing a range of sleep complaints and sleep disorders. The most widely used sleep questionnaires demonstrate good psychometric properties in terms of reliability and criterion validity and have been used to examine sleep complaints in both sleep disorder and psychiatric patients (Sateia, Doghramji, Hauri, & Morin, 2000). The main limitations of these instruments are in regard to substantive validity and structural validity (Clark & Watson, 1995; Loevinger, 1957).

Substantive validity: lack of comprehensiveness

If an instrument comprehensively measures each of the content areas hypothesized to be within the assessed domain, it is said to have substantive validity. This includes having items specific to each content area within the domain, as well as having enough items within each content area to capture it adequately. Achieving substantive validity begins with the creation of a broad, initial item pool that is based on theories of what should be included (and excluded) from the domain. Although items will eventually be discarded from this initial pool based on psychometric analyses, beginning the process with an over-inclusive item pool ensures that the final instrument will adequately cover all content areas (Clark & Watson, 1995; Loevinger, 1957).

Many existing sleep questionnaires do not provide comprehensive assessment of the domains they propose to measure. For example, the Pittsburgh Sleep Quality Index (PSQI; Buysse, Reynolds, Monk, Berman, & Kupfer, 1989) has seven subscales measuring subjective sleep quality, sleep latency, sleep duration, sleep efficiency, sleep disturbances, use of sleeping medication, and daytime dysfunction. Despite the broad content areas represented by the subscales (e.g., daytime dysfunction), the majority of them are assessed with two or fewer items. Similarly, the Global Sleep Assessment Questionnaire (GSAQ; Roth, et al., 2002) assesses 7 sleep disorders using 11 items. Finally, the Epworth Sleepiness Scale (ESS; Johns, 1991) consists of 8 items designed to measure sleepiness in a variety of situations. However, the concern has been raised that the instrument may be missing important items that assess daytime sleepiness, given that an initial item pool was not used to ensure adequate coverage of the domain (Miletin & Hanly, 2003).

In addition to lacking enough items to measure specific content areas adequately, a more fundamental problem with existing sleep instruments is that many are limited to a narrow range of content (e.g., insomnia, lassitude). This becomes particularly problematic when trying to assess the relations between psychological disturbances and more unusual sleep complaints, such as nightmares. To continue studying the relation between sleep complaints and psychological disorders, researchers need instruments that provide a more comprehensive assessment of sleep disturbances both in terms of items and content areas.

Structural validity: lack of well-defined scales and subscales

An instrument is considered to have structural validity if the internal structure of the items parallels the structure of the underlying domain. For example, if several content areas can be identified within a domain, each content area ideally should have its own scale. Factor analytic techniques can be particularly helpful in creating these unidimensional scales (Floyd & Widaman, 1995). Although most existing sleep instruments were not created using factor analysis, sleep questionnaires have been submitted to factor analysis in order to examine the underlying structure of the items and to determine if this structure matches the existing scales.

Given the difficulty of creating unidimensional scales and subscales without the use of factor analysis (Clark & Watson, 1995), it is not surprising that many existing sleep scales have been found to be multidimensional. For example, several widely used questionnaires including the PSQI and the Insomnia Severity Index (ISI; Morin, 1993) combine nighttime sleep problems and daytime fatigue in a total sleep disturbance score, which is suggestive of a single, broad factor of sleep disturbances. However, structural analyses have established that daytime and nighttime disturbances represent distinct dimensions that are only moderately correlated (Koffel & Watson, 2009a).

Cole et al. (2006) used confirmatory factor analysis to establish the existence of three separate factors of sleep disturbance from the component scores of the PSQI: Sleep Efficiency, Perceived Sleep Quality, and Daily Disturbances. On the basis of these results, it appears that the PSQI is measuring relatively independent dimensions; it may be more informative to obtain scores on these separate dimensions as opposed to a combined total score. Similarly, various factor analyses conducted on the ISI items have found several factors related to nighttime and daytime problems, again suggesting that the instrument is measuring multiple distinct dimensions (Bastien, Vallieres, & Morin, 2001; Savard, Savard, Simard, & Ivers, 2005). Finally, several scales from the Sleep-50 (Spoormaker, Verbeek, van den Bout, & Klip, 2005), a questionnaire that provides scores on a variety of sleep disturbances, did not replicate when the items were submitted to factor analysis. This suggests that the scales do not adequately capture the underlying dimensions of sleep disturbances (e.g., sleep apnea, nightmares). Overall, these analyses clearly demonstrate that it is advantageous to submit items to factor analysis during the initial scale development to ensure that the final scales will be unidimensional and parallel the underlying structure of sleep complaints.

Overview of the Current Research

The primary goal of the current study was to create a new measure of sleep disturbances that would complement existing measures and that would address their limitations. Similar to instruments such as the Sleep-50 and the GSAQ, our goal was to create a series of scales assessing specific sleep disturbances. However, our strategy in creating this instrument differed in two key ways from previous approaches.

First, we wanted to create an instrument that had enough items to provide comprehensive assessment of the various dimensions or content areas within self-reported sleep disturbances. This was achieved by including multiple markers of all the sleep disturbance dimensions that could potentially emerge in structural analyses. To ensure that our initial item pool was comprehensive, we arranged the items in homogeneous item composites (HICs) or groupings of hypothetically related items (Hogan, 1983). For example, one HIC had items referring to initial insomnia, whereas another had items referring to sleep quality. Although we grouped the initial items based on our assumptions of which items potentially might cohere into scales, the final selection of items and creation of scales was done using structural analyses. It is important to note that in many cases the structure that emerged in factor analysis did not parallel our rational organization of items; this again highlights the importance of combining theory with rigorous psychometric analyses of the data.

Second, our aim was to create an instrument with unidimensional scales reflecting the underlying structure of sleep disturbances. Beginning with our comprehensive item pool, we used exploratory factor analyses to shape and finalize the scales. This represents a major departure from the creation of previous sleep disturbance scales, which typically were submitted to factor analysis only after the scales already had been finalized. We report on the creation of the ISDI using 2 scale development samples (students), as well as an examination of the psychometric properties of the ISDI using 3 additional scale validation samples (students, psychiatric patients, sleep disorder patients).

In addition to creating a new measure of sleep disturbances, a secondary goal was to gain a better understanding of the structure of self-reported sleep disturbances. Previous research examining the structure of sleep disturbances that coincide with depression and anxiety (e.g., insomnia, hypersomnia, fatigue) has shown that these sleep disturbances can be organized into two broad factors of Lassitude and Insomnia (Koffel & Watson, 2009a). We hypothesized that we would obtain these same two higher order factors with our comprehensive measure of sleep disturbance. However, we were also open to the possibility that we would obtain a more complex, expanded structure given that the current study included a broader variety of sleep complaints.

Method

Participants and Procedure

Scale development sample 1

The participants were 298 undergraduate students enrolled in an introductory psychology course. They received research credit upon completion of the study. All participants completed the initial ISDI item pool in small group sessions. This item pool will be discussed in greater detail subsequently. The sample included 243 women and 54 men (the sex of one participant was unreported). Age ranged from 18–40 and the mean age was 19. The sample included 270 Whites (90.6%), 15 Asian Americans (5.0%), 5 multiracial participants (1.7%), 3 African Americans (1.0%) and 5 participants (1.7%) whose racial status was either unknown or from another category.

Scale development sample 2

The participants were 335 college students enrolled in an introductory psychology course. They received research credit upon completion of the study. All participants completed a revised ISDI item pool in small group sessions. This revised item pool will be discussed in greater detail subsequently. The sample included 217 women and 117 men (the sex of one participant was unreported). Age ranged from 18–27 and the mean age was 19. The sample included 308 Whites (91.9%), 10 Asian Americans (3.0%), 6 multiracial participants (1.8%), 5 African Americans (1.5%) and 6 participants (1.8%) whose racial status was either unknown or from another category.

Student sample

This sample consisted of 374 college students enrolled in an introductory psychology course. They received research credit upon completion of the study. All participants completed the final 86-item ISDI using an internet based data collection program. The sample included 218 women and 156 men. Age ranged from 18–32 and the mean age was 19. The sample included 338 Whites (90.4%), 16 Asian Americans (4.3%), 11 multiracial participants (2.9%), and 9 participants (2.4%) whose racial status was either unknown or from another category. In addition, 319 students completed a second assessment approximately two weeks later, which allowed us to examine the short-term retest reliability of the ISDI.

Psychiatric patient sample

The participants were 188 psychiatric patients recruited from the Community Mental Health Center of Mideastern Iowa and the Adult Psychiatry Clinic at the University of Iowa Hospitals and Clinics. Patients at these sites were approached and asked if they were interested in enrolling in a research study. Individuals who agreed to participate were given a packet of questionnaires, including the final 86-item ISDI, which they completed at home and returned through the mail in a prepaid envelope. The additional questionnaires in this packet are described in the following section. The patients were paid $15 for participating in the study. The sample included 131 women and 56 men (the sex of one participant was unreported). Age ranged from 18–81 and the mean age was 41. The sample included 168 Whites (89.4%), 10 multiracial participants (5.3%), 5 African Americans (2.6%), 2 American Indians (1.1%), and 3 participants (1.6%) whose racial status was either unknown or from another category. In a sample of 575 patients recruited from the same locations, the diagnoses were as follows: 42.4% major depression, 21.9% generalized anxiety disorder, 13.7% social phobia, 13.0% PTSD, 13.0% agoraphobia, 11.8% specific phobia, 11.5% panic disorder, 12.3% dysthymic disorder, 10.1% substance use disorders, and 9.2% obsessive compulsive disorder (see Table 1 in Watson, et al., 2008).

Sleep disorder patient sample

The participants were 205 patients who had been seen in the Neurology Department of the University of Iowa Hospitals and Clinics and received a sleep disorder diagnosis within the last year. Patients were mailed a letter explaining the study and were asked to either call or return a postcard if they were interested in enrolling in a research study. Individuals who agreed to participate were mailed a packet of questionnaires, including the final 86-item ISDI, which they completed at home and returned through the mail in a prepaid envelope. The additional questionnaires in this packet are described in the following section. The patients were paid $15 for participating in the study. The sample included 116 women and 89 men. Age ranged from 20–83 and the mean age was 52. The sample included 190 Whites (92.7%), 6 multiracial participants (2.9%), 2 American Indians (1.0%), 2 African Americans (1.0%), 2 Asian Americans (1.0%), and 3 participants (1.5%) whose racial status was either unknown or from another category. In this sample, 64.4% were diagnosed with obstructive sleep apnea, 23.9% were diagnosed with hypersomnia, 18.5% were diagnosed with insomnia, 9.8% were diagnosed with restless legs syndrome, 5.4% were diagnosed with narcolepsy, 2.4% were diagnosed with periodic limb movements or sleep disorder NOS, and less than 1.0% were diagnosed with circadian rhythm sleep disorder or a sleep disorder related to a mental condition.

Measures

Women’s Health Initiative Insomnia Rating Scale (WHIIRS)

The psychiatric and sleep disorder patients completed the WHIIRS (Levine, et al., 2003), a 5-item scale measuring nighttime sleep complaints over the past 4 weeks, including early, middle, and late insomnia. Coefficient alphas were .83 and .85 for the psychiatric patients and sleep disorder patients, respectively.

ESS

To obtain a measure of self-reported sleepiness, the patient samples also completed the ESS (Johns, 1991). The ESS is an 8-item questionnaire that asks participants to rate their general tendency to doze off during the day using a 4-point scale ranging from would never doze to high chance of dozing. Coefficient alphas were .85 and .89 in the psychiatric patients and sleep disorder patients, respectively.

PSQI

The PSQI (Buysse, et al., 1989) was used to obtain information about both nighttime and daytime complaints over the past month in the patient samples. The PSQI has seven components: subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleeping medication, and daytime dysfunction. The PSQI component scores had a coefficient alpha of .77 and .79 in the psychiatric patients and sleep disorder patients, respectively.

Inventory of Depression and Anxiety Symptoms (IDAS)

The patient samples also completed the IDAS (Watson, et al., 2007), a multi-dimensional measure of depression and anxiety that contains 10 specific symptom scales (Lassitude, Insomnia, Traumatic Intrusions, Panic, Social Anxiety, Suicidality, Appetite Loss, Appetite Gain, Well-Being, Ill Temper) and 2 broad scales of General Depression and Dysphoria. Participants choose a response that reflects the way they have been feeling during the past two weeks. Responses are rated on a five point scale, ranging from 1 (not at all) to 5 (extremely). The Dysphoria scale was used as a general measure of depression in this study because it does not contain items asking about sleep problems or fatigue. Similarly, we calculated the score on the Traumatic Intrusions scale after removing an item related to nightmares to avoid inflating the correlation between these scales and the ISDI. The coefficient alphas for these scales all were above .76 in these samples.

Results

Development of the ISDI

Scale development: Phase I

An item pool was created based on an extensive literature review of previous sleep questionnaires, a review of the sleep disturbances listed in the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association, 2000) and the International Classification of Sleep Disorders (ICSD-2; American Academy of Sleep Medicine, 2005), and a sleep item bank from the University of Pittsburgh containing nearly 3,000 items. This item bank was created as part of a National Institutes of Health (NIH) initiative to develop health outcomes questionnaires. The items from our item pool were answered using a true/false format. This dichotomous format was selected for several reasons. First, the major advantage of a dichotomous format over a Likert-type scale is that it allows more questions to be asked in a shorter period of time (Clark & Watson, 1995). Given that our instrument was developed to be a comprehensive measure of sleep disturbances, speed of responding was an important consideration. Second, many of the items included in our item pool referred to infrequent phenomena which would be difficult to rate on a Likert-type scale (e.g., sleep-related hallucinations). One disadvantage of using a dichotomous response format, however, is that extreme items, endorsed as either true or false by the large majority of participants, can lead to distorted correlational results; to address this issue, we examined response distributions in the second phase of scale development.

As mentioned previously, items originally were arranged into rational HICs in order to ensure proper coverage of each potential dimension of sleep disturbance. The items contained in the initial HICs were submitted to a series of principal factor analyses in the first scale development sample in order to select items for preliminary scales. The prior communality estimates were calculated using squared multiple correlations (SMCs). We rotated all factors using both varimax (which constrains the factors to be orthogonal) and promax (which allows the factors to be correlated). In the following analyses we report on the varimax rotations in order to help identify problematic items that “split” between two or more factors. The goal in these analyses was to extract the greatest number of factors that were interpretable and distinct from one another and to create preliminary scales based on these factors. In general, we used the following guidelines when selecting items for the preliminary scales. First, items that loaded below .40 were considered weak markers of a dimension and were removed. Second, items that loaded strongly onto more than one factor were considered a threat to the discriminant validity of the scales and also were eliminated.

In the first analysis, 79 items referring to difficulties initiating and maintaining sleep (HICs were labeled Initial Insomnia, Middle Insomnia, Late Insomnia, General Sleep Quality, Light Sleep, Negative Cognitions About Sleep, Physiological Arousal at Night, Cognitive Arousal at Night, Anxiety at Night, and Panic Attacks at Night) were submitted to factor analysis. Three preliminary scales were created based on this analysis: Initial Insomnia (e.g., “It takes me a long time to fall asleep”), Anxiety at Night (e.g., “I sometimes lie awake worrying”), and Light Sleep (e.g., “I am easily awakened by noises”).

In the second analysis, 78 items referring to lassitude (HICs were labeled Excessive Sleep, Daytime Confusion, Daytime Dysfunction, Daytime Sleepiness, Daytime Fatigue, Nonrestorative Sleep, Phase Delayed, Phase Advanced, and Irregular Schedule) were submitted to factor analysis. Three preliminary scales were created based on this analysis: Fatigue (e.g., “I have a hard time focusing during the day because I am tired”), Excessive Sleep (e.g., “I take long naps”), and Irregular Schedule (e.g., “My bedtime is very irregular”).

In the third analysis, 42 items referring to unusual events and behaviors during the night (HICs were labeled Sensations at Night, Movement at Night, Excessive Movements at Night, Walking at Night, Talking at Night, Teeth Grinding at Night, and Enuresis) were analyzed together and resulted in two preliminary scales: Movement at Night (e.g., “I move around a lot in my sleep”) and Sensations at Night (e.g., “I sometimes have cramps or pain in my legs during the night”). Finally, 22 items referring to nightmares and night terrors were analyzed together and resulted in a single preliminary Nightmares scale (e.g., “I frequently have frightening dreams”).

At this point, items were added to obtain more comprehensive coverage of several target factors (e.g., fragmented sleep and fatigue). In addition, some items that did not define distinct dimensions were tentatively retained to see if they would perform better in a new sample. A revised item pool, consisting of items in the preliminary scales plus new and retained items, was used in Phase II of the scale development with the second scale development sample.

Scale development: Phase II

These analyses were based on responses from the second scale development sample. As in Phase I, we used principal factor analysis with both varimax and promax rotations to identify dimensions of sleep disturbances and to select items for the final scales based on each of these dimensions. Once again, the goal in these analyses was to extract the greatest number of factors that were interpretable and distinct from one another.

First, the preliminary scales from Phase I were each submitted to factor analysis. The preliminary Anxiety at Night and Initial Insomnia scales were analyzed together since items from these scales had a tendency to overlap in Phase I. Due to the addition of items, a new dimension emerged from the Anxiety at Night and Initial Insomnia items and was used to create the Fragmented Sleep scale (e.g., “If I wake up during the night, I find it difficult to fall asleep again”). Similarly, an additional interpretable factor emerged from the analysis of the preliminary Fatigue scale. This dimension was used to create the Nonrestorative Sleep scale (e.g., “I have a hard time waking up during the week”). The remaining ISDI scales continued to define single dimensions of sleep disturbance.

The items from the preliminary scales were also submitted to a single overall factor analysis. We obtained 11 interpretable factors that represented distinct sleep disturbances. Nine of these dimensions had previously emerged in Phase I (Initial Insomnia, Anxiety at Night, Light Sleep, Fatigue, Excessive Sleep, Irregular Schedule, Movement at Night, Sensations at Night, Nightmares). The two additional dimensions of Nonrestorative Sleep and Fragmented Sleep also emerged in this analysis. The final ISDI scales were based on these 11 dimensions. When selecting items for the final scales, we followed several guidelines to ensure that the items would be maximally informative and distinct: (1) items that loaded too strongly on a factor (i.e., .90 or above) were considered redundant and removed; (2) items that loaded below .40 were considered weak markers of a dimension and were removed; (3) items that loaded strongly onto more than one factor were considered a threat to the discriminant validity of the scales and were removed. In addition, frequency data were calculated for each item to remove items that were worded poorly or worded too strongly; items that were endorsed too frequently (greater than 90% of the sample) or too infrequently (less than 10% of the sample) were discarded. Ultimately, however, a total of only two items from the Nightmares scale were removed based on this criterion.

The final ISDI contains the following specific scales: a 12-item measure of Nightmares; an 11-item measure of Initial Insomnia; a 10-item measure of Fatigue; a 9-item measure of Fragmented Sleep; an 8-item measure of Nonrestorative Sleep; a 7-item measure of Anxiety at Night; 6-item measures of Light Sleep, Movement at Night, Sensations at Night, and Excessive Sleep; and a 5-item measure of Irregular Schedule. Table 1 reports the coefficient alphas and average interitem correlations (AICs) for these scales in this second scale development sample. All of the scales show evidence of good internal consistency, with coefficient alphas of .80 and above, with the exception of Excessive Sleep (.77). AICs provide a measure of internal consistency that is not affected by the number of items in a scale. AICs should fall in the range of .15 (broad constructs) to .50 (narrow constructs) (Clark & Watson, 1995). All of the scales have AICs that fall within the upper limits of this range (with the exception of Light Sleep at .51), suggesting that the ISDI scales are measuring relatively narrow and specific dimensions of sleep disturbances.

Table 1.

Internal Consistency Reliabilities (Coefficient Alphas) and Average Interitem Correlations (AICs) of the Iowa Sleep Disturbances Inventory (ISDI) Scales

ISDI Scale	Scale Development Sample		Scale Validation Samples
	Student		Student		Psychiatric patient		Sleep disorder patient
	α	AIC	α	AIC	α	AIC	α	AIC
Daytime Disturbances (18 items)	.91	.36	.91	.36	.94	.47	.94	.47
Nightmares (12 items)	.87	.36	.85	.32	.94	.57	.93	.53
Initial Insomnia (11 items)	.90	.45	.93	.55	.93	.55	.95	.63
Fatigue (10 items)	.89	.45	.89	.45	.92	.53	.92	.53
Fragmented Sleep (9 items)	.84	.37	.82	.34	.87	.43	.89	.47
Nonrestorative Sleep (8 items)	.83	.38	.88	.48	.87	.46	.89	.50
Anxiety at Night (7 items)	.81	.38	.84	.43	.86	.47	.88	.51
Light Sleep (6 items)	.86	.51	.89	.57	.89	.57	.87	.53
Movement at Night (6 items)	.83	.45	.86	.51	.88	.55	.89	.57
Sensations at Night (6 items)	.80	.40	.80	.40	.89	.57	.88	.55
Excessive Sleep (6 items)	.77	.36	.77	.36	.84	.47	.81	.42
Irregular Schedule (5 items)	.80	.44	.81	.46	.84	.51	.87	.57
Median values	.84	.39	.85	.44	.89	.52	.89	.53

Note. n =335 (students, scale development), 374 (students, scale validation), 188 (psychiatric patients), 205 (sleep disorder patients).

Table 2 presents the correlations among the 11 specific scales in this second scale development sample. The majority of correlations fall in the low to moderate range, which indicates good discriminant validity. All correlations were less than .50 with the exception of Fatigue and Nonrestorative Sleep (r = .57). This correlation suggests that the Fatigue and Nonrestorative Sleep scales are closely related. In addition, when 10 factors were extracted from the ISDI items instead of 11, the Fatigue and Nonrestorative Sleep items combined into a single factor. In order to reflect the close relationship between these two dimensions, we created a broader Daytime Disturbances scale. This scale is the summation of the Fatigue and Nonrestorative Sleep scales.

Table 2.

Iowa Sleep Disturbances Inventory (ISDI) Scale Correlations in the Scale Development Sample 2

ISDI Scale	1	2	3	4	5	6	7	8	9	10	11
1. Initial Insomnia	.—
2. Anxiety at Night	.48	.—
3. Fragmented Sleep	.45	.36	.—
4. Nonrestorative Sleep	.38	.25	.25	.—
5. Light Sleep	.35	.28	.44	.12	.—
6. Irregular Schedule	.32	.21	.15	.39	.10	.—
7. Fatigue	.27	.35	.25	.57	.17	.34	.—
8. Sensations at Night	.17	.17	.24	.23	.19	.10	.29	.—
9. Nightmares	.16	.31	.26	.17	.18	.06	.17	.29	.—
10. Movement at Night	.04	.07	.14	.10	.01	.03	.05	.27	.24	.—
11. Excessive Sleep	−.01	.15	.05	.27	−.04	.20	.48	.13	.19	.05	.—

Note. n = 335. Correlations of |.50| and greater are in bold. Correlations of |.11| and greater are significant at p < .05.

We retained the Fatigue and Nonrestorative Sleep scales as separate subscales based on several considerations. First, the average interitem correlations (AICs) of these two subscales are higher than the AIC of the overall scale (AIC for Fatigue = .45, AIC for Nonrestorative Sleep = .38, AIC for Daytime Disturbances = .36). This suggests that it is appropriate to create subscales since the items on each subscale are more closely related than all the items combined (Clark & Watson, 1995). Second, nonrestorative sleep is often conceptualized as a dimension of insomnia that is distinct from difficulty initiating sleep, maintaining sleep, and daytime fatigue (American Psychiatric Association, 2000). Retaining a separate measure of nonrestorative sleep will help determine if this specific sleep problem is informative to study in the context of psychopathology, as has been suggested in previous research (Ohayon, 2005).

To clarify the underlying structure of the ISDI, the 11 specific scales were submitted to principal factor analysis; we report on the promax rotations in the following analyses in order to examine the correlations among the factors. Based on previous research examining the higher order structure of sleep complaints, we expected that the scales would load onto two general factors, one representing Insomnia and the other representing Lassitude (Koffel & Watson, 2009a; Watson, et al., 2007). Our analyses initially revealed the presence of a large and relatively broad factor of sleep complaints, which was defined by the majority of scales and accounted for 80% of the common variance (all scales except Movement at Night loaded onto this factor at .30 or above). In the next step, we extracted two clear, well-defined factors representing Insomnia and Lassitude. These two factors jointly accounted for all of the common variance. When a third factor was extracted, it was defined by the Nightmares, Movement at Night, and Sensations at Night scales, however, none of these scales loaded above .47 on this factor. This suggests that the third factor is not well defined with the current ISDI scales, and as such, we report results from the two-factor solution.

Table 3 reports the factor loadings from the two-factor solution in the second scale development sample. Insomnia was represented by the Fragmented Sleep, Initial Insomnia, Light Sleep, and Anxiety at Night scales, and to a lesser extent, the Nightmares scale. Lassitude was represented by the Fatigue, Excessive Sleep, Nonrestorative Sleep, and Irregular Schedule scales. The Movement at Night and Sensations at Night scales did not load strongly onto either factor. The Lassitude and Insomnia factors were only moderately correlated at .42, suggesting that these factors represent distinct dimensions of sleep disturbance. These analyses have implications for understanding both the ISDI and the structure of self-reported sleep disturbances. Our findings suggest that self-reported sleep disturbances have a hierarchical structure. Lower order dimensions of sleep disturbances, as measured by the specific ISDI scales, can be grouped under two higher order factors of Insomnia and Lassitude.

Table 3.

Factor Structure of the Iowa Sleep Disturbance Inventory (ISDI) Scales

ISDI Scale	Scale Development Sample		Scale Validation Samples
	Student		Student		Psychiatric patient		Sleep disorder patient
	I	II	I	II	I	II	I	II
Fragmented Sleep	.66	−.03	.65	−.12	.79	−.06	.75	−.03
Initial Insomnia	.65	.04	.64	.08	.72	−.06	.73	.05
Light Sleep	.59	−.14	.53	−.23	.57	−.09	.59	−.13
Anxiety at Night	.53	.14	.48	.12	.67	.14	.49	.12
Nightmares	.33	.13	.36	.12	.30	.16	.24	.22
Sensations at Night	.28	.20	.32	.05	.41	.11	.44	.06
Movement at Night	.15	.07	.18	.05	.26	.25	.31	.28
Fatigue	.09	.73	.21	.65	.04	.80	.08	.84
Excessive Sleep	−.18	.62	−.20	.48	−.14	.56	−.27	.70
Nonrestorative Sleep	.16	.59	.05	.72	.13	.75	.18	.72
Irregular Schedule	.13	.39	.02	.54	.30	.25	.29	.35

Note. n = 335 (students, scale development), 374 (students, scale validation), 188 (psychiatric patients), 205 (sleep disorder patients). I = Insomnia, II = Lassitude. Factor loadings of |.30| and higher are highlighted.

Replicating the ISDI Structure

We will now examine the psychometric properties of the final ISDI scales in our three scale validation samples. However, before reporting reliability and validity analyses, it is important to demonstrate that the structure of the ISDI replicates in samples that traditionally report high levels of sleep disturbances (e.g., psychiatric and sleep disorder patients). In this regard, however, it is important to note that the college students used to finalize the ISDI tended to have above average scores on measures of sleep disturbance. For example, the mean score on the PSQI was 6.16 in scale development sample 2. The mean score on this instrument for normal controls has been reported as 2.67, with a cut-off score of 5 or above indicating sleep difficulties (Buysse et al., 1989). It is also important to demonstrate that the structure of the ISDI replicates in older adults, represented in this study by the psychiatric patient and sleep disorder patient samples (mean age of 41 and 52, respectively). Both older age and patient status may influence the number of sleep dimensions that can be identified, particularly if these samples tend to report a greater variety of sleep complaints than younger, non-patient samples. In our previous work, however, we have found that patients and older adults have quantitatively but not qualitatively different sleep compared to non-patients and young adults, which leads to structural invariance across samples (Koffel & Watson, 2009a).

Item-level factor analysis

In Phase II of the scale development, we obtained 11 dimensions of sleep disturbances, which were used to create the 11 specific ISDI scales. We also created the more general Daytime Disturbances scale to reflect the high correlation between the two specific scales of Fatigue and Nonrestorative Sleep; this broader scale also reflected the fact that Fatigue and Nonrestorative Sleep items loaded onto the same factor when 10 factors were extracted.

To examine whether these sleep dimensions replicated in our scale validation samples, we submitted the 86 ISDI items to principal factor analysis with varimax rotation. When the final ISDI items from the student and psychiatric patient samples were submitted to independent factor analyses, we could extract a maximum of 10 factors in each sample. This structure corresponded to the 10-factor structure that was obtained in the scale development sample 2. Specifically, the first factor corresponded to the general Daytime Disturbances scale (i.e., items from the Nonrestorative Sleep and Fatigue scales loaded onto the same factor), while the remaining 9 factors represented specific ISDI scales. We were unable to extract 11 factors, which once again suggests that Fatigue and Nonrestorative sleep are strongly related and should be combined in a Daytime Disturbances scale.

When the final ISDI items from the sleep disorder patient sample were submitted to factor analysis, we could extract a maximum of nine factors. Items from the Nonrestorative Sleep and Fatigue scales loaded onto the same factor and items from the Sensations at Night and Movement at Night scales loaded onto the same factor. The other seven factors were very similar to those observed in the other samples. This nine-factor structure did not replicate in the other scale validation samples (i.e., the Sensations at Night and Movement at Night scales did not load onto the same factor in the other samples). We report on the 10-factor structure in this paper since it appears to be the most robust and replicable structure. It is likely that small sample sizes and corresponding measurement error contributed to the difficulty in replicating this factor structure in the sleep disorder patient sample.

We computed comparability coefficients to examine the degree of similarity for the regression-based factor scores in the student and psychiatric patient samples (Everett & Entrekin, 1980). We generated a set of regression-based factor scoring weights for the ten-factor, varimax-rotated solution in each sample. The scoring weights from each sample were then applied to the item responses for the student sample and the psychiatric patient sample, which resulted in a total of 20 factor scores per sample (10 representing the student factors and 10 representing the patient factors). The two solutions then were compared in each sample. If the solutions are similar (e.g., Nightmare weights from the student data vs. the parallel weights from the psychiatric patient data), then the corresponding weights for each factor will produce highly correlated scores. Table 4 shows the correlations between the two sets of factor scores in the two samples. Overall, the structures are very similar with 16 out of 20 correlations meeting or exceeding the benchmark of .90 (Everett, 1983). Two of the correlations were slightly below .90 in both the student sample (Fragmented Sleep = .88, Anxiety at Night = .88) and the psychiatric patient sample (Anxiety at Night = .84, Movement at Night = .89), suggesting that not all of the factors are identical, although they all define the same basic dimensions. After transforming the correlations to Fisher’s z to correct for non-normal distribution (Fisher, 1921), the average convergent correlation was .94 in the student data and .95 in the psychiatric patient data. These values establish that the factor structure replicated quite well across these samples; furthermore, they strongly suggest that the sleep complaints of (a) young versus older adults and (b) non-patients versus patients do not differ qualitatively from one another. ¹

Table 4.

Analyses of Factor Similarity: Factor Score Correlations in Students and Psychiatric Patients

Patient Factor Scores	Student Factor Scores
	1	2	3	4	5	6	7	8	9	10
Student Data
1. Daytime Disturbances	.97	.08	.10	.05	−.05	−.02	−.06	.06	.10	.08
2. Nightmares	.02	.98	−.03	.07	−.05	.02	.05	−.04	.00	−.01
3. Initial Insomnia	.09	−.02	.97	.06	.15	−.01	.06	−.01	−.09	.06
4. Fragmented Sleep	−.16	.13	−.03	.88	.07	−.01	.04	.07	.10	−.21
5. Anxiety at Night	−.03	−.11	−.08	−.19	.88	−.03	−.07	.05	−.02	.11
6. Light Sleep	−.07	.03	−.05	.11	−.00	.97	−.12	.01	−.04	−.06
7. Movement at Night	−.06	.06	.10	.02	−.04	−.03	.92	−.10	.03	−.08
8. Sensations at Night	−.01	−.03	−.02	.01	.01	−.08	.14	.92	−.03	−.03
9. Excessive Sleep	−.04	−.06	−.11	−.02	.13	.01	−.01	.00	.90	−.05
10. Irregular Schedule	.16	−.03	.03	−.08	−.03	−.05	−.12	−.05	.05	.92
Patient Data
1. Daytime Disturbances	.98	.01	−.01	.09	.13	.11	.08	.04	.15	−.11
2. Nightmares	−.06	.99	.01	−.05	.13	−.01	.00	−.05	.06	.07
3. Initial Insomnia	−.07	.03	.98	.12	.18	.04	−.05	−.01	−.00	.13
4. Fragmented Sleep	−.02	.03	−.03	.92	.38	.02	.02	.09	−.05	−.03
5. Anxiety at Night	.07	.04	−.03	.11	.84	.01	.03	−.02	−.08	.07
6. Light Sleep	.04	.00	.01	.11	.04	.97	.02	.12	−.04	.09
7. Movement at Night	.07	−.02	−.06	.02	.11	.07	.89	.13	.05	.09
8. Sensations at Night	−.03	−.02	−.01	.08	−.03	.06	.40	.93	−.00	.18
9. Excessive Sleep	.03	.02	−.00	−.10	.03	.02	.05	.04	.93	.05
10. Irregular Schedule	−.04	.01	.04	.07	−.03	.07	.05	.15	.16	.92

Note. n = 374 (students), 188 (psychiatric patients). Convergent correlations are highlighted.

Scale-level factor analysis

To determine if the higher order two-factor structure of Insomnia and Lassitude that we obtained in the second scale development sample replicated in the scale validation samples, we submitted the 11 specific ISDI scales to principal factor analyses with promax rotation. In all three samples, we found that the ISDI scales loaded onto a large and relatively broad factor of sleep complaints, which accounted for 69% of the common variance in students, 70% of the common variance in psychiatric patients, and 67% of the common variance in sleep disorder patients. In the next step, we extracted two clear, well-defined factors representing Insomnia and Lassitude in all samples. These two factors jointly accounted for all of the common variance in students, 93% of the common variance in psychiatric patients, and 92% of the common variance in sleep disorder patients. When a third factor was extracted in students, it was defined by the Movement at Night, Sensations at Night and Nightmares scales. When a third factor was extracted in the psychiatric and sleep disorder patients, it was defined by the Sensations at Night and Movement at Night scales. We refer to this third factor as Unusual Sleep Experiences.

Figure 1 presents the hierarchical structure of these factors in the scale validation samples, displaying the relations among these factors both within and between levels using regression-based factor scores. The ISDI scales listed under each factor are those markers that loaded onto the factor at |.30| or greater in all three scale validation samples. For ease of presentation, we transformed the correlations to Fisher’s z to correct for non-normal distributions (Fisher, 1921) and averaged them across the three scale validation samples. It is noteworthy that the correlations among the Insomnia, Lassitude and Unusual Sleep Experiences factors were all in the .42 to .49 range, indicating that these three factors are moderately related, but still distinct and independent from one another. It is important to note that across our various samples, the Unusual Sleep Experiences factor appears to be under-defined and unstable (e.g., defined by the Sensations at Night and Movement at Night scales in patients and the Sensations at Night, Movement at Night, and Nightmares scales in students). Thus, we will continue to report results on the two-factor model in the scale validation samples as we did in the scale development sample.

Figure 1.

First three levels of sleep disturbances extracted from scale level factor analyses of the Iowa Sleep Disturbances Inventory (ISDI). Within level correlations are represented in parentheses, whereas between level correlations are presented without parentheses. Correlations are among regression-based factor scores resulting from analysis with promax rotation and represent averages across the three scale validation samples. All correlations are significant at p < .01.

Table 3 shows the factor loadings of the ISDI scales in the scale validation samples. The ISDI scales show the same basic structure across all of the scale validation samples. Specifically, of the 11 nonoverlapping ISDI scales, 5 loaded primarily on Insomnia at |.30| or greater in all three samples (Fragmented Sleep, Initial Insomnia, Light Sleep, Anxiety at Night, Sensations at Night), and another 3 loaded primarily on Lassitude at |.30| and greater in all samples (Fatigue, Excessive Sleep, and Nonrestorative Sleep). The three remaining scales (Nightmares, Movement at Night, and Irregular Schedule) tended to split across the two factors in these samples. Once again, we computed comparability coefficients to examine the degree of similarity for the regression-based factor scores across the samples (Everett & Entrekin, 1980). These comparability correlations ranged from .98 to .99 across the three samples. Given that comparability coefficients greater than .90 indicate that the factor structure replicates across samples, these data demonstrate that the higher order factor structure of the ISDI replicates well across (a) young versus older adults and (b) non-patients versus patients.

Internal Consistency

Table 1 presents the coefficient alphas and AICs for each scale in the scale validation samples. As in the second scale development sample, all of the scales show good internal reliabilities in these samples, with coefficient alphas of .80 and above with the exception of the Excessive Sleep scale in the student sample (alpha = .77). Across both scale development and scale validation samples, 46 of the 48 coefficient alphas reach a level of .80 or above. In the scale validation samples, all of the scales have AICs that fall within or above the upper limits of the appropriate range (18 out of the 36 AICs are above .50). In general, the AICs are higher in the patient samples compared to the student samples.

Short-term Retest Reliability

As noted earlier, a subset of the student sample (n = 319) completed the ISDI a second time approximately two weeks after the initial assessment. A relatively short retest interval was chosen in order to determine the dependability of the instrument (i.e., the extent to which instability in scores reflects measurement error) (Watson, 2004). The Daytime Disturbances scale had a retest correlation of .83. The retest correlations for the 11 non-overlapping scales are shown in Table 5, which represents the hetero-occasion block in a multitrait-multioccasion matrix (Longley, Watson, & Noyes, 2005; Watson, et al., 2007). The convergent correlations ranged from .71 to .83 and were all above the minimum benchmark value of .70 (Joiner, Walker, Pettit, Perez, & Cukrowicz, 2005). The mean retest correlation, after transforming the correlations to Fisher’s z, was .79.

Table 5.

Multitrait-Multioccasion Matrix of Retest Correlations in the Student Sample

Time 2 Scale	Time 1 Scale
	1	2	3	4	5	6	7	8	9	10	11
1. Initial Insomnia	.83	.21	.43	.21	.17	−.10	.29	.17	.29	.39	.15
2. Light Sleep	.25	.82	.36	.05	−.03	−.13	−.04	.09	−.08	.22	.02
3. Fragmented Sleep	.43	.35	.80	.22	.15	−.13	.11	.30	.08	.24	.20
4. Fatigue	.25	.06	.26	.79	.15	.36	.56	.29	.34	.24	.18
5. Movement at Night	.19	−.05	.19	.10	.79	.10	−.00	.11	.07	.07	.27
6. Excessive Sleep	−.03	−.18	−.07	.44	.09	.78	.22	.09	.21	.13	.13
7. Nonrestorative Sleep	.35	.02	.14	.60	.08	.23	.78	.17	.46	.23	.11
8. Nightmares	.19	.11	.34	.31	.16	.00	.19	.77	.11	.17	.09
9. Irregular Schedule	.22	−.02	.04	.29	.06	.21	.46	.09	.72	.06	.10
10. Anxiety at Night	.51	.22	.26	.34	.11	.11	.23	.26	.17	.71	.14
11. Sensations at Night	.24	.02	.21	.17	.27	.09	.09	.05	.13	.17	.71

Note. n = 319. Retest correlations are in bold along the diagonal. Discriminant correlations of |.50 | and greater are underlined. Mean retest correlation = .79. Correlations of |.11| and greater are significant at p < .05.

To establish discriminant validity, the convergent or retest correlations should be higher than the other correlations within the same row and column of the block. Although an examination of Table 5 shows that this is the case, we tested the discriminant validity more formally by comparing the convergent correlations with the 20 discriminant correlations for each scale (a total of 220 comparisons overall), using the Williams modification of the Hotelling test for dependent correlations (Kenny, 1987). All of these comparisons were significant at the p < .01 level, two-tailed.

Convergent and Discriminant Validity

Table 6 shows correlations between the ISDI scales and several widely used measures of sleep disturbances in the psychiatric and sleep disorder patient samples. We compared the correlations of each ISDI scale with the WHIIRS and ESS using the Williams modification of the Hotelling test for dependent correlations (Kenny, 1987). Within each row and sample, the stronger convergent correlation with these two sleep measures is highlighted. We chose to compare these two sleep instruments because they measure distinct dimensions of sleep disturbances (i.e., insomnia and lassitude, respectively), whereas the PSQI combines both daytime and nighttime disturbances.

Table 6.

Correlations Between the Iowa Sleep Disturbances Inventory (ISDI) Scales and Sleep Measures in the Scale Validation Samples

ISDI scale	Sleep disorder patient				Psychiatric patient
	WHIIRS	ESS	PSQI	R2	WHIIRS	ESS	PSQI	R2
Fragmented Sleep	.80 a	.06 b	.67	.65	.80 a	.08 b	.69	.64
Initial Insomnia	.64 a	.02 b	.73	.56	.62 a	−.07 b	.64	.48.
Light sleep	.49 a	−.06 b	.44	.26	.47 a	−.01 b	.45	.25
Sensations at Night	.41 a	.19 b	.34	.20	.33 a	.16 a	.42	.18
Anxiety at Night	.39 a	.05 b	.41	.18	.63 a	.12 b	.58	.42
Excessive Sleep	−.06 b	.74 a	.08	.56	.00 b	.65 a	.11	.43
Fatigue	.33 b	.59 a	.51	.52	.33 a	.49 a	.40	.33
Daytime Disturbances	.35 b	.53 a	.54	.48	.37 a	.46 a	.44	.34
Irregular Schedule	.33a	.19 a	.40	.17	.25 a	.09 a	.40	.17
Nonrestorative Sleep	.33 a	.38 a	.49	.33	.35 a	.35 a	.43	.26
Nightmares	.21 a	.21 a	.26	.10	.35 a	.17 a	.41	.18
Movement at Night	.36 a	.26 a	.42	.22	.19 a	.19 a	.28	.10
R2	.72	.62	.73		.73	.52	.69

Note. n = 205 (sleep disorder patients), 188 (psychiatric patients). Within a row and sample, WHIIRS and ESS correlations not sharing the same subscript differ from one another at p < .05, 2-tailed. The stronger convergent correlation is in bold. Correlations of |.14| and greater are significant at p < .05 for sleep disorder patients. Correlations of |.15| and greater are significant at p < .05 for psychiatric patients. WHIIRS = Women’s Health Initiative Insomnia Rating Scale. ESS = Epworth Sleepiness Scale. PSQI = Pittsburgh Sleep Quality Index.

In terms of convergent validity, several ISDI scales, including Fragmented Sleep, Initial Insomnia, Light Sleep, and Anxiety at Night show moderate to strong correlations with the WHIIRS in both samples (rs ranged from .39 to .80). The correlations of these ISDI scales with the WHIIRS are significantly stronger than the correlations of these scales with the ESS (rs ranged from −.07 to .12). In addition, the Sensations at Night scale had a significantly stronger correlation with the WHIIRS (r = .41) compared to the ESS (r = .19) in the sleep disorder patient sample. This indicates good convergent and discriminant validity and suggests that these ISDI scales tap content that is normally associated with insomnia and poor sleep at night.

Conversely, the ISDI scale of Excessive Sleep had significantly stronger correlations with the ESS in both samples (rs ranged from .65 to .74) compared to the WHIIRS (rs ranged from −.06 to .00). In the sleep disorder sample, the Fatigue and Daytime Disturbances scales show significantly stronger correlations with the ESS (rs ranged from .53 to .59) than the WHIIRS (rs ranged from .33 to .35). This again demonstrates good convergent and discriminant validity and suggests that these ISDI scales represent content related to daytime sleepiness and fatigue. Taken together, these findings parallel the scale-level factor analysis that we reported earlier, in which the ISDI scales fall under the two higher order dimensions of Lassitude and Insomnia.

Finally, a number of ISDI scales had moderate to strong correlations with the PSQI, including Initial Insomnia (rs ranged from .64 to .73) and Daytime Disturbances (rs ranged from .44 to .54). Overall, these data suggest that the ISDI scales capture much of the variance in the most widely used measures of nighttime disturbances (i.e., the WHIIRS) and daytime disturbances (i.e., the ESS), as well as measures that combine both (i.e., the PSQI). To further quantify this, we used simultaneous multiple regression and calculated multiple R²s using the 11 specific ISDI scales as predictors of the WHIIRS, ESS, and PSQI (see Table 6). Together the specific ISDI scales capture 72 to 73% of the variance in the WHIIRS, 52 to 62% of the variance in the ESS, and 69 to 73% of the variance in the PSQI across the two samples.

Interestingly, a number of ISDI scales—including the Nightmares and Movement at Night scales—tended to have uniformly lower correlations with other measures of sleep disturbance. This suggests that these ISDI scales may be capturing dimensions of sleep disturbances that are not measured with the existing instruments. Table 6 reports the multiple R²s using the WHIIRS, ESS, and PSQI to predict the 11 specific ISDI scales and the Daytime Disturbances Scale. Although these existing instruments predict 64 to 65% of the variance of the Fragmented Sleep scale and 48 to 56% of the variance of the Initial Insomnia scale, they predict much less of the variance in the remaining ISDI scales (R²s range from .10 to .56). In particular, the existing sleep measures account for little of the variance in Sensations at Night (18 to 20%), Irregular Schedule (17%), Nightmares (10 to 18%), and Movement at Night (10 to 22%). These data provide further evidence that the ISDI measures a broader range of content than traditional measures of sleep disturbances.

Criterion Validity

Group differences on the ISDI scales

Because the ISDI scales are designed to detect sleep disturbances in psychiatric patients, we would expect this sample to have higher scores than other scale validation samples. Table 7 presents the mean scores and standard deviations for all of the ISDI scales in the scale validation samples. We conducted one-way analyses of variances with Scheffe’s test for post hoc comparisons. Psychiatric patients showed consistently higher scores than the other groups on the majority of ISDI scales. They scored significantly higher than either the students or the sleep disorder patients on 10 of the 12 scales. When comparing the sleep disorder patients with the students, there were four scales on which the patients scored higher (Fragmented Sleep, Light Sleep, Sensations at Night, and Fatigue) and four on which the students scored higher (Initial Insomnia, Anxiety at Night, Nonrestorative Sleep, and Irregular Schedule). Two scales, Movement at Night and Excessive Sleep, did not distinguish among the three groups, suggesting that these experiences are relatively common in the general population. Although one might expect a sleep disorder patient sample to show elevations on the majority of ISDI scales, particularly in comparison with a student sample, it is important to keep in mind the nature of the diagnoses in our sleep disorder sample. The primary diagnosis was obstructive sleep apnea (64.4%) and none of the ISDI scales specifically address this sleep disorder.

Table 7.

Descriptive Statistics for the Iowa Sleep Disturbances Inventory (ISDI) Scales in the Scale Validation Samples

Instrument and scale	Psychiatric patient		Sleep disorder patient		Student
	M	SD	M	SD	M	SD
ISDI
Fragmented Sleep	4.87a	3.14	3.86b	3.22	2.25 c	2.47
Light Sleep	3.46a	2.35	2.99a	2.30	2.23 b	2.27
Sensations at Night	1.98a	2.28	2.01a	2.20	.99 b	1.56
Fatigue	6.16a	3.70	5.58a	3.75	4.30 b	3.34
Initial Insomnia	6.16a	4.22	3.78c	4.24	4.84 b	4.11
Anxiety at Night	4.68a	2.38	3.04b	2.55	4.49 a	2.33
Nonrestorative Sleep	4.81a	2.77	3.83b	2.89	4.89 a	2.85
Irregular Schedule	2.10a	1.91	1.36b	1.77	2.27 a	1.83
Nightmares	4.30a	4.37	2.28b	3.48	2.16 b	2.73
Daytime Disturbances	10.97a	6.04	9.41b	6.24	9.19 b	5.52
Movement at Night	2.59a	2.24	2.23a	2.29	2.45 a	2.18
Excessive Sleep	2.17a	2.12	2.40a	2.04	2.01 a	1.89
WHIIRS	10.24 a	5.35	8.47 b	5.50
PSQI	10.03 a	4.77	8.14 b	4.82
ESS	8.36 b	5.12	9.76 a	5.74

Note. n = 188 (psychiatric patients), 205 (sleep disorder patients), and 374 (students). Within a row, means not sharing the same subscript differ from one another at p < .05, 2-tailed. WHIIRS = Women’s Health Initiative Insomnia Rating Scale. PSQI = Pittsburgh Sleep Quality Index. ESS = Epworth Sleepiness Scale.

To further examine the differences between the psychiatric patients and the other two groups, we computed effect sizes using Cohen’s d (Cohen, 1988). Comparing psychiatric patients with sleep disorder patients, we found medium effect sizes for Anxiety at Night (d = .63), Initial Insomnia (d = .54), and Nightmares (d = .50). We found small effect sizes for Irregular Schedule (d = .40), Nonrestorative Sleep (d = .34), Fragmented Sleep (d = .31), Daytime Disturbances (d = .25), and Light Sleep (d = .20). Comparing the psychiatric patients and students, we found a large effect size for Fragmented Sleep (d = .88), medium effect sizes for Nightmares (d = .61), Light Sleep (d = .52), Sensations at Night (d = .52), and Fatigue (d = .52), and small effect sizes for Initial Insomnia (d = .32) and Daytime Disturbances (d = .31).

To compare the psychiatric and sleep disorder patients on existing sleep questionnaires, we present the means and standard deviations on the WHIIRS, PSQI, and ESS at the bottom of Table 7. Once again, psychiatric patients scored higher than sleep disorder patients on both the WHIIRS and the PSQI. Comparing psychiatric patients and sleep disorder patients, we found small effect sizes for the WHIIRS (d = .32), PSQI (d = .39) and ESS (d = −.25). None of these differences reached the magnitude of effect sizes that we found for several of the ISDI scales, including Anxiety at Night (d = .63) and Initial Insomnia (d = .54).

Relations of the ISDI scales with measures of psychopathology

Next, we examined the relations of the ISDI scales with the IDAS scales to see whether sleep disturbances showed specificity to certain symptoms of psychological disorders. In this analysis, we included scales measuring symptoms of depression (i.e., IDAS Dysphoria) and symptoms of anxiety (i.e., IDAS Traumatic Intrusions, Panic, and Social Anxiety). Based on previous research, nightmares should show specificity to non-sleep-related symptoms of PTSD compared to depression and other anxiety disorders (Neylan et al., 1998). Accordingly, we expected the ISDI Nightmares scale to show specificity to the IDAS Traumatic Intrusions scale. Previous research has also shown that scales that define the higher order factor of Lassitude show specificity to depression, whereas scales that define the Insomnia factor show more moderate, nonspecific association with both depression and anxiety (Koffel & Watson, 2009a). Consequently, we expected the ISDI scales that load on the Lassitude factor to show specificity to the Dysphoria scale.

Table 8 presents these correlations in the psychiatric and sleep disorder patient samples. To examine specificity, we selected the highest correlation of each ISDI scale with the IDAS scales. This correlation was compared to all other IDAS scale correlations using the Williams modification of the Hotelling test for dependent correlations (Kenny, 1987). Correlations in bold are significantly stronger than other correlations within the same row and sample.

Table 8.

Correlations Between the Iowa Sleep Disturbances Inventory (ISDI) Scales and the Inventory of Depression and Anxiety Symptoms (IDAS) in the Scale Validation Samples

ISDI scale	IDAS scale
	Psychiatric patient				Sleep disorder patient
	Traumatic Intrusions	Dysphoria	Panic	Social Anxiety	Traumatic Intrusions	Dysphoria	Panic	Social Anxiety
Nightmares	.55	.41	.42	.38	.48	.43	.37	.37
Daytime Disturbances	.28	.53	.37	.39	.19	.50	.38	.33
Fatigue	.30	.52	.37	.39	.19	.52	.43	.35
Anxiety at Night	.32	.50	.38	.39	.39	.46	.35	.32
Nonrestorative Sleep	.22	.46	.31	.33	.16	.42	.27	.26
Sensations at Night	.13	.19	.37	.19	.12	.20	.37	.17
Fragmented sleep	.18	.35	.33	.36	.27	.29	.37	.21
Initial Insomnia	.21	.27	.32	.20	.29	.39	.31	.21
Irregular Schedule	.29	.32	.39	.26	.23	.38	.26	.31
Movement at Night	.18	.26	.31	.17	.13	.35	.37	.22
Excessive Sleep	.15	.20	.21	.06	.12	.25	.22	.17
Light Sleep	.12	.21	.24	.16	.17	.12	.08	.08

Note. n = 188 (psychiatric patients), 205 (sleep disorder patients). Correlations in bold are significantly stronger than the corresponding coefficients for the other IDAS scales at p < .05, 2-tailed. Correlations of |.15| and greater are significant at p < .05 for psychiatric patients. Correlations of |.14| and greater are significant at p < .05 for sleep disorder patients.

As we hypothesized earlier, the Nightmares scale had a significantly stronger correlation with the Traumatic Intrusions scale than with other IDAS scales in the psychiatric patient sample. Nightmares also had its strongest correlation with Traumatic Intrusions (r = .48) in the sleep disorder patient sample, although the differences here were not significant. Overall, re-experiencing traumatic events at night appears to show some specificity to re-experiencing these events during the day. The Daytime Disturbances, Fatigue, Anxiety at Night, and Nonrestorative Sleep scales all had stronger correlations with the Dysphoria scale than with the other IDAS scales in the psychiatric patient sample. Similarly, in the sleep disorder patient sample the Daytime Disturbances and Nonrestorative Sleep scales had stronger correlations with the Dysphoria scale than the other IDAS scales. With the exception of the Anxiety at Night scale, these ISDI scales all loaded onto the higher order Lassitude factor in the scale-level factor analysis. Thus, measures of Lassitude do appear to show specificity to depression/dysphoria compared to anxiety symptoms (e.g., panic). Finally, the Sensations at Night scale correlated more strongly with the Panic scale than with other IDAS scales in both samples. These data suggest that symptoms of restless legs syndrome, particularly unusual limb sensations during the night, show specificity to physiological sensations of panic during the day.

It is important to note that several of the ISDI scales showed nonspecific associations with the IDAS scales. These ISDI scales included Fragmented Sleep, Initial Insomnia, Irregular Schedule, Movement at Night, Excessive Sleep, and Light Sleep. Overall, it appears that certain dimensions of sleep disturbance are broadly associated with depression and anxiety symptoms, whereas other dimensions show greater specificity vis-à-vis certain depression and anxiety symptoms.²

Discussion

Goals in the Creation of the ISDI: Comprehensive, Well-defined Scales

The ISDI is a self-report measure of sleep complaints that was designed to address the limitations of existing sleep measures. It provides a comprehensive assessment of 11 specific dimensions of sleep complaints, which represents broader coverage than is provided by most instruments (see Table 6). In addition, the ISDI scales are based on structural analyses, with each scale representing a single, underlying dimension of sleep disturbance. Using factor analysis in the development of the ISDI was crucial for several reasons. First, submitting our item pool to factor analysis allowed us to examine the underlying structure of self-reported sleep disturbances and create distinct and unidimensional scales based on this structure. Without these types of analyses, it is difficult to determine how many dimensions of sleep disturbances can be reliably assessed. As is the case with many of the existing sleep instruments, rationally created scales often do not map onto the underlying dimensions within the domain. Second, factor analysis allowed us to develop a more complex, hierarchical structure of sleep complaints that would be difficult to obtain with other methods of scale creation. On one level, the 11 ISDI scales can be viewed as specific measures of sleep disturbances; however, they can also be viewed as measuring the two higher order factors of Insomnia and Lassitude (see Table 3). A limitation of this study is that the factor analyses of dichotomous items were based on Pearson correlations, which can lead to attenuated correlational results and difficulty factors (i.e., factors that are based on endorsement rates instead of the target constructs) (Floyd & Widaman, 1995).

Creation and Psychometric Properties of the ISDI Scales

The ISDI was developed using two initial samples of students and subsequently was validated in three additional samples (students, psychiatric patients, sleep disorder patients). We identified 10 replicable dimensions of sleep disturbances, which were used as the basis for 9 specific ISDI scales (Nightmares, Initial Insomnia, Fragmented Sleep, Anxiety at Night, Light Sleep, Movement at Night, Sensations at Night, Excessive Sleep, Irregular Schedule) and 1 broader scale of Daytime Disturbances. This general scale can be further divided into the subscales of Nonrestorative Sleep and Fatigue. The 11 specific ISDI scales each measure a single, relatively narrow dimension of sleep disturbance, as indicated by their coefficient alphas and AICs (see Table 1). Furthermore, they are distinct from one another, with most inter-scale correlations falling within the low to moderate range (see Table 2); in fact, the highest inter-scale correlation was for the two Daytime Disturbances subscales of Nonrestorative Sleep and Fatigue. The ISDI also shows good evidence of short-term retest reliability, with a mean retest correlation of .79 over two weeks (see Table 5). Finally, the ISDI scales show good convergent and discriminant validity with some of the most widely used sleep measures, including the PSQI, the WHIIRS, and the ESS; in addition to accounting for much of the variance in these existing measures, the ISDI also appears to measure sleep disturbances not captured by these traditional measures (e.g., Sensations at Night, Nightmares, Movement at Night) (see Table 6).

ISDI and Psychopathology

One of the primary goals in creating the ISDI was to develop the necessary assessment tools to examine the overlap of sleep disturbances and psychopathology. Previous research has shown that a variety of sleep disturbances are significantly related to psychological disorders. In some cases, sleep disturbances appear to show specificity to certain disorders (e.g., lassitude with depression), whereas other types of sleep disturbances are broad, nonspecific markers of general distress (e.g., insomnia). More research is clearly needed to determine how self-reported sleep disturbances map onto psychological disorders; without the availability of comprehensive sleep instruments, however, this research cannot move forward.

In this study, we report some initial data examining the association of the ISDI with psychopathology. Specifically, we compared the mean ISDI scores of psychiatric patients to those of students and sleep disorder patients. We also examined the associations of the ISDI scales with symptom measures of depression and anxiety. Overall, the majority of the ISDI scales are significantly higher in psychiatric patients than in either sleep disorder patients or college students (see Table 7). In addition, individual ISDI scales show specificity to certain types of symptoms (see Table 8). For example, the Nightmares scale was specific to the IDAS Traumatic Intrusions scale, and several ISDI scales measuring Lassitude showed specificity to the IDAS Dysphoria scale. One unexpected finding was that the Sensations at Night scale showed specificity to the IDAS Panic scale. Although some previous research has shown a link between restless legs syndrome and symptoms of various anxiety disorders (Lee, et al., 2008; Picchietti & Winkelman, 2005; Winkelmann, et al., 2005), a specific connection between restless legs syndrome and panic disorder is not well-established. This is clearly an area that will require additional research.

In contrast to the ISDI scales that showed strong specific association with single symptom dimensions, a number of scales show more general, nonspecific relations with symptoms of depression and anxiety. However, it is important to note that our assessment of psychopathology was limited to measures of anxiety and depression. An expanded assessment battery will be necessary to continue examining the relations between the ISDI scales and psychopathology. The current studies represent a first step in the continuing search for evidence of specificity.

Directions for Future Research

Further validation of the ISDI

Although the ISDI has shown good psychometric properties in terms of internal consistency, retest reliability, convergent and discriminant validity, and criterion validity in our samples, it will be necessary to replicate these findings in additional samples. The current study was also limited to questionnaire measures of depression and anxiety. It will be necessary to examine the ISDI in relation to (1) a larger variety of psychological symptoms and (2) interview data on sleep and psychological functioning. We are currently collecting data on dissociation, schizotypy, and substance use to examine how these daytime symptoms relate to the ISDI scales. In addition, we are gathering interview data on sleep problems and psychological symptoms to determine if the ISDI shows convergent, discriminant, and criterion validity across assessment methods.

Additional higher order factors of sleep disturbances

It is clear that the two-factor structure of sleep disturbances, consisting of the higher order dimensions of Insomnia and Lassitude, is a robust finding that replicates across samples. The 11 specific ISDI scales can be viewed as lower order facets of the higher order dimensions of Insomnia and Lassitude; these higher order dimensions, in turn, show differential relations with measures of depression and anxiety (Koffel & Watson, 2009a). However, we also have evidence from the current study that some sleep dimensions do not fit solidly within this two-factor structure. For example, the Nightmares, Movement at Night, and Sensations at Night scales did not load above .44 on the Insomnia and Lassitude factors in the second scale development sample and the scale validation samples; moreover, these scales tended to cohere together when a third factor was extracted. It is possible that a third higher order factor of sleep disturbances, perhaps representing more unusual sleep experiences, can be obtained. However, this third factor was not well defined in the current study and does not appear to be well represented in the current ISDI scales. As a result, it did not replicate across samples. In our ongoing research, we are developing additional ISDI scales to improve our coverage of higher order factors that go beyond Insomnia and Lassitude.

The ultimate goal of the research presented in this paper is to provide a comprehensive measure of sleep disturbances that has both clinical utility and research applications. In particular, it is our hope that the ISDI will provide useful information in future studies examining the structure of self-reported sleep disturbances and will be used explicate the relations of these disturbances with daytime symptoms. Although we have presented some preliminary evidence in this area using the ISDI and symptom measures of depression and anxiety, there is still a great deal of work that remains to be done.