Psychometric properties of the Disturbing Dream and Nightmare Severity Index–Korean version

Abstract

Study Objectives:

This study aimed to examine psychometric properties of the Disturbing Dream and Nightmare Severity Index (DDNSI) in individuals aged between 18 and 39.

Methods:

All participants (n = 674) were asked to complete the DDNSI, including the modified Nightmare Effects Survey. Additionally, 109 participants were tested for test-retest reliability after 3 months. Among our sample, 229 (33.9%) reported having at least 1 nightmare per month.

Results:

Internal consistency was evaluated for the total sample (Cronbach’s α = .920) and separately for individuals reporting more than once per month (Cronbach’s α = .755). Test-retest reliability after 3 months was .705. Convergent validity of the DDNSI with Nightmare Effects Survey was also satisfactory (r = .638, P < .001). Finally, exploratory factor analysis was conducted to explore the construct of the DDNSI, and results indicated that it consisted of 2 factors, nightmare frequency and nightmare distress [χ²(df) = 2.241(1) ∆χ² (∆df) = 155.575(4), Tucker-Lewis incremental fit index = .980, root mean square error of approximation (90% confidence interval) = .074 (0, .208), standardized root-mean-square residual = .011].

Conclusions:

The DDNSI is a reliable measure of nightmare severity that can be used in various settings.

Citation:

Lee R, Krakow B, Suh S. Psychometric properties of the Disturbing Dream and Nightmare Severity Index–Korean version. J Clin Sleep Med. 2021;17(3):471–477.

BRIEF SUMMARY

Current Knowledge/Study Rationale: Although there is a great abundance of clinical implications for nightmares, there have been few studies examining the psychometric properties of questionnaires to assess nightmares. The Disturbing Dream and Nightmare Severity Index (DDNSI) is a questionnaire that specifically measures nightmares and is used in clinical settings, but there have been no validation studies of the DDNSI.

Study Impact: This study investigated the psychometric properties of the DDNSI in young adults. The DDNSI showed adequate internal consistency and test-retest reliability. The internal structure of the DDNSI consisted of 2 factors, nightmare frequency and nightmare distress, based on exploratory factor analysis. The DDNSI is a reliable measure of nightmare severity that can be used in research and clinical settings.

INTRODUCTION

According to the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5), ¹ nightmares are “typically lengthy, elaborate, story-like sequences of dream imagery that seem real and that incite anxiety, fear, or other dysphoric emotions, and usually terminate with awakening and rapid return of full alertness.” Experiencing nightmares is common, and prevalence estimates for frequent nightmares among general adult population come within range of 2% to 45%.^2–6 Studies investigating the prevalence of nightmares vary in specific sample characteristics and definition on nightmare occurrence, thus, there is a wide range in these estimates. Furthermore, in the psychiatric sample, the prevalence of frequent nightmares is much higher, ranging from 29.9% to 62.3%^7,8 compared to the prevalence found in the general population.

Previous studies have shown that nightmares independently increase the risk of mental disorders^9–11 and suicide even after controlling for depression.^12,13 While there is limited data, 1 prospective longitudinal cohort study reported that the existence of nightmares in predeployment was associated with an increased risk for post-traumatic stress disorder symptoms, unlike insomnia complaints.¹⁴ Additionally, nightmares are often residual symptoms (if not an independent disorder) after the primary mental disorder is treated; and, chronic nightmares play a crucial role in the pathology of post-traumatic stress disorder.^15,16 Furthermore, 1 study suggested that nightmares should be seen as a distinct sleep disorder with specific symptoms, because nightmares are independent from objective sleep disturbances but cause significant impairment during both day and night.¹⁷ Nightmare disorder, characterized by repeated occurrences of vivid dreams that involve threats, usually occur in the second half of a sleep episode, and are accompanied by rapid awakening and alertness, is specified as an independent sleep disorder in the DSM-5. Therefore, nightmares are clinically relevant and are a primary target for intervention in various clinical settings.

Unfortunately, systematic assessment and interventions for nightmares are quite rare in mental health clinics or other clinical realms, as clinicians typically do not ask patients about nightmares.^18,19 Nadorff, Nadorff, and Germain²⁰ found that more than 60% of persons with nightmares once per month or more often were not routinely referred to specialists for nightmare assessment or therapy; and, the very notion of evaluation of and intervention for nightmares was not well known to patients. One study by Krakow¹⁹ investigated nightmare complaints in sleep clinics and found approximately 16% of patients presenting with salient nightmares that disrupted sleep and were also linked to other health outcomes. Despite these findings, a vast majority of individuals complaining of nightmares do not seek professional help for their condition.

In the past, nightmare researchers have been interested in the frequency of nightmares, but research has shown psychological disturbances may be caused more by nightmare distress rather than frequency of nightmares.^21–24 In this context, researchers have suggested nightmare conditions comprise both frequency and distress as separate or potentially overlapping constructs. The DDNSI is a widely used questionnaire for nightmares and measures both nightmare frequency and nightmare distress separately.^12,25–41 However, despite the DDNSI being used in many studies, only internal consistency (0.80–0.93) has been investigated and reported.^{12,13,19,32,33,40–42} In addition, 1 study by Krakow and colleagues found that DDNSI scores decreased to below 10 score at post-treatment, and these improvements were maintained at 12-week follow-up.⁴² Thus, a total score of 10 or greater has been used to predict the presence of a clinical level of disturbing dreams and nightmares. Despite these studies, there has been little information on the psychometric properties of this questionnaire. In this study, we aimed to clarify the internal structure of the DDNSI and investigate clinical correlates of nightmares for future evaluation of nightmares.

METHODS

Participants and procedure

We recruited 229 participants from psychology classes and through community flyers and online advertisement. A total of 695 individuals who expressed an interest completed online surveys. The inclusion criteria were: age from 18 years to 39 years and fluency in the Korean language. We limited the age group to 18–39 because previous studies have reported adults in their 20s and 30s experienced the most frequent nightmares.^43,44 There were no exclusion criteria, and specific psychiatric diagnoses were not excluded. The participants were asked to read a cover sheet informing them of the purpose of the study and their rights prior to completing the survey and were also required to complete the survey in a single sitting. All participants provided online informed consent. Data cleaning resulted in 21 participants being excluded from the analyses due to missing data. Among the sample, 229 participants (33.9%) who reported experiencing at least 1 nightmare per month were selected from the larger pool of 674 individuals. Among the original participants, 109 participants who agreed to participate in further research were tested for test-retest reliability after 3 months. The sample was predominately female (69.4%), with an age range of 18–39 years (mean age 23.72 years, standard deviation 3.76 years). This research was approved by the Sungshin Women’s University Institutional Review Board.

Measures

All questionnaires described in the following were translated from English to Korean, and then translated back from Korean to English and compared with the original version by an independent translator who was bilingual and a native English speaker.

Demographic information

Participants responded to questions about age and sex.

Disturbing Dream and Nightmare Severity Index

The Disturbing Dream and Nightmare Severity Index (DDNSI) is a self-retrospective measure of current nightmare and disturbing dream frequency and severity.⁴² There are 2 sections about the frequency of nightmares and the number of nightmares in a given interval per unit of time (ie, weekly, monthly, or yearly) as well as the severity and intensity of the nightmare problem ranging from no problem (0) to extremely severe problem/intensity (6). The measure also evaluates how often nightmares result in awakenings ranging from never/rarely (0) to always (4). The index score is calculated by summing the number of nightmares per week (up to 14), number of nights with nightmares per week (0–7), and the frequency of nightmare-related awakenings (0–4), ratings of the severity of the nightmares (0–6), and the intensity of the nightmares (0–6). Previous research has determined that an index score ≥ 10 is consistent with a nightmare disorder being present.⁴² Chronicity was determined by a single item question (“Please estimate the number of months or years you have had disturbing dreams and/or nightmares”).

Modified Nightmare Effects Survey

The Modified Nightmare Effects Survey (mNES) is a modified version from the Nightmare Effects Survey, which correlates with other dimensions of mental health or impairment.⁴⁵ The mNES has historically been administered as a part of the DDNSI. The Nightmare Effects Survey consists of 11 self-report questions, including the effect of nightmares on sleep, work, relationships, daytime energy, school, mood, sex life, diet, mental health, physical health, and leisure activities. The mNES consists of 10 self-report questions, and each item is rated on a scale from not at all (0) to a great deal (3). Scores range from 0–30, with higher scores reflecting more impairment attributed to nightmares. The mNES was only administrated to participants who reported experiencing at least 1 nightmare per month. Internal consistency of this measure was acceptable in the sample (α = .92).

Statistical analyses

The data analyses were conducted using Statistical Package for Social Science 21.0 (IBM Corp., Armonk, NY), Mplus version 7,⁴⁶ and Factor 10.4.⁴⁷ The P value was set at < .05 for statistical significance. Descriptive statistics were used to examine demographic data, and internal consistency was examined with Cronbach’s α coefficient (all participants including the test-retest group’s first assessment).

For test-retest reliability, Pearson’s correlation coefficient was used to analyze 109 participants of the total participants. Nightmare chronicity was partialled out for estimating test-retest reliability. “Partial correlation” is measurement of the strength and direction of a linear relationship between 2 continuous variables while controlling for the effect of 1 or more continuous variables. We investigated simple correlation between DDNSI scores at time 1 and time 2, then examined partial correlation between these 2 time points by controlling for nightmare chronicity. This was because nightmare chronicity is one of the mediating factors between nightmare frequency and nightmare distress, and it may have implications for the stability of DDNSI score.

To determine convergent validity, the correlation between the DDNSI and the mNES scores were examined using Pearson’s correlation coefficient. Exploratory factor analysis (EFA) with a partially specified target rotation was conducted to explore the construct of the DDNSI using maximum-likelihood estimator (referred to as ML) (Table 2). We conducted the analysis based on Browne’s partially specified target rotation, which entails the researcher first performing exploratory rotation and then rotating the same data based on the results with an exploratory rotation by specifying the small factor coefficients as zero and other estimates as free parameters, according to the hypothesis of a researcher.^48,49 The analysis procedure was performed in the following 3 stages: (1) investigating the possibility of factor analysis, (2) determining the number of factors by an exploratory rotation, using geomin oblique rotation because the correlation between each factor was not 0, and (3) conducting factor analysis based on oblique rotation to a partially specified target.^48–50 We consulted the following fit indices to determine global model fit: (1) comparative fit index (CFI),⁵¹ (2) Tucker-Lewis incremental fit index (TLI),⁵² (3) root mean square error of approximation (RMSEA),⁵³ and (4) standardized root-mean-square residual (SRMR).⁵⁴ To determine good fit of the model, the following values were used: CFI and TLI above .90, and RMSEA below .08, and SRMR below .10.⁵⁵

Table 2.

EFA factor loadings using partially specified target rotation.

Item	Mean ± SD	F1a	F2b
1. How many nights in a week/a month/a year do you have disturbing dreams and/or nightmares?	1.545C ± 1.611	1.833***	–0.128
2. How many disturbing dreams and/or nightmares do you have in a week/a month/a year?	2.220C ± 2.568	1.999***	0.314
3. On average, do your nightmares wake you up?	2.066 ± 1.184	–0.109	0.540***
4. How would you rate the severity of your disturbing dreams and/or nightmare problem?	2.686 ± 1.330	0.082	1.032***
5. How would you rate the intensity of your disturbing dreams and/or nightmares?	3.109 ± 1.192	0.004	1.003***
The interfactor correlation coefficient	0.406***	0.406***	0.406***

N = 229. ***P < .001. ^aNightmare frequency. ^bNightmare distress. ^COn a weekly basis EFA = exploratory factor analysis, SD = standard deviation.

RESULTS

Reliability and validity of the DDNSI

In this college-based sample, etc., nightmares were found to occur yearly in 30.9%, monthly in 20.5%%, and weekly in 13.5%. Internal consistency was adequate for the DDNSI total score (0.920) of the total sample (0.755). Of the total sample, 109 participants were evaluated for test-retest reliability after 3 months, which was found to be adequate (r =.705, P < .001). Partial correlation of the DDNSI score was .607 (P < .001) after controlling for nightmare chronicity (Table 1). The DDNSI total score was significantly related to the mNES total score (r = .638, P < .001).

Table 1.

Test-retest reliability of the DDNSI.

Variable	r	ra	Mean ± SD
T1 DDNSI	0.705**	0.607***	2.28 ± 4.95
T2 DDNSI	0.705**	0.607***	2.20 ± 5.04

N = 109. **P < .01. ***P < .001. ^aPartial correlation controlling for nightmare chronicity. DDNSI = Disturbing Dreams and Nightmare Severity Index, SD = standard deviation, T₁ = first assessment, T₂ = 3 months after the first administration.

Factor structure of the DDNSI

Factorability of the DDNSI was examined. Several well-recognized criteria for factorability were used. First, it was observed that item–total score correlation was at least .3 for 4 of the 5 items, suggesting reasonable factorability. One of the items correlated below .3 with the total score but was not excluded because the item was related to sleep disturbance associated with nightmares. Second, the Kaiser-Meyer-Olkin measure of sampling adequacy was .626, above the commonly recommended value of .6, and Bartlett’s test of sphericity was significant (χ² = 634.000, df = 10, P < .001). Given these overall indicators, factor analysis was deemed to be suitable for all 5 items.

A scree test and parallel analysis were used because the primary purpose was to identify and compute composite scores for the factors underlying the DDNSI. The eigen value plot and the associated scree test showed evidence for a 1- or 2-factor solution: Eigen values for the first 3 factors were 2.67, 1.23, and 0.71, respectively, showing a clear elbow at the second eigen value. The possible factor solution pointed to 2 factors and was supported statistically, as overall fit for the 1-factor model was not good [χ²(df) = 157.816(5), P < .001; CFI = 0.760; TLI = 0.519; RMSEA = 0.365; SRMR = 0.150), compared to the 2-factor model [χ²(df) = 2.241(1), P = .134; ∆χ² (∆df) = 155.575(4), P < .001; CFI = 0.998; TLI = 0.980; RMSEA = 0.074; SRMR = 0.011].

The 2-factor model was identified as the best solution based on heuristic and statistical approaches. Finally, the factor loadings were examined in the first EFA. The pattern loadings in the 2-factor model ranged from –0.094 to 1.112 and the structural loadings ranged from 0.068 to 1.110. In the 2-factor model, items 1 and 2 loaded highly on factor 1 (1.112 and 0.759 in the pattern loadings and 1.110 and 0.821 in the structural loadings, respectively); the authors labeled this factor as “nightmare frequency”. Items 3, 4, and 5 loaded highly on factor 2 with the pattern loadings ranging from .453 to .871 and the structural loadings ranging from .419 to .869; this factor was labeled as “nightmare distress”. There were no crossloadings of .3 or above. The interfactor correlation coefficient was statistically significant (r = .357, P < .05). Since the interfactor correlation coefficient did not exceed .850, the 2 factors can be considered as independent constructs (Kline, 2015).

To demonstrate the simpler 2-factor structure, a partially specified target rotation (Table 2) was conducted based on the results with an exploratory rotation. The basic structure and model fit with a partially specified target rotation was equal to the results conducted with an exploratory rotation. In addition, the correlations between factors in a partially specified target rotation increased compared to the factors in an exploratory rotation (.406). In this study, given that the modification index value was less than 10 for covariance between measurement errors, no further analysis, such as exploratory structural equation modeling, was performed⁵⁰

DISCUSSION

This study is the first to investigate multiple psychometric properties of the DDNSI. Among the sample, 33.9% reported experiencing more than 1 nightmare per month, indicating that nightmares are a common experience in young adults. The DDNSI showed adequate internal consistency, test-retest reliability, and convergent validity. Additionally, exploratory factor analysis results yielded a 2-factor structure that reflected nightmare frequency and nightmare distress.

Psychometric properties of the DDNSI

Cronbach’s α coefficients for the total sample were adequate, although internal consistency for persons with nightmares once per month or more often yielded lower indices compared to that of the total sample. The observed differences were most likely related to the response structure of the DDNSI. The occurrence of nightmares should precede distress caused by nightmares. In addition, nightmares are relatively frequent in the clinical group, and most people do not experience nightmares. The DDNSI reflects these characteristics by measuring whether the occurrence of nightmares is absent or experienced annually, monthly, or weekly. Only those who respond to monthly or weekly nightmares are asked follow-up questions about the severity and intensity of nightmares and the frequency of awakenings. Therefore, the internal consistency for the total sample was most likely higher than that of individuals with nightmares once per month or more often. Considering that the DDNSI was developed to evaluate the severity of nightmares for people who are predictably more likely to suffer from nightmares, we suggest utilizing the internal consistency for those who responded to all questions of the DDNSI as the most reliable measure. Additionally, the internal consistency for the total sample is higher in comparison with results of earlier studies.^{12,13,19,32,40–42} This may be because the sample selection of the above mentioned studies, with the exception of Krakow et al.,⁴² did not include participants based on nightmare frequency or nightmare distress. As a result, the samples of these studies had a high proportion of individuals who reported not experiencing any nightmares.

Test-retest reliability of the DDNSI at 3 months was acceptable. Because we considered nightmare chronicity to be a mediator between nightmare frequency and nightmare distress, we concluded that the duration of nightmares may affect the stability of the DDNSI score. Thus, we not only considered simple correlation analysis but also partial correlation analysis. The partial correlation coefficient was lower than the simple correlation coefficient but still significantly high. We conclude that the DDNSI is a stable measure even after considering nightmare chronicity.

The DDNSI was also correlated with an independent measure of nightmare impact (Nightmare Effects Survey) that measured the magnitude of dysfunction from nightmare. Higher DDNSI scores were associated with higher mNES scores, supporting the convergent validity of the DDNSI.

Issues with scale refinement

EFA was conducted to explore the internal structure of the DDNSI. First, we investigated whether the data was appropriate for EFA through basic statistical analysis and content review for item refinement. For the third item, which measures the arousal perception of the nightmare, the correlation between the item and total score was lower than .30. This finding can be interpreted with 2 possible explanations. First, this may reflect inconsistencies of previous studies that have questioned whether awakening from nightmares is an indirect indicator of the emotional intensity of the nightmare experience. There has been continuous discussion about whether nightmares can be defined as bad dreams accompanied by awakenings, which has consistently been the case in previous studies.^56–61 Previous studies have noted a difference in the emotional intensity experienced from nightmares based on whether the dream was accompanied by an awakening.^29,62 In contrast, other studies have suggested that most nightmares do not cause awakenings,^63,64 with less than a quarter of chronic nightmare sufferers reporting always waking up when they experience nightmares.⁶⁵ In addition, it has been reported that about 45% of nightmares without accompanied awakenings have shown that the emotional intensity was equal to or even greater than nightmares that cause awakenings.⁶² The discrepancy in these findings implies 2 possibilities—that being awakened by nightmares may be a qualitative difference of 2 different phenomenon or a difference in the degree of the same phenomenon.

Second, the low correlation between the third item and total score may be due to different response scales used for the items. The questions about the severity and intensity of nightmares used a 7-point Likert scale, whereas the item asking about awakening by the nightmares were answered using a 5-point Likert scale. The difference in score ranges may have had the effect of diminishing the impact of nightmare arousal on nightmare severity. Therefore, we suggest unifying the response scale of the third item to a 7-point Likert scale, similar to the fourth and fifth items.

In addition, 2 items on the nightmare frequency scale were very highly correlated (r = .911). Statistically, these 2 items are not considered independent items. However, the total nights of nightmares experienced and the number of nightmares or dreams per night should be separated, because persons with nightmares once per month or more often report experiencing several nightmares per night. Therefore, the present study was conducted without removing the item despite this high correlation. In order to distinguish between the total nights of nightmares experienced and the number of nightmares dreamt per night, we suggest the removal of the restriction on the number of nightmares per night.

Factor structure of the DDNSI

Results from exploratory factor analyses indicated that DDNSI consisted of a 2-factor structure reflecting nightmare frequency and nightmare distress without model modification. The first factor of the DDNSI revealed in this study was labeled “nightmare frequency” and the second factor was labeled “nightmare distress”. The items that are the index of the nightmare frequency and the nightmare distress appeared as dimensions separated from each other. There is a significant positive correlation between the 2 factors, but they are independent psychological constructs. One study by Lee and Suh²⁴ investigating separate effects of nightmare frequency and nightmare distress on suicidal ideation found that nightmare distress fully mediated the relationship between nightmare frequency and suicidal ideation after controlling for insomnia. These results together with our findings suggest that nightmare frequency and nightmare distress should be considered independent psychological constructs, and further investigation of the differential impacts of both factors should be considered in the context of risk for psychopathology and suicidal behavior.

Limitations

This study has several limitations: (1) Test-retest reliability was significant, but the results have limitations in generalizability, as the sample used for test-retest reliability was primarily composed of females. Future studies should include male participants to verify test-retest reliability. (2) Convergent validity was significant, but the interpretation was limited in that the mNES has also not yet been validated. (4) We explored the internal construct of the DDNSI through EFA, but future studies should conduct confirmatory factor analysis with an independent sample to verify factor structure of the DDNSI. (5) We investigated psychometric properties of the DDNSI for general adults aged 18–39 years, but the DDNSI needs to be validated, especially in clinical populations. (6) There was a potential bias of the study sample in that people interested in the topic of nightmares or sleep may have chosen to participate in the study.

Despite these limitations, the DDNSI is a valid measure that assesses nightmares reflecting 2 nightmare dimensions composed of nightmare frequency and nightmare distress. This study has significance in that it verified the reliability and validity of the DDNSI. The DDNSI can be useful for screening patients with nightmare disorder and examining the course of severity of problematic nightmares. Futures studies should verify predictive validity and yield cut-off points to diagnose nightmare disorder.

DISCLOSURE STATEMENT

All authors have read and approved the final manuscript. Work for this study was performed at the authors’ respective institutions. The authors declare no conflicts of interest.

ABBREVIATIONS

CFI

comparative fit index

DDNSI

Disturbing Dream and Nightmare Severity Index

EFA

exploratory factor analysis

mNES

Modified Nightmare Effects Survey

RMSEA

root mean square error of approximation

SRMR

standardized root-mean-square residual

TLI

Tucker-Lewis incremental fit index