A Pre-Screening Questionnaire to Predict Non-24-Hour Sleep-Wake Rhythm Disorder (N24HSWD) among the Blind

Abstract

Study Objectives:

There is currently no questionnaire-based pre-screening tool available to detect non-24-hour sleep-wake rhythm disorder (N24HSWD) among blind patients. Our goal was to develop such a tool, derived from gold standard, objective hormonal measures of circadian entrainment status, for the detection of N24HSWD among those with visual impairment.

Methods:

We evaluated the contribution of 40 variables in their ability to predict N24HSWD among 127 blind women, classified using urinary 6-sulfatoxymelatonin period, an objective marker of circadian entrainment status in this population. We subjected the 40 candidate predictors to 1,000 bootstrapped iterations of a logistic regression forward selection model to predict N24HSWD, with model inclusion set at the p < 0.05 level. We removed any predictors that were not selected at least 1% of the time in the 1,000 bootstrapped models and applied a second round of 1,000 bootstrapped logistic regression forward selection models to the remaining 23 candidate predictors. We included all questions that were selected at least 10% of the time in the final model. We subjected the selected predictors to a final logistic regression model to predict N24SWD over 1,000 bootstrapped models to calculate the concordance statistic and adjusted optimism of the final model. We used this information to generate a predictive model and determined the sensitivity and specificity of the model. Finally, we applied the model to a cohort of 1,262 blind women who completed the survey, but did not collect urine samples.

Results:

The final model consisted of eight questions. The concordance statistic, adjusted for bootstrapping, was 0.85. The positive predictive value was 88%, the negative predictive value was 79%. Applying this model to our larger dataset of women, we found that 61% of those without light perception, and 27% with some degree of light perception, would be referred for further screening for N24HSWD.

Conclusions:

Our model has predictive utility sufficient to serve as a pre-screening questionnaire for N24HSWD among the blind.

Citation:

Flynn-Evans EE, Lockley SW. A pre-screening questionnaire to predict non-24-hour sleep-wake rhythm disorder (N24HSWD) among the blind. J Clin Sleep Med 2016;12(5):703–710.

INTRODUCTION

Non-24-hour sleep-wake rhythm disorder (N24HSWD) is common among blind individuals with no perception of light.¹ This disorder arises from desynchronization between the 24-h solar day and the intrinsic circadian drive to sleep,¹ due to the absence of daily ocular light exposure, which is the primary zeitgeber for synchronization to the 24 h day.²

Although N24HSWD is common among totally blind individuals, not all totally blind individuals manifest this circadian rhythm sleep disorder. Some individuals retain circadian photoreception even in the absence of vision.^1,3–5 In addition, the prevalence of non-circadian sleep disorders is high among the blind population,^6–8 often making it difficult for a clinician to differentiate between this circadian sleep disorder and other common sleep disorders, such as insomnia. It is possible to diagnose N24HSWD from the measurement of a circadian biomarker, such as the melatonin or cortisol rhythm,^1,6,9–14 but such an evaluation requires that a patient collect serial urine at home, or saliva or blood samples under controlled laboratory conditions, over a minimum of 24 hours, over multiple weeks, to measure the change in rhythm timing (circadian period, τ).

BRIEF SUMMARY

Current Knowledge/Study Rationale: Non-24-hour sleep-wake rhythm disorder (N24HSWD) is prevalent among individuals who are totally blind, but diagnosis of this disorder is challenging due to the high prevalence of other sleep disorders experienced by blind individuals. Accurate diagnosis of N24HSWD requires evaluation of circadian phase markers, such as urinary melatonin sampled over several weeks; therefore it is desirable to have a screening tool in order to identify blind patients at the highest risk for N24HSWD.

Study Impact: We developed the first screening tool to discriminate blind individuals at risk for N24HSWD from those unlikely to have the disorder. Our tool provides clinicians with a cost-effective way to pre-screen blind patients with sleep complaints for N24HSWD.

Although we have shown that such evaluations are feasible in the blind population in research studies,^6,10,14,15 given the lack of experience in the medical community diagnosing N24HSWD in the blind, there is a need for a pre-screening tool to identify which patients are likely to have N24HSWD and which patients should be referred for evaluation for other sleep disorders. Such a tool would allow for the targeting of confirmatory melatonin or cortisol rhythm testing, thereby reducing the number of patients needing such a test, and consequently patient burden, and reducing the overall cost of evaluations. A major challenge with developing such a screening tool is that, although N24HSWD is common among the totally blind population, the disease is rare relative to the general population and is considered an orphan disorder. The relatively small population available for study (n ∼100,000 in the United States and ∼130,000 in Europe) limits the sample size available for validation of such screening tools. No study to date has surveyed a sufficient number of blind patients while simultaneously measuring circadian entrainment status for validation of a survey instrument using standard test-retest methods. Our goal therefore was to use statistical bootstrapping techniques in order to develop a survey-based prediction model for N24HSWD among those with visual impairment from a limited dataset. Bootstrapping is a technique whereby a single cohort can be randomly re-sampled repeatedly in order to generate a set of predictors that lead to a disease or outcome of interest.

METHODS

Participants

Details of the survey methods and recruitment procedures are described elsewhere.^6,16 In brief, 130 women were recruited from a cohort of 1,392 blind women who participated in a survey (administered either by ‘phone, or self-administration via large print, braille, or internet), of whom 127 had sufficient biomarker data available for classification of circadian entrainment status. Participants were eligible for the field study if they were not taking medication known to affect sleep, melatonin or estrogen production (e.g., hormone replacement therapy, hypnotics, β-blockers, antidepressants, benzodiazepines). Ethical permission for the study was granted by the Institutional Review Board at Partners Healthcare (2003-P-000263) and by the Department of Defense Human Subjects Research Review Board (HSRRB Log No. A-12744). Written informed consent was obtained from all participants.

Study Schedule and Sample Measurement

Field study participants were asked to complete daily sleep/nap diaries for 8 consecutive weeks. During this time, they were asked to collect 48-h sequential urine samples in ∼4 hourly episodes during the day and ∼8 hours overnight on 2 (n = 90) or 3 (n = 40) occasions.^6,10 The first two 48-h collection periods that exhibited a significant cosine fit were used in analyses for each participant. Urine collection and 6-sulfatoxymelatonin (aMT6s) radioimmunoassay procedures were performed as described by Lockley et al.¹⁰

Assessment of Circadian Rhythm Disorders

Participants were classified as entrained or having N24HSWD as described in Flynn-Evans et al.⁶ Briefly, participants were classified as entrained if their circadian period fell within 23.88–24.12 h.¹⁷ A participant was considered to have a non-24-h rhythm if their circadian period was outside the normal range (i.e., < 23.88 h or > 24.12 h). For the purposes of the current analysis, entrainment status was not sub-classified by phase of entrainment (e.g., advanced, delayed); all those exhibiting a 24-h period were considered entrained.

Statistical Analysis

We used 40 individual questions derived from the larger survey that included information about sleep, light perception, visual impairment, unilateral and bilateral enucleation, scleral shell use, and components of the Pittsburgh Sleep Quality Index¹⁸ and Horne Östberg Questionnaire¹⁹ that were considered for the predictive model (Supplemental Material). These candidate predictors were included in a forward selection stepwise logistic regression model to predict N24HSWD (defined using urinary 6-sulphatoxymelatonin phase or period, a gold standard measure of circadian rhythms, as described above) with the significance level for entry into and exit from the model set at the p < 0.05 level. This model was subjected to 1,000 bootstrapped iterations, with each model using data from 127 participants selected at random with replacement.²⁰ All candidate predictors that were not selected for inclusion at least 10 times in the first 1,000 bootstrapped models were removed from consideration in subsequent models (Supplemental Material). The model was subjected to a second round of 1,000 bootstrapped model iterations using only candidate predictors that were selected at least 10 times in the first round of sampling. Following the second round of forward selection bootstrapping, any candidate predictor that was not selected at least 10% of the time was removed from the final model.

The final model of candidate predictors determined by the forward selection were included in a logistic regression model to predict N24HSWD over 1,000 bootstrapped models using data from 127 participants selected at random with replacement. The concordance statistic (c-statistic) for the area under the receiver operating curve (ROC) was calculated to estimate the discriminative ability of the model. In order to improve generalizability of the study cohort to the population, the optimism of the model (average difference in the c-statistic between model runs, to assess overfitting) was calculated and the c-statistic was adjusted accordingly. We also calculated the sensitivity, specificity, and positive and negative predictive value from the model.

The beta coefficients for the final model were calculated in order to allow for clinical utility and disease prediction. We rounded each beta coefficient to the nearest integer in order to simplify the model scoring for clinical use.

We applied the model to 1,262 blind women who took our larger survey but did not collect urine samples in order to determine how many of those study participants would have been referred for screening using our tool.

Demographic variables were compared using independent, two-sample t-tests for continuous variables and χ² tests for categorical variables.

RESULTS

The characteristics of participants classified as entrained versus non-entrained, and therefore having N24HSWD, are described in Table 1. These groups differed in several expected ways; for example, a higher proportion of those with N24HSWD reported having no perception of light (NPL), bilateral enucleation and cycles of good and bad sleep.

Table 1.

Characteristics of entrained female blind subjects (entrained) and those with non-24-hour sleep-wake rhythm disorder (N24HSWD).

There were 40 survey questions that were considered for inclusion in the model. The first forward selection model resulted in the removal of 23 survey questions (supplemental material). The second forward selection model resulted in the removal of an additional 9 survey questions (supplemental material), leaving a final model that included 8 questions. Table 2 shows the final set of predictors, the number of times that each was chosen in the final forward selection model, and the beta coefficients for the predictors.

Table 2.

Predictors associated with non-24-hour sleep-wake rhythm disorder (N24HSWD) and their associated beta coefficients and the number of times selected in 1,000 bootstrapped logistic regression forward selection models with a p < 0.05 threshold for entry and to remain in the model.

The concordance statistic estimating the area under the ROC, generated from the bootstrapped logistic regression, was 0.96 (Figure 1). Given the potential for overtraining a model based on bootstrapped datasets, the optimism, or average difference in the c-statistic between bootstrapped samples, was calculated to be 0.11, yielding a final adjusted c-statistic of 0.85. These data suggest that the model has good predictive utility.

Figure 1. Receiver operating curve for the eight questions selected for the final model.

In order to simplify the model for ease of scoring in a clinical setting we rounded the beta coefficients to the nearest integer and classified all participants having a score ≥ 0 as meeting criteria for further screening for N24HSWD (Table 3). Patients with sleep complaints and scores < 0 should be screened for other sleep disorders. We chose this cut point in order to maximize the positive predictive value of the model.

Table 3.

N24HSWD questionnaire and outcome scoring.

Using our prediction model on the 116 participants who answered all 8 questions, we found that 88% of participants with N24HSWD were appropriately classified, with scores ranging from 0 to 23. We found that 79% of entrained participants were appropriately classified, with scores ranging from −23 to −1.

Of the 1,262 additional women who took the larger survey, 1,119 answered all 8 questions of interest and were included in the model. We found that 410 (37%) of the participants would be referred for further screening for N24HSWD. Among the participants with NPL, we found that 61% (201/331) would be referred for further screening based on their answers to the questionnaire. We found that 27% (209/788) of those with LP would be referred for screening for N24HSWD.

Using the questionnaire data, we simulated the progression of a proposed screening program for N24HSWD using the screening questions, followed by confirmatory urine collections (Figure 2). When we applied our screening criteria to the 116 field study participants who completed all 8 survey questions, we found that 40 of the 116 participants would be referred for further screening according to the model. Seven of these did not have urine sample collections available and therefore could not be included in this evaluation.

Figure 2. Proposed screening process for identifying individuals with non-24-hour sleep-wake rhythm disorder.

Of the remaining 33 participants, the first urine sample was out of the normal range of 1:18–7:06 for 21 participants, including 14 who were classified as having N24HSWD according to gold standard classification, 3 were entrained at a delayed phase, 3 were entrained at an advanced phase, and 1 was normally phased.

Among the 12 who had a first urine sample within the normal range, requiring a second urine sample, 5 participants had a melatonin acrophase out of phase, and of these 2 participants were classified as having N24HSWD and 3 participants who were classified as entrained at a normal phase according to gold-standard criteria. There were 4 participants who had a second urine sample that occurred within the normal range and all of those were classified as being normally entrained. The remaining 3 participants did not have a second urine sample and were excluded from the analysis.

In summary, among the 30 participants who could be evaluated according to Figure 2, 26 patients would have been treated for N24HSWD, 16 (62%) appropriately, but also 10 who did not have the disorder; 6 were entrained at an abnormal circadian phase over this time frame (Advanced- or Delayed Sleep Wake Rhythm Disorder) and 4 were entrained at a normal phase. The remaining 4 patients were correctly identified as having normally phased rhythms. All 16 patients with N24HSWD would have been selected for treatment using this method. Adding a second urine phase assessment for those patients with an initial out-of-phase urine would improve the specificity of the program.

DISCUSSION

In the current study, we have developed the first predictive pre-screening tool for detecting N24HSWD among the visually impaired. By using a bootstrapped forward-selection model with replacement, we were able to create a model that is capable of generating a probability risk score for blind individuals with sleep complaints. Our model has high predictive utility even after adjustment for overfitting. Our final model included eight questions, making it suitable for screening in a clinical setting. Our tool will provide clinicians with a mechanism for pre-screening blind patients in order to guide additional evaluation and treatment decisions.

Non-24-hour sleep-wake rhythm disorder is a debilitating circadian rhythm disorder that is experienced by up to 40% to 60% of totally blind individuals.^1,6,13 Symptoms of N24HSWD occur when the intrinsic circadian rhythm and the 24-hour social rhythm run out of phase (out of “synch”) with each other. Some of the time, their oscillations coincide, for example when the natural circadian rhythms for sleep coincide with the social night, and adverse symptoms are minimal. At other times, the rhythms are in complete antiphase, for example when the biological clock is promoting sleep in the middle of the social day, causing the characteristic disruption to the sleep-wake cycle.

The duration of time it takes to complete the full cycle of the internal clock with respect to the 24-hour day, or “circadian beat cycle” is determined by the intrinsic circadian period. If an individual's clock is 24.5 h, it “drifts” 0.5 h/day and therefore will take 48 days to cycle around the 24-hour clock (and 49 days to complete a full circadian cycle of 24.5 h). If the circadian period is 25.1 h, this beat cycle is much shorter, only 23 days (∼3 weeks); if the circadian period is 24.13 h, the circa-dian beat cycle is 186 days, or ∼6 months. The range of internal circadian periods between individuals therefore determines the range in how many days it will take for their symptoms to cycle “around the clock,” i.e., from 3–26 weeks.

The circadian basis of N24HSWD distinguishes it from other sleep-wake disorders, and therefore use of hypnotics and stimulants to address the sleep and sleepiness symptoms, respectively, is not appropriate.²¹ The pattern of sleep disruption experienced by patients with the disorder does not always present as a shift in sleep timing each day. A majority of individuals will attempt to maintain sleep at a socially normal time. As a result, some individuals will produce a sleep pattern with the nocturnal sleep episode expanding and contracting as they move in and out of phase and with the build up and pay-back of homeostatic sleep pressure.²² Due to the pleomorphic variation in patient's sleep timing, a review of sleep history may not reveal a clear cyclic pattern to indicate the presence of N24HSWD.²³ These more subtle cyclic changes are termed “relative coordination” and often require an expert to review. Furthermore, given the high prevalence of general sleep disorders among the blind,^6–8 it may be difficult for clinicians to differentiate this circadian rhythm sleep disorder and non-circadian sleep disorders in this population. Our model therefore provides an easy-to-use screening tool for clinicians to identify which patients are at higher risk of this disorder without having to have a detailed understanding of the different types of presentations of the disorder.

It is possible that the differences that have been observed in the sleep-wake patterns of those with N24HSWD relate to the non-ordinal beta coefficients that we observed for question six in our final model (Tables 2 and 3). For example, an individual who succumbs to the internal drive to shift sleep timing each day may be more likely to report that showing enthusiasm to get things done would be “a very big problem,” given that such individuals would be sleeping often during the social day. In contrast, those who attempt to maintain a 24-hour schedule may find it less difficult to show enthusiasm to get things done and may respond to this question in a way that other individuals experiencing sleep loss or sleep disorders might respond. These differences in the ways in which individuals with the same disorder respond to the competing drives for social interaction and sleep (i.e., staying awake during the day versus sleeping), could account for the variation in reporting how the disorder affects social interactions as we observed in question six.

Our screening model should ideally be used as the first step in a two-step process for screening patients for N24HSWD (Figure 2). We chose a threshold score for screening that maximized the number of participants with N24HSWD in order to capture more people who are at risk for the disorder in screening (i.e., maximized false positives while minimizing false negatives). Consequently, our tool classified 27% of women with LP as at risk for N24HSWD, which is an unlikely prevalence in this group based on previous reports.^6,10 While the second phase of screening uses a noninvasive method and is generally well accepted by patients, such an approach is costly and would be inefficient for screening every visually impaired patient with a sleep complaint, given the high prevalence of non-circadian sleep disorders in the visually impaired community.⁸ As such, our pre-screening tool will allow clinicians to identify which patients are at the highest risk of N24HSWD, prior to confirmatory circadian phase testing. For this reason, we felt it was more important to set the sensitivity and specificity of the tool such that more individuals with the disorder would be referred for further screening.

We collected serial urine samples from participants over 48 h, measured over two or three weeks, to classify patients as entrained or having N24HSWD. Collecting urine at home using this method has been validated repeatedly in field studies of more than 400 blind and visually impaired patients, including patients up to 83 years old, and is therefore an appropriate noninvasive biomarker that can be used to estimate circadian phase and period.^12,14,15,24 This method is the only one that can confirm non-entrained rhythms reliably in an outpatient setting. Once a patient is appropriately diagnosed, daily treatments given at a fixed clock time every 24 hours in order to entrain the circadian clock are available for this disorder, including tasimelteon,¹⁴ a melatonin agonist and the only FDA-and EMA-approved treatment, and pharmaceutical-grade melatonin, which has been shown to be capable of entraining the clock in research studies,^15,25–27 but has not been approved.

Although our findings are robust, our study is not without limitation. We conducted this analysis using a subset of data originally intended to identify breast cancer risk among blind women.¹⁶ As a result we do not include men in our model. Given, however, that sighted females have been shown to have a shorter circadian period than sighted males,²⁸ potentially making non-entrained rhythms more difficult to detect in blind females, our findings are likely to be translatable to men with N24HSWD. We also used bootstrapping to sample the same dataset repeatedly, which is arguably a preferable approach for model building, rather than completing the validation using a split-sample analysis.²⁰ Finally, our survey does not replace actual measurement of circadian phase, but should reduce the burden on both physicians and patients in helping to triage patients appropriately for follow-up testing.

In summary, we used a novel statistical technique to identify predictors associated with the presence of N24HSWD. Our final screening tool is concise, suitable for use in a clinical setting, and highly predictive of N24HSWD. We recommend that this measure be included in screening visually impaired and blind patients with sleep complaints to indicate those who need confirmatory circadian phase measurement.

DISCLOSURE STATEMENT

This was not an industry supported study. This work was supported by a US Department of Defense Breast Cancer Research Program Idea Award (BC030928, #W81XWH-04-1-0553 to SWL). Dr. Flynn-Evans was the recipient of a National Heart, Lung and Blood Institute fellowship in the program of training in Sleep, Circadian and Respiratory Neurobiology at Brigham and Women's Hospital (NHLBI; T32 HL079010). Dr. Flynn-Evans is a consultant for Baby Sleep Science. Dr. Lockley was the principal investigator of three recently completed clinical trials of a melatonin agonist for the treatment of non-24-hour sleep-wake rhythm disorder in the blind, sponsored by Vanda Pharmaceuticals. Inc., and has received an investigator-initiated research grant and two service agreements from Vanda Pharmaceuticals, Inc., related to non-24-hour rhythms in the blind. He has also received minor consulting fees from 15 financial companies related to non-24-hour sleep-wake disorder in the blind and the publicly-available clinical trial results. He has also received honoraria from MediCom Worldwide, Inc., for teaching on a CME course sponsored by Vanda Pharmaceuticals, Inc.; for contributing text about non-24-hour sleep-wake disorder for the National Sleep Foundation and textbook chapters published by Elsevier; and in 2007 received an authorship fees from Servier Inc., for writing a review of circadian rhythm disorders in the blind. Dr. Lockley holds current consulting contracts with Delos Living LLC; Environmental Light Sciences LLC; Focal Point LLC; Headwaters Inc.; Hintsa Performance AG; Pegasus Capital Advisors LP; PlanLED; and Wyle Integrated Science and Engineering. In the past 5 years, he has received consulting fees from Carbon Limiting Technologies Ltd for work conducted with PhotonStar LED and from Naturebright; Thomas Jefferson University. He has received unrestricted equipment gifts from Biological Illuminations LLC; Bionetics Corporation; Philips Lighting; an unrestricted monetary gift to support research from Swinburne University of Technology, Australia; a fellowship gift from Optalert, Pty, Melbourne, Australia; and holds equity in iSLEEP, Pty, Melbourne, Australia. Dr Lockley receives royalties from Oxford University Press; and received honoraria for an article in the Wall Street Journal. Dr. Lockley has received honoraria plus support for travel, accommodation or meals for invited seminars, conference presentations or teaching from American Society for Photobiology; Bassett Research Institute; Brookline Adult Education; Brown University; Emergency Social Services Association Conference; Estee Lauder; Harvard University (CME); MediCom Worldwide, Inc (CME); North East Sleep Society; and Portland General Electric. He has received support for travel, accommodation and/or meals only (no honoraria) for invited seminars, conference presentations or teaching from 8th International Conference on Managing Fatigue; 14th Annual Tennessee Perfusion Conference; American Academy of Sleep Medicine; Bar Harbor Chamber of Commerce; Cantifix; Connecticut Business & Industry Association Health and Safety Conference; Emergency Services Steering Committee; Ferrari; Harvard University; Hintsa Performance AG; Illinois Coalition for Responsible Outdoor Lighting; Illuminating Engineering Society; Lighting Science Group Corp; Massachusetts General Hospital; Midwest Lighting Institute; National Research Council Canada; New England College of Occupational and Environmental Medicine; Ontario Association of Fire Chiefs; Rio Tinto; Sleep HealthCenters; University of Connecticut Health Center; UMass Memorial; University of Manchester; University of Texas Medical Branch; Vanda Pharmaceuticals Inc.; Warwick Medical School; Woolcock Institute of Medical Research; Wyle Integrated Science and Engineering (NASA). Dr. Lockley has completed investigator-initiated research grants from Alcon Inc, and has ongoing investigator-initiated research grants from Biological Illumination LLC, Philips Lighting, and Respironics Inc., and has completed service agreement with Rio Tinto Iron Ore. Dr. Lockley holds a process patent for the use of short-wavelength light for resetting the human circadian pacemaker and improving alertness and performance which is assigned to the Brigham and Women's Hospital per Hospital policy. He has also received minor revenue from a patent on the use of short-wavelength light which is assigned to the University of Surrey. Dr. Lockley has served as a paid expert on behalf of seven public bodies and one union in arbitrations in relation to firefighters (6), EMTs (1) and police (1) on the matter of sleep, circadian rhythms and work hours. Current address for Dr. Flynn-Evans is Fatigue Countermeasures Group, Human Systems Integration Division, NASA Ames Research Center, Moffett Field, CA 94035.

ABBREVIATIONS

aMT6s

6-sulfatoxymelatonin

EMA

European Medicines Agency

FDA

Food and Drug Administration

LP

light perception

NPL

no perception of light

N24HSWD

non-24-hour sleep-wake rhythm disorder

ROC

receiver operating curve

τ

circadian period