Abstract
Study Objective:
At least 15% of the full-time work force is shift workers. Working during the overnight hours, early morning start times, and variable or rotating schedules present a challenge to the circadian system, and these shifts are associated with adverse health and safety consequences. Shift work disorder (SWD), a primary (circadian rhythm) sleep disorder indicated by excessive daytime sleepiness and/or insomnia associated with a shiftwork schedule, is under-recognized by primary care physicians. We sought to develop and validate a questionnaire to screen for high risk of SWD in a shift working population.
Design:
Shift workers completed a 26-item questionnaire and were evaluated by a sleep specialist (physician) who diagnosed them as either positive or negative for SWD. The physician assessment of SWD was guided by a flow chart that operationalized the ICSD-2 criteria for SWD.
Setting:
18 sleep clinics in the USA.
Patients or Participants:
311 shift workers.
Interventions:
Not applicable.
Measurements and Results:
Responses to the items in the questionnaire were entered into a series of discrimination function analyses to determine the diagnostic value of the items and the fewest number of questions with the best predictive value. The function was then cross-validated. A final 4-item questionnaire has 89% positive predictive value and 62% negative predictive value (sensitivity = 0.74; specificity = 0.82).
Conclusions:
This Shiftwork Disorder Screening Questionnaire may be appropriate for use in primary care settings to aid in the diagnosis of SWD.
Citation:
Barger LK; Ogeil RP; Drake CL; O'Brien CS; Ng KT; Rajaratnam SMW. Validation of a questionnaire to screen for shift work disorder. SLEEP 2012;35(12):1693–1703.
Keywords: Sleep, nightshift, circadian rhythm, sleepiness, insomnia
INTRODUCTION
Shiftwork, including extended duration shifts and other variable and non-standard hours, comprises approximately 15% of the full-time workforce in the United States,1 almost 23% of the workforce in Japan,2 16% in Australia,3 18 % in the United Kingdom, and 13% in France.4 In the United States,1 approximately 8 million people regularly work overnight hours,5 and approximately 20 million people are estimated to have unusually early work start times (between 02:30 and 07:00).6
Working at night or during unusual hours presents specific physiological challenges to sleep-wake and alertness rhythms. Shift workers' sleep-wake schedules are often out of phase with their endogenous circadian rhythms.7,8 Most workers on permanent night shift do not fully adapt to the shifted sleep-wake schedule required for their work.9–13
Night workers, in particular, are highly prone to vehicular accidents.14,15 Decreased alertness, cognitive ability, and vigilance,16,17 which are likely underlying causes of vehicular accidents, also lead to a substantially higher rate of injuries, industrial accidents, and quality-control errors on the job,18 as well as a general decline in work capacity19 and a higher rate of reported actual and near-miss injuries.20
Shift work is also associated with a number of adverse health outcomes, including cardiovascular disease, diabetes, gastrointestinal problems, mood disturbances, and cancer.21 Night shift workers are reported to have higher body mass index,22,23 elevated cholesterol,22 and elevated triglyceride levels22 when compared to day workers. Gastrointestinal problems include an increased rate of peptic ulcers in night and rotating shift workers compared with day workers,24 and the risk of developing cardiovascular disease was increased by 40% in shift workers.25 Shiftwork has also been associated with increased rates of depression26 and is thought to intensify existing mood disorders.21 Finally, nurses who had performed shift work for many years were significantly more likely to develop breast27,28 and colon cancers.29 In fact, the International Agency for Research on Cancer, a part of the World Health Organization, concluded that shift work is probably carcinogenic to humans.30
The most commonly reported symptoms of shift work are sleep disturbance and excessive sleepiness. There is, however, a wide range in the severity of these symptoms. Some shift workers have extreme difficulty maintaining optimal sleep-wake functioning while performing a shift-work schedule, even while employing appropriate countermeasures, and should be considered for a diagnosis of shift work disorder (SWD).
SWD is a primary sleep disorder, in the category of Circadian Rhythm Sleep Disorders.31 It is characterized by excessive sleepiness and/or insomnia temporally associated with the shift schedule. Although the SWD diagnostic criteria have been in place for more than 20 years, the prevalence of the disorder has only recently been defined. Drake and colleagues have suggested that the prevalence of SWD is at least 10% of those working night and rotating shifts.24
SWD has been poorly diagnosed and most likely under-recognized in primary care settings partly because of lack of standardized screening tools.21,32 Furthermore, individual vulnerability to the adverse health and safety consequences of shift work is not well understood, in part due to the lack of an appropriate screening instrument that identifies those with SWD. The aim in the present study was to develop a questionnaire to assess SWD in a shift working population, suitable for use in clinical practice and research, and validated against clinical diagnosis by sleep specialists in accredited sleep clinics.
METHODS
Participants
Shift workers (n = 311) were recruited to complete the screening questionnaire and to be evaluated by a physician in an accredited sleep clinic. In order to maximize the ability to capture those with and without SWD, we recruited participants in two ways. First, shift workers were recruited from the general population via internet advertising (R; n = 156). Second, flyers at sleep clinics solicited shift workers who were at the clinic for previously scheduled appointments (W; n = 155).
Individuals were eligible to participate in the study if they were aged 18 to 65 years, were not currently being treated by a physician at a sleep clinic for shift work disorder, and, in the past month, worked a non-standard shift schedule (started before 07:00 or after 14:00, rotated, or regularly included hours outside of the standard 07:00 to 18:00 work day). The study protocol was approved by the Partners Human Research Committee. Focus groups were conducted in December 2009, and physician diagnoses on clinic visits were completed from March to October 2010 (spring, summer, and fall).
Questionnaire Development
A previous study assessing the risk of sleep disorders in 4,957 North American police officers33 provided an opportunity to develop a preliminary version of a SWD questionnaire based on The International Classification of Sleep Disorders, Second Edition (ICSD-2) criteria.31 A broad range of questionnaire items for the current project were initially generated and compiled from the police study.
A factor analysis reduced the 37 questions regarding insomnia and excessive sleepiness on the police-focused version of the survey to 24 items that best predicted the final outcome. The 24 items were presented in 10 individual interviews of shift workers to determine the most appropriate questions for discriminating between individuals with and without SWD.34 Shift workers who reported working full-time, aged 18 to 65, were recruited from the general population for these interviews. To be considered shift workers, work had to be scheduled during the habitual hours of sleep, including night shifts, evening and early morning shifts, and rotating shifts,31 and they were required to meet one of the following criteria: (a) begin work between 14:00 and 17:59; (b) begin an 8- to 10-hour shift between 19:00 and 04:00; (c) have frequently rotating shifts; or (d) start work between 04:00 and 07:00. They were required to work these shifts an average of 5 times per month for ≥ 3 months. During the interviews (approximately 60 min), shift workers were asked open-ended questions about their job, the impact of shift work on their life, and their interpretation of the survey questions. Questions were revised based on feedback from shift workers and on the ability of the shift workers to interpret the questions. We also ensured that the items in the questionnaire included all symptoms associated with SWD, as self-reported by the shift working sample.
The final questionnaire consisted of 26 items assessing the following: demographics and work schedule details, insomnia while working non-standard shifts, excessive sleepiness while working nonstandard shifts, and insomnia and excessive sleepiness while on a break (i.e., at least 1 week in the last year) from nonstandard shifts (e.g., vacation or standard day shifts). Non-standard shifts were defined on the questionnaire as those that start before 07:00 or after 14:00, rotate, or regularly include hours outside of the standard 07:00 to 18:00 work day. Response categories for all items except demographics were on a 4 or 5 point Likert scale (see Supplemental Material, Figure S1).
Diagnostic Process
We also conducted 2 focus groups consisting of board-certified sleep specialists (n = 8; 4 per group) to help us better understand how physicians interpret the ISCD-2 criteria and, in practical terms, how they make the diagnosis of SWD. The focus group physicians gave us feedback on a flow chart we developed; the flow chart did not provide any additional guidance outside ICSD-2 criteria. To clarify the diagnosis process, we provided each clinic with a flow chart that operationalized the ICSD-2 criteria for SWD (Figure 1). The flow chart was used to standardize among clinics across the study the clinical evaluation and diagnosis of the disorder. Furthermore, the evaluating physicians did not see the questionnaire that volunteers completed.
Sleep Clinics
The study was conducted across 18 sleep clinics in the United States. All clinics were certified by the American College of Sleep Medicine. The clinics were located in California (Sacramento), Florida (Altamonte Springs, Orlando), Georgia (Macon), Illinois (Chicago, Maryland (Catonsville, Frederick, Westminster), Massachusetts (Brighton, Framingham, Worcester), Pennsylvania (Darby, Lafayette Hill), South Carolina (Charleston, Columbia), Texas (Waco), and Virginia (Portsmouth).
Protocol
Participants completed the questionnaire prior to their appointment with a sleep physician, either at home (R) or in the sleep clinic waiting room (W). Each participant then met individually with a sleep physician who performed a routine clinical evaluation and diagnosis of SWD according to ICSD-2, without access to participant questionnaire data. The diagnosis flow chart was provided to each clinic (Figure 1). Physicians were required to report diagnosis according to the likelihood of SWD (definite, probable, possible, or not likely) and whether a comorbid sleep disorder was present (definite, probable, possible, or not likely), as well as to identify the comorbid sleep disorder(s).
Analysis
The responses to the items in the questionnaire were entered into a series of discrimination function analyses (DFAs) to determine the diagnostic value of the questionnaire items taken together and to identify the smallest number of items contributing significantly to the discriminant function(s). DFA is an analysis of choice for identifying predictors of categorical criterion variables where predictors may be considered non-categorical. It generates a “discrimination function” which may be used to classify new cases into the various criterion categories. However, the predictive value of variables identified by DFA may be inflated by specific characteristics of the validating sample. Use of the function in a new sample may not yield as effective a discrimination as suggested by the original analysis. For this reason, it is essential to cross-validate the findings from the initial DFA with a new sample to determine the likely “true” effectiveness of the function.
For the purposes of cross-validation, the sample was randomly divided into 2 subgroups: an analysis sample (75%) and a cross-validation sample (25%). The random selection was conducted separately for the participants diagnosed as definitely or probably having SWD and those classified as not likely or definitely not having SWD. The final composition and size of samples used in any given analysis was affected by the presence of participants with missing data on one or more questions. T-tests, χ2, and Fisher exact tests were conducted to compare demographics between the analysis and cross validation samples.
To establish the discriminant values of individual items within the questionnaire, t-tests were conducted comparing the individuals in the analysis sample in binary SWD risk groups (type I error rate set at a Bonferroni adjusted α = 0.002). The function was then cross-validated with the cross-validation sample and its adequacy determined by Press's Q test. Receiver operating characteristic (ROC) curves were established for the final questionnaire to determine its sensitivity and specificity. Statistical analysis was conducted with SPSS version 18.0 (SPSS Inc., Chicago, IL, USA).
RESULTS
Sample Characteristics and Clinical Evaluations
A total of 311 (males = 166, females = 124, unknown = 21) shift workers completed the study. Mean (+ SD) age was 41.5 + 12.1 years: 156 shift workers were recruited from the general population (R), and 155 were recruited at previously scheduled sleep clinic appointments (W). The majority of participants started working shifts more than one year ago (82%), with almost half of those (39%) having worked shifts > 5 years. Most participants worked non-standard shifts 5 times per week (37%), 24% worked non-standard shifts 4 times per week, and 15% worked them 3 times per week (see Table 1). Of the sample, 40% reported typically working night shift (start work 19:00–03:59), 30% reported typically working evening shift (start work 14:00–18:59), 15% typically reported working early morning shifts (start work 04:00–06:59), and the remainder were unknown or rotating.
Table 1.
Participants assigned to the cross-validation group (n = 79) were not significantly different from the analysis group (n = 232) in mean age, number of years working shifts, mean number of shifts worked per week, or SWD diagnosis (P > 0.05).
Clinical evaluations were performed in 18 clinics in 17 cities (number of evaluations, percentage): Altamonte Springs, FL (24, 8%); Brighton, MA (24, 8%); Catonsville, MD (5, 2%); Charleston, SC (11, 4%); Chicago, IL (44, 14%); Columbia, SC (33, 11%); Darby, PA (4, 1%); Framingham, MA (8, 3%); Frederick, MD (7, 2%); Lafayette Hill, PA (43, 14%); Macon, GA (3, 1%); Orlando, FL (12, 4%); Portsmouth, VA (20, 6%); Sacramento, CA (26, 8%); Waco, TX (37, 12%); Westminster, MD (3, 1%), Worcester, MA (7, 2%). The physician-assigned SWD diagnosis categories of the 311 participants are shown in Table 1.
After randomly dividing the subjects, the analysis sample consisted of 232 participants (153 diagnosed as likely or definitely with SWD and 79 diagnosed as not likely or definitely not with SWD). The cross-validation sample of 79 consisted of 52 diagnosed as likely or definitely with SWD and 27 diagnosed as not likely or definitely not with SWD (see Table 1).
Questions Included in Final Discriminant Function Analyses
Of the original 26 items in the questionnaire (see Supplemental Materials, Figure S1), only 16 were used in subsequent discriminant function analyses. Questions 1 and 4 were demographic and not appropriate as discriminators. Questions 19–25 were applicable only to participants who had at least one week's break from non-standard shift work in the previous year, as indicated by a positive response to Question 18. Only 179 of the total of 232 participants in the analysis sample answered Question 18 in the positive. The remainder either did not take at least one week's holiday in the past year or did not answer the question. A preliminary stepwise discriminant function analysis with the 179 participants, with all questions included, yielded a significant function (Wilks λ = 0.662, χ2 = 72.44, P < 0.001) discriminating the low and high risk groups, with questions 11, 13, 15, and 26, but not questions 19–25, contributing significantly to the function. Questions 19–25 were, therefore, excluded from subsequent discrimination function analysis to avoid further reduction in sample sizes.
Standard Discriminant Function for Predicting Membership of the Four Diagnostic Categories
Standard multiple discriminant function analysis of the analysis sample yielded one significant discriminant function (Wilks' λ = 0.450, χ2 (72) = 131.927, P = 0.001). The function only yielded 52.2% overall correct classification. While the overall correct classification was poor, the primary sources of errors of classification were in misclassifications between the “definitely” and “probably” subcategories for those diagnosed as “positive” for SWD and between the “not likely” and “definitely not” subcategories for those considered negative for SWD. This suggested that a more effective approach would be to combine the subcategories to form 2 clinical evaluation categories.
Differences between the Two Diagnostic Groups on Individual Questions
Individuals who were diagnosed as either definitely or likely to be suffering from SWD were more inclined to experience insufficient sleep, excessive sleepiness, and a tendency to doze off at work while working non-standard shifts (Table 2). The participants tended to have problems falling asleep and staying asleep, as well as waking up too early. Their overall quality of sleep tended to be unsatisfactory. They had a reduced sense of well-being while they were awake and experienced decreased physical and mental functioning. The positive group tended to doze off while commuting (not driving).
Table 2.
Therefore, only Questions 7 to 15, individually significantly discriminated between the 2 populations. Nonetheless, it is possible that some of the remaining questions may operate as suppressor (moderating) variables in a discriminant function analysis and were, therefore, included in all subsequent discriminant analyses.
Discriminant Function for Predicting Membership of the Two Major Diagnostic Categories: Standard Discriminant Function Analysis
Standard discriminant function analysis of the 185 participants in the analysis sample that had complete data yielded a significant discriminant function (Wilks' λ = 0.646; χ2 = 76.13; df = 17; P < 0.001), with an overall correct classification of 75.1%. This was a considerable improvement on the overall correct classification from the function seeking to discriminate the original 4 diagnostic categories. The percentage of false positives (15.9% falsely classified as positive for SWD) was less than false negatives (29.5%; falsely classified as negative for SWD). The receiver operating characteristics for this function were: sensitivity = 0.71; specificity = 0.84; positive likelihood ratio = 4.44; negative likelihood ratio = 0.35; positive predictive value = 90%; and negative predictive value = 60%. The results suggest that the full function was reasonably sensitive, with the probability of an individual having SWD when he/she was classified as having SWD at 0.90. On the other hand, the probability that an individual did not have SWD when he/she was classified as nothaving SWD was only 0.60, reflecting a higher risk of false negatives.
Stepwise Discriminant Function Analysis
To determine the minimum number of predictors required to significantly discriminate between the diagnostic categories, a stepwise regression analysis was carried out. This resulted in a significant function (Wilks' λ = 0.681; χ2 = 67.65; df = 4, P < 0.001), with positive loadings from questions 11, 13, 15, and 26, with question 26 being a suppressor variable, and yielding 76.4% correct classification (see Table 3).
Table 3.
The reduced function with only 4 variables performed as well as the full function with 16 variables. The receiver operating characteristics for this function were: sensitivity = 0.74; specificity = 0.82; positive likelihood ratio = 4.11; negative likelihood ratio = 0.32; positive predictive value = 89%; and negative predictive value = 62%. Again, the function yielded higher false negatives. The probability that the participant was diagnosed as NOT suffering SWD (Negative) when the function predicted that he/she was not suffering from SWD (low risk) was 0.62. In contrast, where the function predicted that the participant was suffering from SWD (high risk), the probability that he was diagnosed as suffering from SWD (Positive) was 0.89.
Final Discriminant Function
To cross-validate the final discriminant function, a standard discriminant function analysis was first carried out with only the 4 variables extracted in the stepwise analysis. The resulting function (Wilks' λ = 0.70; χ2 = 66.75; df = 4, P < 0.001) yielded the same proportion of cases correctly identified identical to that generated in the stepwise analysis.
Fisher's classification coefficients and the structure matrix for the discriminant function with 4 variables provide the correlation between each question and the function are given in Table 4. The strongest contributor to the function was Question 11. Participants who scored high on the function tended to wake up too early and not be able to go back to sleep when on non-standard shifts. They tended to have a poor sense of well-being (Question 13) and to doze off at work (Question 15). Finally, the negative coefficient for Question 26 in the structure matrix might indicate that participants scoring high on this question were at a lower risk of suffering from SWD. However, χ2 (Table 2) shows that Question 26, on its own, did not discriminate the groups, with a P value of 0.95. This suggests that Question 26 entered the equation as a suppressor variable, possibly for Question 15 with which it has a significant correlation of 0.25 (P = 0.001; see Supplemental Material, Table S1). Of some interest is the finding that Question 3, asking participants when they started working non-standard shifts, did not on its own or in combination with other variables, contribute to discrimination between the diagnostic groups, suggesting that extent of shift-work exposure may not be a major factor in the development of SWD.
Table 4.
Cross-Validation
The function was cross-validated with 61 participants from the cross-validation sample with complete data (see Table 5). The resulting function correctly classified 63.9% of cases, resulting in shrinkage of 12.5%. However, the resulting classification was still better than chance, with a Press's Q test yielding a χ2 = 4.714; df = 1; P < 0.001. Thus, the discriminant function with just 4 variables provided a significantly better than chance agreement with clinical diagnosis of SWD. The following are the receiver operating characteristics for the cross-validation classification: sensitivity = 0.68; specificity = 0.57; positive likelihood ratio = 1.59; negative likelihood ratio = 0.56; positive predictive value = 75%; and negative predictive value = 48%.
Table 5.
Factors Underlying the Discriminators
Although stepwise discriminant analysis indicates that only 4 questions were sufficient to generate a function that discriminated those diagnosed as positive for SWD from those that were diagnosed as negative for SWD, a number of other questions individually significantly discriminated between the diagnostic groups (see Table 2). The likely reason that these questions were not included in the reduced function is that they shared common relevant variance with the questions included. The matrix of significant correlations is given in the Supplemental Material, Table S1.
The correlation matrix was subjected to a principal components analysis with varimax rotation to determine the component structure underlying these correlations (see Supplemental Material, Table S1). In the rotated solution, there were 6 components with eigenvalues > 1.0. The Scree test suggests one major component accounting for 28.62% of the variance (Figure 2, Supplemental Material, Table S2).
Component 1 appears to be a factor reflecting sleep disturbance and associated sense of mental and physical well-being, with high loadings (correlations between each question and the component greater than 0.40) from Questions 7–16. It is reasonable to assume that any predictive value of these questions, other than 11, 13, and 15, are represented by these 3 questions in the discriminant function.
The 4 questions determined by the discriminant function to make up the final questionnaire are shown in Supplemental Materials (Figure S2). The questions are weighted by their discriminant function classification coefficients. To assess an individual's risk of SWD, the number associated with the response to each question is multiplied with the classification function coefficients listed in Table 4 and the constant added, for each column. The resulting score in the High Risk of SWD column are then compared with the resulting score in the Low Risk of SWD column. The higher of the two column scores indicates the individual's risk. For example, if the high risk score is higher than the low risk score, then the individual is at risk for SWD. If the low risk score is higher, then the individual is not considered at risk for SWD. This function can be easily programmed into a standard spreadsheet program.
Insomnia versus Excessive Sleepiness Symptoms
In those clinically diagnosed with SWD, 53.2% of participants responded to the question on the final 4-question survey indicating that they had a considerable or serious problem with waking up too early and not being able to get back to sleep while working non-standard shifts, indicating sleep maintenance insomnia. On the final 4-question survey, 35.6 % of participants responded to the question that they had a moderate chance of or were highly likely to doze off at work during a non-standard shift, indicating excessive sleepiness; 23.4% of participants indicated both insomnia and excessive sleepiness symptoms.
Comorbid Disorders
Of the total participants who were diagnosed as positive for SWD, 56% (115/205) were identified as “definitely” or “probably” having another (comorbid) sleep disorder, with 68% (78/115) of the comorbidities identified as sleep apnea. Shift workers that were recruited from the population (R) were less likely to be diagnosed with SWD than those who visited the sleep clinic on their own (W) (χ2 = 9.42, P = 0.0021). Those who visited the sleep clinic on their own (W) were more likely to be identified as having a comorbid sleep disorder compared to shift workers recruited from the general population (χ2 = 5.54, P = 0.019). Comorbidities included conditions other than sleep disorders such as hypothyroidism, overweight and obesity, anxiety, and depression.
DISCUSSION
We have developed and validated the first questionnaire screening instrument for SWD. The four-item questionnaire correctly identified 76.4% of cases diagnosed to have SWD by a sleep physician, with 89% positive predictive value and 62% negative predictive value (sensitivity = 0.74; specificity = 0.82). This level of accuracy is comparable to other sleep disorders screening tools35,36 and standard screening tools to identify risk of psychiatric conditions.37,38
Consistent with the ICSD2 criteria for SWD, we found the reduced set of four questions assessed both of the hallmark symptoms of the disorder. One question relates to an insomnia complaint characteristic of sleeping at an adverse circadian phase.39 Two questions relate to daytime excessive sleepiness and impaired well-being. The final question relates to the occurrence of sleepiness following days off from work, isolating the symptoms of SWD to those that are temporally associated with the non-standard work schedule.
As sleep disturbances are common in shift workers, the diagnosis of more serious cases that meet criteria for SWD may be challenging. Moreover, the primary symptoms of SWD, insomnia, and excessive sleepiness, are also associated with many other maladies and sleep disorders.21 The minimal criteria for SWD includes a primary symptom of either insomnia or excessive sleepiness that is temporally associated with the work schedule.31 In screening shift workers for sleep disorders, it is essential and often challenging to distinguish between those symptoms associated with or independent of their shift work schedule. In a survey of 673 healthcare professionals, they indicated two-thirds of the cases of SWD are missed or undiagnosed.40 Therefore, this questionnaire to identify those at high risk for SWD will be very beneficial.
Shift workers with SWD experience persistent insomnia and/or excessive sleepiness.41 The disorder negatively affects almost every aspect of the shift worker's work and home life42 and is associated with neurological changes.43 A recent literature review of individual differences indicated that younger shift workers, males, and those that score low on morningness may be more tolerant to shift work.44 Other factors that may increase a shift worker's susceptibility to SWD include genetically based vulnerability to insomnia45–47 and genetic polymorphisms such as that of the Per3 gene that influence sensitivity to sleep loss.48 The length of a shift worker's intrinsic period may also influence the ability to adapt to shift work, with periods longer or shorter than 24 hours significantly decreasing tolerance.49 Until these vulnerabilities are more fully explored, and their association with SWD is determined, our newly validated questionnaire provides a useful method of identifying high risk SWD cases based on self-reported sleep-wake symptoms.
Limitations
In drawing conclusions about the utility of the final four-question screening tool, several caveats should be noted. Although we piloted these questions with almost 5,000 police officers and conducted individual interviews with shift workers to develop the questions for general shift workers, the discriminant functions generated are limited to the questions we chose to include. Additionally, the validity of the function for discriminating between the diagnostic classes is limited by the accuracy of the clinical diagnoses made by physicians; the latter leading to misclassifications that may be due to misdiagnoses of cases. To attempt to control for the variation among physicians and standardize the diagnosis, we limited our study to only sleep specialists at accredited sleep clinics and provided an ICSD-2 criteria-based diagnosis flowchart. To increase the generalizability of the study, we used 18 clinics in multiple geographic regions of the United States. We cannot exclude the fact that participants having just answered a questionnaire addressing symptoms of excessive sleepiness and insomnia might have affected their interaction with the physician.
Because SWD is a symptom-based diagnosis, we cannot totally exclude the issue of circularity in our validation process, as each clinic was provided with a flow chart that operationalized the ICSD-2 symptomatic criteria for SWD. No objective diagnostic test is available for SWD; hence clinical evaluation according to the specified diagnostic criteria remains the current approach in clinical practice. Circularity, however, does not appear to have been a significant factor in our study. For example, item 8 on the screening tool is a question regarding excessive sleepiness, one of the diagnostic criteria for SWD, and has the fifth highest loading on the major component in the Principal Component Analysis (Supplemental Material, Table S1), but it does not appear as a predictor in the discrimination function analysis. We speculate that this is because excessive sleepiness alone is not a major contributor to distinguishing SWD from other sleep disorders—that is, its unique contribution to SWD is not substantial, and its contribution (if any) is represented by other variables with which it correlated, possibly item15 (r = 0.55) or item 13 (r = 0.46). Therefore, if circularity was substantially affecting the results, one would expect that the outcome of a clinician querying a patient about excessive sleepiness and a questionnaire item on this symptom would be highly interrelated and hence become a part of the final questionnaire. The outcome of the analysis we conducted demonstrates that other unique items on the questionnaire more accurately predicted SWD.
Although we attempted to obtain an approximately equal distribution of those diagnosed with and without SWD, positive diagnosis was overrepresented in our sample. Physicians participating in the study may have had a greater tendency towards a positive diagnosis of SWD given their knowledge of the overall aims of the study. The prevalence of SWD in our sample of shift workers (65.9% diagnosed by a clinician as either definitely or probably having SWD) may have also been influenced by the broad range of the non-standard shift types represented in our study sample. Nevertheless, we used well-established methods and statistical techniques to analyze the data, including cross validation. We were, however, limited in the number of participants that could be used to cross-validate the results.
Furthermore, almost one-quarter of our sample did not report having at least one week break from working non-standard shifts (e.g., one week of vacation or one week of standard daytime shifts). This limited our ability to distinguish between excessive sleepiness and/or insomnia symptoms associated with a shift working schedule and the persistence or absence of those symptoms when shift work is not occurring. The lack of a break (vacation, etc.) may be characteristic of some types of shift workers and thus makes a definitive clinical diagnosis of SWD more difficult in those individuals.
The sensitivity and specificity of the instrument in different settings (e.g., primary care clinic versus sleep clinic) remain to be investigated. Future studies should also consider validating the survey against physiological measures. Nocturnal multiple sleep latency tests and daytime polysomnography have been used in past research for study entry criteria and to assess severity.50 It is strongly advised that the results of the classification, at least for the cross-validation sample, be validated against these objective physiological indices of the primary symptoms of SWD.
This Shiftwork Disorder Screening Questionnaire may be appropriate for use in primary care and sleep clinic settings to aid in the diagnosis of SWD. The tool may also be used in research studies to assess the risk of adverse health, safety, and performance outcomes associated with SWD and in the development of appropriate interventions.
ACKNOWLEDGMENTS
This work was supported by research funding from Cephalon, Inc. following submission of a research proposal by the investigators. Cephalon (now Teva Pharmaceutical Industries Ltd.) markets medications that are approved by the U.S., Food and Drug Administration for the treatment of shift work disorder We thank Michael Shreeve for study coordination, Jason Sullivan for technical assistance, and Dr. Richard Bogan and the staff at SleepMed, especially Manley Finch and Allen Boone. We also thank the participating clinics, the sleep physicians and the shift workers who volunteered.
Study data were managed using REDCap electronic data capture tools hosted at Brigham and Women's Hospital. REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing: 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.51
Footnotes
A commentary on this article appears in this issue on page 1591.
DISCLOSURE STATEMENT
This study was supported by Cephalon Inc. Dr. Barger has received a research grant for this study from Cephalon, Inc. Dr. Ogeil has received a research grant for this study from Cephalon, Inc. Dr. Drake has received research grants from Cephalon, Inc. and Zeo, Inc. He has been paid by Cephalon, Inc. and Rockpointe for speaking engagements, has received honoraria from Cephalon, Inc. and Rockpointe, and equipment from Zeo, Inc. Mr. O'Brien has received a research grant for this study from Cephalon, Inc. Dr Rajaratnam reports that he has served as a consultant to Vanda Pharmaceuticals, Philips Respironics, EdanSafe, The Australian Workers' Union, and National Transport Commission, and has received research grants and/or unrestricted educational grants from Vanda Pharmaceuticals, Takeda Pharmaceuticals North America, Philips Lighting, Philips Respironics, Cephalon, and ResMed Foundation, and reimbursements for conference travel expenses from Vanda Pharmaceuticals. His institution has received equipment donations or other support from Optalert, Compumedics, and Tyco Healthcare. He has also served as an expert witness and/or consultant to shift work organizations. Dr Rajaratnam presently serves on the Board of Directors of the Australasian Sleep Association, and has previously served on the Board of Directors of the Sleep Health Foundation.
SUPPLEMENTAL MATERIAL
REFERENCES
- 1.A time to work: Recent trends in shift work and flexible schedules. Mon Labor Rev. 2007;130:3–15. [Google Scholar]
- 2.Dochi M, Suwazono y, Sakata K, et al. Shift work is a risk factor for increased total cholesterol level: a 14-year prospective study in 6886 male workers. Occup Environ Med. 2009;66:592–7. doi: 10.1136/oem.2008.042176. [DOI] [PubMed] [Google Scholar]
- 3.Australian Bureau of Statistics. Australia: 2009. Working time arrangements. [Google Scholar]
- 4.Wedderburn A, editor. Bulletin of European Studies on Time. Vol. 9. Edinburgh: European Foundation for the Improvement of Living and Working Conditions; 1996. Statistics and news; pp. 1–72. [Google Scholar]
- 5.Mellor EF. Mon Labor Rev. 1986. Shift work and flexitime: how prevalent are they? pp. 14–21. [Google Scholar]
- 6.Bureau of Labor Statistics. Workers on Flexible and Shift Schedules in May 2004. [Accessed 5 Nov 2008]; [Google Scholar]
- 7.Richardson GS, Malin HV. Circadian rhythm sleep disorders: Pathophysiology and treatment. J Clin Neurophysiol. 1996;13:17–31. doi: 10.1097/00004691-199601000-00003. [DOI] [PubMed] [Google Scholar]
- 8.Eastman CI, Liu L, Fogg LF. Circadian rhythm adaptation to simulated night shift work: Effect of nocturnal bright-light duration. Sleep. 1995;18:399–407. doi: 10.1093/sleep/18.6.399. [DOI] [PubMed] [Google Scholar]
- 9.Weibel L, Spiegel K, Follenius M, Ehrhart J, Brandenberger G. Internal dissociation of the circadian markers of the cortisol rhythm in night workers. Am J Physiol. 1996;270:E608–13. doi: 10.1152/ajpendo.1996.270.4.E608. [DOI] [PubMed] [Google Scholar]
- 10.L, Follenius M, Spiegel K, Gronfier C, Brandenberger G. Growth hormone secretion in night workers. Chronobiol Int. 1997;14:49–60. doi: 10.3109/07420529709040541. [DOI] [PubMed] [Google Scholar]
- 11.Sack RL, Blood ML, Lewy AJ. Melatonin rhythms in night shift workers. Sleep. 1992;15:434–41. doi: 10.1093/sleep/15.5.434. [DOI] [PubMed] [Google Scholar]
- 12.Quera-Salva MA, Defrance R, Claustrat B, De Lattre J, Guilleminault C. Rapid shift in sleep time and acrophase of melatonin secretion in short shift work schedule. Sleep. 1996;19:539–43. [PubMed] [Google Scholar]
- 13.Roden M, Koller M, Pirich K, Vierhapper H, Walhauser F. The circadian melatonin and cortisol secretion pattern in permanent night shift workers. Am J Physiol. 1993;34:R261–7. doi: 10.1152/ajpregu.1993.265.1.R261. [DOI] [PubMed] [Google Scholar]
- 14.Folkard S. Black times: Temporal determinants of transport safety. Accid Anal Prev. 1997;29:417–30. doi: 10.1016/s0001-4575(97)00021-3. [DOI] [PubMed] [Google Scholar]
- 15.Harris W. Fatigue, circadian rhythm, and truck accidents. In: Mackie R, editor. Vigilance theory, operational performance, and physiological correlates. New York: Plenum; 1977. pp. 133–46. [Google Scholar]
- 16.Czeisler CA, Johnson MP, Duffy JF, Brown EN, Ronda JM, Kronauer RE. Exposure to bright light and darkness to treat physiologic maladaptation to night work. N Engl J Med. 1990;322:1253–9. doi: 10.1056/NEJM199005033221801. [DOI] [PubMed] [Google Scholar]
- 17.Cajochen C, Khalsa SBS, Wyatt JK, Czeisler CA, Dijk DJ. EEG and ocular correlates of circadian melatonin phase and human performance decrements during sleep loss. Am J Physiol. 1999;277:R640–9. doi: 10.1152/ajpregu.1999.277.3.r640. [DOI] [PubMed] [Google Scholar]
- 18.Weiner E. Vigilance and inspection. In: Warm J, editor. Sustained attention in human performance. New York: J Wiley – Sons; 1984. pp. 207–46. [Google Scholar]
- 19.Rosa RR, Bonnet MH, Bootzin RR, et al. Intervention factors for promoting adjustment to nightwork and shiftwork. Occup Med. 1990;5:391–415. [PubMed] [Google Scholar]
- 20.Swanson LM, Drake CL, Arnedt JT. Drowsy driving: A survey of American workers. Behavioral Sleep Medicine. 2011 doi: 10.1080/15402002.2011.624231. In press. [DOI] [PubMed] [Google Scholar]
- 21.Culpepper L. The social and economic burden of shift-work disorder. J Fam Pract. 2010;59:S3–S12. [PubMed] [Google Scholar]
- 22.Biggi N, Consonni D, Galluzzo V, Sogliani M, Costa G. Metabolic syndrome in permanent night workers. Chronobiol Int. 2008;25:443–54. doi: 10.1080/07420520802114193. [DOI] [PubMed] [Google Scholar]
- 23.DiLorenzo L, DePergola G, L'Abbate N, et al. Effect of shift work on body mass index: results of a study performed in 319 glucose-tolerant men working in a Southern Italian industry. Int J Obes Relat Metab Disord. 2003;27:1353–8. doi: 10.1038/sj.ijo.0802419. [DOI] [PubMed] [Google Scholar]
- 24.Drake CL, Roehrs T, Richardson G, Walsh JK, Roth T. Shift work sleep disorder: prevalence and consequences beyond that of symptomatic day workers. Sleep. 2004;27:1453–62. doi: 10.1093/sleep/27.8.1453. [DOI] [PubMed] [Google Scholar]
- 25.Boggild H, Knutsson A. Shift work, risk factors and cardiovascular disease. Scand J Work Environ Health. 1999;25:85–99. doi: 10.5271/sjweh.410. [DOI] [PubMed] [Google Scholar]
- 26.Scott A, Monk TH, Brink L. Shiftwork as a risk factor for depression: a pilot study. Int J Occup Environ Health. 1997;2(Suppl 2):s2–s9. [PubMed] [Google Scholar]
- 27.Davis S, Mirick DK, Stevens RG. Night shift work, light at night, and risk of breast cancer. J Natl Cancer Inst. 2001;93:1557–62. doi: 10.1093/jnci/93.20.1557. [DOI] [PubMed] [Google Scholar]
- 28.Schernhammer ES, Kroenke C, Laden F, Hankinson SE. Night work and risk of breast cancer. Epidemiology. 2006;17:108–11. doi: 10.1097/01.ede.0000190539.03500.c1. [DOI] [PubMed] [Google Scholar]
- 29.Schernhammer ES, Laden F, Speizer FE, et al. Shift work and risk of colorectal cancer in the Nurses' Health Study. J Natl Cancer Inst. 2003;95:825–8. doi: 10.1093/jnci/95.11.825. [DOI] [PubMed] [Google Scholar]
- 30.Straif K, Baan R, Grosse Y, et al. Carcinogenicity of shift-work, painting and fire-fighting. Lancet Oncol. 2007;8:1065–6. doi: 10.1016/S1470-2045(07)70373-X. [DOI] [PubMed] [Google Scholar]
- 31.American Academy of Sleep Medicine. 2nd ed. Westchester, IL: American Academy of Sleep Medicine; 2005. The international classification of sleep disorders; diagnostic and coding manual. [Google Scholar]
- 32.Schwartz JRL. Recognition of shift-work disorder in primary care. J Fam Pract. 2010;59:S18–23. [PubMed] [Google Scholar]
- 33.Rajaratnam SMW, Barger LK, Lockley SW, et al. Sleep disorders, health and safety in police officers. JAMA. 2011;306:2567–78. doi: 10.1001/jama.2011.1851. [DOI] [PubMed] [Google Scholar]
- 34.U.S. Department of Health and Human Services Food and Drug Administration. Washington, DC: 2006. Patient-reported outcome measures: use in medical product development to support labeling claims. [Google Scholar]
- 35.Soldatos CR, Dikeos DG, Paparrigopoulos TJ. The diagnostic validity of the Athens Insomnia Scale. J Psychosom Res. 2003;55:263–7. doi: 10.1016/s0022-3999(02)00604-9. [DOI] [PubMed] [Google Scholar]
- 36.Netzer NC, Stoohs RA, Netzer CM, Clark K, Strohl KP. Using the Berlin Questionnaire to identify patients at risk for the sleep apnea syndrome. Ann Intern Med. 1999;131:485–91. doi: 10.7326/0003-4819-131-7-199910050-00002. [DOI] [PubMed] [Google Scholar]
- 37.Beck AT, Ward CH, Mendelson M. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–71. doi: 10.1001/archpsyc.1961.01710120031004. [DOI] [PubMed] [Google Scholar]
- 38.Gaynes BN, DeVaugh-Geiss J, Weir S, et al. Feasibility and diagnostic validity of the M-3 Checklist: a brief, self-rated screen for depressive, bipolar, anxiety, and post-traumatic stress disorders in primary care. Ann Fam Med. 2010;8:160–9. doi: 10.1370/afm.1092. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Dijk DJ, Czeisler CA. Contribution of the circadian pacemaker and the sleep homeostat to sleep propensity, sleep structure, electroencephalographic slow waves, and sleep spindle activity in humans. J Neurosci. 1995;15:3526–38. doi: 10.1523/JNEUROSCI.15-05-03526.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Anderson C, Sylvester L, Paik S. Recognition and diagnosis of excessive sleepiness associated with shift work disorder: results from shift workers, patients with shift work disorder and healthcare professionals participating in an internet survey. Sleep. 2011;34([Abstract Supplement]):A340. [Google Scholar]
- 41.Drake CL. The characterization and pathology of circadian rhythm sleep disorders. J Fam Pract. 2010;59:s12–7. [PubMed] [Google Scholar]
- 42.Anderson C, Sylvester L, Paik S. The impact of excessive sleepiness associated with shift work: results from shift workers and patients with shift work disorder participating in an internet survey. Sleep. 2100;34([Abstract Supplement]):A167. [Google Scholar]
- 43.Gumenyuk V, Roth T, Korzyukov O, et al. Shift work sleep disorder is associated with an attenuated brain response of sensory memory and an increased brain response to novelty: an ERP study. Sleep. 2011;33:703–13. doi: 10.1093/sleep/33.5.703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Saksvik I, Bjorvatn B, Hetland H, Sandal G, Pallesen S. Individual differences in tolerance to shift work - a systematic review. Sleep Med Rev. 2011;15:221–35. doi: 10.1016/j.smrv.2010.07.002. [DOI] [PubMed] [Google Scholar]
- 45.Drake CL, Friedman NP, Wright KP, Jr., Roth T. Sleep reactivity and insomnia: genetic and environmental influences. Sleep. 2011;34:1179–88. doi: 10.5665/SLEEP.1234. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Bonnet MH, Arand DL. Situational insomnia: consistency, predictors, and outcomes. Sleep. 2003;26:1029–36. doi: 10.1093/sleep/26.8.1029. [DOI] [PubMed] [Google Scholar]
- 47.Watson N, Goldberg J, Arquelles L, Buchwald D. Genetic and environmental influences on insomnia, daytime sleepiness, and obesity in twins. Sleep. 2006;29:645–9. doi: 10.1093/sleep/29.5.645. [DOI] [PubMed] [Google Scholar]
- 48.Viola A, Archer S, James L, et al. PER3 polymorphism predicts sleep structure and waking performance. Curr Biol. 2007;17:613–8. doi: 10.1016/j.cub.2007.01.073. [DOI] [PubMed] [Google Scholar]
- 49.Reinberg A, Motohashi Y, Bourdeleau P, et al. Internal desynchronization of circadian rhythms and tolerance of shift work. Chronobiologia. 1989;16:21–34. [PubMed] [Google Scholar]
- 50.Czeisler CA, Walsh JK, Roth T, et al. Modafinil for excessive sleepiness associated with shift work sleep disorder. N Engl J Med. 2005;353:476–86. doi: 10.1056/NEJMoa041292. [DOI] [PubMed] [Google Scholar]
- 51.Harris P, Taylor R, Thielke R, Payne J, Gonzalez N, Conde J. Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.