Validation of the Behavioral Risk Factor Surveillance System Sleep Questions

Abstract

Study Objective:

Sleep problems may constitute a risk for health problems, including cardiovascular disease, depression, diabetes, poor work performance, and motor vehicle accidents. The primary purpose of this study was to assess the validity of the current Behavioral Risk Factor Surveillance System (BRFSS) sleep questions by establishing the sensitivity and specificity for detection of sleep/ wake disturbance.

Methods:

Repeated cross-sectional assessment of 300 community dwelling adults over the age of 18 who did not wear CPAP or oxygen during sleep. Reliability and validity testing of the BRFSS sleep questions was performed comparing to BFRSS responses to data from home sleep study, actigraphy for 14 days, Insomnia Severity Index, Epworth Sleepiness Scale, and PROMIS-57.

Results:

Only two of the five BRFSS sleep questions were found valid and reliable in determining total sleep time and excessive daytime sleepiness.

Conclusions:

Refinement of the BRFSS questions is recommended.

Citation:

Jungquist CR, Mund J, Aquilina AT, Klingman K, Pender J, Ochs-Balcom H, van Wijngaarden E, Dickerson SS. Validation of the behavioral risk factor surveillance system sleep questions. J Clin Sleep Med 2016;12(3):301–310.

INTRODUCTION

The Behavioral Risk Factor Surveillance System (BRFSS) is an ongoing telephone health survey system conducted by the Centers for Disease Control and Prevention (CDC) in the 50 United States as well as District of Columbia, Puerto Rico, the U.S. Virgin Islands, and Guam. The purpose of the BRFSS is to track health risk behaviors, preventive health practices, and health care access primarily related to chronic disease and injury. The use of the system started in 1984 and continues monthly. More than 350,000 adults are interviewed each year. Information from the surveys is used in research and clinical practice, and is frequently used to support public health policies and health-related legislative efforts.¹

In early 2000, the CDC, in collaboration with the National Sleep Foundation, facilitated a work group charged with raising awareness about, increasing the understanding of, and reducing the impact of sleep deprivation and sleep disorders. In response to the work group's recommendations, the sleep question that was initiated in the BRFSS in 1995 was expanded upon with four optional module questions to capture sleep related symptoms. The current BRFSS sleep questions are: (1) During the past 30 days for about how many days have you felt you did not get enough rest or sleep; (2) On average, how many hours of sleep do you get in a 24-hour period? Think about the time you actually spend sleeping or napping, not just the amount of sleep you think you should get; (3) Do you snore; (4) During the past 30 days, for about how many days did you find yourself unintentionally falling asleep during the day; and (5) During the past 30 days, have you ever nodded off or fallen asleep, even just for a brief moment, while driving?

BRIEF SUMMARY

Current Knowledge/Study Rationale: Accurate and effective questions are needed to screen for sleep disorders in the population. This study was undertaken to assess the validity and reliability of the BRFSS sleep related questions.

Study Impact: The BRFSS telephone survey results are used to establish the prevalence of health problems in society. Validated questions to screen for sleep related health problems are needed to establish national health strategies as well as research funding.

The questions were implemented into the state telephone surveys around 2005, but to date, evidence of their validation is lacking. Therefore, a study was performed to assess the reliability and validity of the BRFSS sleep questions to determine their sensitivity to detect sleep disorders and insufficient sleep in adults. Establishing questions that will accurately identify individuals at risk of poor health outcomes related to sleep/wake disturbance will allow for better appropriation of resources to increase awareness, detection, and treatment, thereby decreasing the health and economic burden to both the individual and the community.

To establish the importance of screening for sleep disturbance and to assure that the questions are succinctly asking about sleep problems likely to result in burden to individual and society health, the sleep literature was reviewed. From the evidence, sleep disorders and lack of sleep opportunity were established as the two main concepts used for comparison analysis in establishing the validity of the BRFSS sleep questions. There is clear evidence that sleep disturbance and deprivation in Americans is closely associated with decreased daytime performance (social, cognitive, psychological), increased morbidity and mortality (hypertension, coronary heart disease, diabetes, cerebral vascular disease), risk of accidents, and diminished quality of life.^2–8

METHODS

A cross-sectional study of community dwelling adults in the Upstate New York region was conducted to assess the sensitivity and specificity of the current 5 BRFSS sleep questions to detect sleep/wake disturbance. BRFSS questions and more objective measures of sleep were obtained at repeated cross-sectional assessments. Screening questions were tested for reliability and validity against the most feasible gold standard measures. Study procedures were approved through the University of Rochester and University at Buffalo institutional review boards.

Subjects

Three hundred fifty subjects were recruited from the community at large. Three hundred subjects were found study eligible and completed all study procedures. Eligibility criteria consisted of English speaking adults over the age of 18 years. Exclusion criteria were use of positive airway pressure or oxygen while sleeping. These criteria were necessary, as study procedures involved screening for sleep disordered breathing, which would not be possible if the person was wearing positive airway pressure interface or oxygen cannula.

Recruitment

Random digit dialing telephone recruitment was initially employed but failed to obtain the estimated recruitment numbers in a timely manner. Recruitment strategies were then redirected to focus on social media, Craigslist, Research Match, the University of Rochester's recruitment website, flyers posted in public areas, and word of mouth. Subjects would contact the research team via telephone or email with their permission to be contacted by the research assistant for study procedures. A telephone call to the potential subject was initiated that included verbal consent to participate, stressing confidentiality. Once found willing and study eligible, the subjects were asked to respond to the BRFSS sleep questions. Responses were recorded as time point 1. Once the first 150 subjects were enrolled, we assessed participants' ethnicity, race, age, and gender, and then compared study enrolled characteristic percentages to United States population-based percentages. Recruitment strategies were then directed at meeting sample demographics that were representative of the population.

Study Procedures

Participants attended one 60-minute study visit, wore an Actiwatch that included electronic sleep diary data entry for 2 weeks, and wore the Apnealink device for one night. During the study visit, questionnaires were administered and the participants underwent a localized physical examination. The Questionnaires consisted of the Insomnia Severity Index (ISI), Epworth Sleepiness Scale (ESS), PROMIS-57 profile, and a questionnaire to gather demographics with medical history and medications. The brief examination included airway Mallampati score, neck and waist circumference, blood pressure, pulse, respiratory rate, height, weight, and functional status. The participants responded to the BRFSS questions at 3 time points (screening telephone call, end of 14 days of data captured via actigraphy and via phone call 30 days from study visit). They were compensated for their participation with $75 in gift cards divided at intake, Apnealink device pick-up, and PRO-Diary device pick up.

Measures and Variables

Insomnia Severity Index

Insomnia Severity Index (ISI) is a 7-item subjective measure of sleep quality, sleep disruption, satisfaction, and worry about sleep and how sleep interferes with daytime function. Time-frame for inquiry was the past 30 days. Each item is rated on a 0–4 scale with 0 representing no symptom and 4 representing very much, resulting in a score ranging from 0–28. The threshold for screening positive for mild insomnia is a score > 7.⁹

Excessive Daytime Sleepiness

Excessive daytime sleepiness was determined as the total Ep-worth Sleepiness Scale (ESS)¹⁰ score that was delivered during intake visit. The ESS is an 8-item questionnaire that reflects the subjective report of likelihood of falling to sleep in 8 situations. The range of scores is 0–24, with 24 the most severe of excessive daytime sleepiness. The timeframe inquired was over the past month. A score > 10 is considered clinically relevant sleepiness¹⁰. The PRO-Diary device¹¹ was also programmed to collect data every evening on the subjects' degree of daytime sleepiness during the day the answer was recorded. As data were collected for 14 days and the time frame of the BRFSS questions are all 30 days, the daily diary responses and BRFSS#4 and BRFSS#5 were standardized to percent of days. See Table 1 for question wording.

Table 1.

Study constructs and measures.

PROMIS-57 Profile

PROMIS-57 Profile questionnaire is a paper-based highly reliable, valid, measure that assesses 8 components including: physical function, anxiety, depression, fatigue, sleep, satisfaction with social function, pain interference, and pain intensity.¹² Component scores were calculated considering missing data via PROMIS scoring guidelines. Each component score was then converted to a standard score with a mean of 50 and a standard deviation (SD) of 10. Therefore a person with a T-score of 40 is one SD below the mean. The standardized T-score is reported as the final score for each participant. Higher scores on the component translate to more of the symptom or higher level of function.

Total Sleep Time

Total sleep time (TST) is the number of hours asleep as measured by actigraphy. The device used was the Camntech Pro-Diary device. The PRO-Diary is a compact wrist-worn electronic diary with integrated activity monitor designed to collect patient reported outcome (subjective data) and actigraphy (objective data) simultaneously.¹³ Subjects were asked to wear the device on their non-dominant wrist continuously for 2 weeks except when showering or swimming. They were instructed to activate the electronic diary and respond to the evening questions as they went to bed and the morning questions when they woke for the last time of their sleep period. Motion-ware captured motion activity counts every minute. Default software calculated sleep per 24-h period. The variables for SL, WASO, NWAK, and TST were downloaded into an Excel database then cleaned, removing any artifacts.

Presence of Obstructive Sleep Apnea

Presence of obstructive sleep apnea (OSA) was determined by a sleep medicine boarded physician using the American Academy of Sleep Medicine criteria of apnea-hypopnea index (AHI) > 5.¹⁴ Apnea was measured using the Apnealink device.^15–17 Subjects wore the device for one night during the 2-week PRO-Diary procedure. Data was scored using device software then confirmed by our sleep physician, who was blinded to the BRFSS responses. The AHI was confirmed by the physician and entered into an excel spreadsheet. A dichotomous variable was developed and coded as (yes/no).

Presence of Snoring

Presence of snoring was measured using the ApneaLink device. The physician scoring the data determined their severity of snoring as mild, moderate, or severe. Variables for presence of snoring (yes/no) and severity were developed and data were entered into the Excel spreadsheet by the physician scoring the Apnealink data. Physician was blinded to the BRFSS responses.

Data Management

All data were collected via paper except PRO-Diary and Apnealink Data. Paper data was double entered and verified for consistency. Any discrepancies were investigated and resolution was clarified by primary investigator. PRO-Diary responses were downloaded into an Excel spreadsheet, cleaned of clinically unreasonable data, verified for appropriate dates and times then aggregated per subject. Averages across all days of data were calculated then used to compute a variable representing subjective report of total sleep time and sleep efficiency. Objectively measured sleep variables were calculated using default Motionware software settings. Study period averages of variables were calculated for each subject in Excel and imported into SPSS for analysis. The sleep medicine physician scored the Apnealink raw data verifying the accuracy of the data that was scored automatically by the software package. All data were de-identified and stored in secured cabinets and/or servers.

Analysis

Descriptive analysis including assessment of distribution was performed on all variables in SPSS version 21. Reliability analysis using Cronbach α coefficient with ANOVA F statistics to test differences over time was performed on the BRFSS questions over 3 time points (intake, 7–14 days, and 30 days). Bland Altman analysis (using PRISM 6 version 6), correlation analysis, and risk indexes were calculated to test construct and/ or discriminate validity per Table 1. Each BRFSS question was analyzed separately to test sensitivity and specificity in predicting the appropriate construct.

Missing Data

Apnealink data were missing on 14 subjects due to software issues and incomplete data at acquisition. Motionware data were missing on 56 subjects and PRO-Diary data were missing on 53 subjects due to device malfunction, software issues, human error while downloading data, subjects not wearing the device, and subjects losing the device. Several subjects did not wear the PRO-Diary device for the entire 14 days/nights as instructed. Missing data were handled by using averages over days of actual data. Reliability testing excluded by pairwise comparisons. Certain variables are reported as “percent of days” to standardize comparisons between daily data and BRFSS responses that are anchored on the past 30 days.

RESULTS

Sample Characteristics

Three hundred fifty subjects contacted the research team for participation. Three hundred subjects were enrolled in the study. One person withdrew after completing a study visit and Apnealink procedure, as they could not tolerate wearing the wrist-worn device.

The enrolled sample was similar to US population percentages on age, gender, and race. Enrollment fell short of the 45 targeted Hispanics. The sample age ranged from 18–96 years, and all were community dwelling residents. Measures of mood, pain, and function revealed an overall healthy sample per PROMIS standardized scores with population mean of 50. See Table 2 for subject characteristics.

Table 2.

Participant characteristics compared to U.S. population (n = 300).

BRFSS Question #1

BRFSS Question #1 (BRFSS#1): During the past 30 days for about how many days have you felt you did not get enough rest or sleep?

Responses (mean [SD]) at each time point (Day 1, Day 14, and Day 30) were: 12.26 (9.7) n = 297; 14.34 (9.7) n = 117; 10.89 (8.8) n = 136.

Test-Rest Reliability

Test-rest reliability was assessed on responses to BRFSS#1 at intake, 14 days, and 30 days. Comparing over 3 time points, Cronbach α was 0.84 (F = 2.17, df = 49, p = 0.12). Comparing intake to 14 days, Cronbach α was 0.75 (F = 1.22, df = 115, p = 0.27). Comparing over intake to 30 days, Cronbach α decreased to 0.67 (F = 0.09, df = 138, p = 0.76). Participants' responses were consistent over time.

Criterion Validity

Criterion validity was assessed on subjects' responses to BRFSS#1 at day 14 (mean = 47%, SD = 33). To meet the assumptions of Bland-Altman procedures, data from BRFSS#1 at day 14 were transformed into percent of days not rested then compared to the mean difference between the percent of days the subject responded no to the morning daily diary question “did you wake feeling refreshed.” Responses to daily diary question averaged over the 14 days were (mean = 46%, SD = 29; see Figure 1). The large SD reveals the weakness of BRFSS#1 in capturing the true daily measure of feeling rested. From this analysis, the responses to BRFSS#1 could differ as much as 77% from asking the patient in the morning on waking if they feel refreshed. The results of this analysis could also reflect that the two questions are accessing different constructs.

Figure 1. Bland-Altman of BRFSS#1 day 14 vs daily diary entry (n = 101).

Figure 1 exhibits the comparisons between responses to BRFSS#1 and daily electronic diary entries. The Y-axis shows the difference in number of days rested comparing daily entry to BRFSS#1 (estimate over the past 30 days); the X-axis shows the average of the differences in number of days not rested identified using the two methods of measurement. The horizontal lines reflect the mean bias of −3.229 (95% CI = −77.54 to 71.08).

To assess if BRFSS#1 was accessing the construct representing duration of sleep, Spearman rho correlation analysis was performed with sleep categories (short, normal, and long sleeper). BRFSS#1 was not significantly correlated with sleep category (rho = −0.01, df = 236, p = 0.85).

Convergent/Discriminant Validity

Convergent/discriminant validity was assessed comparing BRFSS#1 responses to constructs theorized to be different than lack of sleep opportunity. Correlation analyses were performed comparing BRFSS#1 with responses to the ISI total score (mean = 8.13, SD = 5.50), ESS total score (mean = 7.09, SD = 4.50), PROMIS-57 Profile depression (mean = 11.52, SD = 5.14), and pain (mean = 11.40, SD = 5.50) component raw scores and total sleep time in minutes (mean = 416, SD = 52) as measured by actigraphy. BRFSS#1 was positively and significantly correlated with the total score of the ISI, (rho = 0.56, p < 0.05) and excessive daytime sleepiness as measured by the total score on the ESS, (rho = 0.26, p < 0.05). BRFSS#1 responses were significantly but weakly correlated with anxiety (rho = 0.28, p < 0.05), depression (rho = 0.20, p < 0.05), pain interference (rho = 0.16, p < 0.05), satisfaction with social role (rho = −0.27, p < 0.05), and physical functioning (rho = −0.13, p = 0.03). Correlation with sleep (rho = 0.55, p < 0.05) and fatigue (rho = 0.47, p < 0.05) component scores were significant and moderate. The correlation with total sleep time in minutes was not significant (r = 0.02, p = 0.68).

Summary

In summary, BRFSS#1 was reliable over time but not sensitive to detect lack of sleep opportunity or sleep duration. BRFSS#1 was more closely associated with measures of insomnia, excessive daytime sleepiness, function, pain, and mood disorders.

BRFSS Question #2

BRFSS Question #2 (BRFSS#2): On average, how many hours of sleep do you get in a 24-hour period?

Responses (mean [SD]) at each time point (Day 1, Day 14, and Day 30) were: 6.98 (1.3) n = 297; 6.85 (1.2) n = 117; 6.98 (1.2) n = 137.

Test-Rest Reliability

Test-rest reliability was assessed on responses to BRFSS#2 at intake, 14 days, and 30 days. (Cronbach α) and differences among time points (F) was assessed. Comparing over 3 time points, Cronbach α was 0.89 (F = 1.18, df = 46, p = 0.17). Comparing intake to 14 days, Cronbach α was 0.76 (F = 0.31, df = 114, p = 0.57). Comparing intake to 30 days, Cronbach α was 0.85 (F = 4.96, df = 135, p = 0.03). BRFSS#2 is consistent over time.

Criterion Validity

Criterion validity was assessed by comparing BRFSS#2 at time point day 14 responses to total sleep time in hours measured by actigraphy on the previous 14 days (mean = 6.9, SD = 0.8). The Bland Altman comparison indicated the lack of agreement was 0.05 and SD = 1.2 (95% CI −2.1 to 2.2; see Figure 2). The lack of agreement is very small and not clinically relevant. Considering the SD, 95% of people could report their total sleep time subjectively around 2 hours different from what would be measured objectively. This result reflects the common issue with individual variations in subjective and objective comparisons of total sleep time found with other studies.¹⁸

Figure 2. Bland-Altman of BRFSS#2 Day 14 vs actigraphy total sleep time in hours (n = 117).

Figure 2 exhibits the comparisons between responses to BRFSS#2 and total sleep time in hours measured by actigraphy. The Y-axis shows the average of the differences in hours; the X-axis shows the mean using the two methods of measurement. The horizontal lines reflect the mean bias of 0.049 and SD = 1.2 (95% CI = −2.1 to 2.2).

Convergent/Discriminant Validity

Convergent/discriminant validity was assessed against a proposed question that the authors theorized could be more helpful in assessing for self-imposed lack of sleep opportunity. Participants responded to the question “How many hours of sleep do you need to feel rested?” during the study visit. BRFSS#2 was compared with responses to the new question (mean = 7.53, SD = 1.5) using Bland-Altman analysis. The Bland-Altman comparison indicates the lack of agreement is −0.41 and SD = 1.6 (95% CI −3.5 to 2.7; Figure 3). The lack of agreement is overall around 25 min, and 95% of the population may report a sleep opportunity deficit of as much as 3.5 hours.

Figure 3. Bland-Altman of BRFSS#2 vs question about hours needed to feel rested (n = 297).

Figure 3 exhibits the comparisons between responses to BRFSS#2 and the question “How many hours of sleep do you need to feel rested?” The Y-axis shows the average of the differences in hours; the X-axis shows the mean of the two questions. The horizontal lines reflect the mean bias of −0.41 hours, SD = 1.6 (95% CI = −3.5 to 2.7).

To assess if duration of sleep (< 6 h, 6–7.9 h, and > 8 h) was associated with symptoms related to health burden such as excessive daytime sleepiness, depressive symptoms, fatigue, anxiety, satisfaction with social role, physical functioning, or pain, group means were compared using ANOVA analysis. No significant differences were found.

Summary

In summary, BRFSS#2 was found to be a valid question for obtaining data on the number of hours of sleep.

BRFSS Question #3

BRFSS Question #3 (BRFSS#3): Do you snore?

Table 3 presents descriptive data of BRFSS#3 with related variables. BRFSS#3 was difficult for the participants to answer with a definite yes or no. Subjects could not decide on a yes or no answer or did not want to answer because they slept alone and had no way of knowing if they snored. They also stated that “sometimes” they snore, such as when they have allergies or a cold. Therefore, we gave the subjects 4 choices for the answer (yes, no, sometimes, or I don't know). See Table 4.

Table 3.

Participants characteristics, sleep related variables (n = 300).

Table 4.

Variables and alternative snoring questions used in BRFSS#3 analysis.

Test-Rest Reliability

Test-rest reliability was assessed on responses (4 choices of response) to BRFSS#3 at intake, 14 days, and 30 days. To assess test-retest reliability (Cronbach α) and differences among time points (F) was assessed. Comparing over 3 time points, Cron-bach α was 0.603 (F = 1.431, df = 47, p = 0.244). Comparing intake to 14 days, Cronbach α was 0.746. Comparing intake to 30 days, Cronbach α was 0.648. The weaker Cronbach α likely represents the difficulty the participants experienced with answering this question with a definite yes or no, and thus gave inconsistent responses over time.

Criterion Validity and Predictive Values

Criterion validity and predictive values for presence of OSA were assessed using dichotomous responses, we recoded yes, sometimes, and I don't know responses as yes. Then χ² analysis and odds ratios using the OSA diagnosis (yes/no) given the subject by our sleep medicine physician using Apnealink data were performed. There were significant differences within the cells (χ² = 29.59, p < 0.05). BRFSS#3 is 41.55% (CI 33.35% to 50.11%) sensitive and 87.33% (80.57% to 92.25%) specific for detecting the presence of at least mild OSA (AHI > 5). Positive likelihood ratio is 3.25 and negative likelihood ratio is 0.67. Positive predictive value was 76.62%, and negative predictive value was 59.71%.

Criterion validity and predictive values for presence of snoring on Apnealink were assessed. The answers to BRFSS#3 responses of no, sometimes, and I don't know were then recoded to a variable labeled as no snoring. BRFSS#3 was found to be 51.95% (CI 40.26% to 63.48%) sensitive and 76.33% (69.94% to 81.94%) specific for detecting the presence of at least mild OSA (AHI > 5). Positive likelihood ratio was 2.19 and negative likelihood ratio was 0.63. Positive predictive value was 44.94% and negative predictive value was 81.03%.

As the investigators hypothesized a priori that BRFSS#3 would not be a sensitive or specific question to predict the presence of sleep apnea. Three similar questions presented differently were asked during the initial study visit: (1) “Has anyone ever told you that you snore” with Likert responses of never, seldom (1/y), sometimes (1–3/mo), often (1–3/w), or frequently (> 3/w), (2) “Do you snore” with forced answer of yes or no, and (3) “I have been told I snore” with Likert responses of never, seldom (1/y), sometimes (1–3/mo), often (1–3/w), or frequently (> 3/w). Table 5 shows results for predictors of diagnosed OSA, and Table 6 for predictors of snoring on Apnealink.

Table 5.

Snoring questions predictive values for OSA (disease prevalence 27%).

Table 6.

Snoring questions and their predictive values for snoring on Apnealink (disease prevalence 48%).

As this question was meant to screen for untreated or undertreated obstructive sleep apnea and most screening questionnaires for OSA ≥ 2 questions, including the symptom of excessive daytime sleepiness, predictor value of a combination of BRFSS#3 and BRFSS#4 was calculated and compared to single predictive value of just BRFSS#3 (Tables 5 and 6). Combining the two questions did not increase the predictive value beyond just asking BRFSS#3. Unfortunately asking about snoring in this sample resulted in only 30% to 40% chance of the person actually having OSA. Thus, BRFSS#3 was found to be a poor screen for OSA especially if the person must choose a yes or no response. If the response allows for quantification, the question becomes more specific for detecting OSA. In general, snoring questions used in solidarity are poor predictors of obstructive sleep apnea.¹⁹

BRFSS Question #4

BRFSS Question #4 (BRFSS#4): During the past 30 days, for about how many days did you find yourself unintentionally falling asleep during the day?

Responses (mean [SD]) at each time point (Day 1, Day 14, and Day 30) were: 3.78 (6.5) n = 297; 3.46 (6.6) n = 117; 2.74 (5.1) n = 137.

Test-Rest Reliability

Test-retest reliability (Cronbach α) and differences among time points (F) was assessed. Comparing over 3 time points, Cronbach α was 0.90 (F = 0.20, df = 45, p = 0.82). Comparing intake to 14 days, Cronbach α was 0.82 (F = 0.54, df = 114, p = 0.46). Comparing intake to 30 days, Cronbach α was 0.80 (F = 3.8, df = 134, p = 0.05). BRFSS#4 appears to be reliable and consistently answered over a 14-day time frame, but the 30-day time frame was found less reliable.

Criterion Validity

Criterion validity was assessed via Spearman rho correlation analysis on responses to BRFSS#4 time point Day 1 with the total score of the Epworth Sleepiness Scale. Responses to BRFSS#4 and total score of the Epworth Sleepiness Scale were moderately and positively correlated (rho = 0.50, df = 254, p < 0.05).

Convergent/Discriminant Validity

Convergent/discriminant validity was assessed by comparisons to raw scores on the PROMIS depression, physical function, and fatigue component scales. There was not a significant relationship (rho = 0.08, df = 293, p = 0.17) between depression and number of days unintentionally falling asleep during the day. There was a small but significant negative relationship with physical function (rho = −0.19, df = 295, p. < 0.05) and a small but significant positive relationship with fatigue (rho = 0.27, df = 297, p. < 0.05). Additionally, responses to BRFSS#4 were regressed on apnea hypopnea index (AHI). The AHI significantly (p < 0.05) predicted higher BRFSS#4 response. For every 1-point increase in AHI, the number of days unintentionally fell to sleep increased by 0.325.

Summary

In summary, in this sample of subjects who report on average 10% of the days they unintentionally fall asleep, BRFSS#4 was a reliable question over 14 days only. BRFSS#4 correlated with total score on the Epworth Sleepiness Scale, and was found to be associated with higher fatigue, decreased physical function, and higher AHI.

BRFSS Question #5

BRFSS Question #5 (BRFSS#5): During the past 30 days, have you ever nodded off or fallen asleep even just for a brief moment while driving?

Responses (mean [SD]) at each time point (Day 1, Day 14, and Day 30) were: No = 90% n = 290; No = 90% n = 117; No = 93% n = 137.

Test-Rest Reliability

Test-retest reliability (Cronbach α) and differences among time points (F) was assessed. Comparing over 3 time points, Cron-bach α was 0.71 (F = 1.0, df = 137, p = 0.37). Comparing intake to 14 days, Cronbach α was 0.62 (F = 33, df = 225, p = 0.57). Comparing intake to 30 days, Cronbach α was 0.51 (F = 0.25, df = 267, p = 0.62).

The underlying construct of this question was assumed to be drowsy driving. As it is not possible to follow subjects around to see if they nod off while driving, the surrogate measure of excessive daytime sleepiness was used as the comparator. To assess the sensitivity of the question to detect excessive daytime sleepiness per the Epworth Sleepiness Scale (total score > 10) risk analyses was performed. BRFSS#5 was 86.76% (95% CI 76.35 to 93.75) sensitive and 65.46% (95% CI 59.82 to 70.80) specific with a PPV of 35.98% (95% CI 28.94 to 43.83) and NPV of 95.67 (95% CI 91.94 to 98.00).

Summary

In conclusion, BRFSS#4 and BRFSS#5 were found to have a moderate positive relationship with measures of excessive daytime sleepiness. Additionally, the questions were reliable over time.

DISCUSSION

This study was undertaken to assess the performance of the sleep questions on the BRFSS to screen for sleep problems that are associated with health burden. The most serious health burden associated with sleep problems is automobile accidents from drowsy driving, cardiovascular disease, impaired cognitive and work performance, and mood dysregulation.^{3,4,6,20–22} Screening for sleep problems associated with health burden requires attention to two main concepts: sleep disorders and insufficient sleep opportunity. In general, the current BRFSS questions are reliable but requirement refinement to increase their sensitivity and specificity.

Screening for Non-Restorative or Insufficient Sleep

Non-restorative sleep is usually associated with a sleep disorder as opposed to lack of sleep opportunity. Insufficient sleep is a term that means the person is not providing enough opportunity to sleep in a sleep-promoting environment. Lack of sleep opportunity is a common situation in this era of balancing work and financial obligations with family responsibilities.²³ Additionally the influence of social media and technology, especially in young adults, is particularly a problem.²⁴ Lack of sleep opportunity is also found in shift workers and truck/bus drivers. Insufficient sleep is associated with cancer, weight gain, anxiety, depression, suicide, cardiovascular disease,²⁵ and immune dysfunction, and thus is an important health issue in the population at large.^26–32 As individual's sleep need is unique, screening for insufficient sleep requires direct questioning about sleep need in comparison to total sleep achieved and/or inquiring about the presence of the consequences of inadequate sleep as opposed to just asking how much sleep the person obtains. BRFSS#1 attempts to screening for non-restorative sleep or perhaps just not feeling rested, but is not specific for detecting either sleep disorders or lack of sleep opportunity. BRFSS#2 would be a better choice when screening for insufficient, especially now with the recent consensus reporting the need for 7 hours of sleep.²³ Pairing BRFSS#2 with BRFSS#4 may increase the specificity for detecting lack of sleep opportunity that is associated with daytime consequences.

Screening for Total Sleep Time

There is consensus with sleep experts that sleeping less than 7 hours results in serious health and work related burden.²³ It seems reasonable to use a question that will directly assess a person's total sleep time. BRFSS#2 can be used confidently if one considers there are likely some individuals with subjective/objective discrepancies. The question was found reliable over time and correlated nicely with objective measure of total sleep time. If a question inquiring about hours of sleep needed to feel rested was added to the survey, the specificity of determining lack of sleep opportunity could be increased.

Screening for Sleep Disorders

The most common sleep disorders are insomnia and obstructive sleep apnea.³³ Specifically, untreated obstructive sleep apnea is known to contribute to drowsy driving and automobile accidents, decrease in cognitive and work performance, hyper-tension, cardiovascular disease, diabetes, and depression.^34–40 According to previous studies by Chung et al.,¹⁷ screening for obstructive sleep apnea requires more than one question. To further emphasize that point, receiver operating curves for single questions (BRFSS#1, wake gasping, observed apneas, and total Epworth Sleepiness Scale) were all less than 0.6 and thus considered very weak predictors of an AHI > 5. The STOP-Bang questionnaire has been found to be highly predictive of obstructive sleep apnea. The STOP questions are: (1) do you snore loudly, (2) are you often tired, fatigued or sleepy during the day, (3) has anyone said that you stop breathing during sleep, and (4) have you been diagnosed with hypertension. In a sample of patients without a history of sleep disorders and OSA prevalence of 69%, Chung et al.⁴¹ found the STOP questionnaire to have a positive predictive value of 0.78 (95% CI 0.69 to 0.86) and a negative predictive value of 0.44 (95% CI 0.32 to 0.56) for an AHI > 5. In our sample with a disease prevalence of 27% the BRFSS#3 Do you snore had a PPV of 0.41 (95% CI 0.33 to 0.50) and a NPV of 0.87 (95% CI 0.80 to 0.92). For the purpose of detecting probable OSA via telephone screens, in the population at large where the prevalence is around 20% to 25%, just asking if the person snores will detect OSA around 76% of the time if they endorse snoring, and 59% of the time if they deny snoring. Therefore, revisions to the BRFSS sleep questions to make them more sensitive and specific for detecting sleep disorders are recommended.

Screening for Excessive Daytime Sleepiness

Excessive daytime sleepiness is often the result of lack of sleep opportunity or a sleep disorder. The Epworth Sleepiness Scale is commonly used in clinical practice to screen for obstructive sleep apnea. Unfortunately, only 50% of patients with obstructive sleep apnea will endorse excessive daytime sleepiness.⁴² There have been studies showing a single question can be used to screen for excessive daytime sleepiness, such as “please measure your sleepiness on a typical day, 0–10 scale.”⁴³ There is also other research that suggests using BMI > 32 to screen for excessive daytime sleepiness, as responses to the direct question about EDS may be biased with people working in occupations that will remove the person from work if found to have EDS.⁴⁴ BRFSS#4 attempts to screen for excessive daytime sleepiness. Using the responses to this question requires the assumption that the person is actually allowing himself or herself to fall into sleep. Additionally, the interpretation of the responses is subjective; for example, how many days of unintentionally falling asleep is actually abnormal? Therefore, revisions to the BRFSS questions are recommended.

Screening for Drowsy Driving

Bias is inherent in the responses to questions about falling asleep driving, as people rarely will endorse letting themselves actually fall asleep driving. Instead, alert activating activities such as rolling down the window, playing loud music, or talking with someone on the phone are used. If a screening question is seeking drowsy driving as opposed to actual falling asleep at the wheel, the question should include questions about alert activating activities, or at least specifically asking about feeling sleepy while driving. In addition to screening for drowsy driving, it seems prudent to inquire about other occupations or situations where falling sleep asleep will cause serious harm to self or others. Such situations may be falling asleep on a production line in a factory or while piloting an airplane. Therefore, a revision to expand the question about drowsy driving to include other situations where excessive sleepiness may be a problem is recommended.

Limitations

This study was conducted in Upstate New York, and recruitment strategies were mostly limited to electronic avenues, of which some residents do not have access. Targeted enrollment of elders occurred via posters and word of mouth and may have limited participation to like enrollees. Recruitment using the BRFSS telephone strategy would have accessed the same type of individuals who would participate in the BRFSS surveys, but that recruitment strategy did not produce participants in a feasible manner. Additionally, bias towards participants who are interested in research participation likely occurred. The lack of sufficient Hispanic participants was sustained, despite attempts to target Hispanic neighborhoods with posters. Most participants were not engaged enough in the study to respond to the day 30 BRFSS responses. Despite attempts to ensure the participants would answer the phone to respond at the day 30 time point, most did not return our phone calls. Recruitment strategies did not use the word “sleep” in their title, but in the description of study, it was obvious that screening for sleep problems was going to occur. This could have biased the sample towards subjects who thought they might have a problem with their sleep. The use of gold standard measure of in-lab polysomnography was not feasible and may have led to less precise detection of sleep disordered breathing. The use of wrist actigraphy was a challenge, as several subjects did not wear the device for the full 14 days, and thus resulted in incomplete data.

CONCLUSIONS

Sleep disorders and insufficient sleep are serious problems that result in burden to individuals and society at large. Data derived from the BRFSS are used to establish prevalence of health-related problems that drive funding for national education and research priorities. The sleep-related screening questions used by the BRFSS must be sensitive to detect lack of sleep opportunity as well as presence of sleep disorders. The current sleep related questions are a welcome addition to the BRFSS, but results of this study found some of the questions require refinement to increase their sensitivity and specificity.

DISCLOSURE STATEMENT

This was not an industry supported study. This manuscript was supported by Cooperative Agreement Numbers U48DP001910 and U48DP001910-05S1 from The Centers for Disease Control and Prevention (CDC). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention. The authors have indicated no financial conflicts of interest. The site of research study was University of Buffalo, Buffalo, NY under contract with University of Rochester, Rochester, NY.

ABBREVIATIONS

AHI

apnea-hypopnea index

BRFSS

Behavioral Risk Factor Surveillance System

CDC

Centers for Disease Control and Prevention

ESS

Epworth Sleepiness Scale

ISI

Insomnia Severity Index

NWAK

number of awakenings from sleep

OSA

obstructive sleep apnea

PRO-Diary

Patient Reported Outcome-Diary

PROMIS

Patient Reported Outcomes Measurement Information System

SL

sleep latency

TST

total sleep time