Opinion Polls and Science

I AM CONCERNED ABOUT THE CITATION OF THE NATIONAL SLEEP FOUNDATION (NSF) “SLEEP IN AMERICA” POLL AS A SCIENTIFIC REFERENCE USED IN articles published in SLEEP¹ and other peer-reviewed journals. The NSF's data base from the annual “Sleep in America” opinion survey results in findings about sleep in America that are neither necessarily scientifically reliable, nor in agreement with scientific findings from contemporary sleep research.

Opinion polls are not measures of physical or behavioral events. They are personal opinions held by individuals at a given point in time. The basis of each opinion reflects different histories and biases. There are procedures for conducting scientifically sound polls. Consideration of the 2009 Sleep in America poll I believe is an appropriate illustration of the problem of relying on opinion polls that may not meet these scientific standards.

The 2009 NSF poll determined the extent of sleep problems by standard questions regarding night time sleep disturbances: “Difficulty falling asleep,” “Awake a lot during the night,” and “Woke up too early and couldn't get back to sleep.” The respective response levels in 2009 were as follows: difficulty falling asleep (29%), awake a lot (46%), early awakening (30%). The poll also assessed non-restorative sleep via waking up unrefreshed (45%). A spread scale was used with the following categories: every night/almost every night, a few times a week, a few times a month, rarely, and never. A few times a week was accepted as a “sleep problem.” The NSF concluded from these data that 64% of Americans have sleep problems a few times per week and that 41% experience these problems almost every night. More recent survey studies of English-speaking countries have found population-based reports of these kinds of sleep disturbances in the range of 30-42%.^2,3 However, these reports do not necessarily indicate the rate of clinically significant sleep disorders. This point is illustrated by recent epidemiological surveys of Europeans, which revealed that although 34.5% reported having at least one symptom of sleep disturbance (i.e., difficulty initiating, difficulty maintaining sleep, or non-restorative sleep) at least 3 nights per week, only 9.8% report these symptoms and daytime consequences, and only 6.6% satisfied the DSM-IV requirement for positive and differential diagnosis.⁴

The most serious reason for differences in estimates of sleep disturbances between the NSF poll and other polls concerns the response rate (i.e., population sample NSF uses for their conclusions). Although the telephone numbers polled were distributed across the U.S., the completion rate of the NSF poll was 28%, which can easily result in biased conclusions. Thus the completion of 1000 phone calls required nearly 4000 phone calls. As a result the age distribution of the sample was toward the older end: 18-34 (11%), 35-49 (24%), 50-64 (36%), 65 + (28%). Arguably, this was a population in decline as evident by their responses to a question as to whether they had ever been told that they have or had conditions on a chart of medical conditions: 72% reported at least one condition and 49% reported two or more. These included arthritis (31%), depression/anxiety (23%), diabetes (16%), breathing difficulties (16%). In addition to the effect of aging, it is important to recognize respondents may have been unrepresentative of the larger population—for example, opinion polls about sex, religion, race, and gun control typically draw responses primarily from those with strong interests in those topics. In the case of sleep it is very likely that a 20-minute interview would be completed by someone with a problem rather than some one who has no problem or opinion about sleep.

There are also concerns about the analyses of the NSF data. On two different questions, respondents were asked about total sleep time, when they went to bed and what time they got out of bed. From these, the pollsters obtained the difference between the two estimates and concluded that the respondents were spending “almost an hour in bed without sleeping.” Perhaps a more appropriate use of the two measures would have been the averaging of the two measures or a search in the literature to determine the more accurate estimate. However, both self-estimates were used exclusively as data.

A number of items in the NSF poll appear to be push-poll (i.e., items created to obtain responses to modify or change opinions under the guise of research). The press release of the 2009 poll stated: “About 40% of Americans agree that sleep is as important as diet and exercise to overall health and well being.” This used the responses to an item that asked them to rank the importance of sleep, diet and exercise for a person's health or well-being: 25% agreed that sleep was the most important and 17% agreed that it was equally important to diet and exercise. These proportions may have been quite different if this had been a poll about exercise or diet, and likely different if such items as occupation, age, financial resources, locality, housing and the like had been included.

In conclusion, among the various concerns about the scientific legitimacy of the NSF polls, none is more serious than the fact that overall response rates for NSF polls across the years appears to range from 17% to 28%, which would not be considered valid in epidemiological research. At one point the polling organization warned the National Sleep Foundation that such low response rates limited the validity of polls. These low response rates and the other reasons articulated above provide substantial evidence against citing the National Sleep Foundation Sleep in American poll as a scientifically valid and reliable source of information.

DISCLOSURE STATEMENT

The author has indicated no financial conflicts of interest.