Parallel Assessment Challenges in Nutritional and Sleep Epidemiology

Abstract

Sleep has been consistently linked to health outcomes in clinical studies, but only in recent years has sleep become a focus in epidemiologic studies and public health. In particular, the sizable prevalence of insufficient sleep in the population warrants well-designed epidemiologic studies to examine its impact on public health. As a developing field, sleep epidemiology encounters methodological challenges similar to those faced by nutritional epidemiology research. In this article, we describe a few central challenges related to assessment of sleep duration in population-based studies in comparison with measurement challenges in nutritional epidemiology. In addition, we highlight 3 strategies applied in nutritional epidemiology to address measurement challenges and suggest ways these strategies could be implemented in large-scale sleep investigations.

Abbreviations

OSA

obstructive sleep apnea

PSG

polysomnography

Despite consistent reports about the negative impact of insufficient sleep on health (1), more than a third of American adults are sleep deprived (2), representing a major public health burden. Indeed, insufficient sleep has been linked to deleterious health outcomes such as cardiometabolic morbidity (3–7), depression (8), and mortality risk (9) in addition to motor vehicle accidents (10). To maintain optimal health, the American Academy of Sleep Medicine recommends that adults obtain at least 7 hours of sleep per night (11).

Sleep duration is not the only important construct as over the last 70 years, the field of sleep medicine has made significant advances with the nosology of more than 80 sleep disorders and the development of technology for diagnosis and treatment of sleep disorders (12). More recently, the rapid growth of sleep epidemiology has garnered attention from wide audiences, even beyond the scientific community (13). Yet population-based sleep research has methodological challenges surrounding the assessment of sleep. These measurement challenges, inherent in large studies, limit interpretation of prior findings and comparison across studies.

Interestingly, the field of nutritional epidemiology has faced similar methodological obstacles in the assessment of diet. Beyond their mutual measurement challenges, sleep and nutrition are entwined via complex biological, behavioral, and environmental pathways (14). The association between sleep and nutrition is bidirectional; sleep duration and quality affect macronutrient and caloric consumption (15, 16), while intake of particular foods (e.g., tart cherries, kiwi fruit, and fatty fish) (16, 17) and dietary patterns (18) might affect sleep duration, quality, and timing. Sleep and nutrition are influenced by circadian rhythms, socioeconomic status, and physical activity, and both, independently and jointly, contribute to cardiometabolic morbidity. Specifically, sleep duration, variability, and timing as well as dietary composition, quantity, and mealtimes influence adiposity, type 2 diabetes, and cardiovascular disease (19–21).

Population-based nutritional studies date back to the 1940s and have had an impact on dietary recommendations worldwide (22). As nutritional epidemiology has expanded, methods and tools have been developed and refined to minimize the inherent measurement error in diet assessment. We believe that the conceptual framework that guided the evolution of these methods and tools in nutritional epidemiology could inform the growing field of sleep epidemiology, which includes both sleep clinicians and investigators developing population-based sleep research.

This article illustrates 3 parallel methodological challenges in sleep and nutritional epidemiology to guide epidemiologists and clinicians with sleep-research pursuits in large-scale samples. Sleep assessment and analytical approaches are common limitations in population-based sleep research. In particular, implementation of causal frameworks in large-scale sleep research and addressing systematic errors in sleep assessment could improve the accuracy of effect estimates. Here, we provide a practical perspective to address these challenges by implementing methods to minimize bias that are commonly used in nutritional epidemiology. Measurement errors are inevitable in all epidemiologic studies, but careful consideration of assessment instruments and causal pathways during the planning stages could alleviate and perhaps avoid these pitfalls. The aim of this paper is to increase awareness among investigators with an interest in epidemiologic sleep research about techniques widely used in the nutritional epidemiology field that could be implemented primarily in the assessment of sleep duration, either in new studies or with preexisting data.

PARALLEL METHODOLOGICAL CHALLENGES IN SLEEP AND NUTRITIONAL EPIDEMIOLOGY

Application of the causal framework in sleep and nutrition research

Directed acyclic graphs, or causal diagrams, serve as a visual presentation of a specified causal pathway. Causal diagrams aid in the formalization of research questions and identify confounding and mediating pathways between the exposure, outcome, and third variables within a causal framework.

Sleep is a multidimensional state represented by its duration, architecture (sleep stages), timing, continuity, quality, and intraperson night-to-night variability; thus, determining which aspects of sleep to measure in a particular study can be a challenge. Depending on the research question, the different dimensions of sleep might confound, interact with, or mediate one another in relation to a health outcome (7). Here, we use causal diagrams to illustrate the role of sleep duration as a confounder or a mediator in 2 separate research questions (23, 24). In older adults with poor sleep quality, long sleep duration is associated with cardiovascular mortality, suggesting that sleep quality modifies the relationship between sleep duration and cardiovascular mortality (25). Alternatively, sleep duration might be confounded by concurrent sleep disorders such as insomnia, obstructive sleep apnea (OSA), or restless leg syndrome. Indeed, a population-based cohort reported that untreated OSA was associated with high mortality risk, independent of sleep duration (Figure 1A) (24). Sleep architecture, specifically, the percentage time in slow wave sleep, has been inversely associated with body mass index in older men, independent of OSA and total sleep duration (26). Finally, sleep duration might be an intermediate variable on a pathway between an exposure and an outcome. For example, in a large study among employed workers, sleep duration mediated the association between working long hours per week and occupational injury risk (Figure 1B) (23).

Figure 1.

Causal diagrams showing different pathways involving sleep duration. A) Causal diagram representing potential confounding by sleep duration of the association between treatment of obstructive sleep apnea and mortality risk. B) Causal diagram representing potential mediation by sleep duration of the association between working hours per week and occupational injury risk.

There is a parallel in nutritional epidemiology. Diet is similarly a multifaceted behavior that represents quantity, composition, and timing of nutrient intake. In relation to health outcomes, diet can be a predictor, a source of confounding, a mediator, or an effect modifier. For example, total energy intake often confounds the association of red meat intake and cardiovascular morbidity. Thus, nutritional epidemiologists routinely measure and account for total energy intake as a potential confounder with regression techniques (i.e., the residual method or by directly including total energy intake in model adjustment) (27). However, in certain scenarios, total energy intake acts as a mediator on the causal pathway between an exposure and an outcome and thus should not be handled as a confounder due to the risk of overadjustment (28).

To illustrate, associations of sugar-sweetened beverage intake and weight gain might be mediated by total energy intake, in that consumption of sugary beverages can lead to excessive caloric intake that, in turn, can cause weight gain. Further, nutrients might interact with one another in relation to health outcomes. To capture these interactions, nutritional epidemiologists analyze diet patterns, rather than specific foods, to associate food combinations in the diet with disease. These examples highlight the essential role of causal diagrams in diet research as a conceptual framework that guides statistical analysis and collection strategies. A contemporary epidemiologic tool, causal diagrams could also aid investigators in the formulation of research questions, inform collection of sleep measures and other data, guide the statistical analysis, and control for bias. Specifically, causal diagrams allow investigators to assess the role of a sleep variable in the primary study question: a potential confounder (e.g., OSA is an independent predictor of sleep duration and hypertension), a mediator (e.g., sleep quality is on the pathway from shift work to weight gain), or an extraneous source of variation. If sleep confounds the investigated association, adjustment for sleep in the analysis would eliminate the resulting bias. In contrast, if the sleep variable mediates this association, statistical adjustment for this variable will induce overadjustment bias (28). Causal diagrams have been underutilized in epidemiologic sleep research. A recent systematic review (7) has examined the role of diet in the relationship between sleep and cardiometabolic health. This review found numerous studies that considered diet as a confounder rather than a mediator, an analytical strategy that resulted in overadjustment of reported effect estimates.

Validation of sleep instruments

In epidemiologic studies, self-reported instruments are often used because they are quick, affordable, and feasible to administer in large samples. Sleep questionnaires are designed to measure various sleep characteristics, including duration, timing, and sleep quality (e.g., Pittsburgh Sleep Quality Index) (29), or to screen for specific sleep disorders, such as OSA and insomnia. While objective instruments are often used for diagnosis of many sleep disorders, self-reported tools are essential for evaluation of some sleep characteristics (e.g., sleep quality) that cannot be measured otherwise. Further, subjective tools have established associations between sleep and chronic disease (30–32), and despite being prone to systematic and random error, these tools have high sensitivity and moderate to high specificity (33).

Objective assessment of total sleep duration is accomplished during polysomnography (PSG) via electroencephalography that measures sleep onset, awakenings, total sleep time, and sleep stages. While PSG is considered the gold standard for measurement of sleep architecture, its utilization in research settings is often not financially or logistically feasible. Moreover, a night in a sleep laboratory might not be typical of a person’s usual sleep duration, given that bedtimes and wake times are often related to laboratory practices rather than personal routines. In response to these limitations, wearable devices have been designed and optimized to approximate total sleep duration in clinical and research settings. These devices use accelerometry, in addition to temperature and heart rate in some devices, to quantify sleep duration and sleep timing by recording movement as a proxy for sleep according to the individual device’s algorithm. Despite their objective assessment of sleep and wake patterns, these instruments might produce both systematic and random measurement errors in different subpopulations (34, 35). For example, in individuals with insomnia, the use of wearable devices to measure sleep can introduce systematic bias if prolonged inactive wake periods during the night are recorded as sleep time. The resulting overestimation of sleep time could produce systematic misclassification of sleep duration. Further, even in the absence of sleep disorders, occasional inactivity could induce random error when sleep duration is recorded through objective devices.

While wearable devices and questionnaires are useful instruments in sleep research, investigators must interpret them with caution and use validated instruments when available. Yet even “standard” sleep questionnaires are not necessarily validated in all populations, likely due to the dearth of studies that concurrently collected data from questionnaires and gold standard–assessed sleep.

Comparisons of subjective and objective tools to assess sleep duration have demonstrated a moderate correlation between self-reported sleep duration questions (habitual bedtime, wake time on weekdays and weekends), sleep diary, and 7-night actigraphy (36–38). However, these correlations vary by sex, age, and the reported day of the week (36, 38). Associations between sleep duration and self-rated health have been examined across 3 sleep assessment tools (39). For example, a U-shaped relationship has been observed between a single self-reported sleep duration question and self-rated health. Yet the U-shaped association between self-rated health and 3-night sleep logs was significantly attenuated, and it disappeared with actigraphy-based sleep duration (39).

The field of nutritional epidemiology has encountered similar challenges in that dietary questionnaires are not always validated for the specific population under study, and gold standard measurements of diet are often not available. Two of the solutions utilized to address these problems—method of triads and calibration—warrant attention for their potential application to the assessment of sleep duration.

In nutritional epidemiology, the logical parallel to PSG (in which a person’s sleep is directly observed and quantified) is researcher-observed food intake (with weighted portions and known nutrient composition) in a laboratory setting. Yet this assessment strategy is infeasible except in small studies. More commonly, self-reported food intake through questionnaires (food frequency questionnaires, 24-hour recall, or food diaries) is used as a diet measure in nutrition research. Food frequency questionnaires face similar issues as with sleep questionnaires: Both sleep and eating are daily occurrences, but people might find it difficult to recall these behaviors with accuracy and precision. Due to these systematic and random measurement errors, which can vary by population, each study population might need its own validated and culturally appropriate questionnaire. To validate self-reported nutrients without the use of a true gold standard (i.e., direct observation of food intake of known nutrient composition), nutritional epidemiologists might use the method of triads.

The method of triads is used primarily in nutritional epidemiology but has been applied to other fields as well (e.g., validation of self-reported pubertal stages) (40). In nutritional epidemiology, the method of triads assesses the validity of a self-reported nutrient measurement by conducting 3-way comparisons between nutrient data derived from independent assessments (e.g., biomarker, 24-hour recall, and food frequency questionnaire) (41). The primary utility is that measurement errors between self-reported dietary assessments are often correlated (e.g., participants might misremember their intake of a particular food item in the same way on a 24-hour recall as in a food frequency questionnaire), which can lead to a biased estimation of the correlation coefficients of a given nutrient. In contrast, triangulation analyses evaluate the correlations between nutrients assessed with 3 methods simultaneously, and through a series of equations, the correlation coefficients between each measure and the underlying “true value” of a nutrient can be estimated (Figure 2—figure and equations adapted with permission from Kabagambe et al. (42)).

Figure 2.

Representation of the method of triads in nutritional epidemiology. Application of the method of triads to evaluate the performance of food frequency questionnaires and biomarkers as indicators of long-term dietary intake. This figure is reprinted with permission from Kabagambe et al. (42).

In sleep research, the method of triads could be similarly used to assess the validity of sleep duration when PSG measurements (the gold standard) are not available; the 3 independent assessment methods could be questionnaires on typical sleep habits, prospective sleep diaries, and sleep assessed by wearable devices (Figure 3). Although many studies evaluate the correlation between 2 independent measures of sleep duration, using 3 independent measures simultaneously will provide more accurate validity coefficients. If data from all 3 methods are not available on all subjects, it is acceptable to perform the triangulation analysis within a randomly selected subset of the population. Of note, utilizing the method of triads might actually be more feasible in epidemiologic sleep studies than in dietary studies, given that self-reported sleep questions and diaries take considerably less time than dietary recalls or prospective dietary records. Further, there might be some epidemiologic sleep studies that have already collected all 3 types of data and could easily utilize the method of triads (Figure 4). This method might be best suited for assessing the validity of typical sleep duration rather than other aspects of sleep.

Figure 3.

Method of triads adapted for sleep research from Kabagambe et al. (42). Application of the method of triads to evaluate the performance of food frequency questionnaires and biomarkers as indicators of long-term dietary intake. PSG, polysomnography.

Figure 4.

Sample checklist of data for method of triads for sleep research.

Calibration method for correction of systematic errors in sleep duration assessment

Systematic errors (consistent, repeatable errors as opposed to random errors) have often been detected in the assessment of sleep when comparing questionnaire-derived sleep estimates with objective measurements (i.e., PSG or actigraphy).

In comparison with objective amount of sleep, self-reported sleep duration has been over-estimated in adult (43), pediatric (44), pregnant (45), and older (46) populations. In addition, differential misreporting of subjective sleep duration according to the actual amount of sleep obtained has been shown. Short sleepers (≤6 hours per night) tended to underestimate sleep duration, and those who slept >6 hours per night overestimated sleep duration in relation to actigraphy data (36, 47). In contrast to medical-grade devices, such as actigraphy, consumer wearables that estimate heart rate in their sleep assessments have not been well validated against PSG and thus have not been an optimal choice for use in sleep research (48, 49). These wearable devices might have proprietary algorithms that hamper validation efforts (50). However, recent development of mathematical models that incorporate an estimate of circadian phase and utilize raw acceleration and heart rate data from the Apple Watch has shown great promise for the identification of sleep parameters across different device platforms (51). These findings open up the possibility of using consumer-wearable devices to measure sleep on a large scale.

Concurrent sleep disorders, diagnosed or undiagnosed, might also affect self-reported sleep duration in a systematic way (52). Individuals with OSA or insomnia might underestimate sleep duration; in one study, those with insomnia underestimated total sleep time by 81 minutes (53–55). Finally, an overestimate of sleep duration is expected in responses of employees of industries that regulate sleep, such as the transportation industry (56). While these systematic biases are well-described, we believe that few studies have attempted to correct the bias within the collected data, as has been described in the nutritional epidemiology field (57).

Nutritional epidemiology also uses self-reported instruments, and as in sleep research, systematic errors are often present in self-reported dietary instruments. The source of error could be the instrument itself, such as when a food frequency questionnaire neglects to ask about consumption of an important food, which could happen when short food frequency questionnaires are administered or when a population experiences rapid shifts in the food supply. Systematic error can also be the result of social desirability biases. For example, self-reported total energy intake is often underestimated among overweight study participants (58–60). Overestimation of fruit and vegetable intake and underestimation of fat intake have also been shown to differ according to sex, type of dietary instrument used, and dietary-intervention treatment status. While a systematic error with the research instrument can be difficult to correct after data collection, predictable under- or over-reporting by the participants could be addressed by calibration studies (61). In these studies, the following steps are undertaken:

1. Using a subset of the data, measurements of the exposure of interest are taken subjectively (self-report) and objectively (biomarker).

2. A linear regression model is fitted, with the objective marker as the continuous dependent variable and the self-report measure as the continuous independent variable.

3. The β estimate from the linear regression model is applied as a “correction factor” to the self-report measures (i.e., each unit of self-reported nutrient is related to x units of the objectively-assessed nutrient).

Once corrected, the new measurements can be used in data analysis as a comparison with the results obtained without correction.

To identify and correct systematic bias in sleep research, small validation studies with objective measures of sleep could be conducted in conjunction with larger cohorts with self-reported measures. For example, a calibration study could proceed as follows:

1. Collect self-reported sleep duration for 500 children. For 50 of them, conduct actigraphy at the same time.

2. Perform linear regression with objective sleep duration as the outcome and subjective sleep duration as the exposure.

3. Multiply self-reported sleep duration by the β value from linear regression model to compute corrected sleep duration.

Calibration studies might be an effective strategy to correct bias in already-existing large data sets, if validation studies have been conducted within subsets of the data set or in generalizable populations (Figure 5). Indeed, many validation studies of self-reported sleep duration or timing exist within particular populations (62–64), meaning their estimates could be used to calibrate data sets in similar populations.

Figure 5.

Sample checklist of data for calibration method for sleep research. PSG, polysomnography.

FINAL REMARKS

As the field of sleep epidemiology moves forward, we have reviewed and reflected on several methodological challenges it faces in comparison with similar obstacles in nutritional epidemiology. In epidemiologic research, exposures or outcomes are ideally assessed with objective tools, but if such tools are unavailable, self-reported instruments are used. While subjective sleep tools have uncovered the role of poor sleep in morbidity and mortality, they are associated with measurement error. Beyond assessment challenges, analytical errors can stem from an unclear or absent conceptual framework. To minimize these measurement errors and their bias, we propose that investigators pursuing epidemiologic sleep studies borrow methodological approaches used in nutritional epidemiology. Here, we have illustrated the use of causal diagrams, method of triads, and calibration techniques. These methods present a partial view of current research challenges but could inspire future interdisciplinary collaborations among investigators in these fields. For example, self-reported sleep instruments have limited ability to assess sleep physiology, a determinant of healthy aging. This limitation has sparked the development of consumer wearable devices and smartphone applications to mimic the objective sleep assessment of medical devices that are used in sleep laboratories, but methods to analyze these data are constantly evolving. The nutrition field is also working on methods to analyze real-time dietary data from apps.

Thus, as both disciplines advance, assessment and analytical methods developed by sleep researchers could inform nutrition investigators and strengthen collaborative efforts across these disciplines.