Sleep Health as Measured by RU SATED: A Psychometric Evaluation

Abstract

Objective/Background:

Sleep health is a multidimensional construct of sleep and wakefulness which operationalizes optimal sleep as more than the absence disease. Despite its importance to public health promotion efforts, empirical research examining sleep health is currently limited, possibly due to the lack of empirically validated measures. Therefore, the purpose of the current study was to evaluate the psychometric properties of a previously proposed six-item sleep health scale (RU- SATED).

Participants:

3401 adults (Mean Age = 42.77, 47.8% female) completed an online survey of sleep and health.

Methods:

Participants completed the RU-SATED scale, as well as other sleep-related measures including the Insomnia Severity Index (ISI) and the Sleep-Self-Efficacy Scale (SSE).

Results:

An exploratory factor analysis (EFA) revealed a two-factor structure. A confirmatory factor analysis (CFA) using this two-factor structure demonstrated adequate to good model fit indices (X² = 45.96, df = 8, p < .01; RMSEA = .04; CFI= .98; NFI = .98; TLI = .97). Cronbach’s α was .64 and the average inter-item correlation was .22. RU-SATED was negatively associated with insomnia severity and positively associated with both self-reported sleep and sleep self-efficacy.

Conclusions:

RU-SATED appears to be a valid instrument for the assessment of sleep health among adults that is related to, but distinct from, other established sleep constructs. Future research may benefit from examining the test-retest reliability of the measure and assessing the predictive validity of sleep health as it relates to health-related outcomes.

Most individuals are interested in achieving optimal sleep; yet, the field of sleep medicine has historically been focused on identifying and treating sleep disorders (Shepard et al., 2005). Just as psychological well-being is not exclusively defined by the absence of a mental health disorder, sleep health is more than the absence of a sleep disorder or short sleep duration. The concept of sleep health has emerged as a positive framework through which to view individuals’ sleep. Although no universal definition of sleep health exists, it has been previously defined as “a multidimensional pattern of sleep-wakefulness, adapted to individual, social, and environmental demands, that promotes physical and mental well-being” (Buysse, 2014).

Assessing and promoting better sleep health has several proposed advantages. First, by recognizing that individuals’ sleep exists on a continuum, the concept of sleep health enables all individuals’ sleep to be quantified and modified. Secondly, the sleep health framework avoids dichotomizing individuals’ sleep into healthy and unhealthy by capturing graduations in sleep. Finally, identifying and measuring sleep health instead of only focusing on sleep disorders can assist with health education and promotion efforts, allowing for earlier interventions to prevent the adverse downstream effects of sub-optimal sleep.

Despite the aforementioned benefits associated with the measurement of sleep health, validated measures of this construct are lacking. The National Sleep Foundation (NSF) developed a 12-item measure of sleep health designed to capture three dimensions of sleep: duration, quality, and disorder (Knutson et al., 2017). However, the original conceptual model of sleep health (RU-SATED) proposed by Buysee (2014) predates the measure developed by the NSF and proposes six dimensions: sleep regularity, subjective satisfaction, appropriate timing, adequate duration, high sleep efficiency, and sustained alertness during the day. These six dimensions were based on an extensive literature review which found that each of these six factors were significantly associated with a range of adverse health outcomes (Buysee, 2014). The added dimensional sleep components included in RU-SATED, as well as the focus on wellness instead of illness, provides a comprehensive framework for examining sleep health.

A translated version of the RU-SATED sleep health scale has been psychometrically assessed among a sample of community-dwelling Portuguese adults, with results suggesting that the measure is valid and reliable for the assessment of several sleep health indicators (Becker, Martins, Jesus, Chiodelli, & Stephen Rieber, 2018). However, the psychometric properties and clinical utility of this scale remain unexamined among English-speaking adults. Given the aforementioned benefits associated with assessing sleep health, the availability of a brief and psychometrically sound measure of sleep health would be useful for both clinical and research purposes.

The purpose of the present study was to evaluate the psychometric properties of the RU-SATED sleep health scale among a sample of English-speaking individuals across the lifespan. Specifically, the study sought to examine the factor structure, internal consistency, and concurrent validity of the measure.

Methods

Participants

The present study analyzed data from a larger online study examining sleep and health across the lifespan. Specifically, participants were recruited via Amazon’s Mechanical Turk (MTurk), an online platform where individuals are compensated for the completion of a variety of tasks and services. Research examining data collected via MTurk found that participants using this platform tend to be more demographically diverse than standard internet samples and that the data obtained are as reliable as those acquired via traditional methods (Buhrmester, Kwang, & Gosling, 2011). Additionally, other self-reported sleep measures completed online and in-person have been found to have similar psychometric properties (Thorndike et al., 2011), suggesting that online data collection can reasonably assess the properties of a sleep health scale completed online.

Inclusion and exclusion criteria for the larger study were minimal and were based solely on age and gender with the overall goal of recruiting an equal number of men and women living in the United States across the lifespan. The study was approved by the Institutional Review Board at Virginia Commonwealth University. All participants who completed the sleep health measure were included in the present study. Following the completion of informed consent, participants completed measures associated with their sleep and physical and emotional well-being. Two validity checks were implemented to decrease threats to validity associated with online data collection. The first was an instructional manipulation check asking participants to respond to an item by selecting a specific response. The second was a consistency check which compared participants’ responses to a question about their age at the beginning of the survey to a question about their birthdate at the end of the survey. Participants who failed the instructional validity check (N = 347), as well as those who failed the consistency check (N = 59) were removed, leaving 89.33% of the original sample.

Measures

Sleep health.

Sleep health was measured using the RU-SATED scale (Buysse, 2014). The measure consists of six items each assessing one aspect of sleep heath: sleep regularity (bedtime and waketime occurring at the same time, within one hour, every day), satisfaction, timing (asleep or trying to sleep between 2:00 am and 4:00 am), duration (7 to 9 hours per day), efficiency (less than 30 minutes wake time), and alertness during the day (awake all day without dozing). Items are each rated on three-point Likert scale from 0 (Rarely / Never) to 2 (Usually / Always), with higher scores indicating better sleep health.

Sleep disturbance.

Participants completed a single visual analog item rating their sleep in the past two weeks from 0 (Worst Sleep Imaginable) to 100 (Best Sleep Ever). Insomnia symptoms were measured via the Insomnia Severity Index (ISI), a seven-item scale rated on a four-point Likert scale (Bastien, Vallières, & Morin, 2001). Scores on the ISI range from 0 to 28, with higher scores indicating greater levels of insomnia severity. The ISI is a widely used scale which has been deemed a valid and reliable measure of insomnia symptoms among both community and clinical-based samples (Bastien et al., 2001; Morin, Belleville, Bélanger, & Ivers, 2011).

Sleep self-efficacy.

Participants’ rated their confidence in performing various behaviors necessary to obtain sleep using the Sleep Self-Efficacy Scale (SSE) (Lacks, 1987). This scale is comprised of nine items rated on a five-point Likert scale from 1 to 5, with higher ratings reflecting greater confidence. Higher scores on the SSE have been associated with lower scores on the Pittsburg Sleep Quality Index, as well as higher sleep efficiency and total sleep time measured via both sleep diaries and actigraphy (Currie, Wilson, & Curran, 2002; Edinger & Sampson, 2003; Edinger, Wohlgemuth, Radtke, Coffman, & Carney, 2007).

Data Analyses

Data analyses were conducted using SPSS v.24 and AMOS v.23 (Arbuckle, 2014). In order to assess the suitability of the present sample for factor analysis, Kaiser-Meyer-Olkin (KMO) and Bartlett’s tests were performed. The unique factor structure associated with RU-SATED was examined using an exploratory factor analysis (EFA) with principal axis factoring and a Promax rotation assuming no a priori factor structure. Next, a confirmatory factor analysis (CFA) was performed on the data specifying the factor structure that emerged via the EFA. In order to determine whether the CFA differed between males and females, an invariance design was run as a function of gender. Outliers were assessed via the square distance of Malahanobis (D²), and normality was assessed via the univariate and multivariate coefficients of skewness and kurtosis. Missing data was accounted for using full information likelihood estimation. Internal consistency was evaluated using Cronbach’s α and item-total correlations. Finally, in order to examine the scale’s concurrent validity, sleep health was compared to participants’ self-rated sleep, as well as their ISI and SSE total scores.

Results

Descriptive Results

Participants included 3401 adults ranging from 19 to 99 years old (M = 42.77, SD = 16.79). Individuals were predominately White and college educated. Sleep health was within the average range (M = 7.58, SD = 2.68). A frequency histogram of participants’ RU SATED scores is displayed in Figure 1. A sizable portion of individuals reported difficulties with their sleep, with 29.7% reporting insomnia symptoms in the sub-threshold range (8–14 range) and 20% reporting insomnia symptoms in the clinical range (15–28 range) using the standard cutoffs for the ISI (Bastien et al., 2001). Additional descriptive information is presented in Table 1.

Figure 1.

Frequency histogram of participant’s total sleep health score.

Table 1.

Participant Demographic and Sleep Characteristics

	Mean (Std. Deviation)	Range
Agea	42.77 (16.79)	19 – 99
Sex (% Female)	47.80	--
Race (% White)	80.20	--
Education (% College Educated)	63.60	--
RU-SATED	7.58 (2.68)	0 – 12
Self-Rated Sleep	51.98 (22.06)	0 – 100
Insomnia Severity Index	8.51 (6.42)	0 – 28
Sleep Self-Efficacy	29.33 (8.13)	9 – 45

Note.

^aUnit of measurement in years.

Females reported more insomnia symptoms, as well as lower sleep self-efficacy compared to males. However, no gender differences in sleep health were found. In contrast, racial and ethnic minorities reported significantly poorer sleep health but no differences in insomnia symptoms or sleep self-efficacy. Gender and racial differences across sleep measures are reported in Table 2.

Table 2.

Gender and Racial Differences in Self-Reported Sleep

	White		Racial/Ethnic Minority
	M	SD	M	SD	t-test
RU SATED	7.74	2.66	6.93	2.69	−7.07**
Self-Rated Sleep	51.52	21.99	53.65	22.27	2.23*
Insomnia Severity Index	8.49	6.42	8.61	6.48	0.43
Sleep Self-Efficacy	29.27	8.09	29.57	8.32	0.86
	Male		Female
	M	SD	M	SD	t-test
RU SATED	7.69	2.66	7.67	2.68	0.21
Self-Rated Sleep	53.50	21.39	51.82	22.48	2.16*
Insomnia Severity Index	7.82	6.15	8.70	6.51	−3.91**
Sleep Self-Efficacy	30.47	7.88	28.77	8.25	5.94**

^*p < .05 ,

^**p < .001

Note. M = Mean, SD = Standard Deviation.

Exploratory Factor Analysis

KMO (.717) and Barlett (X² = 2363.31; df = 15, p < .001) tests were performed with results suggesting that the data are suitable for factor analysis (Tabachnick & Fidell, 2001). The EFA revealed a scree plot with a pronounced inflection point at the second-highest eigenvalue. Specifically, the first two factors accounted for 53.35% of the items’ cumulative variance, with 35.70% accounted by factor one and 17.38% attributable to factor two.

In accordance with previously established guidelines, an item was chosen to load onto a specific factor if it achieved simple structure, which was defined as the highest loading eigenvalue exceeding an absolute value of .30, with all cross-loadings being at least .15 less than the item’s highest factor loading (DeVellis, 1991; Worthington & Whittaker, 2006). Item loadings for the two factors that emerged from the EFA are presented in Table 3. Items pertaining to factor one (sleep satisfaction, efficiency, and duration) may best be described as “sleep quality and quantity.” Items associated with factor two (sleep regularity, timing, and alertness) may best be described as “circadian rhythm.”

Table 3.

Item Loadings for the Exploratory Factor Analysis

	Factor
Item	1	2
Satisfaction (Q2)	0.851	−0.065
Duration (Q6)	0.590	0.012
Efficiency (Q5)	0.413	0.160
Timing (Q4)	−0.052	0.477
Regularity (Q1)	0.067	0.455
Alertness (Q3)	0.068	0.356

Note. RU-SATED question number is presented in parentheses.

Confirmatory Factor Analysis

A CFA was completed on the two-factor solution that emerged via the EFA (Figure 2). The two latent constructs were significantly correlated (r = .59, p < .001). Model fit was evaluated using previously established guidelines (Meyers, Gamst, & Guarino, 2016). The Chi-squared goodness-of-fit test failed to provide initial evidence that the two-factor model fit the data well, X² (8) = 45.93 p < .01, because the ratio of the Chi-squared statistic to the degrees of freedom in the model was higher than the conventional critical ratio cutoff of 2.0. However, the Chi-squared test is known to be very sensitive to large sample sizes, suggesting that it is probably not a good estimate of fit for our sample (Schermelleh-Engel, Moosbrugger, & Müller, 2003).

Figure 2.

Two-factor model indicated by confirmatory factor analysis with standardized path coefficients between the latent factors (sleep quality and quantity and circadian rhythm) and the six RU-SATED items. Numbers located outside of the rectangles represent the squared multiple correlation of the item.

In contrast, the normed fit index (NFI) was .98, where values of .95 or higher indicate good fit. Similarly, the incremental fit index (IFI), Tucker-Lewis index (TLI), and comparative fit index (CFI) were .98, .97, and .98, respectively, where values close to 1.0 indicate good fit and values above .90 indicate adequate fit. The current model produced a root mean square error of approximation (RMSEA) of .04, where an RMSEA of .05 or less indicates good fit. Taken together, the fit indices were all in the good range and suggest that the two-factor model of sleep health adequately fits the data.

Gender Invariance Analyses

Invariance analyses evaluate the difference between an unconstrained model, which assumes that the groups are yielding different values of the parameters when the model is applied to the data, and a set of constrained models, which assume that the groups are yielding equivalent values of given sets of parameters when the model is applied to the data. When contrasted to the unconstrained model, the measurement residual model yielded a statistically significant chi-square difference test value, χ² (19) = 81.54, p < .001, suggesting that males and females differed in the magnitude of the error terms of the items included in the model. Bonferroni-corrected post-hoc comparisons revealed that only the error terms for item number one (i.e., sleep regularity) and number four (i.e., sleep timing) differed across the two groups with z-scores of −2.60 and 3.90 respectively. Thus, despite this small caveat, the CFA generally showed overall strong invariance across gender.

Internal Consistency

Cronbach’s α and item-total correlations were used to assess the internal consistency of the RU-SATED scale. Cronbach’s α coefficient for the scale was .64, suggesting sub-optimal internal consistency. In contrast, the average inter-item correlation was .22 with past research indicating that average inter-item correlation should fall between .15 and .5 (Clark & Watson, 1995). Item-total correlations were moderate, ranging from .29 to .50.

Concurrent Validity

Inter-correlations of the RU-SATED scale with participants’ self-rated sleep, as well as their scores on the ISI and the SSE were examined to establish concurrent validity. There was a significant negative association between RU-SATED and participants’ total ISI score (r = −.66, p < .001) and a significant positive association between RU-SATED and both self-rated sleep (r = .54, p < .001) and SSE (r = .62 p < .001). Table 4 presents a correlation matrix of the study’s sleep measures.

Table 4.

Correlation Matrix Among Sleep Variables

Measure	ISI	RU-SATED	SSE	Self-Rated Sleep
1. ISI	—
2. RU-SATED	−.66**	—
3. SSE	−.73**	.62**	—
4. Self-Rated Sleep	−.68**	.54**	.62**	—

Note. ISI = Insomnia Severity Index, SSE = Sleep Self-Efficacy.

^**p < .01

Discussion

The current study evaluated the psychometric properties of the RU-SATED scale and provides initial evidence for its use in an American sample. Specifically, a two-factor structure emerged from the EFA, “sleep quality and quantity” and “circadian rhythm”, which accounted for a significant amount of variance in the data. Model fit indices produced by a subsequent CFA indicated that the two-factor structure fit the data well. This factor structure was also largely invariant across gender. Finally, the scale demonstrated adequate convergent validity and reliability.

The psychometric properties observed in the current study differ in some important ways from a validation study of RU-SATED conducted on Portuguese participants (Brandolim Becker et al., 2018). Although evidence from the current study suggest that a two-factor model best explains the data, the previous study found evidence supporting one-factor model, as well as support for the removal of the item pertaining to sleep efficiency (Brandolim Becker et al., 2018). One possible explanation for these discrepancies may be the existence of cultural differences in sleep health, with sleep efficiency being of greater importance to the construct of sleep health in American than Portuguese culture.

The two-factor model of sleep health that emerged largely coincides with historical understandings of sleep, such as the two-process model of sleep (Borbély, 1982). Specifically, sleep regularity, timing, and alertness may be viewed as relating to the circadian process (i.e., Process C), while sleep duration, efficiency, and satisfaction may be conceptualized as relating to the sleep-wake homeostasis (i.e., Process S).

Yet another difference between the present study and the previous psychometric examination of RU-SATED is reliability of the measure. Although the present study found that the average inter-item correlation was within the normal range as established by previous literature, Cronbach’s α of .64 was considerably lower than the .85 found in the Portuguese sample (Becker et al., 2018). One potential explanation for this finding is the small number of items included in the scale which is known to reduce Cronbach’s α. A second explanation for the lower internal consistency values is the multifaceted nature of sleep health and the difficulty associated with assessing each of its relevant components. The lower Cronbach’s α score is likely reflective of the multiple dimensions of sleep health identified via the EFA, suggesting that a lower internal consistency score does not undermine the psychometric validity of the RU SATED scale. Rather, a sub-optimal internal consistency may be expected given the multiple dimensions comprising sleep health. Nonetheless, the acceptable inter-item and item-total reliability indicates consistency among the items, likely reflecting the significant correlation of the two latent constructs. Overall, reliability estimates ranged from sub-optimal to moderate. Considering the dimensionality of sleep health, and the sensitivity of Cronbach’s α to the number of items, the present findings support positive psychometric validation of the reliability of the scale.

Taken as a whole, the psychometric properties of RU-SATED in the present study suggest that it is an adequate measure of sleep health for use among English-speaking participants. Although existing studies have differed in their methods for measuring sleep health, findings consistently indicate that suboptimal sleep health is associated with a range of physical and mental health outcomes (Dong, Martinez, Buysse, & Harvey, 2019; Furihata et al., 2017). Thus, the availability of a brief and psychometrically sound measure of sleep health may be useful for interventions which seek to prevent the onset of sleep disorders.

Group differences in RU-SATED highlight the scale’s potential usefulness. For example, our finding that racial and ethnic minorities report better self-rated sleep but lower levels of sleep health suggest that the presence of sub-optimal sleep is likely to go unnoticed or underreported among these groups. These findings align with past research suggesting that racial minorities report shorter sleep durations and poorer sleep quality but may not differ in terms of insomnia prevalence (Kaufmann et al., 2016; Petrov & Lichstein, 2016). Similarly, while the lack of gender differences regarding sleep health initially seems to contradict with women’s higher levels of insomnia symptoms and poorer self-rated sleep, these findings also coincide with recent research. Women tend to report better sleep quality and longer sleep duration despite being more prone to developing insomnia (Krishnan & Collop, 2006).

Limitations and Future Directions

It is important to note a few potential limitations of the present study. Due to the nature of the cross-sectional study design, we were not able to examine the test-retest reliability of RU-SATED. Examining the test-retest reliability of the RU-SATED scale via a repeated measures longitudinal design would allow us to assess whether sleep health is a relatively stable trait or one subject to rapid changes over time. Such a study could also allow for the measurement of the sensitivity of RU-SATED to changes in sleep health as a result of aging, an intervention, or some other cause.

Furthermore, although we did assess the concurrent validity of the RU-SATED scale by comparing scores to instruments that assess other sleep related constructs, such as the ISI and SSE, future studies should assess individuals’ scores on the RU-SATED as compared to additional well-established measures of sleep, such as the PROMIS sleep questionnaire, to further establish sleep health as a unique construct.

The relative lack of racial and ethnic minorities did not allow for the examination in whether RU SATED differed for members of different racial and ethnic groups. In light of findings that racial and ethnic minorities are more prone to poor sleep (Chen et al., 2015), future research may benefit from examining the psychometric properties of the scale among different racial minority groups.

Relatedly, comparing RU-SATED scores across differing populations such as healthy sleepers and those diagnosed with insomnia could help to establish known groups validity that would provide further psychometric support for the measure. Also, use of alternative analytic techniques such as item response theory may provide more information about the relative value of the different dimensions of sleep health assessed by the scale. Additionally, although RU-SATED does not assess for risk of actual sleep disorders, future studies could examine the validity of RU-SATED in predicting development of sleep disturbance through a variety of sleep health trajectories over time. If the RU-SATED scale is found to have good predictive validity, the scale could be used to detect early decline in sleep health and allow for interventions to prevent development of clinically significant sleep disturbances.

Lastly, although the specific sleep thresholds measured by the scale (e.g., sleep duration of 7 to 9 hours) may be recommended for specific age groups and within certain populations (Hirshkowitz et al., 2015), they may not be optimal for all groups. Consequently, the generalizability of the RU-SATED is limited to those populations that would benefit from these recommendations (e.g., adults).

Conclusions

The present study extends our knowledge of the factors that comprise sleep health. RU-SATED appears to be a valid instrument for the assessment of sleep health among American adults that is related to, but distinct from, other established sleep constructs.