Assessing Psychometric Properties of the PROMIS Sleep Disturbance Scale in Older Adults in Independent-Living and Continuing Care Retirement Communities

Abstract

Purpose:

Sleep disturbances are associated with poor health outcomes in older adults. The PROMIS Sleep Disturbance scale was designed to assess self-reported general sleep and sleep disturbance. The objective of this study was to validate the short form PROMIS Sleep Disturbance scale for use among older adults living in independent-living and continuing care retirement communities.

Methods:

Older adults (N=307) were recruited from retirement communities in San Diego, CA to participate in a physical activity intervention. Study participants were on average 83.6 years (SD 6.4) and predominately female (72.3%). Self-reported health outcomes included: sleep disturbance, depressive symptoms, quality of life, stress, and pain. Internal consistency of the PROMIS Sleep Disturbance scale was determined using Cronbach’s α, individual item means, and inter-item correlations. Construct validity was examined using Exploratory Factor Analysis techniques. Adjusted linear regression models assessed the predictive validity of the Sleep Disturbance scale and associations with health outcomes.

Results:

The PROMIS Sleep Disturbance scale had a Cronbach’s α= .856 and an inter-item correlation of .504. All items loaded on one sole factor. Additionally, the sleep scale was significantly predictive of depressive symptoms, stress, and quality of life at 12 months.

Conclusions:

The PROMIS 6-item Sleep Disturbance scale had acceptable internal consistency and strong construct validity among a sample of elderly older adults in an independent-living community setting. These findings suggest that the PROMIS scale may provide an accurate assessment of sleep disturbance in older adults. Additional validation testing using objective measures of sleep are needed to confirm these findings further.

Introduction

Approximately 50–70 million Americans report suffering from sleep disturbances. (1) Evidence demonstrates that sleep disturbances and insufficient sleep are associated with chronic disease, diminished quality of life, and increased health care utilization. (2–5) Older adults commonly have a higher prevalence of both chronic disease and reported sleep disturbances. (6–9) Some evidence suggests the biological need for sleep may decrease with aging and symptoms of sleepiness or consequences of sleep apnea may attenuate with aging. (10–11) As adults age there is an increased likelihood that disordered sleep will exacerbate disease processes. (12) Due to conflicting literature on the prevalence of sleep disturbances in older adults, and evidence linking sleep disturbances to chronic conditions, identifying a simple tool to assess sleep disturbances in older adults may be critical for clinicians and caregivers trying to manage chronic disease.

Assessing sleep disturbance in older populations is crucial given the unique individual features, and the aging of the US population. According to the CDC, by 2030 the population of adults >60 years will be approximately 71 million, more than double what it is today. (13) The need for supportive housing such as continuing care retirement communities, are likely to increase. (14) Providing staff in such settings with simple tools to assess sleep and sleep related impairment is important.

There is an increasing number of self-reported measures to assess sleep quality and the presence of sleep disturbances. The Pittsburgh Sleep Quality Index (PSQI) was developed for use in psychiatric clinics to assess patient sleep complaints and sleep quality over a 1-month period. (15) The 19-item measure assesses 7 domains of sleep (including duration, quality, use of sleep aids, etc) and has been validated in populations varying in age, race-ethnicity, and language. Despite its popularity and wide use by researchers over the past two decades, a short form of the PSQI was just recently developed and has not been validated for use among older adult populations. (16) The 7-item Insomnia Severity Index (ISI) is a brief screening tool, designed to assess the nature and severity of Insomnia. (17) The ISI has proven valid and reliable for use in both clinical and community samples and has been validated for use among older adults. Unfortunately, the use of the ISI is limited by its application of screening solely for insomnia; the brief tool does not assess sleep generally, or the symptoms that may be relevant to other prevalent sleep disorders. (18) Overall existing self-report sleep measures are limited by their specificity to sleep disorders or their limited validation in some study populations, including the aging population and the oldest old. (19) The gold standard for characterizing sleep patterns, polysomnography (PSG), is time intensive and expensive, making its use unrealistic for large scale screening for sleep disorders. (19) Furthermore, PSG is often used for the diagnosis of some sleep disorders after they have already been identified and, according to published guidelines, is not recommended for the assessment of some sleep disorders, including insomnia and disorders that require the assessment of sleep patterns over time. (20, 21) Accurate and brief self-reported sleep measures are necessary to screen for potentially undiagnosed and unidentified sleep problems in multiple healthcare settings. Currently, there is a limited understanding of the prevalence of sleep disturbances in older adults, perhaps due to lack of a valid, simple screening tool.

The Patient-Reported Outcomes Information System (PROMIS) Sleep Disturbance Scale was developed to assess self-reported perceptions of sleep quality, sleep depth, and any perceived difficulties related to getting and staying asleep over a 7-day period. (19) To develop item banks, selected experts conducted literature searches, qualitative interviews, and item pilot testing. Item banks were narrowed to a 27-item scale that was validated on a sample of 1,993 adults between the ages of 35–66 years (Mean age=52 years) recruited from clinical and community settings. (19) The full-scale instrument was narrowed to create an 8-tem short form version. (22) Validated against the 27-item full-scale, the PSQI, and the Epworth Sleepiness Scale (ESS), the short form correlated strongly with the full-scale instrument and had greater measurement precision than both the PSQI and ESS measures. (22) There are four exchangeable variations of the short-form designed to assess similar domains of sleep disturbances that have high reliability and precision. (23)

The PROMIS sleep disturbance scale was developed for use in both clinical and research settings; however, neither the full-scale nor short form version has been validated on a sample of adults over the age of 66 years. The purpose of this analysis was to assess the reliability and construct validity of the 6-item PROMIS Sleep Disturbances scale among elderly older adults using exploratory factor analysis. We assessed the predictive validity of the scale using the follow-up data of health outcomes strongly associated with sleep disturbances. We hypothesized the psychometric qualities of the 6-item PROMIS sleep scale would hold consistent with previous validation results when tested on an older adult sample in a care setting.

Methods

Study Sample:

The Multi-level Intervention for Physical Activity in Retirement Communities (MIPARC) study was a 12-month physical activity intervention for 307 residents over the age of 65 years residing in in independent living and continuing care retirement communities. Intervention evaluation included self-report surveys, objective assessment of physical activity, and physical and cognitive functioning assessments. The study protocol and evaluation are described in greater detail elsewhere. (24) The current analyses used baseline demographic and sleep disturbance data and 12-month follow up health outcome data.

Measures

Sleep Disturbances (PROMIS 6-item Sleep Disturbance Scale):

The PROMIS 6a scale is composed of 6 items developed to assess the domain of sleep disturbances in the past 7 days. The first two scale items assess sleep quality and the following 4 items assess domains of restfulness, sleep problems, and difficulty falling asleep. Items include: “My sleep quality was…”; “My sleep was refreshing…”; “I had a problem with my sleep…”; “I had difficulty falling asleep…”; “My sleep was restless…” and “I tried hard to get to sleep…”. Each item has a 5-point response scale. Response options for the sleep quality item range from: “Very poor (1)” to “Very good (5)” and for remaining items range from “Not at all (1)” to “Very much (5). The two positively phrased items are reverse coded and sum scores are calculated. The raw sum score is then rescaled on the PROMIS score conversion table to determine the standardized T-score. (23) The PROMIS Sleep Disturbance T-score was calibrated on a large sample during development and has a mean of 50 and a standard deviation of 10. A higher standardized T-score represents greater sleep disturbances. However, this short form was originally developed on a population with greater prevalence of chronic illnesses; therefore, the PROMIS scoring manual advises that a score of 50 may be reflective of even more presence of sleep disturbances than is likely in an average healthy population.

Predictive Validity Measures: Self-reported Health Outcomes – (Depression, Stress, Pain, Quality of Life)

The literature demonstrates that sleep disturbances are highly correlated with depression, stress, pain, and reduced quality of life. (25) These theoretically based correlates of sleep disturbances were incorporated into this analysis to assess the predictive validity of the sleep disturbance items and their expected relationships with health conditions previously related to poor sleep. Depression was assessed with the 10-item short form of the Center for Epidemiological Studies Depression Scale (CES D-10). (25) The 10-item scale was developed for screening depressive symptoms and measuring severity of depression. Participants responded to each item on a 4-level Likert scale. Response options range from “rarely or none of the time (less that 1/day) (0)” to “All of the time (3). Higher scale scores (0–30) indicate greater presence of depressive symptoms. Quality of Life was measured using a 12-item modified version of the Perceived Quality of Life Scale (PQOL). (27) Participants responded to each item on a 1 to 5 scale with response options ranging from “extremely unhappy” to “extremely happy”. Final scores were calculated using item mean score. Higher mean scores are indicative of greater perceived quality of life. Stress: participants’ self-perceived stress was assessed with Cohen’s 4 item Global Measure of Perceived Stress Scale. (28) Responses ranged from “never” to “always” on a 1 to 5 scale. Sum scale scores were calculated in which a higher score indicates a higher level of perceived stress. Pain was assessed using the PROMIS 6-item Pain Interference Scale. The scale measures the degree in which pain interferes with participation in activities, both physical and mental. Participants responded on a 5-point scale ranging from “Not at all” (1) to “very much” (5). Consistent with the PROMIS Sleep Disturbance Scale, a T score for the scale is calculated. The mean of the T-scale for Pain Interference is 50.

Participant Demographics

Participant demographics including age, gender, race, education, and marital status were assessed at baseline.

Analysis

To evaluate the reliability of the PROMIS sleep disturbance scale we calculated the scale’s internal consistency. Internal consistency of the scale was based on Cronbach’s α for the whole scale, as well as for the scale minus each individual item (“α if item deleted”). A priori, a standard raw Cronbach’s α=0.70 was considered the minimum value for adequate internal consistency. To evaluate reliability further, we calculated individual item means, raw corrected item-totals, and inter-item correlations.

Construct validity of the PROMIS sleep disturbance scale was examined using exploratory factor analysis. In scale development, both exploratory and confirmatory factor analysis techniques were used to confirm the unidimensionality of the 27-item scale. (19) Due to our unique study sample, that could potentially generate different item functioning, we chose the exploratory factor analysis approach to examine the scale’s ability to assess the latent construct of sleep disturbance. To test the predictive validity of the PROMIS Sleep Disturbance scale with associated health outcomes, adjusted linear regression models were run for each health outcome. Linear regression models included adjustment for age, race, gender, education, and study condition. All data analyses were performed using R version 3.3.1. (29), and the R Pysch package. (30)

Results

In this analysis of 307 older adults, the mean (SD) baseline age of participants was 83.6 years (SD 6.4). The sample was predominantly female (72.3%), white non-Hispanic (92.5%), and had at least some college education (64.7%). Over half of the sample (59.3%) was not married at the time of assessment (Table 1).

Table 1.

Descriptive Characteristics of 307 MIPARC Study Participants

Demographics	N (%)
Age (Mean [SD])	83.62 [6.42]
Gender
Female	222 (72.3%)
Education
Less than college	106 (35.3%)
College and above	194 (64.7%)
Marital Status
Married	123 (40.7%)
Not married	179 (59.3%)
Race/ethnicity
White	284 (92.5%)
Asian	15 (4.9%)
Sleep Disturbances (Mean [SD])	53.13 [4.21]
Depression CES D-10 Score (Mean [SD])	5.51 [4.07]
Quality of Life (PQOL) Score (Mean [SD])	3.93 [0.64]
Perceived Stress Score (Mean [SD])	4.21 [2.52]
PROMIS Pain Scale Score (Mean [SD])	49.60 [7.91]

The mean standardized PROMIS Sleep Disturbance score at baseline was 53.13 (SD 4.21). Participants’ individual mean scores ranged from 46.4–76.1 with 76.6% of the sample having a standardized score over 50. This mean and range are higher than the scores of the sample in the original development of the PROMIS short form scales. Overall, the study sample reported low depression, stress, and pain scores and high quality of life scores (Table 1).

The internal consistency of the 6 items of the PROMIS Sleep Disturbance scale was high (α= .86). The internal consistency of the scale remained unchanged when any one of the items was removed (α ranging from .82–.84). The mean inter-item correlation coefficient for the entire scale was also high (.5) with individual item means ranging from 1.9–2.2 (Table 2). The items with the highest means were the two positively worded items, “My sleep quality was…” and “My sleep was refreshing…”, both with item means of 2.2. The corrected item total correlations ranged from .57–.71.

Table 2.

PROMIS Sleep Disturbance 6-item Scale Internal Consistency Data

Scale Cronbach’s α: .86
Inter-item correlation:.50
	Item Mean [SD]	Corrected Item-Total Correlation	Cronbach’s Alpha if Item Deleted
Quality Sleep	2.2 [.869]	.67	.83
Refreshing	2.2 [.844]	.57	.84
Problem with sleep	2.1[1.02]	.71	.82
Difficulty falling asleep	1.9 [1.11]	.65	.83
Restless	2.0 [1.10]	.65	.83
Tried hard to get to sleep	1.9 [1.13]	.63	.83

Our exploratory factor analysis revealed one component had an eigenvalue greater than 1.0 (component 1 eigenvalue: 3.522) and this component explained 58% of the total variance (Table 3), confirming our hypothesis that all items would adequately load onto one factor. Factor loadings for the items ranged from 0.68 to 0.84 demonstrating that all items contributed almost equally to the one component.

Table 3.

PROMIS Sleep Disturbance 6-item Scale Exploratory Factor Analysis Results

Component 1
Eigenvalue: 3.522
Item	Loading
Quality Sleep	.76
Refreshing	.68
Problem with sleep	.84
Difficulty falling asleep	.78
Restless	.75
Tried hard to get to sleep	.75

Results of the adjusted linear regression models testing the predictive validity of the PROMIS Sleep Disturbance scale are presented in Table 4. After adjustment, sleep disturbance scores were significantly associated with several of the theoretically determined health outcome measures. In this sample, higher sleep disturbance scores at baseline were significantly associated with higher scores on the depression scale (B= .254; p < .001), a higher stress score (B= .116; p=.004), and a lower quality of life score (B= −.026; p = .003) at the 12-month follow-up.

Table 4.

Regression models for prediction of self-reported health outcomes at 12 months

	B+	p value
Depression CES D-10 Score	.254	< .001***
Quality of Life PQOL Score	−.026	.004**
Perceived Stress Score	.116	.003**
PROMIS Pain Interference Score	.095	.494

⁺Models adjusted for baseline age, race, gender, education, and intervention study condition

Discussion

This study aimed to test the reliability and validity of the PROMIS 6-item Sleep Disturbance scale for the first time among a sample of elderly older adults in an independent-living community setting. The older adults in this analysis had slightly higher standardized sleep disturbance scores than the middle-aged adult sample used in scale development. As hypothesized, the PROMIS 6-item Sleep Disturbance scale had acceptable internal consistency, strong construct validity, and was found to be significantly predictive of theoretically meaningful health outcomes previously established in the literature.

Our construct validity results are consistent with the PROMIS full-scale development and short form development results, demonstrating the scale is properly assessing the latent construct of sleep disturbance. In this analysis, all items loaded significantly onto one component confirming our study hypothesis and previous validation results. In addition to the construct validity testing, we performed a test of predictive validity, a new validation step not included in previous analyses. In our predictive validity analysis, we found that sleep disturbance scores were significantly associated with higher levels of depression and stress, and lower perceived quality of life at 12 months, findings consistent with previous literature. (4, 31,32) These associations, corroborate the need to screen for sleep disturbances, and the need to explore sleep as a mediator of other physical and mental health outcomes prevalent in older adults.

Due to health implications of chronic sleep disturbances and the unclear prevalence of sleep disturbances in older adult populations, validation analyses like this one are important to determine if current self-report measures of sleep are appropriate and accurate for identifying sleep disturbances in older adults. While the sleep disturbance scores in our sample were only slightly higher than the original sample mean, our results support further investigation of unidentified sleep disturbances in older adults. This is supported even more so when considering our sample was made up of relatively healthy older adults interested in participating in a physical activity intervention. With a growing older adult population, and the high prevalence of chronic disease and sleep disturbances in this portion of the population, older adults are a unique population for future sleep health research. Properly assessing sleep disturbances in older adults may be critical to both improving sleep and chronic disease management. (12) Previous research suggests that, for older adults, improvements in sleep is related to better management of pain, (33) reduced inflammatory risk related to chronic disease management (34), and improved mental health. (35) Our predictive validity results suggest that researchers interested in other health outcomes related to chronic disease management in older adults should consider the inclusion of this PROMIS short form in health assessments for identifying sleep disturbances as potential moderators or mediators. Furthermore, increased use of PROMIS short form scale in older adult populations may lead to identifying individuals with chronic sleep disturbances or at risk for sleep disorders who may otherwise go undiagnosed or untreated.

This study was limited by the use of only baseline cross-sectional data. The parent study was a physical activity intervention and sleep was not a primary outcome, therefore a clinical diagnostic assessment of sleep disturbances was not included nor were any additional concurrent sleep measures. Without another subjective or objective measure of sleep we were not able to compare the performance of the PROMIS sleep disturbance scores to an existing validated measure of sleep, limiting our ability to further test the validity of the scale. Additionally, the study sample consisted of primarily white female older adults, resulting in a homogeneous sample in both race and gender, limiting the generalizability of study results. While the age range of this study sample was exceptional for validation testing completed in this analysis, including the oldest group of older adults, study participants were healthy individuals interested in participating in a physical activity intervention or healthy aging condition and therefore were potentially healthier than the average older adult in continuing care and independent-living retirement community settings.

Future validation studies, ideally including the use of concurrent objective measures of sleep, are needed to examine further the validity of the PROMIS 6-item sleep disturbance scale. Previous research tested the measurement precision of the PROMIS 6-item scale against the widely accepted Pittsburgh Sleep Quality Index and the Epworth Sleepiness Scale and found promising results. (22) These efforts should be replicated in a large sample of older adults, including individuals from the oldest old age group.