Abstract
Background
The Patient-Reported Outcomes Measurement Information System (PROMIS) includes two instruments to quantify sleep symptoms (sleep disturbance [SDA] and sleep-related impairment [SRI]) in diverse populations across a wide symptom spectrum. However, the responsiveness of PROMIS measures to treatment of sleep disorders is unknown. We examined the responsiveness of the PROMIS sleep scales to the treatment of OSA.
Methods
We collected SDA, SRI, and Epworth Sleepiness Scale (ESS) before and after initiation of positive airway pressure (PAP) in patients with type 2 diabetes newly diagnosed with OSA. To compare responsiveness, we compared effect sizes and classifications of symptom improvement using both the reliable change method and thresholds of minimum important difference (MID).
Results
A total of 103 patients completed assessments pre- and post-PAP. SDA, SRI, and ESS scores all declined significantly with PAP therapy. We observed the largest effect size for SDA (−0.64; 95% CI, −0.86 to −0.42), followed by SRI (−0.43; 95% CI, −0.63 to −0.23), and ESS (−0.28; 95% CI, −0.42 to −0.15). More patients experienced the reliable change category of symptom remission categorized by the PROMIS measures (SDA: 23.3%; SRI: 31.1%) relative to the ESS (5.8%) (P < .001 for both). Using the MID, SDA and SRI also classified more patients as improved (SDA: 54.4%; SRI: 49.5%) relative to the ESS (35.0%) (P < .001 for both pairwise comparisons).
Conclusions
PROMIS sleep measures were more likely than the ESS to detect an improvement with PAP therapy. Incorporating PROMIS measures into research and clinical care may provide a more sensitive assessment of symptomatic response to OSA treatment.
Key Words: measure responsiveness, OSA, patient-reported outcomes, sleep-related impairment, sleepiness
Abbreviations: ESS, Epworth Sleepiness Scale; MID, minimum important difference; PAP, positive airway pressure; PROMIS, Patient-Reported Outcomes Measurement Information System; SDA, sleep disturbance; SRI, sleep-related impairment
Responsive patient-reported outcome instruments are critical to the evaluation of therapies intended to improve symptoms. OSA is a common disorder that disrupts sleep and often results in excessive daytime sleepiness.1 The patient-reported outcome instrument most commonly used in evaluating OSA therapies in both clinical and research settings is the Epworth Sleepiness Scale (ESS).2 The ESS assesses a patient’s likelihood of dozing in eight different scenarios. Although widely adopted, this instrument has been criticized for focusing on activities the individual may not encounter (eg, sitting quietly after a lunch without alcohol) and not capturing nighttime symptoms of sleep disruption common in OSA.3,4 Evidence suggests the ESS may have poor reproducibility when administered in routine clinical settings.5
The Patient-Reported Outcomes Measurement Information System (PROMIS) is a set of patient-reported outcomes that are efficient, flexible, precise, and appropriate in a wide variety of chronic diseases and conditions. The PROMIS measures can be administered using computerized adaptive testing, and have been validated with classic test theory and item response theory methods.6 Two of these instruments are devoted to sleep symptoms, measuring sleep disturbance (SDA) and sleep-related impairment (SRI).3 SDA assesses symptoms of sleep disruption, whereas SRI assesses waking impairments because of poor sleep. Although not developed for any specific sleep disorder, these domains address the sleep and waking symptoms that may be caused by OSA. These measures have greater precision for assessing sleep symptom severity than legacy instruments, such as the ESS,7 and have internal consistency and construct validity among populations enriched for sleep disorders.3,8 However, the ability of these novel measures to detect treatment-related changes among those with sleep disorders is unknown. As therapies for sleep disorders continue to evolve, it is important that we use measures capable of capturing symptom changes from treatment. Such measures are necessary to determine the efficacy of new treatment options and to compare the effectiveness of different treatment options from a patient-centered framework. Standardized metrics that accurately reflect changes in patient symptoms are also important to help physicians and patients decide whether to continue or alter treatment plans. Our goal in this analysis was to compare the magnitude of change in the ESS and the two PROMIS instruments among patients with type 2 diabetes mellitus and OSA receiving positive airway pressure (PAP) therapy, and to compare the degree to which each instrument identifies clinically meaningful changes in OSA symptoms.
Methods
We analyzed data from a clinical program designed to identify and treat patients with type 2 diabetes for OSA. The rationale, methods, and primary results of the program are described in detail elsewhere.9 Briefly, we assessed OSA symptoms among all patients with type 2 diabetes seen in a large, urban academic primary care clinic. Those at increased risk for OSA based on telephone screening were referred for diagnostic sleep testing. Of those who completed testing and were diagnosed with OSA, we administered surveys at the time of their sleep clinic visits prior to initiation of PAP therapy and at routine clinical follow-up after PAP initiation (median of 154 days after initial clinic visit and a median of 130 days after PAP initiation). At each visit, patients completed the eight-item PROMIS SDA Assessment, the eight-item PROMIS SRI Assessment, and the eight-item ESS. We converted raw SDA and SRI scores to T scores.7 We used electronic medical record data to characterize patient demographics and comorbidities, and wirelessly transmitted data automatically collected from patient’s PAP units to capture PAP adherence. The Beth Israel Deaconess Medical Center institutional review board approved our protocol with a waiver for informed consent (protocol No. 2013P000180).
To assess convergent validity, we calculated Pearson correlation coefficients between SDA, SRI, and ESS scores both prior to PAP and after PAP therapy. We defined significant correlations of 0 to 0.09 to be negligible, 0.10 to 0.39 to be weak, 0.40 to 0.59 to be moderate, and 0.60 to 1.0 to be strong.10
We compared responsiveness of ESS and PROMIS measures to PAP therapy using effect size and compared categorical assessments of values to define recovery. To calculate the effect size of each patient-reported outcome, we divided the change in each measure before and after institution of PAP therapy by the baseline SD. We then calculated 95% CIs for effect size estimates using bootstrapping simulations with 1,000 repetitions. Within each measure, we also compared baseline and follow-up measures using paired t tests. To assess categorical responses to therapy, we classified the magnitude of response using both the reliable change method and a dichotomization of response based on thresholds of minimum important difference (MID).11,12 The reliable change method seeks to identify changes that both exceed differences attributable to measurement error alone and that cross a threshold between clinical and normal samples. We defined this threshold using the mean and SD of the clinical population and established normal values using the method used by Jacobson and Truax11 and Bauer et al.13 For the PROMIS measures, we used the established normal T score value of 50 with an SD of 10,6 and used a normal ESS mean of 5.9 and SD of 2.2.4 We considered using clinical cutoffs (eg, ESS score > 10), but established clinical cutoffs have not yet been established for the PROMIS sleep measures. Furthermore, the sample was identified as part of a screening program rather than a referral program. Therefore, patients would not be expected to have symptoms as severe as previously referenced clinical samples. To calculate reliable change for each measure, we used the established formula of 1.96 × √(2 × [baseline SE]),2 with baseline SE defined as baseline SD × √(1 − reliability of measure).11 We calculated baseline interitem Cronbach alpha for the ESS to define its reliability. Because PROMIS instruments were created using item response theory, Cronbach alpha is not an appropriate measure of reliability. Instead, we calculated reliability for the PROMIS measures using the formula 1 − (SE2/SD2).14 We used the median SE reported for each participant’s PROMIS T score, and calculated the baseline SD of the T score for the population.3 Using the reliable change index method, we defined four mutually exclusive categories: worsening (score worsened ≥ reliable change), no change (|score change| < reliable change), improvement (score improved ≥ reliable change, but posttreatment score is within the abnormal range), and symptom remission (score improved ≥ reliable change index and posttreatment score is within the normative range). In addition to reliable change methods, we also categorized response using the MID defined as one-half SD for each measure at baseline in the sample.12 Individuals were then classified into three mutually exclusive categories: worsening (score worsened ≥ MID), no change (|score change| < MID), and improvement (score improved ≥ MID).
We used χ2 tests to compare the proportion of individuals assigned to categories of change between instruments both overall and among categories of PAP adherence. Within instruments, we also performed tests of trend across PAP adherence categories. We compared agreement in change category assignment between measures using kappa with linear weighting, and defined 0.10 to 0.20 to be slight, 0.21 to 0.40 to be fair, 0.41 to 0.60 to be moderate, and 0.61 to 1.0 to be substantial.15 All analyses were performed using Stata (StataCorp LLC).
Results
Of the 128 patients who were diagnosed with OSA and started PAP, 103 (80%) returned to clinic and completed follow-up questionnaires. Their clinical characteristics (Table 1) describe a late middle-aged (mean, 61.5 ± 10.7 years) and obese cohort (mean BMI, 33.7 ± 6.3 kg/m2), which was evenly split between mild (n = 51, 49.5%) and moderate to severe OSA (n = 52, 50.5%). These 103 patients returning to clinic did not substantially differ from the 25 patients who did not return for follow-up and were not included in our analysis (e-Table 1).
Table 1.
Baseline Sample Characteristics
| Characteristic | Total Sample (N = 103) | Average Nightly PAP Use |
|||
|---|---|---|---|---|---|
| 0.0-1.9 h (n = 25) | 2.0-3.9 h (n = 28) | 4.0-5.9 h (n = 28) | ≥ 6.0 h (n = 22) | ||
| Age, y | 61.5 ± 10.7 | 60.3 ± 12.3 | 60.8 ± 9.6 | 62.9 ± 9.7 | 61.9 ± 11.9 |
| Female | 48 (46.6) | 11 (44.0) | 15 (53.6) | 16 (57.1) | 6 (27.3) |
| Race | |||||
| White | 42 (40.8) | 8 (32.0) | 11 (39.3) | 11 (39.3) | 12 (54.6) |
| Black | 50 (48.5) | 14 (56.0) | 15 (53.6) | 13 (46.4) | 8 (36.4) |
| Other | 11 (10.7) | 3 (12.0) | 2 (7.1) | 4 (14.3) | 2 (9.1) |
| BMI, kg/m2 | 33.7 ± 6.3 | 33.0 ± 5.5 | 33.2 ± 7.2 | 34.0 ± 5.8 | 34.9 ± 6.8 |
| Charlson Comorbidity Index score | 3.5 ± 2.3 | 4.0 ± 3.0 | 4.0 ± 2.6 | 3.0 ± 1.8 | 3.0 ± 1.7 |
| HbA1c, % | 7.7 ± 1.5 | 7.9 ± 1.7 | 8.0 ± 1.8 | 7.7 ± 1.3 | 7.1 ± 0.9 |
| Presenting symptom(s)a | |||||
| Loud snoring | 82 (80.4) | 19 (76.0) | 22 (78.6) | 23 (82.1) | 18 (85.7) |
| Tired | 90 (89.1) | 24 (96.0) | 26 (92.9) | 22 (81.5) | 18 (85.7) |
| Witnessed apneas | 23 (22.8) | 5 (20.0) | 9 (32.1) | 2 (7.4) | 7 (33.3) |
| OSA category | |||||
| Mild | 51 (49.5) | 12 (48.0) | 17 (60.7) | 16 (57.1) | 6 (27.3) |
| Moderate | 23 (22.3) | 6 (24.0) | 5 (17.9) | 4 (14.3) | 8 (36.4) |
| Severe | 29 (28.2) | 7 (28.0) | 6 (21.4) | 8 (28.6) | 8 (36.4) |
| AHI, events/h | 24.3 ± 21.3 | 21.7 ± 15.1 | 24.4 ± 26.7 | 23.8 ± 22.7 | 27.7 ± 18.5 |
| Baseline SDA, T score | 54.9 ± 6.1 | 56.0 ± 5.8 | 55.9 ± 6.5 | 52.3 ± 6.2 | 55.5 ± 5.5 |
| Baseline SRI, T score | 54.5 ± 9.8 | 54.0 ± 8.4 | 56.2 ± 8.7 | 52.4 ± 10.9 | 55.3 ± 11.3 |
| Baseline ESS score | 8.8 ± 5.3 | 8.6 ± 5.6 | 10.8 ± 5.2 | 7.5 ± 4.9 | 8.4 ± 5.0 |
Values are given as mean ± SD or No. (%). AHI = Apnea Hypopnea Index; ESS = Epworth Sleepiness Scale; HbA1c = hemoglobin A1c; PAP = positive airway pressure; SDA = sleep disturbance; SRI = sleep-related impairment.
Symptoms assessed on telephone screening using the STOP-BANG instrument.
To assess convergent validity, we measured correlations across all three measures at baseline and follow-up time periods. SRI and ESS were moderately correlated with each other as were SRI and SDA (Table 2). Except for a small change in the correlation of the ESS with SDA from 0.18 to 0.25, there did not appear to be a change in the correlation structure from pre- to posttreatment.
Table 2.
Correlations Between Patient-Reported Outcomes Measurement Information System Sleep Instruments and ESS
| Instrument | SDA | SRI | ESS |
|---|---|---|---|
| Baseline | |||
| SDA | … | ||
| SRI | 0.67 (0.55 to 0.77)a | … | |
| ESS | 0.18 (−0.02 to 0.36) | 0.45 (0.28-0.59)a | … |
| Follow-up | |||
| SDA | … | ||
| SRI | 0.65 (0.53 to 0.75)a | … | |
| ESS | 0.25 (0.06 to 0.42)a | 0.44 (0.26-0.58)a | … |
Values are given as Pearson correlation coefficient (95% CI).See Table 1 legend for expansion of abbreviations.
Statistically significant correlation (P < .05).
With PAP therapy, patients experienced statistically significant improvements in SDA (follow-up mean, 51.1 ± 6.0; change, −4.0 points; 95% CI, 2.8-5.1), SRI (follow-up mean, 50.9 ± 6.1; change, −4.2 points; 95% CI, 2.3-6.2), and ESS (follow-up mean, 7.4 ± 4.4; change, −1.5 points; 95% CI, 0.8-2.2). We observed the largest effect size with SDA (−0.64; 95% CI, −0.86 to −0.42), then SRI (−0.43; 95% CI, −0.63 to −0.23), and then ESS (−0.28; 95% CI, −0.42 to −0.15). The SDA effect size was significantly greater than the ESS effect size, which was also observed when the cohort was restricted to those with ≥ 2, ≥ 4, and ≥ 6 h of nightly PAP use. The SRI and ESS effect sizes did not significantly differ except when the cohort was restricted to the 78 patients with ≥ 2 h of nightly PAP use (Fig 1).
Figure 1.
Effect sizes for Patient-Reported Outcomes Measurement Information System (PROMIS) sleep measures and ESS with positive airway pressure therapy in patients with OSA and type 2 diabetes mellitus. ESS = Epworth Sleepiness Scale; nt = night; SDA = PROMIS sleep disturbance; SRI = PROMIS sleep-related impairment. *P < .05 for bootstrapped difference in effect size.
Using median SE and SD for PROMIS measures and Cronbach alpha for the ESS, we estimated the reliability of PROMIS SDA as 0.83, PROMIS SRI as 0.94, and ESS as 0.85. Based on this, we then calculated the reliable change indexes, which were changes of 7.03, 6.68, and 5.65 for SDA, SRI, and ESS, respectively. We then categorized PAP response for each subject using these reliable change indexes and the thresholds of normal relative to the population (SDA: 53.01, SRI: 52.25, and ESS: 6.77). Compared with SDA and SRI, the ESS classified fewer patients post-PAP as having symptoms in the normal range. Likewise, the ESS categorized more patients as unchanged with treatment (Table 3). Within strata of PAP usage, we observed differences in reliable change proportions between SRI and the ESS for PAP use strata of < 2.0 h per night and 4.0 to 5.9 h per night. Within each measure, we observed no significant trend across reliable change categories for groups with greater PAP usage (Table 3). We observed agreement between change categories to be slight between SDA and the ESS (κ = 0.18; 95% CI, 0.04-0.33) and between SRI and the ESS (κ = 0.11; 95% CI, 0.0-0.25), whereas we found agreement was fair between SDA and SRI (κ = 0.27; 95% CI, 0.11-0.42). When we dichotomized PAP use strata at 4 h per night, reliable change proportions also differed between both PROMIS measures and the ESS among those with ≥ 4 h per night of PAP use. Among those with < 4 h of PAP use, reliable change proportions only differed between SRI and the ESS (e-Table 2).
Table 3.
Categories of Response Based on Reliable Change to PAP Therapy Stratified by Hours of Usage
| Instrument | Worsening | No Change | Improvement | Symptom Remission |
|---|---|---|---|---|
| SDA | ||||
| Overall (N = 103)a | 3 (2.9) | 69 (67.0) | 7 (6.8) | 24 (23.3) |
| Average hourly PAP use | ||||
| 0.0-1.9 (n = 25) | 2 (8.0) | 17 (68.0) | 1 (4.0) | 5 (20.0) |
| 2.0-3.9 (n = 28) | 0 (0) | 18 (64.3) | 4 (14.3) | 6 (21.4) |
| 4.0-5.9 (n = 28) | 1 (3.6) | 22 (78.6) | 1 (3.6) | 4 (14.3) |
| ≥ 6.0 (n = 22) | 0 (0) | 12 (54.6) | 1 (4.6) | 9 (40.9) |
| SRI | ||||
| Overall (N = 103)a | 11 (10.7) | 53 (51.5) | 7 (6.8) | 32 (31.1) |
| Average hourly PAP use | ||||
| < 2.0 (n = 25)a | 5 (20.0) | 14 (56.0) | 1 (4.0) | 5 (20.0) |
| 2.0-3.9 (n = 28) | 2 (7.1) | 14 (50.0) | 3 (10.7) | 9 (32.1) |
| 4.0-5.9 (n = 28)a | 2 (7.1) | 15 (53.6) | 2 (7.1) | 9 (32.1) |
| ≥ 6.0 (n = 22) | 2 (9.1) | 10 (45.5) | 1 (4.6) | 9 (40.9) |
| ESS | ||||
| Overall (N = 103) | 2 (1.9) | 90 (87.4) | 5 (4.9) | 6 (5.8) |
| Average hourly PAP use | ||||
| < 2.0 (n = 25) | 0 (0) | 22 (88.0) | 2 (8.0) | 1 (4.0) |
| 2.0-3.9 (n = 28) | 1 (3.6) | 23 (82.1) | 1 (3.6) | 3 (10.7) |
| 4.0-5.9 (n = 28) | 0 (0) | 27 (96.4) | 1 (3.6) | 0 (0) |
| ≥ 6.0 (n = 22) | 1 (4.6) | 18 (81.8) | 1 (4.6) | 2 (9.1) |
Values are No. (%). See Table 1 legend for expansion of abbreviations.
P < .05 for difference from ESS proportion by χ2 test. No significant differences were found between SDA and SRI.
We also categorized change using a threshold of MID for each measure (SDA: 3.1; SRI: 4.9; ESS: 2.6). Similar to patterns observed for categories of reliable change, SDA and SRI categorized fewer patients as having unchanged symptoms and more patients with improvement beyond an MID (Table 4). We observed no significant trend in change categories based on MID with greater PAP use (Table 4). Agreement between categorization of response using MID thresholds was slight between SDA and the ESS (κ = 0.11; 95% CI, 0.00-0.27) and SRI and the ESS (κ = 0.10; 95% CI, 0.00-0.25), but was fair between SDA and SRI (κ = 0.34; 95% CI, 0.20-0.49). When we dichotomized PAP usage strata at 4 h of use per night, both PROMIS measures classified a greater proportion of patients as meeting an MID relative to the ESS only among those with ≥ 4 h per night of use (e-Table 3).
Table 4.
Categories of Response Based on Minimum Important Difference to PAP Therapy Stratified by Hours of Usage
| Instrument | Worsening | No Change | Improvement |
|---|---|---|---|
| SDA | |||
| Overall (N = 103)a | 11 (10.7) | 36 (35.0) | 56 (54.4) |
| Average hourly PAP use | |||
| < 2.0 (n = 25) | 4 (16.0) | 10 (40.0) | 11 (44.0) |
| 2.0-3.9 (n = 28) | 2 (7.1) | 10 (35.7) | 16 (57.1) |
| 4.0-5.9 (n = 28) | 5 (17.9) | 11 (39.3) | 12 (42.9) |
| ≥ 6.0 (n = 22)a | 0 (0.0) | 5 (22.7) | 17 (77.3) |
| SRI | |||
| Overall (N = 103)a | 15 (14.6) | 37 (35.9) | 51 (49.5) |
| Average hourly PAP use | |||
| < 2.0 (n = 25) | 5 (20.0) | 11 (44.0) | 9 (36.0) |
| 2.0-3.9 (n = 28) | 3 (10.7) | 10 (35.7) | 15 (53.6) |
| 4.0-5.9 (n = 28) | 4 (14.3) | 10 (35.7) | 14 (50.0) |
| ≥ 6.0 (n = 22) | 3 (13.6) | 6 (27.3) | 13 (59.1) |
| ESS | |||
| Overall (N = 103) | 11 (10.7) | 56 (54.4) | 36 (35.0) |
| Average hourly PAP use | |||
| < 2.0 (n = 25) | 1 (4.0) | 17 (68.0) | 7 (28.0) |
| 2.0-3.9 (n = 28) | 4 (14.3) | 9 (32.1) | 15 (53.6) |
| 4.0-5.9 (n = 28) | 4 (14.3) | 17 (60.7) | 7 (25.0) |
| ≥ 6.0 (n = 22) | 2 (9.1) | 13 (59.1) | 7 (31.8) |
Values are No. (%). See Table 1 legend for expansion of abbreviations.
P < .05 for difference from ESS proportion by χ2 test. No significant differences were found between SDA and SRI.
Discussion
Among patients with OSA and type 2 diabetes, we observed improvement for each patient-reported outcome measure with PAP treatment, but the three instruments differed in the magnitude of improvement. Relative to the ESS, PROMIS sleep measures had larger effect sizes and detected clinically important improvement as defined by both reliable change and MID in a larger proportion of patients. Our results suggest that PROMIS sleep measures have greater sensitivity to capturing symptom changes with PAP therapy for OSA compared with the ESS.
A lack of reliable change in the ESS potentially reflects the nature of its questions. In contrast to the more general nature of sleep and daytime impairments assessed by PROMIS,3 the ESS asks patients to rate their propensity to doze in specific scenarios that may be infrequently encountered (eg, while sitting in traffic, while lying down in the afternoon).4 This characteristic of the ESS may contribute to poor test-retest reliability5 and high variability in its measurement relative to its change over time. A lack of reliable change in the ESS may also reflect a floor effect. The sample had a mean baseline ESS score of 8.8 that was below the pathologic range (> 10),4 potentially limiting our ability to observe reductions in the ESS. PROMIS measures by contrast are standardized on a mean of 50 with possible scores extending to as low as 30.7 This characteristic of the PROMIS instruments illustrates their potential to detect changes that occur among individuals with symptoms that do not fall within the pathologic range.
Our results also support that the three instruments measure unique constructs, as suggested by their moderate to weak convergent validity. This comes as little surprise for SDA and the ESS, given that these two measures are intended to capture disparate constructs—sleep disruption vs daytime sleepiness. Although both SRI and the ESS are intended to quantify daytime symptoms and therefore may measure similar constructs, our baseline correlation of SRI and the ESS was only r = 0.45. This relatively weak correlation has been noted in other investigations,3 and suggests SRI and the ESS measure different aspects of daytime symptoms and impairments. The presence of only fair to moderate agreement between reliable change categories for these three instruments further aligns with the notion that they measure different constructs.
Overall, our results highlight the importance and need for further identification of symptom diversity in OSA. Cohort studies of patients with OSA increasingly support the presence of distinct symptom-based phenotypes, such as subpopulations with a predominance of daytime somnolence, insomnia, or no symptoms.16,17 Importantly, these phenotypes appear to differ in their associations with clinically important outcomes, including adherence to PAP therapy and risk for cardiovascular disease.16, 17, 18 Given these distinct phenotypes and their clinical implications, there is a critical need for future work to validate and assess the responsiveness of multiple patient-reported outcome measures in patients with OSA to further refine these phenotypes in other clinical settings. Given the burden of multiple surveys, future work should also focus on how to identify these constructs in an efficient manner.
Our analyses have implications for both research and clinical practice. Given the greater responsiveness of PROMIS measures to PAP therapy, investigators should consider using the PROMIS score as an end point in studies of OSA treatments. As more responsive measures contribute to greater statistical power, researchers using PROMIS measures would be able to make valid conclusions with fewer participants and lower costs. We also anticipate that SDA and SRI would have particular utility in studies of OSA treatment among patient groups resembling those in the sample presented here (eg, those with less severe symptoms). PROMIS forms have also been translated into multiple languages and are suitable for individuals with low literacy levels.6,19, 20, 21, 22, 23 One potential barrier with PROMIS short forms is the requirement that raw scores be converted to T scores for interpretation using a standard rubric, a process that may require electronic rather than manual scoring. Similarly, the PROMIS measures have the potential to be used in clinical practice. The longitudinal assessment of PROMIS scores can provide physicians and patients a sense of whether patients’ sleep symptoms improve with OSA treatment. Such a role is particularly suited for individuals without pathologic sleepiness, where benefit from OSA treatment may be more difficult to ascertain.24 In this way, the PROMIS measures may offer objective evidence to patients and physicians who are uncertain of the utility of continuing OSA therapies such as PAP. Future work will need to assess the benefit of such integration balanced against patient survey burden.
Our study has several limitations, primarily reflecting issues of generalizability. First, the cohort differs from general populations of OSA because of our inclusion criteria. We only included patients with known type 2 diabetes mellitus. Diabetes can independently contribute to fatigue and SDA, potentially influencing patient-reported sleep symptoms.25,26 Nevertheless, studies of PAP therapy in patients with OSA and type 2 diabetes have generally found improvements in sleep symptoms.27, 28, 29 Second, we identified patients via active screening. Relative to patients typically presenting to sleep clinic for evaluation, the screened population likely included patients with less severe symptoms. Third, we did not compare the PROMIS instruments with other legacy instruments, such as the Pittsburgh Sleep Quality Index or the Insomnia Severity Index, in large part because these instruments are not commonly used in clinical practice to assess response to OSA therapies.30,31 Fourth, our assessment of MID was unable to include an anchor, which is typically a single survey item or end point that defines patient improvement with a high degree of face validity.32 It is possible that anchor-based methods may have defined MID differently. However, a recent study found the MID of the ESS to be 2.5 using anchor-based methods, which is quite similar to our distribution based MID of 2.6.33 Fifth, it is also worth noting that we did not include a control group of patients without CPAP treatment. We were therefore unable to assess the test-retest reliability of these measures, and the degree of change, which would be expected over time among individuals without treatment. Finally, we did not observe a trend toward greater reliable improvement in symptoms with increasing PAP use. This may reflect the relatively small sample size and lack of power to detect differences among categories of PAP usage.
Our results highlight the potential utility of PROMIS measures in tracking the impact of OSA treatment. These measures appear to have greater responsiveness to OSA treatment than the ESS, and address different manifestations of OSA. Future work assessing their responsiveness in other populations of patients with OSA will be helpful in informing the roles of these measures in research and clinical care.
Acknowledgments
Author contributions: L. M. D. is the guarantor of this manuscript and takes responsibility for the content, including the data and analysis. All authors met authorship requirements. S. R. P. and S. M. B designed the intervention. L. M. D., S. M. B., M. R., D. J. B., L. Y., and S. R. P. designed the analyses. L. M. D., S. M. B., and S. R. P. drafted the manuscript. All authors edited the manuscript critically for important intellectual content.
Financial/nonfinancial disclosures: The authors have reported to CHEST the following: outside of this work, S. R. P. has received grant funding through his institution from the American Sleep Medicine Foundation, Bayer Pharmaceuticals, and Philips Respironics. S. M. B. has received grant funding through her institution from Merck, Lockheed Martin, American Sleep Medicine Foundation, and Migraine Research Foundation; and has served as a consultant for Verily. L. M. D. is employed by the VA Puget Sound Health Care System. None declared (L. Y., D. J. B., M. R.).
Role of sponsors: The views expressed here are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. None of the funding sources were involved in the design, conduct, or analysis of this project.
Other contributions: We thank the staff and physicians at Healthcare Associates in Boston, MA, and Adriana Rodriguez, Nida Khan, Afrin Nuzhad, and Alex Choi, who conducted telephone screenings and coordinated OSA care.
Additional information: The e-Tables can be found in the Supplemental Materials section of the online article.
Footnotes
FUNDING/SUPPORT: This work was funded by a grant from the ResMed Foundation, grants from the National Institutes of Health [Grants HL140685-01, HL127307], and an Academic Sleep Pulmonary Integrated REsearch/Clinical Fellowship.
Supplementary Data
References
- 1.Strohl K.P., Redline S. Recognition of obstructive sleep apnea. Am J Respir Crit Care Med. 1996;154(2 Pt 1):279–289. doi: 10.1164/ajrccm.154.2.8756795. [DOI] [PubMed] [Google Scholar]
- 2.Giles T.L., Lasserson T.J., Smith B.J., White J., Wright J., Cates C.J. Continuous positive airways pressure for obstructive sleep apnoea in adults. Cochrane Database Syst Rev. 2006;(1):CD001106. doi: 10.1002/14651858.CD001106.pub2. [DOI] [PubMed] [Google Scholar]
- 3.Buysse D.J., Yu L., Moul D.E. Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep. 2010;33(6):781–792. doi: 10.1093/sleep/33.6.781. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Johns M.W. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540–545. doi: 10.1093/sleep/14.6.540. [DOI] [PubMed] [Google Scholar]
- 5.Campbell A.J., Neill A.M., Scott D.A.R. Clinical reproducibility of the Epworth Sleepiness Scale for patients with suspected sleep apnea. J Clin Sleep Med. 2018;14(5):791–795. doi: 10.5664/jcsm.7108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Cella D., Riley W., Stone A. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63(11):1179–1194. doi: 10.1016/j.jclinepi.2010.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Yu L., Buysse D.J., Germain A. Development of short forms from the PROMIS sleep disturbance and Sleep-Related Impairment item banks. Behav Sleep Med. 2011;10(1):6–24. doi: 10.1080/15402002.2012.636266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Full K.M., Malhotra A., Crist K., Moran K., Kerr J. Assessing psychometric properties of the PROMIS Sleep Disturbance Scale in older adults in independent-living and continuing care retirement communities. Sleep Health. 2019;5(1):18–22. doi: 10.1016/j.sleh.2018.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Donovan L.M., Rueschman M., Weng J. The effectiveness of an obstructive sleep apnea screening and treatment program in patients with type 2 diabetes. Diabetes Res Clin Pract. 2017;134:145–152. doi: 10.1016/j.diabres.2017.10.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Schober P., Boer C., Schwarte L.A. Correlation coefficients: appropriate use and interpretation. Anesth Analg. 2018;126(5):1763–1768. doi: 10.1213/ANE.0000000000002864. [DOI] [PubMed] [Google Scholar]
- 11.Jacobson N.S., Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59(1):12–19. doi: 10.1037//0022-006x.59.1.12. [DOI] [PubMed] [Google Scholar]
- 12.Norman G.R., Sloan J.A., Wyrwich K.W. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41(5):582–592. doi: 10.1097/01.MLR.0000062554.74615.4C. [DOI] [PubMed] [Google Scholar]
- 13.Bauer S., Lambert M.J., Nielsen S.L. Clinical significance methods: a comparison of statistical techniques. J Pers Assess. 2004;82(1):60–70. doi: 10.1207/s15327752jpa8201_11. [DOI] [PubMed] [Google Scholar]
- 14.Pilkonis P.A., Yu L., Dodds N.E., Johnston K.L., Maihoefer C.C., Lawrence S.M. Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS) in a three-month observational study. J Psychiatr Res. 2014;56:112–119. doi: 10.1016/j.jpsychires.2014.05.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 16.Gagnadoux F., Le Vaillant M., Paris A. Relationship between OSA clinical phenotypes and CPAP treatment outcomes. Chest. 2016;149(1):288–290. doi: 10.1016/j.chest.2015.09.032. [DOI] [PubMed] [Google Scholar]
- 17.Pien G.W., Ye L., Keenan B.T. Changing faces of obstructive sleep apnea: treatment effects by cluster designation in the Icelandic Sleep Apnea Cohort. Sleep. 2018;41(3) doi: 10.1093/sleep/zsx201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mazzotti D.R., Keenan B.T., Lim D.C., Gottlieb D.J., Kim J., Pack A.I. Symptom subtypes of obstructive sleep apnea predict incidence of cardiovascular outcomes. Am J Respir Crit Care Med. 2019;200(4):493–506. doi: 10.1164/rccm.201808-1509OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Smith S.G., O'Conor R., Curtis L.M. Low health literacy predicts decline in physical function among older adults: findings from the LitCog cohort study. J Epidemiol Community Health. 2015;69(5):474–480. doi: 10.1136/jech-2014-204915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Evans J.P., Smith A., Gibbons C., Alonso J., Valderas J.M. The National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS): a view from the UK. Patient Relat Outcome Meas. 2018;9:345–352. doi: 10.2147/PROM.S141378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Coste J., Rouquette A., Valderas J.M., Rose M., Leplege A. The French PROMIS-29. Psychometric validation and population reference values. Rev Epidemiol Sante Publique. 2018;66(5):317–324. doi: 10.1016/j.respe.2018.05.563. [DOI] [PubMed] [Google Scholar]
- 22.Flens G., Smits N., Terwee C.B., Dekker J., Huijbrechts I., de Beurs E. Development of a computer adaptive test for depression based on the Dutch-Flemish version of the PROMIS item bank. Eval Health Prof. 2017;40(1):79–105. doi: 10.1177/0163278716684168. [DOI] [PubMed] [Google Scholar]
- 23.Fieo R., Ocepek-Welikson K., Kleinman M. Measurement Equivalence of the Patient Reported Outcomes Measurement Information System® (PROMIS®) applied cognition - general concerns, short forms in ethnically diverse groups. Psychol Test Assess Model. 2016;58(2):255–307. [PMC free article] [PubMed] [Google Scholar]
- 24.Patil S.P., Ayappa I.A., Caples S.M., Kimoff R.J., Patel S.R., Harrod C.G. Treatment of adult obstructive sleep apnea with positive airway pressure: an American Academy of Sleep Medicine Clinical Practice Guideline. J Clin Sleep Med. 2019;15(2):335–343. doi: 10.5664/jcsm.7640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Knutson K.L., Ryden A.M., Mander B.A., Van Cauter E. Role of sleep duration and quality in the risk and severity of type 2 diabetes mellitus. Arch Intern Med. 2006;166(16):1768–1774. doi: 10.1001/archinte.166.16.1768. [DOI] [PubMed] [Google Scholar]
- 26.Sudore R.L., Karter A.J., Huang E.S. Symptom burden of adults with type 2 diabetes across the disease course: diabetes & aging study. J Gen Intern Med. 2012;27(12):1674–1681. doi: 10.1007/s11606-012-2132-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Myhill P.C., Davis W.A., Peters K.E., Chubb S.A., Hillman D., Davis T.M. Effect of continuous positive airway pressure therapy on cardiovascular risk factors in patients with type 2 diabetes and obstructive sleep apnea. J Clin Endocrinol Metab. 2012;97(11):4212–4218. doi: 10.1210/jc.2012-2107. [DOI] [PubMed] [Google Scholar]
- 28.Shaw J.E., Punjabi N.M., Naughton M.T. The effect of treatment of obstructive sleep apnea on glycemic control in type 2 diabetes. Am J Respir Crit Care Med. 2016;194(4):486–492. doi: 10.1164/rccm.201511-2260OC. [DOI] [PubMed] [Google Scholar]
- 29.West S.D., Nicoll D.J., Wallace T.M., Matthews D.R., Stradling J.R. Effect of CPAP on insulin resistance and HbA1c in men with obstructive sleep apnoea and type 2 diabetes. Thorax. 2007;62(11):969–974. doi: 10.1136/thx.2006.074351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Buysse D.J., Reynolds C.F., III, Monk T.H., Berman S.R., Kupfer D.J. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28(2):193–213. doi: 10.1016/0165-1781(89)90047-4. [DOI] [PubMed] [Google Scholar]
- 31.Bastien C.H., Vallieres A., Morin C.M. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med. 2001;2(4):297–307. doi: 10.1016/s1389-9457(00)00065-4. [DOI] [PubMed] [Google Scholar]
- 32.Guyatt G.H., Osoba D., Wu A.W., Wyrwich K.W., Norman G.R. Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc. 2002;77(4):371–383. doi: 10.4065/77.4.371. [DOI] [PubMed] [Google Scholar]
- 33.Patel S., Kon S.S.C., Nolan C.M. The Epworth Sleepiness Scale: minimum clinically important difference in obstructive sleep apnea. Am J Respir Crit Care Med. 2018;197(7):961–963. doi: 10.1164/rccm.201704-0672LE. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.