Long-term health outcomes for patients with obstructive sleep apnea: placing the Agency for Healthcare Research and Quality report in context—a multisociety commentary

Abstract

This multisociety commentary critically examines the Agency for Healthcare Research and Quality (AHRQ) final report and systematic review on long-term health outcomes in obstructive sleep apnea. The AHRQ report was commissioned by the Centers for Medicare & Medicaid Services and particularly focused on the long-term patient-centered outcomes of continuous positive airway pressure, the variability of sleep-disordered breathing metrics, and the validity of these metrics as surrogate outcomes. This commentary raises concerns regarding the AHRQ report conclusions and their potential implications for policy decisions. A major concern expressed in this commentary is that the AHRQ report inadequately acknowledges the benefits of continuous positive airway pressure for several established, long-term clinically important outcomes including excessive sleepiness, motor vehicle accidents, and blood pressure. While acknowledging the limited evidence for the long-term benefits of continuous positive airway pressure treatment, especially cardiovascular outcomes, as summarized by the AHRQ report, this commentary reviews the limitations of recent randomized controlled trials and nonrandomized controlled studies and the challenges of conducting future randomized controlled trials. A research agenda to address these challenges is proposed including study designs that may include both high quality randomized controlled trials and nonrandomized controlled studies. This commentary concludes by highlighting implications for the safety and quality of life for the millions of people living with obstructive sleep apnea if the AHRQ report alone was used by payers to limit coverage for the treatment of obstructive sleep apnea while not considering the totality of available evidence.

Citation:

Patil SP, Billings ME, Bourjeily G, et al. Long-term health outcomes for patients with obstructive sleep apnea: placing the Agency for Healthcare Research and Quality report in context—a multisociety commentary. J Clin Sleep Med. 2024;20(1):135–149.

INTRODUCTION

Obstructive sleep apnea (OSA) is a very common respiratory-related sleep disorder affecting 34% of men and 17% of women between the ages of 30 and 70 years in the US population.¹ OSA has been associated with many consequences, including excessive daytime sleepiness, sleep-related impairments in quality of life, cardiovascular disease (eg, hypertension, coronary heart disease, heart failure, atrial fibrillation), stroke, metabolic disorders (eg, diabetes mellitus, metabolic syndrome), and early mortality (see Figure 1).^2–10 While there are several treatment options for OSA, positive airway pressure (PAP), whether continuous PAP (CPAP) or auto-adjustable PAP, continues to be the most common therapy administered. Treatment of OSA with PAP has been established to improve OSA-related sleepiness and sleep-related quality of life. However, considerable controversy remains as to whether treatment of OSA improves cardiovascular disease, stroke, and metabolic outcomes, due to discordant results between nonrandomized controlled studies (NRCSs) and randomized controlled trials (RCTs), which recruited participants with different characteristics.¹¹

Figure 1. Benefits of treatment of obstructive sleep apnea.

In this setting, and at the request of the Center for Medicare & Medicaid Services, the Agency for Healthcare Research and Quality (AHRQ) published a final report and systematic review, “Long-Term Health Outcomes In Obstructive Sleep Apnea (OSA),” in December 2022.¹² This evidence-based review was completed by the Brown Evidence-based Practice Center and had several stated purposes, which included (1) an evaluation of the evidence on improvement of “long-term” clinical health outcomes with special emphasis on PAP therapy and (2) assessment of the validity of sleep-disordered breathing metric criteria (eg, the apnea-hypopnea index [AHI]) as surrogate outcomes. The Center for Medicare & Medicaid Services requested this analysis¹² to be performed in concordance with the center’s “General Methodological Principles of Study Design,”¹³ which is used in developing national coverage determinations. In this document, the Center for Medicare & Medicaid Services states that they place “greater emphasis on health outcomes actually experienced by patients, such as quality of life, functional status, duration of disability, morbidity and mortality, and less emphasis on outcomes that patients do not directly experience, such as intermediate outcomes, surrogate outcomes, and laboratory or radiographic responses.”

Despite the stated emphasis on patient-experienced health outcomes, the final AHRQ report¹² excluded and de-emphasized “sleepiness or other symptoms,” which are outcomes most sleep medicine professionals and patients with OSA find very important. A unified letter endorsed by 21 professional and patient societies and comments by other stakeholders during the public comment period criticized the draft version of the AHRQ report for failing to clearly acknowledge the established therapeutic benefits of the treatment of OSA for excessive sleepiness, sleep-related quality of life, motor vehicle accidents, and blood pressure (see Figure 1). The overall message conveyed by the final version of the AHRQ report, intentional or not, was that there were no significant benefits, short term or long term, of CPAP treatment. Although many in the sleep research community acknowledge the limitations of current RCTs (see Figure 2), given the challenges of randomizing patients with excessive sleepiness to a nontherapeutic arm,¹⁴ this conclusion does not reflect the totality of available evidence. Furthermore, the AHRQ report represented a missed opportunity to recommend research and study design strategies to public and private funding agencies that balance individual and public safety with the need for longer-term, evidence-based studies to determine if there are other clinical benefits of treating OSA. There is strong concern that the AHRQ report will be misconstrued by stakeholders, such as policymakers, payors, and the public, as justification to no longer support using CPAP for OSA. This could have detrimental repercussions for the care of millions of Americans with OSA experiencing benefit from treatment now and in the future.

Figure 2. Limitations of prior randomized controlled trials evaluating positive airway pressure therapy for sleep apnea and potential solutions.

The purpose of this commentary is to review the AHRQ report and place in context its key findings, acknowledge strengths and highlight limitations of the report, and present a more detailed potential research agenda to advance the care of people living with OSA.

SUMMARY OF THE AHRQ REPORT

The AHRQ report performed a comprehensive review of long-term trials from January 1, 2010, to March 22, 2021. RCTs and only NRCS of CPAP that adjusted for potential confounders were included. Included studies had to report effects on prespecified long-term outcomes (≥ 12 months for death, incident cardiovascular events, hypertension, or diabetes mellitus; ≥ 6 months for mental health conditions, cognitive outcome, some quality of life measures, sexual function, sequelae of sleep deprivation). Excessive daytime sleepiness, one of the most common patient-reported symptoms, as an outcome was specifically excluded.¹² Fifty-two studies met the established inclusion/exclusion criteria, with 31 studies (14 RCTs and 17 NRCS) used to compare the effect of CPAP vs no CPAP on long-term outcomes and 15 studies used to examine the correlation between changes in potential intermediate or surrogate measures (AHI, oxygen desaturation index, Epworth Sleepiness Scale [ESS]) and health outcomes in adults with OSA, with remaining studies included to address other questions raised by the systematic review. A summary of the findings is provided in Table 1 and later in this discussion.

Table 1.

Key findings from the Agency for Healthcare Research and Quality report.

Variability in definition of OSA metrics	Studies were inconsistent in the criteria used to define OSA and OSA severity. Most studies identified did not fully report the definitions used.
Effects of PAP vs no PAP on long-term OSA outcomes	Low strength of evidence that RCTs and adjusted NRCSs have shown benefits of CPAP in patients with OSA in improving long-term health outcomes or there was insufficient evidence to evaluate. Long-term health outcomes assessed included death, incident cardiovascular and cerebrovascular events, or diagnosis—individual and composite, motor vehicle crashes, development or regression of hypertension or diabetes mellitus, incident arrhythmias, mental health conditions, cognitive function, health-related and generic quality of life, sexual function, sleep loss consequences.
PAP vs MAD	Low strength of evidence from the few studies that evaluated measurements of depression, anxiety, quality of life, and functional status do not significantly differ between patients receiving either PAP or MAD.
PAP adverse events	Insufficient evidence was present to assess adverse events related to PAP. Adverse events that are reported in FDA databases were related to oral and dental health, respiratory system, otolaryngology, odors, allergies and rashes, burns, eye health, aspiration, aerophagia, and miscellaneous other adverse events.
AHI as an intermediate or surrogate outcome for long-term OSA-related outcomes	Absence of studies directly evaluating changes in OSA metrics and health-related outcomes. Absence of surrogacy or mediation analyses.

AHI = apnea-hypopnea index, CPAP = continuous positive airway pressure, FDA = Food and Drug Administration, MAD = mandibular advancement device, NRCS = nonrandomized controlled study, OSA = obstructive sleep apnea, PAP = positive airway pressure, RCT = randomized controlled trial.

Variability in definition of OSA metrics and validity as intermediate or surrogate outcomes

The AHRQ report highlighted concerns regarding the variability in definition of OSA-related events among studies over time, the shortcomings of the AHI as a sole measure of sleep apnea severity, and low confidence in whether the AHI is in fact the primary predictor of adverse health outcomes. Approximately one-third of eligible studies omitted definitions of the AHI, while only 40% explicitly provided respiratory event definitions, raising concerns for consistency in inclusion criteria across studies, given the impact of different definitions on classification of disease severity.¹⁵

Furthermore, the AHRQ report was critical of the use of a rate metric like the AHI (ie, respiratory events per unit of time) as an assessment of disease exposure rather than measures of cumulative exposure over the total sleep period (eg duration of respiratory events and hypoxic burden) and absence of accounting for differences in timing and clustering of respiratory events (eg, rapid eye movement vs non-rapid eye movement sleep).¹⁶ The observations, the AHRQ report noted, may explain why the AHI has not been well correlated with symptoms and signs of OSA. The AHRQ report “encourages research on alternative metrics to better capture patients’ respiratory processes during sleep” and is in concordance with published statements by the sleep research community.^17,18

The AHRQ report also assessed the validity of the AHI as an intermediate or surrogate outcome. In general, the available data were too limited to allow an assessment of study-level concordance between changes in OSA measures and health outcomes. The AHRQ report called for studies to demonstrate that current OSA metrics are appropriate surrogate measures and whether they are mediators of clinically relevant OSA-related outcomes.

Effect of CPAP vs no CPAP on certain OSA-related outcomes

The AHRQ report concluded that the included RCT studies did not demonstrate an improvement in all-cause or cardiovascular mortality, stroke, myocardial infarction, composite cardiovascular endpoints, driving accidents, or incident diabetes (all with low strength of evidence). The results of included NRCSs were generally concordant with RCTs apart from all-cause mortality. Specifically, NRCSs indicated stronger, statistically significant associations between CPAP treatment for OSA and reduced all-cause mortality, which remained when analyses included both RCTs and NRCSs. The AHRQ report noted that NRCSs tended to include sleepier participants than the RCTs. However, the report had low confidence in the all-cause mortality analyses and noted that potential differences in effect across subgroups of study participants were rarely evaluated.

The conclusions by the AHRQ report on the low strength of evidence of benefits of CPAP treatment for OSA on certain long-term outcomes are not necessarily new or surprising and generally align with another major systematic review commissioned by the American Academy of Sleep Medicine (AASM) on the outcomes assessed.¹⁹ Potential reasons for the absence of clear cardiovascular benefits in RCTs cited by the AHRQ report and other reviews are summarized in Table 2. The AHRQ report noted the inherent bias and potential for residual confounding of many NRCSs in comparing users to nonusers of CPAP. The AHRQ report ultimately concluded that clinical equipoise remains on whether CPAP improves the long-term outcomes that were considered and called for “high quality, long-term studies that are adequately powered for clinical endpoints.”¹² Furthermore, given the low strength of evidence for all conclusions on the effect of CPAP for OSA-related outcomes, the AHRQ report did state that additional evidence was needed before concluding its findings were “stable or correct” and that “It would not be unexpected for future evidence to alter these conclusions.” The AHRQ report also noted that “the focus of our review should not be interpreted to imply that we consider other evidence, short term studies, or other outcomes including sleepiness, other outcomes and intermediate outcomes to be unimportant for patients and clinicians.”¹²

Table 2.

Potential reasons for absence of benefits for PAP in RCTs for primarily cardiovascular trials.

Insufficient power Insufficient follow-up duration Heterogeneity in the definition of OSA and its severity Presence of cardiovascular disease at baseline Inclusion (or lack of) of patients with symptoms such as daytime sleepiness Suboptimal PAP usage for the study duration Heterogenous responses to PAP True lack of effect of PAP

OSA = obstructive sleep apnea, PAP = positive airway pressure, RCT = randomized controlled trial.

PLACING THE AHRQ REPORT IN CONTEXT

In this section, limitations of the AHRQ report in several key areas are considered and the conclusions placed in context for stakeholders, including policymakers, payors, clinicians, and patients.

Effect of CPAP treatment of OSA on excessive daytime sleepiness

Excessive daytime sleepiness (EDS) is a common symptom of OSA and is a critical adverse health outcome experienced by many people living with OSA. EDS in the AHRQ report was only considered as an intermediate outcome, which resulted in the exclusion of an extensive evidence base demonstrating the improvement of EDS in OSA with CPAP. The repeated statement by the AHRQ report, that CPAP “has no impact on long-term, clinically important outcomes,” is misleading and does not acknowledge the evidence that CPAP is an effective treatment for OSA-related symptoms, including EDS.

EDS is a highly impactful consequence of OSA, which has direct adverse effects on people living with OSA and contributes to worsening of other OSA-related impairments.^20,21 EDS is the most common OSA-related symptom for which people seek treatment, with implications for driving safety, and is the strongest clinical indication for prescription of CPAP by clinicians. It is also a major determinant of persons’ acceptance of, and adherence to, long-term CPAP therapy.^22,23

Short-term studies (< 6 months) assessing EDS were excluded from the AHRQ report on the premise that such data are not relevant to long-term benefit from CPAP treatment. The relief of EDS was considered only to be a “short-term benefit.” However, this patient-centric benefit is a long-term, clinically important effect, which is dependent upon continued use of CPAP therapy. EDS predictably recurs upon interruption of CPAP in the clinical setting. This has been formally demonstrated in studies of 1–2 weeks’ withdrawal from long-term CPAP therapy, which led to recurrence of both self-reported and objective EDS.^24,25 These findings provide compelling evidence that short-term benefits of CPAP have implications for long-term outcomes in that they persist on continued therapy but are reversed if treatment is interrupted.

The evidence supporting the benefits of CPAP in improving EDS was critically evaluated in the recent systematic review conducted by an AASM sponsored task force on CPAP therapy.^11,19 A meta-analysis of 33 RCTs of at least 4 weeks’ duration in participants with EDS yielded a mean improvement of –2.7 (95% confidence interval: –3.2 to –2.15) points in the ESS with CPAP compared to a control condition.¹⁹ The minimal clinically important difference for the ESS is generally accepted to be 2.^11,26,27 While the AASM systematic review included trials of less than 6 months duration, 10 of the 12 RCTs were included in the AHRQ analysis and provide data on the improvement in ESS with CPAP vs a control condition over ≥ 6 months. Thus, there is high strength of evidence for the long-term benefit of CPAP treatment for OSA on EDS. Indeed, this evidence served as the basis for the strong recommendation in the 2019 AASM Clinical Practice Guideline for use of CPAP therapy in treating OSA in adults with EDS.¹¹

Effect of CPAP treatment on motor vehicle crashes

There is abundant evidence linking untreated OSA with an increased rate of motor vehicle crashes (MVCs).²⁸ The AHRQ meta-analyses only considered data from RCTs after 2010 and did not examine NRCSs on MVCs. In the two RCTs analyzed (SAVE²⁹ and PREDICT³⁰ trials), there was no significant effect of CPAP on MVCs, although a trend to a lower rate of crashes causing injury was observed in SAVE (relative risk 0.84; 95% confidence interval: 0.70–1.00). However, neither study was sufficiently powered for MVCs, which was a secondary outcome. Furthermore, SAVE excluded patients with moderate-severe sleepiness (ESS > 15), and while PREDICT included patients with ESS > 9, patients with a history of sleepiness while driving were specifically excluded. Thus, for safety reasons, these studies excluded individuals most likely to experience MVCs, demonstrating the limitations of RCT data to find a reduction in MVCs with CPAP. Given the evidence for the effect of CPAP on EDS, many institutional review boards and investigators are concerned about randomizing study participants with severe EDS to an ineffective control arm given the potential for high risk of harm from sleepiness over extended periods of time. However, this should be placed in context, as excessive sleepiness can be caused by conditions other than OSA, including insufficient sleep, medications, primary hypersomnias, and other medical conditions.

Innovative methodologic approaches clearly need to be explored in future studies. However, if the AHRQ report considered MVCs to be a clinically important long-term outcome, then the total evidence base should have been considered. An unintended and unfortunate consequence would be if federal policies curtailed the need for OSA screening in safety-sensitive occupations due to misinterpretation of findings in the AHRQ report.

In the absence of RCTs specifically designed to assess MVCs as a primary outcome, frameworks exist to allow for NRCSs to be utilized in systematic reviews,^31,32 assuming any recommendations are transparent in their assessment of the quality of the studies, the harms and benefits, resources used, and consideration of patient preference and values. For example, two recent meta-analyses examined data from NRCSs on the effect of CPAP treatment of OSA on MVCs yielded very similar findings.^19,33 In the more recent AASM meta-analysis,¹⁹ which focused on noncommercial MVCs, the 10 studies included consisted predominantly of pre- to post-CPAP comparisons for single groups with follow-up ranging from 2 years before to 0.5–6.0 years after enrollment, thus evaluating the long-term impact of OSA treatment. The rate of MVCs was strikingly reduced following CPAP treatment, with an overall risk ratio of 0.28 (95% confidence interval: 0.18–0.43).¹⁹ The overall quality of evidence from these NRCSs were considered of low quality, predominantly due to absence of a control group. However, the methodologically strongest of these studies³⁴ compared crash rates for 210 patients with OSA before and after CPAP treatment to population control rates during the same time period, with adjustment for annual distance driven and verification of crashes from transport authority records. These authors reported a risk ratio of 0.43 (95% confidence interval: 0.30–0.63) for MVCs following CPAP therapy, similar to the overall point estimate.

These data have informed recommendations by scientific societies and federal policies for OSA screening and management for both noncommercial and commercial drivers.³⁵ Innovative, pragmatic studies evaluating the outcome of OSA screening and treatment programs implemented in commercial trucking companies with control groups have demonstrated reduced MVCs among CPAP-adherent compared with untreated drivers.^36,37 Based on the available evidence, the AASM recently issued a position statement urging broad implementation of OSA screening and treatment programs for commercial vehicle operators, stating “the use of existing paradigms to identify and treat OSA-related crash risk results in clear benefits, including a reduction in crashes, economic gains, symptom relief and general health benefits for the operator.”^38,39 These recommendations underscore the importance of CPAP treatment of OSA as a clinical and public safety outcome. Given the potential for severe consequences from MVCs, such as death or injury resulting in disabilities, due to OSA-related sleepiness, a conservative and cautious approach to policy decisions is justified until additional studies of higher quality are reported.

Effect of CPAP treatment on blood pressure

Hypertension has a high prevalence of approximately 50% in people living with OSA, while 30% of people with hypertension and 83% with refractory hypertension are estimated to have OSA.⁴⁰ Hypertension has been shown to increase the risk of myocardial infarction, stroke, heart failure, dementia, and kidney disease, among other outcomes, as acknowledged in previous AHRQ statements.⁴¹ OSA is associated with the development of incident hypertension,^42,43 and multiple systematic reviews^11,44–47 have shown a small but clinically significant benefit in blood pressure reduction with treatment of OSA with CPAP.

In evaluating long-term blood pressure related outcomes, the AHRQ report narrowly focused on the development or resolution of hypertension and chose not to examine improvements in blood pressure as a long-term outcome. The report concluded that, due to the limited number of studies, there was “insufficient evidence to determine the effect of CPAP on risk of incident HTN or reversion to normotension.” Limiting the focus to the development or resolution of hypertension ignored the salient outcome of the magnitude of blood pressure reduction, which can have important patient-level benefits (eg, reduction in the number of blood pressure medications) and population-level benefits (eg, reduction in mortality and cardiovascular outcomes).⁴⁸ Prior systematic reviews^19,44–47 demonstrated a 2–4 mmHg reduction in blood pressure with CPAP in people with OSA. While a relatively small reduction compared to antihypertensive medications at the individual level, it may have implications at the population level in reducing cardiovascular events. There is a substantial evidence base that sustained reductions in blood pressure by 1–4 mmHg with antihypertensive therapy translates into meaningful long-term cardiovascular risk reduction.^49,50

Hypertension is multifactorial in etiology, with only some intermediate pathways potentially affected by CPAP treatment. While a single antihypertensive drug may be expected to lower blood pressure to normal levels in some patients with mild hypertension, it would not be expected to either resolve or prevent new hypertension in all patients. It is critical to demonstrate an independent blood pressure-lowering effect attributable to a single specific therapy, in the context of RCTs and NRCSs, as has been demonstrated in patients with hypertension and OSA treated with CPAP. Several meta-analyses have shown clinically significant reductions in blood pressure with CPAP, with individual studies suggesting several important factors such as hypertension phenotype (eg, uncontrolled, resistant, or refractory hypertension), younger age, the presence of excessive sleepiness, greater severity of OSA, and higher adherence to CPAP as important factors in predicting the CPAP-mediated reductions in blood pressure.^19,45,51 Furthermore, the blood pressure-lowering effect of CPAP is maintained over the long term. For example, 2 weeks of CPAP withdrawal in patients with OSA on long-term CPAP therapy resulted in significant increases in blood pressure.²⁴ The evidence for clinically significant reduction in blood pressure with CPAP treatment in OSA had previously led to the AASM Clinical Practice Guideline recommendation: “We suggest that clinicians use positive airway pressure, compared to no therapy, to treat OSA in adults with comorbid hypertension. (CONDITIONAL).”¹¹

Continued work is needed to evaluate the direct impact of CPAP-mediated reductions in blood pressure in individuals with OSA on long-term, clinically important cardiovascular outcomes. These studies will be challenging and will require large study samples over multiple centers. It may be important to consider RCTs or NRCSs in high-risk populations where blood pressure control is particularly important, such as those at high risk for a stroke or transient ischemic attack, given emerging data.^52–55 In the interim, the AHRQ report should not be misconstrued to imply that there is an absence of benefit associated with the blood pressure-lowering effects of CPAP in OSA.

MOVING BEYOND THE AHRQ REPORT

Future research priorities and approaches

The AHRQ report provides a strong rationale and useful suggestions for future studies evaluating the long-term benefit of CPAP therapy. However, the recommendations that were put forth for specific future studies were incomplete. The AHRQ report does not fully acknowledge the challenges in conducting research on long-term outcomes in patients with OSA undergoing treatment. These challenges are related to the heterogeneity of OSA itself, the heterogeneity of individual responses to OSA, and the reluctance of patients and physicians to risk randomization to no treatment given the clinical consequences of OSA and the known symptomatic benefits of CPAP and other OSA treatments. In the next sections, areas of opportunity are outlined for additional research on long-term outcomes associated with the treatment of OSA.

Standard reporting of AHI definitions

The AHRQ report noted that only 40% of studies assessed explicitly reported the full definition of the AHI and its components used. In studies that reported use of published criteria for apneas and hypopneas, 17 of 26 (65%) studies, there were inconsistencies in reporting airflow reduction or desaturation thresholds and reporting which published criteria were being used. The inconsistencies noted by the AHRQ report involved early RCTs where the 1997 and 1999 AASM criteria were used. Nevertheless, the field of sleep medicine, journals, and readers would benefit if authors were required to provide the full definitions of the AHI used.

New approaches to defining disease severity

The AHI has been the primary measure of OSA severity utilized in clinical research and care. As discussed, the AHI may not represent the primary determinant of adverse sequelae of OSA and is poorly correlated with OSA-related outcomes.^12,17,18 The sleep research community has recognized the limitations of the AHI and called for the continued development and validation of metrics that provide improved prediction of individuals at risk for cardiovascular consequences and most likely to benefit from treatment of OSA.^17,18 In addition, new metrics when validated should be carefully assessed to ensure they do not further exacerbate health disparities (eg, if race difference in oximeter performance affects measures of hypoxic burden based on pulse oximetry).

Hypoxemic burden:

Patients with OSA who have the same AHI can have very different degrees of hypoxemia. Improved prediction of cardiovascular and all-cause mortality occurs when adjusting the AHI for the degree of event-related hypoxemia.^56–58 More recently, event-related “hypoxemic burden” has been reported to be a better predictor of major adverse cardiovascular outcomes than the AHI in a clinic-based cohort.⁵⁹ Hypoxemic burden in community-based cohorts of people with OSA has been shown to be more predictive of cardiovascular mortality, even after adjustment for AHI, minimum oxygen saturation, and time at saturation below 90%, demonstrating that this measure of disease severity may provide improved prediction of adverse outcomes of OSA beyond that afforded by traditional OSA metrics.⁶⁰

Sympathetic nervous system activation:

Metrics derived from analysis of the photoplethysmography pulse wave signal that are markers of sympathetic nervous system activation (eg, pulse wave attenuation and heart rate acceleration events) and vascular stiffness (eg, pulse transit time) are of increasing interest. These measures are associated with cardiovascular risk profile and have therefore been suggested as potential markers of OSA-related cardiovascular risk.⁶¹ Heart rate response to respiratory events, in particular, is one of the more promising supplemental metrics of OSA and predicts the incidence of both nonfatal and fatal cardiovascular events in two population-based cohorts.⁶² This metric also appears to identify subgroups of people with OSA who have reduced risk of major adverse cardiovascular events with CPAP use.⁶³ Cluster analysis of polysomnographic markers suggest that periodic limb movements are another sign of sympathetic nervous system activation associated with higher cardiovascular risk.⁶⁴

Airflow-derived metrics:

Measures available from airflow signals may also provide useful information on risk beyond that provided by the AHI. For example, respiratory event duration has been associated with mortality.⁶⁵ Automated, advanced signal processing of respiratory parameters from the polysomnogram may permit analyses of more complex measures of sleep-related airway obstruction and clinical outcomes, such as improved discrimination of central vs obstructive events.⁶⁶ Endophenotypes of OSA have been established that characterize varying degrees of pharyngeal collapsibility, pharyngeal dilator muscle compensation, ventilatory control, and arousal threshold.⁶⁷ While approaches to estimating these endophenotypes from routine sleep studies continue to be refined and debated,^67,68 these characteristics may have a role in predicting responsiveness to non-PAP therapies.⁶⁹ Whether they are predictive of cardiovascular outcomes remains to be demonstrated,^67,68 as does their short- and long-term reliability.

Machine learning:

Hypoxic burden, sympathetic nervous system activation, and airflow-derived endophenotypes are not the only alternative metrics of OSA severity that are under evaluation. These hypothesis-driven metrics are likely to be joined by metrics derived from hypothesis-free, machine-learning methods applied to signals available from polysomnography and respiratory polygraphy.^70–74 Critical to the development of both hypothesis-driven and machine learning-derived metrics will be their generalizability beyond the cohorts in which they are developed, their short-term and long-term reliability and agreement, and their validation in predicting responses to OSA treatment; none is yet sufficiently well validated as an alternative or supplement to the AHI. Additional features not routinely collected during sleep testing, such as event-related increases in CO₂ and changes in intrathoracic pressure, may also be important measures of OSA severity and should not be neglected.

Clinical phenotypes to identify at-risk individuals

While the identification of improved polysomnographic metrics of OSA severity is needed, the assessment of patient-reported symptoms and the identification of clinical phenotypes have equally important potential in the selection of at-risk individuals for clinical trials. For example, an important identified limitation of prior RCTs assessing OSA treatment effects on outcomes such as mood or cognition was the failure to target individuals with baseline impairment in the outcomes of interest,¹⁹ while accumulating data indicate benefit from CPAP in patients with, for example, baseline cognitive impairment.^75,76

In addition, clinical phenotypes using cluster analytical approaches have identified subtypes of patients defined by excessive sleepiness, disturbed sleep, or minimal symptoms.^59,77–83 Symptoms are both patient-centered outcomes of clinical importance and also important markers of cardiovascular disease risk in patients with OSA. Longitudinal cohort studies in both clinic-based⁸⁴ and population-based cohorts⁸⁵ and symptom-based cluster analyses^{51,77,81,86,87} support that the excess cardiovascular risk in OSA is seen primarily in those with excessive sleepiness. This has not been observed in all cohorts^59,88,89; however, differences may, in part, reflect limitations to current approaches for measuring sleepiness. Further studies using expanded biomarkers of sleepiness, impaired performance, or other symptoms will be helpful in identifying high-risk individuals.

Biomarkers of sleepiness or impaired performance

Current approaches to assessing sleepiness or impaired performance have been primarily through questionnaires or objective measures that are often expensive, impractical, or inconvenient (eg, multiple sleep latency test). Large variability is evident in population-based cohorts, which have not shown a strong correlation of sleepiness with disease severity by AHI. A majority of individuals with any severity of OSA (ie, AHI > 5), and as much as half of those with severe OSA (ie, AHI > 30), including those in occupational settings who may have concerns about employability, do not report excessive sleepiness.^90–92

Objectively measured, inexpensive, valid, and reliable biomarkers of excessive sleepiness, vigilance, and impaired performance are needed to facilitate research into patient populations that are most likely to obtain robust neurocognitive and cardiovascular benefits from treatment of OSA. Recently, novel metrics derived from analysis of the overnight electroencephalogram recordings obtained during polysomnography have shown some promise as a biomarker of sleep depth,^93,94 and appear to identify increased motor vehicle crash risk in general population samples with OSA.⁹⁵ In addition, the identification of blood biomarkers or non-electroencephalogram physiologic markers that can identify risk of impairment from OSA and response to OSA treatment should be a high research priority.

Molecular biomarkers

OSA is associated with cardiovascular disease through several pathways, including sympathetic activation, oxidative stress, and systemic inflammation. However, studies have not shown that molecular biomarkers of these pathways identify patients at increased OSA-specific cardiovascular risk or subgroups with OSA that have reduced cardiovascular risk with treatment. The lack of available data reflects the relatively early stage of molecular biomarker exploration in OSA and a paucity of longitudinal cohort studies with extensive proteomic, metabolomic, and gene expression data that are also well characterized with respect to OSA phenotype.⁹⁶

Fortunately, public repositories such as the National Sleep Research Resource (https://sleepdata.org/), the Biologic Specimen and Data Repository Information Coordinating Center (https://biolincc.nhlbi.nih.gov), and the Database of Genotypes and Phenotypes (https://www.ncbi.nlm.nih.gov/gap/) are increasing the availability of such data. Therefore, identifying blood biomarkers for OSA-related cardiovascular risk and treatment responsiveness should be a top research priority. Studies to elucidate the mechanisms underlying the association of sleepiness with increased cardiovascular risk are also needed. Such studies will help enhance identification of high-risk patient groups and may also suggest novel treatments targeting these mechanistic pathways.

Genetic risk

OSA has long been recognized to have an important genetic component.⁹⁷ However, the same factors that hinder identifying blood biomarkers of OSA risk have also impeded progress in understanding the genetic architecture of OSA. This is rapidly changing, with recent large-scale genome-wide association studies identifying genetic variants linked to risk of having OSA.^98,99 Expanding this work to develop polygenic risk scores that identify individuals at risk for OSA, as well as those at higher risk for specific OSA-related morbidities, should be a top research priority.

Reconsidering approaches to RCTs for treatment of OSA

The AHRQ report concludes that RCTs do not provide evidence of an effect of CPAP on long-term OSA-related outcomes such as mortality and other cardiovascular-related comorbidities. However, there are limitations of the RCTs performed to date, which require some caution in interpreting (see Figure 2 and Table 2). Most RCTs to date evaluating cardiovascular outcomes have recruited participants from outside of sleep medicine clinics who were not excessively sleepy and, therefore, less likely to adhere to treatment over long periods of time.^100–102 Although the AHRQ report calls for new RCTs, little guidance was provided, and there was not clear recognition of some of the unique challenges in conducting RCTs in this area. A recent American Thoracic Society-sponsored workshop provided insights for both the sleep research field and potential sponsors to consider in developing the necessary evidence base that can guide policy decisions.¹⁴ The following sections summarize some of the challenges and potential solutions.

Challenges to conducting RCTs and maintaining equipoise

The challenges in conducting RCTs on long-term outcomes in OSA include barriers to clinician involvement, patient participation, and regulatory and funding agency approval. Clinicians may lack equipoise and be reluctant to randomize participants to long-term clinical trials with an inactive treatment arm given the known benefits of OSA treatment in reducing excessive sleepiness, other symptoms, and related safety risks.^28,30,33 Participants, similarly, may be appropriately concerned about being assigned to a longer-term inactive treatment arm without the potential for benefits and shouldering burdens associated with participation (eg worsening symptoms, lost time, travel requirements, loss of autonomy, and personal costs). Regulatory and funding agencies, including institutional review boards, encounter similar concerns raised during the peer review process and have hesitated to approve long-term studies.

Absence of equipoise among stakeholders is a barrier to the conduct of long-term OSA-related outcomes. However, as noted by a recent American Thoracic Society workshop report,¹⁴ equipoise does not need to be held universally within a field. While the recent negative RCTs on the long-term cardiovascular benefits of CPAP treatment for OSA are not definitive, they should increase equipoise regarding cardiovascular benefits of treatment. Blanket exclusions of participants due to symptoms of sleepiness is not warranted given that excessive sleepiness can be caused by conditions other than OSA (eg, insufficient sleep, medications, primary hypersomnia). Incorporating methods to mitigate EDS and its consequences during RCTs may allow participants to be randomized in clinical trials aimed at understanding the independent benefits of treatment of OSA.

However, even if investigators implement a rigorous informed consent process that acknowledges the potential for persistent or increased sleepiness in those assigned to a control arm and has appropriate mitigation plans in place to address this,^29,103 patient acceptance will remain a serious challenge to inclusion of symptomatic patients in long-term RCTs.

Optimizing PAP treatment adherence for OSA:

Low levels of adherence to CPAP therapy in asymptomatic patient groups has been an important limitation of prior randomized clinical trials of CPAP therapy for secondary prevention of cardiovascular disease. To improve adherence, several approaches have demonstrated effectiveness. These include identifying a comfortable CPAP mask/interface; addressing minor CPAP side effects such as mucosal dryness, air leak, nasal congestion, and mask discomfort; and implementing educational interventions. Educational interventions can increase CPAP use by approximately 35 minutes per night and reduce the likelihood of CPAP discontinuation when accompanied by companion behavioral, psychosocial, or technological interventions.^104,105 Formal motivation enhancement therapy is one of the best-validated approaches to improving adherence. Motivation enhancement therapy can increase nightly CPAP adherence by approximately 40 minutes at 1 week after CPAP initiation and by 90 to 120 minutes after 3 to 12 months of treatment.^106,107 Telemonitoring, facilitated by modems integrated into many contemporary CPAP devices and coupled with specific feedback to users (telecoaching), can also increase CPAP use by approximately 40 minutes per night.¹⁰⁸ Although intensive application of adherence interventions will significantly increase the cost of clinical trials, the importance of adherence to their success warrants the investment of resources.

Considering innovative approaches to RCTs

A barrier to the conduct of RCTs is their high and continually increasing costs. Diverse strategies have been proposed to reduce costs, including streamlining of institutional review board approval processes, data gathering, site training and monitoring, and selecting trial designs that minimize cost.¹⁰⁹ Innovative approaches will be crucial for studying OSA-related outcomes where potential funding sources are limited (see Table 3).

Table 3.

Alternative study designs.

Short-term randomized controlled trials of treatment withdrawal Pragmatic trials Adaptive trials Patient preference treatment intervention trials Observational studies—quasi-experimental designs ○ Propensity score-based designs ○ Instrumental variable ○ Regression discontinuity ○ Difference-in-difference

Short-term studies of treatment withdrawal for OSA:

Innovative, short-term RCTs of treatment-adherent people with OSA can provide answers to important research questions in a cost-effective manner. Short-term withdrawal trials of CPAP can evaluate the sustained effects of long-term treatment of OSA in suppressing symptoms and improving cardiovascular outcomes (eg, blood pressure).¹¹⁰ These studies have shorter time frames and lower cost and can minimize bias by selecting patients who are adherent to treatment. However, determining the broad applicability of these findings to clinical practice will inevitably require large-scale trials.

Pragmatic study designs:

Pragmatic trial designs are a potential cost-reducing approach to evaluating the real-world effectiveness of therapies in producing long-term benefits.^111,112 Pragmatic trials are conducted in real-world, usual-care settings, with large sample sizes, minimal exclusion criteria, simplified trial structure, and protocols that are well harmonized across sites.¹¹³ Patient-centered outcomes relevant to usual clinical practice are evaluated, and there is increasing use of electronic health record data for both recruitment and ascertainment of study outcomes. Pragmatic RCTs may also be conducted within patient registries by incorporating a randomization module during unselected consecutive enrollment into the registry.¹¹⁴ The enhanced efficiencies of pragmatic trials come with challenges to the integrity of study procedures and data quality across sites with varying expertise, but these challenges can be mitigated with appropriate trial management.^111,112

Adaptive trials:

Adaptive trial designs^100,112,115 are trials that offer “pre-planned opportunities to use accumulating trial data to modify aspects of an ongoing trial while preserving the validity and integrity of that trial.”¹¹⁶ Adaptive enrichment designs could permit enhanced sampling of at-risk groups (eg, identified on the basis of clinical phenotype, physiologic characteristics/endophenotypes, or biomarkers) showing favorable treatment response at prespecified interim analyses. Another response-adaptive approach is to redirect recruitment to treatment arms with more favorable responses at prespecified interim analysis, allowing for abandoning a treatment arm based on a poor interim response and thereby enhancing recruitment to a study arm with a strong interim response, with additional new therapies added as the trial progresses based on prespecified criteria. A variation on this SMART (sequential, multiple assignments, randomized trials) approach would be to allow nonadherent CPAP participants to be rerandomized to an alternative intervention (eg, mandibular advancement, drug therapy, hypoglossal stimulation, etc.).¹⁰⁰ These adaptive approaches, which provide for targeting of responsive subgroups and effective therapies, have the potential to reduce the sample size and trial duration required to demonstrate meaningful effects, thereby reducing the cost of large-scale RCTs.^112,115,116 However, the ability to reduce trial duration may be limited in studies of long-term outcomes given the amount of time needed to accumulate enough data for a meaningful interim analysis. Indeed, this approach has the potential to increase trial duration depending on event and recruitment rates.

Participant engagement and preferences

There is increasing recognition of the importance of patient engagement for a successful trial in all aspects of RCT design and conduct, including the selection of patient-centered outcomes, participant burden, and peer support during trials.^14,117 One promising approach is incorporating patient preference for treatment interventions. For example, research in pain medicine and mental health has demonstrated that allowing patients to choose their preferred method of treatment can improve participant satisfaction and treatment adherence and enhance outcomes.^118,119 In a recent study of participants with OSA, sequential treatment with CPAP and mandibular advancement devices was followed by the option to use either or alternate between treatments over a subsequent 6 months. High rates of adherence were achieved, with patients using alternating treatments showing the highest rate of normalization of outcome scores.¹²⁰ Pragmatic study designs that incorporate this approach may be complicated by how multiple treatments may be reimbursed during a clinical trial. Finally, treatment designs that avoid sham treatment by using an active control arm or that mitigate symptoms of OSA without treating the underlying OSA (eg, pharmacotherapy of EDS)^14,121 may further enhance participant recruitment and retention, although active control arms could increase sample size requirements. Thus, incorporating flexible treatment approaches that allow for patient choice may promote the success of future long-term outcome OSA trials.

Alternative designs to RCTs

As discussed, the implementation of RCTs in symptomatic patients with OSA faces challenges, including large bureaucratic processes, high costs, maintenance of equipoise, and recruitment of the appropriate study population.¹⁰⁹ These issues have led some experts to suggest the need for alternative study designs based on observational data that permit causal inferences (see Table 3).¹⁰¹

A recent American Thoracic Society workshop report¹⁴ considered several approaches to obtain causal inferences from observational data, including instrumental variable, regression discontinuity, difference-in-difference designs, and propensity score-based designs. While these methods can be implemented alone, they can also be utilized together when designing observational studies.

Instrument variable analyses are a statistical methodology,¹²² which can be used to assess causality between a treatment and outcome when observational data is vulnerable to confounders or biases. Instrument variable analyses assume the presence of random, exogenous variation in treatment, which are then used to obtain a causal estimate of the treatment effect and counteract the impact of unobserved confounders. Nonetheless, instrument variable analyses are not without limitations. They are contingent on specific assumptions, such as the instrument’s robust association with the exposure and its absence of direct influence on the outcome. Violating these assumptions can introduce bias into the estimates. Furthermore, identifying valid instruments (eg, distance from care) can be particularly difficult and can sometimes lead to imprecise estimates, making interpretation difficult.

Regression discontinuity design is another quasi-experimental design that can be considered when RCTs are not feasible.^14,123,124 This approach assesses causal effects when a running variable falls below a distinct threshold (eg, AHI of 5 events/h). In this design, individuals above and below the threshold have similar characteristics except for the intervention received,¹²⁵ and the measurement noise of the running variable results in pseudo randomization and allows for the estimate of causal effects. Advantages of regression discontinuity design is that it is relatively easy to understand and harnesses natural experiments around a threshold. Disadvantages of this approach includes the assumption that the running variable is not manipulated, that conclusions regarding intervention effects can only be made around the specific threshold value used, and that results may not be generalizable to other populations.

A difference-in-difference study design can be used to assess outcomes over time after a new program, policy, or other external change is implemented.^126,127 Causal estimates can be determined by comparison of effects before and after the change, if there is a contemporaneous group affected (ie, an intervention group) and unaffected (ie, a control group). One potential example is if there was a sudden change in policy coverage for patients newly diagnosed with OSA. Difference-in-difference approaches have certain advantages such as the ability to estimate causal effects of a sudden change and allowing for control of time-invariant differences between the groups. Limitations of the difference-in-difference study design include the potential for selection bias, difficulty in identifying the timing of the intervention, the requirement that any cross-over between groups is minimized, and the assumption that both groups would have similar outcomes in the absence of the intervention (ie, the parallel trend assumption).

Propensity score-based designs are perhaps the most common approach and aim to replicate a well-conducted RCT by creating treated and untreated groups that are balanced with regard to baseline characteristics associated with clinical outcomes of interest. Statistical theory¹²⁸ has demonstrated that these methods can provide unbiased causal estimates of treatment effects. The main concern with propensity score designs is the potential for unrecognized or unmeasured confounders that could explain some of the differences between the groups. While these unmeasured factors are controlled for to the extent that they are correlated with measured covariates, it cannot be assumed that unmeasured confounders are fully addressed. This is a particular concern for studies that compare treatment-adherent to treatment-nonadherent patients, as the ability of propensity score-based adjustment to adjust for bias from a “healthy user” effect^129–131 has not been demonstrated. Methodologic approaches to describing the robustness of causal inferences from observational studies include the E-value, which quantifies how strong of an association an unrecognized confounder would need to have with both the exposure and outcome (independent of covariate adjustment already performed) to negate the findings.¹³² If this is a large number, it adds confidence to the robustness of results. While caution is warranted, observational studies offer a reasonable alternative design when an RCT is not feasible and they may be more feasible, less costly, and timelier than RCTs in evaluating the efficacy of CPAP therapy in OSA.¹³³ Ultimately, the ability to access CPAP treatment adherence and efficacy data, as well as the to link this data with outcomes in electronic health records, make these approaches particularly promising for OSA-related applications.

To ensure that rigorous observational studies are performed, convening groups of experts in these approaches and sleep medicine clinicians will be necessary.¹³⁴ These collaborations can help identify the appropriate study population, ensure that the study is adequately powered, and identify an appropriate set of measured covariates. As the correlates of CPAP adherence that might affect cardiovascular outcomes remain largely unknown, further investigation of these factors is an important area of investigation to better inform the design of observational studies of CPAP treatment of OSA. Overall, while RCTs remain the gold standard for evaluating the efficacy of treatments, observational studies have important advantages and can serve as a complementary strategy in studying the benefits of OSA treatment.

CONCLUSIONS

The AHRQ report suggests there is insufficient evidence to support treatment of OSA with PAP solely to improve cardiovascular disease and mortality. Furthermore, the AHRQ report did not consider sleepiness as an important, clinically relevant long-term outcome. Improvements in sleepiness and quality of life are key patient-centered outcomes for people living with OSA and a strong motivation for prescribing treatment, which are sustained with long-term CPAP adherence. These important, person-centered benefits were minimally recognized late into the AHRQ report and may be easily missed by readers.¹² Furthermore, the AHRQ report minimized the impact of OSA treatment on MVCs and public safety by failing to include NRCSs designed to address this question. Finally, the AHRQ report narrowly evaluated the potential blood pressure reduction benefits of OSA treatment by examining only the development or resolution of hypertension, rather than also considering changes in blood pressure as a clinically relevant outcome. A consequence of reading the AHRQ report may be that stakeholders erroneously conclude that there are no clinically relevant benefits to the treatment of OSA.

The AHRQ report does provide a compelling rationale for why more studies to address the impact of CPAP and other treatments for OSA on longer-term outcomes are required. Although the report does not provide an explicit roadmap, recent reports, together with the discussion presented here, address that void.^14,17,18 There are important barriers that need to be adequately addressed to safely conduct RCTs in this area, namely management of sleepiness and its consequences during a long-term clinical trial. However, it is equally important to recognize that not all sleepiness is caused by OSA and may be appropriately managed with adequate monitoring and intervention. The failure of recent RCTs of PAP for OSA to demonstrate cardiovascular benefits should allow clinicians, investigators, and patients to be in equipoise for conducting clinical trials in asymptomatic or mildly symptomatic people. Validation of novel metrics that predict cardiovascular risk in such patients should allow the design of RCTs targeting those most likely to obtain cardiovascular benefit. Future trials of OSA should consider innovative RCT designs, targeting specific patient groups while minimizing health care disparities, that are most likely to benefit using biomarkers of risk and response and should incorporate known effective approaches to promote long-term CPAP adherence during a clinical trial. Furthermore, journals should require standardized reporting of the AHI or other metrics of OSA to facilitate direct comparison between studies. In addition to clinical trials, observational studies incorporating quasi-experimental designs should be considered when RCTs are not possible or complementary to RCTs; understanding the factors associated with CPAP nonadherence that might contribute to cardiovascular risk would strengthen causal inferences from such studies. Furthermore, engagement of patient organizations early in the study design process and during study implementation to address patient preference will be critical to ensuring recruitment of reasonably large and representative study samples and ensuring adequate retention. The sleep community including clinicians, investigators, and patients must engage public and private stakeholders to ensure that funding of these initiatives is sustained with an appropriate research roadmap. Until such studies are performed, payors are cautioned against policies reducing coverage for the treatment of OSA, as reduced coverage may have unintended consequences for the millions of people with OSA for whom treatment can provide symptomatic benefit and for the safety of the general public.

DISCLOSURE STATEMENT

All authors have read and approved the manuscript. Susheel Patil reports consulting for Primasun, Inc. and is a current member of the American Academy of Sleep Medicine Board of Directors. Martha Billings reports no disclosures. Ghada Bourjeily reports equipment support from Dreem, Inc. as part of an investigator-initiated grant and honorarium from EBSCO-Dynamed for editorial work. Nancy Collop reports receiving research support from Huxley Medical. Daniel Gottlieb reports service on scientific advisory boards for Signifier Medical Technologies, Inc. and Wesper, Inc. and as a consultant to Powell-Mansfield, Inc. and Apnimed, Inc. and has received research support from Resmed, Inc. Karin Johnson reports no disclosures. R. John Kimoff reports research operating funds from Canadian Institutes of Health Research, Fonds de Recherche du Québec–Santé, Bresotec Inc., Signifier Medical Inc., and Philips-Respironics. He has received consulting fees from Eisai, Powell-Mansfield, and Bresotec Inc. Allan Pack reports that he is a project leader of an National Institutes of Health-funded program project on developing personalized approaches to obstructive sleep apnea.

ABBREVIATIONS

AASM

American Academy of Sleep Medicine

AHI

apnea-hypopnea index

AHRQ

Agency for Healthcare Research and Quality

CPAP

continuous positive airway pressure

EDS

excessive daytime sleepiness

ESS

Epworth Sleepiness Scale

MVC

motor vehicle crash

NRCS

nonrandomized controlled study

OSA

obstructive sleep apnea

PAP

positive airway pressure

RCT

randomized controlled trial