The Challenge of Undiagnosed Sleep Apnea in Low-Risk Populations: A Decision Analysis

Abstract

Objectives:

Obstructive sleep apnea (OSA) may contribute to impaired performance among otherwise healthy active duty military personnel. We used decision analysis to evaluate three approaches to identifying and treating OSA in low-risk populations, which may differ from current standard practice for high-risk populations.

Methods:

We developed a decision tree to compare two simple strategies for diagnosis and management of sleep apnea in a low-risk population. In one strategy, a simple screening inventory was followed by conventional laboratory polysomnography (split-night), whereas the alternative strategy involved performing home testing in all individuals. This allowed us to weigh the costs associated with large-scale diagnostic approaches against the costs of untreated OSA in a small fraction of the population.

Results:

We found that the home testing approach was less expensive than the screen-then-test approach across a broad range of other important parameters, including the annual performance cost associated with untreated OSA, the prevalence of OSA, and the duration of active duty.

Conclusions:

Assuming even modest annual performance costs associated with untreated OSA, a population strategy involving large-scale home testing is less expensive than a screening inventory approach. These results may inform either targeted or large-scale investigation of undiagnosed OSA in low-risk populations such as active duty military.

INTRODUCTION

Obstructive sleep apnea (OSA) is a prevalent disorder associated with morbidity and mortality, as well as impaired work performance and motor vehicle accidents^1–3 with annual cost burden estimates in the range of $50 billion in the United States.⁴ OSA is traditionally diagnosed by laboratory polysomnogram (PSG), according to the frequency of apneas and hypopneas observed during sleep. According to recent guidelines from the American Academy of Sleep Medicine, OSA is diagnosed by having either an apnea–hypopnea index (AHI) > 15 regardless of symptoms, or an AHI > 5 if accompanied by snoring, sleepiness, or related symptoms.⁵

Despite the health and performance consequences of OSA, and the variety of available treatments, most individuals remain undiagnosed.^6,7 Identifying OSA is particularly challenging in low-risk populations, such as young healthy active duty military personnel. The prevalence of OSA depends on the definitions used with more strict definitions yielding 2% to 4% prevalence in adults, whereas more modern definitions yielding 10% to 15% prevalence.⁸ In healthy younger populations, the values may be lower than typically reported in epidemiological studies, but occult OSA can still occur in 5% even after extensive screening.⁹ The challenge involves imperfect screening tools¹⁰ that are vulnerable to false positive findings when applied in low-prevalence populations, as is the case for modern OSA screening tools.¹¹ Although PSG is the gold standard diagnostic test, using PSG to screen for OSA in large populations with low disease prevalence may not be feasible due to limitations of cost and availability. The growing availability of validated home sleep devices has offered an alternative approach,^12,13 but the assumption that a home diagnostic framework is necessarily cost-saving has been questioned.^14,15

Another important challenge is that the standard Epworth Sleepiness Scale correlates poorly with the severity of OSA,^16,17 or with objective sleepiness measured by multiple sleep latency testing.¹⁸ The consequence of this disease–symptom dissociation is that one cannot expect individuals with OSA to “declare themselves” clinically on the basis of symptoms, even when those symptoms are elicited in the structured context of a medical evaluation. In recognition of this challenge, various inventories have been developed to improve the yield of clinical screening in order to triage at-risk individuals for formal testing.¹⁰ However, these tools demonstrate only modest performance characteristics, making them particularly vulnerable from a Bayesian standpoint when applied to either high- or low-risk populations. Specifically, the challenges of interpreting unexpectedly positive results in low-prevalence populations (in which there is an increased risk of false positives), or unexpectedly negative results in high-risk populations (in which there is an increased risk of false negatives), are not straightforward.¹⁹

Here, we performed a decision analysis to evaluate two approaches to undiagnosed OSA in low-risk populations. Although we recognize many health-related motivations for diagnosis and treatment of OSA, our model is intended to emphasize the performance impairments that might accompany OSA in a healthy population. Thus, the overarching goal is to provide a framework for balancing the performance related costs against the costs associated with diagnosis and treatment of OSA in the special setting of a low-risk active duty population.

METHODS

Modeling was performed using TreeAge Pro 2011 (TreeAge Software, Williamstown, Massachusetts). Unlike many models presented in the literature, our model considers only the cost of each decision. Given our intention of offering a framework for approaching diagnostics in a healthy, low-risk population, the main considerations are the costs associated with decreased performance among those with undiagnosed OSA weighed against the costs associated with diagnostic evaluation and treatment. We did not therefore explicitly consider quality of life or health outcomes, which undoubtedly will impact a cost–benefit analysis as one considers older and less healthy military personnel and veterans. However, for modeling purposes in this presumed healthy population, performance-related costs may outweigh health considerations (although health risks are expected to improve the cost–benefit favorability of identifying and treating occult OSA).

Model Structure

The tree consists of a decision node and three main branches representing strategies to approach undiagnosed OSA in a low-risk population. Every branch after the decision node is a chance node, that is, each binary pathway option is determined by a probability. Names of pathways are given above the lines, whereas the probability values associated with the path are given below the lines. The “#” sign indicates the complement of the probability value given on the upper branch of each chance node. The triangles indicate terminal nodes, where the costs of each step in the corresponding pathway are collected.

Model Parameter Estimates and Assumptions

We used model parameter values from the decision analysis of Pietzsch et al.²⁰ The cost of OSA testing with a typical “level 3” home monitor was $210 per person for a single-night test; we included a 10% failure rate, requiring repeat testing, such that the cost was adjusted upward by 10% to $231 per person. The cost of continuous positive airway pressure (CPAP) treatment per year includes regular replacement of disposable equipment such as tubes and filters ($114 per year), and two office visits per year ($180 per year). We estimated the cost of a machine at $1,200, assumed to be spread over 10 years (assuming it would need to be replaced every 10 years). Thus, the total CPAP cost per person per year was thus the sum of these: $114 + $180 + $120 = $414 per year. The cost associated with rejecting CPAP after a trial period of home use was assumed to be the cost of a 3-month rental ($112 × 3 = $336) plus a single clinic visit ($90), for a total one-time cost of $426 per person. The cost of PSG, which was assumed to be a single split-night study, was $891. We did not discount costs over time, nor did we model the possibility of individuals initially accepting CPAP rejecting it at a later date, or of those rejecting it initially reconsidering it at a later date. We did not model the human cost of durable medical equipment company staff performing an initial setup. The sensitivity and specificity values for split-night PSG and for home monitor are shown in Table I. The baseline values for sensitivity and specificity of the screening inventory is taken from Chung et al,²¹ as determined for the binary detection of OSA when defined by the cutoff value of AHI > 5.

TABLE I.

Parameter Estimates for the Baseline Model

Parameter	Base Value	Parameter	Base Value
Sensitivity Home Monitor	0.91a	Cost Screen	$10c
Specificity Home Monitor	0.83a	Cost Home Monitor	$231a
Sensitivity PSG	0.89a	Cost PSG	$891a
Specificity PSG	0.94a	Cost CPAP	$414/Yeara
Sensitivity Screen	0.84b	Cost CPAP Reject	$426a
Specificity Screen	0.56b	Cost UnTx OSA	$10,000/Yearc
Pretest OSA	0.05c	p(Accept CPAP if OSA)	0.70c
Duty Years	20c	p(Accept CPAP if no OSA)	0.25c

^aPietzsch et al.²⁰

^bChung et al.²¹

^cEstimated.

Estimating the performance cost associated with untreated OSA is not straightforward. There is likely to be a distribution of potential costs associated with the spectrum of performance consequences, such as risk for motor vehicle or on-the-job accidents, the costs associated with such accidents, and the impact of poor performance on different types of work-systems. We simplified this complex problem by assuming as a baseline estimate that untreated OSA conferred a 20% reduction in work efficiency. Assuming an annual salary of $50,000, the reduction in work efficiency would amount to $10,000 per year. We performed several sensitivity analyses involving this variable, not only because it is the most uncertain of the model, but also because the plots allow one to see how this value alters the optimal decision. The duration of service no doubt involves many factors and accordingly was subjected to sensitivity analyses.

We estimated the cost associated with administering a simple screening test as $10, the probability of accepting CPAP as 70%. We made the conservative assumption that the probability of continuing with CPAP in the absence of OSA as 25% (the true value is not known, and may be much lower).

At the terminal nodes of each pathway, the costs are calculated in a weighted fashion according to the probabilities at each chance node and are evaluated from the perspective of an individual across the time horizon of the duration of active duty. The costs for the “do nothing” branch include only the costs of untreated OSA per year, multiplied by duty years. For the Screen branch, the one-time costs include administering the screen, the PSG for those testing positive, and the cost of rejecting CPAP. Accumulated costs per year of active duty include the cost of CPAP treatment per year, and the cost of untreated OSA per year. For the Home testing branch, the one-time costs include administering the home monitor (adjusted for the 10% chance of needing to repeat this one more time), and the cost of rejecting CPAP after a brief home trial of treatment. Accumulated yearly costs for this branch, like the others, include the costs of untreated OSA and the cost of ongoing CPAP treatment.

RESULTS

Model Structure

In this decision analysis, we assumed three strategies for approaching undiagnosed OSA (Fig. 1). Each strategy, represented by the first branches of the tree stemming from the decision node, will be evaluated in terms of the expected cost of the decision, which incorporates costs downstream of the decision path. The “do nothing” branch simply reports the cost associated with untreated OSA in the subset of the population defined by the pretest probability of OSA. The “screen” branch employs a screening test to determine who will undergo laboratory PSG, which in this model is assumed to be conducted as a split-night study. This is a conservative estimate of cost, by avoiding the expense associated with a two-PSG approach (one diagnostic and one titration). The “home monitor” branch employs a device used in the home to detect OSA.

FIGURE 1.

Model structure. The Decision Node (square) is shown at the left (“Strategy”). Each chance node is given by a circle from which two branches project, each with a probability under the path line (italics), and a descriptive name above the path line. The “#” sign indicates the complement of the probability given on the other path line of each pair. The terminal nodes, indicated by triangles aligned on the right side, indicate the final step in the model. Costs are accumulated at these nodes (see Methods). “OSA” and “No OSA” refer to the true disease status of an individual; note that it is placed at the first chance node to simplify the tree structure, but would not be known clinically. TP, True Positive; FP, false positive; TN, true negative; FN, false negative; PSG, polosomnogram (split-night); prev, prevalence; sens, sensitivity; spec, specificity. The probability of accepting CPAP given true OSA is p(AcceptCPAP), While the probability of accepting CPAP given no OSA is p(AcceptCPAP*).

For the screen and home monitor branches, the probability of each possible outcome of testing is given by chance nodes, and include true positive and false positive (for each positive test result) and true negative and false negative (for each negative test result). The tree simplifies the clinical approach to OSA as follows. For the screening branch, one administers a simple screening test, and only those with positive results are sent for laboratory PSG. Those testing positive in this setting are given CPAP. For the home monitoring branch, one administers a device to quantify OSA in the home setting, and only those with positive test results are given CPAP. Those receiving CPAP may or may not accept the treatment after a trial period. Note that the pretest probability in this model refers to the prevalence of undiagnosed OSA; we do not consider costs associated with the diagnosis and management of OSA in the course of routine clinical care. Thus, the model is meant to provide insight into how to approach the problem of occult OSA in a large population with presumably low OSA risk. We note that considering whether AHI > 5 or AHI > 15 is the pertinent cutoff is contained in the pretest probability parameter. That is, if the prevalence of AHI > 5 is 5%, then the prevalence of AHI > 15 might be, say, 2%. Thus, one can perform sensitivity analysis of the prevalence of undiagnosed OSA as a proxy to also consider what severity of OSA is relevant to treat.

The baseline parameter values are shown in Figure 1 and also listed in Table I. A detailed description of the assumptions underlying each parameter is given in the Methods section. Analysis of the model using the baseline parameter values indicated that the home testing arm was favored in terms of cost. Specifically, over the 20-year time horizon, doing nothing costs $10,000 per person, whereas the screen option costs $5,468 per person, and the home testing option costs $4,516 per person. Note that these values are per person over 20 years, as time is included as a variable in the model; the per year cost is 10,000/20, or $500 per person per year. Considering there are approximately 1,000,000 active duty military personnel, this would amount to an expected cost savings between doing nothing and the favored approach of population home testing of approximately $5,000 × 1,000,000 or $5 billion ($250 million per year).

We emphasize that the process of decision modeling necessarily involves uncertainty with regards to the model parameters. This can be as a result of uncertainties in the published literature, or in whether a particular patient is sufficiently similar to those enrolled in clinical trials to extrapolate the results to clinical care. Performing sensitivity analyses addresses uncertainty in particular parameters by systematically varying them and re-evaluating the model results. In the subsequent sections, we will perform one-way and two-way sensitivity analysis to evaluate the impact of uncertainty in key parameters on the preferred approach using our model.

One-Way Sensitivity Analysis

To determine how uncertainty in key parameters will influence the costs associated with each branch of the initial decision node, we conducted a series of one-way sensitivity analyses, in which a single variable was evaluated across a range of possible values, while holding all other parameter values constant. In each case, the outcome is expressed as the expected value of cost, across the active duty duration, per individual.

Varying the cost of untreated OSA revealed that doing nothing was the most expensive option, even for a very low annual cost of untreated OSA of approximately $1,000 (Fig. 2A). The crossover point was approximately $1,500 when the duty years value was set at 10 (data not shown). Although we used a 20-year service duration for the base case above, here and subsequently we used a 10-year time horizon that may be more realistic of the spectrum of service durations throughout the population, and will be a more conservative measure of cost (i.e., favoring do-nothing).

FIGURE 2.

One-way sensitivity analyses. The expected costs incurred, per person, is shown across variation of several parameters, including (A) the cost of untreated OSA per year, (B) the pre-test probability or prevalence of untreated OSA, (C) the duration of active duty in years, (D) the cost of home testing per person, (E) the probability of accepting CPAP among those individuals with OSA, and (F) the probability of accepting CPAP among those individuals without OSA. In each panel, the plots include the costs associated with the do nothing branch (blue), the screen branch (red), and the home monitor branch (yellow).

Varying the pretest probability of undiagnosed OSA revealed that doing nothing was the most expensive option for prevalence values above 0.5% (Fig. 2B). The crossover point was approximately 0.8% when the duty years value was set to 10 (data not shown).

Varying the duration of service (duty years) revealed that doing nothing was the most expensive option for durations of service greater than 1 year (Fig. 2C). The crossover point increased to 2 years when the annual cost of untreated OSA was decreased to $5,000 (data not shown).

Varying the cost of home testing (Fig. 2D), the probability of accepting CPAP given the presence of OSA (Fig. 2E), or the probability of accepting CPAP given the absence of OSA(i.e., false positive individuals), each showed that doing nothing remains clearly more expensive across a range of values. Additionally, varying the annual cost of CPAP from $200 to $800 showed continued preference for the home monitor pathway (data not shown), similar to the results of home testing analysis (Fig. 2D).

In all of the one-way analyses (including those indicated as not shown), the screen branch was somewhat more expensive than the home monitor branch.

Two-Way Sensitivity Analysis

In a series of two-way sensitivity analyses, we evaluated how variation in the values of pairs of parameters might alter the preferred strategy. These analyses revealed several interesting findings of practical relevance to approaching OSA in low-risk populations.

Varying service duration and cost of untreated OSA revealed a strong preference for the home monitor branch over doing nothing (Fig. 3A). The contour border between these two strategies indicates that the minimum cost of untreated OSA required to favor the home monitor branch rapidly decreases as service duration increases. In other words, doing nothing is favored from a cost standpoint mainly when service duration is short.

FIGURE 3.

Two-way sensitivity analyses. (A) When varying the cost of untreated OSA and the service duration, the home monitor branch (red) is favored over doing nothing (blue) for most of the parameter space. The screen branch is a thin line at the interface of these two options, and is not well visualized.(B) Similarly, when varying the cost of untreated OSA and the pretest probability of undiagnosed OSA, the home monitor branch (red) is favored over doing nothing (blue) for most of the parameter space (and again the screen option is poorly visualized in between these options. When varying the sensitivity and specificity of the screen tool, the screen (yellow) is favored for a larger parameter space when the pre-test probability of undiagnosed OSA is 2.5% (C) than when it is 5% (D). Conservative estimates of the cost of untreated OSA ($5,000 per year) and duty duration (10 years) were implemented in panels (C) and (D). When varying the cost of home monitor administration and the cost of untreated OSA, the home monitor branch is favored for most of the parameter space when the pre-test probability of OSA is 2.5% (E) and more so if it is 5% (F). In panels (E) and (F), conservative estimates were implemented for screen performance (84% sensitivity and 84% specificity) and active duty duration of 10 years.

Varying pretest probability of undiagnosed OSA and cost of untreated OSA similarly revealed a strong preference for the home monitoring strategy over doing nothing (Fig. 3B). The contour border between these two strategies indicates that the minimum cost of untreated OSA required to favor the home monitor branch rapidly decreases as the pretest probability of undiagnosed OSA increases. Repeating these two-way analyses assuming marked improvement in the screening test specificity to 84%, which would make the screen superior to any current validated inventory, increased the parameter space favoring the screen strategy, but home testing remained the optimal choice for the majority of the parameter space (data not shown).

Given that the performance of the screen branch depends on the performance characteristics, and to allow for the potential for improved (yet inexpensive) screens to be developed in the future, we conducted two-way analyses on the sensitivity and specificity of the screen when the pretest probability of undiagnosed OSA was either 2.5% (Fig. 3C) or 5% (Fig. 3D). We fixed the service duration at 10 years, and the cost of untreated OSA at $5,000 per year, as more conservative estimates that occurred near the “elbow” of the contours shown in the one-way sensitivity analysis (Fig. 2). The results show that an improved hypothetical screening test must be considerably more accurate than currently available screens in order to favor the screen branch in terms of cost. Note that the higher pretest probability (5%) of undiagnosed OSA is shown to require better screen performance in order to favor the screening strategy. This is a consequence of the false negative risk incurring progressively larger cost burden of untreated OSA as the prevalence increases.

Next, we varied the cost of home testing and the cost of untreated OSA, when the pre-test probability of undiagnosed OSA was either 2.5% (Fig. 3E) or 5% (Fig. 3F). For these plots, we made conservative assumptions regarding the screening test performance improving to 84% sensitivity and 84% specificity, and service duration of 10 years (each of which would be expected to favor screening). These plots show the tendency to favor the home monitor branch as cost of untreated OSA increases. However, certain combinations of low cost of untreated OSA and high cost of home monitor use will favor screening with a tool that has superior sensitivity and specificity than those currently available.

DISCUSSION

Our decision model addresses the important question of balancing the costs of untreated OSA against those associated with large-scale diagnostics and disease management. Our model differs from typical decision analyses of involving OSA in that we focused on costs associated with performance impairments related to untreated OSA, rather than the morbidity and mortality linked to OSA. This model provides a framework for answering key questions, including (1) what range of performance costs attributable to OSA warrant large-scale screening?, (2) what range of expected service time maintains the cost favorability of screening?, and (3) should the diagnostic approach begin with simple screening inventories before PSG, or should large-scale home testing be undertaken? This latter question is of special interest, given that the current American Academy of Sleep Medicine guidelines indicate that home sleep devices should only be used in those at high risk of OSA.¹³ The recommendations of this guideline were based on available evidence for diagnostic accuracy (which was felt to be limited), and focused on the use of home devices to target the confirmation of OSA in patients with high clinical suspicion, rather than using the home monitors for screening. In contrast, our results indicate that in certain settings, even modest costs attributable to untreated OSA outweigh the cost of large-scale screening, even for very low OSA prevalence (≤5%). Our results also indicate that screening inventories can theoretically be less expensive than large-scale home testing, though this would require considerable improvements over currently available screening tools.¹³

The Performance Cost of Untreated OSA

This is arguably the most important and also most uncertain parameter in our model. Performance impairment may have consequences spanning work and automobile accidents, as well as general decreased work efficiency. Although the spectrum of potential costs is vast, this is precisely the setting in which sensitivity analyses can prove most useful. Here, we see that the cost of large-scale home testing is less than the cost of untreated OSA even when the performance-associated costs are modest (under $5,000 per year). When considering implementation of OSA testing, a targeted approach to focus on those military personnel serving roles predicted to be most sensitive to sleepiness or attention could further optimize the potential cost savings.

The Role of Risk Stratification With Screening Tools

Ideally, one could use easily acquired demographic or symptom data to risk stratify large populations without incurring substantial cost, such as administering the STOP-BANG inventory to all active duty members. However, our results suggest that the performance of such a screen must have substantially higher performance characteristics (sensitivity and specificity) than currently available tools. Even assuming better performance of the screen strategy in our modeling, the cost of performing home testing on all service members was balanced by even modest costs associated with untreated OSA.

Limitations and Future Directions

Our model has several limitations. We implemented simplifying assumptions to provide a general framework to approach OSA in low-risk populations. For example, we assumed that individuals accepting CPAP continue to use it, and those who reject it do not have the opportunity to reconsider or to pursue CPAP alternatives. We also do not model the infrastructure costs that might be incurred in population screening or capital investment in leasing home monitors. Those who reject CPAP in our scenario might come from either true OSA cases or from false positive cases without OSA—a model incorporating further investigation of CPAP rejection could more closely approximate clinical workflow. Future models may include the long-term costs and treatment benefits associated with OSA, as well as alternatives to CPAP in those who are intolerant but warrant treatment. We limited ourselves here to young healthy population with low OSA prevalence, but as the military population ages, important health problems may accumulate for which OSA is a risk factor (such as hypertension, heart attacks, stroke, and diabetes). Several groups have published cost–benefit decision analyses in this regard.^22–25