Economic Return From the Women’s Health Initiative Estrogen Plus Progestin Clinical Trial

Abstract

Background

The findings of the Women’s Health Initiative (WHI) estrogen plus progestin (E+P) trial led to a substantial reduction in use of combined hormone therapy (cHT) among postmenopausal women in the United States. The economic effect of this shift has not been evaluated relative to the trial’s $260 million cost (2012 U.S. dollars).

Objective

To estimate the economic return from the WHI E+P trial.

Design

Decision model to simulate health outcomes for a “WHI scenario” with observed cHT use and a “no-WHI scenario” with cHT use extrapolated from the pretrial period.

Data Sources

Primary analyses of WHI outcomes, peer-reviewed literature, and government sources.

Target Population

Postmenopausal women in the United States, aged 50 to 79 years, who did not have a hysterectomy.

Time Horizon

2003 to 2012.

Perspective

Payer.

Intervention

Combined hormone therapy.

Outcome Measures

Disease incidence, expenditure, quality-adjusted life-years, and net economic return.

Results of Base-Case Analysis

The WHI scenario resulted in 4.3 million fewer cHT users, 126 000 fewer breast cancer cases, 76 000 fewer cardiovascular disease cases, 263 000 more fractures, 145 000 more quality-adjusted life-years, and expenditure savings of $35.2 billion. The corresponding net economic return of the trial was $37.1 billion ($140 per dollar invested in the trial) at a willingness-to-pay level of $100 000 per quality-adjusted life-year.

Results of Sensitivity Analysis

The 95% CI for the net economic return of the trial was $23.1 to $51.2 billion.

Limitation

No evaluation of indirect costs or outcomes beyond 2012.

Conclusion

The WHI E+P trial made high-value use of public funds with a substantial return on investment. These results can contribute to discussions about the role of public funding for large, prospective trials with high potential for public health effects.

Primary Funding Source

National Heart, Lung, and Blood Institute.

One of the major debates in public funding of research is its overall return to society. In this context, studies that link National Institutes of Health (NIH)–sponsored research to changes in clinical practice patterns, patient outcomes, and costs are uncommon, although such information could inform debate about the role of publicly funded medical research in the nation’s portfolio of federal health spending.

As an example, a decade ago, results of the Women’s Health Initiative (WHI) estrogen plus progestin (E+P) clinical trial changed the understanding of the risk– benefit profile of combined hormone therapy (cHT) use (1, 2). In 2002, approximately 5.5 million U.S. women used cHT, largely based on clinical trial evidence of vasomotor symptom and osteoporosis benefit and observational evidence that suggested reduced cardiovascular disease risk (3– 6). In July 2002, publication of the E+P trial results provided randomized, controlled trial evidence of increased cardiovascular disease, venous thromboembolism, and breast cancer risk among cHT users (1). Investigators from the WHI concluded that “the [cHT] risk– benefit profile is not consistent with a viable intervention for primary prevention of chronic diseases” (1). After publication of these results, cHT use in the United States decreased by approximately 50% and continued to decline at 5% to 10% annually as the U.S. Food and Drug Administration and other groups endorsed the study conclusions (3, 4, 7–11). Although other studies influenced this shift in use, the timing and magnitude of the shift suggests that most is attributable to the WHI E+P trial (3, 4, 8, 9, 12).

At a cost of approximately $260 million (in 2012 U.S. dollars), the WHI E+P trial was one of the most expensive studies ever funded by the NIH. From the trial’s inception, stakeholders within and outside the NIH debated the rationale for investing considerable public resources in a single large trial with little consideration of its potential economic value (13, 14).

Our study objective was to estimate the clinical and economic return of the trial from a payer perspective by comparing actual observed cHT use with a counterfactual scenario in which the E+P trial was not conducted. We calculated the net economic return of the trial as the difference in net economic benefit between scenarios, minus the trial cost. Our findings can contribute to the current debate about funding for the NIH and other federal agencies that sponsor public research.

Methods

Overview

We developed a disease-simulation model to evaluate clinical and economic outcomes for cHT-eligible women since the initial publication of the E+P trial results (2003 to 2012). We defined “cHT-eligible” as women aged 50 to 79 years who were postmenopausal and did not have a hysterectomy. Our model linked trends in cHT use with disease risk estimates from the WHI to simulate 10-year health outcomes for persons who ever or never used cHT by age group (50 to 59, 60 to 69, and 70 to 79 years) and weighted outcomes to represent the U.S. population. Women were “cHT ever users” if they had any cHT use during the model time horizon and “cHT never users” if they had no cHT use during the same interval. We compared disease incidence, survival, health-related quality of life, and direct medical expenditure outcomes between a “WHI” scenario with observed cHT use and a “no-WHI” scenario with a linear extrapolation of cHT use based on the pretrial (1998 to 2002) trend. The outcomes of each scenario were calculated based on corresponding weighted averages of disease incidence among cHT ever and never users. Direct medical expenditures (hereafter “expenditures”) and quality-adjusted life-years (QALYs) were calculated based on the predicted prevalence of disease over the model time horizon. Together, these outcomes were used to calculate the net economic return of the E+P trial from a payer perspective (that is, we did not consider direct nonmedical or indirect expenditures).

Estimation of cHT Use, Disease Incidence, and Outcomes in the WHI and No-WHI Scenarios

We estimated disease incidence and survival in the WHI and no-WHI scenarios using a Markov state–transition model developed in Excel (Microsoft, Redmond, Washington) (15). Women began in a disease-free state and either remained there or transitioned into one of 11 disease states, which represented all major disease outcomes reported in the E+P trial (Figure 1). The model tracked clinical and economic outcomes in 1-year cycles between the calendar years of 2003 and 2012.

Figure 1.

Disease-simulation model simplified schematics for the cHT decision tree and Markov state transition model structure.

Panel A illustrates the structure of the simulation model decision tree. At the end of the decision tree, women entered the Markov model (“M”) (panel B) and were tracked for long-term health outcomes. Once women transitioned into a disease state, survival, expenditure, and quality-adjusted life-year outcomes were stratified into initial vs. subsequent years in that state. Note that the cHT ever user Markov models divide the “no disease of interest” state into current cHT users and cHT stoppers. cHT = combined hormone therapy; HT = hormone therapy.

* Combined hormone therapy– eligible women entered the model from 2003 until 2012. The first cohort (2003) comprised prevalent hormone therapy users and never users, and subsequent cohorts (2004 –2012) comprised never users.

† The outcomes of interest are coronary heart disease, coronary artery bypass graft/percutaneous transluminal coronary angioplasty, stroke, deep venous thrombosis, pulmonary embolism, breast cancer, endometrial cancer, colorectal cancer, hip fracture, vertebral fracture, and other osteoporotic fracture. Each outcome of interest is tracked as a separate health state in the Markov model.

cHT Use

For both the WHI and no-WHI scenarios, we estimated cHT use over time. In the WHI scenario, year-to-year cHT use was estimated from pharmacy claims information about use in the United States between 2000 and 2009, when 27% of overall HT users received cHT (3). We held cHT use in 2010 to 2012 constant at its 2009 level due to a lack of specific estimates for those years. We first assumed that current cHT users had a 50% rate of cHT cessation in the initial year of use. Then, among women who had been using cHT for at least 1 year, we based cessation on the observed annual decline in prevalent minus incident users.

A similar approach was used in the no-WHI scenario, but 2003 to 2012 cHT use rates were based on a linear extrapolation of the pre-2003 trend (reduction of 1.5% per year) (3, 4, 8). We assumed that 37% of overall HT users received cHT, based on 2000 to 2002 data (4).

We distributed incident and prevalent cHT users into different age groups (50 to 59, 60 to 69, and 70 to 79 years) in accordance with the reported age distributions of cHT users before and after the publication of the WHI results (8).

Disease Risks of cHT Never Users and cHT Users

We estimated age-dependent annualized disease incidence for cHT never users from a previous analysis of combined outcomes in the WHI clinical trial and observational study (n = 40 845) (16). We also analyzed age-dependent disease incidence among cHT users from a previous analysis of combined outcomes in the WHI clinical trial and observational study (16). The model allowed disease risks to depend on duration of cHT use after 2003 (<2 years, 2 to 5 years, and ≥5 years); previous HT use (yes or no); years between menopause and cHT initiation (<5 years and ≥5 years based on age <60 or ≥60 years); and in women who had stopped using cHT, on the interval since cessation.

We also conducted a similar analysis using only WHI clinical trial disease risks (Supplement, available at www.annals.org) (2).

Mortality Inputs

Disease-attributable mortality rates were based on previously reported analyses (Supplement). Age-dependent, other-cause (background) mortality rates were derived from 2010 U.S. life tables. We adjusted background mortality estimates to reflect disease-attributable mortality rates.

Medical Expenditures and Health State Utilities

We based expenditures for each disease of interest on previous economic analyses (Supplement) and calculated cHT prescription expenditures based on the wholesale acquisition cost for conjugated estrogens–medroxyprogesterone acetate tablets (Pfizer, New York, New York) (17). We included additional bisphosphonate and antidepressant prescription expenditures in the WHI scenario to account for use of these medications for primary prevention of fracture and vasomotor symptom relief, respectively (Supplement). We also applied a disease-specific death cost to reflect variable end-of-life expenditures across the included diseases (Supplement).

With information from the Short Form-36 Health Survey from trial participants, we used a validated algorithm to derive Short Form-6D utility index values for health state utilities (18, 19). Utility values range from 0 (dead) to 1 (perfect health) and are intended to reflect relative preferences for given health states. The quality-of-life effect of cHT use was based on the difference between within-participant change scores in cHT and placebo groups from baseline to year 1 of the trial, stratified by utilities by baseline age group (20). Utility values for disease states were based on previous studies involving women of similar age (Supplement).

Effect on Clinical Outcomes and QALYs

We calculated U.S. population-level clinical and economic outcomes by scaling our per-woman outcomes with the estimated number of U.S. cHT-eligible women between 2003 and 2012 (that is, the “affected population”). Our affected population calculations were anchored to U.S. census projections for the number of women aged 50 to 79 years during each year of the model time horizon.

We calculated total expenditures by applying annual disease-attributable expenditures from the health economics literature to the model-predicted prevalence of disease health states (Supplement). We tracked the year that expenditures were incurred and inflation-adjusted values to 2012 U.S. dollars using the medical care component of the Consumer Price Index.

Using survival and utility values, we calculated QALYs, which reflect both duration and quality of life (21–23). In the QALY framework, 10 years of life with a utility of 0.5 is valued equally to 5 years of life with a utility of 1.0 (5 QALYs). We calculated total QALYs by applying health state utilities to the prevalence of each health state.

Net Economic Return

The population net monetary benefit reflects how much economic value was generated (in QALYs) in excess of societal willingness to pay (per QALY) (24 –27). The net monetary benefit can be expressed as the difference between the expenditures and the QALYs realized, scaled by a societal willingness to pay per QALY. In our base case, we used a societal willingness-to-pay level of $100 000 per QALY. Our sensitivity analyses also consider values ranging from $10 000 to $200 000 per QALY, reflecting the plausible range of implied willingness-to-pay levels in the United States (28, 29).

The net economic return reflects how much value the WHI E+P trial generated for postmenopausal women in the United States in excess of the cost of conducting the trial. We calculated the net economic return of the E+P trial as the difference between the population net monetary benefit in the WHI and no-WHI scenarios, minus the cost of the trial. We estimated the cost of the trial using research budget allocations from the National Heart, Lung, and Blood Institute to the WHI study sites and clinical coordinating center between 1992 and 2002. We assumed that 75% of the decline in cHT use (and thus value) was attributable to the WHI E+P trial in the base case, and evaluated alternative scenarios with 50% and 100% of the decline attributed to the trial.

Characterizing Uncertainty

We used Monte Carlo simulation methods to create distributions for input values using 95% CIs or plausible ranges derived from expert opinion (15, 30, 31). Value distributions were sampled 100 000 times and propagated through the model, and expected incidence, expenditure, QALY, net economic return outcomes, and 95% credible intervals were calculated.

We also evaluated alternative scenarios with cHT health-related quality-of-life effect 2- and 4-fold greater than the base-case scenario. These scenarios reflect the possibility that the quality-of-life benefit reported for E+P trial participants could be attenuated because of inclusion criteria that may have excluded women with the most severe vasomotor symptoms.

Validation

We validated our model disease results by comparing simulated annualized disease incidence rates with those from independent studies (Supplement). The results showed that predicted incidence estimates in the WHI scenario closely matched observed population-based incidence estimates for U.S. postmenopausal women across disease states.

Additional information about our model structure, variables, and outcomes is available in the Supplement.

Role of the Funding Source

The WHI program is funded by the National Heart, Lung, and Blood Institute. The funding source had no role in the design, conduct, or analysis of the study or the decision to submit the manuscript for publication.

Results

Hormone Therapy Use

We estimated that there were approximately 39.1 million women eligible for cHT between 2003 and 2012. Among these women, the model estimated that 5.2 million and 9.5 million persons had ever used cHT in the WHI and no-WHI scenarios, respectively. Most (64%) of the 4.3 million additional cHT users in the no-WHI scenario were aged 50 to 59 years. In the WHI and no-WHI scenarios, the mean durations of cHT use among ever users were 3.1 and 4.3 years, respectively.

Clinical Outcomes

Relative to the no-WHI scenario, 126 000 fewer cases of breast cancer, 76 000 fewer cases of cardiovascular disease (coronary heart disease, coronary artery bypass graft or percutaneous transluminal coronary angioplasty, and stroke), and 80 000 fewer cases of venous thromboembolism (deep venous thrombosis or pulmonary embolism) (Figure 2) occurred in the WHI scenario. There were also 263 000 more fractures (hip, vertebral, and other osteoporotic) and 15 000 more cases of colorectal cancer (Figure 2) in the WHI scenario.

Figure 2.

Absolute differences in 10-y disease incidence between WHI and no-WHI scenarios.

Positive values reflect increased disease incidence in the WHI scenario vs. the no-WHI scenario, and negative values reflect decreased disease incidence in the WHI scenario vs. the no-WHI scenario. Errors bars represent 95% CIs as generated by sensitivity analysis. CABG = coronary artery bypass graft; PTCA = percutaneous transluminal coronary angioplasty; WHI = Women’s Health Initiative.

Economic Outcomes

Compared with the no-WHI scenario, the WHI scenario resulted in $35.2 billion in direct medical expenditure savings. Most of the savings came from fewer cHT users and associated office visits ($26.2 billion), decreased breast cancer incidence ($4.5 billion), and decreased cardiovascular disease incidence ($2.2 billion), offsetting increases in expenditures for greater fracture incidence ($4.8 billion) and colorectal cancer ($1.0 billion) (Figure 3). The WHI scenario resulted in 145 000 more QALYs than the no-WHI scenario. This difference was primarily due to the greater health-related quality-of-life effect associated with decreased breast cancer and cardiovascular disease incidence in the WHI scenario, which greatly offset reductions in QALYs due to increased fracture incidence.

Figure 3.

Differences in 10-y direct medical expenditure between WHI and no-WHI scenarios.

Positive values reflect greater expenditure related to changes in the incidence of the listed condition in the WHI scenario vs. the no-WHI scenario, and negative values reflect reduced expenditure for listed condition. Errors bars represent 95% CIs as generated by sensitivity analysis. CABG = coronary artery bypass graft; PTCA = percutaneous transluminal coronary angioplasty; WHI = Women’s Health Initiative.

Net Monetary Benefit and Net Economic Return Outcomes

In the base case, the net monetary benefit in the WHI and no-WHI scenarios was $23.56 and $23.51 trillion, respectively. Thus, the WHI scenario created $49.5 billion (95% CI, $31.5 to $68.2 billion) in additional net monetary benefit compared with the no-WHI scenario, greatly exceeding the inflation-adjusted E+P trial cost of $260 million ($141 million in nominal terms). Under the assumption that 75% of the decline in cHT use (and economic value) is attributable to the WHI E+P trial and a societal willingness-to-pay level of $100 000 per QALY, the expected net economic return of the trial was $37.1 billion (CI, $23.1 to $51.2 billion) (Table). Savings from reduced use of cHT drove the early economic value of the trial, and later value was driven by a combination of cHT expenditure savings and QALY gains (Figure 4).

Table.

Expected Health Outcomes in the WHI and No-WHI Scenarios^*

Scenario	Persons Who Ever Used cHT, n	Person-Years of cHT Use, n	Life-Years, n	QALYs, n	DME, $	Net Monetary Benefit, $†	Net Economic Return, $‡
WHI trial	5.2 million	16.3 million	330.9 million	242.7 million	711.0 billion	23.56 trillion	37.1 billion
No WHI trial	9.5 million	40.8 million	330.6 million	242.6 million	746.3 billion	23.51 trillion	–
Difference	−4.2 million	−24.5 million	0.3 million	0.1 million	−35.2 billion	49.5 billion	–

cHT = combined hormone therapy; DME = direct medical expenditure; QALY = quality-adjusted life-year; WHI = Women’s Health Initiative.

^*There were 39.1 million women eligible for cHT during the model time horizon.

^†Calculated at a societal willingness-to-pay level of $100 000 per QALY.

^‡Calculated as the difference in net economic benefit minus the cost of the estrogen plus progestin clinical trial ($260 million) and assuming that 75% of the net monetary benefit is attributable to the WHI estrogen plus progestin trial.

Figure 4.

Annual net economic return created by expenditure savings and QALY gains in the WHI and the no-WHI scenarios.

QALY = quality-adjusted life-year; WHI = Women’s Health Initiative.

Alternative Assumptions and Scenarios

The alternative analysis with only WHI clinical trial disease risks resulted in similar expected net economic return but had increased uncertainty because of a smaller sample size ($35.9 billion [CI, $14.8 to $57.1 billion]).

Under alternative assumptions that 50% and 100% of the decline in cHT use could be attributed to WHI E+P trial (vs. other cHT findings reported in the same period), the net economic return was $24.8 billion (CI, $15.4 to $34.1 billion) and $49.5 billion (CI, $30.8 to $68.2 billion), respectively.

At willingness-to-pay levels of $50 000 and $200 000 per QALY, the net economic return was $31.7 billion (CI, $21.2 to $42.2 billion) and $48.1 billion (CI, $26.3 to $69.8 billion), respectively.

With 2- and 4-fold greater cHT quality-of-life benefit, the net economic return was $23.7 billion (CI, $17.2 to $30.2 billion) and $14.4 billion (CI, $10.5 to $18.3 billion), respectively.

Discussion

In this study, we used disease-simulation modeling, informed with observational data of cHT prescribing and risk information from the WHI E+P trial, to estimate the net economic return of one of the largest public research investments of recent decades. Over the 10 years since the main trial findings were released, we estimate that practice changes resulting from the WHI E+P trial led to 126 000 fewer breast cancer cases, 76 000 fewer cardiovascular events, 263 000 more osteoporotic fractures, and 15 000 more colorectal cancer cases. The net health yield for women in the United States was approximately 145 000 more QALYs than would have occurred in the absence of the trial. Together, these clinical and economic outcomes amount to a net economic return of $37.1 billion, a return of approximately $140 on every dollar invested in the trial. In the probabilistic sensitivity analysis, 99.7%, 84.7%, and 32.3% of simulations resulted in net economic returns of $20 billion or greater, $30 billion or greater, and $40 billion or greater, respectively.

Of the $37.1 billion in net economic return attributable to the WHI E+P trial, $26.4 billion was attributable to medical expenditure savings. These savings were driven by 25 million fewer person-years of cHT use, as well as cost savings from avoided diseases. The remaining $10.7 billion represents the value of additional quality-adjusted life expectancy resulting from lower incidence of breast cancer, cardiovascular disease, and venous thromboembolism. The strong influence of breast cancer on the value of the trial is attributable in part to the younger age distribution of cHT users in the population, the relatively high incidence of breast cancer among younger women, the magnitude and persistence of increased risk for breast cancer (especially among early cHT initiators), relatively long postdiagnosis survival, and the high cost of cancer care (16). The ethics of valuing QALYs (and the value of a QALY itself) has been debated widely in the United States (32–35). In this context, even if the monetary value of QALYs gained is not considered, the net economic return from posttrial shifts in cHT use remains very high (approximately $26 billion).

Because the WHI E+P trial was unusual in size and scope, it is important to put our findings in the context of the larger debate about the value of publicly funded health research. Based on observational data that support cardio-protective effects, the primary motivation for the trial was establishing the effect of cHT on coronary heart disease, with hip fractures and breast cancer as secondary and safety outcomes, respectively (36 –38). This trial design was controversial, in part because many believed that the observational data on coronary heart disease were highly compelling. For example, a 1993 Institute of Medicine review noted that “the study is likely to terminate early because of evidence demonstrating protection against coronary heart disease, thereby precluding the identification of later occurring outcomes” (14). Although many trials, both large and small, are motivated by similar clinical controversies, most will result in more modest returns or, perhaps, losses. To generate an economic return similar to the WHI E+P trial, a trial would have to yield findings that changed a widely accepted practice affecting millions of patients and the health gains accrued as a consequence of the practice change would need to be relatively large. Our analysis of the economic return from the WHI E+P trial suggests that, in certain circumstances, public investments in large prospective trials with high clinical and public health relevance could provide a similarly large positive rate of return in the long term (39 – 41).

Our decision model is a simplified representation of complex and interrelated biological, clinical, and economic factors, which does not account for all potential cHT disease associations or possible correlations between model inputs. We focused on the primary outcomes evaluated in the WHI E+P trial because of their strong associations with cHT use and their economic and health-related quality-of-life consequences. We do not consider unopposed estrogen hormone therapy use or health outcomes. We also do not consider direct nonmedical (travel, parking, and lodging) or indirect (productivity loss) disease expenditures or clinical and economic outcomes beyond 2012. Extending the period of analysis would probably increase net economic return several-fold but would also introduce substantial uncertainty. In addition, we used combined WHI clinical trial and observational study baseline disease risks to approximate risk in the U.S. cHT-eligible population, although the U.S. population probably has greater disease risk than these study volunteers. If this were uniformly true across the included diseases, the E+P trial net economic return would be greater than estimated in our analysis. A related limitation is that our disease-simulation model allows women to develop only one disease of interest during the model time horizon. We believe that this approach is reasonable because only a small proportion of women are expected to have multiple diseases of interest and the diseases with increased incidence among cHT users may have greater comorbid effects than those with decreased incidence (resulting in a conservative trial value estimate), in accordance with previous findings (42). Costs and benefits were not discounted to simplify the analysis. We took this approach because it is unlikely that a conventional 3% discount rate would qualitatively alter the substance of our findings over the time horizon of our analysis.

The Women’s Health Initiative E+P trial changed understanding of cHT risk– benefit balance and was a major driver of the precipitous and sustained decline in hormone therapy use in the United States between 2003 and 2012 (3, 4). This substantial decision-making effect has been attributed to the high-level evidence provided by the trial’s randomized design, large sample size, long-term follow-up, and evaluation of many clinically meaningful end points. These design characteristics were feasible because of the $260 million public investment in the trial and dedicated study participants that continue to contribute their time to help answer these important research questions. Our findings suggest that these investments have yielded a return of approximately $37 billion in net economic return in the past decade. This level of value was robust across plausible uncertainty ranges and remained greater than $20 billion in all simulations that we evaluated. Overall, our findings suggest that large public research investments can yield considerable clinical and economic value when targeted to address research questions with great clinical relevance and public health effect.

Context

Use of combined hormone therapy as a prevention strategy for cardiovascular disease significantly declined during the past decade after publication of the results of the Women’s Health Initiative (WHI).

Contribution

Mathematical models compared changes in combined hormone therapy use, morbidity and mortality rates, and costs for WHI versus no-WHI scenarios. The WHI scenario resulted in substantially fewer women with adverse health outcomes and billions of dollars in net economic savings.

Caution

The WHI reversed a widely accepted practice affecting millions of women and resulted in large population health gains.

Implication

Large public investments directed toward trials that address questions with high clinical relevance and public health influence may yield considerable returns.

—The Editors