Valuing vaccination

Abstract

Vaccination has led to remarkable health gains over the last century. However, large coverage gaps remain, which will require significant financial resources and political will to address. In recent years, a compelling line of inquiry has established the economic benefits of health, at both the individual and aggregate levels. Most existing economic evaluations of particular health interventions fail to account for this new research, leading to potentially sizable undervaluation of those interventions. In line with this new research, we set forth a framework for conceptualizing the full benefits of vaccination, including avoided medical care costs, outcome-related productivity gains, behavior-related productivity gains, community health externalities, community economic externalities, and the value of risk reduction and pure health gains. We also review literature highlighting the magnitude of these sources of benefit for different vaccinations. Finally, we outline the steps that need to be taken to implement a broad-approach economic evaluation and discuss the implications of this work for research, policy, and resource allocation for vaccine development and delivery.

The prevention of disease and death through vaccination is commonly regarded as one of the greatest public health achievements of the 20th century (1, 2). Globally, coverage with all major vaccinations has drifted up since 2000 (Fig. 1) (3). Today more than 100 million children are vaccinated annually against diseases such as diphtheria, tetanus, pertussis, tuberculosis, polio, measles, and hepatitis B (4). These and other vaccinations prevent an estimated 2.5 million deaths each year (4). Vaccination programs have also led to the eradication of smallpox, the near eradication of polio, and an estimated 74% reduction in measles deaths over the last 10 years (4, 5).

Fig. 1.

Worldwide coverage of several vaccinations over time (2000–2012); presented as the percent of the target population that has received a given vaccination.

Despite these successes, an estimated 23 million infants did not receive routinely recommended vaccinations in 2012. Even larger coverage gaps are seen among newer vaccinations such as those that protect against Haemophilus influenzae type b (Hib), pneumococcal disease, and rotavirus. This situation reflects the fact that three of the four countries with the largest under-five populations in the world—China, India, and Indonesia—have yet to incorporate several such vaccinations in their national immunization programs. New vaccinations against human papillomavirus (HPV) have been introduced into national immunization programs in 45 countries, but coverage varies and remains relatively low in several of them (6–8).

According to recent estimates, it will require roughly USD $50–60 billion to scale up coverage for routinely recommended and new vaccinations—including those against HPV and prospective vaccinations against dengue and malaria—in 94 low- and middle-income countries from 2011 to 2020 (9). By comparison, the current biennial budget of the World Health Organization is roughly USD $4 billion (10). Major financial commitments are required from governments and other stakeholders to fund this scale-up of vaccination. Such commitments can be justified on many grounds, including the fact that vaccination safeguards health, which is a fundamental human right and intrinsically valuable. However, when governments are faced with difficult decisions about how to allocate scarce resources, systematic comparisons of the benefits and costs of each option can be quite important.

In recent years, the instrumental value of health for economic development has been well researched and documented (11). It has been shown that population health can operate through multiple channels to provide a significant boost to economic growth, which can in turn generate additional resources to invest in health. Healthy adults tend to work longer and harder; healthy children tend to have better records of school attendance and educational attainment and better cognitive function (11, 12); and healthy populations tend to save more and to attract more foreign direct investment (FDI) contributing to capital accumulation, job creation, and technological progress (13, 14). In addition, healthy populations tend to have relatively low fertility rates and a correspondingly reduced burden of youth dependency (15).

This article builds on two key premises: first, that vaccination has had, and can continue to have, a potent role in promoting population health, and second, that health is a robust and powerful driver of economic well-being. We describe a theoretical framework that highlights the full economic benefits of vaccination, which extend well beyond the benefits traditionally captured by economists in economic evaluations of vaccinations. We also review evidence on the magnitude of these benefits and outline an approach to the economic evaluation of vaccination that identifies and takes account of these benefits. We conclude with a critical discussion of the implications of this work for research, policy, and resource allocation.

Broader Perspective on the Value of Vaccination

Many health interventions, including vaccinations, have been subjected to economic evaluation. Historically, economists have taken a narrow approach to valuing vaccination’s benefits by focusing strictly on a subset of the potential benefits, mainly averted health care spending. Some existing studies also capture those productivity gains that arise because vaccination protects people from losing productive time due to their own health care utilization or the need to provide or seek care for their children or other household members. These narrow sources of economic benefit associated with vaccination are closely linked to health care and, for most routinely recommended vaccinations, are large relative to the cost of vaccination (16). Nonetheless, failing to account for the full spectrum of benefits—including some important instrumental effects of vaccination on economic well-being—will result in an undervaluation of vaccination. The failure to measure the full benefits of vaccination could be especially important when evaluating the new generation of more expensive vaccines, at least some of which will require heavy ancillary investment in nonvaccine costs such as cold chain storage space and human resource capacity (17). To address this undervaluation bias, we argue for expanding the conceptualization of vaccination benefits, moving from a narrow to a broad (i.e., full benefits) approach (Table 1).

Table 1.

Framework of vaccination benefits

Perspective		Benefit categories	Definition
Broad	Narrow	Health care cost savings	Savings of medical expenditures because vaccination prevents illness episodes
		Care-related productivity gains	Savings of patient’s and caretaker’s productive time because vaccination avoids the need for care and convalescence
		Outcome-related productivity gains	Increased productivity because vaccination improves physical or mental health
		Behavior-related productivity gains	Vaccination improves health and survival, and may thereby change individual behavior, for example by lowering fertility or increasing investment in education
		Community health externalities	Improved outcomes in unvaccinated community members, e.g., through herd effects or reduction in the rate at which resistance to antibiotics develops
		Community economic externalities	Higher vaccination rates can affect macroeconomic performance and social and political stability
		Risk reduction gains	Gains in welfare because uncertainty in future outcomes is reduced
		Health gains	Utilitarian value of reductions in morbidity and mortality above and beyond their instrumental value for productivity and earnings

Modified from refs. 18–21.

This broad approach considers the benefits that come from avoiding the long-term mental, physical, or cognitive impairments that many vaccine-preventable diseases can cause (22–26), for example, blindness resulting from measles infection (27, 28), hearing loss from mumps (29), or cognitive diminution from intrauterine rubella (30). Naturally, these health problems (and avoiding them via vaccination) can impact educational attainment, adult earnings, and social functioning (31–35). Thus, outcome-related productivity gains are benefits that follow on from improved health due to vaccination. The broad approach also considers behavior-related productivity gains, which result because reducing the burden of vaccine-preventable disease can lead to behavior change affecting productivity (36). For example, if a couple believes their children’s chances of survival have increased as a result of their being vaccinated against disease, the couple may decide to have fewer children and to invest more resources in each child (e.g., spending on health care and education) (37). These behaviors are presumed to improve household well-being and may spur economic growth through realization of a demographic dividend (15, 38).

Commonly, not only the people who receive a vaccination but also the unvaccinated derive benefits from widespread vaccination. Community health externalities include herd effects, whereby unvaccinated members of a community incur protection from disease through the vaccination of others (39–47). They also include reduced use of antibiotics to fight vaccine-preventable diseases and, as a consequence, slower development of antibiotic resistance (48, 49). Community economic externalities occur because high rates of vaccination and reduced disease transmission can make a country more desirable for domestic investment and FDI, as well as for tourism and immigration (14). Finally, vaccination reduces risk, and risk reduction implies lower need for ensuring against the future possibility of incurring a disease, as well as welfare gains due to reduced anxiety and worry. Along with the utilitarian value of health gains (i.e., those above and beyond their instrumental value for economic well-being), all these vaccination effects are valuable to individuals and the societies in which they live; ignoring these effects in economic evaluation studies will lead to systematic underestimation of the value of vaccination.

Value of Vaccination: Building the Evidence Base

Despite the potential importance of the broad benefits of vaccination at the individual and population levels, recent reviews of the literature reveal that many of these benefits are typically neglected in economic evaluations of vaccination (18, 50, 51). The existing reviews on this topic contribute two major findings: First, only a small portion of existing economic evaluation studies takes a broad view of the benefits of vaccination, and even those that do are relatively confined in how far they go beyond the narrow benefit categories. For instance, Bärnighausen et al.’s review of the types of benefits that have been captured in existing benefit-cost analyses of Hib vaccination finds only one study each that accounts for outcome-related productivity gains and community externalities (18, 50, 51), whereas none of the other studies account for any benefits beyond the traditionally captured health care cost savings or care-related productivity gains. In another systematic review, Deogaonkar et al. searched the literature for economic evaluation studies of vaccinations in low- and middle-income settings that take a broader perspective and find that, of the 26 articles published between 1990 and 2011, 8 capture community health externalities and a separate 8 studies capture outcome-related productivity gains. No study takes into account behavior-related productivity gains or community economic externalities (50). In a third systematic review, Ozawa et al. examined economic evaluations of vaccinations in low- and middle-income countries (51) and conclude that “[t]here were little data on long-term and societal economic benefits such as morbidity-related productivity gains, averting catastrophic health expenditures, growth in gross domestic product (GDP), and economic implications of demographic changes resulting from vaccination” (51).

Second, the existing reviews demonstrate that new research is needed to produce more robust evidence on the full benefits of vaccination. A number of scholars around the world have started to conduct such studies (19, 31, 33, 50–55).

Table 2 provides an overview of the emerging research on the full benefits of vaccination. Some studies focus on individual vaccinations, whereas others focus on clusters or packages of vaccinations. Some focus on existing vaccinations, whereas others seek to understand the benefits of prospective vaccination (20, 55). Different studies also focus on various subsets of the full benefits framework. Notably, no study to date captures all sources of benefit from vaccination. However, the ones that do focus on measuring full benefits generally find them to be substantial in magnitude (18, 31, 33, 55). For example, many existing studies of the Hib vaccination show the vaccination costs to be below or to barely outweigh the benefits (20, 56). However, Bärnighausen et al. show that incorporating broad benefits of Hib vaccination would drive its benefit-cost ratios well above 1 in a range of studies, indicating that the vaccination should be included in national immunization programs because it is net beneficial to society to do so (18). A study by Canning et al. that analyzed antenatal tetanus vaccination in mothers of low socioeconomic status in Matlab, Bangladesh, demonstrates that, in addition to improving child survival and preventing cognitive impairment in children of vaccinated mothers, vaccination can also promote educational attainment and subsequent wage gains (on the order of 2.5%) in those children (31). The authors find that the benefits and costs of maternal tetanus vaccination compare favorably to other potential investments to improve educational attainment for children from low-income households. Similarly, a study that calculated the return on investment in a GAVI Alliance program to extend coverage of new and underused childhood vaccination in 75 low-income countries during 2005–2020 estimates that the rate of return would be 12% in the first year of the program and rise to 18% in year 15 (55). These estimated rates of return compare favorably with average rates of return to schooling, which is a well-known and potent driver of economic well-being. Notwithstanding these strong results, the authors argue that the figures are conservative because they did not account for additional benefits such as reduced pain and suffering among survivors or demographic dividend effects and that a full benefits approach would lead to even more favorable estimates of return on investment.

Table 2.

Important benefits for specific vaccinations: The broad view

Vaccination	Health outcomes targeted	Benefits	Examples
PCV	Pneumococcal diseases, including pneumonia, meningitis, and otitis media	Outcome-related productivity gains	Episodes of pneumococcal pneumonia will keep children out of school, impeding cognitive development and learning (19, 22).
			Survivors of pneumococcal meningitis can suffer from severe cognitive and neurological sequelae (19, 23).
			Pneumococcal otitis media can impair cognitive development and lead to hearing loss (19, 26, 35).
		Community health externalities	PCV coverage decreases the use of antibiotics and thus the rate of occurrence of antibiotic-resistant pneumococcal infections (19, 49).
			Herd effects: Childhood PCV vaccination is likely to lead to substantial reduction in pneumococcal disease in population groups that will not routinely receive the vaccination, because it prevents the spread of the infection to these groups, e.g., the elderly and HIV-infected middle-aged adults (19, 46). Evidence from studies of PCV7 and PCV10 introduction indicate reduced circulation of vaccine-specific serotypes in the population (42–44).
Hib vaccine	Bacteremia, meningitis, epiglotittis, cellulitis, and infectious arthritis	Outcome-related productivity gains	Hib vaccination can avert long-term neurological sequelae of Hib infection, such as deafness, blindness, mental retardation, seizures, and paralysis, which affect a child’s ability to attend school and to learn (18, 24, 25, 30).
			Avoiding meningitis-related long-term disability through vaccination could result in averting very large productivity losses (estimated at up to US$ 910 million over a ten-year period in 72 low-income countries) (54).
		Behavior-related productivity gains	As Hib vaccination can reduce child mortality, mothers of vaccinated children can achieve their target family size through fewer births, allowing parents to invest more resources in each child and, as a consequence, improving children’ nutrition, health, and educational attainment. These improvements can increase earning potential and adult labor productivity (18, 38).
		Community health externalities	Herd effects: studies have documented marked reductions in the incidence of Hib infection in unvaccinated persons following the introduction of Hib vaccine into national immunization programs (18, 45, 47).
			Hib vaccination can prevent disease and thus obviate the need for antibiotic use, reducing the prevalence of antibiotic-resistant strains (18, 48).
		Community economic externalities	Demographic dividend: At the population level, reductions in fertility rates will decrease the number of youth dependents relative to the size of the adult labor force. This age structure change can lead to increased savings, which in turn can be used to invest in physical and human capital, stimulating economic growth (38).
HPV quadrivalent vaccine	HPV 6/11/16/18 infection; HPV16/18-related cervical, anal, vaginal, and vulvar precancers and cancers; HPV 6/11-related genital warts	Behavior-related productivity gains	Households in which a member has cervical cancer have reported changes in behaviors such as daily food consumption and school attendance, both of which could negatively impact educational attainment and earnings (21, 36).
		Community health externalities	Herd effects: Data suggest declines in incidence of male genital warts as the result of widespread female vaccination (21, 41).
Antenatal maternal vaccination with tetanus-diphtheria toxoid	Neonatal tetanus	Outcome-related productivity gains	Antenatal maternal vaccination against tetanus leads to significant schooling gains for children whose parents had no schooling. This effect is important, as families with low socioeconomic status may also have poor nutrition, which can compound the negative effect of tetanus (31).
MCV	Measles	Outcome-related productivity gains	Evidence from Matlab,Bangladesh shows that childhood measles vaccination appears to increase the school enrollment of boys, (but not of girls) (37). See also evidence from South Africa: Anekwe, 2011 (57).
Dengue vaccine (prospective)	Dengue fever, severe dengue	Outcome-related productivity gains	Persistent symptoms following the acute phase of dengue infection, which can include prolonged fatigue, muscle and joint pain, weakness, and depressive symptoms, could affect labor productivity (20, 34, 58, 59).
		Community health externalities	Unvaccinated community members could benefit from reduced dengue transmission between human hosts and the dengue mosquito vector (20).
		Community economic externalities	Health improvements resulting from dengue vaccination could make an economy more attractive for foreign direct investment and tourism (20).
			Reductions in dengue incidence could reduce public and private spending on outbreak control; these monies can be invested elsewhere stimulating economic growth (20).
Traditional EPI vaccines: DTP, BCG, MCV, polio	Diptheria, pertussis, tetanus, measles, polio, tuberculosis complications	Outcome-related productivity gains	A study of data from the Cebu Longitudinal Health and Nutrition Survey in the Philippines examined the impact of traditional EPI vaccines on gains in cognition, as measured by language, mathematics, and intelligence test scores and finds that children who were fully immunized with the EPI package scored higher on the cognitive tests than children who did not receive traditional EPI vaccines. Using international evidence to translate test score gains into earnings gains as adults, the researchers estimate a 21% return on investment (ROI) for vaccination spending (33, 55).
GAVI Alliance program to extend coverage of new and underused childhood vaccines–traditional EPI vaccines, plus Hib, Hep B, yellow fever, PCV, meningococcal A/C conjugate	Diptheria, pertussis, tetanus, measles, polio, tuberculosis complications; Hib (Bacteremia, meningitis, epiglotittis, cellulitis, and infectious arthritis), Hep B, and yellow fever; pneumococcal disease (including pneumonia, meningitis and otitis media), rotavirus-related diarrhea, meningitis	Outcome-related productivity gains	The GAVI Alliance program to extend coverage of new and underused childhood vaccination in 75 low-income countries during 2005–2020 could improve life expectancy and, in turn, earnings (with an estimated ROI of 12% in 2005, rising to 18% by 2020) (55).

BCG, Bacille Calmette-Guérin vaccination against tuberculosis; dengue, prospective dengue vaccination; DTP, diphtheria-tetanus-pertussis vaccine; EPI, expanded program on immunization; HepB, hepatitis B; Hib, Haemophilus influenzae type B; HPV, human papilloma virus vaccine; MCV, measles-containing vaccine; PCV, pneumococcal conjugate; ROI, return on investment.

Some Considerations Involved in Conducting a Broad Economic Evaluation

Implementing an economic evaluation of a vaccination intervention that adopts a full-benefits approach requires attention to five key items.

Understanding the Disease and the Vaccination.

Researchers need to understand the epidemiological background of the vaccine-preventable disease (or diseases), including disease incidence, prevalence, and duration, as well as all of the different natural courses of the disease (e.g., acute and chronic phases and long-term sequelae). In addition, the diagnosis of the disease, the portion of all cases that go undiagnosed, and the efficacy and side effects of existing treatments need to be studied carefully. Finally, the researchers need to understand vaccination efficacy and safety and the precise preventive effects [e.g., does a vaccination prevent infection completely, such as the measles vaccination, or only particular complications of infection, such as the Bacille Calmette–Guérin (BCG) vaccination against tuberculosis]. Data must also be collected on the costs of the vaccine itself, including costs associated with its delivery, storage, and administration.

Understanding the Population and the Social and Economic Context.

The full benefits of a vaccination will not only depend on clinical parameters describing the vaccine-preventable disease, but also on the social and economic roles of the people affected by the disease. For instance, the externalities associated with influenza vaccination among older adults or pneumococcal vaccination among children will depend on whether older adults care for younger household members or not. Understanding of the health systems context is also needed, for example, to distinguish a vaccination’s effectiveness when implemented through a national immunization program from the efficacy reported in randomized controlled trials (RCTs).

Specifying Pathways from Disease to Health, Social, and Economic Outcomes.

Following the broad benefits framework set forth in Table 1, it is crucial to specify all major health, social, and economic outcomes associated with different disease profiles. For instance, a vaccination that prevents a disease that can cause temporary or permanent impairments, such as hearing impairments due to Hib infection, will likely lead to long-term improvements in cognitive function, educational attainment, and labor market productivity. Adverse events associated with vaccination also need to be considered.

Quantifying and Monetizing the Broad Benefits.

The size of effects on different health, social, and economic outcomes caused by the vaccination or set of vaccinations being analyzed needs to be specified and monetized, with attention to institutions, policies, and other interventions that can serve to magnify or mitigate the effect sizes. When evaluating a vaccination that does not yet exist (such as an HIV, malaria, or dengue vaccination), potential benefits can be quantified using estimates of the causal effects of the vaccine-preventable disease on the outcome of interest in conjunction with hypothesized vaccination effectiveness. For instance, the employment benefits of an HIV vaccination can be estimated using measures of the effect of HIV on employment for different socioeconomic strata, along with information on average earnings by stratum and an assumed effectiveness of the HIV vaccination. Future benefits need to be discounted to present value using a range of discount rates chosen to reflect the length of the time horizon and the riskiness of the benefit profile.

Comparison of Benefits and Costs.

Although cost-effectiveness analysis (CEA) is traditionally the most widely used tool for economic evaluation of vaccination, benefit-cost analysis (BCA) lends itself more naturally to the full-benefits approach. Three virtues of BCA are that it can account for a diverse set of health and nonhealth outcomes, it can be used to compare health and nonhealth interventions (which is particularly important for decision makers such as Ministers of Planning who have to allocate funds across many sectors), and it directly generates a recommendation regarding the desirability of a health intervention based on the value of the estimated benefit-cost ratio. It does all of this by translating the diverse effects associated with vaccination interventions into dollar measures that can be combined, an exercise that often requires the imposition of strong (and not uncontroversial) assumptions, such as placing a monetary value on life itself. By contrast, CEA cannot be used to compare health and nonhealth interventions and is not well suited to handling situations in which there are multiple outcomes of interest. In addition, estimates of cost-effectiveness do not offer any guidance on the advisability of an intervention in the absence of an externally determined cost-effectiveness threshold or a budget constraint.

Discussion

The emergence of new ideas, theoretical models, and empirical evidence on the economic benefits of health has added health interventions to the arsenal of major instruments for promoting economic well-being. For some health interventions, outcome-related productivity gains, community health externalities, community economic externalities, and the value of pure health gains have been explored (51, 60–63) but studies like these are more the exception than the rule. By contrast, economic evaluations of education typically adopt a broad approach, commonly focusing on outcome-related productivity gains—education’s impact on work participation, productivity, and earnings throughout the life course (64). Some studies have also highlighted other broad benefits, including spillovers of education on political stability, poverty reduction, and crime (65, 66), although the literature has yet to capture the full benefits of education in a single pecuniary measure.

By routinely failing to account for the full spectrum of benefits of health interventions, economists have unwittingly undervalued many health interventions (18, 20, 21). The ensuing biases are likely to be especially large in the case of vaccination. First, vaccinations are commonly given in early life phases, and the returns on vaccination investments, such as improved school attainment, economic productivity, and social functioning, are reaped throughout the life course (31–33). Second, vaccinations often disrupt transmission chains of infectious diseases throughout the community, leading to multiplier effects for the broader economic and social benefits of vaccination (20, 39–44, 67). Third, the child survival benefits of vaccinations typically catalyze or accelerate fertility decline and create potentially sizable opportunities for economic growth.

Overall, the studies reviewed herein suggest that important and large vaccination benefits have been routinely ignored in economic evaluation studies. Although the resulting undervaluation bias may not have had important practical implications for well-established and low-cost vaccinations, such as those in the World Health Organization (WHO)’s Expanded Program on Immunization, it will likely guide policy makers toward underinvestment in expanding coverage with a new generation of more costly vaccinations, such as those against rotavirus, pneumococcal disease, and HPV—to the peril of the populations whose health and welfare these vaccinations could improve. A harder to estimate but potentially even more detrimental effect of systematic vaccination undervaluation is underinvestment in discovery and development of new vaccines (68). Potential market size is an important motivator for private industry to develop new products (69–71); adoption of vaccinations into national immunization plans will thus be a powerful signal to industry that could influence research investment decisions (68).

Three general recommendations flow from our arguments and related synthesis of existing evidence on broad benefits of vaccination. First, many economic evaluation studies of vaccinations should be redone to capture the full benefits generated by the vaccination in question. Second, the evidence to date on the full value of vaccination has been focused on measuring the total social benefits generated. It would also be useful to explore the distribution of vaccination’s benefits among different possible beneficiaries. Third, the primary empirical evidence on broad vaccination benefits will need to be considerably expanded and improved (72). Although many studies have shown that these types of benefits can be substantial, for many benefit categories, the evidence base has not been firmly established. For instance, it seems highly plausible that dengue vaccination could increase tourism flows in dengue-endemic countries, such as Brazil and Malaysia. Dengue outbreaks are typically highly visible in the international media, and tourists may decide to avoid travel to countries experiencing an outbreak (58, 59, 73, 74). Although a dengue vaccine is not yet available, promising candidates are under development (75). Studies are needed of the causal effect of dengue outbreaks on tourism streams and revenues to test the claim that dengue vaccination can generate positive community economic externalities by reducing the frequency and intensity of outbreaks.

One largely unexploited strategy holds particular promise for providing rigorous evidence on broad vaccination benefits. Modern vaccinations have routinely been investigated in RCTs, but these trials have mostly focused on safety, immunogenicity, and efficacy end points and have ignored the impacts of vaccination on educational, economic, and social outcomes. Follow-up studies in the trial populations to assess impacts on broad outcomes could produce a compelling body of new evidence, although this approach may be limited by the ethical obligation to deliver successful interventions to control groups once efficacy is established. The fact that in the case of many such trials long periods of time will have passed since the original exposure assignment might pose challenges for ensuring good follow-up of the trial populations. On the other hand, with respect to the goal of establishing broad vaccination benefits, a long time between exposure assignment and outcome assessment will likely be an advantage, because many of the broad benefits, such as school attainment and labor market outcomes, will only manifest themselves over longer periods of time. Collecting economic and social indicators as part of vaccination RCTs, or after the main trial has been completed, offers a promising and practical approach for guarding against the undervaluation bias that has plagued economic evaluation of vaccinations for decades (31, 72). Observational studies can also be informative, especially insofar as they adjust statistically for nonrandom vaccination status using, for example, propensity score analysis or instrumental variables, or focus on within-family differences in vaccination status and related outcomes (33, 55, 57).

Insofar as biomedical advances hold great promise with regard to the introduction of new and improved vaccines, building a body of objective evidence on the full benefits and costs of vaccination will be essential to assessing the desirability of pursuing these innovations.