Bridging the Gap: Need for a Data Repository to Support Vaccine Prioritization Efforts

Abstract

As the mechanisms for discovery, development, and delivery of new vaccines become increasingly complex, strategic planning and priority setting have become ever more crucial. Traditional single value metrics such as disease burden or cost-effectiveness no longer suffice to rank vaccine candidates for development. The Institute of Medicine—in collaboration with the National Academy of Engineering—has developed a novel software system to support vaccine prioritization efforts. The Strategic Multi-Attribute Ranking Tool for Vaccines—SMART Vaccines—allows decision makers to specify their own value structure, selecting from among 28 pre-defined and up to 7 user-defined attributes relevant to the ranking of vaccine candidates. Widespread use of SMART Vaccines will require compilation of a comprehensive data repository for numerous relevant populations—including their demographics, disease burdens and associated treatment costs, as well as characterizing performance features of potential or existing vaccines that might be created, improved, or deployed. While the software contains preloaded data for a modest number of populations, a large gap exists between the existing data and a comprehensive data repository necessary to make full use of SMART Vaccines. While some of these data exist in disparate sources and forms, constructing a data repository will require much new coordination and focus. Finding strategies to bridge the gap to a comprehensive data repository remains the most important task in bringing SMART Vaccines to full fruition, and to support strategic vaccine prioritization efforts in general.

Introduction

Development of new vaccines—irrespective of a country’s income level—often falls prey to competing demands, shrinking budgets and lengthening development timelines [1]. The tradeoffs inherent to new vaccine discovery, development, and delivery are shaped by public health needs, and such factors as technical feasibility, financial yields, affordability, regulation, and also public opinions concerning the diseases. These dynamics create a complex maze of choices with limited data to support and coordinate vaccine development efforts. Information deficiency also challenges strategic and transparent decision making in vaccine prioritization efforts. In a broad sense, decision making processes employed by various stakeholders remain opaque.

Consider the case of tuberculosis. An estimated 8.6 million new incident cases and 1.3 million deaths were reported worldwide in 2012 [2]. Although a vaccine is currently available and still used to vaccinate newborns, Bacille Calmette-Guérin (BCG) does not confer consistent protection against the infection in adults [3,4]. Drug-resistant strains of tuberculosis further challenge effectiveness in adults. A comprehensive analysis toward an improved vaccine for tuberculosis would ideally involve an understanding of—among other factors—how BCG imparts immunity, and why its effectiveness varies widely among infants, children and adults. In addition, the financial implications to develop a new vaccine, public awareness, and vaccine adoption are some of the many factors needed to evaluate a vaccine for development. In South Africa, for example, where the tuberculosis epidemic causes significant health and economic burden [2], this information is largely fragmented or inconsistent, but remains integral to the vaccine development process. The paucity and quality of data pose a significant challenge especially in the context of developing countries.

There is an enormous gap in estimating disease burden and vaccine candidate characteristics required to support effective vaccine development decisions. Consider a simple alternative for potential new vaccines to enhance protection against pneumococcal infection. Existing vaccines have used two approaches—either a multivalent polysaccharide vaccine or a protein conjugate vaccine. The 23-valent pneumococcal polysaccharide (PPS23) is recommended in the United States for at-risk children over age two and adults over age 64, using a single dose primary vaccination followed by a booster shot at five years for persons at high risk. Three conjugate vaccines are currently marketed, the broadest spectrum having 13 serotypes (PCV13), with three doses recommended in the United Kingdom and a four-dose sequence in the United States [5].

The disparate effectiveness rates and immunization schedules—with the associated costs of vaccine purchase and administration—raise obvious questions about desirable directions for further vaccine development. Should we seek to reduce dosage frequency, or expand the number of serotypes involved? How does the rising risk of antibiotic-resistant bacterial populations influence these choices? Can we develop one vaccine appropriate both for infants and older children as well as senior adults, or is it best to rely on a combination of these strategies? And with each of these choices come concerns about the risk of scientific failure, potential risks of adverse effects, and the potential for prevention of pandemic outbreaks. All of these issues represent tradeoffs that can be considered, but they also indicate the need for comprehensive data that could be used in a formal systems-based approach for priority setting.

Approaches to New Vaccine Prioritization

Data-related challenges repeatedly surfaced during our work on an Institute of Medicine (IOM) project—pursued in collaboration with the National Academy of Engineering—that has resulted in a software product for prioritization called SMART Vaccines—short for Strategic Multi-Attribute Ranking Tool for Vaccines (available for free at www.nap.edu/smartvaccines).

Previous IOM efforts have relied on a single metric approach to produce a rank-ordered listing of vaccines. A set of publications issued in 1985 [6] and 1986 [7] used infant mortality equivalents as the sole benefit measure to rank new vaccines for development that are of interest to the United States and the developing countries, respectively. A subsequent report released in 2000 employed cost-effectiveness as an efficiency criterion to produce a priority list of vaccines for development [8]. Recent stakeholder feedback has indicated that both these approaches have been limited in their use because of the narrowness of employed measures to help prioritize vaccine candidates. To help create a broader evaluation mechanism that would go beyond the traditional health and economic measures, the IOM, in its recent multi-phase effort, has employed a multi-attribute utility theory based approach to rank vaccines.

The multi-attribute utility theory is a special class of multi-criteria decision analysis tools, the previous applications of which have ranged from environmental engineering and academic program evaluation to energy and national security resource planning (see for example [9, 10]). The application of this method represents a novel mechanism for prioritizing new vaccines, and by extension—with further work—potentially to strategic planning and allocation of public health resources and interventions.

Uniquely, SMART Vaccines allows specification of numerous programmatic, policy, intangible and other attributes—from the total of 28 built-in and up to seven user defined vaccine attributes—that are traditionally omitted from cost-effectiveness and similar analyses in the comparative evaluation of vaccines. SMART Vaccines then elicits the set of attributes the user wishes to include in the analysis, and leads the user to set weights on how much each of these attributes should matter in the final evaluation. Next, SMART Vaccines calculates a SMART Score for each vaccine candidate, displayed graphically, and then allows users to conduct dynamic sensitivity analysis to see how SMART Scores vary as attributes and weights are changed. The software structure, use, and interpretation of the SMART Scores among other details can be found in the Ranking Vaccines reports [11–13].

Over the course of laying the axiomatic groundwork using multi-attribute utility theory [11], and prototyping and testing of SMART Vaccines 1.0 [12] coupled with application evaluation with some user groups [13], the need for systematically collected datasets for comparing vaccine candidates became apparent. Data were sparse for disease burdens, associated treatment costs as well as careful characterization of potential new vaccine candidates that often need to be compared for go or no-go executive decisions for investment and development. The need for a coordinated and systematic way to expand vaccine data collection efforts, especially in developing countries, was evident.

Data Demands

Published studies, reports, and publicly available datasets provided focused data for population cohorts used in SMART Vaccines. Extrapolation of findings to country-level populations with a wider range of demographics was challenging. Data for SMART Vaccines are entered by the user in a three step process that considers population, disease, and vaccine characteristics, shown as screenshots in Figure 1, Figure 2, and Figure 3. However, these data may be conceptually organized into four groups:

Figure 1.

Screenshot of the demographic data page in SMART Vaccines. Standard life table information along with productivity estimates are required as part of the definition of the population for which a vaccine is being developed.

Figure 2.

Screenshot of the disease burden data page in SMART Vaccines. For the selected disease, information regarding annual incidence, case fatality rate, and other illness related data such as disutility, disability, and costs are required for analysis.

Figure 3.

Screenshot of the vaccine characteristics entry page in SMART Vaccines. Vaccine product profile information—anticipated coverage, effectiveness, duration of immunity, number of doses, and their research, development, and administration costs are sought from the user.

Demographic Data

Common life table data describing age composition and life expectancy are needed entries for specifying populations of interest (Table 1). This first group of data may be obtained from publicly available sources such as the United Nations World Population Prospects and the World Health Organization (WHO) Global Health Observatory. This is supplemented with standard life expectancy as a constant benchmark (i.e., Japanese women with the greatest longevity). Hourly wage rates must also be estimated and input. For pre-loaded populations, these were available from the International Labor Organization. Average hourly earnings to all adults was applied—whether working at home, in the labor force, unemployed or some combination—using standard economic approaches that assign a value of productive time to all persons. Adult-like values of time to children were imputed on the premise that a sick child would demand the attention of an adult, hence costing the adult the opportunity cost of that time involved in child caring. Locating and compiling these demographic data may be cumbersome, but a necessary step in understanding a vaccines candidates potential within a population.

Table 1.

Demographic Data Needs for SMART Vaccines 1.1

Demographic Information	Description
1. Total population (N)	Total number of people in a population by sex and five-year age groups for a selected country.
2. Number of people alive at age x (lx)	These three variables are standard population life- table attributes, and each country’s life table includes lx, nLx, and ex for both sexes, by age groups.
3. Person-years lived between ages x and x+n (nLx)
4. Life expectancy (ex)
5. Standard life expectancy (sx)	Life expectancy for the Japanese population is used as the standard.
6. Hourly wage rate (USD)	Hourly wage rate for a population is calculated by dividing average income by total hours worked per year.

Disease Burden

Information about the disease is specified into health burden—incidence, case fatality rate, and other complications due to the disease (Table 2). This second group of data relating to disease burden and mortality can be sourced from the following: WHO health statistics and information systems; the Institute for Health Metrics and Evaluation’s Global Burden of Disease study [14]; and the National Vital Statistics Report and the Morbidity and Mortality Weekly Report of the U.S. Centers for Disease Control and Prevention. Disutility and disability weights were captured from published literature and the Global Burden of Disease study, respectively, to account for disease morbidity [15]. The availability of disability weights—which are independent of specific populations—may make disability-adjusted life years (DALYs) a preferred option over quality-adjusted life years (QALYs) for many decision makers, despite the stronger theoretical justification for the QALYs.

Table 2.

Disease Burden Data Needs for SMART Vaccines 1.1

Disease Burden Information	Description
1. Incidence (per 100,000)	New cases of a specified disease during a given time period divided by the number of persons in a stated population in which the cases occurred.
2. Case fatality rate (probability)	Probability of death, conditional on the disease being present. Thus the number of expected deaths equals the annual incidence rate times the case fatality rate.
3. Death Costs (USD)	Costs per case diagnosed with a disease resulting in death; includes medication and outpatient and inpatient costs.
4. Permanent impairment Percent of cases Disutility Disability weight Duration (days) Costs	Out of all disease cases, what percent result in permanent impairment? Disutility tolls represent the difference between HUI2 of the healthy state prior to illness (0.99) and the state during sickness. Disability weights quantify health losses for non-fatal consequences of diseases. Duration of permanent impairment. Costs per case diagnosed with a disease resulting in permanent impairment; includes medication and outpatient and inpatient costs.
5. Morbidity Percent of cases Disutility Disability weight Duration Costs	(i) Out of all disease cases, what percent result in morbidity? (ii) and (iii) Disutility tolls and disability weights quantify health losses for non-fatal consequences of diseases. (iv) Duration of morbidity. (v) Costs per case diagnosed with a disease resulting in morbidity; includes medication and outpatient and inpatient costs.

Incidence and case fatality rate are defined by age groups because these measures reflect the outputs of premature deaths averted and incident cases averted per year due to a potential vaccine. Since the diseases affect infants and elderly differently, SMART Vaccines allows for variability in disease burden among different age groups.

Disease morbidity also defines its effect on health related quality of life. Morbidities can include complications as a consequence of the principal disease, such as meningitis, sinusitis, and otitis media due to pneumococcal infection or fever and abdominal pain due to rotavirus. These secondary conditions are consequential because their treatment costs and health implications are used to calculate expected benefits of a vaccine. SMART Vaccines requires disability weights or disutility (tolls) and duration of the complication along with the costs associated in treating each condition. From the cases diagnosed with a particular disease, the percent of cases refer to a particular morbidity or permanent impairment caused due to the disease. The probabilities attached to user-selected and specified morbidities and permanent impairment must add up to 100 percent. Since most of these data are obtained from national health surveillance systems, high-income countries with strong infrastructure are presumably better-positioned to gather timely and congruous information than low- and middle-income countries.

Economic Data

A third group—economic data relating to cost of treating diseases (also Table 2)—are population-specific, since aggregate costs relating to the treatment of disease and its complications, including inpatient and outpatient hospital services, prescription drugs and medications vary in different parts of the world. Country-level costs associated with health services were obtained from World Health Organization-Choosing Interventions that are Cost Effective (WHO-CHOICE) database. Information about health resources utilization was obtained from published studies and expert estimates during the software development and testing. Future uses of SMART Vaccines will rely on economic data specific to each relevant population, some of which will likely have a strong empirical basis, and others of which may only available as expert-informed estimates. Sensitivity analysis conducted by users can inform them of how such data may or may not alter their final priority rankings, and hence also where investment in better data may prove to be most useful.

Vaccine Characteristics

A fourth group—vaccine data (Table 3)—may require conjectural data about specific attributes of vaccines, including their potential performance (coverage and effectiveness in producing immunity), costs per dose, doses required per person, vaccine administration costs, research, development, and licensure costs. These vaccine attributes typically form the basis for target product profiles that manufacturers and donors often use to guide their vaccine development efforts.

Table 3.

Vaccine Product Profile Information for SMART Vaccines 1.1

Vaccine Product Profile Information	Description
1. Coverage (percentage)	Anticipated coverage rate for the new the vaccine.
2. Effectiveness (percentage)	Anticipated effectiveness for the new vaccine.
3. Length of immunity (years)	Anticipated length of immunity from the new vaccine.
4. Doses required per person (number)	Anticipated doses required per person.
5. Cost per dose (USD)	Expected costs per vaccine dose.
6. Cost to administer per dose (USD)	Expected costs to administer a dose.
7. R&D and licensure costs (USD)	Anticipated costs for a vaccine manufacturer to develop and license a vaccine.

Effects of a vaccine vary widely depending on the population—age, sex, and environment—and these measures in SMART Vaccines are typically in the form of estimates. Since new vaccines do not exist yet, length of immunity, doses required per person, cost per dose, and cost to administer per dose are anticipated measures. Changing the vaccine characteristics allows sensitivity analysis for various “what if” scenarios in SMART Vaccines—how do the costs change as vaccine effectiveness increases or what is the change in coverage as costs decrease, among others.

Research, development, and licensure costs options also typically would rely on estimates surrounding vaccine development as manufacturing and licensure costs tend to vary significantly across high- and low-income countries.

From Data to Decisions

In numerous demonstrations of the SMART Vaccines software in diverse stakeholder settings, observers often comment on the large amount of data that must be acquired, verified, and imported into the software model before it can be used. But the software itself does not create the data demands; it brings them into clear view. One cannot analyze these types of strategic planning options without using the very data that SMART Vaccines requires.

Indeed, in the design of SMART Vaccines, numerous tradeoffs that reduce the data burdens for users at the expense of reduced software features were made. Examples of such tradeoffs include the modeling of herd immunity, the modeling of diseases that create permanent disability, and the age brackets available to characterize disease burden compared with those available to describe the population itself. Ultimately, with each data group presenting different aspects of vaccines, it would be ideal to expand these data to include more countries and country-specific information, which would require a significant expansion of the SMART Vaccines database.

Even though several vaccine databases currently exist, they either remain private or unique to an organization’s mission. The Pan American Health Organization’s OLIVES, an online international vaccine economics and statistics database, is focused on low-and middle-income countries for three vaccines: human papillomavirus, pneumococcal conjugate, and rotavirus. UNAIDS, the Joint United Nations Programme on HIV/AIDS, contains tools and data that are available for international use but are also limited by their disease scope. LiST: The Lives Saved Tool estimates the impact of maternal, newborn and child health interventions in middle and low income countries, but falls short of considering long-term and non-fatal effects which contribute toward disease burden [16].

The Need for a Data Repository: A Call to Action

The scarcity of data coupled with the need for evidence-based decisions motivate the development of an exhaustive data repository focused on vaccine prioritization. Moreover, a general purpose data repository suitable for different geographical regions can help increase standardization, transparency, and comparability around data, and guide various decision makers in arriving at realistic estimates for their analyses. This public data infrastructure could offer a systematic approach to gather information via templates as starting points for relevant parameters for vaccine candidates and different populations. Building a data repository will need to involve various stakeholders to provide differing perspectives and technical expertise. SMART Vaccines could help serve as an initial platform to stimulate further work and much needed coordination in this area.

The need for complete and consistent data is not unique to SMART Vaccines, but is a prerequisite for quality decision making. Investing in new vaccines and related public health technologies have time and again demonstrated the possibilities of large pay-offs—both in terms of health and wealth [17]. In reality, decisions are made and will continue to be made despite the information deficiency, and hence a data repository—created and curated by an open community of users—could effectively bridge the existing gaps in informed decision making.