Key issues for estimating the impact and cost-effectiveness of seasonal influenza vaccination strategies

Abstract

Many countries have considered or are considering modifying their seasonal influenza immunization policies. Estimating the impact of such changes requires understanding the existing clinical and economic burden of influenza, as well as the potential impact of different vaccination options. Previous studies suggest that vaccinating clinical risk groups, health care workers, children and the elderly may be cost-effective. However, challenges in such estimation include: (1) potential cases are not usually virologically tested; (2) cases have non-specific symptoms and are rarely reported to surveillance systems; (3) endpoints for influenza proxies (such as influenza-like illness) need to be matched to case definitions for treatment costs, (4) disease burden estimates vary from year to year with strain transmissibility, virulence and prior immunity, (5) methods to estimate productivity losses due to influenza vary, (6) vaccine efficacy estimates from trials differ due to variation in subtype prevalence, vaccine match and case ascertainment, and (7) indirect (herd) protection from vaccination depends on setting-specific variables that are difficult to directly measure. Given the importance of knowing the impact of changes to influenza policy, such complexities need careful treatment using tools such as population-based trial designs, meta-analyses, time-series analyses and transmission dynamic models.

Introduction

In the wake of the 2009 A/H1N1 influenza pandemic, many countries have considered or are considering modifying their seasonal influenza immunisation policies.^1,2 For example, both the US³ and the UK⁴ recently expanded their recommendations to include vaccination of children. Several other European countries, Canada and Hong Kong recommend childhood vaccination, while Austria, Estonia and several Canadian provinces recommend vaccination for all age groups.^5-7The World Health Organization promotes the use of seasonal influenza vaccination through its Global Action Plan. However, they recently identified lack of evidence on disease burden and cost-effectiveness as a barrier to decision making in this area.⁸

A range of options for influenza immunization policy exist, including the type of program (universal or targeted), groups to target for age or risk-based strategies (health care workers, clinical risk groups, children, the elderly, working adults), type of vaccine to use (inactivated or live, adjuvanted or unadjuvanted) and the mechanism for funding (public sector, insurance, out-of-pocket, employers). The large number of potential targets and program types complicates evaluation and decision making in the area, especially as each option should ideally be evaluated incrementally (i.e., compared with the next best feasible alternative).

Estimating the potential impact of a change in policy can be separated into two steps. The first is to estimate the existing burden, both in terms of influenza-attributable disease as well as the resulting economic burden associated with disease to the health care system and the wider economy. Once this baseline is established, the proportion of this disease and economic loss that can potentially be prevented using different vaccination strategies can be estimated. This allows the potential benefit of different options to be quantified and compared with each other.

In this article, we describe key issues and challenges involved in each stage of this process. We also discuss the extent to which they are addressed in existing economic evaluations of influenza vaccination in different target groups. Table 1 summarizes the steps involved in establishing the impact of influenza vaccination along with the key components, potential difficulties and tools that can be used at each stage. Our purpose is to guide analysts conducting studies to estimate influenza vaccine impact, and policy makers interpreting such studies; for recommendations on vaccination policy itself we refer readers to recent World Health Organization guidelines.⁹

Table 1. Stages in establishing the impact of seasonal influenza vaccination.

Stage	Key components	Potential difficulties	Available data and tools
1. Establish the influenza-attributable disease burden.	Community (non-health seeking) infection	Cases are not generally recorded	Serological and virological surveys combined with statistical and mathematical models
	Healthcare utilization and mortality	Cases have non-specific symptoms and are rarely tested	Longitudinal data sets Statistical time-series models
2. Establish the economic burden of influenza.	Direct costs to healthcare providers and households	Volume estimates of outcomes must be matched with unit costs	Health care records Patient surveys Market prices (or shadow prices), tariffs
	Indirect costs (loss of productivity and leisure time)	Estimates differ widely depending on the method used and inter-country differences in work absenteeism.	Human capital estimates Friction costs Stated or revealed preference studies
3. Establish vaccine impact in a population	Efficacy in vaccinated individuals	Clinical trials estimates vary due to changes in subtype prevalence, vaccine match and case ascertainment	Randomized clinical trials Meta-analyses
	Indirect impact on non-vaccinated individuals	Indirect protection depends on setting-specific variables that are difficult to directly measure	Household and cluster randomized studies Transmission dynamic models

Estimating the Epidemiological Burden of Disease

Estimating influenza disease burden is complicated by a lack of routine laboratory testing and non-specific symptoms.¹⁰ This means the number of confirmed cases that are detected by routine surveillance systems substantially underestimates the true disease burden.¹¹ The level of underestimation is most evident when considering mild disease in the community but is also a major issue when trying to estimate more severe influenza disease burden. This has led to the use of proxy disease categories, as well as other more complex modeling methods, to assess the influenza disease burden.

Proxy categories vary in their sensitivity and specificity, typically relying on clinical diagnosis, with influenza causing only a proportion of the total disease. Commonly used categories include acute respiratory illness (ARI)¹² or influenza-like illness (ILI),¹³ which typically includes fever (of ≥ 38°C) and cough and/or sore throat.¹⁴ Other studies have used the category of pneumonia and influenza from hospitalization or mortality records.¹⁵ Information on such categories does not require additional testing and is often available in routinely collected data, if for example, they have corresponding ICD code categories.

While these categories may not provide a realistic measure of the influenza disease burden, they can be used to help monitor disease patterns over time¹⁴ or as precursors to laboratory testing in clinical trials.¹² There have been efforts to try and standardize definitions such as ILI to increase their reliability as proxies for influenza.¹⁴ However, the specificity and sensitivity may still vary by age-group and setting,¹⁶ as well as from season to season dependent on the circulation of causative agents over time.¹⁷

The severity of disease that can be caused by influenza varies from asymptomatic or mild disease (non-medically attended), to more severe infections which require ambulatory or hospital care and which may result in death. The ways in which disease burden is typically estimated also vary by the differing levels of severity. Direct estimates of community (non-health seeking) clinical attack rates require virological testing of those reporting symptoms, while serological sampling may be used to estimate the overall (clinical and non-clinical) infection rate.^11,18

Time-series methods are often used to estimate more severe influenza disease that requires healthcare use or that is recorded in routine mortality statistics. These models examine changes in broader disease categories over time alongside the known seasonal variation in influenza activity. Essentially the models try to predict the influenza-attributable disease by establishing a non-influenza background rate of disease. The models can broadly be grouped into two categories,¹⁸ those that use measures of influenza activity (e.g., laboratory reports) and other independent variables^11,19 to establish the background rate, and those that assume that the background rate follows a consistent seasonal pattern.^20,21

An alternative direct approach would be routine virological testing of all patients with respiratory symptoms seeking healthcare, but this is usually too resource-intensive to do outside a research setting. Whatever method is used to establish disease burden, they should be made over several years (rather than a single season) due to variation in strain transmissibility, virulence, and prior immunity over time.¹⁷

Estimating the Economic Burden

A broad distinction can be made between the direct and indirect cost burden. Direct costs are for medical consumption (consultations, medication, diagnostic tests, surgery, etc.) related to diagnosis and treatment of disease against which vaccination offers protection. They also comprise costs, directly related to the implementation of the vaccination program, consisting of administration costs (consultations, needles, syringes, screening, information campaign, cold chain, infrastructure (buildings, vehicles, training, etc.) and vaccine purchasing costs (based on the number of required doses, including wastage). Treatment costs for vaccine associated adverse events may also be included here. Direct personal costs, which are often ignored, include transport costs incurred to receive vaccination or treatment.^22,23

Indirect costs encompass costs of lost time (either productive time or leisure time, or a combination of both) in patients. They may also include such time losses in caregivers of patients (e.g., parents, spouse), depending on a jurisdiction’s guidelines for economic evaluation.

Cost estimates are usually derived from the quantity of resources consumed and their unit costs. The former estimate (i.e., number of consultations, hospital days, medicine packages, days of sick leave etc) can be based on case report forms, patient charts, hospital records as well as surveys among patients and caregivers. Unit cost estimates are usually based on the latest available market prices and tariffs, or their shadow prices to approach opportunity costs. Unit cost estimates can be sensitive to the category of disease being used (self-reported ILI, clinician reported ILI, clinician confirmed influenza, laboratory confirmed influenza etc.). Therefore it is essential that estimates of volume (e.g., age-specific number of symptomatic ILI cases who have attended primary care at least once, derived from primary care reporting systems) are carefully matched with appropriate unit cost estimates (eg, costs for patients who have symptomatic ILI and attend primary care at least once, derived from surveys among patients attending primary care). Failure to do so can lead to substantial under or overestimation of the cost burden arising from a specific patient group.

Although it is widely accepted that indirect costs in the form of time losses do represent some given quantity of costs (to the individual, to an employer, to society), currently these costs are usually ignored in applied economic evaluations. Time losses arise whenever individuals interrupt their normal activities, because of illness or premature mortality (to themselves and/or friends/relatives), or to be subjected to an intervention, for instance, to be vaccinated. During illness or for part of the rest of the expected lifetime (or very briefly during the vaccination consultation), this individual and society then incur time costs. Although there are variations in what guidelines describe as societal and health care sector perspectives, in theory a societal perspective requires the inclusion of all costs, no matter who they accrue to. A health care sector (and health care payer/provider) perspective usually excludes all indirect costs. However, the use of a measure of health utilities such as quality adjusted life-years (QALYs) may implicitly incorporate some non-monetized valuation of lost productive and leisure time.

For influenza, the estimates of the mean number of working days lost per episode typically range between 1.5 and 4.9 d (laboratory-confirmed influenza), 3.7–5.9 d (physician diagnosis), and < 1 d to 4.3 d (self-reported influenza).²⁴ These ranges again illustrate the need to match the number of episodes that occur with the diagnosis and severity of those episodes.

The human capital method is the oldest and historically the most popular method to estimate the value of time losses. With this approach one’s earnings are considered to be a good estimate of the opportunity costs of one’s lost time. In other words the value of lost production is considered to equal the present value of future earnings during the period of lost (or impaired) ability to work or to enjoy leisure activities.²²

An adaptation to the human capital approach is made with the friction cost method, by which the period of interruption of productivity is limited to the friction period (i.e., the time it takes to replace a sick or deceased individual by somebody else performing the same tasks).^25,26 The friction period depends on the nature of the activity and on the labor market equilibrium (e.g., the friction period is shorter for unskilled laborers than for skilled laborers, and longer in periods with less unemployment). With the friction cost method, absence from work during the friction period is assumed to cause a less than proportional decrease in labor productivity, because of internal labor reserves, catching-up work upon return and diminishing returns for labor.²⁷ This method leads to lower and arguably more realistic estimates of lost production due to disease than the human capital method.²⁸ A limitation of the friction cost method is that it ignores the value of lost working time (and income) and lost leisure time to the sick individual, because it focuses on the employer.²⁷ Therefore, while using the friction cost method to estimate lost productivity time in the numerator, non-monetized (leisure) time should implicitly be included in the measurement of health gains in the denominator of the cost-effectiveness ratio. A third alternative is using stated preference studies (contingency valuation or willingness to pay methods, through which estimates are elicited by questionnaires) or revealed preference studies (for which estimates are derived from observed behavior). However these yield diverging results and are much less used in applied analyses.

As for other highly frequent and often self-limiting illness (e.g., varicella²⁹ and rotavirus³⁰), when indirect cost savings by vaccination are included in the analysis, they tend to dominate the direct costs saved, particularly when the human capital method is used.²³ This is also the case for influenza vaccination strategies that impact on absenteeism of working adults (either directly by preventing adult’s illness, or indirectly by preventing their children’s illness).³¹

Indirect cost estimates are also used to value the loss of life years due to fatalities. Here too, the human capital, friction cost and willingness to pay methods can all be used, again likely yielding substantially different estimates. The friction cost method then values the time period needed to replace a deceased employee, and not the other life-years a deceased person has lost. The latter life-years should therefore be included in the denominator of the cost-effectiveness ratio when using the friction cost approach.

Inter-country differences in rates of influenza-associated work absenteeism and likely also in ILI consultation rates in general, can be due to country-specific differences in requirements for physician certification that an employee is unable to work. In some countries, employees are required to consult a physician to obtain such a statement from their first day of illness onwards, whereas in other countries, absence from work can be up to 14 d before a physician’s statement is required to justify the absence.³² Particularly for mild influenza cases, these differences may influence a patient’s inclination to consult a physician, and possibly also the total period of absenteeism estimated by a physician for a patient presenting for care early during the illness period.

Estimating Vaccine Impact

Clinical trials provide an estimate of vaccine efficacy in vaccinated individuals within the trial setting. Efficacy varies depending on characteristics of both vaccinee (age, immunocompetence) and vaccine (dosage, presence of adjuvant, live or inactivated). However, there are several problems around directly applying trial results to obtain an estimate of population-level vaccine impact. First, seasonal influenza virus consists of three subtypes (influenza A/H1, A/H3 and B), each of which has a separate profile in terms of virulence, transmissibility, seasonality and potential for vaccine effect. The relative prevalence of each subtype, and hence overall vaccine effect, differs between seasons and across locations. Furthermore, the genetic composition of each subtype changes from season to season (antigenic drift). Vaccines are updated each year to account for drift, but their degree of match to each season’s strains differs by season and location. In some years, larger genetic changes (antigenic shifts) occur, which can lead to particularly poorly matched vaccines. One study suggests that, in between 1987–1997, a good match between vaccine and the predominant strain only occurred in 50% of seasons.³³

Second, immune correlates of vaccine protection are not always reliable endpoints for estimating impact because their association with efficacy against disease is variable. Evidence about such links is particularly weak in target groups that may respond least well to vaccination, such as young children, the elderly and immunocompromised.³⁴ Hence, impact estimation relies on infection and disease endpoints. However, case ascertainment varies because the accuracy of virological testing changes over time and between laboratories.³⁵ Sometimes proxies such as influenza-like illness are used as trial outcomes instead or as precursors to laboratory confirmation.¹² However, this makes case ascertainment dependent on clinical judgment, which can be even more variable than laboratory testing accuracy. Also, vaccine impact on severe outcomes such as hospitalization and death is likely to differ from impact on all cases, and may be particularly difficult to measure due to the large sample sizes needed to precisely estimate these relatively rare events.³⁶

Third, population-level vaccine effectiveness can be greater than individual-level vaccine efficacy because of indirect (herd) impact. That is, preventing infection in an individual protects not only that person but also others who would otherwise have been infected by the person now vaccinated. Some study designs can be used to estimate indirect impact.³⁷ Household trials record influenza episodes not just in vaccinated individuals but also in members of the same household,³⁸ while cluster trials randomize the entire eligible population of a geographically contiguous area to be vaccinated or not.³⁹ A third method is to use population surveillance to compare influenza incidence before and after a vaccination program is introduced. For example, post-vaccination surveillance in Ontario suggested that influenza incidence decreased after the introduction of universal vaccination to a level beyond that which could be predicted from direct protection alone.⁴⁰

However, these studies are limited by lack of external validity.⁴¹ Indirect protection depends on variables such as household structure, age distribution, population mixing, infectivity of influenza strains, susceptibility of individuals and vaccine coverage which differ between settings and are mostly difficult to measure in trials. Consequently, mathematical models have been used as an alternative to primary studies to estimate influenza vaccine impact. Such models, called transmission dynamic models, relate the risk (or force) of infection at different time points to the proportion of the population infected at that time.^42,43 For instance, a model-based analysis of the English influenza immunization program concluded that most of its benefits were obtained through indirect protection rather than by directly protecting vaccine recipients.⁴⁴ However, results of models are highly dependent on assumptions made about key parameters which are difficult to measure, such as the probability of transmission, degree of initial seeding and clinical attack rate. Hence sensitivity analysis to investigate the robustness of conclusions to changes in such assumptions is vital, but is rarely done.¹⁷

Application to Existing Studies

To investigate the extent to which the issues outlined above are addressed by economic evaluations of influenza vaccination, we conducted a review of existing reviews published between January 2006 and October 2012. We searched PubMed and Web of Science (SCI and SSCI) using the general search string “(cost OR costs OR cost-effectiveness OR cost-benefit OR cost-utility OR economic) AND influenza AND (vaccine OR vaccines OR vaccination OR immunisation OR immunization)” in the title field. Articles were shortlisted if they were systematic reviews published in peer-reviewed journals in English, reviewing full economic evaluations of seasonal influenza vaccination.

The combined search identified 129 publications after removal of duplicates. However, 119 were excluded for various reasons (61 primary research articles, 23 meeting abstracts, 12 about pandemic influenza, 10 without economic evaluations, 4 not about influenza vaccination, 4 opinion pieces, 2 news items, 2 not about vaccination, 1 in Mandarin). The remaining 10 articles focused on either people with underlying illness,⁴⁵ children,^17,46-48 healthy (working) adults,^49-51 health care workers⁵² or the elderly.^45,53

The reviews concluded that targeted seasonal influenza vaccination programmes were generally found to be cost-effective and even cost saving in some circumstances (see Table 2). The exception was targeting healthy working adults, where the evidence was more mixed. However, all reviews concluded that results of analyses were sensitive to assumptions made about many of the issues discussed in this article, including herd (indirect) protection, incorporation of the cost of productivity loss, estimating vaccine effectiveness against different endpoints, estimating the influenza-attributable proportion of different endpoints and estimating vaccination costs. Herd protection was identified as a key issue by all articles about vaccination of young children, while productivity implications were identified as key issues for both childhood and working adult programmes.

Table 2. Key issues considered in economic evaluations of seasonal influenza vaccination.

				Key issues relating to outcomes*
Paper	Target group	Number of studies reviewed	Main study conclusions	Herd	Endp	Prod	VCost	VEff	Other key issues
Newall 201217	Children ≤ 18 y	20	11/20 cost saving; remainder mostly cost-effective	√	√	√	√		Difficulty estimating severe outcomes.
Coleman 200646	Children	7	Not discussed	√		√			Consumer choice Direct expenses paid by households Trade-off between paying for prevention and treatment
Nichol 201147	Children ≤ 18 y	20	11/20 cost saving; remainder mostly cost-effective	√
Savidan 200848	Children < 18 y	15	All cost saving or cost-effective	√			√	√
Burls 200652	Healthy adults, healthcare workers	14	10/14 cost saving (including 2/2 on health care workers)	√
Gatwood 201249	Healthy adults 18–64 y	7	“Generally not cost saving”			√		√	Variability in outcomes Setting of vaccine delivery Severe adverse events Estimating less severe endpoints
Hogan 201250	Healthy adults	10	8/10 favored vaccination	√	√		√	√	Perspective (employer only or employee as well)
Newall 200951	Adults 50–64 y	6	All cost-effective		√	√	√	√	Life expectancy in people with co-morbidities
De Waure 201245	Adults > 50 y and high-risk populations	20	All cost saving or cost-effective in both elderly and high-risk groups	√		√
Postma 200653	Elderly	18	15/18 cost saving, 16/18 cost-effective			√	√	√	Definition of influenza-attributable hospitalisation or death

^* Herd: herd protection; Endpt: mismatch in endpoints used to estimate incidence and vaccine effectiveness (e.g., acute respiratory illness, influenza-like illness, laboratory-confirmed influenza); Prod: productivity loss due to influenza; VCost: drivers of vaccination costs (purchase and administration); VEff: drivers of vaccine effectiveness.

Conclusion

Our review highlights a number of important issues in each of these steps. The issues we discuss around estimating the epidemiological and economic burden of influenza, as well as the impact of different vaccination programs, can have an important effect on the conclusion of studies such as economic evaluations that are crucial for policy making. Policy making for vaccines is subject to particular complexities and uncertainties and influenza vaccination is no exception to this for the reasons we outlined.²³ Besides the broad principles in Table 1, Table 3 provides further recommendations for estimating the impact and cost-effectiveness of seasonal influenza vaccination in specific groups.

Table 3. Specific recommendations for estimating the impact and cost-effectiveness of seasonal influenza vaccination in particular groups.

Target group	Recommendations
Healthy young children (0–5 y)	Consideration of household and pre-school transmission is important in this key transmission group.
Healthy school age children	Consideration of household and school-based transmission is important in this key transmission group.
Healthy adults	This is not a usual target group for seasonal influenza vaccination. However, the economic burden of influenza to families and employers may be substantial due to its effect on productivity.
Elderly	Account needs to be taken of the substantial burden of morbidity and mortality that may not necessarily be attributed to influenza.
Population with high risk of complications due to chronic conditions	Appropriate assessment of the risk and vaccine effectiveness in these groups, which are often small in comparison with the overall population.
Pregnant women	Potential protective effect mothers on their infants should be considered.
Health care workers	Potential protective effect on high-risk patients should be considered.

Glossary

Abbreviations:

ILI

influenza-like illness