Identifying signatures of the impact of rotavirus vaccines on hospitalizations using sentinel surveillance data from Latin American countries

Abstract

Background.

Passive surveillance data are often the only available source of data that can be used to evaluate the population-level impact of vaccination, but such data often suffer from important limitations such as changes in surveillance efforts. This study provides an example of how to identify important signatures of rotavirus vaccine impact, including evaluating the overall effectiveness and changes in rotavirus seasonal dynamics.

Methods.

We used data from a standardized sentinel rotavirus surveillance network in six Latin American countries (Bolivia, El Salvador, Guatemala, Honduras, Paraguay, and Venezuela) from 2004–2017. A random-effects model was used to evaluate changes in the proportion of rotavirus-associated hospitalizations following vaccine introduction. Harmonic regression models were used to estimate vaccine impact on the number of rotavirus hospitalizations, controlling for trends in rotavirus-negative cases. Changes to rotavirus seasonality were evaluated using center of gravity analysis, wavelet analysis, and harmonic regression.

Results.

All countries observed declines in the proportion of rotavirus-positive acute diarrhea samples with a mean reduction of 16% (95% confidence interval: 10–22%). We estimate that each 10% increase in vaccine coverage was associated with declines in the number of rotavirus-positive cases, ranging from 4.3% (1.3–7.2%) in Honduras to 21.4% (16.8–25.9%) in Venezuela. The strength of the seasonal peak in rotavirus incidence became smaller after vaccine introduction in Guatemala, Honduras, and Venezuela. Seasonal peaks also shifted later in the surveillance year, especially in higher-mortality countries.

Conclusions.

The combination of methods we applied have different strengths that allow us to identify common signatures of rotavirus vaccine impact.

Vaccines are generally licensed and introduced into national immunization programs following a series of carefully designed randomized controlled trials to demonstrate individual-level vaccine efficacy. Following vaccine introduction, it is important to accurately assess the population-level impact of vaccines—which is influenced by vaccine effectiveness as well as vaccine uptake—in order to fully understand their benefits and justify the costs. In many parts of the world, passive surveillance data are the only available source of data that can be used to evaluate population-level vaccine impact. However, such surveillance data often suffer from important limitations, such as missing data and inconsistency in the surveillance system over time. Despite these issues, surveillance data can provide important information on population-level trends in disease incidence, which is especially important for vaccine evaluations. Using rotavirus as an example, this study aims to identify signatures of the population-level impact of vaccines with imperfect surveillance data by employing a combination of methods that have different strengths. These measures include evaluating changes in the proportion of viral-positive tests, evaluating changes associated with vaccine uptake, and evaluating changes in the seasonal dynamics of the virus.

This study evaluates the population-level impact of rotavirus vaccines in Latin America, as countries in this region exhibit a broad range of development levels. Both vaccine efficacy and effectiveness are lower in low-income settings where the burden of rotavirus-associated gastroenteritis (RVGE) is greatest [1–4]. In Latin America and the Caribbean, 19 countries and one territory have introduced the rotavirus vaccines (the monovalent Rotarix vaccine [RV1; GlaxoSmithKline Biologicals] and the pentavalent RotaTeq vaccine [RV5; Merck]) into the routine immunization schedule as of December 2017 [5, 6]. Vaccine efficacy for RV1 against rotavirus-associated hospitalization was 83.0% (95% confidence interval (CI): 73.1–89.7%) over two years of follow-up in 10 Latin American countries [7], while vaccine efficacy for RV5 against rotavirus hospitalizations and ED visits was 90.0% (95% CI: 29.4%−99.8%) in the region [8]. Case-control studies have estimated that vaccine effectiveness for the two vaccines was 63.5%−73% in Latin American countries, according to recent meta-analyses [9, 10].

In this study, we aim to evaluate the population-level impact of vaccines on rotavirus-associated hospitalizations (overall effectiveness), which incorporates not only direct protection from the vaccines but also indirect protection (herd immunity) and vaccine coverage [11, 12]. We use data from a regional rotavirus sentinel surveillance network in Latin American and Caribbean countries [13]. The data had important caveats, such as inconsistent surveillance effort. We employ a combination of methods that have different strengths to estimate vaccine impact, while accounting for these limitations. We also evaluate the effect of vaccination on rotavirus seasonality, as the timing and frequency of rotavirus epidemics have changed following vaccine introduction in some settings, providing evidence of herd immunity [14–16].

METHODS

Surveillance data and participating countries

Monthly data on hospitalizations for acute diarrhea among children <5 years of age were collected at sentinel hospitals through the regional rotavirus sentinel surveillance network in Latin American and Caribbean countries [13]. We evaluated the data available from 2004 through 2017. Standardized protocols for case recruitment and specimen testing have been described previously (Supplementary Methods) [17]. A total of 17 Latin American and Caribbean countries and territories have participated in the rotavirus sentinel surveillance network, with some countries intermittently participating at various time points (Supplementary Table 1). Of these, only Bolivia, El Salvador, Guatemala, Guyana, Honduras, Paraguay, and Venezuela had both pre- and post-vaccination data available. Six countries (accounting for ~25% of the population <5 years old) were included in our analyses after excluding Guyana, for which only seven cases were tested for rotavirus per month on average. The number of hospitals participating in the surveillance network as well as testing effort varied over time (Supplementary Figure 1 and 2A). Vaccine coverage estimates were obtained from WHO-UNICEF (Supplementary Figure 3) [18]. Countries are classified into high-mortality developing (HMD) and low-mortality developing (LMD) countries according to the WHO mortality strata (Supplementary Table 1) [19].

Evaluating the impact of vaccination on the proportion of rotavirus-positive diarrhea hospitalizations

One way to overcome variation in surveillance effort over time—particularly before and after vaccine introduction—is to evaluate vaccine impact in terms of the proportion of cases tested that were positive for the pathogen of interest, which is indicative of the impact of vaccination on the non-specific illness that the pathogen causes. The proportion rotavirus-positive was evaluated using a random-effects model, in which a random intercept for each country was included to adjust for baseline differences in proportion rotavirus-positive. The denominator of the proportion was the cumulative number of cases hospitalized for acute diarrhea whose stool samples were collected for laboratory testing, while the numerator was the cumulative number of laboratory-confirmed RVGE hospitalizations. The proportion was calculated for the pre-vaccine period (from the beginning of surveillance until—but not including—the month of vaccine introduction) as well as the evaluation period (from two years after vaccine introduction to the end of the surveillance period reported to WHO/PAHO when the data were downloaded). The two-year window between vaccine introduction and the beginning of the evaluation period was chosen based on the time until vaccine coverage became high at the national level (Supplementary Figure 3).

Since the countries included in this study are located in both the Northern and Southern Hemispheres, and rotavirus typically peaks during the winter months (or dry season) [20, 21], the surveillance year was defined as follows: from July to June in the Northern Hemisphere (El Salvador, Guatemala, Honduras, and Venezuela) and from January to December in the Southern Hemisphere (Bolivia and Paraguay). When aggregating the data in the pre-vaccine period and evaluation period, data were included only if there were >10 months of data in each surveillance year (Supplementary Methods). We calculated an absolute difference between the proportion rotavirus-positive in the pre-vaccine period and evaluation period in each country, and then obtained a pooled estimate of the difference across countries.

Evaluating the impact of vaccination on the number of RVGE hospitalizations

We evaluated changes in the number of RVGE hospitalizations per 10% increase in vaccine coverage using harmonic regression, controlling for surveillance effort by adjusting for the number of rotavirus-negative acute diarrhea hospitalizations. We fit quasi-Poisson generalized linear models with harmonic terms to data from each country separately, as follows:

$$\begin{gathered} Y\left(t\right)~quasiPoisson\left({\lambda }_t\right) \\ log\left({\lambda }_t\right)={\beta }_0+{\beta }_{1}sin12\left(t\right)+{\beta }_{2}cos12\left(t\right)+{\beta }_{3}v\left(t\right)+{\beta }_{4}n\left(t\right)+ \\ {\beta }_5{I}_{post}\left(t\right)sin12\left(t\right)+{\beta }_6{I}_{post}\left(t\right)cos12\left(t\right)+ϵ \end{gathered}$$ (Equation 1)

where Y(t) was the number of RVGE hospitalizations in each country in month t since the start of surveillance, v(t) was the estimated monthly vaccine coverage among all children <5 years old, and n(t) was the log-transformed monthly number of rotavirus-negative diarrhea hospitalizations. The harmonic terms sin12 and cos12 correspond to the 12-month period of oscillations, i.e. sin12 = sin (2πt/12); these were included in the model because a strong 12-month period of oscillations in RVGE cases was observed in all countries, which was also supported by wavelet analysis. To account for possible changes to the seasonality of rotavirus in the post-vaccine period, we included interactions between the harmonic terms and I_post(t), an indicator variable which was 1 for months in the post-vaccination era and 0 otherwise.

To estimate the monthly vaccine coverage among all children <5 years old, we adjusted the yearly national coverage of infants receiving the last dose of rotavirus vaccines estimated by WHO-UNICEF (Supplementary Figure 4 and Supplementary Methods) [18]. To calculate the relative reduction in the number of RVGE hospitalizations per 10% increase in vaccine coverage, v_t was divided by 10 before being included in the model.

In order to adjust for various temporal factors that might affect the outcome, such as changes in surveillance effort and the size of the population under surveillance (which was unknown), we included the log-transformed number of rotavirus-negative acute diarrhea hospitalizations in each month (n(t)) as a covariate in the model (Supplementary Figure 2B). While the incidence of diarrhea, and hence the total number of tests performed, is expected to decline as a result of vaccine introduction, the number of rotavirus-negative cases should not be affected by rotavirus vaccine introduction, and thus, it allows us to adjust for temporal trends unrelated to the vaccine.

Changes in the average number of RVGE hospitalizations following the vaccine introduction was also evaluated using a different harmonic regression model, in which we replaced the vaccine coverage term with an indicator variable for the post-vaccine period to estimate the overall vaccine impact (Equation 3 in Supplementary Methods).

Evaluating the impact of vaccination on rotavirus seasonality

Vaccination has the potential to alter the dynamics of the disease by reducing the rate of pathogen transmission. To assess the change in timing of rotavirus seasonality after vaccine introduction, we estimated the mean timing of rotavirus activity in each surveillance year using the “center of gravity,” in which each month is weighted according to the number of RVGE hospitalizations occurring that month [16]. Since the data are circular (i.e. December occurs both 11 months after and one month before January), this is estimated in radians according to a weighted circular object. The analysis was performed using the package “circular” in R version 3.4.1 (Vienna, Austria) and 95% CIs were calculated using 1,000 bootstrap iterations [22].

We also assessed whether the seasonal frequency of rotavirus activity was altered after vaccine introduction. Wavelet analysis, a powerful method to analyze non-stationary time series data, was used to identify the dominant frequency in a time series at each specific point in time (Supplementary Methods) [23, 24].

The harmonic regression models (Equation 1) were also used to explicitly test whether the strength (amplitude) and timing (phase angle) of annual seasonal oscillations of RVGE hospitalizations changed following vaccine introduction (Supplementary Methods) [25].

RESULTS

Changes in the proportion rotavirus-positive diarrhea hospitalizations

The proportion of RVGE hospitalizations declined in the six Latin American countries (Figure 1). The proportion rotavirus-positive was between 30% and 60% before vaccine introduction, except for one year (2008) in Paraguay in which the proportion was <20%. According to the random-effects model, all six countries observed a decline in the proportion rotavirus-positive after vaccine introduction (Figure 2). The largest decline was observed in El Salvador (23%, 95% CI: 21%−26%), while the smallest decline was seen in Bolivia (8%, 95% CI: 5%−10%). The pooled estimate of the absolute reduction among six countries was 16% (95% CI: 10%−22%). As a sensitivity analysis, we ran the model using only data from large hospital(s) that consistently reported cases in each country, and found that the pooled estimate of the reduction in the proportion rotavirus-positive did not vary (Supplementary Figure 5).

Figure 1.

Proportion of rotavirus-positive samples among children hospitalized for acute diarrhea whose samples were collected for laboratory testing by surveillance year in six Latin American countries.

Dashed lines represent the timing of rotavirus vaccine introduction. Blue, grey, and red dots represent data points included in the pre-vaccine period, vaccine roll-out window, and post-vaccine period, respectively. Proportions were not calculated for a given surveillance year if there was more than one month in which the number of hospitalized cases with samples collected was missing. Abbreviations: BOL, Bolivia; SLV, El Salvador; GTM, Guatemala; HND, Honduras; PRY, Paraguay; VEN, Venezuela.

Figure 2.

Results of the random-effects model to estimate changes in the proportion of rotavirus-positive samples pre- versus post-vaccine introduction in six Latin American countries. The forest plot shows the absolute difference in the proportion rotavirus-positive before versus after vaccine introduction for each country as well as a pooled estimate. The cumulative numbers of RVGE hospitalizations and tested samples in the pre- and post-vaccine periods are also shown. Black boxes represent point estimates of the absolute different in proportion and their sizes are proportional to the sample size of each country. Horizontal bars represent 95% CIs.

Abbreviations: RVGE, rotavirus-associated gastroenteritis; RE model, random-effects model; CI, confidence interval; BOL, Bolivia; SLV, El Salvador; GTM, Guatemala; HND, Honduras; PRY, Paraguay; VEN, Venezuela.

Changes in the number of RVGE hospitalizations

The relative reduction in the number of RVGE hospitalizations per 10% increase in vaccine coverage was 8.3% in Bolivia (95% CI: 5.0%−11.5%), 6.9% in Guatemala (95% CI: 1.7%−11.9%), 5.4% in El Salvador (95% CI: −0.2%−10.6%), 4.3% in Honduras (95% CI: 1.3%−7.2%), 6.5% in Paraguay (95% CI: −0.2%−12.7%), and 21.4% in Venezuela (95% CI: 16.8%−25.9%). The overall reduction in the average number of RVGE cases pre- versus post-vaccination was 64% in Bolivia (95% CI: 49%−75%), 56% in El Salvador (95% CI: 15%−77%), 33% in Honduras (95% CI: 7%−51%), and 82% Venezuela (95% CI: 66%−90%); more modest reductions were observed in Guatemala (20%, 95% CI: −15%−44%) and Paraguay (9.5%, 95% CI: −24%−34%).

Changes in RVGE seasonality

The seasonality of RVGE hospitalizations shifted following vaccine introduction (Figure 3). In El Salvador, Honduras, Paraguay, and Venezuela (LMD countries), the height of the peaks in the post-vaccine time series (red lines) appeared to be remarkably lower than those for the pre-vaccine time series (grey lines). In Bolivia and Guatemala (HMD countries), the change in the height of the RVGE peaks was modest compared to LMD countries; however, timing of the peaks clearly shifted by a few months to later in the surveillance year.

Figure 3.

Epidemic curve of the monthly number of RVGE hospitalizations for each surveillance year in six Latin American countries.

Grey lines represent the epidemic curves in the pre-vaccine period and red lines are for the post-vaccine period. Colors become darker in later years. For example, the darkest grey line represents the epidemic curve for the last year prior to vaccine introduction, while the lightest red line represents the epidemic curve for the first year post-vaccine introduction. Abbreviations: RVGE, rotavirus-associated gastroenteritis; BOL, Bolivia; SLV, El Salvador; GTM, Guatemala; HND, Honduras; PRY, Paraguay; VEN, Venezuela.

A shift in the timing of rotavirus activity was also supported by the center of gravity analysis (Figure 4). In most countries, peak timing appeared later in the surveillance year after vaccine introduction. Error bars became wider later in the post-vaccine period because there were fewer cases per month. According to wavelet analysis (Figure 5), the one-year period became less pronounced after vaccine introduction in Guatemala, Honduras, and Paraguay. There was some evidence of a biennial period emerging in the last few years of available data in Honduras. Unlike other countries, the one-year period became more pronounced after vaccine introduction in Bolivia.

Figure 4.

Timing of peak rotavirus activity for each surveillance year as measured by center of gravity in six Latin American countries.

Purple dots represent the mean center of gravity and grey bars represent 95% confidence intervals by bootstrap for each surveillance year. Orange dashed lines represent the timing of vaccine introduction. There is no data point for a given surveillance year if there was more than one month in which the number of hospitalized cases with samples collected was missing. Abbreviations: BOL, Bolivia; SLV, El Salvador; GTM, Guatemala; HND, Honduras; PRY, Paraguay; VEN, Venezuela.

Figure 5.

Frequency of RVGE cases described by local and global wavelet power spectra in six Latin American countries.

Local wavelet power spectrum is shown in a color-coded, two-dimensional graph. The x-axis represents year of data collection, while the y-axis is the period in years. Power increases from blue to red, with redder colors indicating greater power in support of a given period of oscillations. Black lines indicate the α=0.05 significance level. White dashed lines indicate the month of vaccine introduction. Shaded areas represent regions outside the cone of influence, in which computed power spectra are less accurate due to boundary effects. Global wavelet power spectra are shown in the line graphs on the right. Grey lines represent the 95% significance level according to a chi-squared distribution. If the black line is above (i.e. to the right of) the grey line, the global wavelet is significant at that period. Abbreviations: RVGE, rotavirus-associated gastroenteritis; BOL, Bolivia; SLV, El Salvador; GTM, Guatemala; HND, Honduras; PRY, Paraguay; VEN, Venezuela.

The average amplitude of seasonal RVGE oscillations either became smaller or did not change after vaccine introduction in all countries other than Bolivia (Supplementary Table 2), consistent with the wavelet and descriptive analyses. The amplitude became larger in the post-vaccine period in Bolivia because the number of RVGE hospitalizations during the non-rotavirus season decreased following vaccine introduction. Changes in the phase angle after vaccine introduction were especially clear among HMD countries. Average peak timing was delayed by 1–2 months in the post-vaccine period in HMD countries.

DISCUSSION

Despite limitations with passive surveillance data, we were able to demonstrate the signatures of the population-level impact of rotavirus vaccines by employing a combination of methods with different strengths. All six countries observed substantial declines in the proportion rotavirus-positive after vaccine introduction. Furthermore, all countries observed declines in the number of RVGE hospitalizations as vaccine coverage increased as well as reductions in the average number of RVGE hospitalizations following vaccine introduction. Changes in RVGE seasonality were demonstrated by a combination of methods, including center of gravity, wavelet analysis, and harmonic regression.

Surveillance data are essential for measuring the population-level impact of vaccines and evaluating changes in disease seasonality. However, results should be interpreted with caution, especially when comparing to estimates of individual-level vaccine effectiveness from randomized clinical trials or case-control studies. The population-level vaccine impact could differ from the individual-level effect for various reasons. As the population-level impact is a combination of direct effect, indirect effect, and vaccine coverage, it could be larger than individual-level direct effect as a result of herd immunity, or could be smaller due to various reasons such as low vaccine coverage. For example, in Bolivia where vaccine coverage was high, the population-level impact estimated in our study (64%) was similar to previously estimated vaccine effectiveness [26]. However, in Guatemala where vaccine coverage remained <80%, our estimate of vaccine impact was lower than the vaccine effectiveness estimated in a case-control study [27]. Thus, measures of vaccine impact based on surveillance data can provide decision-makers with valuable information on the overall vaccine program, which could not be obtained from other types of data, provided the analysis carefully addresses the limitations and potential biases of the data. Such analyses are especially important in settings where more robust studies cannot be conducted due to limited resources.

The major limitation with our surveillance network was inconsistent reporting. The number of hospitals reporting information to the surveillance system varied from year to year during the study period in each country for reasons that are unknown. This variation in surveillance effort was addressed by using the random-effects models to estimate changes in the proportion of tests positive for rotavirus before versus after vaccine introduction and by using the number of rotavirus-negative acute diarrhea hospitalizations as a covariate in harmonic regression models to control for trends unrelated to vaccine introduction.

Other factors affecting the overall trend in the number of RVGE cases may also have changed over time, such as healthcare access and utilization, underlying health conditions, and hygiene practices. By evaluating vaccine impact using the proportion positive, these factors were controlled for to some extent, as they affect both the numerator and the denominator of the proportion positive. We included rotavirus-negative hospitalizations as a covariate in the harmonic regression to adjust the vaccine impact for these temporal factors, as rotavirus-negative hospitalizations should be affected by such factors but not by the introduction of rotavirus vaccines. The use of “synthetic controls,” in which a composite time-series of unrelated diagnoses is used to generate a counterfactual estimate of incidence in the absence of vaccination, has been recently proposed as a more robust way to control for these temporal factors [28, 29]; however, as we did not have data on other diseases, rotavirus-negative hospitalizations can serve as an appropriate control.

Previous studies also evaluated the population-level impact of rotavirus vaccines in Latin American countries. One study found that the number of RVGE hospitalizations decreased by 84% in 2008 and by 69% in 2009 compared to 2006 in El Salvador [30]. We estimated a lower vaccine impact in El Salvador because our study included more recent data and the number of RVGE hospitalizations increased after 2009. Another recent systematic review estimated that the proportion rotavirus-positive decreased from 24.3% pre-vaccination to 16.1% post-vaccination in Latin American countries [10]. As a sub-analysis, we calculated the pooled estimate of the proportion rotavirus-positive for the pre-vaccine period using the data from countries that have not yet introduced rotavirus vaccines and the pre-vaccine data from the six countries included in the main analysis. We compared this to the pooled estimate of the proportion rotavirus-positive for the post-vaccine period using the data from countries that only had post-vaccination data as well as the post-vaccine data from the six countries. The pooled estimate of the proportion rotavirus-positive was 36% (95% CI: 29%−42%) in the pre-vaccine period and 21% (95% CI: 17%−25%) in the post-vaccine period, suggesting a 15% overall difference between the pre- and post-vaccine periods (Supplementary Figure 6). However, it is difficult to directly compare our results to those of the systematic review, as the case definition and denominator of the proportions are different. Also, their study calculated the proportion by pooling data from studies that used different study designs (cohort, case-control, cross-sectional, case series and surveillance), while all countries in our study used the standardized surveillance protocol.

Although the magnitude varied, reductions in the incidence and prevalence of RVGE were found in all studies, which is reassuring that rotavirus vaccines have decreased RVGE among children in Latin American countries. Although the magnitude of decrease in the proportion positive might not seem substantial, the actual number of hospitalizations prevented by the vaccine could be considerable because of the high baseline rate of RVGE in these countries. Also, it has been reported that rotavirus vaccines were more effective in preventing severe gastroenteritis [31], but our sentinel surveillance network recruited all hospitalized diarrhea cases regardless of the severity of diarrhea, which could help explain the small magnitude of decrease.

Another advantage of using surveillance data is that we can evaluate changes in disease seasonality. We observed two major patterns of the effect of vaccine introduction on RVGE seasonality. In some countries, the height of the epidemic curves became lower and the annual oscillations became less pronounced following vaccine introduction. In others, there was a modest reduction in the height of epidemics, but a clear shift in the timing of peak rotavirus activity towards later in surveillance year. The first pattern was commonly observed among LMD countries (Honduras and Venezuela), while the second pattern was more common among HMD countries (Bolivia and Guatemala). The annual oscillations persisted in the post-vaccine period in HMD countries, which could be explained by the higher baseline number of RVGE hospitalizations and/or lower effectiveness of the vaccines due to poorer underlying health in these countries. Peak rotavirus activity was delayed following vaccine introduction because of the reduction in the proportion of susceptible children in the population, as has been demonstrated by mathematical models, and is indicative of herd immunity [16].

Although surveillance data are often not perfect and suffer from various data quality issues, they are typically the only means by which vaccine impact, accounting for both direct and indirect protection from vaccination as well as vaccine coverage, can be quantified. We have illustrated how different methods enabled us to evaluate and compare the impact of rotavirus vaccines on RVGE hospitalizations among children in Latin American countries. Our results showed that the burden of RVGE at the hospital level decreased after vaccine introduction, with changes to the seasonality of disease that reflect a decrease in rotavirus transmission. With a careful understanding of data issues and various methods, imperfect surveillance data can still help to identify the signatures of vaccine impact.

Highlights.

• Rotavirus hospitalizations declined after vaccine introduction in PAHO countries

• Changes to rotavirus seasonality occurred post-vaccination and varied by country

• Various methods provide different advantages for quantifying vaccine impact

Alphabetical List of All Abbreviations and Their Definitions

CI

confidence interval

HMD

high-mortality developing

LMD

low-mortality developing

PAHO

Pan American Health Organization

RVGE

rotavirus-associated gastroenteritis

WHO

World Health Organization