Determining the effectiveness of the pentavalent rotavirus vaccine against rotavirus hospitalizations and emergency department visits using two study designs

Abstract

The objective of this study is to determine the vaccine effectiveness (VE) of the pentavalent rotavirus vaccine (RV5) for preventing rotavirus-related hospitalizations and emergency department (ED) visits during the 2006–07 and 2007–08 rotavirus seasons using two study designs. Active, prospective population-based surveillance was conducted to identify cases of laboratory-confirmed rotavirus-related hospitalizations and ED visits to be used in case-cohort and case-control designs. VE was calculated using one comparison group for the case-cohort method and two comparison groups for the case-control method. The VE estimates produced by the three analyses were similar. Three doses of RV5 were effective for preventing rotavirus-related hospitalizations and ED visits in each analysis, with VE estimated as 92% in all three analyses. Two doses of RV5 were also effective, with VE ranging from 79% to 83%. A single dose was effective in the case-cohort analysis, but was not significant in either of the case-control analyses. The case-cohort and the case-control study designs produced the same VE point estimates for completion of the three dose series. Two and three doses of RV5 were effective in preventing rotavirus-related hospitalizations and ED visits.

1. Introduction

Rotavirus is the most common cause of diarrhea-related illness and death among infants and young children [1]. Prior to the introduction of a rotavirus vaccine in the United States (US), nearly all children were infected with rotavirus during the first five years of life [2–5]. The introduction of a three dose, oral, pentavalent, human-bovine reassortant vaccine (RV5) (RotaTeq, Merck and Co. Inc., Whitehouse Station, NJ), in the US led to significant reductions in rotavirus hospitalizations and emergency department (ED) visits [6–12]. RV5, which contains G1–G4 and P8 strains, was licensed for use in the US in February 2006 and subsequently recommended by the Centers for Disease Control and Prevention (CDC)’s Advisory Committee on Immunization Practices and the American Academy of Pediatrics for routine use in infants [13,14]. RV5 is recommended for infants at 2, 4 and 6 months of age, and all doses should be administered before a child reaches 32 weeks of age [15]. Coverage with ≥1 dose of RV5 reached 50–60% during the first year following its introduction in 2006 and increased steadily (2.7% per quarter on average) to 74% by mid-2009 [6].

Because vaccine effectiveness (VE) may be reduced under field conditions [16], post-licensure evaluation of VE is important. Different study designs have been used to estimate post-licensure VE and each has advantages and disadvantages. While case-control studies are most frequently used to determine post-licensure VE [17–27], the case-cohort design is also used for this purpose [28–33]. The vaccination status of diseased cases is compared to disease-free controls with the traditional case-control design, and to a representative subcohort from the community regardless of disease status that produced the cases with the case-cohort design. A direct comparison of these designs for rotavirus VE has not been done.

As part of the CDC-funded New Vaccine Surveillance Network (NVSN), seasonal, population-based acute gastroenteritis (AGE) surveillance was conducted to determine the burden and epidemiology of rotavirus disease and to monitor the impact of the rotavirus vaccine program. This system provided a unique opportunity to determine the effectiveness of RV5 for preventing rotavirus-associated hospitalizations and ED visits using two different study designs.

2. Materials and methods

Active, prospective, population-based surveillance for hospitalizations and ED visits due to AGE in non-immunocompromised children <3 years was conducted during two rotavirus seasons: 2007 (December 2006–June 2007) and 2008 (December 2007–June 2008). Surveillance sites included University of Rochester Medical Center (Monroe County, New York), Cincinnati Children’s Hospital Medical Center (Hamilton County, Ohio) and Vanderbilt University Medical Center (Davidson County, Tennessee), hereafter referred to as “Rochester”, “Cincinnati”, and “Nashville”. Institutional Review Board approvals were obtained from the CDC and study sites. Details of NVSN AGE surveillance have been previously described [34,35].

2.1. Identification of rotavirus-positive inpatient and ED cases

Surveillance for hospitalized children occurred five days each week. In the ED, systematic sampling was performed 2–4 days each week for 10–12 h. Bulk stool samples were obtained within 14 days of the child’s visit for AGE symptoms (>95% collected within 7 days) and tested using Rotaclone, a commercial enzyme immunoassay (EIA) by Meridian Bioscience, Inc. Rotavirus-positive samples were genotyped by reverse transcription polymerase chain reaction and nucleotide sequencing [36–39]. To be eligible as cases, children were required to have a laboratory-confirmed rotavirus-related hospitalization or ED visit, birth on or after April 1, 2006, and age of ≥52 days at symptom onset (old enough to have received one dose of RV5 with two weeks for an immune response).

2.2. Identification of comparison groups

Two study designs, a case-cohort design with one comparison group and a case-control design with two comparison groups, were used to calculate the VE in three separate analyses.

2.2.1. Comparison group #1: subcohort comparison group (case-cohort design)

The first comparison group was a representative subcohort of children registered with primary care practices in each of the three counties. Probability proportional to size (PPS) cluster sampling was used to select a representative subcohort from each county [40]. Subcohort children were required to be county residents, active members of their practice (≥1 visit during the first seven months of life), have a date of birth on or after April 1, 2006 and not be immunocompromised.

2.2.2. Comparison group #2: AGE controls (case-control design)

The second comparison group consisted of children with AGE, found rotavirus-negative by EIA, who were enrolled using the same surveillance protocol during the same timeframe and at the same medical institutions as the rotavirus-positive cases.

2.2.3. Comparison group #3: acute respiratory infection (ARI) controls (case-control design)

The third comparison group consisted of children enrolled in NVSN active surveillance for ARI during the same timeframe and at the same medical institutions as the rotavirus-positive cases. Subjects enrolled in both the AGE and ARI surveillance systems were excluded, thus reducing the potential for misclassification of cases and controls. Details of NVSN ARI surveillance have been previously described [17,41–43].

2.3. Determination of vaccination status

Parents were asked to provide names of all health care providers where their children received immunizations and documentation of the immunization record (electronic or written) was requested. In addition, state immunization registries were queried for documentation of immunizations. For subcohort subjects, the subjects’ practices were visited and medical charts were reviewed for vaccination documentation. For all subjects, administration dates and types of rotavirus vaccine were obtained.

2.4. Statistical analyses

All analyses were conducted using SAS, version 9.3 (SAS Institute Inc., Cary, North Carolina). Significance was defined as P < 0.05, and all P values were 2-sided.

2.4.1. Case-cohort design

VE was calculated for 1, 2 and 3 doses of RV5 against laboratory-confirmed rotavirus-related hospitalizations and ED visits. Vaccination status and number of days spent at risk during rotavirus season were included as time-dependent variables. Time between rotavirus seasons (July–November 2007) was not considered time at risk since rotavirus rarely circulates during those months [10]. Other covariates included date of birth, insurance status, and breastfeeding status. RV5 doses given <14 days before failure (onset of rotavirus symptoms) or censoring were not included, as it takes two weeks for vaccine-induced immunity to fully develop. Subjects entered the risk set at the beginning of the rotavirus season or, if <52 days of age at the beginning of the season, at 52 days of age, and the time axis was age at failure or censoring. There were no subjects selected as members of the subcohort who were also identified by NVSN surveillance as a rotavirus-positive case. A stratified Cox model using study site as the strata variable was used to estimate the adjusted hazard ratio for a rotavirus-related hospitalization or ED visit in vaccinated subjects relative to unvaccinated subjects, with the following formula: VE = (1 − Adjusted Hazard Ratio) × 100. Ninety-five percent confidence intervals (CI) were calculated using the robust sandwich estimator to account for intra-cluster correlation resulting from PPS cluster sampling [44].

2.4.2. Case-control designs using AGE and ARI controls

In both case-control analyses, cases were matched to a variable number of controls (1–5) according to date of birth (±30 days). Controls were also time-matched to cases at failure time (i.e. controls were matched to a case if they visited the hospital or ED on or after the date that their matched case failed). Doses a control received after the failure date of its matched case were not included, and age was calculated as age at the matched case’s failure date. In a small number of instances (1 AGE control and 4 ARI controls), the control visited the hospital or ED before their matched case. The assumption was made that these controls did not receive vaccine in the short period (<6 weeks) between their own visit and their matched case’s visit. This assumption allowed us to match all 76 cases to at least one control. In each instance that this occurred, the matched control was >9 months of age at their visit, so it is unlikely that they would have received vaccine following their visit. In the instance that a control did receive vaccine following their visit, the bias is toward the null. In the event that a child had multiple visits during which they were enrolled by surveillance within the same season, only their initial visit was included in this study. None of the rotavirus-positive cases had multiple visits within the same season. Conditional logistic regression was used to calculate VE for 1, 2 or 3 doses of RV5 with the following formula: VE = (1 − Adjusted Odds Ratio) × 100. RV5 doses given <14 days before the onset of rotavirus symptoms for the cases were not included. For controls, RV5 doses given <14 days before the date of their matched case’s symptom onset were not included. Covariates included age, breastfeeding status, insurance status and site.

3. Results

3.1. Demographics

3.1.1. Cases

Seventy-six cases met the surveillance enrollment inclusion criteria, and had verified vaccination status; 21 from Rochester, 39 from Cincinnati, and 16 from Nashville. Table 1 displays the characteristics of the cases compared to each of the comparison groups.

Table 1.

Demographic characteristics of infants and young children from Rochester, Cincinnati and Nashville, 2007–2008.

Variables	Cases		Case-cohort design			Case-control designs
	N	%	Comparison group #1 Subcohort			Comparison group #2 Matched AGE controls			Comparison group #3 Matched ARI controls
			N	%	P-valuea	N	%	P-valuea	N	%	P-valuea
Gender
Female	42	55	358	48	0.24	89	50	0.45	131	45	0.13
Male	34	45	385	52		90	50		157	55
Insuranceb
Public/none	49	64	286	40	<0.0001	137	77	0.09	224	78	0.02
Private	27	36	436	60		42	23		64	22
Ever breastfedc
No	38	50	221	31	<0.001	75	42	0.17	128	44	0.41
Yes	38	50	490	69		104	58		160	56
NVSN site
Rochester	21	28	204	27	–d	30	17	0.26	89	31	0.02
Nashville	16	21	153	21		55	31		106	37
Cincinnati	39	51	386	52		94	53		93	32
RV5 doses
0 Doses	66	87	249	34	<0.0001	117	65	0.03	190	66	0.01
One dose	5	7	72	10		28	16		39	14
Two doses	3	4	93	13		19	11		32	11
Three doses	2	3	329	44		15	8		27	9

Abbreviations: ED, emergency department; RV5, pentavalent rotavirus vaccine.

^aCompared to cases.

^b21 from the subcohort missing insurance information.

^c32 from the subcohort missing breastfeeding information.

^dAt each site, 1 cluster of 10 subcohort children was selected per case.

3.2. VE using the case-cohort design and subcohort comparison group (comparison group #1)

3.2.1. Comparison group #1: subcohort comparison group (case-cohort design)

The subcohort consisted of 21 clusters (10 children each) from Rochester, 39 clusters from Cincinnati, and 16 clusters from Nashville. After removing subcohort members not meeting inclusion criteria and/or those excluded due to a missing chart or vaccination verification (N = 17), there were 743 subjects included: 204 from Rochester, 386 from Cincinnati and 153 from Nashville. Table 1 compares the demographic characteristics of the cases and subcohort groups. Compared to cases, subcohort subjects were more likely to be privately insured (P < 0.0001) and to have ever been breastfed (P < 0.001).

3.2.2. Comparison of diphtheria, tetanus and acellular pertussis vaccine (DTaP) vaccination status and practices of the cases and subcohort (comparison group #1)

To assess whether cases had the same opportunity to receive RV5 as subcohort subjects, the first DTaP vaccination status was compared; 99% of cases and 93% of subcohort subjects received an initial DTaP (P = 0.0505). There was no difference in age at first DTaP (9 weeks for both groups) (P = 0.3291). To determine whether the selected clusters represented the practices from which the cases arose, the practices of the cases and subcohort were compared. Five of the 21 cases from Rochester (24%), 17 of the 39 cases from Cincinnati (44%) and 4 of the 16 cases from Nashville (25%) were from practices that had at least one cluster in the study.

3.2.3. VE results using the case-cohort design and subcohort comparison group (comparison group #1)

The initial multivariable Cox model included a time-dependent vaccination status variable, a time-dependent variable for number of days spent in a rotavirus season, date of birth, gender, insurance status (public or none versus private) and breastfeeding status (ever breastfed). Gender was removed due to its insignificant effect on VE. The recommended, three-dose course of RV5 was 92% (95% CI: 60%, 99%) effective in preventing rotavirus-related hospitalizations and ED visits. Two doses of RV5 vaccine were 81% (95% CI: 9%, 96%) effective, and one dose was 77% (95% CI: 14%, 94%) effective (Table 2).

Table 2.

Comparison of vaccine effectiveness against laboratory-confirmed rotavirus-related hospitalizations and emergency department visits using three types of comparison groups in infants and young children from Rochester, Cincinnati and Nashville, 2007–2008.

Variable	Case-cohort design	Case-control designs
	Comparison group #1 Subcohorta VE (95% CI)	Comparison group #2 AGE controlsb VE (95% CI)	Comparison group #3 ARI controlsc VE (95% CI)
Three doses	92% (60%, 99%)	92% (21%, 99%)	92% (33%, 99%)
Two doses	81% (9%, 96%)	83% (17%, 96%)	79% (9%, 95%)
One dose	77% (14%, 94%)	68% (−18%, 91%)	58% (−38%, 87%)

Abbreviations: CI, confidence intervals; VE, vaccine effectiveness.

^aAdjusted for number of days at risk (time-dependent), date of birth, insurance status and breastfeeding status.

^bMatched ±30 days on date of birth and adjusted for age, breastfeeding, insurance status and site.

3.3. VE using the case-control design (comparison groups #2 and #3)

3.3.1. Comparison group #2: matched AGE controls

There were 873 children born on or after April 1, 2006, screened by AGE surveillance. Of these, 714 (82%) were enrolled. Controlling for site, clinical setting (inpatient or ED) and enrollment year, non-enrolled children were less likely to have insurance (P < 0.05), but there were no significant differences by age or gender. Major reasons for non-enrollment included refusal (37%), lack of interpreter (35%), and absence of a parent or legal guardian (11%). Of those enrolled by AGE surveillance, 331 were eligible for this study, and 314 had their rotavirus vaccination history verified. Of these 314, there were 179 AGE controls matched to a rotavirus-positive case. Matched AGE controls were similar to cases with regard to all demographic factors (Table 1).

3.3.2. Comparison group #3: matched ARI controls

There were 2129 children born on or after April 1, 2006, screened by ARI surveillance. Of these, 1284 (60%) were enrolled. After controlling for site, clinical setting and enrollment year, there were no significant differences between enrolled and non-enrolled children except non-enrolled children were less likely to have insurance (P < 0.01). Major reasons for non-enrollment included protocol deviation at the Cincinnati site (48%), refusal (20%) and lack of interpreter (14%). Of those enrolled by ARI surveillance, 896 were eligible for this study, and 871 had their rotavirus vaccination history verified. Of these 871, there were 288 ARI controls matched to a rotavirus-positive case. ARI controls were more likely to be publicly insured compared to cases (P = 0.02), and the proportion of subjects from each site differed between ARI controls and rotavirus-positive cases (P = 0.02) (Table 1).

3.3.3. VE results using the case-control design (comparison groups #2 and #3)

Both of the initial conditional logistic regression models (matched on the case’s failure time and ±30 days on date of birth) included: vaccination status, insurance status, breastfeeding status, gender, site and age (at failure for cases, and at time of the matched cases’ failures for controls). Though cases were matched to controls ±30 days on date of birth and exactly on the cases’ failure dates, age remained significant in the multivariable model. Therefore, age was retained in the final model in order to completely control for any residual confounding not completely controlled for by matching. In both models, gender was removed due to its insignificant effect on VE. In the model using AGE controls (comparison group #2), the VE in preventing rotavirus-related hospitalization and ED visits for 3, 2, and 1 dose was 92% (95% CI: 21%, 99%), 83% (95% CI: 17%, 96%), and 68% (95% CI: −18%, 91%), respectively. In the model using ARI controls (comparison group #3), the VE for 3, 2, and 1 dose was 92% (95% CI: 33%, 99%), 79% (95% CI: 9%, 95%), and 58% (95% CI: −38%, 87%), respectively (Table 2).

3.4. Genotype distribution of cases and VE against strains G1–G4

Genotype distribution of the cases varied by year (Table 3). In general, point estimates for VE against serotypes G1–G4 were similar to VE estimates against all serotypes (Table 4).

Table 3.

Distribution of P and G genotypes of rotavirus in infants and young children from Rochester, Cincinnati and Nashville, 2007–2008.

Genotypes	Study years
	2007		2008		2007–2008
	N	%	N	%	N	%
P[8]G1	20	35.1	19	100	39	51.3
P[4]G2	8	14.0	0		8	10.5
P[8]G3	1	1.8	0		1	1.3
P[8]G9	2	3.5	0		2	2.6
P[8]G12	18	31.6	0		18	23.7
Mixeda	2	3.5	0		2	2.6
Rareb	4	7.0	0		4	5.3
Otherc	2	3.5	0		2	2.6
Overalld	57	100	19	100	76	100

^aMixed types – one P[4]P[8]G1G2; one P[4]P[8]G2.

^bRare types – two P[6]G2; two P[8]G2.

^cOther – two P[8].

^dP and G genotypes of rotavirus by site: Rochester: P[8]G3: 4.8%; P[8]G9: 4.8%; P[8]G12: 85.7%; Other: 4.8%; Cincinnati: P[8]G1: 61.5%; P[4]G2: 18.0%; P[8]G9: 2.6%; Other: 17.9%; Nashville: P[8]G1: 93.8%; P[4]G2: 6.3%.

Table 4.

Comparison of vaccine effectiveness against laboratory-confirmed rotavirus-related hospitalizations and emergency department visits caused by strains G1–G4 using three different comparison groups in infants and young children from Rochester, Cincinnati and Nashville, 2007–2008.

Variable	Case-cohort study design	Case-control study designs
	Comparison group #1 Subcohorta VE (95% CI)	Comparison group #2 AGE controlsb VE (95% CI)	Comparison group #3 ARI controlsc VE (95% CI)
Three doses	89% (3–99%)	91% (12–99%)	d
Two doses	78% (−31–96%)	85% (14–98%)	82% (−21–97%)
One dose	66% (−41–92%)	61% (−65–91%)	41% (−141–85%)

Abbreviations: CI, confidence intervals; VE, vaccine effectiveness.

^aAdjusted for number of days at risk (time-dependent), date of birth, insurance status, breastfeeding status and gender.

^bMatched ±30 days on date of birth and adjusted for age, breastfeeding, insurance status and gender.

^cMatched ±30 days on date of birth and adjusted for age, breastfeeding, insurance status, site and gender.

^dThree-dose estimate could not be calculated as the single case strain G1–G4 case with three doses was matched to a single control with three doses.

4. Discussion

This is the first study to evaluate the VE of RV5 using two study designs and three comparison groups with the same set of rotavirus-positive cases. Each of the VE methods found three doses of RV5 to be 92% effective in preventing rotavirus-related hospitalizations and ED visits with similar point estimates for partial doses. Our three-dose VE estimates were comparable to the pre-licensure efficacy trial (94.5%) [45], and similar to other post-licensure RV5 VE studies using case-control designs [21–23,27,46,47].

As part of a population-based surveillance system, cases in our study were selected to represent affected individuals in a defined county population, and the subcohort was selected to represent the county population. The Cox model allows for time-dependent variables, making it possible to consider the changing vaccination status as children receive multiple rotavirus vaccinations and more precisely accounting for the timing and dose effects relative to the disease incidence [48]. This approach is advantageous for VE studies since the vaccination status of children changes over time.

Despite the advantages of using a case-cohort design, certain limitations also exist. The process of selecting a representative subcohort and verifying immunizations can be costly and time-consuming. Because PPS sampling was used to select clusters, intracluster correlation was present. Our model did correct for design effects from our clustering methodology, thus the likelihood that this introduced a significant source of bias in our study is low. Another potential limitation is the varying propensity to seek medical care, which may differ among families who do and do not have their children vaccinated [49]. However, the likelihood of a bias relating to propensity to seek care may be lower in a study such as ours which reflects a somewhat serious outcome (e.g. hospitalization or an emergency department visit). Subcohort subjects were required to be active patients in their practice, which may have introduced a selection bias resulting in participating subjects being better vaccinated than those considered ineligible. To examine this, we reviewed the literature and found that following the RV5 recommendation in February 2006, coverage with ≥1 dose of vaccine among infants aged 5 months rose quickly to about 50–60% within the first year [50]. In our subcohort, of the 40 children born in September 2006, who would have been 5 months of age in February 2007, 19 (47.5%) had received at least one dose of RV5 before 5 months of age. Therefore, we do not feel as though this inclusion criterion resulted in our subcohort being significantly better vaccinated than the actual cohort. Also, no difference existed in the age of the first DTaP dose for cases and subcohort subjects, which suggests that this bias was not present. DTaP, however, may not be the best measure of vaccination availability for a newly introduced vaccine such as RV5 [6]. Unfortunately, we did not have enough specific details for each of the practices to know whether there were differences in RV5 availability and provider recommendations. Finally, although the subcohort group theoretically should have best represented the source population from which the rotavirus-positive cases arose, it appears to have been the least similar to the rotavirus-positive cases in terms of demographics. One study concluded that a subcohort selected to be representative of the community from which hospitalized or ED cases arose was demographically less similar to the cases than test-negative controls enrolled in the hospital or ED, and that a case-cohort study design may not be optimal for determining post-licensure VE [30]. All subjects were required to be residents of surveillance counties, in which 95% of all pediatric hospital admissions are served by NVSN surveillance hospitals. Therefore, if subcohort subjects required hospitalization, they likely had access to these surveillance hospitals.

Demographically, the AGE control group (comparison group #2) was the most comparable of the three comparison groups to the cases. Since AGE controls were enrolled using the same surveillance system that identified rotavirus-positive cases, studies utilizing AGE controls would likely be less expensive than studies using the other two comparison groups. A potential bias with this method is that AGE controls may be misclassified if there are issues with the sensitivity or specificity of the rotavirus EIA results. In our study misclassification is unlikely to have substantially impacted our results since the rotavirus EIA is reported to be >95% sensitive, and >95% of bulk stool specimens were collected within 7 days [51]. Lastly, non-enrolled children were less likely to have insurance compared with enrolled children, so our analysis may under-represent the non-insured, which may be a less well-vaccinated population.

The case-control design utilizing ARI controls (comparison group #3) provided a comparison group that was demographically more similar to the cases than the subcohort, but less similar than the AGE controls. Since ARI controls did not have AGE, misclassification of cases and controls is unlikely. While ARI subjects were enrolled using a surveillance system of children in the same age group as the AGE subjects, the distribution of age and children over the seasons differed. Similar to the AGE analysis, non-enrolled children were less likely to have insurance compared with enrolled children, so this study population may under-represent the noninsured, a potentially less well-vaccinated population. We found no advantage to using ARI controls over AGE controls.

In our study the case-cohort design produced VE point estimates similar to those produced by the more traditional case-control designs. This finding differs from two studies evaluating influenza VE, where a case-cohort method had substantially lower VE than the case-control method using the same cases [17,30]. For influenza vaccination, propensity to seek care may be more important than for rotavirus vaccination in which the cases and subcohort subjects were more closely aligned, perhaps making the case-cohort analysis more robust.

RV5 was found to be highly effective in preventing rotavirus-related hospitalizations and ED visits and, despite methodologically specific strengths and weaknesses, both the case-cohort and case-control study designs produced the same VE point estimates for a three-dose vaccination course of RV5.