Abstract
Background
Case-control studies to quantify oral cholera vaccine effectiveness (VE) often rely on neighbors without diarrhea as community controls. Test-negative controls can be easily recruited and may minimize bias due to differential health-seeking behavior and recall. We compared VE estimates derived from community and test-negative controls and conducted bias-indicator analyses to assess potential bias with community controls.
Methods
From October 2012 through November 2016, patients with acute watery diarrhea were recruited from cholera treatment centers in rural Haiti. Cholera cases had a positive stool culture. Non-cholera diarrhea cases (test-negative controls and non-cholera diarrhea cases for bias-indicator analyses) had a negative culture and rapid test. Up to four community controls were matched to diarrhea cases by age group, time, and neighborhood.
Results
Primary analyses included 181 cholera cases, 157 non-cholera diarrhea cases, 716 VE community controls and 625 bias-indicator community controls. VE for self-reported vaccination with two doses was consistent across the two control groups, with statistically significant VE estimates ranging from 72 to 74%. Sensitivity analyses revealed similar, though somewhat attenuated estimates for self-reported two dose VE. Bias-indicator estimates were consistently less than one, with VE estimates ranging from 19 to 43%, some of which were statistically significant.
Conclusions
OCV estimates from case-control analyses using community and test-negative controls were similar. While bias-indicator analyses suggested possible over-estimation of VE estimates using community controls, test-negative analyses suggested this bias, if present, was minimal. Test-negative controls can be a valid low-cost and time-efficient alternative to community controls for OCV effectiveness estimation and may be especially relevant in emergency situations.
Keywords: cholera vaccine, case-control study, vaccine effectiveness, test-negative, community controls, Haiti
BACKGROUND
Case-control studies are commonly conducted to quantify vaccine effectiveness (VE)(1–4); however, care must be taken to minimize bias during control selection. Studies of oral cholera vaccine (OCV) effectiveness have often selected matched-controls through community-based recruitment of individuals without diarrhea.(5–9) Community control recruitment is appealing because it facilitates matching on key confounders such as age, neighborhood, and calendar time and can occur until target matching ratios are achieved. Among the disadvantages of community control recruitment are its time- and resource-intensiveness and the potential for attenuation of VE estimates if controls are less likely than cases to engage in health-seeking behaviors, such as vaccination and presentation to a health facility for diarrhea treatment.(10)
Patients who test negative for the condition under study (i.e., test-negative controls) are an alternative control group commonly used in VE studies.(4,11,12) Because test-negative controls can be identified and recruited from the same health facilities recruiting cases, this approach may minimize bias due to differential health-seeking behavior and is time- and resource-efficient.(12) While test-negative controls are used less commonly than community controls in OCV effectiveness research,(13) one review found them to be valid for estimation of rotavirus VE,(14) and others have found that VE estimates generated from test-negative analyses were comparable to those generated by cohort and case-cohort analyses of the same data. (10,15) While the use of test-negative controls in lieu of community controls has great potential to produce rapid and cost-efficient VE estimates, it is unknown how estimates derived from these control groups might compare to one another in the context of cholera.
Using data from an OCV effectiveness case-control study conducted in rural Haiti, we compared VE estimates derived from case-control analyses using community controls to those using test-negative controls. We also conducted a bias-indicator analysis to assess potential bias in VE estimates derived using community controls.(1,16)
METHODS
Study setting
We conducted an OCV effectiveness case-control study in two areas of Haiti where OCV campaigns were directed by the Ministry of Health and Population of Haiti as part of a public health response to cholera. The first 50,000-person campaign took place in the Artibonite Department between April and June, 2012 using killed whole-cell oral cholera vaccine, Shanchol (Shantha Biotechnics, India) and targeted two rural regions (Bocozel and Grande Saline) irrigated by branches of the Artibonite River. An estimated 77–93% of the Bocozel community and 63% of the Grande Saline community were vaccinated against cholera in the 2012 campaign.(17) Early OCV effectiveness estimates from this campaign suggested effectiveness of 58 to 63%.(6) Further OCV campaigns using the same vaccine occurred in subsequent years targeting an additional 395,000 people in high cholera-incidence communities.
Study design
From October 2012 through November 2016, we conducted two case-control studies using community controls: a primary study of OCV effectiveness and a bias-indicator. Bias-indicator analyses are commonly deployed alongside primary VE studies with community controls as a tool to discern the presence of bias.(1,4–6,8,16) Briefly, bias-indicator analyses examine the association between vaccination and a condition to which the vaccine is not expected to afford protection, in this instance, non-cholera diarrhea. In the absence of bias, we expect a null association between oral cholera vaccination and non-cholera diarrhea.
Nested within the primary and bias-indicator case-control studies was a test-negative design, in which non-cholera diarrhea cases (i.e., bias-indicator cases) served as controls to cholera cases.
Participants
All eligible participants were required to meet two conditions: (1) residency in the vaccine catchment area where they were recruited at the start of the study; and (2) eligible for the vaccination campaign when it was implemented in their catchment area (i.e., age ≥12 months, not pregnant, and living in the region at the time of the vaccine campaign).
Identification of cholera and non-cholera diarrhea cases
Patients with acute watery diarrhea—defined as three or more watery, non-bloody stools in a 24-hour period with an onset of three days or fewer before presentation— were recruited from one cholera treatment center and two cholera treatment units in the Artibonite Department beginning in October 2012 and in two cholera treatment centers the Central Department beginning in November 2014. Participants were asked to provide a stool sample for testing by culture and rapid test. Participants with a stool sample that was culture positive for Vibrio cholerae O1 were classified as cholera cases and those whose stool samples tested negative for V. cholerae O1 by both culture and rapid test were classified as non-cholera diarrhea cases. These non-cholera diarrhea cases served as controls in the test-negative OCV effectiveness analysis and also as cases for bias-indicator analyses. Figure 1 shows an overview of enrollment and classification of participants with diarrhea.
Figure 1.
Overview of enrollment of cholera and non-cholera diarrhea cases
*A patient was considered to have complete vaccination data if s/he had data on the number of doses received, both for self-report and a verified source (vaccination card or registry)
Enrollment of community controls
For both cholera cases and non-cholera diarrhea cases, up to four community-based controls were recruited from their residences. Community controls were individuals who could be matched to a case by location of residence, enrollment time (within two weeks of the case), and age group (1–4 years, 5–15 years, and >15 years) and did not seek treatment for diarrhea between the first day of study enrollment and the date of onset of symptoms in their corresponding case. When more than one eligible control was available in a household, an individual of the same sex was selected when possible. If more than one eligible control was available but they were both of different sex to the case, the one most closely matching the case in age was chosen. In rural Haiti, households are often grouped in a cluster of multigenerational families called “lakou”.(18)
In choosing controls, study workers approached the home nearest to the case’s home, excluding homes within the same “lakou” because we anticipated that exposure to the cholera vaccine was likely to be highly correlated within the “lakou”. Study workers then approached the next closest residence until up to four matched controls were enrolled.
Ethical considerations
Ethical approval for participation in the case-control studies were obtained from Partners Institutional Review Board (Boston, MA, USA) and the Haitian National Bioethics Committee (Port-au-Prince, Haiti).
Procedures
Stool samples were collected in sterile containers, and rapid tests were conducted immediately according to the manufacturer’s protocol. An additional specimen was transported in Cary-Blair media to the Haitian National Public Health Laboratory in Port-au-Prince or the Enteric Diseases Laboratory in Saint Marc for subsequent culture on thiosulfate-citrate-bile salts-sucrose agar. Identification of V. cholerae serogroup O1 at the serotype level was done using a standard slide agglutination method.(19)
To collect data on sociodemographic characteristics, cholera risk factors, and self-reported vaccination, study workers interviewed participants with diarrhea at the health facility, upon enrollment. Because culture results were not available at the time of enrollment, participants were unaware of their cholera status at the time of interview. Community controls were interviewed at home. For child participants and those who were unable to respond to interview questions, guardians or a family member proxy responded to questions on behalf of the participant. A study worker abstracted clinical data from the medical charts of participants with diarrhea.
Assessment of vaccination
Oral cholera vaccination was first assessed by self-report during the face-to-face interview. Study workers described the vaccine to study participants in terms of its function, timing of delivery, and mode of administration to differentiate it from other vaccines. If the participant reported receiving the oral cholera vaccine, they were asked how many doses they received. Individuals who reported receipt of at least one dose of the vaccine were asked to produce their oral cholera vaccination card during the home visit. The paper vaccination card was provided to vaccinees during a pre-campaign census, or at the time of their first dose of vaccine. The cards include the individuals name, age, sex, the number of doses and time each dose was received, corresponding vaccine batch number, and a vaccine card bar-code. For participants who enrolled from sites in the Artibonite Department through March 2014, vaccination registries were additionally used to verify vaccination status for individuals who reported vaccination but could not produce a vaccine card or who reported no vaccination.
Statistical analyses
The primary outcome was the effectiveness of two OCV doses as compared to zero doses. Data completeness was high for both vaccination status and covariates. Therefore, primary analyses were restricted to cases and controls that had complete vaccine dose information by both self-report and verified sources, and multivariable analyses were complete case analyses. We calculated VE using the formula as (1-relative risk).(20)
Due to individual matching, we conducted conditional logistic regression for analyses with community controls (i.e., VE analyses and bias-indicator analyses). We reported separate VE estimates for self-reported and verified vaccination and used indicator variables to model the number of vaccine doses received (none, one, or two). The small percentage of individuals who reported receipt of only one dose of the vaccine resulted in imprecise effectiveness estimates for a single OCV dose, particularly for the test-negative design. Therefore, we focused analyses on two-dose effectiveness, relative to no vaccination. Because we matched broadly by age category, we included age as a continuous variable in multivariable models to adjust for residual confounding by age within an age category. We also controlled for cholera risk factors, identified a priori, that were found to be associated with cholera at a p-value of less than 0.20. This approach was used to narrow a long list of biologically plausible cholera risk factors while still allowing control for the most likely confounders in this dataset.
Analyses including test-negative controls were conducted as above, with two exceptions. First, because test-negative controls (i.e., participants with non-cholera diarrhea) comprised the minority of diarrhea cases recruited from participating facilities, we forewent matching and included all eligible non-cholera diarrhea cases in order to maximize statistical power. Therefore, we conducted logistic regression analyses. Because cholera disease is rare (i.e., the yearly incidence is low) in Haiti, the odds ratio derived from logistic regression approximates the relative risk. Second, we additionally considered as potential confounders indicators of diarrheal disease severity (e.g., duration of stay in the health facility, treatment received). Treatment center was not a confounder after adjusting for these severity indicators and was therefore not included the final model. We compared Akaike Information Criterion (AIC) when age was modelled as continuous and as five categorical variables; we elected to model it as continuous because this resulted in the lowest AIC.
Sensitivity analyses
We conducted sensitivity analyses whereby we included all participants with data on the number of doses received, by at least one source (i.e., self-report or vaccination card), rather than requiring that participants have data by both sources. Because complete case analyses may be biased when data are not missing completely at random, to account for the small amount of missing covariate data, we performed multivariable analyses on data sets multiply imputed (N=25) using covariate and outcome data and pooled effect estimates across data sets.(21) Last, we allowed for the possibility that the set of confounders for the relationship between vaccination and non-cholera diarrhea in the bias -indicator study was not identical to the set of confounders for vaccination and diarrhea due to cholera.(4,16) This might be true if different etiologic or social factors contributed to cholera and non-cholera diarrhea or if there were chance variations in the distribution of cholera risk factors. To address this, we examined whether bias-indicator results were similar when we adjusted for diarrhea risk factors identified a priori that were associated with non-cholera diarrhea at a p-value <0.20, rather than risk factors we adjusted for in primary VE analyses with community controls.
RESULTS
Of 184 and 160 participants who met the definition for cholera and non-cholera diarrhea, respectively, 181 (98%) and 157 (98%) were included in primary VE analyses (Figure 1). Clinical characteristics of these participants are shown in Table 1. Cholera patients generally presented with more severe illness, with greater than half presenting with severe dehydration, as compared to 22% of non-cholera diarrhea cases. Patients with cholera were also more likely to receive treatment with intravenous fluids and antibiotics and tended to experience longer stays in the health facility. Sociodemographic and epidemiologic characteristics of study participants are shown in Table 2. Self-reported OCV completion of two doses was 39% among cholera cases; 54% among non-cholera diarrhea cases; and 59% among VE community controls (Table 3). Self-reported vaccination was verified in a minority of participants, with two-dose vaccination rates of 15%, 27%, and 28% respectively (Table 3).
Table 1.
Clinical presentation and treatment of cholera and non-cholera diarrhea cases
| Cholera diarrhea cases N=181a |
Non-cholera diarrhea cases N=157a |
|
|---|---|---|
|
| ||
| Variable | n (%) | n (%) |
| Time from symptom onset to admission (days)b,c | 0 [0 – 1] | 0 [0 – 1] |
| Serotype | ||
| Ogawa | 148 (82) | Not applicable |
| Inaba | 33 (18) | Not applicable |
| Cholera treatment center | ||
| A | 63 (35) | 43 (28) |
| B | 40 (22) | 36 (23) |
| C | 28 (15) | 45 (29) |
| D | 49 (27) | 32 (21) |
| Dehydration stage at presentation | ||
| A (Mild) | 19 (11) | 48 (31) |
| B (Moderate) | 64 (36) | 74 (47) |
| C (Severe) | 97 (54) | 34 (22) |
| Treatment received at clinic | ||
| Oral rehydration solution | 174 (97) | 153 (98) |
| Intravenous fluids | 158 (88) | 85 (54) |
| Antibiotics | 93 (52) | 50 (32) |
| Amount of oral rehydration solution given in clinic (liters)b,d | 8 [4 – 12] | 4 [2 – 6] |
| Amount of intravenous fluids fluid given in clinic (liters)b,e | 7 [3 – 15] | 2 [1 – 5] |
| Admitted overnight to the cholera treatment unit | 175 (97) | 125 (80) |
| Duration of stay at cholera treatment unit (days)b,f | 3 [2 – 5] | 1 [1 – 2] |
| Outcome | ||
| Discharged | 166 (92) | 144 (92) |
| Transferred | 2 (1) | 6 (4) |
| Died | 7 (4) | 1 (1) |
| Left against medical advice | 5 (3) | 5 (3) |
With the exception of cholera serotype (N=181), clinical data was not obtained for one cholera diarrhea case and one non-cholera diarrhea case, thus effective samples sizes are 180 and 156, respectively, for all other variables, unless otherwise noted.
Median [interquartile range]
Data available for 178 cholera diarrhea cases
Among those given oral rehydration solution. Data available for 167 cholera diarrhea cases, 150 non-cholera diarrhea cases
Among those given intravenous fluids. Data available for 158 cholera diarrhea cases, 85 non-cholera diarrhea cases.
Among those admitted overnight. Data available for 175 cholera diarrhea cases, 125 non-cholera diarrhea cases
Table 2.
Selected characteristics of study participants
| Cholera cases (N=181a) |
Community controls (N=716a) |
Bias indicator cases / Test-negative controls (N=157a) |
Bias indicator community controls (N=625a) |
|
|---|---|---|---|---|
| Median age [IQR] | 26 [10, 45] | 27 [10, 43] | 31 [13, 49] | 30 [12, 46] |
| Female sex | 77 (43) | 395 (55) | 85 (54) | 372 (60) |
| Respondent was self | 105 (58) | 467 (65) | 110 (70) | 447 (72) |
| Ever attended school | 90 (50) | 416 (58) | 95 (61) | 352 (56) |
| Main toilet is latrine | 120 (66) | 436 (61) | 107 (68) | 394 (63) |
| Household has electricityb | 68 (38) | 250 (35) | 44 (28) | 168 (27) |
| Makes a living by agriculture | 111 (61) | 395 (55) | 96 (61) | 352 (56) |
| Makes a living by fishing | 20 (11) | 110 (15) | 17 (11) | 94 (15) |
| Reports knowing how to treat water | 175 (97) | 706 (99) | 154 (98) | 621 (99) |
| Reports always treating water | 89 (49) | 422 (59) | 78 (50) | 346 (55) |
| Reports washing hands 4 or more times per day | 57 (31) | 214 (30) | 39 (25) | 145 (23) |
| Household buys waterc | 36 (20) | 121 (17) | 37 (24) | 133 (21) |
| 30 minutes or more on foot from home to riverd | 54 (30) | 174 (24) | 47 (30) | 177 (29) |
| Ate leftover unheated rice in past week | 38 (21) | 177 (25) | 43 (27) | 179 (29) |
| Consumed food or beverage outside of home in the last weeke | 115 (64) | 488 (68) | 109 (69) | 431 (69) |
| Ate raw fruits or vegetables in the last week (N=159)f | 42 (23) | 213 (30) | 49 (31) | 230 (37) |
| Type of cover for household water vesselg | ||||
| Covered | 145 (83) | 597 (84) | 130 (83) | 524 (84) |
| Uncovered | 24 (14) | 105 (15) | 22 (14) | 88 (14) |
| Partially covered | 6 (3) | 9 (1) | 4 (3) | 12 (2) |
| Household water vessel has a taph | 28 (16) | 84 (12) | 21 (13) | 80 (13) |
| Size of opening on household water vessel (narrow versus wide)i | 63 (36) | 215 (30) | 61 (39) | 200 (32) |
| Antacid usage in the past 14 daysj | 11 (6) | 52 (8) | 19 (12) | 54 (9) |
| Previous hospitalization for diarrheak | 24 (13) | 85 (12) | 29 (18) | 73 (12) |
| Prior admission to a cholera treatment unitl | 19 (11) | 84 (13) | 24 (16) | 73 (12) |
| Member of household had diarrhea in last 7 daysm | 29 (16) | 75 (10) | 11 (7) | 39 (6) |
| Member of household previously admitted to cholera treatment unit | 52 (29) | 195 (27) | 59 (38) | 194 (31) |
Unless otherwise noted
Household has electricity: N=715 for community controls; N=156 for bias indicator cases
Household buys water: N=714 for community controls
30 minutes or more on foot from home to river: N=178 for cholera cases; N=714 for community controls; N=155 for bias indicator cases; N=619 for bias indicator controls
Consumed food or beverage outside of home in the last week: N=180 for cholera cases; N=624 for bias indicator controls
Ate raw fruits or vegetables in the last week: N=156 for bias indicator cases
Water vessel cover: N=175 for cholera cases; N=711 for community controls; N=156 for bias indicator cases; N=624 for bias indicator controls
Water vessel has a tap: N=176 for cholera cases; N=714 for community controls; N=156 for bias indicator cases; N=624 for bias indicator controls
Size of opening on water vessel: N=176 for cholera cases; N=714 for community controls; N=156 for bias indicator cases; N=624 for bias indicator controls
Antacid usage in the past 14 days: N=170 for cholera cases; N=672 for community controls; N=154 for bias indicator cases; N=617 for bias indicator controls
Previous hospitalization for diarrhea: N=624 for bias indicator controls
Prior admission to a cholera treatment unit: N=170 for cholera cases; N=672 for community controls; N=154 for bias indicator cases; N=617 for bias indicator controls
Member of household had diarrhea in last 7 days: N=180 for cholera cases, N=715 for community controls
Table 3.
Oral cholera vaccine effectiveness for two doses, versus none, in rural Haiti, complete case analyses
| Cases n (%) |
Controls n (%) |
Crude RR [95% CI] |
Adjusted RR [95% CI] |
Adjusted vaccine effectiveness, % [95% CI] |
p-value | |
|---|---|---|---|---|---|---|
| Cholera vaccine effectiveness case-control study | N=181 | N=716 | ||||
| Self-reported vaccination | 71 (39) | 423 (59) | 0.32 [0.21 – 0.49] | 0.27 [0.16 – 0.45]a | 73 [55 – 84] | <0.0001 |
| Verified vaccination | 28 (15) | 198 (28) | 0.30 [0.17 – 0.51] | 0.26 [0.13 – 0.51]a | 74 [49 – 87] | <0.0001 |
| Bias-indicator case-control study | N=157 | N=625 | ||||
| Self-reported vaccination | 84 (54) | 385 (62) | 0.62 [0.39 – 0.99] | 0.62 [0.37 – 1.03]b | 38 [−3 – 63] | 0.07 |
| Verified vaccination | 42 (27) | 192 (31) | 0.70 [0.41 – 1.19] | 0.71 [0.39 – 1.29]b | 29 [−29 – 61] | 0.26 |
| Test negative design | N=181 | N=157 | ||||
| Self-reported vaccination | 71 (39) | 84 (54) | 0.51 [0.32, 0.81] | 0.26 [0.12 – 0.61]c | 74 [39 – 88] | 0.002 |
| Verified vaccination | 28 (15) | 42 (27) | 0.49 [0.29, 0.84] | 0.38 [0.12 – 1.16]c | 62 [−16 – 88] | 0.09 |
N=875; Adjusted for matching factors (location of residence, enrollment time, age group) via conditional logistic regression and the following covariates: female sex, age (continuous), respondent was self, ever attended school, main toilet is latrine, reports knowing how to treat water, reports always treating water, household buys water, water source (from pump, treated water, bottled water, rain water, well), water treatment method (tablets, boiling, chlorine), same water source used for washing and drinking, makes a living by agriculture, makes a living by fishing, consumed food or beverage outside of home in the last week, ate raw fruits or vegetables in the last week, knowledge on how to avoid cholera (heating food, not going to the bathroom near water source, “other” way not included in list), hand washing habits (before and after touching a baby, at “other” time not included in list), member of household had diarrhea in last 7 days, water vessel cover (uncovered, covered, partially covered), water vessel has a tap, size of opening on water vessel (narrow versus wide), 30 minutes or more on foot from home to river
N=770; Adjusted for same variables listed above.
N=310; Adjusted for female sex, age (continuous), respondent was self, ever attended school, household has electricity, catchment area (first vaccination campaign versus second), water source (from tap, river), water treatment method (tablets, “other” way not included in list), must leave house to get water, same water source used for washing and drinking, antacid usage in the past 14 days, ate leftover unheated rice in past week, ate raw fruits or vegetables in the last week, knowledge on how to avoid cholera (heating food, treating water), hand washing habits (after using toilet, before eating), previous hospitalization for diarrhea, prior admission to a cholera treatment unit, member of household had diarrhea in last 7 days, member of household previously admitted to cholera treatment unit, length of current admission to health facility (days), volume of intravenous fluid provided at clinic, volume of oral rehydration salts provided at clinic, dehydration stage (A, B or C), reports washing hands 4 or more times per day, reports knowing cholera can be transmitted through uncooked food.
Vaccine effectiveness
Figures 2 and 3 compare OCV effectiveness estimates with community and test-negative controls for two doses versus none, stratified by whether vaccination was self-reported or verified. In complete case and multiply imputed analyses of the primary datasets, VE for self-reported vaccination with two doses was robust and consistent across both control groups, with statistically significant point estimates ranging from 72 to 74% (Figure 2, Tables 3 and 4, Supplemental Material Table 1). In sensitivity analyses including all participants, self-reported two dose VE remained the same for community controls but decreased to 66% with test-negative controls for both complete case [CI: 26–85] and multiply imputed analyses [CI: 27–84] (a 9.6% difference from the VE estimate for community controls) but remained statistically significant (Figure 2, Supplemental Material Table 1). In community control analyses, estimates for two dose vaccination verified via a vaccination card or register were the same as estimates from self-reported vaccination (Tables 3 and 4, Supplemental Material Table 1). On the other hand, in test-negative analyses, VE point estimates for verified vaccination were as low as 58%, a 21.6% difference from the corresponding estimate of 74% derived from community controls (Figure 3).
Figure 2.
Oral cholera vaccine effectiveness estimates for two self-reported doses versus none
Figure 3.
Oral cholera vaccine effectiveness estimates for two verified doses, versus none
Table 4.
Oral cholera vaccine effectiveness for two doses, versus none, in rural Haiti, data multiply imputeda
| Adjusted RR [95% CI] | Adjusted vaccine effectiveness, % [95% CI] |
p-value | |
|---|---|---|---|
| Cholera vaccine effectiveness case-control study | N=181 | N=716 | |
| Self-reported vaccination | 0.27 [0.17 – 0.45] | 73 [55 – 83] | <0.0001 |
| Verified vaccination | 0.26 [0.14 – 0.49] | 74 [51 – 86] | <0.0001 |
| Bias-indicator case-control study | |||
| Self-reported vaccination | 0.63 [0.38 – 1.04] | 37 [−4 –62] | 0.07 |
| Verified vaccination | 0.69 [0.38 – 1.25] | 31 [−25 –62] | 0.22 |
| Test negative design | |||
| Self-reported vaccination | 0.27 [0.12 – 0.61] | 73 [39 – 88] | 0.002 |
| Verified vaccination | 0.31 [0.10 – 0.91] | 69 [9 – 90] | 0.03 |
Analyses adjusted for the same variables shown in Table 2
Bias-indicator estimates
Point estimates from bias-indicator analyses were consistently non-null and less than one, ranging from 35 to 43% for self-reported and 26 to 32% for verified vaccination with two doses, indicating a protective association between vaccination and non-cholera diarrhea (Figure 4, Tables 3 and 4, Supplemental Material Tables 1 and 2). While confidence intervals were generally wide, particularly for verified vaccination status, the upper bound for self-reported vaccination hovered around and occasionally excluded one, indicating a statistically significant inverse relationship between two-dose oral cholera vaccination and non-cholera diarrhea.
Figure 4.
Bias indicator estimates for two doses, versus none, using community controls
DISCUSSION
We found that case-control analyses using community and test-negative controls yielded two dose OCV effectiveness estimates that were generally comparable and ranged from 58%–74% up to four years post-vaccination. These estimates are consistent with reported two-dose VE for the Shanchol vaccine.(5,6,22–25) Interestingly, and unexpectedly, bias-indicator estimates were consistently non-null (less than one) in primary and sensitivity analyses and in some cases were statistically significant, raising the possibility of downward bias in estimates derived using community controls. Indeed, while VE estimates derived using each of the control groups were generally similar, those derived with test-negative controls were attenuated relative to community controls in some analyses.
The inverse association observed in the bias-indicator study was unexpected given the hypothesis that those who seek treatment for diarrhea might have increased health-seeking behavior and therefore be more likely to be vaccinated. Operationalized, this bias would overestimate vaccination rates in diarrhea cases seeking care, manifesting as a positive association between non-cholera diarrhea and vaccination in bias-indicator analyses. This bias, if present in primary VE analyses would be expected to attenuate VE estimates. In contrast, we found that non-cholera diarrhea cases were less likely than their community controls to be vaccinated against cholera. This could happen if individuals with diarrhea under-reported vaccination or healthy controls over-reported vaccination (i.e., recall bias). The bias indicator study observation that self-reported vaccination was higher in healthy controls in spite of similar verified vaccination rates among cases and controls is consistent this hypothesis. Of note, the potential for differential reporting of vaccination by cases and controls is minimized in the test-negative design(12) because all participants have diarrhea and, because confirmation of the cause of diarrhea is not available at the time, are unaware of their case status at time of interview. A second explanation for the inverse association in the bias-indicator study is that vaccinated households were less prone to diarrhea due to other factors, such as health-related knowledge or practices or access to clean water, which we were unable to fully account for in spite of statistical adjustment for a broad array of reported cholera risk factors. The plausibility of this explanation is supported by our prior work demonstrating that that cholera-related knowledge and practice improved following vaccination campaign implementation, which included a strong cholera education component.(26) If either of these two mechanisms explain the observed inverse relationship between vaccination and non-cholera diarrhea, there is concern for over-estimation of VE in primary analyses using community controls.
In contrast, there are at least two additional possible explanations that would not undermine the validity of primary VE estimates using community controls. The first is chance. The observed non-null associations may be chance findings. This is supported by p-values of >0.05 in most analyses. Second, it is possible that a small number of cholera cases were misclassified as non-cholera diarrhea cases. We limited misclassification by requiring a negative rapid test and negative culture for inclusion as a non-cholera diarrhea case. However, the negative predictive value of the rapid test in Haiti has been reported to be 91%(27) and while culture is known to be highly specific, the negative predictive value may be jeopardized if individuals took antibiotics prior to stool sample collection. The two-hour transport time to Port au Prince may also have reduced the negative predictive value of cultures processed there, but because samples were transported on ice, and we did not learn of transit disruptions, we expect this occurred rarely, if at all. Regardless of the mechanism, if cases of cholera were inadvertently classified as non-cholera diarrhea cases, this would attenuate VE estimates with test-negative controls by misclassifying cases as controls and could also lead to an inverse association between vaccination and non-cholera diarrhea, such as that which we observed.
Fortunately, because the cases from the bias-indicator study could be used as controls for test-negative analyses, we were able to examine whether VE estimates using community controls were greater than those using test-negative controls. We found that they were often similar with percent differences ranging from 1–10% with self-reported vaccination and 7–22% for verified vaccination. Of note, estimates for verified vaccination results should be interpreted with caution because some individuals were enrolled more than four years post-vaccination and vaccination could be verified in only a minority of individuals who reported it. In contrast, self-reported vaccination rates for controls enrolled in the Artibonite (86%) fell within the estimated vaccine coverage rates of 77 to 93%.(17)
Although our findings suggest that test-negative controls may be a valid and less resource-intensive alternative to community controls, methodologic limitations to this approach have been previously described.(4,12) Additionally, we encountered several implementation challenges with the use of test-negative controls. For example, in the community control design we maximized statistical power and efficiency by matching up to four community controls to each case by age group, neighborhood and calendar time. Nearly all cases (98%) had four community controls. In contrast, use of test-negative controls yielded less than one control per case because non-cholera diarrhea cases, testing negative by both rapid test and culture, represented the minority of diarrhea cases that enrolled in our study, resulting in less precise estimates relative to those derived with community controls. In this context, community controls facilitated achievement of our targeted matching ratio and desired sample size. If a sufficient number of test-negative controls (i.e., non-cholera cases) cannot be recruited efficiently, investigators may be forced to extend study duration to achieve the target sample size, which may undermine the potential economic and time efficiency of the test-negative design.
Furthermore, depending on the epidemiology of non-cholera diarrhea in the area, it is possible that test-negative controls may differ from cholera cases in key ways that may be difficult to measure. For example, while ensuring geographic proximity to cases is straightforward in the community, it may be more challenging to obtain accurate residence locations for patients recruited from a health center. Given that temporal spatial characteristics can determine vaccine uptake and cholera,(28,29) failure to adjust for these variables may bias results. In the present test-negative analyses, we adjusted for age but were unable to precisely control for area of residence or calendar time (because there were months in which a non-cholera diarrhea case was enrolled but a cholera case was not enrolled).
In conclusion, we compared OCV estimates using two control groups with different strengths and weaknesses, both of which yielded similar findings that were consistent with prior studies of OCV effectiveness. While the non-null bias-indicator finding suggested possible overestimation of estimates using community controls, our test-negative design suggested this bias, if present, was minimal, highlighting the challenges of interpreting a non-null finding in a bias-indicator study, in spite of its usefulness as a bias assessment tool. Although limitations of the test-negative design must be considered, we conclude that the test-negative design can be a valid low-cost, time-efficient alternative to community controls for OCV effectiveness estimation. The differences in cost between the test-negative and community-matched case-control design, assuming equal sample sizes, will likely vary across settings and depend on the cost of local transport, the time it takes to travel to communities for control recruitment, and local wages for study workers. The test-negative design may offer the greatest economic advantage in settings where community controls must be recruited from remote areas, requiring extensive time and/or transport costs for study workers, and where there is a sufficient number of test-negative controls (i.e., non-cholera diarrhea cases) that can be efficiently recruited to meet target sample sizes. The lower cost and ease of implementation of the test-negative design be especially relevant in the context of emergency situations where time and resources are particularly limited.
Supplementary Material
Supplemental Material. Table 1. Oral cholera vaccine effectiveness for two doses, versus none, in rural Haiti, all available observations.
Supplemental Material. Table 2. Bias indicator estimates comparing two doses, versus none, using community controls and all available observations, confounders specific to non-cholera diarrhea.
HIGHLIGHTS.
Effectiveness estimates using test-negative and matched community controls were similar
Test-negative controls may be a valid and efficient alternative to community controls
Non-null bias-indicator findings may not necessarily reflect bias in effectiveness estimates
Acknowledgments
We acknowledge and thank Gertrude Cene, Jean Saint Fleur, Jackson Larose and Djencia Augustin for their participation.
SOURCES OF FINANCIAL SUPPORT
This work was supported by the U.S. National Institutes of Allergy and Infectious Diseases [grant number R01AI099243 to LCI and JBH].
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
CONFLICTS OF INTEREST
The authors declare no conflicts of interest.
We report no competing interests.
References
- 1.Shapiro ED. Case-control studies of the effectiveness of vaccines: validity and assessment of potential bias. Pediatr Infect Dis J. 2004;23(2):127–31. doi: 10.1097/01.inf.0000109248.32907.1d. [DOI] [PubMed] [Google Scholar]
- 2.Concato J, Shah N, Horwitz RI. Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs. N Engl J Med. 2000;342(25):1887–92. doi: 10.1056/NEJM200006223422507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Orenstein WA, Bernier RH, Dondero TJ, Hinman AR, Marks JS, Bart KJ, et al. Field evaluation of vaccine efficacy. Bull World Health Organ. 1985;63(6):1055–68. [PMC free article] [PubMed] [Google Scholar]
- 4.Lipsitch M, Jha A, Simonsen L. Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies. Int J Epidemiol. 2016 doi: 10.1093/ije/dyw124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Luquero FJ, Grout L, Ciglenecki I, Sakoba K, Traore B, Heile M, et al. Use of Vibrio cholerae Vaccine in an Outbreak in Guinea. N Engl J Med. 2014;370(22):2111–20. doi: 10.1056/NEJMoa1312680. [DOI] [PubMed] [Google Scholar]
- 6.Ivers LC, Hilaire IJ, Teng JE, Almazor CP, Jerome JG, Ternier R, et al. Effectiveness of reactive oral cholera vaccination in rural Haiti: a case-control study and bias-indicator analysis. Lancet Glob Heal. 2015;3(3):e162–8. doi: 10.1016/S2214-109X(14)70368-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Moradi G, Rasouli MA, Mohammadi P, Elahi E, Barati H. A cholera outbreak in Alborz Province, Iran: a matched case-control study. Epidemiol Health. 2016;38:e2016018. doi: 10.4178/epih.e2016018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lucas MES, Deen JL, von Seidlein L, Wang X-Y, Ampuero J, Puri M, et al. Effectiveness of Mass Oral Cholera Vaccination in Beira, Mozambique. N Engl J Med. 2005;352(8):757–67. doi: 10.1056/NEJMoa043323. [DOI] [PubMed] [Google Scholar]
- 9.Thiem VD, Deen JL, von Seidlein L, Canh DG, Anh DD, Park J-K, et al. Long-term effectiveness against cholera of oral killed whole-cell vaccine produced in Vietnam. Vaccine. 2006;24(20):4297–303. doi: 10.1016/j.vaccine.2006.03.008. [DOI] [PubMed] [Google Scholar]
- 10.Ali M, You YA, Sur D, Kanungo S, Kim DR, Deen J, et al. Validity of the estimates of oral cholera vaccine effectiveness derived from the test-negative design. Vaccine. 2016;34(4):479–85. doi: 10.1016/j.vaccine.2015.12.004. [DOI] [PubMed] [Google Scholar]
- 11.Belongia EA, Simpson MD, King JP, Sundaram ME, Kelley NS, Osterholm MT, et al. Variable influenza vaccine effectiveness by subtype: a systematic review and meta-analysis of test-negative design studies. Lancet Infect Dis. 2016;16(8):942–51. doi: 10.1016/S1473-3099(16)00129-8. [DOI] [PubMed] [Google Scholar]
- 12.Sullivan SG, Tchetgen Tchetgen EJ, Cowling BJ. Theoretical Basis of the Test-Negative Study Design for Assessment of Influenza Vaccine Effectiveness. Am J Epidemiol. 2016;184(5):345–53. doi: 10.1093/aje/kww064. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Wierzba TF, Kar SK, Mogasale VV, Kerketta AS, You YA, Baral P, et al. Effectiveness of an oral cholera vaccine campaign to prevent clinically-significant cholera in Odisha State, India. Vaccine. 2015;33(21):2463–9. doi: 10.1016/j.vaccine.2015.03.073. [DOI] [PubMed] [Google Scholar]
- 14.Tate JE, Patel MM, Cortese MM, Payne DC, Lopman BA, Yen C, et al. Use of Patients With Diarrhea Who Test Negative for Rotavirus as Controls to Estimate Rotavirus Vaccine Effectiveness Through Case-Control Studies. Clin Infect Dis. 2016;62(suppl 2):S106–14. doi: 10.1093/cid/civ1014. [DOI] [PubMed] [Google Scholar]
- 15.Azman AS, Parker LA, Rumunu J, Tadesse F, Grandesso F, Deng LL, et al. Effectiveness of one dose of oral cholera vaccine in response to an outbreak: a case-cohort study. Lancet Glob Heal. 2016 Nov;4(11):e856–63. doi: 10.1016/S2214-109X(16)30211-X. [DOI] [PubMed] [Google Scholar]
- 16.Lipsitch M, Tchetgen Tchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology. 2010;21(3):383–8. doi: 10.1097/EDE.0b013e3181d61eeb. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ivers LC, Teng JE, Lascher J, Raymond M, Weigel J, Victor N, et al. Use of oral cholera vaccine in haiti: a rural demonstration project. Am J Trop Med Hyg. 2013 Oct;89(4):617–24. doi: 10.4269/ajtmh.13-0183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Edmond YM, Randolph SM, Richard GL, Edmond Y, Randolph S, Richard G. The Lakou System: A Cultural, Ecological Analysis of Mothering in Rural Haiti. J Pan African Stud. 2007;2(1) [Google Scholar]
- 19.WHO. WHO. World Health Organization; Manual for the laboratory identification and antimicrobial susceptibility testing of bacterial pathogens of public health concern in the developing world. [Google Scholar]
- 20.Halloran ME, Struchiner CJ, Longini IM. Study designs for evaluating different efficacy and effectiveness aspects of vaccines. Am J Epidemiol. 1997;146(10):789–803. doi: 10.1093/oxfordjournals.aje.a009196. [DOI] [PubMed] [Google Scholar]
- 21.Schafer JL. Analysis of incomplete multivariate data. New York: Chapman and Hall; 1997. [Google Scholar]
- 22.Sur D, Lopez AL, Kanungo S, Paisley A, Manna B, Ali M, et al. Efficacy and safety of a modified killed-whole-cell oral cholera vaccine in India: an interim analysis of a cluster-randomised, double-blind, placebo-controlled trial. Lancet. 2009;374(9702):1694–702. doi: 10.1016/S0140-6736(09)61297-6. [DOI] [PubMed] [Google Scholar]
- 23.Sur D, Kanungo S, Sah B, Manna B, Ali M, Paisley AM, et al. Efficacy of a low-cost, inactivated whole-cell oral cholera vaccine: results from 3 years of follow-up of a randomized, controlled trial. PLoS Negl Trop Dis. 2011;5(10):e1289. doi: 10.1371/journal.pntd.0001289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Bhattacharya SK, Sur D, Ali M, Kanungo S, You YA, Manna B, et al. 5 year efficacy of a bivalent killed whole-cell oral cholera vaccine in Kolkata, India: a cluster-randomised, double-blind, placebo-controlled trial. Lancet Infect Dis. 2013;13(12):1050–6. doi: 10.1016/S1473-3099(13)70273-1. [DOI] [PubMed] [Google Scholar]
- 25.van Loon FP, Clemens JD, Chakraborty J, Rao MR, Kay BA, Sack DA, et al. Field trial of inactivated oral cholera vaccines in Bangladesh: results from 5 years of follow-up. Vaccine. 1996;14(2):162–6. doi: 10.1016/0264-410x(95)00122-h. [DOI] [PubMed] [Google Scholar]
- 26.Aibana O, Franke MF, Franke M, Teng JE, Teng J, Hilaire J, et al. Cholera vaccination campaign contributes to improved knowledge regarding cholera and improved practice relevant to waterborne disease in rural Haiti. PLoS Negl Trop Dis. 2013;7(11):e2576. doi: 10.1371/journal.pntd.0002576. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Boncy J, Rossignol E, Dahourou G, Hast M, Buteau J, Stanislas M, et al. Performance and utility of a rapid diagnostic test for cholera: notes from Haiti. Diagn Microbiol Infect Dis. 2013;76(4):521–3. doi: 10.1016/j.diagmicrobio.2013.03.010. [DOI] [PubMed] [Google Scholar]
- 28.Ivers LC, Teng JE, Lascher J, Raymond M, Weigel J, Victor N, et al. Use of oral cholera vaccine in Haiti: a rural demonstration project. Am J Trop Med Hyg. 2013;89(4):617–24. doi: 10.4269/ajtmh.13-0183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Page A-L, Ciglenecki I, Jasmin ER, Desvignes L, Grandesso F, Polonsky J, et al. Geographic Distribution and Mortality Risk Factors during the Cholera Outbreak in a Rural Region of Haiti, 2010–2011. PLoS Negl Trop Dis. 2015;9(3):e0003605. doi: 10.1371/journal.pntd.0003605. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Allan M, Grandesso F, Pierre R, Magloire R, Coldiron M, Martinez-Pino I, et al. High-resolution spatial analysis of cholera patients reported in Artibonite department, Haiti in 2010–2011. Epidemics. 2016;14:1–10. doi: 10.1016/j.epidem.2015.08.001. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Material. Table 1. Oral cholera vaccine effectiveness for two doses, versus none, in rural Haiti, all available observations.
Supplemental Material. Table 2. Bias indicator estimates comparing two doses, versus none, using community controls and all available observations, confounders specific to non-cholera diarrhea.