Abstract
Background
Ten years ago, we formulated two hypotheses about whole-cell diphtheria-tetanus-pertussis (DTP) vaccination: first, when given after BCG, DTP increases mortality in girls and, second, following DTP there is an increase in the female/male mortality rate ratio (MRR). A recent review by WHO found no convincing evidence that DTP increases mortality in females.
Methods
We used previous DTP reviews as well as the recent WHO review for assessing the hypotheses. As pre-specified we excluded studies with survival or frailty bias; if children had received BCG and DTP simultaneously; and if the children had received neonatal vitamin A.
Results
In seven studies of BCG-vaccinated children, DTP vaccination was associated with a 2.54 (95% CI 1.68–3.86) increase in mortality in girls (with no increase in boys [ratio 0.96, 0.55–1.68]). In 10 studies of BCG-vaccinated children, the female-to-male mortality ratio was 2.45 (1.48–4.06) times higher after DTP than before DTP. In 15 studies of children who had received DTP after previous BCG vaccination, mortality was 1.53 (1.21–1.93) times higher in girls than boys. The findings were similar in studies conducted before and after formulation of the hypotheses.
Conclusions
The two hypotheses were confirmed in the studies that fulfilled pre-specified criteria.
Keywords: BCG, Diphtheria-tetanus-pertussis vaccine, DTP, Non-specific effects of vaccines, Sex-differential effects, Strategic Advisory Group of Experts on Immunization (SAGE)
Introduction
In the early 2000 s, we suggested that diphtheria-tetanus-pertussis vaccine (DTP) was associated with increased mortality.1-3 Subsequently, WHO's Global Advisory Committee on Vaccine Safety (GACVS) sponsored the reanalysis of some existing data sets4-9 and concluded in 2004 that there was no basis for the suggestion that DTP might be associated with increased female mortality.10 However, the analysis of the WHO-sponsored studies introduced survival bias.1,11,12 Following recognition of this methodological error 10 years ago, together with researchers at the London School of Hygiene & Tropical Medicine, we formulated a set of testable hypotheses about the non-specific effects of vaccines, which were published in Tropical Medicine and International Health (the ‘TMIH-hypotheses’).1
The hypotheses were largely based on previous reports which had suggested, first, that DTP was associated with increased child mortality in rural Guinea-Bissau2 and, second, that the real reason why the high-titre measles vaccine (HTMV) had been associated with increased female mortality was that children had received DTP after measles vaccination (MV).3
Two of these hypotheses dealt with DTP and its potential sex-differential effects: 1. among girls who have received BCG vaccine, DTP vaccination, given alone or with oral polio vaccine, is causally associated with 25–50% higher mortality from causes other than diphtheria, tetanus or pertussis than that in girls who have not received DTP (Hypothesis-1); 2. the female mortality rate following DTP vaccination is higher than the corresponding male mortality rate (Hypothesis-2 [Box 1]).
Box 1. Hypotheses and methodologies relating to the sex-differential effects of diphtheria-tetanus-pertussis (DTP).
The ‘TMIH hypotheses’1 and methodology used to test them (the present paper)
Hypothesis 1. ‘Among girls who have received BCG vaccine, DTP vaccinations, given alone or with oral polio vaccine (OPV), are causally associated with substantial higher mortality (25–50%) (from causes other than diphtheria, tetanus or pertussis) than that in girls who have not received DTP. This lasts for at least 6 months after DTP vaccination or until a different vaccine is given within 6 months after vaccination.’1
Estimate assessed: MRR=mortality rate ratio in DTP vaccinated females/ mortality rate in DTP unvaccinated females
Hypothesis 2. ‘Among BCG-vaccinated children there is an increase in the female/male (F/M) mortality ratio when DTP vaccinations are given (alone or with OPV). This increase lasts for at least 6 months after the initial DTP vaccination, or until a different vaccine is given within 6 months after vaccination.’1
In several situations the interval between BCG vaccination and DTP vaccination may be too small to assess the F/M mortality ratio with precision. A variant of Hypothesis 2 is that the female mortality rate following DTP vaccination is higher (by at least 30%) than the corresponding male mortality rate.
Estimates assessed: Change in female-male MRR following DTP-vaccination assessed through:
F/M mortality ratio for DTP-vaccinated children, divided by the F/M mortality ratio for DTP-unvaccinated children (which is mathematically equivalent to the female DTP-vaccinated/DTP-unvaccinated mortality ratio, divided by the male DTP-vaccinated/DTP-unvaccinated mortality ratio).
The F/M MRR following DTP vaccination.
Conditions for both hypotheses as pre-specified in the original statement1
Exclusion criteria
• Any other vaccine given with any dose of DTP (other than OPV). • Infants given hepatitis B vaccine at birth. • Antimalarial drugs and/or other interventions given in trials or routinely with DTP (that will confound comparisons). • Vitamin A supplementation given at birth. • Situations in which boys and girls are treated differentially in ways that may affect mortality, including differential access to vaccination.
Factors affecting the choice of suitable study situations for addressing hypotheses
• Good vaccination records accessible to investigators for a high proportion of study children, held centrally or by parents/guardians. • If data are captured longitudinally, the date of recording vaccine status must be known. • It should be possible to distinguish between absence of records and absence of vaccination. • Vaccination records of dead children should be captured/copied, where possible. • Information should be collected on potential confounders, which may indicate that DTP may be preferentially given to infants at high or low risk of death.
The SAGE review of Hypothesis 2 and the methodology used to test it17
The SAGE review assessed ‘the difference in vaccine effect between boys and girls, which is equivalent to the comparison in boy‐girl mortality ratios between DTP-vaccinated and DTP-unvaccinated children.’17
Estimate assessed: Mortality rate in DTP vaccinated females/ mortality rate in DTP unvaccinated females divided by mortality rate in DTP vaccinated males/ mortality rate in DTP unvaccinated males.
No exclusion criteria. Studies considered very high risk of bias included.
Following further observational studies and randomised trials suggesting non-specific effects of vaccines,13-16 WHO's Strategic Advisory Group of Expert on Immunization (SAGE) sponsored a thorough review of the literature on the possible non-specific effects of BCG, DTP and MV on mortality of children less than 5 years of age.17,18 In 2014, the review concluded that BCG and MV were associated with nearly a halving of mortality, an effect which if true could not be explained by prevention of TB or measles infection. For DTP the majority of studies (7 of 10)19 suggested that DTP had a deleterious effect but the literature was considered inconsistent because two studies4,8 indicated a beneficial effect. These studies dealt with whole-cell pertussis DTP and not acellular pertussis vaccine. Regarding the potential sex-differential effect of DTP the SAGE review reached the conclusion ‘one study reported a large differential effect in girls, but overall there was not convincing evidence of a differential between girls and boys’.17
In the present paper, we review the evidence for the two hypotheses on DTP, 10 years after their formulation. We based our literature search on previous reviews and the recent literature search for the SAGE review. Given the contradiction between our previous observations and the SAGE review conclusion, we examined whether differences in methodology, including the choice of studies, could be the explanation.19,20
Methods
Studies of DTP vaccination, sex and mortality
Our previous review of the sex-differential effect of DTP covered studies up to 2010.13 The review had 14 studies of mortality after DTP administered before MV; three of these studies did not have information on sex and several did not have information on a DTP-unvaccinated group.
In 2013–2014 SAGE organised a comprehensive review of the potential non-specific effects of DTP on mortality of children less than 5 years of age. The SAGE review identified 16 studies of DTP: 10 were used to compare DTP-vaccinated to DTP-unvaccinated children (six of the 16 were excluded because of perceived very high risk of bias). The SAGE reviewers used 12 of the 16 studies in the analysis of the sex-differential effects of DTP (four studies had no information on sex); apparently the reviewers thought that the ‘very high risk of bias’ did not affect the quality of data on sex. The SAGE reviewers excluded some studies from our previous review because there was no DTP-unvaccinated group, they were hospital case fatality studies or there was fear of overlap with other cohort studies. Supplementary Table 1 2,4,6–8,21–46 lists all studies used in the SAGE review and in the present review, along with the reasons for including or excluding them in the main analyses. Box 1 presents the TMIH hypotheses and proposed methodology,1 and the hypothesis and methodology used in the SAGE review, respectively.
Assessment of risk of bias and relevance of studies
The SAGE review emphasised the assessment of risk of bias. Since all DTP studies were observational, they all had at least ‘medium’ or ‘high risk’ of bias. Six of the 16 studies had ‘very high risk’ and were excluded from the SAGE review's main comparison of DTP-vaccinated versus DTP-unvaccinated children. Bias was assessed within seven domains: 1. bias due to confounding (including frailty bias); 2. bias in participation into the study (including inception bias); 3. bias in measurement of intervention (including survival bias); 4. bias due to departure from intended interventions (performance bias)—e.g., were critical co-interventions balanced over intervention groups. If all children received a co-administered vaccine (other than oral polio vaccine with DTP) the study was excluded; 5. bias in measurement of outcome (detection bias)—with all-cause mortality as the main outcome no major problem in this domain; 6. bias due to missing outcome data (attrition bias)—not considered to be problems; and 7. bias in selection of the reported results (reporting bias)—the reviewers had problems in assessing this but assumed that all studies had ‘moderate risk of bias’.17
The SAGE review did not assess the direction of bias. Nearly all studies suggest that the healthiest children are vaccinated first;13,19 this would lead to DTP-vaccinated children having lower mortality than DTP-unvaccinated children. Apart from reporting bias, the only proposed biases working in the opposite direction have been that starting follow-up after vaccination (as would happen with data from demographic surveillance systems) could mean that frail children had already died in the unvaccinated group leading to higher measured mortality in the DTP group; secondly, censoring for MV during follow-up could mean that the healthiest DTP-children were measles vaccinated first and the frail were left to die in the DTP group. It has been shown specifically in one study that the delay in starting follow-up had no impact on the estimate27 and several studies have started follow-up at the time of DTP vaccination23,26 and found similar strong negative effects, so there is no evidence that this bias is important. Similarly, the studies which have tested whether censoring for MV matters have found no effect8 and many studies have not censored for MV and still found a strong negative effect of DTP2,6,23,25,40 so there is no evidence that this bias is important.
Survival and frailty bias
Building on the previous discussion of methodological issues in relation to DTP-vaccination studies,1,11–13 there are other biases which are far more important, particularly survival and frailty bias. As documented in detail elsewhere,19 the SAGE-review included several studies in which the DTP-unvaccinated group had an implausibly high mortality rate (see Supplementary Table 1).
Frailty bias occurs when the healthy children are vaccinated and the unhealthy children are left in the unvaccinated group, which therefore has an implausibly high mortality rate.19 Survival bias occurs in studies that have retrospectively updated the dates of vaccinations for survivors, but were unable to obtain retrospective vaccine information from children who died because their vaccination card was no longer available. The latter group of children have then been classified with the vaccination status they had when they were last seen. Hence, any vaccine given between household visits would not be captured, and some ‘vaccinated deaths’ would therefore be misclassified as ‘unvaccinated deaths’. Furthermore, many studies of DTP have unfortunately not documented whether and when the child was ‘unvaccinated’; if a child's vaccination status had not been documented, the child was assumed to be unvaccinated. As a consequence, many children are left in the unvaccinated group because they have already died, whereas survivors were assigned to their respective vaccination groups. Hence, studies in which the immunisation status of ‘unvaccinated’ control group has not been actively determined are generally of poor quality.
Both types of bias lead to an implausibly high mortality rate in the unvaccinated group and to underestimation of the mortality rate ratio (MRR) for DTP-vaccinated children compared to DTP-unvaccinated children. We have suggested the use of a bias index calculated by dividing the mortality rate for children classified as unvaccinated by the mortality rate of all children who have received any vaccination. As discussed in detail elsewhere, studies with adequate assessment of vaccination status (unvaccinated versus vaccinated) and prospective follow-up have had a bias index below 2.0, whereas studies with important survival bias or frailty bias have had a bias index above 2.0.19 For example, in three of the studies included in the SAGE review the ‘unvaccinated’ group was a default group not verified by inspection of the vaccination card, and the bias index was 2.3, 3.4 and 7.5 in these studies (Table 1).4,7,8 In the presentation of the hypotheses of the non-specific effects of DTP, it was indicated that care should be taken that DTP was not preferentially given to infants with either a high or low risk of death.1 Thus, as explained in Table 1, we have excluded these three studies with poor data quality from both Hypothesis 1 and Hypothesis 2 analyses of the possible sex-differential effects of DTP. Two other studies were excluded from both the SAGE review and from this analysis: one compared older vaccinated children with younger unvaccinated children without correcting for age and had a bias index of 6.8,21 and in the other study 63% of children received BCG and DTP simultaneously.22 Furthermore, we have excluded one study23 in which the children were DTP-vaccinated, but had not been BCG-vaccinated.
Table 1.
Study used in SAGE review17; period; age group; length of follow-up (FU); and vaccine comparison | Mortality rate per 1000 person-years (deaths/person-years) or case fatality % (deaths/cases) | MRR DTP vaccinated vs DTP-unvaccinated females |
Comments; adjustments; vaccine during FU | Bias index, i.e. MRR for unvaccinated vs vaccinated children (95% CI) | |
---|---|---|---|---|---|
Included studies | DTP-vaccinated females | DTP-unvaccinated females | |||
Studies in which most or all DTP-unvaccinated had received BCG | |||||
In SAGE analysis: Guinea-Bissau II2
1990–96; children aged 0–6 and followed for 6 months; DTP1 (mostly after BCG) vs BCG or unvaccinated |
Also BCG 4.1% (28/680)a
Only DTP 18.2% (2/11) |
BCG-vaccinated 2.0 (11/544)a
Unvaccinated 4.0 (38/943) |
2.31 (1.16–4.59)a,b | Non-vaccinated likely to receive DTP during FU. Estimates similar when follow-up was censored at 9 months. VAS not given. Adjusted – see paper. |
1.35 (1.0–1.9). |
Excluded in SAGE analysis: Guinea-Bissau V40 1990–1996; Hospital case fatality; children aged 1.5–8 mo; DTP1-3 (mostly after BCG) vs BCG |
17.8% (35/197)a | BCG 12.1% (4/33)a | 1.47 (0.56–3.85) | No vaccine during FU. VAS not given. |
Vaccination status defined by card at entry; hence, no unvaccinatedI |
In SAGE analysis: Senegal I24
1997–2001; age 0–24 mo; DTP1 after BCG vs BCG (Table 1) |
130 (10/77.0)a | 75 (162/2160.8)a | 3.55 (0.92-17.57)b | Vaccination coverage low so other vaccines during FU were unlikely. Females with BCG=DTP1 excluded. VAS not given. Adjustment: Age, season, health centre. |
1.48 (1.2–1.8). |
Excluded in SAGE analysis: Guinea-Bissau IV25
Case fatality during 3 months of war (1998), 1.5–6 months; DTP1-3 vs no DTP (mostly BCG vaccinated) |
6.1% (14/228)a | 2.9% (1/35)a | 2.33 (0.14–39.17)b | Vaccine unlikely during FU due to war. VAS not given. |
0.67 (0.1–5.4)II |
In SAGE analysis: Malawi26
1995–1997; mortality between 1 week and 8 months; monthly anthropometrics; only children present; DTP1 (mostly after BCG) vs no DTP (BCG vaccinated) |
198 (10/50.4) | No DTP 87 (6/68.9) | 5.44 (0.88-33.7)a,b | Vaccines unlikely during FU with one month FU. Adjusted age, HIV status of mother VAS probably not given. |
1.45 (0.7–3.2). |
In SAGE analysis: Guinea-Bissau III27
2004–2008; LBW cohort; vaccination status at 2 month FU to 6 monthc DTP1-2 after BCG vs BCG |
115 (13/113.3)a | BCG 34 (2/58.1)a | 7.18 (1.53–33.7)a,b | Only BCG-arm of study with BCG first and then DTP. DTP unvaccinated children likely to receive DTP during FU. Some may have received NVAS. Adjusted age, MUAC. |
1.45 (0.7–3.2). |
In SAGE analysis: India II6
1998–2001; 1 week to 5 months; NVAS trial DTP and BCG vs BCG |
2.52 (1.04–6.09)b | A few may have received BCG and DTP simultaneously. SAGE used the estimate for all children (both NVAS and placebo recipients). If restricted to placebo recipients the MRR was 3.15 (0.95–10.46). Since the difference is small we have maintained estimate used by SAGE. |
1.38 (0.8–2.3). | ||
Excluded studies | |||||
Studies in which DTP-unvaccinated had mostly not received BCG | |||||
In SAGE analysis: Guinea-Bissau23
1984–87: age 2–8 mo; FU 6 mo; DTP1-3 versus unvaccinated (few BCG vaccinated) |
126 (27/215.0) | NA | 2.34 (1.04-5.27)a,b | Controls were unvaccinated. Not likely to receive other vaccines during FU. Adjustment: age, sex, season, BCG, region, period, BCG. VAS not given. |
0.49 (0.3–0.9)III |
Studies in which most or all DTP–unvaccinated had received BCG | |||||
In SAGE analysis: Bangladesh I8,31
1985–1999; 6 weeks to 8 months DTP after BCG vs BCG only |
31 (13/416)a | 77 (34/440)a | 0.36 (0.18–0.72)a | 3.40 (2.9–3.9)I,V | |
In SAGE analysis: Burkina Faso7
1985–1993; vaccination status at first visit and 6 months FU DTP and BCG vs BCG only |
51 (15/295)a | 64 (16/251)a | 0.81 (0.39–1.66) | 2.29 (1.7–3.0)V | |
In SAGE analysis: Papua New Guinea4
1989–1994; 4 weeks to 5 months DTP after BCG vs BCG only |
13 (4/309)a | 26 (4/155)a | 0.50 (0.13–2.01) | 7.52 (5.2–11.0)V,I
Very high mortality in unvaccinated. |
|
Excluded in SAGE analysis: India I21
2006–2011; 6 weeks to 8 months; DTP after BCG vs BCG only |
29 (73/2523)a | 78 (19/245)a | 0.37 (0.22–0.62) | 6.82 (4.4–10.5). Very high mortality in unvaccinated. |
|
Excluded in SAGE analysis: Philippines22
1988–1991; 0–30 months BCG+DTP vs BCG only |
Data not presented in paper | 0.96 (0.26–5.15) | MRR(unvaccinated/vaccinated) could not be calculated because the study only included children who had received BCG 63% received BCG and DTP simultaneously so this study did not compare DTP after BCG versus BCG. |
DTP: diphtheria-tetanus-pertussis; FU: follow-up; LBW: low birth weight; MRR: mortality rate ratio; MUAC: mid-upper-arm-circumference; MV: measles vaccination; NA: not applicable; NVAS: neonatal vitamin A supplementation; SAGE: Strategic Advisory Group of Experts on Immunization; VAS: vitamin A supplementation.
The SAGE review had 16 studies of DTP; 5 studies mentioned in the table were excluded because they had survival and frailty bias with a bias index higher than 2.0; 4 studies, India,30 Ghana,44 Ghana45 and Benin46 had no information on sex. The Roman numerals are used when there are several studies from the same country; the same numerals have been used in a previous analysis of the SAGE review of DTP.19
a Estimate or numbers are in the paper; & calculated by the person responsible for the data set.
b Estimate reported in the SAGE review.17
c Estimate for BCG is from a randomised comparison of BCG at birth versus BCG later (as currently recommended for LBW children). The estimate is only from the BCG at birth arm of the trial because most children in the control groups received BCG and DTP simultaneously or closely spaced.
I: Guinea-Bissau V was not included in the SAGE review. This study is not overlapping with any of the other studies included in the SAGE review. The bias index could not be assessed because all children had been vaccinated. The study included children sick enough to be admitted to hospital, so inclusion depended on degree of illness and not vaccination status. This is a strong version of a case-control study where the population is children admitted to hospital, so it is a far more robust design than the Benin case-control study that was included in the SAGE review. There is no reason to suspect that high-risk children who had received DTP or low-risk children who had not received DTP were more likely to be admitted. Note that community-based studies are also a sample because not all children are seen, and the reason for not being seen is often linked to vaccination status (for example, did not attend clinic or away travelling), whereas admission is very unlikely to have been linked to vaccination status in this study. We therefore included the study in our review. The hospital case fatality study included a few readmissions, as 6% (12/185) had been admitted more than once with DTP as their last vaccination; the similar figure is not reported for the BCG group. Hence, it was not possible to make the comparison for only the last admission. However, this is unlikely to have exaggerated the results as there were probably fewer readmissions with BCG as last vaccination because BCG-vaccinated children will become eligible for DTP-vaccination shortly after being discharged. Hence, the DTP/no-DTP estimate in this study is likely to be conservative.
II: Guinea-Bissau IV was not included in the SAGE review. This study is not overlapping with any of the other studies included in the SAGE review. The study has a bias index of only 0.7. Due to war in Guinea-Bissau in 1998, routine immunisation was suspended and DTP was not given when it was due. Whether a child received DTP or not depended on the war and not on factors that may be associated with the health of the child (such as attendance at clinic, or too ill to vaccinate) as in most cohort study or case-control study. This is not a typical cohort study, and many of the criteria used to assess bias in cohort studies do not apply. Virtually all children were vaccinated with BCG. We therefore included the study in our review.
III: This study was included in the SAGE review. However, since the DTP-unvaccinated children were mostly not BCG-vaccinated the study is not included in the testing of Hypothesis 1.
IV: In the Bangladesh study ‘unvaccinated’ was the default classification - no child was excluded for lack of information, and ‘unvaccinated’ was not verified by inspection of the vaccination card or interview. Consequently, children who were vaccinated and died may have been incorrectly coded as unvaccinated if vaccination was not documented before death (for example, because of travel). Similarly, some children who died after receiving DTP may not have had DTP vaccination registered, which would make mortality too high in the no-DTP group and too low in the DTP group. We have therefore not included the study in our review.
V: In Burkina Faso, when the vaccination card was not seen, the child was assumed to be unvaccinated; as in Bangladesh, children who had been given DTP and died may therefore have been incorrectly classified as unvaccinated. According to the authors 54% of follow-up time (72938 person-months) was assigned to unvaccinated, 11% (14238 person-months) to BCG, 3% (3411 person-months) and 33% (44515 person-months) to BCG and DTP (Table 3).5 Hence, a large proportion of children received DTP with BCG. First, administration with BCG is likely to reduce harm from DTP. Second, the hypothesis being tested is the effect of DTP given after BCG. Hence, mortality was over-estimated in the no-DTP group and under-estimated in the DTP group, and it is impossible to deduce who received DTP after BCG. We have therefore not included the study in our review.
VI: PNG study performed in Tari has extraordinarily high frailty bias (and perhaps survival bias), and this is reflected in its very high bias index of 7.5. The 1-5 month mortality rate in unvaccinated children was 233 per 1000 person-years, a staggeringly high rate for this community, compared to only 30.9 in vaccinated children. At the time of the study, infant (0-12 month) mortality in Tari was only 68 per 1000 live births,4 so 1-11 month mortality was probably around 34 per 1000 as approximately 50% of infant deaths are before 1 month of age in these circumstances. Frailty bias occurred in the PNG study because vaccines were given only at health centres4 and health centres were instructed not to vaccinate sick children. In addition, the SAGE review used an estimate from this study for DTP where, among children aged 1-5 months who got DTP (Table 3), only 69% got DTP after BCG (2185/3153); 22% (686/3153) got DTP before or with BCG - which biases the estimate in favour of DTP. BCG is purported (implausibly) to reduce infant mortality by 83% in this study. Frailty bias very strongly influenced the MRR for both BCG and DTP because both vaccinations were delayed in frail children. We have therefore not included the study in our review.
Hypothesis 1: studies of DTP-vaccinated and DTP-unvaccinated girls
We included seven studies of children (who had previously received BCG) that compared the mortality rate of DTP-vaccinated and DTP-unvaccinated girls2,6,24-27,40 (Table 1); the bias index varied between 0.5 and 1.7 in these studies. These studies had some form of age-adjustment. Five of the studies had been included in our previous review of DTP13 and two studies were identified in the SAGE review.6,24 There may have been a slight survival bias in the two studies identified by the SAGE review because the status of ‘unvaccinated’ was not documented, but in one study the interval between household visits to collect vaccination information was only 2 weeks6 and in the other study there were so few vaccinated children24 that a few dead children incorrectly classified as unvaccinated will have limited impact on the estimate; the bias index was 1.38 and 1.48 in these two studies. We have analysed the data both including and excluding these two studies.
Hypothesis 2: studies assessing the effect of DTP for girls and boys among BCG-vaccinated children
There are two ways to assess whether there is an increase in the female/male (F/M) mortality ratio when DTP is given among BCG-vaccinated children as specified in hypothesis 2 (Box 1).
First, the F/M mortality ratio for DTP-vaccinated children, divided by the F/M mortality ratio for DTP-unvaccinated children (Supplementary Table 2) is mathematically equivalent to the female DTP-vaccinated/DTP-unvaccinated mortality ratio, divided by the male DTP-vaccinated/DTP-unvaccinated mortality ratio (Table 1 and Supplementary Table 3). Five studies reported data for both ratios and they were similar.2,6,24,26,40 Five studies reported only data for one or the other ratio; we have assumed that they would also be similar.21,25,27,39,42 Hence, 10 studies had data to assess the change in F/M mortality ratio when DTP is administered to BCG-vaccinated children.
Second, since many studies do not report data for BCG-vaccinated children or have little data on BCG-vaccinated children, we had also formulated a variant of hypothesis 2 that the female mortality rate following DTP vaccination is higher (by at least 30%) than the corresponding male mortality rate (Box 1).1 Fifteen studies had estimates of female and male mortality rates following the primary DTP vaccinations and before receiving MV (Supplementary Table 4). Seven studies were also part of the SAGE analysis,2,6,24-27 four were included in our previous review of DTP39-42 and the last four were found in the SAGE literature search but sex estimates were not included in the SAGE analysis.30,35,43,44 The studies not included in the SAGE analysis are explained in Supplementary Table 1.
Statistical analyses and methodological considerations
We have indicated in the tables to what extent the studies have adjusted their estimates for socio-economic and health related background factors. As specified in the hypotheses, DTP should not have been co-administered with other vaccines than oral polio vaccine or with other interventions. The length of follow-up is critical as it relates to whether the DTP-vaccinated and the DTP-unvaccinated children were likely to receive other vaccines during follow-up (Supplementary File 1).
For Hypothesis 1, we compared the mortality rate of DTP-vaccinated and DTP-unvaccinated (but BCG-vaccinated) girls.
For Hypothesis 2, we compared: the change in mortality rate following DTP-vaccination after BCG-vaccination for both females and males; the F/M mortality ratio for BCG-vaccinated and DTP-vaccinated children, respectively; and the mortality rate of DTP-vaccinated females and DTP-vaccinated males within the same study.
Combined estimates of vaccine efficacy against death were obtained using meta-analysis in Stata 12.1 (StataCorp LP, College Station, TX, USA).
Results
Hypothesis 1: DTP-vaccinated girls have higher mortality than DTP-unvaccinated girls
As seen in Figure 1, all seven studies showed higher mortality in DTP vaccinated versus DTP-unvaccinated but BCG-vaccinated girls. In the meta-analysis, DTP-vaccinated girls had 2.54 (1.68–3.86) times higher mortality than DTP-unvaccinated girls. Since the results were homogeneous (p for homogeneity=0.69), a fixed effects model was used. In the studies from Guinea-Bissau, the MRR was 2.30 (1.37–3.86), whereas it was 3.05 (1.51–6.14) in the studies from elsewhere (Senegal, Malawi, India). In a further study from Guinea-Bissau,23 the DTP-unvaccinated girls had not received BCG (Table 1); if that study was included in the comparison of DTP-vaccinated girls versus DTP-unvaccinated girls, the meta-estimate for the eight studies was 2.50 (1.73–3.62).
The effect was slightly less in the studies2,25,40 completed before the hypotheses were formulated in 2005 (MRR=2.00 [1.15–3.46]) than in more recent studies (MRR=3.53 [1.86–6.67]). Excluding studies in which some had received neonatal vitamin A supplementation (NVAS)6,27 the MRR was 2.29 (1.40–3.76). Excluding two studies with possibly slight survival bias6,24 made little difference to the estimate (MRR=2.46 [1.49–4.04]).
The MRR for DTP-vaccinated boys and DTP-unvaccinated (but BCG-vaccinated) boys in the same seven studies was 0.96 (0.55–1.68) (random effects model as p for homogeneity=0.08), significantly different from the effect among girls (p=0.006, test of interaction) (Supplementary Table 3).
Hypothesis 2: following DTP vaccination females have higher mortality than males
Using the estimate for all 10 studies in a meta-analysis, the increase in the F/M MRR was 2.45 (1.48–4.06) when children received DTP after BCG (Table 2)(random effects model, p for homogeneity=0.15).
Table 2.
Study name used by SAGE17; period | Vaccines | F/M MRR for DTP-vaccinated/BCG-vaccinated children (95% CI) |
---|---|---|
Papers published before the hypotheses formulated | ||
Guinea-Bissau and Senegal: Female-male twin pairs; 1978–200039 | BCG and DTP1-3 | 29.3 (3.80–226.3) |
Guinea-Bissau II;1990–19962 | BCG and DTP1 | 1.51 (0.64–3.58) |
Guinea-Bissau V; 1990–199640 | BCG and DTP1-3 | 3.02 (0.92–9.90) |
Guinea-Bissau IV; 199825 | BCG and DTP1-3 | 11.10 (0.19–634.9) |
Papers published after the hypotheses | ||
Malawi; 1995–199726 | BCG and DTP1 | 2.34 (0.34–16.3) |
Guinea–Bissau VI; 2002–200442 | BCG and DTP1-3 | 1.04 (0.35–3.07) |
Guinea–Bissau III, 2004–200827 | BCG and DTP1-2 | 2.90 (0.36–23.28) |
Senegal I; 1997–200024 | BCG and DTP1 | 15.94 (2.00–126.8) |
India I; 2008–201121 | BCG and DTP1-3 | 1.94 (0.91–4.14) |
India II; 1998–20016 | BCG and DTP1-3 | 1.96 (0.79–4.84) |
Meta-estimate (random effects model) | 2.45 (1.48–4.06) |
DTP: diphtheria-tetanus-pertussis; FM: female/male; MRR: mortality rate ratio; SAGE: Strategic Advisory Group of Experts on Immunization.
The Roman numerals are used when there are several studies from the same country; the same numerals have been used in a previous analysis of the SAGE review of DTP19
We also tested the variant of Hypothesis 2 by comparing the F/M MRR among children who had DTP as their last vaccination. As seen in Figure 2 (and Supplementary Table 4), 15 studies could be used for this analysis; 13 of the 15 studies found higher female than male mortality. The F/M MRR was 1.53 (1.21–1.93) (random effects model). In seven studies from Guinea-Bissau it was 1.58 (0.98–2.54) whereas it was 1.56 (1.26–1.94) in the eight studies from elsewhere. In the four studies published before Hypothesis2 was formulated,2,25,39,40 the F/M MRR was 1.90 (0.94–3.88). In the following 11 studies the MRR was 1.49 (1.21–1.84). The F/M MRR was similar in the seven studies used for testing Hypothesis 1 (MRR=1.54 [1.05–2.28]) and in the other eight studies (MRR=1.54 [1.13–2.10]).
Excluding the five studies in which some children had received NVAS or might have received MV during follow-up (Table 2)2,6,27,35,44 did not change the estimate (MRR=1.72 [1.28–3.11]).
In the seven studies (Table 2) in which there was mortality information for unvaccinated children, the F/M MRR was 1.53 (1.05–2.24) for DTP-vaccinated children and 0.91 (0.72–1.15) among DTP- and BCG-unvaccinated children (test of interaction, p=0.02).
Discussion
Main observation
When the two hypotheses regarding higher female mortality after DTP were formulated, only a few studies had been published. The subsequent studies have provided further support for both hypotheses. Hence, girls are likely to have higher mortality if they are DTP-vaccinated rather than DTP-unvaccinated, and DTP-vaccinated girls are likely to have higher mortality than DTP-vaccinated boys.
Strengths and weaknesses
SAGE recently sponsored a comprehensive review of the literature on DTP17 to which we have added the literature from previous reviews11,13 providing little doubt that we have covered all relevant studies. Even if a few more studies were found, it is unlikely that they would change the trend in the studies.
Since the non-specific effects have been labelled a Guinea-Bissau phenomenon,10 it is worth emphasizing that the trend for both hypotheses were equally strong in data from outside Guinea-Bissau, including studies from Bangladesh, India, Malawi, Senegal, and Ghana (Figures 1 and 2).
Given the uncertainties in observational studies, where the children may have received other vaccines during follow-up, it is likely the estimates presented here are conservative. Furthermore, due to frailty bias, which can be difficult to control, a comparison of DTP-vaccinated children versus those who have not yet received DTP will most likely overestimate the beneficial estimate of the effect on survival. Hence, the effect of DTP for females may in fact be worse, and DTP could also have a negative effect for boys even though we found no effect in the present study.
Comparison with previous studies
The SAGE review evaluated the potential sex-differential effect of DTP only by assessing interactions between the separate MRRs for DTP-vaccinated and DTP-unvaccinated girls and boys, respectively, corresponding to the analysis we presented Table 2.17 The SAGE review was unsatisfactory because it included studies with major survival bias (Table 1, Supplementary Table 1). Since these studies have implausibly high mortality in the unvaccinated groups there will be a lot of misclassification of deaths in the ‘unvaccinated’ group. Many estimates will therefore be false and they should clearly not be used to assess whether DTP has a differential effect for girls and boys. In three studies4,7,8 with major survival bias girls benefitted more than boys from DTP-vaccination, and the SAGE reviewers therefore concluded that there was no convincing evidence of a differential effect for girls and boys.17,18
Furthermore, the SAGE analysis limited the number of eligible studies because some studies had no information on the mortality rate of BCG-vaccinated children, but only mortality rates of DTP-vaccinated females and DTP vaccinated males. We therefore also tested the variant version of Hypothesis 21 which is less affected by these problems (Box 1).
Though not reviewed here, the studies of DTP administered after MV likewise suggest that DTP has worse effects for girls than for boys.3,13,21 The most spectacular was the trials of HTMV in which the increased female mortality after HTMV turned out to be due to the fact that it was given early and many children got DTP after HTMV; the excess female mortality was only seen among children who received DTP after HTMV.3 Several other studies have also suggested that DTP after MV is linked to excess female mortality. In India, where booster DTP is recommended, a recent study showed that the F/M MRR was 2.21 (1.24–3.93) after booster DTP even though there was no sex-differential effect after MV.21
Interpretation
The test of Hypothesis 1 supports that DTP is associated with an absolute increase in mortality for females in areas with high mortality. Since there was no reduction in mortality for males after DTP vaccination, the comparison of mortality for females and males in Hypothesis 2 would further support that DTP is associated with an absolute increase in mortality. This increase in mortality was clearly stronger for girls (Table 2, Figure 2).
Vaccines may reprogram both the innate and the adaptive immune systems to non-targeted infections, thus producing non-specific changes in susceptibility to a broad range of infections.32,33 It may be easier to accept that a vaccine can reduce susceptibility to other infections, as is the case with BCG and MV. However, if a system can be programmed beneficially it is also likely that it can be misdirected, increasing susceptibility to other infections. The ways in which the female and the male immune systems may respond differently to vaccinations in infants are only beginning to be studied.47
Implications
Fifteen years ago, we published the first paper that suggested that DTP might be associated with a general increase in mortality.2 In response, the Global Advisory Committee on Vaccine Safety (GACVS) sponsored a series of reanalyses of existing data sets to examine possible sex-differential and non-specific effects of DTP. GACVS dismissed the hypotheses as unfounded in 2004.10 However, it was subsequently acknowledged that many studies on which GACVS based its conclusion had used an incorrect methodology and introduced survival bias into the analysis.1,11,12 This process led to the formulation of the hypotheses published in 2007.1
In 2013–2014, SAGE undertook a comprehensive review of the potential non-specific effects of BCG, DTP and MV.17 This review is still marred by the inclusion of studies which have extraordinarily high mortality rates for the unvaccinated group. The reviewers handled the inadequacy of some studies by assigning different grades of bias. However, the studies with survival bias or frailty bias should have been excluded, because they grossly misrepresent reality by having an implausibly high mortality in the control group, thereby inflating the beneficial effects of vaccines. By including these studies, the SAGE review has introduced a notion of inconsistency. But as demonstrated in Tables 1 and 2 there are really no inconsistencies in the effect of DTP, when using the methodology proposed along with the TMIH hypotheses in 2007. We have identified no inconsistent results – only inconsistent and incorrect methodologies in some studies.
The finding that DTP is associated with increased mortality in girls is of major importance. DTP is the most widely used vaccine, and coverage with DTP3 has been used to monitor the performance of the immunization program. The emphasis on DTP3 as the monitoring target has helped promote DTP coverage, but has sometimes reduced delivery of other vaccines. This process has not contributed to better health.48 It would seem more logical to use BCG and MV, which are associated with better child survival,17,18 to monitor performance of the vaccination program.
In conclusion, the association of DTP with increased mortality has been clouded by flawed studies to suggest that trends are inconsistent.1,10,17-19 The TMIH hypotheses published in 2007 were formulated in 2005.1 Now, ten years later, we have tested the two hypotheses using studies from before, but also those published since formulation of the hypotheses. All studies published after the formulation of the hypotheses have supported them. It is now imperative to vigorously examine the link between DTP and susceptibility to infection. Is a new vaccine needed, can the damage be minimized by co-administering DTP with other vaccines or by providing a live vaccine shortly after DTP?
Supplementary data
Supplementary data are available at Transactions online (http://trstmh.oxfordjournals.org/).
Acknowledgments
Authors’ disclaimers: The funding agencies had no role in the study design, data collection, data analysis, data interpretation, or the writing of the report.
Authors’ contributions: HR, ABF, AR, CSB, and PA have all been involved in examining the SAGE review from 2014 and agreed that a revised review was necessary. HR, CSB, and PA took part in formulating the original Trop Med Int Health hypotheses which are the basis for the present review. HR, ABF, AR, CSB, and PA were co-authors of previous reviews of DTP and took part in several of the studies of DTP and sex-differential effects which have contributed to the present review. The first draft was written by PA; all authors critically revised the manuscript for intellectual content. HR was responsible for the statistical analyses. PA is guarantor of the paper.
Funding: The work on vaccination has been supported by the Danish Council for Development Research, Ministry of Foreign Affairs, Denmark [grant number 104.Dan.8.f.]. The project also received support from Fonden til Lægevidenskabens Fremme, Novo Nordisk Foundation and European Union FP7 support for OPTIMUNISE [grant: Health-F3-2011-261375]. CSB holds a starting grant from the ERC [ERC-2009-StG-243149]. CVIVA is supported by a grant from the Danish National Research Foundation [DNRF108]. PA holds a research professorship grant from the Novo Nordisk Foundation.
Competing interests: None declared.
Ethical approval: Not required.
References
- 1.Fine PEM, Smith PG. ‘Non-specific effects of vaccines´ - an important analytical insight, and a call for a workshop. Trop Med Int Health 2007;12:1–4. [DOI] [PubMed] [Google Scholar]
- 2.Kristensen I, Aaby P, Jensen H. Routine vaccinations and child survival: follow up study in Guinea-Bissau, West Africa. BMJ 2000;321:1435–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Aaby P, Jensen H, Samb B et al. . Differences in female-male mortality after high-titre measles vaccine and association with subsequent vaccination with diphtheria-tetanus-pertussis and inactivated poliovirus: re-analysis of West African studies. Lancet 2003;361:2183–8. [DOI] [PubMed] [Google Scholar]
- 4.Lehmann D, Vail J, Firth MJ et al. . Benefits of routine immunisations on childhood survival in Tari, Southern Highlands Province, Papua New Guinea. Int J Epidemiol 2004;34:138–48. [DOI] [PubMed] [Google Scholar]
- 5.Elguero E, Simondon F, Simondon K, Vaugelade J. Non-specific effects of vaccination on survival: a prospective study in Senegal. Trop Med Int Health 2005;10:956–60. [DOI] [PubMed] [Google Scholar]
- 6.Moulton LH, Rahmathullah L, Halsey NA et al. . Evaluation of non-specific effects of infant immunizations on early infant mortality in a southern Indian population. Trop Med Int Health 2005;10:947–55. [DOI] [PubMed] [Google Scholar]
- 7.Vaugelade J, Pinchinat S, Guielle G et al. . Lower mortality in vaccinated children: follow up study in Burkina Faso. BMJ 2004;329:1309–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Breiman RF, Streatfield PK, Phelan M et al. . Effect of infant immunization on childhood mortality in rural Bangladesh: analysis of health and demographic surveillance data. Lancet 2004; 364:2204–11. [DOI] [PubMed] [Google Scholar]
- 9.Nyarko P, Pence B, Debpuur C Immunization status and child survival in rural Ghana. Population Council. Working papers No. 147. New York: Population Council; 2001.
- 10.Global Advisory Committee on Vaccine Safety Wkly Epidemiol Rec 2004;79:269–72. [PubMed] [Google Scholar]
- 11.Aaby P, Benn CS, Nielsen J et al. . DTP vaccination and child survival in observational studies with incomplete vaccination data. Trop Med Int Health 2007;12:15–24. [DOI] [PubMed] [Google Scholar]
- 12.Jensen H, Benn CS, Lisse IM et al. . Survival bias in observational studies of the impact of routine vaccinations on childhood survival. Trop Med Int Health 2007;12:5–14. [DOI] [PubMed] [Google Scholar]
- 13.Aaby P, Benn CS, Nielsen J et al. . Testing the hypothesis that diphtheria-tetanus-pertussis vaccine has negative non-specific and sex-differential effects on child survival in high-mortality countries. BMJ Open 2012;2:e000707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Aaby P, Whittle HC, Benn CS. Vaccine programmes must consider their effect on general resistance. BMJ 2012;344:e3769 doi:10.1136/bmj.e3769. [DOI] [PubMed] [Google Scholar]
- 15.Aaby P, Roth A, Ravn H et al. . Randomized trial of BCG vaccination at birth to low-birth-weight children: beneficial nonspecific effects in the neonatal period. JID 2011;204:245–52. [DOI] [PubMed] [Google Scholar]
- 16.Aaby P, Martins CL, Garly ML et al. . Non-specific effects of standard measles vaccine at 4.5 and 9 months of age on childhood mortality: Randomised controlled trial. BMJ 2010;341:c6495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Higgins JPT, Soares-Weiser K, Reingold A Systematic review of the non-specific effects of BCG, DTP and measles containing vaccines. http://www.who.int/immunization/sage/meetings/2014/april/presentations_background_docs/en/ [accessed 1 October 2016]. [DOI] [PMC free article] [PubMed]
- 18.Strategic Advisory Group of Experts on Immunization Week Epidemiol Rec 2014;89: 233–5. [PubMed] [Google Scholar]
- 19.Aaby P, Ravn H, Benn CS. The WHO review of the possible non-specific effects of diphtheria-tetanus-pertussis vaccine. Pediatr Infect Dis J 2016;35:1247–57. [DOI] [PubMed] [Google Scholar]
- 20.Farrington CP, Firth MJ, Moulton LH et al. . Epidemiological studies of the non-specific effects of vaccines: II - methodological issues in the design and analysis of cohort studies. Trop Med Int Health 2009;14:977–85. [DOI] [PubMed] [Google Scholar]
- 21.Krishnan A, Srivastava R, Dwivedi P et al. . Non-specific sex-differential effect of DTP vaccination may partially explain the excess girl child mortality in Ballabgarh, India. Trop Med Int Health 2013;18:1329–37. [DOI] [PubMed] [Google Scholar]
- 22.Chan GJ, Moulton LH, Becker S et al. . Non-specific effects of diphtheria-tetanus-pertussis vaccination on child mortality in Cebu, The Philippines. Int J Epidemiol 2007; 36:1022–9. [DOI] [PubMed] [Google Scholar]
- 23.Aaby P, Jensen H, Gomes J et al. . The introduction of diphtheria-tetanus-pertussis vaccine and child mortality in rural Guinea-Bissau: an observational study. Int J Epidemiol 2004,33:374–80. [DOI] [PubMed] [Google Scholar]
- 24.Aaby P, Nielsen J, Benn CS, Trape JF. Sex-differential and non-targeted effects of routine vaccinations in a rural area with low vaccination coverage: Observational study from Senegal. Trans Roy Soc Trop Med Hyg 2015;109:77–85. [DOI] [PubMed] [Google Scholar]
- 25.Aaby P, Jensen H, Garly M et al. . Routine vaccinations and child survival in war situation with high mortality: effect of gender. Vaccine 2002;21:15–20. [DOI] [PubMed] [Google Scholar]
- 26.Aaby P, Vessari H, Nielsen J et al. . Sex differential effects of routine immunizations and childhood survival in rural Malawi. Pediatr Inf Dis J 2006;25:721–7. [DOI] [PubMed] [Google Scholar]
- 27.Aaby P, Ravn H, Roth A et al. . Early diphtheria-tetanus-pertussis vaccination associated with higher female mortality and no difference in male mortality in a cohort of low birthweight children: an observational study within a randomised trial. Arch Dis Child 2012:97:685–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Aaby P, Rodrigues A, Biai S et al. . Oral polio vaccination and low case fatality at the paediatric ward in Bissau, Guinea-Bissau. Vaccine 2004;22:3014–7. [DOI] [PubMed] [Google Scholar]
- 29.Aaby P, Hedegaard K, Sodemann M et al. . Childhood mortality after oral polio immunisation campaign in Guinea-Bissau. Vaccine 2005;23:1746–51. [DOI] [PubMed] [Google Scholar]
- 30.Hirve S, Bavdekar A, Juvekar S et al. . Non-specific and sex-differential effects of vaccinations on child survival in rural western India. Vaccine 2012;30:7300–8. [DOI] [PubMed] [Google Scholar]
- 31.Higgins JPT, Soares-Weiser K, López-López JÁ et al. Association of BCG, DTP, and measles containing vaccines with childhood mortality: systematic review. BMJ 2016;355:i5170. [DOI] [PMC free article] [PubMed]
- 32.Kleinnijenhuis J, Quintin J, Preijers F et al. . Bacille Calmette-Guerin induces NOD2-dependent nonspecific protection from reinfection via epigenetic reprogramming of monocytes. Proc Natl Acad Sci U S A 2012;109:17537–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Benn CS, Netea MG, Selin LK, Aaby P. A small jab – a big effect: non-specific immunomodulation by vaccines. Trends Immunol 2013;34:431–9. [DOI] [PubMed] [Google Scholar]
- 34.Benn CS, Bale C, Sommerfelt H et al. . Hypothesis: Vitamin A supplementation and childhood mortality: amplification of the non-specific effects of vaccines. Int J Epidemiol 2003:32:822–8. [DOI] [PubMed] [Google Scholar]
- 35.Fisker AB, Bale C, Rodrigues A et al. . A randomised trial of high-dose vitamin A at vaccination contacts after 6 months of age. Pediatrics 2014;134(3):e739–48. [DOI] [PubMed] [Google Scholar]
- 36.Martins CL, Benn CS, Andersen A et al. . A randomized trial of a standard dose of Edmonston-Zagreb measles vaccine given at 4.5 months of age: effect on total hospital admissions. JID 2014;209;1731–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Benn CS, Diness BR, Roth A et al. . Effect of 50,000 IU vitamin A given with BCG vaccine on mortality in infants in Guinea-Bissau: randomised placebo controlled trial. BMJ 2008;336:1416–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Benn CS, Fisker A, Napirna BM et al. . Vitamin A supplementation and BCG vaccination at birth in low birthweight neonates: two by two factorial randomised controlled trial. BMJ 2010;340:c1101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Aaby P, Jensen H, Rodrigues A et al. . Divergent female-male mortality ratios associated with different routine vaccinations among female-male twin pairs. Int J Epidemiol 2004;33:367–73. [DOI] [PubMed] [Google Scholar]
- 40.Veirum JE, Sodemann M, Biai S et al. . Routine vaccinations associated with divergent effects on female and male mortality at the paediatric ward in Bissau, Guinea-Bissau. Vaccine 2005;23:1197–203. [DOI] [PubMed] [Google Scholar]
- 41.Benn CS, Fisker AB, Jørgensen MJ, Aaby P. Why worry: Vitamin A with DTP vaccine. Vaccine 2007;25(5):777–9. [DOI] [PubMed] [Google Scholar]
- 42.Benn CS, Rodrigues A, Yazdanbakhsh M et al. . The effect of high-dose vitamin A supplementation administered with BCG vaccine at birth may be modified by subsequent DTP vaccination. Vaccine 2009;27:2891–8. [DOI] [PubMed] [Google Scholar]
- 43.Aaby P, Nielsen J, Benn CS, Trape JF. Sex-differential effects of BCG and diphtheria-tetanus-pertussis vaccine in a rural area with high vaccination coverage: Observational study from Senegal. Trans R Soc Trop Med Hyg 2016;110:527–33. [DOI] [PubMed] [Google Scholar]
- 44.Welega P, Nielsen J, Adjuik M et al. . Non-specific effects of diphtheria-tetanus-pertussis and measles vaccinations? An analysis of surveillance data from Navrongo, Ghana. Trop Med Int Health 2012;17:1492–505. [DOI] [PubMed] [Google Scholar]
- 45.Bawah AA, Phillips JF, Adjuik M et al. . The impact of immunization on the association between poverty and child survival: Evidence from Kassena-Nankana district of northern Ghana. Scand J Pub Hlth 2010;38:951–03. [DOI] [PubMed] [Google Scholar]
- 46.Velema JP, Alihonou EJ, Gandaho T, Hounye FH. Childhood mortality among users and non- users of primary health care in a rural West African community. Int J EpidemioI 1991;20:474–9. [DOI] [PubMed] [Google Scholar]
- 47.Flanagan KL, van Crevel R, Curtis N et al. . Heterologous (“Nonspecific”) and sex-differential effects of vaccines: epidemiology, clinical trials, and emerging immunological mechanisms. CID 2013;57:283–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Fisker AB, Hornshøj L, Rodrigues A et al. . Effects of the introduction of new vaccines in Guinea-Bissau on vaccine coverage, vaccine timeliness, and child survival: an observational study. Lancet Global Health 2014;2:e478–87. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.