Abstract
Previously, the Fiji Pneumococcal Project (FiPP) evaluated reduced dose immunization schedules that incorporated pneumococcal protein conjugate and/or polysaccharide vaccine (PCV7 and 23vPPV, respectively). Immune hyporesponsiveness was observed in children vaccinated with 23vPPV at 12 months of age compared with children who did not receive 23vPPV.
Here we assess the long-term impact of 23vPPV vaccination on nasopharyngeal carriage rates and densities of Streptococcus pneumoniae, Haemophilus influenzae, Staphylococcus aureus and Moraxella catarrhalis. Nasopharyngeal swabs (n=194) were obtained from healthy children who participated in FiPP (now aged 5–7 years). S. pneumoniae were isolated and identified by standard culture-based methods, and serotyped using latex agglutination and the Quellung reaction. Carriage rates and densities of S. pneumoniae, H. influenzae, S. aureus and M. catarrhalis were determined using real-time quantitative PCR.
There were no differences in the rate or density of S. pneumoniae, H. influenzae or M. catarrhalis carriage by PCV7 dose or 23vPPV vaccination in the vaccinated participants overall. However, differences were observed between the two main ethnic groups: Fijian children of Indian descent (Indo-Fijian) were less likely to carry S. pneumoniae, H. influenzae and M. catarrhalis, and there was evidence of a higher carriage rate of S. aureus compared with indigenous Fijian (iTaukei) children. Polysaccharide vaccination appeared to have effects that varied between ethnic groups, with 23vPPV vaccination associated with a higher carriage rate of S. aureus in iTaukei children, while there was a lower carriage rate of S. pneumoniae associated with 23vPPV vaccination in Indo-Fijian children.
Overall, polysaccharide vaccination had no long-term impact on pneumococcal carriage, but may have impacted on S. aureus carriage and have varying effects in ethnic groups, suggesting current WHO vaccine schedule recommendations against the use of 23vPPV in children under two years of age are appropriate.
Keywords: Pneumococcal polysaccharide vaccine, Pneumococcal conjugate vaccine, Nasopharyngeal carriage, Streptococcus pneumoniae, Staphylococcus aureus, Ethnicity
1. Introduction
Streptococcus pneumoniae (the pneumococcus) is the most common cause of pneumonia, which is responsible for an estimated 1.3 million deaths annually in children under five years of age [1]. There are over 90 different serotypes of S. pneumoniae [2, 3]. Prior to the introduction of pneumococcal conjugate vaccines (PCVs), the vast majority of pneumococcal disease was caused by a limited number of serotypes [4]. Nasopharyngeal carriage of S. pneumoniae is considered a pre-requisite for the development of pneumococcal disease [5]. The introduction of PCVs has led to a dramatic reduction in invasive pneumococcal disease caused by vaccine-type S. pneumoniae [6–8]. However, in many settings, there has been little change in overall S. pneumoniae carriage due to replacement of vaccine type S. pneumoniae with non-vaccine types (serotype replacement) [9, 10]. The potential for species replacement following pneumococcal vaccination is also of concern, particularly given that some studies have found a negative relationship between vaccine-type S. pneumoniae and Staphylococcus aureus carriage [11, 12]. In some settings, pneumococcal vaccination has also affected colonization of the respiratory pathogens Moraxella catarrhalis and Haemophilus influenzae [13, 14].
The Fiji Pneumococcal Project (FiPP) was a single-blind, open-labelled randomized Phase II vaccine trial conducted in Suva, Fiji, designed to identify a pneumococcal vaccination schedule more suited to resource-poor countries. Specifically, FiPP evaluated a reduced dose 7-valent pneumococcal conjugate vaccine (PCV7) primary series in infancy, followed by the 23-valent pneumococcal polysaccharide vaccine (23vPPV) booster at 12 months of age [15]. One of the key findings was immune hyporesponsiveness in 23vPPV-vaccinated children compared with children not vaccinated with 23vPPV, observed following challenge with a micro-dose (20%) of 23vPPV at 17 months of age [16]. Nasopharyngeal carriage rates of S. pneumoniae were unaffected by 23vPPV vaccination at the same time-point [17].
FiPP also highlighted the differences in carriage rates between the two main ethnicities of Fiji, indigenous Fijians (iTaukei) and Fijians of Indian descent (Indo-Fijians) [18], consistent with an earlier carriage survey in the same setting [19]. In FiPP, carriage of S. pneumoniae, H. influenzae and M. catarrhalis was higher in iTaukei children than Indo-Fijian children, and higher M. catarrhalis densities were found in iTaukei children vaccinated with 23vPPV compared with iTaukei children who were not vaccinated with 23vPPV [18].
In the follow-up study to FiPP, a subset of children were enrolled to investigate the long-term impact of 23vPPV vaccine on carriage, immunity and clinical outcomes. Here we report the long-term effect of pneumococcal vaccination on nasopharyngeal carriage of S. pneumoniae and other respiratory pathogens, and examine differences in carriage between the two main ethnicities of Fijian children.
2. Materials and Methods
2.1 Study Design
The original FiPP study design is described elsewhere [15]. In brief, healthy Fijian infants (n=552) were randomized to receive either 0, 1, 2 or 3 doses of PCV7 (Prevnar®, Pfizer Inc., USA) at 6, 10 and/or 14 weeks, with or without a 12 month dose of 23vPPV (Pneumovax®, Merck & Co., Inc., USA). Children who were randomized to receive 0 or 1 dose of PCV7 were given a catch-up dose of this vaccine at 2 years of age. The long-term implications of the immune hyporesponsiveness observed at 18 months in children receiving 23vPPV were investigated in the Fiji follow-up study by enrolling all FiPP participants that were contactable (n=195), now aged 5–7 years old. One hundred and ninety five nasopharyngeal swabs were collected, of which 194 were available for microbiological analysis (Table 1). The characteristics of this subset, such as ethnicity, gender and number of children in each group, were similar to that of FiPP (see Supplemental Table S1 and Table 2 in Russell, et al. 2010 [17]), and were similar when considering each vaccine alone in this study (see Supplemental Table S2) - although children who received 3 doses of PCV7 were slightly older than children who received 2 doses of PCV7 (median age 6.21 years vs. 5.92 years, p=0.017). The geometric mean antibody levels in this study for 23vPPV vaccinated and 23vPPV unvaccinated children were similar to those observed in these children at 18 months of age in FiPP (data not shown). This study was approved by the Human Research Ethics Committee, Royal Children’s Hospital, Melbourne and the Fiji National Research Ethics Review Committee.
Table 1.
Group | Initial PCV7 doses |
Initial PCV7 dose timing (wks) |
23vPPV (12 mo) |
End of FiPP PCV7 dose* |
Total PCV7 doses |
NP swabs collected in this study (n) |
---|---|---|---|---|---|---|
A | 3 | 6, 10, 14 | − | 0 | 3 | 26 |
B | 3 | 6, 10, 14 | + | 0 | 3 | 28 |
C | 2 | 6 and 14 | − | 0 | 2 | 25 |
D | 2 | 6 and 14 | + | 0 | 2 | 30 |
E | 1 | 14 | − | 1 | 2 | 15 |
F | 1 | 14 | + | 1 | 2 | 25 |
G | 0 | None | + | 1 | 1 | 19 |
H | 0 | None | − | 1 | 1 | 26 |
PCV7 dose given at 2 years of age
Table 2.
Isolate | aroE | gdh | gki | recP | spi | xpt | ddl | ST | Associated serotypes^ |
---|---|---|---|---|---|---|---|---|---|
0534-01 | 8 | 37 | 9 | 29 | 2 | 12 | 14 | 1619 | 19A, NT |
0075-01 | 1 | 5 | 4 | 5 | 5 | 1 | 229 | 10183* | Closest match ST9 (14) |
0119-04 | 8 | 29 | 4 | 15 | 17 | 12 | 31 | 1106 | 14, NT |
0541-01 | 8 | 29 | 4 | 15 | 17 | 12 | 31 | 1106 | 14, NT |
0051-01 | 8 | 29 | 4 | 15 | 17 | 12 | 31 | 1106 | 14, NT |
0173-01 | 7 | 139 | 4 | 8 | 13 | 38 | 216 | 10184* | |
0586-02 | 8 | 37 | 9 | 29 | 2 | 12 | 53 | 344 | NT |
0586-01 | 8 | 37 | 9 | 29 | 2 | 12 | 31 | 10185* | Closest match ST344 (NT) |
0918-03 | 8 | 37 | 9 | 29 | 2 | 12 | 31 | 10185* | Closest match ST344 (NT) |
0598-03 | 8 | 37 | 9 | 29 | 2 | 12 | 31 | 10185* | Closest match ST344 (NT) |
0009-01 | 8 | 37 | 9 | 29 | 2 | 12 | 31 | 10185* | Closest match ST344 (NT) |
0500-01 | 213 | 5 | 223 | 1 | 17 | 1 | 14 | 10182* | Closest match ST3516 (19F) |
0082-03 | 16 | 9 | 193 | 226 | 6 | 20 | 5 | 10181* |
Newly identified in this study;
ST = sequence type;
Serotypes associated with ST or closest match for newly identified STs
2.2 Nasopharyngeal swabs
Buffered cotton nasopharyngeal swabs (Sarstedt, Australia) were collected and transported as described previously [17], in line with World Health Organization guidelines [20, 21]. Swabs were stored frozen in 1 ml of skim milk tryptone glucose glycerol (STGG) medium, and later transported on dry ice to the Pneumococcal Research laboratory at the Murdoch Childrens Research Institute in Melbourne, where they were stored at −80°C until processing.
2.3 Culture, identification and serotyping
Samples were cultured on Columbia horse blood agar plates containing 5 µg/ml of gentamicin (gHBA; Oxoid brand, Thermo Fisher Scientific, Australia) and incubated for 36–44 h (37°C, ~5% CO2) [22]. Two α-hemolytic colonies were randomly selected for subculture using a method previously described [22]. These colonies, plus any additional morphologically distinct α-hemolytic colonies, were subcultured onto Columbia horse blood agar plates (HBA; Oxoid brand, Thermo Fisher Scientific, Australia) and incubated for 24 h (37°C, ~5% CO2).
Colonies that were optochin sensitive were presumptively identified as S. pneumoniae. Other α-hemolytic colonies that were non-susceptible to optochin (intermediate or resistant) were tested for bile solubility and with the Phadebact® Pneumococcus test (Boule Diagnostics AB, Huddinge, Sweden) to enable identification.
Presumptive pneumococci were serotyped by latex agglutination using a combination of commercial reagents (Denka-Seiken Co., Ltd., Japan) and reagents produced in-house [23, 24] using Statens Serum Institute antisera (SSI, Copenhagen, Denmark). Equivocal reactions were confirmed using the Quellung reaction with antisera from SSI [25]. Isolates that were non-typeable were tested for the presence of the lytA gene by real-time PCR [26]. lytA-positive non-typeable isolates were further examined by multilocus sequencing typing (MLST) [27]. lytA-negative non-typeable isolates were excluded from further analysis. MLST allelic profiles were submitted to the Streptococcus pneumoniae MLST database (http://pubmlst.org/spneumoniae/) sited at the University of Oxford [28] where new sequence types (STs) were assigned for those isolates with allelic profiles not matching any existing ST. A serotype 14-specific PCR [29] was used to test non-typeable isolates that had MLST STs associated with serotype 14.
2.4 Quantitative PCR
Genomic DNA was extracted from 100 µl STGG using the QIAmp DNA Minikit (Qiagen) as previously described [18]. S. pneumoniae, S. aureus, M. catarrhalis and H. influenzae were examined by real-time quantitative PCR (qPCR) using two duplex assays [18], see Supplemental Table S4. Bacterial densities were determined using a standard curve from a dilution series prepared with genomic DNA from reference isolates [18].
2.5 Statistical Analysis
Logistic regression was used to examine the impact of PCV7 (1, 2 or 3 doses) and 23vPPV (0 or 1 dose) on carriage of S. pneumoniae, H. influenzae, S. aureus and M. catarrhalis, with each species examined in separate models. To determine if the effect of 23vPPV was dependent on the number of doses of PCV7 received, an interaction between 23vPPV and the number of doses of PCV7 was also considered in each model. Regression models were used to examine the effect of PCV7 and 23vPPV on density, with bacterial density data log transformed prior to analysis. To examine bacterial associations, logistic regression models were fitted to determine the odds of carrying one bacterial species given carriage of another species. For analyses of vaccine-type carriage culture data were used, while qPCR data were used for all other analyses. For ethnicity based analyses, due to the small number of individuals who were of ‘other’ ethnicity (n=6), only those of iTaukei (n=139) and Indo-Fijian (n=49) ethnicity were included. Where zero counts were observed, exact logistic regression was used to obtain the median unbiased estimate of the odds ratio [30]. Characteristics of study participants by ethnic group are shown in Supplemental Table S3. An adjusted analyses for potential confounders for comparisons between iTaukei and Indo-Fijian children was considered, however as these variables are considered intermediary steps in the relationship between ethnicity and carriage, and not true confounders, this analysis was not conducted [31].
Where analyses examine vaccine-type serotypes, only those serotypes included in each vaccine were considered (potentiallycross-reactive serotypes were considered non-vaccine type). As serotype 15B is present in 23vPPV, 15B and 15C were considered separately despite the potential of these serotypes to interconvert [32]. Analyses were conducted using the Stata 12 software (release 12; StataCorp, College Station, TX, USA).
3. Results
Of the 194 swabs tested, 85 (44%) were positive for S. pneumoniae, 99 (51%) for H. influenzae, 142 (73%) for M. catarrhalis and 53 (27%) for S. aureus by qPCR. Using culture, 88 (45%) swabs contained pneumococci, of which 13 (15%) contained PCV7 vaccine types, 37 (42%) contained 23vPPV types (including PCV7 types), and 51 (58%) only contained serotypes absent from either vaccine. Overall, 103 isolates were identified; non-typeable S. pneumoniae were most common (n=13, 12.6%), followed by serotypes 11A (n=10, 9.7%), 6A (n=8, 7.8%) and 16F (n=7, 6.8%) (Figure 1).
3.1 Non-typeable S. pneumoniae
All 19 α-hemolytic isolates that were presumptively identified as non-typeable S. pneumoniae were tested by lytA real-time PCR. Six were negative for the lytA gene and were designated as non-S. pneumoniae. The remaining 13 isolates were examined by MLST (Table 2) and represented eight STs, including five new STs. Four isolates had STs commonly associated with serotype 14; however, they were negative when tested using a serotype 14-specific PCR and thus designated non-typeable pneumococci.
3.2 Impact of pneumococcal vaccination on nasopharyngeal carriage rates and densities
None of the models considering an interaction term between 23vPPV and PCV7 doses showed significant interaction (each p>0.05; details not shown). Adjustment for PCV7 doses and 23vPPV/PCV7 dose interaction gave results that were similar to the results from unadjusted analysis of the effect of 23vPPV. As such unadjusted results are presented unless specified.
The number of PCV7 doses received had no statistically significant impact on pneumococcal carriage rate or density (each p>0.05). Similarly, there were no differences in the rate or density of pneumococcal carriage between 23vPPV recipients and non-recipients, regardless of adjustment for the number of PCV7 doses received (each p>0.05). There were no statistically significant differences in the carriage rates (Figure 2) or densities of vaccine type (PCV7 and 23vPPV) carriage by doses of either vaccine (each p>0.05).
We found no long-term impact of 23vPPV or PCV7 vaccination on H. influenzae carriage rates or densities. S. aureus carriage rates were higher in 23vPPV recipients (OR=2.41, 95% CI (1.24, 4.68); p=0.010) compared with children that did not receive 23vPPV, with 36 (35%) 23vPPV recipients carrying S. aureus compared with 17 (18%) children that did not receive 23vPPV. M. catarrhalis density was higher (p=0.036) in children that received 1 dose of PCV7 (median density 1.51 × 105 cfu/ml) compared with children who received 3 doses of PCV7 (median density 1.38 × 106 cfu/ml).
3.3 Bacterial associations
Carriage of S. pneumoniae was positively associated with carriage of H. influenzae and M. catarrhalis (Table 3). Positive associations were also observed between H. influenzae and M. catarrhalis. A negative association, where carriage of one species is less likely to occur given the presence of another species, was observed between S. aureus and M. catarrhalis, and between S. aureus and H. influenzae.
Table 3.
Overall | H. influenzae | M. catarrhalis | S. aureus | ||||
---|---|---|---|---|---|---|---|
n (%) | n (%) | OR (95% CI); p-value | n (%) | OR (95% CI); p-value | n (%) | OR (95% CI); p-value | |
S. pneumoniae (all serotypes) | 85 (44) | 65 (34) | 7.17 (3.76, 13.66); p<0.001* | 72 (37) | 3.09 (1.52, 6.27); p=0.002* | 19 (10) | 0.64 (0.33, 1.22); p=0.172 |
S. pneumoniae (PCV7 VT) | 13 (7) | 9 (5) | 2.28 (0.68, 7.65); p=0.184 | 10 (5) | 1.24 (0.33, 4.68); p=0.754 | 3 (2) | 0.79 (0.21, 2.97); p=0.723 |
S. pneumonia (23vPPV VT) | 37 (19) | 26 (13) | 2.72 (1.26, 5.88); p=0.011* | 33 (17) | 3.63 (1.22, 10.83); p=0.021* | 8 (4) | 0.69 (0.29, 1.62); p=0.389 |
H. influenzae | 99 (51) | 84 (43) | 3.57 (1.80, 7.10); p<0.001* | 20 (10) | 0.48 (0.25, 0.91); p=0.024* | ||
M. catarrhalis | 142 (73) | 29 (15) | 0.30 (0.15, 0.59); p=0.001* | ||||
S. aureus | 53 (27) |
Significant (p<0.05); VT= vaccine type; Percentage is out of the 194 participants
VT= vaccine type; Percentage is out of the 194 participants
3.4 Effect of ethnicity
Overall, 1 (2%) Indo-Fijian child and 35 (25%) iTaukei children were carrying 23vPPV vaccine types, with 1 (2%) Indo-Fijian child and 11 (8%) Indo-Fijian children carrying PCV7 vaccine types. Significant differences in overall carriage rates were found between the two main ethnicities. Indo-Fijian children had lower odds of carrying S. pneumoniae (OR=0.20, 95% CI (0.09, 0.44); p<0.001), H. influenzae (OR=0.18, 95% CI (0.08, 0.38); p<0.001) and M. catarrhalis (OR=0.12, 95% CI (0.06, 0.24); p<0.001), and higher odds of carrying S. aureus (OR=2.31, 95% CI (1.15, 4.61); p=0.018) compared with iTaukei children. Carriage densities between the two ethnic groups differed for M. catarrhalis (median density of 7.00 × 105 cfu/ml for iTaukei vs 9.49 × 104 cfu/ml for Indo-Fijians; p=0.008) but not for the other three bacterial species.
No Indo-Fijian 23vPPV recipients were found to carry S. pneumoniae, while 39% of Indo-Fijian children who did not receive 23vPPV were carriers (Table 4). There was no evidence of a difference in the odds of carrying S. pneumoniae among iTaukei who did and did not receive 23vPPV; however, the odds of carrying S. aureus were significantly higher in 23vPPV recipients compared with non-recipients (Table 4). There was no effect of 23vPPV vaccination on carriage of H. influenzae or M. catarrhalis (after adjustment for PCV7 doses received) for either ethnic group. There was no evidence that PCV7 had an effect on carriage for any of the bacterial species by ethnic group.
Table 4.
All | no 23vPPV recipients |
23vPPV recipients |
OR | 95% CI | p-value | ||
---|---|---|---|---|---|---|---|
iTaukei (n = 139+) | S. pneumoniae n (%) | 74 (53) | 35 (55) | 39 (52) | 0.90 | 0.46, 1.75 | 0.752 |
H. influenzae n (%) | 86 (62) | 41 (64) | 45 (60) | 0.84 | 0.42, 1.68 | 0.623 | |
M. catarrhalis n (%) | 119 (86) | 59 (92) | 60 (79) | 0.34 | 0.12, 0.99 | 0.048^ | |
S. aureus n (%) | 32 (23) | 6 (9) | 26 (35) | 5.13 | 1.95, 13.47 | 0.001* | |
Indo-Fijian (n = 49++) | S. pneumoniae n (%) | 9 (18) | 9 (39) | 0 (0) | 0.06† | 0.00, 0.45 | 0.003* |
H. influenzae n (%) | 11 (22) | 5 (22) | 6 (23) | 1.48 | 0.39, 5.71 | 0.567 | |
M. catarrhalis n (%) | 20 (41) | 13 (57) | 7 (27) | 0.44 | 0.14, 1.42 | 0.168 | |
S. aureus n (%) | 20 (41) | 11 (48) | 9 (35) | 0.88 | 0.28, 2.75 | 0.821 |
Significant (p<0.05);
Not significant after adjusting for PCV7 dose, with (p=0.310) or without (p=0.053) including interaction between PCV7 and 23vPPV;
Of the 139 iTaukei children, there were 75 with 23vPPV and 64 with no 23vPPV;
Of the 49 Indo-Fijian children, there were 26 with 23vPPV and 23 with no 23vPPV.
Exact logistic regression used to obtain the median unbiased estimate due to the zero count.
4. Discussion
Immune hyporesponsiveness following pneumococcal polysaccharide vaccination at 12 months of age was described previously for the children in this study [16]. The long-term implications of hyporesponsiveness on nasopharyngeal carriage are unknown, despite concerns that this may lead to increased pneumococcal disease susceptibility in these children. Currently the World Health Organization does not recommend 23vPPV to children under the age of two years based on immunological findings [33], however there are limited data on the effects of 23vPPV vaccination on nasopharyngeal carriage in children and no published studies looking at long-term effects. In this study, we found no long-term impact of 23vPPV on S. pneumoniae carriage. Early studies of other pneumococcal polysaccharide vaccines reported no effect of polysaccharide vaccination on nasopharyngeal carriage of S. pneumoniae [34–36]. More recently, two short-term studies, the original FiPP study [17] and another study in children aged 1–7 years with acute otitis media [37], also found that 23vPPV vaccination had no effect on carriage.
Of the 88 children in our study carrying pneumococci, 37 (42%) were carrying serotypes included in 23vPPV, a coverage rate which may suggest a justification for the use of 23vPPV in this population. However, 21 of the 41 (51%) children vaccinated with 23vPPV were carrying 23vPPV types compared with 16 of the 47 (34%) children not vaccinated with 23vPPV, suggesting that 23vPPV does not reduce carriage of 23vPPV serotypes in the long-term.
In contrast to the original FiPP study, which found a dose-dependent effect of PCV7 on pneumococcal carriage [17], we found no long-term differences in pneumococcal carriage in children who received 1, 2 or 3 PCV7 doses. However, due to the catch-up dose at the end of the previous study, there was no PCV7-unvaccinated group in this study to serve as a comparator. PCV7 vaccination appears to have eliminated carriage for three of the seven PCV7 serotypes (serotypes 4, 6B and 9V).
Consistent with previous studies, we found differences in carriage between the two main ethnic groups in Fiji [18, 19]. These differences were most striking when examining the effect of 23vPPV vaccination after stratifying by ethnicity, as the association between 23vPPV receipt and increased S. aureus carriage appears to only occur in iTaukei children. Additionally, there was evidence to suggest that 23vPPV may have reduced S. pneumoniae carriage in Indo-Fijians. In FiPP, there was no evidence that 23vPPV vaccination had varying effects on carriage rates in the two ethnic groups but iTaukei 23vPPV recipients had higher carriage densities of M. catarrhalis [18], findings not replicated in our study. However, the carriage rate of S. aureus was very low in FiPP (3.3%, typical for 17-month old children) compared with our study (27%). Additionally, analyses of carriage for FiPP were limited to the groups given 3 doses of PCV7 with 23vPPV, 3 doses of PCV7 without 23vPPV and the unvaccinated control group, whereas our study used data from all groups.
Ethnic differences in susceptibilities to infectious diseases such as tuberculosis, dengue fever and malaria [38–40], as well as the immune response to rubella vaccination [41], have been observed in other populations. There was no difference between iTaukei and Indo-Fijian children in this study for many pneumococcal carriage risk factors such as antibiotic use and exposure to cigarette smoke [42], although household income was higher for Indo-Fijian children (median weekly income ($US) $82.50 vs. $57.75, p=0.001). While socio-economic status is a risk factor for pneumococcal carriage [43], the differences in carriage between iTaukei and Indo-Fijian children in our study are consistent with previous studies in Fiji [18, 19], where household income levels were not different. It is therefore unlikely that the differences observed in the two ethnic groups in our study is attributable to economic status. The reasons underpinning the differences between the two main groups in Fiji are unknown but maybe due to unknown household and social factors, or genetic susceptibility.
Some, but not all, studies have reported increased S. aureus carriage following PCV7 vaccination [44, 45], consistent with observations of a negative relationship between carriage of S. aureus and vaccine-type S. pneumoniae [13, 14, 46]. In this study, there was no clear evidence that the association between 23vPPV receipt and increased S. aureus carriage in iTaukei children was due to bacterial interactions or species replacement given that there was not a corresponding reduction in vaccine-type S. pneumoniae carriage. Most of the bacterial interactions we found are consistent with other studies; such as the positive relationships between S. pneumoniae and M. catarrhalis, and S. pneumoniae and H. influenzae [43, 47, 48]. We found no association between S. aureus and S. pneumoniae, vaccine-type or otherwise. However, as all children were vaccinated with PCV7, PCV7 vaccine-type carriage was uncommon. While bacterial interactions or species replacement do not appear to have an effect at this time point, we have no carriage data between 17 months and 5–7 years of age, and it is possible that the changing carriage dynamics during this period may have played a role that is no longer apparent.
There are limited data on the impact of 23vPPV vaccination on S. aureus carriage and so the mechanism behind this observation is unknown. One study in children with a history of otitis media noted higher acute otitis media attributable to S. aureus in children vaccinated with PCV7 and 23vPPV compared with children that received no pneumococcal vaccination, however carriage of S. aureus was not examined [49]. The presence of pilus in S. pneumoniae [50] and, to a lesser degree, hydrogen peroxide-mediated killing of S. aureus by S. pneumoniae [51] in vitro, have both been implicated in the negative association between S. pneumoniae and S. aureus. However, it is clear the immune system also plays an important role as studies examining the effect of PCV in HIV-infected and HIV-uninfected children have shown no negative association between S. pneumoniae and S. aureus in the immunocompromised children [52, 53]. Additionally, prior S. pneumoniae colonization in a mouse model was found to inhibit S. aureus acquisition due to cross-reactive antibodies to a pneumococcal dehydrogenase [54].
The clinical implication of higher S. aureus carriage in iTaukei 23vPPV vaccinated individuals is also unclear, although our preliminary analyses show no change in hospital admissions between groups (Licciardi et al., unpublished results). However, it is possible this may affect diseases for which hospitalization would be unlikely, such as otitis media and skin infections. It should also be noted that although 23vPPV was associated with higher carriage of S. aureus in iTaukei children, the carriage rate of S. aureus in iTaukei 23vPPV recipients was similar to that of Indo-Fijian children who did not receive 23vPPV (35% vs. 48%, respectively). Immunologically, preliminary analyses show that there were no differences in antibody levels (geometric mean antibody concentrations) between 23vPPV vaccinated and 23vPPV unvaccinated children, and that the antibody levels seen in each ethnic group were similar, consistent with findings from the original FiPP study (Licciardi et al., unpublished results). Given the relatively small numbers in this study, larger studies would be required to fully explore some observations and relationships, such as differences in ethnicity and the impact of 23vPPV on S. aureus carriage.
We found a relatively large number of non-typeable pneumococci in this study and identified four new MLST STs. Other studies with a similar methodology have also found a relatively large proportion of non-typeables [55]. Most of our non136 typeable isolates had STs (or close matches to STs) commonly associated with non-typeables, such as ST344. However, three were ST1106 which is associated with serotype 14. Although previous studies have found non-typeable isolates genetically related to serotype 14 [56, 57], further investigation of our isolates using a serotype 14-specific PCR suggest that these isolates may lack part of the capsule locus or have capsule sequences that are sufficiently divergent that they are not detected by the serotype 14-specific PCR.
5. Conclusions
Although 23vPPV vaccination was associated with immune hyporesponsiveness in children at 17 months, it had no impact on S. pneumoniae carriage rates or densities at 17 months and no long-term impact on S. pneumoniae carriage rates or densities in children now aged 5–7 years. However, our data suggest that 23vPPV vaccination may be associated with higher S. aureus carriage, and may have varying effects in different ethnic groups. Despite observing no long-term impact on S. pneumoniae carriage rates or density, 23vPPV vaccination results in immune hyporesponsiveness, has potential long-term effects on S. aureus carriage, and there is a lack of evidence showing a clear benefit of using 23vPPV in children under two years of age. As such, current guidelines recommending against the use of 23vPPV in children under 2 years of age are appropriate.
Supplementary Material
Acknowledgements
We thank all the participants, families and staff involved in FiPP and this study. This work was funded by National Institutes of Health (NIH), USA (NIH Grant 1R01AI085198-01A1) and was supported by the Victorian Government’s Operational Infrastructure Support Program. Laura Boelsen is supported by the Fay Marles Scholarship (The University of Melbourne). Paul Licciardi and Fiona Russell are recipients of the Australian National Health and Medical Research Council (NHMRC) Early Career Fellowship, and Catherine Satzke is a recipient of the Australian NHMRC Career Development Fellowship. Karen Lamb was supported under a National Health and Medical Research Council Centre of Research Excellence grant (ID31035261) to the Victorian Centre for Biostatistics (ViCBiostat).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflicts of interest: none.
References
- 1.Walker CL, Rudan I, Liu L, Nair H, Theodoratou E, Bhutta ZA, et al. Global burden of childhood pneumonia and diarrhoea. Lancet. 2013;381:1405–1416. doi: 10.1016/S0140-6736(13)60222-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Henrichsen J. Six newly recognized types of Streptococcus pneumoniae. J Clin Microbiol. 1995;33:2759–2762. doi: 10.1128/jcm.33.10.2759-2762.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Park IH, Geno KA, Yu J, Oliver MB, Kim K-H, Nahm MH. Genetic, biochemical, and serological characterization of a new pneumococcal serotype, 6H, and generation of a pneumococcal strain producing three different capsular repeat units. Clin Vaccine Immunol. 2015;22:313–318. doi: 10.1128/CVI.00647-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Johnson HL, Deloria-Knoll M, Levine OS, Stoszek SK, Freimanis Hance L, Reithinger R, et al. Systematic evaluation of serotypes causing invasive pneumococcal disease among children under five: the pneumococcal global serotype project. PLoS Med. 2010;7:e1000348. doi: 10.1371/journal.pmed.1000348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Simell B, Auranen K, Kayhty H, Goldblatt D, Dagan R, O'Brien KL. The fundamental link between pneumococcal carriage and disease. Expert Rev Vaccines. 2012;11:841–855. doi: 10.1586/erv.12.53. [DOI] [PubMed] [Google Scholar]
- 6.Whitney CG, Farley MM, Hadler J, Harrison LH, Bennett NM, Lynfield R, et al. Decline in invasive pneumococcal disease after the introduction of protein–polysaccharide conjugate vaccine. N Engl J Med. 2003;348:1737–1746. doi: 10.1056/NEJMoa022823. [DOI] [PubMed] [Google Scholar]
- 7.Miller E, Andrews NJ, Waight PA, Slack MP, George RC. Herd immunity and serotype replacement 4 years after seven-valent pneumococcal conjugate vaccination in England and Wales: an observational cohort study. Lancet Infect Dis. 2011;11:760–768. doi: 10.1016/S1473-3099(11)70090-1. [DOI] [PubMed] [Google Scholar]
- 8.Williams SR, Mernagh PJ, Lee MH, Tan JT. Changing epidemiology of invasive pneumococcal disease in Australian children after introduction of a 7-valent pneumococcal conjugate vaccine. Med J Aust. 2011;194:116–120. doi: 10.5694/j.1326-5377.2011.tb04192.x. [DOI] [PubMed] [Google Scholar]
- 9.Weinberger DM, Malley R, Lipsitch M. Serotype replacement in disease after pneumococcal vaccination. Lancet. 2011;378:1962–1973. doi: 10.1016/S0140-6736(10)62225-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Vestrheim DF, Høiby EA, Aaberge IS, Caugant DA. Impact of a Pneumococcal Conjugate Vaccination Program on Carriage among Children in Norway. Clin Vaccine Immunol. 2010;17:325–334. doi: 10.1128/CVI.00435-09. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bogaert D, van Belkum A, Sluijter M, Luijendijk A, de Groot R, Rümke HC, et al. Colonisation by Streptococcus pneumoniae and Staphylococcus aureus in healthy children. Lancet. 2004;363:1871–1872. doi: 10.1016/S0140-6736(04)16357-5. [DOI] [PubMed] [Google Scholar]
- 12.Regev-Yochay G, Dagan R, Raz M, Carmeli Y, Shainberg B, Derazne E, et al. Association between carriage of Streptococcus pneumoniae and Staphylococcus aureus in children. JAMA. 2004;292:716–720. doi: 10.1001/jama.292.6.716. [DOI] [PubMed] [Google Scholar]
- 13.Dunne EM, Smith-Vaughan HC, Robins-Browne RM, Mulholland EK, Satzke C. Nasopharyngeal microbial interactions in the era of pneumococcal conjugate vaccination. Vaccine. 2013;31:2333–2342. doi: 10.1016/j.vaccine.2013.03.024. [DOI] [PubMed] [Google Scholar]
- 14.Shak JR, Vidal JE, Klugman KP. Influence of bacterial interactions on pneumococcal colonization of the nasopharynx. Trends in microbiology. 2013;21:129–135. doi: 10.1016/j.tim.2012.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Russell FM, Balloch A, Tang MLK, Carapetis JR, Licciardi P, Nelson J, et al. Immunogenicity following one, two, or three doses of the 7-valent pneumococcal conjugate vaccine. Vaccine. 2009;27:5685–5691. doi: 10.1016/j.vaccine.2009.06.098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Russell FM, Carapetis JR, Balloch A, Licciardi PV, Jenney AWJ, Tikoduadua L, et al. Hyporesponsiveness to re-challenge dose following pneumococcal polysaccharide vaccine at 12 months of age, a randomized controlled trial. Vaccine. 2010;28:3341–3349. doi: 10.1016/j.vaccine.2010.02.087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Russell FM, Carapetis JR, Satzke C, Tikoduadua L, Waqatakirewa L, Chandra R, et al. Pneumococcal nasopharyngeal carriage following reduced doses of a 7-valent pneumococcal conjugate vaccine and a 23-valent pneumococcal polysaccharide vaccine booster. Clin Vaccine Immunol. 2010;17:1970–1976. doi: 10.1128/CVI.00117-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Dunne EM, Manning J, Russell FM, Robins-Browne RM, Mulholland EK, Satzke C. Effect of pneumococcal vaccination on nasopharyngeal carriage of Streptococcus pneumoniae, Haemophilus influenzae, Moraxella catarrhalis, and Staphylococcus aureus in Fijian children. J Clin Microbiol. 2012;50:1034–1038. doi: 10.1128/JCM.06589-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Russell FM, Carapetis JR, Ketaiwai S, Kunabuli V, Taoi M, Biribo S, et al. Pneumococcal nasopharyngeal carriage and patterns of penicillin resistance in young children in Fiji. Ann Trop Paediatr Int Child Health. 2006;26:187–197. doi: 10.1179/146532806X120273. [DOI] [PubMed] [Google Scholar]
- 20.O'Brien KL, Nohynek H The WHO pneumococcal vaccine trials carriage working group. Report from a WHO Working Group: standard method for detecting upper respiratory carriage of Streptococcus pneumoniae. Pediatr Infect Dis J. 2003;22:e1–e11. doi: 10.1097/01.inf.0000049347.42983.77. [DOI] [PubMed] [Google Scholar]
- 21.Satzke C, Turner P, Virolainen-Julkunen A, Adrian PV, Antonio M, Hare KM, et al. Standard method for detecting upper respiratory carriage of Streptococcus pneumoniae: updated recommendations from the World Health Organization Pneumococcal Carriage Working Group. Vaccine. 2013;32:165–179. doi: 10.1016/j.vaccine.2013.08.062. [DOI] [PubMed] [Google Scholar]
- 22.Satzke C, Dunne EM, Porter BD, Antonio M, O'Brien KL, Robins-Browne RM, et al. PneuCarriage Project: identifying the optimum pneumococcal serotyping method(s), including detection of multiple serotype carriage. Ninth International Symposium on Pneumococci and Pneumococcal Disease (ISPPD9); Hyderabad, India. 2014. [Google Scholar]
- 23.Ortika BD, Habib M, Dunne EM, Porter BD, Satzke C. Production of latex agglutination reagents for pneumococcal serotyping. BMC Res Notes. 2013;6:49. doi: 10.1186/1756-0500-6-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Porter BD, Ortika BD, Satzke C. Capsular serotyping of Streptococcus pneumoniae by latex agglutination. J Vis Exp. 2014:e51747. doi: 10.3791/51747. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Habib M, Porter BD, Satzke C. Capsular serotyping of Streptococcus pneumoniae using the Quellung reaction. J Vis Exp. 2014:e51208. doi: 10.3791/51208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Carvalho MdGS, Tondella ML, McCaustland K, Weidlich L, McGee L, Mayer LW, et al. Evaluation and improvement of real-time PCR assays targeting lytA, ply, and psaA genes for detection of pneumococcal DNA. J Clin Microbiol. 2007;45:2460–2466. doi: 10.1128/JCM.02498-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Dunne EM, Ong EK, Moser RJ, Siba PM, Phuanukoonnon S, Greenhill AR, et al. Multilocus sequence typing of Streptococcus pneumoniae by use of mass spectrometry. J Clin Microbiol. 2011;49:3756–3760. doi: 10.1128/JCM.05113-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Jolley K, Maiden M. BIGSdb: Scalable analysis of bacterial genome variation at the population level. BMC Bioinformatics. 2010;11:595. doi: 10.1186/1471-2105-11-595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Dias CA, Teixeira LM, Carvalho MdGS, Beall B. Sequential multiplex PCR for determining capsular serotypes of pneumococci recovered from Brazilian children. J Med Microbiol. 2007;56:1185–1188. doi: 10.1099/jmm.0.47347-0. [DOI] [PubMed] [Google Scholar]
- 30.Mehta CR, Patel NR. Exact logistic regression: theory and examples. Stat Med. 1995;14:2143–2160. doi: 10.1002/sim.4780141908. [DOI] [PubMed] [Google Scholar]
- 31.Schisterman EF, Cole SR, Platt RW. Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology. 2009;20:488–495. doi: 10.1097/EDE.0b013e3181a819a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.van Selm S, van Cann LM, Kolkman MAB, van der Zeijst BAM, van Putten JPM. Genetic basis for the structural difference between Streptococcus pneumoniae serotype 15B and 15C capsular polysaccharides. Infect Immun. 2003;71:6192–6198. doi: 10.1128/IAI.71.11.6192-6198.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.World Health Organization. 23-valent pneumococcal polysaccharide vaccine. WHO position paper. Wkly Epidemiol Rec. 2008;83:373–384. [PubMed] [Google Scholar]
- 34.Douglas RM, Hansman D, Miles HB, Paton JC. Pneumococcal carriage and type-specific antibody: failure of a 14-valent vaccine to reduce carriage in healthy children. Am J Dis Child. 1986;140:1183–1185. doi: 10.1001/archpedi.1986.02140250109044. [DOI] [PubMed] [Google Scholar]
- 35.Rosén C, Christensen P, Hovelius B, Prellner K. A longitudinal study of the nasopharyngeal carriage of pneumococci as related to pneumococcal vaccination in children attending day-care centres. Acta Otolaryngol. 1984;98:524–532. doi: 10.3109/00016488409107593. [DOI] [PubMed] [Google Scholar]
- 36.Wright PF, Sell SH, Vaughn WK, Andrews C, McConnell KB, Schiffman G. Clinical studies of pneumococcal vaccines in infants. II. Efficacy and effect on nasopharyngeal carriage. Rev Infect Dis. 1981;3:S108–S112. doi: 10.1093/clinids/3.supplement_1.s108. [DOI] [PubMed] [Google Scholar]
- 37.Veenhoven RH, Bogaert D, Schilder AGM, Rijkers GT, Uiterwaal CSPM, Kiezebrink HH, et al. Nasopharyngeal pneumococcal carriage after combined pneumococcal conjugate and polysaccharide vaccination in children with a history of recurrent acute otitis media. Clin Infect Dis. 2004;39:911–919. doi: 10.1086/422651. [DOI] [PubMed] [Google Scholar]
- 38.Modiano D, Petrarca V, Sirima BS, Nebié I, Diallo D, Esposito F, et al. Different response to Plasmodium falciparum malaria in West African sympatric ethnic groups. Proc Natl Acad Sci. 1996;93:13206–13211. doi: 10.1073/pnas.93.23.13206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Sierra BdlC, Kourí G, Guzmán MG. Race: a risk factor for dengue hemorrhagic fever. Arch Virol. 2007;152:533–542. doi: 10.1007/s00705-006-0869-x. [DOI] [PubMed] [Google Scholar]
- 40.Stead WW, Senner JW, Reddick WT, Lofgren JP. Racial differences in susceptibility to infection by Mycobacterium tuberculosis. N Engl J Med. 1990;322:422–427. doi: 10.1056/NEJM199002153220702. [DOI] [PubMed] [Google Scholar]
- 41.Haralambieva IH, Salk HM, Lambert ND, Ovsyannikova IG, Kennedy RB, Warner ND, et al. Associations between race, sex and immune response variations to rubella vaccination in two independent cohorts. Vaccine. 2014;32:1946–1953. doi: 10.1016/j.vaccine.2014.01.090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Ghaffar F, Friedland IR, McCracken GH., Jr Dynamics of nasopharyngeal colonization by Streptococcus pneumoniae. The Pediatric infectious disease journal. 1999;18:638–646. doi: 10.1097/00006454-199907000-00016. [DOI] [PubMed] [Google Scholar]
- 43.Jourdain S, Smeesters PR, Denis O, Dramaix M, Sputael V, Malaviolle X, et al. Differences in nasopharyngeal bacterial carriage in preschool children from different socio-economic origins. Clin Microbiol Infect. 2011;17:907–914. doi: 10.1111/j.1469-0691.2010.03410.x. [DOI] [PubMed] [Google Scholar]
- 44.Spijkerman J, Prevaes SM, van Gils EJ, Veenhoven RH, Bruin JP, Bogaert D, et al. Long-term effects of pneumococcal conjugate vaccine on nasopharyngeal carriage of S. pneumonia, S. aureus, H. influenzae and M. catarrhalis. PLoS One. 2012;7:e39730. doi: 10.1371/journal.pone.0039730. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.van Gils EJ, Hak E, Veenhoven RH, Rodenburg GD, Bogaert D, Bruin JP, et al. Effect of seven-valent pneumococcal conjugate vaccine on Staphylococcus aureus colonisation in a randomised controlled trial. PLoS One. 2011;6:e20229. doi: 10.1371/journal.pone.0020229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Lijek RS, Weiser JN. Co-infection subverts mucosal immunity in the upper respiratory tract. Curr Opin Immunol. 2012;24:417–423. doi: 10.1016/j.coi.2012.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Xu Q, Almudervar A, Casey JR, Pichichero ME. Nasopharyngeal bacterial interactions in children. Emerg Infect Dis. 2012;18:1738–1745. doi: 10.3201/eid1811.111904. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Bae S, Yu J-Y, Lee K, Lee S, Park B, Kang Y. Nasal colonization by four potential respiratory bacteria in healthy children attending kindergarten or elementary school in Seoul, Korea. J Med Microbiol. 2012;61:678–685. doi: 10.1099/jmm.0.040584-0. [DOI] [PubMed] [Google Scholar]
- 49.Veenhoven R, Bogaert D, Uiterwaal C, Brouwer C, Kiezebrink H, Bruin J, et al. Effect of conjugate pneumococcal vaccine followed by polysaccharide pneumococcal vaccine on recurrent acute otitis media: a randomised study. Lancet. 2003;361:2189–2195. doi: 10.1016/S0140-6736(03)13772-5. [DOI] [PubMed] [Google Scholar]
- 50.Regev-Yochay G, Lipsitch M, Basset A, Rubinstein E, Dagan R, Raz M, et al. The pneumococcal pilus predicts the absence of Staphylococcus aureus co-colonization in pneumococcal carriers. Clin Infect Dis. 2009;48:760–763. doi: 10.1086/597040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Regev-Yochay G, Trzciński K, Thompson CM, Malley R, Lipsitch M. Interference between Streptococcus pneumoniae and Staphylococcus aureus: in vitro hydrogen peroxide-mediated killing by Streptococcus pneumoniae. J Bacteriol. 2006;188:4996–5001. doi: 10.1128/JB.00317-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Madhi SA, Adrian P, Kuwanda L, Cutland C, Albrich WC, Klugman KP. Long-term effect of pneumococcal conjugate vaccine on nasopharyngeal colonization by Streptococcus pneumoniae—and associated interactions with Staphylococcus aureus and Haemophilus influenzae colonization—in HIV-infected and HIV-uninfected children. J Infect Dis. 2007;196:1662–1666. doi: 10.1086/522164. [DOI] [PubMed] [Google Scholar]
- 53.McNally LM, Jeena PM, Gajee K, Sturm AW, Tomkins AM, Coovadia HM, et al. Lack of association between the nasopharyngeal carriage of Streptococcus pneumoniae and Staphylococcus aureus in HIV-1–infected South African children. J Infect Dis. 2006;194:385–390. doi: 10.1086/505076. [DOI] [PubMed] [Google Scholar]
- 54.Lijek RS, Luque SL, Liu Q, Parker D, Bae T, Weiser JN. Protection from the acquisition of Staphylococcus aureus nasal carriage by cross-reactive antibody to a pneumococcal dehydrogenase. Proc Natl Acad Sci. 2012;109:13823–13828. doi: 10.1073/pnas.1208075109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Turner P, Turner C, Jankhot A, Helen N, Lee SJ, Day NP, et al. A longitudinal study of Streptococcus pneumoniae carriage in a cohort of infants and their mothers on the Thailand-Myanmar border. PLoS One. 2012;7:e38271. doi: 10.1371/journal.pone.0038271. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Chewapreecha C, Harris SR. Dense genomic sampling identifies highways of pneumococcal recombination. Nat Genet. 2014;46:305–309. doi: 10.1038/ng.2895. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Andrade AL, Franco CM, Lamaro-Cardoso J, Andre MC, Oliveira LL, Kipnis A, et al. Non-typeable Streptococcus pneumoniae carriage isolates genetically similar to invasive and carriage isolates expressing capsular type 14 in Brazilian infants. J Infect. 2010;61:314–322. doi: 10.1016/j.jinf.2010.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.