Abstract
Objectives
Hospital-based studies of community-onset bloodstream infections (CO-BSI) are less resource-intensive to carry out than population-based incidence studies. We examined several metrics capturing the potential role of Salmonella Typhi as a cause of CO-BSI for making inferences about incidence.
Methods
We systematically reviewed three databases for hospital-based studies of CO-BSI. We determined, by study, the prevalence and rank order of Salmonella among pathogenic bloodstream isolates, and the prevalence ratio of Salmonella Typhi to Escherichia coli (S:E ratio). We then describe these hospital-based study metrics in relation to population-based typhoid fever incidence data from a separate systematic review.
Results
Forty-four studies met the inclusion criteria, of which 23 (52.3%) isolated Salmonella Typhi at least once. Among studies isolating Salmonella Typhi, the median (interquartile range) prevalence and rank order of Salmonella Typhi compared to other pathogens isolated in BSI was 8.3% (3.2–37.9%) and 3 (1–6), respectively. The median (interquartile range) S:E ratio was 1.0 (0.4–3.0). With respect to incidence, in Pemba Island, Tanzania, prevalence, rank order, S:E ratio, and incidence was 64.8%, 1, 9.2 and 110 cases per 100 000, respectively, and in Boulkiemdé, Burkina Faso, was 13.3%, 3, 2.3 and 249 cases per 100 000.
Conclusions
We describe considerable variation in place and time for Salmonella Typhi prevalence, rank order, and S:E ratio among hospital-based studies of CO-BSI. Data from simultaneous typhoid prevalence and incidence studies are limited. We propose that hospital-based study metrics warrant evaluation for making inference about typhoid incidence and as covariates in typhoid incidence models.
Keywords: Salmonella Typhi, typhoid fever, prevalence, modelling
Introduction
Typhoid fever is a serious systemic infection caused by the organism Salmonella enterica subspecies enterica serovar Typhi (Salmonella Typhi). Salmonella Typhi is transmitted predominantly through fecally contaminated food and water [1]. Typhoid fever is an important source of morbidity and mortality globally. It is estimated to cause more than 10 million illnesses and 116 000 deaths [2,3] worldwide, with most illnesses in low-resource areas in Asia and sub-Saharan Africa [4–6]. With the recent prequalification of typhoid conjugate vaccines [7], countries are faced with making decisions about vaccine introduction based on incidence data that are often either scarce, of insufficient quality, or that offer an incomplete picture. Such decisions are complicated by the substantial variation in typhoid incidence not just between regions, but between countries within the same region, and within the same country [8–10].
The reference standard method for estimating the incidence of typhoid fever is prospective, population-based active surveillance in a large cohort, but such studies are costly and time-consuming to implement. Prospective, passive sentinel site surveillance study designs that use healthcare utilisation surveys, called ‘multiplier studies’ [11,12] or ‘hybrid surveillance’ [13,14], make adjustments for under-ascertainment to estimate incidence rates from sentinel site data. These studies are less resource-intensive but yield results that may be susceptible to selection and recall bias, compromise precision, and the type of multipliers implemented are not standardised [8].
Statistical models using historical disease patterns and covariates on the causal pathway of transmission (e.g. water supply, sanitation, and chronic carriers) are an additional avenue for estimating typhoid incidence [15–19]. Surrogates of economic development such as infrastructure (e.g. proportion of road paved), access to improved water and sanitation, prevalence of stunting, and percent of the population living in extreme poverty have also been explored for use in models [5], along with seasonal and environmental factors that may influence typhoid transmission dynamics [20–25].
Covariates for incidence that might be directly related to disease occurrence include metrics from hospital-based studies of community-onset bloodstream infections (CO-BSI), which take into account the prevalence of Salmonella Typhi versus that of other BSI, and the rank order of Salmonella Typhi among BSIs. To assess the influence of study design and temporal changes, it could be useful to compare the prevalence of Salmonella Typhi to the prevalence of non-Salmonella organisms, a strategy that has been implemented in epidemiologic studies of pneumococcal disease [26]. We performed an analysis of a systematic review of the prevalence of CO-BSI among hospitalised febrile inpatients with the objective to describe the three hospital-based metrics of Salmonella Typhi, to compare them to high-quality primary incidence data from the literature, and to create a resource for future modelling efforts.
Methods
Search strategy and selection criteria
The protocol for the systematic review on the prevalence of CO-BSI among febrile hospitalised patients has been published [27] and was registered on PROSPERO on 28 September 2018 (CRD42018109388; Appendix S1). In brief, on 19 September 2018, we searched PubMed, Web of Science and Scopus to identify studies of CO-BSI with no restriction on language, country or date. Keywords used were fever, bacteremia, septicemia, epidemiology, incidence and prevalence, as well as spelling alternatives and related terms. Prospective studies with consecutive series of hospitalised febrile patients using aerobic blood culture as the reference standard diagnostic test were included.
Two authors screened the titles and abstracts for inclusion. Full-text articles and data abstraction of included articles were independently screened in parallel by two authors with discrepancies resolved by discussion or a separate, third author. Quantitative data abstracted were the number of participants in the study, number of hospitalised participants with BSI, and number and type of each pathogenic organism causing BSI. We subsequently stratified the data of included articles by whether Salmonella Typhi was isolated at least once and performed a sub-analysis on these studies. Recognising the importance of negative studies, we also describe the studies from which Salmonella Typhi was not isolated.
Data analysis
Among studies that isolated Salmonella Typhi, the number of unique pathogen types was counted by totalling the number of different isolates by species and serovar. Groups of organisms that were not typed or differentiated by the original study, such as ‘non-typhoidal Salmonella’ or ‘Streptococcus species,’ were counted as a single type. For example, if a study reported isolating 100 Salmonella Typhi, 100 Escherichia coli, and 100 ‘non-typhoidal Salmonella,’ we reported three pathogen types in that study.
We then calculated the prevalence of Salmonella Typhi and E. coli among pathogens causing BSI and ranked the organisms by proportion of pathogens isolated, where the most frequently isolated organism was ranked first. In studies that also isolated E. coli, we compared it to Salmonella Typhi by calculating the ratio of Salmonella Typhi prevalence to E. coli prevalence (S:E ratio). We chose E. coli as a comparator organism because of its high prevalence as a cause of bloodstream infection [28,29] and the lack of a vaccine in programmatic use for extraintestinal pathogenic E. coli. We did not examine Staphylococcus aureus because of the expected low prevalence in our dataset and we ruled out Streptococcus pneumoniae and Haemophilus influenzae B due to the widespread but incomplete introduction of vaccines for these pathogens. Because not all studies included mycobacterial blood culture in addition to standard aerobic blood culture, mycobacterial isolates were excluded when calculating prevalence, rank, and S:E ratio. Organisms not identified but still attributed as a cause of BSI were also excluded. We reported each hospital-based metric by individual study and the median and interquartile range (IQR 25–75%) of the metrics stratified by United Nations (UN) sub-region [30]. We also documented the UN sub-region and country of negative studies to describe the locations that did not isolate Salmonella Typhi and compared the locations on regional maps to studies that isolated Salmonella Typhi.
Our previous systematic review reported studies that used population-based surveillance to estimate typhoid fever incidence by location [8]. Due to the observed heterogeneity of typhoid incidence [8–10], we were confident only in comparing hospital-based metrics to typhoid incidence if the studies overlapped by both place and time. Analysis for trends, correlations, and associations were planned but could not be completed due to the lack of overlapping data. We instead describe the number and location of studies from both reviews that overlapped in place and provide a descriptive summary of those that overlap by place and time.
Results
In our systematic review of the prevalence of CO-BSI among febrile hospitalised patients, we screened 7886 titles and abstracts, of which 7634 were excluded [27]. We then screened the full text of 252 articles, resulting in 44 studies that were included. Among the 44 included studies, 23 (52.3%) studies isolated Salmonella Typhi at least once and were eligible for sub-analysis [31–53] (Figure 1).
Study characteristics
The 23 studies that isolated Salmonella Typhi collected data between 1984 and 2014 in 13 countries in Africa (7) and Asia (6) (Table 1). By UN sub-region, 14 (60.9%) studies were done in Eastern Africa, three (13.0%) in South-eastern Asia, two (8.7%) in Southern Asia, and the remaining four (17.4%) in Middle Africa, Northern Africa, Western Africa, and Eastern Asia. There were 23 526 hospitalised febrile participants, of whom 2385 (10.1%) had BSI; the median (IQR) prevalence of BSI was 13.1 (7.0–17.6%).
Table 1.
UN subregion | Locality, Country [ref] | Inclusion Age | Data collection year(s) | Number of febrile participants | BSI (% of febrile participants) | Count of pathogen types | Three most frequently isolated pathogens (number isolated) |
---|---|---|---|---|---|---|---|
Eastern Africa | Mumias, Kenya [46] | >5 y | 1994 | 229 | 51 (22.3) | 9 |
|
Nairobi, Kenya [36] | 3 m–12 y | 2001 | 264 | 32 (12.1) | 10 |
|
|
Blantyre, Malawi [39] | Children (no age provided) | 1996–1997 | 2123 | 365 (17.2) | 17 |
|
|
Blantyre, Malawi [35] | Adults (no age provided) | 1997–1998 | 2789 | 449 (16.1) | 17 |
|
|
Blantyre, Malawi [31] | ≥14 y | 2000 | 352 | 69 (19.6) | 13 |
|
|
Lilongwe, Malawi [48] | ≥14 y | 1998 | 238 | 54 (22.7) | 13 |
|
|
Dar es Salaam, Tanzania [43] | ≥15 y | 1995 | 517 | 84 (16.2) | 20 |
|
|
Dar es Salaam, Tanzania [34] | 0–7 y | 2001–2002 | 1787 | 127 (7.1) | 23 |
|
|
Moshi, Tanzania [45] | ≥13 y | 2007–2008 | 403 | 54 (13.4) | 12 |
|
|
Moshi, Tanzania [44] | 2 m–<13 y | 2007–2008 | 467 | 16 (3.4) | 5 | 1
|
|
Muheza, Tanzania [52] | 2 m–13 y | 2006–2007 | 3639 | 341 (9.4) | 8 |
|
|
Muheza, Tanzania [49] | ≥13 y | 2007 | 198 | 26 (13.1) | 9 |
|
|
Pemba Island, Tanzania [50] | >2 m | 2009–2010 | 2209 | 79 (3.6) | 5 |
|
|
Jinja, Uganda [37] | 6 m– <60 m | 2012 | 250 | 45 (18.0) | 10 |
|
|
Middle Africa | Bangui, Central African Republic [38] | All ages | 1999 | 131 | 35 (26.7) | 8 |
|
Northern Africa | Port Sudan, Sudan [42] | ≥12 y | 1984 | 100 | 22 (22.0) | 3 |
|
Western Africa | Boulkiemde, Burkina Faso [47] | 2 m–15 y | 2013–2014 | 1339 | 118 (8.8) | 13 |
|
Eastern Asia | Taipei, Taiwan [53] | ≤15 y | NR | 300 | 6 (2.0) | 5 |
|
Southeastern | Jayapura, Northeastern Asia Papua, Indonesia [41] | All ages | 1997–2000 | 226 | 34 (15.0) | 6 |
|
Siem Reap, Cambodia [32] | <16 y | 2009–2010 | 1225 | 76 (6.2) | 13 |
|
|
Multiple, Thailand [40] | >2 y | 1991–1993 | 1137 | 36 (3.2) | 13 |
|
|
Southern Asia | Multiple, India [33] | ≥5 y | 2011–2012 | 1564 | 124 (7.9) | 16 |
|
Kathmandu, Nepal [51] | ≤12 y | 2005–2006 | 2039 | 142 (7.0) | 19 |
|
Ref, reference; (t), tied; NR, not reported; BSI, bloodstream infection; NTS, non-typhoidal Salmonella; y, years; m, months.
From participants with BSI, 2413 pathogenic organisms were isolated. The median (IQR) count of pathogen types per study was 12 (8–15). Salmonella Typhi was the most frequently isolated organism in nine (39.1%) of the studies, followed by Salmonella serovar Typhimurium in six (26.1%), and E. coli in three (13.0%) studies (Figure 2).
Hospital-based metrics
Of 2413 pathogens isolated, 317 (13.1%) were Salmonella Typhi. Overall median (IQR) prevalence of Salmonella Typhi among pathogens causing BSI was 8.3% (3.2–37.9%); 5.7% (2.7–37.5%) in Africa and 32.7% (19.7–37.8%) in Asia (Table 2). In the UN sub-regions Eastern Africa, South-eastern Asia, and Southern Asia, the median (IQR) prevalence of Salmonella Typhi among pathogens was 3.7% (2.3–30.1%), 28.9% (18.6–33.6%), and 37.8% (37.2–38.3), respectively. Overall median (IQR) rank of Salmonella Typhi among pathogens causing BSI was 3 (1–6) and was 6 (2–7), 1 (1–3), and 1 (1–1) in Eastern Africa, South-eastern Asia, and Southern Asia, respectively.
Table 2.
Locality, Country (last obs year) [ref] | Number of pathogens isolated causing BSI | Proportion of isolates that were Salmonella Typhi (%) | Salmonella Typhi rank | Proportion of isolates that were E. coli (%) | E. coli rank | Salmonella Typhi: E. coli Ratio |
---|---|---|---|---|---|---|
Eastern Africa | ||||||
Pemba Island, Tanzania (2010) [50] | 71 | 64.8 | 1 | 7.0 | 3 | 9.2 |
Mumias, Kenya (1994) [46] | 52 | 46.2 | 1 | 3.8 | 6 | 12.0 |
Moshi, Tanzania (2008) [45] | 58 | 44.8 | 1 | 12.1 | 2 | 3.7 |
Moshi, Tanzania (2008) [44] | 16 | 37.5 | 1 | 18.8 | 3 | 2.0 |
Muheza, Tanzania (2007) [49] | 25 | 8.0 | 5 | 16.0 | 3 | 0.5 |
Lilongwe, Malawi (1998) [48] | 49 | 6.1 | 5 | 6.1 | 5 | 1.0 |
Blantyre, Malawi (1997) [39] | 365 | 4.1 | 6 | 0.0 | NR | * |
Muheza, Tanzania (2007) [52] | 341 | 3.2 | 7 | 6.7 | 5 | 0.5 |
Nairobi, Kenya (2001) [36] | 32 | 3.1 | 6 | 3.1 | 6 | 1.0 |
Blantyre, Malawi (1998) [35] | 450 | 2.7 | 8 | 9.6 | 3 | 0.3 |
Jinja, Uganda (2012) [37] | 45 | 2.2 | 5 | 0.0 | NR | * |
Blantyre, Malawi (2000) [31] | 75 | 1.3 | 7 | 5.3 | 4 | 0.3 |
Dar es Salaam, Tanzania (1995) [43] | 92 | 1.1 | 10 | 13.0 | 3 | 0.1 |
Dar es Salaam, Tanzania (2002) [34] | 155 | 0.6 | 16 | 15.5 | 1 | 0.0 |
Eastern Africa median (IQR) | 64.5 (46.0–139.3) | 3.7 (2.3–30.1) | 6 (2–7) | 6.9 (4.2–12.8) | 3 (3–5) | 0.8 (0.3–2.4) |
Middle Africa | ||||||
Bangui, Central African Republic (1999) [38] | 35 | 5.7 | 3 | 5.7 | 3 | 1.0 |
Northern Africa | ||||||
Port Sudan, Sudan (1984) [42] | 22 | 59.1 | 1 | 0.0 | NR | * |
Western Africa | ||||||
Boulkiemde, Burkina Faso (2014) [47] | 120 | 13.3 | 3 | 5.8 | 5 | 2.3 |
Africa median (IQR) | 58.0 (35.0–120.0) | 5.7 (2.7–37.5) | 5 (1–7) | 6.1 (3.8–12.1) | 3 (3–5) | 1.0 (0.4–2.2) |
Eastern Asia | ||||||
Taipei, Taiwan [53] | 6 | 16.7 | 2 | 33.3 | 1 | 0.5 |
South-eastern Asia | ||||||
Siem Reap, Cambodia (2010) [32] | 76 | 28.9 | 1 | 10.5 | 3 | 2.8 |
Jayapura, Northeastern Papua, Indonesia (2000) [41] | 34 | 38.2 | 1 | 23.5 | 2 | 1.6 |
Multiple, Thailand (1993) [40] | 36 | 8.3 | 4 | 36.1 | 1 | 0.2 |
South-eastern Asia median (IQR) | 36.0 (35.0–56.0) | 28.9 (18.6–33.6) | 1 (1–3) | 23.5 (17.0–29.8) | 2 (2–3) | 1.6 (0.9–2.2) |
Southern Asia | ||||||
Kathmandu, Nepal (2006) [51] | 145 | 36.6 | 1 | 2.8 | 10 | 13.3 |
Multiple, India (2012) [33] | 113 | 38.9 | 1 | 9.7 | 3 | 4.0 |
Southern Asia median (IQR) | 129.0 (121.0–137.0) | 37.8 (37.2–38.3) | 1 (1–1) | 6.3 (4.5–8.0) | 7 (5–8) | 8.7 (6.3–11.0) |
Asia median (IQR) | 56.0 (34.5–103.8) | 32.8 (19.8–37.8) | 1 (1–2) | 17.0 (9.9–30.9) | 3 (1–3) | 2.2 (0.8–3.7) |
Overall median (IQR) | 58.0 (34.5–116.5) | 8.2 (3.2–37.9) | 3 (1–6) | 7.0 (4.6–14.3) | 3 (3–5) | 1.0 (0.4–3.0) |
Ref, reference; BSI, bloodstream infection; NR, not reported; IQR, interquartile range.
Unable to calculate because demoninator for S:E ratio is zero.
E. coli accounted for 186 (7.7%) of 2413 pathogens isolated. Overall median (IQR) prevalence of E. coli among pathogens causing BSI was 7.0% (4.6–14.3%); 6.1% (3.8–12.1%) in Africa and 17.0% (9.9–30.9%) in Asia. In the UN sub-regions Eastern Africa, South-eastern Asia, and Southern Asia, the median (IQR) prevalence was 6.9% (4.2–12.8%), 23.5% (17.0–29.8%) and 6.3% (4.5–8.0%), respectively. Overall the median (IQR) rank of E. coli among pathogens causing BSI was 3 (3–5) in Eastern Africa, 2 (2–3) in South-eastern Asia and 7 (5–8) in Southern Asia.
The overall median (IQR) S:E ratio was 1.0 (0.5–3.0). Among studies done in Africa, the median (IQR) S:E ratio was 1.0 (0.4–2.2); in Asia it was 2.2 (0.8–3.7). The highest S:E ratio was 13.3 in a study in Kathmandu, Nepal, in 2006, where Salmonella Typhi accounted for 53 (36.6%) and E. coli for 4 (2.8%) of 145 pathogens isolated [51]. In contrast, the lowest S:E ratio was <0.1 in a study in Dar es Salaam, Tanzania in 2002, where Salmonella Typhi accounted for 1 (0.6%) and E. coli for 24 (15.5%) of 155 pathogens isolated [34]. Three studies did not isolate E. coli, precluding calculation of a S:E ratio [37,39,42].
Our systematic review yielded 21 (47.7%) studies that did not isolate Salmonella Typhi, of which seven were done in Africa [54–60] and four in Asia [61–64] (Table 3). The seven studies in Africa were conducted in Kenya, Mozambique, Nigeria, Tanzania, and Uganda. Among these five countries, we identified studies at other locations and times in Kenya, Tanzania, and Uganda that did report isolating Salmonella Typhi [34,36,37,43–46,49,50,52]. Five studies in Africa [55,57–60] and one in Asia [61] reported isolating Salmonella species but did not specify the species or serovar. Among the 11 studies in Africa and Asia not reporting isolation of Salmonella Typhi, all but two reported E. coli [54,57]. The median (IQR) prevalence and rank of E. coli among these studies were 15.0% (4.6–34.6%) and 2 (1–3), respectively.
Table 3.
UN subregion | Locality, Country [ref] | Data collection year(s) | Number of febrile participants | BSI (% of febrile participants) | Count of pathogen types | Three most frequently isolated pathogens (number isolated) |
---|---|---|---|---|---|---|
Eastern Africa | Mwanza, Tanzania [59] | 2011–2012 | 317 | 21 (6.6) | 8 |
|
Nyanza region, Kenya [54] | 2013–2014 | 148 | 5 (3.4) | 2 |
|
|
West Kenya, Kenya [55] | 1987–1990 | 449 | 58 (12.9) | 10 |
|
|
Maputo, Mozambique [56] | 2011–2012 | 841 | 63 (7.5) | 15 |
|
|
Kampala, Uganda [60] | 1997 | 305 | 39 (12.8) | 11 |
|
|
Western Africa | Benin City, Nigeria [57] | 1988–1989 | 642 | 67 (10.4) | 10 |
|
Ibadan, Nigeria [58] | 1998 | 102 | 39 (38.2) | 7 |
|
|
Eastern Asia | Tainan, Taiwan [63] | 2006–2007 | 396 | 60 (15.2) | 10 |
|
Okinawa, Japan [62] | NR | 526 | 40 (7.6) | 7 |
|
|
South eastern Asia | Bangkok, Thailand [64] | 1997 | 246 | 119 (48.4) | 19 |
|
Southern Asia | Pune, India [61] | 2013–2015 | 1524 | 59 (3.9) | 16 |
|
Ref, reference; (t), tied; NR, Not reported; BSI, bloodstream infection; NTS, non-typhoidal Salmonella.
Prevalence studies compared to incidence studies
Three hospital-based prevalence studies were done in the same location as a population-based surveillance study of typhoid incidence from our earlier systematic review [8]; two were located in Africa [44,45] (Figure 3) and one in Asia [51] (Figure 4). In Moshi, Tanzania, from 2007 through 2008, typhoid prevalence among pathogens was 37.5%, rank order was 1, and S:E ratio was 2.0 in children aged two months to under 13 years [44]. In rural and urban Moshi in 2011 among children under 15 years, typhoid incidence was 18 and 155 cases per 100 000, respectively. Among adults 13 years and older from 2007 through 2008, typhoid prevalence among pathogens was 44.8%, 1 and 3.7, respectively [45], and typhoid incidence in ages greater than 14 years was 28 and 201 cases per 100 000 in rural and urban Moshi, respectively [65]. In Kathmandu, Nepal from 2005 through 2006 [51], typhoid prevalence among pathogens was 36.6%, rank order was 1, and S: E ratio was 13.3 while incidence was 655 per 100 000 in 1986 [66].
Two locations, Pemba Island, Tanzania in 2010 [50] and Boulkiemdé, Burkina Faso in 2013 [47], had both hospital-based prevalence and a population-based incidence data collected during the same year. Small numbers of concurrent prevalence and incidence studies precluded a statistical examination for an association or trend. In Pemba Island, typhoid prevalence among pathogens was 64.8%, rank order was 1, S:E ratio was 9.2, and typhoid incidence was 110 cases per 100 000. In Boulkiemdé, typhoid prevalence among pathogens was 13.3%, rank order was 3, and S:E ratio was 2.3, and adjusted typhoid incidence was 249 cases per 100 000.
Discussion
We found that Salmonella Typhi prevalence, rank order and prevalence ratio among CO-BSI in hospitalised febrile patients vary substantially in place and time. For example, in three locations in Tanzania, Salmonella Typhi prevalence was 37% and the organism ranked first among pathogens isolated in Moshi [44,45]; prevalence was less than 1% and ranked 14 in Dar es Salaam [34] and Salmonella Typhi was not isolated in Mwanza [59]. We were only able to directly compare hospital-based prevalence data to studies of population-based incidence in two locations. Because we identified few locations that implement or report on both strategies simultaneously, we were unable to fully investigate the hypothesis that there is a relationship between hospital-based prevalence and population-based incidence.
Based on studies that overlap in place but not time [44,45,51] and also studies not included in our incidence review [10,67–69], it is plausible that areas with high typhoid incidence also observe a high proportion of Salmonella Typhi among pathogens isolated from blood cultures. It should be noted that in the only two locations we were able to directly compare the place and time of prevalence to population-based incidence of Salmonella Typhi, there was an inverse association, 64.8% prevalence with 110 cases per 100 000 incidence [50] vs. 13.3% with a 249 per 100 000 incidence [47]. However, in both of these locations, incidence would be classified as ‘high’ (i.e., greater than 100 cases per 100 000) and Salmonella Typhi was among the most frequent pathogens isolated. Blood culture sensitivity [70], proportion of febrile patients seeking hospital care [11,13], and seasonality [21] can lead to varying estimates of incidence [8] and prevalence, limiting the conclusions that can be drawn about the relationship until further investigation, especially given the sample size. We encourage concurrent prevalence and incidence studies to not only examine associations between the two, but also to provide more comprehensive data including on all isolates recovered to assist with informing policy decisions on typhoid control.
Statistical modelling is becoming increasingly important in predicting disease burden in areas where data are lacking [71]. These modelling techniques use what is previously known about a disease and observed data from one location to extrapolate estimates to other locations [72]. For example, epidemiologic studies demonstrate that unsafe water and food, and poor sanitation are associated with increased risk for typhoid fever and are on the causal pathway to infection [73]. Other covariates not directly on the causal pathway, such as population density, wealth distribution, and proportion of roads paved have been used in typhoid modelling [5,15,16]. To our knowledge, covariates that capture the disease state such as those presented in our review, including the hospital-based metrics of prevalence, rank, and ratio compared to other pathogens causing BSI, have not been explored in such models. Generating incidence data by hybrid surveillance requires conducting a representative healthcare utilisation survey in the catchment area of the sentinel surveillance site. Because typhoid prevalence data are considerably easier to collect compared with typhoid incidence data, they may represent an untapped information resource for making inferences about typhoid disease occurence in an area. We call for further data collection and reporting in order to gain further insight into the usefulness of these hospital-based metrics and to test these metrics in typhoid burden models. We anticipate that doing so will deliver more robust and accurate models for estimating typhoid incidence and insights into typhoid occurence outside of the few locations with rigorous incidence studies.
While the majority of studies in the original systematic review isolated Salmonella Typhi, a large proportion of studies in our review did not isolate Salmonella Typhi. Search strategies for systematic reviews of prevalence and incidence are designed to collect studies in which the pathogen of interest is reported. Because our review was on the prevalence of any CO-BSI, we were able to capture 21 studies that did not isolate Salmonella Typhi. It is reasonable to conclude that typhoid fever incidence is unlikely to be substantial in a place where a large prevalence study fails to isolate any Salmonella Typhi. Although small studies should be viewed with caution due to their limited power to confirm absence, studies isolating no Salmonella Typhi represent important potential sources of information about locations with little or no disease at the time of the study. There are also studies in which participants fit the inclusion criteria for a BSI, but the study only reported on a single pathogenic species, such as S. pneumoniae [74,75]. Such studies were not only excluded from our review, but also represent missed opportunities to report the full range of pathogens that were or were not isolated [76].
Our search strategy only included studies on hospitalised participants, where the prevalence of bloodstream infection tends to be considerably higher overall than that found in the outpatient setting. In doing so, we likely missed a proportion of patients that have mild disease, who either do not present to the hospital or are treated as an outpatient or other facilities. We elected not to combine outpatient studies to avoid study location becoming a source of bias but did not attempt to make any adjustments to our analysis to account for underascertainment.
Additionally, we planned to examine the S:E ratio to control the effect of study design on apparent Salmonella Typhi prevalence. However, the prevalence and rank of E. coli were not stable across our dataset, limiting the usefulness of this metric in our review. An alternative approach would have been to create a composite variable of bloodstream infections other than the target organism for benchmarking. In our view, this approach is confounded by the influence of both other major epidemicprone causes of bloodstream infection such as non-typhoidal S. enterica as well as vaccine-preventable infections such as S. pneumoniae for which prevalence changes may be driven by vaccine introductions. Given comparators have proven effective for other pathogens [26], we suggest that investigators continue to examine and investigate their performance.
We provide additional evidence through hospital-based prevalence surveillance studies that Salmonella Typhi varies in both place and time. Hospital-based studies of CO-BSI may provide a useful window on local disease burden. Continued use of hospital-based prevalence, sentinel site surveillance and active, population-based incidence studies is central to recognising changes in disease dynamics, antimicrobial resistance, and to monitor the impact of vaccine introduction. This review serves as a resource for typhoid disease modellers, and policy makers. We anticipate that hospital-based study metrics warrant consideration as covariates in statistical models and as evidence for decision making for areas beyond those with rigorous studies of typhoid incidence.
Supplementary Material
Acknowledgements
The findings of this study were presented in part at the 11th International Conference on Typhoid and Other Invasive Salmonelloses, Hanoi, Vietnam, 26-28 March 2019 (poster 87).
JAC and CSM received support from Bill & Melinda Gates Foundation (BMGF) grant OPP1151153. JAC also received support from BMGF (OPP1125993 and OPP1158210), the US National Institutes of Health (grant number R01AI121378) and the New Zealand Health Research Council through the e-ASIA Joint Research Program (grant number 16/697).
Supporting Information
Additional Supporting Information may be found in the online version of this article:
Appendix S1 Final search strategy 19 September 2018.
References
- 1.Crump JA, Sjölund-Karlsson M, Gordon MA, Parry CM. Epidemiology, clinical presentation, laboratory diagnosis, antimicrobial resistance, and antimicrobial management of invasive salmonella infections. Clin Microbiol Rev 2015: 28: 901–937. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.GBD 2017 Causes of Death Collaborators Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018: 392: 1736–1788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.GBD 2017 Disease and Injury Incidence and Prevalence Collaborators Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018: 392: 1789–1858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mogasale V, Mogasale VV, Ramani E et al. . Revisiting typhoid fever surveillance in low and middle income countries: lessons from systematic literature review of population-based longitudinal studies. BMC Infect Dis 2016: 16: 35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Antillón M, Warren JL, Crawford FW et al. . The burden of typhoid fever in low- and middle-income countries: A metaregression approach. PLoS Negl Trop Dis 2017: 11: e0005376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kim J-H, Mogasale V, Im J, Ramani E, Marks F. Updated estimates of typhoid fever burden in sub-Saharan Africa. Lancet Glob Health 2017: 5: e969. [DOI] [PubMed] [Google Scholar]
- 7.World Health Organization Typhoid vaccines: WHO position paper, March 2018 – Recommendations. Vaccine 2019: 37: 214–216. [DOI] [PubMed] [Google Scholar]
- 8.Marchello CS, Hong CY, Crump JA. Global typhoid fever incidence: a systematic review and meta-analysis. Clin Infect Dis 2019:68(Supplement_2): S105–S116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Feikin DR, Olack B, Bigogo GM et al. . The burden of common infectious disease syndromes at the clinic and household level from population-based surveillance in rural and urban Kenya. PLoS ONE 2011: 6: e16085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Andrews JR, Vaidya K, Bern C et al. . High rates of enteric fever diagnosis and lower burden of culture-confirmed disease in peri-urban and rural. Nepal J Infect Dis 2018: 218 (suppl_4): S214–S221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Crump JA, Youssef FG, Luby SP et al. . Estimating the incidence of typhoid fever and other febrile illnesses in developing countries. Emerg Infect Dis 2003: 9: 539–544. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Crump JA, Kirk MD. Estimating the burden of febrile illnesses. PLoS Negl Trop Dis 2015: 9: e0004040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Andrews JR, Barkume C, Yu AT et al. . Integrating facility-based surveillance with healthcare utilization surveys to estimate enteric fever incidence: methods and challenges. J Infect Dis 2018: 218(suppl_4): S268–S276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Luby SP, Saha S, Andrews JR. Towards sustainable public health surveillance for enteric fever. Vaccine 2015: 33: C3–C7. [DOI] [PubMed] [Google Scholar]
- 15.Stanaway JD, Reiner RC, Blacker BF et al. . The global burden of typhoid and paratyphoid fevers: a systematic analysis for the Global Burden of Disease Study 2017. Lancet Infect Dis 2019: 19: 369–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pitzer VE, Feasey NA, Msefula C et al. . Mathematical modeling to assess the drivers of the recent emergence of typhoid fever in Blantyre, Malawi. Clin Infect Dis 2015: 61(suppl 4): S251–S258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pitzer VE, Bowles CC, Baker S et al. . Predicting the impact of vaccination on the transmission dynamics of typhoid in South Asia: a mathematical modeling study. PLoS Negl Trop Dis 2014: 8: e2642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lo NC, Gupta R, Stanaway JD et al. . Comparison of strategies and incidence thresholds for Vi conjugate vaccines against typhoid fever: a cost-effectiveness modeling study. J Infect Dis 2018: 218(suppl_4): S232–S242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Watson CH, Edmunds WJ. A review of typhoid fever transmission dynamic models and economic evaluations of vaccination. Vaccine 2015: 33(Suppl 3): C42–C54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Dewan AM, Corner R, Hashizume M, Ongee ET. Typhoid fever and its association with environmental factors in the Dhaka metropolitan area of Bangladesh: a spatial and time-series approach. PLoS Negl Trop Dis 2013: 7: e1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Saad NJ, Lynch VD, Antillón M, Yang C, Crump JA, Pitzer VE. Seasonal dynamics of typhoid and paratyphoid fever. Sci Rep 2018: 8: 6870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kelly-Hope LA, Alonso WJ, Thiem VD et al. . Geographical distribution and risk factors associated with enteric diseases in Vietnam. Am J Trop Med Hyg 2007: 76: 706–712. [PubMed] [Google Scholar]
- 23.Wang L-X, Li X-J, Fang L-Q, Wang D-C, Cao W-C, Kan B. Association between the incidence of typhoid and paratyphoid fever and meteorological variables in Guizhou. China. Chin Med J (Engl) 2012: 125: 455–460. [PubMed] [Google Scholar]
- 24.Akullian A, Ng’eno E, Matheson AI, et al. . Environmental transmission of typhoid fever in an urban slum. PLoS Negl Trop Dis. 2015: 9: e0004212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.de Alwis R, Watson C, Nikolay B et al. . Role of environmental factors in shaping spatial distribution of Salmonella enterica serovar Typhi, Fiji. Emerg Infect Dis 2018: 24: 284–293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mackenzie GA, Hill PC, Jeffries DJ et al. . Effect of the introduction of pneumococcal conjugate vaccination on invasive pneumococcal disease in The Gambia: a population-based surveillance study. Lancet Infect Dis 2016: 16: 703–711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Marchello CS, Dale AP, Pisharody S, Rubach MP, Crump JA. A Systematic Review and Meta-analysis of the Prevalence of Community-Onset Bloodstream Infections among Hospitalized Patients in Africa and Asia. Antimicrob Agents Chemother. 2019: in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Deen J, von Seidlein L, Andersen F, Elle N, White NJ, Lubell Y. Community-acquired bacterial bloodstream infections in developing countries in south and southeast Asia: a systematic review. Lancet Infect Dis 2012: 12: 480–487. [DOI] [PubMed] [Google Scholar]
- 29.Reddy EA, Shaw AV, Crump JA. Community-acquired bloodstream infections in Africa: a systematic review and meta-analysis. Lancet Infect Dis 2010: 10: 417–432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.United Nations Statistics Division Standard country or area codes for statistical use (M49). (Available from: https://unstats.un.org/unsd/methodology/m49/) [5 Feb 2019].
- 31.Peters RPH, Zijlstra EE, Schijffelen MJ et al. . A prospective study of bloodstream infections as cause of fever in Malawi: clinical predictors and implications for management. Trop Med Int Health 2004: 9: 928–934. [DOI] [PubMed] [Google Scholar]
- 32.Chheng K, Carter MJ, Emary K et al. . A prospective study of the causes of febrile illness requiring hospitalization in children in Cambodia. PLoS ONE 2013: 8: e60634. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Morch K, Manoharan A, Chandy S et al. . Acute undifferentiated fever in India: a multicentre study of aetiology and diagnostic accuracy. BMC Infect Dis 2017: 17: 665. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Blomberg B, Manji KP, Urassa WK et al. . Antimicrobial resistance predicts death in Tanzanian children with bloodstream infections: a prospective cohort study. BMC Infect Dis 2007: 7: 43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Gordon MA, Walsh AL, Chaponda M et al. . Bacteraemia and mortality among adult medical admissions in Malawi-predominance of non-typhi salmonellae and Streptococcus pneumoniae. J Infect 2001: 42: 44–49. [DOI] [PubMed] [Google Scholar]
- 36.Okwara FN, Obimbo EM, Wafula EM, Murila FV. Bacteraemia, urinary tract infection and malaria in hospitalised febrile children in Nairobi: is there an association? East Afr Med J 2004: 81: 47–51. [DOI] [PubMed] [Google Scholar]
- 37.Kibuuka A, Byakika-Kibwika P, Achan J et al. . Bacteremia among febrile Ugandan children treated with antimalarials despite a negative malaria test. Am J Trop Med Hyg 2015: 93: 276–280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Kassa-Kelembho E, Mbolidi C-D, Service Y-B, Morvan J, Minssart P. Bacteremia in adults admitted to the Department of Medicine of Bangui Community Hospital (Central African Republic). Acta Trop 2003: 89: 67–72. [DOI] [PubMed] [Google Scholar]
- 39.Walsh AL, Phiri AJ, Graham SM, Molyneux EM, Molyneux ME. Bacteremia in febrile Malawian children: clinical and microbiologic features. Pediatr Infect Dis J 2000: 19: 312–318. [DOI] [PubMed] [Google Scholar]
- 40.Leelarasamee A, Chupaprawan C, Chenchittikul M, Udompanthurat S. Etiologies of acute undifferentiated febrile illness in Thailand. J Med Assoc Thai 2004: 87: 464–472. [PubMed] [Google Scholar]
- 41.Punjabi NH, Taylor WRJ, Murphy GS et al. . Etiology of acute, non-malaria, febrile illnesses in Jayapura, northeastern Papua, Indonesia. Am J Trop Med Hyg 2012: 86: 46–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Hyams KC, Oldfield EC, Scott RM et al. . Evaluation of febrile patients in Port Sudan, Sudan: isolation of dengue virus. Am J Trop Med Hyg 1986: 35: 860–865. [DOI] [PubMed] [Google Scholar]
- 43.Archibald LK, den Dulk MO, Pallangyo KJ, Reller LB. Fatal Mycobacterium tuberculosis bloodstream infections in febrile hospitalized adults in Dar es Salaam, Tanzania. Clin Infect Dis Off Publ Infect Dis Soc Am 1998: 26: 290–296. [DOI] [PubMed] [Google Scholar]
- 44.Crump JA, Ramadhani HO, Morrissey AB et al. . Invasive bacterial and fungal infections among hospitalized HIV-infected and HIV-uninfected children and infants in northern Tanzania. Trop Med Int Health 2011: 16: 830–837. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Crump JA, Ramadhani HO, Morrissey AB et al. . Invasive bacterial and fungal infections among hospitalized HIV-infected and HIV-uninfected adults and adolescents in northern Tanzania. Clin Infect Dis 2011: 52: 341–348. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Dougle ML, Hendriks ER, Sanders EJ, Dorigo-Zetsma JW. Laboratory investigations in the diagnosis of septicaemia and malaria. East Afr Med J 1997: 74: 353–356. [PubMed] [Google Scholar]
- 47.Guiraud I, Post A, Diallo SN et al. . Population-based incidence, seasonality and serotype distribution of invasive salmonellosis among children in Nanoro, rural Burkina Faso. PLoS ONE 2017: 12: e0178577. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Bell M, Archibald LK, Nwanyanwu O et al. . Seasonal variation in the etiology of bloodstream infections in a febrile inpatient population in a developing country. Int J Infect Dis IJID Off Publ Int Soc Infect Dis 2001: 5: 63–69. [DOI] [PubMed] [Google Scholar]
- 49.Nadjm B, Mtove G, Amos B et al. . Severe febrile illness in adult hospital admissions in Tanzania: a prospective study in an area of high malaria transmission. Trans R Soc Trop Med Hyg 2012: 106: 688–695. [DOI] [PubMed] [Google Scholar]
- 50.Thriemer K, Ley B, Ame S et al. . The burden of invasive bacterial infections in Pemba, Zanzibar. PLoS ONE 2012: 7: e30350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Kelly DF, Thorson S, Maskey M et al. . The burden of vaccine-preventable invasive bacterial infections and pneumonia in children admitted to hospital in urban Nepal. Int J Infect Dis 2011: 15: e17–23. [DOI] [PubMed] [Google Scholar]
- 52.Nadjm B, Amos B, Mtove G et al. . WHO guidelines for antimicrobial treatment in children admitted to hospital in an area of intense Plasmodium falciparum transmission: prospective study. BMJ 2010: 340: c1350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Lin TY. Bacteremia in febrile children. Chang Yi Xue Za Zhi 1991: 14: 44–49. [PubMed] [Google Scholar]
- 54.Pavlinac PB, Naulikha JM, John-Stewart GC et al. . Mycobacterium tuberculosis bacteremia among acutely febrile children in western Kenya. Am J Trop Med Hyg 2015: 93: 1087–1091. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Petit PL, Haarlem JV, Poelman M, Haverkamp MC, Wamola IA. Bacteraemia in patients presenting with fever. East Afr Med J 1995: 72: 116–120. [PubMed] [Google Scholar]
- 56.Preziosi M, Zimba TF, Lee K et al. . A prospective observational study of bacteraemia in adults admitted to an urban Mozambican hospital. S Afr Med J 2015: 105: 370–374. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Akpede GO, Abiodun PO, Sykes RM. Relative contribution of bacteraemia and malaria to acute fever without localizing signs of infection in under-five children. J Trop Pediatr. 1992: 38: 295–298. [DOI] [PubMed] [Google Scholar]
- 58.Ayoola OO, Adeyemo AA, Osinusi K. Aetiological agents, clinical features and outcome of septicaemia in infants in Ibadan. West Afr J Med 2003: 22: 30–34. [DOI] [PubMed] [Google Scholar]
- 59.Christopher A, Mshana SE, Kidenya BR, Hokororo A, Morona D. Bacteremia and resistant gram-negative pathogens among under-fives in Tanzania. Ital J Pediatr 2013: 39: 27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Ssali FN, Kamya MR, Wabwire-Mangen F et al. . A prospective study of community-acquired bloodstream infections among febrile adults admitted to Mulago Hospital in Kampala, Uganda. J Acquir Immune Defic Syndr Hum Retrovirology Off Publ Int Retrovirology Assoc 1998: 19: 484–489. [DOI] [PubMed] [Google Scholar]
- 61.Mave V, Chandanwale A, Kagal A et al. . High burden of antimicrobial resistance and mortality among adults and children with community-onset bacterial infections in India. J Infect Dis 2017: 215: 1312–1320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Tokuda Y, Miyasato H, Stein GH, Kishaba T. The degree of chills for risk of bacteremia in acute febrile illness. Am J Med 2005: 118: 1417. [DOI] [PubMed] [Google Scholar]
- 63.Lee C-C, Wu C-J, Chi C-H et al. . Prediction of community-onset bacteremia among febrile adults visiting an emergency department: rigor matters. Diagn Microbiol Infect Dis 2012: 73: 168–173. [DOI] [PubMed] [Google Scholar]
- 64.Archibald LK, McDonald LC, Rheanpumikankit S et al. . Fever and human immunodeficiency virus infection as sentinels for emerging mycobacterial and fungal bloodstream infections in hospitalized patients ≥15 years old, Bangkok. J Infect Dis 1999: 180: 87–92. [DOI] [PubMed] [Google Scholar]
- 65.Marks F, von Kalckreuth V, Aaby P et al. . Incidence of invasive salmonella disease in sub-Saharan Africa: a multicentre population-based surveillance study. Lancet Glob Health 2017: 5: e310–e323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Acharya IL, Lowe CU, Thapa R et al. . Prevention of typhoid fever in Nepal with the Vi capsular polysaccharide of Salmonella typhi. A preliminary report. N Engl J Med 1987: 317: 1101–1114. [DOI] [PubMed] [Google Scholar]
- 67.Petersiel N, Shresta S, Tamrakar R et al. . The epidemiology of typhoid fever in the Dhulikhel area, Nepal: A prospective cohort study. PLoS ONE 2018: 13: e0204479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Barkume C, Date K, Saha SK et al. . Phase I of the Surveillance for Enteric Fever in Asia Project (SEAP): an overview and lessons learned. J Infect Dis 2018: 218(suppl_4): S188–S194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.John J, Van Aart CJC, Grassly NC. The burden of typhoid and paratyphoid in India: systematic review and meta-analysis. PLoS Negl Trop Dis 2016: 10: e0004616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Antillon M, Saad NJ, Baker S, Pollard AJ, Pitzer VE. The relationship between blood sample volume and diagnostic sensitivity of blood culture for typhoid and paratyphoid fever: a systematic review and meta-analysis. J Infect Dis 2018: 218(suppl_4): S255–S267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Heesterbeek H, Anderson RM, Andreasen V et al. . Modeling infectious disease dynamics in the complex landscape of global health. Science 2015: 347: aaa4339–aaa4339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Keeling MJ, Rohani P. Modeling infectious diseases in humans and animals. Princeton: Princeton University Press, 2011. [Google Scholar]
- 73.Crump JA. Progress in typhoid fever epidemiology. Clin Infect Dis 2019: 68: S4–S9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Williams EJ, Thorson S, Maskey M et al. . Hospital-based surveillance of invasive pneumococcal disease among young children in urban Nepal. Clin Infect Dis 2009: 48(Suppl 2): S114–S122. [DOI] [PubMed] [Google Scholar]
- 75.Rhodes J, Dejsirilert S, Maloney SA et al. . Pneumococcal bacteremia requiring hospitalization in rural Thailand: an update on incidence, clinical characteristics, serotype distribution, and antimicrobial susceptibility, 2005–2010. PLoS ONE 2013: 8: e66038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Crump JA. Time for a comprehensive approach to the syndrome of fever in the tropics. Trans R Soc Trop Med Hyg 2014: 108: 61–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.MapChart: Africa - Countries [Internet]. (Available from: https://mapchart.net/africa.html) [29 May 2019].
- 78.MapChart: Asia- Countries [Internet]. (Available from: https://mapchart.net/asia.html) [29 May 2019].
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.