Skip to main content
PMC home page
eLife logoLink to eLife
. 2015 Dec 8;4:e09186. doi: 10.7554/eLife.09186

Mathematical modeling of the West Africa Ebola epidemic

Jean-Paul Chretien 1,*, Steven Riley 2, Dylan B George 3
Editor: Mark Jit4
PMCID: PMC4769159  PMID: 26646185

Abstract

As of November 2015, the Ebola virus disease (EVD) epidemic that began in West Africa in late 2013 is waning. The human toll includes more than 28,000 EVD cases and 11,000 deaths in Guinea, Liberia, and Sierra Leone, the most heavily-affected countries. We reviewed 66 mathematical modeling studies of the EVD epidemic published in the peer-reviewed literature to assess the key uncertainties models addressed, data used for modeling, public sharing of data and results, and model performance. Based on the review, we suggest steps to improve the use of modeling in future public health emergencies.

DOI: http://dx.doi.org/10.7554/eLife.09186.001

Research Organism: Human

eLife digest

The outbreak of Ebola that started in West Africa in late 2013 has caused at least 28,000 illnesses and 11,000 deaths. As the outbreak progressed, global and local public health authorities scrambled to contain the spread of the disease by isolating those who were ill, putting in place infection control processes in health care settings, and encouraging the public to take steps to prevent the spread of the illness in the community. It took a massive investment of resources and personnel from many countries to eventually bring the outbreak under control.

To determine where to allocate people and resources during the outbreak, public health authorities often turned to mathematical models created by scientists to predict the course of the outbreak and identify interventions that could be effective. Many groups of scientists created models of the epidemic using publically available data or data they obtained from government officials or field studies. In some instances, the models yielded valuable insights. But with various groups using different methods and data, the models didn’t always agree on what would happen next or how best to contain the epidemic.

Now, Chretien et al. provide an overview of Ebola mathematical modeling during the epidemic and suggest how future efforts may be improved. The overview included 66 published studies about Ebola outbreak models. Although most forecasts predicted many more cases than actually occurred, some modeling approaches produced more accurate predictions, and several models yielded valuable insights. For example, one study found that focusing efforts on isolating patients with the most severe cases of Ebola would help end the epidemic by substantially reducing the number of new infections. Another study used real-time airline data to predict which traveler screening strategies would be most efficient at preventing international spread of Ebola. Furthermore, studies that obtained genomic data showed how specific virus strains were transmitted across geographic areas.

Chretien et al. argue that mathematical modeling efforts could be more useful in future pubic health emergencies if modelers cooperated more, and suggest the collaborative approach of weather forecasters as a good example to follow. Greater data sharing and the creation of standards for epidemic modeling would aid better collaboration.

DOI: http://dx.doi.org/10.7554/eLife.09186.002

Introduction

On March 23, 2014, the Ministry of Health Guinea notified the World Health Organization (WHO) of a rapidly evolving outbreak of Ebola virus disease (EVD), now believed to have begun in December 2013. The epidemic spread through West Africa and reached Europe and the United States. As of November 4, 2015, WHO reported more than 28,000 cumulative cases and 11,000 deaths in Guinea, Liberia, and Sierra Leone, where transmission had been most intense (World Health Organization, 2016).

As the emergency progressed, researchers developed mathematical models of the epidemiological dynamics. Modelers have assessed ongoing epidemics previously, but the prominence of recent EVD work, enabled by existing research programs for infectious disease modeling (National Institutes of Health, 2016a; National Institutes of Health, 2016b) and online availability of EVD data via WHO (World Health Organization, 2016), Ministries of Health of affected countries, or modelers who transcribed and organized public WHO or Ministry of Health data (Rivers C) may be unprecedented. The efforts for this outbreak have been numerous and diverse, with major media incorporating modeling results in many pieces throughout the outbreak. U.S. Government decision making has benefited from modeling results at key moments during the response (Robinson R).

We draw on this vigorous response of the epidemiological modeling community to the EVD epidemic to review (Moher et al., 2009) the application of modeling to public health emergencies, and identify lessons to guide the modeling response to future emergencies.

Results

Overview of modeling applications

We identified 66 publications meeting inclusion criteria (Figure 1).

Figure 1. Literature search flow.

Figure 1.

Open in a new tab

DOI: http://dx.doi.org/10.7554/eLife.09186.003

Models addressed 6 key uncertainties about the EVD epidemic: transmissibility, typically represented by the reproduction number (R, the average number of people each infected person infects; assessed in 41 publications); effectiveness of various interventions that had been or might be implemented (in 29 publications); epidemic forecast (in 29 publications); regional or international spreading patterns or risk (in 15 publications); phylogenetics of EVD viruses (in 9 publications); and feasibility of conducting vaccine trials in West Africa (in 2 publications) (Table 1, Supplementary file 1).

Table 1.

Overview of modeling publications on the 2013-present EVD epidemic.

DOI: http://dx.doi.org/10.7554/eLife.09186.004

Ref. Date of latest EVD data Date published EVD data was pre-existing and public Uncertainties addressed
R Interventions Forecast Spread Phylogenetics Clinical trials
Baize et al., 2014 3/20/14 4/16/14 No *
Dudas and Rambaut, 2014 3/20/14 5/2/14 Yes *
Alizon et al., 2014 6/18/14 12/13/14 Yes * *
Gire et al., 2014 6/18/14 8/28/14 No * *
Stadler et al., 2014 6/18/14 10/6/14 Yes *
Volz and Pond, 2014 6/18/14 10/24/14 Yes *
Pandey et al., 2014 8/7/14 10/30/14 Yes * * *
Gomes et al., 2014 8/9/14 9/2/14 Yes * * *
Valdez et al., 2015 8/15/14 7/20/15 Yes * * * *
Merler et al., 2015 8/16/14 1/7/15 Yes * * * *
Rainisch et al., 2015 8/16/14 2/18/15 Yes *
Althaus, 2014 8/20/14 9/2/14 Yes *
Fisman et al., 2014 8/22/14 9/8/14 Yes * *
Nishiura and Chowell, 2014 8/26/14 9/11/14 Yes * *
Poletto et al., 2014 8/27/14 10/23/14 Yes * *
Meltzer et al., 2014 8/28/14 9/26/14 Yes * * *
Agusto et al., 2015 8/29/14 4/23/15 Yes * *
Althaus, 2015 8/31/14 4/19/15 Yes *
Scarpino et al., 2014 8/31/14 12/15/14 Yes *
Weitz and Dushoff, 2015 8/31/14 3/4/15 Yes * *
Drake et al., 2015 9/2/14 10/30/14 Yes * * *
Towers et al., 2014 9/8/14 9/18/14 Yes * *
Bellan et al., 2014 9/14/14 10/14/14 Yes *
Chowell et al., 2015 9/14/14 1/19/15 Yes *
Cooper et al., 2015 9/14/14 4/14/15 Yes *
Read et al., 2015 9/14/14 11/12/14 Yes * *
WHO Ebola Response Team, 2014 9/14/14 9/23/14 No * * *
Faye et al., 2015 9/16/14 1/23/15 No * *
Bogoch et al., 2015 9/21/14 10/21/14 Yes * *
Yamin et al., 2014 9/22/14 10/28/14 No * *
Lewnard et al., 2014 9/23/14 10/24/14 Yes * * *
Webb et al., 2015 9/23/14 1/30/15 Yes * * *
Shaman et al., 2014 9/28/14 10/27/14 Yes * *
Chowell et al., 2014 10/1/14 11/20/14 Yes * *
Fasina et al., 2014 10/1/14 10/9/14 Yes *
Khan et al., 2015 10/1/14 2/24/15 Yes *
Rivers et al., 2014 10/5/14 10/16/14 Yes * * *
Xia et al., 2015 10/7/14 9/8/15 Yes * *
Majumder et al., 2014 10/11/14 4/28/15 Yes * *
Kiskowski, 2014 10/15/14 11/13/14 Yes * *
Fisman and Tuite, 2014 10/18/14 11/21/14 Yes * * *
Althaus et al., 2015 10/20/14 1/15/15 Yes * * *
Simon-Loriere et al., 2015 10/25/14 6/24/15 No *
Rainisch et al., 2015 10/31/14 6/16/15 Yes *
Fast et al., 2015 11/1/14 5/15/15 Yes *
Kucharski et al., 2015 11/1/14 2/18/15 Yes * * *
Tong et al., 2015 11/11/14 5/13/15 No * *
Hoenen et al., 2015 11/21/14 3/26/15 No *
Cope et al., 2014 12/3/14 12/10/14 Yes * *
White et al., 2014 12/3/14 1/30/15 Yes * * *
WHO Ebola Response Team, 2015 12/14/14 12/24/14 No * *
Chowell et al., 2014 12/17/14 1/21/15 Yes *
Siettos et al., 2014 12/21/14 3/9/15 Yes * *
Park et al., 2015 12/26/14 6/18/15 No *
Nadhem and Nejib, 2015 12/30/14 6/14/15 Yes *
Camacho et al., 2015 1/18/15 2/10/15 Yes * *
Carroll et al., 2015 1/31/15 6/17/15 No * *
Bellan et al., 2015 2/9/15 4/15/15 Yes * *
Barbarossa et al., 2015 2/13/15 7/21/15 Yes * * *
Kugelman et al., 2015 2/14/15 6/12/15 No *
Cleaton et al., 2015 2/28/15 9/3/15 Yes *
Wang and Zhong, 2015 3/18/15 3/24/15 Yes *
Toth et al., 2015 3/31/15 7/14/15 Yes * *
Dong et al., 2015 4/3/15 9/5/15 Yes * *
Browne et al., 2015 4/12/15 5/14/15 Yes * *
Zinszer et al., 2015 5/13/15 9/1/15 Yes *
Open in a new tab

The number of publications with models to estimate R increased rapidly early in the epidemic, along with those including intervention, forecasting, and regional and international spread models; the growth rate of publications with phylogenetic modeling applications and clinical trial models increased later in the epidemic (Figure 2).

Figure 2. Cumulative number of modeling applications by date of most recent EVD data used.

Figure 2.

Open in a new tab

The figure includes 125 modeling applications across the 66 publications.

DOI: http://dx.doi.org/10.7554/eLife.09186.005

Of the 125 models reported across the studies, 74% included mechanistic assumptions about disease transmission (e.g., compartmental, agent-based, or phylogenetic models), while 26% were purely phenomenological (Supplementary file 2).

Data sources

For 54 (82%) of the 66 publications, the only EVD data used was pre-existing and publicly-available (Table 1). Typically, these were aggregate case data posted online by the WHO or affected countries, or Ebola virus genetic data released previously during the epidemic. Twelve studies used original EVD epidemiological data (Baize et al., 2014; WHO Ebola Response Team, 2014; 2015; Faye et al., 2015; Yamin et al., 2014) or genomic data (Baize et al., 2014; Gire et al., 2014; Simon-Loriere et al., 2015; Tong et al., 2015; Hoenen et al., 2015; Park et al., 2015; Carroll et al., 2015; Kugelman et al., 2015).

Examples of additional data used for some modeling applications include official reports of social mobilization efforts (Fast et al., 2015), media reports of case clusters (Cleaton et al., 2015), media reports of events that may curtail or aggravate transmission (Majumder et al., 2014), and international air travel data (Gomes et al., 2014; Poletto et al., 2014; Read et al., 2015; Bogoch et al., 2015; Rainisch et al., 2015; Cope et al., 2014). Several studies incorporated spatial data on EVD cases into models of regional EVD spread (Gire et al., 2014; Merler et al., 2015; Rainisch et al., 2015; Tong et al., 2015; Carroll et al., 2015; Zinszer et al., 2015).

Data and results sharing

Of the 12 studies that collected original EVD data, 9 released those data before or at the time of publication (8 with Ebola virus genetic data deposited in GenBank (Baize et al., 2014; Gire et al., 2014; Simon-Loriere et al., 2015; Tong et al., 2015; Hoenen et al., 2015; Park et al., 2015; Carroll et al., 2015; Kugelman et al., 2015) and 1 with detailed epidemiological data in the online publication (Yamin et al., 2014). Many publications used results from the WHO Ebola Response Team investigations (WHO Ebola Response Team, 2014; 2015) (for example, estimates of the generation time, case fatality rate, or other epidemiological parameters as model inputs), but the detailed epidemiological data from these studies, to date, are not publicly available.

Accumulation of shared EVD data over successive studies was evident especially in the phylogenetic analyses. For example, all phylogenetic studies published after release of the initial Ebola virus sequences by (Baize et al., 2014) (Guinea) and (Gire et al., 2014) (Sierra Leone) incorporated those sequence data.

Across all studies, the publication lag (defined as date of most recent EVD data used to date of online publication) was almost 3 months (median [interquartile range] = 85 [30–157] days). The lag varied across modeling applications, and was considerably shorter in studies that included models to estimate R (median = 58 days for publications with R estimation versus 118 days for others) or to forecast (median = 50 versus 125 days) (Figure 3).

Figure 3. Publication lag by type of modeling application.

Figure 3.

Open in a new tab

The vertical red and turquoise lines indicate the median lag for publications including and not including, respectively, the type of modeling application.

DOI: http://dx.doi.org/10.7554/eLife.09186.006

Lags were longest for studies with phylogenetic and clinical trials applications (median = 125 and 108 days, respectively), although there were fewer publications with these models.

Modeling results: R and forecasts

Forty-one publications characterized epidemic dynamics using epidemiological (N=36), genomic (N=4), or news report data (N=1). Twenty-four of these provided estimates of the basic reproduction number (R0) for Guinea, Liberia, Sierra Leone, or West Africa, using epidemiological or genomic data (Figure 4, Supplementary file 3).

Figure 4. R0 estimates by type of model input data.

Figure 4.

Open in a new tab

Aggregate, case counts released by the WHO or Ministries of Health; Line-level, individual-level data from epidemiological investigations; Genomic, Ebola virus sequence data. The Figure excludes an outlier estimate of 8.33 for Sierra Leone (Fisman et al., 2014).

DOI: http://dx.doi.org/10.7554/eLife.09186.007

There were 16 country-specific estimates of R0 for Guinea, Liberia, or Sierra Leone that used EVD epidemiological data (aggregate or line-level) and provided 95% confidence or credible intervals (CIs). Median CI width was about 85% smaller for models that used cumulative EVD counts (N=11 models in 5 publications) than for models that used disaggregated EVD case data, such as weekly counts (N=5 models in 3 publications) (Figure 5).

Figure 5. R0 estimates and CIs by type of epidemiological input data.

Figure 5.

Open in a new tab

Disaggregated data typically were weekly counts. Top row: Vertical lines indicate 95% CIs. Bottom row: Horizontal bars indicate median CI width.

DOI: http://dx.doi.org/10.7554/eLife.09186.008

Although CIs were also narrower for models when deterministic rather than stochastic methods were used to estimate parameter uncertainty, all of the deterministic results came from a single study (Figure 6).

Figure 6. R0 estimates and CIs by model fitting method.

Figure 6.

Open in a new tab

Top row: Vertical lines indicate 95% CIs. Bottom row: Horizontal bars indicate median CI width.

DOI: http://dx.doi.org/10.7554/eLife.09186.009

Fifteen publications provided numerical forecasts of cumulative EVD incidence for West African countries. Of 22 models that assumed no additional response measures beyond those implemented at the time (i.e., 'status quo' assumptions), 18 overestimated the future number of cases (Figure 7, Supplementary file 4).

Figure 7. Accuracy of cumulative incidence forecasts.

Figure 7.

Open in a new tab

Accuracy is shown as the ratio of predicted incidence to incidence subsequently reported by the WHO. 'Dampening' refers to various approaches to restrict the growth of forecasted incidence over time. Top row: Accuracy by date of forecast. Bottom row: Accuracy by forecast lead time ('Horizon'). The Figure excludes one forecast with horizon > 1 year (Fisman and Tuite, 2014).

DOI: http://dx.doi.org/10.7554/eLife.09186.010

In multivariate analysis, forecast error was lower for forecasts made later in the outbreak (14% reduction in mean absolute percentage error [MAPE] per week, P<0.001), higher for forecasts with longer time horizons (29% increase in MAPE per week, P<0.01), and lower for forecasts that used decay terms, spatially heterogeneous contact patterns, or other methods that served to constrain projected incidence growth (90% reduction in MAPE, P<0.01). Country and number of parameters in the model were not statistically significant predictors of forecast accuracy.

Discussion

We identified 66 modeling publications during approximately 18 months of the EVD response that assessed trends in the intensity of transmission, effectiveness of control measures, future case counts, regional and international spreading risk, Ebola virus phylogenetic relationships and recent evolutionary dynamics, and feasibility of clinical trials in West Africa. We found a heavy dependence on public data for EVD modeling, and identified factors that might have influenced model performance. To our knowledge, this review is one of the most comprehensive assessments of mathematical modeling applied to a single real-world public health emergency.

An important caveat of our review is that it only captures published results. We are aware of additional EVD epidemiological investigations and modeling not yet published. Some modelers providing direct support to operational response efforts have not published results, possibly because of operational demands.

Also, we could not account comprehensively for the sources of variation across studies. For example, studies that estimated R0 using the same data sources at about the same time reported varied results. Such variation may, in part, reflect the problem of identifiability, with different R0 estimates possible for models that perform equally well depending on other parameter values (Weitz and Dushoff, 2015). Ideally, an investigation into this heterogeneity would include implementation of models in a common testing environment.

Our review suggests several possible steps for improving the application of epidemiological modeling during public health emergencies. First, agreement on community best practices could improve the quality of modeling support to decision-makers. For example, our analysis is consistent with simulation studies showing underestimation of uncertainty in estimating R0 with cumulative (as opposed to disaggregated) incidence data, and supports the recommendation to use disaggregated data and stochastic models (King et al., 2015). Additionally, incidence forecasts provided reasonable prospective estimates several weeks forward in time during the initial phase; however, given available data and methodologies these forecasts became progressively more inaccurate as they projected dynamics beyond several weeks. Validation of incidence forecasts against other relevant data, such as hospital admissions and contacts identified, also could provide evidence that the assumptions are sound.

The 2014 onwards ebola outbreak in West Africa clearly highlights the need for a better understanding of how increasing awareness of severe infections within a community decreases their transmissibility even in the absence of specific interventions. Advancing methodological approaches to capture this effect, such as dampening approaches, might help account for behavioral changes, interventions, contact heterogeneity, or other factors that can be expected in a public health emergency which likely will improve forecasting accuracy. Establishing best practices within the community will allow decision-makers the ability to more quickly accept methodologies and results that have been generated via these best practices. Hence, decisions based on these results can happen more quickly.

Second, modeling coordination could facilitate direct comparison of modeling results, identifying issues on which diverse approaches agree and areas of greater uncertainty. Epidemiological modelers might learn from comparison initiatives in modeling of influenza (Centers for Disease Control and Prevention, 2013), dengue (US Department of Commerce), and HIV (HIV modeling consortium); and in other fields such as climate forecasting Intergovernmental Panel on Climate Change, 2010). For epidemiological application, an ensemble approach should preserve methodological diversity to exploit the full range of state-of-the-art modeling methods, but include enough standardization to enable cross-model comparison. Establishing an initial architecture for a coordinated, ensemble effort now could assist the response to EVD, and future public health emergencies.

Perhaps most importantly, outbreak modeling efforts would be much more fruitful if data and analytical results could be made available more quickly to all interested parties (Yozwiak et al., 2015). The publication timelines for academic journals typically will not be consistent with decision-making needs during public health emergencies like the EVD epidemic, where the epidemiological situation was highly dynamic and the usefulness of data and forecasts time-constrained. Establishing mechanisms for modelers without special access to the official epidemiological teams to share interim results would expand the evidence base for response decision-making. Ideally, data should be made available online in machine-readable form to facilitate use in analyses. Modelers and other analysts expended enormous effort during the EVD epidemic transcribing data posted online in pdf documents.

New norms for data-sharing during public health emergencies (World Health Organization, 2015) would remove the most obvious hurdle for model comparison. The current situation where groups either negotiate bilaterally with individual countries or work exclusively with global health and development agencies is understandable, but highly ineffective. The EVD outbreak highlights again – after the 2003 Severe Acute Respiratory Syndrome epidemic and 2009 influenza A (H1N1) pandemic – that an independent, well-resourced global data observatory could greatly facilitate the public health response in many ways, not least of which would be the enablement of rapid, high quality, and easily comparable disease-dynamic studies.

Materials and methods

For this review, we adapted the PRISMA methodology (Moher et al., 2009) to identify quantitative modeling studies of the 2013-present West Africa EVD epidemic. We searched PubMed on September 24, 2015, for publications in English since December 1, 2013, using the term ‘Ebola’ in any field. We reviewed all returned abstracts and selected ones for confirmatory, full-text review that mentioned use of quantitative models to characterize or predict epidemic dynamics or evaluate interventions. We included studies that met this criterion in full-text review.

We excluded studies of clinical prediction models, viral or physiological function models, ecological niche models, animal reservoir models, and publications that did not use data from the 2013-present West Africa EVD epidemic.

For included publications, we recorded the geographic settings, date of most recent EVD data used and date of publication, type of EVD data used, questions the models addressed, modeling approaches, and key results, including estimates of the basic reproduction number (R0) and forecasts of future EVD incidence provided in the main text of the publications. To assess forecast accuracy, we compared predictions of models made under ‘status quo’ assumptions (i.e., without explicit inclusion of additional interventions or behavioral changes) to EVD incidence data subsequently released by the WHO (World Health Organization, 2016), using the WHO figures dated soonest after the forecast target date.

Acknowledgements

We thank the reviewers for excellent comments that improved the manuscript. The views expressed are those of the authors and do not necessarily reflect the views of any part of the US Government.

Funding Statement

No external funding was received for this work.

Additional information

Competing interests

The authors declare that no competing interests exist.

Author contributions

J-PC, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

SR, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

DBG, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article.

Additional files

Supplementary file 1. Overview of publications.

DOI: http://dx.doi.org/10.7554/eLife.09186.011

elife-09186-supp1.zip (1.3KB, zip)
DOI: 10.7554/eLife.09186.011
Supplementary file 2. Overview of models.

DOI: http://dx.doi.org/10.7554/eLife.09186.012

elife-09186-supp2.zip (1.1KB, zip)
DOI: 10.7554/eLife.09186.012
Supplementary file 3. Models estimating R.

DOI: http://dx.doi.org/10.7554/eLife.09186.013

elife-09186-supp3.zip (1.3KB, zip)
DOI: 10.7554/eLife.09186.013
Supplementary file 4. Forecast models.

DOI: http://dx.doi.org/10.7554/eLife.09186.014

elife-09186-supp4.zip (816B, zip)
DOI: 10.7554/eLife.09186.014

References

  1. Agusto FB, Teboh-Ewungkem MI, Gumel AB. Mathematical assessment of the effect of traditional beliefs and customs on the transmission dynamics of the 2014 ebola outbreaks. BMC Medicine. 2015;13:96. doi: 10.1186/s12916-015-0318-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alizon S, Lion S, Murall CL, Abbate JL. Quantifying the epidemic spread of ebola virus (eBOV) in sierra leone using phylodynamics. Virulence. 2014;5:825–827. doi: 10.4161/21505594.2014.976514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Althaus CL, Low N, Musa EO, Shuaib F, Gsteiger S. Ebola virus disease outbreak in nigeria: transmission dynamics and rapid control. Epidemics. 2015;11:80–84. doi: 10.1016/j.epidem.2015.03.001. [DOI] [PubMed] [Google Scholar]
  4. Althaus CL. Estimating the reproduction number of ebola virus (eBOV) during the 2014 outbreak in West Africa. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.91afb5e0f279e7f29e7056095255b288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Althaus CL. Ebola superspreading. The Lancet. Infectious Diseases. 2015;15:507–508. doi: 10.1016/S1473-3099(15)70135-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Baize S, Pannetier D, Oestereich L, Rieger T, Koivogui L, Magassouba N, Soropogui B, Sow MS, Keïta S, De Clerck H, Tiffany A, Dominguez G, Loua M, Traoré A, Kolié M, Malano ER, Heleze E, Bocquin A, Mély S, Raoul H, Caro V, Cadar D, Gabriel M, Pahlmann M, Tappe D, Schmidt-Chanasit J, Impouma B, Diallo AK, Formenty P, Van Herp M, Günther S. Emergence of zaire ebola virus disease in guinea. The New England Journal of Medicine. 2014;371:1418–1425. doi: 10.1056/NEJMoa1404505. [DOI] [PubMed] [Google Scholar]
  7. Barbarossa MV, Dénes A, Kiss G, Nakata Y, Röst G, Vizi Z. Transmission dynamics and final epidemic size of ebola virus disease outbreaks with varying interventions. PloS One. 2015;10:e09186. doi: 10.1371/journal.pone.0131398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bellan SE, Pulliam JR, Pearson CA, Champredon D, Fox SJ, Skrip L, Galvani AP, Gambhir M, Lopman BA, Porco TC, Meyers LA, Dushoff J. Statistical power and validity of ebola vaccine trials in sierra leone: a simulation study of trial design and analysis. The Lancet. Infectious Diseases. 2015;15:703–710. doi: 10.1016/S1473-3099(15)70139-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bellan SE, Pulliam JRC, Dushoff J, Meyers LA. Ebola control: effect of asymptomatic infection and acquired immunity. The Lancet. 2014;384:1499–1500. doi: 10.1016/S0140-6736(14)61839-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bogoch II, Creatore MI, Cetron MS, Brownstein JS, Pesik N, Miniota J, Tam T, Hu W, Nicolucci A, Ahmed S, Yoon JW, Berry I, Hay SI, Anema A, Tatem AJ, MacFadden D, German M, Khan K. Assessment of the potential for international dissemination of ebola virus via commercial air travel during the 2014 west african outbreak. The Lancet. 2015;385:29–35. doi: 10.1016/S0140-6736(14)61828-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Browne C, Gulbudak H, Webb G. Modeling contact tracing in outbreaks with application to ebola. Journal of Theoretical Biology. 2015;384:33–49. doi: 10.1016/j.jtbi.2015.08.004. [DOI] [PubMed] [Google Scholar]
  12. Camacho A, Kucharski A, Aki-Sawyerr Y, White MA, Flasche S, Baguelin M, Pollington T, Carney JR, Glover R, Smout E, Tiffany A, Edmunds WJ, Funk S. Temporal changes in ebola transmission in sierra leone and implications for control requirements: a real-time modelling study. PLoS Currents. 2015;7 doi: 10.1371/currents.outbreaks.406ae55e83ec0b5193e30856b9235ed2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Carroll MW, Matthews DA, Hiscox JA, Elmore MJ, Pollakis G, Rambaut A, Hewson R, García-Dorival I, Bore JA, Koundouno R, Abdellati S, Afrough B, Aiyepada J, Akhilomen P, Asogun D, Atkinson B, Badusche M, Bah A, Bate S, Baumann J, Becker D, Becker-Ziaja B, Bocquin A, Borremans B, Bosworth A, Boettcher JP, Cannas A, Carletti F, Castilletti C, Clark S, Colavita F, Diederich S, Donatus A, Duraffour S, Ehichioya D, Ellerbrok H, Fernandez-Garcia MD, Fizet A, Fleischmann E, Gryseels S, Hermelink A, Hinzmann J, Hopf-Guevara U, Ighodalo Y, Jameson L, Kelterbaum A, Kis Z, Kloth S, Kohl C, Korva M, Kraus A, Kuisma E, Kurth A, Liedigk B, Logue CH, Lüdtke A, Maes P, McCowen J, Mély S, Mertens M, Meschi S, Meyer B, Michel J, Molkenthin P, Muñoz-Fontela C, Muth D, Newman EN, Ngabo D, Oestereich L, Okosun J, Olokor T, Omiunu R, Omomoh E, Pallasch E, Pályi B, Portmann J, Pottage T, Pratt C, Priesnitz S, Quartu S, Rappe J, Repits J, Richter M, Rudolf M, Sachse A, Schmidt KM, Schudt G, Strecker T, Thom R, Thomas S, Tobin E, Tolley H, Trautner J, Vermoesen T, Vitoriano I, Wagner M, Wolff S, Yue C, Capobianchi MR, Kretschmer B, Hall Y, Kenny JG, Rickett NY, Dudas G, Coltart CE, Kerber R, Steer D, Wright C, Senyah F, Keita S, Drury P, Diallo B, de Clerck H, Van Herp M, Sprecher A, Traore A, Diakite M, Konde MK, Koivogui L, Magassouba N, Avšič-Županc T, Nitsche A, Strasser M, Ippolito G, Becker S, Stoecker K, Gabriel M, Raoul H, Di Caro A, Wölfel R, Formenty P, Günther S. Temporal and spatial analysis of the 2014–2015 ebola virus outbreak in west africa. Nature. 2015;524:97–101. doi: 10.1038/nature14594. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Centers for Disease Control and Prevention Announcement of requirements and registration for the predict the influenza season challenge. https://www.federalregister.gov/articles/2013/11/25/2013-28198/announcement-of-requirements-and-registration-for-the-predict-the-influenza-season-challenge 2013
  15. Chowell D, Castillo-Chavez C, Krishna S, Qiu X, Anderson KS. Modelling the effect of early detection of ebola. The Lancet. Infectious Diseases. 2015;15:148–149. doi: 10.1016/S1473-3099(14)71084-9. [DOI] [PubMed] [Google Scholar]
  16. Chowell G, Simonsen L, Viboud C, Kuang Y. Is west africa approaching a catastrophic phase or is the 2014 ebola epidemic slowing down? different models yield different answers for liberia. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.b4690859d91684da963dc40e00f3da81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Chowell G, Viboud C, Hyman JM, Simonsen L. The western africa ebola virus disease epidemic exhibits both global exponential and local polynomial growth rates. PLoS Currents. 2014;7 doi: 10.1371/currents.outbreaks.8b55f4bad99ac5c5db3663e916803261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Cleaton JM, Viboud C, Simonsen L, Hurtado AM, Chowell G. Characterizing ebola transmission patterns based on internet news reports. Clinical Infectious Diseases. 2015:civ748. doi: 10.1093/cid/civ748. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Cooper BS, Boni MF, Pan-ngum W, Day NP, Horby PW, Olliaro P, Lang T, White NJ, White LJ, Whitehead J. Evaluating clinical trial designs for investigational treatments of ebola virus disease. PLoS Medicine. 2015;12:e09186. doi: 10.1371/journal.pmed.1001815. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Cope RC, Cassey P, Hugo GJ, Ross JV. Assessment of the risk of ebola importation to australia. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.aa0375fd48a92c7c9422aa543a88711f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Dong F, Xu D, Wang Z, Dong M. Evaluation of ebola spreading in west africa and decision of optimal medicine delivery strategies based on mathematical models. Infection, Genetics and Evolution. 2015;36:35–40. doi: 10.1016/j.meegid.2015.09.003. [DOI] [PubMed] [Google Scholar]
  22. Drake JM, Kaul RB, Alexander LW, O'Regan SM, Kramer AM, Pulliam JT, Ferrari MJ, Park AW. Ebola cases and health system demand in liberia. PLoS Biology. 2015;13:e09186. doi: 10.1371/journal.pbio.1002056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Dudas G, Rambaut A. Phylogenetic analysis of guinea 2014 EBOV ebolavirus outbreak. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.84eefe5ce43ec9dc0bf0670f7b8b417d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Fasina FO, Shittu A, Lazarus D, Tomori O, Simonsen L, Viboud C, Chowell G. Transmission dynamics and control of ebola virus disease outbreak in nigeria, july to september 2014. Euro Surveillance. 2014;19:20920. doi: 10.2807/1560-7917.ES2014.19.40.20920. [DOI] [PubMed] [Google Scholar]
  25. Fast SM, Mekaru S, Brownstein JS, Postlethwaite TA, Markuzon N. The role of social mobilization in controlling ebola virus in lofa county, liberia. PLoS Currents. 2015;7 doi: 10.1371/currents.outbreaks.c3576278c66b22ab54a25e122fcdbec1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Faye O, Boëlle PY, Heleze E, Faye O, Loucoubar C, Magassouba N, Soropogui B, Keita S, Gakou T, Bah el HI, Koivogui L, Sall AA, Cauchemez S. Chains of transmission and control of ebola virus disease in conakry, guinea, in 2014: an observational study. The Lancet. Infectious Diseases. 2015;15:320–326. doi: 10.1016/S1473-3099(14)71075-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Fisman D, Khoo E, Tuite A. Early epidemic dynamics of the west african 2014 ebola outbreak: estimates derived with a simple two-parameter model. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.89c0d3783f36958d96ebbae97348d571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Fisman D, Tuite A. Projected impact of vaccination timing and dose availability on the course of the 2014 west african ebola epidemic. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.06e00d0546ad426fed83ff24a1d4c4cc. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Gire SK, Goba A, Andersen KG, Sealfon RS, Park DJ, Kanneh L, Jalloh S, Momoh M, Fullah M, Dudas G, Wohl S, Moses LM, Yozwiak NL, Winnicki S, Matranga CB, Malboeuf CM, Qu J, Gladden AD, Schaffner SF, Yang X, Jiang PP, Nekoui M, Colubri A, Coomber MR, Fonnie M, Moigboi A, Gbakie M, Kamara FK, Tucker V, Konuwa E, Saffa S, Sellu J, Jalloh AA, Kovoma A, Koninga J, Mustapha I, Kargbo K, Foday M, Yillah M, Kanneh F, Robert W, Massally JL, Chapman SB, Bochicchio J, Murphy C, Nusbaum C, Young S, Birren BW, Grant DS, Scheiffelin JS, Lander ES, Happi C, Gevao SM, Gnirke A, Rambaut A, Garry RF, Khan SH, Sabeti PC. Genomic surveillance elucidates ebola virus origin and transmission during the 2014 outbreak. Science. 2014;345:1369–1372. doi: 10.1126/science.1259657. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Gomes MF, Pastore Y Piontti A, Rossi L, Chao D, Longini I, Halloran ME, Vespignani A. Assessing the international spreading risk associated with the 2014 west african ebola outbreak. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.cd818f63d40e24aef769dda7df9e0da5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. HIV modeling consortium Available: http://www.hivmodelling.org/
  32. Hoenen T, Safronetz D, Groseth A, Wollenberg KR, Koita OA, Diarra B, Fall IS, Haidara FC, Diallo F, Sanogo M, Sarro YS, Kone A, Togo AC, Traore A, Kodio M, Dosseh A, Rosenke K, de Wit E, Feldmann F, Ebihara H, Munster VJ, Zoon KC, Feldmann H, Sow S. Virology. mutation rate and genotype variation of ebola virus from mali case sequences. Science. 2015;348:117–119. doi: 10.1126/science.aaa5646. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Intergovernmental Panel on Climate Change. IPCC expert meeting on assessing and combining multi model climate projections. Good practice guidance paper on assessing and combining multi model climate projections https://www.ipcc-wg1.unibe.ch/guidancepaper/IPCC_EM_MME_GoodPracticeGuidancePaper.pdf 2010
  34. Khan A, Naveed M, Dur-E-Ahmad M, Imran M. Estimating the basic reproductive ratio for the ebola outbreak in liberia and sierra leone. Infectious Diseases of Poverty. 2015;4:13. doi: 10.1186/s40249-015-0043-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. King AA, Domenech de Celles M, Magpantay FMG, Rohani P. Avoidable errors in the modelling of outbreaks of emerging pathogens, with special reference to ebola. Proceedings of the Royal Society B: Biological Sciences. 2015;282:20150347. doi: 10.1098/rspb.2015.0347. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kiskowski MA. A three-scale network model for the early growth dynamics of 2014 west africa ebola epidemic. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.c6efe8274dc55274f05cbcb62bbe6070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kucharski AJ, Camacho A, Checchi F, Waldman R, Grais RF, Cabrol JC, Briand S, Baguelin M, Flasche S, Funk S, Edmunds WJ. Evaluation of the benefits and risks of introducing ebola community care centers, sierra leone. Emerging Infectious Diseases. 2015;21:393–399. doi: 10.3201/eid2103.141892. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Kugelman JR, Wiley MR, Mate S, Ladner JT, Beitzel B, Fakoli L. Monitoring of ebola virus makona evolution through establishment of advanced genomic capability in liberia. Emerging Infectious Diseases. 2015;21:1135–1143. doi: 10.3201/eid2107.150522. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Lewnard JA, Ndeffo Mbah ML, Alfaro-Murillo JA, Altice FL, Bawo L, Nyenswah TG, Galvani AP. Dynamics and control of ebola virus transmission in montserrado, liberia: a mathematical modelling analysis. The Lancet. Infectious Diseases. 2014;14:1189–1195. doi: 10.1016/S1473-3099(14)70995-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Majumder MS, Rivers C, Lofgren E, Fisman D. Estimation of MERS-coronavirus reproductive number and case fatality rate for the spring 2014 saudi arabia outbreak: insights from publicly available data. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.98d2f8f3382d84f390736cd5f5fe133c. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Meltzer MI, Atkins CY, Santibanez S, Knust B, Petersen BW, Ervin ED, Nichol ST, Damon IK, Washington ML. Estimating the future number of cases in the ebola epidemic--liberia and sierra leone, 2014-2015. Morbidity and Mortality Weekly Report. Surveillance Summaries (Washington, D.C. : 2002) 2014;63 Suppl 3:1–14. [PubMed] [Google Scholar]
  42. Merler S, Ajelli M, Fumanelli L, Gomes MF, Piontti AP, Rossi L, Chao DL, Longini IM, Halloran ME, Vespignani A. Spatiotemporal spread of the 2014 outbreak of ebola virus disease in liberia and the effectiveness of non-pharmaceutical interventions: a computational modelling analysis. The Lancet. Infectious Diseases. 2015;15:204–211. doi: 10.1016/S1473-3099(14)71074-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine. 2009;6:e09186. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Nadhem S, Nejib HD. The ebola contagion and forecasting virus: evidence from four african countries. Health Economics Review. 2015;5:47. doi: 10.1186/s13561-015-0047-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. National Institutes of Health Fogarty international center. Research and policy for infectious disease dynamics. http://www.fic.nih.gov/about/staff/pages/epidemiology-population.aspx 2016a
  46. National Institutes of Health National Institute of General Medical Sciences. Models of Infectious Disease Agent Study (MIDAS) http://www.nigms.nih.gov/Research/SpecificAreas/MIDAS/Pages/default.aspx 2016b
  47. Nishiura H, Chowell G. Early transmission dynamics of ebola virus disease (eVD), West Africa, march to august 2014. Euro Surveillance : Bulletin Européen Sur Les Maladies Transmissibles = European Communicable Disease Bulletin. 2014;19:20894. doi: 10.2807/1560-7917.ES2014.19.36.20894. [DOI] [PubMed] [Google Scholar]
  48. Pandey A, Atkins KE, Medlock J, Wenzel N, Townsend JP, Childs JE, Nyenswah TG, Ndeffo-Mbah ML, Galvani AP. Strategies for containing ebola in west africa. Science. 2014;346:991–995. doi: 10.1126/science.1260612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Park DJ, Dudas G, Wohl S, Goba A, Whitmer SL, Andersen KG, Sealfon RS, Ladner JT, Kugelman JR, Matranga CB, Winnicki SM, Qu J, Gire SK, Gladden-Young A, Jalloh S, Nosamiefan D, Yozwiak NL, Moses LM, Jiang PP, Lin AE, Schaffner SF, Bird B, Towner J, Mamoh M, Gbakie M, Kanneh L, Kargbo D, Massally JL, Kamara FK, Konuwa E, Sellu J, Jalloh AA, Mustapha I, Foday M, Yillah M, Erickson BR, Sealy T, Blau D, Paddock C, Brault A, Amman B, Basile J, Bearden S, Belser J, Bergeron E, Campbell S, Chakrabarti A, Dodd K, Flint M, Gibbons A, Goodman C, Klena J, McMullan L, Morgan L, Russell B, Salzer J, Sanchez A, Wang D, Jungreis I, Tomkins-Tinch C, Kislyuk A, Lin MF, Chapman S, MacInnis B, Matthews A, Bochicchio J, Hensley LE, Kuhn JH, Nusbaum C, Schieffelin JS, Birren BW, Forget M, Nichol ST, Palacios GF, Ndiaye D, Happi C, Gevao SM, Vandi MA, Kargbo B, Holmes EC, Bedford T, Gnirke A, Ströher U, Rambaut A, Garry RF, Sabeti PC. Ebola virus epidemiology, transmission, and evolution during seven months in sierra leone. Cell. 2015;161:1516–1526. doi: 10.1016/j.cell.2015.06.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Poletto C, Gomes MF, Pastore y Piontti A, Rossi L, Bioglio L, Chao DL, Longini IM, Halloran ME, Colizza V, Vespignani A. Assessing the impact of travel restrictions on international spread of the 2014 west african ebola epidemic. Euro Surveillance. 2014;19:20936. doi: 10.2807/1560-7917.ES2014.19.42.20936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Rainisch G, Asher J, George D, Clay M, Smith TL, Kosmos C, Shankar M, Washington ML, Gambhir M, Atkins C, Hatchett R, Lant T, Meltzer MI. Estimating ebola treatment needs, united states. Emerging Infectious Diseases. 2015;21:1273–1275. doi: 10.3201/eid2107.150286. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Rainisch G, Shankar M, Wellman M, Merlin T, Meltzer MI. Regional spread of ebola virus, west africa, 2014. Emerging Infectious Diseases. 2015;21:444–447. doi: 10.3201/eid2103.141845. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Read JM, Diggle PJ, Chirombo J, Solomon T, Baylis M. Effectiveness of screening for ebola at airports. The Lancet. 2015;385:23–24. doi: 10.1016/S0140-6736(14)61894-8. [DOI] [PubMed] [Google Scholar]
  54. Rivers C, GitHub https://github.com/cmrivers/ebola
  55. Rivers CM, Lofgren ET, Marathe M, Eubank S, Lewis BL. Modeling the impact of interventions on an epidemic of ebola in sierra leone and liberia. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.4d41fe5d6c05e9df30ddce33c66d084c. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Robinson R. Statement before the US senate committee on health, education, labor, and pensions. joint full committee hearing - ebola in West Africa: a global challenge and public health threat. http://www.help.senate.gov/hearings/hearing/?id=9522d319-5056-a032-5290-5a537121c053
  57. Scarpino SV, Iamarino A, Wells C, Yamin D, Ndeffo-Mbah M, Wenzel NS, Fox SJ, Nyenswah T, Altice FL, Galvani AP, Meyers LA, Townsend JP. Epidemiological and viral genomic sequence analysis of the 2014 ebola outbreak reveals clustered transmission. Clinical Infectious Diseases. 2015;60 doi: 10.1093/cid/ciu1131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Shaman J, Yang W, Kandula S. Inference and forecast of the current west african ebola outbreak in guinea, sierra leone and liberia. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.3408774290b1a0f2dd7cae877c8b8ff6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Siettos C, Anastassopoulou C, Russo L, Grigoras C, Mylonakis E. Modeling the 2014 ebola virus epidemic – agent-based simulations, temporal analysis and future predictions for liberia and sierra leone. PLoS Currents. 2015;7 doi: 10.1371/currents.outbreaks.8d5984114855fc425e699e1a18cdc6c9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Simon-Loriere E, Faye O, Faye O, Koivogui L, Magassouba N, Keita S, Thiberge JM, Diancourt L, Bouchier C, Vandenbogaert M, Caro V, Fall G, Buchmann JP, Matranga CB, Sabeti PC, Manuguerra JC, Holmes EC, Sall AA. Distinct lineages of ebola virus in guinea during the 2014 West African epidemic. Nature. 2015;524:102–104. doi: 10.1038/nature14612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Stadler T, Kühnert D, Rasmussen DA, du Plessis L. Insights into the early epidemic spread of ebola in sierra leone provided by viral sequence data. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.02bc6d927ecee7bbd33532ec8ba6a25f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Tong Y-G, Shi W-F, Liu D, Qian J, Liang L, Bo X-C, Liu J, Ren H-G, Fan H, Ni M, Sun Y, Jin Y, Teng Y, Li Z, Kargbo D, Dafae F, Kanu A, Chen C-C, Lan Z-H, Jiang H, Luo Y, Lu H-J, Zhang X-G, Yang F, Hu Y, Cao Y-X, Deng Y-Q, Su H-X, Sun Y, Liu W-S, Wang Z, Wang C-Y, Bu Z-Y, Guo Z-D, Zhang L-B, Nie W-M, Bai C-Q, Sun C-H, An X-P, Xu P-S, Zhang X-L-L, Huang Y, Mi Z-Q, Yu D, Yao H-W, Feng Y, Xia Z-P, Zheng X-X, Yang S-T, Lu B, Jiang J-F, Kargbo B, He F-C, Gao GF, Cao W-C, China Mobile Laboratory Testing Team in Sierra Leone Genetic diversity and evolutionary dynamics of ebola virus in sierra leone. Nature. 2015;524:93–96. doi: 10.1038/nature14490. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Toth DJ, Gundlapalli AV, Khader K, Pettey WB, Rubin MA, Adler FR, Samore MH. Estimates of outbreak risk from new introductions of ebola with immediate and delayed transmission control. Emerging Infectious Diseases. 2015;21:1402–1408. doi: 10.3201/eid2108.150170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Towers S, Patterson-Lomba O, Castillo-Chavez C. Temporal variations in the effective reproduction number of the 2014 west africa ebola outbreak. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.9e4c4294ec8ce1adad283172b16bc908. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. US Department of Commerce Dengue forecasting. http://dengueforecasting.noaa.gov/
  66. Valdez LD, Aragão Rêgo HH, Stanley HE, Braunstein LA. Predicting the extinction of ebola spreading in liberia due to mitigation strategies. Scientific Reports. 2015;5:12172. doi: 10.1038/srep12172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Volz E, Pond S. Phylodynamic analysis of ebola virus in the 2014 sierra leone epidemic. PLoS Currents. 2014;6 doi: 10.1371/currents.outbreaks.6f7025f1271821d4c815385b08f5f80e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Wang XS, Zhong L. Ebola outbreak in west africa: real-time estimation and multiple-wave prediction. Mathematical Biosciences and Engineering. 2015;12:1055–1063. doi: 10.3934/mbe.2015.12.1055. [DOI] [PubMed] [Google Scholar]
  69. Webb G, Browne C, Huo X, Seydi O, Seydi M, Magal P. A model of the 2014 ebola epidemic in west africa with contact tracing. PLoS Currents. 2015;7 doi: 10.1371/currents.outbreaks.846b2a31ef37018b7d1126a9c8adf22a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Weitz JS, Dushoff J. Modeling post-death transmission of ebola: challenges for inference and opportunities for control. Scientific Reports. 2015;5:8751. doi: 10.1038/srep08751. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. White RA, MacDonald E, de Blasio BF, Nygård K, Vold L, Røttingen J-A. Projected treatment capacity needs in sierra leone. PLoS Currents. 2014 doi: 10.1371/currents.outbreaks.3c3477556808e44cf41d2511b21dc29f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. WHO Ebola Response Team Ebola virus disease in West Africa--the first 9 months of the epidemic and forward projections. The New England Journal of Medicine. 2014;371:1481–1495. doi: 10.1056/NEJMoa1411100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. WHO Ebola Response Team West African ebola epidemic after one year--slowing but not yet under control. The New England Journal of Medicine. 2015;372:584–587. doi: 10.1056/NEJMc1414992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. World Health Organization Developing global norms for sharing data and results during public health emergencies. http://www.who.int/medicines/ebola-treatment/data-sharing_phe/en/ 2015 doi: 10.1371/journal.pmed.1001935. [DOI] [PMC free article] [PubMed]
  75. World Health Organization Situation reports: ebola response roadmap. http://www.who.int/csr/disease/ebola/situation-reports/en/ 2016
  76. Xia ZQ, Wang SF, Li SL, Huang LY, Zhang WY, Sun GQ, Gai ZT, Jin Z. Modeling the transmission dynamics of ebola virus disease in liberia. Scientific Reports. 2015;5:13857. doi: 10.1038/srep13857. [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Yamin D, Gertler S, Ndeffo-Mbah ML, Skrip LA, Fallah M, Nyenswah TG, Altice FL, Galvani AP. Effect of ebola progression on transmission and control in liberia. Annals of Internal Medicine. 2015;162:11–17. doi: 10.7326/M14-2255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  78. Yozwiak NL, Schaffner SF, Sabeti PC. Data sharing: make outbreak research open access. Nature. 2015;518:477–479. doi: 10.1038/518477a. [DOI] [PubMed] [Google Scholar]
  79. Zinszer K, Morrison K, Anema A, Majumder MS, Brownstein JS. The velocity of ebola spread in parts of west africa. The Lancet. Infectious Diseases. 2015;15:1005–1007. doi: 10.1016/S1473-3099(15)00234-0. [DOI] [PubMed] [Google Scholar]
eLife. 2015 Dec 8;4:e09186. doi: 10.7554/eLife.09186.015

Decision letter

Editor: Mark Jit1

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your work entitled "Improving epidemiological modeling coordination for Ebola and other public health emergencies" for peer review at eLife. Your submission has been favorably evaluated by Prabhat Jha (Senior editor), and three reviewers, one of whom (Mark Jit) is a member of our Board of Reviewing Editors. Another of the reviewers, Marco Ajelli, has also agreed to reveal their identity.

The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission.

The reviewers all agreed that this is a timely, comprehensive and useful review of modelling studies that have described and/or forecasted the Ebola epidemic in West Africa. We all enjoyed reading the paper, and agreed that it made some important points about good modelling practice in general. In particular, we agree that data sharing and complete access to best available epidemiological data (especially patient databases) for all modelling groups is of paramount importance in the case of future outbreaks of infectious diseases.

We believe that the way the information and discussion was presented could be improved, and by doing so would enhance the influence that the paper may have. In particular:

1) More detail could be provided on how Ebola modelling was actually used to support decision making both internationally and for individual national governments. Many of the recommendations in the Discussion are predicated on the premise that the modelling results were actually useful to inform policy (or if they weren't, could be made useful by improving the modelling process). Some historical discussion on what happened in practice would help here. Currently only a single US-centric reference is provided.

2) The paper needs a proper Materials and methods section providing details about the search terms/strategy, inclusion/exclusion criteria and information extracted from the papers. Given that this is a review, compliance with PRISMA guidelines would normally be expected. If a non-standard approach to searching was used, this needs to be justified and the method explained.

3) Each section of the paper seems to contain a mixture of (i) data directly extracted from the reviewed papers, (ii) direct inferences from these data, (iii) more general discussion points around the inferences and (iv) broader discussion (e.g. recommendations for standardisation and funding for future outbreak response). While each section is helpful, the way the paper is written makes it difficult to distinguish the provenance of the different points made. For instance, there are statements like "Modelers welcomed the release of these data, but expended considerable effort curating them for analysis", "accurate epidemic forecasting for Ebola proved to be quite challenging, especially for long time horizons, because the behavior of people infected and at risk was unpredictable", "incomplete information from the field can make it difficult to determine if failure to end the epidemic reflects sub-optimal implementation of effective measures, or over-optimistic assumptions about their effectiveness" where it is unclear whether these are statements drawn from the reviewed papers, or based on the authors' own experiences.

Using a traditional separation into Results and Discussion may also help this as most scientists are used to the idea of drawing immediate inferences from data, and then suggesting broader implications of these inferences. The Discussion section may be lengthy and structured if necessary. The broader discussion around funding etc. which is not really an extrapolation from the data (however welcome the points may be) should probably be given a section of its own.

4) Table 2 shows a very large variation in the baseline scenario forecast, spanning orders of magnitude. The authors seem to suggest that this stems from different assumptions used about epidemic growth forecasting e.g. random mixing, incidence decay or spatial/sociodemographic structuring. It would be useful if you could explicitly structure the presentation of results to show whether these different assumptions had an effect on the outbreak size. This should at minimum be done in the visual presentation but if some sort of meta-analysis/meta-regression could help that would be even better.

5) Additional information about the spatial structure (if any) of the models reviewed would be useful. Spatial structure in these models ranges from homogeneous (non-spatial models) to detailed agent-based models such as the one developed by Merler et al. Lancet ID.

6) Some models employed single streams of data (e.g., all case counts by country) while other models employed multiple data streams (e.g., all cases and cases among health-care workers such as the multi-type branching process model by Drake et al.). This would be useful to highlight.

7) The paper makes the point that there are differences between the results of various modelling groups because of different methodologies used. However, in some cases modelling methodology has been improperly used, and this would be useful to highlight. In particular the following areas could be given attention:

a) Parameter estimation. In Box 1, you have underlined that the different models analyzed in this review have a remarkably different number of "unknown" parameters to be estimated. This represents a very important topic that could be given more space in this manuscript). For example:

A clear distinction should be made between estimated and imputed parameters. For instance the number of estimated parameters in Pandey et al. (2014) is 4, while the other parameters are somehow imputed from the literature. That said, still the 4 unknown parameters estimated in Pandey et al. (2014) are not univocally identifiable by using as input the dataset considered in that work.

In Rivers et al. (2014) about 10 (surely not all unequivocally identifiable) parameters have been estimated on the basis of case counts only.

b) Uncertainty. A good modelling practice (which is not always followed in the reviewed papers) is the assessment of the uncertainty due to i) the stochasticity of the infection transmission process, and ii) the uncertainty in parameter estimates. In particular, as regards the uncertainty in parameter estimates, methods providing estimates of the parameters distribution (e.g., MCMC) should be preferred with respect to methods providing point estimate of the parameter values only (e.g., MSE, ML). This would avoid (or at least limit) the spread in the community of biased findings never confirmed on the field or by subsequent studies (e.g., the estimate that the main driver of the Ebola epidemic was the transmission during traditional burials found by Pandey et al. (2014) which was based on a point estimate made through a MSE fitting procedure).

c) Model validation. Another major point is that most (though not all) the reviewed modelling studies lack in model validation against other epidemiologically relevant quantities (e.g., ETU admissions, number of individuals included in contact tracing). Validation against other quantities is a key indicator to understand whether the model is able to explain the reason of an observed pattern of infection spread or it is just able to reproduce case counts "by chance". In fact, although it is welcome to show that the model is in agreement with the next few incidence points after stopping model calibration, modelers should not limit model validation to that.

8) It would also be good to clarify that the reviewed modelling studies can be divided in two categories: the ones aimed at explaining the observed pattern, for instance by accounting for actual data on intervention measures, and those that do not provide any explanation for the observed pattern (e.g., the model in Fisman, Khoo and Tuite (2014)). In fact, the point here is not to fit some data points, but to understand the mechanisms behind the observed infection spread as their knowledge is key for predictions and preparedness.

9) The importance for modelling groups of having access to online epidemiological data in an easy to export format (e.g., CSV file) could be highlighted. In fact, the Ministries of Health of the three most affected countries provided PDF files only, some of which were images, thus implying remarkable efforts by modelers to readily use such datasets. Moreover, the case of Guinea was even worse as, at the beginning of the epidemic, there was not an official website of the Guinean Ministry of Health with their reports uploaded.

eLife. 2015 Dec 8;4:e09186. doi: 10.7554/eLife.09186.016

Author response

1) More detail could be provided on how Ebola modelling was actually used to support decision making both internationally and for individual national governments. Many of the recommendations in the Discussion are predicated on the premise that the modelling results were actually useful to inform policy (or if they weren't, could be made useful by improving the modelling process). Some historical discussion on what happened in practice would help here. Currently only a single US-centric reference is provided.

We recast the manuscript as more of a standard scoping literature review, with less emphasis on anecdotal evidence of the practical utility of modeling during the outbreak. We mostly have only our personal experiences to draw on for such evidence; the revision draws much more heavily on the peer-reviewed literature (please see response to Comment 2 below).

2) The paper needs a proper Materials and methods section providing details about the search terms/strategy, inclusion/exclusion criteria and information extracted from the papers. Given that this is a review, compliance with PRISMA guidelines would normally be expected. If a non-standard approach to searching was used, this needs to be justified and the method explained.

We revised the manuscript as a PRISMA-compliant literature review.

3) Each section of the paper seems to contain a mixture of (i) data directly extracted from the reviewed papers, (ii) direct inferences from these data, (iii) more general discussion points around the inferences and (iv) broader discussion (e.g. recommendations for standardisation and funding for future outbreak response). While each section is helpful, the way the paper is written makes it difficult to distinguish the provenance of the different points made. For instance, there are statements like "Modelers welcomed the release of these data, but expended considerable effort curating them for analysis", "accurate epidemic forecasting for Ebola proved to be quite challenging, especially for long time horizons, because the behavior of people infected and at risk was unpredictable", "incomplete information from the field can make it difficult to determine if failure to end the epidemic reflects sub-optimal implementation of effective measures, or over-optimistic assumptions about their effectiveness" where it is unclear whether these are statements drawn from the reviewed papers, or based on the authors' own experiences. Using a traditional separation into Results and Discussion may also help this as most scientists are used to the idea of drawing immediate inferences from data, and then suggesting broader implications of these inferences. The Discussion section may be lengthy and structured if necessary. The broader discussion around funding etc. which is not really an extrapolation from the data (however welcome the points may be) should probably be given a section of its own.

We agree that the original organization was non-standard and confusing. The revision is structured as a standard research paper, with Introduction, Materials and methods, Results, and Discussion. The broader implications and recommendations are included in the Discussion.

4) Table 2 shows a very large variation in the baseline scenario forecast, spanning orders of magnitude. The authors seem to suggest that this stems from different assumptions used about epidemic growth forecasting e.g. random mixing, incidence decay or spatial/sociodemographic structuring. It would be useful if you could explicitly structure the presentation of results to show whether these different assumptions had an effect on the outbreak size. This should at minimum be done in the visual presentation but if some sort of meta-analysis/meta-regression could help that would be even better.

We replaced the table with a figure that identifies several factors that may have influenced forecasting results in a clearer and more compelling way. Figure 6 shows the association between more accurate forecasts and earlier forecasts, shorter time horizons, and approaches that constrained predicted epidemic growth (for which we suggest the term “dampening”). We also performed a regression to quantify these effects, reported in the Results.

5) Additional information about the spatial structure (if any) of the models reviewed would be useful. Spatial structure in these models ranges from homogeneous (non-spatial models) to detailed agent-based models such as the one developed by Merler et al. Lancet ID.

We identified modeling studies that incorporated spatial data (see Results, subsection “Data sources”).

6) Some models employed single streams of data (e.g., all case counts by country) while other models employed multiple data streams (e.g., all cases and cases among health-care workers such as the multi-type branching process model by Drake et al.). This would be useful to highlight.

We added two paragraphs in the Results section on sources of data (Results, subsection “Data sources”) and hope this addresses the spirit of the excellent comment, which we interpret as a call for more thorough characterization of the types of model input data.

7) The paper makes the point that there are differences between the results of various modelling groups because of different methodologies used. However, in some cases modelling methodology has been improperly used, and this would be useful to highlight. In particular the following areas could be given attention: a) Parameter estimation. In Box 1, you have underlined that the different models analyzed in this review have a remarkably different number of "unknown" parameters to be estimated. This represents a very important topic that could be given more space in this manuscript). For example: A clear distinction should be made between estimated and imputed parameters. For instance the number of estimated parameters in Pandey et al. (2014) is 4, while the other parameters are somehow imputed from the literature. That said, still the 4 unknown parameters estimated in Pandey et al. (2014) are not univocally identifiable by using as input the dataset considered in that work. In Rivers et al. (2014) about 10 (surely not all unequivocally identifiable) parameters have been estimated on the basis of case counts only.

We provided the number of parameters for models estimating R, and distinguished between estimated and imputed values (Supplementary file 3).

b) Uncertainty. A good modelling practice (which is not always followed in the reviewed papers) is the assessment of the uncertainty due to i) the stochasticity of the infection transmission process, and ii) the uncertainty in parameter estimates. In particular, as regards the uncertainty in parameter estimates, methods providing estimates of the parameters distribution (e.g., MCMC) should be preferred with respect to methods providing point estimate of the parameter values only (e.g., MSE, ML). This would avoid (or at least limit) the spread in the community of biased findings never confirmed on the field or by subsequent studies (e.g., the estimate that the main driver of the Ebola epidemic was the transmission during traditional burials found by Pandey et al. (2014) which was based on a point estimate made through a MSE fitting procedure).

We added a detailed analysis of how use of cumulative versus disaggregated input data, and deterministic versus stochastic approaches to parameter estimation, may have influenced the reported uncertainty of R estimates (see Results, subsection “Modeling results: R and forecasts” as well as Figure 4 and Figure 5).

c) Model validation. Another major point is that most (though not all) the reviewed modelling studies lack in model validation against other epidemiologically relevant quantities (e.g., ETU admissions, number of individuals included in contact tracing). Validation against other quantities is a key indicator to understand whether the model is able to explain the reason of an observed pattern of infection spread or it is just able to reproduce case counts "by chance". In fact, although it is welcome to show that the model is in agreement with the next few incidence points after stopping model calibration, modelers should not limit model validation to that.

We added this excellent point to the Discussion (paragraph four):

“Validation of incidence forecasts against other relevant data, such as hospital admissions and contacts identified, also could provide evidence that the assumptions are sound.”

8) It would also be good to clarify that the reviewed modelling studies can be divided in two categories: the ones aimed at explaining the observed pattern, for instance by accounting for actual data on intervention measures, and those that do not provide any explanation for the observed pattern (e.g., the model in Fisman, Khoo and Tuite (2014)). In fact, the point here is not to fit some data points, but to understand the mechanisms behind the observed infection spread as their knowledge is key for predictions and preparedness.

We added this distinction to our characterization of the models (see Results, subsection “Overview of modeling applications”):

“Of the 125 models reported across the studies, 74% included mechanistic assumptions about disease transmission (e.g., compartmental, agent-based, or phylogenetic models), while 26% were purely phenomenological (Supplementary file 2).”

9) The importance for modelling groups of having access to online epidemiological data in an easy to export format (e.g., CSV file) could be highlighted. In fact, the Ministries of Health of the three most affected countries provided PDF files only, some of which were images, thus implying remarkable efforts by modelers to readily use such datasets. Moreover, the case of Guinea was even worse as, at the beginning of the epidemic, there was not an official website of the Guinean Ministry of Health with their reports uploaded.

We’re grateful for the chance to address this very important point, and added the recommendation to the Discussion (paragraph seven):

“Ideally, data should be made available online in machine-readable form to facilitate use in analyses. Modelers and other analysts expended enormous effort during the EVD epidemic transcribing data posted online in PDF documents.”

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Supplementary file 1. Overview of publications.

    DOI: http://dx.doi.org/10.7554/eLife.09186.011

    elife-09186-supp1.zip (1.3KB, zip)
    DOI: 10.7554/eLife.09186.011
    Supplementary file 2. Overview of models.

    DOI: http://dx.doi.org/10.7554/eLife.09186.012

    elife-09186-supp2.zip (1.1KB, zip)
    DOI: 10.7554/eLife.09186.012
    Supplementary file 3. Models estimating R.

    DOI: http://dx.doi.org/10.7554/eLife.09186.013

    elife-09186-supp3.zip (1.3KB, zip)
    DOI: 10.7554/eLife.09186.013
    Supplementary file 4. Forecast models.

    DOI: http://dx.doi.org/10.7554/eLife.09186.014

    elife-09186-supp4.zip (816B, zip)
    DOI: 10.7554/eLife.09186.014

    Articles from eLife are provided here courtesy of eLife Sciences Publications, Ltd

    RESOURCES