Exploring uncertainty and risk in the accelerated response to a COVID-19 vaccine: Perspective from the pharmaceutical industry

Highlights

• •Understanding uncertainty in the COVID-19 pandemic is a priority for the industry.
• •Policy and population behavioral affect risk management in the short/mid-term.
• •Information around natural history unknowns will guide long-term risk management.
• •Our framework allows risk management and risk-sharing strategies discussions.

1. Introduction

A series of pneumonia cases of unknown origin were reported to the World Health Organization (WHO) at the end of 2019 [1]. Since then Corona Virus Disease (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become a pandemic with over 33 million cases and 1,000,000 deaths reported worldwide within the span of nine months despite unprecedented control measures [2]. Mitigation measures and fiscal policies that include social distancing, border closures, shelter-in-place, widespread testing, active contact tracing, and macro-economic stimulus packages have been implemented to various degrees in most countries. These restrictions have seriously impacted economies globally [3] and it is uncertain how countries will react to a persisting transmission risk. Access to diagnostics, treatments, and vaccines produced and accessible at scale will be critical to control the disease, save lives, and restore economic growth.

In this global emergency, we have seen innovative and cooperative efforts across sectors to accelerate the repurposing of existing drugs (i.e. drugs already in use but with a different indication than COVID-19), large scale production of diagnostics and serology tests, and development of new treatments and vaccines [4]. Notable examples are the coordination of large scale multi-arm clinical trials by the World Health Organization (WHO) for existing drugs, and the exploration of modern vaccine platforms such as cell-expressed protein subunit vaccines and mRNA vaccines [5], [6]. Encouraging partnerships have also emerged between biopharmaceutical industries, non-governmental organizations, foundations, and governments to advance vaccine technology development and access [7], [8], [9], [10]. Despite this level of initial activities, there is an intrinsic level of uncertainty related to the evolving epidemiology of an emergent disease and the impact of the policy response that present a risk for research and development (R&D) programs and the planning of future manufacturing capacity, as previously seen in the case of SARS and Zika epidemics where there were a substantial decrease in cases before vaccine was finalized [11].

Disease transmission and statistical models have provided a framework to investigate the initial dynamics of COVID-19 transmission and predict the possible short-term future of the pandemic. The first-generation modeling analyses have been invaluable in rapidly guiding global and national policy on the possible impact of mitigation interventions. While all predictions are, by nature, associated with uncertainty, the timeframe of such predictions is directly correlated to the extent of uncertainty. At this juncture the pharmaceutical industry needs to reduce the long-term uncertainty space for short term decisions, so that this could be accounted for in decision processes for R&D and manufacturing needing months of leeway.

Our aim here is to illustrate how epidemiological uncertainty and potential trade-offs could be addressed through modeling to enable manufacturers’ decisions in the global response to the pandemic. The quantitative examples presented here should be considered as illustration rather than forecasting.

2. Methods

2.1. Using modeling to illustrate uncertanties

Considering the need to compress timelines for vaccines to be discovered, produced and made accessible at scale, scenario planning can be central to risk management efforts. We organized scenarios over three periods: (i) a short-term period, where modeling can inform the required level of scale-up of manufacturing capacity for short-term response, and provide support to early decisions on clinical trials for new vaccines; (ii) a mid-term period to support decision making on where and when to test new vaccine candidates and about the scale of the production capacity that may be needed; and (iii) a long-term period when SARS-CoV-2 could become a public health problem with recurrent epidemics to plan for long-range vaccine manufacturing capacity needs (Fig. 1 , adapted from [11]).

Fig. 1.

Epidemiological trajectory of a pandemic and pharmaceutical industry questions and timelines. This figure maps the evolution of a pandemic and the relevant industry decisions that will be discussed across this time periods.

We developed an age-stratified disease transmission model to illustrate how possible scenarios in the short- and mid-term can define the industry decision landscape and review current knowledge of possible drivers of uncertainty in the long term. Further description of the model, parameters and assumptions can be found in the Supplementary material. The model code is available online (https://gitlab.com/SPMEGModels/covid19-model, R code). Briefly, this model, which expands on the standard Susceptible-Exposed-Infectious-Recovered (SEIR) model, allows us to take into account age specific contact-matrix, risk of hospitalizations, and risk of deaths to inform risk management strategies through scenario planning. We developed the model as a tool for uncertainty analysis that can be adapted to any settings as needed. We account for levels of severity of the infection, the possibility of reinfection with a different level of severity compared to the primary infection, and the protection conferred by natural infection. For each age class in this model, eight compartments where considered to represent the infection process (Fig. 2 ). We fitted the model to illustrate two country settings selected to represent a range of epidemic progression and policy responses: France had the first epidemic wave under control as a result of a national level policies implementing a package of mitigation measures including shelter-in-place. In the USA, the first epidemic wave has been more heterogenous in its resolution and the country has had a response implemented to different extent across states.

Fig. 2.

Flow diagram of the model structure. Susceptible ($S$), incubation ($E$) period of primary infection, Infectious and asymptomatic ($I^A$), Infectious with mild symptoms ($I^M$), Infectious with severe symptoms ($I^S$), Recovered ($R$), Susceptible after waning of natural immunity ($W$), Exposed after waning of natural immunity ($W^E$).

2.2. Short-term considerations

Scenarios to guide decision-making for industry’s short-term needs are influenced by a combination of uncertainty related to epidemiological information that resolves over time and of how the response to the epidemic evolves which is less predictable. In the short term, the need to reduce risks from a global manufacturer perspective depends on the timeline for local R&D site selection and preparation (which in the context of speed effort to develop a vaccine could include decisions related to phase 1/2 and 3 trials).

To illustrate the trade-offs in the short-term, we retrospectively looked at what were our estimated more-probable scenario depending at different time points of observed data reported from the USA (end of March – when mitigation measures only started to be implemented -, April, May, and June). At each time point, we used all data available at the specific time to calibrate the model, assuming constant social distancing post-calibration at the level of the most recent days of the calibration period (detailed description of assumptions can be found in supplementary material, Table S1).

2.3. Mid- and long-term considerations

The COVID-19 epidemiological outlook in the mid-term will affect sample size, site and population selection for phase 2/3 trials of vaccine candidates and the risk implications are highest. Government policy decisions and the level of the population adherence will drive the shape of country-specific epidemics in the mid-term. Governments are under pressure to limit the impact of control strategies, and these considerations may influence subsequent policy decisions. Social acceptance in the long run is also an important consideration since it will impact the effectiveness of such policies. Nonetheless, a certain degree of mitigation measures will likely be at hand through the period post first wave to avoid overwhelming health systems and potentially high numbers of deaths. How stringent these measures will be is not yet known. We used data from France to address mid-term uncertainties. It is possible that, in the mid-term, as countries try out new socially and economically acceptable interventions, we will observe varying levels of control measures applied, varying from strong measures (that we modeled as aiming to reduce transmission within five days of implementation followed by a slow relaxing phase over a three month period, starting on thresholds below what was observed for first wave), to weak measures (modeled with a high threshold to start the implementation of control measures and a relaxing period over only a month). In our illustration, our base case scenario would fall in between, with a hypothesized reduction of 50% in population mobility over a 10-day period and considering a two-month relaxing phase starting on medium threshold. We have also considered a scenario including targeted measures for the elderly labelled risk-based response (Supplementary material, Table S2).

In this mid-term period we also explored the effect of varying seasonality (from marked to no seasonality) and the level of waning of natural immunity (1 year, 2 years and 5 years duration) on resurgences and level of population susceptibility by August 2021. The herd immunity threshold (i.e., proportion of people in the population that need to be immune, either by natural infection or from vaccination) to avoid a large COVID-19 outbreak has been estimated to be around 50% to 70% [12]. These parameters were also relevant when considering scenarios for the long-term.

3. Results

3.1. Examples from modeling to scenario planning and uncertainty implications for the industry

3.1.1. Short-term considerations

As expected, uncertainty around the resolution and magnitude of the first pandemic wave is lower in countries having already experienced most of their wave, such as France, than in settings like the US that were less advanced in their pandemic progression when this analysis was done in June 2020 (Fig. 3 ). Policy changes in control measures (or adherence by populations to these policies) drive the evolution and total magnitude of the pandemic, more so than the uncertainty around underlying epidemiological parameters. For the US for example, we observed more than double the number of hospitalized cases between the scenario using data up to April when compared to the estimates based on data up to March (Table S4). Whereas varying the underlying epidemiological parameters lead to an increase of 81% in number of hospitalized cases between best- and worst-case scenarios (with equivalent fit on the calibration period up to mid-April, Table S5).

Fig. 3.

Prediction accuracy of short-term projections of symptomatic cases by country. In the short-term, all panels explore the impact of projections based on varying time points of the pandemic first wave. Panel A represents France which has resolved its first wave and show little uncertainty; Panel C represents the US which data have led to different evolution projections. Panel B and D illustrate variation on predicted number of susceptible populations for France and the US respectively. Variation on projections of susceptible population after the first wave depends on the magnitude of the wave.

Importantly for industry is that variations on projections of country-specific first wave resolution (Fig. 3) can affect the estimation of the proportion of the population that will remain susceptible. This is a direct indicator of the risk of future waves that could be used for short-term clinical site evaluation. General consensus, based on modeling studies to date, is that the number of susceptible population after the first wave is likely to remain high [13], [14] (as shown in Fig. 3, panel B and D, all scenario lead to percentages well above the threshold estimated range for herd immunity), although substantial changes could still occur in the US if the trends in the dynamics between transmission of SARS-CoV-2 and policy continue. Preliminary results from population-based seroprevalence studies are emerging and thus far reiterate that a high proportion of the population who has experienced the first wave remains susceptible 4]. Although pockets of high COVID-19 seroprevalence have been identified in some areas (33% in Mumbai, India, and 52% in Manus, Brazil), and could affect the success of vaccine trials [12], [15], [16]. As the pandemic evolves, such seroprevalence studies will be key to improve knowledge on the level of infection from COVID-19 and reduce related uncertainty in model-based predictions. Nonetheless, site selection for future vaccines need to be decided in the mist of the short-term pandemic evolution, bringing up important risk considerations to the pharmaceutical industry.

3.1.2. Mid-term considerations

The four scenarios modeled for France in the mid-term, based on observed data until June 28th, showed that two to three outbreak episodes leading to reinstatement of more stringent control measures over the considered period (Fig. 4 ) could be necessary. We observed (Fig. 4, Panel A) a reduction in transmission during summer months due to our assumption of seasonality, pushing cases to accrue over the cold months of October/November. The thresholds observed (Fig. 4, Panel A) reflect different levels of tolerance to SARS-CoV-2 community transmission based on the country’s health service capacity as well as community responses. In these scenarios, depending on the response to the epidemic, by August 2021 we could have 50–90% of the population still susceptible (Fig. 4, Panel B). The estimated ranges in proportion of susceptible population are still higher than the proportion of susceptible population needed to possibly prevent or strongly limit the risk of new epidemic waves through herd immunity as indicated by the red dotted line in Fig. 4, Panel B based on our estimates [17]. If a risk-based response is applied, i.e., sheltering those with comorbidity or people >65 years, we would likely see comparable evolution of the epidemic to that observed in our medium response scenario. However, there will be a substantially lower number of observed deaths as the high-risk group is differentially protected (Fig. 4, Panel C-D). This is important as manufacturers are expected to demonstrate the benefit of vaccination among high risk groups and such policy could reduce the number of people with comorbidity and 65 years and older contributing data to vaccine efficacy trials.

Fig. 4.

Mid-term scenarios for subsequent epidemic waves and projections of susceptible population. In the mid-term, these figures represent the projections of subsequent waves in France assuming seasonality and a calibration until June. Panel A shows the varying threshold for public health response (medium response initiated at an incidence in symptomatic cases of >150 cases/100,000 population, strong response triggered by >50 cases/ 100,000 population and a weak response triggered by >200 cases/100,000 population), and their impact on the frequency and magnitude of the following waves, in term of symptomatic cases. Panel B illustrate the proportion of susceptible population as a reflect of the different response thresholds. Panel C shows the number of symptomatic cases projected to subsequent waves comparing a public health response applied to the full population vs. risk-based response (i.e., focus on at risk population), assuming a medium response threshold. Panel D illustrates the same assumptions than panel C but focused on the impact of a risk-based response on the number of deaths observed in subsequent waves.

Seasonality and waning of natural immunity do not have an important role in the mid-term period (respectively ±13% and ±2% in term of total hospitalizations on the mid-term period – Table S6), and resurgence of cases are likely driven primarily by the public health response to the pandemic in terms of policy recommendations and adherence (Fig. 5 ). Nonetheless, a short duration in natural immunity could imply in higher threshold for reaching herd immunity (Fig. 5, Panel D), leading to a large pool of susceptible individuals vulnerable for further disease resurgence by July 2021. This would need to be considered to assess the value of a vaccine arriving late in the market (either adding supply or improved efficacy) and when exploring scenarios for implementation of vaccination in the long-term (if COVID-19 becomes a recurrent public health problem).

Fig. 5.

Mid-term scenarios of the impact of seasonality and duration of natural immunity on the subsequent epidemic waves and projections of susceptible population. Panels A and B illustrate the impact of seasonality on subsequent pandemic waves and proportion of susceptible population, varying from no seasonality (0%) to strong seasonality (reduction of 40% in transmission during the summer). Panels C and D show the impact of duration of natural immunity (assumptions varying from 1 year, to 5 years) on subsequent pandemic waves and on the proportion of susceptible population. All results were generated with France settings.

The mid-term model scenarios and their associated uncertainty are essential for informing production planning at scale. During this timeframe, the pharmaceutical industry needs to accept and plan for higher risk than in the short term as there are higher variable costs associated with long production processes and additional opportunity costs as production capacity and other resources may be diverted for a longer period.

The information available regarding risk of outbreaks in the absence of effective interventions is currently not directly applicable to decision on testing the efficacy of vaccine candidates, as policy decisions, population adherence and evolution of patient management directly impact the mid-term epidemiological outlook. The uncertainty permeating clinical development is significant and the higher risk in R&D at this stage when site selection occurs several months in advance of the initiation of any study, and success in trials for vaccines relies on the timely accrual of cases in sufficient numbers to reach conclusions. A low threshold for reinstating mitigation measures, for instance, could risk the enrolment or endpoint accrual in a vaccine trial. In this context, predicting where and when subsequent waves will occur depends on close monitoring of policy decisions, local population behaviors and local surveillance systems evolutions and characteristics at well-defined geographic areas (e.g., county or district level) where modeling could be applied, should there be enough surveillance data. The uncertainty related to prediction of local epidemics could lead manufactures to activate a larger number of clinical sites, encompassing a wide geographic area, than in more traditional study approach where disease prevalence is known.

3.1.3. Long-term considerations

Insights on the long-term evolution of this pandemic are essential to support infrastructure investments. In addition to other considerations, the key epidemiological question will relate to the potential of COVID-19 to become a recurrent event and therefore the need for supply of vaccine to be sustainable in the long term If SARS-CoV-2 continues circulating, the response to this question may rely on duration of immunity, seasonality and severity of subsequent infections.

At the time of writing, we have no evidence about the duration and quality of an antibody response afforded by SARS-CoV-2 infection. Studies of SARS-CoV-1 – virus identified in 2003 and closely related to SARS-CoV-2 – have suggested limits to the body’s immune response [18], [19]. Evidence from earlier studies on human coronaviruses (HCoVs), which are associated with the common cold, indicate that primary infection does not lead to lasting immunity, suggesting waning of immunity or relative immunity that allows for reinfections [20], [21]. Data also demonstrate that HCoVs circulate with marked seasonality, with seasonal waves starting in December, peaking in January or February, and subsiding in March [22].

In the event of natural infection affording long-lasting immunity, once the population reaches a level of herd immunity, SARS-CoV-2 circulation would be minimal, and may not payoff substantial investments in industrial capacity that is made at an early phase of the pandemic to expand or redirect production investments. If immunity afforded by infection wanes over a year and the disease is seasonal, we could see an annual resurgence of the virus that could mimic influenza seasons. Such scenario would signal the need for robust and sustainable investment in vaccine production over time. However, such investment in infrastructure could lead to competing priorities for companies also producing other vaccines with shared platforms. If immunity protects from reinfection for five years, the virus could return erratically at first, until the virus establishes a more regular five-year interval pattern. If this virus does not show marked seasonality, sporadic surges could occur, with no predictable pattern. This last scenario would be particularly problematic for planning of timing and capacity of ongoing production of vaccines and therapeutics and would probably require stockpiling. Illustration of long-term effect of waning of immunity and circulation of SARS-CoV-2 have been previously discussed [23].

The impact of vaccination in the long-term, and the risk of fixed investments such as those aiming to increase production capacity, will depend on the duration and quality of immunity afforded by vaccination. The duration of protection will still be unknown at time of registration. Vaccine producers are required by regulatory authorities to monitor safety and effectiveness of their products post-licensure, which imply in investments to monitor for rare adverse events following immunization. Public-private partnerships that can maximize the use of existing vaccine effectiveness and safety evaluation platforms will be key to address the long-term risk/benefit analyses of potentially several COVID-19 vaccines in large cohorts across several countries.

4. Discussion

There are high expectations for pharmaceutical products to help curb the effect of this pandemic on society. In the context of public health emergencies, the industry is being challenged to develop and produce safe and effective vaccines within accelerated timelines (of 12–18 months) which can only be achieved through investments in innovative technologies, and flawless planning under substantial risk.

The risk associated with epidemiological uncertainty has important financial implications that can be materialized by failure of costly trials, missed reallocation of production lines, or the expansion of production capacity that could become underutilized or could have been more beneficial for products associated to other diseases. These implications reflect both opportunity costs and sunk costs. Sunk costs at early stages of development do represent a major risk for investment decisions, such as failure of costly trials. Incentives from governments aiming to de-risk these investments have been unprecedented and grounded on key economic analyses [24]. Currently, the value of a vaccine can be considered in terms of morbidity, mortality averted and the need for non-pharmaceutical interventions. In the mid- to long-term, treatment and earlier access to rapid diagnostics can affect transmission, health impact and change the population perception about risk/benefit COVID-19 vaccines, impacting uptake and value of a future vaccine.

While at present this risk cannot be avoided, the framework presented here allow us to discuss risk management and partial risk-sharing strategies through partnerships within industry and with the public sector. Operation Warp Speed, the Access to COVID-19 Tools (ACT) Accelerator, and the European Union coronavirus vaccines strategy are examples of the willingness of countries and international health organizations to contribute to and de-risk vaccine and drug development for COVID-19 [25], [26], [27]. Working with uncertainty scenarios and moving through decisions iteratively allows for reassessment of risks as new data become available, ultimately informing rapid and flexible adaptation of trial design and production.

We underscore the central role of policy decisions and population’s adherence to these policies in planning risk management strategies in short- and mid-term, while natural history unknowns (such as duration of immunity post-infection and seasonality) should be guiding long-term discussions. Future mitigation policies and the sustained adherence to these are difficult to predict. Country-specific experience documented through the first wave of this pandemic and current approaches to resurgence could be used as a qualitative indication of mid-term directions. However, the assumptions and scenarios are regularly evolving as new information becomes available, such as large-scale mobility data, to reduce the uncertainty landscape progressively.

Conducting clinical trials during a pandemic poses important challenges [28]. Not only is it difficult to predict where and when outbreaks will occur and to prepare trial sites to match vaccine readiness for testing, but also the burden of the outbreak on the healthcare system may not allow for dedicated trial resources and will likely impact enrolment, site visits and data collection. In addition, if multiple vaccines and drugs are ready for testing at the same time, it will be important not to crowd sites or burden countries and their ethics and regulatory authorities with multiple trials, as was the case with Ebola therapeutic development during the 2013–2016 outbreak [29], [30]. Collaborations and coordination will be important to achieve successful and accessible solutions as the pandemic generates demand for vaccines around the world. As underlined by our modeling exercise, clinical and epidemiological studies will be needed to identify populations at highest risk, ensure the appropriate outcomes are collected in standardized and optimized ways and guide program implementation decisions that ensure equitable access to vaccines, as well as reducing uncertainties in projections. The prominent role of policy changes and social behavior, that are not predictable, directly affect the robustness of the projections obtained with the different forecasting tools developed for COVID-19. To account for this, most groups limit their projections to short-time periods, with changes in their modeling approach over time [31]. Health authorities have promoted standardized scenarios for model comparisons to avoid relying on one unique set of projections and increase comparability [32], [33]. Collection of real-world evidence will also be instrumental to provide more granularity to the local epidemiology, allowing for potentially faster adaptation of vaccine trial design and more accurate benefit/risk measurement. As we observe an increase of open data policies by governments and public institutions, such as [34], the speed of data availability and its quality will need to be considered to ensure effective decision making, specifically reducing uncertainties in models supporting those decisions. For these local decisions, model estimates based on short timeframes, but greater geographical granularity can be helpful. The iterative revision of scenarios as new data become available can introduce agility in the planning process but also demonstrates level of uncertainties throughout the evolution of the first wave. Addressing data gaps will help improve substantially the assessment of uncertainty during this pandemic phase. Going forward, and specifically when a vaccine becomes available, there will be added value to prioritize research gaps and data collection based on techniques such as expected value of perfect information analysis. Our modeling could help industry define research gaps key for the development and deployment of these technologies.

The quantitative examples presented here should be considered as illustrations rather than forecasts by country. As such, we did not position the presentation of results to exhaust sensitivity analyses required for scenario planning. In addition, modeling is a dynamic process, especially as new knowledge becomes available daily and parameters need to be refined. Precise projections have therefore a very short duration of veracity, especially at mid-term horizon. Although other calibration methods could potentially provide a more detailed exploration of the parameter space, the method selected provided a good fit to the data and an exploration of key disease parameters separately. We limited the illustrative analysis at a country level, while acknowledging that there is significant heterogeneity in disease transmission at the subnational level. Country-specific mitigation measures encompassed interventions with substantial unknowns about their effectiveness either separately or in combination. Importantly, if countries decide to respond in unexpected ways in subsequent waves compared to their first pandemic response, for example by not re-instating any mitigation measures or continuing lockdowns, both trial design and production capacity planning will be affected. Our mid-term scenarios of control measures aim to explore this important unknown, but iterative analyses following a close tracking of stringency and timings of control measures is needed. In the long-term, our exploration does not include the potential impact of cross-reactivity or cross-protection as frequent co-circulation of HCoVs types and virus co-detection within individuals do not suggest high cross-reactivity [22]. Finally, we did not account for possible future mutations in the virus that could have implications on transmissibility and pathogenesis or interfere with immune response as data are currently limited [35].

5. Conclusions

We discussed the uncertainties related to information on the natural history of COVID-19 and on future mitigation policies through scenarios and underscore the central role that policy decisions and population adherence have in the evolution of this pandemic and industry associated risks. As detailed, disease transmission models can facilitate this risk assessment and guide accelerated pharmaceutical industry response to COVID-19. These models do require adequate data to be able to support decisions efficiently, and could then play a major role in reducing decision risk associated with R&D and manufacturing of COVID-19 vaccines. Once these new vaccines become available, these tools could then be valuable in addressing further uncertainty related to the distribution, access and uptake. Our first response to this pandemic relies on strategic choices that cannot be delayed. Public and private cooperation is mandatory to ensure appropriate data and tools are available to address timely and efficiently this unique public health, social and economic challenge.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary material

The following are the Supplementary data to this article: