Quantitatively assessing early detection strategies for mitigating COVID-19 and future pandemics

Abstract

Researchers and policymakers have proposed systems to detect novel pathogens early by monitoring samples from hospital patients, wastewater, and air travel, in order to mitigate future pandemics. How much benefit would such systems offer? We developed, empirically validated, and mathematically characterized a quantitative model that simulates disease spread and detection time for any given disease and detection system. We find that hospital monitoring could have detected COVID-19 in Wuhan 0.4 weeks earlier than it was actually discovered, at 2,300 cases compared to 3,400. Wastewater monitoring would not have accelerated COVID-19 detection in Wuhan, but provides benefit in smaller catchments and for asymptomatic or long-incubation diseases like polio or HIV/AIDS. Monitoring of air travel provides little benefit in most scenarios we evaluated. In sum, early detection systems can substantially mitigate some future pandemics, but would not have changed the course of COVID-19.

It has been widely debated which policies, if any, could have mitigated the initial stages of the COVID-19 pandemic in late 2019 and early 2020 as community transmission became established and widespread. Early studies compared non-pharmaceutical interventions (NPIs) such as mobility restrictions (1, 2), school closures (3, 4), voluntary home quarantine (5) and testing policies (6), and optimized NPI parameters like testing frequency (7), quarantine length (8), testing modality (9), test pooling (10) and intervention timing and ordering (11). While such NPIs undoubtedly slowed the early spread of COVID-19 (12) and previous outbreaks (13, 14), there has been little investigation of whether a separate strategy focused on earlier detection of COVID-19 would have enabled more successful mitigation. In theory, earlier detection enables a response when the outbreak is smaller: thus resource-intensive mitigation strategies like test-trace-isolate become less costly, and the earlier interventions are applied, the larger the number of infections and deaths that can be delayed until healthcare capacity is increased (15). However, the relevant question is not whether early detection helps, but quantitatively how much of a difference it would make. This question is especially urgent given current international and national policy proposals to invest billions of dollars in such systems (16, 17).

Researchers and policymakers have proposed immediate investments in systems to continuously monitor for novel pathogens in (i) patients with infectious symptoms in hospitals (18), (ii) community wastewater treatment plants (19, 20), and (iii) airplane sewage or bridge air on international flights (21–23), as well as other sites (24–28). These three sites have attracted interest because they have been frequent testing sites in COVID-19: hospitals since the pandemic’s beginning (29), and wastewater and air travel more recently (30, 31) because hospital cases can lag community cases (32). COVID-19 also spurred methodological innovation and characterization of sampling from these sites, particularly wastewater (33–35). Detecting novel pandemics at these sites has occasionally been piloted (20, 36) but has not been implemented at scale, in part because it is unclear if these proposed systems sufficiently expedite detection of outbreaks. The systems under consideration would use multiplex testing for conserved nucleic acid sequences of known pathogen families, exploiting the fact that many past emerging diseases belonged to such families, including SARS-CoV-2 (2019), Ebola (2013), MERS-CoV (2012), and pandemic flu (2009). Proposed technologies include multiplex PCR (37–40), CRISPR-based multiplex diagnostics (41), and metagenomic sequencing (42), possibly implemented with pooling (10).

To determine whether early detection of novel pathogens at these sites could be effective in changing the course of a pandemic, we first examined whether COVID-19 could have been detected earlier in Wuhan if systems had been in place in advance to monitor hospital, wastewater or air travel. To do this, we developed, empirically validated, and mathematically characterized a quantitative, branching process simulation-based model that predicts the number of cases at the time of detection given a detection system and a set of outbreak epidemiological parameters. We then used this model and COVID-19 epidemiological parameters (43) to estimate how early COVID-19 would initially have been detected in Wuhan by the three early detection systems, and compare this to the actual date of COVID-19 detection. Finally, we use our model to estimate detection times of infectious agents with different epidemiological properties, such as monkeypox and polio in recent outbreaks (44, 45), to inform pathogen-agnostic surveillance for future pandemics.

Model to estimate earliness of detection

Previous research (15) and our analysis (Supplementary text, figs. S1—5 and table S1) suggest that earlier COVID-19 lockdowns could have delayed cases and deaths. Thus it is critical to understand which early detection systems, if any, could have effectively enabled earlier response. To do this, we built a model that simulates outbreak spread and earliness of detection for a given outbreak and detection system (Materials and methods, Supplementary materials). This builds upon branching process models that have previously been used to model the spread of COVID-19 (46, 47) and other infectious diseases (48). A traditional branching process model starts from an index case and iteratively simulates each new generation of infections. Our model follows this pattern, but with each new infection we also simulate whether the infected person is detected by the detection system with some probability (Fig. 1A), and the simulation stops when the number of detected individuals equals the detection threshold and the detection delay has passed. Thus each detection system is characterized by these three parameters of detection probability, threshold, and delay (table S2). For example, in hospital monitoring, an infected individual’s detection probability is the probability they are sick enough to enter the hospital, which is the hospitalization rate (assuming testing has a negligible false negative rate). In systems that test individuals (hospital and air travel individual monitoring), the threshold is measured in an absolute number of cases. In systems that test wastewater (wastewater monitoring), the threshold is measured in terms of outbreak prevalence because wastewater monitoring can only sample a small percentage of sewage flows (49); thus a higher number of cases is required to trigger detection in a bigger community. We gathered literature estimates of detection system and outbreak parameters (tables S2 and S3) and validated wastewater monitoring sensitivity in independent data (fig. S6 and Materials and methods, Supplementary materials). We then empirically validated the model by testing its ability to predict the detection times for the first COVID-19 outbreaks in 50 US states in 2020. We gathered the dates of the first COVID-19 case reported by the public health department of each US state (table S4) as well as literature estimates of true (tested and untested) statewide COVID-19 case counts in early 2020 (50). Using our model, we were able to predict the number of weeks until travel-based detection in each US state to within a mean absolute error of 0.97 weeks (fig. S7 and S8).

Fig. 1. Comparison of COVID-19 detection times in the actual pandemic versus with proposed early detection systems.

(A) Schematic of first 20 infections in a simulated run of the detection model. In this run, Person 1 seeds an outbreak in a community covered by a hospital detection system. Each person infects a number of individuals determined by a draw from a negative binomial distribution. Each person is then detected by the detection system with probability $p_{\text{test}}$ (gold) or goes undetected (olive); in the hospital system, $p_{\text{test}}$ equals the hospitalization rate. (B) Estimated cases until COVID-19 detection in the actual pandemic versus model-simulated cases until detection for proposed detection systems. Estimates for the actual pandemic are drawn from (51). Points for proposed detection systems are simulated case counts from the model (actual pandemic (black), hospital (teal), wastewater (orange) and air travel (purple)) assuming a Wuhan-sized catchment (650,000 people). Three, two, and one asterisk(s) signify statistically significant differences at the 0.001, 0.01, and 0.05 levels, respectively, in one-sided t-tests of between each detection system and the actual pandemic. NS. signifies not statistically significant at p=0.05. Equivalent weeks until detection are shown on the right y-axis.

Early detection’s impact on COVID-19 detection in Wuhan

Next we use our model to examine the detection systems’ ability to detect the first major COVID-19 outbreak in Wuhan (Fig. 1B and table S2). To estimate cases at detection in the actual pandemic, we used literature estimates of total (tested and untested) COVID-19 case counts in Wuhan in late 2019 and early 2020 (51). Our model shows that, on average, hospital monitoring could have detected COVID-19 after 2,292 cases. In reality, the pandemic was identified after 3,413 cases on average. Thus, hospital monitoring would have caught the outbreak 1,121 cases earlier (approximately 0.43 weeks earlier), a statistically significant difference with p = 1.9e-09 and t = −6.3 in one-sided Welch two-sample t-test. Wastewater monitoring would have lagged detection in the actual pandemic; it caught the outbreak after 4,575 cases, or 1,162 cases later, on average (p = 0.018 and t = 2.1). We tested this wastewater prediction empirically by calculating the cases until COVID-19 wastewater detection in Massachusetts in early 2020, using literature-estimated Massachusetts COVID-19 cases (50) and Massachusetts wastewater SARS-CoV-2 PCR data (52); our model prediction was consistent with this analysis (fig. S9). Because we model wastewater monitoring to detect later in larger communities (Materials and methods, Supplementary materials), the Wuhan result is in part due to Wuhan’s 650,000-person catchments. Wastewater monitoring would lead status quo detection of COVID-19 in catchments smaller than 480,000 people, well above the global mean catchment size of 25,000 people (53). Air travel monitoring did not provide any acceleration of detection because of the low probability of simultaneously traveling and being sick.

Early detection’s impact for other diseases: mathematical analysis and simulation

To make our model easily usable for outbreaks beyond COVID-19, we derived a compact formula that approximates the model’s simulations. We observed that, without accounting for the delay of $g$ generations between the threshold case’s infection and detection, the number of cases until detection, $C$, is a random variable that follows a negative binomial distribution by definition: each infected case is a Bernoulli trial, “success” in that trial occurs when that case enters the detection system (with a probability we name $p_{test}$), and we count the number of cases until the number of successes equals the detection threshold $d$. After accounting for $g$, we derived a formula approximating the mean of $C$ when the outbreak starts in a community covered by the detection system (see Supplementary Text for full derivation):

$$\mathbb{E}[C]\approx \frac{d\times R_0^g}{p_{test}}$$ (1)

We confirmed our formula approximates the simulation model closely by comparing the detection times predicted by both for all the detection systems for multiple diseases (fig. S10). Thus the formula allows us to interpret the model and the quantitative relationships between detection times and various variables: the formula shows that the number of cases until detection increases linearly with the detection threshold, increases polynomially with $R_0$ and exponentially with the delay $g$ as $R_0{}^g$, and decreases as the fraction of cases being tested increases. This formula also makes the model easily usable for detection systems beyond the ones studied here.

We applied our model to several outbreaks of recent interest–including COVID-19, monkeypox (2022), polio (2013–2014), Ebola (2013–2016) and flu (2009 pandemic)–and found that the detection systems vary in their success depending on the epidemiological parameters of the agent (Figs. 2, S11 and S12, and table S3). For example, in our model hospital monitoring tends to outperform wastewater monitoring when the hospitalization rate is high, as in the case of Ebola, but tends to underperform for diseases like polio, in which the hospitalization rate is low and when there is high asymptomatic spread in the delay from detection to hospitalization. This is consistent with Equation (1), as well as previous observations that Ebola was first detected in hospitals (54) and that wastewater monitoring has been more effective than hospital monitoring for detecting polio (55). We also modeled the status quo detection times for these outbreaks: the number of cases until these outbreaks were detected in the status quo, without the proposed detection systems in place. We found that early detection systems can catch outbreaks when they are up to 52% smaller (wastewater for polio) or 110 weeks earlier (hospital for HIV/AIDS) (figs. S13, S14, S15 and S16).

Fig. 2. Comparison of detection systems for different infectious diseases.

(A) Earliness of detection for detection systems in cases across infectious diseases (hospital (teal), wastewater (orange), air travel (purple) and status quo (black)) in a 650,000-person catchment. (B) Earliness of detection for detection systems in weeks across infectious diseases in a 650,000-person catchment. (C) Epidemiological parameters of the studied diseases.

Because future infectious diseases are likely to have different epidemiological parameters, we generalized the previous analysis and calculated detection times for many possible diseases spanning the epidemiological parameter space (Figs. 3 and S17). As expected, hospital monitoring is the best system for diseases with higher hospitalization rates and lower times to hospitalization. For diseases with higher R0s and times to hospitalization, wastewater monitoring emerges as the best system more often, because hospital monitoring has a longer detection delay (mainly the time from infection to hospitalization) than wastewater (mainly the time from infection to fecal shedding), during which cases grow exponentially with R0. However, this holds mainly for diseases with high probability of fecal shedding and low hospitalization rate. Air travel monitoring, which did not perform well in the previous modeled diseases (Figs. 1 and 2), actually performed best for a few diseases for which fecal shedding is low (disadvantaging wastewater monitoring) and the time to hospitalization and R0 are too large (disadvantaging hospital monitoring).

Fig. 3. Comparison of detection systems across the space of possible diseases of varying epidemiological parameters.

(A) Average weeks gained over status quo detection by the proposed detection systems across the epidemiological space of possible diseases. Within each panel, each uniformly colored cell corresponds to a specific disease with the hospitalization rate and probability of fecal shedding indicated on the x- and y-axes, as well as the R0 and time to hospitalization (generations) indicated by the panel row and column. The cell has a hue corresponding to the detection system that detects the disease the earliest (hospital (teal), wastewater (orange) and air travel (purple)) and an intensity corresponding to the number of weeks gained by the earliest system over status quo detection. Times are calculated by the derived mathematical approximation in a 650,000-person catchment.

Our results show that the benefits of early detection systems vary from marginal (0.4 weeks earlier for COVID-19) to significant (110 weeks earlier for HIV/AIDS) (Figs. 1B, 2, and S16). Our detection time model (Fig. 1A) can be used for many diseases and detection systems, including other systems beyond this study (24, 25), by varying the fraction of the infected population being tested in each system. Two further points are worth emphasizing. First, early detection only aids mitigation if it leads to a coordinated early response. Many factors beyond detection affect the pace of response, including the economic and political feasibility of lockdowns, the availability of medicines and personal protective equipment, and whether there are pre-determined policies to be implemented upon detection. Second, when deciding to invest in these systems, one must consider factors such as cost-effectiveness and whether the system provides evidence of disease severity. Although wastewater monitoring gives earlier detection than hospital monitoring in multiple diseases (Fig. 3A), it does not discriminate between mild and severe disease. In contrast, hospital monitoring provides evidence that the detected pathogen produces symptoms that require hospital treatment.

These results can inform ongoing international and national policy debates about which policies are needed to mitigate future pandemics. In the wake of COVID-19, the World Health Organization Intergovernmental Negotiating Body is actively negotiating a new treaty on international pandemic preparedness which updates the International Health Regulations (2005). Drafts of this treaty highlight “early warning and alert systems” as key measures (16). Similarly, the presidential administration of the United States has proposed investing $5.3 billion over 7 to 10 years in early warning and real-time monitoring systems, including in hospitals and wastewater (17). In this study, we have assessed detection systems’ detection times and have developed a model to assess current and future detection system proposals. Along with additional cost-effectiveness analysis and technical pilots (20), these results can help inform which detection systems are most effective and thus worth funding in pandemic preparedness efforts.