Why estimating population-based case fatality rates during epidemics may be misleading

Abstract

Different ways of calculating mortality ratios during epidemics can yield widely different results, particularly during the COVID-19 pandemic. We formulate both a survival probability model and an associated infection duration-dependent SIR model to define individual- and population-based estimates of dynamic mortality ratios. The key parameters that affect the dynamics of the different mortality estimates are the incubation period and the length of time individuals were infected before confirmation of infection. We stress that none of these ratios are accurately represented by the often misinterpreted case fatality ratio (CFR), the number of deaths to date divided by the total number of infected cases to date. Using available data on the recent SARS-CoV-2 outbreaks and simple assumptions, we estimate and compare the different dynamic mortality ratios and highlight their differences. Informed by our modeling, we propose a more systematic method to determine mortality ratios during epidemic outbreaks and discuss sensitivity to confounding effects and errors in the data.

I. INTRODUCTION

The mortality ratio is a key metric describing the severity of a viral disease. It changes in time and can be measured in a number of ways during an epidemic. One common metric is the case fatality ratio (CFR), found by dividing the total number of deaths to date, D(t), by the total number of all cases to date N(t) [1, 2].

In the recent outbreaks of SARS-CoV-2, the CFR has been estimated from aggregated population data. We show examples of CFR curves in Fig. 1 and in the Supplemental Information (SI). As of March 26, 2020, the global CFR(t) = D(t)/N(t) = 21,306/472,762 ≈ 4.5% [3], while CFRs in individual regions vary significantly. Clearly, this estimate would correspond to an actual mortality ratio only if all the remaining unresolved individuals recover. However, some of these unresolved cases will lead to death and thus to a gradual increase of the estimated mortality ratio over time. Despite the underestimation of this type of population-based measurement, it is still commonly being used by various health officials and is often inconsistently defined as deaths/(deaths + recovereds) even though this difference has been clearly distinguished [4].

FIG. 1. Mortality-ratio estimates.

(a) Evolution of the cumulative number of infected (red), death (black), and recovered (green) cases. The size of the circles indicates the number of cases in the respective compartments on a certain day. (b–c) Estimates of mortality ratios (see Eqs. (8) and (14)) of SARS-CoV-2 infections in China and Italy. The “delayed” mortality-ratio estimate CFR_d corresponds to the number of deaths to date divided by total number of cases at time t − τ_res. Many studies use CFR_d, although this metric underestimates the individual-based mortality (defined below). Another population-based mortality ratio is M_p(t), the number of deaths divided by the sum of death and recovered cases, up to time t. The data are based on Ref. [5].

During the severe 2003 acute respiratory syndrome (SARS) outbreak in Hong Kong, the World Health Organization (WHO) also used the aforementioned estimate to obtain a CFR of 4.5% while the final values approached 17.0% [6, 7]. For the ongoing SARS-CoV-2 outbreaks, analyses by WHO and other institutions still use the CFR = D(t)/N(t) metric (see Table I). Since actual mortality probabilities are important measures for assessing the risks associated with epidemic outbreaks, typical underestimations by CFRs may lead to insufficient countermeasures and a more severe epidemic [8, 9].

TABLE I.

Different CFR estimates of COVID-19.

reference	CFR
Xu et al. [1, 11] and Mahase [12]	2 %
Wu et al. [2]	0.1–1 % (outside Wuhan)
World Health Organization [13, 14]	2–4 %

An unambiguous, physiologically-based definition of mortality ratio is the probability that a single, newly infected individual will eventually die of the disease. If there are sufficient individual-level or cohort data, these probabilities can be further stratified according to patient age, gender, health condition, etc. [10]. The mortality ratio or probability of death should be an intrinsic property of the virus and the infected individual, depending on age, health, access to health care, etc. This intrinsic probability ought not to be directly dependent on the population-level dynamics of infected and recovered individuals. Thus, it can in principle be framed by a model for the survival probability of a single infected individual. Whether this individual infects others does not directly affect his probability of eventually dying. In section IIA, we derive a model describing the probability M₁(t) that an infected individual dies or recovers before time t. Importantly, these models incorporate the duration of infection (including an incubation period) before a patient tests positive at time t = 0.

However, as mentioned above, the CFR and other mortality measures are typically reported based on population data. Do these population-based measures, including CFR, provide reasonable measures of the probability of death of an individual? In section IIB, we describe how mortality ratios are defined within population-level models, specifically, a disease duration-structured SIR model. We will show that population-based estimates are typically not a meaningful measure of mortality, but that under simplifying assumptions, the mortality ratio M_p(t) is more closely related to the number of deaths to date divided by the number of dead plus the number of recovered individuals to date [4]. In the simplest approximation, the mortality ratio is currently (as of March 26, 2020) 21,306/(21,306+114,749) ≈ 15.7% [3], significantly higher than the March 26, 2020 CFR≈ 4.5% estimate.

We use the same estimates for the rate parameters in our individual and population models to compute the different mortality ratios. Note that in general, both the individual mortality probability M₁(t) and the population-based estimates M_p(t) depend on the time of measurement t. By critically analyzing these estimates and another ad hoc “delayed” ratio CFR_d, we illustrate and interpret the differences among these measures and discuss how changes or uncertainty in the data affect them. In section III, we summarize our results and identify a correction factor to transform population-level mortality estimates into individual mortality probabilities.

II. RESULTS

A. Intrinsic individual mortality rate

Consider an individual that, at the time of positive testing (t = 0), had been infected for a duration τ₁. A “survival” probability density can be defined such that P(τ,t|τ₁)dτ is the probability that the patient is still alive and infected (not recovered) at time t > 0 and has been infected for a duration between τ = t+τ₁ and τ +dτ. Since τ₁ is unknown, it must be estimated or averaged over. The individual survival probability evolves according to

$$\frac{\partial P\left(\tau ,t|{\tau }_1\right)}{\partial t}+\frac{\partial P\left(\tau ,t|{\tau }_1\right)}{\partial \tau }=-\left(\mu \left(\tau ,t|{\tau }_1\right)+c\left(\tau ,t|{\tau }_1\right)\right)P\left(\tau ,t|{\tau }_1\right),$$ (1)

where the death and recovery rates, μ(τ,t|τ₁) and c(τ,t|τ₁), depend explicitly on the duration of infection at time t and implicitly on patient health and age a [15]. They may also depend explicitly on time t to reflect changes in clinical policy or available health care. For example, enhanced medical care may decrease the death rate μ, giving the individual’s intrinsic physiological processes a chance to cure the patient. These intrinsic individual-based death and recovery rates do not directly depend on population-level viral transmission.

Equation (1), assuming an initial condition of one particular individual who has been infected for time τ₁ at the time of positive test, can be solved using the method of characteristics. From the solution P(τ,t|τ₁) one can derive the probabilities of death and recovery by time t as

$$P_{\text{d}}\left(t|{\tau }_1\right)={\int }_0^t\text{d}s\mu \left({\tau }_1+s,s\right)P\left({\tau }_1+s,t|{\tau }_1\right),P_{\text{r}}\left(t|{\tau }_1\right)={\int }_0^t\text{d}sc\left({\tau }_1+s,s\right)P\left({\tau }_1+s,t|{\tau }_1\right).$$ (2)

The probability that an individual died before time t, conditioned on resolution (either death or recovery), is then defined as

$$M_1\left(t|{\tau }_1\right)=\frac{P_{\text{d}}\left(t|{\tau }_1\right)}{P_{\text{d}}\left(t|{\tau }_1\right)+P_{\text{r}}\left(t|{\tau }_1\right)},$$ (3)

where we have explicitly indicated the dependence on the duration of the infection prior to confirmation of infection. These formulae also depend on all other relevant patient attributes such as age, accessibility to health care, etc. In the long-time limit, when resolution has occurred (P_d(∞) + P_r(∞) = 1), the individual mortality ratio is simply M₁(∞) = P_d(∞). This result relies only on intrinsic individual rate parameters and is completely independent of disease transmission at the population level. In order to capture the dependence of death and recovery rates on the time an individual has been infected, we propose a constant recovery rate c and a simple piece-wise constant death rate μ(τ|τ₁) that is not explicitly a function of time t:

$$c\left(\tau ,t|{\tau }_1\right)=c,\mu \left(\tau |{\tau }_1\right)=\{\begin{array}{ll}0\hfill & \tau \le {\tau }_{\text{inc}}\hfill \\ {\mu }_1\hfill & \tau >{\tau }_{\text{inc}}\hfill \end{array},$$ (4)

where τ_inc is the incubation-time parameter, the time after infection during which an individual remains asymptomatic. During this incubation period, the patient has zero death rate but can recover by clearing the virus. In other words, some patients fully recover without ever developing serious symptoms.

For coronavirus infections, the incubation period appears to be highly variable with a mean of τ_inc ≈ 6.4 days [17]. We can estimate μ₁ and c using individual patient data where 19 patients (outside Hubei) had been tracked from the date on which their first symptoms occurred until the disease resolved [16].

Two out of 19 patients died, on average, 20.5 days after first symptoms occurred and the mean recovery time of the remaining 17 patients is 16.8 days. We show the recovery-time distribution in Fig. 2(a). Since we know that the mortality ratio in this dataset is 2/19, we can determine the dependence between μ₁ and c according to μ₁/(μ₁ +c) ≈ 2/19 (or c/μ₁ ≈ 8.5). The constant recovery and after-incubation period death rates [18] are estimated to be

$$c=\frac{1}{20.5}/\text{day }\approx 0.049/\text{day and }{\mu }_1=c/8.5\approx 0.006/\text{day}.$$ (5)

FIG. 2. Individual mortality ratio.

(a) Recovery time after first symptoms occurred based on individual data of 17 patients [16]. (b) Death- and recovery rates as defined in Eq. (4). The death rate μ(τ₁) approaches μ₁ for τ₁ > τ_inc, where τ_inc is the incubation period and τ₁ is the time the patient has been infected before first being tested positive. (c) The individual mortality ratio M₁(t|τ₁) for τ_inc = 6.4 days at different values of τ₁. Note that the individual death probability P_d(t|τ₁) and M₁(t|τ₁) are nonzero only after t > τ_inc − τ₁. (d) The asymptotic individual mortality ratio M₁(∞) (see Eq. (3)) as a function of τ₁.

Using these numbers, the recovery and death rate functions c(τ,t|τ₁) and μ(τ|τ₁) are plotted as functions of τ in Fig. 2(b). We show the evolution of M₁(t|τ₁) at different values of τ₁ in Fig. 2(c). The corresponding long-time limit M₁(∞) is readily apparent in Fig. 2(d): for τ₁ ≥ τ_inc, M₁(∞) = μ₁/(μ₁+c) ≈ 0.105, while M₁(∞) < μ₁/(μ₁+c) when τ₁ < τ_inc. The smaller expected mortality associated with early identification of infection arises from the remaining incubation time during which the patient has a chance to recover without possibility of death. When conditioned on testing positive at or after the incubation period, the patient immediately suffers a positive death rate, increasing his M₁(∞).

Finally, in order to infer M₁ (and also indirectly μ and c) during an outbreak, a number of statistical issues must be considered. First, if the outbreak is ongoing, there may not be sufficient long-time cohort data. Second, τ₁ is unknown. Since testing typically occurs at the onset of symptoms, most positive patients will have been infected a few days earlier. The uncertainty in τ₁ can be represented by a probability density ρ(τ₁) for the individual. The expected mortality can then be constructed as an average over ρ(τ₁):

$${\overline{M}}_1(t)=\frac{{\overline{P}}_{\text{d}}(t)}{{\overline{P}}_{\text{d}}(t)+{\overline{P}}_{\text{r}}(t)},$$ (6)

where ${\overline{P}}_{\text{d}}(t)$ and ${\overline{P}}_{\text{r}}(t)$ are the the τ₁-averaged probabilities death and cure probabilities. Note that this averaging is different from the population-level averaging ${\overline{M}}_1(t)={\int }_0^{\infty }{M}_1\left(t|{\tau }_1\right)\rho \left({\tau }_1\right)\text{d}\tau$, which would describe the average of mortality ratios over a population with heterogeneous initial durations τ₁.

Some properties of the distribution ρ(τ₁) can be inferred from the behavior of patients. Before symptoms arise, only very few patients will know they have been infected, seek medical care, and get their case confirmed (i.e., ρ(τ₁) ≈ 0 for τ₁ ≈ 0). The majority of patients will contact hospitals/doctors when they have been infected for a duration of τ_inc. The distribution ρ(τ₁) thus reaches its maximum near or shortly after τ_inc. Since patients are most likely to test positive after experiencing symptoms, we choose a gamma distribution

$$\rho \left({\tau }_1;n,\gamma \right)=\frac{{\gamma }^n}{\Gamma (n)}{\tau }_1^{n-1}{e}^{-\gamma {\tau }_1}$$ (7)

with shape parameter n = 8 and rate parameter γ = 1.25/day so that the mean n/γ is equal to τ_inc = 6.4.

Upon using the rates in Eqs. (4) and averaging over ρ(τ₁), we derived expressions for $\overline{P}(t)$, ${\overline{P}}_{\text{d}}(t)$, and ${\overline{P}}_{\text{r}}(t)$ which are explicitly given in the SI. Using the values in Eq. (5) we find an expected individual mortality ratio ${\overline{M}}_1(t)$ (which are subsequently plotted in Fig. 3) and its asymptotic value ${\overline{M}}_1(\infty )={\overline{P}}_{\text{d}}(\infty )=0.101$. Of course, it is also possible to account for more complex time-dependent forms of c and μ₁ [19], but we will primarily use Eqs. (4) in our subsequent analyses.

FIG. 3. Population-level mortality-ratio estimates.

Outbreak evolution and mortality ratios without containment measures (a,c) and with quarantine (b,d). The curves are based on numerical solutions of Eqs. (9) using the initial condition I(τ,0) = ρ(τ;8,1.25) (see Eq. (7)). The death and recovery rates are defined in Eqs. (4) and (5). We use a constant infection rate β₁S(0) = 0.158/day, which we estimated from the basic reproduction number of SARS-CoV-2 [17]. To model quarantine effects, we set β₁ = 0 for t > 50. We show the mortality-ratio estimates $M_{\text{p}}^0(t)$ and $M_{\text{p}}^1(t)$ (see Eq. (14)) and CFR_d(t,τ_res) (see Eqs. (8), (11), (12), and (14)).

In the next section, we define population-based estimates for mortality ratios, M_p(t), and explore how they can be computed using SIR-type models. By comparing M₁(t) to M_p(t), we gain insight into whether population-based metrics are good proxies for individual mortality ratios. We will outline the mathematical differences and additional errors that confound population-level estimates.

B. Infection duration-dependent SIR model

While individual mortalities can be estimated by tracking many individuals from infection to recovery or death, oftentimes, the available data are not resolved at the individual level and only total populations are given. Typically, one has the total number of cases accumulated up to time t, N(t), the number of deaths to date D(t), and the number of cured/recovered patients to date R(t) (see Fig. 1). The CFR is simply D(t)/N(t). Note that N(t) includes unresolved cases and that N(t) ≥ R(t)+D(t). Resolution (death or recovery) of all patients, N(∞) = R(∞)+D(∞), occurs only well after the epidemic passes.

A variant of the CFR commonly used in the literature [1, 2] is the delayed CFR

$${\text{CFR}}_{\text{d}}\left(t,{\tau }_{\text{res}}\right)=\frac{D(t)}{N\left(t-{\tau }_{\text{res}}\right)},$$ (8)

where τ_res is a corresponding time lag that accounts for the duration from the day when first symptoms occurred to the day of cure/death. Many estimates of the COVID-19 mortality ratio assume that τ_res = 0 [1, 2] and thus underestimate the number of death cases D(t) that result from a certain number of infected individuals. Similar underestimations using CFR_d have been reported in previous epidemic outbreaks of SARS [4, 6] and Ebola [20].

Alternatively, a simple and interpretable population-level mortality ratio is M_p(t) = D(t)/(R(t)+D(t)), the death ratio of all resolved cases. To provide a concrete model for D(t) and R(t), and hence M_p(t), we will use a variant of the standard infection duration-dependent susceptible-infected-recovered (SIR)-type model described by [21]

$$\begin{gathered} \frac{\text{d}S(t)}{\text{d}t}=-S(t){\int }_0^{\infty }\text{d}{\tau }^{\prime }\beta \left({\tau }^{\prime },t\right)I\left({\tau }^{\prime },t\right), \\ \frac{\partial I(\tau ,t)}{\partial t}+\frac{\partial I(\tau ,t)}{\partial \tau }=-(\mu (\tau ,t)+c(\tau ,t))I(\tau ,t), \end{gathered}$$ (9)

and $\text{d}R(t)/\text{d}t={\int }_0^{\infty }\text{d}\tau c(\tau ,t)I(\tau ,t)$, where S(t) is the number of susceptibles, I(τ,t) is density of individuals at time t who have been infected for time τ, and R(t) is the number of recovered individuals. The rate at which an individual infected for time τ at time t transmits the infection to a susceptible is denoted by β(τ,t)S(t).

Note that the equation for I(τ,t) is identical to the equation for the survival probability described by Eq. (1). It is also equivalent to McKendrick age-structured models [22, 23] [24]. Infection of susceptibles is described by the boundary condition

$$I(\tau =0,t)=S(t){\int }_0^{\infty }\text{d}{\tau }^{\prime }\beta \left({\tau }^{\prime },t\right)I\left({\tau }^{\prime },t\right),$$ (10)

which is similar to that used in age-structured models to represent birth [22]. Finally, we use an initial condition consistent with the infection duration density given by Eq. (7): I(τ,0) = ρ(τ;n = 8,γ = 1.25). Note that Eq. (10) assumes that all newly infected individuals are immediately identified; i.e., these newly infected individuals start with τ₁ = 0. After solving for the infected population density, the total number of deaths and recoveries to date can be found via

$$D_0(t)={\int }_0^t\text{d}{t}^{\prime }{\int }_0^{\infty }\text{d}\tau \mu \left(\tau ,t^{\prime }\right)I\left(\tau ,t^{\prime }\right),R_0(t)={\int }_0^t\text{d}{t}^{\prime }{\int }_0^{\infty }\text{d}\tau c\left(\tau ,t^{\prime }\right)I\left(\tau ,t^{\prime }\right).$$ (11)

The corresponding total number of cases N(t) in Eq. (8) is

$$N_0(t)=R_0(t)+D_0(t)+{\int }_0^{\infty }\text{d}\tau I(\tau ,t).$$ (12)

In the definitions of D₀(t), R₀(t), and N₀(t), we account for all possible death and recovery cases to date (see SI) and that newly infected individuals are immediately identified. We use these case numbers as approximations of the reported case numbers to study the evolution of mortality-ratio estimates. Mortality ratios based on these numbers underestimate the actual individual mortality M₁ (see section IIA) since they involve individuals that have been infected for different durations τ, particularly recently infected individuals who have not yet died.

An alternative way to compute populations is to exclude the newly infecteds and consider only the initial cohort. The corresponding populations in this case are defined as

$$D_1(t)={\int }_0^t\text{d}{t}^{\prime }{\int }_{t^{\prime }}^{\infty }\text{d}\tau \mu \left(\tau ,t^{\prime }\right)I\left(\tau ,t^{\prime }\right),R_1(t)={\int }_0^t\text{d}{t}^{\prime }{\int }_{t^{\prime }}^{\infty }\text{d}\tau c\left(\tau ,t^{\prime }\right)I\left(\tau ,t^{\prime }\right).$$ (13)

Since D₁(t) and R₁(t) do not include infecteds with τ < t, they exclude the effect of newly infected individuals, but may yield more accurate mortality-ratios as they are based on an initial cohort of individuals in the distant past. The infections that occur after t = 0 contribute only to I(τ < t,t); thus, D₁(t) and R₁(t) do not depend on the transmission rate β or the number of susceptibles S(t). Note that all the populations derived above implicitly average over ρ(τ₁;n,γ) for the first cohort of identified infecteds (but not subsequent infecteds). Moreover, the population density I(τ ≥ t,t) follows the same equation as $\overline{P}\left(t|{\tau }_1\right)$ provided the same ρ(τ₁;n,γ) is used in their respective calculations.

The two different ways of partitioning populations (Eqs. (11) and (13)) lead to two different population-level mortality ratios

$$M_{\text{p}}^0(t)=\frac{D_0(t)}{D_0(t)+R_0(t)}\text{ and }{M}_{\text{p}}^1(t)=\frac{D_1(t)}{D_1(t)+R_1(t)}.$$ (14)

Since the populations D₀(t) and R₀(t), and hence $M_{\text{p}}^0(t)$, depend on disease transmission through β(τ,t) and S(t), we expect $M_{\text{p}}^0(t)$ to carry a different interpretation from M₁(t) and $M_{\text{p}}^1(t)$.

In the special case in which μ and c are constants, the time-integrated populations ${\int }_0^t\text{d}{t}^{\prime }{\int }_0^{\infty }\text{d}\tau I\left(\tau ,t^{\prime }\right)$ and ${\int }_0^t\text{d}{t}^{\prime }{\int }_{t^{\prime }}^{\infty }\text{d}\tau I\left(\tau ,t^{\prime }\right)$ factor out of $M_{\text{p}}^0(t)$ and $M_{\text{p}}^1(t)$, rendering them time-independent and

$$M_{\text{p}}^{0,1}=\frac{{\mu }_1}{{\mu }_1+c}=M_1.$$ (15)

Thus, only in the special time-homogeneous case do both population-based mortality ratios become independent of the population (and transmission β) and coincide with the individual death probability.

To illustrate the differences between M₁(t), $M_{\text{p}}^{0,1}\left(t\right)$, and CFR_d(t,τ_res) in more general cases, we use the simple death and cure rate functions given by Eqs. (4) in solving Eqs. (1) and (9). For β(τ,t) in Eq. (10), we account for incubation effects by neglecting transmission during the asymptomatic incubation period (τ ≤ τ_inc) and assume

$$\beta (\tau ,t)=\{\begin{array}{cc}0& \tau \le {\tau }_{\text{inc}}\\ {\beta }_1& \tau >{\tau }_{\text{inc}}\end{array}.$$ (16)

We use the estimated basic reproductive number ${\mathcal{R}}_0={\beta }_{1}S(0)/\left({\mu }_1+c\right)\approx 2.91$ [17] to fix ${\beta }_{1}S(0)=\left({\mu }_1+c\right){\mathcal{R}}_0\approx 0.158/\text{day}$. We also first assume that the susceptible population does not change appreciably before quarantine and set S(t) = S(0). Thus, we only need to solve for I(τ,t) in Eqs. (9) and (10). We solve Eqs. (9) and (10) numerically (see the Methods section for further details) and use these numerical solutions to compute D_0,1(t), R_0,1(t), and N_0,1(t) (see Fig. 3(a) and (b)), which are then used in Eqs. (14) and CFR_d(t − τ_res). To determine a realistic value of the time lag τ_res, we use data on death/recovery periods of 36 tracked patients [16] and find that patients recover/die, on average, τ_res = 16.5 days after first symptoms occurred.

We show in Figs. 3(c) and (d) that $M_{\text{p}}^1(t)$ approaches the individual mortality ratio ${\overline{M}}_1(\infty )\approx 0.1$ of section IIA. This occurs because the model for P(τ,t) and I(τ,t) are equivalent and we assumed that the initial distribution of τ for both quantities are given by ρ(τ;8,1.25). However, the population-level mortality ratios CFR_d(t,τ_res) and $M_{\text{p}}^0(t)$ also take into account recently infected individuals who may recover before symptoms. This difference yields different mortality ratios because newly infecteds are implicitly assumed to be detected immediately and all have τ₁ = 0. Thus, the underlying infection-time distribution is not the same as that used to compute ${\overline{M}}_{\text{p}}^1(t)$ (see SI for further details). The mortality ratios CFR_d(t,τ_res) and $M_{\text{p}}^0(t)$ should not be used to quantify the individual mortality probability of individuals who tested positive after their incubation period. During the course of an outbreak, the measures CFR_d(t,τ_res) and $M_{\text{p}}^0(t)$ are subject to another confounding influence. Since D(t), R(t), and N(t) do not change with the same rates at the same time, these population-level mortality estimates only reach their steady state after sufficiently long times (see Fig. 3(c) and (d)).

To summarize, we described two confounding factors that complicate the direct use of population-level mortality ratio to estimate individual mortality probabilities. First, infection-time distributions ρ(τ;n,γ) that are meaningful on an individual level may not correspond to those in population-level data. Second, population-level mortality ratios are often time-dependent and most informative only in the steady state after the outbreak stopped.

The evolution of the mortality ratios in Fig. 3 qualitatively resembles the behavior of the mortality-ratio estimates in Fig. 1. As shown in Fig. 1, the population-based estimates for coronavirus varies, decreasing in time for China but fluctuating for Italy. These changes could result from changing practices in data collecting, or from explicitly time-inhomogeneous parameters μ(τ,t), c(τ,t), and/or β(τ,t).

Although population-level quarantining does not directly affect the individual mortality M₁(t|τ₁) or ${\overline{M}}_1(t)$, it can be easily incorporated into the SIR-type population dynamics equations through changes in β(τ,t)S(t). For example, we have set S(t > t_q) = 0 to represent implementation of a quarantine after t_q = 50 days of the outbreak. After t_q = 50 days, no new infections occur and the estimates CFR_d(t,τ_res) and $M_{\text{p}}^0(t)$ start converging immediately towards their steady-state values (see Fig. 3(d)). Since the number of deaths decreases after the implementation of quarantine measures, the delayed CFR_d(t,τ_res = 17) is first decreasing until t = t_q +τ_res = 67. For t > 67, the CFR_d(t,τ_res = 17) measures no new cases and is thus equal to the CFR.

III. DISCUSSION AND SUMMARY

After an outbreak, it is important to assess the severity of the disease by estimating its mortality and other disease characteristics. Assuming accurate data, the often-used CFR and delayed CFR typically underestimate the true, final death ratio. For example, during the SARS outbreaks in Hong Kong, the WHO first estimated the fatality rate to 2.5% (March 30, 2003) whereas the final estimates reached values of about 17.0% (June 30, 2003) [7]. Standard metrics like the CFR are seen to be easily confounded by and sensitive to uncertainty in intrinsic disease parameters such as the incubation period and the time τ₁ a patient had been infected before clinical confirmation of infection. For the recent COVID-19 outbreaks, CFR-based measures may still provide reasonable estimates of the actual mortality across different age classes due to a counter-acting error in the numbers of unreported mild-symptom cases.

Here, we stress that more mechanistically meaningful and interpretable metrics can be defined and be as easily estimated from data as CFRs. Our proposed mortality ratios for viral epidemics are defined in terms of (i) individual survival probabilities and (ii) population ratios using numbers of deaths and recovered individuals. Both of these measures are based on the within-host evolution of the disease, and in the case of $M_{\text{p}}^{0,1}(t)$, the population-level transmission dynamics. Thus, these metrics directly incorporate key parameters operating on the weeks or months timescale, the incubation time τ_inc and time of prior infection τ, through the solution of age-structured PDEs. Among the metrics we describe, $M_{\text{p}}^1(t)$ is structurally closest to ${\overline{M}}_1(t)$ in that both are independent of transmission β since new infections are not considered. Both of these converge after an incubation time τ_inc to a value smaller than or equal to μ₁/(μ₁ + c).

The most accurate estimates of M₁ can be obtained if we keep track of the fate of cohorts that were infected within a small time window in the past. By following only these individuals, one can track how many of them died as a function of time. As more cases arise, one should stratify them according to estimated τ to gather improving statistics for M₁(∞). These data should also be collated according to the other central factor in COVID-19 mortality: patient age. With the further spread of SARS-CoV-2 in different countries, data on more individual cases of death and recovery can be more easily stratified by age, health condition, and other individual characteristics. Using identical initial infection time distributions ρ(τ₁;n,γ) (see Eq. (7)), the long-time limit of $M_{\text{p}}^1(t)$ approaches the individual mortality ${\overline{M}}_1(\infty )$ (see Eq. (6)).

Besides accurate cohort data, for which at present there are few for coronavirus, cumulative population data has been used to estimate the mortality ratio. The metrics $M_{\text{p}}^0(t)$ and CFR(t) are based on these aggregate populations but implicitly depend on new infections and the transmission rate β. Despite this confounding factor, $M_{\text{p}}^0(t)$ and CFR_d(t,τ_res) approach $e^{-c{\tau }_{\text{inc }}}{\mu }_1/\left({\mu }_1+c\right)$ as t → ∞, where $e^{-c{\tau }_{\text{inc }}}$ is the probability that no recovery occurred during the incubation time τ_inc. Based on these results, we can establish the following connection between the different mortality ratios for initial infection times with distribution ρ(τ₁;n,γ) and mean $\overline{\tau }=n/\gamma$:

$${\text{CFR}}_{\text{d}}(\infty )=M_{\text{p}}^0(\infty )\approx e^{-c\overline{\tau }}{M}_{\text{p}}^1(\infty )=e^{-c\overline{\tau }}{\overline{M}}_1(\infty ).$$ (17)

According to Eq. (17), population-level mortality estimates (e.g., CFR and $M_{\text{p}}^0$ can be transformed, at least approximately, into individual mortality probabilities using the correction factor $e^{-c\overline{\tau }}$ with $\overline{\tau }\approx {\tau }_{\text{inc}}$.

Besides the mathematical differences between M₁(t) and $M_{\text{p}}^0(t)$, CFR, estimating $M_{\text{p}}^0(t)$ and CFR(t) from aggregate populations implicitly incorporate a number of confounding factors that lead to variability in these estimates. In Fig. 4, we plot the population-level mortality-ratio estimates $M_{\text{p}}^0$ against the CFR for different regions and observe large variations and very little correlation between countries [25]. As of March 26, 2020, the value of $M_{\text{p}}^0$ in Italy is almost 45% and can increase further if the current conditions (e.g., treatment methods, age group proportion of infecteds, etc.) do not change. Differences between the mortality ratios in China and Italy (see Figs. 1(b) and (c)) might be a result of varying medical treatment strategies, different practices in data collecting (e.g., post-mortem testing), and differences in the age demographics between the countries.

FIG. 4. Region-dependence of COVID-19 mortality-ratio estimates.

Mortality-ratio estimates of COVID-19 in different regions (see Eqs. (8) and 14 (τ_res = 0)). We used data on the cumulative number of cases, recoveries, and deaths in Ref. [5] as of March 24, 2020. The marker sizes indicate the population of the corresponding countries. The metrics $M_{\text{p}}^0(t)$ and and CFR are largely uncorrelated with correlation coefficient 0.33.

In general, even if the cohort initially tested was only a fraction of the total infected population, tracking ${\overline{M}}_1\left(t\right)$ or $M_{\text{p}}^1(t)$ of this cohort still provides an accurate estimation of the mortality rate. However, the newly infecteds that contribute to CFR and $M_{\text{p}}^0(t)$ at later times may not all be tested or may be tested at different times after they were infected. A reported/tested fraction f < 1 would not directly affect the CFRs or mortality ratios if the unreported/untested individuals die and recover in the same proportion as the tested infecteds. Undertesting will overestimate true CFR or mortality rates if the untested infecteds are less likely to die than the tested infecteds. In other words, if the untested (presumably because they were mildly or asymptomatic) population predominantly recovers instead of dying, the actual CFR and mortality ratios would be significantly lower than those based on tested individuals. If untested infecteds do not die, the asymptotic mortality of all infected individuals ${\mathcal{M}}_{\text{p}}^{0,1}(\infty )\approx M_{\text{p}}^{0,1}(\infty )$ (see the SI). Current estimates show that only a minority of SARS-CoV-2 infections are reported (e.g., f ≈ 14% in China before January 23, 2020) [26].

Besides under-reporting, the delay in transmission after becoming infected will also affect $M_{\text{p}}^0(t)$. Although we have assumed that transmission occurs only after the incubation period when symptoms arise, there is evidence of asymptomatic transmission of coronavirus [26, 27]. Asymptomatic transmission can be modeled by setting β(τ) > 0 even for τ < τ_inc. An undelayed transmission in a nonquarantine scenario causes relatively more new infecteds who have not had the chance to die yet, leading to a smaller mortality ratio $M_{\text{p}}^0(t)$. Within our SIR model, delaying transmission reduces the number of infected individuals and deaths at any given time but increases the measured mortality ratio $M_{\text{p}}^0(t)$. Without quarantine, the asymptotic values $M_{\text{p}}^0(\infty )$ and CFR(∞) will also change as a result of changing the transmission latency period, as shown in the SI. With perfect quarantining, the asymptote $M_{\text{p}}^0(\infty )$ is eventually determined by a cohort that does not include new infections and is thus independent of the transmission delay.

In this work, we have explicitly defined a number of interpretable mathematical metrics that represent the risk of death. By rigorously defining these metrics, we are able to reveal the inherent assumptions and factors that affect their estimation. Within survival probability and SIR-type models, we explicitly illustrate how physiologically important parameters such as incubation time, death rate, cure rate, and transmissibility influence the metrics. We also discussed how statistical factors such as time of testing after infection (τ₁) and testing ratio (f) affect our estimates. Given the uncertainty in the testing fraction, we conclude that M₁(t) and $M_{\text{p}}^1(t)$ is best interpreted as approximately the mortality probability conditioned on being tested positive. In practice, these are probably also good estimates of mortality of patients conditioned on showing symptoms. In addition to our metrics and mathematical models, we emphasize the importance of curating individual cohort data. These data are more directly related to the probability of death M₁(t) and are subject to the fewest confounding factors and statistical uncertainty.

METHODS

To numerically solve Eqs. (9) and (10), we used a uniform discretization τ_k = kΔτ,k = 0,1,…,K. A backward difference operator [I(τ_k,t) − I(τ_k−1,t)]/(Δτ) is used to approximate ∂_τI(τ,t) and a predictor-corrector Euler scheme is used to advance time [28]. Setting the cut-off I(−Δτ,t) ≡ 0 and I(KΔτ,t) ≡ 0, the resulting discretized equations for the full SIR model are

$$\begin{gathered} S(t+\Delta t)=S(t)-\Delta tS(t)\sum _{k=0}^K\beta \left({\tau }_k,t\right)I\left({\tau }_k,t\right)\Delta \tau , \\ \tilde{I}\left({\tau }_k,t\right)=I\left({\tau }_k,t\right)-\Delta t\frac{I\left({\tau }_k,t\right)-I\left({\tau }_{k-1},t\right)}{\Delta \tau }-\Delta t\left(c\left({\tau }_k,t\right)+\mu \left({\tau }_k,t\right)\right)I\left({\tau }_k,t\right), \\ I\left({\tau }_k,t+\Delta t\right)=I\left({\tau }_k,t\right)-\frac{\Delta t}{2}\left[\frac{I\left({\tau }_k,t\right)-I\left({\tau }_{k-1},t\right)}{\Delta \tau }+\left(c\left({\tau }_k,t\right)+\mu \left({\tau }_k,t\right)\right)I\left({\tau }_k,t\right)+\frac{\tilde{I}\left({\tau }_k,t\right)-\tilde{I}\left({\tau }_{k-1},t\right)}{\Delta \tau }+\left(c\left({\tau }_k,t+\Delta t\right)+\mu \left({\tau }_k,t+\Delta t\right)\right)\tilde{I}\left({\tau }_k,t\right)\right]+{\delta }_{k,0}\frac{\Delta t}{\Delta \tau }S(t)\sum _{j=0}^K\beta \left({\tau }_j,t\right)I\left({\tau }_j,t\right)\Delta \tau , \end{gathered}$$ (18)

where $\tilde{I}$ is the initial predicted guess, and the last term proportional to δ_k,0 encodes the boundary condition Eq. (10). Note that we use ${\sum }_{k=0}^K\beta \left({\tau }_k,t\right)I\left({\tau }_k,t\right)\Delta \tau$ to indicate the numerical evaluation of ${\int }_0^{\infty }\text{d}{\tau }^{\prime }\beta \left({\tau }^{\prime },t\right)I\left({\tau }^{\prime },t\right)$. Quadrature methods such as Simpson’s rule and the trapezoidal rule can be used to approximate the integral more efficiently.

The total deaths, recovereds, and infecteds at time t are found by

$$D_0(m\Delta t)=\frac{1}{2}\sum _{j=0}^m\sum _{k=0}^{K}c(k\Delta \tau ,j\Delta t)[I(k\Delta \tau ,j\Delta t)+\tilde{I}(k\Delta \tau ,j\Delta t)]\Delta \tau \Delta t,$$

$$R_0(t)=\frac{1}{2}\sum _{j=0}^m\sum _{k=0}^K\mu (k\Delta \tau ,j\Delta t)[I(j\Delta \tau ,j\Delta t)+\tilde{I}(k\Delta \tau ,j\Delta t)]\Delta \tau \Delta t,$$

$$I(m\Delta t)=\sum _{k=0}^{K}I(k\Delta \tau ,m\Delta t)\Delta \tau ,$$

with analogous expressions for D₁(mΔt) and R₁(mΔt). To obtain a stable integration scheme, the time steps Δt and Δτ have to satisfy Δt/(2Δτ) < 1. In all of our numerical computations, we thus set Δt = 0.002,Δτ = 0.02, and K = 10⁴. In the SI, we show additional plots of the magnitude of I(τ,t) in the t − τ plane.