DDE: Deep Dynamic Epidemiological Modeling for Infectious Illness Development Forecasting in Multi-level Geographic Entities

Abstract

Understanding and addressing the dynamics of infectious diseases, such as coronavirus disease 2019, are essential for effectively managing the current situation and developing intervention strategies. Epidemiologists commonly use mathematical models, known as epidemiological equations (EE), to simulate disease spread. However, accurately estimating the parameters of these models can be challenging due to factors like variations in social distancing policies and intervention strategies. In this study, we propose a novel method called deep dynamic epidemiological modeling (DDE) to address these challenges. The DDE method combines the strengths of EE with the capabilities of deep neural networks to improve the accuracy of fitting real-world data. In DDE, we apply neural ordinary differential equations to solve variant-specific equations, ensuring a more precise fit for disease progression in different geographic regions. In the experiment, we tested the performance of the DDE method and other state-of-the-art methods using real-world data from five diverse geographic entities: the USA, Colombia, South Africa, Wuhan in China, and Piedmont in Italy. Compared to the state-of-the-art method, DDE significantly improved accuracy, with an average fitting Pearson coefficient exceeding 0.97 across the five geographic entities. In summary, the DDE method enhances the accuracy of parameter fitting in epidemiological models and provides a foundation for constructing simpler models adaptable to different geographic areas.

Introduction

The coronavirus disease 2019 (COVID-19), caused by the virus SARS-CoV-2, has witnessed a catastrophic global spread, affecting over 214 countries and territories across all seven continents. This unprecedented pandemic has led to more than 103 million confirmed cases and an excess of 2 million fatalities [1]. The profound impact of COVID-19 extends beyond public health, significantly influencing economic and social development worldwide. The diverse circumstances surrounding the outbreak, including variations in quarantine measures, social distancing protocols, population density, and healthcare infrastructure, have resulted in distinct patterns of COVID-19 transmission across different regions [2, 3]. Given these disparities, the development of analytical models is imperative for comprehensively understanding the disease’s trajectory and projecting future trends. Such models play a pivotal role in analyzing the spread dynamics and formulating effective strategies to mitigate the outbreak’s impact on various geographic scales.

Many studies propose mathematical models to simulate the progression of infectious diseases, with the Susceptible-Infected-Removed (SIR) and Susceptible-Exposed-Infected-Removed (SEIR) models (Fig. 1) being widely employed by epidemiologists due to their simplicity, utilizing only infectious, exposed, and recovered rates to characterize disease dynamics [5]. However, the complex nature of real-world infection dynamics often challenges existing parameter estimation methods, resulting in suboptimal fitting performance for SIR and SEIR models [5]. To address these limitations, some studies have introduced variants to SIR or SEIR models, aiming to better capture diverse population trends with multiple parameters tailored to specific geographic areas, yielding higher precision in fitting performance [6–11]. Nevertheless, due to the unique characteristics of each region (with some regions experiencing nascent outbreaks while others observe diminishing cases), designing distinct models for individual regions is time-intensive. Moreover, precise modeling is crucial for estimating the spread of COVID-19. Therefore, the development of simple yet accurate models, adaptable to a wide range of countries or regions and easily transferable to new areas, holds significant value in COVID-19 modeling efforts.

Fig. 1.

The architectures of Susceptible-Infected-Removed (SIR) and Susceptible-Exposed-Infected-Removed (SEIR) models. A Structure figure for SIR model. B Structure figure for SEIR model. $\beta$ in the figures represents the transmission rate of infection, $\sigma$ is the rate of moving from exposed to infectious class, and $\gamma$ shows the recovery rate [4]

Machine learning has made significant strides in addressing numerous complex data-driven challenges, including the prediction of medical data. Consequently, it has been increasingly utilized to model COVID-19 data, offering robust data fitting capabilities particularly in early-stage modeling [12–16]. However, there remains a gap in exploring the forecasts under varied development scenarios during the middle and late periods of the pandemic. Moreover, the inherent black-box nature of these machine learning models enables users to obtain predicted results directly, yet it hampers interpretability as the reasons behind predictions remain unclear. Consequently, these models exhibit low explainability, and their prediction process lacks analytical and quantifiable assessment.

Due to the limitations of existing methods, we introduce a novel deep dynamic epidemiological modeling (DDE) method, which integrates neural ordinary differential equations (Neural ODE) [17] and epidemiological equations (EE) for COVID-19 data fitting. Recognizing the diverse characteristics of countries and regions, we propose four variant models based on the SIR and SEIR models, namely SIRD, SEIRD, SMCRD, and SEMCRD. These models accommodate variations in population group classifications and can handle different data sources, such as instances where specific numbers of mild and severe cases are not explicitly provided. Furthermore, we developed the DDE framework to fit real-world data based on these four variants. We designed an auxiliary neural network within the Neural ODE solver to refine the effect function, thereby capturing the diverse impacts of regional and national intervention policies on the infection rate. In summary, the contributions of this paper can be delineated as follows.

1. We introduce the DDE method, which integrates neural networks (NN) and Neural ODE to facilitate parameter fitting in SEIR-like models. This approach enables the derivation of transmission rates across various population groups through network training and provides visualizations depicting the evolution of these rates. Notably, DDE demonstrates superior performance, yielding Pearson correlation coefficients exceeding 0.97 across all datasets.

2. We develop four variants of the SIR/SEIR models, namely SIRD, SEIRD, SMCRD, and SEMCRD, designed to accommodate the diverse characteristics observed across regions and countries. Employing the DDE method enhances the precision of parameter estimation compared to state-of-the-art approaches.

3. We conduct a comparative analysis of the DDE method against other learning-based models. Our findings reveal that the DDE method consistently outperforms state-of-the-art methods, achieving significantly higher average Pearson coefficients (P < 0.05). Furthermore, we examine the output trends of the neural networks and demonstrate their correlation with real-world data.

Related Work

SEIR and their Variants

The SEIR model and its variants are widely employed in epidemiological analyses. Within the SEIR model, the population is segmented into four categories: susceptible individuals (S), exposed individuals (E), infected individuals (I), and recovered individuals (R). The dynamics of these compartments are governed by a system of coupled ordinary differential equations. Previous studies investigating infectious diseases, such as SARS and MERS, which have sparked global epidemics, often rely on the SEIR model for epidemiological analysis [5, 18, 19]. Similarly, in the context of COVID-19 research, numerous studies have focused on variants of the SEIR model tailored to specific regions or cities [6, 8–11]. For instance, Peng et al. [6] proposed a generalized SEIR model to analyze the COVID-19 epidemic in China, while Choi et al. [8] introduced an enhanced SEIR model, termed the SEIAQIm model, based on data from Korea. Furthermore, novel extensions of the SEIR framework have been proposed to capture additional complexities in disease transmission dynamics. Watson et al. [9] presented a population-based, age-structured Susceptible-Exposed-Infectious-Recovery-Susceptible (SEIRS) model, employing a Metropolis-Hastings Markov Chain Monte Carlo sampling scheme to fit the model to COVID-19 mortality data within a Bayesian framework. Additionally, Leung et al. [10] developed an age-structured susceptibility-exposure-infection-removal (SEIR) model to simulate the spread of the Omicron variant in Beijing. The proposed SEIR variants [8–10] are tailored to particular countries or regions and often feature intricate parameter designs.

Statistical Methods

Statistical models play a vital role in modeling specific aspects of real-world data. For instance, Kraemer et al. [20] utilized generalized linear models to examine the impact of human mobility and control measures during the early stages of the COVID-19 outbreak in Wuhan, China. Additionally, Chinazzi et al. [21] proposed the global epidemic and mobility model to analyze the effects of travel restrictions on the global spread of COVID-19, utilizing parameters derived from previous epidemiological studies of SARS and MERS coronaviruses. Furthermore, Kucharski et al. [22] investigated the repercussions of cases exported from Wuhan on other regions using a stochastic transmission dynamic model. While these statistical models have provided valuable insights into specific aspects of the pandemic, they exhibit diversity in their designs and complexity.

Learning-Based Methods

Some studies addressing the issue of COVID-19 data trend prediction have turned to machine learning approaches, driven by perceived deficiencies in SEIR models and statistical methods. For instance, Yang et al. [13] integrated population migration data before and after January 23, along with the latest COVID-19 epidemiological data, into the SEIR model to forecast the epidemic curve. Similarly, Rustam et al. [12] proposed the utilization of various machine learning models, including linear regression, least absolute shrinkage and selection operator, support vector machine, and exponential smoothing, to predict infection, recovery, and mortality rates. However, these approaches either relied on historical infectious disease data for training or lacked interpretability. Furthermore, Dandekar et al. [23] attempted to leverage the Neural ODE method to model changes in the infection and recovery groups. However, this method primarily focused on enhancing the performance of the SIR model and fitting real-world data, with experimentation limited to the early stages of the COVID-19 pandemic (before April). Moreover, comparative analyses with other methods were lacking, and the method did not assess data fitting across various scales of regions (countries, regions, cities), hindering its practical application.

Method

In this section, we propose a data-driven model for fitting real-world data. Firstly, we introduce four variants derived from the SEIR and SIR models to enhance the fitting accuracy of COVID-19 data. Subsequently, we elaborate on the DDE method for dynamic parameter estimation. Within our DDE framework, multi-layer neural networks are incorporated to model intervention influence functions, enabling the assessment of the impact of quarantine policies across different countries. Figure 2 illustrates the computational process involved in our DDE method for dynamic parameter estimation.

Fig. 2.

The overall structure of the proposed DDE model. A The model structure of DDE method. The real data are represented by solid frames, and the predictions are shown by dotted frames. The system consists of four parts: Firstly, the current values of the equations are inputted into the parameter generation network to derive corresponding parameters. Subsequently, these generated parameters, along with the current values, are utilized to compute the function value at the subsequent time step using the Neural ODE solver. This process iterates multiple times until the solution converges to the predicted value. Finally, the loss function between the predicted value and the ground truth is calculated, enabling optimization through gradient descent. B The schematic plot of the process of solving the epidemiological equations. C The process plot provides a detailed illustration of parameter estimation

The SIR model partitions the population into three distinct groups: the susceptible (S), the infectious (I), and the recovered (R). Here, the S group represents healthy individuals, while the I group comprises those who have been infected. The R group consists of individuals who have successfully recovered from the infection. The SEIR model incorporates an additional compartment for the exposed group (E), representing individuals who have been exposed to the virus but are not yet infectious. The formulation of the SEIR model is depicted below:

$$\begin{array}{cc}\hfill \frac{\mathit{dS}}{\mathit{dt}}& =-\beta ·\frac{S·I}{N}\hfill \\ \hfill \frac{\mathit{dE}}{\mathit{dt}}& =\beta ·\frac{S·I}{N}-\sigma ·E\hfill \\ \hfill \frac{\mathit{dI}}{\mathit{dt}}& =\sigma ·E-\gamma ·I\hfill \\ \hfill \frac{\mathit{dR}}{\mathit{dt}}& =\gamma ·I\hfill \end{array}$$ 1

where $\beta$ is the transmission rate of infection, $\sigma$ is rate of moving from exposed to infectious class, $\gamma$ is the recovery rate, and $N=S+I+E+R$ is the number of the total population.

Four Variants: SIRD, SEIRD, SMCRD, and SEMCRD

The classic SEIR model has been extensively utilized in numerous prior studies [5, 18, 19]. Moreover, in the context of COVID-19 research, the SEIR model remains prevalent [24–26]. Given the distinctive characteristics of COVID-19, we identify several areas for enhancing the traditional SEIR and SIR models:

1. We introduce a novel approach to categorize the recovery population, distinguishing between fatalities and individuals undergoing disease rehabilitation. This refinement involves partitioning the original recovery group into two distinct subgroups: the deceased population and the recovery (disease rehabilitation) population. To accommodate the varied death and recovery scenarios observed across different countries, we propose the incorporation of two parameters: mortality rate and recovery rate.

2. We refine the definition of the infected population to encompass two subcategories: the mild population and critical patients. The mild population encompasses asymptomatic individuals and those with self-recovering mild symptoms, whereas critical patients include severe cases requiring hospitalization. To capture the dynamics of real infections more accurately, we introduce three parameters: the mild infectious rate, the transition rate from mild to critical, and the recovery rates for both mild and critical cases. This refinement enhances the fidelity of our model in simulating real-world infection dynamics.

Therefore, we implement four variants: SIRD, SEIRD, SMCRD, and SEMCRD. Figure 3 illustrates the model structures of these four variants. Compared to the SIR and SEIR models, the SIRD and SEIRD models incorporate considerations for the deceased population, facilitating a comprehensive understanding of mortality growth attributed to the disease. Additionally, the SMCRD and SEMCRD models, derived from the SIRD and SEIRD models respectively, further delineate the infected group I into mild and critical cases. This segmentation aligns with the nuanced characteristics of COVID-19, enabling more accurate modeling of real-world data. The formulations of the SMCRD and SEMCRD models are depicted below:

$$\begin{array}{cc}\hfill \frac{\mathit{dS}}{\mathit{dt}}& =-\beta ·\frac{S·(M+C)}{N}\hfill \\ \hfill \frac{\mathit{dM}}{\mathit{dt}}& =\beta ·\frac{S·(M+C)}{N}-\alpha ·M-{\gamma }_{\mathit{MR}}·M\hfill \\ \hfill \frac{\mathit{dC}}{\mathit{dt}}& =\alpha ·M-{\gamma }_{\mathit{CR}}·C-{\gamma }_D·C\hfill \\ \hfill \frac{\mathit{dR}}{\mathit{dt}}& ={\gamma }_{\mathit{MR}}·M+{\gamma }_{\mathit{CR}}·C\hfill \\ \hfill \frac{\mathit{dD}}{\mathit{dt}}& ={\gamma }_D·C\hfill \end{array}$$ 2

where $\alpha$ is the rate of moving from people with mild symptoms to people with severe symptoms, ${\gamma }_{\mathit{MR}}$ is the rate of moving from people with mild symptoms to recovered, ${\gamma }_{\mathit{CR}}$ is the rate of moving from people with severe symptoms to recovered, ${\gamma }_D$ is the rate of moving from people with severe symptoms to death, and $N=S+M+C+R+D$ is the number of the total population.

$$\begin{array}{cc}\hfill \frac{\mathit{dS}}{\mathit{dt}}& =-\beta ·\frac{S·(M+C)}{N}\hfill \\ \hfill \frac{\mathit{dE}}{\mathit{dt}}& =\beta ·\frac{S·(M+C)}{N}-\sigma ·E\hfill \\ \hfill \frac{\mathit{dM}}{\mathit{dt}}& =\sigma ·E-\alpha ·M-{\gamma }_{\mathit{MR}}·M\hfill \\ \hfill \frac{\mathit{dC}}{\mathit{dt}}& =\alpha ·M-{\gamma }_{\mathit{CR}}·C-{\gamma }_D·C\hfill \\ \hfill \frac{\mathit{dR}}{\mathit{dt}}& ={\gamma }_{\mathit{MR}}·M+{\gamma }_{\mathit{CR}}·C\hfill \\ \hfill \frac{\mathit{dD}}{\mathit{dt}}& ={\gamma }_D·C\hfill \end{array}$$ 3

where $N=S+E+M+C+R+D$ is the number of the total population.

Fig. 3.

The architectures display the four variants based on SEIR and SIR models: SIRD, SMCRD, SEIRD, SEMCRD. In the SIRD model, the transmission rate of infection $\beta$, recovery rate ${\gamma }_R$, and mortality rate ${\gamma }_D$ are considered. Furthermore, the SEIRD model takes the exposed group E into account and assumes the exposed rate as $\sigma$. The SMCRD and SEMCRD models divide the infection group into people with mild symptoms M and people with severe symptoms C. Thus, the rate of moving from people with mild symptoms to people with severe symptoms is $\alpha$, the recovery rate of the people with mild symptoms is ${\gamma }_{\mathit{MR}}$, the recovery rate of the people with severe symptoms is ${\gamma }_{\mathit{CR}}$, the death rate of the people with severe symptoms ${\gamma }_D$ [4]

The unified equation form of these models is as follows:

$$\begin{array}{c}\hfill \frac{d\mathbf{Z}(t)}{\mathit{dt}}=\mathcal{F}(\mathbf{Z}(t),t,\theta ),\text{with}\mathbf{Z}(t_0)={\mathbf{Z}}_0\end{array}$$ 4

where $t\in \{t_0,...,t_i,...,t_T\}$ ($t_0$ stands for the initial day, and $t_i$ represents the ith day from $t_0$), $\mathbf{Z}(t)\in {\mathbb{R}}^D$, ${\mathbf{Z}}_0=[S_0,E_0,I_0,R_0,D_0]$ for SEIRD model, ${\mathbf{Z}}_0=[S_0,I_0,R_0,D_0]$ for SIRD model, ${\mathbf{Z}}_0=[S_0,E_0,M_0,C_0,R_0,D_0]$ for SEMCRD model, and ${\mathbf{Z}}_0=[S_0,M_0,C_0,R_0,D_0]$ for SMCRD model, ${\mathbf{Z}}_0$ is the function value at time $t_0$. The $\mathcal{F}(·)$ is a known and continuous function with parameter $\theta$, and $\mathbf{Z}(t)$ is the unknown function that must be approximated. For the SIRD model, the parameter set $\theta$ includes $\beta ,{\gamma }_R,{\gamma }_D$. The SEIRD model’s parameter set $\theta$ includes $\beta ,\sigma ,{\gamma }_R,{\gamma }_D$. Further, for SMCRD model, $\theta$ includes $\beta ,\alpha ,{\gamma }_{\mathit{MR}},{\gamma }_{\mathit{CR}},{\gamma }_D$. In SEMCRD model, $\theta$ includes $\beta ,\sigma ,\alpha ,{\gamma }_{\mathit{MR}},{\gamma }_{\mathit{CR}},{\gamma }_D$.

From a computational point of view, knowing that $\mathbf{Z}(t_0)={\mathbf{Z}}_0$, you can calculate the value of $\mathbf{Z}(t_i)={\mathbf{Z}}_i$ in any step $t_i$ by performing piecewise integration from previously known points:

$$\begin{array}{c}\hfill {\mathbf{Z}}_i={\mathbf{Z}}_{i-1}+{\int }_{t_{i-1}}^i\mathcal{F}({\mathbf{Z}}_{i-1},t_{i-1},\theta )\text{with}i\in \{1,...,T\}\end{array}$$ 5

When $\mathrm{\Delta }\tau =t_i-t_{i-1}$ is small enough, we can get the approximation result:

$$\begin{array}{c}\hfill {\mathbf{Z}}_i={\mathbf{Z}}_{i-1}+\mathrm{\Delta }\tau ·\mathcal{F}({\mathbf{Z}}_{i-1},t_{i-1},\theta )\end{array}$$ 6

Thus, in each time step $t_i$, the value of function ${\mathbf{Z}}_i$ can be obtained by deduction of the function value ${\mathbf{Z}}_{i-1}$ at the previous moment $t_{i-1}$.

DDE Method Applying in SMCRD and SEMCRD Models

Neural networks are renowned for their robust fitting capabilities. In our DDE model, we employ multi-layer neural networks (MNNs) to model the effect function’s values, utilizing them as dynamic parameters within the Neural ODE solution process. Figure 4 elucidates the utilization of multi-layer neural networks in parameter design. The estimation of these designed parameters draws inspiration from the Neural ODE method [17]. In the preceding section, we introduced four variants based on the SEIR or SIR model and outlined the parameter designs for SIRD, SEIRD, SMCRD, and SEMCRD models. However, in real-world data fitting, diverse countries and regions often implement varying intervention and isolation measures, eliciting different responses from the populace. Consequently, we revise the parameter set $\theta$ to account for the myriad changes in infection scenarios:

$$\begin{array}{c}\hfill \theta =[\beta \ast ,W,{\gamma }_R,{\gamma }_D]\text{, for SIRD}\\ \hfill \theta =[\beta \ast ,W,\sigma ,{\gamma }_R,{\gamma }_D]\text{, for SEIRD}\\ \hfill \theta =[\beta \ast ,W,\alpha ,{\gamma }_{\mathit{MR}},{\gamma }_{\mathit{CR}},{\gamma }_D]\text{, for SMCRD}\\ \hfill \theta =[\beta \ast ,W,\sigma ,\alpha ,{\gamma }_{\mathit{MR}},{\gamma }_{\mathit{CR}},{\gamma }_D]\text{, for SEMCRD}\end{array}$$ 7

where the $\beta \ast$ is the transmission rate of infection which could be influenced by effect function $Ef{f}_t$. The effect function $Ef{f}_t$ uses the MNNs to fit. The W represents the weights in the MNNs. The structure of the MNNs is shown in Fig. 5.

Fig. 4.

The DDE method is implemented in SMCRD and SEMCRD models. In the DDE-SMCRD and DDE-SEMRD models, neural network (NN) models are used to calculate dynamic $\beta$ (initial $\beta \times EE{f}_t$) based on the neural ODE method

Fig. 5.

The structure figure of the MNNs that are used to fit the effect function $Ef{f}_t$ on the dynamic transmission rate of infection $\beta$

According to Fig. 5, we can obtain the output of the first hidden layer of the MNNs as follows:

$$\begin{array}{c}\hfill H_{k_1}^1(t_i)=a(\sum _{j=1}^{inpu{t}_u}{W}_{j·k_1}^1·Inpu{t}_j(t_i)+b_{k_1}^1)\end{array}$$ 8

where $H_{k_1}^1(t_i)$ is the $k_1$th of unit of the first hidden layer output in time step $t_i$, the input tensor $Input(t_i)$ =[$Inpu{t}_0(t_i)$, $Inpu{t}_1(t_i)$, $Inpu{t}_2(t_i)$, $Inpu{t}_3(t_i)$, $Inpu{t}_4(t_i)$] for SEMCRD ($Inpu{t}_0$ is $E_{t_i}$, $Inpu{t}_1(t_i)$ is $M_{t_i}$, $Inpu{t}_2(t_i)$ is $C_{t_i}$, $Inpu{t}_3(t_i)$ is $R_{t_i}$, and $Inpu{t}_4(t_i)$ is $D_{t_i}$) and Input(t) =[$Inpu{t}_0(t_i)$, $Inpu{t}_1(t_i)$, $Inpu{t}_2(t_i)$, $Inpu{t}_3(t_i)$] for SMCRD ($Inpu{t}_0(t_i)$ is $M_{t_i}$, $Inpu{t}_1(t_i)$ is $C_{t_i}$, $Inpu{t}_2(t_i)$ is $R_{t_i}$, and $Inpu{t}_3(t_i)$ is $D_{t_i}$), the $inpu{t}_u$ is the number of $Input(t_i)$, it is 5 for SEMCRD model or is 4 for SMCRD model. The $W_{j·k_1}^1$ represent the weight of the input layer and the first hidden layer in the connection weight of the jth input unit and the $k_1$th of the first hidden unit. The $b_{k_1}^1$ is the bias of the $k_1$th unit of the first hidden layer, and the $a(·)=ELU(·)$ is the activation function of the hidden layer.

The output of the 2nd hidden layer to the Nth hidden layer also can be deduced in the following:

$$\begin{array}{cc}\hfill H_{k_2}^2(t_i)=& a(\sum _{k_1=1}^{h_1\_u}{W}_{k_1·k_2}^2·H_{k_1}^1(t_i)+b_{k_2}^2)\hfill \\ \hfill & ......\hfill \\ \hfill H_{k_N}^N(t_i)=& a(\sum _{k_{N-1}=1}^{h_{N-1}\_u}{W}_{k_{N-1}·k_N}^N·H_{k_{N-1}}^{N-1}(t_i)+b_{k_N}^N)\hfill \end{array}$$ 9

where $a(·)$ is the activation function, and all hidden layers use the same activation function, $h_{N-1}\_u$ is the number of hidden units in the $N-1$th hidden layer, $b_{k_N}^N$ is the bias of the $k_N$th unit of the Nth hidden layer, $W_{k_{N-1}·k_N}^N$ represents the parameters between the $N-1$th hidden layer and Nth hidden layer. And, the $H_{k_{N-1}}^{N-1}(t_i)$ is the $k_{N-1}$th units of the $N-1$th hidden layer in time step $t_i$, and $H_{k_N}^N(t_i)$ is the $k_N$th units of the Nth hidden layer in time step $t_i$. Similarly, replace N with 2 to get the mean of $H_{k_2}^2(t_i)$, $h_1\_u$, $W_{k_1·k_2}^2$, $b^2$, and $H_{k_1}^1(t_i)$.

Thus, we can calculate the output of these MNNs:

$$\begin{array}{c}\hfill Outpu{t}_{\beta }(t_i)={(1+e^{-{\sum }_{k_N=1}^{N\_u}{W}_{k_N}^O·H_{k_N}^N(t_i)+b^O})}^{-1}\end{array}$$ 10

where $Outpu{t}_{\beta }(t_i)$ in the ith time step $t_i$ as the output of N-layer NN, $h_N\_u$ is the number of hidden units in the Nth hidden layer, $W^O$ represents the parameters between the Nth hidden layer and the output layer and the $W_{k_N}^O$ stands for the $k_N$th of the weight matrix $W^O$, $H^N(t_i)$ is the output of the Nth hidden layer in time step $t_i$ and the $H_{k_N}^N(t_i)$ is the output of the $k_N$th of the hidden layer output, and the $b^O$ stands for the bias of the output layer. Moreover, we can present a simper form of $Outpu{t}_{\beta }(t_i)$ with weight of N hidden layer is $W=\{W^1,W^2,...,W^N\}$, and $W^O$ is the weight of output layer as:

$$\begin{array}{c}\hfill Outpu{t}_{\beta }(t_i)=a_O(W^O(a(W^N...a(W^1·Input(t_i)+b^1)...)+b^N)+b^O)\end{array}$$ 11

where the activation function of output layer is $a_O(·)$, and the activation function of hidden layers is $a(·)$. The bias $\{b^1,...,b^N\}$ is of the 1st to Nth of hidden layers.

The output of the MNNs represents the effect function $Ef{f}_t$. To solve the ODE equations $\mathbf{Z}(t)$, we need to use an ODE solver. Efficient and accurate ODE solvers have been used for 120 years and can guarantee the growth of approximation error, monitor the level of error, and adapt their evaluation strategy on the fly to achieve the requested level of accuracy [17]. In specific time $t_i$, the function ${\mathbf{Z}}_i$ can be obtained into the form:

$$\begin{array}{c}\hfill {\mathbf{Z}}_i=ODESlover({\mathbf{Z}}_{i-1},\mathcal{F}({\mathbf{Z}}_{i-1},t_{i-1},\theta ),t_{i-1},\theta ),\text{with}\mathbf{Z}(t_0)={\mathbf{Z}}_0\end{array}$$ 12

According to Eq. 12, each ${\mathbf{Z}}_i$ can be calculated. Different models have various parameter sets $\theta$. The specific content of $\theta$ has been given in Eq. 7.

Further, we design the loss function. The loss function L with MNNs attempts to minimize the value of the loss function by adjusting parameters for SMCRD and SEMCRD models. The simper loss function is:

$$\begin{array}{cc}\hfill L(X,\widehat{X})=& \frac{1}{3}\sum _{t=0}^T\underset{\theta }{min}\{{(log(X_t)-log(\widehat{X_t}))}^2\}\hfill \\ \hfill =& \frac{1}{3}\sum _{t=0}^T\underset{\theta ,W}{min}\{||log(I_t)-log({\widehat{M}}_t+{\widehat{C}}_t){||}^2\hfill \\ \hfill & +||log(R_t)-log({\widehat{R}}_t){)||}^2+||log(D_t)-log({\widehat{D}}_t){||}^2\}\hfill \end{array}$$ 13

where $X=\{X_0,X_1,...,X_T\}$ represent real data considered in the loss function in each time step and $\widehat{X}=\{{\widehat{X}}_0,{\widehat{X}}_1,...,{\widehat{X}}_T\}$ stands for predicted data per day. In the ith of time $t_i$, the real data used in calculate loss is $X_i=[I_i,R_i,D_i]$, and the predicted data is ${\widehat{X}}_i=[{\widehat{M}}_i+{\widehat{C}}_i,{\widehat{R}}_i,{\widehat{D}}_i]$ ($I_i$, $R_i$, and $D_i$ represent the number of real infection, recovery, death group, relatively, and ${\widehat{M}}_i$, ${\widehat{C}}_i$, ${\widehat{R}}_i$, and ${\widehat{D}}_i$ stand for the number of predicted mild patients, critical patients, recovery, death group.).

To minimize L, understanding how the gradient of the loss depends on equation set $\mathbf{Z}(t)$. According to the chain rules, we can know that:

$$\begin{array}{c}\hfill \frac{d\mathbf{a}(t)}{\mathit{dt}}=-\mathbf{a}{(t)}^T\frac{\partial f(\mathbf{Z}(t),t,\theta )}{\partial \mathbf{Z}}\text{where,}\mathbf{a}(t)=\frac{\partial L}{\partial \mathbf{Z}(t)}\end{array}$$ 14

The $\frac{\partial L}{\partial \mathbf{Z}(t_0)}$ can be computed by the ODESolver mention above. The ODESolver must start from the initial value of $\frac{\partial L}{\partial \mathbf{Z}(t_T)}$ and run backward. A complicated situation is that solving this ODE requires knowing the value of $\mathbf{Z}(t)$ along its entire trajectory. However, we can simply recalculate $\mathbf{Z}(t)$ from its final value $\mathbf{Z}(t_T)={\mathbf{Z}}_T$ and work backward with time.

The gradients of loss function L depends on both $\mathbf{Z}(t)$ and $\mathbf{a}(t)$:

$$\begin{array}{c}\hfill \frac{\mathit{dL}}{d\theta }=-{\int }_{t_T}^0\mathbf{a}{(t)}^T\frac{\partial f(\mathbf{Z}(t),t,\theta )}{\partial \theta }dt\end{array}$$ 15

The $\mathbf{a}{(t)}^T\frac{\partial \mathcal{F}}{\partial \mathbf{Z}}$, $\mathbf{a}{(t)}^T\frac{\partial \mathcal{F}}{\partial \theta }$ and $\mathcal{F}$ can be obtained through automatic differentiation of computing vector-Jacobian products. Finally, all gradients including $\frac{\partial L}{\partial \mathbf{Z}(t_0)}$ and $\frac{\partial L}{\partial \theta }$ can be calculated at once by calling the ODE solver. From the steps, we first define the initial state, and then design the dynamics system. After that, we compute vector-Jacobian products to obtain $\mathbf{a}{(t)}^T\frac{\partial \mathcal{F}}{\partial \mathbf{Z}}$, $\mathbf{a}{(t)}^T\frac{\partial \mathcal{F}}{\partial \theta }$ and $\mathcal{F}$. In the end, the ODESolver is used to solve the reverse-time SMCRD and SEMCRD equations and gain the gradients.

Experiments and Discussions

Implemental Details

In our experiment, we gathered two types of data: country-level and regional-level. For countries, we utilized data from three countries for model training and testing: the United States of America (USA), Colombia (CO), and South Africa (ZA). Additionally, our model implementation considered data from two specific regions: Wuhan City in China (CN-WH) and Piedmont in Italy (PD). Data collection involved obtaining information on the total cumulative infection cases, recoveries, and deaths, sourced from [9, 27]. For CN-WH, the data spanned from January 24th, 2020 to April 15th, 2020, while for PD, it ranged from February 24th, 2020 to August 4th, 2021. Regarding country-level data collection, we gathered data from January 23rd, 2020 to December 14th, 2020 for the USA, and from March 6th, 2020 to August 11th, 2020 for CO, and from March 7th, 2020 to August 12th, 2020 for ZA. All statistical data were sourced from official notifications provided by various countries, the World Health Organization (WHO), the National Health Commission, among others. For model testing, we utilized the data from the last 20 days to assess model performance, while the remaining data were used for model training.

We employed experienced initial parameter values based on [28], as both traditional parameter estimation methods and our DDE algorithm require initialization. Based on experimental results, the median time from onset to clinical recovery for mild cases is approximately two weeks, leading us to select 0.07 as the initial value for parameter ${\gamma }_{\mathit{MR}}$. Considering the recovery time for patients with severe or critical diseases ranging from 3 to 6 weeks, we set the initial ${\gamma }_{\mathit{CR}}$ to 0.03. According to a previous study [28], the period from onset to the development of severe diseases is one week, leading us to choose 0.15 as the initial $\alpha$. Additionally, the time from symptom onset to outcome among deceased patients varies from 2 to 8 weeks, with a median of six weeks. Thus, we set the initial value of ${\gamma }_D$ to 0.03. For the models incorporating MNNs, we chose 0.15 as the initial value for $\sigma$ and $\alpha$, 0.07 for ${\gamma }_{\mathit{MR}}$, 0.03 for ${\gamma }_{\mathit{CR}}$, and 0.03 for ${\gamma }_D$. For fitting $\beta$ using MNNs, we selected 0.5 as the initial bias of MNNs, 0.0 as the mean value of the initial weight of MNNs, and 0.01 as the standard deviation (std) of the initial weight of MNNs. All experiments were conducted on an Intel XeonE5-2630 v4 @ 2.20GHz CPU and NVIDIA RTX 2080Ti GPU running ArchLinux. We implemented all models using PyTorch. The DDE algorithms implemented in all epidemiological equations were trained for 5000 iterations with a selected learning rate (LR) of 1e-3, which decayed to 0.95 times the LR after 400 iterations. All methods followed a standardized approach, utilizing the initial 80$\%$ of the data for training and fitting purposes, allocating the middle 10$\%$ for validation, and assessing the optimal model’s performance on the remaining 10$\%$ of the data.

To evaluate and demonstrate the efficacy of our DDE algorithm, we conducted several experiments. Initially, we implemented the DDE algorithm based on two classical epidemiological equations, SIR and SEIR, along with four variants devised by us, namely SIRD, SEIRD, SMCRD, and SEMCRD. These four models of DDE algorithms are referred to as DDE-SIRD, DDE-SEIRD, DDE-SMCRD, and DDE-SEMCRD, respectively. In Part A of the experiment, we investigated the disparities between our DDE algorithm and traditional parameter estimation methods, including neural ODE, Nelder-Mead, Truncated Newton Conjugate-Gradient (TNC), Metropolis-Hastings Markov chain Monte Carlo-based method (MCMC-MH) [9], Adaptive-Metropolis Markov chain Monte Carlo-based method (MCMC-AM), and Delay Rejecting Adaptive Metropolis Markov chain Monte Carlo-based method (MCMC-DRAM). Subsequently, in Part B of the experiment, we compared the performance of the four models with state-of-the-art machine learning techniques such as Decision Tree (DT), Extremely randomized trees (ET), Random Forest, and deep learning methods including RNN, LSTM, and GRU. Finally, in Part C, we presented the estimated parameters obtained by the DDE method, providing further insight into its capabilities.

Part A: Comparisons of Traditional Parameter Estimation Methods and DDE

Using parameter estimation methods to assess essential parameters, including infection, mortality, and recovery rates, is crucial for SEIR and its variants to accurately model disease dynamics. Therefore, we conducted a comparison between traditional parameter estimation methods and the DDE method proposed in this study. The comparison performance of different estimation methods based on SIR and its variants in the national data is presented in Table 1. In the USA, the mean squared error (MSE) obtained by the best traditional parameter estimation method is 693.137, while the best MSE achieved by the DDE method is 44.209. Similarly, for the recovery rate estimation, the MSE obtained by the best traditional method is 19.575, whereas the DDE model yields a significantly lower MSE of 0.173. On ZA, the MSE obtained by the best traditional parameter estimation method is 25.209, whereas the DDE method achieves a substantially lower MSE of 0.370. These results highlight the superior performance of the DDE method in accurately estimating essential parameters compared to traditional parameter estimation methods.

Table 1.

MSE of SIR model and its variants in different optimization methods, including Neural ODE (Adam) [17], Nelder-Mead [29], BFGS [30], TNC [31]), MCMC-MH [9], MCMC-AM, MCMC-DRAM, and DDE. The unit of MSE is ten thousand people

Models	Parameter estimation method	Countries			Regions
		CO	USA	ZA	CN-WH	Italy-PD
SIR [18]	Neural ODE	19.317*	16550.121*	183.378*	2.407*	4.697*
	Nelder-Mead	40.253*	2979.545*	166.426*	5.875*	11.609*
	BFGS	28.419*	2094.101*	196.961*	1.125*	8.182*
	TNC	27.632*	2703.972*	54.257*	1.099*	5.102*
	MCMC-MH	68.071*	1971.760*	211.275*	7.214*	1.740*
	MCMC-AM	67.836*	2070.959*	216.647*	8.262*	1.809*
	MCMC-DRAM	72.410*	1951.311*	218.704*	6.882*	1.991*
	DDE	$\mathbf{0.210}$	$\mathbf{52.799}$	$\mathbf{1.143}$	$\mathbf{0.140}$	$\mathbf{0.068}$
SIRD	Neural ODE	29.148*	10190.655*	137.386*	2.910*	4.063*
	Nelder-Mead	59.437*	2722.980*	177.995*	5.303*	8.003*
	BFGS	57.518*	2693.094*	176.266*	1.067*	8.582*
	TNC	29.590*	2218.704*	59.395*	1.128*	3.179*
	MCMC-MH	42.029*	1265.510*	140.070*	4.657*	10.523*
	MCMC-AM	46.686*	1178.305*	137.213*	6.350*	12.384*
	MCMC-DRAM	42.202*	1228.337*	138.031*	4.331*	0.949*
	DDE	$\mathbf{0.173}$	$\mathbf{53.628}$	$\mathbf{1.047}$	$\mathbf{0.096}$	$\mathbf{0.034}$
SMCRD	Neural ODE	30.634*	13553.997*	137.763*	2.899*	4.043*
	Nelder-Mead	28.687*	7476.677*	85.259*	2.884*	0.705*
	BFGS	28.687*	7476.649*	85.260*	2.786*	7.909*
	TNC	24.860*	7901.017*	84.018*	3.678*	0.722*
	MCMC-MH	22.140*	746.631*	82.189*	1.883*	5.119*
	MCMC-AM	19.575*	693.137*	67.917*	4.973*	4.038*
	MCMC-DRAM	21.372*	944.910*	83.602*	5.933*	3.299*
	DDE	$\mathbf{0.526}$	$\mathbf{44.209}$	$\mathbf{2.074}$	$\mathbf{0.027}$	$\mathbf{0.028}$

*Represents metrics with statistically significant differences compared to the proposed DDE method (p < 0.05)

The bold values in the table represent the best results on the corresponding dataset for that column

Furthermore, Table 2 presents the comparison results of different estimation methods based on SEIR and its variants in the national data. In the USA, MSE obtained by the best traditional parameter estimation method is 547.042, while the best MSE achieved by the DDE method is 41.598. Similarly, for the recovery rate estimation, the MSE obtained by the best traditional method is 5.303, whereas the DDE model yields a significantly lower MSE of 0.182. On ZA, the MSE obtained by the best traditional parameter estimation method is 37.618, whereas the DDE method achieves a substantially lower MSE of 0.370. These results underscore the superior performance of the DDE method in accurately estimating essential parameters compared to traditional parameter estimation methods.

Table 2.

MSE of SEIR model and its variants in different optimization methods, including Neural ODE (Adam) [17], Nelder-Mead [29], BFGS [30], TNC [31]), MCMC-MH [9], MCMC-AM, MCMC-DRAM, and DDE. The unit of MSE is ten thousand people

Models	Parameter estimation method	Countries			Regions
		CO	USA	ZA	CN-WH	Italy-PD
SEIR [5]	Neural ODE	23.231*	10523.219*	38.832*	0.967	6.812*
	Nelder-Mead	22.111*	2172.293*	195.785*	5.677*	8.903*
	BFGS	33.297*	2767.012*	78.357*	8.627*	24.941*
	TNC	30.671*	2882.301*	36.361*	6.777*	3.573*
	MCMC-MH	27.029*	1033.137*	69.987*	1.433*	7.662*
	MCMC-AM	22.506*	828.373*	66.287*	1.605*	3.799*
	MCMC-DRAM	22.681*	958.117*	74.007*	0.984*	7.521*
	DDE	$\mathbf{0.192}$	$\mathbf{56.901}$	$\mathbf{0.967}$	$\mathbf{0.250}$	$\mathbf{0.034}$
SEIRD	Neural ODE	9.664*	10858.052*	39.060*	0.366	1.271*
	Nelder-Mead	57.171*	2727.172*	184.118*	5.298*	8.083*
	BFGS	63.887*	2657.202*	70.900*	8.750*	54.941*
	TNC	29.307*	2611.273*	32.233*	6.500*	3.598*
	MCMC-MH	16.443*	550.370*	48.904*	7.340*	4.847*
	MCMC-AM	22.648*	547.749*	60.919*	5.717*	4.388*
	MCMC-DRAM	14.273*	547.042*	43.801*	3.268*	3.889*
	DDE	$\mathbf{0.182}$	$\mathbf{52.377}$	$\mathbf{0.370}$	$\mathbf{0.138}$	$\mathbf{0.028}$
SEMCRD	Neural ODE	5.475*	5521.649*	69.415*	0.727*	0.720*
	Nelder-Mead	16.453*	5899.235*	85.259*	2.464*	0.405*
	BFGS	30.886*	7476.546*	99.747*	17.681*	1.311*
	TNC	27.640	2094.233*	115.132*	3.072*	3.403*
	MCMC-MH	6.586*	3447.659*	192.867*	2.075*	2.289*
	MCMC-AM	13.488*	1796.291*	163.301*	1.884*	2.197*
	MCMC-DRAM	5.303*	1627.123*	37.618*	1.928*	2.345*
	DDE	$\mathbf{0.459}$	$\mathbf{41.598}$	$\mathbf{0.386}$	$\mathbf{0.033}$	$\mathbf{0.014}$

*Represents metrics with statistically significant differences compared to the proposed DDE method (p < 0.05)

The bold values in the table represent the best results on the corresponding dataset for that column

Table 1 presents the comparison results of different estimation methods based on SEIR and its variants on the regional data. The best method among the parameter estimation methods obtains an MSE of 0.705 on PD, while the best MSE of DDE models is 0.028. Similarly, on CN-WH, the best method among the parameter estimation methods achieves an MSE of 1.067, whereas the best MSE of the DDE method is 0.027. Additionally, Table 2 illustrates the comparison performance of different estimation methods based on SEIR and its variants in the regional data. The best method among the parameter estimation methods obtains an MSE of 0.405 on PD, while the best MSE of DDE models is 0.014. Similarly, on CN-WH, the best method among the parameter estimation methods achieves an MSE of 0.366, whereas the best MSE of the DDE method is 0.033. These results demonstrate the superior performance of the DDE method in accurately estimating essential parameters compared to traditional parameter estimation methods, particularly on regional data.

Table 3 presents the Pearson coefficient to evaluate the performance of all methods based on SIR and its variants. It is interesting to note the difference in performance between the US data and other regional data. In the US data, while the DDE method still achieves a high Pearson correlation of 97.23$\%$, it is intriguing that the traditional parameter estimation method performs slightly better with a correlation of 98.45$\%$. The lack of significant difference (P < 0.05) between the two methods suggests that, in the context of US data, both approaches may be equally effective for parameter estimation. Similar trends are observed in CO, ZA, CN-WH, and PD, where the DDE method consistently achieves higher Pearson coefficients compared to traditional methods.

Table 3.

Pearson coefficients of SIR model and its variants in different optimization methods, including Neural ODE (Adam) [17], Nelder-Mead [29], BFGS [30], TNC [31], MCMC-MH [9], MCMC-AM, MCMC-DRAM, and DDE. N represents the prediction with no correlation to real data

Models	Parameter estimation method	Countries			Regions
		CO	USA	ZA	CN-WH	Italy-PD
SIR [18]	Neural ODE	91.95$\%$*	72.04$\%$*	82.92$\%$*	34.71$\%$*	49.77$\%$*
	Nelder-Mead	84.53$\%$*	86.98$\%$*	86.98$\%$*	29.60$\%$*	46.82$\%$*
	BFGS	88.23$\%$*	80.29$\%$*	85.82$\%$*	56.80$\%$*	47.27$\%$*
	TNC	80.80$\%$*	86.92$\%$*	91.69$\%$*	47.28$\%$*	39.98$\%$*
	MCMC-MH	92.72$\%$*	94.42$\%$*	11.29$\%$*	$-$00.09$\%$*	$-$00.41$\%$*
	MCMC-AM	92.75$\%$*	94.89$\%$*	11.29$\%$*	$-$00.04$\%$*	$-$00.57$\%$*
	MCMC-DRAM	92.74$\%$*	94.52$\%$*	11.27$\%$*	$-$00.19$\%$*	$-$00.47$\%$*
	DDE	$\mathbf{99.17}\%$	$\mathbf{97.14}\%$	$\mathbf{97.93}\%$	$\mathbf{98.33}\%$	$\mathbf{95.18}\%$
SIRD	Neural ODE	90.82$\%$*	70.90$\%$*	85.09$\%$*	31.79$\%$*	42.98$\%$*
	Nelder-Mead	89.95$\%$*	82.56$\%$*	86.01$\%$*	29.72$\%$*	40.54$\%$*
	BFGS	90.02$\%$*	82.00$\%$*	85.89$\%$*	23.86$\%$*	40.10$\%$*
	TNC	92.58$\%$*	85.08$\%$*	90.61$\%$*	44.17$\%$*	46.93$\%$*
	MCMC-MH	92.59$\%$*	94.61$\%$*	39.18$\%$*	28.42$\%$*	29.43$\%$*
	MCMC-AM	92.61$\%$*	94.77$\%$*	39.21$\%$*	28.25$\%$*	29.37$\%$*
	MCMC-DRAM	92.65$\%$*	94.69$\%$*	39.19$\%$*	28.38$\%$*	29.38$\%$*
	DDE	$\mathbf{99.47}\%$	$\mathbf{97.23}\%$	$\mathbf{98.88}\%$	$\mathbf{97.33}\%$	$\mathbf{94.22}\%$
SMCRD	Neural ODE	90.77$\%$*	70.89$\%$*	85.06$\%$*	31.81$\%$*	42.94$\%$*
	Nelder-Mead	1.73$\%$*	1.86$\%$*	1.49$\%$*	8.90$\%$*	2.34$\%$*
	BFGS	$-$2.78$\%$*	6.16$\%$*	$-$5.87$\%$*	$-$6.09$\%$*	40.57$\%$*
	TNC	$-$1.62$\%$*	17.34$\%$*	14.24$\%$*	8.36$\%$*	5.98$\%$*
	MCMC-MH	93.57$\%$*	94.78$\%$*	40.22$\%$*	29.10$\%$*	29.78$\%$*
	MCMC-AM	93.81$\%$*	96.37$\%$	40.79$\%$*	29.92$\%$*	29.64$\%$*
	MCMC-DRAM	93.94$\%$*	$\mathbf{98.45}\%$	40.04$\%$*	29.11$\%$*	29.70$\%$*
	DDE	$\mathbf{98.95}\%$	$97.15\%$	$\mathbf{98.71}\%$	$\mathbf{97.42}\%$	$\mathbf{95.38}\%$

*Represents metrics with statistically significant differences compared to the proposed DDE method (p < 0.05)

The bold values in the table represent the best results on the corresponding dataset for that column

Similarly, Table 4 shows the Pearson coefficient to evaluate the performance of all methods based on SEIR and its variants. The DDE method outperforms traditional parameter estimation methods across all regions. For example, in the USA, the best DDE model achieves a Pearson coefficient of 98.75$\%$, while the best traditional parameter estimation method achieves 98.08$\%$. Similar trends are observed in CO, ZA, CN-WH, and PD, where the DDE method consistently achieves higher Pearson coefficients compared to traditional methods. Therefore, we can conclude that the DDE model outperforms all traditional parameter estimation methods in correlation and precision.

Table 4.

Pearson coefficients of SEIR model and its variants in different optimization methods, including Neural ODE (Adam) [17], Nelder-Mead [29], BFGS [30], TNC [31]), MCMC-MH [9], MCMC-AM, MCMC-DRAM, and DDE. N represents the prediction with no correlation to real data

Models	Parameter estimation method	Countries			Regions
		CO	USA	ZA	CN-WH	Italy-PD
SEIR [5]	Neural ODE	90.64$\%$*	78.31$\%$*	90.64$\%$*	90.77$\%$*	59.33$\%$*
	Nelder-Mead	90.18$\%$*	82.98$\%$*	88.69$\%$*	22.02$\%$*	32.67$\%$*
	BFGS	63.17$\%$*	84.23$\%$*	26.39$\%$*	55.88$\%$*	47.55$\%$*
	TNC	57.44$\%$*	80.28$\%$*	91.25$\%$*	69.52$\%$*	35.62$\%$*
	MCMC-MH	94.30$\%$*	95.40$\%$*	13.39$\%$*	$-$00.06$\%$*	$-$00.49$\%$*
	MCMC-AM	94.50$\%$*	95.82$\%$	13.53$\%$*	$-$00.14$\%$*	$-$00.35$\%$*
	MCMC-DRAM	94.49$\%$*	95.32$\%$*	13.34$\%$*	$-$00.01$\%$*	$-$00.58$\%$*
	DDE	$\mathbf{99.77}\%$	$\mathbf{98.75}\%$	$\mathbf{99.01}\%$	$\mathbf{98.03}\%$	$\mathbf{94.47}\%$
SEIRD	Neural ODE	95.97$\%$*	76.20$\%$*	91.66$\%$*	93.27$\%$*	54.99$\%$*
	Nelder-Mead	90.14$\%$*	82.56$\%$*	85.80$\%$*	29.72$\%$*	40.45$\%$*
	BFGS	53.76$\%$*	82.03$\%$*	29.45$\%$*	59.88$\%$*	47.91$\%$*
	TNC	92.57$\%$*	82.26$\%$*	94.00$\%$*	68.70$\%$*	45.98$\%$*
	MCMC-MH	94.18$\%$*	95.88$\%$	41.01$\%$*	47.29$\%$*	29.97$\%$*
	MCMC-AM	93.88$\%$*	96.16$\%$	40.67$\%$*	29.09$\%$*	30.06$\%$*
	MCMC-DRAM	94.37$\%$*	95.98$\%$	41.21$\%$*	29.45$\%$*	30.30$\%$*
	DDE	$\mathbf{99.47}\%$	$\mathbf{97.35}\%$	$\mathbf{99.61}\%$	$\mathbf{97.32}\%$	$\mathbf{95.47}\%$
SEMCRD	Neural ODE	92.13$\%$*	N*	86.98$\%$*	58.02$\%$*	52.35$\%$*
	Nelder-Mead	26.34$\%$*	N*	$-$2.32$\%$*	11.07$\%$*	61.82$\%$*
	BFGS	92.03$\%$*	N*	88.81$\%$*	23.07$\%$*	48.45$\%$*
	TNC	92.80$\%$*	N*	87.80$\%$*	33.07$\%$*	46.15$\%$*
	MCMC-MH	95.51$\%$*	96.82$\%$	42.79$\%$*	30.29$\%$*	30.49$\%$*
	MCMC-AM	94.35$\%$*	97.87$\%$	43.14$\%$*	28.57$\%$*	30.61$\%$*
	MCMC-DRAM	95.82$\%$*	$\mathbf{98.08}\%$	41.34$\%$*	30.07$\%$*	30.52$\%$*
	DDE	$\mathbf{99.17}\%$	$97.17\%$	$\mathbf{99.48}\%$	$\mathbf{96.94}\%$	$\mathbf{97.59}\%$

*Represents metrics with statistically significant differences compared to the proposed DDE method (p < 0.05)

The bold values in the table represent the best results on the corresponding dataset for that column

The comparison of the DDE method with the MCMC methods on longer-term data for the USA and PD regions is indeed insightful. As shown in Table 5, DDE exhibits better accuracy and stability in both fitting the long-term data and predicting future trends compared to the MCMC methods (MCMC-HM, MCMC-AM, MCMC-DRAM). This suggests that the DDE method may offer advantages in handling longer time series data and capturing underlying trends more effectively. The robust performance of the DDE method across different time spans underscores its reliability and potential applicability in modeling and forecasting epidemiological data over extended periods. This finding further strengthens the case for the utility of the DDE method in real-world applications where accurate long-term predictions are crucial for informing public health interventions and policy decisions. In the comparison of state-of-the-art parameter estimation methods, the DDE method attained the best MSE and Pearson coefficient. Contrasted with other parameter estimation methods, MCMC methods achieve more stable fitting performance. Additionally, the parameter estimation method based on the Stochastic Epidemic Model with Covariates and Random Dispersal (SEMCRD) model obtained the most favorable average fitting effect among the six models, irrespective of whether utilizing the DDE or MCMC method.

Table 5.

Fitting performance comparison results (mean ± deviation) of SEMCRD models in different optimization methods using long time data (USA and PD), including MCMC-MH [9], MCMC-AM, MCMC-DRAM, and our DDE method

Areas	Data	Parameter estimation methods
		DDE		MCMC-HM		MCMC-AM		MCMC-DRAM
		MSE	Pearson	MSE	Pearson	MSE	Pearson	MSE	Pearson
TESTING
USA	I	1020.793	$\mathbf{99.93}\%$	222.493*	96.18$\%$*	407.405*	99.89$\%$	$\mathbf{75.465}\ast$	99.79$\%$
		±329.906	$\pm \mathbf{0.00}\%$	±186.014	±7.24$\%$	±375.282	±0.04$\%$	$\pm \mathbf{40.699}$	±0.15$\%$
USA	R	$\mathbf{1141.487}$	$\mathbf{99.78}\%$	1821.911	99.75$\%$	4733.41*	99.76$\%$	1413.249	99.74$\%$
		$\pm \mathbf{251.271}$	$\pm \mathbf{0.00}\%$	±1914.596	±0.04$\%$	±3196.488	±0.03$\%$	±2540.935	±0.03$\%$
USA	D	$\mathbf{1153.170}$	$\mathbf{99.71}\%$	2095.054*	95.28$\%$*	7688.638*	99.69$\%$	2845.97*	98.73$\%$
		$\pm \mathbf{245.170}$	$\pm \mathbf{0.00}\%$	±2018.805	±7.75$\%$	±4409.223	±0.12$\%$	±3499.636	±2.13$\%$
Italy-PD	I	$\mathbf{0.107}$	$\mathbf{98.18}\%$	5.27*	89.11$\%$*	650.641*	15.25$\%$*	870.435*	75.18$\%$*
		$\pm \mathbf{0.124}$	$\pm \mathbf{0.36}\%$	±4.535	±14.22$\%$	±1222.895	±104.5$\%$	±1884.292	±49.77$\%$
Italy-PD	R	$\mathbf{0.556}$	$\mathbf{97.50}\%$	7.098*	94.35$\%$*	1211.114*	97.01$\%$	891.288*	91.06$\%$*
		$\pm \mathbf{0.589}$	$\pm \mathbf{0.05}\%$	±6.155	±4.06$\%$	±1666.59	±0.77$\%$	±1924.663	±10.03$\%$
Italy-PD	D	$\mathbf{0.748}$	52.24$\%$	9.906*	84.35$\%$*	1347.102*	$\mathbf{92.17}\%\ast$	897.757*	91.81$\%$*
		$\pm \mathbf{0.528}$	±0.05$\%$	±8.777	±8.01$\%$	±1861.873	$\pm \mathbf{0.24}\%$	±1927.825	±1.01$\%$
FITTING
USA	I	$\mathbf{6.639}$	$\mathbf{99.68}\%$	20.904*	98.38$\%$	86.311*	94.77$\%$*	27.205*	97.78$\%$
		$\pm \mathbf{1.662}$	$\pm \mathbf{0.03}\%$	±21.293	±0.91$\%$	±41.702	±0.21$\%$	±24.062	±1.59$\%$
USA	R	$\mathbf{8.542}$	$\mathbf{99.44}\%$	27.151*	97.73$\%$	101.492*	93.34$\%$*	32.793*	97.58$\%$
		$\pm \mathbf{2.326}$	$\pm \mathbf{0.03}\%$	±26.469	±2.19$\%$	±45.181	±4.11$\%$	±28.904	±1.58$\%$
USA	D	$\mathbf{8.651}$	$\mathbf{89.41}\%$	28.757*	89.32$\%$	112.675*	81.31$\%$*	39.52*	89.27$\%$
		$\pm \mathbf{2.337}$	$\pm \mathbf{0.29}\%$	±26.633	±3.95$\%$	±49.617	±5.81$\%$	±35.19	±7.54$\%$
Italy-PD	I	$\mathbf{0.009}$	$\mathbf{99.07}\%$	0.102*	92.33$\%$*	4.855*	71.39$\%$*	0.689*	32.31$\%$*
		$\pm \mathbf{0.002}$	$\pm \mathbf{0.16}\%$	±0.058	±6.64$\%$	±6.447	±2.11$\%$	±0.812	±78.16$\%$
Italy-PD	R	$\mathbf{0.016}$	$\mathbf{99.29}\%$	0.146*	96.53$\%$*	6.47*	80.88$\%$*	0.906*	82.77$\%$*
		$\pm \mathbf{0.002}$	$\pm \mathbf{0.04}\%$	±0.081	±2.54$\%$	±8.382	±20.27$\%$	±1.022	±25.03$\%$
Italy-PD	D	$\mathbf{0.017}$	$\mathbf{93.023}\%$	0.165*	89.85$\%$*	6.628*	81.66$\%$*	0.951*	69.98$\%$*
		$\pm \mathbf{0.001}$	$\pm \mathbf{0.214}\%$	±0.087	±6.56$\%$	±8.528	±12.21$\%$	±1.023	±35.05$\%$

*Represents metrics with statistically significant differences compared to the proposed DDE method (p < 0.05)

The bold values in the table represent the best results on the corresponding dataset for that column

Part B: Comparisons of DDE with Learning-Based Methods

To further assess the fitting capability of our DDE method, we conducted a comparative analysis with four learning-based models: DDE-SIRD, DDE-SEIRD, DDE-SMCRD, and DDE-SEMCRD, alongside state-of-the-art learning models. Figure 6 presents the overall forecast results using national data from countries such as CO, ZA, and USA. The observed trends suggest that while LSTM, representative of deep learning methods, may achieve superior results when the 20-day test data changes slowly, its stability is compromised. In contrast, the DDE models exhibit a superior ability to capture the evolving patterns of the data, with some demonstrating closer adherence to real-world developments, as evidenced by the infection curve in ZA. Additionally, Fig. 7 illustrates the results obtained from regional data, such as WH, and PD, over a 20-day test period. Our analysis indicates that the DDE models consistently outperform other deep learning methods, such as GRU or LSTM, in most scenarios. While these deep learning methods may occasionally achieve similar fitting performance to DDE, there are instances where they produce trends diverging from the actual data, as observed in the death curve of Piedmont.

Fig. 6.

Comparative performance of state-of-the-art methods in predicting the progression of infectious diseases at the country level (machine learning methods: RF, ET, DT, deep learning methods: LSTM, RNN, GRU, our models: DDE-SIRD, DDE-SEIRD, DDE-SMCRD, DDE-SEMCRD). Each column of the figure contains six subfigures, arranged in pairs representing different aspects of model predictions. The top subfigure in each pair depicts the overall trend fitting, while the bottom subfigure illustrates the model’s predictions for the subsequent 20 days of testing data. Within each pair of subfigures, the top group represents the model’s prediction of I, the middle group represents predictions of R, and the bottom group represents predictions of D. Furthermore, in the subfigures depicting predictions for the 20-day testing period, the name of the best-performing model corresponding to the predicted performance is provided. From the overall predictions, our DDE model has better prediction performance in the prediction of three countries

Fig. 7.

Comparative performance of state-of-the-art methods in predicting the progression of infectious diseases at the region level. Each column of the figure contains six subfigures, arranged in pairs representing different aspects of model predictions. The top subfigure in each pair depicts the overall trend fitting, while the bottom subfigure illustrates the model’s predictions for the subsequent 20 days of testing data. Within each pair of subfigures, the top group represents the model’s prediction of I, the middle group represents predictions of R, and the bottom group represents predictions of D. Furthermore, in the subfigures depicting predictions for the 20-day testing period, the name of the best-performing model corresponding to the predicted performance is provided. From the overall predictions, our DDE model has better prediction performance in the prediction of two regions

Additionally, the 20-day test results pertaining to national data from CO, ZA, and the USA are illustrated in Fig. 6. Upon analysis of the USA data, it is evident that our DDE-SEIR models exhibit superior fitting capability for the I and R trends. However, there is a slight tendency for the growth of D to be overestimated. Conversely, deep learning methods demonstrate minimal increases in I, R, and D, with limited effectiveness in trend estimation. Furthermore, concerning the CO data, both the DDE-SIRD and DDE-SEIRD models effectively capture the basic trend of I, albeit with some minor fluctuations left unanticipated. While the fitting of R is more accurate, there is a tendency towards slight overestimation of D. In contrast, the deep learning approach still struggles to capture discernible trends in CO data. In the case of ZA, the decreasing trend in I poses a challenge for accurate estimation. Both the DDE-SIRD and DDE-SMCRD models exhibit erroneous estimations of trends, while the DDE-SEIRD and DDE-SECRD models with neural network components accurately predict the downward trend in I. Moreover, all four of our models provide accurate estimations for the trends in R and D. However, deep learning methods display better prediction accuracy for the downward trend in I, with RNN estimates closely approximating the true values. Despite this, in the fitting of R and D, deep learning methods not only fail to predict the correct direction of the data but also yield results opposite to the true values.

Furthermore, as depicted in Fig. 7, the 20-day test results on regional data from PD and CN-WH are presented. Across both regions, all methods demonstrate the ability to accurately estimate the trends of the predicted data. Additionally, LSTM and RNN exhibit superior performance, particularly in CN-WH and PD. Overall, it is observed that all models achieve a better fitting effect on regional data. In contrast, due to the inherent complexity, the deep learning method does not perform as effectively as our proposed DDE method on national data.

Table 6 illustrates the Pearson correlation coefficient for our DDE models, namely DDE-SIRD, DDE-SEIRD, DDE-SMCRD, DDE-SEMCRD, along with other learning-based models (deep learning methods: RNN, LSTM, GRU, and traditional machine learning methods: RF, ET, DT) over a 20-day test period. According to the Pearson metric for real-world data presented in Table 6, deep learning methods such as LSTM and RNN models have demonstrated favorable results in regional data fitting. Specifically, in the fitting of data from CN-WH, the highest Pearson correlations for I and R are achieved by GRU and LSTM, respectively. A similar trend is observed in the data fitting for PD, where the highest Pearson correlations for I and R are obtained by RNN and LSTM, respectively. However, these deep learning algorithms still exhibit instability in fitting, often resulting in negative correlations between predicted data and real data.

Table 6.

Pearson coefficient results of RNN, LSTM, GRU, DT, ET, RF, and DDE models based on SIRD, SEIRD, SMCRD, and SEMCRD. N represents the prediction results and the real data are without correlation

Areas	Data	Methods
		GRU	LSTM	RNN	DT	ET	RF	DDE-	DDE-	DDE-	DDE-
								SIRD	SEIRD	SMCRD	SEMCRD
CO	I	92.68$\%$*	92.96$\%$*	94.52$\%$	N*	N*	N*	$\mathbf{95.77}\%$	95.13$\%$	94.97$\%$	94.98$\%$
	R	88.72$\%$*	90.97$\%$*	90.86$\%$*	N*	N*	N*	$\mathbf{99.86}\%$	99.85$\%$	99.78$\%$	99.80$\%$
	D	93.85$\%$*	95.28$\%$*	98.51$\%$*	N*	N*	N*	98.83$\%$	98.77$\%$	98.52$\%$	$\mathbf{98.53}\%$
USA	I	N*	$-$64.96$\%$*	N*	N*	N*	N*	$\mathbf{97.81}\%$	97.55$\%$	97.54$\%$	97.49$\%$
	R	N*	62.31$\%$*	N*	N*	$-$87.91$\%$*	N*	$\mathbf{99.76}\%$	99.73$\%$	99.73$\%$	99.71$\%$
	D	N*	64.81$\%$*	N*	N*	N*	N*	$\mathbf{99.71}\%$	99.69$\%$	99.69$\%$	99.67$\%$
ZA	I	72.39$\%$*	$-$77.85$\%$*	95.69$\%$	92.24$\%$*	79.95$\%$*	93.10$\%$*	$-$95.06$\%$*	94.83$\%$	$-$91.85$\%$*	$\mathbf{96.39}\%$
	R	$-$43.67$\%$*	88.58$\%$*	$-$92.86$\%$*	$-$92.80$\%$*	$-$82.24$\%$*	$-$90.96$\%$*	99.35$\%$	$\mathbf{99.70}\%$	99.61$\%$	99.69$\%$
	D	98.63$\%$	82.90$\%$*	$-$92.61$\%$*	$-$92.92$\%$*	$-$90.31$\%$*	$-$95.18$\%$*	$\mathbf{99.80}\%$	99.66$\%$	99.79$\%$	99.73$\%$
CN-WH	I	$\mathbf{0.9985}$	99.79$\%$	99.84$\%$	N*	85.52$\%$*	N*	98.23$\%$	99.78$\%$	98.87$\%$	99.46$\%$
	R	99.78$\%$	$\mathbf{99.84}\%$	99.80$\%$	N*	$-$94.92$\%$*	N*	98.23$\%$	94.67$\%$	98.73$\%$	98.08$\%$
	D	97.35$\%$	$-$98.93$\%$*	$-$93.32$\%$*	N*	$-$73.68$\%$*	N*	$\mathbf{98.89}\%$	96.17$\%$	99.17$\%$	98.75$\%$
Italy-PD	I	99.82$\%$	99.76$\%$	$\mathbf{99.82}\%$	N*	98.45$\%$	96.66$\%$*	99.23$\%$	99.34$\%$	98.15$\%$	98.15$\%$
	R	99.73$\%$	$\mathbf{99.84}\%$	99.78$\%$	N*	$-$94.92$\%$*	$-$70.83$\%$*	99.33$\%$	99.41$\%$	98.74$\%$	99.44$\%$
	D	$-$85.10$\%$*	$-$98.71$\%$*	99.20$\%$	N*	$-$92.77$\%$*	$-$75.72$\%$*	$\mathbf{99.83}\%$	99.81$\%$	99.78$\%$	99.77$\%$

*Represents metrics with statistically significant differences compared to the proposed DDE method (p < 0.05)

The bold values in the table represent the best results on the corresponding dataset for that column

Furthermore, traditional machine learning methods such as RF, ET, and DT have shown limited effectiveness in fitting regional data. Instances arise where predictions exhibit no correlation or even negative correlation with real data. In contrast, our DDE models (DDE-SIRD, DDE-SEIRD, DDE-SMCRD, and DDE-SEMCRD) have consistently demonstrated stable and remarkable performance in overall regional predictions of I, R, and D. Utilizing CN-WH data as a benchmark, the four DDE models have achieved average correlation coefficients of 98.45$\%$, 96.87$\%$, 98.92$\%$, and 98.76$\%$ for DDE-SIRD, DDE-SEIRD, DDE-SMCRD, DDE-SEMCRD, respectively. Similarly, on the PD dataset, these models attained average Pearson correlation coefficients of 99.46$\%$, 99.52$\%$, 98.89$\%$, and 99.12$\%$ for DDE-SIRD, DDE-SEIRD, DDE-SMCRD, DDE-SEMCRD, respectively.

In summary, both the traditional ML and DL methods have demonstrated capable predictive performance. As evidenced by the data fitting results across five geographical entities, these methods have surpassed the performance of traditional EE models parameter estimations, representing a significant improvement. However, the black-box nature of these models poses challenges in understanding the underlying analysis rationale, making it difficult for users to discern the basis of their predictions. Consequently, providing actionable guidance to epidemiologists regarding subsequent management strategies for the COVID-19 epidemic becomes challenging. In contrast, the proposed DDE method, which draws inspiration from EE model design, utilizes neural networks solely as a means of parameter fitting. This approach yields parameters that can offer valuable insights to epidemiological experts. Moreover, the DDE method exhibits higher fitting accuracy and stability compared to existing parameter fitting methods. Thus, it represents a promising avenue for enhancing the precision and interpretability of COVID-19 epidemic trend estimation.

Part C: The Estimated Parameters Obtained by DDE Method

To demonstrate the interpretability of the proposed DDE method, we present the parameter estimation results in four EE variants (SIRD, SEIRD, SMCRD, SEMCRD models) fitted by DDE (Fig. 8). In the USA, the average initial transmission rate of infection $\beta \ast$ is calculated to be 0.469, while in CO, it is found to be 0.222. Similarly, the average initial transmission rate of infection $\beta \ast$ is estimated to be 0.254 in ZA. In regional analyses, the average $\beta \ast$ in Wuhan is determined to be 0.4578, while in PD the average $\beta \ast$ obtains 0.367. These infection rates obtained by our DDE method are deemed suitable for real-world scenarios. Figure 8 also illustrates that DDE can simulate various infection rate changes using neural networks to fit the effect function. The DDE models depict the fundamental trend of a gradual decline in the infection rate and provide more accurate estimates of the initial infection rate.

Fig. 8.

Schematic diagram of the infection rate beta generated by each model and different countries and regions. Models include DDE-SIRD, DDE-SEIRD, DDE-SMCRD and DDE-SEMCRD. The dark red line in the figure represents the initial infection rate, and the red dotted curve line represents the change in the infection rate under the influence of the diversity of each country. The curves are simulated by MNNs

Conclusion

An indispensable tool for modeling and estimating EE is required for fitting and analyzing epidemic disease data. In response, we propose a DDE method that integrates NN into EE modeling, achieving high performance across multi-level geographic regions. Experimental findings demonstrate that our DDE model exhibits robust and superior capabilities in various regions, surpassing traditional parameter estimation methods. While learning-based models such as LSTM and RF demonstrate comparable data fitting performance, they lack interpretability crucial for epidemiologists. In addressing this limitation, our DDE method combines EE with neural networks and leverages the Neural ODE approach to enhance data fitting. Consequently, the output of the NN carries specific epidemiological significance. For instance, in the SEMCRD model utilizing DDE, the network’s output represents the effective function of the infection rate. Our model has demonstrated enhanced accuracy and correlation with real-world data. Moreover, our method introduces novel ideas and tools for future analysis of infectious diseases.

Author Contributions

B. S., H. L., P. Z., and D. D. F. conceived and supervised the project. R. L. and J. L. designed the deep learning algorithm and the computational framework. R. L., J. L., and Y. W. designed the study and contributed to the initial drafting of the manuscript. R. L. collected and organized data and performed statistical analysis. P. Z., H. L., and B. S. provided critical comments and reviewed the manuscript. All authors discussed the results and approved the final version before submission.

Funding

We would like to thank all the investigators and study subjects for this study. The Project of Intelligent Management Software supports this work for Multimodal Medical Big Data for New Generation Information Technology, Ministry of Industry and Information Technology of People’s Republic of China (TC210804V). This work is also supported by the Postdoctoral Fellowship Program of CPSF under Grand No. GZC20231604. The research presented in this paper was also supported by Shanghai Jiao Tong University “Jiao Tong University Star” plan key project of the Medical and Industrial Cross under grant number YG2022ZD007.

Data Availability

The data being used in the current study are provided at https://doi.org/10.5281/zenodo.10572455.

Code Availability

The code being used in the current study for developing the algorithm is provided at https://github.com/Liuruhan/DENN.

Declarations

Competing Interests

The authors declare no competing interests.