Electrocardiogram classification using delay differential equations

Abstract

Time series analysis with nonlinear delay differential equations (DDEs) reveals nonlinear as well as spectral properties of the underlying dynamical system. Here, global DDE models were used to analyze 5 min data segments of electrocardiographic (ECG) recordings in order to capture distinguishing features for different heart conditions such as normal heart beat, congestive heart failure, and atrial fibrillation. The number of terms and delays in the model as well as the order of nonlinearity of the model have to be selected that are the most discriminative. The DDE model form that best separates the three classes of data was chosen by exhaustive search up to third order polynomials. Such an approach can provide deep insight into the nature of the data since linear terms of a DDE correspond to the main time-scales in the signal and the nonlinear terms in the DDE are related to nonlinear couplings between the harmonic signal parts. The DDEs were able to detect atrial fibrillation with an accuracy of 72%, congestive heart failure with an accuracy of 88%, and normal heart beat with an accuracy of 97% from 5 min of ECG, a much shorter time interval than required to achieve comparable performance with other methods.

Cardiovascular diseases are the main cause of deaths worldwide and a major cost in health care. Better diagnosis methods are much needed. Here, delay differential equations (DDEs) are used to discriminate short (5 min) electrocardiography (ECG) data segments. This method does not require any preprocessing of the data, is performed on the time series themselves, is computationally fast, and could be the basis for a real time diagnostic system. DDEs reveal non-linear properties as well as spectral properties of the data. A DDE relates a differential and delay embedding in a complex manner to extract distinctive dynamical properties of the underlying dynamical system. DDE analysis is a time domain tool that combines aspects of nonlinear dynamics, Fourier analysis, and higher-order statistics.

INTRODUCTION

In global vector field reconstruction,^{1, 2, 3, 4} the recorded data are used to generate a model, whose dynamical behavior is equivalent to the original system. Equivalence is not required for our data analysis method. Nonetheless, the identification technique provides a global model that captures some essential features of the underlying dynamics.

The techniques introduced here are based on Delay Differential Equations (DDEs). DDEs are a generalization of ordinary differential equations (ODEs) with time delays. DDEs have to be used to describe the underlying dynamics in particular physical and biological processes. Such processes are typically characterized by a delayed reaction (see Driver⁵ for a list of examples). Delays also have an important role in analyzing ECG data.⁶

Solving even the simplest linear DDE $\dot{x}(t)=\alpha x(t-\tau )$ is complicated (see, e.g., Ref. 7) and not within the scope of this paper. We do not seek DDEs that predict time series but rather global DDE models that capture distinguishing features of data for different heart conditions such as normal heart beat, congestive heart failure, and atrial fibrillation. The question whether an ECG is best modeled by a linear or non-linear process is directly related to the structure selection of the DDE: Is a linear DDE sufficient or are non-linear terms needed? How many terms, how many delays and what kind of non-linearity should be used?

DDEs can be seen as a flavor of a autoregressive (AR) model or an autoregressive moving average (ARMA) model^{8, 9, 10, 11} where the time series on the left side of the equation is replaced by the derivative. Lately delay systems have been used in the context of reservoir computing (RC).^{12, 13, 14, 15} RC is a recently introduced, bio-inspired, machine-learning paradigm for processing empirical data that is mimicking neuronal networks. A DDE in this context is the simplest nonlinear delay system with a singular node.

A motivation for DDE analysis of non-linear data comes from embedding theory in non-linear time series analysis. An embedding converts a single time series into a multidimensional object in an embedding space (Whitney,¹⁶ Packard et al.,¹⁷ Takens,¹⁸ and Sauer et al.²). The reconstructed attractor reveals basic properties (dimension, Lyapunov spectrum, and entropy) of the true attractor of the system. It allows valuable information to be obtained about the dynamics of the system without having direct access to all the systems variables.

There are two basic embeddings: delay and derivative embeddings. For a delay embedding, the time series itself and its delayed versions are used to construct the embedding; for the derivative embedding the time series and its successive derivatives are used. Judd and Mees¹⁹ introduced the idea of non-uniform embeddings for time series with components of multiple time-scales. DDE analysis then relates aspects of the different embeddings: the derivative of the time-series is related to functions of non-uniformly delayed versions of that time series.

DDE data analysis can be also seen as a novel way of combining Fourier analysis and higher-order statistics in a time domain framework. The relationship between frequency analysis and analysis of frequency and/or phase couplings in the time domain is poorly understood (see, e.g., Refs. ^{20, 21, 22, 23, 24}). The linear terms of a DDE correspond to the main frequencies in the signal. For n independent frequencies in a signal a linear DDE with 2 n–1 terms is needed to describe such data. The nonlinear terms in the DDE are related to nonlinear couplings between the harmonics. DDEs can also be expanded in a Yule-Walker-like way^{25, 26} and the DDE coefficients then can be rewritten as functions of dynamical higher-order data correlations. These dynamical higher-order data correlations are generalizations of Nth order data moment functions such as, e.g., the auto-correlation (2nd order moment) and the bi-correlation (3rd order moment). The paper is organized as follows: Sec. 2 shows the connection between DDEs and classical Fourier analysis and higher order statistics (HOS). In Sec. 3, good classifiers for ECG data are found via DDE analysis. Section 4 is the discussion.

DELAY DIFFERENTIAL EQUATIONS AND TIME-DOMAIN FREQUENCY ANALYSIS

DDE analysis is done in the time domain on the time-series themselves and not in the frequency domain. The DDE framework combines linear and non-linear information from the data in a complex and not easily understandable way. To gain some insight in the meaning of the different terms of a DDE, we show the correspondence of the linear terms to the main time-scales or frequencies in the signal and how the non-linear terms contain information about non-linear couplings.

Linear DDEs

The simplest linear DDE is

$$\dot{x}=a{x}_{\tau }\hspace{0.17em},$$ (1)

where $x_{\tau }=x(t-\tau )$. Solving this equation in general is nontrivial (see, e.g., Ref. 7) and beyond the scope of this paper. However, looking at special solutions can lead to understanding the terms in a DDE as used here for detection/classification purposes. A special solution of Eq. 1 is (see Ref. 7)

$$x(t)=\cos (\omega t)\hspace{0.17em};\hspace{1em}a={(-1)}^n\omega \hspace{0.17em};\hspace{1em}\tau =\frac{\pi \hspace{0.17em}(2n-1)}{2\omega },$$ (2)

where $n\in \mathbb{N}$. The coefficient a is proportional to the frequency and the time delay τ is inversely proportional to the frequency. For a signal with a frequency f and $\omega =2\pi f$ the delay is then

$$\tau =\frac{(2n-1)}{4\hspace{0.17em}f}\hspace{0.17em}.$$ (3)

The delay is inversely proportional to the frequency and the coefficient a is directly proportional to the frequency.

A special solution of the linear DDE

$$\dot{x}={\displaystyle \sum _{i=1}^N{a}_i}{x}_{{\tau }_i}\hspace{0.17em}$$ (4)

is

$$x(t)={\displaystyle \sum }_{k=1}^{2 N-1}\cos ({\omega }_{k}t)\hspace{0.17em};\hspace{1em}{\tau }_i=\frac{\pi (2n-1)}{2{\omega }_j},$$ (5)

where $n\in \mathbb{N}$ are arbitrary integers and all delays ${\tau }_i$ are related to one of the frequencies. The expressions for the coefficients a are more complicated than in Eq. 2 and each depends on all the frequencies in the signal.

Equations 4, 5 imply that we need a DDE with 2N–1 linear terms to describe a harmonic signal with N frequencies. If we consider Eq. 1 and a sum of three harmonics, $x(t)={\sum }_{i=1}^3\cos ({\omega }_{i}t)$, Eq. 1 cannot be solved analytically. To estimate the value of the coefficient a we expand Eq. 1 as a Yule-Walker-like equation:^{25, 26} We multiply both sides of Eq. 1 with $x_{\tau }$ and apply the expectation operator $⟨F(t)⟩\equiv \underset{T\to \infty }{\lim }\left(\frac{1}{T}\hspace{0.17em}{\int }_0^{T}F(t)\hspace{0.17em}\hspace{0.17em}\text{dt}\right)$ and get

$$a=\frac{⟨\dot{x}\hspace{0.17em}{x}_{\tau }⟩}{⟨x_{\tau }^2⟩}\hspace{0.17em}.$$ (6)

The numerator in Eq. 6 looks like a “dynamical” version of the autocorrelation function $⟨x\hspace{0.17em}{x}_{\tau }⟩$ and it can be rewritten as delay derivatives of the autocorrelation function in the case of a bounded stationary signal,

$$\begin{array}{c}⟨\dot{x}{x}_{\tau }⟩=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{0}{\overset{T}{\int }}\dot{x}}{x}_{\tau }dt\\ =\underset{T\to \infty }{\lim }\frac{1}{T}{\left[x{x}_{\tau }\right]}_0^T-\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{0}{\overset{T}{\int }}x}\frac{d{x}_{\tau }}{\mathit{d}\mathit{t}}dt\\ =\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{0}{\overset{T}{\int }}x}\frac{d{x}_{\tau }}{d\tau }d\tau \\ =\frac{d}{d\tau }⟨x{x}_{\tau }⟩\hspace{0.17em}.\end{array}$$ (7)

For $x(t)=\sum _{i=1}^3\cos ({\omega }_{i}t)$, the expressions in Eq. 6 are

$$\begin{array}{c}⟨\dot{x}\hspace{0.17em}{x}_{\tau }⟩\stackrel{{\omega }_3\ne {\omega }_j}{=}-\frac{1}{2}\sum _{i=1}^3\sin ({\omega }_i\tau ){\omega }_i,\\ ⟨\dot{x}\hspace{0.17em}{x}_{\tau }⟩\stackrel{{\omega }_3={\omega }_1}{=}-\frac{1}{2}\hspace{0.17em}{\omega }_2\sin \left(\tau {\omega }_2\right)-2{\omega }_1\sin \left(\tau {\omega }_1\right),\\ ⟨\dot{x}\hspace{0.17em}{x}_{\tau }⟩\stackrel{{\omega }_3={\omega }_2}{=}-\frac{1}{2}\hspace{0.17em}{\omega }_1\sin \left(\tau {\omega }_1\right)-2{\omega }_2\sin \left(\tau {\omega }_2\right),\\ ⟨x_{\tau }^2⟩\stackrel{{\omega }_3\ne {\omega }_j}{=}\frac{3}{2},\\ ⟨x_{\tau }^2⟩\stackrel{{\omega }_3={\omega }_j}{=}\frac{5}{2}\hspace{0.17em};\hspace{1em}j=1,2\hspace{0.17em}.\end{array}$$ (8)

The coefficient a in Eq. 6 is a smooth function with singularities at ${\omega }_3={\omega }_i$, i = 1,2. Therefore estimating a numerically can be used as a time domain frequency detection tool: To detect the two frequencies in the signal $\mathcal{D}=\cos ({\omega }_1\tau )+\cos ({\omega }_2\tau )$ ($f_1=31\hspace{0.17em}\text{Hz},f_2=69\hspace{0.17em}\text{Hz},\omega =2\pi f$) the term $\cos ({\omega }_3\tau )$ was added for a range of frequencies $f_3=\frac{{\omega }_3}{2\pi }$. In Fig. 1, we estimated the coefficient a numerically by a singular value decomposition (SVD) algorithm²⁷ and then computed the least square error $\sqrt{\sum {(\dot{x}-a\hspace{0.17em}{x}_{\tau })}^2}$ for Eq. 6 for $\mathcal{D}+\cos ({\omega }_3\tau )$ with $f_3$ varying from 0 to 150 Hz. The delay was 10 $\delta t$, where $\delta t=\frac{1}{f_s}$ with a sampling rate $f_s=1000\hspace{0.17em}\text{Hz}$. The two singularities at the two frequencies $f_1=31\hspace{0.17em}\text{Hz}$ and $f_2=69\hspace{0.17em}\text{Hz}$ are clearly visible in both plots. The choice of a different delay would change the shape of curve (see Eqs. 7, 8). The error $\rho =\left|\dot{x}-\frac{⟨\dot{x}\hspace{0.17em}{x}_{\tau }⟩}{⟨x_{\tau }^2⟩}\hspace{0.17em}{x}_{\tau }\right|$ is nonzero since Eq. 6 is not an exact solution.

Figure 1.

Error ρ and coefficient a for the linear DDE $\dot{x}=a\hspace{0.17em}{x}_{\tau }$ with $\tau =10\hspace{0.17em}\delta t$ vs. frequency f₃ for the signal $x(t)=\cos ({\omega }_{1}t)+\cos ({\omega }_{2}t)+\cos ({\omega }_{3}t)$ with $f_1=31\hspace{0.17em}\text{Hz}$ and $f_2=69\hspace{0.17em}\text{Hz}$ ($\omega =2\pi f$).

The possible advantages for the use this time-domain frequency analysis tool are that it can be applied for short time series and for sparse data: a can be estimated for the set of all points for which the derivative can be computed and the delayed point exists. Missing points can be left out.

The method is also fairly noise insensitive. In Fig. 2, we added to the signal white noise η with a signal-to-noise ratio of SNR = −10 dB which is more noise than signal. We then did the same numerical experiment as we did in Fig. 1: we estimated the coefficient a numerically by a singular value decomposition (SVD) algorithm and then computed the least square error for Eq. 6 for $D+\cos ({\omega }_3\tau )$ with f₃ varying from 0 to 150 Hz. Again, the two singularities at the two frequencies $f_1=31\hspace{0.17em}\text{Hz}$ and $f_2=69\hspace{0.17em}\text{Hz}$ are clearly visible in both plots (Fig. 2).

Figure 2.

Error ρ and coefficient a for the linear DDE $\dot{x}=a\hspace{0.17em}{x}_{\tau }$ with $\tau =10\hspace{0.17em}\delta t$ vs. frequency $f_3$. White noise η was added to the signal: $D=\cos ({\omega }_{1}t)+\cos ({\omega }_{2}t)+\eta$, where the signal-to-noise ration SNR = −10 dB and $f_1=31\hspace{0.17em}\text{Hz}$ and $f_2=69\hspace{0.17em}\text{Hz}$ ($\omega =2\pi f$). The coefficient a and error ρ were then plotted for $x(t)=\mathcal{D}+\cos ({\omega }_{3}t)$ with f₃ varying between 0 and 150 Hz.

Nonlinear DDEs

In real world data, various frequency components do not always appear completely independently of one another. Those non-linear interactions of frequencies and their phases (e.g., quadratic phase coupling) cannot be detected by a power spectrum, the Fourier transform of the autocorrelation function (second-order cumulant), since phase relationships and frequency couplings of signals are lost. Such couplings are usually detected via higher order spectra or bispectral analysis.^{28, 29, 30, 31, 32, 33, 34, 35} The bispectrum or bispectral density is the Fourier transform of the third-order cumulant (bicorrelation function). Consider the signal $x(t)=A_1\cos ({\omega }_{1}t+{\phi }_1)+A_2\cos ({\omega }_{2}t+{\phi }_2)$ which is passed through a quadratic nonlinear system $h(t)=b\hspace{0.17em}{x}^2(t)$ where b is a non-zero constant. On the output of the system, the signal will include the harmonic components: $(2{\omega }_1,2{\phi }_1),(2{\omega }_2,2{\phi }_2),({\omega }_1+{\omega }_2,{\phi }_1+{\phi }_2)$, and $({\omega }_1-{\omega }_2,{\phi }_1-{\phi }_2)$. These phase relations are called quadratic phase coupling (QPC). Since we are here interested in the couplings of frequencies we will consider the special case of quadratic frequency coupling (QFC) when the phases are zero (${\phi }_1={\phi }_2=0$): For a signal $x(t)=\cos ({\omega }_{1}t)+\cos ({\omega }_{2}t)+\cos ({\omega }_{3}t)$ frequency coupling occurs when ${\omega }_3$ is a multiple of one of the frequencies or of the sum or difference of the two frequencies.

A simple DDE with one non-linear term,

$$\dot{x}=a\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2},$$ (9)

cannot be solved analytically, but as shown for the linear case in Sec. 1, it can be expanded as a Yule-Walker-like equation:^{25, 26} We multiply both sides of Eq. 9 with $x_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}$ and apply the expectation operator and get

$$a=\frac{⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩}{⟨x_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}^2⟩}\hspace{0.17em}.$$ (10)

The numerator $⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩$ in Eq. 10 looks like a dynamical version of the bicorrelation³⁶$⟨x\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩$. It can be rewritten as delay derivatives of the moments in the case of a bounded stationary signal,

$$\begin{array}{c}⟨\dot{x}{x}_{{\tau }_1}{x}_{{\tau }_2}⟩=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{0}{\overset{T}{\int }}\dot{x}}{x}_{{\tau }_1}{x}_{{\tau }_2}dt=\underset{T\to \infty }{\lim }\frac{1}{T}{\left[x{x}_{{\tau }_1}{x}_{{\tau }_2}\right]}_0^T-\underset{T\to \infty }{\lim }\frac{1}{T}\underset{0}{\overset{T}{\int }}x\frac{d}{\mathit{d}\mathit{t}}(x_{{\tau }_1}{x}_{{\tau }_2})dt=\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{0}{\overset{T}{\int }}x}\frac{d{x}_{{\tau }_1}}{d{\tau }_1}{x}_{{\tau }_2}dt\\ +\underset{T\to \infty }{\lim }\frac{1}{T}{\displaystyle \underset{0}{\overset{T}{\int }}x}{x}_{{\tau }_1}\frac{d{x}_{{\tau }_1}}{d{\tau }_2}dt=\frac{d}{d{\tau }_1}⟨x{x}_{{\tau }_1}{x}_{{\tau }_2}⟩+\frac{d}{d{\tau }_2}⟨x{x}_{{\tau }_1}{x}_{{\tau }_2}⟩\hspace{0.17em}.\end{array}$$ (11)

For $x(t)=\cos ({\omega }_{1}t)+\cos ({\omega }_{2}t)+\cos ({\omega }_{3}t)$, the bicorrelation is only non-zero when ${\omega }_3$ is a multiple of one of the frequencies or of the sum or difference of the two frequencies. In these cases, the expressions for the dynamic bicorrelation are

$$\begin{array}{c}⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩\stackrel{{\omega }_3={\omega }_1\pm {\omega }_2}{=}\frac{1}{4}({\omega }_1(-\sin ({\tau }_2{\omega }_1\pm {\tau }_1{\omega }_2)-\sin ({\tau }_1{\omega }_1\pm {\tau }_2{\omega }_2)+\sin (\pm {\tau }_2{\omega }_2-{\tau }_1({\omega }_1\pm {\omega }_2))\\ +\sin (\pm {\tau }_1{\omega }_2-{\tau }_2({\omega }_1\pm {\omega }_2)))+{\omega }_2(-\sin ({\tau }_2{\omega }_1\pm {\tau }_1{\omega }_2)-\sin ({\tau }_1{\omega }_1\pm {\tau }_2{\omega }_2)\\ +\sin ({\tau }_2{\omega }_1-{\tau }_1({\omega }_1\pm {\omega }_2))+\sin ({\tau }_1{\omega }_1-{\tau }_2({\omega }_1\pm {\omega }_2)))),\\ ⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩\stackrel{{\omega }_3=2{\omega }_i}{=}\frac{1}{4}({\omega }_i\sin (({\tau }_1-2{\tau }_2){\omega }_i)-{\omega }_i\sin ((2{\tau }_1-{\tau }_2){\omega }_i)-2{\omega }_i\sin (({\tau }_1+{\tau }_2){\omega }_i)),\\ ⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩\stackrel{{\omega }_3=\frac{{\omega }_i}{2}}{=}\frac{1}{8}{\omega }_i\left(\sin \right(\frac{1}{2}({\tau }_1-2{\tau }_2){\omega }_i)-\sin (\frac{1}{2}(2{\tau }_1-{\tau }_2){\omega }_i)-2\sin (\frac{1}{2}({\tau }_1+{\tau }_2){\omega }_i));\hspace{0.17em}i=1,2.\end{array}$$ (12)

and for the denominator of Eq. 10

$$\begin{array}{c}{x}_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}^2\stackrel{{\omega }_3\ne {\omega }_i}{=}\frac{1}{8}(\cos (2({\tau }_1-{\tau }_2){\omega }_1)+4\cos (({\tau }_1-{\tau }_2)({\omega }_1-{\omega }_2))+\cos (2({\tau }_1-{\tau }_2){\omega }_2)+4\cos (({\tau }_1-{\tau }_2)({\omega }_1+{\omega }_2))\\ +4\cos (({\tau }_1-{\tau }_2)({\omega }_1-{\omega }_3))+4\cos (({\tau }_1-{\tau }_2)({\omega }_2-{\omega }_3))+\cos (2({\tau }_1-{\tau }_2){\omega }_3)+4\cos (({\tau }_1-{\tau }_2)({\omega }_1+{\omega }_3))\\ +4\cos (({\tau }_1-{\tau }_2)({\omega }_2+{\omega }_3))+18),\\ x_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}^2\stackrel{{\omega }_3={\omega }_1}{=}2\cos (2({\tau }_1-{\tau }_2){\omega }_1)+2\cos (({\tau }_1-{\tau }_2)({\omega }_1-{\omega }_2))+\frac{1}{8}\cos (2({\tau }_1-{\tau }_2){\omega }_2)+2\cos (({\tau }_1-{\tau }_2)({\omega }_1+{\omega }_2))+\frac{25}{4},\\ x_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}^2\stackrel{{\omega }_3={\omega }_2}{=}\frac{1}{8}\cos (2({\tau }_1-{\tau }_2){\omega }_1)+2\cos (({\tau }_1-{\tau }_2)({\omega }_1-{\omega }_2))+2\cos (2({\tau }_1-{\tau }_2){\omega }_2)+2\cos (({\tau }_1-{\tau }_2)({\omega }_1+{\omega }_2))+\frac{25}{4}.\end{array}$$ (13)

The coefficient a in Eq. 10 is only non-zero for QFC frequencies. We therefore can use this equation to detect non-linear couplings in the time domain in the same way as we used the linear DDE Eq. 6 to detect frequencies.

Fig. 3 shows how non-linear terms can detect frequency couplings. For $x(t)=\cos ({\omega }_{1}t)+\cos ({\omega }_{2}t)+\cos ({\omega }_{3}t)$ ($\omega =2\pi f$) where the frequencies $f_1$ and $f_2$ were 31 Hz and 69 Hz, respectively, and $f_3$ was varied from 0 to 150 Hz. For each $f_3$ we estimated the coefficient a numerically by a singular value decomposition (SVD) algorithm²⁷ and then computed the least square error $\sqrt{\sum {(\dot{x}-a\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2})}^2}$ for Eq. 9. The nonlinear coefficient a (top plot in Fig. 3) is only non-zero when there is QFC and is zero otherwise. The least square error (bottom plot in Fig. 3) shows a slope with spikes when there is QFC as well as at the 2 frequencies $f_1$ and $f_2$.

Figure 3.

Error and coefficient a for the nonlinear DDE $\dot{x}=a\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}$ with ${\tau }_1=5\hspace{0.17em}\delta t$ and ${\tau }_2=11\hspace{0.17em}\delta t$ vs. frequency f₃ for the signal $x(t)=\cos ({\omega }_{1}t)+\cos ({\omega }_{2}t)+\cos ({\omega }_{3}t)$ with $f_1=31\hspace{0.17em}\text{Hz}$ and $f_2=69\hspace{0.17em}\text{Hz}$ ($\omega =2\pi f$).

Equation 9 can therefore be used to detect QFC. Since the coefficient a in Eq. 10 is not an exact solution of Eq. 9, the error (lower plot in Fig. 3)) is $\rho =|\dot{x}-a\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}|$. For all values of $f_3$ that are different from any coupling cases (sum or multiples of the two other frequencies) it should be $\rho =\dot{x}$ since a = 0 for all non-coupling values of $f_3$. Numerically a will never be exactly zero, but a small value. Therefore, the error is $\rho =\left|\dot{x}-\frac{⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩}{⟨x_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}^2⟩}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}\right|$. $⟨x_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}^2⟩$ has a spike when $f_3$ is equal to $f_1$ and $f_2$ and $⟨\dot{x}\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}⟩$ has spikes for QFC cases. Therefore the error (lower plot in Fig. 3)) shows bumps for QFC cases and the frequencies. The delays ${\tau }_{1,2}$ only change the shape of the error function.

In this section, we wanted to show the connection of DDEs to spectral analysis: Delays connected to linear terms in the DDE relate to frequencies (see Eqs. 6, 7, 8 and Fig. 1) and delays connected to non-linear terms in the DDE relate to couplings between those frequencies (see Eqs. 10, 12, 13, and Fig. 3). Here, we are not aiming to interpret these plots quantitatively. Different delays would only change the shape of the curves, but not the fact that there are spikes for the frequencies in the linear case and spikes for the frequency couplings in the non-linear case. Therefore, any arbitrary choice of delays will give the same qualitative behavior.

DDE ANALYSIS OF ECG DATA

In Sec. 2, we showed how linear terms of a DDE relate to the time scales or frequencies in the signal and the non-linear terms relate to non-linear couplings of those time scales or frequencies. Any DDE can be expanded in a Yule-Walker-like way and the coefficients will then be combinations of dynamical data correlations.

A DDE can also be interpreted as generic non-uniform embedding. Such a generic non-uniform embedding (the DDE model) unfolds timescales for the linear terms and couplings between frequencies for the non-linear terms as a spectrogram unfolds frequencies and higher order statistics (HOS) unfolds the couplings between frequencies. DDEs and spectral analysis are connected as we showed in the previous section. They are just operating in two different domains, the time domain and the spectral domain.

A DDE is a nonlinear extension of a non-uniform embedding with linear and/or nonlinear functions of the timeseries. Therefore, it can be tailored to the dynamics. HOS would have to be extended to a network of all possible HOS moments combined with linear spectral analysis to do the same. Therefore, a DDE is the simpler approach.

The structure or model form of the DDE and the delays for classification of heart data were in this section selected by an exhaustive search.

Data

We analyzed 24 h data from 15 young healthy persons in normal sinus rhythm (NSR) (ECG sample frequency: 128 Hz) of 15 congestive heart failure (CHF) patients (ECG sample frequency: 250 Hz) as well as of 15 subjects suffering from atrial fibrillation (AF) (ECG sample frequency: 128 Hz) selected from the Physionet database.³⁷ Table TABLE I. lists the files. The first five subjects of each group were used for the CHAOS Controversial Topics in Nonlinear Dynamics “Is the Normal Heart Rate Chaotic?” (http://physionet.org/challenge/chaos). The other ten subjects from each group are randomly selected records from the same databases.

TABLE I.

ECG data used. The three conditions are normal sinus rhythm (NSR), congestive heart failure (CHF), and atrial fibrillation (AF). The data were downloaded from the physionet database.³⁷

Supervised structure selection

Typically, a nonlinear delay differential equation has the form

$$\begin{array}{c}\dot{x}=f(a_i,x_{{\tau }_j})=a_1\hspace{0.17em}{x}_{{\tau }_1}+a_2\hspace{0.17em}{x}_{{\tau }_2}+a_3\hspace{0.17em}{x}_{{\tau }_3}+\cdots +\hspace{0.17em}{a}_{i-1}\hspace{0.17em}{x}_{{\tau }_n}\\ +a_i\hspace{0.17em}{x}_{{\tau }_1}^2+a_{i+1}\hspace{0.17em}{x}_{{\tau }_1}{x}_{{\tau }_2}+a_{i+2}\hspace{0.17em}{x}_{{\tau }_1}{x}_{{\tau }_3}\\ +\cdots +\hspace{0.17em}{a}_{j-1}\hspace{0.17em}{x}_{{\tau }_n}^2+a_j\hspace{0.17em}{x}_{{\tau }_1}^3+\hspace{0.17em}{a}_{j+1}\hspace{0.17em}{x}_{{\tau }_1}2x_{{\tau }_2}+\cdots \\ ⋮\\ \cdots +a_l\hspace{0.17em}{x}_{{\tau }_n}^m,\end{array}$$ (14)

where $x=x(t)$ and $x_{{\tau }_j}=x(t-{\tau }_j)$. The DDE Eq. 14 has n delays, l monomials with coefficients $a_1,a_2,\mathrm{...},a_l$, and a degree m of nonlinearity. By a k-term DDE, we mean a DDE with k monomials selected from the right-hand side of Eq. 14. Although quite flexible, as for any global modeling technique, there is a significant gain in accuracy by carefully selecting the structure of the model.^{38, 39, 40} By structure selection or model learning, we mean retaining only those monomials that make the most significant contribution to the data dynamics. An equally important task is to select the right time-delays, since they are directly related to the primary time-scales and non-linear couplings between them of the dynamics under study.

Lainscsek et al.⁴⁰ used a genetic algorithm to find a single DDE model for the classification of Parkinson movement data. Here, we want to do an exhaustive search of models and delays and find the models and delays that best separate classes of data. To do so we look at all possible polynomial DDE models

$$\begin{array}{c}\dot{x}=a_1\hspace{0.17em}{x}_{{\tau }_1}+a_2\hspace{0.17em}{x}_{{\tau }_2}+a_3\hspace{0.17em}{x}_{{\tau }_1}^2+a_4\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}+a_5\hspace{0.17em}{x}_{{\tau }_2}^2+\\ a_6\hspace{0.17em}{x}_{{\tau }_1}^3+a_7\hspace{0.17em}{x}_{{\tau }_1}^2\hspace{0.17em}{x}_{{\tau }_2}+a_8\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}^2+a_9\hspace{0.17em}{x}_{{\tau }_2}^3,\end{array}$$ (15)

with some of the $a_i$ equal to zero. Only models with up to three terms were considered. If the analysis did not give satisfactory results we added additional delays, increased the order of non-linearity and/or used DDEs with more than three terms. There were 5 one-term models, 18 two-term models, and 32 three-term models.

Tables 2, TABLE III. list all these models. Note that e.g. the DDE models $\dot{x}=a_1\hspace{0.17em}{x}_{{\tau }_1}+a_2\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}$ and $\dot{x}=a_1\hspace{0.17em}{x}_{{\tau }_2}+a_2\hspace{0.17em}{x}_{{\tau }_1}\hspace{0.17em}{x}_{{\tau }_2}$ are the same with exchanged delays ${\tau }_1$ and ${\tau }_2$. Therefore, only the first of these two models were used. All such redundant DDE models were omitted. There were only two linear DDEs (model 1 and 5) while all others are non-linear. Seven of the DDEs had only one delay (models 1, 2, 4, 7, 9, 17, and 30) and nine models were symmetric (models 3, 6, 16, 22, 23, 25, 43, 52, 53) with two interchangeable delays.

TABLE II.

One- and two-term models. An “x” denotes that the term $a_{\mathrm{\star }}$ is nonzero. The different types of models are: “L”—linear, “S”— symmetric, “1”—single delay DDE. All other DDEs are non-linear and have two non-interchangeable delays. To save space in the table $x_{{\tau }_i}$ was written in the short form $x_i$.

TABLE III.

Three term models. An “x” denotes that the term $a_{\mathrm{\star }}$ is nonzero. The different types of models are: “L”—linear, “S” —symmetric, “1”—single delay DDE. All other DDEs are non-linear and have two non-interchangeable delays. To save space in the table $x_{{\tau }_i}$ was written in the short form $x_i$.

Data analysis

The data were analyzed without filtering and no artifacts were removed from the data. The downloaded NSR and AF data were sampled at 128 Hz, but the CHF data were sampled at 250 Hz. To use the same DDE with the same delays for all data, the NSR and AF data were up-sampled using the matlab function resample⁴¹ with the default options. Throughout this paper, we use 5 min non-overlapping data windows for our analysis. Each window was re-normalized to zero mean and unit variance to be able to compare data of different origin.

For the model selection task, we have to choose a classifier, select training data, select a classification tool, and do some cross-validation to take the small number of subjects into account. In this manuscript, we chose seven different classifiers and tested the performance of each separately. Those classifiers were: (1) NSR vs. AF vs. CHF, (2) NSR vs. AF, (3) NSR vs. CHF, (4) AF vs. CHF, (5) NSR vs. (AF and CHF), (6) AF vs. (NSR and CHF), and (7) CHF vs. (NSR and AF). As training data we selected one 5 min data window every 20 min (e.g., for a 20 h recording of one subject 60 5 min data windows were used). We used a repeated random sub-sampling validation⁴² where we trained on 10 subjects of each group and tested on the remaining 5 subjects of each group. This was repeated 300 times with each subject equally often used as training and testing subject. As classifier we used singular value decomposition (SVD).²⁷ As measure of performance we use Cohen's kappa κ^{43, 44, 45, 46} which can be computed directly from the confusion matrix.⁴⁷ A confusion matrix (also known as matching matrix, contingency table, or error matrix) is a specific table layout that allows visualization of the performance. Each column of the matrix represents the instances in a predicted class, while each row represents the instances in an actual class.

The random sub-sampling validation gives 300 values for Cohen's kappa for each model and each set of delays. To choose the best model we searched the highest minimum of the mean of the 300 values for each classifier. The best models and delays for the seven classification tasks are listed in Table TABLE IV..

TABLE IV.

Best DDE models selected for the seven classification tasks. The kappa values κ reported here were computed directly from the confusion matrices.⁴⁷ The area under the ROC curve $A\prime$ can only be computed for the 6 binary classifiers. $A\prime$ was computed from the ROC curves in Fig. 5. The units of the delays are time steps $\delta \tau =\frac{1}{f_s}$, where $f_s$ is the sampling frequency.

To test the performance of the classifiers in Table TABLE IV., we computed for each of the 7 models the 300 SVD-weights from the training data and then took the mean of those weights. These weights were then applied to the whole data set. Fig. 4 shows the computed features for all data for all seven classification tasks. The corresponding Cohen kappa values as well as the areas under the ROC curves are listed in Table TABLE IV.. The separating hyperplanes between the conditions were selected by SVD. In Fig. 4, the distance d from these hyperplanes are shown. NSR is best separated from the 2 diseases ($\kappa =0.96-0.99$). All other classifiers were also quite good.

Figure 4.

Distances d from the separating hyperplanes for all 5 min data windows for all 15 subjects for the three conditions (left plots) for the best DDE models reported in Tab. TABLE IV.. The mean value for each subject is shown as a black line. The histograms of these plots are shown on the right column. Blue refers to NSR, red to AF, and green to CHF. The black horizontal lines indicate the separating lines between the conditions selected by SVD. The vertical lines separate the subjects.

The model quality was also assessed by computing the area under the receiver operating characteristic (ROC) curves $A\prime$ (see Fig. 5).

Figure 5.

Receiver operating characteristic (ROC curves) and A' (area under the ROC curve) for the classifiers NSR vs. AF, NSR vs. CHF, AF vs. CHF, NSR vs. (AF, CHF), AF vs. (NSR,CHF), and CHF vs. (NSR,AF). The abbreviations N, A, and C in the legend correspond to NSR, AF, and CHF.

A receiver operating characteristic (ROC), or simply ROC curve,^{48, 49, 50, 51} is a graphical plot which illustrates the performance of a binary classifier system as its discrimination threshold is varied. It is created by plotting the fraction of true positives out of the positives vs. the fraction of false positives out of the negatives at various threshold settings. An $A\prime$ above 0.5 indicates classification performance above chance. In Fig. 5, all curves for the classifiers NSR vs. AF, NSR vs. CHF, AF vs. CHF, NSR vs. (AF,CHF), AF vs. (NSR,CHF), and CHF vs. (NSR,AF) are shown and $A\prime$ is for all binary classifiers above 0.92, which is excellent. Normal heart rate is better distinguishable from the two diseases: $A\prime$ is above 0.97 for NSR vs. AF, NSR vs. CHF, and NSR vs. (AF,CHF).

SVD is a simple classification tool. Performance could be easily improved by using a more sophisticated classifier such as support vector machine. Here, we wanted to emphasize on the results from the DDE analysis rather than best performance, which is already excellent.

DISCUSSION

We analyzed 24 h ECG data from healthy subjects and patients with either atrial fibrillation or congestive heart failure downloaded from the physionet database.³⁷ These data were analyzed using delay differential equations (DDEs). First, we made a connection between DDEs in the time domain and spectral analysis tools such as Fourier analysis and higher-order statistics. We then used the outputs of DDE models with a maximum of three terms to build good classifiers for the three conditions. For 5 min data windows of the ECG data from 2 electrodes we were able to separate the three heart conditions with high accuracy.

In other studies using the same dataset, separation of the three heart conditions was only achieved using all of the data for each subject;^{52, 53, 54} in comparison, we only needed 5 min of data from each condition. In our analysis, we needed non-linear terms in the DDE models to separate the three heart conditions. Non-linear methods were also better at distinguishing the data classes in Refs. ^{52, 53}. We found that a purely non-linear model with three non-linear terms was needed for the classification of NSR vs. AF vs. CHF, but a model with two linear and only one nonlinear term was needed for distinguishing CHF from NSR and AF in Refs. ^{55, 54}. This is consistent with other studies showing that CHF is a more regular heart condition.

We have thus shown that the three heart conditions are dynamically different (different DDE models are selected for the different classifiers), that the three heart conditions have different characteristic time-scales (different delays are selected for the different classifiers), and that non-linear models are needed for classification. In other studies, the same dataset was used to determine whether the heart rate is more chaotic in normal subjects, but this remains an open question.

Our analysis shows that a DDE can be considered a generic nonuniform embedding: “non-uniform” because the delays should reflect the dominant time-scales of the dynamical system and “generic” because it is a combination of a delay and a derivative embedding. The models combine functions of delayed versions of the signal with the derivative of the signal. A DDE model can unfold dynamical structures that are relevant for a single time series and the underlying dynamical system, which may be unknown as in the ECG.