Measuring Disease Burden with Individual Cumulative Incidence in Patients with Cirrhosis

Abstract

We introduce a new method for evaluating the health history of a patient through the use of multi-type recurrent events data that can aid in the assessment of disease progression and quality of life in a healthcare setting. The Disease Burden Score (DBS) is characterized as the area under the Disease Burden Curve (DBC), a monotone, increasing stepwise graph, that measures the weighted or unweighted number of health events that occur during a patient’s follow-up period (i.e. cardiovascular events or decompensation events). This measure is not only a valuable tool in cases where death data is not available or reliable but also as an interim measurement when death is infrequent. A DBS can be computed for all patients present in the data and thus can be used as a means of comparing subgroups and as the outcome variable in regression. We evaluate the performance of our method in simulation studies and illustrate the applications with an EHR dataset.

1. Introduction

Time-to-event endpoints serve as a valuable outcome measurement for summarising health data. Time-to-event analysis (also known as survival analysis) has regularly been employed in both clinical trials and observational studies and continues to motivate methodological advancements to accommodate the evolving paradigms in medicine. The basic principle of survival analysis is to measure how long a patient under study can survive “event-free”, where an event is a predefined, clinically relevant incidence such as death or disease progression. In studies with time-to-event endpoints, survival can be summarised in many different ways such as survival probability, median survival, and mean survival. Generally, what is being measured is the time until the first event occurs after entering the study.

The analysis of survival data would be relatively straightforward if the event times of all patients under study were observed. In practice, this is unlikely since many studies will end before all patients can experience an event, or patients may exit the study before an event is observed. The type of failure to observe an event time in survival analysis is referred to as censoring, and estimating numeric summaries of survival becomes complicated in the presence of censored data. The most commonly used method for evaluating survival probability or median survival is the Kaplan-Meier (KM) method. [1] Introduced by Edward Kaplan and Paul Meier in 1958, the KM method established a nonparametric method for estimating survival probabilities in the presence of censoring. The favored method for estimating and comparing mean survival in the presence of censoring has been restricted mean survival time (RMST) or the adjacent method, window mean survival time (WMST). RMST, first introduced by Irwin, establishes a time horizon $\tau$ to serve as an upper bound in computing a mean survival time (WMST utilizes two time horizons to create a “window” in which to evaluate mean survival).[2, 3] In each of these cases, the estimates are only measures of survival in relation to the first event to be observed.

In many clinical settings, multiple events can occur at the patient level in competition with each other (known as competing risks) or in succession (known as recurrent events). In competing risks analysis, the time until the first of a set of multiple, mutually exclusive (competing) events is measured, and estimation aims to identify the probability of each of the particular events occurring. For instance, one might wish to measure the risk of mortality due to cardiac events or liver-related events. In repeated events analysis, the event of interest can occur multiple times in succession. For example, patients at risk for cerebrovascular events may experience transient ischemic attacks,[4] cancer patients may experience recurrent superficial tumors [5] and individuals with psychiatric disorders may experience repeated episodes of hospitalization. A standard way to model repeated events is through their intensity functions, where the intensity at time $t$ describes the probability of a new occurrence at $t$, given an individual’s past history of occurrences. [6]

The repeated events methodology is most appropriate when there is a single type of event that recurs, such as cardiac infarction or hospitalization, but there are also scenarios where the timing of a set of differing health events is of interest. In the case of liver cirrhosis, which affects over two million American adults, decompensating events often serve as an important surrogate measure for death. [7] These decompensations are defined as an acute deterioration in liver function in a patient with cirrhosis and are characterized by ascites (asc), spontaneous bacterial peritonitis (sbp), variceal bleeding (vb), hepatic encephalopathy (he), hepatorenal syndrome (hrs), or hepatopulmonary syndrome (hps). These six different decompensating events can occur more than once (recurrent events) over time and are not mutually exclusive. In retrospective studies that utilize Electronic Health Records (EHR) datasets, the timing of these events is subject to left censoring and interval censoring due to patients only being diagnosed with a decompensating event at doctor visits. This results in events being present at the time of cirrhosis diagnosis and multiple decompensations can be recorded simultaneously (ie an occurrence of ascites might be diagnosed at the same time as an occurrence of hrs). In theory, no events occur at the very same time, but in practice, due to limitations in data collection, they might be observed at the same time. The repeated events methodology is not able to utilize ties at the patient level, nor is it able to utilize events that are coded at time 0. Hence, certain observations are ignored if repeated events methods are employed for analysis, losing valuable patient-level information.

There are several methods that directly address interval censoring in recurrent events settings, especially in the univariate point process case. [8, 9, 10, 11] Also, there have been methods research when considering the multi-type recurrent event paradigm. [12] Chen et al. utilize joint modeling methodology based on mixed Poisson models and marginal methods. In the Poisson models, patients are assumed to have a latent vector-valued subject-specific effect. These latent random effects are thought to represent unobserved covariates which, if available, can be used to characterize the variation in the Poisson rates between event types and among subjects in the population. The marginal method instead constructs generalized estimating equations from means and variances of the unconditional distribution obtained by marginalizing over the random effects. The latter is a commonly used method in the general multi-type recurrent events situation. [13] Ultimately, these methods are generally statistically complicated and laden with assumptions that may not be verifiable in practice. While they can recover information that may otherwise be lost due to interval censoring, their corresponding interpretation may be difficult for clinicians, physicians, and patients to understand, and if certain assumptions are invalid, the resulting analysis may suffer from bias.

In an attempt to utilize all available event information in these complex EHR settings in a manner that is intuitive and readily useful to biomedical researchers and physicians, we have developed a new statistical methodology for measuring and modeling a patient’s health progression that can incorporate any event that takes place during the study period (including ties and baseline events) and can serve as a surrogate measure for death when unavailable, or when death is not a meaningful endpoint (such as in the case of choosing candidates for organ transplantation). This method computes a score for each patient based on their event history during their follow-up period and can be used as a basis for comparing subgroups and as an outcome variable while performing regression. This score can be thought of as a given disease’s overall burden on a patient through the various clinically relevant health events and is thus named the disease burden score (DBS). Unlike its counterparts, it is computationally straightforward, can be used to compare subgroups and individuals, and doesn’t require distributional assumptions that can complicate application and interpretation.

The methods for computing DBS and constructing models with DBS as an outcome variable are described in Section 2. Various simulation studies were performed to evaluate the quality of the DBS models as well as the power of hypothesis tests that use DBS. The formulation of these simulation settings and their results are described and displayed in Section 3, a real data example is provided in Section 4, and a brief discussion of the utility and limitations of this method is in Section 5.

2. Methods

Cumulative disease burden is an individual measurement of a disease’s longitudinal impact on a patient over time and is computed by calculating the area underneath a patient’s incidence curve (disease burden curve). As in every time-to-event analysis, the disease burden curve (DBC) is bounded by 0 and can continue indefinitely (in theory). At each time point where an event occurs, the DBC will increase in height by whatever weight is applied to the specific event that occurred. In this setting, multiple events can occur at the same time point. The disease burden score (DBS), graphically represented as the area under the DBC, can be computed as ${\varphi }_i={\int }_0^{\infty }{f}_i(t)dt$ where $f_i(t)$ represents a function characterizing the $i^{\text{th}}$ patient’s DBC. Because follow-up times will differ between patients, a time horizon, $\tau$, can be used as a restriction that will make the corresponding $\tau$-year DBS directly comparable. In practice, the area under patient $i$’s DBC can simply be calculated by using the following summation:

$${\hat{\varphi }}_i=\sum _{k=1}^{m_i}\left(\tau -t_{ik}\right)\omega \left(e_{ik}\right)$$ (1)

where $t_{ik}$ is the time of patient $i$’s $k^{\mathit{\text{th}}}$ event incidence with $k=1,\dots ,m_i$ with $m_i$ being the total number of event incidences for patient $i,\tau$ is an upper-bound restriction time (which can also be selected as the last follow-up time for each patient, $e_{ik}$ is patient $i$’s $k^{\mathit{\text{th}}}$ event type, and $\omega \left(e_k\right)$ is a scalar of weights that assigns higher or lower weights to the vertical increase of the DBC based on what event has occurred at time $t_k$.

The DBC is easily visualized as a step-wise graph where a vertical step occurs at every incidence time. For any given weight function $\omega \left(e_k\right)$, the $k^{\mathit{\text{th}}}$ incidence will cause the DBC to increase by $\omega \left(e_k\right)$ units. In the most basic setting, when the identity weight function $\omega \left(e_k\right)=(\mathrm{1,\; 1},\dots ,1)$ is applied, every incidence causes the DBC to increase by one unit in the y-axis. In this simple case, it is similar to a patient-specific cumulative incidence curve where the final height of the DBC represents the total number of events that occurred during the follow-up period. In this paper, we will establish the basic methodology for the general weight function case but will primarily focus on the identity weight function in our simulation study.

Figure 1 displays the results from an analysis performed on two different patients who both experienced 15 decompensating events during the first 8 years of follow-up. The top figures display results when the identity weight vector is used, while the bottom figures display the results from when a non-identity weight vector is applied where more severe decompensating events are coded with a higher weight $(\text{asc},\text{sbp},\text{vb},\text{he},\text{hrs},\text{hps})\to \left(\frac{1}{3},1,\frac{1}{2},\mathrm{1,\; 1},1\right)$. These weights were chosen for illustrative purposes following general medical knowledge about the severity of differing decompensation events.

Figure 1:

Disease burden curves (DBC) with the area under the curve (DBS) shaded blue or green for patients A and B respectively. Both patients experienced 15 distinct decompensating events during the first 8 years of follow-up. The top two figures show the DBC and DBS in the identity weight case while the bottom figures display the DBC and DBS after a non-identity weight vector is applied.

Despite the equality in the total number of decompensating events that occur for patients A and B shown in Figure 1, this analysis reveals a distinct difference in the timing and severity of decompensating events. If we only consider the top figures where equal weights are applied for all decompensating events, patient B has a higher DBS than patient A. This is strictly a result of the timing of those decompensating events. By year 5, patient A has only experienced a total of 7 events while patient B has already experienced 11. The timing of these events is important because even though the events themselves may be acute, the consequences of these health events can be chronic or lead to other residual health problems. In other words, experiencing events earlier in the follow-up period results in dealing with the chronic outcomes for a longer period of time. Thus, the technique of analyzing time-to-event data through the disease burden paradigm penalizes events that happen earlier in the follow-up period (like a standard survival analysis) and often (like recurrent events analysis). Naturally, the DBS has a lower bound of 0, but, in theory, it doesn’t necessarily have an upper bound since the follow-up period could extend indefinitely or the total number of events that occur could be infinite. In practice, the follow-up period and total number of events will be finite.

Due to differences in follow-up times, different disease burden scores may not be directly comparable. For example, a patient who has experienced a single decompensating event at baseline and is followed up for 8 years will have a DBS of 8 (using the identity weight vector), whereas a patient who experiences an event at years 1, 2, and 3, but is lost to follow-up at year four will only have a DBS of 6. Thus, we implement an upper bound $\tau$ at which to perform our analysis. While this does result in not using all available information, it produces a measurement that is directly comparable from one patient to the next.

Another intuitive option for dealing with the different follow-up times might be to normalize the score based on the length of follow-up time. Several different choices were considered, including the most obvious of dividing the DBS by the length of follow-up time or taking the ratio of the area above and below the curve. In the case where patients do not experience any events (DBS of 0), the normalized score would still be 0 regardless of follow-up length, possibly biasing comparisons. Thus, we found it best from a practical and interpretation standpoint to proceed using a time horizon at which to restrict our comparison. Future work can be aimed at establishing a coherent, unbiased score that can be normalized appropriately based on follow-up length.

2.1. Modeling

A disease burden score can be computed for every sampled patient, therefore, it can be used as an outcome variable in a regression model. In the case where no events are observed for a patient a DBS score can still be computed rather than being considered a censored data point (DBS = 0).

Let $T_i$ be a vector of event times that includes the final observed follow-up time for patient $i$, where $i=1,\dots ,n$. In many survival settings, the final observation in $T_i$ would be defined as $D_i\wedge C_i=min\left(D_i,C_i\right)$ where $D_i$ is patient $i$’s death time and $C_i$ is patient $i$’s censoring time, assumed to be independent of $D_i$ conditional on the baseline covariates. In certain EHR data sets, death data may be unknown or unverifiable, therefore, we generalize to the case where the last entry of $T_i$ simply measures the final time when patient $i$ was present in the data and will be denoted as $\mathit{\text{tail}}\left(T_i\right)$. We then define $K_i$ as a vector of event-type information where each entry corresponds to any of the event times or the final observation time recorded in $T_i$. The covariates for prediction will be denoted as $Z_i$. Our observed data is then $\left(T_i,K_i,Z_i\right)$ for $i=1,\dots ,n$. Also, let $\tau$ represent a time horizon that forms an upper bound on the analysis.

We are interested in the area under the DBC up to time $\tau ,{\varphi }_i\left(T_i,K_i,\tau \right)$, modeled through an individual’s covariates, $Z_i$. Thus, for our purposes, we will consider a patient as having been censored whenever $\mathit{\text{tail}}\left(T_i\right)<\tau$ and will denote this by the indicator function ${\delta }_i^C=1\left(\mathit{\text{tail}}\left(T_i\right)<\tau \right)$. Then, if we let $\Phi (z,\tau )=E(\varphi (T,K,\tau )\mid Z=z)$, we can directly model this as:

$$g\{\Phi (z,\tau )\}={\beta }^{\prime }X$$ (2)

where $g(\cdot )$ is a smooth and strictly increasing link function, ${\beta }^{\prime }$ is a $(p+1)$-dimensional vector and $X^{\prime }=\left(1,Z^{\prime }\right)$. The trivial choice for the link function would be the identity link, but since the support of the DBS is bounded in practice, it may also be appropriate to consider some link function $g(\cdot )$, such as the log link, that ensures the estaimted mean DBS be positive.

For the general link function $g(\cdot )$, following the least squares principle, an inverse probability censoring weighted estimating function of $\beta$ is:

$$S_n(\beta )=n^{-1}\sum _{i=1}^n\frac{\left(1-{\delta }_i^C\right)}{{\hat{G}}_i(\tau )}{X}_i\left\{{\varphi }_i-g^{-1}\left({\beta }^{\prime }{X}_i\right)\right\}$$ (3)

where ${\hat{G}}_i(\tau )$ is the probability estimate of patient $i$ being lost to follow-up prior to $\tau$ based on $\left\{\left(\mathit{\text{tail}}\left(T_i\right),{\delta }_i^C\right),i=1,\dots ,n\right\}$. A Cox model can be utilized for generating ${\hat{G}}_i(\tau )$, but several other modelling procedures may be used in its place since, in theory, every patient $i$ will have a final observation time $\mathit{\text{tail}}\left(T_i\right)$ that represents when they are lost to follow-up. Further discussion on this is presented in Section 5. Now, let $\hat{\beta }$ be the unique root of $S_n(\beta )=0$.

The variance of $\hat{\beta }$ can be estimated as:

$$Var(\hat{\beta })\approx {\left[S_n^{\prime }(\hat{\beta })\right]}^{-1}{\Sigma }_{S_n(\hat{\beta })}{\left[S_n^{\prime }(\hat{\beta })\right]}^{-1}$$ (4)

where $S_n^{\prime }(\beta )$ is the first order derivative of the estimating equation with respect to $\beta$ solved at $\hat{\beta }$ and ${\Sigma }_{S_n(\hat{\beta })}$ is the covariance matrix of $S_n(\hat{\beta })$, which can be estimated using the empirical variance of $S_n(\hat{\beta })$. Since $S_n(\hat{\beta })$ is a mean of an iid sample, it follows a multivariate normal distribution with mean, $\beta$ and variance, ${\Sigma }_{S_n(\hat{\beta })}$.

3. Simulation Study

3.1. Model Evaluation

In order to assess the quality of the DBS modeling procedure described in the previous section, we created a simulation study to evaluate the bias of $\hat{\beta }$ as well as the precision of our variance estimates. We performed these simulations under different sample sizes, different levels of censoring, both the identity and the log link, and with varying levels of covariate effect.

To start this simulation we first generated baseline covariates, ${\mathbf{X}}_i={\left(X_{i1},X_{i2},X_{i3},X_{i4}\right)}^{\prime }$ for each patient, $i=1,\dots ,n$, with the first two elements generated from Unif(0, 1) and Normal(1, 0.4) respectively, and the third and fourth elements generated as Bern(0.3) and Bern(0.65) respectively. The DBS, ${\varphi }_i$, was then generated from:

$${\varphi }_i=g^{-1}\left({\beta }_0+{\beta }_1{X}_{i1}+{\beta }_2{X}_{i2}+{\beta }_3{X}_{i3}+{\beta }_4{X}_{i4}\right)+ϵ$$ (5)

where $ϵ$ is an error term generated from Normal(0, 1). For $g(x)=x$, we set $\beta =[4,2.5,-0.75,0,-3.5{]}^{\prime }$. For $g(x)=\text{log}(x)$ we set $\beta =[1.25,\text{log}(2.5),-\text{log}(1.25),\text{log}(1),-\text{log}(3){]}^{\prime }$. Two of the four covariates were selected to have a “strong” effect, one was selected to have a “weak” effect, and one was selected to have no effect in both the identity and log link cases.

With respect to censoring, while every patient with any follow-up time can have a DBS, not all can be measured at time horizon $\tau$. So, we generated loss-to-follow-up times using an exponential distribution with mean time of 6.67 years $(\lambda =0.15)$. Then we select different values of $\tau$ as our time horizon for computing DBS. At $\tau =1,2$, and 4 years, the estimated probability of being censored prior to the analysis time is roughly 14%, 26%, and 45% respectively.

We present the results for two different sample sizes, a large sample of 2000 and a small sample of 500 under low, moderate, and high censoring scenarios. For each scenario, 1000 iterations were generated. In Table 1, we display the results for the large sample size case with both the identity and log links. This table contains the bias, empirical standard error (ESD), asymptotic standard error (ASE), and the empirical coverage probabilities (CP) corresponding to the asymptotic 95% confidence intervals.

Table 1:

Simulation results for identity and log link: sample size = 2000 at three different levels of censoring. For each parameter, the bias, empirical standard deviation (ESD), the asymptotic standard error (ASE), and the empirical coverage probabilities (CP) corresponding to the asymptotic 95% confidence intervals are displayed.

Link	Censoring %	Parameter	Bias	ESD	ASE	CP
Identity	14%	${\beta }_0$	0.0015	0.0870	0.0852	0.946
		${\beta }_1$	−0.0015	0.0826	0.0834	0.947
		${\beta }_2$	0.0011	0.0592	0.0602	0.943
		${\beta }_3$	−0.0021	0.0518	0.0525	0.962
		${\beta }_4$	−0.0019	0.0525	0.0505	0.950
	26%	${\beta }_0$	0.0003	0.0916	0.0918	0.954
		${\beta }_1$	−0.0010	0.0902	0.0898	0.951
		${\beta }_2$	0.0015	0.0636	0.0648	0.958
		${\beta }_3$	−0.0022	0.0551	0.0566	0.965
		${\beta }_4$	−0.0013	0.0558	0.0544	0.943
	45%	${\beta }_0$	0.0015	0.1161	0.1151	0.948
		${\beta }_1$	−0.0032	0.1098	0.1125	0.957
		${\beta }_2$	0.0020	0.0801	0.0812	0.956
		${\beta }_3$	−0.0031	0.0690	0.0708	0.957
		${\beta }_4$	−0.0008	0.0699	0.0682	0.951
Log	14%	${\beta }_0$	0.0002	0.0324	0.0321	0.952
		${\beta }_1$	−0.0004	0.0342	0.0346	0.954
		${\beta }_2$	0.0004	0.0228	0.0233	0.947
		${\beta }_3$	−0.0009	0.0200	0.0202	0.963
		${\beta }_4$	−0.0009	0.0219	0.0215	0.943
	26%	${\beta }_0$	−0.0002	0.0344	0.0346	0.952
		${\beta }_1$	0.0001	0.0375	0.0373	0.954
		${\beta }_2$	0.0005	0.0246	0.0251	0.963
		${\beta }_3$	−0.0009	0.0213	0.0218	0.964
		${\beta }_4$	−0.0008	0.0234	0.0231	0.949
	45%	${\beta }_0$	0.0002	0.0435	0.0434	0.947
		${\beta }_1$	−0.0011	0.0460	0.0468	0.956
		${\beta }_2$	0.0007	0.0310	0.0315	0.954
		${\beta }_3$	−0.0014	0.0267	0.0274	0.959
		${\beta }_4$	−0.0003	0.0293	0.0290	0.952

The conclusion from Table 1 is that, in large samples, the proposed model produces unbiased estimates. Furthermore, the ESD and ASE match closely and the empirical coverage probabilities are similar to the nominal level, supporting the accuracy and reliability of the method under both link functions.

Additional simulation results for the smaller sample size case are displayed in Table A.4 in the Appendix. Similarly, the results from that set of simulations also demonstrate that the model produces unbiased estimates of regression coefficients and their corresponding standard errors.

3.2. Power

One of the main motivators for this method is the utilization of more information than standard survival techniques with repeating events, especially in an EHR setting. In theory, the use of more information should ultimately result in higher power to detect differences that exist between subgroups. We set up a set of simulations that investigate this potential advantage of using disease burden scores when there are multiple and recurring health events available in the data, as is the case with liver cirrhosis.

For this set of simulations, for simplicity and clarity, we only compare a treatment and control arm rather than investigating multiple variables. We have modeled our simulation approach based on the liver cirrhosis paradigm, and thus have six different types of events to reflect each of the six decompensating events. We investigate three different settings that are intended to mimic the different types of data collection schemes that take place in medical research. The first (S1) is like a randomized-controlled trial (RCT) where the true timing of the start of the disease and the true timing of all events are known. The second (S2) also mimics a RCT, but without knowing the true start time of the disease. Thus, patients enter the study at some time after they have contracted the disease and may have already experienced some decompensating events. The result is event times are potentially coded at entry into the trial (time = 0). Standard survival modeling procedures such as Cox regression and recurrent events regression need to disregard these events. After entry into the study, all events in S2 are coded continuously. The third setting (S3) reflects the EHR setting where the true disease start time is unknown as in S2, but also, data is only collected at discrete time points (when a patient visits a doctor or interacts with the health care system). Events occur continuously, but due to a discrete monitoring scheme, multiple events might be coded at the same time point (tied events). Again, Cox regression and recurrent events regression are unable to utilize all information if there are tied event times or events at baseline. Figure 2 shows a timeline example of how data is coded in S3. The blue points are the true, underlying event times, and the red points are the observed data that would be recorded at a doctor’s visit. Note that some events occur before the disease is discovered, thus they are recorded at what is the observed baseline of the disease (time = 0). Also, multiple events may occur between visits, but they are only recorded at discrete visit times. Lastly, note that between visit 1 and visit 2, this patient experienced the first event type twice, but when it is recorded, it is only coded once. This is done because, from a physician’s perspective, multiple events of the same type would not necessarily be distinguishable in the same interval of time. In the instances they are (such as in the case where events might be self-reported), this favors DBS rather than competing methods since it results in more tied information.

Figure 2:

Timeline representing how events occur versus how they are recorded in S3 (mimicking an EHR setting).

We equally allocate patients into the treatment and control groups with sample sizes of $n=300$ in each. In all three settings, we simulate every patient’s censoring time based on an exponential distribution with a mean of 24, 26, and 48 months. In S2 and S3, we additionally simulate a disease discovery time. What this represents is the lag time between the start of the disease and the point in time that the patient is officially diagnosed. It is then from this diagnosis/discovery time that the observation period begins. The disease discovery time is always simulated as exponential with a mean of 12 months. Lastly, in S3, for each patient, we simulate a collection of visit times based on a uniform distribution. While the decompensating events occur continuously between patient visits, they are only coded at the following visit time. For example, if a patient were to experience three different types of decompensating events at times 1.3, 2.5, and 3.2 and visits the doctor at times 2 and 4, the data would show that they experienced one decompensating event at time 2 and two decompensating events at time 4. If that patient were to experience the same type of decompensating event multiple times between visits, it would only be coded once at the following visit.

Each of the six different types of decompensating events has its own event-rate pattern. It is often the case in practice that once an event takes place, the likelihood of experiencing that event again increases. Thus, we simulate our events such that the event rates increase with each occurrence up to a certain point where the event rates begin to plateau. Figure 3 shows an example of how the exponential event rates change as an event recurs. In this example, the event rates between the treatment and control groups differ at the start of follow-up but eventually converge if the event occurs often enough. We will call this the converging event-rate scenario.

Figure 3:

Example of event-rate progression. The y-axis represents the exponential event rate and the x-axis represents the number of events that have occurred.

For each of the six different types of decompensating events, we set a starting event rate that characterizes the time to the first event, and an ending event rate that characterizes the $\lambda$ value that will be converged upon as events continue to occur. The rate at which events converge is based on the CDF of an exponential distribution with rate parameter of $\lambda =0.01$. We consider both a scenario where event rates converge and another where they cross. The starting and ending event rates for both cases are displayed in Table 2. Figures displaying the event rates for each event type and scenario, similar to Figure 3 can be found in the supplemental material.

Table 2:

Event rate parameters for power simulation. ${\lambda }_s$ denotes the starting event rate for the first event and ${\lambda }_e$ denotes the ending event rate being converged upon.

	Converging				Crossing
Event Type	Control Arm		Treatment Arm		Control Arm		Treatment Arm
	${\lambda }_s$	${\lambda }_e$	${\lambda }_s$	${\lambda }_e$	${\lambda }_s$	${\lambda }_e$	${\lambda }_s$	${\lambda }_e$
1	0.02	0.04	0.016	0.04	0.02	0.04	0.0125	0.045
2	0.025	0.06	0.021	0.06	0.025	0.06	0.02	0.065
3	0.03	0.055	0.025	0.055	0.03	0.055	0.025	0.06
4	0.015	0.045	0.0125	0.045	0.02	0.045	0.015	0.05
5	0.025	0.055	0.0185	0.055	0.025	0.055	0.025	0.06
6	0.0275	0.04	0.0225	0.04	0.03	0.04	0.02	0.045

Figure 4 displays the results of our power study based on the three different censoring distributions for each of the three scenarios with the same choice of $\tau$ for computing DBS. It is apparent that in S1, DBS can experience a power advantage over RE regression when event rates cross, but is outperformed when they merely converge. It is still valuable to see that DBS, under slower censoring rates, tends to produce higher power than does the Cox model/LR test even in these RCT style scenarios. Considering Cox models and LR tests are what are most often used in survival projects, this is an important advantage of DBS to consider. Importantly, in S3, the DBS produces uniformly better power than alternative methods. This is due to DBS being able to capture more of the decompensation information than RE regression because it doesn’t need to discard events at baseline or event ties. Results of the same simulation using the log link rather than the identity link are presented in Figure A.7 in the appendix.

Figure 4:

Power of DBS across three different censoring distributions. Tau was chosen as 24 for all of the simulations.

Because a time horizon needs to be chosen in advance of analysis, we have also conducted a simulation study to evaluate power across different choices of $\tau$. Figure 5 displays the results of our power study based on different choices of time horizon, $\tau$, with a fixed censoring rate of exponential with a mean of 36 months. When event rates cross, we see that for several choices of $\tau$, the DBS method has better power than RE regression even in S1, while it performs better in terms of power than RE regression for all choices in S3. Unsurprisingly, in the converging event rates scenario, RE regression performs best in S1, but suffers in S3. The reason for the DBS power dropping off for larger choices of $\tau$ is likely due to the sample size diminishing. Though the IPCW procedure will ensure unbiased estimates under any choice of $\tau$, the standard errors will grow larger with higher censoring as can be seen in Table 1 and A.4. Results of the same simulation using the log link rather than the identity link are presented in Figure A.8 in the appendix.

Figure 5:

Power of DBS vs. recurrent events regression and Cox model. The x-axis represents different choices of tau and the y-axis displays the corresponding power. Censoring was constant for all simulations as exponential with a mean of 36 months.

4. Real Data Analysis

An analysis of real data applying the DBS methodology was performed. We used the Chicago Area Patient-Centered Outcomes Research Network (CAPriCORN) data, which captures the diverse population of the greater Chicago metropolitan area, for several illustrative examples of using the DBS. The dataset encompasses 30 hospitals, and 10 health systems, totaling 12.8 million patients, which allows us to model and assess outcomes in patients with cirrhosis of the liver. The longitudinal nature of this EHR data allows us to investigate health outcomes based on time-dependent events. The CAPriCORN data merges institutional databases while protecting patient health information.

The study population includes patients who were diagnosed with cirrhosis between January 1, 2011, and December 31, 2021, based on a selection of ICD-9 and ICD-10 codes. Each patient’s study period began at the earliest appearance of a cirrhosis diagnosis code and continued until the latest date in their chart prior to the study end or loss-to-follow-up (whichever occurs first). Decompensating events were then recorded using relevant ICD codes throughout the study period for each patient.

For this analysis, we chose to perform a regression analysis using three numeric variables (age, frailty score, and Charlson comorbidity score) and two categorical variables (sex and hcc status). The Hospital Risk Frailty Score (HFRS) was used to establish a patient-specific frailty score, computed as a composite score of weighted frailty events (such as Alzheimer’s, dementia, falls, fractures, etc.). [14] The Charlson comorbidity score is computed based on 17 comorbidities, with two subcategories for diabetes and liver disease. Comorbidities are weighted from 1 to 6 for mortality risk and disease severity, and then summed to form the total Charlson score. [15]

Table 3 displays the results from various regression analyses using this real data. We selected $\tau =2$ and 5 years as the time horizon for computing DBS, and performed the regression analysis with both the identity and the log-link functions. For the identity-link function, the estimates of covariate effects, $\hat{\beta }$, can be easily interpreted as the unit increase in $\tau$-year DBS while holding all other variables constant. For example, with every unit increase in frailty score, a patient is expected to experience an increase in their 2-year DBS of 0.05, or male patients are expected to experience a 5-year DBS that is 0.95 higher than their female counterparts.

Table 3:

Outcomes from CAPriCORN data regression analysis using both the identity and log-link functions at two different choices of $\tau$.

Link	$\tau$	Covariate	$\hat{\beta }$	${\hat{\sigma }}^2$	Z	P-value
Identity	2	Intercept	1.941	0.069	28.10	< 0.0001
		Age	−0.015	0.001	−13.73	< 0.0001
		Frailty	0.050	0.002	20.93	< 0.0001
		Charlson	−0.004	0.003	−1.42	0.1550
		HCC	0.282	0.066	4.30	< 0.0001
		Sex (male)	0.289	0.026	11.03	< 0.0001
	5	Intercept	6.268	0.315	19.88	< 0.0001
		Age	−0.043	0.005	−8.55	< 0.0001
		Frailty	0.128	0.011	11.98	< 0.0001
		Charlson	−0.014	0.014	−0.97	0.3306
		HCC	0.281	0.276	1.02	0.3090
		Sex (male)	0.950	0.116	8.22	< 0.0001
Log	2	Intercept	0.668	0.044	15.19	< 0.0001
		Age	−0.010	0.001	−14.15	< 0.0001
		Frailty	0.023	0.001	21.27	< 0.0001
		Charlson	0.002	0.002	0.91	0.3622
		HCC	0.174	0.041	4.22	< 0.0001
		Sex (male)	0.207	0.019	11.01	< 0.0001
	5	Intercept	1.844	0.062	29.94	< 0.0001
		Age	−0.009	0.001	−8.78	< 0.0001
		Frailty	0.020	0.002	13.24	< 0.0001
		Charlson	−0.001	0.003	−0.22	0.8234
		HCC	0.053	0.058	0.929	0.3531
		Sex (male)	0.206	0.025	8.18	< 0.0001

As for the log link, the coefficients can instead be interpreted as relative changes rather than absolute changes. The change in DBS is multiplicative rather than additive, and $e^{\hat{\beta }}$ represents the relative difference in $\tau$-year DBS difference with a change in the respective covariate level. For example, with a unit increase in frailty score, the 2-year DBS score is expected to increase by a factor of $e^{0.023}=1.023$. In other words, the 2-year DBS is expected to increase by 2.3% for every unit increase in frailty score.

Overall, from Table 3 we can see that age, frailty, and sex all have significant effects on the DBS based on equally weighted decompensating events. HCC appears to have a significant effect on the 2-year DBS but not the 5-year DBS, indicating that its effect gets washed out over time.

5. Discussion

The DBS has several great advantages in theory and application. It relies on no distributional assumptions (such as proportional hazards), can account for all recorded events of interest (even those recorded at baseline or as ties), maintains the same interpretation in all circumstances, and can experience improved power over alternative methods (especially in an EHR setting). Not only can DBS be used as a means of comparing subgroups, but can be used to compare individual patients. Within hepatology, and with liver cirrhosis patients in particular, evaluating candidates for transplant is commonplace and has serious implications. The model for end-stage liver disease (MELD) score has been used to rank and prioritize liver transplant candidates for many years,[16] where the MELD score estimates disease severity in LT candidates based on serum creatinine, bilirubin, and the International Normalized Ratio (INR) of the prothrombin time.[17] This score has also been updated to include sodium levels, but it doesn’t account for any decompensating events. Hepatologists have long recognized the relationship between decompensating events and mortality and have begun calling for their use in evaluating candidates for liver transplants.[18] DBS is a scoring method that can afford the physician exactly that.

DBS also has its limitations. In order to ensure comparability between subgroups and individuals, an upper-bound time horizon, $\tau$ needs to be chosen. While this may be seen as a limitation, it is one that is shared by many other methods in survival analysis (RMST, WMST, quantiles, milestones). Also, the power of DBS to detect a treatment effect in an RCT setting is not uniformly better than the alternative methods. This is unsurprising, as the main motivation for establishing this method was to maximize the information available in the EHR setting. In most circumstances where there is a continuous monitoring scheme and all events can be coded at their correct times, then DBS will likely not be as powerful as RE regression (though this may not hold if the underlying intensity functions for events cross). Despite this limitation in power, DBS maintains a clinically intuitive interpretation which may not be true of the more powerful methods (especially when their underlying model assumptions are violated).

One of the important facets of the DBS modeling procedure is the IPCW. The purpose of incorporating IPWC into the procedure is to “recover” the patients we otherwise lose to censoring. In our case, censoring cannot be thought of in the traditional time-to-event sense (where censoring prevents the observation of the relevant event), since we are actually interested in a score that is a composite of events, and the score can be computed even when events don’t occur or aren’t captured during the follow-up period. For our score, a patient is censored if they are lost-to-follow-up before the prespecified time horizon $\tau$. Thus, how we think about computing IPCW values doesn’t necessarily need to follow traditional convention. Naturally, we can use a Cox model to estimate the weights, but because, in theory, the time at which each patient becomes unobservable is measured (whether as a death or as a traditional loss-to-follow-up), we can potentially use more straightforward modeling techniques for estimating the probability any given patient will be observable at the critical time point $\tau$. A $\tau$–year logistic regression could be performed, for example, since the outcome of interest (whether a patient $i$ is observable at $\tau$ can be measured for all $i$). There is greater nuance that can be imposed on this question, and we aim to investigate this more in future work.

6. Conclusion

The disease burden score is a strong candidate for investigating multi-event recurring data, especially in EHR settings when events can be coded at baseline or tied due to the observation scheme. In using DBS, researchers can take advantage of more of the available EHR data in their analyses and establish a more expansive view of the information present in those data sets. The primary value of the DBS is when there are varied and re-occurring health events that provide insight into the progression of a disease, such as Cirrhosis, especially in the absence of reliable death data. We have demonstrated through simulation that there are certain paradigms where standard survival techniques will fail to identify differences in health trajectories and disease progression between different subgroups of a population as effectively as DBS. In these circumstances, DBS can be leveraged to provide a fuller picture of the time-to-event nature of the data and create a measurement that can serve as an outcome variable in regression.

As a novel method, there are many avenues in which DBS can benefit from future work. One major aim is establishing data-driven approaches for selecting weight functions and possibly incorporating the time horizon, $\tau$, as a regression coefficient so that researchers do not need to prespecify it. Also, consideration for methods that normalize DBS scores rather than truncating them at $\tau$ is of interest.

Appendix A.

Table A.4:

Simulation results for identity and log link: sample size = 500 at three different levels of censoring. For each parameter, the bias, empirical standard deviation (ESD), the asymptotic standard error (ASE), and the empirical coverage probabilities (CP) corresponding to the asymptotic 95% confidence intervals are displayed.

Link	Censoring %	Parameter	Bias	ESD	ASE	CP
Identity	14%	${\beta }_0$	0.0002	0.1814	0.1700	0.937
		${\beta }_1$	−0.0088	0.1660	0.1665	0.948
		${\beta }_2$	0.0013	0.1255	0.1198	0.932
		${\beta }_3$	−0.0001	0.1032	0.1047	0.956
		${\beta }_4$	0.0010	0.1005	0.1008	0.954
	26%	${\beta }_0$	−0.0003	0.1945	0.1839	0.940
		${\beta }_1$	−0.0063	0.1792	0.1795	0.951
		${\beta }_2$	0.0008	0.1340	0.1291	0.941
		${\beta }_3$	0.0001	0.1111	0.1129	0.947
		${\beta }_4$	−0.0006	0.1083	0.1087	0.954
	45%	${\beta }_0$	0.0036	0.2399	0.2292	0.935
		${\beta }_1$	−0.0060	0.2298	0.2242	0.944
		${\beta }_2$	−0.0070	0.1709	0.1613	0.934
		${\beta }_3$	0.0078	0.1460	0.1409	0.938
		${\beta }_4$	0.0017	0.1312	0.1357	0.946
Log	14%	${\beta }_0$	−0.0001	0.0681	0.0643	0.934
		${\beta }_1$	−0.0022	0.0689	0.0696	0.949
		${\beta }_2$	0.0002	0.0490	0.0467	0.937
		${\beta }_3$	−0.0004	0.0402	0.0406	0.956
		${\beta }_4$	−0.0012	0.0424	0.0430	0.956
	26%	${\beta }_0$	−0.0006	0.0734	0.0695	0.942
		${\beta }_1$	−0.0009	0.0750	0.0752	0.950
		${\beta }_2$	−0.0001	0.0525	0.0504	0.935
		${\beta }_3$	−0.0004	0.0432	0.0439	0.948
		${\beta }_4$	−0.0022	0.0458	0.0464	0.959
	45%	${\beta }_0$	0.0002	0.0914	0.0873	0.941
		${\beta }_1$	−0.0001	0.0966	0.0944	0.941
		${\beta }_2$	−0.0032	0.0675	0.0636	0.937
		${\beta }_3$	0.0024	0.0569	0.0550	0.942
		${\beta }_4$	−0.0019	0.0564	0.0580	0.953

Figure A.6:

Event-rate pattern for the six different event types used in the power simulation. The y-axis is the event rate of an exponential distribution that determines the time to first event, or the gap time between subsequent events, and the x-axis represents the number of events that have occurred.

Figure A.7:

Power of DBS using log link across three different censoring distributions. Tau was chosen as 24 for all of the simulations.

Figure A.8:

Power of DBS with log link vs. recurrent events regression and Cox model. The x-axis represents different choices of tau and the y-axis displays the corresponding power. Censoring was constant for all simulations as exponential with a mean of 36 months.