Learning from local to global: An efficient distributed algorithm for modeling time-to-event data

Abstract

Objective

We developed and evaluated a privacy-preserving One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) without sharing patient-level information across sites.

Materials and Methods

Using patient-level data from a single site combined with only aggregated information from other sites, we constructed a surrogate likelihood function, approximating the Cox partial likelihood function obtained using patient-level data from all sites. By maximizing the surrogate likelihood function, each site obtained a local estimate of the model parameter, and the ODAC estimator was constructed as a weighted average of all the local estimates. We evaluated the performance of ODAC with (1) a simulation study and (2) a real-world use case study using 4 datasets from the Observational Health Data Sciences and Informatics network.

Results

On the one hand, our simulation study showed that ODAC provided estimates nearly the same as the estimator obtained by analyzing, in a single dataset, the combined patient-level data from all sites (ie, the pooled estimator). The relative bias was <0.1% across all scenarios. The accuracy of ODAC remained high across different sample sizes and event rates. On the other hand, the meta-analysis estimator, which was obtained by the inverse variance weighted average of the site-specific estimates, had substantial bias when the event rate is <5%, with the relative bias reaching 20% when the event rate is 1%. In the Observational Health Data Sciences and Informatics network application, the ODAC estimates have a relative bias <5% for 15 out of 16 log hazard ratios, whereas the meta-analysis estimates had substantially higher bias than ODAC.

Conclusions

ODAC is a privacy-preserving and noniterative method for implementing time-to-event analyses across multiple sites. It provides estimates on par with the pooled estimator and substantially outperforms the meta-analysis estimator when the event is uncommon, making it extremely suitable for studying rare events and diseases in a distributed manner.

INTRODUCTION

Real-world data (RWD) such as electronic health records (EHRs) and health plan claims are playing an increasing role in generating real-world evidence to support healthcare decision making.¹ Patient data (eg, diagnoses, medications, labs, and clinical notes) are routinely collected and entered into the EHRs during clinical care delivery. When compared with cross-sectional observational data, the detailed longitudinal information contained in the EHR enables time-to-event modeling, also known as survival analysis. Incorporating time into statistical models that predict occurrence of outcomes enables a better understanding of the predictors of when an event occurs rather than merely whether it occurred.^2–4 The Cox proportional hazards model (hereafter referred to as the Cox model) is one of the most commonly used time-to-event models, and has been widely applied in EHR-based studies for treatment evaluation, risk factor identification, and individual risk prediction.⁵

The last few years have witnessed an increasing number of clinical research networks, curating and using immense collections of health system EHRs and health plan claims data. Two prominent examples are (1) the Observational Health Data Sciences and Informatics (OHDSI) network—an international network of observational health databases that cover more than half a billion patient records in a common data model (CDM) and (2) the national Patient-Centered Clinical Research Network (PCORnet)—a network covering more than 100 million patients in the United States.^6,7 These large data consortia strive to provide platforms and tools to integrate heterogenous RWD from a diverse range of healthcare organizations. Multicenter analyses using RWD from these clinical research networks have been increasing rapidly in large part because they can improve the generalizability of the study results by increasing the sample size.

Despite the benefits of multicenter analyses, direct sharing of patient-level data across institutions may be challenging because of concerns related to patient and institutional privacy. Studies that include institutional data from multiple countries face legal and regulatory barriers to sharing patient-level data. The OHDSI network, which is an international consortium, uses a federated model in which the patient-level data are stored at local institutions and only aggregated information are shared. Thus, multicenter analyses have to be based on the summary statistics obtained from each site.^8–11 The results are often combined through meta-analysis, specifically, some form of weighted average where the weight for each estimate, eg, could be the inverse of the site-specific variance (for a fixed-effect model). However, when the outcomes or exposures are rare, or some of the sites may have small sample size, the accuracy (in terms of bias and precision) of the meta-analysis may be poor, as will be shown later in both the simulation studies and with real-world examples.

Distributed computing has been considered in many applications,^12–17 in which the model estimation is decomposed into smaller computational tasks that are computed locally at each site and then returned to the site instituting the study. For example, a distributed algorithm for conducting logistic regression, GLORE (Grid Binary Logistic Regression),¹³ and the WebDISCO (a web service for distributed Cox model learning) for fitting the Cox model¹⁴ were developed and successfully implemented in the pSCANNER (patient-centered Scalable National Network for Effectiveness Research) consortium.¹⁸ Through multiple rounds of iterative communication of information across sites, these algorithms can be lossless (ie, the results are equivalent to the results from fitting the model on the dataset created by pooling the individual-level data across all sites). However, owing to the need to iteratively transfer data across sites, it is time-consuming and labor-intensive in practice. Thus, development of noniterative distributed algorithms, which require neither data sharing nor iterative communication back and forth between sites, is an active research. For example, Chen et al¹² developed a lossless noniterative distributed algorithm for linear regression. Duan et al¹⁷ proposed a noniterative distributed algorithm for logistic regression through the construction of a surrogate likelihood.

In this article, we propose the One-shot Distributed Algorithm for Cox model (ODAC). A unique challenge in developing distributed algorithms for the Cox model, compared with our earlier work on logistic regression,¹⁷ is that the log partial likelihood function is not equivalent to the summation of local log partial likelihood functions from each site. Each component of the log partial likelihood function involves a nonlinear function of data from all patients who are at risk at a certain time point. By designing a novel initialization step, our algorithm extends the surrogate likelihood approach¹⁶ to deal with the more complicated Cox partial likelihood function, while maintaining the property of only requiring communication of aggregated information between sites. We show that our proposed algorithm is noniterative while achieving high accuracy (small bias) in both a simulation study and a real-world use case that studies the risk factors of acute myocardial infarction (AMI) and stroke using claims data from 4 different databases in the OHDSI network.

MATERIALS AND METHODS

Cox model

We first introduce the notation and basic assumptions of the Cox model. Let $X$ be a vector denoting $p$ risk factors and let $T$ be the time-to-event for the outcome of interest. The Cox model assumes the hazard at time $t$ follows

$$\lambda \left(t|X\right)={\lambda }_0\left(t\right)\exp \left({\beta }^{T}X\right),$$

where$\mathrm{ }{\lambda }_0\left(t\right)$ is the baseline hazard function, and $\beta$ is the vector of regression coefficients which are interpreted as the log hazard ratios (HRs). The observed data are $\{T_i,\mathrm{ }{\delta }_i,\mathrm{ }{x}_i\}$ for the $i$-th subject, with ${\delta }_i=0\mathrm{ }$indicating censoring and ${\delta }_i=1$ indicating an event, $i\mathrm{ }=\mathrm{ }1,\mathrm{ }\dots ,\mathrm{ }N$. For a given time t, we denote $R\left(t\right)=\mathrm{ }\{i;\mathrm{ }{T}_i\ge t\}$, which is the risk set at time t. The log partial likelihood function is constructed as

$$L\left(\beta \right)=\frac{1}{n}\sum _{i=1}^n{\delta }_i\log \frac{\exp \left({\beta }^T{x}_i\right)}{{\sum }_{j\in R\left(T_i\right)}\exp \left({\beta }^T{x}_j\right)},$$

which does not require estimation of the baseline hazard ${\lambda }_0$(t). The parameter $\beta$ is estimated by the value that maximizes the log partial likelihood function.⁵

Parameter estimation in a distributed network via ODAC

Now suppose that we have data stored in K different clinical sites and denote $n_j$ to be the sample size of the data at the j-th site. Denote the total sample size to be $N={\sum }_{j=1}^K{n}_j$. For the i-th patient at the j-th site, we observe $\{T_{ij},\mathrm{ }{\delta }_{ij},\mathrm{ }{x}_{ij}\}$. The risk set in site j at time t is defined as $R_j\left(t\right)=\{i;T_{ij}\ge t\}$, which contains all the subjects in site j that have not experienced an event or been censored at time $t$. The combined risk set over all the K sites is $R\left(t\right)=\left\{\right(i,j);T_{ij}\ge t\}$. With a slight abuse of notation, we further denote $R_{ij}$ to be the risk set at time $T_{ij}$, ie, $R_{ij}=R\left(T_{ij}\right).\mathrm{ }$Denote the set of event times at the j-th site to be ${\mathcal{T}}_j=\mathrm{ }\{T_{ij}:\mathrm{ }{\delta }_{ij}=1\}$, and the total set of unique event time to be $\mathcal{T}={\cup }_j{\mathcal{T}}_j$. Assume there are in total $d$ unique event time points $t_1,\mathrm{ }\dots ,\mathrm{ }{t}_d\in \mathcal{T}$. Ideally, if all the data could be pooled together, the combined partial likelihood function could be expressed as

$$L\left(\beta \right)=\frac{1}{N}\sum _{j=1}^K\sum _{i=1}^{n_j}{\delta }_{\mathit{ij}}\log \frac{\exp \left({\beta }^T{x}_{ij}\right)}{{\sum }_{(s,k)\in R_{ij}}\exp \left({\beta }^T{x}_{sk}\right)}$$ (1)

and a pooled estimator $\widehat{\mathrm{\beta }}$ could be obtained by maximizing the combined partial likelihood function. However, in reality, transferring patient-level data across sites is almost always impossible due to patient privacy concerns; thus, each site can only access their own data.

Inspired by the surrogate likelihood approach developed in Jordan et al¹⁶ and Duan et al,¹⁷ we aim to construct a proxy of the combined partial likelihood function, which we call a surrogate likelihood function. The idea is to construct a function that is close to the combined partial likelihood function defined in equation 1 near a neighborhood of the true parameter value. One naive choice of the surrogate likelihood function (eg, at the j-th site) is the local partial likelihood function $L_j\left(\beta \right)$ constructed as

$$L_j\left(\beta \right)=\frac{1}{n_j}\sum _{i=1}^{n_j}{\delta }_{\mathit{ij}}\log \frac{\exp \left({\beta }^T{x}_{ij}\right)}{{\sum }_{s\in R_j\left(T_{ij}\right)}\exp \left({\beta }^T{x}_{sj}\right)}$$ (2)

which only utilizes the patient-level data at the j-th site. However, this approximation does not utilize any information from other sites. Our goal is to further improve the accuracy of the approximation by borrowing aggregated information from other sites.

Define $\stackrel{-}{\beta }$ to be an initial value that is in a neighborhood of the true value of the parameter $\beta$. We propose the surrogate likelihood function obtained at site j to be

$${\stackrel{\sim }{L}}_j\left(\beta \right)=\mathrm{ }{L}_j\left(\beta \right)+〈\nabla L\left(\stackrel{-}{\beta }\right)-\nabla L_j\left(\stackrel{-}{\beta }\right),\beta 〉+\frac{1}{2}{\left(\beta -\stackrel{-}{\beta \mathrm{ }}\right)}^T\{{\nabla }^{2}L\left(\stackrel{-}{\beta }\right)-{\nabla }^2{L}_j\left(\stackrel{-}{\beta }\right)\}\left(\beta -\stackrel{-}{\beta \mathrm{ }}\right)$$ (3)

for$\mathrm{ }j=1,\mathrm{ }\dots K,\mathrm{ }$where $L_j(\beta )$ is the local likelihood function defined in equation (2), $\nabla \mathrm{ }$and ${\nabla }^2$ denote the first and second order gradients of a function (explicit forms of $\nabla L_j\left(\stackrel{-}{\beta }\right)$, $\nabla L\left(\stackrel{-}{\beta }\right),\mathrm{ }{\nabla }^2{L}_j\left(\stackrel{-}{\beta }\right)$ and ${\nabla }^{2}L\left(\stackrel{-}{\beta }\right)$ can be found in the Supplementary Appendix). Intuitively, the surrogate likelihood function in equation (3) utilizes the local likelihood function $L_j\left(\beta \right)$ as a baseline function, and it also adds a first-order term and a second-order term, $〈\nabla L\left(\stackrel{-}{\beta }\right)-\nabla L_j\left(\stackrel{-}{\beta }\right),\beta 〉$ and $\frac{1}{2}{\left(\beta -\stackrel{-}{\beta \mathrm{ }}\right)}^T\{{\nabla }^{2}L\left(\stackrel{-}{\beta }\right)-{\nabla }^2{L}_j\left(\stackrel{-}{\beta }\right)\}\left(\beta -\stackrel{-}{\beta \mathrm{ }}\right)$, to alter the shape of the local likelihood function arround $\stackrel{-}{\beta }$ toward the combined partial likelihood function.

In the construction of the surrogate likelihood function ${\stackrel{\sim }{L}}_j\left(\beta \right)$, once $\stackrel{-}{\beta }$ is obtained, the terms $L_j\left(\beta \right)$, $\nabla L_j\left(\stackrel{-}{\beta }\right)\mathrm{ }$and ${\nabla }^2{L}_j\left(\stackrel{-}{\beta }\right)$ can be calculated at the j-th site without needing to transfer information. And the constructing componenets $\nabla L\left(\stackrel{-}{\beta }\right)$ and ${\nabla }^{2}L\left(\stackrel{-}{\beta }\right)$ can be calculated distributively from all sites with the help of some preliminary summary-level statistics $U_j\left(\mathcal{T}\right)={(U}_j\left(t_1\right),\dots U_j\left(t_d\right))$, $W_j\left(\mathcal{T}\right)={(W}_j\left(t_1\right),\dots W_j\left(t_d\right))\mathrm{ }$and $Z_j\left(\mathcal{T}\right)={(Z}_j\left(t_1\right),\dots Z_j\left(t_d\right))$, where for each time point $t\mathrm{ }$in $\mathcal{T}$, $U_k(t)={\sum }_{i\in R_k\left(t\right)}\exp \left({\stackrel{-}{\beta }}^T{x}_{ik}\right)$, $W_k(t)={\sum }_{i\in R_k\left(t\right)}\hspace{0.17em}\text{exp}\hspace{0.17em}\left({\stackrel{-}{\beta }}^T{x}_{ik}\right)x_{ik}$, and $Z_k\left(t\right)={\sum }_{i\in R_k\left(t\right)}\hspace{0.17em}\text{exp}\hspace{0.17em}\left({\stackrel{-}{\beta }}^T{x}_{ik}\right)x_{ik}{x}_{ik}^T$. The detailed steps of distributively calculating $\nabla L\left(\stackrel{-}{\beta }\right)$ and ${\nabla }^{2}L\left(\stackrel{-}{\beta }\right)$ can be found in the Supplementary Appendix. Importantly, these gradients are all aggregated such that patient privacy is protected.

Regarding the initial value $\stackrel{-}{\beta }$, we recommend using the inverse variance weighted average of the estimates obtained by fitting a Cox model at each site, that is,

$$\stackrel{-}{\beta }={\left(\sum _{j=1}^K{\widehat{V}}_j^{-1}\right)}^{-1}\sum _{j=1}^K{\widehat{V}}_j^{-1}{\widehat{\beta }}_j$$ (4)

where ${\widehat{\beta }}_j=\mathrm{ }{\mathrm{argmax}}_{\beta }{L}_j\left(\beta \right)\mathrm{ }$is the estimator from of Cox model fitted on data at the j-th site, and ${\hat{V}}_j\mathrm{ }$is the estimated variance of ${\widehat{\beta }}_j$. After constructing the surrogate likelihood function, we can obtain a surrogate likelihood estimator at each site by

$${\stackrel{\sim }{\beta }}_j=\mathit{argma}{x}_{\beta }\mathrm{ }{\stackrel{\sim }{L}}_j\left(\beta \right).$$ (5)

The variance of ${\stackrel{\sim }{\mathrm{\beta }}}_{\mathrm{j}}$ is defined to be ${\stackrel{\sim }{\mathrm{V}}}_{\mathrm{j}}$, and it can be calculated using equation S.5 in the Supplementary Appendix. Finally, we combine all the local surrogate estimator ${\stackrel{\sim }{\mathrm{\beta }}}_{\mathrm{j}}$ using inverse variance weighted averaging in the same format as in equation 5. We summarize the ODAC algorithm in the following and also provide a schematic illutration in Figure 1.

ODAC algorithm

(1) Initialization

  In site k = 1 to K,

  do

    Fit a Cox model and obtain the local estimate ${\widehat{\beta }}_k$ and the variance estimate$\mathrm{ }{\widehat{V}}_k$; broadcast ${\widehat{\beta }}_k$, ${\widehat{V}}_k$, and the set of unique event time points in site k.

  end

(2) Local surrogate estimator

  In Site k = 1 to K,

  do

   obtain $\stackrel{-}{\beta }$ using (4), and all the unique event time points across all sites ${\mathrm{ }t}_1\dots t_d$; calculate and broadcast the intermediate summary-level statistics $U_j\left(\mathcal{T}\right),\mathrm{ }{W}_j\left(\mathcal{T}\right)$ and $Z_j\left(\mathcal{T}\right)$; construct the surrogate likelihood ${\stackrel{\sim }{L}}_k\left(\beta \right)$ by (3) treating the k-th site as the local site; obtain and broadcast ${\stackrel{\sim }{\beta }}_k$ and the variance ${\stackrel{\sim }{\mathrm{V}}}_{\mathrm{k}}$;

  end

(3) Evidence synthesis

  Obtain $\stackrel{\sim }{\beta }$ using (5) by plugging in ${\stackrel{\sim }{\mathrm{\beta }}}_{\mathrm{k}}$ and ${\stackrel{\sim }{\mathrm{V}}}_{\mathrm{k}}$.

Return $\stackrel{\sim }{\beta }$.

Figure 1.

Schematic illustration of ODAC to fit a multicenter Cox proportional hazards model algorithm. The first step is initialization, in which each site reports the local estimate of the log hazard ratio (${\widehat{\beta }}_j$), and the set of event times (${\mathcal{T}}_j$). In the second step, each site calculates the average of all local estimates, and obtain the union of all event times. Using this information, each site shares the intermediate results ($U_j\left(\mathcal{T}\right),\mathcal{ }{Z}_j\left(\mathcal{T}\right),\mathcal{ }{W}_j\left(\mathcal{T}\right)$). Next, each site combines the intermediate results and the local patient-level information to construct a surrogate likelihood function defined in equation 3, and obtains the local surrogate estimates by optimizing the surrogate likelihood function. The last step is the evidence synthesis, in which all the local surrogate estimates are combined through a weighted average.

A step-by-step illustration of ODAC using a simulated network with 2 sites

To better explain each step in the ODAC algorithm, we simulate 2 datasets that mimic a distributed network with 2 clinical sites. Site A has 100 patients and site B has 50 patients, and the goal is to fit a Cox model to study the association between the survival time and treatment (new drug vs placebo) adjusting for age. We generated age from a uniform distribution with a range from 20 to 60, and the treatment status was generated from a Bernoulli distribution with probability 0.5 of being in each arm. Before fitting the model, we standardized age by subtracting the mean age and dividing by the standard error. The survival time of each patient was then generated from a Weibull proportional hazards model, in which the baseline hazard follows a Weibull-distribution with scale 200 and shape 20 and the regression parameters for the treatment option and age are set to –1 and 1, respectively. We generated noninformative censoring from a Weibull distribution to keep the event rate at approximately 5%. Figure 2 provides a detailed step-by-step illustration of the implementation of the ODAC algorithm to this simulated dataset.

Figure 2.

Step-by-step illustration of ODAC to fit a multicenter Cox proportional hazards model algorithm using a simulation dataset. In the first step, the 2 sites share the estimated log hazard ratios and their variances [${\widehat{\beta }}_A=\left(-1.69,\mathrm{ }1.58\right),{\widehat{\beta }}_B=\left(-0.60,\mathrm{ }2.02\right),\mathrm{ }{\widehat{V}}_A=\left(1.20,\mathrm{ }0.90\right),{\widehat{V}}_B=(1.51,\mathrm{ }1.91)]$, as well as their set of unique event times, ${\mathcal{T}}_A=(21,\mathrm{ }23,\mathrm{ }24,\mathrm{ }31,\mathrm{ }32,\mathrm{ }36)$ and ${\mathcal{T}}_B=(25,\mathrm{ }27,\mathrm{ }39)$. In the second step, each site calculates the inverse variance–weighted initial estimator $\stackrel{-}{\beta }=(-0.68,\mathrm{ }1.72)$, and obtains the combined set of event times $\mathcal{T}$, which contains 9 unique time points. Then, each site shares 63 numbers in this step. Using these numbers, sites A and B obtain and share the surrogate likelihood estimates ${\stackrel{\sim }{\beta }}_A=\left(-\mathrm{0.75,1.54}\right),{\stackrel{\sim }{\beta }}_B=\left(-0.76,\mathrm{ }1.53\right)$, as well as the covariance matrix ${\stackrel{\sim }{V}}_A$ and ${\stackrel{\sim }{V}}_B$. In the evidence synthesis step, the final estimator ${\stackrel{\sim }{\beta }}_{\mathit{all}}$ is obtained by the weighted average of ${\stackrel{\sim }{\beta }}_A$ and ${\stackrel{\sim }{\beta }}_B$ where ${\stackrel{\sim }{\beta }}_{\mathit{all}}=(-0.76,\mathrm{ }1.54)$.

In the first step, the 2 sites fit the Cox model locally and share the initial estimates of the log HRs and their variances [${\widehat{\beta }}_A=\left(-1.69,\mathrm{ }1.58\right),\mathrm{ }{\widehat{\beta }}_B=\left(-0.60,\mathrm{ }2.02\right),\mathrm{ }{\widehat{V}}_A=\left(1.20,\mathrm{ }0.90\right),\mathrm{ }{\widehat{V}}_B=(1.51,\mathrm{ }1.91)].\mathrm{ }$In addition, each site reports their set of unique event times. In site A, there are 6 time points at which an event occurs, that is, ${\mathcal{T}}_A=(21,\mathrm{ }23,\mathrm{ }24,\mathrm{ }31,\mathrm{ }32,\mathrm{ }36)$, and in site B, we have ${\mathcal{T}}_B=(25,\mathrm{ }27,\mathrm{ }39)$. Therefore, in the first step, site A shares ${\widehat{\beta }}_A$, ${\widehat{V}}_A$, ${\mathcal{T}}_A$ and site B shares ${\widehat{\beta }}_B$, ${\widehat{V}}_B$, ${\mathcal{T}}_B$.

In the second step, each site calculates the inverse variance weighted initial estimator $\stackrel{-}{\beta }={(\widehat{V}}_A^{-1}{\widehat{\beta }}_A+\mathrm{ }{\widehat{V}}_B^{-1}{\widehat{\beta }}_B)/{(\widehat{V}}_A^{-1}+{\widehat{V}}_B^{-1})=(-0.68,\mathrm{ }1.72)$, and obtains the combined set of event times $\mathcal{T}$ which contains 9 unique time points. Then for each time $t\in \mathcal{T}$, site A calculates a scalar $U_A\left(t\right)$, a p-dimensional vector $W_A\left(t\right)$, and a matrix containing $p^2$ numbers $Z_A\left(t\right)$ (for simple illustration, we vectorize $Z_A\left(t\right)$ in Figure 2). Therefore, in this step, in total each site shares 42 numbers that do not contains patient-level information. Using these numbers, sites A and B are able to construct the surrogate likelihood function within each site, and then obtain and share the surrogate likelihood estimates ${\stackrel{\sim }{\beta }}_A=\left(-\mathrm{0.75,1.54}\right),{\stackrel{\sim }{\beta }}_B=\left(-0.76,\mathrm{ }1.53\right)$, as well as their corresponding covariance matrices ${\stackrel{\sim }{V}}_A$ and ${\stackrel{\sim }{V}}_B$.

In the evidence synthesis step, the final estimator ${\stackrel{\sim }{\beta }}_{\mathit{all}}$ is obtained as the weighted average of ${\stackrel{\sim }{\beta }}_A$ and ${\stackrel{\sim }{\beta }}_B$ with weights equal to the inverse of ${\stackrel{\sim }{V}}_A$ and ${\stackrel{\sim }{V}}_B$. In this example, we obtain ${\stackrel{\sim }{\beta }}_{\mathit{all}}=(-0.76,\mathrm{ }1.54)$, and the variance of ${\stackrel{\sim }{\beta }}_{\mathit{all}}$ is estimated by ${\stackrel{\sim }{V}}_{\mathit{all}}=\mathrm{ }{(\stackrel{\sim }{V}}_A^{-1}+{\stackrel{\sim }{V}}_B^{-1}{)}^{-1}$, which is closer to the true parameter value than the local estimates.

A real-world use case using OHDSI data

We used ODAC to study the risk factors of AMI and stroke in a population with pharmacologically treated major depressive disorder using data from 4 different U.S. insurance claims databases that have been converted to the Observational Medical Outcomes Partnership CDM¹⁹ in the OHDSI network. Both AMI and stroke were defined as the occurrence of the respective diagnosis codes in an inpatient or emergency room setting. We only counted the first occurrence of each condition per patient in order to preserve independence.

We fit Cox regression models for the 2 outcomes and corresponding risk factors. For AMI, we included known observed risk factors: age, gender, alcohol dependence, hyperlipidemia, hypertensive disorder, depression, obesity, and type 2 diabetes.^20,21 Similarly, for stroke the following known risk factors were included: congestive heart failure, coronary arteriosclerosis, hyperlipidemia, ischemic heart disease, renal failure, hypertensive disorder, transient cerebral ischemia, and type 2 diabetes.²² We compared our ODAC estimator with the pooled estimator and the estimator from meta-analysis.

A numerical experiment to demonstrate the benefit of ODAC in studying rare events

To demonstrate the benefit of ODAC comparing to meta-analysis in study rare events, we designed the following numerical experiment. A pooled dataset of N = 10 000 subjects were generated based on Weibull proportional hazards model, where the baseline hazard follows a Weibull-distribution with scale 200 and shape 20. We generated 2 covariates from independently and identically distributed uniform distributions and the true log HRs were set to be $\beta$= (–1, 2). We set the event rate (number of cases over number of subjects) as 20%, 5%, 2%, and 1% by appropriately modifying the distribution of censoring times. The pooled data were distributed over K = 10 clinical sites, with 5 large and 5 small sites. We set the relative sizes of the large to small sites to be 1000/1000, 1250/750, 1500/500. Under each scenario, we compared the ODAC estimator with the meta-analysis estimator over 1000 replications. Because the pooled estimator can be considered a gold standard, the bias to the pooled estimator is used as the metric to evaluate the performance of each method. For simplicity of illustration, we only present the results for estimation of coefficient ${\beta }_2$, as the simulation results for ${\beta }_1$ are similar.

RESULTS

Application to OHDSI data

The summaries of patients’ characteristics of the 4 datasets are listed in Table 1. The overall event rates are below 1% for both outcomes. The Medicare data (ie, IBM MarketScan Medicare Supplemental Database) were observed to have more elderly patients, and therefore a higher prevalence for both the outcome and risk factors of interest.

Table 1.

Characteristics of the 4 claims datasets at the Observational Health Data Sciences and Informatics

Dataset	CCAE	MDCD	MDCR	Optum
Subjects	64 222	59 861	69 164	62 348
Median age, y	43	35	71	47
Female, %	69.21	73.82	68.08	69.68
Congestive heart failure, %	0.70	3.06	7.58	2.61
Hypertensive disorder, %	20.81	31.80	57.70	32.96
Ischemic heart disease, %	1.70	3.82	10.27	4.10
Type 2 diabetes mellitus, %	7.49	14.63	21.83	12.71
Coronary arteriosclerosis, %	2.39	4.92	18.43	5.75
Renal failure syndrome, %	0.69	2.67	2.31	2.49
Transient cerebral ischemia, %	0.41	0.64	2.32	0.71
Hyperlipidemia, %	20.96	22.00	43.21	33.85
Obesity, %	7.15	16.54	6.71	9.62
Alcohol dependence, %	1.79	2.94	1.01	2.29
Major depressive disorder, %	4.17	3.55	3.16	3.34
Acute myocardial infarction, %	0.26	0.75	2.03	0.51
Stroke, %	0.24	0.73	1.75	0.58

^aThe 4 claims datasets are the IBM MarketScan Commercial Claims and Encounters Data (CCAE), IBM MarketScan Medicaid Multi-State Medicaid Database, IBM MarketScan Medicare Supplemental Database (MDCR), and OptumDe-IdentifiedClinformatics.

Figure 3 shows the estimated log HRs from the 3 methods as well as their 95% confidence intervals (CIs). The ODAC provided HR estimates are nearly identical to the pooled estimates for most of the risk factors, while meta-analysis estimates have substantial biases compared with the pooled estimator. The differences in estimated effect sizes can lead to potentially different conclusions in the investigation of risk factors. For example, the estimated log HR of alcohol dependence for AMI changed signs when comparing the pooled estimate (0.571; 95% CI, 0.322-0.820) to the meta-analysis estimate (−0.053; 95% CI, −0.361 to 0.255). In the HR scale, the pooled estimates was 1.770 (95% CI, 1.380-2.271) and the meta-analysis estimate was 0.948 (95% CI, 0.697-1.290). Furthermore, the meta-analysis estimates were sometimes qualitatively different from the pooled analysis and ODAC estimates. For example, both the pooled and the ODAC estimates suggested that alcohol dependence is significantly associated with the time to AMI, while the meta-analysis estimate showed it is not a significant risk factor. Similar contrasts were also present regarding risk factors such as hyperlipidemia, obesity, and depression. For all the risk factors of stroke, the relative bias of ODAC was <2%, whereas the relative bias of meta-analysis estimates were as high as 109%. This real-world example suggests that the ODAC estimator is preferred over meta-analysis in the investigation of risk factors of rare time-to-event outcomes.

Figure 3.

(A) Estimated log hazard ratios with 95% confidence intervals for risk factors of acute myocardial infarction using pooled analysis (red), meta-analysis (green), and One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) (blue). (B) Estimated log hazard ratios with 95% confidence intervals for risk factors of stroke using pooled analysis on the combined dataset across all sites (red), meta-analysis (green), and ODAC (blue).

Numerical expiriments

Our numerical study shows that ODAC has better performance than the meta-analysis estimator especially when the event is rare. The conclusion is supported by Figure 4, which shows the boxplots of the bias to the pooled estimator for different event rates and sample sizes. We observe that for all scenarios, ODAC obtains relative biases close to 0, meaning that it provides almost identical results to the pooled estimator. When the event rate is fixed and the sample size changes, the performance of the ODAC and meta-analysis estimators stay unchanged. When the event rate decreases, meta-analysis estimator is observed to have larger bias. When event rate is 1%, the average bias of the meta-analysis estimator is around –0.14, while the largest absolute bias of ODAC is 0.01. In addition to the difference in magnitude of bias, the variation of the meta-analysis estimator is much larger compared with that of the ODAC estimator.

Figure 4.

Boxplot of relative bias to the gold standard (pooled estimator obtained by fitting Cox model on the combined dataset across all sites) in different simulation settings. The 2 methods illustrated in the plot are meta-analysis (red) and One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) (blue). The event rate varies from 20% to 1%, and there are 3 different sample size distributions: (1) all 10 sites have 1000 samples, (2) 5 sites with 1250 samples and 5 with 750 samples, and (3) 5 sites with 1500 samples and 5 with 500 samples. Under each setting, the boxplots are based on 1000 replications of the experiment.

DISCUSSION

In this article, we developed and evaluated ODAC, a distributed, privacy-preserving algorithm to fit a Cox model in a federated multicenter setting. ODAC does not require iterative communication across sites, which minimizes the communication cost, and therefore is easy to implement in large research networks such as OHDSI and PCORnet as well as smaller networks such as those within PCORnet, such as PEDSnet (A National Pediatric Learning Health System)^23,24 and OneFlorida.²⁵

ODAC was shown to be significantly more accurate than conventional meta-analysis, particular when event rates were low, 1% or below; even though the communication cost (defined as the quantity of numbers needed to be transferred or shared) is greater than that of meta-analysis. Specifically, for a model with p parameters, the meta-analysis requires each site to share 2 numbers (a point estimate and a standard error for each parameter), while ODAC requires transferring ${{(M}_t+1\left)\right(p}^2+p+1)\mathrm{ }$numbers from each site, where $M_t$ is the number of unique event time points across all sites. Usually, $M_t$ is not very large in the EHR data setting, as the event time is often measured in days. For example, for a 1-year follow-up, the largest possible $M_t=\min (365,\mathrm{ }{n}_e)$, where $n_e$is the total number of events. When studying extremely rare events, $n_e$ is likely to be much smaller than the number of days during the follow-up period. As a consequence, when the number of covariates in the model is not extremely large, transferring $O\left(M_t{p}^2\right)$ numbers is not considered a burden in multicenter analysis. Yet, the improvement in estimation accuracy is substantial, compared with meta-analysis, particularly when the event rate is low. Compared with iterative methods such as WebDISCO, the communication cost of ODAC is low.

Implementing ODAL in distributive networks such as OHDSI is easy, as it does not require iterative communications. The initialization step is fairly standard in a multicenter analysis where each site shares their local estimates. After the initialization, each site is required to share the intermediate quantities $U_k,\mathrm{ }{Z}_k,\mathrm{ }{W}_k$ to all the other sites. These quantities all have closed forms and can be obtained using prewritten code or software packages distributed across the network. Both OHDSI and PCORnet have the necessary infrastructure—ARACHNE (A distributed OHDSI research network and study workflow orchestration)²⁶ and the PCORnet Query Tool²⁷ based on PopMedNet (an open-source application used to facilitate multisite health data networks)²⁸ to distribute these analytical codes. The surrogate partial likelihood estimator can also be obtained using an optimization function that is commonly found in prewritten code or software packages. The code for ODAC is available from the authors upon request.

The impact of this work is clear, on several dimensions. First, ODAC affords researchers the ability to conduct Cox analyses across many sites in a federated network. Second, ODAC provides a means to preserve the privacy and confidentiality of patients in the context of research conducted on federated data networks. Rather than require individual patient-level data from all network sites, such data are required from only 1 site. The other sites a proxied by aggregate data from each of the other sites. Third, federated data networks are essential to the study of rare diseases. ODAC estimators were shown to be superior to meta-analysis across a range of sampling regimes including rare outcomes.

Our ODAC algorithm is based on the pooled analysis, which is to fit a unified Cox regression model on the combined dataset. Therefore it essentially requires the data are homogenous distributed across sites. In the future, we plan to extend our method to handle heterogeneity across clinical sites by allowing site-specific effects and covariates. We are also developing open-source software packages to support studies that wish to use ODAC for multicenter analysis in existing distributed networks such as OHDSI and PCORnet. One key task is to implement ODAC according to the networks’ Observational Medical Outcomes Partnership CDM for OHDSI and the PCORnet CDM to alleviate the potentially complicated preprocessing procedures. We believe that ODAC is a significant contribution to the fast-growing distributed research networks with enormous patient-level RWD who are facing data sharing issues and privacy concerns, and ultimately will help facilitate collaborative efforts to investigate risk factors for time to event outcomes in healthcare systems. Some of the numerous applications in which it may be a particularly powerful tool involve use in modeling risk factors for disease progression and treatment effectiveness for uncommon and rare diseases. Because it minimizes communication costs, it will also be an excellent tool for international studies that require time-to-event modeling of outcome.

CONCLUSION

The proposed ODAC algorithm for multicenter Cox model is privacy-preserving and noniterative. Through a simulation study and a real-world use case using OHDSI data, ODAC was shown to have higher estimation accuracy compared with the meta-analysis, especially for studying rare events.

FUNDING

This work was supported in part by the National Institutes of Health grants 1R01LM012607, 1R01AI130460, 5R01LM010098, UL1TR001427, R21AG061431, R01CA246418, and 1R01AI116794, a grant from the Patient-Centered Outcomes Research Institute (PCORI) for the PEDSnet Clinical Research Infrastructure (RI-CRN-2020-007), and the Cancer Informatics and eHealth Core program at the University of Florida Health Cancer Center.

AUTHOR CONTRIBUTIONS

RD, CL, and YC designed methods and experiments; MJS provided the dataset from the Observational Health Data Sciences and Informatics; CL and JT conducted simulation experiments; MJS conducted data analysis; all authors interpreted the results and provided instructive comments; RD, CL, and YC drafted the main article. All authors have approved the article.

CONFLICT OF INTEREST STATEMENT

None declared.