SurvMaximin: Robust federated approach to transporting survival risk prediction models

Abstract

Objective:

For multi-center heterogeneous Real-World Data (RWD) with time-to-event outcomes and high-dimensional features, we propose the SurvMaximin algorithm to estimate Cox model feature coefficients for a target population by borrowing summary information from a set of health care centers without sharing patient-level information.

Materials and Methods:

For each of the centers from which we want to borrow information to improve the prediction performance for the target population, a penalized Cox model is fitted to estimate feature coefficients for the center. Using estimated feature coefficients and the covariance matrix of the target population, we then obtain a SurvMaximin estimated set of feature coefficients for the target population. The target population can be an entire cohort comprised of all centers, corresponding to federated learning, or a single center, corresponding to transfer learning.

Results:

Simulation studies and a real-world international electronic health records application study, with 15 participating health care centers across three countries (France, Germany, and the U.S.), show that the proposed SurvMaximin algorithm achieves comparable or higher accuracy compared with the estimator using only the information of the target site and other existing methods. The SurvMaximin estimator is robust to variations in sample sizes and estimated feature coefficients between centers, which amounts to significantly improved estimates for target sites with fewer observations.

Conclusions:

The SurvMaximin method is well suited for both federated and transfer learning in the high-dimensional survival analysis setting. SurvMaximin only requires a one-time summary information exchange from participating centers. Estimated regression vectors can be very heterogeneous. SurvMaximin provides robust Cox feature coefficient estimates without outcome information in the target population and is privacy-preserving.

1. Introduction

Electronic health records (EHR) have been widely adopted in the U. S. and other countries [1-5]. The EHR contains a wealth of patient medical information collected over time by health care providers, and common structured data types include demographics, diagnoses, laboratory test results, medications, and vital signs. Given its longitudinal nature, EHR data have been utilized for various research purposes, including survival analysis [6-8]. For example, the Cox proportional hazards model is used commonly and has been applied to EHR risk prediction [9].

With the increasing availability of EHR data, there is a great interest in integrating knowledge from a diverse range of health care centers to improve generalizability and accelerate discoveries. There now exist multiple collaborative consortia each composed of diverse health care centers seeking to leverage their EHR data in unison. For example, the Consortium for Clinical Characterization of COVID-19 by EHR (4CE consortium) is an international research collaborative that collects patient-level EHR data to study the epidemiology and clinical course of COVID-19 [10]. The consortium comprises more than 300 hospitals across seven countries with 83,178 patients, representing a broad range of multi-national health care centers serving diverse patient populations.

However, EHR data obtained from multiple diverse health care centers often exhibit a high degree of heterogeneity due to variability in EHR and data warehouse platforms, patient populations, health care practices, coding, and documentation. Further, patient-level data often cannot be shared directly between health care centers in a timely manner due to patient and institutional privacy laws [11]. Thus, there is a need for robust analytic strategies to overcome the barriers to conduct multi-center EHR studies.

Our objective is to jointly leverage multi-center, high-dimensional EHR data to make more precise inferences for a target population in the survival analysis setting by sharing only summary statistics obtained from each center, such as Cox feature coefficients and covariance matrices. The target population may be the entire population inclusive of all centers, a subset of centers, or a new, separate population. Integrative analysis approaches that only require individual sites to share summary statistics are often referred to as federated learning [12-14].

Most existing federated learning methods focus on settings with a small number of predictors and/or homogeneous settings where the underlying predictive models are shared across sites [12-14]. In addition, existing methods generally require several rounds of communication between sites, which can be inefficient and labor-intensive. To ensure transportability of models across sites, transfer learning methods have been proposed to transfer knowledge from separate but related centers to provide robust and precise estimates for patients in a new center. This approach has widespread applications in medical studies such as drug sensitivity prediction, integrative analysis of “multi-omics” data, and natural language processing [15-18]. However, most transfer learning methods require outcome labels from the target population, which may be difficult and expensive to obtain, and do not consider the federated learning scenario where individual-level data cannot be shared across sites. In the absence of outcome labels in the target population, transfer learning methods require stringent assumptions that the target and source populations share the same underlying risk model, leading to potential transfer failure when the risk model for the target population is similar to only a subset of source populations [19-21].

With heterogeneous training datasets from multiple centers, one potential limitation of the existing federated transfer learning methods is that the performance of the prediction model can vary substantially across centers. Thus, although the overall performance may be satisfactory, the performance of the model in a particular center might be low. Moreover, when trained models are applied to a new population, transferability and portability are not guaranteed. To improve the robustness of prediction models, the maximin effect approach was first proposed in [22-24], and used as a metric to build a robust prediction model for continuous outcomes across heterogeneous training datasets. Instead of optimizing the average performance across all training datasets, the maximin effect method aims to train a model that maximizes the minimum gain over the null model among all training datasets. The maximin approach was further extended to a setting that allows for covariate shift between the source and target populations [25]. The group distributional robustness optimization in [26,27] is closely related to the maximin effect, which builds a robust prediction model by minimizing the worst-case training loss over a class of distributions. The maximin projection has been developed in [28] to construct the optimal treatment regimen for new patients by leveraging training data from different groups with heterogeneity in optimal treatment decision.

In this paper, motivated by the maximin algorithm for continuous outcomes in [22,25], we propose a maximin transfer learning algorithm for predicting a survival outcome (SurvMaximin) in a target population with high-dimensional features by robustly combining multiple prediction models trained in different source populations. This algorithm only requires sharing of summary statistics across centers and can easily accommodate high-dimensional features. SurvMaximin can be viewed as a robust federated approach to transfer models trained at multiple external centers to a target population, so we refer to it as a federated transfer learning method. SurvMaximin differs from existing transfer learning methods in that it does not require the target population to share the same underlying model with the source population, a highly desirable property when learning with multiple heterogeneous health care systems. The training of the SurvMaximin algorithm also does not require the target population to have gold-standard outcome labels.

2. Methods

2.1. SurvMaximin: Federated robust transfer learning for survival outcomes

The main aim of the SurvMaximin algorithm is to derive a robust risk prediction model for an unlabeled target population based on labeled data from the L source populations under the data sharing constraints. Suppose there are L source populations, indexed by l ∈ {1, …, L}, representing L studies and one target population, denoted by Q. The observed data from the l th source population consists of n_l independent and identically federated random vectors ${𝒟}_l=\{({\mathit{Z}}_{li},X_{li},{\delta }_{li}){\}}_{1\le i\le n_l}$, where Z_li denotes the p-dimensional standardized baseline risk factors with p potentially large relative to the sample sizes, the censored survival times are observed as X_li = min(T_li, C_li) and δ_li = I(T_li ≤ C_li) with T_li and C_li denoting the survival time and follow up time for the i th subject in the lth population, respectively, and I(·) is an indicator function which equals to 1 if the inside is true and equals to 0 otherwise. In the target population Q, only the baseline features {Z_Qi}_1≤i≤nQ are observed. We assume that the survival time in the l th source population, T_l, given the baseline features Z_l follows a Cox proportional hazards model, ${\Lambda }_l(t\mid {\mathit{Z}}_{li})={\Lambda }_{0,l}(t)\text{exp}\left({\mathit{b}}_l^T{\mathit{Z}}_{li}\right)$, which can be equivalently expressed [29] as.

$$\begin{gathered} {\tilde{T}}_{li}\equiv \log {\Lambda }_{0,l}(T_{li})=-{\mathit{b}}_l^T{\mathit{Z}}_{li}+{\in }_{li}\text{with}{\in }_{li}\perp {\mathit{Z}}_{li}\text{and}P({\in }_{li}>x) \\ =\text{exp}\{-\text{exp}(x)\} \end{gathered}$$ (1)

where Λ_l(t∣Z_li) is the conditional cumulative hazard function given Z for the l th population, Λ_0,l(t) is the cumulative baseline hazard function, b_l ∈ R^p denotes the vector of unknown log hazard ratio parameters associated with the risk factors Z, and ⊥ denotes independence, ∈ is a random variable with distribution specified in (1). We assume that distributions of the baseline risk factors, hazard ratio parameters, and the baseline hazard functions may vary across the source populations due to study heterogeneity. Similarly, we assume that the survival time from the target population, T_Q, follows a Cox model with unknown baseline cumulative hazard Λ_0,Q(·) and feature effect function β_Q:

$${\tilde{T}}_{Qi}\equiv \log {\Lambda }_{0,Q}(T_{Qi})=-{\beta }_Q^T{\mathit{Z}}_{Qi}+{\in }_{Qi}\text{with}{\in }_{Qi}\perp {\mathit{Z}}_{Qi}\text{and}P({\in }_{Qi}>x)=\text{exp}\{-\text{exp}(x)\}.$$ (2)

The SurvMaximin algorithm aims to identify a robust approximation to β_Q based on the estimated hazard ratio parameters trained from $\{{𝒟}_l{\}}_{1\le l\le L}$ as well as the target feature distribution. Due to the lack of gold standard labels on Q and the unspecified heterogeneity among {b_l}_1≤l≤L and β_Q, the target β_Q cannot be identified with the observed data. Instead of targeting β_Q directly, the central idea of the SurvMaximin algorithm is to identify an approximation to β_Q that maximizes the minimum reward across all L source populations. Following [25], we define the hypothetical outcomes for the n_Q subjects in Q generated from the lth source models as.

$$\begin{gathered} {\tilde{T}}_{{}_{li}}^{{}^{\star }}\equiv -{\mathit{b}}_l^T{\mathit{Z}}_{Qi}+{\in }_{li}^{\star }\text{with}{\in }_{li}^{\ast }\perp {\mathit{Z}}_{Qi}\text{and}P\left({\in }_{li}^{\star }>x\right)=\text{exp}\{-\text{exp}(x)\}\text{for}1\le l \\ \le L, \end{gathered}$$ (3)

where ${\tilde{T}}_{li}^{{}^{\star }}$ can be viewed as the hypothetical outcome (transformed survival time) if the individual, Z_Qi was assigned to the l th source population. Then we define a robust prediction model as.

$${\beta }_Q^{\star }=\underset{\beta \in {\mathbb{R}}^{\mathrm{p}}}{\text{arg}\max }{R}_Q(\beta )\text{with}{R}_Q(\beta )=\underset{1\le l\le L}{\text{min}}\left\{\mathrm{E}{\left({\tilde{T}}_{{}_{li}}^{{}^{{}^{{}^{\star }}}}\right)}^2-E{\left({\tilde{T}}_{{}_{li}}^{{}^{{}^{{}^{\star }}}}+{\beta }^T{\mathit{Z}}_{Qi}\right)}^2\right\}$$ (4)

the expectation is taken with respect to Z_Qi and ${\left\{{\tilde{T}}_{{}_{{}_{li}}}^{{}^{{}^{{}^{\star }}}}\right\}}_{1\le l\le L}$ defined in (3). Such a covariate shift maximin effect was defined in [25] for the linear model and is now extended to the Cox regression model. We shall note that $E{\left({\tilde{T}}_{{}_{{}_{li}}}^{{}^{{}^{{}^{\star }}}}\right)}^2-E{\left({\tilde{T}}_{{}_{{}_{li}}}^{{}^{{}^{{}^{\star }}}}+{\beta }^T{\mathit{Z}}_{Qi}\right)}^2$ is a reward function of β, which represents the variance of ${\tilde{T}}_{li}^{{}^{\star }}$ explained by the linear prediction −β^TZ_Qi. Our targeted maximin effect is maximizing the adversarial reward R_Q(β) across the L groups. The SurvMaximin estimate ${\beta }_Q^{{}^{\star }}$ leads to a robust prediction model since the optimization in (4) guards against the worst-case scenario. The maximin effect can be interpreted from an adversarial perspective [23]: in a two-side game, we select an effect vector β and the counter agent then chooses the most challenging scenario for this β; that is, choose the source population such that β has the worst predictive performance. Our goal is to choose β such that the worst-case reward with respect to predicting the transformed survival time returned by the counter agent is maximized.

As shown in the Supplementary Materials, R_Q(β) can be equivalently expressed as:

$$\begin{gathered} R_Q(\beta )=\underset{1\le l\le L}{\text{min}}\left\{E{\left({\mathit{b}}_l^T{\mathit{Z}}_{Qi}\right)}^2-E{\left({\mathit{b}}_l^T{\mathit{Z}}_{Qi}-{\beta }^T{\mathit{Z}}_{Qi}\right)}^2\right\} \\ =\underset{b\in \mathbb{B}}{\text{min}}\left\{2{\mathit{b}}^T{\Sigma }_Q\beta -{\beta }^T{\Sigma }_Q\beta \right\}, \end{gathered}$$ (5)

where Σ_Q = E[Z_Qi(Z_Qi)^T], and $\mathbb{B}=\{{\mathit{b}}_l,l=1,\cdots ,L\}$. Following [25],we may show that the maximin effect ${\beta }_{\mathrm{Q}}^{{}^{\star }}$ as defined by (4) can be expressed as a weighted average of {b_l}_1≤l≤L,

$${\beta }_Q^{\star }=\mathit{B}{\gamma }_Q^{\star }\text{with}{\gamma }_Q^{\star }=\underset{\gamma :\Vert {\gamma }_l{\Vert }_1=1,{\gamma }_l\ge 0}{\text{argmin}}{\gamma }^T{\Gamma }_Q\gamma$$ (6)

where B_p×L = [b₁, …, b_L], ∥·∥_q denotes the L_q norm, Γ_Q = B^TΣ_QB is a similarity matrix, and the minimization above is restricted to the simplex in L-dimension space. The optimal aggregation weight ${\gamma }_{\mathrm{Q}}^{{}^{\star }}$ in (6) depends on both {b_l}_1≤l≤L and the covariance matrix Σ_Q for the target population. The identification equation (6) reveals an important geometric interpretation of the maximin effect: ${\beta }_{\mathrm{Q}}^{{}^{\star }}$ is the point that has the smallest distance to the origin and lies on the convex combination of the regression vectors {b_l}_1≤l≤L [22]. The maximin estimator tends to shrink the components of {b_l}_1≤l≤L whose estimated coefficients vary with different signs across studies to zero and is not as sensitive to the inclusion of sites with an extreme hazard ratio regression vector [22]. In the transfer learning setting, we incorporate the target distribution Z_Q into the definition of the distance.

2.2. Implementation of the SurvMaximin algorithm

The SurvMaximin algorithm involves three key steps: (I) locally train the prediction model for each of the L source sites to obtain {b_l}_1≤l≤L; (II) estimate the covariance matrix Σ_Q and obtain a similarity matrix among $\left\{{\widehat{\mathbf{b}}}_l^T{\mathit{Z}}_Q,l=1,\cdots ,L\right\}$, denoted by ${\widehat{\Gamma }}_Q$; and (III) obtain the final SurvMaximin estimator as an optimal linear combination of $\{{\widehat{\mathbf{b}}}_l{\}}_{1\le l\le L}$ according to (6). The schema of the SurvMaximin algorithm is shown in Fig. 1.

Fig. 1.

Schematic of SurvMaximin algorithm for federated transfer learning.

Step I: Training L local risk prediction models.

We first obtain ${\widehat{\mathbf{b}}}_l$ as the maximizer of the penalized partial likelihood.

$${\widehat{\mathbf{b}}}_l=\underset{\mathbf{b}}{\text{arg}\max }\{{\ell }_l(\mathbf{b})+{\lambda }_l𝒫(\mathbf{b})\}$$ (7)

where ${\ell }_l(\mathbf{b})$ is the log partial likelihood associated with ${𝒟}_l$ and $𝒫(\mathbf{b})=\alpha \Vert \mathbf{b}{\Vert }_1+(1-\alpha )\Vert \mathbf{b}{\Vert }_2^2$ is the elastic net penalty function, which is frequently used to overcome high dimensionality and collinearity of features with α = 1 corresponding to the standard LASSO and α = 0 corresponding to the ridge penalty [30]. The non-negative penalty parameter λ_l can be selected via standard tuning criteria including the AIC, BIC, or cross-validation.

Step II: Estimate the similarity matrix among.$\left\{{\mathbf{b}}_l^T{\mathit{Z}}_Q,l=1,\cdots ,L\right\}$

We estimate the similarity matrix of $\left\{{\mathbf{b}}_l^T{\mathit{Z}}_Q,l=1,\cdots ,L\right\}$, ${\Gamma }_Q={\mathbf{B}}^T{\Sigma }_Q\mathbf{B}$, as ${\widehat{\Gamma }}_Q={\widehat{\mathbf{B}}}^T{\widehat{\Sigma }}_Q\widehat{\mathbf{B}}$, where ${\widehat{\mathbf{B}}}_{p\times L}=[{\widehat{\mathbf{b}}}_1,\cdots ,{\widehat{\mathbf{b}}}_{\mathit{L}}]$ and ${\widehat{\Sigma }}_Q$ is the empirical variance covariance matrix of Z_Q estimated based on the unlabeled target population data.

Step III: Maximin aggregation via (6).

Finally, we obtain the SurvMaximin aggregated log hazard ratio estimator as.

$${\widehat{\beta }}_Q=\widehat{\mathbf{B}}{\widehat{\gamma }}_Q\text{with}{\widehat{\gamma }}_Q=\underset{\gamma :\Vert {\gamma }_l{\Vert }_1=1,{\gamma }_l\ge 0}{\text{argmin}}{\gamma }^T{\widehat{\Gamma }}_Q\gamma +\eta \Vert \gamma {\Vert }_{{}_2}^{{}^2},$$ (8)

where η ≥ 0 is the tuning parameter and the ridge penalty is included to account for the potential high collinearity among $\{{\widehat{\mathbf{b}}}_l{\}}_{1\le l\le L}$. See Supplementary Materials for additional information on data adaptive approach to selecting η. In practice, we find that when there is some heterogeneity observed as in our 4CE studies, setting η = 0 works well and the results are not sensitive to the choice of η when a relatively small η is chosen.

2.3. Transfer to a target site with missing features

A substantial challenge in transfer learning across different health care centers is that certain risk predictors, such as laboratory test results or demographic information, may be available in one center but not in a different center. For example, in the 4CE consortium, all U.S. centers report data on race while European centers do not, causing race data to be entirely missing for European centers. To transport a risk prediction model for a target center Q with only a subset of features available, one may fit a reduced model limited to only the available features for each source center and transport the reduced risk models from the source centers. When the target center changes, essentially, we will need to retrain the model at each source center according to the feature availability of the target center. Such an approach is not computationally efficient as each center needs to fit multiple models, and also increases the number of communications required across centers.

To enable transfer learning in the context of differential feature availability, we propose a simple projection approach that only requires each source center to additionally compute the empirical covariance matrix of the features, $\{{\widehat{\Sigma }}_l{\}}_{l=1,\cdots ,L}$, where ${\widehat{\Sigma }}_l=n_l^{-1}{\sum }_{i=1}^{n_l}{\mathit{Z}}_{li}{\mathit{Z}}_{li}^T$. Let $𝒜\subset \{1,\cdots ,p\}$ index features that are available at the target site, ${𝒜}^{{}^c}=\{1,\cdots ,p\}\setminus 𝒜$. Let ${\mathit{Z}}^{[𝒜]}$ denote the subvector Z corresponding to $𝒜$. The key step is to project ${\widehat{\mathbf{b}}}_l^T{\mathit{Z}}_l$ to the subspace spanned by ${\mathit{Z}}_l^{[𝒜]}$, ${\widehat{\mathbf{b}}}_l^T{\mathit{Z}}_l={\widehat{\theta }}_l^T{\mathit{Z}}_l^{[𝒜]}+e_l$, and predict T_l based on ${\widehat{\theta }}_l^T{\mathit{Z}}_l^{[𝒜]}$. Since the features are all assumed to be centered, we obtain ${\widehat{\theta }}_l={\widehat{\mathbf{b}}}_l^{[𝒜]}+{\widehat{\alpha }}_l{\widehat{\mathbf{b}}}_l^{[{𝒜}^c]}$ and ${\widehat{\alpha }}_l={\left({\widehat{\Sigma }}_l^{[𝒜,𝒜]}\right)}^{-1}{\widehat{\Sigma }}_l^{[𝒜,{𝒜}^c]}$, where ${\widehat{\Sigma }}_l^{[𝒜,𝒜]}$ and ${\widehat{\Sigma }}_l^{[𝒜,{𝒜}^c]}$ denote the submatrices of Σ_l corresponding to $\{𝒜,𝒜\}$ and $\{𝒜,{𝒜}^{{}^c}\}$. The final SurvMaximin estimator for the feature effects of ${\mathit{Z}}_Q^{[𝒜]}$ can be constructed by replacing $\{{\widehat{\mathbf{b}}}_l{\}}_{1\le l\le L}$ with $\{{\widehat{\theta }}_l{\}}_{l=1,\cdots ,L}$ and ${\widehat{\Sigma }}_Q$ with ${\widehat{\Sigma }}_Q^{[𝒜,𝒜]}$. If ${\mathit{Z}}_l^{[𝒜]}$ is of high dimension or the vectors in ${\mathit{Z}}_l^{[𝒜]}$ are colinear, in which case the covariance ${\widehat{\Sigma }}_l^{[𝒜,𝒜]}$ is not invertible, we can apply regularization methods in [31,32] to stably invert ${\widehat{\Sigma }}_l^{[𝒜,𝒜]}$ and construct ${\widehat{\alpha }}_l$.

2.4. Validation of SurvMaximin algorithm

We validated the performance of SurvMaximin in federated transfer learning using both simulation studies and a real-world study where we transported COVID-19 mortality risk prediction models to target centers using EHR data from hospitalized patients with COVID-19.

2.4.1. Simulation studies

Simulation studies were conducted to assess the performance of SurvMaximin and to compare its performance against existing federated learning methods. Since SurvMaximin transports a risk prediction model to a future target center without survival outcomes observed, we used other federating learning methods that also do not require supervised training on the target data as comparisons. Specifically, we consider the standard random effect meta-analysis estimator (herein referred to as Meta); the One-shot Distributed Algorithm (ODAC) for the Cox model [12]; and the locally trained risk prediction model with varying training sizes of n_Q = 200, 400, and 600. We considered simulation scenarios with L = 15 centers each with sample size $n_l={300}^{\star }\left[\frac{l}{3}\right]$, and p = 20 or 50 features in the risk prediction model.

We generated Z_l from a multivariate normal distribution MVN(0, Σ), where Σ is either $\Sigma ={\left[{0.5}^{-\mid \mathrm{j}-{\mathrm{j}}^{\prime }\mid }\right]}_{j=1,\cdots ,p}^{j^{’}=1,\cdots ,p}$ with an autoregressive correlation (AR) structure, or where Σ is a compound symmetry covariance matrix with variance 1 and covariance 0.5. We then generated Z_Q from MVN(0, Σ_Q) with Σ_Q = 0.1 + Σ. Subsequently, we generated T_l and T_Q from:

$$\begin{gathered} 2\log T_{li}=-\log \{0.125(1+0.05l)\}-{\mathbf{b}}_l^T{\mathit{Z}}_{li}+{\in }_{li},l=1,\cdots ,L; \\ 2\log T_{Qi}=-\log 0.225-{\mathbf{b}}_l^T{\mathit{Z}}_{Qi}+{\in }_{Qi}, \end{gathered}$$

where ε_li and ε_Qi were generated from extreme value distributions. We let.

$$b_Q={\left[{\beta }_{8\times 1}^{†},0_{1\times (p-8)}\right]}^T,b_l={\left[{\beta }_{8\times 1}^{†}+e_l^{†},0_{1\times (p-8)}\right]}^T,e_l^{†}=[e_{l1},\dots ,e_{l8}{]}^T$$

and consider a range of scenarios for β^† and {e_l}_l=1,…,L to explore how the signal strength, the heterogeneity among the source sites, as well as the degree of similarity between the target site and the source sites affect the performance of SurvMaximin relative to other methods. Specifically, we consider β^† = [0.5, 0.4, 0.3, 0.2, −0.2, −0.3, −0.4, −0.5]^T and [0.25, 0.2, 0.15, 0.1, −0.1, −0.15, −0.2, −0.25]^T to represent moderate and weak signals. We consider two settings for $\{e_l^{†}\}$ under each with three levels of heterogeneity among source sites for e_l. In setting (I), we let e_lj = τ{1(l ≤ 5) +3I(l > 5)}(−1)^j, which results in the first 5 sites being more similar to the target site than the remaining 10 sites. In setting (II), we let e_lj = τ(l−1) such that a majority of the source sites are substantially different from the target site. We let τ = 0.05, 0.1, and 0.2 to reflect a low, medium and high degree of heterogeneity among the source sites. As τ increases, the target site also becomes more dissimilar to the source sites. We generated censoring time from Exponential (1) distribution, leading to about 20 % to 30 % event rates across the L source sites under setting (I) and 20 % to 40 % under setting (II).

To evaluate the performance of SurvMaximin for missing features, we considered setting (I) with moderate signal, covariance matrix Σ being AR (1), p = 20, and τ = 0.05, 0.1, 0.2. We let the first feature of the target site be missing and calculated the projected SurvMaximin estimator as described in Section 2.3, denoted by SurvMaximin_project. For comparison, we also fitted penalized Cox models to each site with covariates ${\mathit{Z}}^{[𝒜]}$ with $𝒜=\{2,\cdots ,p\}$ to obtain the corresponding effect estimates ${\widehat{\mathbf{b}}}_l^{[𝒜]}$ for ${\mathit{Z}}^{[𝒜]}$; and then constructed SurvMaximin based on ${\left\{{\widehat{\mathbf{b}}}_l^{[𝒜]}\right\}}_{l=1,\cdots ,L}$ . As naïve benchmarks, we additionally constructed the ODAC and Meta models based on Z and transported these models to the target site by removing the component associated with the first covariate. Such naive approaches are often adopted in practice due to the inability to refit the reduced models on source sites.

We also generated censoring time from Exponential (2.5) to consider scenarios with rare events, ranging from 4 % to 10 % for the 15 sites under setting (I) and 3 % to 35 % under setting (II).

We evaluate the overall performance of the estimated risk score from each method in predicting the survival time T_Q for the target site, based on the survival C-statistic with a truncation time close to the largest observed survival time from the target site [33]. We estimate the C-statistics based on an independent validation data of size N_Q = 2,000 generated from the target distributions. For each configuration, we summarize results based on 500 iterations.

2.4.2. Improving Cross-system portability of COVID-19 mortality risk prediction models with SurvMaximin

We further validated the performance of SurvMaximin by deriving robust and transportable mortality risk prediction models for patients hospitalized with COVID-19 using international, multi-institutional EHR data from the 4CE consortium [10,34]. Baseline risk factors and mortality information were available for 83,178 patients from L₀ = 17 participating health care centers of the consortium across three countries: France, Germany, and the U.S. Eligibility criteria for the study included a positive SARS-CoV-2 reverse transcription polymerase chain reaction (PCR) test result; an admission date between March 1, 2020 and January 31, 2021; and the admission occurring 7 days before to 14 days after the date of their first positive PCR test result recorded in their EHR. Each health care center performed analyses locally and then reported summary results to the central institution. We considered each of the individual health care centers as a potential target population and sought to derive a mortality risk prediction model that is transportable to this population from multiple external models. Given the multinational nature of our data, we anticipated a significant amount of between-health care center heterogeneity in their mortality risk models.

Baseline risk predictors considered included: age groups (18–25, 26–49, 50–69, 70–80, 80+), sex, and race (White, Black, Asian, Hispanic, and other); the pre-admission Charlson comorbidity index (CCI) derived from diagnostic codes; and laboratory test values at admission [35]. We focused on 10 commonly measured laboratory tests (with missing rates < 30 %), including C-reactive protein (CRP), albumin, aspartate aminotransferase (AST), AST to alanine aminotransferase ratio (AST/ALT), total bilirubin, creatinine, d-dimer, white blood cell count (WBC), lymphocyte count, and neutrophil count. Values of AST, d-dimer, and CRP were log-transformed due to their skewed distributions. Missing baseline laboratory values and CCI were imputed via the multivariate imputation by chained equation method and averaged over five imputed sets [36]. In total, we considered p = 19 potential risk predictors. A few predictors, including race data for the European centers, were not available (Supplementary Figure S4). When a variable was not ascertained at a site, the local Cox model fitting excluded it. We derived and evaluated prediction models for all-cause mortality by 3, 7, and 14 days after the admission date. We excluded patients who died on the day of admission in the survival analysis.

For each L₀ = 17 health care centers, we transported mortality risk prediction models trained from external analyses via SurvMaximin to the patient population in this center. Specifically, for the lth healthcare center, we fit LASSO penalized Cox models to estimate the effect of Z_l coefficients $\widehat{{\mathit{b}}_l},l=1,\cdots ,L_0$ on survival outcome. For l = 1, …, L₀, we let the l th site be the target site and then trained the SurvMaximin algorithm based on the source data from all remaining sites that have predictors ${\mathit{Z}}^{{𝒜}_l}$ ascertained, where ${\mathit{Z}}^{{𝒜}_l}$ denotes the covariate vector that is available at site l. We used the proposed projection method when the target center had an incomplete set of features. After obtaining the SurvMaximin risk model for the target center, we compared the SurvMaximin risk model against each of the L₀ supervised locally trained models with respect to the accuracy in predicting t₀ = 3, 7, 14-day mortality in the target population. We quantified the accuracy of predicting t₀-day mortality based on the area under the receiver characteristic curve (AUC). We repeated this analysis for all L₀ = 17 centers each time considering one of them as the target population Q.

3. Results

3.1. Results for Simulation studies

Simulation results are summarized in Fig. 2 for the moderate signal scenario. In setting (I), where 5 source sites have feature coefficients like the target site, SurvMaximin results in models with accuracy comparable to those from ODAC and Meta when the heterogeneity is low (τ = 0.05, 0.1) and outperforms other methods when the heterogeneity is high (τ = 0.2). Since there are 5 source sites relatively like the target site, the transported model from SurvMaximin attained accuracy higher than the locally trained model with n_Q = 200 and comparable to those trained with n_Q = 600. When p or the correlation among the features increases, the estimated models generally attain lower prediction performance. Nevertheless, SurvMaximin remains to attain a robust performance relative to the other federated learning methods across different levels of heterogeneity.

Fig. 2.

Average C-statistics under settings (I) and (II) with Σ being either AR(1) or compound symmetry; p = 20or50; and tau = 0.05, 0.10, or 0.20 (local coefficients’ heterogenicity) for predicting survival in the target population with risk models trained by SurvMaximin, Meta, ODAC, as well as supervised penalized Cox regression with n_Q = 200, 400, 600 labeled target data (Local₂₀₀, Local₄₀₀, Local₆₀₀).

In setting (II), where only 1 other site has similar feature coefficients to the target site, SurvMaximin exhibits a substantially better predictive performance when compared to Meta and ODAC across all settings, further highlighting the robustness of SurvMaximin to varying degrees of similarity between the target site and the source sites. Across all levels of heterogeneity, the Meta and ODAC estimators suffer from very small C-statistics indicating poor predictive performance. We observed similar trends regardless of using p = 20, 50 features or covariance matrix structure. Further, the performance of SurvMaximin remains better than the supervised model trained withn_Q = 200 labeled target site data and comparable to the locally trained models with n_Q = 400 and 600. This suggests that SurvMaximin may improve estimation performance when the target population sample size is small.

With weaker signals, the cross-site heterogeneity is more pronounced, leading to more apparent distinctions between SurvMaximin and other federated methods (Figure S1 of the Supplement). Only when the heterogeneity is very low with τ = 0.05 under Setting (I), all methods perform similarly. Under all other settings, SurvMaximin substantially outperforms ODAC and Meta. With weaker signals, locally trained models also require a larger sample size to attain performances comparable to SurvMaximin. This further illustrates the advantage of transporting existing models in a robust fashion over training a supervised model when the training sample size is not large relative to the feature dimension. Results for low event rates are shown in Figure S2, which show similar trends.

Results for assessing the performance of the projected SurvMaximin algorithm in the presence of missing features are summarized in Figure S3. The projected SurvMaximin model attains prediction performance comparable to the SurvMaximin model trained by aggregating the locally fit sub-models with Z. Thus, the projection method provides a comparable alternative SurvMaximin estimator when features may be missing for some sites, without the need to unify the set of features for all the centers all the time. The projected SurvMaximin estimator also outperforms the naïve approach of removing the component associated with the first covariate from the ODAC or Meta estimators.

3.2. Results for transporting COVID mortality risk models

For each covariate, we compared the L₀ local estimates of its log hazard ratio to those based on SurvMaximin in Fig. 3. While these two sets of estimators are generally consistent, SurvMaximin estimators tend to be more concentrated at the center, while local estimators exhibit higher variability in part due to unstable estimates from some sites. For example, the log hazard ratio (HR) of the age group (18–25) ranges from −6.58 to 0 for the local estimates while the SurvMaximin estimates range from −1.43 to −0.7.

Fig. 3.

Density plots of local versus SurvMaximin effect estimators for healthcare systems..l = 1, …, L₀ = 15

The AUC estimates associated with the risk models obtained based on SurvMaximin and local supervised training for predicting 3, 7, and 14-day mortality are shown in Fig. 4. For each site, we also compared the AUC of models trained in each of the external sites, the locally trained model, and the SurvMaximin model for predicting 14-day mortality (Figure S5). The accuracy of the risk models transported by SurvMaximin, which does not utilize the outcome information of the target local site, is comparable or even sometimes higher than that of the locally trained models. The AUCs of SurvMaximin are more concentrated at a comparatively higher AUC, suggesting the robustness of the SurvMaximin approach.

Fig. 4.

Density plots of estimated AUCs for models trained via local estimation versus SurvMaximin for healthcare systems l = 1, …, L₀ = 17.

4. Discussion

We proposed the SurvMaximin approach to deriving a robust risk prediction model for a target population by robustly synthesizing information from estimated risk models from multiple sites. For the target site, the SurvMaximin estimator ${\widehat{\beta }}_{maximin}$ is a linear combination of the coefficient estimators of the local sites $\{{\widehat{\mathit{b}}}_l{\}}_{1\le l\le L}$, or it lies in the convex hull of all the $\{{\widehat{\mathit{b}}}_l{\}}_{1\le l\le L}$ and is closest to a zero point with respect to some distance related to the target population. The method enables us to safely transport a set of existing risk models to a target population in the presence of high cross-site heterogeneity.

Compared with existing federated learning methods, such as Meta and the federated learning methods proposed by [12], the proposed maximin method can handle high-dimensional covariates and is very robust to heterogeneity between sites. It is also robust to sample size differences and improves the inference when the sample size of the target population is small as seen from the simulation studies. The SurvMaximin algorithm is efficient in time and cost as it only requires one-time sharing of the summary statistics. Compared with existing transfer learning methods, the proposed maximin method can help to preserve the privacy and confidentiality of patients in different centers. Further, it requires less information, including only feature information of the target site and feature effect estimators from other sites. Thus, SurvMaximin is very flexible and generalized such that it can adapt to a variety of scenarios while achieving high accuracy with limited information.

5. Conclusion

In this paper, we developed a SurvMaximin covariate effect estimator for multi-center survival data with high-dimensional covariates. Simulation studies and real EHR data analysis show that the proposed estimator achieves high accuracy in a range of settings with different levels of heterogeneity between sites and different sample sizes. The SurvMaximin is a highly flexible and robust approach for multi-center survival analysis, which enables federated learning, transfer learning, as well as federated transfer learning.