Re-calibrating Pure Risk Integrating Individual Data from Two-Phase Studies with External Summary Statistics

Summary:

Accurate risk assessment is critical in clinical decision-making. It entails the projected risk based on a risk prediction model agreeing with the observed risk in the target cohort. However, the model often over- or under-estimates the risk. Building a new model for the target cohort would be ideal but costly. It is therefore of great interest to recalibrate an existing model for the target cohort. Existing methods have been proposed to recalibrate the model by leveraging the disease incidence rates from the target cohort. However, they assume the same covariate distribution across cohorts and when the assumption is violated, the recalibrated model can be substantially biased. Further, recalibration is also complicated by the two-phase sampling design that is commonly used for developing risk prediction models. In this paper, we develop a weighted estimating-equation approach to accounting for the two-phase design and combine it with a weighted empirical likelihood that leverages the summary information on both disease incidence rates and covariates from the target cohort. We provide a resampling-based inference procedure. Our extensive simulation results show that using the summary information from the target population, the proposed recalibration method yields nearly unbiased risk estimates under a wide range of scenarios. An application to a colorectal cancer study also illustrates that the proposed method yields a well-calibrated model in the target cohort.

1. INTRODUCTION

Accurate assessment of the risk of a patient for developing a disease in a future time given their risk profile is critical in clinical decision-making and has been a major focus in chronic disease research (Alba et al., 2017). To assess the accuracy of risk prediction, calibration is often used as an important performance measure to quantify how well the projected risk based on a prediction model developed from a source study agrees with the observed risk in a target cohort. In practice, due to discrepancy of patient risk profiles between cohorts, the source study used for model development does not necessarily reflect well the target cohort to which the model will be applied. As a result, poor calibration occurs frequently when the model is applied directly to the target cohort. For example, Collins and Altman (2012) found the Framingham cardiovascular model developed based on the US population over-estimated the cardiovascular disease risk, especially in men, in several United Kingdom (UK) cohorts. In our real data example of colorectal cancer, there was an over-estimation of risk in the target population when we applied the model developed from the source cohort directly. To circumvent the under- or over- estimation of risk, it would be ideal to develop a new risk prediction model in the target cohort; however, it can be both logistically and cost prohibitive to collect detailed individual-level information on a large number of individuals from the target cohort. Therefore, there is a great interest in recalibrating an existing risk prediction model for the target cohort.

Let T denote the time to disease onset and Z is a p × 1 vector of covariates. Under the Cox proportional hazards model (Cox, 1972), the pure risk of developing the disease before t₁ given being disease-free at t₀ can be expressed as

$$\text{Pr}\left(t_0⩽T<t_1\mid T⩾t_0,Z\right)=1-\text{exp}\left[\left\{{\Lambda }_0\left(t_0\right)-{\Lambda }_0\left(t_1\right)\right\}\text{exp}\left({\beta }_0^{T}Z\right)\right],$$ (1)

where Λ₀(t) is the cumulative baseline hazard function and β₀ are the log-hazard ratios of Z. Relative risks (here, hazard ratios) β₀ are usually similar between the source study and the target population, but Λ₀(t) can be quite different (Keiding and Louis, 2016). This has been noted by clinical reports and general guidelines for medical statistics (Moher et al., 2010; Vedula and Altman, 2010) based on surveys of meta-analyses showing that tests reject the null hypothesis of homogeneity less often for risk ratios than for risk differences, which involve baseline risks. Suitable disease incidence rates from the target population have been used to recalibrate or estimate Λ₀(t) through a time-dependent attributable risk function (Gail et al., 1989; Liu et al., 2014). These approaches require that the joint distribution of all covariates f(Z) be common for the source study and the target population. However, this is generally not the case and violation of this assumption can lead to substantially biased baseline hazard function estimation.

Recalibration of an existing risk prediction is also markedly complicated by how the model is built in the source study. It is common that risk prediction models are built using data collected under the two-phase sampling design (Neyman, 1938). In particular, the nested case-control (NCC) study and case-cohort (CCH) study designs (Thomas, 1977; Prentice and Breslow, 1978; Prentice, 1986) have been proposed as cost-effective alternatives to the standard cohort design. Under the NCC design, the more expensive covariates are measured only on cases and a subset of controls randomly sampled from the risk sets of the cases without replacement. Under the CCH design, these covariates are measured for cases and a randomly selected subcohort. Statistical methods for estimating regression coefficients from NCC/CCH studies have been developed extensively for the Cox proportional hazards model, for example, based on a conditional logistic regression model (Thomas, 1977) or a pseudolikelihood approach (Prentice, 1986). Another popular approach is based on the idea of inverse probability weighting (IPW) (Samuelsen, 1997; Chen and Lo, 1999). This approach has the advantage that it can additionally calculate quantities beyond the regression coefficients, like the baseline hazard and covariate distribution of the full cohort (Cai and Zheng, 2013). By using the information from covariates and auxiliary variables that are measured on the entire cohort, sampling weights can be refined to further improve the efficiency of parameter estimates (Deville and Särndal, 1992; Breslow et al., 2009; Rivera and Lumley, 2016; Shin et al., 2020).

Our work is motivated by the need to recalibrate a colorectal cancer (CRC) risk prediction model built from the Women’s Health Initiative (WHI), one of the largest research cohorts in the US, to the UK Biobank (UKB). In this model, both environmental and genetic risk factors were included. While environmental risk factors were collected on the entire cohort, genetic information was only collected on a subset of CRC cases and disease-free individuals due to cost and logistic constraints. We observed evidence of discrepancy in the disease incidence rates (Figure 1) between WHI and UKB, indicating a potential need to recalibrate Λ₀(t). However, existing methods that assume a common covariate distribution can not meet this need due to different covariate distributions observed between the two cohorts (Table 3).

Figure 1.

Probabilities (point estimate and 95% pointwise CI) of developing colorectal cancer for the WHI and UK Biobank cohorts obtained by the Kaplan-Meier estimator. This figure appears in color in the electronic version of this article, and any mention of color refers to that version.

Table 3.

Descriptive Statistics for WHI and UKB. The Hazard ratio estimates (95% confidence interval [CI]) for CRC are based on the WHI data.

	Prevalence		HR (95% CI)
Risk Factor/Outcome	WHI	UKB	WHI
No. of subjects	76,733	198,058
Endoscopy history	56.2%	36.2%	0.81 (0.68, 0.96)
No. of relatives with CRC (⩾1)	14.7%	11.9%	1.18 (0.94, 1.49)
Vigorous leisure exercise 0–2 hr/wk	14.9%	33.7%	0.99 (0.78, 1.26)
Vigorous leisure exercise >2 hr/wk	12.1%	14.3%	0.88 (0.66, 1.17)
Aspirin/NSAID use (Regular user)	85.0%	24.7%	0.74 (0.58, 0.93)
Vegetable intake (⩾ median portion/day)	53.0%	45.4%	0.96 (0.81, 1.14)
BMI, kg/m2 (⩾30)	22.8%	24.3%	1.38 (1.13, 1.68)
Estrogen status (Positive)	44.9%	8.1%	0.84 (0.70,1.02)
Polygenic risk score, mean (SD)	0 (1)	0 (1)	1.31 (1.20,1.43)
Mean follow-up years	6.7	5.8
CRC pure risk (5-year)	0.7%	0.5%
No. of risk factors for constraints		4

NOTE: Gray context indicates the prevalence difference compared to WHI is greater than 10%.

In this article, we propose a novel method to recalibrate Λ₀(t) and thus project pure risk for the target cohort, utilizing only the summary-level information from the target population together with the individual-level data from the source study under a two-phase study design. We use an inverse-probability weighted estimating equation for Λ₀(t). To accommodate the differential covariate distribution between the source and target cohorts, we propose a novel approach that combines the weighted estimating equation with the empirical likelihood method (Owen, 2001; Qin and Lawless, 1994), reassigning the probability masses for the covariate observations from the source data such that the summary statistics of the covariate distribution is same as that for the target cohort. By solving the re-weighted estimating equation, we obtain an estimator that has comparable performance to the existing methods that use (time-dependent) attributable risk function (Gail et al., 1989; Liu et al., 2014) when the covariate distribution is same for two populations, and reduces the bias considerably when it differs. The robustness comes from the spirit that the reassigned probability masses shift the observed covariate distribution of the source cohort towards that of the target population and therefore make the sample more representative of the target population (Zheng et al., 2021).

The rest of this article is organized as follows. We describe the proposed estimation and inference procedures in Section 2. We evaluate the finite sample performance of the proposed estimator through extensive simulation studies and present the results in Section 3. We illustrate the proposed method by an application to recalibrate a risk prediction model for colorectal cancer developed using the two-phase data from WHI to the target cohort of UKB in Section 4. We provide some concluding remarks in Section 5. Additional technical details and numeric results are provided in the Web Appendices.

2. METHODS

2.1. Notation and Models

Suppose a risk prediction model for T conditional on $Z={\left(Z_a^T,Z_b^T\right)}^T$ is developed based on a two-phase designed study. In the first phase, the outcome and limited characteristics, which may include part of Z (say, Z_a) and auxiliary variables W that can be used for stratification or as surrogates for components of Z, are collected for all participants in a well-defined cohort. In the second phase, components of Z (say, Z_b) that are not collected for all subjects in the first phase are obtained for only a subset of the cohort for the cost-effectiveness purpose. We denote the underlying population for this study by ${\mathcal{P}}^{*}$, and refer Pr* and E* generically as the probability and expectation of random variables with respect to ${\mathcal{P}}^{*}$. The interest is to generalize this risk prediction model to a target population $\mathcal{P}$, with Pr and E denoting the corresponding probability and expectation, respectively. Assume that for $\mathcal{P}$ we only have summary information such as disease probability for the outcome and basic statistics like mean/prevalence for some of the covariates, while for ${\mathcal{P}}^{*}$ individual-level data under the two-phase design are available.

For both populations, we postulate the Cox proportional hazards model (Cox, 1972) for the effect of Z on the failure time T. Specifically, for the source population ${\mathcal{P}}^{*}$,

$${\lambda }^{*}(t\mid Z)={\lambda }_0^{*}(t)\text{exp}\left({\mathit{\beta }}_0^{T}Z\right),$$ (2)

where λ*(t|Z) = lim_dt→0 Pr*(t ⩽ T < t + dt|T ⩾ t, Z)/dt, ${\lambda }_0^{*}(t)$ is an unspecified baseline hazard function for ${\mathcal{P}}^{*}$, and exp(β₀) is a p × 1 vector of hazard ratios. Following the convention, we assume β₀ are common between the two populations, while the distribution of Z is not necessarily the same. For the target population $\mathcal{P}$, we have

$$\lambda (t\mid Z)={\lambda }_0(t)\text{exp}\left({\beta }_0^{T}Z\right),$$ (3)

where λ(t|Z) = lim_dt→0 Pr(t ⩽ T < t + dt|T ⩾ t, Z)/dt and λ₀(t) is an unspecified baseline hazard function for $\mathcal{P}$.

Assume the source cohort for the two-phase study includes n participants. For the ith participant, i = 1, …, n, let T_i, L_i, C_i, W_i, Z_ai and Z_bi be the failure time, left truncation time (e.g., study entry), right censoring time, auxiliary variables, covariates available for the entire cohort, and covariates available only for subjects selected into the second phase, respectively. Without loss of generality, we assume there are no ties in failure time. Define X_i = min(T_i, C_i) if X_i > L_i, and disease status δ_i = I(L_i < T_i ⩽ C_i) where I(·) is an indicator function. Therefore δ_i = 1 if the failure time is observed and 0 otherwise. We assume that both L_i and C_i are independent of T_i conditional on $Z_i={\left(Z_{ai}{}^T,Z_{bi}{}^T\right)}^T$. Let V_i = 1 indicate that the ith participant is selected into the second phase and V_i = 0 otherwise. Therefore the data available for this cohort is $\mathcal{D}=\left\{D_i=\left(L_i,X_i,{\delta }_i,W_i,Z_{ai},V_i,V_i{Z}_{bi}\right),i=1,\dots ,n\right\}$. In practice, to enhance sampling efficiency, the second phase sampling is often based on information (denoted as A) collected at the first phase. Accordingly, the probability of being selected into the second phase Pr*(V_i = 1|A), denoted hereafter as ${\widehat{p}}_i$ for simplicity, is generally known or estimable. Here A may include outcomes and the covariates available for the whole cohort, depending on sampling design. For the NCC design, the sampling is based on A = {(L_i, X_i, δ_i), i = 1, …, n}, and for the CCH design, A = {δ_i, i = 1, …, n} includes only the outcome indicator as the subcohort is selected at the study entry. More details for these two well-known designs are provided in Web Appendix A.1 and A.2.

Define counting process N_i(t) = I(L_i < X_i ⩽ t, δ_i = 1) and at-risk process Y_i(t) = I(L_i < t ⩽ X_i). To account for missing Z under the two-phase design, we follow the IPW principle and weigh the ith subject by ${\pi }_i=V_i/{\widehat{p}}_i$. The maximum pseudo-likelihood estimator for β₀, denoted by $\widehat{\beta }$, is the solution to

$$U(\beta )=\sum _{i=1}^n{\int }_0^{\tau }\left\{{\pi }_i{Z}_i-\frac{{\sum }_{j=1}^n{\pi }_j{Y}_j(t)Z_j\text{exp}\left({\beta }^T{Z}_j\right)}{{\sum }_{j=1}^n{\pi }_j{Y}_j(t)\text{exp}\left({\beta }^T{Z}_j\right)}\right\}d{N}_i(t).$$ (4)

Then, given β, a modified Breslow estimator for ${\Lambda }_0^{*}(t)\equiv {\int }_0^t{\lambda }_0^{*}(u)du$, is

$${\widehat{\Lambda }}_0(t,\beta )=\sum _{i=1}^n{\int }_0^t\frac{d{N}_i(s)}{{\sum }_{j=1}^n{\pi }_j{Y}_j(s)\text{exp}\left({\beta }^T{Z}_j\right)},0⩽t⩽\tau .$$ (5)

For the NCC design, Samuelsen (1997) derived the true sampling weights. Cai and Zheng (2013) established the asymptotic consistency and normality of these estimators, based on which they proposed a resampling procedure to estimate the variances.

For the target population $\mathcal{P}$, the available summary information may include S(t) = Pr(T > t), the overall disease-free probability at time t for a series of time points t₁, …, t_s, and μ₀ ≡ E{h(Z)}, where h(Z) ≡ (h₁(Z), …, h_q(Z))^T is a q × 1 (known) mapping function. For example, a common function h(·) is h(Z) = Z_j, and μ₀ is then the mean of Z_j. Therefore, the available data for recalibration include individual-level data $\mathcal{D}=\left\{D_i=\left(L_i,X_i,{\delta }_i,W_i,Z_{ai},V_i,V_i{Z}_{bi}\right),i=1,\dots ,n\right\}$ from the source cohort and summary information {S(t), t = t₁, …, t_s; μ₀} from the target cohort. The summary information of S(t) and μ₀ can be from different sources of the target population. For example, the incidence rates may be obtained from a disease registry and the summary information of covariates may be obtained from published literature.

2.2. Proposed Estimation Methods

Let Υ(t) be a generic cumulative baseline hazard function and

$$\Phi (Z;{\rm Y}(t),\beta ,S(t))=\text{exp}\left\{-{\rm Y}(t)\text{exp}\left({\beta }^{T}Z\right)\right\}-S(t),$$

for t ∈ [0, τ]. It is easy to see that for the target population E{Φ(Z; Λ₀(t), β₀, S(t))} = 0, where ${\Lambda }_0(t)={\int }_0^t{\lambda }_0(u)du$. Assuming a common distribution of Z between $\mathcal{P}$ and ${\mathcal{P}}^{*}$, we can estimate the expectation by using the weighted covariate distribution estimator based on the two-phase data from ${\mathcal{P}}^{*}$, placing π_i density mass on the ith observed data point. We can recalibrate Λ₀(t), t = t₁, …, t_s, by solving the following estimating equation for Υ(t):

$$\sum _{i=1}^n{\pi }_i\Phi \left(Z_i;{\rm Y}(t),\widehat{\beta },S(t)\right)=\sum _{i=1}^n{\pi }_i\left[\text{exp}\left\{-{\rm Y}(t)\text{exp}\left({\widehat{\beta }}^T{Z}_i\right)\right\}-S(t)\right]=0.$$ (6)

It is worth noting that for recalibrating Λ₀(t) at a specific time t, one only needs S(t) at t. Since $\Phi (Z;{\rm Y}(t),\widehat{\beta },S(t))$ is a continuous and strictly decreasing function of Υ(t) for any fixed $\left\{Z,\widehat{\beta },S(t)\right\}$, an unique non-negative solution to equation (6) exists when S(t) > 0. We denote this solution as ${\widehat{\Lambda }}_0^{\text{c}}(t)$ and call it a constrained estimator. The Newton-Raphson algorithm can be used to solve equation (6). Note that ${\widehat{\Lambda }}_0^{\text{c}}(t)$ is approximately equivalent to the well-known attributable-hazard-function-based estimator for which the attributable hazard is assumed constant across time under the rare disease situation (Gail et al., 1989).

In practice, the assumption of having a same covariate distribution for both the source and target populations is often violated. Thus the observed covariates {Z_i : V_i = 1, i = 1, …, n} with weights of {π_i : V_i = 1, i = 1, …, n} will not be representative of f(Z) in the target population $\mathcal{P}$, and equation (6) would yield biased estimates. To reduce bias, a natural way is to reassign probability mass to the weighted covariate data, aiming at making the data more representative of f(Z) in $\mathcal{P}$. Using the information of μ₀ = E{h(Z)} from $\mathcal{P}$, we employ the empirical likelihood (Owen, 2001; Qin and Lawless, 1994) to obtain the probability masses and reassign them to {Z_i : V_i = 1, i = 1, …, n} in equation (6). Under this framework, we obtain the probability mass for each observation Z_i|V_i = 1 by maximizing the log empirical likelihood ${\sum }_{i=1}^n{V}_i\text{log}\left(w_i\right)$ under the constraints of

$$\sum _{i=1}^n{w}_{i}h\left(Z_i\right)={\mu }_0,w_i>0,\sum _{i=1}^n{w}_i=1.$$ (7)

However, the approach generally yields biased estimators of w_i. This is because under the two-phase study design where cases are over-sampled, the joint distribution of covariates in the subsample is distorted compared to that in the whole cohort. If the constraints were not on all the covariates, equation (7) could lead to a substantially biased estimator. This can be seen from a simple example. Suppose that in both source and target cohorts, there are two independent covariates and both are associated with failure time. Under the NCC or CCH design, these covariates are correlated in the subsample due to oversampling of cases. If we only constrain on one covariate, it will not only inadvertently break the independence between the covariates but also change the distribution of the other covariate, even when the other covariate has the same distribution between the source and target cohorts. In this situation, it can yield a more biased estimator than the estimator without the covariate constraint. To overcome this problem, we instead propose to maximize the weighted log empirical likelihood ${\sum }_{i=1}^n{\pi }_i\text{log}\left(w_i\right)$ with the same constraints as in (7). By this, we maintain the correlation structure of the entire source cohort.

In what follows we describe the estimation procedure. Using the Lagrange multiplier, we have ${\widehat{w}}_i={\pi }_i{\left({\sum }_{i=1}^n{\pi }_i\right)}^{-1}/\left[1+{\widehat{\gamma }}^T\left\{h\left(Z_i\right)-{\mu }_0\right\}\right]$, i = 1, …, n , where $\widehat{\gamma }$, the Lagrange multiplier for the restriction ${\sum }_{i=1}^n{w}_i\left\{h\left(Z_i\right)-{\mu }_0\right\}=0$, is the solution to ${\sum }_{i=1}^n\left[{\pi }_i\left\{h\left(Z_i\right)-{\mu }_0\right\}\right]/\left[1+{\gamma }^T\left\{h\left(Z_i\right)-{\mu }_0\right\}\right]=0$. Standard empirical likelihood programs (e.g., R functions scel.R and scelcount.R to compute empirical likelihood using a self-concordant convex criterion, see Owen (2021)) can be used to obtain $\left\{{\widehat{w}}_i,i=1,\dots ,n\right\}$ and we have ${\widehat{w}}_i=0$ given π_i = 0. By solving the estimating equation with reassigned probability masses for Υ(t), i.e., ${\sum }_{i=1}^n{\widehat{w}}_i\left[\text{exp}\left\{-{\rm Y}(t)\text{exp}\left({\widehat{\beta }}^T{Z}_i\right)\right\}-S(t)\right]=0$, we obtain an estimator of Λ₀(t), denoted by ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ and we call it a doubly constrained estimator. The estimation procedure can be equivalently formulated by solving for (Υ(t), γ, β) the following unified estimating equations that explicitly incorporate the two-phase sampling weights {π_i, i = 1, …, n},

$$\sum _{i=1}^n\left(\begin{array}{c}{\pi }_i\rho \left(Z_i;{\rm Y}(t),\gamma ,{\mu }_0,S(t),\beta \right)\\ {\int }_0^{\tau }\left\{{\pi }_i{Z}_i-\frac{\sum _{j=1}^n{\pi }_j{Y}_j(t)Z_j\text{exp}\left({\beta }^T{Z}_j\right)}{\sum _{j=1}^n{\pi }_j{Y}_j(t)\text{exp}\left({\beta }^T{Z}_j\right)}\right\}d{N}_i(t)\end{array}\right)=0,$$ (8)

where

$$\rho (Z;{\rm Y}(t),\gamma ,\mu ,S(t),\beta )=\left(\begin{array}{c}\frac{\text{exp}\left\{-{\rm Y}(t)\text{exp}\left({\beta }^{T}Z\right)\right\}-S(t)}{1+{\gamma }^T\left\{h(Z)-\mu \right\}}\\ \frac{h(Z)-\mu }{1+{\gamma }^T\left\{h(Z)-\mu \right\}}\end{array}\right).$$

We offer two remarks regarding our estimation procedure. First, generally the more constraints are imposed, the closer is the artificial covariate distribution $\left\{{\widehat{w}}_i,i=1,\dots ,n\right\}$ to the target distribution. However, based on our simulation and real data example as shown in Sections 3 and 4, imposing constraints (e.g., means and variances) on a few covariates that have large differences between the two populations is usually adequate for recalibration. Second, as the true values of S(t) and μ₀ are generally not available, we can replace them by their respective estimates $\widehat{S}(t)$ and $\widehat{\mu }$. Once we obtain ${\widehat{\Lambda }}_0^{\text{dc}}(t)$, we can estimate the pure risk of developing the disease in formula (1) for a new subject with covariates ${\tilde{Z}}_i$ by $\widehat{\text{Pr}}\left(t_0⩽T<t\mid T⩾t_0,{\tilde{Z}}_i\right)=1-\text{exp}\left[\left\{{\widehat{\Lambda }}_0^{\text{dc}}\left(t_0\right)-{\widehat{\Lambda }}_0^{\text{dc}}(t)\right\}\text{exp}\left({\widehat{\beta }}^T{\tilde{Z}}_i\right)\right]$.

2.3. Inference

Based on the estimating equation theory, for a fixed set of time points t = t₁, …, t_k, both ${\widehat{\Lambda }}_0^{\text{c}}(t)$ and ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ converge strongly to the limiting values ${\Lambda }_0^{†}(t)$ and ${\Lambda }_0^{‡}(t)$ (but may not to Λ₀(t)), respectively, and they are asymptotically normal (Web Appendix A.5). Despite the asymptotic normality, it is rather complicated to derive the analytical plug-in variance estimators under the two-phase design. Instead, we consider a perturbation-based resampling method to estimate the variances and pointwise confidence intervals (CI) of ${\widehat{\Lambda }}_0^{\text{c}}(t)$ and ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ at any specific t. Specifically, for each resampling process, we generate the perturbed counterparts of $\left\{{\pi }_i,i=1,\dots n;\widehat{S}(t);\widehat{\mu }\right\}$, denoted by $\left\{{\pi }_i^{†},i=1,\dots n;{\widehat{S}}^{†}(t);{\widehat{\mu }}^{†}\right\}$. If $\widehat{S}(t)$ and $\widehat{\mu }$ are obtained from sufficiently large samples (e.g., population registry) with high precision, we only need to generate the perturbed weights $\left\{{\pi }_i^{†},i=1,\dots n\right\}$. Replacing {π_i, i = 1, …n; S(t); μ₀} by $\left\{{\pi }_i^{†},i=1,\dots n;{\widehat{S}}^{†}(t);{\widehat{\mu }}^{†}\right\}$ in equations (8), we obtain a perturbed estimate ${\widehat{\Lambda }}_0^{{\text{dc}}^{†}}(t)$. Repeat this procedure B times to obtain B realizations of ${\widehat{\Lambda }}_0^{{\text{dc}}^{†}}(t)$, denoted by $\left\{{\widehat{\Lambda }}_0^{{\text{dc}}_b^{†}}(t),b=1,\dots ,B\right\}$. The empirical distribution of $\left\{{\widehat{\Lambda }}_0^{{\text{dc}}_b^{†}}(t)-{\widehat{\Lambda }}_0^{\text{dc}}(t)\right\}$ can then be used to approximate the distribution of $\left\{{\widehat{\Lambda }}_0^{\text{dc}}(t)-{\Lambda }_0^{‡}(t)\right\}$. The variance of ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ can be estimated by perturbation based variance $B^{-1}{\sum }_{b=1}^B{\left({\widehat{\Lambda }}_0^{{\text{dc}}_b^{†}}(t)-{\widehat{\Lambda }}_0^{\text{dc}}(t)\right)}^2$. The 95% CI can be constructed based on this variance estimator or quantiles of the B perturbed values. Similarly, the empirical distribution, variance and CI for ${\widehat{\Lambda }}_0^{\text{c}}(t)$ can be obtained.

The perturbed weights $\left\{{\pi }_i^{†},i=1,\dots n\right\}$ can be generated by the following procedure. First, generate n random realizations of ${\mathcal{I}}_i$ from a known non-negative distribution with $\text{E}\left({\mathcal{I}}_i\right)=1$ and $\text{var}\left({\mathcal{I}}_i\right)=1$ and $\mathcal{I}=\left\{{\mathcal{I}}_i,i=1,\dots ,n\right\}$. The standard exponential distribution was used in all our numerical studies reported later. Then assign the weight ${\mathcal{I}}_i$ to the ith participant (i = 1, …, n), and follow the sampling mechanism of the design to generate the perturbed weights $\left\{{\pi }_i^{†},i=1,\dots n\right\}$. We provide description of steps for generating perturbed weights in detail for both the NCC and CCH designs in Web Appendix A.3 and A.4.

In practice, the variance estimates of $\widehat{S}(t)$ and $\widehat{\mu }$ are usually available or estimable. To generate ${\widehat{S}}^{†}(t)$, one can use logit transformation by first generating a random value $u~N\left(\text{log}\left[\widehat{S}(t)/\left\{1-\widehat{S}(t)\right\}\right],\text{var}(\widehat{S}(t))/{\left\{\widehat{S}(t)(1-\widehat{S}(t))\right\}}^2\right)$, where $\text{var}(\widehat{S}(t))$ is the variance estimate of $\widehat{S}(t)$, and then ${\widehat{S}}^{†}(t)=\text{exp}(u)/(1+\text{exp}(u))$. If a constraint is a mean estimator, say, ${\widehat{\mu }}_k$ that is approximately normal-distributed as generally justified by asymptotic normality, one may obtain one realization from a normal distribution $N\left({\widehat{\mu }}_k,{\widehat{\sigma }}_{{\widehat{\mu }}_k}^2\right)$, where ${\widehat{\sigma }}_{{\widehat{\mu }}_k}^2$ is the variance estimator of ${\widehat{\mu }}_k$. If a constraint is a variance estimator, one may generate realizations from a χ² distribution, as the variance estimator follows the χ² distribution if the samples are from the normal distribution. By this, the variability due to estimation of variance is also incorporated in the inference process. For other scenarios, appropriate distributions can be used to generate perturbed counterparts. If additional information of the correlation (like correlation coefficient) between components of $\left\{\widehat{S}(t),\widehat{\mu }\right\}$ is available, one may generate these perturbed values based on a multivariate distribution.

3. SIMULATION STUDIES

We conducted extensive simulation to evaluate the proposed methods under various scenarios for both NCC and CCH. Specifically, we assumed the Cox proportional hazards models (2) and (3) for the source and target cohorts, respectively. We considered a Weibull distribution for Λ₀(t) = (θt)^ν. For the target population $\mathcal{P}$, we set θ = 0.002 and ν = 2. For the source population ${\mathcal{P}}^{*}$, we considered two situations: (A1) θ = 0.002, ν = 1.5, i.e., ${\Lambda }_0(t)\ne {\Lambda }_0^{*}(t)$; (A2) θ = 0.002, ν = 2, i.e., ${\Lambda }_0(t)={\Lambda }_0^{*}(t)$. Under A1, the true relative difference between ${\Lambda }_0^{*}(t)$ and Λ₀(t) is 400%, 253%, and 189% at t = 20, 40, and 60, respectively.

We generated two covariates Z = (Z₁, Z₂)^T. Let N^T (μ, σ, a, b) denote the truncated normal distribution with mean μ and standard deviation σ and lie within the interval (a, b). For the source cohort ${\mathcal{P}}^{*}$, we generated Z₁ ~ Bernoulli(0.5), Z₂ ~ N^T (0, 0.4, −0.8, 0.8), and Z₁ ⫫ Z₂. For the targeted cohort $\mathcal{P}$, we considered the following three configurations: (C1) Z₁ ~ Bernoulli(0.5), Z₂ ~ N^T (0, 0.4, −0.8, 0.8) and Z₁ ⫫ Z₂; (C2) Z₁ ~ Bernoulli(0.2), Z₂ ~ N^T (0, 0.4, −0.8, 0.8) and Z₁ ⫫ Z₂; (C3) Z₁ ~ Bernoulli(0.2), Z₂|Z₁ = 1 ~ N^T (0, 0.4, −0.8, 0.8) and Z₂|Z₁ = 0 ~ N^T (−0.4, 0.3, −0.8, 0.8). Under C1, f(Z) = f*(Z). Scenarios C2–C3 showed various level of disparity of covariate distribution between the two populations. The corresponding coefficients β₀ = (β₁, β₂) = (log(1.5), log(1.5)). We generated right censoring time C = C*I(1 ⩽ C* ⩽ 100) + I(C* < 1) + 100I(C* > 100), where C* ~ N(40 + 10Z₁, 15). This yielded 96.4% and 98.8% censoring rates for A1 and A2, respectively, representing typically high censoring situation in the NCC/CCH studies. The observed failure time was rounded to the nearest integer to mimic the real-world data. We simulated a cohort sufficiently large to yield ~ 100 cases and all cases were selected into the NCC/CCH data. For NCC, we sampled the controls with a case/control ratio of 1 : 1 following the conventional NCC risk-set sampling procedure. For CCH, we randomly sampled a subcohort such that the total sample size of CCH is same as the NCC sample size. So we can compare the efficiency of both designs given a common sample size. To generate the summary information from the target population, we generated a cohort from the target population to obtain the Kaplan-Meier estimates of disease-free probabilities and means of (Z₁, Z₂), with sample size M = 2, 000 and 100, 000, representing moderate and large sample sizes of the external information resources. A total of 200 perturbation-based resampling samples were used for inference and 2, 000 simulated data sets were generated under each simulation setting.

For the two-phase studies, it is common that some of the covariates in the risk prediction model are available for the entire cohort and utilizing this information from the entire cohort can improve efficiency for model development. Therefore, we considered the following three practical scenarios regarding covariate availability at the first phase: (S1) only Z₁ was available; (S2) only Z₂ was available; (S3) neither Z₁ nor Z₂ was available. In addition, we considered the ideal situation, (S0) both Z₁ and Z₂ were available, to serve as the benchmark for efficiency comparison. Under S3, we calculated the true NCC weights as the sampling weights. We estimated the sampling weights for controls under S1 by logistic regression incorporating Z₁ and log(X) and under S2 by logistic regression incorporating Z₂ and log(X). Since log(X) and Z₂ were both continuous, we used the linear B-splines with 3 internal knots to allow for potential non-linear effects.

We compared four methods: Breslow estimator ${\widehat{\Lambda }}_0^{\text{b}}(t)$, constrained estimator ${\widehat{\Lambda }}_0^{\text{c}}(t)$, doubly constrained estimator ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ with two different constraints on Z: E(Z₁) and {E(Z₁), E(Z₂)} which are denoted by ${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$ and ${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$, respectively. Among the four methods, ${\widehat{\Lambda }}_0^{\text{b}}(t)$ was served as a basic model for comparison, since it represented the conventional solution that no external information about the target cohort was used. We assessed the performance of these estimators by the following metrics: relative bias (PBias), empirical standard deviation (ESD), resampling-based standard error (RSE), square root of mean squared error (sMSE), and probability of 95% pointwise CIs that cover the true value of Λ₀(t) (CP) at selected ages t = 20, 40, and 60. We also calculated the average cumulative absolute deviation (CAD), ${\sum }_{t=1}^{T_{\text{max}}}\left|{\widehat{\Lambda }}_0(t)-{\Lambda }_0(t)\right|$, where T_max = 60 and ${\widehat{\Lambda }}_0(t)$ was an estimator of Λ₀(t) from each of the methods, to assess the overall performance across a wide range of time.

Table 1 summarizes the results for the four covariate availability scenarios S0–S3 under $\text{A}1:{\Lambda }_0^{*}(t)\ne {\Lambda }_0(t)$ and C1: common covariate distribution for the NCC design. The summary information from the target cohort was obtained based on a very large sample size M = 100, 000. The regression coefficient estimators (${\widehat{\beta }}_1$, ${\widehat{\beta }}_2$) under S0–S3 were all nearly unbiased. As expected, (${\widehat{\beta }}_1$, ${\widehat{\beta }}_2$) based on the full-cohort (S0) was most efficient, while S1 that had Z₁ fully observed showed a similar efficiency of ${\widehat{\beta }}_1$ to S0, and S2 that had Z₂ fully observed showed a similar efficiency of ${\widehat{\beta }}_2$ to S0. The scenario S3 with both covariates observed only at the second phase was least efficient for estimating (β₁, β₂). These results indicate that incorporating fully observed covariates in estimating sampling weights improves the estimation efficiency for the regression coefficients of the corresponding covariates. All estimators of Λ₀(t) were almost unbiased, except for the Breslow estimator ${\widehat{\Lambda }}_0^{\text{b}}(t)$, which had poor performance with relative bias ranging from 179% to 398%. Under each of the scenarios S0–S3, both the constrained estimator and the two doubly constrained estimators had very similar variances.

Table 1.

Summary statistics of simulation results based on the NCC data for the scenario of ${\Lambda }_0^{*}(t)\ne {\Lambda }_0(t)$, f*(Z₁, Z₂) = f(Z₁, Z₂) and M = 100, 000.

			(S0) When both Z1 and Z2 are known for the full source cohort				(S3) When neither Z1 nor Z2 is known at the first phase
t	Λ0(t)		${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$	${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$
20	1.7	PBias	387.5%	−1.0%	−0.9%	−0.9%	397.9%	0.6%	−0.1%	−1.0%
		ESD	1.88	0.24	0.25	0.25	2.24	0.32	0.33	0.33
		RSE	1.87	0.25	0.25	0.25	2.26	0.32	0.32	0.32
		sMSE	6.78	0.25	0.25	0.25	7.05	0.32	0.33	0.33
		CP	0.4%	94.6%	94.7%	94.7%	0.3%	93.0%	92.8%	93.5%
40	6.6	PBias	245.1%	−0.6%	−0.6%	−0.5%	252.1%	0.9%	0.3%	−0.7%
		ESD	4.07	0.87	0.88	0.88	5.14	1.19	1.22	1.21
		RSE	4.13	0.88	0.89	0.90	5.27	1.20	1.20	1.19
		sMSE	16.59	0.87	0.88	0.88	17.32	1.19	1.22	1.21
		CP	0.2%	94.4%	94.2%	94.3%	0.5%	93.9%	93.4%	93.2%
60	14.6	PBias	179.3%	−0.9%	−0.9%	−0.8%	185.9%	0.6%	0.0%	−1.0%
		ESD	7.56	1.90	1.91	1.92	9.70	2.62	2.69	2.66
		RSE	7.68	1.93	1.93	1.95	9.94	2.64	2.64	2.62
		sMSE	27.31	1.90	1.92	1.93	28.89	2.62	2.69	2.67
		CP	1.5%	94.5%	94.1%	94.6%	1.6%	93.2%	92.9%	93.6%
		CAD	716.4	32.2	32.5	32.7	738.1	42.8	44.7	44.5
			(S1) When Z1 is known at the first phase, and two-phase sampling weights are estimated by incorporating Z1				(S2) When Z2 is known at the first phase, and two-phase sampling weights are estimated by incorporating Z2
t	Λ0(t)		${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$	${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$
20	1.7	PBias	390.3%	−0.8%	−0.6%	−1.7%	394.8%	0.1%	−0.5%	−0.6%
		ESD	1.92	0.26	0.26	0.26	2.15	0.32	0.33	0.33
		RSE	1.98	0.28	0.28	0.28	2.17	0.33	0.33	0.33
		sMSE	6.84	0.26	0.26	0.26	6.98	0.32	0.33	0.33
		CP	1.4%	94.4%	94.1%	93.9%	1.6%	92.9%	92.2%	92.2%
40	6.6	PBias	246.9%	−0.5%	−0.3%	−1.3%	250.4%	0.4%	−0.2%	−0.3%
		ESD	4.24	0.93	0.93	0.92	5.07	1.19	1.22	1.21
		RSE	4.43	1.03	1.02	1.02	5.09	1.23	1.21	1.21
		sMSE	16.75	0.93	0.93	0.93	17.19	1.19	1.22	1.21
		CP	0.2%	94.2%	94.7%	94.5%	0.8%	93.2%	93.0%	92.8%
60	14.6	PBias	180.5%	−0.8%	−0.6%	−1.6%	183.9%	0.1%	−0.5%	−0.6%
		ESD	7.87	2.04	2.04	2.02	9.70	2.63	2.69	2.68
		RSE	8.28	2.26	2.25	2.25	9.68	2.71	2.65	2.67
		sMSE	27.57	2.04	2.04	2.04	28.62	2.63	2.69	2.68
		CP	1.6%	94.8%	94.4%	94.3%	2.5%	92.9%	92.5%	92.8%
		CAD	721.5	34.2	34.4	34.4	732.4	43.0	44.7	44.5
			(S0)	(S3)	(S1)	(S2)
		$\text{ESD}\left({\widehat{\beta }}_1\right)$	0.21	0.31	0.23	0.31
		$\text{ESD}\left({\widehat{\beta }}_2\right)$	0.29	0.43	0.43	0.31

NOTE: Λ₀(t), true value (×1000); PBias, relative bias; ESD, empirical SD (×1000) of the 2000 estimates; RSE, mean of resampling-based standard errors (×1000); sMSE, square root of mean squared error (×1000); CP, coverage probability of a 95% confidence interval for Λ₀(t); CAD, average cumulative absolute deviation (×1000) between the estimates and true values for t ∈ [1, 60].

Interestingly, the variances of all estimators ${\widehat{\Lambda }}_0(t)$ were similar under S0 and S1, and larger under S2 and S3. That is probably because the censoring variable also depended on Z₁ and incorporating Z₁ in estimating the sampling weights may improve the efficiency not only for ${\widehat{\beta }}_1$ but also ${\widehat{\Lambda }}_0(t)$.

Table 2 shows the results for the three covariate scenarios S1–S3 under A1 ${\Lambda }_0^{*}(t)\ne {\Lambda }_0(t)$ but now the covariate distributions are different (C2 and C3) for the NCC design. As in Table 1, the Breslow estimator ${\widehat{\Lambda }}_0^{\text{b}}(t)$ was substantially biased. The constrained estimator ${\widehat{\Lambda }}_0^{\text{c}}(t)$ that assumed f(Z₁, Z₂) = f*(Z₁, Z₂) was now also significantly biased with relative bias as high as 21%. In contrast, the doubly constrained estimators that imposed constraints on the means of covariates were significantly less biased, especially ${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$, which was almost unbiased with maximum relative bias 1.9%, and the CAD and sMSE were always the lowest. The resampling-based variance estimator was close to the empirical variance estimator and the coverage probability of 95% CIs was close to 95%. As expected, the more constraints were imposed on Z, the less bias was for ${\widehat{\Lambda }}_0^{\text{dc}}(t)$. Further, we note that despite that the correlation of Z₁ and Z₂ was substantially different between the source and target populations under C3, constraining the marginal means of covariates yielded nearly unbiased estimates.

Table 2.

Summary statistics of simulation results based on the NCC data for the two scenarios of ${\Lambda }_0^{*}(t)\ne {\Lambda }_0(t)$, f*(Z₁, Z₂) ≠ f(Z₁, Z₂) and M = 100, 000.

			(C2) f(Z1) ≠ f*(Z1), and f(Z2) = f*(Z2)				(C3) f(Z1) ≠ f*(Z1), and f(Z2) ≠ f*(Z2)
(S3) When neither Z1 nor Z2 is known at the first phase
t	Λ0(t)		${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$	${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$
20	1.7	PBias	398.1%	−11.7%	−1.0%	−1.8%	396.8%	−20.4%	−10.8%	−0.9%
		ESD	2.23	0.28	0.20	0.19	2.23	0.26	0.18	0.26
		RSE	2.26	0.29	0.21	0.20	2.26	0.26	0.19	0.26
		sMSE	7.05	0.34	0.20	0.20	7.03	0.43	0.26	0.27
		CP	0.3%	86.4%	94.2%	93.4%	0.3%	70.9%	81.8%	93.7%
40	6.6	PBias	251.1%	−11.5%	−0.8%	−1.6%	251.9%	−20.1%	−10.4%	−0.6%
		ESD	5.05	1.04	0.67	0.65	5.07	0.93	0.59	0.94
		RSE	5.24	1.05	0.68	0.66	5.26	0.95	0.62	0.92
		sMSE	17.23	1.28	0.67	0.66	17.29	1.62	0.90	0.94
		CP	0.3%	85.6%	94.1%	94.0%	0.7%	70.1%	77.2%	93.8%
60	14.6	PBias	184.7%	−11.7%	−1.0%	−1.9%	186.9%	−20.3%	−10.6%	−0.7%
		ESD	9.49	2.27	1.41	1.37	9.74	2.06	1.27	2.05
		RSE	9.86	2.31	1.46	1.42	9.95	2.09	1.32	2.00
		sMSE	28.66	2.84	1.42	1.40	29.05	3.61	2.00	2.05
		CP	2.1%	85.4%	94.6%	93.9%	2.4%	68.3%	73.2%	93.5%
		CAD	734.7	49.4	24.5	24.1	738.3	65.8	35.1	33.8
(S1) When Z1 is known at the first phase and two-phase sampling weights are estimated by incorporating Z1
t	Λ0(t)		${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$	${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$
20	1.7	PBias	392.9%	−12.7%	−0.8%	−1.7%	391.4%	−21.3%	−10.6%	−0.8%
		ESD	1.97	0.23	0.18	0.17	1.95	0.21	0.16	0.24
		RSE	1.98	0.24	0.19	0.18	1.98	0.23	0.18	0.24
		sMSE	6.89	0.32	0.18	0.17	6.86	0.42	0.24	0.25
		CP	0.8%	83.7%	94.6%	93.7%	0.8%	62.1%	79.6%	94.2%
40	6.6	PBias	247.3%	−12.5%	−0.6%	−1.5%	248.1%	−21.0%	−10.3%	−0.5%
		ESD	4.29	0.85	0.57	0.55	4.29	0.75	0.50	0.85
		RSE	4.42	0.87	0.61	0.58	4.44	0.81	0.57	0.85
		sMSE	16.78	1.18	0.57	0.56	16.83	1.57	0.84	0.85
		CP	0.5%	82.3%	93.9%	94.0%	0.4%	56.6%	72.1%	93.8%
60	14.6	PBias	180.3%	−12.7%	−0.8%	−1.7%	182.6%	−21.2%	−10.5%	−0.6%
		ESD	7.95	1.84	1.19	1.14	8.11	1.65	1.05	1.86
		RSE	8.24	1.91	1.29	1.24	8.34	1.77	1.20	1.84
		sMSE	27.57	2.62	1.20	1.17	27.94	3.51	1.86	1.86
		CP	1.6%	80.9%	94.8%	94.3%	1.8%	54.5%	66.5%	94.0%
		CAD	722.3	45.6	20.8	20.3	726.0	65.4	33.5	30.6
(S2) When Z2 is known at the first phase and two-phase sampling weights are estimated by incorporating Z2
t	Λ0(t)		${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$	${\widehat{\Lambda }}_0^{\text{b}}(t)$	${\widehat{\Lambda }}_0^{\text{c}}(t)$	${\widehat{\Lambda }}_{0,1}^{\text{dc}}(t)$	${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$
20	1.7	PBias	395.5%	−12.2%	−1.5%	−1.5%	394.4%	−20.8%	−11.2%	−0.7%
		ESD	2.17	0.28	0.20	0.19	2.15	0.26	0.18	0.25
		RSE	2.17	0.29	0.20	0.20	2.17	0.26	0.18	0.24
		sMSE	6.99	0.35	0.20	0.20	6.97	0.43	0.26	0.25
		CP	1.1%	84.3%	93.6%	93.5%	1.2%	68.9%	78.2%	93.7%
40	6.6	PBias	249.5%	−12.0%	−1.3%	−1.3%	250.4%	−20.5%	−10.8%	−0.3%
		ESD	5.00	1.05	0.67	0.65	5.00	0.94	0.59	0.85
		RSE	5.05	1.05	0.67	0.65	5.09	0.95	0.60	0.83
		sMSE	17.11	1.31	0.67	0.66	17.17	1.64	0.92	0.85
		CP	0.7%	83.4%	93.1%	93.2%	1.1%	66.7%	70.5%	93.5%
60	14.6	PBias	182.5%	−12.2%	−1.5%	−1.5%	185.1%	−20.7%	−11.0%	−0.5%
		ESD	9.53	2.30	1.41	1.38	9.77	2.07	1.26	1.85
		RSE	9.58	2.31	1.45	1.40	9.70	2.10	1.29	1.80
		sMSE	28.37	2.91	1.43	1.39	28.81	3.67	2.04	1.85
		CP	2.5%	83.0%	93.2%	92.8%	3.5%	65.0%	67.3%	93.8%
		CAD	729.0	50.8	24.5	24.0	733.4	66.9	35.9	30.3
			(S0)	(S3)	(S1)	(S2)	(S0)	(S3)	(S1)	(S2)
		$\text{ESD}\left({\widehat{\beta }}_1\right)$	0.22	0.31	0.23	0.31	0.22	0.31	0.23	0.31
		$\text{ESD}\left({\widehat{\beta }}_2\right)$	0.29	0.44	0.45	0.31	0.29	0.44	0.44	0.32

NOTE: Λ₀(t), true value (×1000); PBias, relative bias; ESD, empirical SD (×1000) of the 2000 estimates; RSE, mean of resampling-based standard errors (×1000); sMSE, square root of mean squared error (×1000); CP, coverage probability of a 95% confidence interval for Λ₀(t); CAD, average cumulative absolute deviation (×1000) between the estimates and true values for t ∈ [1, 60].

We examined the performance of all the estimators for NCC under A2: ${\Lambda }_0(t)={\Lambda }_0^{*}(t)$ when the covariate distributions between the source and target cohorts were the same (Web Table 1) and when they were different (Web Table 2). Under this situation, the Breslow estimator ${\widehat{\Lambda }}_0^{\text{b}}(t)$ was consistent to Λ₀(t) and had little bias regardless of whether the covariate distributions were same or different (C1–C3). The constrained estimator ${\widehat{\Lambda }}_0^{\text{c}}(t)$ had little bias when f(Z₁, Z₂) = f*(Z₁, Z₂), but was substantially biased when f(Z₁, Z₂) ≠ f*(Z₁, Z₂) even when ${\Lambda }_0^{*}(t)={\Lambda }_0(t)$. The doubly constrained estimator ${\widehat{\Lambda }}_{0,2}^{\text{dc}}(t)$ always had little bias.

We also examined the performance when the sample sizes for obtaining the summary information from the target cohort were moderate (M = 2, 000). The results had similar patterns to that with a large target cohort of size M = 100, 000 (Web Table 3–6 under A1–A2 and C1–C3). The doubly constrained estimators reduced the bias substantially when ${\Lambda }_0(t)\ne {\Lambda }_0^{*}(t)$, but may lose some efficiency compared to the Breslow estimator when ${\Lambda }_0(t)={\Lambda }_0^{*}(t)$. Note that for rare diseases, precise estimates of disease incidence rates required a substantially larger sample size than that of covariate summary statistics, where a considerably smaller sample size (e.g., 100) was often sufficient (Web Appendix B.1). Furthermore, we examined the performance of all the estimators for the CCH design, and the results had similar patterns to the NCC design, but the estimators were slightly less efficient for $\left({\widehat{\beta }}_1,{\widehat{\beta }}_2\right)$, ${\widehat{\Lambda }}_0^{\text{c}}(t)$, and ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ (Web Table 7). In practice, covariates may be missing due to causes beyond the sampling design, such as unavailable biospecimens or genotyping failures. Under this situation, our simulation showed the proposed estimator still had satisfactory performance (Web Appendix B.2). In addition, simulation with two continuous covariates showed similar patterns (Web Appendix B.3).

Another issue is that if the components of external summary statistics $\widehat{\mu }$ were estimated from the same target sample, they would be correlated. While the variances of these estimates were often available, the covariance was not and we thus did not take into account the covariance of the estimates in the inference procedure. However, based on our extensive simulation, we observed that the mean of standard error estimates was close to the empirical standard deviation, indicating that ignoring the correlation of external summary statistics did not have meaningful impact on the variance estimation of the proposed recalibrating estimator, even when the external covariate summary information was generated from samples of sizes as low as 100 (Web Table 9).

In a two-phase design, incorporating auxiliary covariate information from Phase I can refine the sampling weights and gain efficiency in the estimation of both hazard ratios and baseline hazard functions for the source cohort. In our simulation, we used the logistic regression model to incorporate the covariate information that is available from the entire source cohort to estimate the two-phase sampling weights and observed an efficiency gain in regression coefficient estimation, as shown by comparing the empirical standard deviations of $\widehat{\beta }$ between different data availability scenarios S0 – S3. With the same aim of gaining efficiency for the source cohort, Shin et al. (2020) recently proposed a weight calibration approach for NCC. We compared our weighting method with theirs under their simulation settings. Interestingly, both methods had a very comparable efficiency (Web Appendix B.4), indicating that in practice the weights estimated by a conventional logistic regression model that incorporates all available sampling-relevant variables could achieve an efficiency close to a design-elaborated weighting method. Therefore, we expect that adopting either weighting approach in our proposed recalibration will have comparable performance in the baseline hazard function recalibration.

4. APPLICATION

4.1. Study Populations and Risk Factors

We illustrate our methods by recalibrating a colorectal cancer (CRC) risk prediction model developed from the WHI to a target cohort assembled from the UKB to demonstrate the practical utility. Specifically, we built a genetic and environmental risk prediction model for CRC using the data from the WHI observational cohort and recalibrated the model by leveraging the overall UKB disease-free probabilities and some summary information of risk factors from UKB.

The WHI observational cohort (Womens Health Initiative Study Group, 1998) is a prospective study including 93,676 women aged 50 to 79 years in the U.S. recruited between 1993 and 1998. The study collected information on socio-demographic and epidemiologic factors using standardized questionnaires and biological measurements at clinic visits. Due to budgetary constraint, only a subset of WHI participants were genotyped following the NCC design. In this analysis, due to the limited numbers of minorities, we focused on only white women for building the risk prediction model. There were a total of 76,733 white women and the mean follow-up time was 6.7 years. Among these, 1,073 developed CRC during the follow-up. All of these women had environmental risk factor information at the baseline; however, only 2,591 including 879 CRC cases had genotyping information due to the NCC design, inadequate specimens or assay failure. The disease outcome was age (in integer years) at the CRC diagnosis, which was subject to left truncation (age at enrollment) and right censoring (last follow-up or death).

The target cohort for prediction, the UKB, is a large long-term biobank study in the United Kingdom. The study is aimed to improve the prevention, diagnosis and treatment of a wide range of illnesses including cancer and heart diseases (Sudlow et al., 2015). It recruited 500,000 people aged between 40–69 years in 2006–2010 from across UK and has been following the health of these participants to date. To be consistent with WHI, we included only white women (198,058 participants) with enrollment age greater than 50. Among these, 1,150 developed CRC during the follow-up. The mean follow-up length is 5.8 years.

The risk prediction model includes both polygenic risk score (PRS) and environmental risk factors. PRS is a weighted sum of 140 known CRC loci identified to date through genome-wide association studies with weights being the effect sizes. It has been shown to improve the prediction of models that included only the environmental risk factors (Jeon et al., 2018). We included the following environmental risk factors (Freedman et al., 2009): history of endoscopy (sigmoidoscopy or colonoscopy) in last 5 years (yes, no); number of first-degree relatives with CRC (0, ⩾ 1); current leisure-time vigorous activity (0, 0–2, > 2 hours per week); use of aspirin and other nonsteroidal anti-inflammatory drugs (NSAIDs) (nonuser, regular user); vegetable consumption (⩾ vs. < medium servings/day); BMI (< 30, ⩾ 30kg/m²); and estrogen status within the last two years (negative, positive). We fit the model starting at age 50 because no women below age 50 were recruited in WHI. For women who enrolled after age 50, they were treated as left truncated at the enrollment age, which can be appropriately handled by defining the at-risk set for each subject as described in Section 2.1.

In practice, due to the complexity of real data, it is often unrealistic to follow exactly the ideal two-phase designs like NCC/CCH such that true sampling weights are obtainable. Alternatively, one can use a working model like logistic regression to estimate the sampling weights. The performance of such a working model was satisfactory in our simulation study that under the exact NCC/CCH sampling scheme when the true sampling weights were obtainable, the performance with estimated weights using the observed data by logistic regression was very comparable to that with the true weights. Therefore, we estimated the sampling weights by logistic regression including all environmental risk factors, age at enrollment, disease status, and age at CRC onset or censoring. For continuous variables, we used the linear B-splines with 3 internal knots to allow for potential non-linear effects.

4.2. Results

Figure 1 shows the probabilities of developing CRC from 50 to 75 years old for the WHI and UKB cohorts. Both cohorts have very similar disease probabilities before 67 years old but the UKB has lower probabilities than WHI after that, indicating a potential poor calibration if we were to apply the WHI model directly to the UKB participants. Table 3 presents the descriptive statistics of environmental risk factors for the entire WHI and UKB cohorts. The PRS were standardized after incorporating the sampling weights from the second phase. The hazard ratio estimates with 95% CIs were calculated from the weighted Cox model based on the WHI second phase data. The prevalences of environmental risk factors were mostly similar between the two cohorts, except for four of them. Fewer UKB participants had endoscopy (WHI 56.2%, UKB 36.2%), used NSAIDS (WHI 85.0%, UKB 24.7%), and had positive estrogen status (WHI 44.9%, UKB 8.1%), but more had moderate exercise (0–2 hours/week) than WHI participants (WHI 14.9%, UKB 33.7%). Generally, having higher PRS, positive family history, and being obese increased the CRC risk, while having had endoscopy, more exercise, Aspirin/NSAIDS use, greater vegetable intake and positive estrogen status reduced the risk.

To investigate the calibration performance, for each UKB participant, we calculated 5-year pure risk of developing CRC from the enrollment age based on the risk prediction model. We stratified UKB participants into ten equally sized groups according to their model-based risk estimates. For each group, we calculated the empirical pure risk by one minus the Kaplan-Meier estimator of disease-free probability and compared it with the average model-based pure risk. When the model-based risk estimates match well to the empirical risk across all risk groups, it indicates the model has good calibration.

Figure 2 shows the comparison of the average model-based and empirical pure risk estimates in each risk group for four models: the WHI model, the model with the constrained estimator ${\widehat{\Lambda }}_0^{\text{c}}(t)$, two models with the doubly constrained estimator ${\widehat{\Lambda }}_0^{\text{dc}}(t)$ (the first having the constraints on the four selected environmental risk factors marked by gray color in Table 3 and the second having the constraints on all environmental risk factors). The WHI model and the constrained estimator performed poorly with the model-based risk estimates far above the upper bounds of the 95% CIs of the empirical risk estimates in most risk groups. For the two doubly constrained estimators, the model-based and empirical risks agreed well across all groups, with O/E 0.95 (95% CI: 0.89–1.01) and 0.98 (95% CI: 0.91–1.04), respectively, where the CIs were obtained by 200 bootstrap samples, and the average (maximum) of absolute differences across risk groups were 0.032% (0.074%) and 0.030% (0.059%), respectively. The 95% CIs of empirical 5-year pure risk estimates covered the average model-based estimates across all risk groups. In addition, the Hosmer-Lemeshow test indicates the two doubly constrained estimators were appropriately calibrated (p-value: 0.59 and 0.81), while the WHI model and the model with ${\widehat{\Lambda }}_0^{\text{c}}(t)$ were miscalibrated (both p-values < 0.001). This highlights the importance of accounting for the potential differences in the risk factor distribution when recalibrating the model, rather than only considering the difference of the overall disease incidence rates.

Figure 2.

Calibration plots for the target UK Biobank cohort: empirical (observed) pure risk stratified by the deciles of model-based predicted pure risk with the dashed 45-degree line indicating perfect calibration. The 95% CIs of empirical pure risk estimates are shown as vertical error bars for each risk group. H-L test, the Hosmer-Lemeshow test.

While we assume common hazard ratios (HR) between populations, we observed some risk factors had inconsistent HRs between WHI and UKB (Web Table 18), e.g., the HR of BMI was 1.38 (95% CI: 1.13, 1.68) in WHI and 0.97 (95% CI: 0.83, 1.13) in UKB. We evaluated the calibration of the relative risk (RR) score, i.e., ${\widehat{\beta }}^{T}Z$ with $\widehat{\beta }$ estimated from the source cohort WHI, on the relative risk in the target UK Biobank cohort. The purpose was to examine how well the calibration was for the overall risk score, instead of individual risk factor HRs. Participants in UK Biobank were divided equally into 7 strata based on risk scores, and the middle stratum that included the 50th percentile of the risk score was set as the reference. We chose 7 strata to ensure a fair number of CRC cases within each stratum for reliable estimation of RR. The predicted or expected RR for a risk score stratum was the ratio of the within-stratum geometric average of individuals model-based RR between that stratum and the reference stratum. The observed RR and its 95% CI for a stratum were derived by fitting a Cox model with a 0–1 stratum indicator as a covariate including only the specific stratum and the reference stratum. We plotted the observed and the expected RRs across risk score strata in Figure 3. The slope fell on 45-degree line and the confidence interval of observed RR covered expected RR for each stratum, indicating the risk score based on the WHI model calibrates well in the UK Biobank.

Figure 3.

Calibration plot of relative risk (RR) score in the UK Biobank with hazard ratio estimates from the WHI

5. DISCUSSION

In the same vein as Zheng et al. (2021) who recalibrated absolute risk when the source individual data were from a cohort study, we proposed an empirical likelihood-based doubly constrained estimating equation approach to recalibrating the risk prediction model developed from a two-phase source study to a target cohort. The summary-level information from the target cohort was utilized in the recalibration and integrated with two-phase sampling weights in the estimating equations. Extensive simulation results and the real data application showed that the proposed two-phase doubly constrained estimator reduced bias substantially compared to existing methods, which can lead to an unacceptably large bias when the baseline hazard function and covariate distribution differ between the source and target cohorts. When the summary information is precise, the proposed estimator also has the potential to gain efficiency. Our proposed doubly constrained estimator provides a practical tool to handle the common situation of two-phase data for model building and recalibration to a target cohort.

In our real data analysis, the target UKB cohort has sufficient individual-level data to build its own prediction model. However, in practice such large cohorts from target populations rarely exist. Therefore, recalibrating a developed risk prediction model with high-level information from the target is a much more feasible and appealing solution than collecting individual-level data. In fact, the proposed recalibration approach does not require summary information from all the covariates. For computational simplicity one may filter out highly correlated constraints and select key constraints on covariates that show large differences in summary statistics between populations and have strong effects. The real data analysis in Section 4 shows that incorporating constraints of just a few key risk factors yields comparable performance to incorporating constraints of all risk factors. Our extensive simulation also shows that simple summary statistics like mean and variance are generally adequate to yield almost unbiased estimator for the baseline hazard function.

We assume common hazard ratios between populations. The transportability of hazard ratios from one cohort to another depends on adequacy of the prediction model, including but not limited to, whether all important confounders and effect modifiers have been appropriately accounted for. In practice, since it is impossible to measure and incorporate all confounders and modifiers in the risk prediction model, a moderate heterogeneity of hazard ratios is expected but may not have meaningful impact when predicting the pure risk for developing a disease. In the real data analysis, we demonstrated this point that the hazard ratios of some risk factors appear to be different between the two cohorts; however, the risk score, importantly, the pure risk seems to be calibrated well.

DATA AVAILABILITY STATEMENT

The Women’s Health Initiative (WHI) data are available with an application to the WHI at http://whi.org/ and the UK Biobank data are also available with an application to the UK Biobank at https://www.ukbiobank.ac.uk/.