Improved risk prediction via cross-domain calibration in a retrospective case-control study

Abstract

Given a well calibrated standard model involving standard predictors, we propose a constrained maximum likelihood approach to incorporate the biomarker sampled from a different population without having to correctly model them to improve the prediction. The constraints ensure that the averaged risk with the biomarker included is as close as that from the standard model. Our method allows different distributions of the standard risk predictors between the target population and the source population of the study data, and relies on the study data to inform the relationship between biomarker and standard predictors. We develop large sample theory of our method for parameters and measures for predictive accuracy, and perform simulation studies to assess the finite sample performance of our method. We apply our method to analyze a case-control study of breast cancer to develop a model that includes both standard breast cancer risk predictors and the biomarker mammographic density.

1. Introduction

Accurate risk prediction is an important research objective. When comprehensive biomarker information is available, incorporating it into predictive models can substantially improve accuracy. However, collecting biomarker data for a large number of patients is often prohibitively expensive, limiting the feasibility of building stable models that use all biomarkers. It is even more challenging in studies of rare diseases, where only case-control samples are available. Such samples do not represent the target population, complicating the model development for risk prediction. Therefore, calibration after the initial model development is essential for creating a practically useful prediction tool. A well-calibrated model not only improves prediction but also supports patients in making risk-based decisions. Calibration is usually carried out using data independent of those used in model development, which integrates information from two different domains. In our setting, the source domain consists of fewer observations but includes biomarker information, while the target domain has more observations but lacks biomarker information. We build an initial model in the source domain first, then calibrate it toward the target domain, with the primary goal of improving the prediction accuracy in the target domain. The well-calibrated model that relies only on the standard predictors in the target domain is referred to as the base model.

Incorporating data from different sources is commonly referred to as domain adaptation (Blitzer et al., 2007; Blanchard et al., 2011), including approaches such as transfer learning (Ben-David et al., 2010) and change of support (Gelfand et al., 2001). These methods aim to improve model performance in one domain by leveraging knowledge from another. Our approach is motivated similarly by integrating external information. However, it differs from traditional domain adaptation in that we also incorporate new predictors, i.e. the biomarkers, in the source domain. This broadens the standard domain adaptation framework.

Specifically, in this paper we develop novel statistical methods for predicting the risk of binary outcomes in case-control studies using logistic regression, while integrating information from multiple data sources. Importantly, we do not require the model defined on the source population to be correct. The standard method for analyzing case-control data is to fit a prospective logistic regression model with an offset term to adjust for case-control sampling (Prentice and Pyke, 1979). In contrast, we develop a novel constrained maximum likelihood method that differs from the standard method in five ways. First, our method ensures that the new model achieves risk prediction performance comparable to that from the base model. To this end, we model both the effect of the standard predictors on the biomarkers, as well as the effect of the standard predictors and the biomarkers on the binary outcome. A retrospective likelihood is employed to account for the case-control sampling structure, following ideas similar to those of Chen et al. (2006). Second, our method explicitly allows for differences between the source population from which the study sample is assembled and the target population. Third, our method accommodates the distribution knowledge of the standard risk predictors in the target population, while leveraging biomarker data from the source population. Fourth, our method is highly statistically efficient, making it suitable for a wide range of sample sizes. Fifth, by incorporating external information, it further mitigates the identifiability issues that commonly arise when estimating the odds ratio (OR) parameters in case-control studies.

Our method for estimating the OR parameters is based on maximizing the conditional retrospective likelihood while incorporating the constraints derived from the base model. Constrained maximum likelihood methods have recently been developed to increase statistical efficiency for estimating OR parameters by exploiting external information through constraints (Qin et al., 2014; Chatterjee et al., 2016). In particular, Qin et al. (2014) increased efficiency by incorporating known outcome prevalence across strata defined by standard predictors, under the assumption that the source population shares the same stratum-specific risks and predictor distributions as the target population. Similarly, Chatterjee et al. (2016) leveraged a known regression relationship between the outcome and standard predictors, assuming the joint distribution of predictors and biomarkers is known. More recently, Zheng et al. (2021) proposed a weighted estimating equation approach that recalibrates risk using a re-weighted sample. Our method differs from these approaches in several important ways. We only require the source domain to contain the biomarker information, without assuming shared distributions between the source and target populations. In terms of the form of the constraints, our method flexibly accommodates both point-wise and interval constraints, as well as functional constraints. On the other hand, in terms of data availability, our framework can incorporate both individual level information (Chatterjee et al., 2016) and summary level information (Zheng et al., 2021). These features make our method highly flexible and broadly applicable.

The rest of this paper is organized as follows. We describe the proposed method in Section 2. The parameter estimation and its asymptotic results are discussed in Section 3. In Section 4, we apply our method to predict the 5-year risk of breast cancer of the Breast Cancer Detection and Demonstration Project (BCDDP) data. In Section 5, we assess the finite sample performance of our method using simulated data.

2. The Constrained Maximum Profile Likelihood Estimator

2.1. Case-Control Modeling

Let $Y$ be the binary outcome of interest ($Y=1$: case; $Y=0$: noncase) and ${\left({\mathbf{\text{X}}}^T,{\mathbf{\text{Z}}}^T\right)}^T$ denote the covariates of interest, where $\mathbf{\text{X}}={\left(X_1,\dots ,X_p\right)}^T$ is the standard risk predictors and $\mathbf{\text{Z}}={\left(Z_1,\dots ,Z_q\right)}^T$ is the candidate predictors, for instance the biomarkers. We consider the situation that data for ${\left({\mathbf{\text{X}}}^T,{\mathbf{\text{Z}}}^T\right)}^T$ is collected from a frequency matched case-control sample, where cases and controls are matched on a random variable $S$ with $M$ strata. Here $S$ is an established predictor, such as age and ethnicity categories. Let $n_{1s}$ and $n_{0s}$ denote the respective number of cases and controls in stratum $s,s\in \{1,\dots ,M\}$. We denote the underlying population for this study by ${𝒫}^{*}$, and refer ${\text{Pr}}^{*}$ and $E^{*}$ as the probability and expectation of random variables with respect to ${𝒫}^{*}$. The superscript “*” here and after stands for “Source”. We have the individual level observed data on $(Y_i,{\mathbf{\text{X}}}_i,{\mathbf{\text{Z}}}_i,S_i)$ for $i\in \{1,\dots ,n\}$ randomly selected subjects from the study source population ${𝒫}^{*}$.

We consider a logistic regression working model to predict the binary outcome $Y$ with ${\left({\mathbf{\text{X}}}^{\top },{\mathbf{\text{Z}}}^{\top },S\right)}^T$ from the source population ${𝒫}^{*}$:

$${\text{Pr}}^{*}\left(Y=1∣\mathbf{\text{X}}=\mathbf{\text{x}},\mathbf{\text{Z}}=\mathbf{\text{z}},S=s\right)\equiv P_{1\mathbf{\text{xz}}s}^{*}=\frac{\text{exp}\left\{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{X}}}^{\top }\mathbf{\text{x}}+{\mathit{\beta }}_{\mathbf{\text{Z}}}^{\top }\mathbf{\text{z}}\right\}}{1+\text{exp}\left\{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{X}}}^{\top }\mathbf{\text{x}}+{\mathit{\beta }}_{\mathbf{\text{Z}}}^{\top }\mathbf{\text{z}}\right\}},$$ (1)

where superscript “$\top$” indicates vector transpose. Here ${\alpha }_s,s\in \{1,\dots ,M\}$ is the intercept parameter corresponding to strata $s$, and $\mathit{\beta }={\left({\mathit{\beta }}_{\mathbf{\text{X}}}^{\mathbf{\top }},{\mathit{\beta }}_{\mathbf{\text{Z}}}^{\mathbf{\top }}\right)}^{\mathbf{\top }}$ are the log odds ratio parameters for ${\left({\mathbf{\text{X}}}^T,{\mathbf{\text{Z}}}^T\right)}^T$. Without external information from target population, we can directly use traditional prospective likelihood method to analyze stratified case-control data (Prentice and Pyke, 1979; Weinberg and Wacholder, 1993; Scott and Wild, 2001; Chen et al., 2008b). But the intercept parameter $\alpha ={\left({\alpha }_1,\dots ,{\alpha }_M\right)}^T$ is unidentifiable. Moreover, nonparametric modeling of ${\text{Pr}}^{*}(\mathbf{\text{Z}}\mid \mathbf{\text{X}},S)$ is practically challenging because of the possible large number of $\mathbf{\text{X}}$ values. We adopt a parametric model for ${\text{Pr}}^{*}(\mathbf{\text{Z}}\mid \mathbf{\text{X}},S)$, denoted as $f_{\tau }(\mathbf{\text{Z}}\mid \mathbf{\text{X}},S)$, with $\tau$ being the index parameter vector. Suppose that $\mathbf{\text{X}}$ and $\mathbf{\text{Z}}$ have $K$ and $L$ unique values observed in the data, ${\mathbf{\text{x}}}_k,k\in \{1,\dots ,K\}$ and ${\mathbf{\text{z}}}_l,l\in \{1,\dots ,L\}$, respectively. The stratum specific probability of covariates is $\mathit{\pi }=\left\{{\pi }_{\mathit{\text{sk}}},s\in \{1,\dots ,M\},k\in \{1,\dots ,K\}\right\}$ with ${\pi }_{\mathit{\text{sk}}}={\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k\mid S=s\right)$.

The empirical loglikelihood function of the case-control data can be derived as

$$\begin{gathered} \ell (\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\pi })=\text{log}\prod _{i=0}^1\prod _{s=1}^M\prod _{k=1}^K\prod _{l=1}^L{\text{Pr}}^{*}{\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\mid Y=i,S=s\right)}^{n_{\mathit{\text{ikls}}}} \\ =\text{log}\prod _{i=0}^1\prod _{s=1}^M\prod _{k=1}^K\prod _{l=1}^L{\left\{\frac{P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)}{{\text{Pr}}^{*}(Y=i\mid S=s)}\right\}}^{n_{\mathit{\text{ikls}}}} \\ =\text{log}\prod _{i=0}^1\prod _{s=1}^M\prod _{k=1}^K\prod _{l=1}^L{\left\{\frac{P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)}{\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)}\right\}}^{n_{\mathit{\text{ikls}}}} \end{gathered}$$ (2)

where $P_{0{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}S}^{*}=1-P_{1{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}S}^{*}$ and $n_{\mathit{\text{ikls}}}$ is the number of observations having $Y=i,\mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l$ within stratum $s$. The likelihood function (2) is derived in detail in Appendix A. We emphasize that a stratified design is not required for our method. Our methodology works well by removing the index $S$ in the likelihood if there is no stratification. For a large number of strata $S$, our methodology can treat it as an ordinal variable equivalent to $\mathbf{\text{X}}$ to simplify the computation if needed.

2.2. Profile Likelihood

Because the number of nuisance parameters in $\mathit{\pi }$ can potentially be large, we derive the profile likelihood for parameters ${\left({\mathit{\alpha }}^T,{\mathit{\beta }}^T,{\mathit{\tau }}^T\right)}^T$ using the method of Lagrange multipliers.

Lemma 1. Define

$${\mu }_{\mathit{\text{is}}}\equiv \frac{n_{\mathit{\text{is}}++}}{n_{+s++}\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathit{x}}_k{\mathit{z}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathit{z}}_l\mid {\mathit{x}}_k,s\right)},$$

for $s\in \{1,\dots ,M\}$. Denote $\mathit{\mu }$ as the vector of $\left\{{\mu }_1,\dots ,{\mu }_M\right\}$, the profile log likelihood $\ell (\mathit{\alpha },\mathit{\beta },\mathit{\tau },\stackrel{\mathit{ˆ}}{\mathit{\pi }}(\mathit{\alpha },\mathit{\beta },\mathit{\tau }\left)\right)$ can be computed as ${\mathit{\text{sup}}}_{\mathit{\mu }}\tilde{\ell }(\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\mu })$, where

$$\tilde{\ell }(\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\mu })=\sum _{s=1}^M\text{log}\prod _{ikl}\widetilde{Pr}\left(Y=i,S=s\mid \mathit{X}={\mathit{x}}_k,\mathit{Z}={\mathit{z}}_l\right),$$

and

$$\widetilde{Pr}\left(Y=i,S=s\mid \mathit{X}={\mathit{x}}_k,\mathit{Z}={\mathit{z}}_l\right)=\frac{n_{+\mathit{\text{sk}}+}{P}_{i{\mathit{x}}_k{\mathit{z}}_{l}s}^{*}{f}_{\tau }\left({\mathit{z}}_l\mid {\mathit{x}}_k,s\right){\mu }_{\mathit{\text{is}}}}{n_{\mathit{\text{is}}++}\sum _{i=0}^1{\mu }_{\mathit{\text{is}}}\sum _{l=1}^L{P}_{i{\mathit{x}}_k{\mathit{z}}_{l}s}^{*}{f}_{\tau }\left({\mathit{z}}_l\mid {\mathit{x}}_k,s\right)}.$$ (3)

According to the definition, ${\mu }_s=\text{log}\left({\mu }_{1s}/{\mu }_{0s}\right)=\text{log}\left\{\left(n_{1s++}\text{pr}(Y=0\mid s)\right)/\left(n_{0s++}\text{pr}(Y=1\mid s)\right)\right\}$ adjusts the bias of computing the population prevalence from the unrepresentative case-control sampling. This greatly simplifies the computation because $\mathit{\mu }$ has dimension $M$, much smaller than the dimension of $\mathit{\pi }$, which is $MK$, in practice. We provide the detailed calculations of the above finding in Appendix B. In the above development, we note that constraint (6) did not contribute to the derivation because it does not involve ${\pi }_{\mathit{\text{sk}}}$.

It is well known that the intercept parameter $\mathit{\alpha }$ is unidentifiable from stratified case-control data (Prentice and Pyke, 1979; Cao et al., 2024). As noted as Chatterjee and Carroll (2005) and Chen et al. (2008b), the parameter $\mathit{\alpha }$ is separable from $\mathit{\mu }$ via the retrospective likelihood (2) or (3) and hence is theoretically identifiable. The external knowledge in the constraints further assures the practical identifiability of $\mathit{\alpha }$. The detailed discussion is provided in the Appendix.

We need to point it out that the profile probability $\widetilde{\text{Pr}}\left(Y=i,S=s\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\right)$ can be viewed as a new pmf of $Y,Z\mid \mathbf{\text{X}},s$ but not from model (1). Meanwhile, $\widetilde{\text{Pr}}\left(Y=i,S=s\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\right)$ depends on $\mathit{\mu }$ only through the free parameters ${\mu }_s,\in \{1,\dots ,M\}$. The above treatment of free parameter switching, together with the incorporation of the parametric relationship between $\mathbf{\text{Z}}$ and $(\mathbf{\text{X}},S)$ without complicating the working model (2), defines the novelty of our constrained profile likelihood function.

2.3. Constraints Design

We transport a prediction model incorporating $\mathbf{\text{X}}$ and $\mathbf{\text{Z}}$ in the target population $𝒫$, with Pr and $E$ denoting the corresponding probability and expectation. We particularly allow different characteristics in terms of the source population ${𝒫}^{*}$ and the target population $𝒫$, that may have differences not only in the distribution of individual predictor variables, but also in the outcome rate and predictor outcome association (Chi et al., 2019). Assume that we have a practical risk prediction model based on the risk predictors $\mathbf{\text{X}},S$, denoted as $\phi (\mathbf{\text{X}},S)$, and it is well calibrated in the target population. The stratum specific distribution of $\mathbf{\text{X}}$ in the target population of prediction, ${\delta }_S\left(\mathbf{\text{X}}\right)=\text{Pr}(\mathbf{\text{X}}=\mathbf{\text{x}}\mid S=s)$, is known from external sources. Distinct ${\delta }_S\left(\mathbf{\text{x}}\right)$ and ${\pi }_{\mathit{\text{sk}}}$ is known as “covariate shift” in domain adaptation work (Li et al., 2021). Despite the presence of between-population heterogeneity, it is reasonable to believe that the population specific models share some degree of similarity.

In order to calibrate the aforementioned case-control OR parameters, constraints are imposed on the source risks via external knowledge. We consider the calibration in assessing the external validity, that is, the agreement between the predicted risk and observed outcomes of the target population (Pfeiffer and Gail, 2017). For each stratum $s$, we categorize the predicted risk $\phi (\mathbf{\text{X}},S)$ into $I_S$ intervals within which the left and right end points are denoted by the $a_{\mathit{\text{sr}}}$ and $b_{\mathit{\text{sr}}}$ for $r\in \left\{1,\dots ,I_s\right\}$, respectively. Thus, the averaged predicted risk within each risk interval is

$$P_{\mathit{\text{sr}}}^e\equiv P^e\left(Y=1\mid a_{\mathit{\text{sr}}}\le \phi \left(\mathbf{\text{x}},s\right)\le b_{\mathit{\text{sr}}}\right),s\in \left\{1,\dots ,M\right\},r\in \left\{1,\dots ,I_s\right\},$$ (4)

in which “e” stands for the computation being based on the target knowledge externally. When tailoring the new model (1) to the target population, we hope to ensure a superb calibration performance in each stratum defined by ${\left({\mathbf{\text{X}}}^{\top },S\right)}^{\top }$. To this end, we impose the equality of these averaged risks computed based on (1) and that based on $\phi (\mathbf{\text{X}},S)$, i.e. the resultant constraints are

$$\frac{\sum _{\mathbf{\text{x}}:a_{\mathit{\text{sr}}}\le \phi (s,\mathbf{\text{x}})\le b_{\mathit{\text{sr}}}}\sum _{\mathbf{\text{z}}}{P}_{1{\mathbf{\text{xz}}}_{l}s}^{*}{\delta }_s\left(\mathbf{\text{x}}\right)f_{\tau }(\mathbf{\text{z}}\mid \mathbf{\text{x}},s)}{\sum _{\mathbf{\text{x}}:a_{\mathit{\text{sr}}}\le \phi (s,\mathbf{\text{x}})\le b_{\mathit{\text{sr}}}}{\delta }_s\left(\mathbf{\text{x}}\right)}=P_{\mathit{\text{sr}}}^e$$ (5)

for $s\in \{1,\dots ,M\},r\in \left\{1,\dots ,I_s\right\}$. In practice, equality constraints might be too stringent to apply, especially when the computation of $P^e\left(Y=1\mid a_{\mathit{\text{sr}}}\le \phi (s,\mathbf{\text{x}})\le b_{\mathit{\text{sr}}}\right)$ itself may contain errors due to various issues associated with the external information. Thus we allow the equality constraints to be relaxed to the inequality constraints

$$\left|\frac{\sum _{\mathbf{\text{x}}:a_{\mathit{\text{sr}}}\le \phi (\mathbf{\text{x}},s)\le b_{\mathit{\text{sr}}}}\sum _{\mathbf{\text{z}}}{P}_{1\times {\mathbf{\text{z}}}_{l}s}^{*}{\delta }_s\left(\mathbf{\text{x}}\right)f_{\tau }(\mathbf{\text{z}}\mid s,\mathbf{\text{x}})}{\sum _{\mathbf{\text{x}}:a_{\mathit{\text{sr}}}\le \phi (s,\mathbf{\text{x}})\le b_{\mathit{\text{sr}}}}{\delta }_s\left(\mathbf{\text{x}}\right)}-P_{\mathit{\text{sr}}}^e\right|\le d_{\mathit{\text{sr}}}{P}_{\mathit{\text{sr}}}^e$$ (6)

for $s\in \{1,\dots ,M\},r\in \left\{1,\dots ,I_s\right\}$. Here, $d_{\mathit{\text{sr}}}$ is a buffer coefficient that controls the degree to which we rely on the external risk $P^e\left(Y=1\mid a_{\mathit{\text{sr}}}\le \phi \left({\mathbf{\text{x}}}_k,s\right)\le b_{\mathit{\text{sr}}}\right)$. A smaller $d_{\mathit{\text{sr}}}$ implies more trust in the external information. Obviously the inequality constraint (6) reduces to the equality constraint (5) when $d_{\mathit{\text{sr}}}=0$.

2.4. Constrained Maximum Profile Likelihood Estimator

Building upon the previous discussions, we introduce the constrained maximum profile likelihood estimator via solving the optimization problem

$$\begin{gathered} \underset{\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\mu }}{\text{max}}{\ell }^{*}(\mathit{\alpha },\mathit{\beta },\tau ,\mathit{\mu })=\sum _{i,k,l,s}{n}_{\mathit{\text{ikls}}}\text{log}\left\{\frac{n_{+\mathit{\text{sk}}+}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right){\mu }_{\mathit{\text{is}}}}{n_{\mathit{\text{is}}++}\sum _{i=0}^1{\mu }_{\mathit{\text{is}}}\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)}\right\} \\ \text{subject}\text{to}{C}_{\mathit{\text{sr}}}=0,s\in \left\{1,\dots ,M\right\},r\in \left\{1,\dots ,I_s\right\}, \end{gathered}$$ (7)

where $C_{\mathit{\text{sr}}}$ is the equality constraint given in (5) that

$$C_{\mathit{\text{sr}}}\equiv \frac{\sum _{\mathbf{\text{x}}:a_{\mathit{\text{sr}}}\le \phi \left(s,\mathbf{\text{x}}\right)\le b_{\mathit{\text{sr}}}}\sum _{\mathbf{\text{z}}}{P}_{1\mathbf{\text{xz}}l}^{*}{\delta }_s\left(\mathbf{\text{x}}\right)f_{\tau }\left(\mathbf{\text{z}}∣\mathbf{\text{x}},s\right)}{\sum _{\mathbf{\text{x}}:a_{\mathit{\text{sr}}}\le \phi \left(s,\mathbf{\text{x}}\right)\le b_{\mathit{\text{sr}}}}{\delta }_s\left(\mathbf{\text{x}}\right)}-P_{\mathit{\text{sr}}}^e.$$

It is equivalent to maximizing (2) subject to the constraints in terms of estimating $(\mathit{\alpha },\mathit{\beta },\mathit{\tau })$.

For inequality constraints given in (6), slack parameters $\mathit{\xi }=\left[{\xi }_{\mathit{\text{sr}}}^{+},{\xi }_{\mathit{\text{sr}}}^{-}:s\in \{1,\dots ,M\},r\in \left\{1,\dots ,I_s\right\}\right]$ are introduced (Luenberger et al., 1984; Boyd et al., 2004) and the constraints are denoted by

$$C_{\mathit{\text{sr}}}^{+}\equiv \frac{\sum _{k:a_{\mathit{\text{sr}}}\le \phi \left(s,{\mathbf{\text{x}}}_k\right)\le b_{\mathit{\text{sr}}}}\sum _{l=1}^L{P}_{1{\mathbf{\text{xz}}}_{l}s}^{*}{\delta }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}{\sum _{k:a_{\mathit{\text{sr}}}\le \phi \left(s,{\mathbf{\text{x}}}_k\right)\le b_{\mathit{\text{sr}}}}{\delta }_{\mathit{\text{sk}}}}-\left(1+d\right)P_{\mathit{\text{sr}}}^e+{\xi }_{\mathit{\text{sr}}}^{{+}^2},$$

and

$$C_{\mathit{\text{sr}}}^{-}\equiv \left(1-d\right)P_{\mathit{\text{sr}}}^e-\frac{\sum _{k:a_{\mathit{\text{sr}}}\le \phi \left(s,{\mathbf{\text{x}}}_k\right)\le b_{\mathit{\text{sr}}}}\sum _{l=1}^L{P}_{1\mathbf{\text{x}}{\mathbf{\text{z}}}_{l}s}^{*}{\delta }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}{\sum _{k:a_{\mathit{\text{sr}}}\le \phi \left(s,{\mathbf{\text{x}}}_k\right)\le b_{\mathit{\text{sr}}}}{\delta }_{\mathit{\text{sk}}}}+{\xi }_{\mathit{\text{sr}}}^{-2}.$$

Therefore the inequality constraints (6) can be replaced by the equality constraints. For notational simplicity, the constraints are denoted by $C_{\mathit{\text{sr}}}$ for both situations.

Hence, we propose a very flexible model that imposes the external validity to calibrate the OR parameter estimation further to improve the risk prediction.

3. Estimation and Asymptotic Result

3.1. Augmented Lagrangian Multiplier Method

In model (7), the imposed constraints can be replaced by other meaningful criteria in the given intervals, and the calibration framework is still reasonable. In order to overcome the possible loss of the convexity property due to the very flexible selection of the constraints, we apply augmented Lagrangian multiplier algorithm to (7) via solving

$$\underset{\mathit{\theta }}{\text{max}}\sum _{i,k,l,s}{n}_{\mathit{\text{ikls}}}\text{log}\left\{\frac{n_{+\mathit{\text{sk}}+}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right){\mu }_{\mathit{\text{is}}}}{n_{\mathit{\text{is}}++}\sum _{i=0}^1{\mu }_{\mathit{\text{is}}}\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}\right\}+{\mathit{\lambda }}^{\top }\mathbf{\text{C}}\left(\mathit{\theta }\right)-\frac{\sigma }{2}\Vert \mathbf{\text{C}}\left(\mathit{\theta }\right){\Vert }^2,$$ (8)

where $\mathit{\theta }$ is the collection of parameters $\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\mu }$, the Lagrangian multipliers $\mathit{\lambda }=\left[{\lambda }_{\mathit{\text{sr}}},s\in \{1,\dots ,M\},r\in \right.\left.\left\{1,\dots ,I_s\right\}\right]$, and the slack parameters $\mathit{\xi }$ if the inequality constraints are considered. The addition of the quadratic penalty term $-\sigma \Vert \mathbf{\text{C}}\left(\mathit{\theta }\right){\Vert }^2/2$ guarantees the local convexity when $\sigma$ is sufficiently large (Nocedal and Wright, 1999). For simplicity, we assume when $n\to \infty$, the constrained maximum profile likelihood in (7) has a unique solution.

3.2. Large Sample Property of the Optimization Solution

Theorem 1. The estimator $\stackrel{\mathit{ˆ}}{\mathit{\theta }}$ obtained by maximizing (8) converges to the “true parameter” ${\mathit{\theta }}_0={\left({\mathit{\alpha }}_0^{\top },{\mathit{\beta }}_0^{\top },{\mathit{\tau }}_0^{\top },{\mathit{\mu }}_0^{\top }\right)}^{\top }$, where ${\mathit{\theta }}_0$ is the unique solution of the constrained optimization problem (7), if the KKT condition and linear independence constraint qualification hold.

The linear independence constraint qualification requires the linear independence of the gradients of the active constraints, for example, $\nabla \mathbf{\text{C}}\left(\mathit{\theta }\right)$ has full rank when $\mathbf{\text{C}}\left(\mathit{\theta }\right)$ is active. The proof of Theorem 1 is a direct corollary to Theorem 17.5 in Nocedal and Wright (1999).

The augmented Lagrangian multiplier method implemented in our study is one of the most popular methods for addressing nonlinear constrained optimization problems. Many other strategies for nonlinear optimization problems with nonlinear constraints are used in practice. For example, Rosen (1960) and Fletcher (2013) approximate nonlinear constraints with linear ones when the constraints slightly deviate from linearity. For severely nonlinear problems, sequential quadratic programming shows its advantages (Nocedal and Wright, 1999). Nonsmooth optimization was proposed (Fletcher, 2013) and was further studied by Lewis and Overton (2013) for special nonsmooth constraints. Many other strategies exist to handle such optimization problems and we refer the readers to Ruszczynski (2011); Fletcher (2013) and Gill et al. (2019).

We name the set of the $\mathit{\theta }$ values that satisfy the constraints in (5) as the feasible set. If the maximizer to the loglikelihood in (2) belongs to the feasible set, then the constraints in (5) are not effective. However, as soon as the solution falls out of the feasible set, the constraints are effective and the target population calibrates the solution. The active constraints $C\left(\mathit{\theta }\right)$ can be recognized by nonzero Lagrange multipliers. According to the KKT condition, when the former situation happens, the constraints imposed in (8) may still affect the inference results even though it does not change the point estimation. When the latter situation happens, both estimation and inference are affected. We provide the following theorem on the asymptotic properties of the optimizer of (8).

Theorem 2. Let $\mathit{C}\left(\mathit{\theta }\right)$ be the constraints defined in Section 2.4 and $\mathit{U}\left(\mathit{\theta }\right):𝚯\mapsto {\mathit{R}}^{\left|\mathit{\theta }\right|\times \left|C\right|}$ be a function that satisfies ${\mathit{U}}^{\top }\left(\mathit{\theta }\right)\partial \mathit{C}(\mathit{\theta }{)}^{\top }/\partial \mathit{\theta }=0$ and ${\mathit{U}}^{\top }\left(\mathit{\theta }\right)\mathit{U}\left(\mathit{\theta }\right)=\mathit{I}$. We also assume $n_{\mathit{\text{is}}}/n\to r_{\mathit{\text{is}}}$ as $n\to \infty$, where $r_{\mathit{\text{is}}},i=\mathrm{0,1},s\in \{1,\dots ,M\}$ are constants in [0, 1]. The variance-covariance matrix of the estimator converges to

$$\mathit{V}\left({\mathit{\theta }}_0\right)\left\{𝒥\left({\mathit{\theta }}_0\right)-\Omega \left({\mathit{\theta }}_0\right)\right\}{\mathit{V}}^{\top }\left({\mathit{\theta }}_0\right),$$

where ${\mathit{\theta }}_0$ is defined in Theorem 1, $\mathit{V}\left(\mathit{\theta }\right)=\mathit{U}\left(\mathit{\theta }\right){\left[{\mathit{U}}^{\top }\left(\mathit{\theta }\right)𝒥\left(\mathit{\theta }\right)\mathit{U}\left(\mathit{\theta }\right)\right]}^{-1}{\mathit{U}}^{\top }\left(\mathit{\theta }\right)$,

$$\begin{gathered} 𝒥\left(\mathit{\theta }\right)=\sum _s{r}_{+s}E\left[\left.{\mathit{\text{var}}}_{Y,Z}^{*}\left\{\left.\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })}{\partial \mathit{\theta }}\right|\mathit{X}\right\}h(\mathit{X},s)\right|s\right], \\ \Omega \left(\mathit{\theta }\right)=\sum _{\mathit{\text{is}}}{r}_{\mathit{\text{is}}}E{\left[\left.E\left\{\left.\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })}{\partial \mathit{\theta }}\right|X\right\}\right|Y=i,s\right]}^{\otimes 2}, \\ {P}_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })=\frac{P_{i{\mathit{x}}_k{\mathit{z}}_{l}s}^{*}{f}_{\tau }\left({\mathit{z}}_l\mid {\mathit{x}}_k,s\right){\mu }_{\mathit{\text{is}}}}{h\left({\mathit{x}}_k,s\right)}, \end{gathered}$$

and $h\left({\mathit{x}}_k,s\right)=\sum _{\mathit{\text{il}}}{P}_{i{\mathit{x}}_k{\mathit{z}}_l}^{*}{\mu }_{\mathit{\text{is}}}{f}_{\tau }\left({\mathit{z}}_l\mid {\mathit{x}}_k,s\right)$.

3.3. Measures of Prediction Accuracy for Model (1) in the Target Population

We now describe the criteria in Pfeiffer and Gail (2017) for assessing the risk prediction accuracy in the target population. Let ${\left\{y_i,{\mathbf{\text{x}}}_i,s_i\right\}}_{i=1}^m$ be a cross-sectional sample generated from the well calibrated external model $\phi (\mathbf{\text{x}},s)$. Let $E$ be the expected number of cases associated with an interval $[a,b]$, given by

$$E=P^e\{Y=1\mid a\le \phi (\mathbf{\text{x}},s)\le b\}\sum _{i=1}^{m}I\left\{a\le \phi \left({\mathbf{\text{x}}}_i,s_i\right)\le b\right\},$$

where $I$ is the indicator function. Various prediction methods will lead to various approximations of $P^e\{Y=1\mid a\le \phi (s,\mathbf{\text{x}})\le b\}$. Based on the solution to (7) and (8), we predict the number of cases as

$$\widehat{E}=\frac{\sum _{k:a_{\mathit{\text{sr}}}\le \phi \left({\mathbf{\text{x}}}_k,s\right)\le b_{\mathit{\text{sr}}}}\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\delta }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)}{\sum _{k:a_{\mathit{\text{sr}}}\le \phi \left({\mathbf{\text{x}}}_k,s\right)\le b_{\mathit{\text{sr}}}}{\delta }_{\mathit{\text{sk}}}}\sum _{i=1}^{m}I\left\{a\le \phi \left({\mathbf{\text{x}}}_i,s_i\right)\le b\right\}.$$

Obviously, the difference between $\widehat{E}$ and $E$ measures the accuracy of the prediction, a smaller value implies a better prediction.

4. Analysis of Data from the Breast Cancer Detection and Demonstration Project (BCDDP)

We apply our proposed method to analyze the BCDDP data with calibration from the Breast Cancer Risk Assessment Tool (BCRAT). The purpose is to develop a logistic regression model for predicting the five-year risk of breast cancer in the U.S. White European women. The BCRAT, in which the participants were recruited from women who volunteered to undergo mammographic screening, has been widely used for assessing breast cancer risk for individual White European women in the United States. There are two major differences between the two sources. The target domain from BCRAT is well calibrated but impossible to obtain the biomarker, i.e. percent mammographic density (PD) in our study, anymore. The source data are small and from an ongoing study that evaluates the predictiveness of volumetric breast density, in which only cases and controls were recruited from the University of Pennsylvania health system. Although PD was incorporated as a strong risk predictor into the BCRAT, the model with which has yet to be validated after more than 10 years since the updated model was published (Chen et al., 2006). Hence it is meaningful to detect the predicted risks of the risk distribution after the validation.

The BCRAT predictors include age at menarche (Agemen), age at first live birth (Ageflb), number of previous breast biopsies (Nbiops), and number of first degree relatives (mother/sisters; Numrel) who had breast cancer. PD has been established as one of the strongest risk predictors for breast cancer (Byrne et al., 1995; Boyd et al., 1995). The OR function used in the BCRAT includes 3,154 cases and 3,506 controls. A subset of 1,386 cases and 1,744 controls have PD measurements. The details of this study were reported previously (Chen et al., 2006, 2008a), where it was shown that the availability of PD depends only on the case-control status and age strata. Here, we analyze data for the 3,130 (1,386+1,744) subjects who had complete predictor data, as inclusion of subjects who did not have PD data requires nontrivial extension of our method. Because weight confounds the relationship between breast cancer risk and PD, we also include weight as a predictor as in the previous work (Chen et al., 2006, 2008a). The codings of BCRAT predictors and weight are the same as those in Chen et al. (2006) and are fitted as ordinal variables. In the model, we consider two age strata defined as $S=1$ if age ≤ 50 and $S=2$ otherwise, and finer categorization can be adopted in the same manner. In our study of BCDDP data, 50 years old is chosen as the cutoff because many researchers (Seidman et al., 1987; Byrne et al., 1994; Smart et al., 1997) have noticed the differences for women younger versus older than 50 years in terms of survival rates, modalities, or benefits gained from the breast cancer screening. We fit model (1) with $\mathbf{\text{X}}=(\text{Ageflb},\text{Agemen},\text{Nbiops},\text{Numrel},\text{Weight}{)}^{\top }$ and $Z=\text{PD}$. In the data, around 10% of the subjects have zero values for PD, which, we believe, reflect both truly no dense tissues and PD values that are below the detection limit. Therefore, we use a zero-inflated Beta regression model for the distribution of $Z$ to accommodate excess zero values (Chen and Chen, 2015). Specifically, we set

$$\text{Beta}(Z\mid \mathbf{\text{X}},S;\mathit{\gamma },\mathit{\omega })=\text{Beta}(Z\mid \kappa ,\varphi ),\text{where}\text{logit}\left(\kappa \right)=\left(1,S,{\mathbf{\text{X}}}^{\top }\right)\mathit{\gamma },\text{log}\left(\varphi \right)=\left(1,S,{\mathbf{\text{X}}}^{\top }\right)\mathit{\omega },$$

where $\kappa$ is the mean parameter and $\varphi$ is the precision parameter of the beta distribution. Let $\rho$ denote the probability that $Z$ is truly zero and let $Z_{\text{min}}$ the minimum detectable value of PD. The probability density function of $Z$ can then be written as follows:

$$\begin{gathered} P(Z\mid \mathbf{\text{X}},S;\mathit{\gamma },\mathit{\omega },\rho )=p(Z=0{)}^{I\{Z=0\}}p(Z>0{)}^{I\{Z>0\}} \\ {=\left[\rho +(1-\rho )p\left(Z<Z_{\mathit{\text{min}}}\mid \mathbf{\text{X}},S;\mathit{\gamma },\omega \right)\right]}^{I\{Z=0\}}\times \left[\right(1-\rho \left)\text{Beta}\right(Z\mid \mathbf{\text{X}},S;\mathit{\gamma },\mathit{\omega }){]}^{I\{Z>0\}}. \end{gathered}$$

We used a categorized version of $Z$, denoted by $Z^c$, to fit model (1), where $Z^c$ took integer values 0 to 9, corresponding to $Z=0$ or $Z\in \left(0,Z_{\text{min}}\right),\left[Z_{\text{min}},0.1\right),\left[\mathrm{0.1,0.2}\right),\dots$, and [0.8, 1), respectively.

To ensure that the predicted numbers of breast cancer cases in the target population subgroups are sufficiently accurate (“good calibration”), we impose constraints based on 5-year absolute risks projected from the BCRAT as follows. Since the BCRAT has been shown to be generally well calibrated (Costantino et al., 1999; Rockhill et al., 2001; Bondy et al., 1994), by equating the predicted number by our new model to that of the BCRAT, the new model would be similarly well calibrated. Let ${\phi }^B\left(T_1,T_2,{\mathbf{\text{X}}}^B\right)$ denote the BCRAT absolute risk estimate in age interval $(T_1,T_2)$ given predictors ${\mathbf{\text{X}}}^B={(\text{Ageflb},\text{Agemen},\text{Numrel},\text{Nbiops})}^{\top }$, which can be calculated online (http://www.cancer.gov/bcrisktool). The superscript “B” indicates “BCRAT”. We calculate ${\phi }^B\left(S,{\mathbf{\text{X}}}^B\right)$ as the weighted average of ${\phi }^B\left(T_1,T_2,{\mathbf{\text{X}}}^B\right)$ with $T_1,T_2\in S,T_2-T_1=5$, and weight equal to $P\left(\left[T_1,T_2\right]\right)$ as estimated from the NHIS. Further, the quartiles of ${\phi }^B\left(S,{\mathbf{\text{X}}}^B\right)$ larger than 1.67% were selected as the $a^{\prime }s$ and $b^{\prime }s$ in constraints (5) and the corresponding external risk is calculated via $P^e\left(Y=1\mid a_{\mathit{\text{sr}}}\le {\phi }^B\left(S,{\mathbf{\text{X}}}^B\right)\le b_{\mathit{\text{sr}}}\right),s=\mathrm{1,2},r=\mathrm{1,2},\mathrm{3,4}$. Therefore, we have a total of 8 constraints, 4 for each age stratum. We estimate the distribution $P^e\left({\mathbf{\text{X}}}^B\right.,\text{Weight}\mid S$) from NHIS, whose variance is ignored in the current analysis. Then we calculate $P^e\left({\mathbf{\text{X}}}^B\mid S\right)$ as $\sum _{\mathit{\text{Weight}}}{P}^e\left({\mathbf{\text{X}}}^B\right.,\text{Weight}\mid S$). We then obtain the averaged 5-year risk as

$$\begin{gathered} P^e\left(Y=1\mid {\phi }^B\left(S=1,{\mathbf{\text{X}}}^B\right)\in \left[{\mathit{a}}_1,{\mathit{b}}_1\right]\right)=(1.8\%,2.0\%,2.6\%,3.7\%), \\ {P}^e\left(Y=1\mid {\phi }^B\left(S=2,{\mathbf{\text{X}}}^B\right)\in \left[{\mathit{a}}_2,{\mathit{b}}_2\right]\right)=(1.8\%,2.7\%,3.3\%,5.7\%). \end{gathered}$$

We further consider two constraints, $P^e(Y=1\mid S=1)=0.8\%$ and $P^e(Y=1\mid S=2)=1.8\%$, to control the overall risks

We use these 10 numbers as the benchmark for risk prediction calibration. The left hand side of the resultant constraints below is in a form that is slightly different from (5), because predictor “Weight” was not included in ${\mathbf{\text{X}}}^B$ and therefore have to be averaged in the constraints:

$$\frac{\sum _{{\mathbf{\text{X}}}^B:a_{\mathit{\text{sr}}}\le {\phi }^B\left(S,{\mathbf{\text{X}}}^B\right)\le b_{rs}}\sum _{\text{Weight}}\sum _{Z^c}P\left(Y=1\mid S,\mathbf{\text{X}},Z^c\right)P^e(\mathbf{\text{X}}\mid S)P\left(Z^c\mid S,\mathbf{\text{X}};\mathit{\gamma },\mathit{\omega },\rho \right)}{\sum _{{\mathbf{\text{X}}}^B:a_{\mathit{\text{sr}}}\le {\phi }^B\left(S,{\mathbf{\text{X}}}^B\right)\le b_{\mathit{\text{sr}}}}{P}^e\left({\mathbf{\text{X}}}^B\mid S\right)}$$

The results are summarized in Table 1. We conduct the following Ad Hoc method for comparison and use its result as our initial value. First, a simple logistic regression model is fitted to obtain the OR estimates for $\left(\mathbf{\text{X}},Z^c\right)$. Then we fit the $P(Z\mid \mathbf{\text{X}},S)$ model using controls only to obtain an approximate estimate of the PD distribution. Lastly, we solve for the corresponding stratum specific intercept parameters by plugging these estimates back into (5). The estimated log ORs obtained by our method can be quite different from those by the Ad Hoc method. In the mean model for the PD distribution, Ageflb and Nbiops are positively associated, while age strata, Agemen, and Weight are negatively associated. Weight is negatively associated in the variance model. As indicated by the estimate of the mixture probability $\rho$, which is significantly different from zero, only around 10% subjects truly have no dense breast tissue as reflected on the mammogram image, while the remaining 90% have positive measurements. The estimated parameter values involved in the PD distribution do not change greatly across different methods. This indicates that the constraint itself does not provide too much extra information to estimate these parameters. The predicted risk is provided in Table 2. It is easily to see from Section 3.3 that assessing the accuracy via risk is equivalent to that via number of cases. Although there are slight discrepancies in some intervals, the Ad Hoc method predicts the number of cases as well as ours in general. Chen et al. (2006) have discussed a similar behavior. Applying the estimated model in (1) to a very large sample (100, 000) overemphasizes the insubstantial difference. AUC comparison confirms the similarity with p-value of 0.521. We also compare our method with the constraint proposed in Chatterjee et al. (2016). This constraint performs poorly in estimating the OR parameter because it cannot bring in the information contained in the target biomarker data, resulting in a bad case prediction (Table 2) and a significantly small AUC with p-value < 0.001 (Figure 2). Both AUC tests are completed by using permutation test to avoid bias in DeLong’s algorithm (Zhu et al., 2024).

Table 1:

Analysis of the BCDDP data: estimates of the stratum specific intercept parameters and log ORs for the BCRAT predictors, weight, and PD, together with estimates of parameters in the zero-inflated Beta regression model for the distribution of PD. “Proposed” stands for the proposed method; “Ad Hoc” stands for the Ad Hoc method; “Chatterjee’s” stands for the method using constraints from Chatterjee et al. (2016). “est”: Parameter estimation. “$\widehat{\text{sd}}$”: Estimated standard deviation.

Parameter		Proposed		Ad Hoc		Chatterjee’s
		est	$\widehat{\mathbf{\text{sd}}}$	est	$\widehat{\mathbf{\text{sd}}}$	est	$\widehat{\mathbf{\text{sd}}}$
log OR parameter	Age<50	−6.824	1.192	−6.719	0.243	26.340	0.014
	Age≥50	−5.870	0.901	−5.747	0.086	−10.489	0.008
	Agemen	0.176	0.066	0.185	0.057	0.323	0.012
	Ageflb	0.036	0.053	0.127	0.046	0.234	0.008
	Numrel	0.717	0.087	0.654	0.086	−3.371	0.032
	Nbiops	0.203	0.073	0.156	0.067	0.392	0.014
	Weight	0.224	0.046	0.181	0.041	1.238	0.005
	PD	0.200	0.019	0.180	0.017	0.002	0.002
Mean model $\mathit{\gamma }$	Intercept	0.579	0.088	0.553	0.115	0.672	0.017
	Age≥50	−0.479	0.045	−0.447	0.059	−0.505	0.010
	Agemen	−0.089	0.029	−0.079	0.042	−0.076	0.005
	Ageflb	0.115	0.023	0.130	0.034	0.131	0.004
	Nbiops	0.253	0.031	0.260	0.046	0.266	0.006
	Weight	−0.408	0.023	−0.413	0.032	−0.387	0.004
Variance model $\omega$	Intercept	1.662	0.115	1.647	0.139	1.629	0.020
	Age≥50	0.106	0.065	0.120	0.087	0.109	0.011
	Weight	−0.106	0.033	−0.141	0.043	−0.123	0.006
Mixture probability	$\rho$	0.105	0.008	0.101	0.009	0.081	0.001

Table 2:

Risk prediction assessment of the BCDDP data. The reported values are absolute error. “Interval”: absolute risk intervals defined in (4); “Risk”: averaged risk within each interval; “Cases”: predicted number of cases. Followed by the absolute error in the parentheses. “True” represents risks and number of cases from the well calibrated external model; “Proposed” stands for the proposed method; “Ad Hoc” stands for the Ad Hoc method; “Chatterjee’s” stands for the method using constraints from Chatterjee et al. (2016).

Intervals	Risk				Cases
	True	Proposed	Ad Hoc	Chatterjee’s	True	Proposed	Ad Hoc	Chatterjee’s
$\phi (S=1,\mathbf{\text{x}})\in [0.32\%,8.18\%)$	0.008	0.000	0.000	0.992	270	12	16	29491
$\phi (S=2,\mathbf{\text{x}})\in [0.85\%,11.22\%)$	0.018	0.000	0.000	0.012	1439	36	113	445
$\phi (S=1,\mathbf{\text{x}})\in [1.58\%,1.93\%)$	0.018	0.001	0.001	0.982	15	2	1	898
$\phi (S=1,\mathbf{\text{x}})\in [1.93\%,2.24\%)$	0.020	0.002	0.002	0.980	5	0	2	323
$\phi (S=1,\mathbf{\text{x}})\in [2.24\%,3.15\%)$	0.026	0.002	0.002	0.974	17	3	0	641
$\phi (S=1,\mathbf{\text{x}})\in [3.15\%,8.18\%)$	0.037	0.010	0.010	0.963	7	1	2	255
$\phi (S=2,\mathbf{\text{x}})\in [1.59\%,2.34\%)$	0.018	0.000	0.000	0.009	515	40	26	239
$\phi (S=2,\mathbf{\text{x}})\in [2.34\%,2.95\%)$	0.027	0.001	0.001	0.023	260	3	19	40
$\phi (S=2,\mathbf{\text{x}})\in [2.95\%,4.46\%)$	0.033	0.003	0.003	0.032	180	3	27	5
$\phi (S=2,\mathbf{\text{x}})\in [4.46\%,11.22\%)$	0.057	0.003	0.003	0.057	98	8	16	0

Figure 2:

ROC curves and AUC of the proposed method and Ad Hoc method.

5. Simulation Studies

We performed simulations to illustrate: (i) The source data are collected by the stratified case-control design from the source population. (ii) The underlying population of the source data are different from the target population. The simulation scheme is devised in two steps. First, we define a target population where $P^e(\mathbf{\text{X}}\mid S)$ is known, $P(\mathbf{\text{Z}}\mid \mathbf{\text{X}},S)$ is identical to the one defined in the later section referring to the source population, and a well-calibrated risk prediction model based on $(\mathbf{\text{X}},S)$ is available. Second, we generate $(\mathbf{\text{X}},\mathbf{\text{Z}},S)$ from the source population through case-control sampling, mimicking the BCDDP case-control study described above. The details are depicted as follows.

5.1. External Information from the Target Population

We set $\mathbf{\text{X}}={\left(X_1,X_2,X_3,X_4\right)}^T$, where $X_1$ is generated by categorizing data generated from $N\left(\mathrm{0,1}\right)$ into 2 groups by its 0.8 quartile, and $X_2$ is generated from a multinomial distribution with probabilities (0.8, 0.1, 0.1). For $X_3$, we first generate a Poisson random variables with mean $-\text{log}\left(0.75\right)$, and assign values 0 or 1 to $X_3$ according to whether the Poisson variable is 0. The last component $X_4$ is similarly generated from a Poisson variable with mean $-\text{log}\left(0.7\right)$. For the stratum variable $S$, we generate a Uniform random variable in the range of (30, 80), and assign $S=1$ if this variable is less than 50 and $S=2$ otherwise. Therefore, $\mathbf{\text{X}}$ and $S$ are mutually independent. The conditional distribution $P^e(\mathbf{\text{X}}\mid S)$ can then be expressed as ${\prod }_{i=1}^4{P}^e\left(X_i\right)$, which is treated as known. A single biomarker $Z$ is generated based on $S$ and $\mathbf{\text{X}}$, following the Beta model $P(Z\mid \mathbf{\text{X}},s)~\mathit{\text{Beta}}(\kappa \varphi ,\varphi -\kappa \varphi )$, where $\text{logit}\left(\kappa \right)={\gamma }_0+{\gamma }_{1}I\{s=2\}+{\gamma }_2{x}_1+{\gamma }_3{x}_3$, and $\text{log}\left(\varphi \right)={\omega }_0+{\omega }_{1}I\{s=2\}+{\omega }_2{x}_2$. We set ${\left({\gamma }_0,{\gamma }_1,{\gamma }_2,{\gamma }_3\right)}^{\top }=(\mathrm{0.6,0.4,0.2},-0.2{)}^{\top }$ and ${\left({\omega }_0,{\omega }_1,{\omega }_2\right)}^{\top }=(\mathrm{0.6,0.2,0.1}{)}^{\top }$. The categorized variable $Z^c$ takes integer values 0 to 9, corresponding to $Z\in \left[\mathrm{0,0.1}\right),\left[\mathrm{0.1,0.2}\right),\dots$, and [0.9, 1), respectively. The population risk of $Y$ based on ${\left({\mathbf{\text{X}}}^T,Z^{c^T},S\right)}^T$ is given by (1) where ${\alpha }_s=(-4.5,-3.5{)}^{\top },{\mathit{\beta }}_{\mathbf{\text{X}}}=(\mathrm{0.3,0.6},-\mathrm{0.4,0.6}{)}^{\top }$ and ${\mathit{\beta }}_z=0.06$.

Thus, a well calibrated risk prediction model based on ($\mathbf{\text{X}},S$) is

$$\phi (\mathbf{\text{x}},s;\mathit{\eta })\equiv p^e(Y=1\mid \mathbf{\text{x}},s;\eta )=\int P(Y=1\mid \mathbf{\text{x}},\mathbf{\text{z}},s)P\left(Z^c\mid \mathbf{\text{x}},s\right)d\mathbf{\text{z}}$$

Consistent with the common goodness-of-fit test of calibration, we choose the quartiles of $\phi (s,\mathbf{\text{x}};\eta )$ as the cutoffs for each stratum used in the constraint (6). Incorporating $P^e(\mathbf{\text{x}}\mid S)$, we are able to obtain these cutoff values $\left[a_{\mathit{\text{sr}}},b_{\mathit{\text{sr}}}\right],s=\mathrm{1,2},r=\mathrm{1,2},\mathrm{3,4}$ within each risk interval. They are considered as the calibration benchmark, leading to corresponding constraints as described in (6). In order to mimic the real data set, $P^e(Y=1\mid \left.a_{\mathit{\text{sr}}}\le \phi \left(s,{\mathbf{\text{x}}}_k\right)\le b_{\mathit{\text{sr}}}\right)$ in (6) is computed as an empirical risk $\sum _{i=1}^{n^e}I\left\{y_i=1,a_{\mathit{\text{sr}}}\le \phi \left(s_i,{\mathbf{\text{x}}}_i\right)\le b_{\mathit{\text{sr}}}\right\}/n^e$ from a cross-sectional sample ${\left\{y_i,s_i,{\mathbf{\text{x}}}_i,z_i^c\right\}}_{i=1}^{n^e}$ randomly drawn from the target population with size $n^e=\mathrm{100,000}$.

5.2. Generate the Source Data and Evaluate the Model

We generate the case-control sample from distributions that differ from those in Section 5.1. First, $X_1$ is generated by grouping $N\left(\mathrm{0,1}\right)$ into 2 groups by its 0.4 quartile, and $X_2$ is generated from a multinomial distribution with probabilities (0.3, 0.4, 0.3). For $X_3$, we first generate a Poisson variable with mean 1.25 and assign values 0 or 1 to $X_3$ according to whether the variable is $0.X_4$ is similarly generated as $X_3$, where the Poisson distribution has mean 1. Here, we deliberately incorporated a source population that has different distribution for $\mathbf{\text{X}}$ than the target population so that we can demonstrate the performance of our method under nonidentical ${\pi }_i’\text{s}$ and ${\delta }_i’\text{s}$. The stratum variable $S$ as in Section 5.1. The biomarker $Z$ and its categorization are created in the same way except the parameters ${\left({\gamma }_0,{\gamma }_1,{\gamma }_2,{\gamma }_3\right)}^{\top }=(0.5,0.45,0.25,-0.25{)}^{\top }$ and ${\left({\omega }_0,{\omega }_1,{\omega }_2\right)}^{\top }=(0.5,0.1,0.2{)}^{\top }$. The OR parameters in the source population are ${\alpha }_s=(-4.3,-3.2{)}^{\top },{\mathit{\beta }}_{\mathbf{\text{x}}}=(0.35,0.65,-\mathrm{0.35,0.65}{)}^{\top }$ and ${\mathit{\beta }}_z=0.07$, different from that in the target population. We retain 500 cases and 1000 controls for each stratum, thus, a total of 1000 cases and 2000 controls are included in the analysis. We apply the augmented Lagrangian multiplier to the proposed model to estimate the parameters $\mathit{\alpha },\mathit{\beta },\mathit{\gamma },\mathit{\omega }$. We repeat the simulation 1000 times.

5.3. Numerical Results

We provide the parameter estimation and standard deviation estimation results in Table 3, and demonstrate the predicted risk within each risk interval $\left[a_{\mathit{\text{sr}}},b_{\mathit{\text{sr}}}\right],s=\mathrm{1,2},r=\mathrm{1,2},\mathrm{3,4}$ calculated from Step 1 in Table 3. The parameter estimation exhibits two distinct characteristics. The estimates $\widehat{\alpha },\widehat{{\beta }_2},\widehat{{\beta }_4}$, and $\widehat{\mathit{\mu }}$ show the first characteristic of small absolute errors between the estimates and the parameter values in the target population. This reflects the benefit of incorporating the constraints. In particular, the constraints contribute substantially to improving the estimation of $\stackrel{\mathit{ˆ}}{\mathit{\alpha }}$, since they correct the inconsistency that arises when relying solely on the case–control sample from the source (Cao et al., 2024). The improvement further extends to $\widehat{\mathit{\mu }}$, as it contains $\stackrel{\mathit{ˆ}}{\mathit{\alpha }}$. The estimates $\widehat{{\beta }_2}$ and $\widehat{{\beta }_4}$ also benefit from the implementation of the constraints. In contrast, the $\text{AdHoc}$ method estimates ${\beta }_2$ and ${\beta }_4$ with large absolute errors because it does not leverage external information. Ad Hoc estimates $\mathit{\alpha }$ and $\mathit{\mu }$ with similar absolute errors to our method because it uses the same external information, but leads to larger variabilities.

Table 3:

Parameter estimation and variance estimation in simulation. “Source” and “Target” stand for the true parameters in two populations respectively. “Proposed” stands for the proposed method; “Ad Hoc” stands for the Ad Hoc method. “AE”: Absolute error between the estimated and target parameters. “sd”: Empirical standard deviation. “$\widehat{\text{sd}}$”: Estimated standard deviation. “ratio of $\widehat{\text{sd}}$”: Ratio of $\widehat{\text{sd}}$ of the proposed method to that of the Ad Hoc method.

Parameter	Source	Target	Proposed			Ad Hoc			ratio of $\widehat{\mathbf{\text{sd}}}$
			AE	sd	$\widehat{\mathbf{\text{sd}}}$	AE	sd	$\widehat{\mathbf{\text{sd}}}$
${\alpha }_1$	−4.30	−4.50	0.13	0.12	0.13	0.20	0.17	0.24	0.5
${\alpha }_2$	−3.20	−3.50	0.17	0.13	0.13	0.23	0.17	0.08	1.7
${\beta }_1$	0.35	0.30	0.05	0.07	0.05	0.05	0.08	0.08	0.6
${\beta }_2$	0.65	0.60	0.02	0.02	0.03	0.05	0.05	0.05	0.5
${\beta }_3$	−0.35	−0.40	0.05	0.04	0.05	0.05	0.09	0.08	0.6
${\beta }_4$	0.65	0.60	0.01	0.03	0.04	0.05	0.08	0.08	0.5
${\beta }_z$	0.07	0.06	0.01	0.02	0.02	0.01	0.02	0.02	0.9
${\gamma }_0$	0.50	0.60	0.10	0.07	0.06	0.10	0.09	0.09	0.7
${\gamma }_1$	0.45	0.40	0.05	0.05	0.05	0.05	0.08	0.07	0.7
${\gamma }_2$	0.25	0.20	0.05	0.03	0.03	0.05	0.04	0.04	0.7
${\gamma }_3$	−0.25	−0.20	0.05	0.03	0.03	0.04	0.04	0.05	0.7
${\omega }_0$	0.50	0.60	0.10	0.05	0.05	0.10	0.08	0.07	0.7
${\omega }_1$	0.10	0.20	0.10	0.05	0.05	0.09	0.08	0.07	0.7
${\omega }_2$	0.20	0.10	0.10	0.02	0.02	0.11	0.03	0.03	0.7
${\mu }_1$	1.85	2.47	0.21	0.04	1.40	0.23	0.07	/	/
${\mu }_2$	0.76	1.37	0.05	0.04	0.46	0.08	0.07	/	/

$\widehat{{\beta }_1},\widehat{{\beta }_3},\widehat{{\beta }_z},\widehat{\mathit{\gamma }}$, and $\widehat{\mathit{\omega }}$ has a different characteristic in that they remain close to the source estimates and exhibit large absolute errors. Specifically, $\widehat{{\beta }_z},\widehat{\mathit{\gamma }}$, and $\widehat{\mathit{\omega }}$ benefit little from the constraints, as no relevant external information is available for these parameters for both methods. The effects on $\widehat{{\beta }_1}$ and $\widehat{{\beta }_3}$ are also limited, suggesting only a weak influence of the constraints. Although the two methods show similar performance in terms of absolute bias, Ad Hoc method has larger variability than our method. Further details on how the constraints affect parameter estimation are provided in Section 5.4.

The initial value of the augmented Lagrangian multiplier is provided by the Ad Hoc method mentioned in Section 4 that is also used for comparison. Following the description in Section 3.3, we computed the observed and expected numbers of cases from the cross-sectional case-control sample, which is reported in Table 4. The proposed method has a slightly better performance in terms of case predictions. The ROC plot in Figure 2 shows that the two methods perform almost identically, indicating highly similar predictions. Note that Ad Hoc requires much more target population information than our method. To formally assess the difference between the proposed method and the Ad Hoc method, we compared the AUCs of the ROC curves based on both methods. The resulting p-value of 0.784 confirms their similarity. This p-value was obtained using a permutation test instead of DeLong’s algorithm, which has recently been criticized by researchers (DeLong et al., 1988; Zhu et al., 2024).

Table 4:

Risk prediction and accuracy measurement of simulation. The reported values are absolute error to the target truth and ratio of standard deviation. “Interval”: absolute risk intervals defined in (4); “Risk”: averaged risk within each interval; “Cases”: predicted number of cases. “True” represents risks and number of cases from the well calibrated external model; “Proposed” stands for the proposed method; “Ad Hoc” stands for the Ad Hoc method; “ratio of $\widehat{\text{sd}}$” stands for the ratio of standard deviation of the proposed method to that of the Ad Hoc method.

Intervals	Risk				Cases
	True	Proposed	Ad Hoc	ratio of $\widehat{\mathbf{\text{sd}}}$	True	Proposed	Ad Hoc	ratio of $\widehat{\mathbf{\text{sd}}}$
$\phi (S=1,\mathbf{\text{x}})\in [2.28\%,23.34\%)$	0.054	0.004	0.007	0.99	2194	37	52	0.65
$\phi (S=2,\mathbf{\text{x}})\in [6.18\%,45.51\%)$	0.134	0.005	0.003	0.99	8011	247	711	0.61
$\phi (S=1,\mathbf{\text{x}})\in [2.28\%,5.20\%)$	0.035	0.003	0.005	0.44	930	46	54	0.89
$\phi (S=1,\mathbf{\text{x}})\in [5.20\%,7.92\%)$	0.063	0.004	0.008	0.50	581	25	24	0.85
$\phi (S=1,\mathbf{\text{x}})\in [7.92\%,11.91\%)$	0.097	0.008	0.014	0.46	423	1	3	0.81
$\phi (S=1,\mathbf{\text{x}})\in [11.91\%,23.34\%)$	0.162	0.013	0.019	0.22	260	32	28	0.56
$\phi (S=2,\mathbf{\text{x}})\in [6.18\%,13.09\%)$	0.093	0.004	0.003	0.48	3378	37	414	0.83
$\phi (S=2,\mathbf{\text{x}})\in [13.09\%,19.11\%)$	0.158	0.005	0.003	0.49	2074	64	203	0.73
$\phi (S=2,\mathbf{\text{x}})\in [19.11\%,27.06\%)$	0.229	0.011	0.007	0.42	1623	122	38	0.73
$\phi (S=2,\mathbf{\text{x}})\in [27.06\%,45.51\%)$	0.349	0.014	0.007	0.21	936	25	54	0.47

5.4. Additional Numerical Studies

In addition to the simulation mimicking the real BCDDP data, we conduct extensive simulation studies to evaluate the performance of the proposed method under more settings. We consider two strata, $S=\mathrm{1,2}$, with probability 0.6 and 0.4 respectively. We include four covariates ${\mathbf{\text{X}}}^{\top }={\left(X_1,X_2,X_3,X_4\right)}^T$, where $X_1,X_3,X_4$ are Bernoulli with success probabilities 0.8, 0.75, 0.7 respectively, and $X_2$ is ternary with probabilities (0.8, 0.1, 0.1). The biomarkers $\mathbf{\text{Z}}$ are generated from 3 independent Bernoulli trials: $Z_1~\text{Bernoulli}\left(\text{logit}\left({\gamma }_1{X}_1+{\gamma }_2{X}_2\right)\right),Z_2~\text{Bernoulli}\left(\text{logit}\left({\gamma }_3{X}_1+{\gamma }_4{X}_3\right)\right)$, and $Z_3~\text{Bernoulli}\left(\text{logit}\left(I\{S=2\}+{\gamma }_5{X}_4\right)\right)$. Let ${\gamma }^{\top }={\left({\gamma }_1,{\gamma }_2,{\gamma }_3,{\gamma }_4,{\gamma }_5\right)}^{\top }$. Then, the response variable $Y$ is generated from Bernoulli $\left(\text{logit}\left\{\alpha +{\mathit{\beta }}_{\mathbf{\text{X}}}^{\mathbf{\top }}\mathbf{\text{X}}+{\mathit{\beta }}_{\mathbf{\text{Z}}}^{\mathbf{\top }}\right\}\right)$ with parameter ${\mathit{\alpha }}^{\mathbf{\top }}={\left({\alpha }_1,{\alpha }_2\right)}^{\mathbf{\top }}$ and ${\mathit{\beta }}^{\top }=\left({\mathit{\beta }}_{\mathbf{\text{X}}}^{\top },{\mathit{\beta }}_{\mathbf{\text{Z}}}^{\top }\right)={\left({\beta }_{X1},{\beta }_{X2},{\beta }_{X3},{\beta }_{X4},{\beta }_{Z1},{\beta }_{Z2},{\beta }_{Z3}\right)}^{\top }$. The true parameters are ${\mathit{\alpha }}^{\top }=(-3.5,-4{)}^{\top },{\mathit{\beta }}_{\mathbf{\text{X}}}^{\top }=(-0.25,0.65,-0.55,-0.15{)}^{\top },{\mathit{\beta }}_{\mathbf{\text{Z}}}^{\top }=(0.4,-\mathrm{0.3,0.5}{)}^{\top }$, and ${\mathit{\gamma }}^{\top }=(0.5,-0.2,-0.4,0.2,-0.3{)}^{\top }$ in the basic setting based on which we generate the target population. Further, the constraints are created from the target population following (5) with two overall risks and four grouped risks in each stratum, yielding 10 equality constraints in total with values (0.067, 0.047, 0.036, 0.059, 0.095, 0.151, 0.024, 0.041, 0.067, 0.1084). Then the case-control sample is generated with 1:1 cases-controls ratio from the source population whose distributions and parameters differ from those of the target population. Specifically, in the source population, $X_1,X_3,X_4$ are independent Bernoulli random variables with success probabilities 0.4, 0.2865, 0.3679 respectively, and $X_2$ is ternary with probabilities (0.3, 0.4, 0.3). The parameter values are $\left({\mathit{\alpha }}^{\mathbf{\top }},{\mathit{\beta }}^{\mathbf{\top }}\right)=(-4.5,-3.5,0.35,0.65,-0.3,0.5,0.3,-0.4,0.4{)}^{\mathbf{\top }}$. We adopt an identical $\mathit{\gamma }$ in the source population due to the correctly specified model assumption for $Z\mid S,\mathbf{\text{X}}$. All scenarios are repeated 1000 times independently on a system with 2 Dual Intel Xeon E2630 v4, 10 cores @ 2.2 GHz CPU and 128G memory.

We first examine the asymptotic performance of the proposed method under the basic setting and provide the result in Table S5. The results show a consistent pattern in parameter estimation and, as expected, a decreasing trend in standard deviations as sample size increases. We also report the algorithmic convergence performance including the averaged running time and iterations executed. Here, the iteration stops when the change in the derivative along the searching direction is less than 10⁻⁶, a software default threshold value. This generally leads to between 400 and 600 iterations throughout our implementation. The detailed results are in Table S5, Figure S2 and Figure S3. Interestingly, the algorithm converges faster for larger sample sizes than for smaller ones. This behavior is explained by the fact that larger datasets provide smoother likelihood surfaces, while the computational burden is not substantially affected by sample size due to the discretized profile likelihood (3). In particular, the computation complexity depends on the number of distinct combinations of $S,\mathbf{\text{X}}$, and $\mathbf{\text{Z}}$ rather than on the total size $n$.

In the following additional simulations, we maintain the baseline setting with sample size 3000. The convergence speed is roughly 470 iterations and 9 seconds for all simulation scenarios. First, we assess the robustness of our method across two scenarios. First, we vary the distribution of $\mathbf{\text{X}}$ in the source population to evaluate the robustness of our method under various degrees of distributional shift. The difference between the source and target populations is quantified using Kullback-Leibler (KL) divergence. In the left panel of Table S6, the distributions of $\mathbf{\text{X}}’\text{s}$ are identical in the source and target, resulting in a zero KL-divergence. The middle panel is the basic setting with KL = 1.557. In the right panel, we generate source $\mathbf{\text{X}}$ as below: $X_1,X_3,X_4~\text{Bernoulli}\left(0.01\right)$ and are independent, $X_2=\mathrm{0,1},2$ with probabilities 0.01, 0.01 and 0.98 respectively. This yields KL = 12.203. Even in this extreme scenario, our method performs well, with biases and standard deviations remaining stable. These results demonstrate the robustness of our method under different degrees of distributional shift between the source and target populations of $\mathbf{\text{X}}$. Next, we vary the case-control ratios. Specifically, we considered case-control ratios of 1:4, 4:1, 1:20, and 20:1. Based on the results in Table S7, our estimates remain reliable and perform well across all case-control ratios.

Further, we relax the equality constraints to inequality constraints as proposed in (6) and experimented with different levels of inequality constraints. Here, the buffer coefficient $d_{\mathit{\text{sr}}}$ is used to represent the reliance on the constraint. A larger $d_{\mathit{\text{sr}}}$ implies less reliance while a smaller $d_{\mathit{\text{sr}}}$ represents more reliance. In the extreme case, when we set $d_{\mathit{\text{sr}}}=0$, the inequality constraints degenerate to the equality constraints, representing a full reliance on the constraint. In practice, if the target population information is fully trustable, we recommend using $d_{\mathit{\text{sr}}}=0$ or very small (such as ≤ 0.2), while setting $d_{\mathit{\text{sr}}}$ to be a large value (such as 2) when one is unwilling to fully trust the information. For illustration, we vary $d_{\mathit{\text{sr}}}$ while keeping all other settings fixed in the baseline setting. Table S8 reports the performances of our method at $d_{\mathit{\text{sr}}}=0.2,\mathrm{1,2},4,10$. When $d_{\mathit{\text{sr}}}$ is larger, say 4 or 10, the parameter estimation is farther from the target truth. As $d_{\mathit{\text{sr}}}$ decreases, the estimation moves closer to the target population. We also compare the performance of our method when the conventional Sequential Quadratic Programming (SQP) is adopted, which does not augment an additional quadratic term to guarantee convex Lagrangian. See Table S8 in the Supplement. Although SQP completes the optimization faster, its performance is slightly worse when inequality constraints with large $d_{\mathit{\text{sr}}}$ are considered. When $d_{\mathit{\text{sr}}}$ is small or even tighten into equality constraints, SQP performs as well as ours. We believe this is caused by the nonconvexity of the resulting optimization problem solved by SQP.

Finally, we compare the performance of our method in two different types of target population information, which we name Negative Control and Positive Control. The Negative Control refers to the situation when no target biomarker distribution information is available. In this case we adopt Chatterjee et al. (2016). In contrast, the Positive Control refers to the situation where the full biomarker data is available from the target population hence the biomarker distribution can be considered fully known. We adapt our method to incorporate such additional information. In our simulation, the corresponding external data is generated using parameters where $\mathit{\beta }$ is estimated from a large target sample of $Y\mid S,\mathbf{\text{X}},\mathbf{\text{Z}}$, and $\mathit{\gamma }$ is estimated from the model of $\mathbf{\text{Z}}\mid S,\mathbf{\text{X}}$ based on the same sample. Table S9 presents the parameter estimates. As expected, Positive Control provides highly accurate parameter estimation. On the other hand, Negative Control shows large bias in estimating $\mathit{\alpha }$ and $\mathit{\beta }$. This is because Chatterjee et al. (2016) incorporates additional mean zero requirement on the reduced score function, which is not satisfied in our case-control setting. Examining the AUC and ROC curves in Figure S4 further highlights the advantage of our method, which clearly outperforms the Negative Control, indicating substantially better risk prediction. Similarly as in Section 5.3, a permutation test is conducted to compare the proposed method with the others, yielding p-values of 0.23, 0.90, and 0.04, respectively. These results demonstrate good performance of our method in parameter estimation and risk prediction, confirming that our model is well calibrated.

6. Conclusion

We develop a method for risk prediction in a target population using a case-control sample from a potentially different source population, together with external data from the target population containing fewer predictors. By imposing constraints derived from the external data on the case-control sample based maximum likelihood estimates, our method achieves good calibration even when the working prediction model is potentially misspecified.

We analyzed a subset of the BCDDP cases and controls to demonstrate our methods, where the remaining subjects were excluded due to the lack of mammographic density data. Benichou et al. (1997); Chen et al. (2006, 2008a) showed that the mammographic density in the BCDDP data depends only on the case–control status, hence is missing completely at random given case-control status. Because our retrospective likelihood function is conditioned on the same case–control status, it remains to be valid, although the efficiency is lower compared to the full data analysis if it had been applicable. On the other hand, if the missingness of mammographic density had depended on elements in addition to case-control status, our method would lead to selection bias. Further studies are needed regarding how to handle various missingness.

In the context of predicting human diseases, a key interest is identifying population subgroups with very high risk. Consequently, it is particularly important for the model to exhibit good calibration in the tails of the risk distribution. Our method naturally accommodates this by allowing the use of tailored risk intervals. For instance, stringent constraints can be imposed on risk intervals in the tail regions, while looser constraints can be applied elsewhere, providing flexibility to prioritize accuracy where it matters most.

Another important feature of our proposed method is its ability to accommodate differences between the source and target populations. For example, in the BCDDP study, women enrolled in BCDDP differed from the general White European population both in composite breast cancer rates and in the distribution of breast cancer risk factors. However, the BCRAT method assumed a common odds ratio function for breast cancer across both populations. In contrast, our method provides a framework to identify a working model that achieves desirable calibration even when the source data distribution deviates from that of the target population. This flexibility opens the possibility of improving risk prediction by incorporating heterogeneous sources of data with more complex structures.

The likelihood function in our method can be extended. For example, the likelihood function does not have to be the retrospective likelihoods or logistic regression. The likelihood function can also be generalized to any loss functions with respect to the parameters of interest from the source population (Cao et al., 2024). This method can be applied to calibrate the outcome rather than parameters, leading to a special loss function, as in isotonic regression, although such applications require substantial external knowledge (Jiang et al., 2011).

Beyond the constraints discussed above, many other forms of external information can be incorporated to enhance calibration. For instance, missingness in the source population can be addressed by borrowing information from an external model built on a richer dataset. The flexibility of our framework also allows calibration across data types, such as using continuous data to calibrate discrete data, or vice versa. This idea is closely related to calibration regression, which minimizes calibration error by quantifying the discrepancy between the source and calibrated outcomes (Guo et al., 2017). Furthermore, specially designed investigations into the relationship between the distributions of the source and target populations are of interest, naturally linking our framework to broader topics in domain adaptation.

Another natural direction for further improvement is to accelerate computation while maintaining accuracy. Current implementation does not take advantage of the structure of objective function and constraints. Incorporating more advanced optimization algorithms, such as the Alternating Direction Method of Multipliers (Boyd et al., 2011), offers a promising way to enhance both efficiency and scalability.

Supplementary Material

See Supplemental Material at online for more numerical simulations.

Figure 1:

ROC curves and AUC of the proposed method and Ad Hoc method.

A. Derivation of the loglikelihood function (2)

The retrospective likelihood function at a single observation is rewritten as

$$\begin{array}{l}=\frac{\begin{array}{c}{\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\mid Y=i,S=s\right)\\ {\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l,Y=i\mid S=s\right)\end{array}}{{\text{Pr}}^{*}(Y=i\mid S=s)}\\ =\frac{{\text{Pr}}^{*}\left(Y=i\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l,S=s\right){\text{Pr}}^{*}\left(\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right){\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k\mid S=s\right)}{{\text{Pr}}^{*}(Y=i\mid S=s)}.\end{array}$$

The denominator ${\text{Pr}}^{*}(Y=i\mid S=s)$ is attained by summing up $\mathbf{\text{X}}$ and $\mathbf{\text{Z}}$ from ${\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l,Y=i\mid S=s\right)$, i.e., $\sum _{k=1}^K\sum _{l=1}^L{\text{Pr}}^{*}\left(Y=i\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l,S=s\right){\text{Pr}}^{*}\left(\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right){\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k\mid S=s\right)$. Note that ${\text{Pr}}^{*}\left(Y=i\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,\mathbf{\text{Z}}={\mathbf{\text{z}}}_l,S=s\right)=P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}S}^{*}$ is the model of $Y$ given ${\left({\mathbf{\text{X}}}^{\text{T}},{\mathbf{\text{Z}}}^{\text{T}},S\right)}^{\text{T}}$ in the source population in (1), ${\text{Pr}}^{*}\left(\mathbf{\text{Z}}={\mathbf{\text{z}}}_l\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right)$ is denoted as $f_{\tau }(\mathbf{\text{Z}}\mid \mathbf{\text{X}},S)$, and ${\text{Pr}}^{*}\left(\mathbf{\text{X}}={\mathbf{\text{x}}}_k\mid S=s\right)$ in the source population is denoted as ${\pi }_{\mathit{\text{sk}}}$. Thus (2) holds.

B. Proof of Lemma 1

We consider the constrained optimization problem from (2) with an intrinsic constraint $\sum _{k=1}^K{\pi }_{\mathit{\text{sk}}}-1=0$ that is induced by the legit probability ${\pi }_{\mathit{\text{sk}}}$. Therefore, the Lagrangian multiplier method requires to maximize the function

$$g^{*}(\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\pi })=l(\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\pi })+\sum _{s=1}^M{\lambda }_s^{*}\left(\sum _{k=1}^K{\pi }_{\mathit{\text{sk}}}-1\right),$$ (9)

where ${\lambda }^{*}$ denote the set of Lagrange multipliers ${\lambda }_s^{*}:s\in \{1,\dots ,M\}$.

In maximizing ((9)) over $\mathit{\pi }$ with parameters $(\mathit{\alpha },\mathit{\beta },\mathit{\tau })$ fixed, we differentiate it with respect to $\mathit{\pi }$ and obtain the following

$$\frac{\partial g^{*}}{\partial {\pi }_{\mathit{\text{sk}}}}=\frac{n_{+\mathit{\text{sk}}+}}{{\pi }_{\mathit{\text{sk}}}}-\sum _{i=0}^1{n}_{\mathit{\text{is}}++}\frac{\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}{\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}+{\lambda }_s^{*}=0.$$

Multiplying ${\pi }_{\mathit{\text{sk}}}$ by both sides of the equation and summing over $k$, we obtain

$$\begin{gathered} \sum _{k=1}^K{n}_{+\mathit{\text{sk}}+}-\sum _{k=1}^K{\pi }_{\mathit{\text{sk}}}\sum _{i=0}^1{n}_{\mathit{\text{is}}++}\frac{\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}{\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}+{\lambda }_s^{*}\sum _{k=1}^K{\pi }_{\mathit{\text{sk}}}=0, \\ {n}_{+s++}-\sum _{i=0}^1{n}_{\mathit{\text{is}}++}\sum _{k=1}^K{\pi }_{\mathit{\text{sk}}}\frac{\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}{\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}+{\lambda }_s^{*}=0, \\ {n}_{+s++}-\sum _{i=0}^1{n}_{\mathit{\text{is}}++}\sum _{k=1}^K\frac{\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_l}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}{\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}+{\lambda }_s^{*}=0, \\ {n}_{+s++}-\sum _{i=0}^1{n}_{\mathit{\text{is}}++}+{\lambda }_s^{*}=0, \\ {\lambda }_s^{*}=0. \end{gathered}$$

Therefore, we are able to express ${\pi }_{\mathit{\text{sk}}}$ as a function of (${\alpha }_s,\mathit{\beta },\mathit{\tau },{\mu }_{\mathit{\text{is}}}$):

$$\begin{gathered} {\hat{\pi }}_{\mathit{\text{sk}}}\left({\alpha }_s,\mathit{\beta },\mathit{\tau },{\mu }_{\mathit{\text{is}}}\right)=\frac{n_{+\mathit{\text{sk}}+}}{n_{+s++}\sum _{i=0}^1{\mu }_{\mathit{\text{is}}}\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}, \\ \text{where}{\mu }_{\mathit{\text{is}}}=\frac{n_{\mathit{\text{is}}++}}{n_{+s++}\sum _{k=1}^K\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\pi }_{\mathit{\text{sk}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid s,{\mathbf{\text{x}}}_k\right)}. \end{gathered}$$

Plugging ${\mathit{\lambda }}^{*}=\mathbf{0}$ and ${\widehat{\pi }}_{\mathit{\text{sk}}}$ back into $g^{*}\left(\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\pi },\mathit{\lambda },{\mathit{\lambda }}^{\mathbf{*}},\mathit{\xi }\right)$ then gives the profile likelihood (3). Further, let ${\mu }_s=\text{log}\frac{{\mu }_{1s}}{{\mu }_{0s}},s\in \{1,\dots ,M\}$, and they are free parameters. Thus, instead of constructing ${\mu }_{\text{is}}’\text{s}$ as a function of the free parameters ${\pi }_{\mathit{\text{sk}}}’\text{s}$, we view ${\mu }_{\mathit{\text{is}}}’\text{s}$ as free parameters and treat ${\widehat{\pi }}_{\mathit{\text{sk}}}$ as functions of the ${\mu }_{\mathit{\text{is}}}’\text{s}$.

C. Identifiability of parameter $\mathit{\alpha }$

We first show the theoretical identifiability. The identifiability of $\mathit{\beta }$ is easily shown by computing the odds ratio (Roeder et al., 1996; Chatterjee and Carroll, 2005). In order to show the identifiability of $\mathit{\alpha }$, rewrite the profile loglikelihood function (3) as

$$\begin{gathered} \tilde{l}(\mathit{\alpha },\mathit{\beta },\mathit{\tau },\mathit{\mu })=\sum _{\mathit{\text{ikls}}}\text{log}\frac{n_{+\mathit{\text{sk}}+}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right){\mu }_{\mathit{\text{is}}}}{n_{\mathit{\text{is}}++}\sum _{i=0}^1{\mu }_{\mathit{\text{is}}}\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)} \\ =\sum _{\mathit{\text{ikls}}}\text{log}\frac{n_{+\mathit{\text{sk}}+}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)e^{{\mu }_{s}i}}{n_{\mathit{\text{is}}++}\sum _{i=0}^1\sum _{l=1}^L{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)e^{{\mu }_{s}i}}. \end{gathered}$$

The term $P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}S}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)e^{{\mu }_{s}i}=\text{exp}\left\{i\left({\kappa }_s-{\alpha }_s\right)\right\}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}S}^{*}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)$ where ${\kappa }_s={\alpha }_s+\text{log}\left\{\left(n_{1s++}/\left(n_{0s++}\right\}-\right.\right.\text{log}\left\{\text{pr}\right(Y=1\mid s\left)\right)/\text{pr}(Y=0\mid s)\left)\right\}$. Therefore, as noted as Chatterjee and Carroll (2005) and Chen et al.(2008b), the parameter ${\alpha }_s$ is separatable from ${\kappa }_s$ in the likelihood and hence is theoretically identifiable.

In practice when external information available, for example $P^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right)$ is given, and suppose two sets of parameters $\mathit{\alpha },\mathit{\tau },\mathit{\pi }$ and $\tilde{\mathit{\alpha }},\tilde{\mathit{\tau }},\tilde{\mathit{\pi }}$ both satisfying the constraints (5), we have

$$\begin{gathered} \frac{\frac{e^{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\mathbf{\text{z}}}^{\top }{\mathbf{\text{z}}}_l}}{1+e^{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\mathbf{\text{z}}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}}{\sum _k{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}}=\frac{\frac{e^{{\tilde{\alpha }}_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\mathbf{\text{z}}}^{\top }{\mathbf{\text{z}}}_l}}{1+e^{{\tilde{\alpha }}_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\mathbf{\text{z}}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tilde{\tau }}\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}}{\sum _k{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}}, \\ \frac{\frac{1}{1+e^{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\mathbf{\text{z}}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}}{\sum _k{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}}=\frac{\frac{1}{1+e^{{\tilde{a}}_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\mathbf{\text{z}}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tilde{\tau }}\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}}{\sum _k{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}}. \end{gathered}$$ (10)

Summing the above equation over $l$ on both sides, we obtain

$$\begin{gathered} \frac{{\pi }_{\mathit{\text{sk}}}{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right)}{\sum _k{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}}=\frac{{\tilde{\pi }}_{\mathit{\text{sk}}}{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right)}{\sum _k{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}}, \\ \frac{{\pi }_{\mathit{\text{sk}}}{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right)}{\sum _k{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}}=\frac{{\tilde{\pi }}_{\mathit{\text{sk}}}{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right)}{\sum _k{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}}. \end{gathered}$$

Adding the two equations

$$\begin{gathered} {\pi }_{\mathit{\text{sk}}}\sum _k{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}={\tilde{\pi }}_{\mathit{\text{sk}}}\sum _k{P}^e\left(Y=1\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}, \\ {\pi }_{\mathit{\text{sk}}}\sum _k{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right){\tilde{\pi }}_{\mathit{\text{sk}}}={\tilde{\pi }}_{\mathit{\text{sk}}}\sum _k{P}^e\left(Y=0\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \end{gathered}$$

we have ${\pi }_{\mathit{\text{sk}}}={\tilde{\pi }}_{\mathit{\text{sk}}}$ because the external prevalence $P^e\left\{Y=1\mid {\mathbf{\text{x}}}_k,s\right\},s\in \{1,\dots ,M\},k\in \{1,\dots ,K\}$ is fixed for either $\mathit{\theta }$ or $\tilde{\mathit{\theta }}$. Plugging this back to (10) leads to

$$\begin{gathered} \frac{e^{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\text{z}}^{\top }{\mathbf{\text{z}}}_l}}{1+e^{{\alpha }_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\mathbf{\top }}{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\text{z}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)=\frac{e^{{\tilde{\alpha }}_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\text{z}}^{\top }{\mathbf{\text{z}}}_l}}{1+e^{{\tilde{\alpha }}_s+{\mathit{\beta }}_{\mathbf{\text{x}}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\text{z}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tilde{\tau }}\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right), \\ \frac{1}{1+e^{{\alpha }_s+{\mathit{\beta }}_{\text{x}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\text{z}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)=\frac{1}{1+e^{{\tilde{\alpha }}_s+{\mathit{\beta }}_{\text{x}}^{\top }{\mathbf{\text{x}}}_k+{\mathit{\beta }}_{\text{z}}^{\top }{\mathbf{\text{z}}}_l}}{f}_{\tilde{\tau }}\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right) \end{gathered}$$

and adding these two results in $f_{\tilde{\tau }}\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)=f_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)$ for all $l,k,s$, hence $\mathit{\tau }=\tilde{\mathit{\tau }}$. It further implies $\tilde{\mathit{\alpha }}=\mathit{\alpha }$.

D. Proof of Theorem 2

According to the KKT condition, the solution to the constrained profile likelihood ${\widehat{\mathit{\theta }}}_n$ satisfies the first order condition that

$$\frac{\partial l\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)}{\partial \mathit{\theta }}+\left\{\frac{\partial \mathbf{\text{C}}{\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)}^{\top }}{\partial \mathit{\theta }}\right\}\mathit{\lambda }=0.$$ (11)

where $\partial l\left({\widehat{\mathit{\theta }}}_n\right)/\partial \mathit{\theta }\in {\mathbf{\text{R}}}^{\left|\mathit{\theta }\right|},\partial \mathbf{\text{C}}{\left({\widehat{\mathit{\theta }}}_n\right)}^{\top }/\partial \mathit{\theta }\in {\mathbf{\text{R}}}^{\left|\mathit{\theta }\right|\times \left|C\right|},\left|\mathit{\theta }\right|$ is the length of $\mathit{\theta }$ and $\left|C\right|$ is the number of equality constraints. Interestingly, (11) implies that $\partial l\left({\widehat{\mathit{\theta }}}_n\right)/\partial \mathit{\theta }$ is in the column space of $\partial \mathbf{\text{C}}{\left({\widehat{\mathit{\theta }}}_n\right)}^{\top }/\partial \mathit{\theta }$. Thus, as long as $\left|\mathit{\theta }\right|>\left|C\right|$, we can use any differentiable function $\mathbf{\text{U}}\left(\mathit{\theta }\right):𝚯\mapsto {\mathbf{\text{R}}}^{\left|\mathit{\theta }\right|\times \left|C\right|}$ that satisfies ${\mathbf{\text{U}}}^{\top }\left(\mathit{\theta }\right)\partial \mathbf{\text{C}}(\mathit{\theta }{)}^{\top }/\partial \mathit{\theta }=0,{\mathbf{\text{U}}}^{\top }(\mathit{\theta }\left)\mathbf{\text{U}}\right(\mathit{\theta })=\mathbf{\text{I}}$ for any $\mathit{\theta }\in 𝚯$, and we will automatically have

$${\mathbf{\text{U}}}^{\top }\left({\widehat{\mathit{\theta }}}_n\right)\frac{\partial l\left({\widehat{\mathit{\theta }}}_n\right)}{\partial \mathit{\theta }}=\mathbf{0}.$$ (12)

For sufficiently large $n$, taking Taylor expansion of (12) about ${\widehat{\mathit{\theta }}}_n$ at ${\mathit{\theta }}_0$ gives us

$$\begin{gathered} \mathbf{0}={\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\left[\frac{1}{\sqrt{n}}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial {\mathit{\theta }}^{\top }}+\frac{1}{n}\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)+O_p\left\{\sqrt{n}{‖{\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0‖}^2\right\}\right] \\ ={\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\frac{1}{\sqrt{n}}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial {\mathit{\theta }}^{\top }}+{\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\frac{1}{n}\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)+o_p\left\{\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)\right\} \\ ={\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\frac{1}{\sqrt{n}}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial {\mathit{\theta }}^{\top }}+\left\{{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)+{\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)}^{\top }\frac{\partial {\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\right\}\frac{1}{n}\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)+o_p\left\{\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)\right\} \\ ={\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\frac{1}{\sqrt{n}}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}+{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)\frac{1}{n}\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)+o_p\left\{\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)\right\}. \end{gathered}$$ (13)

Following the theories in Crowder (1984); Stoica and Ng (1998); Moore et al. (2008), let $\mathit{\psi }\left(t\right):\mathbf{\text{R}}\mapsto 𝚯$ be a continuous differentiable map representing the feasible arc and $\mathit{\psi }\left(0\right)={\mathit{\theta }}_0,\mathit{\psi }(1/n)={\widehat{\mathit{\theta }}}_n,\forall n$. Then we have

$${\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0=\mathit{\psi }\left(\frac{1}{n}\right)-\mathit{\psi }\left(0\right)={\left.\frac{1}{n}\frac{d\mathit{\psi }\left(t\right)}{dt}\right|}_{t=\frac{1}{n^{*}}},$$

for some $0<1/n^{*}<1/n$. Note that $\mathbf{\text{C}}\left\{\mathit{\psi }\right(t\left)\right\}=\mathbf{0}$ for all $t$. Therefore $\mathbf{\text{C}}\left\{\mathit{\psi }\left(1/n^{*}\right)\right\}=0$ and

$$0={\left.\frac{\partial \mathbf{\text{C}}\left\{\mathit{\psi }\right(t\left)\right\}}{\partial t}\right|}_{t=1/n^{*}}={\left.\frac{\partial \mathbf{\text{C}}\left\{\mathit{\psi }\right(t\left)\right\}}{\partial \mathit{\psi }}\frac{d\mathit{\psi }\left(t\right)}{dt}\right|}_{t=1/n^{*}}.$$

This implies $d\mathit{\psi }\left(1/n^{*}\right)/dt$ is in the column space of $U\left({\mathit{\theta }}^{*}\right)$, where ${\mathit{\theta }}^{*}=\mathit{\psi }\left(1/n^{*}\right)$, i.e. $d\mathit{\psi }\left(1/n^{*}\right)/dt=\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right){\mathbf{\text{Q}}}_n$ for some ${\mathbf{\text{Q}}}_n\in {\mathbf{\text{R}}}^{\left|C\right|}$. Hence

$${\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0=\frac{1}{n}\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right){\mathbf{\text{Q}}}_n.$$ (14)

Inserting (14) into (13), we have

$$\mathbf{0}={\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\frac{1}{\sqrt{n}}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}+\frac{1}{n}{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)\frac{1}{n}\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\sqrt{n}\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right){\mathbf{\text{Q}}}_n+o_p\left\{\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)\right\}.$$

First, under regular condition (Stoica and Ng, 1998; Moore et al., 2008), $-\frac{1}{n}{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right)$ is invertible for sufficient large $n$ with ${\mathit{\theta }}^{\mathbf{*}}$ being in a neighborhood of ${\mathit{\theta }}_0$, so

$${\mathbf{\text{Q}}}_n={\left\{-\frac{1}{\sqrt{n}}{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)\frac{1}{n}\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\mathbf{\text{U}}\left({\mathit{\theta }}^{\mathbf{*}}\right)\right\}}^{-1}{\mathbf{\text{U}}}^{\top }\left({\widehat{\mathit{\theta }}}_n\right)\frac{1}{\sqrt{n}}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}+o_p\left\{n\left({\widehat{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)\right\}.$$ (15)

Combining (14) and (15), we get

$$\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)=\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right){\left\{-\frac{1}{n}{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)\frac{{\partial }^{2}l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top }}\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right)\right\}}^{-1}\frac{1}{\sqrt{n}}{\mathbf{\text{U}}}^{\top }\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}+o_p\left\{\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)\right\}.$$

Next, we show the limiting distribution of $n^{-1/2}\partial l\left({\mathit{\theta }}_0\right)/\partial \mathit{\theta }$. The variance of the score function is

$$\begin{gathered} \text{var}\left\{n^{-1/2}\frac{\partial l\left({\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\right\} \\ =\sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}\text{var}\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\} \\ =\sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}E{\left[\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}-E\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}\right]}^2 \\ =\sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}E\left[{\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}}^{\otimes 2}\right] \end{gathered}$$ (16)

$$-2\sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}E\left[\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}\Phi \left({\mathit{\theta }}_0;i,s\right)\right]$$ (17)

$$+\sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}E\left\{\Phi {\left({\mathit{\theta }}_0;i,s\right)}^{\otimes 2}\right\}$$ (18)

where $\Phi \left({\mathit{\theta }}_0;i,s\right)=E\left\{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)/\partial \mathit{\theta }\mid Y=i,S=s\right\}$. Term (16) is

$$\begin{gathered} \sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}E\left[{\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}}^{\otimes 2}\right] \\ =\sum _{\mathit{\text{is}}}\frac{n_{+s++}}{n}{\mu }_{\mathit{\text{is}}}\sum _k\sum _l{\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}}^{\otimes 2}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left(z_l\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _i\sum _k\sum _l{\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}}^{\otimes 2}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)h\left({\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _k{E}_{Y,Z}^{*}\left[{\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}}^{\otimes 2}\right]h\left({\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}E\left[\left.{\text{var}}_{Y,Z}^{*}\left\{\left.\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\right|\mathbf{\text{X}}\right\}h(\mathbf{\text{X}},s)\right|s\right] \end{gathered}$$

with $h\left({\mathbf{\text{x}}}_k,s\right)=\sum _{\mathit{\text{il}}}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{\mu }_{\mathit{\text{is}}}{f}_{\tau }\left({\mathbf{\text{z}}}_l\mid {\mathbf{\text{x}}}_k,s\right)$. The first equality is because of ${\mu }_{\mathit{\text{is}}}P(Y=i\mid s)n_{+s++}=n_{\mathit{\text{is}}++}$. Notation ${\text{var}}_{Y,Z}^{*}$ is the variance of the random variable $Y,Z\mid \mathbf{\text{X}},s$ defined by $P_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })$. Similarly, term (17) can be rewritten as

$$\begin{gathered} \sum _{\mathit{\text{is}}}\frac{n_{\mathit{\text{is}}++}}{n}E\left[\left\{\left.\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right|Y=i,S=s\right\}\Phi \left({\mathit{\theta }}_0;i,s\right)\right] \\ =\sum _s\sum _i\frac{n_{+s++}}{n}{\mu }_{\mathit{\text{is}}}\sum _k\sum _l\left\{\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right\}\Phi \left({\mathit{\theta }}_0;i,s\right)P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\mathit{\tau }}\left(z_l\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _i\sum _k\sum _l\left\{\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right\}\Phi \left({\mathit{\theta }}_0;i,s\right)P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left(z_l\mid {\mathbf{\text{x}}}_k,s\right){\mu }_{\mathit{\text{is}}}{\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _k\sum _i\sum _l\left\{\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right\}\Phi \left({\mathit{\theta }}_0;i,s\right)P_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)h\left({\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _k{E}_{Y,Z}^{*}\left[\left.\left\{\frac{\partial }{\partial \mathit{\theta }}\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)\right\}\Phi \left({\mathit{\theta }}_0;i,s\right)\right|\mathbf{\text{X}}={\mathbf{\text{x}}}_k,s\right]h\left({\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}, \end{gathered}$$

where $E_{Y,Z}^{*}$ is the expectation of the random variable $Y,Z\mid \mathbf{\text{X}},s$ defined by $P_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })$. It can be further rewritten as

$$\begin{gathered} \sum _s\frac{n_{+s++}}{n}\sum _k{E}_Y^{*}\left\{\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)\Phi \left({\mathit{\theta }}_0;i,s\right)\mid \mathbf{\text{X}}={\mathbf{\text{x}}}_k,s\right\}h\left({\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _k\sum _i\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)\Phi \left({\mathit{\theta }}_0;i,s\right)h\left({\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}{\text{pr}}^{*}\left(Y=i\mid s,\mathbf{\text{X}}={\mathbf{\text{x}}}_k\right) \\ =\sum _s\frac{n_{+s++}}{n}\sum _k\sum _i\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)\Phi \left({\mathit{\theta }}_0;i,s\right){\pi }_{\mathit{\text{sk}}}\text{pr}\left(Y=i\mid s,\mathbf{\text{X}}={\mathbf{\text{x}}}_k\right){\mu }_{\mathit{\text{is}}} \\ =\sum _s\frac{n_{+s++}}{n}\sum _i{\mu }_{\mathit{\text{is}}}\text{pr}(Y=i\mid s)\Phi \left({\mathit{\theta }}_0;i,s\right)\sum _k\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)\frac{{\pi }_{\mathit{\text{sk}}}\text{pr}\left(Y=i\mid s,\mathbf{\text{X}}={\mathbf{\text{x}}}_k\right)}{\text{pr}(Y=i\mid s)} \\ =\sum _s\frac{n_{+s++}}{n}\sum _i{\mu }_{\mathit{\text{is}}}\text{pr}(Y=i\mid s)\Phi \left({\mathit{\theta }}_0;i,s\right)E\left\{\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)\mid Y=i,s\right\} \\ =\sum _s\frac{n_{+s++}}{n}\sum _i\frac{n_{\mathit{\text{is}}++}}{n_{+s++}}\Phi \left({\mathit{\theta }}_0;i,s\right)E\left\{\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)\mid Y=i,s\right\} \\ =\sum _s\sum _i\frac{n_{\mathit{\text{is}}++}}{n}\Phi {\left({\mathit{\theta }}_0;i,s\right)}^{\otimes 2} \end{gathered}$$

where $\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)=E_Z\left\{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)/\partial \mathit{\theta }\mid Y,\mathbf{\text{X}},s\right\}=E_Z^{*}\left\{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)/\partial \mathit{\theta }\mid Y,\mathbf{\text{X}},s\right\}$. The first equality applies the law of total expectation. The second equality is because of ${\text{pr}}^{*}\left(Y=i\mid s,\mathbf{\text{X}}={\mathbf{\text{x}}}_k\right)=\text{pr}(Y=i\mid \left.s,\mathbf{\text{X}}={\mathbf{\text{x}}}_k\right){\mu }_{\mathit{\text{is}}}/h\left({\mathbf{\text{x}}}_k,s\right)$. The second last equality applies the definition of ${\mu }_{\mathit{\text{is}}}$ and the last equality is because of the definition of $\mathbf{\text{W}}\left({\mathit{\theta }}_0;i,k,s\right)$. The fact that the probability of $Z\mid Y,\mathbf{\text{X}},s$ defined by $P_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}$ is equivalent to that defined by $P_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })$ can be shown easily by expanding ${\text{pr}}^{*}(Z\mid Y,\mathbf{\text{X}},s)$. Similarly, term (18) can be also rewritten as $\sum _s\sum _i{n}_{\mathit{\text{is}}++}\Phi {\left({\mathit{\theta }}_0;i,s\right)}^{\otimes 2}/n$.

Therefore, $\text{var}\left\{n^{-1/2}\partial l\left({\mathit{\theta }}_0\right)/\partial \mathit{\theta }\right\}$ converges to $𝒥\left({\mathit{\theta }}_0\right)\equiv 𝒥\left({\mathit{\theta }}_0\right)-\Omega \left({\mathit{\theta }}_0\right)$ where

$$\begin{gathered} 𝒥\left({\mathit{\theta }}_0\right)=\sum _s{r}_{+s}E\left[\left.{\text{var}}_{Y,Z}^{*}\left\{\left.\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\right|\mathbf{\text{X}}\right\}h(\mathbf{\text{X}},s)\right|s\right], \\ \Omega \left({\mathit{\theta }}_0\right)=\sum _{\mathit{\text{is}}}{r}_{\mathit{\text{is}}}E{\left[\left.E\left\{\left.\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\right|X\right\}\right|Y=i,s\right]}^{\otimes 2}. \end{gathered}$$

Last, let $P_{\mathit{\text{il}}}^{*}(k,s;\mathit{\theta })=e^{{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}/\sum _{\mathit{\text{il}}}{e}^{{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}$ with ${\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })=\text{log}\left\{P_{i{\mathbf{\text{X}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left(z_l\mid {\mathbf{\text{X}}}_k,s\right){\mu }_{\mathit{\text{is}}}\right\}$, by chain rule and algebra,

$$\begin{gathered} -n^{-1}{\partial }^{2}l\left({\mathit{\theta }}_0\right)/\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top } \\ =-n^{-1}\sum _{\mathit{\text{is}}kl}{\partial }^2\text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)/\partial \mathit{\theta }\partial {\mathit{\theta }}^{\top } \\ =-n^{-1}\sum _{\mathit{\text{is}}kl}\left\{\frac{{\partial }^2{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial {\mathit{\theta }}^2}-\sum _{\mathit{\text{il}}}\frac{e^{{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial }^2{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })/\partial {\mathit{\theta }}^2}{{\sum }_{l^{\prime }{l}^{\prime }}{e}^{{\zeta }_{i^{\prime }{i}^{\prime }}(k,s;\mathit{\theta })}}\right\} \\ -n^{-1}\sum _{\mathit{\text{is}}kl}\left[\sum _{\mathit{\text{il}}}\frac{e^{{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\left\{\partial {\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })/\partial \mathit{\theta }\right\}}^{\otimes 2}}{{\sum }_{l^{\prime }{l}^{\prime }}{e}^{{\zeta }_{i^{\prime }{i}^{\prime }}(k,s;\mathit{\theta })}}-{\left\{\sum _{\mathit{\text{il}}}\frac{e^{{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}\partial {\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })/\partial \mathit{\theta }}{{\sum }_{l^{\prime }{l}^{\prime }}{e}^{{\zeta }_{i^{\prime }{i}^{\prime }}(k,s;\mathit{\theta })}}\right\}}^{\otimes 2}\right] \\ =-n^{-1}\sum _{\mathit{\text{is}}kl}\left[\frac{{\partial }^2{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial {\mathit{\theta }}^2}-E_{Y,Z}^{*}\left\{\left.\frac{{\partial }^2{\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial {\mathit{\theta }}^2}\right|\mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right\}\right] \\ -n^{-1}\sum _{\mathit{\text{is}}kl}\left(E_{Y,Z}^{*}\left[\left.{\left\{\frac{\partial {\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial \mathit{\theta }}\right\}}^{\otimes 2}\right|\mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right]-{\left[E_{Y,Z}^{*}\left\{\left.\frac{\partial {\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial \mathit{\theta }}\right|\mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right\}\right]}^{\otimes 2}\right) \\ =o_p\left(1\right)-n^{-1}\sum _{\mathit{\text{is}}kl}{\text{var}}_{Y,Z}^{*}\left\{\left.\frac{\partial {\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })}{\partial \mathit{\theta }}\right|\mathbf{\text{X}}={\mathbf{\text{x}}}_k,S=s\right\} \\ \to -𝒥\left({\mathit{\theta }}_0\right). \end{gathered}$$

The last convergence is because of the definition of ${\zeta }_{\mathit{\text{il}}}(k,s;\mathit{\theta })$

When $n\to \infty$, we have $\mathbf{\text{U}}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n\right)\to \mathbf{\text{U}}\left({\mathit{\theta }}_0\right)$ and $\mathbf{\text{U}}\left({\mathit{\theta }}^{*}\right)\to \mathbf{\text{U}}\left\{\mathit{\psi }\right(0\left)\right\}=\mathbf{\text{U}}\left({\mathit{\theta }}_0\right)$ by consistency of ${\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n$ and continuity of $\mathbf{\text{U}}$. Thus, $\sqrt{n}\left({\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_n-{\mathit{\theta }}_0\right)$ converges to a normal distribution with mean zero and variance $\mathbf{\text{V}}\left({\mathit{\theta }}_0\right)𝒥\left({\mathit{\theta }}_0\right){\mathbf{\text{V}}}^{\top }\left({\mathit{\theta }}_0\right)$, where

$$\mathbf{\text{V}}\left({\mathit{\theta }}_0\right)=\mathbf{\text{U}}\left({\mathit{\theta }}_0\right){\left[{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)𝒥\left({\mathit{\theta }}_0\right)\mathbf{\text{U}}\left({\mathit{\theta }}_0\right)\right]}^{-1}{\mathbf{\text{U}}}^{\top }\left({\mathit{\theta }}_0\right)$$

To compare the constrained model and the unconstrained version, the difference

$${𝒥}^{-1}\left({\mathit{\theta }}_0\right)𝒥\left({\mathit{\theta }}_0\right){𝒥}^{-1}\left({\mathit{\theta }}_0\right)-\mathbf{\text{V}}\left({\mathit{\theta }}_0\right)𝒥\left({\mathit{\theta }}_0\right){\mathbf{\text{V}}}^{\top }\left({\mathit{\theta }}_0\right)$$

is positive definite, because it is the inverse of the upper-left block of the positive definite matrix

$${\left(\begin{array}{cc}{𝒥}^{-1}\left({\mathit{\theta }}_0\right)𝒥\left({\mathit{\theta }}_0\right){𝒥}^{-1}\left({\mathit{\theta }}_0\right)& \mathbf{\text{V}}\left({\mathit{\theta }}_0\right)\\ {\mathbf{\text{V}}}^{\top }\left({\mathit{\theta }}_0\right)& {𝒥}^{-1}\left({\mathit{\theta }}_0\right)\end{array}\right)}^{-1},$$

where ${𝒥}^{-1}\left({\mathit{\theta }}_0\right)𝒥\left({\mathit{\theta }}_0\right){𝒥}^{-1}\left({\mathit{\theta }}_0\right)$ is the variance of the unconstrained MLE. This means the constrained estimator is more efficient than the unconstrained estimator unless $\mathbf{\text{U}}$ is identical, that is there is no the active constraint thus does not contribute any additional knowledge to estimating $\mathit{\theta }$.

To further look at the property of $𝒥\left(\mathit{\theta }\right)$, we decompose $\Omega \left(\mathit{\theta }\right)$ as follows for any $\mathit{\theta }$.

$$\begin{gathered} \Omega \left(\mathit{\theta }\right) \\ =\sum _{\mathit{\text{is}}}{r}_{\mathit{\text{is}}}E{\left\{\left.\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\right|Y=i,s\right\}}^{\otimes 2} \\ =\sum _{\mathit{\text{is}}}{r}_{\mathit{\text{is}}}{\left\{\sum _{kl}\frac{\partial \text{log}{P}_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}\frac{\text{pr}(Y=i,\mathbf{\text{X}},Z\mid S=s)}{\text{pr}(Y=i\mid S=s)}\right\}}^{\otimes 2} \\ =\sum _{\mathit{\text{is}}}{r}_{\mathit{\text{is}}}{\left(e^{{\mu }_s}\frac{n_{0s++}}{n_{1s++}}+1\right)}^2{\left\{\sum _{kl}\frac{1}{P_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}\frac{\partial P_{\mathit{\text{il}}}^{*}\left(k,s;{\mathit{\theta }}_0\right)}{\partial \mathit{\theta }}{P}_{i{\mathbf{\text{x}}}_k{\mathbf{\text{z}}}_{l}s}^{*}{f}_{\tau }\left(z_l\mid {\mathbf{\text{x}}}_k,s\right){\pi }_{\mathit{\text{sk}}}\right\}}^{\otimes 2} \\ \to \sum _s{\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)}^2\sum _i{r}_{\mathit{\text{is}}}{\left\{\sum _{k}h\left({\mathbf{\text{x}}}_k,s\right)\frac{\partial }{\partial \mathit{\theta }}\sum _l{P}_{\mathit{\text{il}}}^{*}(k,s)\right\}}^{\otimes 2} \\ =\sum _s{\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)}^2\sum _i{r}_{\mathit{\text{is}}}{\left[E\left\{h(\mathbf{\text{X}},s)\frac{\partial }{\partial \mathit{\theta }}{\text{pr}}^{*}(Y=i\mid \mathbf{\text{X}},s)\right\}\right]}^{\otimes 2} \\ =\sum _s{\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)}^2{r}_{+s}{\left[E\left\{h(\mathbf{\text{X}},s)\frac{\partial }{\partial \mathit{\theta }}{\text{pr}}^{*}(Y=1\mid \mathbf{\text{X}},s)\right\}\right]}^{\otimes 2}. \end{gathered}$$

For $𝒥\left(\mathit{\theta }\right)$, let $\mathbf{\text{K}}=\sum _s{\mathbf{\text{K}}}_s$ and ${\mathbf{\text{K}}}_s=(\underset{|\mathit{\theta }\setminus \mu |+s-1}{\underbrace{\mathrm{0,0},\dots ,0}},\left(e^{{\mu }_s}{r}_{0s}/r_{1s}+1\right)r_{+s}^{-1/2},\underset{M-s}{\underbrace{\mathrm{0,0},\dots ,0}})$ where $|\mathit{\theta }\setminus \mathit{\mu }|$ is the number of parameters in $\mathit{\theta }$ excluding $\mathit{\mu }$. Then we have

$$\begin{gathered} {𝒥}_s\left(\theta \right){\text{K}}_s \\ =r_{+s}\left(\sum _{ikl}{\left\{\frac{\partial {\zeta }_{\mathit{\text{il}}}\left(k,s;\theta \right)}{\partial \theta }\right\}}^{\otimes 2}{\text{K}}_{s}h\left({\text{x}}_k,s\right)P_{\mathit{\text{il}}}^{\text{*}}\left(k,s\right){\pi }_{\mathit{\text{sk}}}\left.-\left[\sum _{ikl}\left\{\frac{\partial {\zeta }_{\mathit{\text{il}}}\left(k,s;\theta \right)}{\partial \theta }\right\}h\left({\text{x}}_k,s\right)P_{\mathit{\text{il}}}^{\text{*}}\left(k,s\right){\pi }_{\mathit{\text{sk}}}\right]\left[\sum _{ikl}\left\{\frac{\partial {\zeta }_{\mathit{\text{il}}}\left(k,s;\theta \right)}{\partial \theta }\right\}{\text{K}}_{s}h\left({\text{x}}_k,s\right)P_{\mathit{\text{il}}}^{\text{*}}\left(k,s\right){\pi }_{\mathit{\text{sk}}}\right]\right)\right. \\ =\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)r_{+s}^{-1/2}\left(\sum _{ikl}\left\{\frac{\partial {\zeta }_{\mathit{\text{il}}}\left(k,s;\theta \right)}{\partial \theta }\right\}ih\left({\text{x}}_k,s\right)P_{\mathit{\text{il}}}^{\text{*}}\left(k,s\right){\pi }_{\mathit{\text{sk}}}\right.\left.-\left[\sum _{ikl}\left\{\frac{\partial {\zeta }_{\mathit{\text{il}}}\left(k,s;\theta \right)}{\partial \theta }\right\}h\left({\text{x}}_k,s\right)P_{\mathit{\text{il}}}^{\text{*}}\left(k,s\right){\pi }_{\mathit{\text{sk}}}\right]\left[\sum _{ikl}ih\left({\text{x}}_k,s\right)P_{\mathit{\text{il}}}^{\text{*}}\left(k,s\right){\pi }_{\mathit{\text{sk}}}\right]\right) \\ =\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)r_{+s}^{-1/2}\sum _k\left[\sum _l\left\{\frac{\partial {\zeta }_{1l}\left(k,s;\theta \right)}{\partial \theta }{P}_{1l}^{\text{*}}\left(k,s;{\theta }_0\right)\right\}-{\text{pr}}^{\text{*}}\left(Y=1\mid {\text{x}}_k,s\right)\sum _{\mathit{\text{il}}}\frac{\partial {\zeta }_{\mathit{\text{il}}}\left(k,s;\theta \right)}{\partial \theta }{P}_{\mathit{\text{il}}}^{\text{*}}\left(k,s;{\theta }_0\right)\right]h\left(\text{X},s\right){\pi }_{\mathit{\text{sk}}} \\ =\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)r_{+s}^{-1/2}\sum _k\frac{{\text{pr}}^{\text{*}}\left(Y=1\mid \text{X},s\right)}{\partial \theta }h\left(\text{X},s\right){\pi }_{\mathit{\text{sk}}} \\ =\left(e^{{\mu }_s}\frac{r_{0s}}{r_{1s}}+1\right)r_{+s}^{-1/2}E\left\{h\left(\text{X},s\right)\frac{\partial }{\partial \theta }{\text{pr}}^{\text{*}}\left(Y=1\mid \text{X},s\right)\right\}. \end{gathered}$$

Combine $\Omega \left(\mathit{\theta }\right)$ and $\sum _s{\partial }_s\left(\mathit{\theta }\right){\mathbf{\text{K}}}_s=𝒥\left(\mathit{\theta }\right)\sum _s{\mathbf{\text{K}}}_s$, we have $\Omega \left(\mathit{\theta }\right)=𝒥\left(\mathit{\theta }\right)\left(\sum _s{\mathbf{\text{K}}}_s{\mathbf{\text{K}}}_s^{\top }\right)𝒥(\mathit{\theta }{)}^{\top }$. Therefore the asymptotic variance matrix of unconstrained MLE can be further simplified as ${𝒥}^{-1}\left({\mathit{\theta }}_0\right)-\sum _s{\mathbf{\text{K}}}_s{\mathbf{\text{K}}}_s^{\top }$.