Correlational analyses of biomarkers that are harmonized through a bridging study due to measurement errors

Abstract

Evaluating correlations between disease biomarkers and clinical outcomes is crucial in biomedical research. During the early stages of many chronic diseases, changes in biomarkers and clinical outcomes are often subtle. A major challenge to detecting subtle correlations is that studies with large sample sizes are usually needed to achieve sufficient statistical power. This challenge is even greater when biofluid and imaging biomarker data are used because the required procedures are burdensome, perceived as invasive, and/or expensive, limiting sample sizes in individual studies. Combining data across multiple studies may increase statistical power, but biomarker data may be generated using different assay platforms, scanner types, or processing protocols, which may affect measured biomarker values. Therefore, harmonizing biomarker data is essential to combining data across studies. Bridging studies involve re-processing of a subset of samples or imaging scans to evaluate how biomarker values vary by studies. This presents an analytic challenge on how to best harmonize biomarker data across studies to allow unbiased and optimal estimates of their correlations with standardized clinical outcomes. We conceptualize that a latent biomarker underlies the observed biomarkers across studies, and propose a novel approach that integrates the data in the bridging study with the study-specific biomarker data for estimating the biological correlations between biomarkers and clinical outcomes. Through extensive simulations, we compare our method to several alternative methods/algorithms often used to estimate the correlations. Finally, we demonstrate the application of this methodology to a real-world multi-center Alzheimer disease (AD) biomarker study to correlate cerebrospinal fluid biomarker concentrations with cognitive outcomes.

1. Introduction

A major challenge in biomedical research is that studies with large sample sizes are required for sufficient statistical power to detect biological associations between disease biomarkers and clinical outcomes. This is particularly true for chronic diseases such as Alzheimer disease (AD), which includes a very long asymptomatic stage during which changes in biomarkers and clinical outcomes are often subtle. The need for large sample sizes is problematic, because many of the procedures for collecting biofluid samples and imaging scans are burdensome, perceived as invasive, and/or expensive, resulting in limited sample sizes for individual studies. Whereas clinical outcomes are typically standardized and reliably measured across studies, biofluid and imaging biomarkers are often measured using different assays, scanners, and protocols across studies. Therefore, accounting for differences in biomarker data across studies is crucial to using these multiple datasets to enable greater power to examine correlations between biomarkers and clinical outcomes. A major analytic challenge is how to harmonize and integrate biomarker data from multiple studies to make unbiased estimates of the biological correlations between these biomarkers and clinical outcomes, as demonstrated by the following motivating example.

2. A Motivating Example

Alzheimer disease (AD) is an irreversible neurodegenerative disease that currently affects ~6 million mostly older Americans, and this number is projected to more than double by 2050¹, making AD a major public health crisis. Amyloid plaques are a hallmark of AD brain pathology, and concentrations of amyloid proteins in the cerebrospinal fluid (CSF) reflect the presence of amyloid plaques². There are two major forms of amyloid-β (Aβ) in the CSF, the 42-residue (Aβ42) and the 40-residue Aβ40 that have been linked with AD^2,3. They differ because Aβ42 has two extra residues at the C-terminus, and the amyloid plaques in AD brains consist of mostly Aβ42 and some plaques contain only Aβ42, even though Aβ40 concentration is several-fold more than Aβ42.

Whereas much have been reported about how these CSF biomarkers may aid diagnosing the disease and tracking its progression, it has been well reported that fundamental measurement issues remain with these biomarkers. Even with the same assay platforms, considerable measurement variations on the same CSF samples may exist across different labs or studies^7–9.

The Study of Race to Understand Alzheimer Biomarkers (SORTOUT-AB; NIH/NIA R01 AG067505) is a multi-center biomarker study the includes primarily non-Hispanic White and Black or African American individuals. CSF and imaging biomarker data are harmonized across multiple studies to enable adequate power to determine whether these biomarkers are consistently correlated with clinical and cognitive outcomes in non-Hispanic White and Black or African American individuals. SORTOUT-AB leverages existing CSF biomarker studies conducted by both Washington University (WU) and University of Pennsylvania (UPenn) Alzheimer Disease Research Centers (ADRC). Before the inception of SORTOUT-AB, both ADRCs had already measured Aβ42 and Aβ40 concentrations in their CSF samples. Re-analyzing all CSF samples from both centers together was impractical because of the expense and limited quantity of CSF available. Therefore, a bridging study has been designed that allowed a subset of 69 “bridging” samples from a total of 295 samples collected at UPenn to be shared with WU ADRC and analyzed using the same conditions used for analysis of the WU samples (with a total of 1176 samples). These samples were collected over the time windows from 1998 to 2022 and processed in 2020-2022, and the bridging samples were processed at WU ADRC in 2021-2022.

The key question was how to use values of Aβ42 and Aβ40 concentrations in the bridging samples that were measured at both UPenn and WU, in combination with the already available biomarker data from both centers, to correlate these biomarkers with clinical and cognitive outcomes. Several analytic challenges emerge when both retrospectively collected biomarker data and prospectively collected bridging data from a sub-cohort are integrated for joint analyses. First, how should measurement error be taken into account in the integrated analyses, given that most of the CSF samples were only processed within a single ADRC? Second, there are multiple intuitive approaches to link/impute biomarker data directly across studies, i.e., by simple regression models or other imputation methods. Standard correlational analyses can proceed after data across studies are directly linked or imputed. However, it is important to understand whether these methods introduce any bias in estimating the biological correlations. Third, what are the efficiency and statistical power for estimating and testing correlations between biomarkers and clinical outcomes when using the intuitive approaches, especially when compared against approaches that are designed to account for measurement error?

Using bridging samples to harmonize biomarker data between two studies is not unique to AD research, and many other fields have encountered very similar challenges. For example, the Vitamin D Pooling Project of Breast and Colorectal Cancer (VDPP)¹⁰ combined biomarker data from 17 studies to investigate the association between the circulating vitamin D (25(OH)D) and colorectal cancer. It measured 25(OH)D values in a subset of bridging samples, which it used to harmonize 25(OH)D values across studies^11–13. Surprisingly, the existing statistical literature is quite limited on harmonizing biomarker data across studies with bridging samples that can account for measurement error. Hence, we seek to rigorously examine how best to use bridging samples to harmonize datasets. In Section 3, we propose a new latent variable model to formally incorporate data from the bridging samples into statistical inferences for a correlation coefficient. We also discuss several intuitive and conventional methods for statistical inference on correlation. In Section 4, we conduct extensive simulations to compare the performance of the conventional methods and the new approach. We re-visit our motivating example in Section 5 to implement the methods to the real-world AD database of SORTOUT-AB to estimate the correlation coefficients between CSF amyloid biomarkers and cognitive performance and to compare the results. We finally conclude the paper with discussions and recommendations.

3. Statistical inferences on the correlation

Although we consider the case of two independent biomarker studies, similar methodologies can be extended to more than two studies. Each of the two studies measures the same biomarker, $B$ (i.e., the true concentration of a protein in CSF that can not be perfectly measured) from a randomly selected individual of the population under study. Different assays and protocols lead to heterogeneity in the measurements across the two studies. Let $B_k,k=1,2$, be the measurement of $B$ from study $k$. The goal is to integrate the biomarker data from both studies for a joint correlational analysis with a clinical outcome, $W$, which is standardized and can be reliably measured by both studies. Both studies agree to share a subset of their biofluid samples from their subjects with the other study to bridge the biomarker measurements, and hence the paired measurements ${(B}_1,B_2)$ obtained by both studies are only available for the bridging cohort. We assume that a bridging cohort (called cohort $C_{12}$) of size $n$ from both studies is randomly selected from the two study cohorts, resulting in bivariate data, ${(B}_{11},B_{21}){,(B}_{12},B_{22}),\dots ,{(B}_{1n},B_{2n}).$ Note that here $B_{kj},k=1,2,j=1,2,\dots$ denote the measurement from the j-th participant of the k-th study. In addition to the bridging cohort, assume that $B_1$ is already obtained from $n_1$ other subjects in Study 1 (called cohort $C_1$), $B_{1,n+1}{,B}_{1,n+2},\dots ,B_{1,n+n_1}$, and that $B_2$ is already obtained from $n_2$ other subjects in Study 2 (called cohort $C_2$), $B_{2,n+1}{,B}_{2,n+2},\dots ,B_{2{,n+n}_2}.$ We assume that the subjects in both studies are independent and that both cohorts of subjects are random samples from the same population. We further assume that each study has its unique assay platform and processing protocol that are the main sources of measurement errors, and hence study-specific biomarker measures are not directly comparable. A toy example data set is provided in Table 1 to visualize the specific data structure.

Table 1.

A toy dataset illustrating the typical data structure with biomarker measurements from two studies under a bridging design

Cohort	Sample ID	Study	Measurements of $B$ from study 1 (${\mathit{B}}_1$)	Measurements of $B$ from study 2 (${\mathit{B}}_2$)	Measurement of clinical outcome ($W$)
$C_{12}$ (Bridging)	1	Study2	x	x	x
$C_{12}$ (Bridging)	2	Study2	x	x	x
$C_{12}$ (Bridging)	3	Study2	x	x	x
$C_{12}$ (Bridging)	4	Study1	x	x	x
$C_{12}$ (Bridging)	5	Study1	x	x	x
$C_{12}$ (Bridging)	6	Study1	x	x	x
$C_1$ (Cohort 1)	7	Study1	x		x
$C_1$ (Cohort 1)	8	Study1	x		x
$C_1$ (Cohort 1)	9	Study1	x		x
$C_2$ (Cohort 2)	10	Study2		x	x
$C_2$ (Cohort 2)	11	Study2		x	x
$C_2$ (Cohort 2)	12	Study2		x	x

3.1. A latent variable model

We propose a latent variable model framework to make statistical inferences for the true correlation between $B$ and $W$, $Corr\left(B,W\right)=r$. We conceptualize that, at the subject level, the latent biomarker $B$ follows a normal distribution, and without loss of generality with a mean of 0, $B~N\left(0,{\sigma }^2\right).$ Note that in situations where raw biomarkers do not typically follow a normal distribution, some transformations (e.g., log) can often be used to approximate a normal distribution reasonably well.

We use the following latent variable model, similar to the measurement error model^14–15, to link the study-specific biomarker measurements, $B_1$ and $B_2$, with the true but unknown latent biomarker $B$,

$$B_k={\mu }_k+B+e_k,$$ (1)

where the measurement errors are assumed i.i.d., $e_k~N\left(0,{{\sigma }_e}^2\right)$. Hence, $Var\left(B_k\right)={\sigma }^2+{\sigma }_e^2$, for $k=1,2,$ and $Cov\left(B_1,B_2\right)={\sigma }^2$ under the usual assumption that $B$ and $e_k$ are independent. The correlation between $B_1$ and $B_2,{\rho }_{12}=\frac{{\sigma }^2}{{\sigma }^2+{\sigma }_e^2}$, is the well-known intraclass correlation coefficient (ICC). Note here that different means (${\mu }_k$) from the biomarker measured by different studies are allowed to reflect the fact that different assay platforms and processing protocols may contribute differentially at the mean level of the measurements, in addition to the measurement errors.

We further assume that the clinical outcome, $W$, together with $B_1$ and $B_2$, follow a trivariate normal distribution with a mean of $\mathit{\mu }=({\mu }_1,{\mu }_2,{\mu }_W)\prime$, and a covariance matrix of $\mathrm{\Sigma }$,

$$\left(\begin{array}{c}{B}_1\\ B_2\\ W\end{array}\right)~N\left(\begin{array}{cc}\mathbf{\mu },& \Sigma \end{array}\right).$$

Denote the variance of $W$ by ${\sigma }_W^2$, and let ${\rho }_k=Corr\left(B_k,W\right).$ Hence,

$$\Sigma =\left(\begin{array}{ccc}{\sigma }_e^2+{\sigma }^2& {\rho }_{12}({\sigma }_e^2+{\sigma }^2)& {\rho }_1{\sigma }_W\sqrt{{\sigma }_e^2+{\sigma }^2}\\ {\rho }_{12}({\sigma }_e^2+{\sigma }^2)& {\sigma }_e^2+{\sigma }^2& {\rho }_2\sqrt{{\sigma }_e^2+{\sigma }^2}\\ {\rho }_1{\sigma }_W\sqrt{{\sigma }_e^2+{\sigma }^2}& {\rho }_2{\sigma }_W\sqrt{{\sigma }_e^2+{\sigma }^2}& {\sigma }_W^2\end{array}\right).$$

Now we assume that $Cov\left(e_k,W\right)=0$. Then,

$${\rho }_k=Corr\left(B_k,W\right)=\frac{Cov(B_k,W)}{\sqrt{({\sigma }_e^2+{\sigma }^2){\sigma }_W^2}}=\frac{Cov(B,W)}{\sqrt{({\sigma }_e^2+{\sigma }^2){\sigma }_W^2}}=r\times \sqrt{{\rho }_{12}}.$$

Let ${\rho }_1={\rho }_2=\rho$ be the shared correlation. The true correlation $r$ between $B$ and $W$ can be derived as,

$$r=Corr\left(B,W\right)=\frac{Corr\left(B_k,W\right)}{\sqrt{{\rho }_{12}}}=\frac{\rho }{\sqrt{{\rho }_{12}}}.$$ (2)

Note that, although Equation (2) can also be derived from classical measurement error models, the estimation of true correlation (r) is different with a bridging design. This is because our approaches, as detailed in Sections below, must integrate biomarker data from multiple studies to estimate $\rho ,$ and only data from the bridging samples is available to estimate the ICC, ${\rho }_{12},$ whereas the standard measurement models typically referred to a study in which repeated measures were obtained on all subjects (i.e., without missing data). If $0<{\rho }_{12}<1$, then $Corr\left(B_k,W\right)\le r$. This implies that the error in the biomarker measurements attenuates the correlation between the biomarker and the clinical outcome. In fact, if ${\sigma }_e^2/{\sigma }^2=0,$ i.e., there is no measurement error, $ICC={\rho }_{12}=1$, then there is no attenuation on the correlation, $Corr\left(B_k,W\right)=r$. Further, a larger ${\sigma }_e^2/{\sigma }^2>0$ implies that the correlation $Corr\left(B_k,W\right)$ from the observed biomarker is further attenuated from the true $Corr\left(B,W\right)$ in magnitude. Also note, for testing $H_0:r=Corr\left(B,W\right)=0,$ it is mathematically equivalent to testing $H_0:\rho =0$.

3.1.1. Estimation of the true correlation

One way to estimate the true correlation, r, between the latent and the true biomarker $B$ and $W$ is to first estimate the shared correlation ${\rho }_k$ between each of the observed $B_k,k=1,2,$ and $W$. Specifically, let ${\widehat{\rho }}_1$ be the estimator to $\rho$ obtained from data in $C_1$ alone, and let ${\widehat{\rho }}_2$ be the estimator to $\rho$ obtained from data in $C_2$ alone, calculated as,

$${\widehat{\rho }}_k=\frac{{\sum }_{i=n+1}^{n+n_k}{(B}_{ki}-{\overline{B}}_k){(W}_i-{\overline{W}}_k)}{\sqrt{{\sum }_{i=n+1}^{n+n_k}{{(B}_{ki}-{\overline{B}}_k)}^2{\sum }_{n+1}^{n+n_k}{{(W}_i-{\overline{W}}_k)}^2}},k\text{=}1,2,$$ (3)

where

$${\overline{B}}_k=\frac{{\sum }_{i=n+1}^{n+n_k}{B}_{ki}}{n_k}$$

$${\overline{W}}_k=\frac{{\sum }_{i=n+1}^{n+n_k}{W}_{ki}}{n_k}.$$

Let $Z_k$ be the Fisher’s $Z$ transformed version of ${\widehat{\rho }}_k$¹⁶,

$$Z_k=0.5ln\frac{1+{\widehat{\rho }}_k}{1-{\widehat{\rho }}_k},$$ (4)

which can be approximated by a normal distribution with a mean of

$$z=0.5ln\frac{1+\rho }{1-\rho }$$ (5)

and a variance of

$${\sigma }_k^2=\frac{1}{n_k-3}.$$

Using the data from the bridging cohort $C_{12},\rho$ can also be estimated using the samples from each $B_k$, for $k=1,2$, as

$${\tilde{\rho }}_k=\frac{{\sum }_{i=1}^n{(B}_{ki}-{\tilde{B}}_k){(W}_i-{\tilde{W}}_k)}{\sqrt{{\sum }_i^n{{(B}_{ki}-{\tilde{B}}_k)}^2{\sum }_{i=1}^n{{(W}_i-{\tilde{W}}_k)}^2}},k\text{=}1,2,$$ (6)

where

$${\tilde{B}}_k=\frac{{\sum }_{i=1}^n{B}_{ki}}{n}$$

$${\tilde{W}}_k=\frac{{\sum }_{i=1}^n{W}_{ki}}{n}.$$

Given that

$$U_k=0.5ln\frac{1+{\tilde{\rho }}_k}{1-{\tilde{\rho }}_k}$$ (7)

can be approximated by a normal distribution with a mean of $z$ and a variance of

$${\delta }_k^2=\frac{1}{n-3},$$

an optimal estimator to $z$ within bridging cohort $C_{12}$ can be obtained by

$$U=0.5U_1+0.5U_2,$$

and the variance of U is

$$Var(U)=\frac{1}{2(n-3)}+0.5COV(U_1,U_2).$$ (8)

$COV(U_1,U_2)$ in Equation (8) can be estimated in multiple ways. First, Hotelling¹⁷ and Dunn and Clark¹⁸ derived it asymptotically as,

$$COV\left(U_1,U_2\right)=\frac{0.5{\rho }_1{\rho }_2\left({\rho }_1^2+{\rho }_2^2+{\rho }_{12}^2-1\right)-{\rho }_{12}\left({\rho }_1^2+{\rho }_2^2-1\right)}{\left(1-{\rho }_1^2\right)\left(1-{\rho }_2^2\right)\left(n-3\right)}.$$ (9)

Second, William¹⁹ and Dunn and Clark²⁰ proposed a finite or small sample (SS) approximation as below,

$$COV(U_1,U_2)=\frac{1}{n-3}-\frac{D}{\left(n-3\right)\left(1+{\rho }_{12}\right)}-\frac{{\left({\rho }_1{+\rho }_2\right)}^2{\left(1-{\rho }_{12}\right)}^3}{8\left(n-1\right)\left(1+{\rho }_{12}\right)},$$ (10)

where $D\text{=}1-{\rho }_1^2-{\rho }_2^2-{\rho }_{12}^2+2{\rho }_1{\rho }_2{\rho }_{12}.$

Finally, with $Z_1,Z_2,U,$ and their variances derived, an optimal unbiased estimator to the true correlation $\rho$ in Fisher’s Z scale that uses all available data from the three cohorts, $C_1,C_2$, and $C_{12}$, can be derived as a weighted average of $Z_1,Z_2,$ and $U$ in a linear form,

$$V=a{Z}_1+b{Z}_2+cU,$$ (11)

where the weights $a,b,$ and $c$ are derived to minimize $Var\left(V\right)$ over all choices under the constraints,

$$a\ge 0,b\ge 0,c\ge 0,a+b+c=1.$$

Note that

$$Var\left(V\right)=\frac{a^2}{n_1-3}+\frac{b^2}{n_2-3}+c^{2}Var\left(U\right),$$ (12)

where $Var\left(U\right)$ is given by Equation (8). It can be easily shown, similar to the fixed effect meta-analysis estimator (Equation (5) of Normand²¹), that the variance-minimizing weights $a,b,$ and $c$ are,

$$a=\frac{n_1-3}{{Var\left(U\right)}^{-1}+n_1+n_2-6},$$

$$b=\frac{n_2-3}{{Var\left(U\right)}^{-1}+n_1+n_2-6},$$

$$c=1-a-b=\frac{{Var\left(U\right)}^{-1}}{{Var\left(U\right)}^{-1}+n_1+n_2-6}.$$

An estimate to the shared correlation $\widehat{\rho }$ at the original scale is obtained via an application of the inverse Fisher transformation to $V$ in Equation (11): $\widehat{\rho }=F^{-1}\left(V\right)=\frac{e^{2V}-1}{1+e^{2V}}$. An estimate to ${\rho }_{12}$, the ICC, can be directly obtained by the correlation of $B_1$ and $B_2$ using the paired observations from the bridging cohort $C_{12}$ or by fitting the linear mixed effects models for repeated measures. With these two estimates, the true correlation between $B$ and $W$ at the original scale can be estimated as,

$$\widehat{r}=\frac{\widehat{\rho }}{\sqrt{{\widehat{\rho }}_{12}}}.$$ (13)

3.1.2. Variance and confidence interval

We continue to construct confidence interval (CI) to $r$. Once $\widehat{V}$ and $\widehat{Var}\left(V\right)$ are obtained after replacing the unknowns in Equations (11) and (12) by their respective estimates, a 95% CI to the shared correlation $\rho$ in Fisher’s Z scale is,

$$\widehat{V}\pm 1.96\sqrt{\widehat{Var}\left(V\right)}.$$ (14)

An application of the inverse Fisher transformation to the lower and upper limits of the 95% CI in Equation (14) above gives the 95% lower and upper CI limits in original scale for $\rho =r\sqrt{{\rho }_{12}},$ and then an application of Slutsky Theorem²² gives an asymptotic 95% confidence interval to r as

$$\frac{F^{-1}\left[\widehat{V}\pm 1.96\sqrt{\widehat{Var}\left(V\right)}\right]}{\sqrt{{\widehat{\rho }}_{12}}}.$$ (15)

The variance to the estimated r may also be obtained by the Delta method²³, and subsequently, a 95% CI can be constructed. Specifically, the variance of $\widehat{r}=\frac{F^{-1}(\widehat{V})}{\sqrt{{\widehat{\rho }}_{12}}}$ may be obtained by,

$$Var\left(\widehat{r}\right)=\left(\frac{4e^{2V}}{{(1+e^{2v})}^2\sqrt{{\widehat{\rho }}_{12}}},\frac{\mathbf{-}\mathbf{0.5}{F}^{-1}(\widehat{V})}{\sqrt{{\left({\widehat{\rho }}_{12}\right)}^3}}\right)\varnothing {\left(\frac{4e^{2V}}{{(1+e^{2v})}^2\sqrt{{\widehat{\rho }}_{12}}},\frac{\mathbf{-}\mathbf{0.5}{F}^{-1}(\widehat{V})}{\sqrt{{\left({\widehat{\rho }}_{12}\right)}^3}}\right)}^{\prime },$$

where the first vector on the right is the vector of derivatives, and the prime symbol denotes matrix transpose, and $\varnothing$ is the covariance matrix of ($\widehat{V},{\widehat{\rho }}_{12}$),

$$\varnothing =\left(\begin{array}{cc}{\sigma }_{\widehat{V}}^2& COV(\widehat{V},{\widehat{\rho }}_{12})\\ COV(\widehat{V},{\widehat{\rho }}_{12})& {\sigma }_{{\widehat{\rho }}_{12}}^2\end{array}\right).$$

${\sigma }_{\widehat{V}}^2$ is given by Equation (12), and

$${\sigma }_{{\widehat{\rho }}_{12}}^2=Var({\widehat{\rho }}_{12})=\frac{{(1-{{\widehat{\rho }}_{12}}^2)}^2}{(n-3)}.$$ (16)

$$COV(\widehat{V},{\widehat{\rho }}_{12})=COV\left(cU,{\widehat{\rho }}_{12}\right)=0.5\times c\times COV\left(U_1,{\widehat{\rho }}_{12}\right)+0.5\times c\times COV\left(U_2,{\widehat{\rho }}_{12}\right)=\frac{0.5c}{\left(1-{{\tilde{\rho }}_1}^2\right)}COV({\tilde{\rho }}_1,{\widehat{\rho }}_{12})+\frac{0.5c}{\left(1-{{\tilde{\rho }}_2}^2\right)}COV({\tilde{\rho }}_2,{\widehat{\rho }}_{12}),$$

where $c$ is used to construct $V$ in Equation (11), and it is well known that^17–18

$$COV({\tilde{\rho }}_1,{\widehat{\rho }}_{12})=\frac{0.5{\rho }_1{\rho }_{12}\left({\rho }_1^2+{\rho }_2^2+{\rho }_{12}^2-1\right)-{\rho }_2\left({\rho }_1^2+{\rho }_{12}^2-1\right)}{\left(n-3\right)},$$

and

$$COV({\tilde{\rho }}_2,{\widehat{\rho }}_{12})=\frac{0.5{\rho }_{12}{\rho }_2\left({\rho }_1^2+{\rho }_2^2+{\rho }_{12}^2-1\right)-{\rho }_1\left({\rho }_{12}^2+{\rho }_2^2-1\right)}{\left(n-3\right)}.$$

Hence, another 95% CI to r is

$$\widehat{r}\pm 1.96\sqrt{Var\left(\widehat{r}\right)}.$$ (17)

The proposed latent variable model-based correlation estimator to r as given in Equation (13) has two versions, depending on whether the large sample-based Equation (9) or the small sample-based Equation (10) is used for calculating $a,b,$ and $c$ in Equation (11). We label the two versions of estimator as “latent” involving Equation (9) and “latent_SS” involving Equation (10). Consequently, the use of Equation (9) or Equation (10) also leads to two versions of the associated variance and 95% CIs for the estimated true correlation. Besides the 95% CI in Equation (17) that is based on the Delta method, another 95% CI to r can be derived according to Equation (15) by applying the Slutsky theorem, for which the two versions were labeled as “latent_slutsky” and “latent_SS_slutsky”.

3.1.3. Hypothesis testing

We can also test the hypothesis involving the true correlation between the latent biomarker and the clinical outcome, for example, $H_0:r=Corr\left(B,W\right)=0,$ against the alternative hypothesis $H_a:r=Corr\left(B,W\right)\ne 0.$ We assume $0<{\rho }_{12}<1.$ Given that $H_0:r=Corr\left(B,W\right)=0$ is mathematically equivalent to $H_{0k}:{\rho =\rho }_k=Corr\left(B_k,W\right)=0$, and that $H_a:r=Corr\left(B,W\right)\ne 0$ is mathematically equivalent to $H_{ak}:\rho ={\rho }_k=r\sqrt{{\rho }_{12}}\ne 0.$ A test of $H_0$ at a significance level of $0<\alpha <1$ can be obtained by computing the Wald statistic

$$T=\frac{V}{\sqrt{\widehat{Var}\left(V\right)}},$$

which rejects the null hypothesis at the $\alpha$ level when

$$\left|T\right|>z_{\alpha /2}.$$

The power function of the test is

$$P\left(r,{\rho }_{12}\right)=\mathrm{\Phi }\left(-z_{\alpha /2}-\frac{z}{\sqrt{Var\left(V\right)}}\right)+1-\mathrm{\Phi }\left(z_{\alpha /2}-\frac{z}{\sqrt{Var\left(V\right)}}\right),$$ (18)

where

$$z=0.5ln\frac{1+r\sqrt{{\rho }_{12}}}{1-r\sqrt{{\rho }_{12}}},$$

and $Var\left(V\right)$ is obtained from Equation (12), and $z_{\alpha /2}$ is the upper $100(1-\alpha /2)\%$ percentile of the standard normal distribution whose cumulative distribution function is $\mathrm{\Phi }.$

3.1.4. Covariates

It is often important to estimate the true correlation after adjusting for the effect of important covariates, i.e., age in the SORTOUT-AB. Under the normality assumption, given the covariates, X, the adjusted correlation between the true biomarker and a clinical outcome can be obtained by considering the conditional distribution. Specifically, assuming a multivariate normal distribution,

$$\left(\begin{array}{c}{B}_1\\ B_2\\ \begin{array}{c}W\\ X\end{array}\end{array}\right)~N\left(\begin{array}{cc}\mathit{\mu },& \mathbf{\Sigma }\end{array}\right),$$

we can obtain

$$\left(\begin{array}{c}{B}_1\\ B_2\\ W\end{array}\right)|X=x~N\left(\begin{array}{cc}\mathit{\mu }|x,& \mathbf{\Sigma }|\end{array}x\right),$$

where $\mathit{\mu }|x$ is the three-dimensional vector of regression functions, and $\mathbf{\Sigma }|x$ is a function of $\mathrm{\Sigma }$. Our proposed inferences can be directly generalized to estimate the partial correlations, conditioning on X. The only adjustment in these derivations is that the conditional distribution of the Fisher-transformed estimate to each partial correlation ($Corr\left(B_k,W\right),k=1,2,Corr\left(B_1,B_2\right)$), given X, is now approximated by a normal distribution whose mean is the Fisher-transformed true partial correlation and whose variance is equal to $1/(S-q-3),$ where S is the sample size of the cohort used in the estimation (i.e., the sample size from one of the 3 cohorts, $\left(C_1,C_2,C_{12}\right)$), and q is the dimension of X ²⁴.

3.1.5. Non-Normal Data

Our proposed inferential results were based on the assumption of normal distributions, and normality is a strong parametric assumption. When data deviate severely from normal distributions, however, nonparametric correlation can be considered. Specifically, in Model (1), $B_k={\mu }_k+B+e_k,$ instead of using the original scales of the biomarkers, we can conceptualize a similar latent variable model that is based on the ranks of true biomarker $B$ and the study-specific biomarkers $B_k$ from a large population. This new conceptualization then allows the use of Spearman correlation with a standard clinical outcome $W$ which is based on the ranks of data from any bivariate distributions. Importantly, the same relationship as given in Equation (2), $r=\rho /\sqrt{{\rho }_{12}},$ remains valid when all the correlations in the Equation are Spearman. The asymptomatic normality of the rank-based statistics²⁵ support the use of Fisher transformation on each estimated Spearman correlation from non-normal data, which can also be approximated by a normal distribution whose mean is the Fisher-transformed true Spearman correlation and whose variance is equal to $1/(S-3)$ ^26–27, where S is the sample size of the cohort used in estimating the Spearman correlations (i.e., the sample size from one of the 3 cohorts, $\left(C_1,C_2,C_{12}\right)$). Therefore, our proposed inferences as described above can be genarlized to estimate and test the true Spearman correlation between the true biomarker $B$ and the clinical outcome, $W$.

3.2. Alternative methods and algorithms

In addition to the estimates to the true correlation we propose above (labeled as latent), for comparison purpose, we describe below several intuitive alternative methods and algorithms. Note that all the methods discussed below only differ from the above latent method on how the shared correlation (the numerator in Equation (13)) between the observed biomarker from both studies, i.e., $\rho ={\rho }_1={\rho }_2,$ is estimated. Once $\rho$ is estimated, the next step is the same, i.e., to estimate the true biological correlation, r, by Equation (13) that is based on the latent variable model, i.e., $\widehat{r}=\frac{\widehat{\rho }}{\sqrt{{\widehat{\rho }}_{12}}},$ where ${\widehat{\rho }}_{12}$ is estimated correlation of $B_1$ and $B_2$ using the observations from the bridging cohort.

3.2.1. The regression bridging method

The regression (labeled as reg) method is probably the most straightforward and commonly used method in practice. It starts with first fitting a simple linear regression model, using the samples in the bridging cohort $C_{12}$, to predict $B_1$ from $B_2$ (or, symmetrically, $B_2$ from $B_1)$,

$$B_1=\alpha +\beta B_2+\epsilon .$$

This estimated equation is then applied to the subjects outside the bridging cohort, those with $B_2$ but missing $B_1$ in the $C_2$ cohort (or the other way if $B_2$ is regressed on $B_1$). After obtaining the complete data set on $B_1$ (or $B_2$) from both studies, a single estimate to the correlation between $B_1$ and $W$ (or between $B_2$ and $W$) from the combined cohort (including the bridging cohort) can be calculated with its variance (in Fisher’s Z scale) estimated as $\frac{1}{n_1+n_2+n-3}$. An estimate to r is then obtained by Equation (13), with its variance derived via the Delta method, and a 95% CI obtained by applying the Slutsky Theorem to the counterparts in Fisher’s Z scale.

3.2.2. The meta-analysis method

The meta analysis method (labeled as meta) is another natural choice and has two steps. The first step simply ignores the bridging study, i.e., uses only existing data for a meta analysis on the shared correlation, $\rho ,$ from the two studies. Notice that samples originally from each study but used as part of the bridging cohort $C_{12}$ will be included in the study-specific estimate of the correlation from the study. Using the same notations as before, $B_1$ is measured from subjects in Study 1 with measurements $B_{1,1}{,B}_{1,2},\dots ,B_{1,n_1}$, and $B_2$ is measured from the subjects in Study 2 with $B_{2,1}{,B}_{2,2},\dots ,B_{2{,n}_2}$. Note here $n_1,n_2$ include those later used as part of the bridging cohort. The study-specific estimate to the shared correlation with the clinical outcome $W$ can be calculated as ${\widehat{\rho }}_k,$ the same as in Equation (3), as well as the Fisher transformed counterpart $Z_k,k=1,2$. Then, we can apply the fixed effect meta-analysis model to the study-specific correlation estimates to estimate the shared correlation $\rho$ in Fisher’s $Z$ scale as,

$$\widehat{z}=\frac{{\sigma }_2^2}{{\sigma }_1^2+{\sigma }_2^2}{Z}_1+\frac{{\sigma }_1^2}{{\sigma }_1^2+{\sigma }_2^2}{Z}_2,$$

which can be approximated by normal distribution with a mean of $z$ and a variance of

$${\delta }^2=\frac{1}{n_1+n_2-6}.$$

The second step then uses the estimate of the ICC from the bridging cohort, $C_{12}$, to obtain the estimate to r using Equation (13) while the variance to the estimator can be derived via the Delta method, and a 95% CI can be obtained by applying the Slutsky Theorem to the counterparts in Fisher’s Z scale.

3.2.3. Method Of Variance Estimates Recovery (MOVER)

Newcombe ²⁸ has extended the MOVER CI²⁹, originally intended for the CI to the difference of two independently estimated quantities, to provide a CI for the ratio of two independently estimated parameters. In Equation (2), if we allow the shared correlation $\rho$ and ${\rho }_{12}$ to be independently estimated using independent samples, namely, the ${\rho }_{12}$ will be estimated from the bridging cohort only, while the estimate to $\rho$ (labeled as ${\widehat{\rho }}^{\ast }$) will be calculated similarly as in the meta method above, but using the samples from the two studies that are not in the bridging cohort, i.e., $C_1$ and $C_2.$ Subsequently r can be estimated following Equation (13). The variance to the numerator is $Var\left({\widehat{\rho }}^{\ast }\right)=\frac{1}{n_1+n_2-6}$, and the variance to the denominator is $Var(\sqrt{{\widehat{\rho }}_{12}})=\frac{{(1-{{\widehat{\rho }}_{12}}^2)}^2}{(n-3)\times 4\times {\widehat{\rho }}_{12}}$. Using these variance estimates, the lower and upper confidence limits of the 95% CI to $\rho$ are labeled as $L_1$ and $U_1$, respectively, and those for $\sqrt{{\rho }_{12}}$ are labeled as $L_2$ and $U_2$, respectively. Using the estimates and the CI limits to the numerator and denominator and following the equations of Newcombe ²⁸ (page 1776), we can derive the CI to the true correlation r (labeled as mover).

3.2.4. The imputation algorithm

Similar to the reg method which fills in the “missing” values via a simple regression equation, multiple imputations (labeled as imp) by chained equations can be employed to impute the missing data multiple times^30–31. All the data across all the cohorts will be collectively provided as input to have the missing values of $B_1$ imputed using all available information (or vice versa to impute the missing values of $B_2$) to obtain multiple complete dataset(s). The correlation between $B_1$ (or $B_2$) and $W$ averaged across the multiple imputed data can then be used as an estimate to the shared correlation $\rho$, which, along with the estimated ${\rho }_{12}$, leads to an estimate to r as given in Equation (13). Note that when multiple imputed datasets are generated, the variance to the shared correlation in Fisher’s $Z$ scale will be $\frac{1}{n_1+n_2+n-3}$ plus the between-imputation variance across the multiple imputed datasets^30–31.

4. Simulation studies

We conducted simulation studies to evaluate and compare the performance of our latent variable based estimator (latent) and the other alternative methods (reg, meta, mover, imp) in Section 3.

4.1. Simulation setting

The total number of samples from each of the two studies is assumed to be equal to $N,$ and thus total sample size is $2N=n_1+n_2+n.N$ takes a value from ($50,100,200,500$)$.$ From each study, an equal percentage of samples, i.e., $\mathrm{bridge\; percentage}=p=(0.2,0.3,0.4)$, is randomly selected to constitute the bridging cohort, $C_{12},$ and thus the size of the bridging cohort is $n=2N\times p$. The sample size for the cohort $C_k$ (those not in the bridging cohort) is thus $n_k=N-\frac{n}{2}=N\times (1-p)$ for $k=1,2$. The true correlation between the latent biomarker $B$ and a clinical outcome $W$ is assumed at $r=\left(-0.6,-0.3,0,0.3,0.6\right),$ and the true intra-class correlation coefficient takes values ${\rho }_{12}=(0.3,0.6,0.9)$. For each scenario of combinations by $N,p,r,$ and ${\rho }_{12}$, we first generated a random sample of size $2N$ from the trivariate normal distribution

$$\left(\begin{array}{c}{B}_1\\ B_2\\ W\end{array}\right)~N\left(\begin{array}{cc}\mu ,& \mathrm{\Sigma }\end{array}\right),$$

as specified in Section 3.1, where the mean is a zero vector, and ${\sigma }_e^2={\sigma }_w^2=1$, and ${\sigma }^2$ was chosen to achieve the specified ${\rho }_{12}.$ From the complete data set of $(B_1,B_2,W),$ a randomly selected subset of size n was chosen as the bridging cohort which had both $B_1$ and $B_2$ available, a randomly selected subset of $n_1$ samples was chosen to represent samples form Study 1 with $B_1$ available but $B_2$ missing, and the remaining $n_2$ to represent Study 2 samples with $B_2$ available but $B_1$ missing. The data simulation under each scenario was independently repeated for 200 times. All the methods discussed above were applied to each simulated dataset to estimate the true correlation between $B$ and $W$, as well as their associated variance and 95% CI. The performance of the methods were evaluated based on bias (mean difference between an estimate and the true $r$), root mean square error (RMSE), coverage probability (the probability of a 95% CI covering the true $r$ ), and statistical power (the probability of rejecting the null hypothesis ${\text{H}}_0:r=0$ when the true $r$ is not equal to 0) which becomes Type I error rate when the true $r=0$. For the imp algorithm, the R package mice was used for imputation via their random forest univariate imputation method and we set the number of imputed dataset to be either 1 (labeled as imp1) or 5 (labeled as imp5).

Besides the trivariate normal distribution, we also generated data from trivariate exponential distribution using R package “lcmix” function “rmvexp()” and trivariate log normal distribution using the R package “compositions” function “rlnorm.rplus()”, and conducted similar simulation analyses. The rate parameters for the marginal distributions of the trivariate exponential variates were all set =1, with the same correlation matrix as used for the trivariate normal scenario. For the trivariate log-normal distribution, the log-transformed trivariate random variables were set to follow the same pre-configured mean and covariance matrix as for the trivariate normal scenario.

4.2. Simulation results

For all the methods that estimate the true correlation $r$, Figure 1 to 4 presents bias, RMSE, coverage probability, and Wald test power (size), respectively, under the trivariate normal distributions. The two latent variable model based estimators, latent and latent_SS (see definitions at the end of Section 3.1.2) gave roughly close performance, and imp1 also performed similarly to imp5. Thus, we presented results for latent and imp5 in the main Figures, and included all methods’ results in Supplemental Figure 1 to 4. All the methods showed improvement in terms of bias, RMSE, coverage, power, and Type I error with increasing sample size or/and increasing ICC (i.e., ${\rho }_{12}$). Note that the latent variable model-based estimators to $r$ are also a meta-analysis type of estimator encompassing three components, in comparison to conventional meta which encompasses two components. Not surprisingly, they yielded similar and mostly best performance with the smallest biases approaching zero and RMSE decreasing faster than the other methods across simulation scenarios when the sample size increased, although the RMSE seemed highest at smallest N=100. The bias of these methods appeared to be mostly driven by the small sample sizes as well as by poor ICC. When the ICC reached at least 0.6 and sample sizes per cohort reached at least 200, the magnitude of the bias became very small (+/−0.02) for all the methods.

Figure 1.

Bias is plotted against the sample size per cohort (N at x-axis), the true biological correlation r (at row panels), the ICC, and the size of bridging cohort (in %, at column panels) for the estimators of r from multiple statistical methods (indicated by colors): latent (in red color), meta (in gold color), reg (in green color), imp5 (in blue color), and mover (in pink color)

Figure 4.

Empirical power of the Wald test is plotted against the sample size per cohort (N at x-axis), the true biological correlation r (at row panels), the ICC, and the size of bridging cohort (in %, at column panels) for testing r against 0 from multiple statistical methods (indicated by colors): latent (in red color), meta (in gold color), reg (in green color), imp5 (in blue color), and mover (in pink color).

The size of the bridging cohort within the range we considered (20%, 30%, and 40% of the sample sizes) did not seem to affect the bias very much when the ICC was 0.6 or higher, but a larger size of the bridging cohort seemed to reduce the RMSE when the ICC is very low (${\rho }_{12}\text{=}0.3$). On the other hand, the reg method rendered the worst bias at small or mediocre ICC at ${\rho }_{12}\text{=}0.3$ and 0.6, although its bias was comparable to or sometimes even better than the latent variable model approach and the meta analysis approach when ICC is around 0.9 or higher. When the ICC is 0.9, the RMSE of the reg method was comparable to that of meta and latent, but reg and meta both seemed better performers than latent at very small ICC ${\rho }_{12}\text{=}0.3$. Finally, the mover method tended to be the worst with the largest bias for $r=0$ and the largest RMSE across all the scenarios, and the imp5 seemed to have a slightly larger bias when the true $r$ is positive than latent and meta, but comparable RMSE to other methods which is superior to latent when the ICC and sample size were small.

The 95% CI to the true correlation estimated by the latent variable model along with its variance estimator by the Delta method (latent) rendered high coverage mostly around 95% with larger sample sizes, while the coverage from the 95% CI using the Slutsky’s theorem was comparatively lower than the counterpart from the Delta method, but both better than the coverage from the other methods. The coverage of the 95% CI ranged from 0.82 to 0.985 using the meta method and from 0.8 to 0.99 using the imp5 method. The worst coverage of 95% CI was consistently observed at low ICC=0.3 using the mover method, although its coverage approximated others with a large sample size of at least 300 or with a larger ICC of 0.6 or 0.9.

Figure 4 plots the statistical power for testing whether the true correlation r equals to 0 as a function of sample size under various ICC (${\rho }_{12}$; row panel) and highlights the attenuating effect of ${\rho }_{12}$ in the statistical power for detecting the true correlation r at a significance level of 5%. All methods attained good power at large sample sizes, especially when $r$ is as large as 0.6 (in absolute magnitude). Statistical tests based on our proposed estimator (latent) and meta usually yielded better power, while the power for mover seemed to be the worst. When true $r=0$, Type I error rates were generally well controlled even with small sample sizes and poor ICC, with the exception of imp5 whose Type I error rate constantly stayed above 5% across the ranges of sample size and ICC, while the latent variable model-based estimator was mostly around 5%.

The simulation results (bias, RMSE, coverage, power) were presented in Supplemental Figure 5 to 8 for the trivariate exponential and Supplementla Figure 9 to 12 for log-normal distribution. Because the two distributions deviates markedly from the normal distributions, the performance became much worse for all the methods including our proposed methods. These results highlight the importance of first assessing the joint distribution of biomarkers and clinical outcomes, and then only applying our proposed methods and others when data are reasonably normally distributed. These results also suggest that when data deviate quite severely from normal distributions, our proposed methods should be applied to estimate rank-based correlations such as Spearman correlations to allow more robust inferences, as we demonstrate below.

5. Application to correlating Alzheimer disease biomarkers with cognition

Amyloid plaques are one of the key pathological hallmarks of AD, and CSF biomarkers including Aβ42 and Aβ40 concentrations are strongly correlated with AD brain pathology ^2–6. However, there remain challenges in measuring CSF biomarkers consistently for research studies and considerable lab-to-lab variation exists, preventing a simple combination of biomarker data from different studies. In the multi-center SORTOUT-AB study, a subset of 69 “bridging” CSF samples from UPenn were analyzed at the WU ADRC using the same conditions used for analysis of the WU samples. Concentrations of CSF Aβ42 and Aβ40 were correlated with performance on two standardized cognitive tests. The tests, Animals and Vegetables, ask participants to name as many vegetables and animals as they can within a certain time limit. Higher numbers of animals and vegetables correspond to better cognition. The baseline CSF data was merged with cognitive data collected within two years or less of the CSF samples. The data set consisted of 1176 participants from WU and 295 participants from UPenn ADRC. There were 69 UPenn participants with CSF Aβ42 and Aβ40 concentrations measured by both ADRCs, i.e., both $B_1$ and $B_2$, in addition to the cognitive tests. A total of 950 participants were cognitively normal with a Clinical Dementia Rating^®™ (CDR^®™)³² of 0, and 226 were cognitively impaired (CDR>0). Detailed demographic characteristics of the entire cohort are summarized in Supplemental Table 1. Because of the concern that some of the CSF biomarkers and cognitive scores may deviate from the normal distribution, we chose to use the rank-based Spearman correlations. The ICC was estimated at 0.7573 and 0.8251 for CSF Aβ40 and Aβ42, respectively. The Lin’s concordance correlation coefficient^33–34 between $B_1$ and $B_2$ from the bridging cohort was 0.7238 (95% CI: 0.6235~0.8007) and 0.8495 (95% CI: 0.7810~0.8977) for CSF Aβ40 and Aβ42, respectively, suggesting mediocre test-retest reliability for the two biomarkers. Table 2 presents the estimated true Spearman correlation r and its 95% CI from all the methods between each latent CSF biomarker and each cognitive test, which are also illustrated by Figure 5. Table 2 also includes the p-values for testing whether each correlation equals to 0. The correlation estimates were quite consistent across the methods. CSF Aβ42 showed a correlation of about of 0.30 to 0.37 with Animals and a correlation of about 0.40~0.45 with Vegetables, suggesting that larger values of the CSF Aβ42 are associated with better cognitive performance, consistent with our previous reports^5,35–36. The latent variable model-based point estimators (latent and latent_SS) for the correlation between CSF Aβ42 and the two cognitive tests were more conservative , compared to the regression based (reg) point estimators. The 95% CIs were of similar widths for most methods, but MOVER method seemed to lead to a wider CI. We further found that CSF Aβ40 did not correlate with Animals from all the methods (except for the reg method), but was very weakly but significantly correlated with Vegetables (ranging from 0.16 to 0.21). This positive correlation between CSF Aβ40 and cognition, to the best of our knowledge, has not been reported previously, including a recent large study of CSF biomarkers³⁷ and another recent meta analysis on these biomarkers³⁸. Additionally, age is the biggest risk factor of AD, and is known to affect AD markers. To evaluate whether CSF markers and cognitive tests were correlated beyond the contribution from age, we first regressed out the effect of age on biomarkers, and then derived the residuals. Finally, we applied our methods to the residuals and reported the results in Supplemental Table 2. Similar correlations between CSF biomarkers and cognitive test remained after accounting for the age effect.

Table 2.

Estimated Spearman correlations (labeled as r) between two CSF biomarkers (Aβ40 and Aβ42) and two cognitive tests (Animals and Vegetables), with associated standard error (SE), 95% CI and p-value for testing true r against 0

Cognition	Biomarker	latent	latent_SS	meta	mover	reg	imp5
Animals	Aβ40	r=0.02 SE=0.03 −0.04~0.08 p=0.522	r=0.02 SE=0.03 −0.04~0.08 p=0.521	r=0.04 SE=0.03 −0.02~0.10 p=0.175	r=0.03 SE=0.03 −0.03~0.09 p=0.369	r=0.08 SE=0.03 0.02~0.14 p=0.008	r=0.02 SE=0.03 −0.05~0.08 p=0.602
Animals	Aβ42	r=0.32 SE=0.03 0.26~0.37 P<0.0001	r=0.31 SE=0.03 0.26~0.37 P<0.0001	r=0.33 SE=0.03 0.27~0.38 P<0.0001	r=0.30 SE=0.03 0.25~0.38 P<0.0001	r=0.37 SE=0.03 0.32~0.42 P<0.0001	r=0.34 SE=0.03 0.29~0.39 P<0.0001
Vegetables	Aβ40	r=0.16 SE=0.04 0.09~0.22 P<0.0001	r=0.16 SE=0.04 0.09~0.22 P<0.0001	r=0.17 SE=0.03 0.10~0.23 P<0.0001	r=0.17 SE=0.04 0.10~0.26 P<0.0001	r=0.21 SE=0.03 0.14~0.28 P<0.0001	r=0.16 SE=0.04 0.09~0.24 P<0.0001
Vegetables	Aβ42	r=0.41 SE=0.03 0.35~0.47 P<0.0001	r=0.41 SE=0.03 0.35~0.47 P<0.0001	r=0.41 SE=0.03 0.36~0.47 P<0.0001	r=0.40 SE=0.03 0.34~0.49 P<0.0001	r=0.45 SE=0.03 0.40~0.51 P<0.0001	r=0.41 SE=0.03 0.35~0.47 P<0.0001

Figure 5.

Forest plot of the estimated correlations along with the 95% CI by various methods between two CSF biomarkers (Aβ40 labeled as Ab40 and Aβ42 labeled as Ab42) and two cognitive tests (Animals and Vegetables)

6. Discussion

The literature is limited on using bridging samples and data from multiple studies to make statistical inferences of correlation coefficients when retrospectively collected biomarker data were pooled from multiple studies and linked by the prospectively designed bridging study. We proposed a novel latent variable model framework to incorporate the measurement errors of biomarkers, and derived an optimal estimator in which all data were used for statistical inferences of the correlation. Our proposed approaches have several strengths. First, the simple conceptualization of a latent biomarker underlying all the measurements of the biomarker across the studies allows us to estimate a single biological correlation between a biological trait and a clinical outcome that encompasses all study-specific correlations. Second, the introduction of measurement error in the biomarker across the studies enables us to establish a mathematical linkage between the biological correlation (with a clinical outcome) and the correlation from the observed version of the biomarker within each study. This linkage depends on the magnitude of the correlation between the observed versions of the same biomarker across the studies, namely, the ICC, and hence provides statistical justification on why a bridging study must be designed to estimate the true correlation between the latent biomarker trait and the clinical outcomes. Specifically, without a bridging study, the ICC would not be estimable, and no statistical inferences can be made on the true biological correlation. Third, although a bridging study is typically used to link biomarker values from one study to those from another study, when it comes to establishing the biological correlation of a biomarker with a clinical outcome, there is no need to mathematically transform one observed biomarker to the other. If data are not transformed, this avoids many analytic challenges such as which study should be considered as the ‘gold standard’ to which the other study’s data should be transformed, and what transformations may be the most appropriate. Fourth, whereas the standard statistical paradigm of biomarker studies has been to first establish the measurement properties such as reliability and reproducibility, and then proceed with the use of the biomarker to address critical scientific questions such as its association with clinical outcomes only after the reliability of the biomarker is good or excellent, the use of the latent biomarker approach can address critical scientific questions even when the measurement properties are poor. For example, our simulation studies allowed the ICC of biomarker measurements across the studies to be from low (0.3) to mediocre (0.6), and the results showed that when the total sample size is 100 to 200, substantial bias existed in estimating the true biological correlation. However, when the sample sizes increase, even with a poor ICC as low as 0.3, our proposed correlation estimator showed very small bias and small RMSE, and most importantly, the 95% CI performed well with the expected level of coverage, and the Type I error for the statistical test was also well controlled at the nominal level.

We further compared several potential methods and algorithms for statistical inferences on the true biological correlation coefficient when a bridging study was designed. We found that our latent variable model-based approach generally performed better in terms of bias than the other methods such as those based on regression based method, multiple imputation, but similar to the meta-analysis model. In terms of RMSE, our latent variable model-based estimator was comparable to the others, with the exception when the sample size is very small. Most importantly, our latent variable model-based CI and statistical tests performed better or comparable to the other methods in terms of the coverage and Type I error control, even when the ICC was poor and sample sizes were small. The simple two-stage approaches such as the regression method and the imputation method ignored the variation associated with the prediction at the first stage (i.e., variation in the estimated $B_1,B_2$ conversion), which may be a main reason why these simple methods did not perform well, in comparison to our proposed latent variable model which allowed a joint analysis of all data and avoids the 2-stage approach, leading to better coverages.

Our latent biomarker model used all measurements from the bridging samples, but the meta analysis model did not use bridging samples in the estimation of shared correlation, ${\rho }_1={\rho }_2={\rho }_{.}$ We found that both methods performed almost the same in terms of bias and RMSE, but latent variable model-based CI estimator seemed to outperform in terms of the coverage and statistical power. This may be due to the fact that the participation of repeated measures of the same biomarker from the bridging cohort in the inferential analyses improves the efficiency of the estimator to the shared correlation between the observed biomarker and the clinical outcome.

This methodological work was directly motivated by the SORTOUT-AB, a multi-center biomarker study in AD, and we applied our proposed estimators back to the real world database of the study, and found that CSF Aβ42 showed a correlation of about of 0.3 to 0.45 with the two cognitive tests, consistent with the current literature. While CSF Aβ40 did not correlate with Animals with estimates to $r$ around 0, it was very weakly but significantly correlated with Vegetables, which represents likely the first time that CSF Aβ40 alone is biologically linked with cognitive decline. This important finding may be attributed to the improved statistical power when data from multiple studies were pooled together for joint analyses, and also the improved efficiency of statistical inferences that linked the true biological correlation to the study-specific correlation, overcoming the drawback of the latter which ignored the measurement errors and attenuated the correlation estimates. The latent variable model framework, and the resulting estimators to the true biological correlations, hence represent a novel and better approach to make statistical inferences when retrospective data from multiple biomarker studies are integrated and linked by a prospective bridging study. We recently learned, after we already submitted our own manuscript, that another repeated measures approach was proposed to address joint analyses to pooled and calibrated biomarker data³⁹, using similar study design as already reported by the same group of authors^11–12. Our proposed methods are fundamentally different from the recently published work whose focus was on a case-control design with a logistic regression model to predict a clinical classification, i.e., cases (vs. controls) by using biomarkers, whereas our focus is on a (symmetric) correlation between biomarkers and clinical outcomes without the need to specify an outcome (Y) or a predictor (X). This is important because AD is a neurodegenerative disease lasting decades in its progression, it is likely that there is a two-directional relationship between some biomarkers and cognitive changes at certain stages of the disease. Further, whereas recently published work^{39, 11–12} modeled a binary disease outcome, our approach focused on a continuous cognitive score. Finally, the bridging cohort from recently published work came from only part of their controls, whereas our bridging cohort is designed as a random sample of the entire population under study. This is a major difference because the former will have to make some strong assumptions extrapolating the bridging from only controls to cases in their populations, whereas our approach does not need any such strong assumptions.

Albeit not the primary focus of the current study, the selection of bridging samples is very important in designing bridging studies such as SORTOUT-AB. One fundamental requirement is the random sampling from the pool of samples, which assures the validity of the proposed statistical inferences. Another more practical consideration is that the bridging samples should cover the full ranges of biomarker measurements under study, which avoids potential extrapolations in the statistical inferences. Further, the sample size of the bridging cohort is also an important practical question for designing bridging studies. Whereas some portions of our proposed estimators are valid even with relatively small sample sizes from the bridging cohort, i.e., the normal approximation of Fisher transformaed correlation coefficient, others do require large sample approximation, i.e., Slutsky Theorem-based CI for the correlation which requires a large size of bridging cohort. More research is needed in this direction.

Figure 2.

RMSE is against the sample size per cohort (N at x-axis), the true biological correlation r (at row panels), the ICC, and the size of bridging cohort (in %, at column panels) for the estimators of r from multiple statistical methods (indicated by colors): latent (in red color), meta (in gold color), reg (in green color), imp5 (in blue color), and mover (in pink color)

Figure 3.

Empirical coverage of 95% CI is plotted against the sample size per cohort (N at x-axis), the true biological correlation r (at row panels), the ICC, and the size of bridging cohort (in %, at column panels) for the estimators of r from multiple statistical methods (indicated by colors): latent (in red color), latent_slutsky (in gold color), meta (in green color), reg (in blue color), imp5 (in faint blue color), and mover (in pink color). The light gray horizontal line indicates 0.95.