Weighted Repeated Measures Correlation Coefficient: A New Correlation Coefficient for Handling Missing Data With Repeated Measures

ABSTRACT

The relationship between two variables measured multiple times per individual has often been evaluated in clinical studies. These data are not independent; therefore, the Pearson correlation coefficient is inappropriate, and some correlation coefficients for these data have been proposed. However, in the presence of missing data, the existing methods can be biased. In this article, we proposed a weighted repeated measures correlation coefficient that provides an accurate estimate, even with missing data, in a study in which participants ideally have the same number of measurements. We also provided a bootstrap confidence interval for the weighted repeated measures correlation coefficients. We evaluated the performance of the proposed and existing methods (i.e., simple Pearson correlation coefficient, the Pearson correlation coefficient for average, average of the Pearson correlation coefficient, correlation coefficient based on analysis of covariance, and correlation coefficient based on the linear mixed‐effects model) through simulations and application to actual data. In numerical evaluations using simulations, the proposed method consistently outperformed existing methods. We recommend using a weighted repeated measures correlation coefficient to handle missing values in multiple‐measurement data.

Abbreviations

ANCOVA

analysis of covariance

BSS

bed sensor vital‐sign monitoring system

CI

confidence interval

TA‐BSS

respiratory‐related tidal amplitude by the bed sensor vital‐sign monitoring system

TV‐PN

tidal volume by a ventilator pneumotachograph

1. Introduction

The Pearson correlation coefficient is a well‐known indicator for evaluating the association between two variables for independent data [1] and is appropriate when each participant has only one pair of data. However, this is inadequate for data with two or more pairs of measurements per participant. Previous studies have investigated the correlation coefficients for such non‐independent data [2]. Two major correlation coefficients for multiple pairs of data have been proposed: one based on the analysis of covariance (ANCOVA) [3, 4, 5] and the other based on linear mixed‐effects models [6, 7, 8]. These methods estimate the correlation coefficient by adjusting for interindividual variability or individual‐level effects. In medical research, these correlation coefficients have been applied to studies with continuous monitoring [9, 10, 11, 12, 13, 14]. Intraclass correlation coefficients are often used in reliability tests with multiple pairs of data [15]. However, the intraclass correlation coefficient for repeated measurements assumes that true values of the measurement are constant at different timepoints.

When estimating the relationship between two variables, participants should be considered equally in ideal clinical research. The estimand is the individual average of the two‐variable correlation at a single timepoint in the target population. A situation in which one participant has several times as much measurement data as the other participants is not reasonable in typical situations. However, in clinical studies with repeated measurements, the amount of observed measurement data is not always the same among participants. When participants underwent differing numbers of measurements, the Pearson correlation coefficient and existing correlation coefficients for repeated measurement data could disproportionately reflect the features of participants with more data because these methods do not adjust for the number of measurements per participant. In studies where the same number of measurements is ideally planned for each individual, variations in the number of measurements can be considered a missing data problem. Although differences in the number of measurements can introduce bias, the influence of missing data on correlation coefficients with repeated measures has not been adequately investigated.

We propose a weighted repeated measures correlation coefficient, a new correlation coefficient for repeated measurement data that handles bias depending on the variation in the number of measurements. We added the averaging operation for each individual in the estimation to the existing correlation coefficients based on ANCOVA to equalize the degree of individual impact on the estimator. We also provide a bootstrap confidence interval for the weighted repeated measures correlation coefficients. In this study, we evaluated the performance of correlation coefficients based on the Pearson correlation coefficient, existing correlation coefficients for repeated measurement data, and weighted repeated measures correlation coefficients under both non‐missing and missing data conditions.

2. Correlation Coefficients for Repeated Measurement Data

In our study, we considered six correlation coefficients: three based on the Pearson correlation coefficient ($r_P$, $r_{\mathit{PA}}$, and $r_{\mathit{AP}}$), a correlation coefficient based on ANCOVA ($r_{\mathit{\text{AN}}}$), a correlation coefficient based on the linear mixed effects model ($r_{\mathit{LM}}$), and the weighted repeated measures correlation coefficient ($r_W$).

We assumed a clinical study in which two variables were repeatedly measured in participants to estimate the correlation between them. Let the pair of random variables $\left(X_{\mathit{ij}},Y_{\mathit{ij}}\right)$ $\left(i=1,2,3,\dots ,n;j=1,2,3,\dots ,t_i\right)$ denote the variables to be evaluated, where $i$ refers to the participant and $j$ refers to the timepoint of the measurement. We let $t_i$ be the number of measurements obtained for participant $i$ in the study and $T_i$ be the number of measurements that should be obtained in the ideal study. We assumed that the ideal number of measurements was equal for all participants: $T_1=T_2=\dots =T_n=T$. If missing data occurred at timepoint $j$, then both $X_j$ and $Y_j$ were considered missing.

There are some assumptions in each correlation coefficient. Pearson correlation coefficient ($r_P$) assumes that the multivariate distribution of $X_{\mathit{ij}}$ and $Y_{\mathit{ij}}$ is constant across all measurements. We estimate the correlation of two variables at a single timepoint by repeated measurement data, therefore, $r_{\mathit{AP}}$, $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$ assume the consistency in the conditional multivariate distribution of $X_{\mathit{ij}}$ and $Y_{\mathit{ij}}$ given the participant.

2.1. Methods Based on the Pearson Correlation Coefficient

We considered three methods based on the Pearson correlation coefficient. The first method was the simple Pearson correlation coefficient ($r_P$), which is computed as $r_P=\frac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)}\sqrt{\mathrm{Var}(Y)}}$ where $\mathrm{Cov}(X,Y)$ is the covariance between $X$ and $Y$, and $\mathrm{Var}(X)$ is the variance of $X$. The Pearson correlation coefficient estimates the measurement average of the two‐variable correlation, which differs from the targeted estimand in our study. The second method was the Pearson correlation coefficient for averaged variables ($r_{\mathit{PA}}$), which is the correlation between $\bar{X}$ and $\bar{Y}$. This is computed as $r_{\mathit{PA}}=\frac{\mathrm{Cov}(\bar{X},\bar{Y})}{\sqrt{\mathrm{Var}(\bar{X})}\sqrt{\mathrm{Var}(\bar{Y})}}$, where ${\bar{X}}_i=\frac{1}{t_i}\sum _j{X}_{\mathit{ij}}$ and $\overline{Y_i}=\frac{1}{t_i}\sum _j{Y}_{\mathit{ij}}$. The third method was the average Pearson correlation coefficient ($r_{\mathit{AP}}$), which is the mean value of the Pearson correlation coefficient for each participant. This is computed by $r_{\mathit{AP}}=\frac{1}{n}\sum _i\frac{\mathrm{Cov}\left(X_i,X_i\right)}{\sqrt{\mathrm{Var}\left(X_i\right)}\sqrt{\mathrm{Var}\left(Y_i\right)}}$.

2.2. Correlation Coefficient Based on ANCOVA

The correlation coefficient based on ANCOVA ($r_{\mathit{\text{AN}}}$) is estimated as the ratio of the sums of squares for $X$ and error, calculated using the linear regression model ($Y_{\mathit{ij}}=\mu +X_{\mathit{ij}}\beta +{\theta }_i+{ϵ}_{\mathit{ij}}$), where $\mu$ refers to the intercept and ${\theta }_i$ refers to the individual effect of $i$‐th participant [5]. Let the sum of squares of $X_{\mathit{ij}}$ be ${\mathit{SS}}_x$ and the sum of squares of errors be ${\mathit{SS}}_e$. Let the observed value of $Y_{\mathit{ij}}$ be $y_{\mathit{ij}}$, the predicted value of $y_{\mathit{ij}}$ based on the linear regression ($Y_{\mathit{ij}}=\mu +X_{\mathit{ij}}\beta +{\theta }_i+{ϵ}_{\mathit{ij}}$) be ${\widehat{y}}_{\mathit{ij},0}$, and the predicted value of $y_{\mathit{ij}}$ based on the linear regression $\left(Y_{\mathit{ij}}=\mu +{\theta }_i+{ϵ}_{\mathit{ij}}\right)$ be ${\widehat{y}}_{\mathit{ij},1}$. The sum of the squares is defined as

$$\begin{gathered} \begin{array}{cc}& {\mathit{SS}}_x\mathbf{=}\sum _i\sum _{t_i}\left\{{\left(y_{\mathit{ij}}-{\widehat{y}}_{\mathit{ij},1}\right)}^2-{\left(y_{\mathit{ij}}-{\widehat{y}}_{\mathit{ij},0}\right)}^2\right\}, \\ \\ & {\mathit{SS}}_e=\sum _i\sum _{t_i}{\left(y_{\mathit{ij}}-{\widehat{y}}_{\mathit{ij},0}\right)}^2.\end{array} \end{gathered}$$

The correlation coefficient is estimated with the following equation:

$$r_{\mathit{\text{AN}}}=\mathrm{sgn}(\beta )\sqrt{\frac{{\mathit{SS}}_x}{{\mathit{SS}}_x+{\mathit{SS}}_e}},\text{where}\mathrm{sgn}(x)=\left\{\begin{array}{l}1,x\ge 0\\ -1,x<0.\end{array}\right.$$

The sign of the correlation coefficient corresponds to the $\beta$, which is the coefficient of $X_{\mathit{ij}}$ in the regression analysis.

2.3. Correlation Coefficient Based on the Linear Mixed Effects Model

The correlation coefficient, based on the linear mixed‐effects model ($r_{\mathit{LM}}$) is calculated from the covariance matrix estimated using the maximum likelihood method [8]. Let ${\mathbf{M}}_i={\left(X_{i1},Y_{i1},X_{i2},Y_{i2},\dots ,X_{i{t}_i},Y_{i{t}_i}\right)}^t$ be the outcomes from the ith participant, where $t$ denotes the transpose, and ${\mathbf{A}}_i$ and ${\mathbf{Z}}_i$ are the design matrices for fixed and random effects. The linear mixed effects model is presented as ${\mathbf{M}}_i={\mathbf{A}}_i\mathbf{\beta }+{\mathbf{Z}}_i{\mathbf{b}}_i+{\mathbf{\epsilon }}_i$. In this study, we do not specify any random effects and use only the intercept and variable to identify $X$ and $Y$ as fixed effects. The random effects ${\mathbf{b}}_i$ follows a multivariate normal distribution $\mathit{MVN}(\mathbf{0}\mathit{,}\mathbf{D})$ and the error term ${\mathbf{\epsilon }}_i$ follows a multivariate normal distribution $\mathit{MVN}\left(\mathbf{0},{\mathbf{R}}_i\right)$ where ${\mathbf{R}}_i$ is the Kronecker product (${\mathbf{R}}_i\mathit{=}\mathbf{V}\mathit{\otimes }\mathbf{\Omega }$) between the correlation structure $\mathbf{V}$ across timepoints and the following correlation structure $\mathbf{\Omega }$:

$$\mathbf{\Omega }\mathbf{=}\left(\begin{array}{ll}{\sigma }_x^2& {\sigma }_{\mathit{xy}}\\ {\sigma }_{\mathit{xy}}& {\sigma }_y^2\end{array}\right)$$

Using the estimated value $\widehat{\mathbf{\Omega }}$, the correlation coefficient is calculated to be

$$r_{\mathit{LM}}\mathit{=}\frac{{\widehat{\sigma }}_{\mathit{xy}}}{{\widehat{\sigma }}_x{\widehat{\sigma }}_y}$$

In our study, we assumed a compound symmetry structure in $\mathbf{V}$.

2.4. Weighted Repeated Measures Correlation Coefficient

We proposed a weighted repeated measures correlation coefficient ($r_W$), which is a modification of the correlation coefficient based on ANCOVA. We added a new procedure for averaging the measurements for each participant when decomposing the sums of squares. In the proposed method, we calculate the modified sum of squares as

$${\overline{\mathit{SS}}}_x=\sum _i\frac{1}{t_i}\sum _{t_i}\left\{{\left(y_{\mathit{ij}}-{\widehat{y}}_{\mathit{ij},1}\right)}^2-{\left(y_{\mathit{ij}}-{\widehat{y}}_{\mathit{ij},0}\right)}^2\right\},$$

$${\overline{\mathit{SS}}}_e=\sum _i\frac{1}{t_i}\sum _{t_i}{\left(y_{\mathit{ij}}-{\widehat{y}}_{\mathit{ij},0}\right)}^2.$$

Using the modified sum of squares, we can adjust for the variation in the contribution of the participants to the sum of squares resulting from the missing data. We estimated the weighted repeated measures correlation coefficient with the following equation:

$$r_W=\mathrm{sgn}(\beta )\sqrt{\frac{{\overline{\mathit{SS}}}_x}{{\overline{\mathit{SS}}}_x+{\overline{\mathit{SS}}}_e}}.$$

This formula is equivalent to the ANCOVA‐based correlation coefficient, with ${\mathit{SS}}_x$ replaced by ${\overline{\mathit{SS}}}_x$ and ${\mathit{SS}}_e$ replaced by ${\overline{\mathit{SS}}}_e$.

2.5. Confidence Intervals

One option for estimating the confidence intervals for $r_{\mathit{\text{AN}}}$ in the rmcorr package [16] in R is the bootstrap method. When estimating confidence intervals of confidence level $\gamma$ (0.95 is the typical value), we sample $B$ times and estimate the correlation coefficient, and the lower and upper limits correspond to $\frac{(1-\gamma )}{2}$ and $1-\frac{(1-\gamma )}{2}$ percentile points respectively. We applied this estimation method to other correlation coefficients. In this study, we set $B=1000$ and $\gamma =0.95$.

3. Simulation Study

3.1. Data Generation

We assumed clinical research with multiple timepoints to estimate the correlation coefficient for the target population. Although there may be heterogeneity in the correlation coefficient could be associated with missing measurements in the target population, this is not always evident in advance. For example, in a study of 14 participants (12 males and 2 females), described in Section 4, $r_{\mathit{\text{AN}}}$ and the mean missing probability were 0.567 and 18.5% for males and 0.219 and 3.3% for females, respectively. We considered a situation in which there were subgroups with different correlations and missing probabilities in the target population.

We assumed a study with 30 participants, each with multiple measurements ($T=10,50,100$) of $X$ and $Y$. The simulation was repeated 10 000 times. The 30 participants consisted of 15 participants from each of the two latent classes, denoted as $kϵ\{1,2\}$. For each participant, ${\mathbf{M}}_i$ was generated from the multivariate normal distribution $\mathit{MVN}\left({\mathbf{\mu }}_k,{\mathbf{\sum }}_k\right)$ with the following mean structure and covariance matrix:

$${\mathbf{\mu }}_k\mathbf{=}\left(\begin{array}{l}{\mu }_{\mathit{kx}}\\ {\mu }_{\mathit{ky}}\\ ⋮\\ {\mu }_{\mathit{kx}}\\ {\mu }_{\mathit{ky}}\end{array}\right),$$

and

$${\mathbf{\sum }}_k\mathbf{=}\left(\begin{array}{lllllll}{\sigma }_{\mathit{kx}}^2& r_k{\sigma }_{\mathit{kx}}{\sigma }_{\mathit{ky}}& 0& 0& \cdots & 0& 0\\ r_k{\sigma }_{\mathit{kx}}{\sigma }_{\mathit{ky}}& {\sigma }_{\mathit{ky}}^2& 0& 0& \cdots & 0& 0\\ 0& 0& {\sigma }_{\mathit{kx}}^2& r_k{\sigma }_{\mathit{kx}}{\sigma }_{\mathit{ky}}& \cdots & 0& 0\\ 0& 0& r_k{\sigma }_{\mathit{kx}}{\sigma }_{\mathit{ky}}& {\sigma }_{\mathit{ky}}^2& \cdots & 0& 0\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& 0& \cdots & {\sigma }_{\mathit{kx}}^2& r_k{\sigma }_{\mathit{kx}}{\sigma }_{\mathit{ky}}\\ 0& 0& 0& 0& \cdots & r_k{\sigma }_{\mathit{kx}}{\sigma }_{\mathit{ky}}& {\sigma }_{\mathit{ky}}^2\end{array}\right).$$

The missing probability $p_i$ for participant $i$ was generated from $\mathit{\text{Beta}}\left({\alpha }_k,{\mathrm{\beta }}_k\right)$. The missing probability was constant at all timepoints. Parameters ${\alpha }_k$ and ${\mathrm{\beta }}_k$ were defined from the expected value and variance of $p_i$, and the variance was always set to 0.04. We considered four scenarios varying ${\mathbf{\mu }}_1$ and relationships between missing probability and correlations. Table 1 lists the parameter values used in the data generation.

TABLE 1.

Parameters in the simulation study.

	Scenario 1	Scenario 2	Scenario 3	Scenario 4
${\mu }_{1x}$	0	1	0	1
${\sigma }_{1x}$	1	1	1	1
${\mu }_{1y}$	0	1	0	1
${\sigma }_{1y}$	1	1	1	1
$r_1$	0.8	0.8	0.8	0.8
$E\left[p_{k=1}\right]$	0.3	0.3	0.1	0.1
${\mu }_{2x}$	0	0	0	0
${\sigma }_{2x}$	1	1	1	1
${\mu }_{2y}$	0	0	0	0
${\sigma }_{2y}$	1	1	1	1
$r_2$	0.4	0.4	0.4	0.4
$E\left[p_{k=2}\right]$	0.3	0.3	0.5	0.5

3.2. Evaluation

To evaluate the performance of the point estimation, we estimated six correlations ($r_P,r_{\mathit{PA}},\dots ,r_W$) for 10 000 datasets with and without missing data and calculated their means and standard deviations.

We also evaluated the performance of the confidence intervals for $r_{\mathit{\text{AN}}}$ and $r_W$ in Scenarios 1 to 4 with 50‐timepoints. The number of simulations for the confidence intervals was set to 1000. We calculated the proportion of the confidence interval containing 0.6 as the coverage probability and the average width of confidence intervals. SAS 9.4 was used for all statistical analyses.

3.3. Results

The mean and standard deviation of the point estimates for each correlation when measurements were repeated 10, 50, and 100 times were shown in Tables 2, 3, and 4, respectively. When there were no missing data, $r_{\mathit{\text{AN}}}$ and $r_{\mathit{LM}}$ were valid inferences. They were consistently 0.6 regardless of the number of timepoints and scenario. In all settings without missing data, $r_W$ was similar to $r_{\mathit{\text{AN}}}$. However, $r_P$ and $r_{\mathit{PA}}$ were overestimated in Scenarios 2 and 4, and $r_{\mathit{AP}}$ was slightly underestimated when the number of measurements was 10.

TABLE 2.

Mean and standard deviation of the point estimates for each correlation when measurements were 10.

Correlation method		Scenario 1	Scenario 2	Scenario 3	Scenario 4
Full data	$r_P$	0.60 (0.039)	0.68 (0.032)	0.60 (0.039)	0.68 (0.032)
	$r_{\mathit{PA}}$	0.60 (0.128)	0.89 (0.036)	0.60 (0.128)	0.89 (0.036)
	$r_{\mathit{AP}}$	0.58 (0.041)	0.58 (0.041)	0.58 (0.041)	0.58 (0.041)
	$r_{\mathit{\text{AN}}}$	0.60 (0.040)	0.60 (0.040)	0.60 (0.040)	0.60 (0.040)
	$r_{\mathit{LM}}$	0.60 (0.040)	0.60 (0.040)	0.60 (0.040)	0.60 (0.040)
	$r_W$	0.60 (0.040)	0.60 (0.040)	0.60 (0.040)	0.60 (0.040)
Missing data	$r_P$	0.60 (0.048)	0.68 (0.040)	0.66 (0.044)	0.72 (0.036)
	$r_{\mathit{PA}}$	0.59 (0.152)	0.84 (0.064)	0.52 (0.184)	0.80 (0.087)
	$r_{\mathit{AP}}$	0.56 (0.064)	0.56 (0.064)	0.57 (0.075)	0.57 (0.075)
	$r_{\mathit{\text{AN}}}$	0.60 (0.052)	0.60 (0.052)	0.67 (0.047)	0.67 (0.047)
	$r_{\mathit{LM}}$	0.60 (0.051)	0.60 (0.051)	0.66 (0.047)	0.66 (0.047)
	$r_W$	0.60 (0.054)	0.60 (0.054)	0.62 (0.060)	0.62 (0.060)

Note: Mean estimates of 0.6 are highlighted in bold.

TABLE 3.

Mean and standard deviation of the point estimates for each correlation when measurements were 50.

Correlation method		Scenario 1	Scenario 2	Scenario 3	Scenario 4
Full data	$r_P$	0.60 (0.017)	0.68 (0.015)	0.60 (0.017)	0.68 (0.015)
	$r_{\mathit{PA}}$	0.60 (0.127)	0.97 (0.009)	0.60 (0.127)	0.97 (0.009)
	$r_{\mathit{AP}}$	0.60 (0.017)	0.60 (0.017)	0.60 (0.017)	0.60 (0.017)
	$r_{\mathit{\text{AN}}}$	0.60 (0.018)	0.60 (0.018)	0.60 (0.018)	0.60 (0.018)
	$r_{\mathit{LM}}$	0.60 (0.018)	0.60 (0.018)	0.60 (0.018)	0.60 (0.018)
	$r_W$	0.60 (0.018)	0.60 (0.018)	0.60 (0.018)	0.60 (0.018)
Missing data	$r_P$	0.60 (0.024)	0.68 (0.020)	0.66 (0.022)	0.72 (0.017)
	$r_{\mathit{PA}}$	0.59 (0.145)	0.96 (0.018)	0.53 (0.188)	0.94 (0.032)
	$r_{\mathit{AP}}$	0.59 (0.023)	0.59 (0.023)	0.59 (0.029)	0.59 (0.029)
	$r_{\mathit{\text{AN}}}$	0.60 (0.024)	0.60 (0.024)	0.66 (0.022)	0.66 (0.022)
	$r_{\mathit{LM}}$	0.60 (0.024)	0.60 (0.024)	0.66 (0.022)	0.66 (0.022)
	$r_W$	0.60 (0.023)	0.60 (0.023)	0.60 (0.027)	0.60 (0.027)

Note: Mean estimates of 0.6 are highlighted in bold.

TABLE 4.

Mean and standard deviation of the point estimates for each correlation when measurements were 100.

Correlation method		Scenario 1	Scenario 2	Scenario 3	Scenario 4
Full data	$r_P$	0.60 (0.012)	0.68 (0.010)	0.60 (0.012)	0.68 (0.010)
	$r_{\mathit{PA}}$	0.59 (0.128)	0.99 (0.004)	0.59 (0.128)	0.99 (0.004)
	$r_{\mathit{AP}}$	0.60 (0.012)	0.60 (0.012)	0.60 (0.012)	0.60 (0.012)
	$r_{\mathit{\text{AN}}}$	0.60 (0.013)	0.60 (0.013)	0.60 (0.013)	0.60 (0.013)
	$r_{\mathit{LM}}$	0.60 (0.013)	0.60 (0.013)	0.60 (0.013)	0.60 (0.013)
	$r_W$	0.60 (0.013)	0.60 (0.013)	0.60 (0.013)	0.60 (0.013)
Missing data	$r_P$	0.60 (0.019)	0.68 (0.015)	0.66 (0.017)	0.72 (0.013)
	$r_{\mathit{PA}}$	0.59 (0.145)	0.98 (0.009)	0.53 (0.186)	0.97 (0.018)
	$r_{\mathit{AP}}$	0.60 (0.016)	0.60 (0.016)	0.60 (0.020)	0.60 (0.020)
	$r_{\mathit{\text{AN}}}$	0.60 (0.019)	0.60 (0.019)	0.66 (0.017)	0.66 (0.017)
	$r_{\mathit{LM}}$	0.60 (0.019)	0.60 (0.019)	0.66 (0.017)	0.66 (0.017)
	$r_W$	0.60 (0.016)	0.60 (0.016)	0.60 (0.020)	0.60 (0.020)

Note: Mean estimates of 0.6 are highlighted in bold.

In Scenarios 1 and 2, in which the missing probability did not depend on the latent class, bias was not observed for $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$ even with missing data. However, in Scenarios 3 and 4, where the missing probability depended on the latent class, $r_{\mathit{\text{AN}}}$ and $r_{\mathit{LM}}$ were biased, whereas the bias of $r_W$ was smaller for the missing data. The bias of $r_{\mathit{AP}}$ was also small if the number of measurements was 50 or more. The correlation coefficient based on the mixed‐effects model did not converge by up to 20.2%.

Table 5 showed the coverage probabilities of the 95% confidence intervals using the bootstrap method for $r_{\mathit{\text{AN}}}$ and $r_W$. When there were no missing data, the coverage probability was 95.3%. When there was missing data, the coverage probability of $r_W$ was 93.7%, the coverage probability of $r_{\mathit{\text{AN}}}$ was 90.4% for Scenarios 1 and 2, and 16.8% for Scenarios 3 and 4. Table 6 shows the average width of the 95% confidence intervals. When missing data were present, the confidence interval for $r_W$ was consistently wider than the interval for $r_{\mathit{\text{AN}}}$.

TABLE 5.

Coverage probabilities of the 95% confidence intervals by the bootstrap method.

Correlation method		Scenario 1	Scenario 2	Scenario 3	Scenario 4
Full data	$r_{\mathit{\text{AN}}}$	95.3%	95.3%	95.3%	95.3%
	$r_W$	95.3%	95.3%	95.3%	95.3%
Missing data	$r_{\mathit{\text{AN}}}$	90.4%	90.4%	16.8%	16.8%
	$r_W$	93.7%	93.7%	93.7%	93.7%

TABLE 6.

Average width of the 95% confidence intervals by the bootstrap method.

Correlation method		Scenario 1	Scenario 2	Scenario 3	Scenario 4
Full data	$r_{\mathit{\text{AN}}}$	0.069	0.069	0.069	0.069
	$r_W$	0.069	0.069	0.069	0.069
Missing data	$r_{\mathit{\text{AN}}}$	0.082	0.082	0.079	0.073
	$r_W$	0.087	0.087	0.099	0.099

4. Application Study

4.1. The Bed Sensor Vital‐Sign Monitoring System in ICU

We applied our method to a study of contact‐free respiratory rate and volume assessments using load cells under the bed legs (UMIN‐CTR: UMIN000047793) [17]. The Bed Sensor Vital‐Sign Monitoring System (BSS) was developed as an unconstrained monitoring system to measure the respiratory‐related tidal amplitude and respiratory rate using data from load cells at the four legs of the bed. This study assessed the correlation between respiratory‐related tidal amplitude by BSS (TA‐BSS) and the tidal volume by a ventilator pneumotachograph (TV‐PN) in 14 patients under mechanical ventilation in the ICU. Patients who provided consent and had available data for at least 24 h were included. Respiratory‐related data were recorded as one measurement every 10 min. Measurements were taken at 288 timepoints for 13 subjects and 248 timepoints for one subject. The median missing probability was 11.6%, with an interquartile range of 4.9% to 22.9%.

4.2. Statistical Method

We applied the correlation coefficients considered in the simulation study to assess the relationship between TA‐BSS and TV‐PN with the data from this study. Initially, we attempted to apply the correlation methods to the overall data; however, the method based on the mixed‐effects model could not obtain a convergence solution. Consequently, we estimated the point estimates using methods other than the mixed‐effects model for the overall data.

We also applied the correlation methods to a subsampled dataset, random sampling 50 data points per participant without replacement. The median missing probability in the subsampled dataset was 14.0%, with an interquartile range of 4.0% to 32.0%. Point estimates and 95% confidence intervals (CIs) were estimated using the bootstrap method. In the post hoc analysis, we excluded one participant with a negative Pearson correlation coefficient and reassessed the correlations.

5. Results

Table 7 shows the correlation coefficients for both the overall and subsampled datasets. In the sampled data, $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$ were approximately 0.54; $r_P$ and $r_{\mathit{PA}}$ were slightly higher; and $r_{\mathit{AP}}$ was lower. The widths of the confidence intervals for $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$ were similar and were slightly wider for $r_{\mathit{AP}}$. We performed 1000 bootstrap samplings to estimate the confidence intervals but obtained only 656 solutions for $r_{\mathit{LM}}$. Among the 14 Pearson correlation coefficients calculated for each participant in the estimation of $r_{\mathit{AP}}$, one was negatively correlated. A post hoc analysis excluding this participant was conducted, and the results were shown in Table 8. The exclusion did not affect the values for $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$, but influenced $r_P$, $r_{\mathit{PA}}$, and $r_{\mathit{AP}}$, especially $r_{\mathit{AP}}$.

TABLE 7.

Repeated measures correlation between TA‐BSS and TV‐PN.

Correlation method	Overall data	Sampling data
	Estimate	Estimate	95% CI
$r_P$	0.593	0.566	[0.518, 0.612]
$r_{\mathit{PA}}$	0.622	0.593	[0.535, 0.645]
$r_{\mathit{AP}}$	0.360	0.385	[0.320, 0.463]
$r_{\mathit{\text{AN}}}$	0.552	0.544	[0.486, 0.605]
$r_{\mathit{LM}}$	NA	0.544	[0.485, 0.605] a
$r_W$	0.551	0.537	[0.477, 0.598]

Abbreviations: CI, confidence interval; TA‐BSS, tidal amplitude by the bed sensor vital‐sign monitoring system; TV‐PN, tidal volume by a ventilator pneumotachograph.

^a Estimates were obtained only 656 times when bootstrapping.

TABLE 8.

Repeated measures correlation between TA‐BSS and TV‐PN excluding one participant with a negative Pearson correlation coefficient.

Correlation method	13 data	14 data	Difference
$r_P$	0.603	0.566	0.037
$r_{\mathit{PA}}$	0.635	0.593	0.042
$r_{\mathit{AP}}$	0.444	0.385	0.059
$r_{\mathit{\text{AN}}}$	0.553	0.544	0.009
$r_{\mathit{LM}}$	0.553	0.544	0.009
$r_W$	0.544	0.537	0.007

Abbreviations: TA‐BSS, tidal amplitude by the bed sensor vital‐sign monitoring system; TV‐PN, tidal volume by a ventilator pneumotachograph.

6. Discussion

The correlation coefficients for non‐independent repeated measurement data, where the Pearson correlation coefficient is not appropriate, have been proposed. However, in these methods, the difference in the number of measurements per participant is not adjusted and this could introduce bias in the estimation. In this study, we evaluated the performance of existing correlation coefficients for repeated measurement data with and without missing data in the simulation study and proposed the weighted repeated measures correlation coefficient for handling the variation in the number of measurements. After the simulation, these methods were applied to the actual data to evaluate the relationship between the TA‐BSS and TV‐PN.

In our simulations, when the missing data were independent of other variables, the proposed method and correlation coefficients based on ANCOVA and the mixed‐effects model showed minimal bias. However, in scenarios in which the missing probability and correlation were related, the correlation coefficients based on ANCOVA and the mixed‐effects model were equally overestimated. The proposed method, however, resulted in negligible bias.

We revealed that the proposed confidence interval is valid with and without missing data. Through simulation, we confirmed that the coverage proportion of the proposed confidence interval was approximately 95%. The proposed method showed a wider 95% confidence interval than that of the existing method, but the confidence interval based on the ANCOVA exhibited possible under‐coverage properties under missing data. This indicates that the width of the confidence interval of the existing method is underestimated. When there were no missing data (i.e., the number of timepoints was the same for all participants), the proposed method yielded the same value as the correlation coefficient based on ANCOVA.

Methods based on the Pearson correlation coefficient were only reasonable only in specific situations, even without missing data. As expected, when estimating the Pearson correlation coefficient from the whole dataset without considering repeated measures, or when estimating the Pearson correlation coefficient for the mean value for each participant, these estimators were not valid. This result is consistent with previous studies [2, 5]. The average Pearson correlation coefficient for each participant was underestimated $\rho$ when the number of timepoints was small. The Pearson correlation coefficient estimated from the samples is not an unbiased estimator, and the approximation $E[r]=\rho -\rho \left(1-{\rho }^2\right)/2n$ is known, where $r$ is the correlation coefficient estimated from the sample and $\rho$ is the correlation coefficient for the population [18, 19]. This association could explain this underestimation. When the number of timepoints was sufficient, that is, 50 or more, the average Pearson correlation coefficient was an acceptable estimator, even with missing data.

We applied these methods to actual data to evaluate the relationship between the TA‐BSS and TV‐PN. As the results for $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$ were almost the same, the influence of missing data were considered small, and the correlation was approximately 0.54. Despite the sufficient number of timepoints, the estimate of the $r_{\mathit{AP}}$ was less than 0.54. Sensitivity analysis, excluding one participant with a negative Pearson correlation coefficient, showed that the change in $r_{\mathit{AP}}$ was greater than that in $r_{\mathit{\text{AN}}}$, $r_{\mathit{LM}}$, and $r_W$. Although further studies are needed, the results suggest that $r_{\mathit{AP}}$ may not provide a stable estimate in situations where the sample size is small and includes outliers.

When applied to the overall actual data, $r_{\mathit{LM}}$ could not provide a convergent solution. In our simulation, the correlation coefficient based on the mixed‐effects model did not converge up to 20.2% of the time. In our analysis, we assumed compound symmetry in the structure between timepoints; however, convergence problems arose. In studies with many measurements, such as digital health studies [20, 21], methods based on the mixed‐effects model may not be appropriate because of computational volume and convergence concerns.

Our proposed method's concept of averaging operation is similar to the informative cluster size in cluster‐randomized trials. The bias due to missing data in our study is referred to as the difference in the estimand for the participant‐average treatment effect and cluster‐average treatment effect in cluster‐randomized trials [22, 23, 24]. When estimating the correlation coefficient from repeated measurements, paying attention to the estimand is crucial. As mentioned in the Methods section, the estimation of Pearson's correlation coefficient and the repeated measures correlation coefficient are different. In addition, the difference between the ideal data and actual data should be adequately understood, and one should select an appropriate estimator for the research purpose in practical settings.

Our study had several limitations and areas for future research. First, the proposed method assumed independence within the participant; however, we did not evaluate the performance when this assumption was invalid. Assuming independence of measurement errors within the participant could be reasonable in settings where data are measured at multiple timepoints within a short period. However, when estimating the correlation coefficient for repeated measurement data followed for months or years, we may need to consider a non‐independent variance structure of the error. Only the correlation coefficient based on the mixed‐effects model can incorporate the complex variance structure. However, the mixed‐effects model would also be challenging when the number of timepoints is large, or when assuming a uniform structure among participants is unreasonable because the number of timepoints differs. Evaluating the correlation coefficient in these situations is a topic for future research. Second, the proposed method's efficiency and the confidence interval estimation still require further exploration. In this study, we have yet to show the proposed method's theoretical variance and confidence interval. There may be a better weighting that would reduce the variance because we proposed for the unbiased correlation coefficient, not for achieving the minimum variance of the estimator. Theoretical work to clarify the distribution and variance of the estimators we have proposed and discuss the optimal weights is one of the future research topics. These issues could provide the basis for constructing better confidence intervals and discussing sample size calculations. Confidence intervals for the correlation coefficient for repeated measurement data are often estimated using the bootstrap method [5, 25], which we also used to estimate the confidence intervals for the weighted repeated measures correlation coefficient. We emphasize that the proposed and existing estimators provide the same value when the number of observed measurements for each participant is the same. When there was variation in the number of measurements, the proposed method showed a slightly wider confidence interval than that of the existing method. However, a confidence interval based on the bootstrap method could provide sufficient performance for clinical and epidemiological studies, and this result did not necessarily show that the existing estimator would be superior in efficiency because the coverage probability of the existing estimator was under 95%.

In conclusion, we proposed the weighted repeated measures correlation coefficient and evaluated the performance of various correlation coefficients for repeated measurement data under conditions with and without missing data. Existing methods cannot estimate a valid correlation coefficient in some situations, depending on the number of timepoints, the expected value of the variables, and the mechanism of missing data. In contrast, the proposed method consistently provided valid estimates. Therefore, we recommend using the weighted repeated measures correlation coefficient for multiple data points with missing values.

Conflicts of Interest

This study used data obtained during collaborative research between Chiba University and MinebeaMitsumi Inc. in the application section. Shiroh Isono has an advisory contract with MinebeaMitsumi Inc. Shiroh Isono is one of the inventors of the vital sign monitoring system. None of the other authors has any conflicts of interest.

Supporting information

Data S1. Supporting Information.

Data S2. Supporting Information.

Data S3. Supporting Information.

Data Availability Statement

We provided SAS and R programs as supportive information for applying the proposed method (Weighted repeated measures correlation coefficient). The codes used in the Simulation study section and the Application study section are available from the corresponding author upon reasonable request. The Bed Sensor Vital‐Sign Monitoring System in ICU data is not publicly available.