Conservative confidence intervals for the intraclass correlation coefficient for clustered binary data

Abstract

Asymptotic approaches are traditionally used to calculate confidence intervals for intraclass correlation coefficient in a clustered binary study. When sample size is small to medium, or correlation or response rate is near the boundary, asymptotic intervals often do not have satisfactory performance with regard to coverage. We propose using the importance sampling method to construct the profile confidence limits for the intraclass correlation coefficient. Importance sampling is a simulation based approach to reduce the variance of the estimated parameter. Four existing asymptotic limits are used as statistical quantities for sample space ordering in the importance sampling method. Simulation studies are performed to evaluate the performance of the proposed accurate intervals with regard to coverage and interval width. Simulation results indicate that the accurate intervals based on the asymptotic limits by Fleiss and Cuzick generally have shorter width than others in many cases, while the accurate intervals based on Zou and Donner asymptotic limits outperform others when correlation and response rate are close to their boundaries.

1. Introduction

The intraclass correlation coefficient (ICC), denoted by ρ, is widely used to estimate the correlation between multiple measures from one cluster. For example, CNS Vital Signs are often used to define the presence or absence of impairment for professional fighters [1]. Recently, several new measures are added to the battery to define impairment: clinical diagnosis, and brain image data at baseline or longitudinal change [18]. When multiple assessment tools were used to define the impairment status of each fighter, one of the first research questions is the ICC among these measures. When the outcome is binary, many methods were developed to estimate the ICC: moment generating functions, and quasi-likelihood approach [13]. It should be noted that some of these estimates may not exist for some data [3]. In addition to point estimate, confidence intervals for the ICC should be reported in studies with binary correlated data.

For binary data, the exact one-sided limit for the ICC by Buehler [2] is preferable [6,10,15,19]. The Buehler's exact one-sided limit guarantees the nominal level with the computed converge bounded by the nominal level. It has been successfully applied to studies with independent samples or paired samples [7,11], and single-arm two-stage designs with binary outcome [15]. But, it quickly becomes too computationally intensive for a study with clustered binary outcomes. The computation intensity of the Buehler's exact limit comes from the massive size of the complete sample space [16,21,23,25]. For these reasons, we propose using importance sampling (IS) to calculate confidence intervals for ρ. The IS approach reduces the computational intensity by avoiding that challenge while the computed IS intervals are highly accurate with the actual coverages close to the nominal level. In the IS approach, a statistical quantity has to be used for sample space ordering. We use the following four existing intervals as statistical quantities: the ANOVA interval, the modified Wald interval by Zhou and Donner [30], the interval by Fleiss and Cuzick [4], and the Pearson interval that gives equal weight to each pair of observations [8].

In the confidence interval calculation for ρ, there is another parameter in the tail probability: the event rate π which is considered as the nuisance parameter. Kabaila and LLoyd [5] developed the profile confidence limit by using the profile maximum likelihood estimate (MLE) of π given ρ, instead of all the possible π values. This profile confidence limit is computationally effective, and has very good statistical properties [5]. Therefore, we use a combination of the IS approach for discrete data [9] and profile limits by Kabaila and LLoyd [5] to construct confidence intervals for ρ. We conduct extensive simulation studies to compare their performances with regard to coverage and interval width [22,24]. The psychiatric data set reported by Lipsitz et al. [8] is used to illustrate the application of the proposed accurate intervals.

The rest of the article is organized as follows. In Section 2, we first introduce the four existing asymptotic intervals, and then propose accurate intervals for ρ by using the asymptotic limits as statistical quantities for sample space ordering. In Section 3, we compare the performance of accurate intervals with a common cluster size and multiple cluster sizes through simulation studies. The proposed accurate intervals are then applied to the data set from a psychiatric study. Finally, we provide some comments on the ICC confidence interval calculation as well as a paradox of the ICC estimate in Section 4.

2. Methods

Suppose a study has k clusters with the cluster size of $n_i$ individuals for the ith cluster, where $i=1,2,\dots ,k$. When the outcome is binary, the number of events among $n_i$ individuals from the ith cluster is $Y_i$, with data $\mathbf{Y}=(Y_1,Y_2,\dots ,Y_k)$, see Table 1. The total number of individuals in the study is $N=\sum _{i=1}^k{n}_i$. The ICC provides a quantitative measure of correlation between individuals from the same cluster. This is the parameter of interest here in this article.

Table 1.

Data for a clustered binary study with k clusters.

Events	Non-events	Total
$Y_1$	$n_1-Y_1$	$n_1$
$Y_2$	$n_2-Y_2$	$n_2$
···	···	···
$Y_k$	$n_k-Y_k$	$n_k$

Under the exchangeable correlation model [12], the probability for the number of events from a cluster with the size of $n_i$ is computed as

$$P(Y_i=y|n_i)=\{\begin{array}{ll}\rho (1-\pi )+(1-\rho )(1-\pi {)}^{n_i},& y=0,\\ {\displaystyle (\genfrac{}{}{0ex}{}{n_i}{y})(1-\rho ){\pi }^y(1-\pi {)}^{n_i-y},}& y\in [1,n_i-1],\\ \rho \pi +(1-\rho ){\pi }^{n_i},& y=n_i.\end{array}$$ (1)

This probability density function is going to be used in the simulation studies.

2.1. Asymptotic intervals

Traditionally, the statistical inference for ρ is based on asymptotic approaches. For this reason, we first introduce four commonly used methods to construct asymptotic intervals for ρ.

2.1.1. ANOVA interval and ZD interval

In a one-way ANOVA, the between-cluster mean square (MSB) and the within-cluster mean square (MSW) are calculated as

$$\begin{array}{rl}MSB& =\frac{1}{k-1}[\sum _{i=1}^k\frac{Y_i^2}{n_i}-\frac{1}{N}{\left(\sum _{i=1}^k{Y}_i\right)}^2],\text{and}\\ MSW& =\frac{1}{N-k}[\sum _{i=1}^k{Y}_i-\sum _{i=1}^k\frac{Y_i^2}{n_i}].\end{array}$$

Then, the ICC estimate based on the ANOVA method is

$${\hat{\rho }}_{ANOVA}=\frac{MSB-MSW}{MSB+(n_0-1)MSW},$$ (2)

where $n_0=\frac{1}{k-1}(N-\sum _{i=1}^k{n}_i^2/N)$. Smith [29] developed the large sample variance estimator for ${\hat{\rho }}_{ANOVA}$ as

$$\begin{array}{rl}\text{var}({\hat{\rho }}_{ANOVA})& =\frac{2(1-\rho {)}^2}{n_0^2}[\frac{[1+\rho (n_0-1){]}^2}{N-k}+\frac{(1-\rho )[1+\rho (2n_0-1)]}{k-1}\\ & +\frac{{\rho }^2(M_2-2M_3/N+M_2^2/N^2)}{(k-1{)}^2}],\end{array}$$ (3)

where $M_2=\sum _{i=1}^k{n}_i^2$ and $M_3=\sum _{i=1}^k{n}_i^3$. A $100(1-\alpha )\mathrm{\%}$ confidence interval for ρ can be computed as

$$\hat{\rho }\pm z_{1-\alpha /2}\sqrt{\widehat{\text{var}}(\hat{\rho })},$$ (4)

where $z_{1-\alpha /2}$ is the $(1-\alpha /2)$ quartile of the standard normal distribution. The ANOVA interval is obtained by replacing $\hat{\rho }$ and its variance estimator with ${\hat{\rho }}_{ANOVA}$ in Equation (2) and $\widehat{\text{var}}({\hat{\rho }}_{ANOVA})$ in Equation (3).

Later, Zou and Donner [30] derived a variance estimator for ρ by using the moment functions and the delta method under the exchangeable correlation model:

$$\begin{array}{rl}\text{var}({\hat{\rho }}_{ANOVA})& =\{2k+[\frac{1}{\pi (1-\pi )}-6]\sum _{i=1}^k\frac{1}{n_i}+Z_1\rho +Z_2{\rho }^2+Z_3{\rho }^3\}\\ & \times \frac{{[(k-1)n_{0}N(N-k)]}^2}{{\lambda }^4},\end{array}$$ (5)

where $Z_1=[\frac{1}{\pi (1-\pi )}-6]\sum _{i=1}^k\frac{1}{n_i}-2N+7k-\frac{8k^2}{N}-\frac{2k(1-k/N)}{\pi (1-\pi )}+[\frac{1}{\pi (1-\pi )}-3]M_2$, $Z_2=\frac{N^2-k^2}{N\pi (1-\pi )}-2N-k+\frac{4k^2}{N}+[7-8\frac{k}{N}-\frac{2(1-k/N)}{\pi (1-\pi )}]M_2$, $Z_3=(\frac{1}{\pi (1-\pi )}-4)(\frac{N-k}{N}{)}^2(M_2-N)$, and $\lambda =(N-k)[N-1-n_0(k-1)]\rho +N(k-1)(n_0-1).$ When the variance estimator in Equation (4) is replaced by the variance estimator by Zou and Donner in Equation (5), this confidence interval is referred to be as the ZD interval.

2.1.2. FC interval

Fleiss and Cuzick [4] developed a kappa-type estimator for the ICC based on the probability of two individuals having the same outcome (both 0 or 1), ${\rho }_o$ for the two individuals from the same cluster and ${\rho }_e$ for them from different clusters. This approach is referred to be as the FC approach, with the ICC estimator as

$${\hat{\rho }}_{FC}=1-\frac{\sum _{i=1}^k\frac{Y_i(n_i-Y_i)}{n_i}}{(N-k)\pi (1-\pi )}.$$

Since π and $1-\pi$ are in the denominator, this estimator is not defined when $\pi =0$ or 1. The variance of ${\hat{\rho }}_{FC}$ is calculated as

$$\begin{array}{rl}\text{var}({\hat{\rho }}_{FC})& =(1-\rho )\{[\frac{1}{\pi (1-\pi )}-6]\frac{\sum _{i=1}^{k}1/n_i}{(N-k{)}^2}\\ & +[2N+4k-\frac{k}{\pi (1-\pi )}]\frac{k}{N(N-k{)}^2}+\rho F_1+{\rho }^2{F}_2\phantom{\frac{\sum _{i=1}^{k}1/n_i}{(N-k{)}^2}}\},\end{array}$$ (6)

where $F_1=\frac{M_2}{N^2\pi (1-\pi )}-\frac{(3N-2k)(N-2k)M_2}{N^2(N-k{)}^2}-\frac{2N-k}{(N-k{)}^2},$ and $F_2=[4-\frac{1}{\pi (1-\pi )}]\frac{M_2-N}{N^2}$. The FC interval is computed by using ${\hat{\rho }}_{FC}$ and its variance estimator $\text{var}({\hat{\rho }}_{FC})$ in Equation (4). Zou and Donner [30] found that the FC interval has coverages close to nominal over a wide range of parameter configurations as compared to the ANOVA interval and the ZD interval.

2.1.3. Pearson interval

Pearson's approach computes the ICC from all possible combinations of pairs within each cluster. For a cluster with the size of n, the total possible number of pairs is $n(n-1)/2$. For each pair, its correlation is computed. These $n(n-1)/2$ correlations are added as the overall correlation within that cluster. Pearson's ICC estimator gives equal weight to each pair of individuals to avoid giving more weights to clusters with larger sizes.

The ICC based on the Pearson's approach is presented as

$${\hat{\rho }}_{Pearson}=\frac{1}{\hat{\mu }(1-\hat{\mu })}[\frac{\sum _{i=1}^k{Y}_i(Y_i-1)}{\sum _{i=1}^k{n}_i(n_i-1)}-{\hat{\mu }}^2],$$

where $\hat{\mu }=\frac{\sum _{i=1}^k{Y}_i(n_i-1)}{\sum _{i=1}^k{n}_i(n_i-1)}$. The variance of ${\hat{\rho }}_{Pearson}$ is

$$\text{var}({\hat{\rho }}_{Pearson})=\frac{1-\rho }{\sum _{i=1}^k{[n_i(n_i-1)]}^2}\times [2\sum _{i=1}^k{n}_i(n_i-1)+\rho P_1+{\rho }^2{P}_2],$$ (7)

where $P_1=[\frac{1}{\pi (1-\pi )}-3]\sum _{i=1}^k{n}_i^2(n_i-1{)}^2$ and $P_2=[4-\frac{1}{\pi (1-\pi )}]\sum _{i=1}^k{n}_i(n_i-1{)}^3$. The Pearson interval is computed by using ${\hat{\rho }}_{Pearson}$ and its variance estimator $\text{var}({\hat{\rho }}_{Pearson})$ in Equation (4).

When $\hat{\rho }=1$, the variance estimates are zero for the ANOVA method, the FC method, and the Pearson method in Equations (3), (6), and (7). This occurs when all the individuals in each cluster have the same outcome.

2.2. Accurate intervals

The original data set $\mathbf{Y}$ in Table 1 can be re-organized by the size of clusters. Let W be the maximum number of cluster sizes: $W=max\{n_i,i=1,2,\dots ,k\}$. The re-organized data $\mathbf{M}$ is

$$\mathbf{M}=(m_{10},m_{11},m_{20},m_{21},m_{22},\dots ,m_{{}_{W0}},\dots ,m_{WW}),$$

where $m_{wj}$ is the number of clusters that have j events and the cluster size of w, $w=1,2,\dots ,W$, and $j=0,1,\dots ,w$. Then, $N_w=\sum _{j=0}^w{m}_{wj}$ is the number of clusters having the cluster size of w.

For clusters having the same cluster size of w, their numbers of events follow a multinomial distribution with the probabilities in Equation (1), specifically

$$P_w={\displaystyle (\genfrac{}{}{0ex}{}{N_w}{m_{w0},m_{w1},\dots ,m_{ww}})\prod _{j=0}^w{[P(Y=y_j|w)]}^{m_{wj}},}$$

It follows that the probability of data $\mathbf{Y}$ is

$$P(\mathbf{Y}\mathbf{|}\rho \mathbf{,}\pi )=\prod _{w=1}^W{P}_w,$$

which is a function of the parameter of interest ρ and the nuisance parameter π.

Let T be a statistical quantity to order the upper limits. The tail probability can be expressed as

$$G(\rho ,\pi )=P(T(\mathbf{Y})\le T({\mathbf{Y}}_{\mathbf{o}\mathbf{b}\mathbf{s}})|\rho ,\pi )$$ (8)

where ${\mathbf{Y}}_{\mathbf{o}\mathbf{b}\mathbf{s}}$ is an observed data. Exact one-sided upper limit by Buehler [2] can be computed as the supremum of ρ such that

$$G(\rho ,\pi )>\alpha /2.$$ (9)

In order to compute Buehler's exact one-sided limit [2], the complete sample space has to be enumerated, which is not computationally feasible for a study with clustered binary data due to the massive size of the sample space. Additionally, the tail probability has to be calculated for all possible values of π that lead to the computational intensity issue.

To overcome these challenges, we consider the importance sampling (IS) method described by Lloyd and Li [9] to estimate the tail probability in Equation (8). Importance sampling is able to reduce computational intensity by avoiding enumerating all data sets from the complete sample space [14,16]. Its estimate has a very good approximation to the true tail probability [9].

The importance sampling distribution is assumed to follow a multinomial distribution, with ρ and π estimated from the observed data set. From the fitted ANOVA model by using the observed data, the estimates of ${\hat{\rho }}_{obs}$ and ${\hat{\pi }}_{obs}$ are obtained and then used in the probability density functions in Equation (1). In the ANOVA model, the ${\hat{\pi }}_{obs}$ is calculated as the ratio of the total number of events and the total number of individuals. The number of events for the ith cluster with the size of $n_i$ is then generated from the estimated multinomial distribution. This step is repeated by k times to generate one IS data set having the same cluster sizes as the observed data. Suppose we have B importance data sets: ${\mathbf{Z}}_{\mathbf{1}},{\mathbf{Z}}_{\mathbf{2}},\dots ,{\mathbf{Z}}_{\mathbf{B}}$, where ${\mathbf{Z}}_{\mathbf{b}}=(Y_{1b},Y_{2b},\dots ,Y_{kb})$. These B importance samples are used to estimate the tail probability as

$$\hat{G}(\rho ,\pi )=\frac{1}{B}\sum _{b=1}^{B}I[T({\mathbf{Z}}_{\mathbf{b}})\le T({\mathbf{Y}}_{\mathbf{o}\mathbf{b}\mathbf{s}})]w({\mathbf{Z}}_{\mathbf{b}}|\rho ,\pi ),$$

where $w({\mathbf{Z}}_{\mathbf{b}}|\rho ,\pi )=p({\mathbf{Z}}_{\mathbf{b}}|\pi ,\rho )/p({\mathbf{Z}}_{\mathbf{b}}|{\hat{\rho }}_{obs},{\hat{\pi }}_{obs})$ is the importance weight and $I(.)$ is the indicator function.

When $\hat{G}(\rho ,\pi )$ is used as the estimator for G, one has to compute the upper limit as the supremum of ρ, such that $\hat{G}(\rho ,\pi )$ is larger than $\alpha /2$ for all the possible π values, which is computationally intensive. To further reduce the computational intensity, Kabaila and Lloyd [5] proposed the upper profile limit as the supremum of ρ such that

$$G(\rho ,\hat{\pi }(\rho ))>\alpha /2,$$ (10)

where $\hat{\pi }(\rho )$ is the MLE of π for a given ρ. In addition to that, the profile limit would help reduce the conservativeness of the confidence intervals as the lower limit and the upper limit are computed separately at the nominal of $1-\alpha /2$ for each limit. The accurate lower limit can be computed similar to the method for computing accurate upper limit. The aforementioned four asymptotic limits can be used as statistical quantities for sample space ordering in the accurate interval calculation. We refer to the accurate intervals based on the ANOVA limits, the ZD limits, the FC limits, and the Pearson limits as the ANOVA-IS interval, the ZD-IS interval, the FC-IS interval, and the Pearson-IS interval, respectively.

3. Results

We first compare the performance of asymptotic intervals and accurate intervals for the ICC with regard to coverage probability when all the clusters have the same cluster size. The coverage probability is calculated as the proportion of the simulated data sets whose confidence intervals contain the pre-specified ρ value. When all the clusters have the same size, the asymptotic FC intervals and the asymptotic Pearson intervals are identical. For this reason, we only need to compare the performance of the following three intervals: the ANOVA interval, the ZD interval, and the FC interval. Figure 1 shows the coverage probabilities of these asymptotic intervals with the number of clusters: k = 20, 50 and 100, and the common cluster size 5 or 8, at the nominal level of 95%. The parameter of interest is set to be as $\rho =0.1$, 0.5, and 0.9, and the event rates π are 0.1, 0.3, 0.6, and 0.9. For each configuration, we simulate 2,000 data sets using the probability density functions in Equation (1). For each simulated data set, we fit the ANOVA model to obtain the parameter estimates: $\hat{\pi }$ and $\hat{\rho }$. We then simulate 5,000 importance samples from the multinominal distribution whose parameters are replaced with $\hat{\pi }$ and $\hat{\rho }$.

Figure 1.

Coverage probabilities of asymptotic intervals when the number of clusters is $k=20,50,$ and 100 and a common cluster size is 5 or 8.

It can be seen from Figure 1 that the ANOVA interval has the lowest coverage as compared to the other two methods. At the nominal level of 95%, the coverage probability of the ANOVA interval could be as low as 40% in some configurations. Both the ZD interval and the FC interval have the coverage around the nominal level. The ZD interval often has a higher coverage than the FC interval. When the number of cluster is k = 100, the ZD interval controls the coverage well in many cases with the actual coverage probabilities above 95%. The FC interval has the coverages close to the nominal level, but the coverages are often below the nominal level.

As compared to the coverage of asymptotic intervals in Figure 1, the proposed accurate intervals generally control well for coverage, see Figure 2. Although the accurate intervals based on the IS approach do not theoretically guarantee the coverage, the computed accurate intervals often have very good performance with regard to coverage. The accurate ANOVA-IS intervals occasionally have the coverage slightly below the nominal level when ρ is high.

Figure 2.

Coverage probabilities of accurate intervals when the number of clusters is $k=20,50,$ and 100 and a common cluster size is 5 or 8.

We then compare their interval widths in Figure 3 for accurate intervals. The results for a study with the cluster size of 5 are similar to those with the cluster size of 8. When the number of clusters is small (k = 20), the ZD-IS interval has the shortest width as compared to the ANOVA-IS interval and the FC-IS interval in many configurations. The ANOVA-IS interval has the shortest width when ρ is 0.9 and π is near the boundary, and a study with small number of clusters. As the number of clusters increases from k = 20 to k = 100, the FC-IS interval and the ZD-IS interval have similar interval width, and they have shorter width than the ANOVA-IS interval. When k = 100, the width of the ZD-IS interval is the shortest when ρ is 0.9 and π is near the boundary. The simulations for a study with a common cluster size indicate that the ZD-IS interval and the FC-IS interval outperform the ANOVA-IS interval with regard to coverage and interval width. For this reason, we exclude the ANOVA-IS interval in the following simulations for a study with multiple cluster sizes.

Figure 3.

Interval width of accurate intervals when the number of clusters is $k=20,50,$ and 100 and a common cluster size is 5 or 8.

The FC-IS interval and the Pearson interval are different from each other when a study has more than one cluster size. Figure 4 shows the coverage probability and interval width of the three asymptotic intervals and their associated accurate intervals, with 60 clusters (20 clusters each for the cluster sizes of 3, 4, and 5). The same configurations of ρ and π as the aforementioned simulations are studied. The asymptotic ZD interval is very conservative when ρ and π are near the boundary, having the actual coverage much larger than the nominal level. The FC interval always has a higher coverage than the Pearson interval, and both are below the nominal level when $\rho =0.1$ or 0.5. When $\rho =0.9$, the FC interval and the Pearson interval have coverages around the nominal level, while the coverage of the ZD interval ranges from 0.9 to 1. Meanwhile, accurate intervals control the coverage much better than asymptotic intervals as seen in the figure.

Figure 4.

Coverage probability and interval width of asymptotic intervals (left) and accurate intervals (right) for a study with 3 different cluster sizes: 3, 4, and 5. The number of clusters is 20 for each cluster size with a total of 60 clusters. The computed coverage probability and interval width are plotted as a function of ρ and π.

Among the accurate intervals, the ZD-IS interval has the shortest width when both ρ and π are near the boundary. When ρ is large and π is near the boundary, the ZD-IS interval is the best among them, in which the other two intervals often have very small accurate lower limits. The accurate FC-IS interval often performs better than others with shorter widths in other cases. Similar results are observed in Figure 5 when the number of clusters is increased to k = 90.

Figure 5.

Coverage probability and interval width of asymptotic intervals (left) and accurate intervals (right) for a study with 3 different cluster sizes: 3, 4, and 5. The number of clusters is 30 for each cluster size with a total of 90 clusters. The computed coverage probability and interval width are plotted as a function of ρ and π.

3.1. Example

We use the psychiatric data reported by Lipsitz et al. [8] to illustrate the application of the proposed accurate intervals for ρ. That study had 26 patients who are considered as clusters here. Each patient with psychiatric disorders is assessed by at least three different psychiatrists, with the outcome as the status of disorder (neurosis versus other disorder). The cluster sizes are 3, 4, 5, and 6. The detailed data set can be found in Lipsitz et al. [8].

The estimated ρ values based on the ANOVA method, the FC method, and the Pearson method are presented in Table 2. The estimate based on the FC method is similar to that based on the Pearson method, and they are slightly less than the estimate based on the ANOVA method. The accurate lower limits of the ZD-IS interval, the FC-IS interval, and the Pearson interval are larger than their associated asymptotic lower limits, while the differences in upper limits are small. The accurate ANOVA-IS interval is wider than the other accurate intervals. Suppose we are interested in testing whether ρ is above 20% under the alternative. Three IS intervals (the ZD-IS interval, the FC-IS interval, and the Pearson-IS interval) reject the null hypothesis while their asymptotic intervals fail to reject the null hypothesis. The ANOVA-IS interval is the only IS interval that fails to reject the null hypothesis among the four IS intervals.

Table 2.

Asymptotic and accurate intervals for the psychiatric study.

ρ	Method for interval	Asymptotic	Accurate
${\rho }_{ANOVA}=0.422$	ANOVA	(0.231,0.613)	(0.182,0.690)
	ZD	(0.195,0.649)	(0.214,0.639)
${\rho }_{FC}=0.409$	FC	(0.187,0.632)	(0.209,0.637)
${\rho }_{Pearson}=0.408$	Pearson	(0.179,0.637)	(0.206,0.642)

4. Discussion

We study the four accurate confidence intervals for ρ in a clustered binary study. Four asymptotic limits are used as statistical quantities in computing these accurate intervals. Another asymptotic interval proposed by Chakraborty and Hossain [3] is a simulation based method that would require more computational resources [28]. That approach would introduce additional simulation errors in computing the confidence interval for ρ. We do not have the computational resources to conduct simulations using that method.

In addition to the Wald-type confidence intervals studied in this article, confidence intervals can be solved from cubic functions of ρ, similar to the Wilson type confidence intervals [30]. The proposed accurate confidence intervals are computed by using the ordering information of upper limits or lower limits [20]. We do not expect significant changes with regard to sample space ordering between Wald-type limits and Wilson type limits. In addition, more computational resources are required to solve cubic functions [15,17,26,27]. This could be considered as future research on the difference between Wald type limits and Wilson type limits for accurate intervals calculation.

Similar to the κ statistic for testing agreement, there is a paradox about the ICC estimate: a high probability that two individuals from the same cluster have the same number of events, but a low ρ estimate. For example, a study with k = 50 clusters having a common cluster size of 5, the outcome is $\mathbf{M}=(m_{50},m_{51},m_{52},m_{53},m_{54},m_{55})=(0,0,0,0,1,49)$, where 1 cluster has 4 events and 1 non-event, and the remaining 49 clusters all have 5 events. Given that almost every cluster has 5 events, the probability of two individuals from the same cluster having the same results would be very high, and it should be close to 1. However, the estimated ρ is negative: -0.004 for the FC method and -3.4 $\times {10}^{-14}$ for the ANOVA method. This could be caused by the large value of π that leads to a very large denominator in the ${\rho }_{FC}$ estimate. When π is near 0.5, the estimated ${\rho }_{FC}$ is closer to the probability.

Funding Statement

Research reported in this publication was supported by the National Institutes of Health under Award Number R01AG070849, and R03CA248006.

Disclosure statement

No potential conflict of interest was reported by the author(s).