Sample Size Calculation Guide - Part 4: How to Calculate the Sample Size for a Diagnostic Test Accuracy Study based on Sensitivity, Specificity, and the Area Under the ROC Curve

Introduction

In the previous educational articles, we explained how to calculate the sample size for a rate or a single proportion, for an independent cohort study, and for an independent case-control study (1–3). In this article, we will explain how to calculate the sample size for a diagnostic test accuracy study based on sensitivity, specificity, or the area under the ROC curve.

When to use the sample size calculation procedure of diagnostic performance

The methods explained hereafter should be used in the case that the diagnostic performance of your new test (new device, survey, or biomarker) is expressed as sensitivity, specificity, or area under the ROC curve. The definitions of sensitivity, specificity, or area under the ROC curve were explained by us in previous education editorials (4, 5).

• Sample Size Calculation based on sensitivity or specificity

We will use the sample size calculation methods of Buderer et al.1996 (6). In this method, we need first to calculate the TP+FN for sensitivity and the TN+FP for specificity through the following equation.

$$\begin{array}{l}TP+FN=Z^2\hspace{0.17em}x\frac{\mathit{Sensitivity}\hspace{0.17em}(1-\mathit{Sensitivity})}{W^2}\\ TN+FP=Z^2\hspace{0.17em}x\frac{\mathit{Specificity}\hspace{0.17em}(1-\mathit{Specificity})}{W^2}\end{array}$$

Where Z, the normal distribution value, is set to 1.96 as corresponding with the 95% confidence interval, W, the maximum acceptable width of the 95% confidence interval, is set to 10%, and the expected sensitivity and specificity are defined based on the estimates from previous studies.

The next step is to calculate N required for sensitivity and N required for specificity through the following equations:

• N required for sensitivity

  $$\frac{TP+FN}{P}$$
• N required for specificity

  $$\frac{TN+FP}{1-P}$$

Example: a study to evaluate the accuracy of blood pressure to height ratio as a diagnostic tool for hypertension among adolescents

Assume that we will conduct a study to estimate the accuracy of blood pressure to height ratio as a diagnostic tool for hypertension in adolescents in Egypt. Therefore, we will enroll a group of adolescents including those with hypertension and those without hypertension. Each subject will be screened twice, first time by the gold standard test (reference test), then by the new test (blood pressure to height ratio).

A previous similar study reported a sensitivity of 90% and specificity of 90% while the prevalence rate of hypertension in Egyptian adolescents was 5% (7).

To calculate the sample size required for this study, we apply the above-mentioned equations and the results were as follows:

• TP + FN = 34.5

• TN + FP = 34.5

Then, we calculate the N required for sensitivity and the N required for specificity, as follows:

• N required for sensitivity

  $$\frac{TP+FN}{P}=\frac{34.5}{0.05}=691\hspace{0.17em}\text{participants}$$
• N required for specificity

  $$\frac{TN+FP}{1-P}=\frac{34.5}{1-0.05}=36\hspace{0.17em}\text{participants}$$
• Total required sample size

  $$691+36=728\hspace{0.17em}\text{participants}$$

Therefore, in this study, should include 691 participants with hypertension and 36 participants without hypertension yielding a total sample size of 728 participants.

These equations were programmed by a Vietnamese biostatistician into an android app named “statistics and sample size pro”. By providing the same inputs, we obtain similar estimates (Figure 1).

Figure 1:

shows calculating the sensitivity and specificity by an android app

• Sample size calculation based on the area under the ROC curve

This will require to provide the following inputs in MedCalc software

1. Expected AUC

2. Null value of the AUC (usually 50% is the null value)

3. Ratio between negative and positive cases

Example: a study to evaluate the accuracy of CSF lactate in discriminating the bacterial meningitis from enteroviral meningitis.

Assume that we will conduct a study to estimate the accuracy of CSF lactate to discriminate bacterial meningitis from enteroviral meningitis. Therefore, we will enroll a group of patients with acute meningitis including those with bacterial meningitis and those with enteroviral meningitis. For each CSF specimen, bacterioscopy, bacterial antigen latex agglutination test and CSF bacterial culture will be performed as a standard test (reference test), then the CSF lactate will be estimated (new test).

A previous study by Manomaivat et al. showed that the AUC of CSF lactate was 94% for discriminating bacterial meningitis from enteroviral meningitis (8). The ratio between negative and positive cases was 525/662.

In order to calculate the sample size required for our new study, we will provide the inputs to MedCalc software as follows:

First, open the software then select “sampling” for sample size calculation options then, select “area under the ROC curve” (Figure 2). Finally, submit the data and check the table for the calculation results. As shown in figure 3, the results table shows a sample size of 11 patients (5 cases of enteroviral meningitis and 6 cases of bacterial meningitis) corresponding with a 5% alpha error and a 10% beta error.

Figure 2:

MedCalc menu

Figure 3:

The results table