Reference range: Which statistical intervals to use?

Abstract

Reference ranges, which are data-based intervals aiming to contain a pre-specified large proportion of the population values, are powerful tools to analyse observations in clinical laboratories. Their main point is to classify any future observations from the population which fall outside them as atypical and thus may warrant further investigation. As a reference range is constructed from a random sample from the population, the event ‘a reference range contains $\left(100P\right)\%$ of the population’ is also random. Hence, all we can hope for is that such event has a large occurrence probability. In this paper we argue that some intervals, including the P prediction interval, are not suitable as reference ranges since there is a substantial probability that these intervals contain less than $\left(100P\right)\%$ of the population, especially when the sample size is large. In contrast, a $(P,\gamma )$ tolerance interval is designed to contain $\left(100P\right)\%$ of the population with a pre-specified large confidence γ so it is eminently adequate as a reference range. An example based on real data illustrates the paper’s key points.

1 Introduction

The ‘Choose Wisely’ campaign was developed in the United States in 2012 by the American Board of Internal Medicine Foundation and was launched in the United Kingdom in 2016 by the Academy of Medical Royal Colleges. It aims to encourage a dialogue between clinicians and patients regarding the risk and benefits of interventions, and the practice of evidence-based treatment regimens.¹ As described recently,² this conversation often refers to the patient’s observed values of relevant clinical markers. Since the clinical laboratory provides comparator intervals to assist the clinician in determining a context for an individual value, a natural question from the patient is ‘Are my test results typical with respect to a healthy population?’. Although such assessment values are often referred to as the test’s normal range, this terminology should be discouraged as it implies that such a result has a binary ‘normal or abnormal’ quality which may lead to an arbitrary dichotomous interpretation of the patient’s health status.² Instead, the terms ‘reference limits’ or ‘reference range’ should be used in this context.

Reference ranges are powerful tools in laboratory medicine to aid decision making³ and their use has become increasingly prevalent in clinical practice. Searching in the Web of Science engine at the time of writing for articles published between 1999 and 2019 with ‘reference range’ as a topic, we found 5431 articles of which 469 appeared in 2019, in contrast to 268 articles that appeared in 2009. These articles have collectively been cited by 91,034 publications of which 11,270 appeared in 2019, 2.4 times more than the number of citing publications 10 years earlier.

Apart from the important individual overtones for patients, incorrectly estimating the reference range of a sensitive clinical marker of physiological function has enormous public health implications. For example, underestimating the upper limit of a reference range would mean classifying a large number of people as diseased, thus affecting the doses of medication prescribed.⁴ Construction of appropriate reference ranges is therefore crucial in laboratory medicine practice. Well-known general references^3,5–9 and a case for teaching tolerance intervals in introductory statistics courses¹⁰ are available.

It is common practice to assume that clinical markers related to a disease follow a normal distribution among healthy subjects. If there is evidence against this assumption we could fit models to specify optimal transformations to normality, e.g. logarithmic or square root though this might still result in biased estimates of the upper or lower limits of the reference range depending on whether the distribution is right or left skewed.⁹ Alternatively we could construct reference ranges under specific parametric assumptions different to normality, or follow a nonparametric procedure. The focus of this paper is on the construction of parametric and nonparametric reference ranges for a selected reference population based on a random sample from the population. The problems related to selecting a reference population have been discussed elsewhere.⁶

A P (commonly set to 95%) reference range is a data-based interval that purports to include $\left(100P\right)\%$ of the values in the population of interest. Their main point is to classify any future observations from the population which fall outside these intervals as atypical and thus may warrant further investigation.

Let $F(·)$ denote the continuous cumulative distribution function (cdf) of the population, and $F^{-1}(\gamma )$ denote the $(100\gamma )$-th percentile of the population for a given $\gamma \in (0,1)$. The interval $\left[F^{-1}\left((1-P)/2\right),F^{-1}\left((1+P)/2\right)\right]$ contains exactly $\left(100P\right)\%$ of the population and would be used as the P reference range had F been known. Since $F(·)$ is usually not known completely in real problems, the reference range has to be estimated from a random sample $X_1,\dots ,X_n$ from the population, i.e. $X_1\dots ,X_n$ are independent random variables identically distributed $F(·)$. Note that we follow the notation in Krishnamoorthy and Mathew¹¹ thus denoting the interval’s content level by P instead of the commonly used $1-\alpha$, and its confidence level by γ.

When $F(·)$ is assumed to have a normal distribution $N(\mu ,{\sigma }^2)$ with unknown mean μ and unknown variance ${\sigma }^2$, we have $F^{-1}(\gamma )=\mu +z_{\gamma }\sigma$ where $z_{\gamma }$ denotes the $(100\gamma )$-th percentile of the standard normal distribution N(0, 1). When $F(·)$ is not assumed to have a parametric form, nonparametric (or distribution free) methods can be used. In this paper, both normal-based and nonparametric methods are considered.

As a reference range depends on the random sample, the proportion of the population contained in it is also random. Thus the question is ‘which statistical intervals should be used as reference ranges?’

In this article we argue that a P prediction interval, which continues to be used as a reference range in the literature,^6,12,13 is not fit for the purpose of interest since there is a substantial probability (due to the randomness in the sample) that the prediction interval contains less than $\left(100P\right)\%$ of the population.

We then argue that a $(P,\gamma )$ tolerance interval, with confidence $\gamma \in (0,1)$ set at a pre-specified large value, $\gamma =0.95$ say, is valid as a reference range since it guarantees, with large confidence γ due to the randomness in the sample, to contain $\left(100P\right)\%$ of the population values. Several authors have proposed to use tolerance intervals as reference ranges.^5,14,15 With almost 80 years of research on tolerance intervals or regions, various parametric and nonparametric procedures are readily available for use as reference ranges.

The next two sections discuss reference ranges based on the normal distribution, and nonparametric reference ranges. They are followed by a section considering a numerical example, and a final one with concluding remarks.

2 Reference ranges based on the normal distribution

2.1 Reference ranges currently in use

Based on the sample, one reference range that has been widely used is the P prediction interval for a future observation Y from a population with $N(\mu ,{\sigma }^2)$ distribution^6,12,13

$$R{R}_1=\overline{X}\pm t_{(1+P)/2,\nu }S\sqrt{1+1/n}=\overline{X}\pm c_{1}S$$

where $\overline{X}=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{X}_i}$ is the sample mean, $S^2=\frac{1}{n-1}{\displaystyle {\sum }_{i=1}^n{(X_i-\overline{X})}^2}$ is the sample variance, $t_{\delta ,\nu }$ is the $(100\delta )$-th percentile of the t distribution with ν degrees of freedom (df), $\nu =n-1$, and $c_1=t_{(1+P)/2,\nu }\sqrt{1+1/n}$.

A relevant guide on prediction intervals for reference regions is available,⁷ and we note that the prediction interval RR₁ has also been called the P expectation tolerance interval.^16,17

Other reference ranges are based on estimators of the percentiles $\mu \pm z_{(1+P)/2}\sigma$ and include

$$\begin{array}{l}R{R}_2=\overline{X}\pm z_{(1+P)/2}S=\overline{X}\pm c_{2}S\\ R{R}_3=\overline{X}\pm z_{(1+P)/2}S/{\lambda }_{\nu }=\overline{X}\pm c_{3}S\\ R{R}_4=\overline{X}\pm z_{(1+P)/2}S{\lambda }_{\nu }=\overline{X}\pm c_{4}S\end{array}$$

where ${\lambda }_{\nu }=\sqrt{2/\nu }\Gamma \left((\nu +1)/2\right)/\Gamma (\nu /2),c_2=z_{(1+P)/2}$, $c_3=z_{(1+P)/2}/{\lambda }_{\nu }$ and $c_4=z_{(1+P)/2}{\lambda }_{\nu }$.^9,12 Now $\overline{X}+c_{2}S$ is a naïve estimator of $\mu +z_{\gamma }\sigma ,\overline{X}+c_{3}S$ has the minimum variance among unbiased estimators of $\mu +z_{\gamma }\sigma$, and $\overline{X}+c_{4}S$ has minimum mean squared error among estimators of the form $\overline{X}+cS$ where c is a constant.¹²

One immediate question is whether these reference ranges RR_i contain $\left(100P\right)\%$ of the values in the population, which is the objective of a reference range. Note that the proportion of the population within the reference range $R{R}_i=\overline{X}\pm c_{i}S$ is given by

$$K_i=\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\left\{Y\in \overline{X}\pm c_{i}S\right\}=\Phi \left(\frac{\overline{X}-\mu }{\sigma }+c_i\frac{S}{\sigma }\right)-\Phi \left(\frac{\overline{X}-\mu }{\sigma }-c_i\frac{S}{\sigma }\right)$$ (1)

where $Y\sim N(\mu ,{\sigma }^2)$ and is independent of the sample $X_1,\dots ,X_n$, $\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\{·\}$ is the conditional probability of Y conditioning on the sample $X_1,\dots ,X_n$, and $\Phi (·)$ is the cdf of a N(0, 1) random variable. Hence the objective of a reference range is to have $K_i\ge P$. It is clear from equation (1) that K_i is a random variable depending on the random sample via $\overline{X}$ and S so whether ‘$K_i\ge P$’ is also random. As a result, all we can hope is that the event $\left\{K_i\ge P\right\}$ has a large probability of occurrence.

We note from equation (1) that K_i increases as c_i increases. Hence, among the RR_i ($1\le i\le 4$) given above, the one that has the largest c_i contains the largest proportion of the population. Figure 1 compares the c_i for given sample sizes $n=2:150$ and P = 0.95. Clearly, c₁ is the largest among the c_i ($1\le i\le 4$), and so RR₁ contains the largest proportion of the population among the four reference ranges. We therefore investigate whether or not ‘$K_1\ge P$’ has a large probability to occur in order for RR₁ to be used as a reference range.

Figure 1.

The value of c_i as a function of the sample size n.

First, note that

$$\text{E}(K_1)=\text{E}\left\{\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\left\{Y\in \overline{X}\pm c_{1}S\right\}\right\}=\mathrm{Pr}\left\{Y\in \overline{X}\pm c_{1}S\right\}$$ (2)

$$=\mathrm{Pr}\left\{\frac{\left|Y-\overline{X}\right|/\left(\sigma \sqrt{1+1/n}\right)}{S/\sigma }<t_{P/2,\nu }\right\}=P$$ (3)

where the equality in equation (2) results directly from the well-known conditional expectation formula,¹⁸ and the equality in equation (3) follows from the fact that $\left(Y-\overline{X}\right)/\left(\sigma \sqrt{1+1/n}\right)$ is distributed N(0, 1) and is independent of $S/\sigma$ which has the distribution $\sqrt{{\chi }_{\nu }^2/\nu }$, with ${\chi }_{\nu }^2$ denoting a chi-squared random variable with $\nu =n-1$ df. That the probability in equation (3) is equal to P qualifies RR₁ as a P prediction interval for a future observation Y from the same population that the sample $X_1,\dots ,X_n$ is drawn.

Second, the distribution of K₁ can be studied by simulating a large number, $R_{\mathit{sim}}=1,000,000$ say, of independent realisations of K₁. Note from equation (1) that

$$K_1=\Phi \left(\frac{Z}{\sqrt{n}}+c_1\sqrt{\frac{{\chi }_{\nu }^2}{\nu }}\right)-\Phi \left(\frac{Z}{\sqrt{n}}-c_1\sqrt{\frac{{\chi }_{\nu }^2}{\nu }}\right)$$ (4)

where $Z=\sqrt{n}(\overline{X}-\mu )/\sigma$ is a standard N(0, 1) random variable, ${\chi }_{\nu }^2=\nu S^2/{\sigma }^2$ is a chi-squared random variable with $\nu =n-1$ df, and Z and ${\chi }_{\nu }^2$ are statistically independent. From equation (4), K₁ can easily be simulated. For given P and n, $R_{\mathit{sim}}=1,000,000$ replicas of K₁ are simulated, based on which the probability density function (pdf) of K₁ can be accurately approximated. In Figure 2, the kernel density estimate¹⁹ of the pdf of K₁ based on the simulated K₁ values is plotted (by using the R package KernSmooth)²⁰ for $n=\mathrm{20}$, 50, 100 and 150. Based on the simulated $K_1$ values, we approximated $\mathrm{Pr}\{K_1<P\}$ by the proportion of the $K_1$ values that are less than $P=\mathrm{0.95}$, which are given by 0.385, 0.429, 0.450 and 0.459 for $n=\mathrm{20}$, 50, 100 and 150, respectively. Note that $\mathrm{Pr}\{K_1<P\}$ is given in Figure 2 by the area under the pdf to the left of the vertical line at $P=\mathrm{0.95}$.

Figure 2.

The pdf’s of K₁ for various sample sizes n.

Given equation (3), it can be shown by the delta method that $\sqrt{n}\left(K_1-P\right)$ tends when $n\to \infty$ to a normal distribution with zero mean and finite variance. This is supported by Figure 2 which shows that the pdf of K₁ is getting closer to be symmetric and centered with decreasing variance at P as n increases. Note that n = 150 is not large enough yet for the pdf of K₁ to converge to a normal pdf. From a brief simulation study we found that in order to achieve this satisfactorily the sample size must be very large indeed. Even for n = 10, 000 the skewness and kurtosis values suggest a significant lack of normality. The coefficient of variation of K₁ for n = 150 is 0.014, and becomes smaller than 0.01 for $n\ge 300$, and is around 0.002 for n = 10,000. This asymptotic normal distribution implies that $\mathrm{Pr}\{K_1<P\}\to 0.5$ as $n\to \infty$, that is, the probability of the reference range RR₁ containing less than $\left(100P\right)\%$ of the population is about 1/2 when the sample size is large.

The argument above means that, due to the sample’s randomness, using RR₁ as the reference range implies that there is a substantive probability, close to 50% when n is sufficiently large, that the reference range does not fulfill its objective of containing $\left(100P\right)\%$ of the population. Its property $\text{E}(K_1)=P$ in equation (3) has the following interpretation. A large number of individuals, I say, collect independent samples, one each, and compute the corresponding reference ranges RR₁ based on their own samples. Then the proportions of the population contained in these I reference ranges, $K_{1,1},\dots ,K_{1,I}$, are random values from the interval (0, 1) and form a random sample from the distribution of K₁ although some values could be very close to 0 and some values could be very close to 1. The property $\text{E}(K_1)=P$ merely says that $(K_{1,1}+\cdots +K_{1,I})/I$ is close to P when I is large. Hence, the proportion of the population that one particular reference range contains could be very small but this is compensated by some very large proportions of the population that some other individuals’ reference ranges might contain in the sense that $(K_{1,1}+\cdots +K_{1,I})/I$ is close to P. This potential for compensation from other reference ranges is unlikely to offer any comfort for knowing that one’s reference range has a substantial probability of containing less than $\left(100P\right)\%$ of the population. It is clearly desirable to have a high confidence that our own reference range contains $\left(100P\right)\%$ of the population. Hence RR₁ falls short on this ground and should not be used as a reference range.

The justification for using prediction intervals as reference ranges^5,13 is that exactly $\left(100P\right)\%$ of the future observations from the population should fall within the prediction intervals. It is clear from the line of reasoning stated in the previous paragraphs that this argument is not valid. The inappropriateness of prediction regions when used as reference regions has also been noted in Sections 2.2 and 3.3 of Dong and Mathew.¹⁵

In the next section we discuss tolerance intervals since several authors^5,14,15 have proposed to use them as reference ranges. For example it has been stated that ‘it would seem that the statistical tolerance interval is what clinical chemists have in mind when they speak of a reference range derived from a sample of individuals representing some defined population’⁵ (p. 55).

2.2 Tolerance intervals

A tolerance interval with content level P is a data-based random interval constructed to contain $\left(100P\right)\%$ of the population with a pre-specified (large) confidence level γ about the randomness in the sample.^{11,16,17,21–23} Specifically, a $(P,\gamma )$ tolerance interval is given by¹¹

$$R{R}_5=\overline{X}\pm c_{5}S$$

where the critical constant $c_5=c_5(P,\gamma ,n)$ is chosen such that

$$\begin{array}{l}\mathrm{Pr}\left\{\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\left\{Y\in \overline{X}\pm c_{5}S\right\}\ge P\right\}=\mathrm{Pr}\left\{\Phi \left(\frac{\overline{X}-\mu }{\sigma }+c_5\frac{S}{\sigma }\right)-\Phi \left(\frac{\overline{X}-\mu }{\sigma }-c_5\frac{S}{\sigma }\right)\ge P\right\}\\ =\mathrm{Pr}\left\{\Phi \left(Z/\sqrt{n}+c_5\sqrt{{\chi }_{\nu }^2/\nu }\right)-\Phi \left(Z/\sqrt{n}-c_5\sqrt{{\chi }_{\nu }^2/\nu }\right)\ge P\right\}=\gamma \end{array}$$ (5)

where the random variables Z and ${\chi }_{\nu }^2$ in equation (5) are the same as those in equation (4). The R package tolerance^24,25 can be used to compute c₅.

Figure 3 compares c₁ and c₅ for given sample sizes $n=2:150$ with P = 0.95 and $\gamma =\left\{0.90,0.95\right\}$. It is clear from Figure 3 that c₅ is considerably larger than c₁ in order that RR₅ contains $\left(100P\right)\%$ of the population with a pre-specified large confidence γ about the randomness in the sample. Also, as expected, c₅ increases with γ as seen in Figure 3.

Figure 3.

The values of c₁ and c₅ for various sample sizes n.

3 Equal-tailed tolerance intervals

The tolerance interval RR₅ contains $\left(100P\right)\%$ of the population with a pre-specified (large) confidence γ about the randomness in the sample. But the proportion P of the population contained in RR₅ may not be the central $\left(100P\right)\%$ interval of the population. If we insist that a reference range should contain that central proportion of the population, i.e. $[\mu -z_{(1+P)/2}\sigma ,\mu +z_{(1+P)/2}\sigma ]$ with pre-specified confidence γ about the randomness in the sample, then we should use the following interval as the reference range

$$R{R}_6=\overline{X}\pm c_{6}S$$

where the critical constant $c_6=c_6(P,\gamma ,n)$ is chosen such that

$$\mathrm{Pr}\left\{\overline{X}-c_{6}S<\mu -z_{(1+P)/2}\sigma \text{ and }\mu +z_{(1+P)/2}\sigma <\overline{X}+c_{6}S\right\}=\gamma$$

This interval is called the equal-tailed or central $(P,\gamma )$ tolerance interval.¹⁵ A formula for values of c₆ is available¹¹ and can be computed using the function K.factor of the R package tolerance.^24,25 This interval can be viewed as a γ confidence simultaneous lower confidence bound on quantile $\mu -z_{(1+P)/2}\sigma$ and upper confidence bound on quantile $\mu +z_{(1+P)/2}\sigma$.²⁶

It is clear that comprising the central $\left(100P\right)\%$ of the population $[\mu -z_{(1+P)/2}\sigma ,\mu +z_{(1+P)/2}\sigma ]$ implies containing $\left(100P\right)\%$ of the population. Hence the equal-tailed RR₆ satisfies a more stringent requirement than RR₅ and, as a result, c₆ is larger than c₅.

Figure 4 compares c₅ and c₆ for given sample sizes $n=2:150$ with P = 0.95 and confidence $\gamma =\left\{0.90,0.95,0.99\right\}$. It is clear from Figure 4 that $c_6>c_5$, as expected.

Figure 4.

The values of c₅ and c₆ for various sample sizes n.

Our view is that the $(P,\gamma )$ tolerance interval should be used as the reference range since its form $\overline{X}\pm c_{5}S$ is centered at $\overline{X}$, mimicking the form of the equal-tailed tolerance interval $\mu \pm c_6\sigma$, and with a large confidence γ it does contain $\left(100P\right)\%$ of the population. Only if we specifically require the reference range to contain the central $\left(100P\right)\%$ of the population, $\mu \pm z_{(1+P)/2}\sigma$, then the equal-tailed $(P,\gamma )$ tolerance interval should be used; otherwise it is unnecessarily wider and flags as atypical fewer individuals than the $(P,\gamma )$ tolerance interval.

4 Nonparametric reference ranges

4.1 Nonparametric prediction intervals

When $F(·)$ is not assumed to have a specific form, nonparametric reference ranges can be considered and are based on the order statistics $X_{[1]}<\dots <X_{[n]}$ of the sample $X_1,\dots ,X_n$, and the sample quantiles have been used to estimate the population quantiles $F^{-1}\left(\left(1-P\right)/2\right)$ and $F^{-1}\left(1+P)/2\right)$.⁶

In what follows, $j^{(p)}$ and $j^{(t)}$ are indices used for prediction and tolerance intervals, respectively.

Let $j^{(p)}$, with $1\le j^{(p)}\le n/2$, be the largest natural number such that

$$\mathrm{Pr}\left\{Y\in (X_{[j^{(p)}]},X_{[n-j^{(p)}+1]})\right\}\ge P$$ (6)

where Y is a future observation from the population $F(·)$ independent of the random sample $X_1,\dots ,X_n$ as before. Using the well-known facts that $U_1=F(X_1),\dots ,U_n=F(X_n)$ are independent, each having a uniform distribution on the interval (0, 1), and that $U_{[k]}=F(X_{[k]})$ is the k-th order statistic of $U_1,\dots ,U_n$ and has a beta distribution with parameters k and $n-k+1$, the probability in (6) is equal¹⁶ to $\left(n+1-2j^{(p)}\right)/(n+1)$. Hence the constraint on $j^{(p)}$ required in equation (6) gives

$$j^{(p)}=〈(n+1)\left(1-P\right)/2〉$$ (7)

where $〈a〉$ denotes the integer part of a. This leads to use the nonparametric prediction interval

$$R{R}_7=\left(X_{[j^{(p)}]},X_{[n-j^{(p)}+1]}\right)$$

as a reference range. An interesting remark is that $X_{[j^{(p)}]}$ and $X_{[n-j^{(p)}+1]}$ are consistent point estimators of the population quantiles $F^{-1}\left((1-P)/2\right)$ and $F^{-1}\left((1+P)/2\right)$, respectively.

The proportion of the population contained in RR₇ is given by

$$\begin{array}{l}{K}_7=\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\left\{Y\in (X_{[j^{(p)}]},X_{[n-j^{(p)}+1]})\right\}\\ =\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\left\{F(Y)\in \left(F(X_{[j^{(p)}]}),F(X_{[n-j^{(p)}+1]})\right)\right\}\\ =U_{[n-j^{(p)}+1]}-U_{[j^{(p)}]}\end{array}$$ (8)

which is a random variable. The important question is whether the probability that this proportion is at least P, given by

$$\mathrm{Pr}\left\{U_{[n-j^{(p)}+1]}-U_{[j^{(p)}]}\ge P\right\}$$ (9)

is sufficiently large to qualify the P prediction interval $R{R}_7=\left(X_{[j^{(p)}]},X_{[n-j^{(p)}+1]}\right)$ as a reference range.

By noting that $U_{[n-j^{(p)}+1]}-U_{[j^{(p)}]}$ and $U_{[n-2j^{(p)}+1]}$ follow the same beta distribution $B_{n-2j^{(p)}+1,2j^{(p)}}$, Tukey’s equivalence blocks result²⁷ directly implies that

$$\mathrm{Pr}\left\{U_{[n-j^{(p)}+1]}-U_{[j^{(p)}]}\ge P\right\}=1-B_{n-2j^{(p)}+1,2j^{(p)}}(P)$$ (10)

where $B_{n-2j^{(p)}+1,2j^{(p)}}(·)$ denotes the cdf of the beta distribution with parameters $n-2j^{(p)}+1$ and $2j^{(p)}$. This probability can be easily calculated using the function pbeta in R.

Note that, as $n\to \infty$, the beta distribution $B_{n-2j^{(p)}+1,2j^{(p)}}$ converges to a normal distribution with mean P thus the probability in equation (10) approaches 0.5 as $n\to \infty$.

Figure 5 plots this probability against n for $P=\left\{0.90,0.95,0.99\right\}$. The plots are saw-tooth shaped due to the discreetness of n and $j^{(p)}$. It is clear from the figure that this probability can be substantially smaller than P, and approaches 0.5 as n is large as expected from the asymptotic normal distribution pointed out above. This shows that the nonparametric prediction interval has a substantial probability, close to 0.5 when n is large, of containing less than $\left(100P\right)\%$ of the population values. Hence, this nonparametric prediction interval should not be used as a reference range for the same reason as the prediction interval based on the normal distribution.

Figure 5.

The probability in equation (10) for various sample sizes n.

5 Nonparametric tolerance intervals

A nonparametric tolerance interval is constructed to contain $\left(100P\right)\%$ of the population with a pre-specified (large) confidence γ about the randomness in the sample. Consider the following nonparametric tolerance interval²¹

$$R{R}_8=(X_{[j^{(t)}]},X_{[n-j^{(t)}+1]})$$

where $j^{(t)}$ satisfies that $1\le j^{(t)}\le n/2$ should be the largest natural number such that the proportion of the population contained in RR₈, given by

$$K_8=\underset{Y|X_1,\dots ,X_n}{\mathrm{Pr}}\left\{Y\in (X_{[j^{(t)}]},X_{[n-j^{(t)}+1]})\right\}=U_{[n-j^{(t)}+1]}-U_{[j^{(t)}]}$$

following similar lines as K₇ in equation (8), is at least P with probability γ about the randomness in the sample $X_1,\dots ,X_n$. It follows therefore from equation (10) that $1\le j^{(t)}\le n/2$ should be the largest natural number that satisfies

$$\mathrm{Pr}\left\{U_{[n-j^{(t)}+1]}-U_{[j^{(t)}]}\ge P\right\}=1-B_{n-2j^{(t)}+1,2j^{(t)}}(P)\ge \gamma$$ (11)

For given n, P and γ, $j^{(t)}$ can be easily computed by a direct search over the natural numbers in the range from 1 to $n/2$. Note that if the sample size n is too small, then the existence of $j^{(t)}$ is not guaranteed unless n satisfies¹¹

$$1-\left(n{P}^{n-1}-(n-1)P^n\right)\ge \gamma$$ (12)

The equal-tailed or central nonparametric tolerance intervals can be constructed in a similar way. Our view is that a $(P,\gamma )$ nonparametric tolerance interval is pertinent as a reference range similar to the normal distribution case. Hence we do not go into the details about the equal-tailed nonparametric tolerance intervals to save space.

Figure 6 compares $j^{(t)}$ and $j^{(p)}$ for given sample sizes n with P = 0.95 and $\gamma =\left\{0.90,0.95,0.99\right\}$. It is clear from Figure 6 that $j^{(t)}$ is considerably smaller than $j^{(p)}$, and so RR₈ is wider than RR₇, in order that RR₈ contains $\left(100P\right)\%$ of the population with a pre-specified large confidence γ about the randomness in the sample. Also, as expected, $j^{(t)}$ decreases as γ increases.

Figure 6.

The values of $j^{(p)}$ and $j^{(t)}$ for various sample sizes n given P and γ.

6 Example

A random sample of n = 210 observations on fasting plasma glucose is taken from the population of interest. The data and the R code for all the computations in this paper are available at http://www.personal.soton.ac.uk/wl/RefRange/.

Suppose that the usual normality tests²⁸ show that it is reasonable to assume the population has a normal distribution. The sample mean and standard deviation are computed to be $\overline{X}=5.31$ and S = 0.41 (in unit mmol/L). If we use the prediction interval as the reference range, then it is given by

$$\begin{array}{l}R{R}_1=\overline{X}\pm t_{(1+P)/2,\nu }S\sqrt{1+1/n}=5.31\pm 1.97\times 0.41\times \sqrt{1+1/210}\\ =[4.49,6.12]\end{array}$$

Note, however, as pointed out above, that the probability of the prediction interval containing less than $\left(100P\right)\%$ of the population can be substantial and is computed to be 47%. So there is a 47% probability that the interval does not do what it purports to do: containing $\left(100P\right)\%$ of the population.

If we use the $(P,\gamma )$ tolerance interval as the reference range, with $\gamma =0.95$, then it is given by

$$R{R}_5=\overline{X}\pm c_{5}S=5.31\pm 2.14\times 0.41=[4.43,6.19]$$

This interval is wider than the prediction interval. But, as we pointed out, the tolerance interval does contain $\left(100P\right)\%$ of the population with probability $\gamma =0.95$. Therefore, any future observations falling outside this interval can be regarded as atypical and should be considered for further investigation.

While the tolerance interval above has a confidence $\gamma =95\%$ of containing $\left(100P\right)\%$ of the population, it has a less than $\gamma =95\%$ probability of containing the central $\left(100P\right)\%$ of the population, $\mu \pm z_{(1+P)/2}\sigma$. This probability is computed to be 86%.

In order to have a $\gamma =95\%$ probability of containing the central $\left(100P\right)\%$ of the population, $\mu \pm z_{(1+P)/2}\sigma$, we can use the equal-tailed $(P,\gamma )$ tolerance interval, which is given by

$$R{R}_6=\overline{X}\pm c_{6}S=5.31\pm 2.21\times 0.41=[4.40,6.22]$$

The confidence that this equal-tailed tolerance interval contains $\left(100P\right)\%$ of the population is computed to be 99%, which is much larger than $\gamma =95\%$. Hence, with a 99% probability, the equal-tailed tolerance interval contains $\left(100P\right)\%$ of the population. Furthermore, we estimated that the equal-tailed tolerance interval $\overline{X}\pm 2.21S$ is the $(0.957,\gamma )$ tolerance interval, that is, the interval contains 95.7% of the population with confidence $\gamma =95\%$.

Now suppose that the distribution of the population cannot be assumed to be normal. Then nonparametric reference ranges should be used. If we use the prediction interval as the reference range, then it is given by

$$R{R}_7=[X_{[5]},X_{[n-5+1]}]=[X_{[5]},X_{[206]}]=[4.62,6.09]$$

with $j^{(p)}=5$. Note, however, as we have pointed out, that the probability of the prediction interval containing less than $\left(100P\right)\%$ of the population can be substantial and is computed to be 39%. So there is a 39% probability that the interval does not do what it purports to do: containing $\left(100P\right)\%$ of the population.

If we use the $(P,\gamma )$ nonparametric tolerance interval as the reference range, with $\gamma =0.95$, then it is given by

$$R{R}_8=[X_{[3]},X_{[n-3+1]}]=[X_{[3]},X_{[208]}]=[4.38,6.27]$$

with $j^{(t)}=3$. This tolerance interval is wider than the nonparametric prediction interval but, as we pointed out, it does contain $\left(100P\right)\%$ of the population with 95% confidence. Therefore, any future observations falling outside this interval can be regarded as atypical and should be considered for further investigation.

Finally, we note that nonparametric intervals are usually wider than the corresponding parametric ones since they require fewer assumptions than the parametric model.

7 Conclusions

The objective of a reference range is to contain a pre-specified large content level $\left(100P\right)\%$ of the population with γ confidence level, so that a future observation falling outside the reference range is regarded as atypical and considered for further investigation. This procedure should be useful as part of screening programmes, whose aim is to identify subjects at sufficient risk of a specific disorder who may benefit from further investigation or direct preventive action to avoid death or disability and to improve their quality of life.²⁹

Since a reference range depends on the random sample, the event ‘a reference range contains $\left(100P\right)\%$ of the population’ is also random and so we can never be certain that a reference range contains $\left(100P\right)\%$ of the population. All we can hope for is that the event ‘a reference range contains $\left(100P\right)\%$ of the population’ occurs with a large probability, γ.

Based on this premise, we have argued that the prediction interval is not suitable as a reference range since there is a substantial probability, close to 50% when n is large, that the prediction interval contains less than $\left(100P\right)\%$ of the population. In contrast, a $(P,\gamma )$ tolerance interval is designed to contain $\left(100P\right)\%$ of the population with a pre-specified large confidence γ so it is eminently adequate as a reference range.

Tolerance intervals or regions have been studied by many statisticians since the 1940s. Various parametric and nonparametric procedures are readily available for use as reference ranges or reference regions.^11,16,17,24 Finally, we note that there is some work on constructing reference ranges specifically assuming that the clinical marker follows a log-normal distribution,³⁰ and on sample size calculation for reference ranges,^31,32 and tolerance intervals.^33–35 These aspects, however interesting, fall beyond the scope of our paper.