Extension of overlap measures for Multi-Class Biomarker Evaluation in Alzheimer's Disease

Abstract

In a progressive disease such as the Alzheimer’s disease (AD), the availability of suitable biomarkers tracking the stages of its progression could markedly accelerate drug development by providing an earlier indication of drug efficacy. Investigators use diagnostic tests to classify disease stages into probable Alzheimer’s disease, mild cognitive impairment (MCI), and normal cognitive aging. In this paper, we focus on developing a proper statistical overlap measure-based method to evaluate the diagnostic accuracy of tests with three diagnostic categories. Parametric and non-parametric approaches for the estimation of the overlap measure (OVL) are presented as well as their bootstrap confidence intervals (CIs). The performance of these estimations and its CIs are evaluated through simulations. Furthermore, it is compared with the Volume Under the ROC Surface (VUS), the most common measure to assess the accuracy of tests with three ordinal diagnostic categories. A neuropsychological data set from a longitudinal cohort study for the detection of biomarkers for identifying stages of Alzheimer’s disease is discussed.

Introduction

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder that leads to the death of brain cells that cannot be replaced once lost. While the use of biomarkers in Alzheimer’s disease (AD) research has expanded significantly, current clinical and ethical guidelines–such as those outlined by the National Institute on Aging and Alzheimer’s Association (NIA-AA) and neurological societies–limit the use of biological biomarker-based methods for presymptomatic diagnosis in routine clinical practice. The development of in-vivo biomarkers has moved the diagnosis of Alzheimer’s disease from the dementia stage towards the prodromal stage and has introduced the potential for preclinical diagnosis (i.e., before symptom onset). These developments are relevant for the testing of potential therapies for secondary prevention of Alzheimer’s disease¹. In the context of current diagnostic practices, challenges such as the high cost and limited accessibility of advanced biomarkers (e.g. PET imaging, CSF assays), as well as diagnostic uncertainty in cases with overlapping clinical features (e.g. MCI vs early AD), remain significant.

While there are many proposals for diagnostic testing of binary markers (healthy vs. diseased), there is a lack of research and statistical tools for three-population scenarios that can limit disease prevention and early detection efforts. The transitional stage in the AD process is critical to detect not only because it is indicative of more serious disease development in the future but also because this is the time for family caregivers to discuss care options and family future planning while the patient can still take part in making decisions². In this paper, we will consider mild cognitive impairment (MCI) and/or early stage Alzheimer’s disease as a transition stage between the cognitive changes of normal aging and the more serious problems caused by Alzheimer’s disease (AD).

When it is critical to classify individuals into three groups (i.e. normal, early disease stage, fully developed disease stage), Scurfield³ introduced the ROC surface as a natural generalization of the ROC curve. Mossman⁴ proposed a polyhedral ROC surface and the volume under the surface as a summary measure of the diagnostic accuracy. Dreiseitl et al.⁵ further provided a non-parametric estimate and the associated variance for the volume under a polyhedral ROC surface with probabilistic rating data. The ROC surface for multi-class diagnostic problems, in a non-parametric context, was proposed by Nakas and Yiannoutsos⁶. Reference 7 also discussed the nonparametric estimation of a single VUS. Both parametric and nonparametric inferences on the VUS have been developed^2,8,9). Kang and Tian¹⁰ offered an extensive study comparing possible parametric and non-parametric approaches for the estimation of VUS in terms of bias and root mean square error.

The VUS is an extension of the total area under the ROC curve (AUC). However, the drawbacks of the AUC as a measure of a biomarker’s diagnostic accuracy to distinguish between two disease states are known and many other alternative summary indices have been proposed Franco Pereira et al.¹⁵. Recently, overlapping coefficients (OVL) are widely used in many fields, for example authors in Mizuno et al.¹¹ compared pharmacokinetic parameters of the same drug for two ethnically different regions; the niche overlap of two species, which uses up to n different resources, is addressed by Mueller and Altenberg¹²; the proportion of devices that have a similar range of failure time has been analyzed by Al-Saleh and Samawi¹³. Pardo and Franco-Pereira¹⁴ explored the advantages of the overlap measures over the ROC summary indices to assess the accuracy of a medical diagnostic test in the binary case. The overlap measures have some desirable properties such as invariance to monotone increasing transformations as the ROC summary indices. AUC does not capture shape differences whereas OVL has the advantage of being able to detect differences in both location and shape between the underlying distributions of diseased and healthy samples. Moreover, AUC yields improper values when it is 0.5 or lower, which occurs when the mean score for diseased individuals is less than the mean score for healthy individuals. Therefore, the overlap measures have been proved to be better than the ROC summary indices for assessing diagnostic tests, not only because they perform well when the ROC summary indices fail but also because they are comparable or even superior when the latter are effective²⁶ and Alcaraz-Peñalba et al.¹⁶ addressed the need for robust methods to construct confidence intervals for OVL when dealing with two-class problem by proposing both parametric and non-parametric approaches and recommended different cutoff values for classifying differentiation quality.

Given the advantages of OVL in the two-class case, the aim of this study is to extend and investigate the OVL as a measure of a biomarker’s ability to distinguish among three disease groups. In Sect. "VUS and OVL measures", we provide the motivation for the use of OVL and highlight its advantages in relation to the VUS. Several parametric and non-parametric estimation approaches are proposed for this measure, along with the construction of bootstrap-based confidence intervals (CIs). Additionally, we introduce inferential methods for OVL to compare two biomarkers, as well as its extension to multiclass classification problems. We then apply our methodology to estimate the diagnostic accuracy of AD biomarkers to classify from MCI to AD. A simulation study is conducted to compare OVL estimation procedures in terms of bias and root mean squared error (RMSE), and to assess the performance of the CI estimates with respect to achieving the nominal coverage probability. Furthermore, we compare the power of OVL with that of VUS in identifying potential biomarkers. We conclude with a discussion of our findings and provide recommendations for the appropriate use of OVL.

VUS and OVL measures

To formally define the classification problem with three classes, let’s assume measurements from the first class, denoted by , following a distribution function , i.e. , measurements from the second class, , and measurements from the third class, . Specifically, if we can assume that then we can define a decision rule , which assigns each element to one of the three classes, by establishing two thresholds, , i.e. if X is a new element to be classified, we assign X to

This classification leads to three true positive fractions (TPF) and six false positive fractions (FPF). We define , , and .

If and , the functional form of the ROC surface could be expressed as follows:

1

As a measure of classifier accuracy with 3 classes, VUS is defined as

2

The minimum VUS for a three-class classifier is , which is the volume of the tetrahedron obtained when , and are identically distributed. Its maximum value is one and corresponds to the case in which the three classes are perfectly discriminated in the correct order.

The drawbacks of VUS are analogous to those associated with the AUC in the binary disease classification context. First, VUS relies on the stochastic order assumption which may not hold for many diseases. In fact, when this assumption is violated, the VUS should not be used, as it can be as low as 1/6, even if the underlying biomarker has perfect discriminatory ability. Second, VUS is sensitive to the number of crossings between the class-conditional density functions. For a detailed discussion, see Samawi et al.¹⁷.

An alternative to the VUS to assess the ability of a marker to differentiate three classes are the overlap measures which does not exhibit the above drawbacks. The most commonly used was introduced by Weitzman¹⁸. It is defined as

3

where is the density function associated to , i=1,2,3. Clearly, in general, . OVL measures the similarity between distributions by quantifying the overlapping area of their respective density functions. In contrast to VUS, its intuitive interpretation and the ability to visualize overlap through sample histograms make it particularly attractive to applied researchers. This measure has been also the base for k-sample tests¹⁹.

The overlap coefficient can be rewritten as

which corresponds to the sum of the three error probabilities–namely, the probabilities of failing to correctly classify an observation into its true class. From a probabilistic standpoint, the OVL quantifies the likelihood that a randomly drawn observation from each group (e.g. cognitively normal, MCI, AD) lies in a region where its group is least distinguishable from the others. This measure captures the degree of uncertainty or ambiguity in group separation based on biomarker distributions, without relying on predefined thresholds or classification rules.

The OVL measure has a limitation in extreme scenarios where the biomarker distributions for the first two classes are nearly indistinguishable, while the third class is clearly separated. In such cases, OVL may misleadingly indicate strong overall discriminatory ability, despite the biomarker’s inability to differentiate between the first two classes. This highlights the need for caution when interpreting low OVL values as evidence of a promising biomarker. A necessary follow-up step is to perform pairwise comparisons to assess whether adequate separation exists between each pair of classes. This can be readily achieved using the two-class version of OVL²⁶. In fact, Nakas and Yiannoutsos⁶ recommended examining pairwise comparisons using ROC curves after computing the VUS value, as this approach provides additional insight into the differences among the three groups. Once a diagnostic test has been proven accurate enough for practical use, a cutoff value (or values) for screening subjects must be determined. As discussed in Sect. 3.4 of Nakas³⁴, both VUS and OVL are useful measures for evaluating the discrimination ability of the marker under study but not to select the cut-off points that may be used for clinical decision-making. A review of several methods for optimal cut-point selection can be found in Hassanzad and Hajian-Tilaki²¹.

Estimation methods

Parametric estimation

When biomarker levels follow normal distributions, i.e. , the volume under the trinormal ROC curve after substituting the unknown parameters by the maximum likelihood estimators (MLE) obtained from samples of the three distributions is given by

where , , , , denotes the standard normal distribution function, and its density function.

The corresponding OVL measure can be written as

where , and are the corresponding maximum likelihood estimators of , and , respectively.

The assumption that biomarkers are normally distributed can be very limiting, resulting in inaccurate outcomes if the assumption is significantly violated. A parametric approach based on the Box–Cox transformation to normality often works well in this context^{26, 22}. It consists of assuming that a common, usually unknown, monotone transformation results in the three distributions being normal. See the Supplementary Information for a detailed description.

Kernel estimation

In practice, it is not always possible to achieve normality for some data sets even after applying the Box-Cox transformation. In such cases, some authors propose a kernel-based approach based on the Gaussian kernel density. The distribution functions in (2) and (3) are replaced by appropriate kernel distribution estimators and the integrals are approximated numerically.

One way to estimate the OVL and the VUS is using the Gaussian kernel approach. Therefore, the estimation of the probability density function is given by

and the cumulative distribution function by

for each class k=1,2,3, where represents the number of observations in class k, is the standard Normal distribution function, is its density function, and is the bandwidth in each class.

We consider the asymptotic bandwidth proposed by Silverman²³ and used by Kang and Tian¹⁰ which is given by

4

where and denote the standard deviation and the interquartile range of the kth sample, respectively. We denote these estimators by and . This choice of bandwidth was found to be effective in estimating the overlapping coefficient²⁴.

Following Martinez-Camblor et al.¹⁹, in order to explore the impact of the bandwidth choice in the performance of the OVL kernel estimator, we have also considered bandwidths of the form , where m is the harmonic mean of the ’s, and where S is allowed to take a number of values, explicitly . See in the Supplementary Information Tables S13-S29.

Non-parametric estimation

A non-parametric alternative is to consider the non-parametric estimation of the ROC surface, proposed by Ref. 8, which involves replacing the distribution functions in (1) with the empirical distribution functions associated with the samples:

5

then

Inference based on the OVL

A formal assessment of the diagnostic accuracy of a marker in a three-class classification problem using either its OVL or VUS can be conducted by testing the null hypothesis or . To perform this, we consider the test statistics given in the preceding section and calculate the corresponding p-values using bootstrap techniques. From the biomarker values for the three-classes, we compute the bootstrap distributions of and under resampling with replacement B = 400 times for the combined sample and computing the test statistics under for each resampled dataset. The corresponding p-values are estimated as the proportion of these 400 values that exceed the test statistics obtained from the original sample.

We further propose two bootstrap-based methods for constructing confidence intervals for the OVL. These methods rely on the bootstrap technique to estimate the variance of the OVL estimator. Throughout this section, the term OVL estimator refers interchangeably to either or .

The method referred to as “AN” (or “BC-AN” when the BC transformation is applied) is described in the following steps:

1. Step 1: For the original data we obtain (or if we apply the BC transformation to the data).

2. Step 2: Sample with replacement independently from the original data (apply the BC transformation to the new data) and obtain ().

3. Repeat Step 2 B=500 times, and obtain the bootstrap-based estimate of the variance of by computing

   

   where . (When using the BC transformation analogously we obtain )
4. Step 3: The proposed confidence intervals are then obtained as:

   

   where refers to the percentile of the standard normal distribution.

The bootstrap estimates, can also be used to obtain percentile bootstrap confidence intervals “PB” or “BC-PB”.

When measurements on two diagnostic markers are available, the question of interest is which marker best discriminates among the three disease classes which may be correlated. For testing the hypothesis against the alternative we can use

where is an estimation of A bootstrap-based approach can be used to obtain

The OVL manifold

Evaluating the overall discrimination capacity of a marker for multi-class classification problems is a challenging task. A seminal work on optimal ROC manifolds was promulgated by Scurfield²⁵. Later studied by Nakas and Yiannoutsos⁶ who defined the hypervolume Under the ROC Manifold (HUM) as the probability that the measurements of the diagnostic test under consideration from k subjects, one from each class, will be correctly ordered.

The OVL measure can also be extended to a multi-class setting (k>3) with the following definition

where is the density function associated to It takes values between 0 and 1, where an overlap of one means that the biomarker is useless and a value of zero is the best possible accuracy for the biomarker.

Interestingly, the computational effort required for this measure does not increase significantly with the value of k, but the computational effort of HUM increases as k increases due to requires a higher dimensional integration. Furthermore, all the desirable properties of OVL for two and three classes hold for k>3.

Clinical workflow for the application of the overlap measure

To bridge the gap between methodological development and practical application, a structured clinical workflow for the implementation of the overlap measure OVL is proposed in Fig. 1. This framework is designed to support clinicians in evaluating the discriminatory ability of biomarkers in a practical, interpretable, and threshold-free manner across a wide range of diagnostic settings, without requiring prior alignment or transformation of biomarker direction.

Fig. 1.

Clinical workflow for applying the OVL in biomarker-based diagnosis.

In contrast to alternative measures such as the VUS, which require a preliminary alignment of biomarker direction (e.g. ensuring a consistent ordering across diagnostic groups), the OVL can be directly computed from the observed data without additional preprocessing steps. This property simplifies its application in practice and reduces the risk of misinterpretation due to incorrect data transformations.

The workflow reflects the typical steps followed in clinical practice, from data collection to biomarker evaluation and decision support. When low overlap is observed, additional pairwise comparisons can be performed to identify specific limitations in discriminatory performance. Unlike alternative approaches such as VUS, the OVL does not require prior alignment of biomarker direction, facilitating its direct application in practice.

The process begins with the collection and preprocessing of patient biomarker data, including clinical, biological, or imaging-derived variables. Next, individuals are classified into clinically meaningful diagnostic categories defined according to the problem under study. These groups may correspond to different disease stages, severity levels, or distinct clinical conditions, and provide the basis for evaluating the separation between distributions.

The OVL is then computed to quantify the degree of overlap among the distributions associated with each diagnostic group. This can be achieved using either parametric approaches, assuming specific distributional forms (optionally supported by transformations such as Box-Cox), or non-parametric methods based on kernel density estimation. The resulting OVL value represents the extent to which the groups are not distinguishable. To support biomarker selection, OVL values can be compared across multiple candidate biomarkers. Those exhibiting lower overlap are preferred, as they provide greater diagnostic discrimination. Interpretation of the OVL is straightforward: lower values indicate reduced overlap between groups and therefore better discriminatory performance of the biomarker, whereas higher values reflect increased ambiguity in classification. The interpretation of OVL values in the three-class setting follows guidelines analogous to the two-class case, as given in Franco-Pereira et al.²⁶ For practical interpretation, we propose the following thresholds: OVL values between 0.8 and 1.0 indicatepoor discriminatory ability, values between 0.5 and 0.8 indicate moderate discrimination, values between 0.3 and 0.5indicate good discrimination, values between 0.1 and 0.3 indicate very good discrimination, and values below 0.1 indicateexcellent discrimination.. These thresholds provide a practical and intuitive framework for assessing the overall discriminatory ability of a biomarker. In practice, when the OVL is low, additional pairwise analyses are recommended to avoid situations in which only a category exhibits good separation. This complementary approach provides further insight into the discriminatory performance of the biomarker and supports more informed clinical interpretation. In addition, OVL can be visually supported through graphical representations such as histograms or density plots, enhancing interpretability.

Therefore, the results derived from OVL analysis can be incorporated as a complementary tool in clinical decision-making. While not intended to replace established approaches such as the volume under the ROC surface (VUS), OVL provides a robust and easily interpretable alternative, particularly in scenarios where the underlying assumptions required by VUS are not satisfied. Its low computational cost and intuitive probabilistic interpretation make it especially suitable for integration into clinical workflows and decision-support systems across diverse medical domains.

To illustrate the practical implementation of this framework, we consider real data collected by the Alzheimer’s Disease Research Center at the University of Washington²⁷. Several neuropsychological biomarkers with different discriminatory profiles are analyzed. The results show that, in scenarios where the assumptions underlying the VUS are reasonably satisfied, both VUS and OVL lead to consistent conclusions. However, in more challenging situations, where the stochastic ordering of the distributions is not clearly fulfilled, OVL remains directly applicable and provides a more reliable and transparent assessment of group separability.

Application to a real data

Alzheimer’s is the most common degenerative dementia, affecting approximately 47% of the population aged 85 or older. Neuropsychometric tests have been used to detect Mild Cognitive Impairment (MCI) compared to normal aging and Alzheimer’s. The database we have considered includes individuals from a longitudinal cohort of the Alzheimer’s Disease Research Center at the University of Washington (ADRC)²⁷ and is available in the GitHub repository²⁸. The severity of each individual’s disease was classified according to the Clinical Dementia Rating (CDR). The dataset contains 14 neuropsychometric markers. These markers were measured in 118 individuals aged 75, divided into three diagnostic categories: the non-demented or healthy group (, N = 45) with CDR = 0; the MCI group (, N = 44) with CDR = 0.5; and the Alzheimer’s group (, N = 29) with CDR = 1.

Temporal function (ktemp), visual retention (zbentd), and language performance as assessed by the Boston Naming Test (zboston) capture complementary cognitive domains that are differentially affected in the early stages of Alzheimer’s disease. The temporal factor (ktemp) primarily reflects functions subserved by the medial temporal lobe, including the hippocampus, a region critically involved in episodic memory and one of the earliest sites of Alzheimer-related neurodegeneration driven by tau pathology. Visual retention (zbentd), derived from the Benton Visual Retention Test, assesses visuospatial memory and perceptual organization, which are associated with temporo-parietal networks and may reveal subtle deficits not captured by verbal memory tasks. The Boston Naming Test (zboston) evaluates lexical retrieval and semantic memory, functions linked to temporal neocortical regions that become increasingly affected as the disease progresses. Clinically, the combination of these markers provides a multidimensional characterization of cognitive decline, capturing early hippocampal dysfunction (ktemp), non-verbal memory impairment (zbentd), and emerging language deficits (zboston). Biologically, their alterations are consistent with the spatial progression of Alzheimer’s pathology, from medial temporal structures to broader cortical areas, supporting their joint relevance for improving early detection and discrimination between normal aging, Mild Cognitive Impairment, and Alzheimer’s disease.

The variable ktemp reflects memory-related cognitive processes, which are known to be critically affected in neurodegenerative conditions such as Alzheimer’s disease. In fact, previous analyses such as Luo and Xiong²⁹ have identified ktemp as a highly effective biomarker for discriminating among cognitively normal individuals, patients with MCI, and patients with Alzheimer’s disease, providing a VUS value of 0.75. In that work, the assumption of normality was also assessed and found to be reasonable for this biomarker. Consistent with this evidence, the distributions of ktemp tend to follow a clear ordering across diagnostic groups (see Fig. S1 in the Supplementary Information), leading to similar conclusions when evaluated using both VUS and OVL. Accordingly, for this biomarker we report parametric estimators under the normality assumption.

The estimators of the VUS and OVL introduced in Sect. "VUS and OVL measures" are shown in Table 1 and 2. We have also included a 95-percentile bootstrap confidence interval (95-PB-CI).

Table 1.

Estimations of OVL for the biomarker ktemp.

Point estimator	0.0679	0.0936
95-PB-CI	(0.0078, 0.1726)	(0.0084, 0.2016)

Table 2.

Estimations of VUS for the biomarker ktemp.

Point estimator	0.7518	0.7207	0.7711
95-PB-CI	(0.6333, 0.8682)	(0.6112, 0.8317)	(0.6585, 0.8745)

Figure 2 shows the kernel-based and the parametric estimates of the density functions for each of the three groups, as well as the overlap area OVL. Figure 3 represents the corresponding ROC surface estimates. The graphical interpretation of OVL is clearly easier than that of VUS. In the Supplementary Information, we have included the pairwise comparison per groups (Table S1) to discard any possible limitation in discriminatory performance.

Fig. 2.

Kernel (left) and parametric (right) estimates of the OVL associated with ktemp.

Fig. 3.

Kernel (left) and parametric (right) estimates of the ROC surface associated with ktemp.

The global OVL value indicates a very high level of separation among the three classes. However, the pairwise OVL values reveal a more nuanced structure: while the overlap between and is very low, indicating excellent discrimination, moderate levels of overlap are observed for the pairs vs and vs . Importantly, these values also indicate that no pair of classes is completely overlapping, which supports the overall discriminatory capacity of the biomarker.

In contrast, zboston and zbentd correspond to standardized scores associated with the Boston Naming Test and Benton-type visuospatial memory tasks, respectively. These cognitive measures have been extensively used in Alzheimer’s disease research. Impairment in naming ability, as measured by the Boston Naming Test, has been consistently reported as a characteristic feature of Alzheimer’s disease, even at early stages (Kaplan et al.³⁰ and Henry et al.³¹). Similarly, deficits in visuospatial memory assessed through Benton-type tests have been associated with cognitive decline in older adults and patients with neurodegenerative conditions (see Ref. 32).

In Figs. S4 and S5 of the Supplementary Information, we present the histograms and the p-values from the Shapiro–Wilk normality test for zboston and zbentd, respectively. For zboston, the analysis suggests that the hypothesis of normality cannot be rejected for the diseased group, , whereas it is rejected for both the Mild Cognitive Impairment group, , and the healthy group, . For zbentd, the hypothesis of normality is rejected for all three diagnostic groups. After applying a Box-Cox transformation, the assumption of normality is still not satisfactorily met across the groups. Therefore, kernel-based estimators are adopted as the primary approach for inference, as they do not rely on distributional assumptions and provide a more robust assessment in this setting. We have considered the bandwidth of Martinez-Camblor et al.¹⁹ (k=1,2,3).

From a methodological perspective, the cumulative distribution functions (CDFs) of zboston and zbentd intersect, thereby violating the stochastic ordering assumption required by the VUS (see Figs. S2 and S3 in the Supplementary Information). In such cases, the VUS can not be calculated, and alternative measures such as the OVL become particularly suitable.

The OVL estimators and their 95%-PB confidence intervals are reported in Table 3 for zboston and zbentd. While the VUS is not reliable in this setting due to the lack of a clear stochastic ordering among the groups, the OVL provides a direct and interpretable assessment of discrimination, indicating good discrimination for zbentd and very good discrimination for zboston. Pairwise comparisons per group are also reported in Tables S2 and S3 in the Supplementary Information, and their results are consistent with the overall three-group analysis.

Table 3.

Estimations of OVL based on kernel, , for the biomarkers zboston and zbentd.

	zboston	zbentd
Point estimator	0.2063	0.4009
95-PB-CI	(0.0917, 0.3088)	(0.2225, 0.5609)

Figure 4 displays the kernel-based estimates of the density functions for each of the three groups, together with the corresponding overlap area OVL.

Fig. 4.

Kernel estimates of the OVL associated with zboston (left) and zbentd (right).

These findings highlight that, while OVL is fully compatible with traditional approaches in well-behaved settings, it can also be applied in a broader range of practical scenarios without requiring restrictive assumptions.

Simulation study

The aim of this Section is three-fold. Firstly, we evaluate the bias and the Root Mean Squared Error (RMSE) of the estimators proposed on Sect. "VUS and OVL measures". Secondly, the coverages of 95% percentile confidence intervals are examined. Finally, the size and the power of the test statistics related to OVL and VUS are compared. R software was employed throughout (https://www.r-project.org/, R Core Team, Vienna, Austria). R code for the estimation of OVL and the construction of CIs is publicly available at https://github.com/vicsiemar/OVL-VUS-Simulation-Study.git

Assessment of the accuracy and bias of the estimators of the OVL

A Monte Carlo simulation study of size 1000 was performed in order to evaluate the finite sample properties of the two estimators of OVL, namely and in terms of bias and RMSE. We consider several distributional scenarios. These are specified in Table S4 in the Supplementary Information. Sample sizes considered were , 50, and 100.

Results are shown in Table 4. In general, the estimator tends to provide better performance in terms of precision (RMSE). This trend holds across most scenarios, with the exception of scenarios 5 and 6, where performs comparably or slightly better. Regarding bias, both estimators exhibit a consistent tendency toward minimizing bias as the sample size increases, with bias values approaching zero for larger sample sizes (e.g. at n=100). In general, both methods provide robust results.

Table 4.

Bias, and RMSE for the scenarios of Table S4.

Scenarios	Sample size	Bias			RMSE
		20	50	100	20	50	100
1.		-0.029	-0.005	-0.002	0.107	0.072	0.053
		-0.031	0.008	0.015	0.099	0.068	0.055
2.		-0.076	-0.039	-0.027	0.129	0.076	0.054
		-0.065	-0.025	-0.015	0.116	0.070	0.051
3.		-0.026	-0.016	-0.008	0.111	0.074	0.049
		-0.003	0.016	0.022	0.100	0.074	0.054
4.		-0.037	-0.021	-0.014	0.083	0.051	0.037
		0.018	0.029	0.028	0.088	0.062	0.050
5.		0.007	0.022	0.028	0.102	0.070	0.055
		0.039	0.072	0.078	0.094	0.091	0.088
6.		-0.043	-0.010	-0.000	0.110	0.067	0.048
		-0.072	-0.028	-0.017	0.118	0.070	0.053
7.		0.009	0.037	0.049	0.093	0.072	0.066
		-0.033	-0.001	0.008	0.095	0.060	0.044

The values closest to zero are highlighted in bold

Confidence intervals for the OVL

In this section, we assess the performance of the four confidence interval procedures introduced in Sect. "Inference based on the OVL", namely “AN”, “BC-AN”, “PB” and “BC-PB” in terms of coverage.

Table 5 reports the 95% empirical coverage probabilities obtained under the different methods and sample sizes, corresponding to the distributional scenarios defined in Table S4. These coverage probabilities are used to assess how well different estimation methods capture the true parameter with the intended confidence level (95%) under various distributional assumptions.

Table 5.

95%-coverages for the scenarios of Table S4 using the methods described in Sect. "Inference based on the OVL".

Scenarios	Sample size	AN			BC-AN			PB			BC-PB
		20	50	100	20	50	100	20	50	100	20	50	100
1.		0.905	0.934	0.942	0.903	0.957	0.951	0.871	0.930	0.939	0.862	0.930	0.935
		0.921	0.919	0.944	0.920	0.919	0.943	0.842	0.944	0.955	0.846	0.942	0.954
2.		0.490	0.463	0.400	0.665	0.562	0.361	0.495	0.474	0.381	0.697	0.735	0.736
		0.352	0.256	0.166	0.884	0.917	0.938	0.187	0.153	0.064	0.743	0.852	0.881
3.		0.892	0.921	0.934	0.901	0.965	0.963	0.886	0.920	0.950	0.881	0.912	0.937
		0.931	0.933	0.934	0.934	0.935	0.923	0.883	0.942	0.960	0.919	0.963	0.950
4.		0.856	0.888	0.897	0.941	0.948	0.951	0.834	0.888	0.896	0.954	0.979	0.930
		0.928	0.916	0.856	0.939	0.929	0.916	0.944	0.951	0.901	0.940	0.949	0.930
5.		0.924	0.905	0.918	0.764	0.791	0.759	0.934	0.938	0.929	0.905	0.959	0.975
		0.941	0.721	0.491	0.943	0.743	0.539	0.967	0.926	0.733	0.967	0.922	0.748
6.		0.948	0.723	0.417	0.966	0.949	0.954	0.955	0.943	0.610	0.852	0.928	0.954
		0.919	0.942	0.946	0.851	0.909	0.934	0.729	0.911	0.955	0.677	0.834	0.890
7.		0.910	0.662	0.348	0.857	0.868	0.950	0.961	0.802	0.443	0.857	0.868	0.950
		0.930	0.942	0.931	0.911	0.949	0.942	0.858	0.962	0.958	0.911	0.949	0.942

Values closest to the nominal values are shown in bold.

Each row in Table 5 corresponds to a scenario (1 through 7) described in Table S4, which involves combinations of normal, log-normal, gamma, and mixture distributions. All proposed methods of constructing confidence intervals for OVL are considered across three sample sizes (20, 50, 100). The best-performing method for each sample size is highlighted in bold, indicating the confidence interval procedure that most closely achieves the nominal 95% coverage under each scenario.

Among the interval construction methods, the percentile bootstrap after the Box-Cox transformation (BC-PB) consistently yields the most accurate and stable coverage probabilities across all scenarios and estimators. The AN method, although effective under the standard normal scenario (scenario 1), often underperforms in the presence of skewness or heavy tails, particularly with the estimator. The inclusion of the Box-Cox transformation (BC-AN) improves coverage moderately, but still fails to consistently reach the nominal level in complex scenarios. The percentile bootstrap (PB) performs better than AN in non-normal settings but is further improved by the Box-Cox transformation.

Comparison of the size and power of the OVL and VUS estimators

In this section, we conduct a Monte Carlo simulation study of size 1,000 using different distributions to compare the performance in terms of power of and as methods for evaluating the discriminatory capacity of biomarkers when working with 3 populations. In each simulation, 500 Bootstrap resamples are generated and the significance level is α =0.05. Sometimes, before the kernel is applied, it is recommended to apply a Box-Cox transformation to the data because the data become more symmetrical³³. Consequently, we have examined for some scenarios the use of the BoxCox power transformation before applying the kernel method and denote the resulting estimates as and .

The null hypothesis for the tests based on the two estimators as test statistics is the same, that the biomarker is not informative (that is, OVL=1 and VUS= 1/6, respectively). We consider a total of 17 scenarios based on the different distributions , , and . In the scenarios in which , , and are the same, the rejection rate corresponds to the Type I error, and in the rest, to the power of and . Within each scenario, 8 different sample sizes have been considered: and and (50, 100, 100). To streamline the presentation, the simulation scenarios A combination of different distributions and Mixture distributions are included in the Supplementary Information.

Normal distributions

We begin the simulation study assuming that the three classes follow normal distributions. First, we assume , , and ; we are actually under the null hypothesis . The theoretical OVL reaches the maximum value, unity. The theoretical value of the VUS, on the other hand, is 1/6, which is the minimum value it can attain and corresponds to the volume of the tetrahedron.

In Table 6, the rejection proportions for the case under study are presented. Since the three classes are identically distributed, so we expect the calculated values to be around α =0.05. Generally, except a few cases that are conservative, the rejection proportions of both statistics are close to the nominal.

Table 6.

Proportions of rejection for , and .

	0.058	0.055	0.045	0.055	0.044	0.043	0.053
	0.056	0.046	0.042	0.052	0.055	0.043	0.052
	0.037	0.049	0.028	0.053	0.049	0.028	0.052
	0.049	0.043	0.045	0.047	0.049	0.047	0.048
	0.052	0.036	0.045	0.037	0.036	0.045	0.036
	0.045	0.057	0.032	0.053	0.057	0.030	0.053
	0.058	0.054	0.049	0.052	0.053	0.050	0.051
	0.049	0.046	0.037	0.050	0.050	0.038	0.050

In the second scenario, , , and . The theoretical OVL is 0.6171, while the theoretical VUS is 0.3372. Table 7 shows that the test power is higher for than for regardless of the estimation method used, with the difference being most evident when the sample size is smaller and when kernel estimation is used: rejects almost 90% of the time for the smallest sample sizes considered (), while does so in less than 50% of these cases. As expected, parametric estimations are the best. However, the results for improve considerably for larger sample sizes, with both measures achieving maximum power when all three classes reach a sample size of 100.

Table 7.

Proportions of rejection for , and .

	0.724	0.890	0.472	0.874	0.861	0.462	0.872
	0.779	0.922	0.504	0.901	0.895	0.501	0.896
	0.872	0.973	0.644	0.960	0.953	0.648	0.956
	0.965	0.991	0.840	0.988	0.986	0.837	0.987
	0.995	0.999	0.937	0.997	0.996	0.936	0.996
	0.997	1.000	0.978	1.000	1.000	0.976	1.000
	0.998	1.000	0.987	1.000	0.999	0.988	1.000
	1.000	1.000	1.000	1.000	1.000	1.000	1.000

The Higher power values are in bold.

In the third scenario, we assume that the normal distributions differ not in mean but in standard deviation: , , and . The theoretical values of OVL and VUS in this new scenario are 0.6773 and 0.1668, respectively. Table 8 shows results that are completely different from those in Table 7. achieves a much higher power than , and its rejection rate improves significantly as the sample size increases. , however, faces serious difficulties in rejecting the null hypothesis, a problem that does not improve with increasing sample size. For normal distributions, transformation does not lead to improvement in the kernel methods.

Table 8.

Proportions of rejection for , and .

	0.469	0.058	0.315	0.053	0.057	0.318	0.052
	0.547	0.040	0.375	0.051	0.050	0.383	0.045
	0.727	0.040	0.512	0.039	0.045	0.518	0.038
	0.902	0.031	0.727	0.035	0.036	0.737	0.033
	0.959	0.034	0.844	0.034	0.039	0.845	0.032
	0.994	0.042	0.939	0.048	0.047	0.939	0.046
	0.996	0.047	0.969	0.045	0.046	0.971	0.046
	0.999	0.041	0.996	0.045	0.048	0.997	0.044

The Higher power values are in bold.

Log-Normal distributions

The Log-Normal distribution is positively skewed and is characterized by the fact that its natural logarithm follows a Normal distribution. Mathematically, if , then . Within this subsection, we will consider three additional scenarios to evaluate the discriminatory power of the two metrics we are comparing, following the structure of Sect. "Normal distributions": a first scenario under and two under the alternative, modifying the means and standard deviations of the Log-Normal distribution.

Let’s consider that the three classes are identically distributed as , that is, , , and . That is, OVL=1 and VUS=1/6. Table 9 shows that both and achieve values close to the significance level α for all sample sizes and estimation methods analyzed. is sometimes conservative.

Table 9.

Proportions of rejection for , and .

	0.070	0.054	0.032	0.058	0.044	0.048	0.057
	0.053	0.045	0.039	0.050	0.055	0.046	0.051
	0.040	0.049	0.028	0.052	0.049	0.029	0.053
	0.052	0.040	0.045	0.043	0.049	0.047	0.046
	0.047	0.035	0.035	0.043	0.036	0.045	0.036
	0.044	0.054	0.036	0.054	0.057	0.032	0.053
	0.061	0.053	0.047	0.051	0.053	0.049	0.051
	0.052	0.044	0.045	0.054	0.050	0.040	0.050

In the following scenario, we slightly modify the previous distributions, altering only the mean of the distribution of the third class: , and . For this scenario, OVL=0.6171 and VUS=0.3169.

The results in Table 10 show that is the best and only overperformed by , with a slight better performance for the small and equal sample sizes.

Table 10.

Proportions of rejection for , and .

	0.837	0.836	0.612	0.846	0.795	0.607	0.819
	0.901	0.856	0.650	0.886	0.811	0.687	0.837
	0.975	0.922	0.766	0.956	0.893	0.829	0.905
	0.994	0.964	0.944	0.985	0.958	0.950	0.961
	1.000	0.995	0.981	0.996	0.987	0.979	0.990
	1.000	0.998	0.996	1.000	0.994	0.997	0.997
	1.000	0.997	0.999	0.999	0.997	1.000	0.998
	1.000	1.000	1.000	1.000	1.000	1.000	1.000

The bold values are intended to indicate higher power.

Likewise, it is worth highlighting that, despite only one of the distributions being different, which makes the discrimination by statistical tests more challenging, both and adequately detect the difference between distributions and reject, in the worst case, in more than 60 out of 100 simulations performed.

We propose, in the next scenario, a situation in which the three distributions have the same mean but different standard deviations: , and . The theoretical overlap measure gives us OVL=0.5157, while the theoretical volume under the ROC surface is VUS=0.1674.

As it was the case with Table 8 for the Normal distribution when the distributions differed in standard deviation, Table 11 for this scenario shows that has a much higher discriminative power than for all sample sizes considered in this work. Moreover, the power associated with rises as the sample size increases, reaching the maximum possible power (unity) for the moderate and large sample sizes analyzed (), whereas does not exceed 0.276 in the best case. Transforming the data before applying the kernel improves the power but still do worse than .

Table 11.

Proportions of rejection for , and .

	0.938	0.057	0.401	0.188	0.069	0.825	0.062
	0.978	0.043	0.334	0.198	0.051	0.868	0.048
	0.992	0.037	0.332	0.200	0.043	0.952	0.039
	1.000	0.035	0.728	0.225	0.036	0.999	0.031
	1.000	0.035	0.928	0.226	0.053	0.999	0.042
	1.000	0.042	0.932	0.275	0.050	1.000	0.044
	1.000	0.040	0.979	0.275	0.046	1.000	0.047
	1.000	0.048	1.000	0.276	0.054	1.000	0.049

The Higher power values are in bold.

Gamma distributions

The Gamma distribution, like the Log-Normal distribution, is asymmetric and only takes positive values. It has two parameters: the shape, α > 0, and the scale, β > 0. For the analysis of their rejection proportions, we consider three scenarios. First, we will analyze the behavior of and when all three classes follow the same distribution: . The theoretical values for the OVL and VUS are 1 and 1/6, respectively.

Table 12 presents the rejection proportions for this scenario. Since we are under the null hypothesis , as expected, the observed values are more or less close to the significance level α = 0.05.

Table 12.

Proportions of rejection for , and .

	0.055	0.059	0.033	0.055	0.046	0.033	0.053
	0.063	0.069	0.039	0.063	0.064	0.045	0.064
	0.054	0.047	0.045	0.048	0.048	0.029	0.046
	0.064	0.049	0.041	0.047	0.050	0.037	0.052
	0.045	0.047	0.027	0.048	0.046	0.031	0.045
	0.055	0.054	0.042	0.055	0.057	0.043	0.057
	0.064	0.049	0.055	0.052	0.051	0.053	0.053
	0.058	0.071	0.048	0.063	0.069	0.039	0.070

In the next scenario, the classes come from distributions with the same scale parameter but different shape parameters: , , and . In this scenario, the values for OVL and VUS are OVL = 0.5295 and VUS = 0.3888. As shown in Table 13, achieves slightly higher powers than for all sample sizes, with the differences being greater when the kernel method is used.

Table 13.

Proportions of rejection for , and .

	0.910	0.980	0.672	0.959	0.967	0.708	0.973
	0.946	0.990	0.749	0.978	0.978	0.775	0.980
	0.978	0.996	0.871	0.993	0.992	0.896	0.994
	0.999	1.000	0.963	1.000	1.000	0.972	1.000
	1.000	1.000	0.996	1.000	1.000	0.997	1.000
	1.000	1.000	1.000	1.000	1.000	1.000	1.000
	1.000	1.000	0.999	1.000	1.000	1.000	1.000
	1.000	1.000	1.000	1.000	1.000	1.000	1.000

The Higher power values are in bold.

In the last scenario, both the shape and scale parameters differ between populations: , and . The theoretical values are 0.6138 for OVL and 0.3056 for VUS. The rejection proportions are presented in Table 14, and we observe that outperforms in terms of power when the kernel estimation is chosen, but the opposite occurs with the parametric method. For gamma distributions, applying the transformation significantly improves the performance of the kernel methods, bringing them close to the highest values.

Table 14.

Proportions of rejection for , and .

	0.741	0.646	0.244	0.457	0.550	0.652	0.763
	0.821	0.741	0.388	0.512	0.635	0.747	0.815
	0.896	0.786	0.527	0.597	0.685	0.869	0.858
	0.980	0.873	0.590	0.732	0.809	0.974	0.933
	0.995	0.948	0.643	0.816	0.885	0.991	0.976
	1.000	0.964	0.839	0.905	0.922	0.998	0.985
	1.000	0.982	0.901	0.941	0.967	1.000	0.990
	1.000	1.000	0.944	0.986	0.996	1.000	1.000

The Higher power values are in bold.

For completeness, a comprehensive analysis of the actual impact of the bandwidth in the given problem is provided in the Supplementary Information. Although the impact of bandwidth variation is noticeable, it does not play a critical role in the comparison between OVL and VUS. That is, the relative performance between these measures remains largely consistent across different bandwidth choices. In general, OVL generally outperforms VUS regardless of the selected bandwidth.

Concluding remarks

This work introduces the Overlap Measure (OVL) as a novel approach to evaluating the discriminatory power of biomarkers in neurodegenerative disease diagnosis. By quantifying the probability that each group’s observations fall in regions where their distribution is least distinct from others, the OVL offers a probabilistically grounded and threshold-free measure of classification ambiguity. While not intended to replace current diag- nostic tools like VUS (Volume Under the ROC Surface), the OVL can serve as a practical adjunct rather than a replacement. Importantly, OVL can be applied in a broader range of scenarios, including cases in which the assumptions required for VUS are not satisfied, allowing the evaluation of biomarkers that would otherwise be difficult to assess. We acknowledge the importance of aligning with current clinical and ethical standards for AD diagnosis. We recognize that clinical integration of OVL would require development of user-friendly software, clinician training, and prospective validation. Nevertheless, the OVL’s low computational cost and interpretability make it a promising candidate for integration into decision-support systems in neurology.

The volume under the ROC surface (VUS) can be regarded as a natural extension of the area under the ROC curve (AUC) to three-class classification problems and has been widely used for evaluating diagnostic markers in this setting. However, our results highlight several important limitations when applying VUS in practice. First, under ideal conditions -namely, equal variances across groups and clear stochastic ordering- VUS provides a reasonable global summary of discrimination. Second, when these assumptions are violated, as is often the case in real clinical data (e.g. heterogeneous variability between normal aging, Mild Cognitive Impairment, and Alzheimer’s disease), the performance of VUS deteriorates substantially. In particular, our simulation results (Tables 14, 17, and S8) show that its statistical power can drop to values close to the nominal significance level (≈ 0.05), indicating a failure to detect truly informative biomarkers. Third, this limitation has direct practical implications: reliance on VUS may lead to the incorrect dismissal of clinically relevant markers, especially those exhibiting overlapping but still meaningful distributions across groups.

In contrast, the proposed OVL, inspired by earlier two-class formulations^14,17, remains well-defined and robust under heteroscedasticity and non-ordered distributions. From a methodological perspective, OVL provides a direct quantification of the shared probability mass among the three groups, offering a more stable and interpretable measure of discrimination in complex scenarios. From a practical standpoint, this translates into improved sensitivity for identifying biomarkers that retain clinical value despite partial overlap between diagnostic categories. Across most simulated scenarios, OVL demonstrates higher statistical power than VUS, and even in cases where VUS performs slightly better, the differences are marginal and do not outweigh its instability in more realistic settings. Importantly, the use of OVL reduces the risk of overlooking potentially useful, less invasive, or cost-effective biomarkers that could support early detection and clinical decision-making. While neither VUS nor OVL directly provides optimal classification thresholds, the improved reliability of OVL makes it a more suitable preliminary tool within diagnostic evaluation pipelines, complementing existing threshold-selection methods (e.g. Hassanzad and Hajian-Tilaki²¹) and enhancing the overall assessment of candidate biomarkers in real-world applications.

Parametric and non-parametric estimations of OVL have been proposed. Bootstrap confidence intervals were also proposed and evaluated in terms of attaining the nominal coverage probability based on a simulation study.

Future research involves the development of more accurate methods for confidence intervals construction for OVL in a three-class disease setting, to account for possible effects of covariates on the OVL and to study deeply the OVL measure for comparing 2 or more biomarkers as well as the extension to the k-classes problem.

Supplementary Information

Supplementary Information. ^{(388KB, pdf)}

Author contributions

All authors contributed substantially and in equal measure to the development of this manuscript. All authors were involved in the conception and/or design of the study, as well as in the acquisition, analysis, or interpretation of data, and in drafting or critically revising the manuscript. All authors approved the final version and agree to be accountable for the accuracy and integrity of the work.

Funding

This work was partially supported by grant PID2022-137050NB-I00, funded by MCIN/AEI/10.13039/501100011033/ and by ERDF A way of making Europe.

Data availability

We did not generate any new datasets during this study. However, we did analyse publicly available data, asindicated in the corresponding section of the manuscript. Therefore, the current statement is not accurate and should berevised accordingly to avoid any imprecision.

Code availability

All simulation scripts and analysis code used to generate the results presented in this study are available at: https://github.com/vicsiemar/OVL-VUS-Simulation-Study.git. The repository contains the code required to reproduce the simulation experiments and figures reported in this article.

Declarations

Competing interests

The authors declare no competing interests.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1038/s41598-026-55865-3.