Reducing overconfident errors in clinical prediction models

Abstract

Machine Learning (ML) models are increasingly being used in clinical workflows. Evaluation of these models tends to focus on global performance metrics, which can obscure error patterns and lead to bias in clinical decision making. Here, we propose Proximal Error-Based Confidence Adjustment (PECA), a framework designed explicitly to improve the safety of ML predictions by reducing a model’s confidence in regions of the feature space associated with historical prediction errors. Rather than optimizing global metrics alone, PECA targets confident misclassifications by attenuating prediction confidence proportional to similarity with previously observed errors. Across extensive simulations, PECA reduced the rate at which a model made confident misclassifications while preserving the overall predictive ability of the model. We applied this framework to a clinical trial enrollment workflow for Alzheimer’s disease and demonstrated consistent results with the simulations while also demonstrating superior statistical power compared to baseline models. The results of this paper suggest that taming a model’s tendency to predict overconfidently using historical error patterns may be a critical step towards safer and more reliable ML systems for digital public health.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12911-026-03681-0.

Introduction

Machine Learning (ML) has become ubiquitous in the modern age of biomedical research and clinical decision-making workflows. It is not uncommon to observe ML models outperforming classical statistical models with regard to average performance metrics such as accuracy, area under the receiver operating curve (AUC), and Brier Score [1]. As these systems transition away from predicting in isolated research settings and towards operational real-world deployment, understanding what “model safety and reliability” truly means becomes paramount. In clinical workflows, predictive models directly influence decisions that affect patient safety [2]. Metrics that summarize average performance, like accuracy or AUC, can obscure how models predict on an individual level. Just because a model performs well on average does not guarantee that it performs well individually for each patient [3]. Therefore, ensuring safe and reliable deployment requires developing models subject to evaluation metrics that capture clinically relevant aspects of error and risk.

The confidence with which a model makes its predictions is a key component in evaluating its safety and reliability. Most predictive classifiers produce probabilities that indicate the likelihood of the patient’s event or outcome [4]. When the probability associated with the model’s chosen class is high (e.g., > 80%), the model is expressing strong confidence in its prediction. In clinical workflows, models don’t directly make final decisions for a patient’s diagnosis but are often used as second opinions that can be interpreted alongside clinical expertise [5]. High probabilities are often interpreted as strong evidence for a particular outcome which can, in turn, shape downstream decision making. When this confidence is wrongly assigned, the consequences extend past simple misclassification. A high-confidence error may counter clinical intuition and systematically bias clinical decisions. Many modern ML methods may exhibit acceptable average accuracy while still placing high confidence in erroneous predictions [6, 7]. This is a dangerous latent property of ML classifiers particularly in high-risk clinical settings.

The tendency for a model to produce overconfident predictions is often inherent in the design of contemporary ML algorithms. Many classifiers are trained to minimize loss functions that reward maximal separation of two or more classes. As an example, gradient boosting methods like XGBoost [8] force the model to “learn from its past mistakes” in subsequent training rounds by oversampling observations where it was previously wrong. Once fully trained, it is not uncommon for these models to assign high confidence to their predictions since they were forced to implicitly correct their errors during each round of training. Furthermore, in high-dimensional settings, over-parametrized models can “memorize” training data and therefore produce sharp decision boundaries with most probability density being near 0 or 1 [9, 10]. Although these characteristics frequently improve measures such as AUC, they can also generate overconfidence in regions where training data are sparse or where distributional shift occurs [9, 11]. Post hoc calibration techniques aim to correct discrepancies between predicted probabilities with the true risk of the event through a global or local monotonic transformation [6, 12–15]. A specific example is LoRe (Localized Recalibration) which seeks to improve model calibration by estimating calibration error within local neighborhoods of similar examples and applying a corresponding local adjustment to predicted probabilities [15]. While these methods often reduce global/local miscalibration, they do not explicitly address a model’s tendency to assign overconfident probabilities. Consequently, calibration may reduce average calibration error without reducing the rate at which a model makes high-confidence errors. In the context of clinical prediction, calibration alone may be insufficient for producing trustworthy, actionable predictions.

In this paper, we propose a Proximal Error-Based Confidence Adjustment (PECA), a framework that directly aims to reduce high-confidence errors made by clinical prediction models. PECA uses an adaptive kernel to determine how similar new observations are to historical error regions defined during training and adjusts a model’s predicted probabilities accordingly. PECA builds on the localized perspective introduced by LoRe but shifts the focus from learning local calibration functions to explicitly identifying and responding to regions of past model failure. Rather than estimating calibration error across all neighborhoods, PECA isolates the error space and uses it to directly attenuate overconfidence. This framework makes predictions more uncertain overall, but more certain and accurate in the most important high-confidence regions. We empirically validate PECA through a simulation study that compares its performance subject to common model generalization challenges with existing methods and approaches. We then demonstrate an application of PECA by developing a clinical trial enrollment model for Alzheimer’s disease (AD). In this paper, we seek to bridge the gap between traditional predictive modeling and robust reliability methods, addressing a fundamental barrier to deploying ML models in clinical applications.

Methods

Proximal Error based Confidence Adjustment (PECA)

Let (X_i,Y_i) denote feature–label pairs with and Y_i ∈ 0,1. Suppose a trained classifier produces estimated probabilities

where denotes the learned model parameters. We define the error space as the space generated by

for a user-defined threshold . The error space is the space defined by the features for training observations where the model’s prediction did not match the true label. Generally, we assume = 0.5, although in some instances (e.g. low event rates) it might be more appropriate to choose .

We define the standard Radial Basis Function (RBF) Kernel as the function:

where σ > 0 is a fixed bandwidth parameter that controls the influence of the kernel, and is the squared Euclidean distance between two points  x and . Unlike the standard RBF kernel with a fixed bandwidth, we employ a local RBF kernel [16]:

where and are local bandwidths computed individually for each point based on its k^th nearest neighbor distance within the combined set of errors and test points. This allows the kernel to adapt to local density: points in dense error regions have smaller σ (sharper influence), while points in sparse regions have larger σ (broader influence). The purpose of this is to strike a balance between the number of points smoothed and the number of points with which we can make a high-confidence decision. For any test point , PECA computes a similarity score s(x) that measures how similar x is to the known errors. For each test point x, we calculate the k-nearest neighbor similarity with respect to the error set as the average similarity over the k most similar error points:

In practice, we choose to ensure sufficient representation of errors for robust similarity estimation. In most tabular settings with modest amounts of features, provides enough local samples to sufficiently estimate kernel similarity without over smoothing the neighborhood with distant points. We note that as the smoothing effect becomes more intense. Therefore, we observe less points being predicted with high confidence. The application area should generally dictate how strict of a to use. If the effect of the model making a high confident error is critical, then k should be chosen to be small.

Given the base model probability and the similarity weight s(x), the PECA probability is defined as a convex combination of the model’s estimate and a neutral confidence level α:

By using local bandwidths, this formulation automatically adapts the smoothing strength to the local error density, avoiding excessive smoothing in sparse regions while preserving strong correction in dense error regions. We provide a brief justification of this formulation from a Bayesian perspective in the Supplementary Material. An algorithmic block describing PECA’s implementation can be found in Supplementary Fig. 1.

Simulation design

We performed several simulation studies to compare the performance of PECA under different data generating structures. The central theme of the simulations was to understand how PECA affected the reliability of a model’s predictions when trained on one dataset and extended to another. Such a task is common in clinical research (e.g. a model developed in one hospital and needs to be applied in another). We compared the performance of PECA to other commonly used model generalization techniques (Kernel Mean Matching (KMM), Local Recalibration (LoRe) [15], and the Mahalanobis Out Of Distribution (OOD) adjustment). We evaluated the model’s performance using Accuracy, AUC, Conditional Misclassification Rate (CMR), Brier Score, and the number of observations with a high confident prediction. CMR is defined by the following equation,

where is the high-confidence threshold. The training and testing covariates were drawn from a multivariate Gaussian distribution:

where the covariance matrix Σ was assumed to be the identity matrix. Binary outcomes were generated from a logistic regression model. The probability of a positive class for an instance was defined as:

where was a sparse coefficient vector. Sparsity was introduced to simulate a high-dimensional setting where many predictors are uninformative. We used sample sizes of 1,000 and 500 for our training and testing sets, respectively. We varied the number of features drawn from 50 to 1,000 to understand how PECA performs in different dimensionalities. The core simulation task was to evaluate how each method performed on unseen data. The choice to do this emulates real world challenges where a model might be developed in one research center but need to be deployed in another. To that end, we perturbed the testing domain’s data generating process to create a shifted dataset. The types of shifts were defined as follows:

1. No Drift: Null case; the data generating mechanism for the training and testing sets was identical.

2. Covariate drift: A shift was induced in the marginal distribution P(X) by adding a scaled, random value to a subset of the features in X_t (where X_t is the feature vector for the testing set).

3. Concept drift: A shift was induced in the conditional distribution P(y | X) by perturbing a subset of the logistic model coefficients β to create β_t (where β_t is the coefficient vector used for generating the outcome labels in the testing set).

4. Hybrid drift: A simultaneous application of both covariate and concept drift.

To demonstrate that the method is model agnostic, we used 3 base models: Logistic Regression (GLM), XGBoost, and Support Vector Machine (SVM). The source data (D_s) was partitioned into a training set (D_s^train, 70%) and a validation set (D_s^val, 30%), and a separate test set (D_t) was used for external evaluation of model performance. Base models were trained exclusively on D_s^train. D_s^val is used to make an initial set of predictions and define the error space used by PECA downstream. We performed 100 simulation rounds to ensure robustness.

Data application description

To demonstrate the applicability of our method to a real-world scenario, we consider the task of predicting future cognitive impairment status. Current AD clinical trials are attempting to evaluate whether anti-amyloid antibodies slow progression of cognitive impairment among cognitively normal AD positive individuals (those with abnormal amyloid levels) [17, 18]. Previous work has shown this to be a complex task resulting in poor predictive performance and low statistical power [19, 20]. The difficulty arises partially from the heterogeneity in the clinical manifestations of AD (some people with similar baseline characteristics may follow significantly different trajectories). Furthermore, confirming amyloid positivity requires biomarker testing, most commonly amyloid positron emission tomography (PET) imaging, which is often costly and inaccessible to those not in a research setting. Developing predictive models for future cognitive impairment can help to (1) increase patient safety and (2) reduce trial enrollment costs since they will identify a subset of the population likely to have a clinically meaningful cognitive progression.

We trained and validated our base model using the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset [21]. After training an enrollment model on the ADNI data, we tested it using the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set (UDS) Version 3.0 [22]. Please see the acknowledgements sections for more information on the NACC and ADNI. We included age, sex, education, APOE 4 allele count, and Montreal Cognitive Assessment (MoCA) scores as features and trained a model using XGBoost (100 rounds, learning rate 0.05, max depth 4, subsample 0.7). The estimand of interest was conversion from normal cognition to MCI/Dementia within a three-year conversion window. For both training and testing, we used baseline clinical data for those who were cognitively normal with no evidence of MCI/Dementia at their baseline assessment. We further required, as an inclusion criterion, a positive PET amyloid scan (for both NACC and ADNI, we used derived variables to determine PET positivity). We considered someone to have “future impairment” if any of their subsequent 3 visits indicated MCI/Dementia. In NACC, we used clinician derived diagnoses for cognitive status. In ADNI, we used a Clinical Dementia Rating (CDR) of 0 or 0.5 to indicate no MCI/Dementia and 1.0 or higher to indicate MCI/Dementia. This diagnostic discrepancy was chosen to mimic real-world data drift, where the definition of the outcome might not be consistent between sites. A CDR of 0.5 was combined with a CDR of 0 to represent normal cognition to very mild impairment that doesn’t rise to the level of MCI. This was done to maintain a moderate sample size for modeling. We applied PECA, KMM, LoRe, and Mahalanobis OOD shrinkage adjustments and compared them to baseline. Standard errors were estimated using 2,000 bootstrap rounds. In each round, the training dataset (NACC) was resampled with replacement and the full modeling pipeline was refit, while the external test set (ADNI) was held fixed. Performance metrics were evaluated on the test set for each replicate, and standard errors were computed as the empirical standard deviation of these bootstrap estimates.

This application provides an example of using non-standard α = 0.5 thresholds. Because the prevalence of 3-year conversion in our training data was low (0.33), we altered our classification threshold to reflect this. We chose α = 0.3 as our standard classification threshold and subsequently used 0.2 and 0.4 as our high-confidence thresholds. These values represent a 50% relative decrease/increase in risk of the outcome. Doing this allows our model’s predictions to be interpreted relative to baseline risk in our training population. As a result, the high-confidence regions can be thought of as clinically meaningful increases/decreases in risk.

To demonstrate the downstream clinical utility of using an approach such as PECA, we conducted a power analysis for a hypothetical clinical trial comparing three models: a standard model (no adjustment using an α = 0.3 classification threshold), a vanilla high-confidence model (no adjustment using an α = 0.4 classification threshold), and a PECA model (PECA adjustment using an α = 0.4 classification threshold). We assumed a sample size of 500 (250 per arm), which is in line with existing phase 2 AD clinical trials [23, 24]. We evaluated changes in statistical power across a range of small effect sizes: [0.01,0.15] in increments of 0.01. Small effect sizes are frequently used in modeling clinical outcomes because many AD and cognitive-decline trials have reported modest standardized effects on cognitive endpoints, often with Cohen’s d values below conventional thresholds for moderate effects [25].

Results

Simulation results

Figure 1 displays a graphical representation of the model performances for four of the metrics of interest. We observed that the relative performance behavior of each model type was consistent. Therefore, we focus on presenting the results of the XGBoost model, since is the current state-of-the-art method in the field of tabular classification. Table 1 displays the results from our core simulation task which compares model generalization techniques across varying data generating shifts. We find that across all data shift scenarios, PECA achieves the lowest CMR, best global calibration, and the most comparable AUC compared to the original model. PECA shows the largest magnitude in CMR reduction when there is no data drift (7.7%). Interestingly, PECA reduces the CMR by 7.5% in the concept shift setting – a setting in which it is particularly difficult for a model to generalize. Paired t-tests revealed that across all drift types, PECA achieved a significantly lower CMR than each of the comparison methods. Furthermore, PECA significantly reduces CMR across all data drift settings. This is an important property of the method as its benefits are not dependent on the presence of data drift but instead remain consistent regardless of the similarity between training and testing data. The improved CMR comes at the cost of fewer total high-confidence predictions, a phenomenon that we also observe for LoRe and Mahalanobis OOB as all three methods attempt to balance retention of high-confidence predictions and minimization of CMR (see Supplementary Fig. 3).

Fig. 1.

Aggregated performance metrics visualized by model type and method. Bars indicate metric values. Error bars indicate 95% confidence intervals (± values)

Table 1.

Aggregated performance metrics grouped by drift type and method

Drift Type	Method	Avg CMR	Avg ACC	Avg AUC	Avg Brier	Avg N
None	Original	0.142 ± 0.044	0.773 ± 0.043	0.857 ± 0.040	0.161 ± 0.028	336 ± 27.8
	PECA	0.065 ± 0.046	0.773 ± 0.043	0.857 ± 0.040	0.159 ± 0.019	168 ± 31.4
	Kmm	0.146 ± 0.047	0.766 ± 0.042	0.851 ± 0.041	0.165 ± 0.028	333 ± 27.7
	LoRe	0.100 ± 0.048	0.759 ± 0.046	0.848 ± 0.041	0.169 ± 0.025	249 ± 31.4
	Mah	0.097 ± 0.066	0.773 ± 0.043	0.838 ± 0.043	0.168 ± 0.018	101 ± 48.6
Covariate	Original	0.134 ± 0.075	0.775 ± 0.081	0.848 ± 0.053	0.157 ± 0.053	327 ± 53.4
	PECA	0.075 ± 0.059	0.775 ± 0.081	0.847 ± 0.052	0.156 ± 0.041	199 ± 70.6
	Kmm	0.172 ± 0.084	0.747 ± 0.089	0.808 ± 0.060	0.180 ± 0.061	332 ± 59.1
	LoRe	0.091 ± 0.063	0.767 ± 0.109	0.841 ± 0.056	0.165 ± 0.049	237 ± 68.4
	Mah	0.090 ± 0.074	0.775 ± 0.081	0.825 ± 0.061	0.167 ± 0.036	92.6 ± 61.0
Concept	Original	0.233 ± 0.182	0.700 ± 0.141	0.772 ± 0.166	0.220 ± 0.116	336 ± 27.8
	PECA	0.158 ± 0.203	0.700 ± 0.141	0.771 ± 0.166	0.198 ± 0.077	168 ± 31.4
	Kmm	0.236 ± 0.176	0.696 ± 0.137	0.767 ± 0.162	0.222 ± 0.111	333 ± 27.7
	LoRe	0.196 ± 0.192	0.691 ± 0.123	0.761 ± 0.155	0.219 ± 0.099	249 ± 31.4
	Mah	0.191 ± 0.218	0.700 ± 0.141	0.754 ± 0.163	0.202 ± 0.069	101 ± 48.6
Hybrid	Original	0.225 ± 0.213	0.706 ± 0.170	0.767 ± 0.153	0.213 ± 0.133	327 ± 53.4
	PECA	0.173 ± 0.237	0.706 ± 0.170	0.766 ± 0.153	0.196 ± 0.096	199 ± 70.6
	Kmm	0.258 ± 0.209	0.681 ± 0.174	0.736 ± 0.145	0.234 ± 0.136	332 ± 59.1
	LoRe	0.189 ± 0.230	0.700 ± 0.181	0.758 ± 0.144	0.213 ± 0.120	237 ± 68.4
	Mah	0.186 ± 0.234	0.706 ± 0.170	0.746 ± 0.148	0.200 ± 0.081	92.6 ± 61.0

Bold values indicate the best performance per drift type for each metric (min CMR, max ACC, max AUC, min Brier). Avg N represents the average number of points with a high confidence prediction. 95% CIs are indicated as ± values. Paired t-tests revealed the reduction in CMR for PECA to be significant in relation to each comparison method across all drift types

We also performed sensitivity analyses to understand whether the ratio of samples to features impacted the performance gains, since it is well known that Euclidean distances become unreliable in high dimensional spaces. The relative improvement we observed with PECA persisted across these scenarios, alongside an expected decrease in accuracy and AUC exhibited by all methods (see Supplementary Fig. 2).

Data application results

Table 2 displays the full results of the clinical trial enrollment tool. The KMM model failed to converge and therefore was excluded. The sample sizes of our data appeared insufficient to reliably estimate the importance weights, leading to numerical instability during optimization. The PECA model achieved the lowest CMR (26.0% ±3.58%) compared to the standard model (27.45% ±3.53%). A trial practitioner might consider using PECA if their trials typically suffer from low power. PECA assists in this regard because it decreases the error rate in the enrollment population allowing for a clearer estimation of the treatment effect. This is shown clearly in Fig. 2, which displays the results of the power analysis, where power is defined as the probability of correctly detecting a nonzero treatment effect at a fixed significance level. For small effect sizes, the PECA adjusted model consistently outperforms the high-confidence vanilla model and the standard model. In the AD clinical trial space, even subtle improvements in power can improve the ability to estimate treatment effects if they exist.

Table 2.

Performance metrics for the standard model versus PECA, LoRe, and Mahalanobis adjusted models predicting normal cognition versus cognitive impairment and AD

Method	CMR	ACC	AUC	Brier	N
Standard	0.275 ± 0.035	0.685 ± 0.037	0.618 ± 0.037	0.200 ± 0.009	233 ± 27
PECA	0.260 ± 0.036	0.685 ± 0.037	0.624 ± 0.035	0.196 ± 0.007	211 ± 30
LoRe	0.293 ± 0.038	0.689 ± 0.039	0.593 ± 0.052	0.236 ± 0.019	278 ± 22
Mahalanobis	0.273 ± 0.037	0.685 ± 0.037	0.608 ± 0.038	0.198 ± 0.008	219 ± 30

Bold values indicate the best performance (min CMR, max ACC, max AUC, min Brier). 95% CIs are indicated as ± values

Fig. 2.

Attainable power simulation using a sample size of 500 (250 per arm) for varying effect sizes between 0.01 and 0.15 (in increments of 0.01) for 3 scenarios: PECA Adjusted High-Confidence Model, Vanilla High-Confidence Model, and Standard α = 0.3 Cutoff Model

An important consideration in our results is the relatively small sample size of our testing cohort (NACC; N = 314). On average, we predicted 8% of these participants to progress within 3 years. While this number is small, it is important to note that modern AD clinical trials screen well over 5,000 people for inclusion into a trial. A4, for example, screened 6,673 people [17]. Under similar screening conditions, we would expect our model to predict about 533 people to be enrolled into the trial (which is reflective of our sample size choices for the power simulation). The use of PECA may increase the number of individuals who must be screened, thereby increasing the screen failure rate. Should the “stringent” nature of PECA become prohibitive, we propose using the workflow pictured in Fig. 3. Doing so allows PECA to function as a second opinion to be used in combination with clinically derived enrollment decisions. This workflow allows researchers to alter the amount of risk that they will tolerate with false positive enrollments. That is, if PECA does not enroll enough people, researchers may elect to enroll more using their own recommendations.

Fig. 3.

Proposed clinical usage of PECA adjusted models. The flow demonstrates how a high-confidence probability from a prediction model can be ultimately leveraged alongside clinical judgements and diagnoses

Discussion

This study presents a novel way of thinking about how ML prediction should ultimately work in clinical workflows. While many models and frameworks focus on maximizing global metrics, we empirically show that doing so masks erroneous predictions made with high confidence. Further, we rigorously showed through simulations and a real-world example that PECA consistently reduces confident misclassifications while maintaining the integrity of other global metrics (like AUC and calibration). This is important because it allows us to develop “safer” models without compromising predictive ability. PECA focuses on attenuating confidence proportional to a new observation’s similarity to historical error regions made during model training. These findings suggest that the approach to developing truly reliable models in clinical settings needs to look beyond average performance metrics and into localized evaluations tailored for clinically relevant aspects of error and risk.

We presented an application of PECA to a clinical trial enrollment workflow as a proof of concept. In AD clinical trials, ensuring that both arms of the study are properly enriched with participants who will have the outcome of interest is crucial to be able to detect a treatment effect, especially when the effect size is small [25]. Beyond just effect detection, erroneous enrollment into a trial may lead to undue drug exposure will erroneous exclusion from a trial could result in the failure to receive a beneficial therapeutic. By smoothing predictions proportional to similarity in historical error prone regions, PECA attenuated the high-confidence error rate by 2% and thereby improved the attainable statistical power uniformly across a range of effect sizes consistent with AD drug development. As mentioned in the results, using PECA inherently increases the screen failure rate for a trial. However, as biomarker tests become more available and the screening process becomes more accessible, having a high screen failure rate may become increasingly acceptable because the number of patients being screened can be increased without a large increase in cost. While this enrollment method did result in enrichment and increased statistical power, it should be seen as a byproduct of the primary goal of PECA which is making safety focused enrollment predictions.

In clinical workflows today, predictive ML models are used for early sepsis detection, triage assignment and management, radiological diagnostics, precision medication dosing, and more [26–29]. In each of these cases, the effect of a model producing overconfidently wrong predictions is a direct threat to patient safety. Therefore, while we present a demonstration of PECA to assist clinical trial researchers, the fundamental lens through which we view prediction in these high-risk clinical settings can be extended far past clinical trials. As such, we argue that anyone developing a predictive model to aid or replace an existing clinical workflow should consider using an approach such as PECA to attenuate biased overconfident predictions.

PECA is not without limitations. A primary limitation of the proposed approach is that it intentionally alters a model’s predictions by reducing its confidence on average. In particular, PECA is designed to reduce errors in high-confidence regions, which necessarily comes at the cost of assigning high confidence to fewer predictions overall. While such behavior is not universally desirable, it may be advantageous in clinical settings, where false positive or false negative predictions can have severe consequences, and a more conservative approach is warranted. An additional limitation of PECA is its sensitivity to hyperparameter selection, as the optimal values are not fixed across applications. Selection of α is generally application-dependent, as demonstrated in our data example where we used α = 0.3 due to observed low prevalence of 3-year conversion in our training data. In extreme cases, the CMR could theoretically be reduced to zero, resulting in no predictions being assigned high confidence. We chose to ensure sufficient representation of the errors in the smoothing procedure. For clinical trial applications, we recommend selecting hyperparameters that align with the sample size requirements necessary to adequately power the study. An alternate approach would be using cross-validation to optimize a joint objective function which minimizes CMR subject to a coverage constraint (number of points predicted with high-confidence). The constraint here ensures that the CMR is not trivially reduced to 0 by reducing coverage to 0 simultaneously. Additionally, PECA assumes that the structure of prediction errors is relatively stable between datasets. In the presence of severe concept drift, where future errors no longer resemble historical errors, its effectiveness may deteriorate. Finally, PECA is intended for use in settings where the underlying model exhibits acceptable average performance. In tasks that are inherently difficult to predict, where the model is already highly uncertain across predictions, the application of PECA is unlikely to provide additional utility.

In summary, as the use of ML systems increases in clinical decision-making workflows, there is an imperative need to ensure models are safe and reliable in addition to being highly accurate. PECA directly addresses this duality by using historical error patterns to minimize the number of high-confidence errors. At a minimum, PECA can help to identify situations where a model’s predictions may be less trustworthy, prompting clinicians to weight other sources of information more heavily when making decisions. Future work might consider alternate approaches to defining historical error regions and automated hyperparameter selection that appropriately balances the tradeoff between effective sample size and reduction of confident misclassifications. Furthermore, future work could explore extending PECA beyond classification settings to regression and survival analysis. This would require reimagining an adjacent CMR metric in the regression setting and formulating a loss function which attenuates predictive variance and risk estimates. Moving beyond average performance and towards precise management of model confidence is a necessary step toward developing truly safe and reliable predictive models for digital public health.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

Concept and design: HB PP, and YT. Acquisition, analysis, or interpretation of data: HB, SL, and YT. Drafting of the manuscript: HB. Critical revision of the manuscript: All authors.

Funding

None reported.

Data availability

The NACC and ADNI data are available online through application and reasonable request. Please visit their respective websites to gain access. All code used to generate the simulation data and analysis can be found here: https://github.com/Henry-Bayly/PECA.

Declarations

Ethics and consent to participate

Not applicable.

Consent for publication

Not applicable.

Role of the funder/sponsor

Not applicable.

Competing interests

The authors declare no competing interests.