Prognostic models will be victims of their own success, unless…

Abstract

Predictive analytics have begun to change the workflows of healthcare by giving insight into our future health. Deploying prognostic models into clinical workflows should change behavior and motivate interventions that affect outcomes. As users respond to model predictions, downstream characteristics of the data, including the distribution of the outcome, may change. The ever-changing nature of healthcare necessitates maintenance of prognostic models to ensure their longevity. The more effective a model and intervention(s) are at improving outcomes, the faster a model will appear to degrade. Improving outcomes can disrupt the association between the model’s predictors and the outcome. Model refitting may not always be the most effective response to these challenges. These problems will need to be mitigated by systematically incorporating interventions into prognostic models and by maintaining robust performance surveillance of models in clinical use. Holistically modeling the outcome and intervention(s) can lead to resilience to future compromises in performance.

Artificial intelligence and machine learning change how patients and healthcare providers interact.^1–3 Many hope that predictive models are one of the technologies that improve patient health by giving insight to our current or future health, thereby promoting patient-provider-caregiver decisions that improve clinical outcomes and quality of life.^4–7 Predictive models show great promise in a number of applications, such as reducing time to accurate diagnosis, reducing complications through automated monitoring and prognosis, and empowering preventive medicine through personalized forecasting.^8–15

Predictive models have a life cycle. This life cycle is iterative and nonlinear. In the simplest form of the model life cycle, it begins with model development, updating, or validation. Initial model development has a large and well-established literature.^16–21 Next, model developers and information technology professionals implement their models into the workflow of patients, clinicians, or administrators, so that the models might be acted on. The model implementation and deployment phase requires a different set of skills from model development. This phase is also one that investigators have studied in the informatics and implementation science literature, although this field is less mature than model development.^22–28 Within this stage, explicit consideration for how and which downstream actions could be triggered by the model implementation are often neglected.^29,30 Once the model is implemented, and impacted process workflows stabilize, the model enters the surveillance and maintenance phase.^31–35 This phase requires skills from both the development and the implementation phase.

Surveillance and maintenance are key to preventing degradation of model performance over time.^32,36 Models degrade in healthcare, because those models attempt to characterize biological, business, or behavioral processes with unknown or unmeasured confounders.³⁷ Those models also rely on readily available data that cannot fully describe the underlying patient case mix, practice of medicine, and work environment. Known deterministic formulae (eg, force = mass × acceleration) rarely govern the processes informatics practitioners attempt to model, leaving those models vulnerable to changes in the processes they attempt to describe.¹ Impactful changes in the data process can also manifest as changes in documentation requirements, medical practice, or database definitions. These data process changes affect the performance measures of a predictive model such as its calibration (how well a model’s forecasts reflect the true values)³⁸ or its discrimination (how well a model can split data into different meaningful categories).³⁹ The extent of causal predictors used in a model will partly dictate the attrition in calibration and discrimination. A model only associated with the target outcome may suffer degradation in both discrimination and calibration, whereas a more causally founded model may only lose calibration.

During maintenance, both social and technical adjustments are made to optimize and maintain model-related workflows over time. Researchers have not studied this phase as well as the preceding 2 phases. The last portion of the model life cycle is the de-implementation phase.^40–44 In de-implementation, the model is removed or replaced due to a change in workflow, information technology, modeling technique, or some combination of factors.⁴⁰Figure 1 visualizes the stages of the model life cycle—development,^16–21 implementation,^22–28 maintenance,^31–35 and de-implementation^40–44—along with some example components for each stage. Further details on each component can be found in the accompanying reference in Figure 1.

Figure 1.

Model life cycle.

The diversity of skills needed to develop, deploy, and maintain a model give reason to silo different parts of the model life cycle to different teams. However, compartmentalization may come at the cost of efficiency and efficacy for prognostic models. Prognostic models seek to predict a future patient state, event, or outcomes.^3,45–47 Some prognostic models incorporate the intervention space into the modeling process or forecast nonmodifiable risk (eg, genetic risk).^48–51 Many prognostic models forecast modifiable risk without accounting for interventions.^25,52–54 For example, a model that forecasts the probability of the onset of sepsis in the next 6 hours for a patient on a medical ward is prognostic. After issuing the forecast, clinicians may act by administering broad-spectrum antibiotics or transfer to intensive care. The fundamental purpose of that prognostic model is to improve clinical outcomes, which will change the outcome distribution over time. Deploying a prognostic model into clinical sociotechnical workflows should change behavior. The implicit assumption of the prognostic model is that providing enough forewarning should allow the forecasted outcome(s) to be modified, optimized, or mitigated. This assumption establishes an intimate relationship between the prognostic model and the intervention(s) the model hopes to activate. Despite this interdependence, many models are developed without specifying intervention(s) or without consideration for the implementation phase (eg, readmission risk and suicide attempt risk).^52,55 This divide may help explain the limited impact of readmission risk models on clinical care. Model developers have reported gains in the initial performance of readmission models along with increased provider adoption of readmission risk models,^56,57 yet 30-day readmission rates and age-adjusted mortality rates have stagnated.^58,60 Even fewer models are developed with consideration to the maintenance phase.

If end users (eg, patients, clinicians, administrators) act in response to a forecast, then the distribution of the outcome should be altered in some way if the response impacts the outcome. Ideally, prognostic models anticipate and attempt to account for expected changes in the data process due to the fact that they are being introduced. Figure 2 shows the evolution of a prognostic model from observing a clinical process while in development to directly interacting with that process after implementation.

Figure 2.

Model interactions with outcome.

Directly changing the outcome process has strong potential implications for prognostic model maintenance. In the idealized case, end users incorporate the model to great effect. The intervention and model deployment reduce the variance of the outcome, and may shrink the relationship of predictors associated with the outcome toward zero. The magnitude of these effects is directly related to how strongly the intervention impacts the outcome, which can range from very mild impacts (if the intervention is less effective than desired) to very strong ones. The variance of the dependent variable is reduced as the model-directed interventions promote the desired outcomes. With significantly fewer poor outcomes, the distribution of outcomes becomes more concentrated. Actions inspired by the prediction may disrupt the association of predictors to outcomes. This disruption takes place as a result of risk-focused interventions modifying the occurrence of outcomes. For this reason, the successful use of the model results in its own decline in performance. The model is a victim of its own success. Figure 3 demonstrates the results of model refit after the deployment of an intervention on patients at risk in a simple univariate regression simulation. For this example, we assume the presence of a perfect discrimination model. As the intervention gains effectiveness, the slope of the regression line moves toward zero.

Figure 3.

Consequences of model success.

This result should also be observed in more realistic scenarios in which the prognostic model and the interventions are only moderately effective. As the incidence of the outcome changes in the desired direction, the baseline event rate also shifts, and almost all models are sensitive to changes in overall population outcome event rate. Event rate shifts particularly impact calibration. Under these circumstances, the performance of standard evaluation metrics for calibration, such as log loss, explained variance,⁶¹ or Brier score,⁶² and discrimination (sensitivity, specificity, or positive predictive value)⁶³ will appear to deteriorate. In the idealized case of Figure 3, the model still performs perfectly years later. However, naive assessment of the model without attention to the transformation in the outcome process will result in a suboptimal refit of the model. The refit model will misclassify some patients in need of intervention. Refitting again can lead to the same issue for a different group of patients creating an iterative cycle. Operating the model in a degraded state results in misclassification, potentially leading to safety concerns. This negative cycle may also hasten a model to its de-implementation, diminishing the return on investment of time and capital put into model development, deployment, and maintenance. One could ignore model updating altogether but would still encounter degradation in performance associated with temporal changes of the underlying data process at some point. The same questions of safety and adequate return on investment are also relevant when avoiding model maintenance. There are strategies to mitigate this conundrum by thinking holistically about the model life cycle during model development.

One strategy to avoid this maintenance complication is to include interventions into the model.^1,51 Modeling the intervention space can enable the model to adjust for the actions recommended by model forecasts. This step also allows the model to maintain calibration longer, thereby extending the utility of the model. Including variables on the intervention pathway is not trivial. The interventions may be poorly defined, unrecorded, or not yet implemented. For example, patient-implemented interventions, such as a food or calorie log, may be difficult to find within accessible data sources. Intervention variability, in which only few patients receive the same treatment(s), can lead to biased estimates of effect. For example, a patient at risk for depression may have a large number of preventative options presented to them. Issues of adequate sampling for intervention effect estimation will also plague rare events or outcomes.

If the intervention cannot be directly modeled, informatics practitioners should consider incorporating intervention surrogates. Intervention surrogates are data documenting the workflow associated with an intervention. An intervention surrogate should be correlated with the intervention. While it may not fully capture complex effects of an intervention, it can serve a rough estimate of the intervention’s effect. Intervention surrogates can suffer from the same modeling challenges as the interventions themselves.

If modeling interventions directly or through a surrogate is not an option, one can think of the interventions as a latent variable in the model. Latent variables are not directly observed but are inferred through a mathematical framework. For example, when building a retrospective cohort to measure the effects on bone density one treatment has against another, there may be confounders that are not present in the available data (eg, whether a subject regularly exercises by jogging or not). These unmeasured confounders will cause variability that cannot be explained using standard modeling techniques. Latent variable models may permit the recovery of other independent variable weights on the outcome but reduce the modeling options available. Generalized mixed effects models, Hidden Markov models and Bayesian frameworks are natural methods for estimating latent variables; however, these methods can add significant complexity to model development or to maintenance efforts.

Model surveillance remains crucial to preserving clinical utility regardless of how one represents interventions in a prognostic model. One cannot begin to diagnosis true and model-mediated performance degradation in a state of ignorance. The efforts to validate and implement a prognostic model into clinical or administrative workflows are not trivial and the potential to cause harm at scale are real.^64–68 It is therefore prudent to design models and their accompanying infrastructure to enable monitoring and maintenance.

Prognostic models produce forecasts on complex and dynamic data processes. Models require surveillance and maintenance over time, so that model developers and implementers can evolve models in concert with the models’ nonstationary targets. Failure to monitor prognostic models over time will limit their effectiveness and can in the worst case pose a hazard.³² However, model surveillance and updating alone are not enough. If a prognostic model does not incorporate the intervention space directly, through surrogates, or as a latent variable, then model updating may be more frequently required, and may result in exposing more patients to poorly performing model predictions. It is important to keep in mind that the more strongly an intervention impacts the outcome, the more vulnerable prognostic models are to this phenomenon. Prognostic models should be able to mitigate these issues by considering the entire model life cycle as well as the entire outcome process.

FUNDING

This work was supported by U.S. National Library of Medicine grant number T15LM007450-15 (MCL).

AUTHOR CONTRIBUTIONS

We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

Conflict of interest statement

None declared.