Drift isn’t a bug, it’s the work: post-deployment surveillance for clinical artificial intelligence

Clinical artificial intelligence (AI) is moving from promising research to operational infrastructure. Algorithms now inform clinical data interpretation, risk prediction, triage, documentation and workflow. Yet the culture surrounding evaluation still centres largely on what happens before deployment, such as model development, internal validation, external testing and regulatory authorisation. Now, that framing is no longer sufficient. In medicine, approval has never been the endpoint. Drugs enter postmarketing surveillance. Laboratories operate under continuous quality control. Imaging systems are subject to routine quality assurance. Clinical AI should be treated no differently 1

The reason is straightforward: healthcare is not static. Patient populations change. Disease prevalence shifts. Coding practices evolve. Scanner software is updated. Clinical guidelines are revised. Clinicians themselves adapt to the presence of a model. Under these conditions, even a well-validated algorithm can become miscalibrated, clinically misleading or inequitable after implementation. Post-deployment monitoring is therefore not an optional technical add-on, but part of the lifecycle1–3

The narrative review by Rotalinti et al is timely and useful. The authors argue that ‘drift’ should not be treated as a single vague phenomenon but understood through a more precise taxonomy, including covariate drift, target drift and concept drift. They then go further, proposing that the practical significance of drift depends not only on whether change is detected but also on how quickly it emerges, how large it is, what causes it and what consequences it carries for patients, clinical workflow and regulation.4 That is an important advance. The central discussion is not a new detection algorithm but a framework for thinking more clearly about what kinds of change matter and when.

This framing is especially valuable because the most consequential failures of clinical AI may not announce themselves as catastrophic collapses in headline performance metrics. A model may preserve discrimination while changing the number of patients pushed across a treatment threshold. A shift may be modest overall but concentrated at one site, on one scanner or within one patient subgroup. A documentation change may quietly destabilise a natural-language model before any formal outcomes are available.5 In other words, drift is often local, delayed and clinically asymmetric. Monitoring must therefore be layered, combining continuous surveillance of inputs, missingness, distributions and alert rates, followed by periodic performance reassessment when labels become available.3 5

The practical implications are substantial. Monitoring needs predefined triggers and predefined responses. Some responses will not require changing the model at all but rather narrowing indications for use, increasing human review or adjusting decision thresholds. Others may require recalibration, fine-tuning or full retraining. The point is not perpetual technical tinkering but proportionate intervention.6 7 The growing policy emphasis on predetermined change control plans is important here. The Food and Drug Administration’s guidance makes clear that developers should prospectively specify what kinds of modifications are anticipated, how they will be developed and validated and how their impact will be assessed.8 In parallel, hospital-based quality improvement frameworks and real-world monitoring programmes suggest that this work must be owned operationally, not left to developers alone.9

This is where the review by Rotalinti et al is strongest. It provides conceptual clarity at a moment when the field has many dashboards but insufficient language for interpreting their signals and judging their clinical significance.4 However, like most frameworks at this stage, it cannot yet specify universally accepted thresholds for when recalibration is sufficient, when retraining is necessary or when deployment should pause. But that reflects the state of the field more than a flaw in the argument. Clinical AI still has too little prospective evidence on what degree of drift matters, for which use cases and on what time scale.

Several next steps could follow from this framework. Each is conceptually straightforward, even if difficult in practice. First, monitoring plans should be designed before deployment, not improvised after failure. Second, action pathways should be predefined, including when to intensify human review, when to adjust thresholds, when to fine-tune and when full retraining is warranted. Third, model documentation should evolve from static model cards into living labels, with explicit change logs, performance summaries and update histories. Fourth, regulatory tools such as predetermined change control plans should be used not merely to permit updates but to make them auditable and safe. Finally, clinicians and operational leaders need education in ‘drift literacy’, so that unusual alert rates, workflow shifts or subgroup anomalies are recognised as signals requiring investigation, not dismissed as background noise.2 3 6–8

The field should no longer treat implementation as the end of the lifecycle. For clinical AI, deployment is the start of a surveillance lifecycle. The question is no longer whether drift will occur but whether health systems, developers and regulators will be ready when it does.

Data availability statement

No data are available.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.