“Doing no harm” in the digital age: navigating tradeoffs and operational considerations for privacy-preserving deep learning in medicine

Abstract

As artificial intelligence (AI) is increasingly deployed in clinical care, protecting patient privacy has become a central challenge. Mohammadi et al. examine the emerging use of privacy-enhancing technologies in medical deep learning, and look at how formal privacy guarantees interact with model performance and fairness. Their findings reveal variability in performance-privacy tradeoffs across data types and tasks, and highlight the risk that privacy constraints may widen existing disparities if not carefully evaluated. Beyond performance, privacy and equity should also be considered as equal pillars of trustworthy medical AI.

Introduction

As artificial intelligence (AI) transitions from proof-of-concept models trained in academic silos to real-world deployment, the need to examine operational considerations beyond performance becomes paramount. AI can inadvertently encode traces of the patient data on which they were trained¹ – data which healthcare systems are legally and ethically obligated to protect². Preserving this is fundamental to maintaining patient trust and the social license for data-driven healthcare^3,4, particularly as health data contains personal identifiers, histories, and uniquely sensitive information that can have profound implications for the individual if they are exposed⁵. Emerging threats from malicious actors such as model inversion attacks – attempts to reconstruct sensitive attributes of the data used to train a model – threaten this balance^6,7.

What is differential privacy?

Differential privacy is a cybersecurity method that has been positioned as one of the leading approaches to address privacy concerns⁸. It describes a mathematical framework designed to preserve the anonymity of individual data contributors⁹. The fundamental concept is straightforward – a model’s outputs should not change significantly whether or not any single individual’s data contributed to the training process. It works by adding carefully calibrated “noise” (or random mathematical variation) to limit how much information about any data point can be learned, whilst still allowing the model to learn meaningful patterns. However, as with any powerful tool, its value depends on how it is deployed and evaluated in the context of clinical priorities.

Mohammadi et al¹⁰. present a scoping review on differential privacy in medical deep learning, which included 74 eligible studies from four databases searched to 1 March 2025. The authors examine how differential privacy-based methods mitigate privacy attacks, and how this influences model performance and fairness in clinical applications. This is particularly pertinent because such mitigation strategies can alter the model learning process, with consequences for downstream utility (performance) and reliability¹¹.

The privacy-utility tradeoff reporting gap

Mathematically, the strength of a differential privacy guarantee is governed by the “privacy budget”, which is typically denoted by parameters such as ε (epsilon). This is a metric quantifying how much a single individual’s data influences the model’s outputs⁹. Lower values of ε confer a stronger guarantee of privacy by injecting more noise into the learning process while risking substantial performance degradation, while larger ε relaxes privacy but improves model utility. This privacy-utility tradeoff is a core challenge for differential privacy, and contextualizing this in clinically meaningful terms is essential for transparent evaluation.

Mohammadi et al. reported that studies involving more complex data and tasks (e.g. histopathology, dermoscopy, and speech) described larger performance drops at high privacy budgets compared to the baseline (prior to applying differential privacy), suggesting that differential privacy might be more readily deployed in some clinical domains than others that would require additional privacy-enhancing techniques to address¹⁰. However, in general, the included studies differed in how they reported performance under different privacy budgets (e.g. AUROC vs sensitivity), with many reporting performance metrics at one ε level alone, limiting direct comparability across specialties, data modalities, and tasks¹⁰. This makes it challenging to determine whether observed performance declines reflect inherent limitations of differential privacy in certain domains, or simply differences in study design and reporting.

This reporting gap is also analogous to evaluating a “dose-response relationship” for drugs, where reporting accuracy at one privacy budget is akin to reporting a drug’s effect at a single dose, and does not suffice to show how efficacy and adverse effects change as the dose increases. In clinical contexts, knowing that privacy is “preserved” is not enough – instead, how much privacy was achieved and what was sacrificed to obtain it is essential for navigating the tradeoffs between privacy and utility. As ε is a continuous parameter, reporting model performance across several pre-defined ε levels (e.g. high, medium, low) to allow transparent characterization of this trade-off may support threshold-based decisions by developers and data governance committees, without substantially increasing analytical complexity.

Reporting of other key parameters such as delta (δ, the probability of privacy leakage) and key details for differential privacy implementation to support reproducibility was also poor.

Equity as an essential (but often missing) design principle in differential privacy

Mohammadi et al. also highlight the importance of examining how differential privacy behaves across sociodemographic subgroups. Limited data suggest that models trained on imbalanced datasets already tend to perform worse on underrepresented subgroups, and injecting noise from differential privacy without fairness constraints into these imbalanced datasets can amplify these disparities, likely because noise interferes more with learning patterns in smaller groups¹². This creates an ethical dilemma wherein the very mechanisms designed to protect patient privacy could inadvertently worsen existing inequities.

Only 5/74 studies audited the fairness of their differential privacy techniques, and typically focused on performance across single attributes (e.g. age, sex, ethnicity) without examining intersectionality, which may be overly reductionist as intersecting identities (e.g. age and sex, or age and ethnicity) may create compounding biases that cannot be predicted from the effect of each identity in isolation^13,14. These issues may relate to poor recording of sociodemographic attributes across multiple datasets^15,16, data imbalance due to underserved groups having limited access to healthcare services¹⁷, or health data poverty in settings with limited resources for data curation¹⁸. As such, incorporating inclusive standards from initiatives such as the STANDING Together (Standards for Data Diversity, Inclusivity, and Generalizability) consensus recommendations¹⁹ into the evaluation of privacy-preserving techniques may help ensure that efforts to protect individual data do not further obscure or amplify existing inequities.

Even though techniques that integrate fairness constraints or subgroup-aware optimization into differential privacy training can mitigate disproportionate harms¹², they do not remove the need for explicit value judgments about acceptable privacy-utility trade-offs. This highlights the role of co-designing privacy and fairness objectives with relevant stakeholders beyond developers (e.g. clinical experts, governance leads, patient and public representatives) rather than treating this as an afterthought to system design and evaluation^20,21. This broader involvement may help ensure that privacy protections are implemented in ways that protect individuals in theory without worsening inequities in practice.

Real-world implications and future directions

Privacy engineering cannot remain an abstract academic exercise, and should also be firmly anchored in governance and regulatory frameworks, although these vary across jurisdictions. Privacy laws such as Europe’s General Data Protection Regulation (GDPR) already embed principles that align with formal privacy-preserving methods, such as data minimization, integrity and confidentiality, and accountability for privacy risks²². The UK Information Commissioner’s Office has also explicitly recognized differential privacy (alongside other privacy-enhancing techniques) as part of responsible AI design²³. While there is no clear regulatory mandate for choice of thresholds (e.g. ε) for privacy-preserving AI in healthcare, developers must typically provide a risk-based justification for their design choices comparable to current “state-of-the-art”. Defining clinically meaningful thresholds for acceptable performance degradation under privacy constraints and developing practical tools for auditing privacy leakage and the impact on performance in key subgroups should be considered as priorities for the field, and reflect emerging best practice that regulators are increasingly likely to scrutinize, even where it is not yet formally codified as a requirement.

Finally, differential privacy is one part of a broader ecosystem of privacy-enhancing technologies. Federated learning offers a way to train models without pooling data centrally, but does not eliminate the risk of information leakage from models²⁴, which suggests a role for combining federated architectures with differential privacy as a complementary safeguard²⁵. In parallel, synthetic data generation under formal privacy constraints is gaining traction as a method of enabling data sharing for model development and benchmarking while preserving privacy. However, ensuring that synthetic datasets preserve clinically meaningful structure (especially for longitudinal and high-dimensional data) without encoding or amplifying biases remains an open challenge²⁶.

Conclusion

Mohammadi et al. highlight that privacy should not be an afterthought but should be considered early in the AI lifecycle, as it involves explicit design choices that directly shape performance, fairness, and user trust. Privacy should not be treated as a binary feature but as a design parameter requiring explicit evaluation. Developers should consider more systematic exploration of privacy-utility curves across diverse tasks, report performance across predefined privacy budgets, and characterize subgroup and intersectional performance under differential privacy. Data governance committees and procurement bodies can then assess whether the selected privacy budget is proportionate to the sensitivity of the data, and whether clinically acceptable performance is maintained. This shifts differential privacy from a technical abstraction to a practical governance tool, aligning privacy, performance, and equity as equal considerations in responsible clinical AI.

Author contributions

AYO wrote the original draft of the manuscript. All authors (AYO, KLR, MS, JCK) reviewed and critically revised subsequent drafts, and have read and approved the submitted manuscript.

Data Availability

No datasets were generated or analysed during the current study.

Competing interests

JCK is the editor-in-chief of npj Digital Medicine and had no role in the peer review or decision to publish this manuscript. The remaining authors do not have any conflicts of interest to declare.