Learning to Diagnose Privately: DP-Powered LLMs for Radiology Report Classification

Abstract

Large Language Models (LLMs) are being widely adopted in different domains including education, healthcare, and finance. In healthcare domain, LLMs are used in disease diagnosis, abnormality classification, remedy suggestions etc. Multi-abnormality classification of radiology reports is essential in healthcare, medical decision-making, and drug discovery. LLMs are increasingly utilized for such tasks because of their remarkable Natural Language Processing (NLP) capabilities, which streamline medical report processing and reduce administrative burden. To enhance the predictive accuracy, LLMs are often fine-tuned on private, locally available datasets, such as medical reports. However, this practice raises significant privacy concerns, as LLMs are prone to memorizing training data, making them susceptible to data extraction attacks even through query-based access. Additionally, sharing fine-tuned models and weights poses adversarial risks, because they may inadvertently reveal sensitive information about the training data. Despite the growing application of LLMs to medical text classification, privacy-preserving fine-tuning for multi-abnormality classification remains underexplored. To bridge this gap, we propose a differentially private (DP) fine-tuning approach that preserves privacy while enabling multi-abnormality classification from text radiology reports through Low Rank Adaptation (LoRA). Our framework leverages DP optimization techniques to fine-tune LLMs on local patient data while mitigating data leakage risks. To our knowledge, this is the first study to incorporate DP fine-tuning of LLMs for multi-abnormality classification using text-based radiology reports. We used labels generated by a larger LLM to fine-tune a smaller LLM, accelerating inference while maintaining privacy constraints. We conducted extensive experiments on the MIMIC-CXR, and CT-RATE datasets to evaluate DP fine-tuning method across varying privacy regimes, analyzing the privacy-utility trade-off and demonstrating the efficacy of our approach. For instance, on the MIMIC-CXR dataset, our proposed DP-LoRA framework achieves weighted F1-scores of up to 0.89 under moderate privacy budgets ($ϵ=10$), approaching the performance of non-private LoRA (0.90) and full fine-tuning (0.96). These results demonstrate that strong privacy protection can be achieved with only moderate performance degradation.

I. INTRODUCTION

Innovations and advancements in Large Language Models (LLMs) continue to redefine the frontiers of artificial intelligence (AI), particularly in critical domains such as healthcare, and clinical diagnosis where precision, interpretability, and efficiency are paramount. Fine-tuning LLMs on task-specific, locally sourced, and often private datasets is a common approach for adapting models for specialized downstream applications. Although such datasets are typically de-identified to remove personally identifiable and HIPAA-protected health information (PHI), certain sensitive elements, such as device identifiers, medication names, or contextual patterns, may still be inadvertently memorized by models. When combined with external data sources, this memorization can amplify privacy risks [3] by enabling partial data reconstruction or inference of sensitive attributes.

Recent research has demonstrated that AI models are prone to memorizing [1], [12] fragments of their training data, which lead to significant privacy vulnerabilities. For instance, querying fine-tuned models can reveal personal details or verbatim text segments from the training corpus. During fine-tuning on clinical data such as free-text radiology reports, models may inadvertently encode and reproduce identifiable content. When cross-referenced with publicly available datasets, this leakage can facilitate attacks such as membership inference (MIA) or jailbreak prompting [1], [12], [13]. As highlighted in previous studies [17], [24], we underscore the inherent risks associated with fine-tuning LLMs on sensitive medical text and emphasize the necessity of Differentially Private Fine-Tuning (DP-FT). To illustrate the practical privacy risks associated with fine-tuning on clinical data, we demonstrate a memorization example using a generative language model trained on radiology reports in the next sub-section.

Text-memorization illustration by LLMs:

As mentioned previously (also shown in Figure 1, and Figure 2), fine-tuning LLMs can pose significant privacy risks when models are exposed to real patient data. In Figures 1, 2, we illustrate these threats via text-memorization using BERT (used for abnormality classification), and GPT2 (causal language model), and show how these fine-tuned LLMs can memorize parts of training/fine-tuning data. The examples highlight that fine-tuned models can reproduce patient-specific measurements verbatim. This behavior demonstrates a direct privacy leakage mechanism and motivates the need for formally private fine-tuning methods rather than relying solely on dataset de-identification. Our findings show fine-tuned LLMs reflect memorized training content compared to the off-the-shelf (non-fine-tuned) LLM, this indicates that fine-tuned models exhibit especially strong tendencies towards such leakage. As shown in Figure 2, when asked to complete a partial report, fine-tuned GPT2 models were able to replicate parts of the original report “Findings” (24×17mm). Moreover, for some other examples, we observe that the fine-tuned model outputs the next patient’s report (for instance patient with patient identifier p1xxxxxxx) from the fine-tuning dataset, highlighting the privacy vulnerabilities of non-privately fine-tuned LLMs. In Figure 1, we observe a similar trend for fine-tuned BERT model used for multi-abnormality classification. It is interesting to note that BERT based models are not causal language models, yet they still have the ability to replicate portions of reports used for fine-tuning. Specifically, the fine-tuned models achieved a higher cosine similarity score (between generated and original report “Findings”) privately fine-tuned ones (refer to Figure 9). Cosine similarity score measures how similar two pieces of text are by comparing the angle between their vector representations in the embedding space; a higher cosine similarity between generated text and the original training data suggests the model may be closely repeating what it saw during training, indicating memorization. This underscores the potential privacy risks posed by LLMs memorizing sensitive portions of medical reports, especially when applied to tasks such as text completion, disease classification, report summarization, and more.

FIGURE 1.

Memorization behavior of the GPT-2 Large model on text-based radiology reports reveals that, during report completion tasks, the model can recall and reproduce patient-specific numerical values (e.g., 24 × 17 mm) from its training data, thereby amplifying privacy risks.

FIGURE 2.

Example illustrates rising Cosine similarity between original report Findings and BERT-base-generated Findings, indicating memorization even in non-causal models, more pronounced in the fine-tuned variant. While the non-privately fine-tuned model gives highest cosine similarity (indicating most memorization), fine-tuning with DP reduces the cosine similarity score indicating lesser memorization and better privacy.

FIGURE 9.

Average cosine similarity between ground-truth MIMIC-CXR reports and BERT-medium based fine-tuned model-generated completions under varying differential privacy budgets ($ϵ$). Lower $ϵ$ values (stronger privacy guarantees) result in reduced semantic similarity, indicating diminished memorization compared to the non-private fine-tuned model.

While prior studies have explored differential privacy (DP) in model training, its integration into LLM fine-tuning for text-based radiology report classification remains underexplored. To the best of our knowledge, this is among the first systematic empirical studies evaluating differentially private LoRA-based fine-tuning for multi-abnormality classification from free-text radiology reports. Motivated by previous works [5], [10] who presented DP-based fine-tuning of LLMs for medical image classification, we propose a privacy-preserving framework that incorporates Low-Rank Adaptation (LoRA) to enable secure fine-tuning of LLMs on domain-specific text-based radiology data. This framework is adaptable to any task involving free-text medical reports while ensuring formal differential privacy guarantees during the fine-tuning process.

Despite the rapid adoption of LLMs in healthcare text processing, existing works largely address either predictive performance or privacy protection independently. Most medical natural language processing (NLP) related studies optimize classification accuracy without formal privacy guarantees, while privacy-focused research typically evaluates generic datasets rather than highly sensitive clinical narratives such as radiology reports. Furthermore, the interaction between differential privacy and parameter-efficient fine-tuning strategies remains insufficiently understood. Consequently, it is unclear whether practical privacy-preserving LLM adaptation can maintain clinically meaningful performance. This paper addresses this gap by systematically evaluating differentially private LoRA-based fine-tuning for multi-abnormality classification from free-text radiology reports.

II. RELATED WORKS

Modern machine learning (ML), deep learning (DL), and natural language processing (NLP) techniques have greatly advanced multi-abnormality classification, particularly in medical imaging and radiology report interpretation. High-dimensional Computed Tomography (CT) and volumetric imaging data [18] are routinely analyzed for abnormality detection, while traditional deep learning frameworks have achieved substantial success in multi-lesion recognition, abdominal disease detection, and tumor identification. Weakly supervised methods [32] further enhance classification accuracy by reducing reliance on extensive labeled datasets.

Beyond imaging, radiology report generation and text-based classification have emerged as critical tasks, leveraging a combination of ML, DL, and rule-based algorithms. The advent of LLMs has transformed medical text processing, particularly for summarization and classification. Extractive summarization methods using models such as BERTSUM [22], TransEXT, and BERT [22] have proven effective for medical narrative analysis. Domainspecific LLMs including BioBERT [7], RadioBERT [8], and ChestXRayBERT [9] further enhance radiology report understanding, while models such as ChexBERT, Chexpert, and Vicuna facilitate multi-abnormality classification through contextualized embeddings.

Despite their promise, fine-tuning large-scale LLMs for medical applications remains computationally intensive due to the vast number of model parameters. Parameter-efficient tuning methods [2] such as LoRA, Adapters, and Compacters have been developed to mitigate computational overhead while maintaining performance. However, fine-tuning on private clinical datasets introduces privacy concerns, as sensitive patient information can be inadvertently exposed. To address these risks, prior studies have proposed differentially private frameworks for federated and multi-modal learning [10], [16]. Recent work on differentially private optimization (DP-OPT) [10] and private prompt tuning underscores the importance of privacy-preserving approaches in medical AI. Recent advances, such as DP-SSLoRA [17], leverage self-supervised representation learning and low-rank reparameterization to enhance the privacy–utility tradeoff in differentially private medical models. This work demonstrates that integrating self-supervision with DP-LoRA can achieve strong privacy guarantees without sacrificing diagnostic accuracy.

In this paper, we focus on multi-abnormality classification using free-text radiology reports. While LLMs excel at text processing, their lack of specialized medical training and the high computational cost of fine-tuning limit their performance in clinical tasks. By integrating differential privacy with parameter-efficient tuning via LoRA, we aim to reduce fine-tuning complexity while maintaining both privacy and predictive accuracy. In summary, prior literature typically studies either performance optimization of medical NLP models or privacy vulnerabilities in isolation. Comprehensive evaluations that jointly analyze performance and privacy in sensitive clinical text remain limited. Radiology reports represent highly sensitive medical narratives where memorization risks are critical but underexplored. Moreover, differential privacy mechanisms combined with parameter-efficient fine-tuning frameworks such as LoRA have not been systematically evaluated for multi-abnormality classification tasks. This work bridges these gaps by providing a unified empirical and methodological study of privacy-preserving LLM adaptation in clinical text analysis.

III. OBJECTIVE AND CONTRIBUTIONS

Leveraging multi-abnormality labels generated from free-text radiology reports for differentially private fine-tuning (DP-FT) of LLMs provides a promising pathway for achieving efficient and privacy-preserving inference. However, this strategy remains underexplored for multi-abnormality classification from free-text radiology reports. To the best of our knowledge, no prior work has incorporated DP-LoRA [5] for multi-abnormality classification from text radiology reports.

In this study, we first evaluate LoRA-based fine-tuning under non-private conditions to establish baseline performance. While non-private fine-tuning yields strong accuracy, it also increases the risk of memorizing sensitive data. To mitigate this, we propose Differentially Private Low-Rank Adaptation (DP-LoRA), a fine-tuning approach that integrates LoRA with the privacy guarantees of differential privacy (DP). Specifically, we employ Differentially Private Stochastic Gradient Descent (DP-SGD) [30], where gradients are clipped to a threshold $C$ and perturbed with Gaussian noise characterized by privacy parameters ($ϵ,\delta$). This ensures formal and quantifiable protection against data leakage while maintaining competitive classification performance.

IV. PRELIMINARIES

In this Section, we provide some preliminaries on the datasets, and models we have used in this paper along with some background on differential privacy and low rank adaptation.

A. DATASETS

In this study, we have utilized two publicly available, widely used radiology datasets: the MIMIC-CXR [11], and the CT-RATE dataset [18]. MIMIC-CXR dataset comprises 227,835 radiographic studies collected from the Beth Israel Deaconess Medical Center in Boston, MA, USA. Subsampling improves privacy guarantees only under randomized sampling assumptions. Our implementation follows standard DP-SGD mini-batch sampling, where each sample is independently included with a fixed probability. In this setting, privacy amplification by sub-sampling reduces the effective privacy loss per iteration. Therefore, we use a subset of the MIMIC-CXR dataset, specifically 20,883 text reports for training and 2,610 reports for testing. We selected a fixed subset of 20,883 training reports to ensure consistent privacy accounting under DP-SGD. Differential privacy guarantees depend on dataset size and sampling rate; therefore, controlling dataset cardinality allows reproducible and interpretable privacy budgets across experiments. For patients with multiple radiographic studies, we aggregate all associated textual reports, creating unified “Findings” and “Impression” sections to ensure a comprehensive representation. The “Findings” and “Impression” sections are aggregated because clinical diagnoses may appear in either section. Combining them ensures that all abnormality evidence is captured, improving label consistency for multi-label classification. Additionally, duplicate entries, if any, are removed. MIMIC-CXR has been widely used as a benchmark dataset in prior research. However, to the best of our knowledge, no existing work has incorporated differentially private fine-tuning of LLMs for multi-abnormality classification. While we apply this framework to MIMIC-CXR, the proposed differentially private fine-tuning approach can be adapted to other medical text report datasets for multi-abnormality classification. The CT-RATE dataset comprises 25,692 non-contrast chest CT scans, sourced from 21,304 unique patients, and the CT scans are expanded to 50,188 volumes through various reconstructions. Each scan is accompanied by associated radiology reports, multi-abnormality labels, and metadata. The patient cohort was partitioned into two subsets: 20,000 patients were assigned to the training set, while 1,304 patients were designated for validation. We have used the entire CT-RATE dataset, and utilized 80% for training, 10% for validation and 10% for testing. Although CT-RATE contains reconstruction-expanded volumes, we partition the dataset at the patient level, preventing leakage of correlated samples across training and testing splits and avoiding evaluation bias.

B. MODELS AND REFERENCE STANDARDS

Figures 3, and 4 illustrate the workflow of the proposed private multi-abnormality classification framework. For MIMIC-CXR, ChexBERT [19] was used to generate 14 diagnostic labels: Enlarged Cardiomediastinum, Cardiomegaly, Lung Opacity, Lung Lesion, Edema, Consolidation, Pneumonia, Atelectasis, Pneumothorax, Pleural Effusion, Pleural Other, Fracture, Support Devices, and No Finding.

FIGURE 3.

Differentially private fine-tuning updates only a subset of low-rank model parameters on local medical data, while keeping the rest of the pre-trained LLM frozen.

For CT-RATE, the RadBERT [20] model performed binary classification (present/absent) over 18 classes, including Cardiomegaly, Pericardial Effusion, Coronary Artery Wall Calcification, Lymphadenopathy, Lung Nodule, Pulmonary Fibrotic Sequela, and Pleural Effusion. The selected models (ALBERT-base, BERT-small, and BERT-medium) were chosen to represent compact transformer architectures with varying parameter sizes (11M-41.7M parameters). In differentially private optimization, the magnitude of injected noise depends on model dimensionality. Therefore, smaller models offer improved privacy-utility trade-offs and are more suitable for clinical deployment environments with limited computational and memory resources. Our study evaluates this relationship by comparing multiple model capacities under identical privacy budgets.

C. DIFFERENTIAL PRIVACY

Differential privacy (DP) provides a statistical guarantee that the inclusion or exclusion of any single individual’s data does not significantly affect the model output.

Formally, a randomized algorithm $ℳ$ with input domain $𝒳$ and output range $ℛ$ is $(\epsilon ,\delta )$-differentially private if, for all neighboring datasets $D$ and $D^{\prime }$ differing by one record, and for all subsets $S\subseteq ℛ$,

$$\mathbb{P}[ℳ(𝒟)\in S]\le e^{ϵ}\cdot \mathbb{P}[ℳ({𝒟}^{\prime })\in S]+\delta .$$ (1)

Smaller $ϵ$ implies stronger privacy, while $\delta$ allows for a negligible probability of privacy loss.

D. LOW-RANK ADAPTATION (LoRA)

Low-Rank Adaptation (LoRA) [2] is a parameter-efficient fine-tuning method that avoids updating the full weight matrix $W\in {\mathbb{R}}^{d\times k}$. Instead, it introduces a low-rank decomposition:

$$W^{\prime }=W+BA,$$ (2)

where $B\in {\mathbb{R}}^{d\times r}$, $A\in {\mathbb{R}}^{r\times k}$, and $r\ll \text{min}(d,k)$. Only low-rank matrices $A$ and $B$ are updated during fine-tuning, while $W$ remains frozen. This reduces the number of trainable parameters and hence the overall computational overhead significantly reduces.

E. LoRA WITH DIFFERENTIAL PRIVACY

To ensure differential privacy during fine-tuning, we employ Differentially Private Stochastic Gradient Descent (DP-SGD) [30] in the pipe-line of LoRA. In DP-SGD, at each optimization step, the gradient $\nabla \theta$ is clipped to a fixed norm bound $C$ to limit sensitivity:

$$\tilde{g}=\frac{g}{\max (1,\frac{{\Vert g\Vert }_2}{C})}.$$ (3)

Gaussian noise with standard deviation $\sigma =1.25\times (1∕ϵ)$ is then added:

$$g^{\prime }=\tilde{g}+𝒩(0,{\sigma }^2{C}^2\mathbf{I}),$$ (4)

ensuring $(ϵ,\delta )$-differential privacy over the entire fine-tuning process. By updating only low-rank parameters, DP-LoRA minimizes the number of noisy updates while maintaining formal privacy guarantees and strong predictive performance.

F. SELECTION OF PRIVACY PARAMETERS

The selection of $(ϵ,\delta )$ governs the privacy–utility trade-off. Smaller values yield stronger privacy but may reduce model accuracy. In healthcare applications, $\epsilon \in (0,10)$ is typical for robust protection [16]. An appropriate choice for $\delta$ is 1/n², where n denotes the number of training samples, ensuring that the probability of privacy failure decreases with dataset size. We evaluate $ϵ\in \{0.01,0.1,1,10\}$ to represent four privacy regimes: very strong, strong, moderate, and weak privacy respectively. These ranges follow commonly adopted values in medical differential privacy studies. Rather than exhaustive sweeping, this selection provides interpretable operating points for practitioners to balance privacy and performance.

V. TECHNICAL CONTRIBUTIONS

In this study, we propose a privacy-preserving framework for multi-abnormality classification from free-text radiology reports by integrating LLM-based weak supervision, Low-Rank Adaptation (LoRA), and Differentially Private Stochastic Gradient Descent (DP-SGD). Figure 3 highlights the broader overview of DP-LoRA framework. Figure 4 illustrates the overall workflow, which consists of three main stages: (1) label generation using a domain-specific labeler LLM, (2) differentially private fine-tuning of a classifier LLM, and (3) performance evaluation of a held-out test dataset.

FIGURE 4.

Workflow of the proposed DP fine-tuning framework for multi-abnormality classification problem from the “Findings” of free-text chest radiology reports.

A. WEAKLY SUPERVISED LABEL GENERATION

We first utilize specialized labeler LLMs (e.g., CheXbert for MIMIC-CXR dataset and RadBERT for CT-RATE dataset) to automatically extract structured abnormality labels from free-text radiology reports. Each report is mapped to a multi-label vector across clinically relevant findings such as atelectasis, edema, cardiomegaly, and pleural effusion and so on. The raw outputs of ChexBERT are standardized into a unified label representation, where: (+1) indicates disease presence, (0) indicates disease absence or no relevant mention, (−1) represents uncertain findings, (+2) denotes missing or no-data entries. This stage enables scalable, annotation-free label generation while maintaining full compatibility with privacy-preserving downstream training. The labels for MIMIC-CXR dataset were binarized by merging “unsure” (−1) and “no data” (+2) into the negative (0) class, and retaining “present” (+1) as positive. For CT-RATE dataset, the RadBERT generated labels have binary labels, present/absent (0/+ 1).

Labeler Bias Consideration:

The labels are generated using CheXbert and RadBERT, which share domain similarity with the evaluation task. This may introduce inductive bias where the classifier partially learns the labeler’s representation rather than purely clinical semantics. However, three factors mitigate this effect: (1) the classifier models (ALBERT/BERT variants) differ architecturally from the labeling models, (2) evaluation uses held-out patient partitions, and (3) DP noise discourages exact pattern imitation. Therefore, the reported performance reflects robust feature learning rather than simple labeler mimicry, though future work with expert annotations would further strengthen validation.

B. DIFFERENTIALLY PRIVATE FINE-TUNING OF THE CLASSIFIER LLM

The generated multi-label vectors (as shown in Figure 4) serve as supervision signals for fine-tuning a base classifier LLM using differentially private optimization. Rather than updating the full parameter set, we adopt DP-LoRA, which inserts low-rank trainable matrices into the transformer layers and freezes all original model weights. In our proposed workflow since only the LoRA parameters are updated, the algorithm significantly reduces the number of noisy updates, improving model utility under strong privacy constraints.

C. INFERENCE AND PERFORMANCE EVALUATION

During inference, the DP-Classifier LLM predicts abnormality labels for unseen radiology reports using only the differentially private low-rank parameters. The predicted label vectors are compared with reference standards to compute disease-wise performance metrics, the weighted F1-score. As illustrated in Figure 7, this enables fine-grained assessment across all abnormality classes and highlights categories most sensitive to privacy-induced noise.

FIGURE 7.

Privacy-utility tradeoff for ALBERT-base, BERT-small and BERT-medium models across varied privacy regimes. As the privacy budgets increase, due to less stringent privacy constraints, the model utility increases.

VI. RESULTS

In this Section, we present the results on private and non-private fine-tuning of 3 different models on 2 different datasets. Specifically, we present the experimental results on the MIMIC-CXR [11] and CT-RATE [18] datasets with 3 models of different sizes: BERT-medium (41.7M parameters), BERT-small (29.1M parameters) [34], and ALBERT-base-v2 (11M parameters) [15]. For the scope of this paper we have considered the weighted F1-score [14] metric while reporting the predictive performance of the fine-tuned models with and without differential privacy. To evaluate model performance in the presence of class imbalance, we use the weighted F1-score, a metric that computes the F1-score for each class and averages them according to their support (i.e., the number of true instances per class). Since the labels are highly imbalanced, we have used the weighted F1-score that considers both the correctly predicted positive labels (+1) and negative labels (0). To evaluate the stability of DP-LoRA training method, we plot training and validation loss curves across training epochs. As shown in Figure 5, the validation loss closely follows the training loss without divergence, indicating that the injected DP noise acts as a regularizer and prevents memorization. Unlike non-private fine-tuning, which shows early overfitting, DP-LoRA exhibits smoother convergence and improved generalization.

FIGURE 5.

Training and validation loss of BERT-small model with varying $ϵ$ values on MIMIC-CXR dataset. The plots highlight there is no evidence of overfitting.

A. EFFECT OF DP ON CLASSIFICATION PERFORMANCE

In this experimental section, we compare the effect of differentially private fine-tuning in different privacy regimes compared to the non-fine-tuned models and models fine-tuned without any privacy constraints. From Figure 6, and Figure 7 we observe that the performance of ALBERT-base, BERT-small, and BERT-medium models across different training conditions, including off-the-shelf (non-fine-tuned), differential privacy (DP-SGD) with varying epsilon values, and non-private fine-tuning methods. When fine-tuned with DP-SGD, the models show improved performance for both the datasets, as epsilon increases, indicating a clear privacy-utility tradeoff. In contrast, non-private fine-tuning significantly boosts accuracy: on MIMIC-CXR, non-private LoRA achieves 0.901 for BERT models and 0.891 for ALBERT-base, while full fine-tuning results in the highest accuracy across all models, with BERT-medium reaching 0.968. On CT-RATE dataset, the non-private LoRA achieves 0.79 F1-score whereas the non-private full fine-tuning achieves 0.97. Overall, full fine-tuning without privacy provides the best performance, while DP-SGD enables comparable F1-score with strong privacy guarantees. We observe a higher F1-score on the MIMIC-CXR dataset compared to CT-RATE, primarily due to two factors. First, the reference standard labels for MIMIC-CXR were generated using CheXbert, a BERT-based model fine-tuned on the MIMIC dataset, which aligns closely with the domain and structure of the test data. Second, CT reports tend to be longer and more complex than CXR reports, introducing additional linguistic and semantic challenges for automated label extraction.

FIGURE 6.

This table shows the mean and standard deviation of the weighted F1-score (over 10 epochs) of 3 models BERT-small, BERT-medium, and ALBERT-base over different privacy regimes with four different privacy budgets = (0.01, 0.1, 1.0, 10.0). This shows the privacy-utility trade-off in different privacy regimes.

B. EFFECT OF LoRA RANK ON UTILITY

Intuitively, as we increase the LoRA ranks, the predictive performance of the models gets better, achieving significantly better F1-scores. The results in Figure 8 highlight the critical role of LoRA rank (r) in navigating the privacy-utility tradeoff during LLM fine-tuning. Under a fixed total privacy budget, increasing the LoRA rank (r) leads to a proportional increase in the number of trainable parameters. As a result, more noise must be added per training step to satisfy differential privacy constraints, which in turn degrades the model’s predictive performance, most notably reflected in lower F1 scores. This demonstrates that the choice of LoRA rank is not merely a tuning parameter but a key factor in balancing model utility and privacy. Careful calibration of the rank is therefore essential to optimize performance under stringent privacy guarantees.

FIGURE 8.

This table represents the predictive performance of 3 models ALBERT-base, BERT-small, and BERT-medium for a fixed privacy budget of = 1.0, and DP-failure probability of for varied LoRA ranks (i.e., number of fine-tuned parameters).

C. IMPACT OF DIFFERENTIAL PRIVACY ON SEMANTIC MEMORIZATION

The empirical trend shown in Figure 2 provides quantitative evidence that differentially private (DP) fine-tuning mitigates memorization in radiology report generation. Specifically, we measure the cosine-similarity between the ground-truth MIMIC-CXR report and the model-generated completion under varying privacy budgets. As the privacy budget decreases (i.e., stronger privacy guarantees with smaller $ϵ$), the average cosine similarity systematically declines relative to the non-private baseline. The non-private fine-tuned model achieves the highest cosine similarity score, whereas progressively smaller $ϵ$ values (e.g., $ϵ=1.0,0.1,0.01$) yield lower similarity scores, indicating reduced reconstruction fidelity to the original report. This trend is consistent with the theoretical objective of differential privacy, which constrains gradient updates and limits the model’s capacity to memorize trainingspecific content. Importantly, the reduction in similarity reflects controlled suppression of memorization patterns rather than arbitrary performance degradation, thereby providing an empirical privacy evaluation of the trained models. These results directly address concerns regarding the absence of privacy-specific metrics by demonstrating that DP fine-tuning measurably reduces semantic proximity to ground-truth reports while preserving clinically coherent generation, thus achieving improved privacy preservation.

VII. DISCUSSION

LLMs offer a scalable and efficient solution for parsing free-text radiology reports, enabling both structured data extraction and multi-abnormality labeling. These capabilities are essential for building high-quality labeled datasets and improving the efficiency of downstream diagnostic tasks, especially tasks that are otherwise difficult to scale through manual annotation. In this study, we propose a privacy-preserving fine-tuning framework that integrates Differential Privacy (DP) with Low-Rank Adaptation (LoRA) for multi-abnormality classification from free-text radiology reports. By combining parameter-efficient adaptation with formal privacy guarantees, our approach directly addresses the privacy–utility trade-off inherent in deploying LLMs within healthcare environments. Through extensive experimentation on the MIMIC-CXR [11] and CT-RATE datasets, we demonstrate that the proposed DP-LoRA framework achieves competitive classification accuracy while maintaining rigorous differential privacy guarantees. The results clearly illustrate that although stricter privacy budgets reduce predictive performance, moderate privacy regimes yield weighted F1-scores comparable to non-private baselines. These findings confirm that privacy protection and model utility can coexist effectively when fine-tuning large language models on sensitive clinical text data. DP-LoRA is particularly suitable for low-resource environments because only low-rank adapter parameters are trained while the backbone model remains frozen. This reduces memory consumption and GPU requirements compared to full fine-tuning. In addition, differential privacy acts as an implicit regularizer, allowing smaller models to achieve competitive performance. Therefore, the framework can be deployed in hospital settings without large-scale compute infrastructure.

Compared with prior work [14], which focused on fine-tuning LLMs for radiology report analysis without privacy considerations, our method achieves comparable weighted F1-scores while simultaneously providing formal DP guarantees. A key contribution of this study lies in systematically analyzing how privacy budgets ($ϵ,\delta$) and LoRA rank ($r$) jointly influence predictive performance. We observe a clear dependency between the number of trainable parameters and downstream utility. In non-private settings, increasing the LoRA rank consistently improves F1-scores due to enhanced fine-tuning capacity. However, under strict differential privacy constraints, increasing the number of trainable parameters amplifies gradient noise injection, thereby diminishing performance gains. This highlights the importance of carefully calibrating privacy budgets and adaptation rank to achieve an optimal privacy-utility tradeoff.

In practical deployments, the selection of the privacy budget ($ϵ,\delta$) should be guided by principled risk assessment rather than arbitrary thresholds. Prior work has emphasized that $ϵ$ quantifies a bounded increase in individual privacy risk and must be interpreted within contextual, regulatory, and institutional constraints [16]. Empirical analyses of differentially private machine learning further demonstrate that the implications of $ϵ$ depend on dataset sensitivity, population size, and adversarial assumptions [4]. Accordingly, privacy parameter selection in medical applications such as radiology report analysis should be grounded in broader risk-management frameworks rather than fixed numerical targets. Our evaluation across multiple privacy regimes provides empirical reference points that are consistent with established theoretical perspectives.

Importantly, the notion of “strong performance” must be contextualized within clinical practice. In radiology report classification literature, weighted F1-scores between 0.80-0.95 are typically considered clinically usable depending on abnormality prevalence and decision-support role. In our setting, the model serves as a triage and pre-annotation tool rather than an autonomous diagnostic system. Therefore, F1-scores in the 0.85-0.90 range under privacy constraints indicate practical clinical utility while preserving patient confidentiality.

Beyond quantitative performance, our findings underscore the practical viability of deploying lightweight models in real-world healthcare workflows. By differentially privately fine-tuning compact architectures such as BERT-small and ALBERT-base, we maintain competitive predictive performance while satisfying computational efficiency and confidentiality requirements critical for hospital environments. Although promising, this study has several limitations that motivate future research. First, although we demonstrate effectiveness on chest X-ray-based and CT-based free-text radiology reports, generalizability to broader clinical contexts and multi-modal datasets remains to be explored. Extending DP-LoRA to multi-modal learning scenarios that jointly model imaging and textual data represents an important direction. Second, although we motivate differential privacy through memorization risks, a deeper empirical evaluation of adversarial attack resilience and membership inference resistance would further strengthen deployment readiness. Third, our implementation relies on standard DP-SGD with Gaussian noise; more advanced strategies such as adaptive privacy budgeting, alternative noise mechanisms, or privacy amplification techniques may offer improved privacy–utility trade-offs. A promising future direction is combining DP-LoRA with federated learning to enable multi-center collaborative training without data sharing. Such integration would allow institutions to jointly train models while maintaining both local data confidentiality and formal differential privacy guarantees, making the framework suitable for large-scale clinical deployment.

In summary, this work presents a differential privacy-preserving fine-tuning framework for multi-abnormality classification from radiology reports that is computationally efficient, empirically validated, and practically deployable. By integrating LoRA with differential privacy mechanisms, we demonstrate that strong privacy protection and competitive clinical performance can be achieved simultaneously. The framework is extensible to multi-organ, multi-abnormality classification, multi-modal medical data analysis, report generation, and summarization tasks. As future work, we plan to conduct a more comprehensive analysis of privacy vulnerabilities and explore robust privacy-preserving strategies tailored to large-scale, multi-institutional, multimodal medical applications.

VIII. CONCLUSION

This paper presents an adaptation of differentially privacy for fine-tuning of LLMs for multi-abnormality classification from free-text radiology reports using DP-LoRA mechanism. Experiments on MIMIC-CXR and CT-RATE demonstrate that strong privacy guarantees can be achieved with only moderate reduction in classification accuracy compared to the non-private setting. The results confirm that privacy-preserving adaptation of language models is feasible for real clinical deployment. Future work will extend the framework to multi-modal medical data, federated and split learning based frameworks and advanced privacy mechanisms.

Biographies

PAYEL BHATTACHARJEE received the B.Tech. degree in electronics and telecommunication engineering from the Kalinga Institute of Industrial Technology (KIIT), Bhubaneswar, India, in 2021. She is currently pursuing the Ph.D. degree with the Department of Electrical and Computer Engineering, The University of Arizona, Tucson, AZ, USA.

She is also a Graduate Research Assistant with the Department of Electrical and Computer Engineering, The University of Arizona, under the supervision of Dr. Ravi Tandon. From 2021 to 2022, she was an Associate Software Engineer with Bosch Global Software Technologies, India, and a Project Intern with Robert Bosch Engineering and Business Solutions. Her research interests include trustworthy machine learning, alignment and fine-tuning of large language models (LLMs), differential privacy, causal inference and causal graph discovery, and multi-modal data analysis.

FENGWEI TIAN received the B.Sc. degree in electrical engineering and actuarial science from the University of Calgary, Canada, in 2020, and the M.S. degree in electrical engineering from the University of Pennsylvania, USA, in 2022. He is currently pursuing the Ph.D. degree with the Department of Electrical and Computer Engineering, The University of Arizona, USA. His research interests include privacy-preserving machine learning, edge AI efficiency, and large language model safety.

GEOFFREY D. RUBIN received the M.D. and M.B.A. degrees from the Fuqua School of Business, Duke University, where he was recognized as a Fuqua Scholar. He is currently a Professor and the Chair of the Department of Medical Imaging, The University of Arizona, and the Service Chief of medical imaging of Banner University Medicine, Tucson, Arizona. He was previously the George B. Geller Distinguished Professor and the Chair of radiology of Duke University. Prior to joining Duke University, in 2010, he was a Professor of radiology with Stanford University, where he was also the Associate Dean for Clinical Affairs of the Stanford School of Medicine and the Chief of Cardiovascular Imaging. In 2014, he represented his class as their commencement speaker. He is the author of more than 300 published works and principal investigator for more than $12M in research grants from the National Institutes of Health. He recognized as an Honorary Member of Japan Radiological Society. He is a FACR, FAAR, FAHA, FSABI, and FNASCI. He has been awarded the Gold Medal from the Association of Academic Radiologists and the Society for Advanced Body Imaging. He is the past President of the Fleischner Society, the Society for Body Imaging, the North American Society for Cardiovascular Imagers, and the President and the Board Chair of the International Society for Computed Tomography, a Founding Board Member of the Radiology Leadership Institute of the American College of Radiology, a Board Member of RAD-AID International, and host of the acclaimed RLI podcast, “Taking the Lead.”

JOSEPH Y. LO received the Ph.D. degree from Duke University, in 1993. He is currently a Professor of radiology, a Professor with the Department of Electrical and Computer Engineering, a Professor of biomedical engineering, and a member with the Duke Cancer Institute, Duke University. He was a Research Associate in radiology with Duke University, from 1993 to 1995. His research is at the intersection of computer vision, machine learning, and medical imaging, with a dual focus on mammography and computed tomography (CT). Together with the industry partners, his team developed deep learning algorithms for breast cancer screening with 2D/3D mammography, and that product is undergoing FDA approval with anticipated rollout to clinics worldwide. He, along with this team, is also pioneering the creation of “digital twin” anatomical models from patient imaging data, using these models to forge new paths in CT scan analysis through virtual readers and deep learning techniques. Additionally, he is developing a computer-aided triage system for detecting diseases across multiple organs in body CT scans, leveraging hospital-scale datasets and integrating natural language processing with deep learning for comprehensive disease classification.

NIRAV MERCHANT received the bachelor’s degree in industrial engineering from the University of Pune, India, and the degree in systems and industrial engineering from The University of Arizona, in 1994. He is currently the Director of the Data Science Institute. For the past three decades at The University of Arizona, his research has been focused on the development of scalable computational platforms (cyberinfrastructure) in support of open science projects. His work is primarily directed towards reducing the socio-technical barriers in the adoption of emerging computational and information sciences advances by domain sciences. He serves as the principal investigator for NSF CyVerse, a national-scale cyberinfrastructure, and co-principal investigator for NSF Jetstream the first user-friendly, scalable cloud environment for NSF XSEDE/Access. His research interests include large-scale data management platforms, data delivery technologies, cloud native methodologies, secure data analysis enclaves, and the use of managed sensors and wearables for health interventions.

HEIDI A. HANSON received the M.S. and Ph.D. degrees in demography and sociology from The University of Utah. She is currently the Group Leader of the Biostatistics and Biomedical Informatics Group, Computing and Computational Sciences Directorate, Oak Ridge National Laboratory (ORNL), Oak Ridge, TN, USA. Prior to her tenure at ORNL, she was the Co-Director of the Surgical Population Analytic Research Core, Department of Surgery, The University of Utah, and the Assistant Director of Research of Utah Population Database. She leads efforts to integrate large-scale clinical, genomic, and environmental data to improve disease surveillance and understand temporal and intergenerational patterns in cancer and other complex diseases. Her research interests include population-scale biomedical informatics, life-course epidemiology, statistical demography, and the development of machine learning and AI methods for multi-modal health data.

JOHN GOUNLEY received the Ph.D. degree in computational and applied mathematics from Old Dominion University, in 2014. He is currently a Computational Scientist and a Group Lead for Scalable Biomedical Modeling, Computational Sciences and Engineering Division, Oak Ridge National Laboratory. His research interests include algorithms and scalability for biomedical simulations and data.

RAVI TANDON (Senior Member, IEEE) received the B.Tech. degree in electrical engineering from IIT Kanpur, in 2004, and the Ph.D. degree in ECE from the University of Maryland, College Park, in 2010. He is currently a Professor with the Department of ECE, The University of Arizona, where he holds a Craig M. Berge Faculty Fellowship and a courtesy appointment in applied mathematics. He was a Postdoctoral Research Associate with Princeton University, from 2010 to 2012. From 2012 to 2015, he was a Research Assistant Professor with Virginia Tech. He was a recipient of the NSF CAREER Award, in 2017, the 2018 Keysight Early Career Professor Award, and the Best Paper Award at the 2011 IEEE GlobeCom Conference. He has served as an Associate Editor for IEEE Transactions on Information Theory, IEEE Transactions on Wireless Communications, and IEEE Transactions on Communications.