Generalizable multilingual medical text anonymization using generative instruction tuning

Abstract

Background

Medical research depends on access to high quality data that protects patient privacy. Free text in health records contains valuable clinical detail, yet it often includes sensitive personal information that must be removed before use. Current approaches rely on manually created training data and focus mainly on narrow domains. They are difficult to scale to new medical fields and languages. This study aims to address these limitations by developing a framework that supports privacy-preserving use of medical text across diverse settings.

Methods

The study introduces an annotation-free framework for training and adapting LLM-based anonymization models across diverse medical domains. Our reproducible framework includes the development of a generative medical anonymization model, leveraging synthetic data and instruction tuning of generative LLMs. Performance is evaluated on both synthetic test sets and on patient requests from a digital triage service. Accuracy, recall, precision, and the ability to maintain the original meaning of non-sensitive text are assessed.

Results

Here we show that generative models trained with the synthetic framework reach performance that exceeds strong baseline systems across several medical domains. The models preserve non-sensitive text with high fidelity and anonymize sensitive information with high accuracy. They perform well even when trained on small datasets, generalize to unseen clinical fields, and support anonymization in multiple languages without requiring additional training data in those languages.

Conclusions

The study presents a reproducible, annotation-free approach that enables the development of effective anonymization models for medical text. The framework reduces reliance on real patient data, lowers the cost of adaptation to new settings, and supports wider use of unstructured clinical information for research and service improvement.

Plain language summary

Health records are a useful source of information in medical research. However these records often contain details such as names and telephone numbers that need to be removed to ensure anonymity of patients before the text can be used for research. We created a computational method that generates artificial representative examples of health records and anonymized health records. These can be used to train further computational models to improve diagnosis and treatment of disease.

Xiao et al. present a framework for building a generative anonymization model for patient requests without labeled data. It outperforms other methods without being trained on real patient records.

Introduction

Digital transformation in healthcare has profoundly influenced the medical research landscape, primarily through the widespread adoption of Electronic Health Records (EHRs)^1,2. This shift has ushered in a new era of data-driven research, offering opportunities to explore and analyze vast amounts of health-related data. Epidemiological studies have extensively leveraged these records, utilizing structured data sources, such as lab results, diagnostic codes, and medication histories to gain insights into disease patterns, treatment outcomes, and healthcare utilization^3–6.

However, structured data represents only a fraction of the information contained within EHRs. Up to 80% of medical data is estimated to be unstructured, such as free-text clinical notes created by healthcare providers⁷. These clinical notes are indispensable, as they capture the subtleties of patient care, including detailed observations, physician assessments, and the reasoning behind clinical decisions. Such nuanced information is often lost in the rigidity of structured data formats, which can oversimplify complex medical narratives^8,9. The exclusion of unstructured data from research risks omitting critical clinical insights that could influence patient outcomes, thereby limiting the scope of studies^10–13. As healthcare continues to evolve, integrating unstructured data into research frameworks will be crucial for harnessing the full potential of EHRs and advancing medical science.

Despite the recognized value of unstructured clinical data, there is an inherent tension between utilizing this rich information and safeguarding patient privacy. The limited use of free-text notes in research is primarily driven by concerns surrounding the potential re-identification of patients from these data sources^14,15. This apprehension is not unfounded, as the unstructured nature of clinical notes presents significant challenges for effective anonymisation¹⁶. Unlike structured data, which can be more easily de-identified through removing, masking or aggregating specific fields, free-text notes often contain complex narratives that may include indirect identifiers or contextual information that could lead to patient re-identification. This complexity necessitates more sophisticated approaches to anonymisation that can preserve the clinical utility of the data while robustly protecting patient privacy.

Regulatory frameworks, such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States¹⁷, the Data Protection Act in the United Kingdom¹⁸, and the The European Health Data Space (EHDS) expansion of the General Data Protection Regulation (GDPR) in the European Union¹⁹ have been established in response to the growing concerns surrounding data privacy in healthcare. These regulations safeguard patient privacy by imposing strict guidelines for handling sensitive health information and ensuring data protection and accountability within the healthcare industry. The HIPAA Safe Harbor standard provides a specific framework for the anonymization of health records. According to this standard, a dataset derived from health records can be considered de-identified if it contains none of 18 specified direct or indirect identifiers (e.g., zip code, date of service, telephone number). To fully comply with the Safe Harbor standard, data custodians must ensure that no additional information could be used to identify individuals beyond what the law explicitly prohibits. As an alternative, HIPAA also provides the Expert Determination standard, which requires a qualified data expert to confirm a very low risk of re-identification in the de-identified dataset¹⁴. These standards provide a crucial framework for balancing the utility of health data for research and analysis with the imperative of protecting individual privacy.

The task of anonymising medical text has seen significant advancements in recent years, with various methodologies proposed to address anonymization challenges. Learning-based methods have emerged as a prominent approach, enabling the development of scalable pipelines for anonymization using machine learning^20,21 and deep learning techniques^22–26. These methods have shown promise in automating the anonymization process, potentially alleviating the labor-intensive nature of manual anonymisation. However, a significant limitation of these approaches has been their tendency to exhibit unstable performance when faced with domain shifts between training and real-world data^22,24. This instability poses a considerable challenge in healthcare settings, where data heterogeneity is common across different institutions, specialties, and geographical regions. While deep learning-based methods have been proposed to enhance anonymization capabilities on cross-institute datasets, several factors have constrained their effectiveness. Notably, these methods are often trained and tested on relatively small datasets, which may not adequately represent the diversity of real-world EHRs. Consequently, significant performance degradation has been observed when these models are applied to out-of-distribution data^22,23,26,27.

Among the various forms of unstructured clinical data, free-text patient requests stand out as both highly valuable and uniquely difficult to anonymize. These requests, which may be submitted through online triage systems or healthcare portals, often contain informal language, typos, and diverse expressions of symptoms and needs. They also include dense personal information, such as names, addresses, contact details, and references to family members, all embedded within natural, unstructured sentences. Despite their clinical utility and potential for large-scale analysis, this text format has received limited attention in previous anonymization efforts. In this work, we focus particularly on this setting, demonstrating that our generative framework performs robustly on real-world patient request data while maintaining generalizability to other domains.

We present an annotation-free framework for training and adapting large language model (LLM) based anonymization models across diverse medical domains. To the best of our knowledge, we propose the first generative medical anonymization model which leverages synthetic data and instruction tuning. This framework achieves state-of-the-art performance on real-world data, demonstrating strong zero-shot generalization and cross-lingual capabilities. Importantly, it enables rapid adaptation to new domains and languages with minimal labeled data, addressing the scalability challenges inherent in traditional anonymization methods. By overcoming the limitations of current approaches while maintaining data utility, our method offers a promising solution to the complex challenge of anonymising free-text clinical notes. This advancement will help unlock the vast reservoir of unstructured medical data for research, contributing to improved patient care and accelerated medical discoveries.

Methods

Our approach consists of two parts: first, we create a synthetic training dataset, and second, we use this dataset to train an anonymization model (Fig. 1).

Fig. 1. Overview of our pipeline.

Prompts are processed by a teacher model to generate synthetic data used for LLM training, producing a trained anonymization model for inference. Inference errors trigger feedback identifying incorrect data or model behavior, which is used to correct the data and update prompts, closing the loop and improving subsequent training cycles.

Synthetic training set generation

Methodology

Training an anonymization model requires a dataset comprising of (raw text, anonymized text) pairs. However, sensitive personal information contained within raw texts significantly limits the availability of public datasets. Even when available, we opt for training the models without proprietary user data due to two reasons: first, LLMs are found to have the risk of information leakage²⁸, wherein generative models may output portions of the training sets verbatim in a ‘hallucination’ process. Therefore, there exists a high risk in exposing raw un-anonymized data, to the training of generative models, even only as inputs. Second, with the high cost of human annotations, leveraging state-of-the-art LLMs, such as GPT-4 to annotate data is a well-studied solution. However, using closed-sourced API models to annotate raw user/patient texts is again not feasible in this case, with the risk of exposing proprietary data to third-party companies.

To overcome these challenges, we propose building the training sets by generating both the raw texts and their anonymized counterparts with state-of-the-art API models. Inspired by previous work on LLM self-instruct²⁹, we first generate a few seed examples through prompting the LLM to brainstorm large categories of patient requests, e.g., repeated prescription; medical health consulting, etc. Samples of the seed examples are shown below in seed_text. We then task the LLM with expanding each of the seed examples, which represents a medical request category, to more diverse examples. The prompt messages in Box 1 demonstrates one specific prompt that utilizes the index 5^th seed text to generate 20 similar texts that fall under the same category. We find that generating a number of examples at once has the inherent advantage of encouraging diverse generation, as the LLM tends to avoid re-generating similar texts given access to what it has generated in previous tokens. We iteratively prompt the LLMs to repeat the process of diversifying the training set based on the seed examples, while simultaneously identifying issues and modifying the prompts.

We focus on the following personally identifiable information (PII): names, telephone numbers, dates of birth, practice/hospital/pharmacy details, NHS numbers, addresses, emails, and social media handles.

The framework enables the generation of high-quality synthetic data that mimics the distribution of real-world user data, while keeping the diversity. In the first round of prompting, there were a few details to note. These issues have been addressed by subsequent rounds of prompting.

1. Due to the dominance of US-based texts in the representation of the English language on the internet, language models are highly biased to US information in generation. Therefore, without specific conditional prompts, language models are biased towards generating US template information, including locations, phone numbers templates, and US-specific information, such as SSNs (Social Security Numbers). This raises the question of the generalizability of anonymization models trained on such synthetic datasets, i.e., will language models trained to anonymize US information be able to generalize to anonymize information in other templates, such as those from the UK?

2. Moreover, with the highly-aligned nature of state-of-the-art LLMs, they mostly generate specific categories of information in fixed formats. For example, we note that dates are always generated in the MM/DD/YYYY format unless instructions to diversify formats are provided, e.g., formats like Jan 01, 2000 or 01 Jan 2000. Again, such issues pose concerns to the generalizability of downstream models trained on the data.

3. Without specific instructions, language models generate texts in formal English, without typos. However, users of various socio-demographic backgrounds verbalize their symptoms and needs with diverse expressions, and may be more likely to write text with typos and misspellings, especially medications and symptoms. With the tokenization of language models, there is a risk that models trained on perfect English are unable to generalize to texts with typos. For example, the word extraordinary can be recognized by LLMs as one standalone token, while extrzaordinary can be tokenized into five. This becomes a more severe concern for terminologies in medical domains, which general-domain language models are not proficient in (even when processing perfect English).

Prompt implementation

The initial prompt template used to generate the first few hundred synthetic raw texts is shown in Box 1.

Starting from these, we iteratively identify issues and enhance prompts as described in the previous sub-section (Box 2). In the follow-up raw dataset generation rounds, we present GPT-4o with randomly sampled examples generated with the initial prompts, and describe more desiderata to it to generate more examples, such as (1) more diverse templates, (2) more diverse orders of information (e.g., not to always start by introducing names and locations), (3) more UK-based information, (4) categories of information that aren’t covered through initial random generations (e.g., NHS numbers in UK), (5) different English speaking styles and intended injection of typos.

To accommodate specific anonymization needs, such as internal needs for companies, more follow-up prompts can be formed to generate texts containing information of missing categories (Box 3). For example, practice names were identified to not be commonly generated by the model.

After we have generated the raw datasets, we prompt the teacher model (GPT-4o) again to anonymize the data and replace sensitive information with corresponding placeholders. Box 4 for example, presents a raw-anonymized pair.

Box 1 Initial prompts based on seed examples. We again highlight that all examples are synthetic data without any real information.

seed_text = [

"I’m Drew Harris, and I have been managing Type 2 Diabetes for the past 5 years. I’ve recently moved to a new city, address is 654 Maple Rd, Austin, TX. My previous GP, Dr. Alex Kim, had me on a specific diet and medication plan that worked well. I’d like to discuss this with you. My contact is (555) 321-0987.",

....,

....,

"Hi, my name is Sage Garcia. I was prescribed Amoxicillin last week for a bacterial infection, and I’ve started noticing a rash developing on my arms and legs. I’m concerned about an allergic reaction. My address is 987 Elm St, Seattle, WA, and my date of birth is 11/11/1986. What should I do?",

....,]

messages=[

{"role": “system", “content": “You are a data quality expert who is working on training a large language model to anonymize medical text."},

{"role": “user", “content": f"We are now creating a training set to train an anonymisation model. Since we are not able to share patient data. We need to generate raw text - anonymised data pair. Please make sure the generated data is diverse in structure, information, and expressions. We are first generating the raw text. Consider text that users would say on a AskMyGP app to communicate to their GPs. Let’s start with this: seed_text[5] Now use this sentence as a seed to generate 20 similar texts generated by other patients. Please make sure the expression, structure, and information are diverse enough. Please only return the answer and do not need to say other things. Please return them in json format"}

]

Box 2 Example follow-up prompts to enhance initial generations.

messages=[

{"role": “system", “content": “You are a data quality expert who is working on training a large language model to anonymize medical text."},

{"role": “user", “content": f"We are now creating a training set to train an anonymisation model. Since we are not able to share patient data, we need to generate raw text - anonymised data pairs. We are first working on generating raw text. You have already generated the first few hundred raw text. But there’s a few problems: 1. The data you generated were highly templated. For example, they always start with name, followed by description of the situation, then followed by other information in similar orders. This could lead to the trained model overfit to some specific formats. Note that users can talk in random format and provide the information in different orders. For example, their might be just one sentence to describe the situation. Or they can first provide the reference, then situation. The ones you generated were like: sampled_text Now please take into consideration my feedback and generate 20 more texts. Again, please make sure the expression, structure, and information are diverse enough. Please only return the answer and do not need to say other things. Please return them in json format"}

]

Box 3 Example follow-up prompts to iteratively identify and generate texts containing examples of missing categories.

sampled_text = random.sample(generated_text[800:1000],5)

messages=[

{"role": “system", “content": “You are a data quality expert who is working on training a large language model to anonymize medical text."},

{"role": “user", “content": f"We are now creating a training set to train an anonymisation model. Since we are not able to share patient data, we need to generate raw text - anonymised data pairs. We are first working on generating raw text. You have already generated the first few hundred raw text. But there’s a few problems: We need to train the model to anonymize a set of information, therefore we need the training set to contain them. Now for the first couple hundreds of data you generated, there isn’t a lot of 1. online identifiers (such as social media usernames and ids)

2. workplace/company/educational institution name 3. company address, workplace address. 4. Practice names. 5. And there can be more local patient identifiers and unique booking reference numbers. The ones you generated were like: sampled_text Now please take into consideration my feedback and generate 20 more texts. Again, please make sure the expression, structure, and information are diverse enough. Please only return the answer and do not need to say other things. Please return them in json format"}

]

Box 4 Example follow-up prompts to iteratively identify and generate texts containing examples of missing categories.

"I’m Drew Harris, and I have been managing Type 2 Diabetes for the past 5 years. I’ve recently moved to a new city, address is 654 Maple Rd, Austin, TX. My previous GP, Dr. Alex Kim, had me on a specific diet and medication plan that worked well. I’d like to discuss this with you. My contact is (555) 321-0987".

"I’m [name], and I have been managing Type 2 Diabetes for the past 5 years. I’ve recently moved to a new city, address is [home address]. My previous GP, [doctor name], had me on a specific diet and medication plan that worked well. I’d like to discuss this with you. My contact is [phone number]".

Synthetic dataset cleaning

We find that using the state-of-the-art model GPT-4o to generate the synthetic dataset can achieve excellent diversity for the raw texts, achieving a selfBLEU³⁰ score (0.44) comparable to that of the real world dataset described in Section 4.1 (0.46). Additionally, we see a wider variety of date and ID number formats than in the real patient data, as well as being of similar medical plausibility as assessed on a Likert scale (4.54) when compared to the real world dataset (4.55)(The medical plausibility was assessed on a likert scale by two UK-based medical professionals with a Krippendorff’s alpha of 0.81). The model achieves high performance in anonymization, with an accuracy of over 99% (proved in our cleaning process shown as follows).

With the high cost of manual checking and the upper-bound in human accuracy, we design an automatic dataset cleaning process, by iteratively training on the training split and identifying patterns of error cases in the test split. Motivated by the scaling law between the number of training examples and the performance, an intuitive assumption gives that when the model’s performance stops improving, there is a chance that errors exist in the test set, incorrectly evaluating the abilities of the trained models.

With a dataset consisting of 2100 examples, we hold out 200 instances as the test split, and train the model by scaling up the training data gradually from 50, 100, 200, 500, 1000, to 1900 instances, while evaluating performance and identifying errors using the same hold-out test subset. We find that training with only 50 examples from the uncleaned dataset can already enable the model to reach an accuracy of 0.937 on the test set. Similarly, 0.976 and 0.984 are achieved with 100 and 200 examples, where we start seeing a plateau by further increasing training examples. By looking at the 1.6% error cases we find that, out of the seven failed fields, four of them are incorrectly labeled by GPT-4o in the creation of the dataset. We manually correct them in the test split, and go through the training split where the same data fields occur (e.g., in our initial defined rules, date of birth should be anonymized whereas date should not), and find that we only have to correct 13 records in the training set. For cases where the model is incorrect on the training set, we include additional prompts and seeds to improve the model’s performance on these examples.

The creation of the training set costs less than 10 USD and minimal manual annotation, demonstrating strong potential in adapting to any application fields with different requirements of information categories to be anonymized.

Implementation details

We employ Qwen1.5-Instruct as the backbone, and experiment with three model sizes with 0.5, 1.8 and 7B parameters to understand the scaling laws between base model capabilities and performance after fine-tuning. Qwen1.5 is a family of LLMs that are pretrained on massive multilingual corpus in the autoregressive fashion^31,32. The models are then fine-tuned with instruction tuning through supervised fine tuning (SFT) and reinforcement learning to align with human preference. The Instruct version of the model already has general instruction-following abilities and thus serves as a better starting point to further finetune for anonymisation.

We finetune the models with a batch size of 50 through gradient accumulation, a learning rate of 2e-4, over ten epochs. We employ low-rank adaptation (LORA) fine-tuning³³, which is not only parameter efficient, but also improves the stability in fine-tuning, preserving the general knowledge of LLMs³⁴. We use a rank of 16 and an alpha of 32, following the rule-of-thumb that alpha should be twice as large as rank for LORA finetuning.

Evaluation

Our approach is evaluated on both the generated synthetic data and a real-world dataset of patient requests using a range of metrics.

Data

As well as evaluating our model using the synthetic data, we additionally evaluate it on a private real-world dataset of anonymized patient requests submitted via a digital triage tool used by over 200 GP surgeries in the UK. This tool is part of Evergreen Life’s online consultation platform which allows patients to contact their GP practice by entering free-text into an online form. The dataset consists of 200 user-generated requests containing rich personal and medical information with personal information labeled. The patient requests have a mean length of 13.89 words, with the majority (66.5%) being a single sentence. Figure 2 shows how the frequency of PII compares to the synthetic dataset used to train our model.

Fig. 2. Comparison of PII prevalence.

Distribution of personally identifiable information (PII) categories are shown for both a synthetic and b real-world patient request datasets. In both datasets, name and practice are the most common PII types. The synthetic dataset also includes more examples of social media handles and email addresses.

Ground truth anonymised texts for evaluation were produced by trained anonymisation experts. For each patient request, annotators removed or replaced all occurrences of personally identifiable information, including patient names, clinician names, contact details, addresses, NHS numbers, dates of birth, practice, hospital, and pharmacy names, email addresses, and social media identifiers. Replacements were made using standardized placeholders (e.g., [name], [tel], [dob], [address]) while preserving all non-sensitive text verbatim. Titles and honorifics were retained, and multi-token entities were replaced with a single placeholder to avoid over-segmentation. Other dates not corresponding to dates of birth were intentionally preserved. The resulting anonymised texts serve as document-level ground truth for evaluation.

Annotation was performed by domain experts with prior experience in medical text anonymisation and clinical data governance. All annotators were UK-based and familiar with NHS documentation practices, including common formats for NHS numbers and practice names. Given the limited size of the evaluation set and the constrained nature of the task, there were no disagreements between annotators which needed to be resolved.

Metrics

We evaluate anonymization performance at both the document level and the item level to capture complementary aspects of model behavior.

At the item level, each instance of personally identifiable information (PII) is treated as an individual target. An item is considered correctly anonymized if it is fully replaced by the appropriate placeholder in the model output. Based on these annotations, we compute precision, recall, and F1 score in the standard way, where precision measures the proportion of anonymized items that correspond to true PII, and recall measures the proportion of true PII instances that are successfully anonymized. For example, anonymizing 23 out of 25 pieces of sensitive information across five documents corresponds to an item-level recall of 0.96. Item-level metrics are particularly informative for assessing performance on sparse or rare PII categories.

At the document level, evaluation reflects whether an entire text is safe for downstream use. A document is counted as correct only if all PII instances within the document are anonymized and no non-sensitive text is altered. Document-level recall is defined as the proportion of documents for which every PII instance is correctly anonymized. Document-level accuracy further requires that the output exactly preserves all non-PII content, making it a stricter metric that captures both privacy protection and text fidelity.

Ethics

The use of the Evergreen Life patient requests data was a retrospective analysis of data collected as part of routine clinical practice for the purpose of improving the provision of therapeutic medicine and related research. Evergreen Life’s IRB waived the need for obtaining consent for this study, as the use is part of its usual business practice and services as a healthcare provider. This aligns with NHS Health Research Authority guidance and GDPR regulations in the UK.

Statistics and reproducibility

The performance of the model was assessed using the commonly used metrics: accuracy, precision, recall, and F1 score. Due to model sizes and training times, parameter values were chosen based on previous work on LORA finetuning.

Results

We first highlight the main results, demonstrating that state-of-the-art anonymization performance is achieved by fine-tuning the backbone model using our framework. This approach surpasses the baseline Stanford De-Identifier model³⁵. Notably, as few as 50 raw-anonymized pairs are sufficient for the 1.8B and 7B models to achieve over 97% accuracy. This result supports the “less is more for alignment” hypothesis in SFT for LLMs³⁶. Furthermore, experiments across various base model and dataset sizes confirm the scaling law for training anonymization models: larger base models with greater capabilities and more diverse datasets lead to improved performance.

Figure 3 illustrates the relationship between the number of training examples, model size, and anonymization performance. To establish a baseline, we evaluate the Stanford deidentifier on our test set, measuring performance through two metrics: document-level recall and item-level recall.

Fig. 3. Impact of scaling on performance.

Scaling laws of model sizes, training data sizes, and test performance (measured by document-level recall, or global recall) show that performance improves with model and dataset size. The Stanford de-identifier model is used as a baseline for performance.

Our results indicate that performance generally improves with larger training sets and model sizes. The effectiveness of generative models is particularly evident: even with only 100 examples, the smallest model variant (Qwen 0.5B) achieves a document-level recall of 0.9416, surpassing the strong open-source baseline. Using the full training set, the document-level recall reaches 0.9973, with a item-level recall of 0.9986. These findings underscore the potential of leveraging generative models for anonymization tasks.

Evaluation on real-world patient requests

To validate our model in a practical healthcare setting, we evaluate it on our private real-world dataset of anonymized patient requests.

Despite being trained only on synthetic patient request data, our model achieves 90.5% accuracy on this dataset, surpassing traditional baselines and demonstrating strong generalization from synthetic to real-world language (Table 1).

Table 1.

Comparison of document-level model performance on real patient data

Model	Accuracy	Precision	Recall	F1
Stanford-Deid	78.0	82.0	93.0	87.2
Deepseek Zero-shot	61.5	66.0	93.0	77.2
Qwen Zero-shot	69.0	83.0	81.5	82.2
Ours	90.5	92.5	98.0	95.2

Bold values indicate the best results for each metric.

This result underscores the framework’s ability to handle the informal, varied language and contextual PHI common in patient-generated messages.

While accuracy highlights the overall correctness of document-level predictions, it can obscure performance differences in settings with substantial class imbalance, as is typical in real patient-generated data where PHI occurrences are relatively sparse. For this reason, we additionally report precision, recall, and F1 score. As shown in Table 1, the proposed model achieves the highest F1 score on real data, though the margin over the Stanford-Deid baseline is smaller than the corresponding gain in accuracy. This reflects the difficulty of improving recall for rare PHI instances without sacrificing precision. Notably, our model improves both precision and recall relative to the baseline, despite being trained exclusively on synthetic data, indicating meaningful generalization to real-world language. Accordingly, the primary contribution of our approach lies in its robustness and generalization under synthetic-only training, rather than a large absolute improvement in F1 score alone.

Table 2 breaks down the results by PII type. The most frequent error observed in the model’s output was the failure to fully anonymize practice names, likely due to the base model’s training, which predominantly relied on US-based data and limited its exposure to UK-specific entities. This discrepancy highlights the need for future improvements, particularly through the inclusion of real UK practice names.

Table 2.

Item-Level Per-Type F1 scores on the real-world patient dataset

PII Type	F1
Name	94.11
Phone number	96.77
Date of birth	85.71
Pharmacy	100.00
NHS number	100.00
Address	100.00
Practice	66.67

Multilingual generalization

We find good zero-shot multilingual generalization of the model brought by only training on English. We hypothesize that sensitive information across languages are aligned in the semantic space such that when the model is optimized on English, other languages are indirectly optimized. Given this starting point, we further prove the language adaptability of our framework by explicitly training the model on a synthetic training set of other languages generated with the same method. We show that, with English being the ‘anchor’ language, the model quickly attains generalizability on other languages with training data on a much smaller scale.

Zero-shot generalization

A surprising pattern we first find is the inherent multilingual generalizability of the model even when it is only trained on English. Using our English-only model, we find that it is able to identify 40 to 60% sensitive information when evaluated on synthetic data depending on the specific language (Arabic, French, German). However, the placeholders that the model generates switch between English or the corresponding language. Below are two of these zero-shot code-switching examples on the Chinese test set, where the English-only model is able to identify and anonymize most of the sensitive information, although without achieving perfect accuracy and with the tendency of occasionally replacing information with English placeholders.

医生您好, 我是[name], 来自[city], 之前住在[district], 今年27岁, 医保号 是[patient id], 我的学生号是10001, 在[company name]就读, 请问可以网上挂号 吗?

昨天我在南京市第一医院看病, 患者号是[患者号], 医生您能帮我查询一下我的检 查结果吗? 万分感谢! 我叫[名字]。

Motivated by this zero-shot generalizability, it is hypothesized that tokens of similar semantics are well-aligned in the semantic space, such that out-of-distribution languages are optimized without being directly trained on. We have the reason to believe, the model is able to achieve a similar level of accuracy as English, by directly training the base model with the target language, or pairing the English training set with OOD language training set of a much smaller scale.

Multilingual enhancement

We specifically choose the Chinese language to study in-depth as some of the authors of this manuscript are able to create high-quality prompts for generating training sets of similar quality as the English training set.

Following the same synthetic training set generation pipeline, we generated 400 Chinese training examples, which we found sufficient to adapt the model to the language. We also find that English-centric LLMs have strong tendency for generating texts in a new language with heavy ‘Translationese’ when the prompts are not designed carefully. For examples, the generated entities are usually transliteration of the English entities, such as “迈阿密, 佛罗里达” which is “Miami, Florida” transliterated to Chinese; or “奥利维亚”, the transliteration of “Olivia” (the most generated name in the English training set, interestingly), which are oftentimes unnatural in the target language. The same applies to all information categories that do not transfer across countries (i.e., except ages and dates). Such examples are not reflective of texts that would be created in real-world patient-doctor interactions in non-English speaking countries. However, prompting the LLMs in the target language, and specifying failure modes to avoid, significantly improves the generation quality (e.g., explicitly ask the LLMs to avoid generating transliterated information, but in turn generate authentic texts that native speakers of the target language would produce). We successfully generate genuinely unique names of Chinese persons, and very diverse hospital names located across provinces in the country. Thus, we highly recommend future practitioners to include native speakers or linguists in the training set generating process when adapting our framework to other languages, especially those low-resource.

Having generated the data, the same supervised fine-tuning can be performed on the target language. We hold out 200 examples for evaluation, and Table 3 concludes the results. In line with English, using only 50 examples can get the model to an accuracy of over 96%, surpassing the anonymization standard for clinical usage. Using 200 examples yields the best document-level recall, making 98% of examples free from sensitive information. In the bilingual setting, we pair the original 2100 English training examples and the 200 Chinese examples. As shown, the Chinese capability is not sacrificed even when the majority of the training examples are given to English. Instead, the item-level recall even improves, demonstrating the unified knowledge representation space about anonymization shared across languages.

Table 3.

Multilingual Transfer Results on the Chinese test set

	c50	c100	c200	Bilingual
Item-level recall	97.05	97.84	97.73	97.82
Document-level recall	96.34	97.80	97.99	97.99

Bold values indicate the best results for each metric.

While our generative framework achieves high anonymization performance, there are inherent limitations related to handling cases outside the training distribution and perfect reconstruction of non-sensitive text. Unlike named entity recognition (NER) models, which predict token positions for sensitive entities, generative models must simultaneously detect and replace sensitive information while reproducing all remaining tokens exactly.

We have evaluated reconstruction fidelity on out-of-distribution medical-domain datasets, PubmedQA³⁷ and NFCorpus³⁸. Overall, 99.94% of characters in NFCorpus texts were reproduced exactly, demonstrating effective reconstruction ability (see the Supplementary Materials for full methodology and detailed results).

Nonetheless, reconstruction errors may still occur in other domains or text styles not represented in training. Rare medical subdomains, unusual linguistic constructions, or languages for which limited synthetic examples are generated may challenge the model’s ability to perfectly preserve non-sensitive text. In addition, generative models may occasionally hallucinate or omit tokens, particularly in long or complex inputs, which could affect downstream utility if undetected. Future work could focus on targeted synthetic data augmentation for such hard-to-generalize cases and on incorporating explicit reconstruction supervision or verification mechanisms to further mitigate these errors.

Conclusion

This work introduces an annotation-free framework for training generalizable multilingual medical anonymization models using generative instruction tuning. By leveraging synthetic data generation with Large Language Models, we developed a reproducible approach surpassing traditional methods which removes any risk of data leakage. Our generative model achieves state-of-the-art performance on real-world datasets, demonstrating strong zero-shot generalization, robust text reconstruction, and remarkable cross-lingual capabilities, both zero-shot and with efficient adaptation using minimal target language data. This framework significantly reduces reliance on manual annotation, which will enable wider access to de-identified medical data for research what might lead to advancements in patient care.

Author contributions

All authors contributed to the design and conceptualization of the study and to the interpretation of the results. C.X. and G.T.H. contributed to the experiments, evaluations, and writing. C.X., G.T.H., B.J., M.W., S.F., J.H.K., and N.A.B. reviewed and approved the final manuscript.

Peer review

Peer review information

Communications Medicine thanks Julia Ive and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Data availability

The patient request dataset used in this study is not openly available due to reasons of patient privacy. More information about this dataset is available by contacting the corresponding author. The patient data used to evaluate the anonymisation technique presented in this study necessarily contains confidential patient information, including private medical details. Therefore, due to data privacy and ethics policies, the data used in this study is not publicly available. PubmedQA (https://pubmedqa.github.io/) and NFCorpus (https://www.cl.uni-heidelberg.de/statnlpgroup/nfcorpus/) are publicly available. The source data for Fig. 2 is in Supplementary Data 1, and the source data for Fig. 3 is in Supplementary Data 2.

Code availability

Code for replicating our work is available at https://github.com/ghomasHudson/medical_deid_via_git³⁹.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s43856-026-01682-8.