Applying clinical natural language processing to lung cancer in Spain: a terminology-based panel approach

Abstract

Background

We present a novel methodological framework for developing and validating a terminology-based Disease Panel to identify and characterize patients with metastatic non-small-cell lung cancer (mNSCLC) in a Spanish cohort by applying clinical natural language processing (cNLP) to electronic health records (EHRs).

Materials and methods

The mNSCLC Disease Panel was built from standardized vocabularies (Systematized Nomenclature of Medicine–Clinical Terms and Anatomical Therapeutic Chemical) enriched with curated alternative expressions such as synonyms, acronyms, and abbreviations. Terms were organized by clinical relevance and refined through clinical expert annotation. Using EHRead® (Medsavana S.L., Madrid, Spain), a cNLP pipeline, clinical concepts from the Disease Panel were extracted from Spanish EHRs. Performance was validated by clinical experts through the assessment of precision, recall, and F1 scores.

Results

The Disease Panel comprised 268 terms (mean of alternative expressions 8.4, range 1-41), including 125 (46.6%) demographic and clinical characteristics, 115 (42.9%) treatments, and 28 (10.5%) outcomes. The mean ± standard deviation (SD) precision across all terms was 0.97 ± 0.04, with values of 0.98 ± 0.04 for characteristics, 0.97 ± 0.04 for treatments, and 0.96 ± 0.06 for outcomes. For key terms, the mean ± SD precision, recall, and F1 score were 0.94 ± 0.05, 0.91 ± 0.08, and 0.92 ± 0.05, respectively.

Conclusions

This is the first validated terminology-based Disease Panel specifically designed for mNSCLC and integrated into a cNLP pipeline. It reliably identifies key clinical features with excellent extraction performance, supporting scalable real-world evidence generation. This approach offers a robust alternative to manual chart review or ‘International Classification of Diseases’ data/claims data in oncology research.

Graphical abstract

Highlights

• •A validated terminology-based Disease Panel for mNSCLC was developed.
• •The Disease Panel includes 268 terms (125 clinical, 115 treatments, and 28 outcomes).
• •Expert refinement and validation improve NLP consistency and reproducibility.
• •cNLP achieved 0.97 precision (full terms) and 0.92 F1 score (key terms) detection.
• •It enables accurate, interpretable extraction of key data from unstructured EHRs.

Introduction

Electronic health records (EHRs) in oncology are clinically complex and multidisciplinary, especially within free-text clinical narratives that capture critical details such as tumoral stage, pathological diagnosis, biomarkers, treatment response, and disease progression. Lung cancer exemplifies this complexity as one of the most common malignancies and the leading cause of cancer deaths,^1,2 with non-small-cell lung cancer (NSCLC) accounting for 80%-85% of cases.³ Owing to its clinical heterogeneity and unmet therapeutic needs, NSCLC generates especially valuable EHR data. Only 30% of patients are diagnosed at the early stage of disease, when prognosis is better, whereas most present at advanced stages, when 5-year survival falls below 10%.^4,5 Despite therapeutic advances in metastatic disease (mNSCLC) and the establishment of standard diagnostic methods,6, 7, 8 important evidence gaps remain, with limited treatment options beyond first-line therapy and heterogeneity in real-world practice regarding treatment selection and outcomes. These gaps highlight the need for high-quality real-world data (RWD) to guide treatment decisions and improve early diagnosis in mNSCLC.

Relevant clinical information is buried in unstructured free text within EHRs, making its secondary use challenging. Traditional approaches—manual chart review (which is resource-intensive, time-consuming, and lacks reproducibility) or reliance on ‘International Classification of Diseases’ codes and claims data, which omit essential clinical detail—fail to fully capture oncology care complexity.^9,10 Clinical natural language processing (cNLP) enables large-scale extraction and structuring of free-text clinical data, facilitating the creation of complex and clinically meaningful cohorts.11, 12, 13 Recent advances allow rapid and automated EHR interpretation, overcoming previous limitations in data access. This approach supports a more accurate and deep understanding of routine disease management and provides new opportunities for early diagnosis, improved patient stratification, and clinical decision support.

To leverage cNLP and artificial intelligence (AI) in clinical information extraction, robust cNLP pipelines require curated terminological foundations, including standardized vocabularies and controlled medical terminologies, to ensure consistent identification, classification, and generation of clinical variables. However, standard resources such as SNOMED CT (Systematized Nomenclature of Medicine–Clinical Terms)¹⁴ or ATC (Anatomical Therapeutic Chemical) fail to account for the linguistic variability inherent to routine clinical documentation, especially for complex diseases such as oncology. They often lack the granularity needed to represent entities such as tumor staging, histological subtypes, molecular biomarkers, combinatorial drugs or regimens, descriptions of adverse events, or evaluation of treatment response. Consequently, expert-driven refinement and oncology-specific extensions are essential to capture clinically relevant concepts that extend beyond standardized clinical terminologies. Robust methodologies enabling standardized and structured identification of relevant oncology concepts across diverse data sources are urgently needed to support deep phenotyping and meaningful real-world evidence (RWE) generation.

This work uses mNSCLC as a representative use case to implement and evaluate a novel methodological framework for developing and utilizing validating a terminology-based Disease Panel to identify and characterize patients with mNSCLC in a Spanish health care setting through the application of cNLP to EHRs. Specifically, the project focuses on assessing the feasibility of constructing and refining a disease-specific, terminology-based Panel and evaluating cNLP performance in extracting key clinical concepts from unstructured EHR data. While mNSCLC serves as the initial use case, the methodology is designed as a foundational step toward determining the viability of conducting future RWD studies in this pathology, with the potential to be scalable to other tumor types and adaptable to different health care settings.

Materials and methods

Data sources and preprocessing

Deidentified EHRs from Spanish hospitals were accessed via a secure platform enabling secondary use through cNLP methods. Uploaded documents were standardized and quality-checked. Structured queries were applied to retrieve EHRs relevant to the study objectives, and only free-text clinical records were analyzed. Records dated between 1 January 2015 and 30 November 2024 were included if they corresponded to adult patients from Oncology departments with at least one mention of (‘NSCLC’ OR ‘lung cancer’) AND ‘metastasis’.

Data processing and analysis were conducted in compliance with data protection regulations. As secondary use was previously authorized, no patient consent or additional ethical approval was required. Further details on data sources and preprocessing are provided in Section 1.1 of Supplementary Materials, available at https://doi.org/10.1016/j.esmorw.2026.100716.

cNLP pipeline (EHRead®)

We used the Named Entity Recognition and Named Entity Linking (NER-NEL) module of the EHRead® technology (Medsavana S.L., Madrid, Spain), which detects mentions of clinical terms in free text and links them to standardized terminologies. Each detected expression (including synonyms, acronyms, and abbreviations) is mapped to a unique identifier within these terminologies, ensuring a consistent representation of clinical concepts and enabling recognition of heterogeneous expressions of the same concept across clinical narratives. The NER-NEL module was trained on large clinical corpora from Spanish hospitals. Terminology development and any required extensions were completed before evaluation. The cNLP evaluation was conducted exclusively on held-out gold standard annotations not used during model development, ensuring clear separation between model development and model evaluation and preventing any circularity or information leakage.

To tailor the NER-NEL module to mNSCLC, we developed a Disease Panel including relevant clinical concepts for this primary tumor. This Disease Panel was integrated into the NER-NEL module, enabling the system to detect and map mNSCLC-related terms and their variants within clinical narratives. The following sections describe Disease Panel construction and validation for terminology-based RWD extraction (Figure 1).

Figure 1.

Terminology-based Disease Panel design.

cNLP, clinical natural language processing; EHR, electronic health record; mNSCLC, metatstatic non-small-cell lung cancer; NER-NEL, Named Entity Recognition - Named Entity Linking.

Conception of the initial list of the disease panel

The initial Disease Panel was created through two complementary approaches. Firstly, multidisciplinary working groups, including medical researchers and cNLP experts, reviewed the literature to identify variables commonly used in mNSCLC studies and clinical trials (e.g. demographic and clinical characteristics, treatment patterns, and outcomes). Clinical terms used in the construction of these variables were mapped to standardized medical terminologies and included in the Disease Panel based on their scientific and clinical relevance, regardless of their frequency of documentation in EHRs. Consequently, clinically meaningful terms were incorporated even if rarely recorded in routine clinical practice. Secondly, a frequency analysis of alternative expressions detected by the cNLP processing pipeline was carried out. Frequently occurring alternative expressions were considered for inclusion and mapped to standardized terminologies or curated vocabularies. This frequency-based approach complemented—without overriding—expert selection, ensuring that the Disease Panel captured both commonly documented concepts and clinically relevant but infrequently recorded terms.

Mapping relied on established standardized terminologies such as SNOMED CT, which provides codes, concepts, and definitions used in clinical documentation and is considered the most comprehensive global clinical terminology,¹⁴ and the ATC classification system, which categorizes drugs by target organ or system and therapeutic and pharmacological properties. Additional terms not covered by these standards but relevant to clinical practice were incorporated as part of the curated terminology developed by medical research experts. Each concept in the Disease Panel included synonyms, acronyms, and abbreviations, thereby generating the first comprehensive list of clinical concepts for an mNSCLC Disease Panel. The initial list of terms included in the Disease Panel was categorized by clinical relevance and their use in clinical practice (see Section 1.2 of Supplementary Materials, available at https://doi.org/10.1016/j.esmorw.2026.100716, for further details).

cNLP terminology enrichment

Once defined, the Disease Panel was enriched through dedicated annotation projects—a structured process in which text is manually labeled with predefined categories or concepts to generate reference data for cNLP model development and evaluation. Based on the terms included in the Disease Panel, a targeted sampling of full documents was carried out and manually reviewed by medical annotators to obtain a corpus representing a large proportion of key terms while keeping the overall number of documents manageable. The corpus was preannotated with NER-NEL predictions for each term (i.e. words identified by NER-NEL as belonging to the Disease Panel appeared as tagged/highlighted text fragments within the selected record’s texts). Medical annotators identified any words that could be considered synonyms of terms included in the Disease Panel that were not preannotated by the NER-NEL pipeline [treated as false negatives (FNs)] and removed tags of highlighted words that were wrongly mapped to any given term of the Disease Panel [treated as false positives (FPs)]. After this assignment was completed, medical and terminology experts evaluated the results and updated the baseline terminology basis of the Disease Panel terminology by adding FNs and removing FPs, based on joint review and consensus.

Enrichment of the terminology by expert oncologists

The enriched Disease Panel was shared with the project’s scientific committee, which included two specialized medical oncologists. Both oncologists independently contributed to the terminology enrichment based on their clinical expertise. Their role was to enhance the terminology by adding alternative expressions commonly used in clinical practice (e.g. synonyms, acronyms, and abbreviations) and to review the final linguistic classification of terms according to their relevance. Expressions suggested by both oncologists were directly incorporated into the updated version of the Disease Panel terminology, whereas terms proposed by only one oncologist or with inconsistent suggestions were resolved through consensus.

cNLP performance evaluation

After the Disease Panel terminology was updated and approved by expert oncologists, the performance of NER-NEL was evaluated for terms eligible for cNLP assessment. This included all key terms and selected important terms. Certain terms were not evaluated through manual annotation, as they were analyzed as independent components (e.g. drug combinations such as pembrolizumab, carboplatin, and pemetrexed within the KEYNOTE-189 regimen) or required extraction using attribute-based models.

Two separate annotation projects were conducted to determine the term-level precision, recall, and F1 score. One of them involved targeted sentence selection for each term included in the Disease Panel, including all its alternative expressions. The other project used full documents based on the set of key terms. Documents to be annotated in both projects were sampled through Smart Linguistic Corpus Extraction (SLiCE), a publicly available software developed by Medsavana (Madrid, Spain), which ensures the robustness and representativeness of performance metrics through a statistically validated gold standard.¹¹ Performance metrics were calculated as indicated in Section 1.3 of Supplementary Materials, available at https://doi.org/10.1016/j.esmorw.2026.100716.

Results

Disease panel design

The mNSCLC Disease Panel included a total of 328 terms. Of these, 22 were designated as key terms for disease identification (Table 1), 246 were important to support comprehensive disease characterization (Supplementary Table S1, available at https://doi.org/10.1016/j.esmorw.2026.100716), and 60 terms were excluded form manual annotation, as they did not meet the eligibility criteria for manual annotation detailed in ‘Methods’ section (Supplementary Table S2, available at https://doi.org/10.1016/j.esmorw.2026.100716, details the reason for exclusion).

Table 1.

mNSCLC Disease Panel: key clinical terms for cohort characterization, selected treatments, and outcomes with alternative expression counts

Term IDa	Key clinical term (English)	Key clinical term (Spanish)	Synonym countb	Acronym countb	Abbreviation countb
Clinical characteristics
254626006	Adenocarcinoma of lung	Adenocarcinoma del pulmón	9	0	14
166749999900100	Anaplastic lymphoma kinase	Cinasa del linfoma anaplásico	1	1	0
39999900100	Epidermal growth factor receptor gene	Gen de Receptor del factor de crecimiento epidérmico	1	3	0
254629004	Large cell carcinoma of lung	Carcinoma de células grandes de pulmón	13	0	1
363358000	Malignant tumor of lung	Tumor maligno de pulmón	32	0	8
128462008	Metastatic malignant neoplasm	Neoplasia maligna metastásica	17	9	0
128632008	Non-small-cell carcinoma	Carcinoma de células no microcíticas	8	1	0
254637007	Non-small-cell lung cancer	Cáncer pulmonar de células no microcíticas	20	5	1
166779999900107	Programmed Death Ligand 1	Ligando 1 de muerte programada	1	0	4
254632001	Small cell carcinoma of lung	Carcinoma de células pequeñas de pulmón	18	3	0
254634000	Squamous cell carcinoma of lung	Carcinoma de células escamosas de pulmón	10	2	8
261612004	Stage	Estadio	4	0	0
Treatments
L01FX99	Datopotamab deruxtecan	Datopotamab deruxtecan	0	0	3
L01XC28	Durvalumab	Durvalumab	1	0	1
L01EB01	Gefitinib	Gefitinib	1	0	1
L01EB04	Osimertinib	Osimertinib	1	3	0
L01FD04	Trastuzumab deruxtecan	Trastuzumab deruxtecán	3	4	0
L01FX20	Tremelimumab	Tremelimumab	0	0	1
Outcomes
999990002431107	Disease progression	Progresión de enfermedad	8	2	0
99999000563101	Patient deceased	Exitus hospitalario	11	0	0
25173007	Recurrent neoplasm	Neoplasia recidivante	10	0	1
999990001799107	Response	Respuesta	3	0	0

mNSCLC, metastatic non-small-cell lung cancer.

^aTerm ID includes ATC codes for drugs, SNOMED-CT codes for clinical terms, and Savana codes for ad-hoc terms do not present in SNOMED-CT.

^bFor each term, alternative terms, acronyms, and abbreviations represent synonymous clinical expressions used in the Spanish context.

Expert oncologists enriched the terminology by adding synonyms, acronyms, and abbreviations commonly used in clinical practice, resulting in comments on 250 terms without discrepancies. Of these suggestions, 73.6% (184 terms) were incorporated into the terminology, whereas the remainder were excluded, primarily due to one-to-one mismatches. The main contributions included the refinement of key terms, the definition of terms needed to identify subgroups, and the inclusion of expressions related to histology, genetic markers, and toxicity, which are frequently underrepresented in standardized terminologies. In contrast, the most common mismatches arose from overly specific or ambiguous expressions that could not be consistently mapped to a unique clinical concept.

A total of 268 terms were considered for cNLP performance evaluation. The average number of alternative expressions—such as synonyms, abbreviations, and acronyms—per term was 8.4, with a range from 1 to 41 (Table 1 and Supplementary Table S1, available at https://doi.org/10.1016/j.esmorw.2026.100716). Among the terms evaluated, 22 (8.2%) were identified as key terms essential for defining the mNSCLC population, including selected treatments and outcomes. Additionally, 246 terms (91.8%) were categorized as important, concerning demographic and clinical characteristics, other treatments, and outcomes. The distribution by section category included clinical characteristics (125, 46.6%), treatments (115, 42.9%), and outcomes (28, 10.5%) (Table 2).

Table 2.

Distribution of mNSCLC terms classified by section category

Terms included in the cNLP performance evaluation	N	%
Clinical characteristics	125	46.6
Anthropometrics	2	0.7
Demographics	4	1.5
Toxic habits/exposures	6	2.2
Comorbidities	26	9.7
mNSCLC diagnosis	32	11.9
mNSCLC signs and symptoms	51	19.0
mNSCLC scales	4	1.5
Treatments	115	42.9
Active treatments	76	28.4
Active procedures	26	9.7
Support treatments	6	2.2
Concomitant medication	7	2.6
Outcomes	28	10.5
Clinical outcomes	13	4.9
Potential toxicities	15	5.6
Total	268	100.0

cNLP, clinical natural language processing; mNSCLC, metastatic non-small-cell lung cancer.

Disease panel enrichment

The data source for this analysis was a screening dataset comprising 131 460 711 EHRs associated with 3 804 636 patients. The following sequential filtering steps were applied to identify relevant records, as detailed in the ‘Methods’ section. Firstly, restricting the period to January 2015 to November 2024 yielded 108 263 178 EHRs. Secondly, limiting the dataset to adult patients resulted in 96 453 231 EHRs. Thirdly, selecting oncology-related records reduced the dataset to 1 933 391 EHRs. Finally, requiring at least one mention of NSCLC or lung cancer together with metastasis further reduced the dataset to 83 716 EHRs. These records were utilized for cNLP enrichment (Figure 2). Among them, 9966 (11.9%) records were included in the cNLP terminology enrichment process based on SLiCE.¹¹ The annotation project conducted to calculate precision for each of the 268 terms in the Disease Panel included 15 965 representative sentences. Additionally, the project for assessing precision, recall, and F1 score metrics included 4111 full-text records obtained through targeted selection, used to evaluate both FPs and FNs for the 22 key terms.

Figure 2.

Flowchart of the EHRs used for the cNLP evaluation.

∗Documents for annotation in the three projects were sampled using SLiCE, a software tool developed by Medsavana (Madrid, Spain) that determines the number of documents needed to create a statistically validated gold standard. cNLP, clinical natural language processing; EHR, electronic hospital record; NSCLC, non-small-cell lung cancer; SLiCE, Smart Linguistic Corpus Extraction.

cNLP performance evaluation

The average ± SD precision for the 268 terms included in the cNLP performance evaluation was 0.97 ± 0.04, with all terms achieving precision scores of 0.8 or higher (Figure 3 and Supplementary Table S1, available at https://doi.org/10.1016/j.esmorw.2026.100716). By category, the average ± SD precision was 0.98 ± 0.04 for characteristics, 0.97 ± 0.04 for treatments, and 0.96 ± 0.06 for outcomes. When the analysis was restricted to the 22 key terms, the average ± SD precision, recall, and F1 score were 0.94 ± 0.05, 0.91 ± 0.08, and 0.92 ± 0.05, respectively. Notably, all key terms achieved F1 scores of 0.8 or higher (Table 3).

Figure 3.

Clinical NLP precision of 268 Disease Panel terms by section category.

NLP, natural language processing.

Table 3.

Clinical NLP performance metrics of 22 key clinical terms of the mNSCLC Disease Panel

Key clinical term	TP	FP	FN	Instances	Precision	Recall	F1
Clinical characteristics
Adenocarcinoma of lung	210	10	22	242	0.95	0.91	0.93
Anaplastic lymphoma kinase	196	0	3	199	1.00	0.98	0.99
Epidermal growth factor receptor gene	292	11	4	307	0.96	0.99	0.97
Large cell carcinoma of lung	14	0	5	19	1.00	0.74	0.85
Malignant tumor of lung	290	26	26	342	0.92	0.92	0.92
Metastatic malignant neoplasm	340	6	39	385	0.98	0.90	0.94
Non-small-cell carcinoma	163	31	2	196	0.84	0.99	0.91
Non-small-cell lung cancer	144	3	18	165	0.98	0.89	0.93
Programmed Death Ligand 1	299	10	8	317	0.97	0.97	0.97
Small cell carcinoma of lung	24	4	4	32	0.86	0.86	0.86
Squamous cell carcinoma of lung	89	1	10	100	0.99	0.90	0.94
Stages	384	12	22	418	0.97	0.95	0.96
Treatments
Datopotamab deruxtecan	18	0	0	18	1.00	1.00	1.00
Durvalumab	10	0	4	14	1.00	0.71	0.83
Gefitinib	13	0	2	15	1.00	0.87	0.93
Osimertinib	33	0	0	33	1.00	1.00	1.00
Trastuzumab deruxtecan	12	0	0	12	1.00	1.00	1.00
Tremelimumab	2	0	0	2	1.00	1.00	1.00
Outcomes
Disease progression	399	39	45	483	0.91	0.90	0.90
Patient deceased	53	9	5	67	0.85	0.91	0.88
Recurrent neoplasm	23	0	1	24	1.00	0.96	0.98
Response	712	60	9	721	0.92	0.99	0.95

TP = number of True Positives; FP = number of False Positives; FN = number of False Negatives; Instances = Sum of TP + FP + FN; Precision = TP/(TP + FP), indicates the accuracy of the information the system retrieves; Recall = TP/(TP + FN), indicates the amount of information the system retrieves; F1 score = (2 × Precision × Recall)/(Precision + Recall), is the overall performance indicator of information retrieval.

mNSCLC, metastatic non-small-cell lung cancer; NLP, natural language processing.

Discussion

In this project, we developed and validated a terminology-based Disease Panel for mNSCLC and integrated it into a cNLP pipeline to enable consistent detection of clinically relevant concepts from unstructured EHR narratives. By addressing key methodological challenges inherent to automated EHR data extraction, including data heterogeneity and inconsistent terminology, this approach provides a structured vocabulary enriched and validated by oncologists and medical research experts. Our results show that incorporating a validated Disease Panel enables high-precision and robust cNLP-based data extraction, providing a scalable alternative to manual chart review for identifying and characterizing patients with mNSCLC from EHR data. This methodology has the potential to reduce clinician workload, promote homogeneous data capture across participating hospitals, and support the generation of high-quality structured datasets for future oncology RWE research.

Compared with manual data collection, this approach enables large-scale and efficient analysis while reducing time and resource requirements. Unlike general natural language processing (NLP), cNLP contends with domain- and language-specific challenges and ambiguity in medical discourse, making model development and validation demanding.15, 16, 17, 18 The structured framework we propose with Disease Panel implementation strengthens cNLP development rather than general NLP, as it is specifically designed to address the complexity and sensitivity of oncologic narratives. By enabling consistent detection of clinically meaningful variables, this approach enhances the reliability of data extraction for oncology research and clinical practice. In this area, where precise capture of tumoral staging, histological subtypes, biomarkers, and treatment response is critical, such tailored methodologies become indispensable. Moreover, alignment with international standards such as SNOMED CT and ATC fosters interoperability across health care systems and enables resource reuse in multicenter and multinational studies, supporting reliable RWE generation.

Our findings align with reviews highlighting the potential of AI in lung cancer research¹⁹ and its potential to enhance diagnosis and treatment outcomes,²⁰ while addressing unmet needs for standardization and reproducibility. The development and validation of the Disease Panel represent a methodological advance over prior NLP applications in lung cancer. For example, Ansoborlo et al.²¹ applied NLP to prescreen patients for oncology trial eligibility using reports from multidisciplinary team meetings. Although their approach demonstrated feasibility and highlighted the value of unstructured narratives, it relied on general NLP pipelines without a validated terminology framework, with precision, recall, and F1 score values ranging from 0.68 to 0.87. In contrast, our validated Disease Panel achieved consistently higher metrics across all evaluated terms, with particularly robust performance for key clinical terms, reflecting the value of combining cNLP with a disease-specific, expert-curated terminology. Wang et al.²² demonstrated the feasibility of NLP for extracting lung cancer stage, histology, grade, and therapies from clinical narratives. Kenney et al.²³ showed that although generalized oncology NLP performs well across cancer types, its accuracy decreases in nuanced tasks such as linking histology to neoplasms. Together, these studies support NLP’s promise for oncology but underscore the need for domain-specific refinement, suggesting that terminology-based Disease Panels may provide greater precision and clinical relevance.

We systematically assessed term detection using precision, recall, and F1 score, ensuring that only terms meeting predefined quality thresholds were retained. This iterative validation process, including expert review of FPs and FNs, enhanced both the Disease Panel terminology and underlying NER-NEL models. This robust validation allows construction of complex cohorts and deep phenotyping, enabling more detailed and clinically meaningful analyses than approaches solely relying on unvalidated term detection. Gayap and Akhloufi²⁴ recently reviewed the application of deep ML for lung cancer diagnosis, focusing on imaging-based detection methods and discussing challenges such as dataset dependency, generalizability, and lack of interpretability. Although these approaches illustrate the power of AI for detection tasks, they often remain ‘black box’ models. By contrast, our methodology combines terminological rigor with quantitative performance validation, providing transparency, reproducibility, accuracy, and interpretability across clinical contexts.

Beyond lung cancer, NLP has been applied to generate oncology RWE. Studies using cNLP have characterized clinical features, management, adverse events, and health care resource utilization in patients with chronic lymphocytic leukemia in Spain,^25,26 whereas others applied NLP to radiology reports using the Ovarian-Adnexal Reporting & Data System lexicon to predict metastatic response.²⁷ These studies highlight NLP’s growing role in oncology research, but also the methodological heterogeneity and lack of standardized terminology. By grounding our methodology in a validated disease-specific panel, we propose a scalable and transferable approach that may enhance comparability across studies and facilitate its application across tumor types and healthcare settings.

Future RWD or RWE applications require reconstruction of patient trajectories and assessment of temporality through integration of temporal elements such as therapy start and stop dates, progression events, and death. These components were beyond the project scope, which focused on developing and validating a disease-specific terminology-based panel. Further steps include the transformation of extracted terms into clinically meaningful variables through contextualization and postprocessing. Contextualization relies on dedicated NLP models to disambiguate overlapping concepts (e.g. planned versus completed procedures and suspected versus confirmed findings), detect negation, assess temporality, and determine whether a term refers to the patient or a relative. Postprocessing maps extract terms to study-specific variables, incorporating mandatory temporal elements to position clinical events over time and capturing relationships between core terms and associated qualifiers such as severity, laterality, or dosage. Robust RWD studies also require quality assurance, including external term-level validation by physicians at participating hospitals, patient-level verification through longitudinal cohort review, and consistency checks with scientific literature.

Limitations include the use of Spanish EHRs, limiting the generalizability to other languages, although the methodology is scalable and adaptable. Secondly, although the Disease Panel was enriched through expert curation and iterative annotation, the detection of some clinical concepts that rely on attributes such as severity or temporality requires validation within the context of RWD studies, where longitudinal patient data are available to verify their accuracy and clinical meaning. This study used de-identified EHR data for terminology development and term-level validation, without defining a full patient cohort; thus, specific cNLP models for attribute extraction were not assessed and should be addressed in future work. Finally, although performance metrics reached high values for most terms, the methodology ultimately depends on EHR completeness and accuracy; cNLP can only extract recorded information, leaving unreported or inconsistently documented variables beyond reach. Despite these limitations, the methodology provides a robust, reproducible framework with strong potential for extension across diseases and settings.

Conclusions

Our results demonstrate the feasibility and value of developing and validating an mNSCLC terminology-based Disease Panel for integration into a cNLP pipeline. Combining expert-driven terminology refinement with systematic validation, our methodology addresses key limitations of previous NLP applications, including a lack of standardization and limited reproducibility. This approach enables accurate and interpretable extraction of clinically relevant data from unstructured EHRs. The Disease Panel demonstrates high performance across a broad set of clinical terms, supporting its robustness and reliability for term-level identification and classification of disease-related information. Compared with manual data collection, this approach reduces human resource requirements and improves consistency across sites, facilitating more efficient and homogeneous RWD extraction processes. Although the present validation focuses on term-level extraction, the methodology lays the foundation for downstream RWD/RWE applications. In this regard, the framework contributes to building auditable and standardized data extraction processes that help ensure extracted data are fit for purpose and scientifically valid, with the potential to support regulatory-grade compliance in alignment with emerging regulatory standards.