Completeness of Common Data Elements for Breast Cancer Clinical Trials in Observational Databases

Abstract

Clinical trials serve as a gold standard for data that shapes the healthcare landscape in oncology, but variance in clinical trial eligibility criteria presents challenges for using the data in downstream applications. Identifying common data elements (CDEs) present in cancer trials’ eligibility criteria is critical to achieve standardized representations of healthcare data and thus develop trial recruitment tools that generalize to oncology. In this study, we curate a set of CDEs contained in the eligibility criteria of breast cancer clinical trials and evaluate their completeness across different observational databases represented in the Observational Medical Outcomes Partnership (OMOP) Common Data Model. We show that between databases, CDEs are captured with differing levels of completeness across OMOP domains and that there is discordance between the frequency of eligibility criteria CDEs and their completeness in observational databases, which characterizes these databases’ utility for subsequent oncology trial matching efforts.

Introduction

Evidence-based decisions are critical to practice modern-day medicine, and clinical trial results provide a gold standard of data to drive healthcare decision-making(1). They assist with informing clinical guidelines along with developing or repurposing existing treatments(2). In particular, clinical trials are critical in shaping outcomes for cancer patients by advancing therapies for the global leading cause of death(3,4). This helps explain why 43% of biopharmaceutical industry-sponsored clinical trials conducted in the United States are oncology trials(5). The majority of cancer patients, however, are uninvolved in oncology trials, as estimates indicate 7% of adults with cancer engage in research studies(6). Roadblocks in patient recruitment contribute substantially to this low participation, as over 80% of clinical trials experience delays and 20% of oncology clinical trials fail due to insufficient enrollment(7,8). Research efforts focus on automating manually intensive trial recruitment processes (e.g., screening patients using multiple data sources, contacting candidates about the trial, etc.)(9) to alleviate these delays, but the free-text nature of trials’ eligibility criteria and lack of an agreed computational standard to represent eligibility criteria hinder the development of software solutions and generalizable research findings(10).

Identifying common data elements (CDEs), generalizable variables used to standardize data collection, across oncology trials’ eligibility criteria addresses the roadblocks detailed above. Within clinical research informatics, CDEs facilitate the development of interoperable healthcare systems and enable effective secondary usage of healthcare data(11). Thus, establishing CDEs pertinent to oncology trials’ eligibility criteria assists data-driven healthcare decisions and research efforts. Prior work to identify oncology CDEs focuses on their research utility. Systematic literature reviews are used to curate oncology CDEs and focus on specific types of cancers or medical disciplines with the aim of progressing towards data interoperability(12,13). Other oncology CDEs are grounded in clinical decision-making by analyzing institution-specific standard operating procedures(14). Groups composed of multiple healthcare and clinical research informatics stakeholders also deliberate to identify oncology CDEs (15,16). These initiatives do not consider how data needs differ in the context of patient recruitment for oncology clinical trials, which presents an opportunity to establish more task-specific CDEs.

Studies that characterize eligibility criteria of cancer clinical trials have two broad limitations. Clinician-centric efforts identify which CDEs are available in clinical data sources, but they come with the tradeoff of using a limited number of oncology clinical trials(17). Secondly, studies that characterize a larger volume of oncology clinical trials conduct analyses from the perspective of trial design rather than enrollment(18–20), so their findings focus less on the suitability of healthcare data for computational trial recruitment solutions. These gaps reinforce the merit of identifying recruitment-specific CDEs of oncology trials’ eligibility criteria through large-scale studies, which can inform the methodology and design of computational trial recruitment tools. In this work, we aim to determine a set of CDEs found in the eligibility criteria of breast cancer clinical trials and evaluate how structured clinical data captures these CDEs. To enable this analysis, we identify CDEs across the eligibility criteria of breast cancer trials stored in Clinical Trial Knowledge Base (CTKB)(21), a comprehensive knowledge base of discrete eligibility criteria, and evaluate these CDEs’ completeness amongst structured healthcare databases.

Data and Methods

CDE Identification

CTKB is a relational database of discrete criteria and attributes that are derived from processing the eligibility criteria text of clinical trials registered in ClinicalTrials.gov, a web-based registry for clinical trials that is adopted on an international scale, by October 1st, 2020(21). CTKB parses entities from eligibility criteria text with a custom information extraction pipeline. These entities are mapped and standardized to concepts in the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM)(22). These two steps are performed using the entity normalization module of Criteria2Query, a software tool for identifying suitable cohorts from free-text eligibility criteria that utilizes the open-source concept mapping tool Usagi(23,24). Based on metadata from ClinicalTrials.gov, each trial in CTKB is linked to a target condition. We identify a dataset of 7,984 breast cancer clinical trials to analyze by querying CTKB for trials with the target condition “neoplasm of breast”. We analyze the processed eligibility criteria of these clinical trials and identify common data elements across the following OMOP domains: Condition Occurrence, Drug Exposure, Measurement, and Procedure Occurrence. For each OMOP domain, we determine the five most frequent CDEs and their frequency across all trials. We do not consolidate semantically similar CDEs across OMOP domains into a single reported CDE but rather treat them as separate CDEs to adhere to how CTKB captures information (e.g., Procedure Occurrence concepts we categorize as “Chemotherapy” are reported distinctly from Drug Exposure concepts we categorize as “Chemotherapy”). For the Drug Exposure domain, we roll up data elements into categories based on Anatomical Therapeutic Chemical (ATC) or HemOnc’s classification groupings(25).

CDE Completeness Evaluation

Using the CDEs we curate through the steps detailed above, we assess the data completeness of these CDEs across the structured data of breast cancer patients in two databases. For each database, we implement a computable breast cancer phenotype from PheKB to establish the cohort we analyze(26). We evaluate the All of Us (AoU) Curated Data Repository (CDR) v8(27) and the Columbia University Irving Medical Center (CUIMC) databases. The AoU CDR represents a racially and ethnically diverse set of over 598,000 participants consenting to contribute health data originating from electronic health records (EHR), wearables, physical measurements, genomic tests, and surveys. The CUIMC database represents the EHR data of patients who interact with the New York-Presbyterian/CUIMC hospital system. Both databases adhere to the OMOP CDM. For each database, we measure the data completeness of each CDE by computing the proportion of patients with at least one data record pertaining to the corresponding CDE. We use the CDEs’ corresponding OMOP domains to organize how we report data completeness values.

Results

There are a total of 9,085 CDEs in the breast cancer clinical trials. Table 1 highlights the five most frequent CDEs in the eligibility criteria for breast cancer trials stored in CTKB across the four chosen OMOP domains.

Table 1.

The five most frequent CDEs, across four different OMOP clinical domains, found in the eligibility criteria of breast cancer clinical trials stored in CTKB.

OMOP Domain	Common Data Element	Trial Count (n = 7,984)
Condition Occurrence (n=4,873)	Breast Cancer	6,285 (78.720%)
	Pregnancy	3,333 (41.746%)
	Hypersensitivity	2,245 (28.119%)
	Congestive Heart Failure	1,543 (19.326%)
	Hepatitis	1,515 (18.975%)
Drug Exposure (n=1,910)	Chemotherapy	2,214 (22.730%)
	Hormone Therapy	1,817 (22.758%)
	Targeted Therapy	1,018 (12.751%)
	Steroids	976 (12.224%)
	Antibiotics	439 (5.498%)
Measurement (n=1,247)	Bilirubin	2,179 (27.292%)
	Platelet Count	2,086 (26.127%)
	Pregnancy Test	1,517 (19.000%)
	HER2 Tumor Marker	1,464 (18.337%)
	Hemoglobin	1,271 (15.919%)
Procedure Occurrence (n=1,055)	Chemotherapy	3,645 (45.654%)
	Radiotherapy	2,174 (27.229%)
	Imaging	1,745 (21.856%)
	Biopsy	1,579 (19.777%)
	Hormone Therapy	1,293 (16.195%)

Figure 1 shows the prevalence of the identified CDEs amongst the breast cancer cohorts in the AoU CDR (n=11,971) and CUIMC database (n=60,837). We observe both databases have strong completeness for most of the Measurement domain CDEs relative to their trial frequency reported above with the exception of the tumor biomarker measurement. We observe the lowest completeness in both databases for Procedure Occurrence domain CDEs relative to their trial frequency reported above with the exception of Biopsy and Imaging, the Procedure Occurrence CDE not specific to oncology.

Figure 1.

The completeness of the five most frequent CDEs, across four different OMOP clinical domains, in breast cancer cohorts defined within the AoU CDR and CUIMC database.

Discussion and Future Work

Throughout this study, we identify CDEs present in breast cancer clinical trials’ eligibility criteria and evaluate how often they are observed in structured clinical data. We curate sets of CDEs across multiple clinical domains as defined through OMOP and observe varying trends in the CDEs’ completeness for breast cancer patients’ records. Firstly, we observe that with the exception of the Procedure Occurrence domain, the AoU cohort universally has stronger coverage of the reported CDEs. This general trend can be attributed to AoU CDR ingesting more sources of healthcare data compared to the CUIMC database. Prior AoU research has found over 25% of participants diagnosed with cancer navigate at least two socioeconomic barriers, which was significantly associated with a lower chances of receiving appropriate follow-up care(28). AoU participants’ more limited access to healthcare helps explain the departure of the trend noted earlier in the Procedure Occurrence domain. Secondly, we notice there are differing levels of disagreement between the CDEs’ frequency in eligibility criteria and their prevalence in structured clinical data. Notably, we observe low prevalences for several CDEs related to cancer care (e.g., Chemotherapy, Radiotherapy, and HER2 Tumor Marker) relative to their corresponding trial frequencies across structured data. Unstructured data (i.e., pathology reports, clinical notes, imaging results, etc.) has been shown to contain detailed and accurate information, so trial matching tools should extend their methodology beyond using solely structured data to query oncology-specific CDEs(29–31). Both takeaways demonstrate the specific research problems each database is suitable for and shed light on issues to address before leveraging real world evidence for oncology trial recruitment. Work to address these gaps can be aligned with parallel efforts such as the OHDSI Oncology Working Group’s aim to support observational cancer research(32). Finally, the databases we analyze demonstrate different completeness characteristics for the identified CDEs, which highlights where limitations exist for each database’s data ingestion and transformation pipelines.

These preliminary CDEs can be used to inform several downstream areas of informatics centered around oncology trial recruitment. Research initiatives to assess the feasibility of an oncology clinical trial’s design can leverage these preliminary CDEs to build out evaluation frameworks. Furthermore, these CDEs can be used as a basis to identify gaps in an observational database’s usability for oncology clinical trial recruitment. Finally, these CDEs can assist in standardizing efforts to develop reliable phenotypes for oncology clinical trials, which enables a wide range of subsequent informatics research.

Limitations

While our work provides preliminary CDEs for the eligibility criteria of breast cancer clinical trials and an evaluation of their data capture in observational databases, there are limitations. This CDE analysis was conducted for strictly breast cancer clinical trials, which limits the generalizability of the CDEs across oncology clinical trials. CTKB’s concept extraction pipeline is not oncology-specific and thus is susceptible to not capturing oncology-specific eligibility criteria such as genetic testing. The dataset of CTKB breast cancer clinical trials used in this analysis does not account for breast cancer trials registered after October 1st, 2020, and we do not employ any data quality filters to analyze subsets of breast cancer clinical trials (e.g., terminated trials, completed trials, etc.). We generate each breast cancer cohort using solely structured data and consequently do not catch patients who would be included by phenotyping with unstructured data.

Conclusion

Clinical trials are the gold-standard of data for oncology healthcare and research, as they are responsible for shaping clinical treatment guidelines and the standard of care. However, there is considerable variation in the reporting of clinical trials, specifically their eligibility criteria, that creates barriers in designing informatics solutions. For these reasons, developing robust methods for normalizing the development of oncology clinical trials’ eligibility criteria is a critical research gap that could be filled by identifying CDEs to establish standard practices. We curate a list of preliminary CDEs present in breast cancer trials’ eligibility criteria represented in CTKB and assess the completeness of these CDEs in the AoU CDR and CUIMC database. Our findings show that there are generalizable trends for which CDEs are captured with stronger completeness in each database for breast cancer patients. Future work would focus on creating an augmented concept mapping pipeline capable of parsing genomic eligibility criteria and that leverages large language models to extract concepts. Another direction would be to extend a database evaluation beyond completeness using data quality assessment methods such as Kahn’s framework(33). Finally, prior work highlights that eligibility criteria found in actual trial protocols are more complex than the corresponding ClinicalTrials.gov entries’ eligibility criteria(34), so future work would investigate the consistency of the reported CDEs in trial protocols.