A use-case analysis of Clinical Data Interchange Standards Consortium/Study Data Tabulation Model in academia in an investigator-initiated clinical trial

ABSTRACT

Submitting data compliant with the Clinical Data Interchange Standards Consortium (CDISC) standards is mandatory for new drug applications (NDAs). The standards set by CDISC are widely adopted in the pharmaceutical business world. Introduction of CDISC standards in academia can be necessary to reduce labor, resolve the shortage of data managers in academia, and gain new knowledge through standardized data accumulation. However, the introduction of CDISC standards has not progressed in communities within the academia that do not apply for NDAs. Therefore, herein, we created study data tabulation model (SDTM)-compliant datasets within the academia, without outsourcing, to reduce costs associated with investigator-initiated clinical trials. First, we input data from paper case report forms (CRFs) into an electronic data capture system with minimal function for paper CRFs, “Ptosh,” which is compatible with SDTM. Then, we developed a generic program to convert data exported from Ptosh into fully SDTM-compliant datasets. The consistency was then verified with an SDTM validator, Pinnacle21 Community V3.0.1 (P21C). This resulted in generation of SDTM datasets, resolving all “Rejects” in P21C, thereby achieving the required quality level. Although Ptosh directly exports data in SDTM format, manual mapping of items on CRFs to SDTM variables prepared in Ptosh is necessary. SDTM mapping requires extensive knowledge and skills, and it was assumed that mapping is challenging for the staff without in-depth knowledge of CDISC standards and datasets. Therefore, for CDISC dissemination in academia, it is crucial to secure the staff, time, and funding to acquire the knowledge.

INTRODUCTION

In Japan, the Ministry of Health, Labour and Welfare, Pharmaceutical and Food Safety Bureau Director published Notification No. 0620-6 “Basic Principles on Electronic Submission of Study Data for New Drug Applications” in 2014. From October 2016, submission of electronic data that conforms to the Clinical Data Interchange Standards Consortium (CDISC) standards, which are the global standards for clinical trial data, is required for new drug applications (NDAs).^1-3 Currently, the CDISC/Study Data Tabulation Model (SDTM), which is the raw data for each study, and CDISC/Analysis Data Model, which is the dataset processed for analysis but not mentioned in this work, often tend to be submitted to the Pharmaceuticals and Medical Devices Agency (PMDA) for NDAs.² The CDISC/SDTM defines a standard structure for clinical trial data tabulations. Each of these tables is called a domain. For example, data on subject demographics are assigned to the domain coded as “DM.” Domains are defined for subject demographics, laboratory data, vital signs, adverse events and so on, according to the attributes of the dataset. Usually, clinical trial data are acquired and mapped to the SDTM variables as part of the data management process. Each domain code, variable name, variable order, and display format is defined in detail in the SDTM Implementation Guide.⁴ When receiving CDISC standard data, the PMDA uses software called Pinnacle21 Enterprise (P21C) to validate whether the data were created according to the rules of the CDISC standard.⁵

Overseas, in the United States of America, the Food and Drug Administration requires the submission of CDISC data⁶; in China, CDISC standards are now the preferred standards for electronic data submission.⁶ European and South Korean authorities are considering adopting these standards.⁷ Thus, the submission of CDISC data for NDAs is becoming a standard. CDISC standards have spread rapidly in the Japanese pharmaceutical industry. As of October 31, 2019, 302 consultations have been submitted for electronic data submission by the PMDA, and electronic application data have already been submitted for 94 product applications.⁸

Originally, CDISC standards were not meant for NDAs. CDISC states that accessibility, interoperability, and reusability of data enable more meaningful and efficient research that has a greater impact on global health.⁹ Barrie Nelson, vice president of CDISC, quotes “The benefits of implementing CDISC standards in research studies are numerous -- fostered efficiency, enhanced innovation, increased predictability, complete traceability, improved data quality, reduced costs, and streamlined processes -- all ensuring the integrity of your data from end to end”.¹⁰ The CDISC standards will promote the standardization of operations, leading to laborsaving and high-quality trial operations. The standards will also enable meta-analysis of accumulated, standardized data, leading to advancements in medicine. PMDA not only requires CDISC data for regulatory review but also envisions the use of accumulated, standardized data.¹¹ These benefits are not only for pharmaceutical companies but also for academia. Further, the Japan Agency for Medical Research and Development (AMED) states that investigator-initiated clinical trials will need to comply with CDISC standards from the planning and implementation stages, and they expect that the clinical trials contracted with them will be performed according to CDISC standards.¹²

In academia, unlike in pharmaceutical companies, clinical trial protocols are created by investigators. In a pharmaceutical company, a description of items to be captured is usually standardized. For instance, the number of white blood cells will be denoted as “WBC” even if companies have different protocols. However, because each clinical trial protocol within a single university is created by a different principal investigator, an item will be denoted in different ways in each protocol, such as “White Blood Cell,” “WBC,” or “Leukocytes.” This leads to different nomenclatures and database structures for clinical trial data. This will increase data management labor, and meta-analysis using multiple datasets from different clinical trials will be more difficult. Therefore, the introduction of CDISC standards will help resolve these problems.¹³

Although there are good reasons to introduce CDISC standards in academia, the standardization is not widespread, at least in Japanese academia. One of the reasons being insufficient research funding,¹⁴ not only for hiring staff but also to cover the expenses of implementing the required information technology (IT) system and the associated staff education. Therefore, human resources, such as data managers, are limited, and the daily workload is overwhelming, which implies that the scope to exceed the necessary minimum work is limited. Furthermore, conditions of employment within academia makes it difficult to hire people with expertise.¹⁵ In addition, because most of the data management staff are employed by hospitals, priority is given to those with medical licenses, in contrast to those with IT knowledge. The use of CDISC standards is not mandatory in investigator-initiated trials that are not used for NDAs. These trials are considered to be the reason for the delayed introduction of CDISC standards in academia.

Therefore, we assessed whether fully SDTM-compliant datasets can be generated within academia without outsourcing the task to contract research organizations in an investigator-initiated clinical trial in the field of oncology and clarified adequate methods and problems.

METHODS

The investigator-initiated clinical trial “A phase I/II, open-label, single-arm study of CH5424802 for patients with advanced non-small-cell lung cancer harboring a RET fusion gene” (University Hospital Medical Information Network (UMIN) ID: UMIN000020628) (hereinafter referred to as the “ALL-RET trial”)¹⁶ was the target of this study (hereinafter referred to as “CDISC-study”). The CDISC-study is neither an activity through understanding the cause of diseases and their pathology, nor is it an activity through improving measures to prevent injury and disease as well as diagnostic and treatment measures in medical care or through verifying those measures’ validity. Therefore, we think that it is outside the scope of the Ethical Guidelines for Medical and Health Research Involving Human Subjects.¹⁷

The ALL-RET trial had already started data collection with paper case report forms (CRFs) at the planning stage of the CDISC-study. We used “Ptosh” as the clinical data management system (CDMS). “Ptosh” was jointly developed by the National Hospital Organization Nagoya Medical Center and the Nonprofit Organization for Supporting Clinical Research.¹⁸ It is an electronic data capture (EDC) system with minimal functions for paper CRFs, and it can export data in SDTM format. An EDC is a system that creates electronic CRFs, and the data entered in the database becomes the original CRF. However, as data collection was already progressing with the paper CRFs in the ALL-RET trial, we decided not to use Ptosh as the EDC system, but rather as a CDMS for paper CRFs. In contrast to the EDC system, which is a web-based application where the data are directly input at trial sites, the CDMS is a system for data management staff to input and manage CRFs at the datacenter and is based on the paper-based CRF data.

A general CDMS or an EDC system exports files for each input screen, and the mapping to SDTM will be performed using those files. However, in Ptosh, regardless of the input screen and structure, variables can be consistently mapped to SDTM (Fig. 1). Therefore, datasets in SDTM-format can be obtained without any programming; however, the datasets are not complete SDTM datasets for NDA. Ptosh can output integrated data in SDTM format from one input screen with multiple domains, or vice versa, and can aggregate data from multiple input screens for one domain.

Fig. 1.

Contrast for concept of creating SDTMs between general EDC or CDMS, and Ptosh

Concept of creating SDTMs using general EDC or CDMS is shown on the left side, and Ptosh on the right side. Ptosh can output data in SDTM format directly, whereas general EDC and CDMS create files for each input screen and later create SDTM datasets using conversion programs.

We mapped CDISC/SDTM variables to CRF items for the ALL-RET trial, built a database and input screens on Ptosh (Fig. 2(A)), and entered CRF data into the database using input screens (Fig. 2(B)).

Fig. 2.

Outline of the CDISC-study

First, (A) we mapped CDISC/SDTM variables to CRF items for the ALL-RET trial and built a database and input screens on Ptosh, (B) entered the CRF data of the ALL-RET trial into the Ptosh database, (C) processed the exported SDTM-format data from Ptosh with SAS Macros, (D) manually created trial design datasets, and then (E) used P21C to validate SDTM datasets. (F) We created three lists using the SDTM datasets.

AE: adverse events

SDTM datasets exported by Ptosh are incomplete; thus, post-processing is required to make them fully SDTM-compliant. Therefore, we developed a program using Base SAS, that is a software provided by SAS Institute Inc., to process these datasets. (Fig. 2(C))

In addition to CRF data, SDTM standards also require a description of a protocol outline called a trial design dataset. We manually created the trial design datasets using Microsoft Excel to avoid rejection by P21C. (Fig. 2(D))

The processed SDTM domain file and the minimum trial design datasets were verified by P21C. (Fig. 2(E))

Finally, as a part of the data management work of the ALL-RET trial, we used the complete SDTM datasets to create a list of patients, adverse events, and response evaluation criteria in solid tumors (RECIST) transitions for safety monitoring (Fig. 2(F)). Although RECIST is used for efficacy evaluation, the transition was monitored so that investigational drug administration would not be continued even though it deserves progressive disease (PD).

In addition to the data management system, the first author, who is in charge of data management, self-learned SDTM using materials published on the web, attended a half-day seminar conducted by UMIN, the official SDTM training provided by CDISC, and participated in a CDISC Japan User Group/SDTM team.

RESULTS

We, the academic staff for data management, mapped the CRF items to SDTM variables and built a database in Ptosh. The contents of paper CRFs were input into the database, and data were exported in SDTM format. The exported data required minimal processing to avoid problems in P21 validation.

In the CDISC study, we obtained 22 domains directly from Ptosh (Fig. 3). In Ptosh, multiple domains can be mapped on the same input screen, and variables from one domain can be distributed on multiple input screens, exporting data in SDTM format in an integrated way.

Fig. 3.

Exported files from Ptosh

State where 22 domain files for the ALL-RET trial were exported from Ptosh (unprocessed). Each domain file for CDISC was created directly from Ptosh.

An adverse event entered into Ptosh in Japanese (Fig. 4a) was exported directly in SDTM format in English as the second row of AE (Adverse Events) domain data (Fig. 4b). With adequate license, a medical dictionary for regulatory activities (MedDRA) can be automatically coded by selecting and inputting the symptom name of the adverse event on the input screen. Even if the input screen was in Japanese, all Ptosh outputs were created using Controlled Terminology in English; the terminologies were specified by CDISC.

Fig. 4a.

Screen for adverse events in Ptosh (test data)

All input screens are shown in Japanese. It also shows that MedDRA coding can be performed directly in the system.

Fig. 4b.

Unprocessed AE domain data output from Ptosh

Part of the unprocessed AE domain data output from Ptosh. As shown in Fig. 4a, the data was entered by selecting the Japanese options, but all the data were stored in English. MedDRA coding stores not only the code but also English terms. AEACN, AEREL, and AEOUT are all stored in the Controlled Terminology terms in English characters. The dataset item names are SDTM-compliant.

RECIST data is divided into three domains, TU (Tumor Identification), TR (Tumor Results), and RS (Disease Response) in SDTM. TU comprises the names of target and non-target lesions. Each TU record represents one lesion. TR includes the size of target lesions or presence of non-target lesions. A TR record represents each lesion for each visit. RS includes the assessments for target lesions, non-target lesions, new lesions, or overall responses. An RS record represents an assessment on a visit. It is necessary to link these TU and TR records using Link ID, and this link can be generated by Link ID on pre-mapped CRFs in Ptosh (Fig. 5).

Fig. 5.

Unprocessed output data from Ptosh for the TU, TR, and RS domains

Data output from Ptosh for the TU, TR, and RS domains for the RECIST evaluation. TU and TR data are appropriately linked by TULNKID and TRLNKID. In this sample data, there are 3 target lesions and 3 non-target lesions, but the input screen allows for 5 target lesions and 7 non-target lesions to be entered. Therefore, the lesions that have not been input are output as blanks, and these records require processing to be deleted later.

The data exported from Ptosh are not completely SDTM-compliant. Most variables for prefixed data or input data will be exported from Ptosh. However, most variables for derived data will be missing or exported as blank. For example, an exported DM (Demographics) domain does not contain any data derived from other domains (eg, study drug administration end date, death status, date of death, etc). We needed to post-process the exported data to fill these derived variables. Additionally, we needed to delete extra records with blank data, which will not be exported from the current version of Ptosh. For example, a patient has 3 target and 3 non-target lesions in the ALL-RET trial (Fig. 5). However, because 5 and 7 input fields were prepared on the input screen for target and non-target lesions, 5 and 7 records are exported with 2 extra-records for target lesions and 4 extra for non-target lesions (Fig. 5). We developed a processing program to complete SDTM datasets using Base SAS. This program was created as a general-purpose macro so that it can be used in other studies.

In the validation of P21C, we reduced “Reject” and “Error” items by modifying the mapping in Ptosh for the ALL-RET trial. Finally, 0 rejects and 8 errors were left. All 8 errors were explainable. They were caused by a lack of data on the cases that dropped out before the start of treatment and the inconsistency in the dates of cases that were discontinued without treatment. PMDA will not accept any data with “Reject,” but it will accept data with “Error” that include explanations regarding the causes of the violations and the reasons why such errors cannot be corrected.¹⁹ Therefore, if the data included no “Reject” by the Pinnacle 21 Community, a free version of Pinnacle 21 Enterprise,²⁰ then it implies that our data is suitable for submission to the PMDA. There was no “Reject” from the P21C validator; therefore, we were able to obtain SDTM-compliant datasets, which is sufficient for NDA.

DISCUSSIONS

This study makes the following contributions:

1. We developed a method with which the data management staff of academia can output fully SDTM-compliant datasets, validated by P21C, in investigator-initiated trials without outsourcing.

2. The software, which exported data in SDTM format, was shown to be useful and an adequate method in academia.

Effects of Standardization and Laborsaving using SDTM Datasets

We developed a program to create three lists from the fully SDTM-compliant datasets (Fig. 2(F)). The program is not specific to the ALL-RET trial, and it can be applied without modification for SDTM datasets from any clinical trial regardless of subspecialties. Likewise, other general programs can be developed for other lists, aggregations, graphs, statistics, etc, if SDTM datasets are available. Standardizing SDTM datasets may lead to reduced labor, which is a significant advantage even for clinical studies without NDA. In addition to the CRF data management work, it seems possible to build a common central monitoring system within academia with SDTM datasets.

How Did We Choose Ptosh?

We usually use DEMAND, a CDMS provided by Densuke Systems Co., Ltd. (currently transferred to G-Link System Consulting K.K. by business transfer), as the CDMS for paper CRFs in Kanazawa University. DEMAND is built using the same design concept as the “general EDC or CDMS” shown in Fig. 1. Therefore, DEMAND has templates corresponding to the SDTM domains and Controlled Terminology, which is a CDISC standard dictionary. For domains with a simple structure, such as an adverse event that consists of one event and one record, a template with the AE domain structure can be easily created. However, clinical trial data are not limited to such simple structures. In particular, data belonging to the “Findings” domain class in SDTM have a single record per item; therefore, the data structure is completely different from the “general EDC and CDMS” referred to in Fig. 1 (Fig. 6 and 7). Also, in SDTM, data must be written in English character strings, and not in codes, such as numerical values (Fig. 7); however, it is cumbersome to manually enter all data in English character strings. If there is no function to convert input code to terminology or select input on the input screens, it cannot be put to practical use. There are other limitations with the data. For instance, the TU, TR, and RS domains must be linked to each other.

Fig. 6.

Contrast for image of output data between general EDC or CDMS, and Ptosh

General CRF for the presence of symptoms and an image of the output data. In the output data from general EDC or CDMS, one record represents the contents of one CRF. In SDTM, however, one record represents each item. Thus, SDTM can be used universally irrespective of the items prepared for each clinical trial.

Fig. 7.

Contrast for image of output data between general EDC or CDMS, and Ptosh

CRF that describes an adverse event and images of output data. In a general EDC or CDMS, choices are stored as code. However, SDTMs must be created as English strings specified by Controlled Terminology. In the case of a system that can be entered only with a code, it will be necessary to create another program later to convert every data point entered by code.

We concluded that using DEMAND alone may not be sufficient to output the data of all CRFs from the ALL-RET trial to SDTM and decided to use Ptosh for SDTM output. However, Ptosh has the following limitations. To use Ptosh, it is necessary that staff who are familiar with SDTM perform SDTM mapping to build the input screens. They must carefully design the input screens. In a general system, shown in Fig. 1, SDTM knowledge is not required for the construction of input screens; this knowledge is required only during the development stage of the conversion program. Also, for long-term clinical studies, Ptosh design will be based on the SDTM version prior to the start of operations, and new SDTM versions or therapeutic areas may be newly released at the time of data fixing. This may require minor modifications of exported data from Ptosh in the future.

Use of Ptosh, an EDC, as a CDMS and Role Sharing Between It and a General CDMS

In the CDISC-study, because we planned to output SDTM datasets with paper CRFs and CDMS, and the ALL-RET trial had started with paper CRFs, we decided to use Ptosh as the CDMS. However, because Ptosh is an EDC system with minimal functionalities as a CDMS, it is designed to create electronic CRFs. Usually, only investigators, sub-investigators, or research coordinators at trial sites are allowed to input data directly into the electronic CRFs in EDC. We used the Ptosh version available as of January 2019 because it had a minimum CDMS function called “substitute input” that allowed data center staff to input data from paper CRFs filed by trial site members. However, this is not recommended as the function is now obsolete and could be abolished in the future.

Ptosh is an EDC system without the general data management functions present in many CDMSs. For example, Ptosh does not have the double-entry-and-compare function (the method of two people entering the same CRFs separately and comparing the data to find and correct input errors). Therefore, we performed collating by reading out all items in all cases. However, collating by reading out is more error-prone than double-entry.²¹ It is more efficient to consider using a CDMS with double-entry function at least for the numerical data in the LB (Laboratory Test Results) and VS (Vital Signs) domains of SDTM.

Issues of SDTM Output by Academia

Regardless of the disease area, many items are easily standardized. These include the patient’s basic background items (such as date of birth and sex), adverse events, concomitant medications, clinical tests and vital signs, RECIST evaluation (oncology area only), and dropouts. However, current disease history, study intervention, and special tests for efficacy evaluation depend largely on the disease and the characteristics of the clinical study, and complete standardization is difficult.¹³ The items that are difficult to standardize are also the items where SDTM mapping is difficult. Advanced knowledge of CDISC standards is required to perform these mappings. Therefore, advanced education on CDISC standards is essential for the staff.

We saved time and costs for the CDISC-study by obtaining scientific research grants. As we had prior knowledge of CDISC standards, we could map items on CRFs to SDTM variables when we started to work on Ptosh. However, it is conceivable that sparing the required time and costs is difficult for most academic institutions, especially in Japan.

The first author is a clinical data manager and has previously worked as a system engineer, having acquired knowledge of programming and relational databases. However, it may be difficult for staff with only user-level knowledge of the system to be ready to work immediately after CDISC training. Therefore, it is reasonable to consider hiring those who have a good working knowledge and experience with databases and IT systems.

It is difficult to establish an employment quota for such human resources. In academia, there are some organizations in which medical staff simultaneously serve as data managers because these organizations belong to a hospital; they may not have an employment quota for the data management department unless they are medical staff. Also, there are many organizations that have employees on short-term contracts, eg, for up to five years. It is futile for the department to educate such employees, given that they will leave the organization after the contract expires, and by that time they may have become full-fledged CDISC experts.

We hope that academia as a whole will take a top-down approach in introducing CDISC standards from the perspective of anticipating future standardization and labor savings, and in securing research funding and staff, as well as staff training. In addition to Ptosh, other SDTM output tools are becoming available for SDTM dataset output in academia. For example, REDCap is an EDC system developed at Vanderbilt University in the US and widely used in academia worldwide. It does not have the capability to output SDTM by itself, but REDCap2SDTM, an SDTM dataset output add-on program, has been developed by Osaka University (now transferred to Osaka City University).²² In this way, an environment for CDISC compliance in academia is being arranged.

To spread the use of CDISC standards in academia, it is important to ensure that they are fully used, sufficient education regarding CDISC is provided, and costs are secured to build and use the system.

ACKNOWLEDGMENTS

The authors express their gratitude to Matsuo Yamamoto and Kaori Nagai of the Clinical Research Center of National Hospital Organization Nagoya Medical Center for their support concerning Ptosh.

Although we did not use DEMAND in the CDISC-study, we received a significant amount of cooperation at the start of the CDISC-study. We thank Atsushi Imamura, Noriyuki Suzuki, and Eiichi Ando of Densuke Systems Co., Ltd. (as of the start of the CDISC-study in 2015).

For the ALL-RET trial, we express our gratitude to Dr. Seiji Yano and Dr. Shinji Takeuchi from the Cancer Research Institute of Kanazawa University for their cooperation in implementing the CDISC. We also thank Prof. Kenichi Yoshimura from the Medical Center for Translational and Clinical Research, Hiroshima University Hospital, and Prof. Satoshi Teramukai of the Department of Biostatistics, Kyoto Prefectural University of Medicine Graduate School of Medical Science, for the guidance provided in the CDISC-study. We appreciate Nanayo Aoki, Mayumi Noda, Yukiko Sawamoto, and Nahoko Okitani of the Innovative Clinical Research Center, Kanazawa University, for the feedback on the data center operation task for the ALL-RET trial.

CONFLICTS OF INTEREST STATEMENT

The authors have no conflicts of interest to disclose.

FUNDING STATEMENT

The CDISC-study was performed using grant-in-aid research (Grant-in-Aid for Scientific Research on Innovative Areas JP15K15250) from the Japan Society for the Promotion of Science.

Abbreviations

ALL-RET trial

a phase I/II, open-label, single-arm study of CH5424802 for patients with advanced non-small-cell lung cancer harboring a RET fusion gene

CDISC

Clinical Data Interchange Standards Consortium

CDMS

clinical data management system

CRF

case report form

EDC

electronic data capture

NDA

new drug application

P21C

Pinnacle21 Community V3.0.1

PMDA

Pharmaceuticals and Medical Devices Agency

RECIST

response evaluation criteria in solid tumors

SDTM

Study Data Tabulation Model