The development and analysis of a Japanese modern comprehensive clinical data management training program

Abstract

Background

Clinical data management (CDM) collects, integrates, and makes data available. It plays a vital role in clinical research. However, there are few opportunities for Japanese clinical data managers to learn about its systematic framework, particularly in academic research organizations. While Japanese-language CDM training exists, its effectiveness in a Japanese context requires clarification.

Objectives

We aimed to develop an advanced program of instruction for professionals to understand CDM and to determine the effectiveness of the training program.

Methods and results

We developed an advanced program including risk-based monitoring and the Clinical Data Interchange Standards Consortium on a trial basis for clinical data managers to provide them with a comprehensive understanding of CDM. Fifty-two people attended the program and reported that they were highly satisfied with it.

Conclusions

To provide comprehensive CDM training in Japan, it is imperative to continue improving the content and develop an advanced program. Due to the recent tightening of clinical research regulations and the development and dissemination of various systems for conducting clinical research, the competency-based educational program requires further development.

1. Introduction

Over the past several decades, different organizations have worked to standardize and improve clinical data management (CDM). In 1967, the Greenberg report underlined the need to monitor and control performance in large multicenter clinical studies [1]; however, it provided little information regarding clinical research data management methods. The systemic framework of CDM is integral for developing pharmaceutical and medical devices. In 2016, the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) sought to enhance efficiency and quality by implementing a Quality Management System (QMS) in ICH-E6 (R2) [2]. Currently, using real-world data such as electronic hospital records to capture patients' real-time data and treatment outcomes has become crucial for pharmacovigilance, safety, developing regulations, clinical development, and medical processes [3]. Meanwhile, regulatory authorities must submit clinical trial data in the standard Clinical Data Interchange Standards Consortium (CDISC) format. That way, data can be electronically compiled while applying for a new drug's approval, and the metadata can be assessed. Moreover, the information can inform healthcare policies [4].

The reference book Good Clinical Data Management Practices (GDCMP©) [5] has been published and revised by the Society for Clinical Data Management (SCDM). It is extensively used in industries and academic research organizations (AROs) that conduct training courses for data managers. It provides a reference to clinical data managers (CDMS) in their implementation of high quality CDM processes and is used as a guidance tool for CDMS when preparing for CDM training and education. It is believed that CDMS in Japan will also benefit from using this book. Abouelenein et al. idenftified 67 professional competencies for CDMS and a need for training in methods and concepts that could be applied across therapeutic areas and types of data [6].

In the U.S., continuous efforts have been made to augment CDM, including implementing the Clinical and Translational Science Award (CTSA) program in approximately 60 institutions across the country [7]. CDMS in Japanese AROs encounter unique challenges, including lower budgets, a lack of experienced personnel, inadequate infrastructure, a lack of control over protocol amendments, excessive data collection, and unreliable schedules. Cumulatively, these factors adversely affect the quality and efficiency of the clinical trials conducted in academic institutions.

In Japan, the budget for clinical research funded by the Japan Agency for Medical Research and Development (AMED) is small [8]. Additionally, regulations related to clinical research have been strengthened, and the Clinical Trials Act [9] has been enforced since April 2018. Consequently, CDMS in Japan have encountered additional challenges after introducing QMS— including a risk-based approach to monitoring— and mandated implementation of CDISC standards and advanced technologies. However, there are fewer opportunities for Japanese CDMS to educate themselves regarding the CDM's systematic framework, particularly in AROs. Only a few universities including Tohoku University offer systematic CDM education. Since 2007, the “New 5 Yearly Clinical Trial Activation Plan” [10], developed by the Ministry of Health, Labor and Welfare, was utilized to develop a CDM training workshop. Since 2016, we have conducted annual CDMS training programs with the support of AMED. However, the effectiveness of the CDM training program and human resources development for the foundation and development of clinical research in Japanese AROs must be clarified.

This study aimed to develop an advanced program on a trial basis for CDMS to comprehensively understand CDM in Japan and quantitatively determine the training program's effectiveness.

2. Methods

2.1. Program participants’ application requirements

In Japan, on-the-job training is the primary education and training method for all personnel in clinical research. The acquisition of systematic and basic knowledge about clinical research varies by organization and personnel during on-the-job training. Therefore, those who engage in CDM need basic knowledge. To operate smoothly, joint recognition of the work of CDMS by collaborators in clinical trials other than CDMS is required. Hence, the inclusion criteria were as follows.

• 1.Each ARO that belonged to a medical institution (in principle, to satisfy criteria a–c)
• (a)have had a track record of clinical research (including trials) in recent years
• (b)ensure that novices have been trained during their preliminary period; and
• (c)expect individuals who have completed their training to perform data management work.
• 2.They must have belonged to the participant's medical institution and satisfy the following: individuals who have engaged in data management work for clinical trials and studies and those who planned to engage with these in the future.

2.2. Training program content

We developed a two-day training program on a trial basis by a committee of experts (facilitators). The content covered the basic CDM procedures outlined in the GCDMP, CDISC information, and other biostatistics textbooks. Participants were divided into several groups, each assigned a facilitator, and all lectures and group seminars were conducted in person.

2.3. Questionnaire survey

We conducted a questionnaire survey, originally developed and manually distributed for this study, to assess the participants' degree of understanding of each lecture before and after the two-day training program (see Supplementary for details). A follow-up survey was conducted seven months later as well. The participants answered questions about their degree of understanding of the 11 processes associated with data management before and after the training on a four-point scale (could not imagine, could imagine, could understand, and could explain). The questionnaire items consisted of the following items: (1) importance of CDM; (2) data quality, source documentation, data entry process; (3) documentation and records management; (4) case report form (CRF) design; (5) CDISC; (6) database validation; (7) data cleaning; (8) data review; (9) RBM; (10) safety data management and coding; and (11) database closure. Seven months after the program, we confirmed the long-term educational effects using a follow-up questionnaire survey regarding the participants' current job situations and tips on our CDM training program still in use. The ethical review was waived because the educational program was not subject to Japan's Ethical Guidelines for Life Sciences and Medical Research Involving Human Subjects. All participants agreed to complete the self-assessment questionnaires to examine their understanding of the program content.

2.4. Statistical analysis

Background information on participants was summarized in terms of their institutional affiliation (university hospital, national hospital organization, or national center hospital), job title, and length of time working in clinical trials and as CDMS. For the short-term educational effectiveness of the training programs, we assessed the change in the participants' degree of understanding of the 11 CDM procedures based on the questionnaire survey using Bowker's test. We described whether the training is useful in business procedures using the follow-up questionnaire survey data. JMP PRO version 12.0 was used for the analyses. We set the significance level of statistical tests at 0.05. Due to the study's exploratory nature, statistical adjustment for multiplicity was not considered.

3. Results

The training was held in November 2017. Table 1 presents the contents of the lectures and seminars (see Supplementary for details). During group seminars, a simulated protocol based on the actual study, “A multicenter, parallel-group, placebo-controlled, double-blind, confirmatory trial of intranasal oxytocin in participants with autism spectrum disorders” [11] was used. The table, list, figure, CRF, and electronic patient-reported outcomes were designed in one seminar, and risks within the trial were evaluated in the other seminar.

Table 1.

The data management training program in 2017.

Day 1
Planning Clinical Trials
GCP and Quality Control
CRF Design and Database
Data Cleaning and Reliability
Safety Data Management
Clinical Data Management: History and Future
Group Seminar: CRF Design and Tables, Listings and Figures (TLF) Development
Day 2
Statistical Data Review
Data Review Example
Collaboration in RBM
Group Seminar: Risk assessment
CDISC
General Discussions
(Summary)

Note: The training program comprised 11 lectures and 2 group seminars in two days. Renowned experts in each field gave lectures.

Of the 52 participants enrolled nationwide, 58% worked as CDMS and 13% as clinical research coordinators (CRCs). Nineteen percent of participants had less than one year of CDM experience, 25% had between one and three years, 19% had between three and ten years, and 21% had more than ten years. Overall, 50 participants responded to our short-term questionnaires, of whom 72% and 84% were “satisfied” with the lectures and group seminars, respectively (Fig. 1). Using the Bowker's test, the degree of understanding of each concept was markedly enhanced in 10 out of 11 lectures (Table 2). In the follow-up questionnaire survey (seven months after the training course) conducted to assess the effectiveness of the CDM training course, 25 participants responded to our questionnaires. The distribution of occupations remained unchanged in the follow-up survey; no one was transferred from the CRC or Institutional Review Board staff positions to CDMS. Overall, 21 participants (84%) reported that our training course was still “very useful” or “useful” in their current business procedures. However, 18 CDMS and CRCs reported that they continued encountering challenges with CDM procedures and were willing to join sustainable and advanced training courses for CDMS.

Fig. 1.

Satisfaction with the training.

Table 2.

Educational effects and representative results (CDISC & RBM).

4. Discussion

The effectiveness of CDM training programs in Japan had not been determined. We developed an advanced program including RBM and the CDISC for CDMS to provide them with a comprehensive understanding of CDM. This study demonstrates that we provided high-quality training, as evidenced by participants’ high satisfaction rates. Given the current trend to promote and build foundations for clinical trials, we anticipate an increase in the number of professionals, such as CDMS [12], clinical trial managers [13], etc., which could promote human resource development if we continue to conduct this basic training. The following factors could have influenced the high satisfaction rate reported in this study: (1) the experts for each data management process gave lectures; (2) participants engaged in a practical group seminar using a simulated protocol; and (3) participants actively discussed how CDMS operates at other facilities. Moreover, participants from various professions talked about daily concerns with CDM and the roles required of CDMS. Notably, they cannot typically have these conversations with staff at their facility. Nevertheless, it was challenging to provide professional knowledge and CDISC skills in a 1-h lecture; therefore, it is critical to participate in the CDISC official training held by the CDISC Japan User Group.

Although our training was for CDMS, it might benefit clinical research to develop curricula for professionals other than CDMS to elucidate CDM procedures. Additionally, this might help CDMS understand other professionals. In this study, the follow-up questionnaires revealed the long-term efficacy of this training and details of the impediments and challenges in the data management processes for each participant's current situation. Within the framework of ICH-E6 (R2) [2], quality management produces reliable trial results efficiently. Therefore, implementation of the QMS is required. Quality based on the design concept enables efficient quality control activities focusing on the quality of trials, especially in academic medical institutions with limited resources. With the recent tightening of clinical research regulations and the development and dissemination of various systems for conducting clinical research, such as electronic data capture, electronic patient-reported outcomes, and electronic sources [14], there is a need for professionals who can flexibly use QMS-based thinking. Hence, we need to consider these concepts for advanced courses.

Furthermore, GCDMP, considered a reference for methodology, is currently undergoing revision [15] and will include more evidence-based scientific CDM. Moreover, CDM education in academia and other industries in Japan must not miss this trend. Recently, the Joint Task Force for Clinical Trials (checking for competency) issued the core competencies of various clinical research professionals [16], and the SCDM previously reported on the competencies and basic knowledge required by the CDMS during the annual meetings (SCDM competency) [17]. Therefore, although the recommendations in this study concern the trial CDM educational program in Japan, we need to develop a competency-based educational program for CDMS. Japan's "language barrier" is a major detriment in providing knowledge that matches global standards and is scientifically sufficient for CDMS. We believe that completing the introduction and basic knowledge and skills in one's native language (here, Japanese), followed by career development toward obtaining the SCDM's Certified Clinical Data Manager (CCDM®), is necessary to support engagement with CDMS in Japan and worldwide. In this sense, we believe it is very significant to cooperate with the SCDM headquarters and its Japanese branch.

This study had several limitations. The number of participants in this program was not large, and this study was conducted in a single setting. However, it was a reasonable number considering the number of potential CDMS in Japan and that their backgrounds were applicable to the Japanese context. We might encourage participants to consider diving deeper into CDISC and RBM rather than interacting at a surface level. Moreover, the training content should have been determined based on competencies. We surveyed the participants’ satisfaction and understanding based on the 11 CDM items before and after the program. Then, we surveyed the participants seven months later to see if the training was useful in their current work, but we might need to examine other measures and long-term effects.

CDMS handle diversified data sources in clinical research and advanced research design, data collection, and analysis methods. Moreover, the scope of work handled by CDMS and the required skills have changed significantly. CDMS need to understand the changes that are underway and follow the essential criteria of CDM. We believe our comprehensive training program could help guide future development in CDM and clinical data science, an evolution of CDM.

5. Conclusions

The proposed training program introduced a method to achieve optimization by adding cross-professional stakeholders, and not just CDMS, based on the ICH-E6(R2) QMS concept. Participants learned practical skills and concepts by implementing the idea of risk-based quality control and risk assessment into the CDM process flow. The introduction of content to enable CDISC implementation in academia also occurred. This study reveals that we can provide comprehensive data management training in Japan; however, it is imperative to continue improving the content and develop an advanced program. The development of a competency-based educational program for CDMS needs to be considered.

Ethics declarations

Review and/or approval by an ethics committee was not needed for this study. The ethical review was waived because the educational program was not subject to the Ethical Guidelines for Life Sciences and Medical Research Involving Human Subjects in Japan. All participants agreed to complete the self-assessment questionnaires to examine their understanding of the contents of the program.

Funding

The AMED supported this training program under Grant Number JP15km0908001.

Additional information

The Data Management Training Program and the Questionnaire for Program evaluation are attached in the supplementary.

Data availability statement

Questionnaire survey data were obtained solely for the evaluation of this program. The data associated with our study have not been deposited in publicly available repositories. The authors do not have permission to share the data.

CRediT authorship contribution statement

Takuhiro Yamaguchi: Writing – review & editing, Writing – original draft, Supervision. Hiroko Yaegashi: Writing – original draft, Visualization, Methodology. Shih-Wei Chiu: Formal analysis. Yukari Uemura: Project administration. Takuya Kawahara: Project administration. Tempei Miyaji: Project administration. Tomoe Mashiko: Project administration. Munenori Takata: Writing – review & editing, Writing – original draft, Supervision, Methodology, Investigation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix ASupplementary data

The following are the Supplementary data to this article.