Survey of Risk-Based Quality Management Status and Establishment of Operational Model in Clinical Trials

Abstract

Purpose

With the release of the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use E6 addendum, it is very important to establish risk-based quality management systems which meet good clinical practice. The aim of this research was to propose for risk-based quality management practices in the organizations that conduct clinical trials in South Korea.

Materials and Methods

The survey participant pool consisted of domestic or multinational pharmaceutical companies and fullservice clinical research organizations operating in South Korea that had one or more clinical trials approved by the Korean Ministry of Food and Drug Safety in 2018.

Results

Of the 97 selected companies, a total of 61 companies completed the survey. A total of 42 companies (68.9%) had employees designated to quality management activities. The minimum and maximum numbers of dedicated personnel for quality management were one and 12, respectively, and the average was three. Regarding the role of quality management personnel in companies, standard operating procedure management, issue/corrective action and preventive action (CAPA) management, and preparing inspection were selected the most (81%). The system considered to be the most important for risk management was monitoring system (41 companies, 67.2%), followed by both vendor management and CAPA management systems (17 companies, 27.9%).

Conclusion

In the future, organizations conducting clinical and subsequent pilot studies trials in South Korea should follow these quality management practice to share information with each other.

INTRODUCTION

The International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Good Clinical Practice (GCP) is a set of harmonized international standards for designing, conducting, performing, monitoring, auditing, recording, analyzing, and reporting clinical trials involving human subjects, introduced in 1996. In South Korea, the Korean Good Clinical Practice (KGCP) was enacted in 1995. In January 2001, the Korean government revised the KGCP based on ICH-GCP (E6-R1).

In 2016, the ICH introduced its revised international guidelines, the E6 (R2) Addendum to GCP, with 26 new standards.¹

The Korean Ministry of Food and Drug Safety (MFDS) officially joined the ICH in 2016; as a member of the ICH, which holds a major role in establishing global pharmaceutical guidelines, it is expected to have global expertise in the regulatory aspect. This also brought the need for meeting international standards for conducting clinical trials in South Korea. Therefore, relevant regulations have been significantly enhanced in recent years, leading to an increased number of laws and requirements that must be fulfilled by the responsible parties, including contract research organizations (CROs) and investigative sites.

The part of a clinical trial process where the risk management approach was first adopted was monitoring.²,³ Many studies, especially those conducted in developed countries such as the U.S., have shown that the implementation of a risk management system in the monitoring process is more effective than full on-site monitoring in identifying data anomalies (e.g., data falsification, fabrication).⁴,⁵,⁶,⁷,⁸,⁹,¹⁰

The current status of activities for quality management in companies conducting clinical trials in South Korea has not been determined. Therefore, investigating current practices in implementing the quality management systems of these companies, as well as the discussion of building a sample risk management tool, should be done.

This research aimed to conduct a survey for the current risk management practices in organizations conducting clinical trials in South Korea and identify the needs of people working in this field. Based on these data, a sample risk management model for the sponsors or CROs was proposed, so that they can proactively adapt to the changing regulatory environment by establishing their risk management system that complies with global standards, and the overall quality of clinical trials conducted in South Korea can be improved.

MATERIALS AND METHODS

Online survey

The survey was designed to gather information about current staffing levels and quality management practices of domestic and multinational pharmaceutical companies and CROs conducting clinical trials in South Korea. The questionnaire included questions on tasks for clinical trial quality, personnel designated to these tasks, operation of quality management systems, implementation of risk-based monitoring (RBM), and its effectiveness. The total number of questions was limited to 18, in order to encourage participation.

The questionnaire was finalized after consultative meetings. Email invitations for the survey were sent on June 17, 2019, and the survey was closed on June 26, 2019.

Survey method

An online survey tool called SurveyMonkey (www.surveymonkey.co.uk) and Google forms (docs.google.com/forms) were used to manage the survey data effectively. The survey participant pool consisted of domestic or multinational pharmaceutical companies and CROs operating in South Korea that had one or more clinical trials approved by the Korean MFDS in 2018. The downloaded file contained the status of clinical trials, sponsor names, product names, title of clinical trials, phase of clinical trials, region of drug development, institution names, and approval dates.¹¹ The specific list of these companies was downloaded from the official pharmaceutical integrated intelligence system of Korean MFDS.

Questionnaire

The questionnaire consisted of 18 questions, and the followings insights were expected from the survey: 1) type of company, 2) number of clinical trials conducted in 2018 and relevant personnel in South Korea, 3) tasks related to quality management, 4) personnel and department designated to quality management activities, 5) quality management outsourcing and possible area for improvement, and 6) risk-based quality management practices, to examine the current status of quality management and implementation of risk-based quality management in different types of companies (Supplementary Material, only online).

Statistical method

The statistical tests were two-sided, and the level of significance was set at 0.05. All categorical variables were described by number and percentages in parentheses. The summary statistics were presented by respondents. Fisher's exact test was used to analyze the difference in the distribution of respondents. All analyses were performed using a statistical software package (SAS, version 9.4; SAS Institute; Cary, NC, USA).

RESULTS

Survey results

Among the companies that could be collected on the Pharmaceutical Integrated intelligence system website, a total of 97 companies were selected. The invitations were sent to only one representative for each company to prevent inconsistent answers. Sixty-one out of 97 companies completed the survey, resulting in a response rate of 62.9%. The rates of domestic and multinational pharmaceutical companies among the non-responders were similar to each other (41.7% for domestic and 38.9% for multinational). The type of companies selected for the survey's pool of participants were domestic and multinational pharmaceutical companies and CROs operating in South Korea. The highest number of 15 companies (24.6%) answered to have had six to 10 clinical trials conducted or terminated in South Korea in 2018. The minimum and maximum numbers of personnel who were involved in the clinical trial activities (for example, clinical operation, data management, and biostatistics) in South Korea were three and 320, respectively, and the mean value was 49. The minimum and maximum numbers of dedicated quality management personnel were one and 12, respectively, and the average number was three (Table 1).

Table 1. Characteristics of Respondents (Companies).

Variables	Number of responders
Response rate (n=97)
Completed	61 (62.9)
Not submitted	32 (3.0)
Mail system error	3 (3.1)
Refusal to answer	1 (1.0)
Type of company (n=61)
Domestic pharma	20 (32.8)
Multinational pharma	16 (26.2)
Domestic CRO	14 (23.0)
Multinational CRO	11 (18.0)
Job characteristics of responders (n=61)
Quality assurance	28 (45.9)
Clinical operation	28 (45.9)
Management	3 (4.9)
Start-up task	1 (1.6)
Medical monitor	1 (1.6)
No. of clinical trials conducted (phases I to III) (n=61)
1 to 5	13 (21.3)
6 to 10	15 (24.6)
11 to 20	13 (21.3)
21 to 30	7 (11.5)
31 to 50	6 (9.8)
51 or more	7 (11.5)
No. of personnel involved in clinical trial activities (n=61)	dd
Mean	49
Median (min., max.)	30 (3, 320)
No. of dedicated quality management personnel (n=61)
Mean	3
Median (min., max.)	2 (1, 12)

CRO, contract research organization.

Data are presented as number (%).

Twenty-three companies (54.8%) responded that quality management departments or teams were fully independent functions, while 17 companies (40.5%) responded that they were independent of all operations but reported at the same level as clinical business (clinical operation department). Other responses for reporting structure included the following: “there is a separate team of auditors responsible for quality assurance (QA) and a clinical operation team responsible for quality management,” and “the quality management personnel is independent of all operations but sometimes they report at the same level as clinical operation department.” Regarding the role of quality management personnel, the most commonly selected options were standard operating procedure (SOP) management, issue/CAPA management, preparing inspection (34 companies, 81%), followed by non-compliance management (33 companies, 78.6%) and then individual trial audit (30 companies, 71.4%) (Fig. 1).

Fig. 1. Role of quality management personnel. SOP, standard operating procedure; CAPA, corrective action and preventive action; QA, quality assurance.

Only 22 companies (36.1%) replied that they were currently outsourcing some quality management tasks to a third party (Fig. 2).

Fig. 2. Outsourced quality management tasks. SOP, standard operating procedure.

The most frequently selected reason was “lack of experts in the company” with 13 companies (59.1%), followed by “flexible personnel management” with six companies (27.3%).

The distribution of respondents based on company types showed a difference in the response rate, and all variables were statistically significant (p-value<0.05) as shown in Table 2. A total of 42 companies (68.9%) had employees designated to quality management activities, and 11 (18.0%) companies had plans for recruiting in the future, with 86.9% of the companies showing positive attitudes towards hiring quality management personnel. A total of 27 companies (44.3%) answered to have undergone some type of risk assessment before starting a clinical trial. A total of 28 companies (45.9%) answered to have implemented risk assessment for all or partial clinical trial systems. The top three systems that companies consider to be the most important for risk management were monitoring system (41 companies, 67.2%), vendor management system (17 companies, 27.9%), and CAPA management system (17 companies, 27.9%). A total of 30 companies (49.2%) adopted RBM, and 17 companies (27.9%) were in progress of adopting RBM. The RBM implementation rate of multinational organizations outnumbered that of domestic organizations (Table 2).

Table 2. Quality Management and Risk Management by Company Type.

Variables	Number of responders (n=61)					p value*
	All	Domestic pharma	Multinational pharma	Domestic CRO	Multinational CRO
Personnel dedicated to quality management						0.0081
Yes	42 (68.9)	11 (55)	12 (75)	12 (85.7)	7 (63.6)
No, and have no plans to recruit	8 (13.1)	1 (5)	4 (25)	0 (0)	3 (27.3)
No, but have plans to recruit	11 (18.0)	8 (40)	0 (0)	2 (14.3)	1 (9.1)
Risk assessment before clinical trials						<0.0001
Yes	27 (44.3)	7 (35)	11 (68.8)	3 (21.4)	6 (54.5)
No	18 (29.5)	6 (30)	4 (25)	6 (42.9)	2 (18.2)
No but in preparation	16 (26.2)	7 (35)	1 (6.3)	5 (35.7)	3 (27.3)
Risk assessment for clinical trial systems						<0.0001
Yes	20 (32.8)	4 (20)	10 (62.5)	1 (7.1)	5 (45.5)
No	22 (36.1)	9 (45)	3 (18.8)	8 (57.1)	2 (18.2)
Yes, but only for some parts	8 (13.1)	2 (10)	3 (18.8)	2 (14.3)	1 (9.1)
No, but in preparation	11 (18.0)	5 (25)	0 (0)	3 (21.4)	3 (27.3)
Prioritized systems for risk assessment						<0.0001
Monitoring system	41 (67.2)	10 (27.0)	11 (34.4)	10 (35.7)	10 (47.6)
Vendor management system	17 (27.9)	8 (21.6)	6 (18.8)	2 (7.1)	1 (4.8)
CAPA management system	17 (27.9)	7 (18.9)	2 (6.3)	3 (10.7)	5 (23.8)
SOP management system	14 (23.0)	4 (10.8)	4 (12.5)	5 (17.9)	1 (4.8)
Document management system	14 (23.0)	4 (10.8)	3 (9.4)	5 (17.9)	2 (9.5)
Personnel and training management system	5 (8.2)	1 (2.7)	1 (3.1)	1 (7.1)	2 (9.5)
IP management system	5 (8.2)	2 (5.4)	3 (9.4)	0 (0)	0 (0)
Computerized system	5 (8.2)	1 (2.7)	2 (6.3)	2 (7.1)	0 (0)
Risk-based monitoring						<0.0001
Yes	30 (49.2)	6 (30)	12 (75.0)	5 (35.7)	7 (63.6)
No	14 (23.0)	8 (40)	1 (6.3)	5 (35.7)	0 (0)
No but in preparation	17 (27.9)	6 (30)	3 (18.8)	4 (28.6)	4 (36.4)

CRO, contract research organization; CAPA, corrective action and preventive action; SOP, standard operating procedure; IP, investigational product.

Data are presented as number (%).

^*p-value derived from Fisher's exact test.

Regarding the required elements to improve the quality of clinical trials, the most favored option was “executive officers' and employees' better perspectives on quality management” (27 companies, 44.3%). Out of these 27 companies, 14 were domestic pharmaceutical companies, five were multinational pharmaceutical companies, seven were domestic, and one was a multinational CRO. Other opinions included: “actual investment for building and improving quality system determined by executive officers”; “stable staffing levels of clinical operation with high turnover rate”; and “operating electric system for quality management including computerized system (e-learning system, CAPA management system, etc.).” For the question asking about the reason for not undertaking risk assessment before starting a clinical trial, 23 companies (67.6%) chose “lack of understanding or information on the specific procedure,” and 16 companies (47.1%) selected “relevant regulations not established yet.” The “relevant regulations” stated here referred to ICH E6 (R2) with more specific procedures and requirements to meet the standards of Korean GCP. Out of the 29 companies that have implemented RBM, most respondents (20 companies, 69.0%) answered that they have been using RBM in some clinical trials (Phases I to III). Out of the companies that have implemented RBM, 10 companies (33.3%) answered that RBM is effective in terms of productivity, and six companies (20.0%) responded that RBM is cost-effective. The rest of the companies (14 companies, 46.7%) did not determine the effectiveness of RBM (Table 3).

Table 3. Quality Management and RBM.

Variables	Number of responders
Types of quality management tasks (n=61)
SOP management	49 (80.3)
Issue/CAPA management	47 (77.0)
Training management	46 (75.4)
Individual trial quality management	46 (75.4)
Vendor oversight and management	46 (75.4)
Non-compliance management	45 (73.8)
Preparing inspection	45 (73.8)
Individual trial audit	39 (63.9)
System audit	33 (54.1)
Quality assurance system set-up	30 (49.2)
Improvement of process though consultation	25 (41.0)
Computerized system validation management	14 (23.0)
Elements to improve clinical trial quality (n=61)
Executive officers’ and employees’ better perspectives on quality management	27 (44.3)
Increasing quality management staffing levels	25 (41.0)
Systemic non-compliance management	18 (29.5)
Enhancing quality management of individual clinical trials	16 (26.2)
Development, compliance, and continuous revision of SOPs	9 (14.8)
Increasing training opportunity	4 (6.6)
Enhancing audit of individual clinical trial	4 (6.6)
Systemic vendor oversight and management	4 (6.6)
Other	4 (6.6)
Reason for not undertaking risk assessment (n=34)
Lack of understanding or information on the specific procedure	23 (67.6)
Relevant regulations not established yet	16 (47.1)
No corresponding personnel	11 (32.4)
Poor understanding among employees of risk assessment and management	7 (20.6)
Other (not applicable)	2 (5.9)
Extent of RBM implementation (n=29)
All clinical trials (phases I to III)	9 (31.0)
Some clinical trials (phases I to III)	20 (69.0)
Effectiveness of RBM (n=30)
Productivity	10 (33.3)
Cost effectiveness	6 (20.0)
Not effective	0 (0)
Don’t know yet	14 (46.7)

RBM, risk-based monitoring; SOP, standard operating procedure; CAPA, corrective action and preventive action.

Data are presented as number (%).

Development of risk management tool for clinical trial systems

Monitoring system

Many risk assessment tools for monitoring have been developed by various organizations and companies, including TransCelerate, as their monitoring systems were the most preferred for implementing risk assessment.¹² However, these existing tools are more appropriate for large companies with systemic computerized system and insider experts, such as statistical experts and medical experts, rather than domestic companies in South Korea that are relatively small compared to multinational companies. For small companies with limited physical and human resources, attempting to establish a RBM system might be difficult. In this research, we developed a simplified risk-based management tool for monitoring system by modifying the existing RBM risk index library tool developed by TransCelerate, so that small companies with limited resources and inefficient computerized system can implement RBM to their monitoring system more easily. This tool comes with several examples to facilitate understanding of the presented risk indicators for organizations that are unfamiliar to this concept. The tool itself cannot be used to implement the RBM system right away; the users must add, delete, or modify the risk indicators included in this tool according to their organization or a specific clinical trial. To accomplish this, certain tolerance limits for each risk indicator should be established based on the company's own experience in conducting clinical trials and relevant reference documents. Also, each risk indicator included in the RBM tool should be checked on a predefined cycle and changed accordingly, if needed. These procedures should be defined in detail in a Risk Based Monitoring Plan (RBMP) before a monitoring visit, and must be followed when conducting the monitoring visit. In the tool generated in this research, the risks were categorized into eight areas: 1) clinical research associate/site workload (on-site and remote monitoring); 2) data quality; 3) essential documents; 4) investigational products; 5) issue management; 6) safety; 7) site staff, facilities, and equipment; and 8) subject recruitment and dropout. There are sub-categories under each main category, and corresponding risk indicators can be assigned to each sub-category. The definition of each risk indicator is also provided. The risk indicators can be grouped as either general risk or trialspecific risk, based on their definitions.

Vendor management system

Various relevant documents were reviewed to develop a tool for a vendor management system, which was the second most preferred system for implementing risk assessment. The risk indicators provided in this tool were derived from a paper titled “Current practice and perspective in CRO oversight based on a survey performed among members of the German Association of Research-Based Pharmaceutical Companies (vfa).”¹³ In this tool, the risks were divided into four categories: 1) vendor selection; 2) outsourcing model; 3) outsourcing areas; and 4) vendor oversight. There are sub-categories under each main category, and the risk indicators corresponding to each sub-category can be added, deleted, or modified. The scoring scale for risk indicators should be determined based on internal standards for risk management.

CAPA management system

The third-most popular option for implementing risk assessment was developing a tool for CAPA management system. To this end, a variety of relevant documents were reviewed. The risk indicators provided in this tool were derived from the CAPA system specified in the CH Q8 and Handbook of investigation and effective CAPA systems.¹⁴ In this tool, the risks were divided into four categories: 1) problem detection & source; 2) initial impact assessment; 3) problem investigation; and 4) CAPA plan. There are sub-categories under each main category, and the risk indicators corresponding to each sub-category can be added, deleted, or modified. The scoring scale for risk indicators should be determined based upon internal standards for risk management.

DISCUSSION

This research aimed to investigate the current risk management practices in the pharmaceutical industry, including CROs in South Korea, as well as to provide a sample risk management tool useful for sponsors or CROs.

There is a lot of information on the quantitative analysis of clinical trials from various studies and documents. However, no single qualitative study about clinical trial evaluation (e.g. QA staffing levels in sponsors or CROs) has been conducted in South Korea, not to mention a lack of understanding about what should be examined to evaluate the current quality management practices. In this research, a survey was conducted to investigate and analyze the current quality management practices. The survey questionnaire consisted of a series of questions about the number of clinical trials conducted as well as staffing levels, quality management tasks, dedicated personnel performing these tasks, operation of quality management systems, implementation of RBM and its effectiveness, and so on.

As for the size of the QA unit, the GCP QA in a German survey was similar to the average for companies demonstrated in the current study. However, this study's results on the independence of the reporting line was higher than the results of the German study.¹⁵ Clinical QA activity was similar in both surveys in terms of auditing, non-compliance investigations, regulatory inspections, GCP/compliance consultancy, process improvement activities, training, SOP management, vendor oversight, and computer system validation.¹⁶ In a U.S. survey from 2009, the percentage (71%) of companies that conducted risk assessment prior to monitoring was higher than the South Korean result.¹⁷ The percentage of companies that were currently using RBMs in this survey was less than the percentage demonstrated in a survey of international organizations in 2018.¹⁸

However, this study had a few limitations. Differences in opinions on the application of quality improvement of clinical trials could not be collected from companies, due to different circumstances depending on the type of organizations (e.g., domestic vs. multinational or pharmaceutical company vs. CROs). While domestic companies have to consider introducing new systems such as risk management systems, multinational companies are different because they have to apply the systems that they receive from their headquarters to the South Korean situation.

The current version of KGCP was released on December 6, 2019, and it has not been adapted to the ICH E6 (R2) addendum. The ICH E6 (R2) describes the basic concept of the risk management process, and states that risks should be considered at both the system level (e.g., SOPs, computerized systems, and personnel) and clinical trial level (e.g., trial design, data collection, and informed consent process). To detect potential internal problems or weaknesses in an organization, risk management needs to be conducted at both levels as early as possible. Unlike ICH E6 (R2), KGCP is legally binding; therefore, the process of conducting risk management as well as the results from this should be documented for inspection. This is why in-depth discussion on the readiness and knowledge of the organizations conducting clinical trials in South Korea is necessary prior to updating KGCP. The starting point of this research was this very need for investigating the current status of the clinical trial industry related to risk management systems in South Korea; and proposing initiatives to implement the risk management system. To ensure that the risk-based approach is adopted by many organizations, ranging from small to large businesses in South Korea, a clear national guidance and minimum compliance requirements must be outlined.

In conclusion, we recognized the need for a model for quality management of clinical trials in South Korea and found that a practical model is needed accordingly. It was also noted that it was necessary to raise the perspective of quality management by employees and the executive officer of an organization for quality improvement, and it is important to train experts in GCP QA continuously. In the future, the quality management practices of organizations conducting clinical trials in South Korea and subsequent pilot studies should be followed to share information between organizations.