Establishing an evaluation index system of nursing quality for clinical drug trials: A Delphi study

Abstract

Aims

To construct a quality evaluation index system for clinical drug trials nursing management and obtain the weight of all indicators.

Design

A mixed‐method research design with a quantitative component was used, primarily qualitative.

Methods

Through a literature review and semi‐structured interview, an expert consultation questionnaire on the quality of nursing evaluation indicators for clinical drug trials was developed in April 2021. Eighteen experts in clinical drug trial nursing, medical, and pharmacy conducted 2 rounds of consultation according to the Delphi method to determine the indicators for evaluating the quality of clinical drug trial nursing. The weights of each indicator were determined using analytic hierarchical analysis.

Results

The established quality evaluation system of clinical drug trial nursing mainly includes 3 first‐level indicators, 12 second‐level indicators, and 59 third‐level indicators. The positive coefficients of the two rounds of expert consultation were 90%–100%, and the authority coefficients were 0.831 and 0.885, respectively; the coordination coefficients were 0.189 and 0.214, respectively. The consulting results and weight settings are reliable. The evaluation index system we constructed is in line with the characteristics of the clinical drug trial nursing profession, with clear index levels and strong clinical operability, which can provide a reference for the assessment, monitoring and improvement of nursing quality in clinical drug trials. It will also clarify how nurses contribute to subjects' safety.

1. INTRODUCTION

The research and development of new drugs have a low success rate, high investment, long cycle, and high risk. According to the American Association of Pharmaceutical Research and Manufacturers, the cycle of developing new drugs in the United States is 10–15 years, and the average cost is US $0.8–1 billion, of which 70% are spent on clinical trials (Brixner et al., 2019). High‐quality clinical trials are the key to research data's authenticity, scientificity, and reliability. Moreover, it can reduce the cost and time spent on research, benefiting patients (Zhao et al., 2020). Correspondingly, China's relevant departments have put forward requirements for improving the quality of clinical trials (National Health Commission of the People's Republic of China, 2019). Nurses are an important part of the clinical drug trial team, and the work of nurses directly affects all aspects of clinical drug trials. Nursing quality is critical to the standardized management and quality control of the entire trial process (Spilsbury et al., 2008). Constructing the nursing quality evaluation system to improve quality in the medical speciality of clinical drug trial is important.

2. BACKGROUND

Clinical drug trials are an essential step in confirming the efficacy and safety of new drugs. During the trial, tolerance and pharmacokinetics of the human body to the new drug are observed (Federal Register USGPO, 2012). Carrying out clinical drug trials requires the cooperation of various professional personnel, and nurses are indispensable to that (Snyder, 1994). Nurses' work involves all aspects of clinical drug trials, including the determination of the protocol, the formulation of the standard operating procedure (SOP), the screening and management of the subjects, administration, monitoring and first aid, the collection and handover of biological samples, the recording of original documents, the sorting of data, etc. Therefore, nursing quality directly affects the accuracy of clinical trial results and subject safety.

Currently, each hospital department has gradually built its speciality quality evaluation system of nursing (Chen et al., 2017; Liu et al., 2020, 2021; Yang et al., 2021). The clinical drug trial centre is a highly specialized department. People who participate in drug trials are usually advanced cancers refractory to standard treatment, with a 90‐day mortality rate of more than 15% (Arkenau et al., 2008); sometimes, they are healthy subjects. Due to the specificity of the disease treatment method and the subject population, there is a big gap with other specialities regarding nursing service content and mode, nursing process, and cooperation between doctors, nurses, pharmacists, and patients. Thus, the nursing quality of clinical drug trials cannot be fully and accurately reflected by traditional indicators for assessing the nursing quality of general hospitals.

The majority of studies on clinical drug trials focussed on drug efficacy (Rheker et al., 2018), subject management (Haddad et al., 2015; Rabin & Tabak, 2006; Tinkler et al., 2018), or nursing role (Hong et al., 2021; McCabe et al., 2019). Nevertheless, there is no report on the nursing quality evaluation system in clinical drug trials. This study constructed a nursing quality evaluation system in clinical drug trials through literature review, semi‐structured interviews, and Delphi expert consultation. We intended to promote the continuous improvement of nursing quality in clinical drug trials, improving the efficiency and quality of clinical trials, and ensuring the safety of subjects.

3. THE STUDY

3.1. Aims and objectives

This study aimed to construct a quality evaluation index system for clinical drug trials nursing management and obtain the weight of all indicators to provide a reference for assessing, monitoring, and improving nursing quality in clinical drug trials.

4. METHODS

The Standards for Reporting Qualitative Research guideline was followed (File S1). A mixed‐method research design with a quantitative component was used, primarily qualitative. The literature review, semi‐structured interview, Delphi, and analytic hierarchy methods are used to construct a quality evaluation system (Figure 1).

FIGURE 1.

Flowchart of the study.

4.1. Preliminary construction of evaluation indicators

We established a working group, including 2 clinical drug trial nursing administrators, 1 research physician, 1 head nurse, and 2 research nurses. The working group is responsible for identifying and contacting experts for consultation, formulating expert consultation questionnaires, sorting out feedback, and statistical analysis of the information returned from the questionnaires.

Based on a systematic literature review, our working group initially developed evaluation indicators according to the existing guiding principles of clinical drug trials and the characteristics of nursing quality. Taking Donabedian's “structure‐process‐outcome” three‐dimensional quality structure model as the theoretical framework, the quality evaluation index system of clinical drug trial nursing was categorized and summarized (Hasson et al., 2000).

4.2. Semi‐structured interview

The purpose of the semi‐structured interview was to perfect the concept and components of the evaluation system. Through purposive sampling and the maximum difference method, we selected stakeholders involved in the nursing quality of clinical drug trials for in‐depth interviews. A total of 4 medical and nursing managers, 2 pharmacy experts, 3 research nurses, and 12 subjects were included.

Several questions are asked during the interview, such as “What do you think the work of a research nurse involves?” and “In what ways do you think the quality of a research nurse's work can be evaluated?” Based on the interview content, corresponding indicators were added and refined. We analysed, compared, and synthesized all indicators. Furthermore, we initially constructed a nursing quality evaluation system for clinical drug trials, including 3 first‐level indicators, 12 second‐level indicators, and 63 third‐level indicators. From this, the expert consultation questionnaire was developed.

4.3. Selection of experts

In this study, we used purposive and snowball sampling methods to recruit 18 experts from 12 hospitals in five cities and provinces. All hospitals are third‐level first‐class hospitals with clinical drug trial wards. The inclusion criteria used for the selection of experts are as follows: (1) Bachelor's degree or above; (2) intermediate professional title or above; (3) should participate in clinical trials for more than 5 years, nursing managers, pharmacists, and quality assurance personnel for more than 1 year, and (4) participate in this research voluntarily and be able to answer the expert consultation questionnaire during the research period. Finally, 1 clinical drug trial medical expert, 2 pharmacy experts, 13 head nurses, and 2 quality assurance officers were included as a consulting experts panel.

4.4. Delphi method

The purpose of the Delphi survey was to clarify and verify the indicators of a quality evaluation system. It involves two or more rounds of release and recovery of the questionnaire for expert consultation. Achieving a consensus on experts' opinions indicates the completion of the consultation.

The questionnaires were distributed to experts by email, WeChat, or in person in April 2021. It contained three parts. Part 1 aimed to determine general information of experts, including age, educational background, work experience, and professional title. Part 2 included experts' opinions of evaluation indicators. In Part 2, experts were required to score each item using a 5‐point Likert scale, from very unimportant to very important. They can also provide comments for the deletion and retention of items. Part 3 was used to investigate the authority of experts according to their basis of making decision and familiarity with the research content.

A total of two rounds of questionnaire consultation were conducted, each of which lasted 2–3 weeks. Our research team carefully checked the collected questionnaires and conducted supplementary surveys by telephone or WeChat for missing items. The results of the two rounds were summarized, analysed and discussed. The criteria for the deletion of indicators contained the mean of importance score of entries <3.5, the coefficient of variation (CV) [(standard deviation/mean) × 100%] >0.30 and the full‐score ratio [the percentage of experts' evaluated items on the score of five (very important)] <10%. Also, we added the indicators suggested by experts and adjusted the indicators they questioned. After two rounds of consultation, the experts' opinions are basically unanimous. We established the quality evaluation system of clinical drug trial nursing.

4.5. Analytic hierarchy process (AHP)

To determine the relative importance of each indicator, we employed the AHP methodology to establish a consensus on the relative weights assigned to each indicator. The AHP is a decision‐making methodology developed by Saaty in the 1970s (Saaty, 1994). At its core, the AHP provides a structured approach to decision‐making by breaking down a complex problem into a hierarchy of criteria and alternatives. The hierarchy consists of a goal, criteria, sub‐criteria, and alternatives. The goal represents the overarching objective or problem that needs to be addressed. Criteria are the different factors or dimensions that contribute to achieving the goal, and sub‐criteria represent the sub‐factors within each criterion. Alternatives refer to the various options or solutions available for consideration.

Our research takes the nursing quality evaluation index of clinical drug trial as the target layer (goal), the first‐level index and the second‐level index as the criterion layer (criteria, sub‐criteria), and the third‐level index as the scheme layer (alternatives), forming a hierarchical structure model. According to the hierarchical structure, the indicators in each level are compared in pairs. The Satty scale method is used to express the relative importance and to construct the judgement matrix. The established hierarchical structure model and the constructed judgement matrix were input into the yaahp 10.3 software to determine the weights and combination weights of each indicator.

4.6. Data analysis

All data were managed and analysed by SPSS 23.0 software. The degree of expert authority is expressed by the authority coefficient (Cr), which is determined by the familiarity coefficient (Cs) and the judgement coefficient (Ca), namely Cr = (Ca + Cs)/2. The degree of concentration of expert opinions is expressed by importance value, CV, and full‐score ratio. The higher importance value, higher full‐score ratio and smaller CV mean better concentration. The degree of coordination of expert opinions is expressed by Kendall's coordination coefficient W. The higher value of Kendall's W means better consistency and credibility of expert opinions (Shen et al., 2019). The yaahp 10.3 software was used for the AHP. Each index's weight and combination weight were obtained using the power method. The consistency ratio (CR), the ratio of consistency index (CI) and average random consistency index (RI), was also obtained for testing. CR <0.1 indicates that the judgement matrix has a good consistency (Yang et al., 2021).

4.7. Ethical considerations

The ethics committee approval was obtained from the local ethics committee. We obtained written informed consent from eligible participants. They were informed about study‐related issues and were aware of their right to withdraw from the study at any time. Anonymity and privacy were ensured through strict confidentiality measures, including the use of ID codes and secure data storage. During data analysis and reporting, all identifying information was removed.

5. RESULTS

The general information of the Delphi experts is shown in Table 1. Their ages ranged from 26 to 59 (38.56 ± 7.99) years, and their working years ranged from 3 to 38 (17.28 ± 9.78). In this study, we conducted two rounds of expert consultation. The effective recovery rates for the first and second rounds of the questionnaire were 90% and 100% (Table 2), respectively. And the proportions of experts' opinions were 77.78% and 16.67%, respectively (Table 3). It shows that experts have high enthusiasm for our research.

TABLE 1.

General information for experts.

Variables	Frequency (N = 18)	Percentage (%)
Age (year)
20–30	3	16.67
30–40	7	38.89
40–50	7	38.89
50–60	1	5.55
Academic degree
Bachelor	13	72.22
Master	5	27.78
Working years
1–10	4	22.22
10–20	5	27.78
20–30	8	44.44
30–40	1	5.56
Working years as a manager
1–5	8	44.44
5–10	5	27.78
10–15	5	27.78
Job title
Intermediate title	16	88.88
Vice‐senior title	1	5.56
Senior title	1	5.56

TABLE 2.

Questionnaire recovery.

	Questionnaires issued	Questionnaires returned	Questionnaire recovery rate (%)
First round	20	18	90
Second round	18	18	100

TABLE 3.

Experts who made recommendations.

	Total number of experts	Expert who made the recommendation	Rate of making recommendations (%)
First round	18	14	77.78
Second round	18	3	16.67

In this study, the authority coefficients of the two rounds of consulting experts were respectively, 0.831 and 0.885 (Table 4), both of which were >0.7, indicating that the experts' judgements were based on years of practical experience and rich theoretical foundations, with high reliability. The coefficients of variation of all indicators were <0.3, and Kendall's W in the two rounds were, respectively, 0.189 and 0.214 (p < 0.01), indicating that experts had a good degree of concentration and coordination.

TABLE 4.

The authority of experts.

	Judgement coefficient (Ca)	Familiarity coefficient (Cs)	Authority coefficient (Cr)
First round	0.756	0.906	0.831
Second round	0.855	0.914	0.885

All the first‐ and second‐level indicators are reserved in the first round of expert consultation. Four third‐level indicators were deleted based on the criteria of deletion and experts' opinions, including “improved quality assurance system”, “correct rate of samples sent out”, “compliance with regular review”, and “patient awareness of informed consent for trials”. In addition, we modified the expression of 17 indicators according to experts' opinions. In the second round of expert consultation, the degree of recognition of the indicators by experts was relatively high, and no new indicators were added.

We summarized and analysed the results of the two rounds of expert consultation, adjusted the indicators, sorted out the final indicators of the nursing quality evaluation of clinical drug trials, and determined the weight coefficients. The consistency test results of the judgement matrix of the indicators at all levels were CR <0.1, indicating satisfactory consistency. The nursing quality evaluation system for clinical drug trials includes 3 first‐level indicators, 12 second‐level indicators, and 59 third‐level indicators (Table 5).

TABLE 5.

Nursing quality evaluation indicators and their weights in clinical drug trial.

First‐level Indictor	Weights	Second‐level indictor	Weights	Combined weight	Third‐level indictor	Weights	Combined weight	Importance value	Standard deviation	Coefficient of variation	Full‐score ratio (%)
A. Structural quality	0.4000	A1. Nursing human resource elements	0.1070	0.0428	A1.1. The ratio of nurses to clinical programs	0.5080	0.0217	4.56	0.98	0.22	72.20
					A1.2. Composition ratio of professional working years	0.0926	0.0040	3.89	1.02	0.26	27.80
					A1.3. Average nursing hours of inpatient subjects per day (h)	0.2449	0.0105	4.17	0.86	0.21	44.40
					A1.4. Average nursing hours of outpatient subjects per day (h)	0.1545	0.0066	4.11	0.90	0.22	44.40
		A2. Nursing staff education and training	0.4155	0.1662	A2.1. Hours of competency training (pre‐job training) per capita	0.0982	0.0163	4.50	0.71	0.16	61.10
					A2.2. Hours of on‐the‐job training per capita for specialist positions	0.0453	0.0075	4.17	0.71	0.17	33.30
					A2.3. Hours of training on knowledge of clinical drug trials	0.1593	0.0265	4.67	0.59	0.13	72.20
					A2.4. Hours of training on first aid knowledge	0.2610	0.0434	4.89	0.32	0.07	88.90
					A2.5. Hours of training for communication skills	0.0553	0.0092	4.22	0.81	0.19	38.90
					A2.6. Hours of training for clinical trial protocols	0.2610	0.0434	4.89	0.32	0.07	88.90
					A2.7. Hours of training for department standard operating procedure	0.1199	0.0199	4.61	0.61	0.13	66.70
		A3. Environmental and equipment management	0.1849	0.074	A3.1. First aid facilities and medicines are complete	0.4086	0.0302	4.94	0.24	0.05	94.40
					A3.2. The temperature and humidity of the ward are consistent, the time of all clocks is synchronized, and there is a backup power supply for the control equipment	0.2339	0.0173	4.67	0.49	0.10	66.70
					A3.3. Regular maintenance, calibration and quality inspection of equipment	0.1857	0.0137	4.61	0.61	0.13	66.70
					A3.4. Establish a good interpersonal communication environment	0.0578	0.0043	4.00	1.14	0.28	38.90
					A3.5. Ward is warm, reflecting family and humanization	0.0411	0.0030	3.83	0.79	0.21	16.70
					A3.6. Perfect subject recruitment platform	0.0728	0.0054	4.06	0.87	0.22	33.30
		A4. GCP and Bylaws	0.2926	0.117	A4.1. Formulate complete systems and norms, modify them regularly, and constantly improve them	0.6667	0.0780	4.78	0.55	0.11	83.30
					A4.2. Awareness of rules and regulations	0.3333	0.0390	4.61	0.50	0.11	61.10
B. Process quality	0.2000	B1. Specialist nursing operation technology	0.1205	0.0241	B1.1. The pass rate for specialized critical emergency operations	0.2580	0.0062	4.78	0.43	0.09	77.80
					B1.2. The pass rate for processing blood samples	0.1025	0.0025	4.50	1.04	0.23	72.20
					B1.3. The pass rate of blood sample collection	0.2580	0.0062	4.78	0.55	0.11	83.30
					B1.4. The correct rate of dispensing the drug	0.1641	0.0040	4.67	0.97	0.21	83.30
					B1.5. The correct rate of operation of the centrifuge	0.0684	0.0016	4.28	1.02	0.24	50.00
					B1.6. The correct rate of blood sample sorting	0.0464	0.0011	4.22	1.06	0.25	50.00
					B1.7. The correct rate of performing the ECG	0.1025	0.0025	4.50	0.79	0.17	61.10
		B2. Evaluation of new drug administration and nursing intervention	0.1906	0.0381	B2.1. New drugs are qualified for short‐term storage and transportation	0.0983	0.0037	4.50	1.04	0.23	72.20
					B2.2. New drug administration operation pass rate	0.1558	0.0059	4.78	0.55	0.11	83.30
					B2.3. Accuracy rate of drug infusion speed	0.1687	0.0064	4.56	0.98	0.22	72.20
					B2.4. The correct rate of use of drug infusion utensils	0.1687	0.0064	4.56	0.98	0.22	72.20
					B2.5. Drug infusion sequence accuracy	0.1687	0.0064	4.56	0.98	0.22	72.20
					B2.6. The correct rate of extravasation treatment of new drugs	0.0802	0.0031	4.39	1.04	0.24	61.10
					B2.7. The pass rate of new drug waste treatment	0.0405	0.0015	4.22	1.17	0.28	55.60
					B2.8. Residual drug recovery pass rate	0.0506	0.0019	4.28	1.07	0.25	55.60
					B2.9. The correct rate for assessment of potential complications	0.0686	0.0026	4.33	1.08	0.25	61.10
		B3. New drug adverse reaction management	0.4182	0.0836	B3.1. The correct rate of evaluation of adverse reactions	0.1958	0.0164	4.67	0.49	0.10	66.70
					B3.2. The correct rate of emergency treatment of allergic reactions to new drugs	0.4934	0.0413	4.83	0.51	0.11	88.90
					B3.3. The implementation rate of nursing measures for specific adverse reactions of new drugs	0.3108	0.0260	4.72	0.57	0.12	77.80
		B4. Original records management	0.2707	0.0541	B4.1. Original care records are accurate and timely	0.3108	0.0168	4.67	0.49	0.10	66.70
					B4.2. Original care record completeness rate	0.4934	0.0267	4.72	0.57	0.12	77.80
					B4.3. Normative modification of the original care record	0.1958	0.0106	4.50	0.62	0.14	55.60
C. Outcome quality	0.4000	C1. Subject‐related outcomes	0.4094	0.1637	C1.1. Incidence of extravasation of intravenous fluids of degree III or above	0.2857	0.0468	4.44	0.70	0.16	55.60
					C1.2. The correct rate of taking medicine at home	0.2857	0.0468	4.44	0.98	0.22	61.10
					C1.3. Compliance with PK blood collection in subjects	0.2857	0.0468	4.44	1.04	0.23	66.70
					C1.4. The participant's awareness rate of knowledge related to the new drug	0.1429	0.0234	4.22	0.88	0.21	44.40
		C2. Nursing staff‐related outcomes	0.0965	0.0386	C2.1. Active departure rate	0.1667	0.0064	3.89	1.02	0.26	27.80
					C2.2. The nurse's awareness of the knowledge related to the new drug	0.8333	0.0322	4.67	0.59	0.13	72.20
		C3. Outcome indicators of new drug use	0.2047	0.0819	C3.1. The total number of deviations from the project plan caused by nurses	0.1164	0.0095	4.56	0.62	0.14	61.10
					C3.2. Incidence of medication errors	0.3055	0.0250	4.83	0.38	0.08	83.30
					C3.3. Incidence of blood sample processing errors	0.1164	0.0095	4.56	1.04	0.23	77.80
					C3.4. The incidence of AE and SAE caused by nurse work	0.2063	0.0169	4.78	0.43	0.09	77.80
					C3.5. Incidence of process errors	0.1164	0.0095	4.56	0.62	0.14	61.10
					C3.6. Blood collection over‐window rate	0.0765	0.0063	4.50	0.86	0.19	66.70
					C3.7. The incidence of SAE underreporting	0.0624	0.0051	4.39	1.09	0.25	66.70
		C4. Satisfaction	0.2895	0.1158	C4.1. Satisfaction of the subject or family with the nursing work	0.2554	0.0296	4.33	0.59	0.14	38.90
					C4.2. Nurse satisfaction with nursing work	0.3381	0.0391	4.39	0.70	0.16	50.00
					C4.3. Physicians' overall satisfaction with nursing work	0.1660	0.0192	4.17	1.04	0.25	44.40
					C4.4. CRC (Research Assistant) overall satisfaction with nurses	0.0746	0.0086	3.89	1.02	0.26	27.80
					C4.5. Number of valid nursing complaints in this year	0.1660	0.0192	4.17	0.62	0.15	27.80

6. DISCUSSION

6.1. The scientificity and reliability of the nursing quality evaluation system in clinical drug trial

The scientificity and reliability of the construction of the nursing quality evaluation system are reflected in the research methods and the selection of experts. This study initially constructed a questionnaire through literature review and interviews with relevant personnel and then strictly followed the steps of the Delphi method to screen nursing quality evaluation indicators.

Clinical trials are required to be conducted in qualified institutions. Experts were selected from 12 qualified third‐level first‐class hospitals in Guangdong, Shanghai, Zhejiang, Hunan, and Beijing, reducing area distribution bias to some extent. These experts are experienced in the field of clinical drug trials, have a high degree of familiarity with nursing management, and can make valuable judgements and suggestions about research content.

The development of clinical drug trials requires the collaboration of multiple personnel. When selecting experts, we choose frontline research nurses and also nursing managers. In addition, we selected some medical experts and pharmacists with extensive experience in clinical drug trials. These guarantee the representation and comprehensiveness of experts.

The effective recovery rates of the two rounds of expert consultation were 90% and 100%, respectively. The percentages of experts who raised constructive ideas were 77.78% and 16.67%, respectively. It shows that the experts are highly motivated, reflecting their interest and support for this research. The authority coefficients of experts were both >0.7, respectively, 0.831 and 0.885. It indicates that the consulting experts have high authority, which guarantees the reliability of the results of the inquiries. The coordination coefficients of experts were 0.189 and 0.214, respectively (p < 0.01), indicating that expert opinions were coordinated and the consultation results were reliable.

6.2. The evaluation index system has specialized characteristics and practicability

The indicators constructed in this study can fully reflect the speciality characteristics of clinical trial drug nursing. Unlike general departments, the main purpose of clinical drug trials is to explore the pharmacokinetics and degree of tolerance of new drugs used by the human body for the first time. Thus, all nursing treatments were performed under the provisions of the clinical drug trial protocol, including subject admission, vital sign measurement, blood sample collection, laboratory examination, and so forth. Moreover, these operations must strictly follow the time window given by the protocol. Thus, there is a second‐level indicator “specialist nursing operation technology” in the process quality evaluation, which includes three‐level indicators such as “the pass rate of blood sample collection” and “the correct rate of performing the ECG”. Moreover, there are corresponding three‐level indicators such as “The total number of deviations from the project plan caused by nurses”, “Incidence of process errors” and “Blood collection over‐window rate” in the evaluation of outcome quality to evaluate the quality of the entire nursing operation process by nurses.

In addition to routine nursing operations, research nurses are also required to participate in the preliminary preparation of the project, including the study of research plans and SOPs, preparation of materials and environment, communication with various personnel, and management of related materials. Correspondingly, in the structural quality evaluation system, we have set secondary indicators of “nursing staff education and training” and “environmental and equipment management”. The corresponding tertiary indicators with speciality characteristics, such as “hours of training on knowledge of clinical drug trials”, “hours of training for clinical trial protocols”, and “the incidence of AE and SAE caused by nurse work”, have relatively high expert scores. They also meet the requirements of the speciality and have practicability and operability (McAdams, 2012).

During a clinical trial, in addition to hospitalization, subjects are required to undergo outpatient follow‐up according to the protocol. The nurse is required to cover both the ward and the outpatient clinic. Thus, the direct nursing hours of subjects during hospitalization and the average nursing hours in outpatient clinics were used together to reflect whether the nurses are sufficient and whether they can ensure the quality of nursing, which is consistent with prior studies (Chang et al., 2019; McAdams, 2012). The evaluation content generally runs through the whole clinical trial nursing process.

During the first and second rounds of consultation, experts made several comments. The working group discussed these comments and made the appropriate changes to facilitate the usage of this quality assessment system in clinical settings. Several experts thought that subjects are not only patients but also healthy people. Clinical trials will also carry out projects related to healthy people. Thus, we change all ‘patients’ in the indicators to ‘subjects’. An expert recommended adding “program completion” to the outcome quality section. Because program completion outcomes were influenced by multiple factors other than nursing, we ultimately did not add it. Some experts suggested deleting the two items, “processing blood sample” and “dispensing the drug”, because the research nurses did not do this. Considering the failure to homogenize the job responsibilities of research nurses in China (Zhang et al., 2022), the work undertaken by research nurses varies from hospital to hospital. Also, most other experts confirmed the importance of these two items, so they were eventually retained. Experts also suggest that the nurse–patient ratio is less applicable in clinical trials. The workload of the research nurses should include the number of projects, the number of people selected and enrolled, and the number of visits during treatment and survival, as well as the amount of document management, the number of sample collection and processing, and so on. Therefore, the nurse‐manpower ratio is different from the conventional nurse–patient ratio or bed ratio, but is the ratio of the number of nurses to clinical projects. Generally, the evaluation indicators revised according to expert opinions are more suitable for clinical practice.

6.3. The weight and combined weight of evaluation indicators of clinical trial nursing quality

6.3.1. Structural quality

Among the secondary indicators of structural quality, nursing staff education and training had the highest weight (0.4155). A scoping review (Hong et al., 2021) noted that training in nursing and clinical trials is a crucial factor not only for the safety of study participants, but also for nurses' performance, confidence, and satisfaction. Standardized training and assessment ensure that nurses operate consistently in clinical trials and follow clinical trial protocols. It also reduces variability and individual differences, reduces trial error, and improves the quality of the trial.

The tertiary indicators with the highest weight in nursing staff education and training are hours of training on first aid knowledge (0.2610) and hours of training for clinical trial protocols (0.2610), which reflects that we should attach more importance to how to respond to emergencies and protocol study. Clinical trials involve the first administration of new drugs to subjects, which carries potential risks. Related serious adverse events, including death, have been reported (Casassus, 2016; Zhang et al., 2021). Furthermore, a clinical trial protocol involves several sections that will specifically outline the nursing care of subjects in a specific clinical trial (Legor, 2020), which varied from trial to trial as guidelines for nursing practice, so nurses had to keep learning.

Meanwhile, among the tertiary indicators of structural quality, the indicator with the highest combined weight is “formulate complete systems and norms, modify them regularly, and constantly improve them” (0.0780). This item is defined as formulating the rules and regulations of the department according to the laws and regulations, guidelines, and expert consensus related to clinical trials and nursing. A previous study (Zhao et al., 2020) determined that government management, including relevant laws and regulations, guidelines, and expert consensus, was significantly and positively associated with trial quality. Hastings et al. (2012) also suggested that clinical research nurses must not only be well‐versed in clinical skills, but must also be knowledgeable about the complex regulatory, ethical, and scientific aspects of a clinical trial in order to achieve the best possible outcome for subjects and trials. This reminds us to pay attention to the perfection of the specialist system, so as to provide us with good guidance.

6.3.2. Process quality

New drug adverse reaction management (0.4182) accounts for the greatest weight in the secondary indicators of process quality, while the correct rate of emergency treatment of allergic reactions to new drugs has the greatest weight in its tertiary indicators, which also had the largest combined weight among all three‐level indicators of process quality. The use of new drugs is often accompanied by an unknown and unpredictable risk of adverse events (Adashek et al., 2019). In line with what we found in the structural quality section, experts were more concerned with the timely and correct handling of emergencies by nurses during drug administration in clinical trials. This warns us to remember that subject safety is the highest priority in clinical trials.

Among all tertiary indicators of process quality, it is worth noting that accurate and timely original nursing records (0.0267) have the second‐largest combined weight. This is because these records are the credential basis for project verification before new drugs are approved for marketing, including subject case report forms, original medication observation, blood collection records, follow‐up records, etc. Supervisors also learn about a clinical trial's entire process by inspecting the trial's raw data. “No record means it didn't happen.” The authenticity, timeliness, and traceability of the original data are key focuses of the clinical trial process. Therefore, raw data should be recorded in the healthcare information system and documentation in a comprehensive, standardized, accurate, and timely manner to ensure the quality of the data.

6.3.3. Outcome quality

Subject‐related outcomes accounted for the highest weight among the secondary indicators of outcome quality. The management of subjects in clinical drug trials is unified and individualized, involving subject recruitment, informed consent, subject screening, enrolment, nursing care, follow‐up, etc. (Ness & Royce, 2017). The quality of each critical component can affect subject outcomes and subsequent clinical trial phases. The nursing staff is prompted to gate every aspect of the clinical trial process and make every effort to promote the physical and mental health of the subjects.

Three of the tertiary indicators in patient‐related outcomes are tied for the top combined weight of all tertiary indicators of outcome quality. They were “incidence of extravasation”, “the correct rate of taking medicine at home”, and “compliance with PK blood collection in subjects”. Clinical trials are a critical step in drug development, providing data for decision‐making by developers and regulatory authorities. As one of the data sources, PK blood samples reflect the pharmacokinetics of the studied drug interactions. Therefore, clinical trials should strive to reflect accurate results as accurately as possible and eliminate or control for factors that may cause bias (Kuehne et al., 2020). Thus, there is a need to ensure subject compliance in the trial. This suggests that nursing staff should explain the purpose of each nursing activity to the subject, inform the patient of its importance and precautions, and provide proper health education before performing it. In addition, nursing staff should focus on compliance not only when patients are hospitalized, but also when they are at home.

6.4. Limitations of the work

There are some limitations to this study. Experts for this study were selected from 12 hospitals in 5 provinces and cities, mainly in Guangdong. It is necessary to expand the research population in future research. In addition, we have not applied this evaluation system to clinical practice and failed to evaluate the effect after application. Later studies should focus on the clinical application of this evaluation system and continue to improve it.

7. CONCLUSION

In this study, we combined qualitative and quantitative research methods, reviewed the massive literature, and used the semi‐structured interview and the Delphi methods to construct the nursing quality evaluation index of clinical drug trials. At the same time, we set the weight of each indicator in combination with the AHP. The constructed index conforms to the characteristics of the clinical drug trial nursing speciality. It can comprehensively measure the nursing quality of clinical drug trials and provide a reference for the reliable monitoring and continuous improvement of nursing quality in clinical drug trials.

AUTHOR CONTRIBUTIONS

Study design: Herui Yao, Suiwen Ye. Data collection: Junyi Chen, Qingxiu Mai, Shuping Chen, Qingyin Liu, Mengyun Shi, Yunni Lin, Lihong Yan, Meiyu Li. Data analysis: Junyi Chen, Huiyu Zhou. Study supervision: Herui Yao, Suiwen Ye. Manuscript writing: Junyi Chen, Huiyu Zhou.

FUNDING INFORMATION

This study was supported by Medical Science and Technology Research Fund in Guangdong Province of China (A2020359).

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

ETHICS STATEMENT

This study was approved by the Ethics Committee at Sun Yat‐Sen Memorial Hospital, Sun Yat‐Sen University (SYSKY‐2023‐001‐01). We obtained written informed consent from all participants. We confirm that all methods were performed in accordance with the relevant guidelines and regulations.

Supporting information

File S1.

Chen, J. , Zhou, H. , Mai, Q. , Chen, S. , Liu, Q. , Shi, M. , Lin, Y. , Yan, L. , Li, M. , Ye, S. , & Yao, H. (2023). Establishing an evaluation index system of nursing quality for clinical drug trials: A Delphi study. Nursing Open, 10, 7266–7278. 10.1002/nop2.1979

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.