Application of Delphi method and analytic hierarchy process to establish indicator system for evaluation of rational drug use in children with primary nephrotic syndrome

Abstract

Nephrotic syndrome (NS) can be divided into primary, secondary, and congenital NS 3 types, and primary nephrotic syndrome (PNS) accounts for about 90% of the total number of NS in children, which is a common childhood glomerular disease one. The treatment of children with PNS has been controversial and confused because of hormone tolerance, complications, multiple drug combinations, and other issues, but there are no indicators to assess the rational drug use (RDU) of children with PNS. This study aims to develop a set of indicators to assess the RDU in children with PNS.

The study is an observational study and the procedure includes 3 steps:

• 1.Systematic review: searched the websites, guidelines, and studies to establish the initial indicators.
• 2.Expert consultation: applied the modified Delphi method among experts in the field of nephrology for a two-round collaborative consensus project. Obtained the final indicators by modifying each round based on the comments provided by the experts.
• 3.Analytic Hierarchy Process: applied the AHP to determine the weight of each indicator.

A consensus was reached after 2 rounds of the Delphi survey and each indicator was weighted. The final indicators included 2 first-rank indicators and 16 second-rank indicators. In round 1, modified 3 indicators, increase 2 indicators and delete 6 indicators. In round 2, reached consensus. The first-rank indicators comprised drug choice (46.96%) and drug usage and dosage (53.04%); The second-rank indicators aimed to the specific drug therapy, including the RDU of hormones, immunomodulators, and adjuvant drug. The score of each indicator met the requirements, therefore, childrens PNS RDU evaluation index system had been established and the index was scientific and credible.

The first set indicators had been established to assess RDU of children with PNS. Monitoring these indicators will guide people towards the promotion of RDU for PNS. Whats more, the indicator provided a methodological reference for the development of other indicator sets.

1. Introduction

In 1986, RDU was first defined by the World Health Assembly (WHA) as “patients receive medications appropriate to their clinical needs, in doses that meet their own individual requirements, for an adequate period of time, and at the lowest cost to them and their community”.^[1] Among the high-risk drug populations, children have always been the focus of attention, but the safety and efficacy of their medication have been challenging.^[2–7] In 2014, our research team conducted a systematic search to evaluate existing drug-related indicators and found that there was only 1 set of medication indicators developed for children.^[4,6] In addition, this set of indicators was designed for children in primary health care, which was not suitable for the treatment of specific diseases.^[4]

We screened diseases in hospitalized children by prevalence and burden of disease and found that childrens NS is one of the most common kidney diseases in pediatrics and the second largest in children with kidney disease.^[8] According to foreign reports, the annual incidence of the population under the age of 16 is about 1/50,000, of which 58.9% of the initial episodes within 1 year indicate that a considerable number of new cases occur each year and are one of the most common kidney diseases in paediatrics.^[9] The number of hospitalized patients has been increasing year by year. PNS accounts for about 90% of the total number of children with NS. Once the incidence of NS, it will have a serious impact on childrens health. At present, the treatment of the disease is mainly in hormone therapy and general treatment, but the hormones dosage and course of treatment have some controversy, while there is a big difference in the general treatment due to the doctor personal medication habits.

Therefore, in this study, we took the PNS as a sample disease, and combined the modified Delphi method with AHP to develop a set of indicators to assess the RDU in children.

2. Methods

2.1. Survey design

We used the Delphi method to reach experts consensus, which was modified by adding a round-table discussion after each email survey. And translated consensus into indicators. The Delphi process took 2 consecutive rounds in the form of an email survey. After each round, we modified the questionnaire based on the advice provided by the experts and presented the previous results anonymously so that the experts could re-evaluate the answers without peer pressure.^[10]

2.2. Review evidence and generate initial indicators

To developed the initial indicators, our group searched the guide library (GIN, NGC, Trip, NICE), English databases (PubMed, EMbase, Cochrane Library), and Chinese databases (CNKI, VIP, Wanfang, CBM). The search terms were “ nephrotic syndrome ”, “primary nephrotic syndrome”, “children”, “pediatric”,“newborn”, “ neonate”, and “infant”. First search time was in May 2017 and updated the search in October 2017. Two researchers (ML, LNZ) independently selected studies.

The included guidelines and studies met the following criteria:

• 1.patients with PNS between 0 to 18 years;
• 2.interventions related to drug treatment;
• 3.guidelines were the latest edition;
• 4.published in English or Chinese;
• 5.guidelines that the drug treatment recommendations could be developed indicators.

Two reviewers (ML, LNZ) independently extracted and classified the drug treatment recommendations according to the included guidelines and studies, and the project team developed the indicators based on treatment recommendations. For example, the Japanese guidelines^[11] suggested that NS is a risk factor for decreases in bone mineral density and compression fractures and suggested the reduction or discontinuation of steroids for the prevention and treatment of pediatric steroid-induced osteoporosis. Therefore, we developed the indicator “Calcium treatment ratio”, which was defined as the number of children received calcium supplementation or vitamin D as a percentage of all children. In this indicator, while calcium and vitamin D, only calcium or vitamin D were all considered as calcium treatment, because many families often had a home storage of calcium tablets or vitamin tablets. A too-low proportion of calcium treatment ratio would serve as a warning for clinical drug use.

2.3. Panel selection

Panel selection played an important role in indicators evaluation. For this Delphi, we assembled a 24-experts panel to participate this project by email survey. They were from 4 hospitals distributed in the eastern, central, and western regions respectively. Each hospital provided 2 experts, 1 clinician and, 1 clinical pharmacist.^[6] The list of hospital selection is through the Group of People with Highest Risk of Drug Exposure of the International Network for the Rational Use of Drugs (INRUD) in China.^[12]

At the same time, we invited another 4 experts to discuss changes in indicators after each survey. They were all from the West China Second University Hospital who were not project members.

Principle of expert selection:

• 1.more than 3 years of practice in a pediatric nephrology department;
• 2.possessed at least an intermediate title;
• 3.were interested and willing to participate in our study; and
• 4.had no direct conflict of interest with this study.

This study was approved by the Institutional Review Board of West China Second University Hospital.

2.4. Delphi process and the weight of each indicator

Two rounds Delphi process were conducted. In the e-mail survey, experts scored 4 aspects of each indicator:

• 1.importance,
• 2.accessibility,
• 3.degree of familiarity, and
• 4.the evidence of judgment.

In addition, experts were given an opportunity to provide comments or suggestions at the end of the questionnaire. After each e-mail survey, we conducted a round-table discussion of the indicators with a mean score below 7 on the importance and accessibility in the questionnaire. The other 4 experts mentioned above assisted to determine whether indicators should be added, rejected or modified based on the scores and recommendations. When all the parameters of the survey met the requirements, the Delphi process was completed.

AHP was to weight each indicator in this study. In the AHP, we obtained the relative weight of the indicator by the geometric mean method, and normalized the elements of each column to calculate the weight by the consistency test.

3. Results

3.1. Study population

The 2 rounds of expert participation rates were 95.8% and 95.7%, respectively. In the first round, 11 (45.8%) clinicians and 12 (50%) clinical pharmacists, completed the questionnaire. In the second round, 22 experts completed the questionnaire with 1 clinician lost. All questionnaires returned were valid.

3.2. Development of indicators

By literature review, 8 guidelines^[11–19] and 35 studies were included. We had developed 22 indicators for the first round surveys, including 4 first-rank indicators and 18 second-rank indicators. Our group conducted the two-round Delphi survey from September 2017 to December 2017, details in Figure 1.

Figure 1.

Flow diagram of quality indicator development. Initial indicators were generated based on a systematic review of guidelines and studies, a 2-round modified Delphi process was then carried out, and some indicators were added, rejected, or modified in each round Delphi survey.

3.3. Final indicators and their weights

After the two-round Delphi survey, 2 first-rank indicators and 16 second-rank indicators were generated. The first-rank consisted of “drug choice” and “drug use and dosage”. Among the second-rank indicators, they were respectively for the assessment of hormonal use (3/16), immunomodulators (5/16), and adjuvant drugs (8/16). In terms of sources, 11 (68.75%) indicators were from guidelines, 3 (18.75%) were self-made and 2 (12.5%) were from experts advice. And all the indicators score met the requirement.

Each indicator was weighted by AHP. The higher the weight value was, the more important the indicator was. In the first-rank indicators, the weight of the “drug choice” was 0.5304, followed by “drug usage and dosage” of 0.4696. More details were shown in Tables 1 and 2.

Table 1.

Final indicators and weight of each indicator.

Table 2.

Weight of the top 10 indicators among the second-rank indicators.

4. Discussion

4.1. Analysis of the indicators

Through the modified Delphi survey, we developed 16 indicators to assess the RDU in PNS children during hospitalization. The indicator set aimed to monitor and assess the actual hospital drug use through hospital information system (HIS) system data. Because hormones, immunomodulators, and adjunct drugs such as diuretics were the primary drugs used to treat PNS in children, indicators were developed for these 3 classes of drugs. As symptomatic treatment part of the doctor medication habits, nearly half of the indicators were used to assess the use of auxiliary drugs. By monitoring these indicators, 1 could judge whether the hospitals drug use was reasonable ().

Table 3.

The process of Delphi method.

4.2. Strengths of this study

The set of indicators is the first set of indicators designed to assess the RDU of treatment in PNS children. The whole development process and method of indicators was scientific and reliable, for the Delphi had been recognized throughout the world.^[6] The indicators were mainly abstracted from guidelines, therefore indicators had high credibility; In addition, the experts involved in the questionnaire all had a wealth of expertise and clinical samples. Combining with clinical experience, the feasibility of indicators became better.

4.3. Limitations of this study

Our results should be considered some limitations. First, it was challenging to develop a set of indicators that apply to all children diagnosed with PNS in different countries.^[6] All experts came from China, and the included literature was only in Chinese or English. Second, the indicators were mainly for the medication mentioned in the literature, not covering all the medical conditions of the disease. Third, AHP scores were determined subjectively by experts. In our study, the weights were mainly based on the judgment of the experts clinical experience, without objective data to prove.^[20] Fourth, patients opinions might not have been adequately included. This indicator set mainly considered literature and expert clinical experience, without consultation of patients. Finally, although details of the project were described fully prior to the questionnaire, experts might vary in their understanding of the questionnaire because the survey was conducted by mail instead of face-to-face.

5. Conclusions

The study developed the first set of indicators designed to assess the RDU of PNS in hospitalized children, including 2 first-level indicators and 16 second-level indicators, each weighted. Most of the indicators were set according to the guidelines, so other countries or regions could use or modify the indicators in practice. In addition, the set of indicators provided a methodological reference for the development of other indicator sets.

Acknowledgments

The authors gratefully acknowledge the Group of People with Highest Risk of Drug Exposure of INRUD in China and the members of the Delphi panel who provided their expertise and experience—Ying Zhou, Xuhui Zhong (Peking University First Hospital), Xiaohuan Du, Qihua Feng (The First Affiliated of Soochow University), Yong Zhang, Liucheng Rong (Nanjing Children's Hospital), Min Zhan, Tingyan He (Shenzhen Children's Hospital), Ruiling Zhao, Lijun Zhao (Children's Hospital of Shanxi Province), Yu Yang, Li Tian (Union Hospital Affiliated with Tongji Medical College of Huazhong University of Science and Technology), Rihan Wu (Inner Mongolia People's Hospital), Ling Fang, Li Yang (the First Hospital of Anhui Medical University), Bo Hu, Lin Yuan (Suining People's Hospital), Daiqin Xiong, Hongtao Zhu (The First Affiliated Hospital of Xinjiang Medical University), Ming Liao, Yuhong Li (Maternal and Child Health Hospital of Guiyang Province), Zhenyu Pan, Zhijuan Li (Xi’an Children's Hospital).

Author contributions

Funding acquisition: Li Ling Zhang.

Methodology: Nan Li Zeng.

Project administration: Nan Li Zeng, Liang Huang, Li Ling Zhang.

Resources: Hong Yu Tao.

Supervision: Li Ling Zhang.

Visualization: Liang Huang.

Writing – original draft: Mao Lin.

Writing – review & editing: Mao Lin.