Abstract
Aims
To assess the impact with time of guidelines on antiemetic use in an 850-bed Paris university hospital with a high proportion of cancer patients.
Methods
Guidelines on the use of antiemetics available in cancer chemotherapy were drafted according to the Delphi technique. Their implementation was based upon a patient-specific antiemetic prescription form. To assess the impact of guideline implementation over time, discrepancies between current practice and the guidelines were compared before guideline implementation (between March and August 1995) and after implementation (between March and August 1997, and March and August 1998).
Results
Before the Delphi panel’s guidelines were implemented, 5-HT3 antagonists were inappropriately administered in 70% of cases. After guideline implementation, this proportion dropped significantly (P <0.0001, Fisher’s exact test) to 22% between March and August 1997 and 28% between March and August 1998.
Conclusions
Implementation of guidelines seems to have resulted in significant changes with time, although a causal relationship has not been demonstrated. The development of guidelines by our hospital’s multidisciplinary working group helped the various consultants to adjust medical practices to take account of these changes.
Keywords: antiemetics, Delphi technique, drug utilization review, impact of guidelines
Introduction
Guidelines for clinical practice are being promoted as a way of helping practitioners to reach clinical decisions, and of improving the effectiveness and reducing unnecessary costs of health care. However, despite increasing enthusiasm for such guidelines, there is limited evidence that they have any lasting impact on clinical practice. Their success depends on several factors, including the methods of their elaboration, dissemination and implementation. Once the guidelines have been worked out and implemented, the question is to ascertain whether or not they are having the impact expected. It is therefore important to carry out studies to determine whether the guidelines actually reach the physicians for whom they are intended.
In 1994, the most costly drugs in the pharmacy of our Paris University Hospital (Hôpital Tenon, 850 beds) were 5-HT3 receptor antagonists. In France, approval of 5-HT3receptor antagonists is not restricted with regard to the indications for use, which are defined as the ‘preventive and curative treatment of chemotherapy-induced nausea and vomiting’.
In January 1995, the oncologists decided to standardize the antiemetic regimens prescribed in the Tenon Hospital. Radiotherapists and respiratoy physicians were also prescribers of chemotherapy in our hospital, and together with the oncologists, constituted a Working Group. These senior clinicians agreed to approve recommendations based on the opinions of a panel of outside experts, according to the Delphi technique, and to participate in an antiemetic utilization study, to be conducted both before and after the implementation of guidelines for this purpose.
The aim of this study was to describe the way in which the following objectives can be achieved: the elaboration, dissemination and implementation of guidelines, and the impact of their implementation over time.
Methods
The process developed by the Tenon Working Group included three stages: the drafting of guidelines based on experts’ recommendations resulting from the Delphi Technique, implementation of the guidelines, and assessment of their impact.
The Delphi technique*
The Delphi technique was adopted by the Tenon Working Group in order to reach a consensus of opinion on recommendations for treating acute chemotherapy-induced nausea and vomiting. Seven oncologists, three radiotherapists, three respiratory physicians and one hospital pharmacist constituted the group of experts. They were chosen by the Tenon Working Group because they were highly qualified and experienced specialists working outside the Tenon Hospital.
The 14 experts agreed to answer questions anonymously, in writing. After the first round of questions, the Tenon Working Group sent each expert a summary of the opinions expressed by the members of the Delphi group without identifying any of them. In the light of this additional information, the experts were asked to reiterate their opinions.
This process was repeated twice (three rounds in all). One expert failed to answer the second round of questions, and another did not respond at all. After the second and third rounds, as after the first round, the Tenon Working Group circulated the Delphi experts with a summary of the issues considered and the views expressed, and also of their comments. It was hoped that opinions would converge after the 3 rounds of consultation and that a consensus would be reached. In any case, the Working Group agreed that the opinions expressed in the third round would constitute the guidelines on 5-HT3 antagonist prescription.
Implementation of the experts’ recommendations
During the 4th quarter of 1996, a specific course of training for clinicians was started. In accordance with the previously published guidelines for medical practice [5–7] an antiemetic prescription order form, which summarized the guidelines (Appendix 1), had to be used at the time of prescription and sent to the hospital pharmacy. After October 1996, no antiemetics were supplied without a completed antiemetic prescription order form.
Appendix 1.
Prescription order form for antiemetics.
Assessing the impact of the guidelines [8, 9]
To assess the impact of the guidelines, a ‘before and after’ study of the antiemetic drugs used in chemotherapy was performed at the Tenon Hospital, in 1995, 1997 and 1998, from March to August. During these three periods, all the consecutive in-patient units where chemotherapy was prescribed were enrolled in the study. Data were entered in a Newton palm-top computer (Apple), that easily fits into a lab coat pocket. The characteristics of this computer were as follows: 4Mb chip, 3 Mb extended life memory, and 32 bit-RISC processor. Special software was devised in collaboration with a software development company (MdEo, Puteaux, France). Scrolling menus were designed by the senior clinicians constituting the Tenon Working Group. Data on drug prescription, sex, age, and cancer sites were recorded at the patient’s bedside. each recorded item was checked before being downloaded into a microcomputer in the central biostatistics unit.
Drug utilization was classified as ‘appropriate’ when the guidelines were strictly applied. Patients were classified as ‘overtreated’, either when their regimen comprised higher doses than those recommended, or when a 5-HT3 antagonist was prescribed but had not been recommended. Patients were classified as ‘undertreated’, either when a corticosteroid or 5-HT3 receptor antagonist was not prescribed although it had been recommended, or when a 5-HT3 receptor antagonist was not prescribed but had been recommended. Because prices were not stable during the study period, we based our study of cost savings on the prices prevailing in November 1998. Note that ondansetron and granisetron, the only 5-HT3 receptor antagonists available in our institution, cost exactly the same.
Results
Points agreed on by the Delphi experts
The prescription form for antiemetogenic agents (Appendix 1) was drafted in accordance with the guidelines obtained from the Delphi experts (Appendix 2).
The points agreed on by these experts were the classification of emetogenic anticancer drugs into the three groups listed in Appendix 1, the medical indication for each group, the list of risk factors for nausea and vomiting, a definition of the failure of antiemetic treatment, and the efficacy and safety characteristics of antimetic drugs.
Emetogenic potential
Failure of the panel of outside experts to agree on the four group classification of emetogenic anticancer drugs proposed by the Tenon Working Group led to the adoption of a three group classification based on three therapeutic approaches (Appendix 1). In the administration of several emetogenic anticancer drugs to the same patient, the experts agreed to classify these drugs according to their strongest emetogenic potential. Eleven of the 14 experts agreed to include cisplatin in the highly emetogenic class, whatever its dosage.
Risk factors
Five of the 14 experts did not agree, in the third round of discussions, to include a history of nausea and vomiting during pregnancy, the intravenous route of administration, or a day care setting. All the experts reached agreement on the risk factors listed in Appendices 1 and 2.
Definition of failure
During the secound round of Delphi consultations, the experts agreed to define the failure of antiemetogenic treatment as the occurrence of one or more attacks of nausea or vomiting during the first 24 h after chemotherapy drug ingestion.
5-HT 3 receptor antagonist receptor efficacy
For the two 5-HT3 receptor antagonists in use at the Tenon hospital, 12 of the 14 Delphi experts considered 3 mg granisetron equivalent to 8 mg ondansetron.
Drug safety
The antiemetogenic agents listed in Appendix 1, including the above 5-HT3 receptor antagonists, were considered by all the experts to be well tolerated. They were aware of the association between high doses of phenothiazine and acute dyskinesia, but did not consider this a contraindication for phenothiazine administration. Three circumstances were considered as contraindications for the prescription of corticosteroids in cancer: diabetes, gastroduodenal ulcer, and a history of psychiatric disorders.
Patients
Pharmacists enrolled 174 consecutive in-patients receiving chemotherapy from March to August 1995, 166 from March to August 1997, and 163 from March to August 1998.
From March to August 1995, highly emetogenic anticancer drugs were prescribed for 125/174 patients (71%), moderately emetogenic anticancer drugs, for 29 patients (17%), weakly emetogenic anticancer drugs for 20 patients (12%).
From March to August 1997, highly emetogenic anticancer drugs were prescribed for 72/166 patients (43%), moderately emetogenic anticancer drugs, for 70 patients (42%) and weakly emetogenic anticancer drugs for 24 patients (14%).
From March to August 1998, highly emetogenic anticancer drugs were prescribed for 68/163 patients (42%), moderately emetogenic anticancer drugs, for 63 patients (39%) and weakly emetogenic anticancer drugs for 32 patients (19%).
Physician compliance before and after guideline implementation
Totally inappropriate use
Before the Delphi panel’s guidelines were implemented, inappropriate use of 5-HT3 receptor antagonists was recorded for 123/174 patients (70%). After guideline implementation, these figures dropped significantly (P <0.0001, Fisher’s exact test,Table 1) to 36/166 (22%) for March to August 1997, and to 45/163 (28%) for March to August 1998.
Table 1.
Appropriateness of antiemetic prescription before and after Delphi guideline implementation.
Undertreatment
Before guideline implementation, 74/174 patients (43%) were ‘undertreated’ for the sole reason that 5-HT3 receptor antagonists were not administered with corticosteroids although this had been recommended. After implementation, these figures dropped significantly (P <0.0001, Fisher’s exact test,Table 1): Thus, from March to August 1997, 12/166 patients (7%) were undertreated, and from March to August 1998, 13/163 (8%) because 5-HT3 receptor antagonists were not administered with corticosteroids when this was recommended.
Overtreatment
From March to August 1995, 49/174 patients (28%) were ‘over treated’. These figures dropped significantly (P <0.05 Fisher’s exact test,Table 1) to 24/166 patients (14%) for March to August 1997, and to 29/163 (18%) for March to August 1998.
Cost saving study
As Figure 1 shows, a rapid but transient decrease in 5-HT3 receptor antagonist delivery occurred between March and August 1995. By the end of the study conducted before guideline implementation, i.e. by August 1995, the level of consumption had returned to its level at the beginning of the study, in March 1995. After the implementation of the Delphi experts’ guidelines began in October 1996, 5-HT3 receptor antagonist use diminished continuously in the hospital. Table 2 shows the annual figures, expressed as the number of 5-HT3 receptor antagonist vials supplied, and their cost in French francs.
Figure 1.
5-HT3 receptor antagonist consumption (expressed in vials containing 8 mg ondansetron per vial or 3 mg granisetron) in the Tenon Hospital from the first quarter of 1994 to the third quarter of 1998. Before=before Delphi guideline implementation. Hawthorne effect: the positive effect on drug performance of a patient’s inclusion in a research project. Abscissae: Time expressed in years (Y) and quarters (Q). — 5-HT3 receptor antagonists, □ granisetron, ▵ ondansetron.
Table 2.
Cost saving during the drug utilization review process.
Discussion
Between 1995 and 1998, a rational procedure for monitoring the prescription of antiemetics was implemented in our 850-bed Paris hospital. This procedure is still in general use, with the agreement of hospital physicians. In 1995, a Working Group was initiated comprising senior physicians from the oncology, radiotherapy and respiratory departments, and a hospital pharmacist, in rather complex circumstances: the drug market was rapidly changing and pressure for cost containment was high. The Working Group implemented recommendations agreed on by an external group of experts in accordance with the Delphi technique, which can be used to determine the qualifications required for admission to further medical education in therapeutics [10]. This technique allowed physicians to take part in courses, to improve their knowledge and also to develop prescribing skills. An appropriate organization is in place to facilitate the process. It was essential to see that the physicians who will use the guidelines take part in the process of guideline development, to ensure their co-operation and support. Accordingly, a strategy for obtaining a consensus was built into the process, and clinical experience provided part of the basis for the guidelines recommended. Furthermore, physician compliance, for instance, with the recommendations to combine corticosteroids with 5-HT3 receptor antagonists, is easier to obtain when strong evidence of beneficial results is available.
The Delphi technique enabled the experts to agree on the classification of anticancer drugs as strongly, moderately or weakly emetogenic. The pharmacological type of anticancer drugs is the most important factor in the onset of emesis. Several classifications have been published, most of them comprising 3–5 groups [11–13]; there are discrepancies between these classifications, because some anticancer drugs are considered to have high emetogenic potential in one classification and moderate in another. Some classifications take the dosage into account. No explicit methodology has been published for the application of these classifications, and there is no international agreement. This complexity lends itself to the use of the Delphi technique. The experts agreed on a three group classification defined above because it corresponded to the three practical therapeutic approaches the groups had in common: (i) administration of 5-HT3 receptor antagonists by the i.v. route (8 mg day−1 ondansetron or 3 mg day−1 granisetron [14]; (ii) their administration by the oral route, and (iii) administration of non 5-HT3 receptor antagonists. Although the efficacy of the combined use of corticosteroids and 5-HT3 receptor antagonists has been established in the literature [15], the results of the study carried out here before guideline implementation highlighted the local underprescription of corticosteroids. The Delphi group of experts agreed that it was safe for cancer patients to use corticosteroids for short periods. Locally, the recommendations for better use of corticosteroids led to a major change in therapeutic attitudes.
The Delphi process is recognized as one of the most valuable ways of determining the skills required of practising physicians [16]. Furthermore, it has been shown that physicians who are willing to participate in expert panels are representative of their colleagues [17]. The 14 external experts on our Delphi panel were chosen by the physicians in the Working Group of the Tenon hospital, according to subjective criteria (i.e. their reputation, and their involvement in the treatment of patients with cancer in French hospitals). Since the main objective of their recommendations for drug utilization was to provide acceptable guidelines, the most important criterion was that they should be recognized as experts by the clinicians. As the experts were asked to express their views in writing, by means of questionnaires, they did not need to meet, thus minimizing the negative effects of direct group interaction. Another advantage of this process is that Delphi panel members were unaware of each others’ identities so no bias was introduced by conferring. The Delphi process relies on the general expertise and experience of leading members of the professions concerned.
Among the factors affecting the successful introduction of guidelines based on the experts’ recommendations, Grimshaw and Russel showed that the best strategy was to implement an antiemetic prescription order form [17]. This form is now available throughout our hospital, but its validity is limited to 1 year. Once a year, the original Delphi panel recommendations are updated by the Working Group, and the updated recommendations are validated by the group of external experts in a single round of discussions.
As in any uncontrolled study, which is the major limitation of the present work, it is possible that factors other than those included in the guidelines defined here helped to reduce drug consumption. One effect of the drug utilization review, described elsewhere as the Hawthorne effect [7] (i.e. the beneficial effect on drug performance of patient inclusion in a research project) was indeed observed during the study conducted before guideline implementation. Although this effect is known to be transient, the 5-HT3 receptor antagonist consumption measured in the study after guideline implementation remained at the lowest level ever observed in our hospital, probably as a result of the implementation of the expert panel’s recommendations. All prescriptions that do not conform to the experts’ recommendations are routinely reviewed by this panel. During the drug utilization review, no significant change was noted in hospital activity: thus, the number of patients admitted to the oncology department only increased by 13% between 1996 and 1997, and decreased by 7% between 1997 and 1998, and the duration of the hospital stay remained stable (6 days ±1). Nevertheless, we cannot assert that the conditions of the patients admitted before guideline implementation was similar to the condition observed thereafter. In addition, 5-HT3 receptor antagonists became more popular with physicians during the study period, their efficacy was better publicised, they caused no serious adverse effects, and their cost decreased. Between 1996 and the present time (April 1999), no new antiemetics have been introduced in our hospital, thus ruling out one possible reason for changes in prescriptions. Oral forms of 5-HT3receptor antagonists were the only innovations during the study period, but they did not supplant the forms administered i.v. (for example, as from 1996, only one unit of oral 5-HT3receptor antagonists was supplied by the pharmacy for every 20 vials of the i.v. form). In our opinion, all these external factors would have increased the use of 5-HT3 receptor antagonists if efficient guidelines for drug prescription had not been implemented. In other Paris hospitals also, the use of 5-HT3 receptor antagonists decreased in 1996 and 1997, but only by 3.5%, compared with 24% in the Tenon Hospital. From 1997 to 1998, the use of 5-HT3 receptor antagonists increased by 15% in the other Paris hospitals, but their consumption at Tenon decreased by 13%.
Conclusion
The study conducted before guideline implementation revealed many cases in which inappropriate drugs had been administered. Although it was not possible here to demonstrate a causal relationship between guidelines and changes in medical practice, it is very encouraging to note that the after implementation of guidelines in the Tenon Hospital, physician compliance with the panel’s recommendations improved and was regular. Our study shows that when practitioners are involved in a dynamic process, which in this case included the Delphi technique, their compliance with the resulting guidelines is effective. However, the ultimate evaluation must involve outcome studies, to assess the impact of the guidelines on patient care.
Acknowledgments
We thank the Delphi expert group who reached agreement on the prescription of antiemetics. The group included the following: Bonneterre J, Centre Oscar Lambret, Lille; Depierre A, Hôpital Saint-Jacques, Besançon; Extra JM, Hôpital Saint-Louis, Paris; Jammet P, Institut Curie, Paris; Quoix E, Hôpital de Strasbourg; Lamanière JG, Hôpital Bretonneau, Tours; Lartigau E, Institut Gustave Roussy, Villejuif; Lemarié E, Hôpital Bretonneau, Tours; Morere JF, Hôpital Avicenne, Bobigny; Pouillard P, Institut Curie, Paris; Simon JM, Hôpital de la Pitié, Paris; Spielman M, Institut Gustave Roussy, Villejuif; Tourani JM, Hôpital Laennec, Paris; Viau P, Hôpital Gauducheau, Saint-Herblain.
In addition, we are grateful to Pr Alain-Jacques Valleron for helpful advice. We also thank resident hospital pharmacists who collected data during the drug utilization review. Lastly, we thank the software development company MdEo for helping us to adapt the palm-top computer to the needs of this review.
Funding support: Grant from the Commission d’Evaluation, Hôpital Tenon, and software developments funded by Institut National de la Santé et de la Recherche Médicale.
Appendix 2
Guidelines for the use of 5-HT3 receptor antagonists in chemotherapy-induced acute nausea and vomiting
High emetic potential chemotherapy (ChT)
Right away
3 mg granisetron i.v. or 8 mg ondansetron i.v. 15 min before ChT, and steroids.
In case of failure
combination of a different class of antiemetic, and steroids, or 16 mg ondansetron i.v. or 6 mg granisetron i.v., and steroids.
Moderate emetic potential chemotherapy
Right away
1 mg granisetron orally, 1 h before ChT and 12 h thereafter, or 8 mg ondansetron orally, 2 h before ChT, with or without steroids. If the oral route is not possible: 3 mg granisetron i.v. or 8 mg ondansetron i.v., 15 min before ChT with or without steroids or a bolus of 2 mg kg−1 day−1 metoclopramide followed by 3 mg kg−1 day−1, with or without steroids.
In case of failure
combination of a different class of antiemetic, and steroids, or 16 mg ondansetron i.v. or 6 mg granisetron i.v., and steroids.
Low emetic potential chemotherapy
Right away
no antiemetic
In case of risk factors
†: 1 mg granisetron orally, 1 h before ChT and 12 h thereafter, or 8 mg ondansetron orally, 2 h before ChT, or 30 mg day−1 metoclopramide.
Footnotes
Briefly, this is a method of obtaining answers to questions regarding issues that are uncertain, even to experts; it has three cardinal features: anonymity (no member of the group knows either who the others are, or the views expressed by any of them on a particular question), reiteration (the round of questions is repeated three times), and feed-back (this consists of the inclusion of additional information in each round, in the form of statistics regarding the earlier answers of the group, and the experts’ comments). The questions, may be the same in each round, or consist of new queries raised by earlier answers [1–4]. Feed-back and reiteration comprise a sequence of rounds, between which a summary of the results of the previous round is communicated to the participants.
List of risk factors agreed on by the experts: a combination of two or more anticancer drugs with the same emetic potential, age <50 years, female sex, anxiety, emesis experienced during previous ChT cycles.
References
- 1.Milholland AV, Wheeler SG, Heieck JJ. Medical assessment by a Delphi group opinion technic. N Engl J Med. 1973;288:1272–1275. doi: 10.1056/NEJM197306142882405. [DOI] [PubMed] [Google Scholar]
- 2.Pill J. The Delphi method: Substance, context, a critique and an annotated bibliography. Socio Econ Plan Sci. 1971;5:57–71. [Google Scholar]
- 3.Gabbay J, Francis L. How much day care surgery ? Delphic predictions. Br Med J. 1988;297:1249–1252. doi: 10.1136/bmj.297.6658.1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Koplan JP, Farer LS. Choice of preventive treatment for Isoniazid-resistant tuberculosis infection. Use of decision analysis and the Delphi technique. JAMA. 1980;244:2736–2740. [PubMed] [Google Scholar]
- 5.Field MJ, Lohr KN, editors. Guidelines for Clinical Practice From Development to Use National Academy of Sciences. USA: 1992. [PubMed] [Google Scholar]
- 6.Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317–1322. doi: 10.1016/0140-6736(93)92244-n. [DOI] [PubMed] [Google Scholar]
- 7.Grimshaw JM, Russel IT. Achieving health gain through clinical guidelines II. Ensuring guidelines change medical practice. Qual Health Care. 1994;3:45–52. doi: 10.1136/qshc.3.1.45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Morgan AS, Brennan PJ, Fishman Impact of a Vancomycin restriction policy on use and cost of Vancomycin and incidence of Vancomycin-resistant enterococcus. Ann Pharmacother. 1997;31:970–973. doi: 10.1177/106002809703100902. [DOI] [PubMed] [Google Scholar]
- 9.Ray-Coquard I, Philip T, Lehmann M, Fervers B, Farsi F, Chauvin F. Impact of a Clinical Guidelines Program of Breast and Colon Cancer in a French Cancer Center. JAMA. 1997;278:1591–1595. [PubMed] [Google Scholar]
- 10.Boath E, Mucklow J, Black P. Consulting the oracle: a Delphi study to determine the content of a postgraduate distance learning course in therapeutics. Br J Clin Pharmacol. 1997;43:643–647. doi: 10.1046/j.1365-2125.1997.00602.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Laszlo J. Nausea and vomiting as major complications of cancer chemotherapy. Drugs. 25(Suppl 1):1–7. doi: 10.2165/00003495-198300251-00002. [DOI] [PubMed] [Google Scholar]
- 12.Craig JB, Powell BD. Review: The management of nausea and vomiting in clinical oncology. Am J Med Sci. 1987;293:33–44. doi: 10.1097/00000441-198701000-00008. [DOI] [PubMed] [Google Scholar]
- 13.Lindley CM, Bernard S, Fields SM. Incidence and duration of chemotherapy-induced nausea and vomiting in the outpatient oncology population. J Clin Oncol. 1989;7:1142–1149. doi: 10.1200/JCO.1989.7.8.1142. [DOI] [PubMed] [Google Scholar]
- 14.Ruff P, Paska W, Goedhals L, et al. Ondansetron compared with granisetron in the prophylaxis of cisplatin-induced acute emesis: a multicentre double-blind, randomised, parallel-group study. Oncology. 1994;54:113–118. doi: 10.1159/000227321. [DOI] [PubMed] [Google Scholar]
- 15.Roila F, Tonato M, Cognetti F, et al. Prevention of cisplatin-induced emesis: a double-blind multicenter randomized crossover study comparing ondansetron and ondansetron plus dexamethasone. J Clin Oncol. 1991;9:675–678. doi: 10.1200/JCO.1991.9.4.675. [DOI] [PubMed] [Google Scholar]
- 16.Dunn WR, Hamilton DD, Harden RM. Techniques of identifying competences needed of doctors. Med Teacher. 1985;7:15–25. doi: 10.3109/01421598509036787. [DOI] [PubMed] [Google Scholar]
- 17.McKee M, Priest P, Ginzler M, Black N. How representative are members of experts panels? Quality Assurance Health Care. 1991;3:89–94. doi: 10.1093/intqhc/3.2.89. [DOI] [PubMed] [Google Scholar]