Evidence-based decisions about herbal products for treating mental disorders

Introduction

Herbal products are becoming increasingly mainstream. This is not simply a comment on their popularity, but on the level and rigor of scientific research and acceptance by clinical investigators, as well as the attention being paid to these therapeutic options by government bodies. A recent trial published in the Journal of the American Medical Association (JAMA) evaluating St. John's wort for depression demonstrates this shift.¹ Several features of this trial are noteworthy. Not only was the trial of rigorous design and published in a highly reputed journal, but it was completed by a group of high profile and experienced US psychiatric researchers who are generally associated with researching pharmaceuticals.

In the past, the quality of trials involving herbal products and other alternative treatment options for psychiatric and other disorders has been criticized.² Criticisms have included failing to account for biases and confounders, using nonstandardized products and poor study design. However, in the last 5 years, MEDLINE (www.ncbi.nlm.nih.gov/PubMed) has indexed over 50 randomized controlled trials (RCTs) investigating complementary medicines for treating mental disorders, suggesting that an increasing number of alternative approaches are being tested using what is approaching gold standard methods. This number is likely an underestimate because MEDLINE search strategies, even those devised by experts, can fail to locate a significant number of relevant trials.³ It is also well recognized that MEDLINE does not index all relevant journals in the area of complementary medicines.⁴ The Cochrane Controlled Trials Register of the Cochrane Library (www.cochranelibrary.com), which lists clinical trials that have been identified through regular searchers of MEDLINE, EMBASE and hand searches of clinical journals, is considered an authoritative source of RCTs.⁵ For illustrative purposes only, a selection of RCTs from the Cochrane Library of herbal products used for psychiatric indications is listed in Table 1.^{1,6,7,8,9,10,11,12,13}

Table 1

Considering that the amount of good-quality research regarding herbals in psychiatric disorders appears to be expanding steadily, treatment reviews, unless updated regularly, quickly become outdated and serve a limited purpose. Such reviews are also not particularly useful for making individual patient treatment decisions; they tend not to address specific clinical questions adequately, but rather, provide the reader with a general awareness of the topic. As such, this paper is not intended to comprehensively review the benefits and risks of herbal products commonly used for psychiatric disorders. Recent examples of such reviews can be found elsewhere.^14,15,16,17 Instead, the purpose of this review is to present the steps involved in taking an evidence-based approach to clinical decision making regarding the use of herbal products in psychiatry. The reader will be oriented to the evidence-based process using examples of herbal product use for mental disorders. Methods and tools that facilitate rapid access to the best clinical research about herbal products will be presented. In addition, special issues concerning herbal therapies that differ from the evaluation of pharmaceutical products will be introduced and discussed.

Effectiveness and safety

In 1999, the Natural Health Products Directorate (NHPD) of Health Canada was created with the mission of ensuring that all Canadians have ready access to natural health products that are safe, effective and of high quality, while respecting freedom of choice and philosophical and cultural diversity.¹⁸ The creation of such a government body at the federal level demonstrates the importance that Canadian society now places on these health care options. It also implies that Canadians are demanding greater scrutiny of these products. This is a tremendous step forward; bringing these commonly used products and services into the mainstream will pave the way for improvements in access and dissemination of information, product quality control and clinical research. Compared with conventional pharmaceutical therapies in North America, the use of herbal products has lacked adequate regulations, standards and controls, and the quality and veracity of information has been inadequate. For broader acceptance into the medical community, changes are necessary to eliminate this double standard.

The NHPD is concerned with assessing the effectiveness and safety of herbal products. As with conventional medicines, answering these questions is hardly straightforward. A medicine can appear effective in an RCT in highly selected and motivated study participants who take the medicine without fail and receive unusually vigilant monitoring and follow-up. However, when taken out of the clinical laboratory and into the general population, the benefits are often diminished, if not completely eliminated.¹⁹ For many herbal therapies, the reverse process is underway. Medicines such as St. John's wort and Ginkgo biloba, after decades if not centuries of clinical use, are now the subjects of intense clinical research. Effectiveness research on other agents such as kava kava and valerian is growing, although to this point, the number of studies is relatively small and study designs have been inadequate.¹⁶ Increasingly, consumers are asking for clear and detailed information on the benefits of various treatment options, particularly for chronic diseases and preventive therapies. The level of inquiry has been highest for pharmaceutical products, but the gap appears to be narrowing as the healthy skepticism surrounding all therapies grows.

Establishing the effectiveness of a therapeutic intervention ideally comes from a study in which there is little room for biasing treatment allocation, measurement of outcomes or interpretation of the findings. The key ingredients for producing an unbiased assessment of a therapy's value are well established; awareness of these ingredients (randomized allocation, allocation concealment, blinding of evaluators of outcomes, high follow-up rates and analyses by intent to treat) has improved in both clinical and lay settings, in large part, because of the advances made in the past 10 years by those promoting evidence-based clinical decision making.^20,21

Identifying a medicine's safety is more challenging. A drug may appear relatively safe under controlled conditions with exposure being limited to a small number of people with a narrow band of characteristics. However, once the controls over who can and who cannot be exposed are loosened, all bets are off. With most pharmaceuticals, the last control over exposure — access through prescriptions only — is maintained after it is approved by Health Canada for public consumption. This control is not maintained for any herbal product available in North America. Is this a concern? The long and growing list of serious herbal product adverse reactions and interactions between herbal product and pharmaceutical drugs suggests that it is.^17,22

Evidence-based decision making

Evidence-based clinical decision making requires consideration of the best available clinical research evidence, the experience of the clinician and the patient's values and preferences.²³ This last factor seems particularly germane to decisions about alternative medicines because it is often the patient who instigates the discussion. It is the first component (consideration of the best evidence) that has been emphasized in the evidence-based decision-making literature. However, this has resulted in criticism of this approach; detractors have suggested that the evidence-based approach is too narrow and is often not appropriate.^24,25 They have also suggested that clinical experience is not sufficiently emphasized.²⁶ The first criticism is unwarranted. Knowing that there are no studies with adequate internal and external validity to guide a therapeutic decision is tremendously informative and is in concert with the evidence-based approach. This tells the practitioner and the patient that the probability of achieving a good outcome with this therapy is uncertain. It should also encourage those involved in the decision to take a more objective approach determining the benefits and risks of an unproven therapy on an individual basis (e.g., using objective criteria to measure treatment outcome and setting timelines for monitoring responsiveness). The second criticism, that practitioner experience is not sufficiently emphasized, is warranted and applies equally to treatment decisions involving herbal, pharmaceutical and nonsomatic interventions.

The foremost skill necessary for initiating an evidence-based approach is recognizing uncertainty in the treatment decision. The steps of the evidence-based process begin when more than one treatment option exists (including doing nothing) but the best option for the patient is unclear. The traditional 5-step process includes:

1. The development of a 4-part clinical question (i.e., population, intervention, alternative, clinical outcome)

2. An efficient and strategic search for the best available evidence to answer the clinical question

3. Critical appraisal of the best evidence for its internal and external validity

4. Application of the evidence. and

5. reflection upon how the first 4 steps could have been completed with greater efficiency.

It has been suggested that an extra step be added between the first 2.²⁷ In this additional step, the clinician answers the clinical question as she would have without completing the process. This transparent approach allows one to identify initial beliefs and possible biases. An interesting conflict is presented if the findings of steps 3 and 4 reveal data that contradict the clinician's initial beliefs. This conflict could be resolved in several ways, but an open discussion with the patient is a good option.

Evidence-based practice and herbal products in psychiatry

This 5-step process applies to herbal products and mental illness as much as it does to pharmaceuticals and other branches of medicine. However, herbal product evaluation involves special considerations that distinguish it from the assessment of conventional pharmaceutical products. The most obvious difference is in the evaluation of the product itself. For pharmaceuticals, the active ingredient is always known, although its mechanism of benefit may be obscure. There are strict measures in place to ensure that the pharmaceutical is of high quality. This is not the case with herbal products. Until regulations are changed, there will continue to be uncertainty about the amount of active ingredient in every nonstandardized batch of herbal product sold in North America. This issue is of less concern in countries such as Germany, where there are strict quality control regulations for the preparation and distribution of herbal products.²⁸

Knowing what the active ingredient is may be less important than knowing that the preparation, from harvest to dosage formulation, is highly standardized and regulated and the methods have been demonstrated to be safe, effective and reproducible. However, even in countries where the preparation of herbal products is strictly regulated, there is no guarantee that the amount of active ingredients in different batches is equal or even similar. Products are typically standardized on the basis of a single marker that is consistent from batch to batch. However, this marker may not be an important contributor to the activity of the product and it may not correlate well with the amount of the (other) active constituents. This concern was demonstrated recently in the United States with St. John's wort products.²⁹ Although the hypericin content was between 0.18% and 0.25% in 6 of 7 products standardized to this ingredient, the putatively active ingredient, hyperforin, varied from 0% to 3.26% (mean 1.2%, standard deviation 1.1%).

The widely recognized JAMA series Users' Guides to the Medical Literature describes how to handle a broad range of situations of uncertainty including questions of effectiveness, harm (e.g., drug interactions, adverse effects), prognosis and diagnosis. These guides can be found online at the Centres for Health Evidence Web site (www.cche.net/usersguides/main.asp), and updates have been published in a new book.³⁰ In each, a set of guiding questions leads the reader through the most important issues to be considered when evaluating and applying research to an individual patient. The first users' guide (and likely the most commonly used) published by this group pertains to effectiveness questions.^19,20 To be applied to herbal products, additional questions need to be included that consider the quality, activity, source and formulation of the product (Box 1).³¹ The activity level and, therefore, the effectiveness of a herbal product will vary significantly, depending on which parts of the plant the ingredients are taken from and the processes involved (e.g., variations in the extraction procedure) in preparing the phytopharmaceutical. Even the timing of the harvest and the drying and storage conditions can affect its potency.²⁸ There are numerous examples of quality control tests by independent laboratories finding inadequate-to-negligible amounts of the presumed active ingredients and, in many cases, pharmaceutical adulterants.^29,32

Box 1.

Differences in product formulation and dosage preparation (e.g., teas, tinctures, extracts, dried roots, capsules) can also profoundly affect the activity of the product. In addition to detailing the preparation process, the standardized dose and dosage form used, a study should fully describe the herb itself, including its genus, species and the parts of the plant used, as well as the name and manufacturer if a commercial product was used. All tests conducted to identify the product's constituents should also be described.²⁸ These factors often limit the generalizability of studies involving herbal preparations, particularly in countries where there is little quality evaluation, regulatory control and standardization of products.

Blinding is a specific issue in clinical trials of herbal products. When outcomes are unambiguous and unlikely to be affected by preconceptions or personal beliefs, blinding is less of an issue (e.g., death in trial of an antilipidemic agent). However, because of the strong convictions (for and against) held by the public and health care providers about herbal products and the subjectivity in rating clinical symptoms and level of disability associated with mental disorders, blinding is imperative for an accurate and unconfounded evaluation of the clinical effect of phytopharmaceuticals. Unfortunately, given the nature of herbal products, it can be difficult to effectively blind participants and researchers in clinical trials of herbal products.³³ Thus, it is essential that attention be paid to this potential source of bias. This can be done by having participants and researchers record which group they believe the participant was randomly assigned to. In a trial involving 2 groups, an accuracy rate approaching 50% indicates that blinding worked.

Application of the evidence-based process

Recognizing uncertainty

This is the most important step of all. Although therapeutic decisions are made regularly in a busy clinician's day, many are made without reasonable awareness of the evidence supporting the decision. In such situations, the clinician ought to recognize this as uncertainty and henceforth begin the process of locating and digesting the best evidence related to the decision. It is not possible to do this for all situations of uncertainty, but this does not justify not doing it at all. It is suggested that clinicians begin applying the evidence-based steps to clinical decision making for selected situations, such as common decisions or cases where uncertainty is profound. Each clinician will therefore activate the process for different decisions.

Asking the question

The formulation of the question to be investigated is surprisingly important. The details of the question effectively direct the search and determine what study design would constitute the best evidence (e.g., RCTs for effectiveness trials). The question should also be used to bring perspective to the critical appraisal of the findings. It should include a description of the patient or population that highlights key history, treatment and risk characteristics; the intervention being considered and alternative options (which can include doing nothing); and the outcomes desired for and by the patient.^23,30 It needs to be emphasized that these outcomes be clinical (e.g., time spent doing rituals), not surrogate (e.g., Yale–Brown Obssesive–Compulsive Scale score).

Example 1A

“I'm worried about a serotonin syndrome in Mr. X who wants to add St. John's wort to citalopram.”

Population: Mr. X, a middle-aged man with depression; taking citalopram, 20 mg/day for 3 months; had mild nausea when treatment started but otherwise has tolerated citalopram well; took St. John's wort monotherapy in the past with partial improvement and no adverse effects; currently takes no other medications.

Intervention: Add St. John's wort to ongoing citalopram (20 mg/day).

Comparison: Avoid the combination.

Outcome: Serotonergic side effects or serotonin syndrome (i.e., gastrointestinal distress, tremor, myoclonus, incoordination, autonomic instability, hypomania, confusion, diaphoresis, hyperthermia, stupor, coma, death).

A similar scenario could just as likely involve the potential for a pharmacokinetic drug–herb interaction. Many clinicians are aware of the recent findings indicating that St. John's wort may induce a cytochrome enzyme pathway that is involved in the metabolism of various drugs and, therefore, has the potential to cause harmful pharmacokinetic drug interactions.^34,35 However, it is often difficult to differentiate clinically meaningful interactions from those that are likely insignificant. What would the 4 parts of the clinical question be if Mr. X was also taking amlodipine for angina and hypertension?

Searching for the best evidence

Tertiary references are usually not useful for making patient-specific therapeutic decisions. They are generally out of date and lack enough detail for a thorough evaluation of the risks and benefits on an individual patient basis. The alternative approach of using the best available evidence has been, until recently, too arduous for practitioners in terms of both the time and skills necessary. However, with improved access to comprehensive health care information databases (e.g., MEDLINE, Cochrane Library), which include information on complementary medicines, and the development of several evidence-based secondary resources (Table 2), many questions can be answered relatively efficiently. In addition, there are several Web sites and books dedicated to teaching practitioners the evidence-based skills of questioning, searching and critical appraisal.

Table 2

Searches should be efficient. For questions that are likely to be common, secondary sources are recommended. If this fails to provide a rapid and useful answer, searching the appropriate databases for the primary literature is encouraged.

Example 1B

“I would more seriously consider Mr. X's request if I knew how well St. John's wort worked.”

Step 1: What kind of question is this?

An effectiveness question.

Step 2: What type of study is best designed to answer this type of question?

An RCT or meta-analysis of RCTs.³⁶

Step 3: Which secondary literature databases are likely to address this question?

Clinical Evidence, Evidence-Based Mental Health, Bandolier, The Natural Pharmacist, among others.

Step 4: If needed, which primary literature databases are likely to address this question?

Cochrane Library (for systematic reviews and RCTs), PubMed (complementary medicine database; to increase efficiency, a search can be limited to RCTs and meta-analyses or by using the clinical queries feature).

In this example, synopses based on the best evidence of the effectiveness of St. John's wort can be located in several sources listed in step 3. The synopsis that was identified focused on a meta-analysis examining the effectiveness of St. John's wort.³⁷ However, the recent RCT referred to earlier¹ could only be located by searching the Natural Pharmacist Web site (see Table 3), the Cochrane Library or PubMed. In this case, efficient searchers without access to subscriber-only databases (i.e., a combined search using Bandolier and PubMed) and using a slow-speed modem connection could have located one of the useful synopses and the RCT in about 5–10 minutes. An efficient searcher with subscriptions to each resource and high-speed Internet access could have located a high-quality synopsis and the full text of the recent RCT within 5 minutes.

Table 3

Critical appraisal

Evidence-based clinical decision making does not absolutely require that all practitioners become astute critical appraisers, although such skills certainly are advantageous.³⁸ For many clinicians who do not have the time or inclination to develop such skills, knowing how to efficiently access preappraised synopses and synthesized information may be more important (and efficient) than learning the strengths and weaknesses of various research trial designs and analytical methods. Thus, it is recommended that clinicians initially give higher priority to developing their questioning and searching skills than they do to improving their critical appraisal skills.

When critically appraising evidence that relates to a clinical question, there are 3 main questions to consider:

· “What are the results?”

· “Are the results of the study believable?”

· “Will the results help me in caring for my patient?”³⁰

The first question focuses the appraiser on the inner workings of the report, including the trial design and method of analysis of the data. For example, the validity of a meta-analysis would be highly questionable if it combined the results from a variety of studies, some involving a well-characterized, standardized, high-quality herbal product and others that provided no information about the product used. The second question asks the appraiser to examine the size or magnitude of benefit and its precision, and the third question determines if the results can be applied to the patient at hand. There are several excellent and widely available resources that teach clinicians how to efficiently work through these questions toward the goal of efficiently transferring research evidence into patient decisions (Table 2).

Example 2A

“Should I support my patient in using kava as their initial somatic therapy for managing agoraphobia and social phobia?”

When searching for evidence to support the use of kava in the management of a patient with social anxiety disorder and agoraphobia, you identify 12 articles in PubMed by using the “Clinical Queries” function (search filters: therapy, specificity; search terms: “kava AND anxiety”). A quick scan identifies the most recent citation to be a Cochrane Library systematic review. Of the other citations, 3 are in German and 1 is in Italian, 3 are review articles, 1 is an earlier systematic review by the same authors as the Cochrane Library citation and the remaining 3 are RCTs. To be efficient, you quickly go to the Cochrane Library to examine the most recent best evidence addressing kava extract in treating anxiety.³⁹

Characteristic of the usual rigor of a Cochrane systematic review, the authors of this meta-analysis explicitly described their search strategy, which included 6 databases, many synonyms for kava and direct contact with manufacturers and experts of kava treatments. Only randomized, double-blind trials of oral kava extract for the treatment of anxiety were included in the analysis. In an attempt to quantify the likelihood of bias inherent in the trials, the authors assigned quality scores of the trials on the basis of the descriptions of randomization, blinding and withdrawals.⁴⁰ However, there were no inclusion limits placed on trials that did not specifically delineate the components, manufacturing process or standard of kava used in the individual trials. There was also no requirement for evaluating the effectiveness of blinding. Unlike the PubMed search that located 3 English-language RCTs, these authors identified and included 7 trials. All of the reviewed trials suggested superiority of kava extract over placebo. The meta-analysis of the 3 studies using the Hamilton Anxiety Score demonstrated a difference of approximately 10 points in favour of kava extract.

The authors described the dosing and specific extract of kava used in each study for which the information was available. In 3 of 7 studies combined in the meta-analysis, the same extract from a single manufacturer was used (i.e., WS 1490, 300 mg/day of extract standardized to deliver a total daily dose of 70% kavapyrones [i.e., 210 mg]), thus identifying an effective kava product.

Integration of the findings

If the findings are considered believable and applicable to the patient, the next step is to incorporate the information together with the clinician's experience and the patient's wishes. In cases when there are several treatment options and the outcomes of the different choices are variable and unpredictable, it is often best to involve the patient in deciding if the benefit of the intervention is worth its costs (e.g., risk for side effects, inconvenience and expenses). To facilitate an informed decision, the patient should be informed of the benefits in absolute terms, whenever possible, and reasonable alternatives should be discussed.

Example 2B

“I'm impressed by the findings about kava and my patient seems to want to give it a try, so is there any reason not to?”

If impressed by these results, both in terms of validity and magnitude of benefit, the next critical issue is to determine if an equivalent formulation of kava is accessible to your patient. This may be most efficiently accomplished by contacting a local pharmacy about the availability of kava extract WS 1490. However, if this specific formulation is not available, the decision to use an alternative formulation of kava would be made knowing that the Cochrane systematic review findings may not be applicable. This would be important information to pass along to the patient, along with a review of the potential adverse effects, both serious and minor, and the overall risk of stopping therapy prematurely (due to inefficacy, adverse effects or other reasons). The alternative somatic therapies (i.e., antidepressants and benzodiazepines) ought to be reviewed similarly.

Conclusions

Rauwolfia for “insanity,” St. John's wort to treat “apparitions” and ginkgo for “brain disorders” have all been part of traditional medicinal treatment for psychiatric conditions for at least 3000 years. However, we should not translate length of use with either effectiveness or safety. Rather, herbal products need to be approached like other drugs, where the chance of doing harm (through side effects, drug interactions or needless expenditure) is respected, and the chance of doing good is approached with a healthy skepticism. This healthy skepticism should include consideration of the best evidence of the effectiveness of the herbal product along with the patient's values and the clinician's experience. More and more, available resources are capable of not only locating but also facilitating an efficient evaluation of the best evidence of an herbal therapy's potential benefit. When evaluating the best evidence specific to phytopharmaceuticals, there are additional issues that need to be considered. The clinician should be assured that a standardized manufacturing process (from harvest to dosage formulation) was followed and that currently available herbal products are manufactured equivalently. An evidence-based approach for responding to uncertainty related to herbal product therapy for mental disorders can be an efficient and effective means for facilitating collaborative and informed treatment decisions between patients and clinicians.