An Evidence-based Elective on Dietary Supplements

Abstract

Objective

To implement and evaluate the effectiveness of a pharmacy elective on dietary supplements that emphasized evidence-based care.

Design

A 3-credit elective that employed both traditional lectures and a variety of active-learning exercises was implemented. The course introduction provided a background in dietary supplement use and evidence-based medicine principles before addressing dietary supplements by primary indication.

Assessment

Student learning was assessed through quizzes, case assignments, discussion board participation, and completion of a longitudinal group project. Precourse and postcourse surveys were conducted to assess students' opinions, knowledge, and skills related to course objectives.

Conclusion

The course was an effective way to increase students' knowledge of dietary supplements and skills and confidence in providing patient care in this area.

INTRODUCTION

Dietary supplements are commonly used and account for substantial healthcare expense. As of 2004, nearly 50% of the US population used a vitamin or mineral supplement and 14% used an herbal supplement consistently for at least 1 year.¹ In 1998, 12% of the US population used herbal medicine and only 15% of that use was guided by a practitioner.² This common use of dietary supplements is reflected through the dollars spent annually by the US consumer. In 2005, more than $4 billion was spent on dietary supplements. Over 50% of these products were purchased at a drug store or supermarket. Specialty stores (eg, GNC, Vitamin Shoppe, Vitamin World) and natural food stores accounted for 12% and 9% of purchases, respectively, according to the Council for Responsible Nutrition (CRN).³ Seventy percent of patients who use complementary and alternative medicine, of which dietary supplement are a component, do not report this to their primary care physician. More than 1 in 3 Americans use a dietary supplement without consulting their doctor and this inarguably poses a risk to patient health. With millions of Americans passing through retail pharmacies to spend billions on dietary supplements, pharmacists can play a key role in addressing this issue.

The extensive use of dietary supplements highlights the importance of additional evidence-based training on dietary supplements for pharmacy students. The magnitude of dietary supplement use in the United States is not paralleled by course content in pharmacy education. As of 2003, only 15 out of 64 colleges/schools of pharmacy offered an elective in natural products and 20 offered an elective in natural products and complementary and alternative medicine.⁴ By 2005, 7 colleges and schools of pharmacy offered a course exclusively covering herbal supplements, 57 offered courses in which at least one third of the content covered herbal supplements, and 13 did not offer any instruction on herbal supplements.^4-7 Of concern, pharmacists and pharmacy students list the Internet, lay people, media, and product labels as their primary sources of information on herbal supplements.^5,8,9 We created an elective course in dietary supplements at the Albany College of Pharmacy and Health Sciences in response to informal student feedback indicating an interest in this area and the lack of any courses covering dietary supplements. In addition to covering current knowledge of dietary supplements, courses should prepare pharmacy students to assess the validity of emerging information. The importance of this is accented by the ongoing work of the National Center for Complementary and Alternative Medicine (NCCAM). As of March 2008, NCCAM had nearly 50 dietary supplements clinical trials that were either recruiting or not yet recruiting patients.¹⁰ While this is only a snapshot from one National Institutes of Health (NIH) center, it represents a significant amount of new information regarding the use of supplements. Future pharmacists need to be able to assess and incorporate this information prospectively.

DESIGN

The course, An Evidence-Based Approach to Dietary Supplements, was offered to third-year doctor of pharmacy (PharmD) students as a 3-credit elective. Course objectives were discussed with students and provided in the course syllabus. Goals were for students to be able to: (1) describe the federal regulatory process for the production and sale of dietary supplements in the United States; (2) discuss the potential problems and concerns with regulation of dietary supplements; (3) compare and contrast the advantages of available dietary supplement resources and references; (4) describe the body of scientific evidence supporting the safety and efficacy of individual dietary supplements reviewed during the course in terms of quantity, quality, and consistency; (5) identify knowledge gaps in the scientific evidence for dietary supplements; and (6) given a patient scenario involving the current or desired use of a dietary supplement, develop an evidence-based recommendation for the patient.

There were no prerequisites and the course met twice a week. The course was offered in spring and fall 2007. The college's computer policy requiring students to bring a laptop to class every day allowed a wide variety of in-class activities. The course had no required textbook but numerous required readings. In spring 2007, instructors included 1 academic pharmacist who had a clinical practice site in a community pharmacy, and 1 nonpharmacist faculty member who focused on research methods and evaluative techniques. Course enrollment, driven by strong student interest, doubled in fall 2007. This second offering was managed by only 1 PhD-trained nonpharmacist. No outside lecturers or guest speakers participated in or contributed to the course.

This study was granted exempt status by the IRB at the Albany College of Pharmacy.

Content

The course included a brief overview of complementary and alternative medicine and the majority of the content focused on dietary supplements. For this course, dietary supplements referred only to the following ingestible products and did not include any topical or aromatherapy applications: herbs, homeopathic substances, vitamins, minerals, and foods.

Topic selection was based on a survey of existing courses and emphasized areas of particular importance to dietary supplement use. Published descriptions of courses on dietary supplements and complementary and alternative medicine were consulted in constructing the course outline and approach.^11-13 Discussions of fundamental topics or issues in a supplements course also were consulted.^4,13 Course time was devoted to areas where there is adequate evidence supporting the use of dietary supplements as well as areas where supplement use is not supported by the evidence. The course content is outlined in Table 1. The course began with a discussion of complementary and alternative medicine and dietary supplement use, as well as a discussion of psychosocial issues relating to their use. Beliefs and attitudes among patients and healthcare professionals were discussed at the outset. This was intended to help students identify and discuss their preconceived ideas about supplements and to foster a better understanding of why people use them. Discussion also focused on instances in which supplements are part of a larger healthcare practice such as traditional Chinese Medicine or Ayurvedic Medicine.

Table 1.

Topics for Elective, Evidence-based Course on Dietary Supplements

The course also included a primer in the principles and practices of evidence-based medicine. Evidence-based medicine is “the integration of best research evidence with clinical expertise and patient values.”¹⁴ The application of evidence-based medicine fits naturally within a science-based curriculum employing the scientific method, which is vital to training pharmacists in care involving dietary supplements.¹³ By preparing students to critically evaluate the literature and identify the best evidence available, the course focuses on skills as much as knowledge. This course content greatly benefitted from and built significantly on previously completed required course content that addressed evidence-based medicine, clinical reviews, and literature analysis.

The evidence-based approach utilized in the course refers to a systematic review of many references on the safety and effectiveness of dietary supplements. Sources included primary literature located through PubMed,¹⁵ as well as the Natural Standard Database and Natural Medicines Comprehensive Database.^16,17 While these tertiary sources are fairly comprehensive and were used extensively, students were also required to consult primary literature, focusing on study design and analysis issues that may influence study results.

Common Internet search engines were also used extensively to find sites and advertisements for dietary supplements, though seldom to establish an evidence base. These lay sites represent consumers' data sources and are also a common source of information for pharmacists.^5,8,9 During the mental health lectures, for example, students searched depression and herbal in popular search engines. Marketing and educational materials found through these searches were evaluated and discussed in small groups and as a class. Discussion focused on how to handle this type of information through patient counseling. The goal was to train students not to memorize evidence on particular dietary supplements, but to learn how to formulate an evidence-based decision.

EVALUATION AND ASSESSMENT

A variety of approaches were used to assess student learning (Table 2). While the course employed an evidence-based approach, course assessment was less structured than a formal evidence-based course evaluation.

Table 2.

Assessment Methods Used in an Elective Course on Dietary Supplements

Quizzes

Quizzes covering lecture material were conducted every 2 to 3 weeks. Each 10-question quiz was completed electronically during class and typically took 15 to 20 minutes. Student performance on quizzes was consistent throughout the semester; average scores were typically around 90%.

Case Assignments

Case assignments were completed in small groups of 3 to 7 students and involved a variety of activities. Case 1 focused specifically on labeling and marketing claims of brand name dietary supplements. Students had to identify brand-name dietary supplements and quickly assess whether product claims could be substantiated and whether the claims were in compliance with federal regulations. In case 2, students had to use an evidence-based approach to evaluate a brand name combination product and assess its safety and effectiveness. Each ingredient in the combination product was investigated and discussed in order to make a general recommendation for the complete product. The groups shared their findings from cases 1 and 2 through in-class presentations. Case 3 required students to create summary grids of the course material. Each group covered a major indication (eg, women's health, bone, and joint) and provided information on commonly used dietary supplements and those that had significant effectiveness or safety features. This served both as a review and to provide the class with readily available summary information for future reference. (Case assignments, class activities, and quiz questions are available from the corresponding author by request.)

Student performance on case assignments varied considerably across the class, particularly on the first case where grades ranged from C to A. Overall student performance improved with each case such that the average grade for Case 3 was an A-.

Class Participation

Class participation was primarily assessed through the use of an online discussion board, which were administered through Blackboard Academic Suite (U.S. Patent No. 6,988,138) course management tools. Students were required to submit at least one new posting that contained a summary with a link to or copy of an article regarding dietary supplements in the popular media. Only popular media sources were allowed in order to simulate the types of information patients would readily have access to. Students also were required to make at least 4 discussion entries on existing posts. These discussions could include opinions and beliefs but had to be fact- or evidence-based. Students were required to complete a portion of their discussion board posts before the midpoint of the semester. Qualitatively, discussion board posts improved as the semester progressed and students became more critical of information regarding dietary supplements in the popular media.

Final Project

For the final project, students were required to design a proprietary blend dietary supplement. The final projects were presented in a poster presentation on the last day of the course. Students had to create 2 sets of promotional material. The first set had to comply with federal regulations and have a strong evidence base, while the second set could contain anything that they found in a real advertisement for dietary supplements. Students were evaluated partly on their ability to justify the inclusion or exclusion of key ingredients in their product. One goal of this project was to show students what patients encounter when they make a decision about using dietary supplements and how that information is created. Final project topics included products enhancing mental acuteness, improving skin quality, lowering cholesterol and blood pressure, and “curing” hangovers. Final project grades ranged from B- to A.

In-class Activities

In addition to the formal student assessment, numerous in-class activities were completed both in groups and individually. Each topic area had a variety of hypothetical case assignments built into it. For example, during women's health, the following case was presented: “A 30-year old woman who is 15 weeks pregnant comes into the pharmacy complaining of nausea. She is looking for a natural way to treat it. What would you recommend to her? What does the literature suggest? What does the manufacturer claim? Is there any safety information? What about during pregnancy?”

Groups were assigned to research an herb, vitamin or mineral, or combination product that the patient might consider. As with all issues pertaining to pregnancy, patient counseling points always started and ended with strongly advising patients to discuss the product with their physician before taking it. Students were given a few minutes to research some or all of these answers and then informally present and defend their answers to the class. In-class activities were not graded but were similar in format and content some graded class activities. Qualitatively, as the semester progressed, student responses improved and students required less prompting for class discussion.

Survey

A survey was administered to students on the first and last day of classes. The survey consisted of 10 knowledge-based questions worth 1 point each and 5 Likert-scale items. Students were asked to rate the statements using a 5-point Likert scale ranging from strongly agree to strongly disagree: (1) Commonly used dietary supplements are generally effective; (2) Commonly used dietary supplements are generally safe; (3) Current regulations for dietary supplements are generally adequate; (4) I am confident in my ability to provide pharmaceutical care to patients using dietary supplements; and (5) The dietary-supplement information resources currently available are adequate to assist in providing care to patients using these agents. The questions contained a sixth “not applicable” category which had no responses.

All statistical analyses were conducted using Stata, version 9.2 (StataCorp, College Station, TX). The Fisher exact test was estimated to compare the distribution of Likert scale responses before and after the course. The Fisher exact test also was used to compare the proportion of students who agreed or strongly agreed with each of the statements before and after the course. The Mann-Whitney test was used to compare Likert scores before and after the course. A t test was used to compare the assessment survey's knowledge-based scores. Unpaired tests were used because some students may have completed only the pre- or postcourse survey instrument. A 2-sided alpha of 0.05 was used for all tests.

Twenty-six students enrolled in the course in spring 2007 and 49 students in fall 2007. Response rates for the course assessment survey were 88% (precourse) and 96% (postcourse) for year 1, and 96% (precourse) and 88% (postcourse) for year 2, for aggregate response rates of 93% (precourse) and 91% (postcourse). Responses to Likert-scale items are given in Table 3 and summarized below.

Table 3.

Pharmacy Students' Achievement of Course Objectives in an Elective on Dietary Supplements (Precourse N = 63; Postcourse N = 69)

Student opinion of dietary supplement effectiveness did not change after the course. The distribution of responses to the statement “Commonly used dietary supplements are generally effective” was the same (p = 0.66), and the mean score was unchanged (3.1 precourse and postcourse, p = 0.79), as was the proportion of students who agreed with the statement (36% precourse vs. 32% postcourse, p = 0.720).

Student opinions of the general safety of dietary supplements appeared to change before and after the course. While the distribution of responses to the item “Commonly used dietary supplements are generally safe” was not significantly different (p = 0.094), the change in mean response was significant (3.3 precourse vs. 2.9 postcourse, p = 0.01), as was the proportion of students who agreed with the item (26% precourse vs. 43% postcourse, p = 0.049).

Students' opinion of the adequacy of dietary supplement regulations did not change through the course as students still firmly believed that regulation of dietary supplements was inadequate upon completion of the course. The distribution of responses was the same (p = 0.41) and the mean response was unchanged (4.2 precourse vs. 4.3 postcourse, p = 0.36), as was the proportion of students who agreed with the statement (6% precourse vs. 3% postcourse, p = 0.681).

By the end of the course, students were more confident about providing pharmaceutical care involving dietary supplements. (p < 0.01). The distribution of responses was significantly different (p < 0.0001), the mean response was greater (3.9 precourse vs. 2.2 postcourse, p = 0.002), and the proportion of students who stated that they were confident providing care involving dietary supplements also significantly increased (4% precourse vs. 79% postcourse, p < 0.0001).

By the end of the course, significantly more students believed that resources for current information on dietary supplements were available to assist in providing patient care using dietary supplements. The distribution of responses was significantly different (p = 0.003) and the mean response was significant (3.1 precourse vs. 2.6 postcourse, p = 0.002), as was the proportion who agreed with the statement (32% precourse vs. 57% postcourse, p = 0.003).

The change in the mean scores on the 10 knowledge-based questions was significant (7.6 precourse vs. 8.0 postcourse, p = 0.048).

Course Evaluation

The course was also assessed using a standard course-evaluation tool at the middle and end of the course. Overall, students reviewed the course very well, giving positive reactions to both the course content and the course format.

DISCUSSION

This course had a clear, though somewhat modest, impact on student knowledge, skills, and attitudes. The course improved student knowledge and skills as evidenced by the course evaluation survey. While this was a basic assessment, it did indicate a strong trend towards increased knowledge. The large shift in opinions regarding dietary supplements may have been due to the vast majority of adverse events regarding dietary supplements involve a relatively small number of supplements. For example, in 2001, the US Food and Drug Administration (FDA) received more than 1,000 adverse event reports on ephedra¹⁸ compared to 654 adverse event reports for all other herbal products.¹⁹ Likewise, kava was responsible for 59 adverse event reports; echinacea, for 69; and ginseng for 46.¹⁹ Learning which dietary supplements commonly cause adverse reactions likely increases the perception that, as a whole, dietary supplements are safe.

Student survey results regarding the effectiveness of dietary supplements are substantively similar if not lower than the estimates found by Shah et al.⁵ Shah conducted a cross-sectional survey of pharmacy students regarding their beliefs and perceptions of herbal supplements not directly related to a course or intervention and found that 43% of pharmacy students agreed that herbal supplements are generally effective. Likewise, student opinions of the safety of dietary supplements were substantively similar to those of Shah, who found that 34% of pharmacy students felt that herbal supplements can be safely taken with a prescription.⁵ Similar to the assessment conducted by Shah, these items assess general attitudes and do not identify specific dietary supplements.

The survey results indicate that pharmacy students did not and still do not believe that regulations on dietary supplements are adequate. The course clearly did not change students' opinions, nor did it intend to. Considering the presence of strong condemnations of dietary supplements regulations, these findings are not surprising.^13,20 It would be surprising if pharmacy students were supportive of dietary supplement regulations. A 1999 survey found that 75% of respondents wanted the government to verify all health claims in advertisements and product labels, as well as review safety data for dietary supplements prior to sale, effectively giving the government increased regulatory authority over dietary supplements.²¹

The course significantly and meaningfully increased student confidence in providing patient care involving dietary supplements. This may be due to the increased knowledge of available information resources or the increased skills developed through class activities and patient cases. This may also be due, in part, to the sheer number of dietary supplements discussed in class. At least as important as the coverage of dietary supplements is instruction in methods to evaluate them. Given the number of ongoing dietary supplements trials, having a course in which students memorize which products do or do not work is an inefficient model and does not prepare them to respond to ever-changing patient information needs.⁴ This course, instead, trained pharmacy students how to determine the level of evidence available for using or not using a dietary supplement and provide patient care that balances safety, effectiveness, and cost. Changes in student confidence may have been due in part to increased knowledge of the available resources about dietary supplements.

While the changes in student knowledge, as assessed using 10 knowledge-based questions, was statistically significant, is unclear if a 4% change is practically significant or meaningful.

One of the first goals in the building of this course was to create a reproducible educational experience; reproducible in the sense that it could be used both within our college as well as at other colleges and schools of pharmacy. This course represents an important component of a future pharmacist's education both in terms of content and approach.

SUMMARY

This 3-hour elective on dietary supplements both addressed a growing student demand for information in this important area and filled a gap in the pharmacy curriculum. Survey results indicated increases in students' confidence, skills, and knowledge concerning dietary supplements. Whether these changes will be lasting or directly impact patient care involving dietary supplements is not known.