Does the Evidence Make a Difference in Consumer Behavior? Sales of Supplements Before and After Publication of Negative Research Results

Abstract

Objective

To determine if the public consumption of herbs, vitamins, and supplements changes in light of emerging negative evidence.

Methods

We describe trends in annual US sales of five major supplements in temporal relationship with publication of research from three top US general medical journals published from 2001 through early 2006 and the number of news citations associated with each publication using the Lexus-Nexis database.

Results

In four of five supplements (St. John’s wort, echinacea, saw palmetto, and glucosamine), there was little or no change in sales trends after publication of research results. In one instance, however, dramatic changes in sales occurred following publication of data suggesting harm from high doses of vitamin E.

Conclusion

Results reporting harm may have a greater impact on supplement consumption than those demonstrating lack of efficacy. In order for clinical trial evidence to influence public behavior, there needs to be a better understanding of the factors that influence the translation of evidence in the public.

BACKGROUND

Herbs, vitamins, and other dietary supplements are popular self-care products in the US.¹ In the last decade, the National Institutes of Health has devoted nearly $2 billion to studying complementary and alternative medicine (CAM),² including several major clinical trials on herbs, vitamins, and supplements (hereafter “supplements”).^3–5 In addition, other federal agencies have sponsored research synthesizing evidence from supplement trials over this same time period.⁶

The social value of clinical research derives from its potential to inform improvements in medical practice and public health.^7–9 For example, in the wake of evidence from large trials demonstrating harm from hormone replacement therapy and alpha-blockers, use of these products declined.^10,11 It is an open question whether research evaluating the safety and efficacy of supplements makes a comparable practical difference, especially as their use is often not mediated by a healthcare provider. Especially in the domain of supplements, whether and how findings published in medical journals make their way into popular information channels is unknown. Indeed, some commentators have challenged the public investment in CAM research altogether on the assumption that the results are unlikely to have any impact on CAM use.¹² Here we briefly describe supplement sales trends for five popular supplements published in the general medical literature in recent years and examine whether publication is temporally associated with changes in the public’s use of these agents.

METHODS

Using Pubmed, we identified key supplements that had been the subject of at least one study published in a major US general medical journal (New England Journal of Medicine, Annals of Internal Medicine, or JAMA) from 2001 through early 2006: St. John’s Wort, vitamin E, echinacea, saw palmetto, and glucosamine/chondroitin. We focused on these medical journals because news services tend to emphasize the articles in the major publications, and the number of news stories was our proxy indicator for public exposure to medical news. These supplements are some of the most commonly used in the US, and they represent a diverse range of supplement consumers and clinical conditions.

We assumed that if the research was having an impact, the sales of a supplement would change in a manner commensurate with the evidence in the subsequent annual sales trends. We acquired annual sales data through 2006 for studied supplements from the Nutrition Business Journal, a monthly executive newsletter that defines and tracks industry trends in the sales of nutritional products and supplements for the entire US market. In addition, we assumed that clinical research makes its way to the public by means of the popular press. Therefore, we determined the total number of US news and wire stories for each study for the period of 1 week prior to publication through 2 months after publication using the Lexis-Nexis database. We plotted annual sales in $US (y-axis) versus time (x-axis) and annotated graphs with time points indicating study publication dates, looking for changes in sales trends in the year immediately subsequent to the date of publication.

To determine the significance of temporal associations with publication, we fit a linear slope to the data for all time points prior to the publication date, calculated the 95% confidence interval (CI) around that slope, and then extrapolated for all time points after the publication date by extending the line and the corresponding CI lines. We then plotted expected and observed sales lines in order to determine whether the observed data post-publication fell outside of an expected range. Only those actual sales that fell outside of the 95% expected sales interval prior to publication were considered significant.

RESULTS

Sales trends for each of the five supplements in relationship to research publication are presented in Figure 1. The number of press citations for each study within 2 months of publication ranged from 39 to 71 (Table 1).

Figure 1.

Annual trends in sales of selected supplements in the US from 2000–2006 with dates of major negative publications.

Table 1.

Supplement studies published in major general medical journal

Agent	Pub Date	Number of Press Citations
St. John’s wort
18 JAMA. 2001;285(15):1978–86.	4/01	63
3 JAMA, 2002; 287:1807–1814.	4/02	57
Vitamin E
19 Ann Intern Med 2005; 142:37–46.	11/04 (epub)	67
20 JAMA 2005; 293:1338–1347.	3/05	40
21 JAMA 2005; 294(1): 56–65.	7/05	71
Echinacea
22 JAMA 2003;290(21): 2824–2830.	12/03	39
4 New Eng J Med 2005; 353(4): 341–348.	7/05	55
Saw palmetto
24 New Eng J Med 2006;354:557–566.	2/06	45
Glucosamine/chondroitin
5 New Eng J Med, 2006;354:795–808.	2/06	58

St. John’s Wort (Hypericum Perforatum)

St. John’s wort plant extract has been used to treat mild to moderate symptoms of depression.^13–15 US sales of St. John’s wort peaked at nearly $315 million in 1998. In 2000 concerns were raised about the safety of St. John’s wort because of its potential to interfere with anti-retroviral medications.^16,17 Two large studies of St. John’s wort for major depression published in 2001 and 2002 cast doubt on its effectiveness for major depression.^3,18 These two studies were covered in 63 and 57 new stories respectively around the time of publication. Changes in sales after publication of the study results in April 2001 and April 2002 were gradual and consistent with the trend prior to publication.

Vitamin E (alpha-tocopherol)

Vitamin E is a fat-soluble anti-oxidant that may mitigate oxidative stress linked to cancer and cardiovascular disease. In January 2005 a meta-analysis of major vitamin E trials raised concerns about higher mortality rates among study participants who took >800 IU of Vitamin E per day.¹⁹ In March and July of 2005, two additional studies reported little or no effect of vitamin E in preventing cancer and cardiovascular disease.^20,21 These studies were mentioned in 67, 40, and 71 news stories respectively for a total of 178 citations in the wake of these studies. These citation numbers included the two highest number of citations among all supplements we examined and together represent 50–300% greater news citations for a given sales year than any other supplement. After the 2005 publications, annual sales dropped sharply (down 33%) from previous sales levels and continued to decline at an accelerated pace the following year, which was significantly below expected.

Echinacea (Echinacea Angustafolia and Echinacea Purpurea)

Echinacea angustafolia and purpurea, two species of purple cone flower, have been used to treat cold symptoms and upper respiratory infections. Two prominent studies, one investigating Echinacea purpurea published in December 2003 and one investigating Echinacea angustafolia published in July 2005, showed no benefit in treatment and prevention of colds.^4,22 The only significant side effects were an allergic rash in children. These studies were mentioned in 39 and 55 news stories respectively. US annual sales of all echinacea products exceeded $200 million in 2000 and 2001. From 2003 to 2004, sales dropped modestly from $177 million to $153 million (down 14%). Sales between 2004 and 2005 reached a plateau and then dropped further in 2006 from $154 to $129 million (down 16%). These modest declines were not significant, suggesting no substantial immediate impact of the research on sales.

Saw Palmetto

The ripe fruit of the saw palmetto plant (serenoa repens) is used to treat urinary symptoms related to benign prostatic hyperplasia (BPH).²³ The largest US study to date published in February 2006 reported that while well tolerated, saw palmetto was no better than placebo in improving symptoms of BPH.²⁴ US annual sales of saw palmetto peaked in 2003 at $145 million. This study was mentioned 45 times in news stories around the time of publication. Annual sales dropped only slightly from $137 to $134 million between 2005 and 2006 (down 2.5%).

Glucosamine and Chondroitin

Supplemental glucosamine alone or taken in combination with chondroitin has gained popularity for treating symptoms of osteoarthritis. In February 2006, results from a large federally funded trial⁵ showed that glucosamine alone or in combination with chondroitin did not improve symptoms of osteoarthritis more than placebo. However, a pre-planned secondary analysis suggested that the glucosamine/chondroitin combination was more beneficial than placebo in a subgroup with moderate-to-severe osteoarthritis. There were no serious adverse events reported. This study was mentioned in 58 news stories around the time of publication. After publication of the study sales did not substantially change.

DISCUSSION

Whether and to what extend supplement research makes a difference to consumers is a multi-faceted public health research question that this study only begins to assess. When supplement research results report harm, they may be more influential to consumers and their physicians than results reporting lack of efficacy. Conversely, when research shows supplements to be safe but not necessarily effective, there may be little or no influence in public consumption. The case of vitamin E suggests it is possible for negative evidence revealing adverse effects from clinical research to cause dramatic declines in the consumption of supplements. Between 2004 and 2005 sales dropped dramatically (33%) comparable to declines in hormone replacement therapy after publication of research by the Women’s Health Initiative.¹¹

However, in the case of echinacea and saw palmetto, where the products were shown to be relatively safe but no better than placebo, declines in sales were more modest and consistent with secular sales trends. For glucosamine, publication of mixed outcome results was associated with no appreciable marginal change in product sales. And for St. John’s wort, reports regarding drug interactions predating publication of the large studies in 2001 and 2002, and smaller studies showing efficacy around the same time and subsequently ^25,26, may have prevented those studies from having any additional marginal impact on sales trends.

For this analysis we assumed that press exposure was an important mediator by which supplement trials might influence public behavior. The prominent studies we report here varied little in their number of Lexus-Nexis citations associated with their publication. However, vitamin E had the largest number of citations (71) and the highest total of citations (178), which exceeded the total of citations for the other four supplements by 50–300 percent. How the media covers a study and how that coverage does or does not persist could not be captured using our simple methods.

Multiple other factors not measured in this study likely contribute to whether results from clinical research on supplements have an impact. It is important to acknowledge that the number and significance of prior positive studies may influence whether sales after a single publication decline. For instance, if a given supplement has no side-effects and multiple previous studies showing potential benefit, one might expect minimal declines in sales after one negative result. Multiple studies on the same supplement published within months of each other, as was the case in vitamin E, may have promoted a stronger association with sales changes not seen in single studies. Furthermore, consumers of different supplements may vary their response to the evidence based on the specific purpose of the supplement, the type of supplement, and the pharmacological alternatives. The purposes for which vitamin E are taken (prevention) differ from saw palmetto and St. John’s wort (chronic symptoms) or echinacea (acute symptoms). Furthermore, vitamin E may be more likely to be recommended by physicians than the other supplements, making them more concerned about advising their patients in light of the emerging safety data.

The type of supplement may make a difference too. As suggested by Nahin,²⁷ consumers who use vitamin and mineral supplements had differences in their characteristics. They found that vitamin or mineral use was associated with female sex, white race, non-smoking, more years of schooling, difficulty walking, a history of osteoporosis, and reading health magazines, whereas non-vitamin, non-mineral dietary supplement use was primarily associated with residing in California and having difficulty with muscle strength. Accordingly, these different consumer groups may respond differently to new evidence. Finally, even if clear persuasive research were optimally reported and disseminated, consumers would still face persistent challenges in interpreting marketing messages they receive regarding supplements. For instance, in the wake of the glucosamine trial, manufacturers highlighted the secondary outcomes of the trial showing benefit, while excluding the results showing placebo equivalence. Consumers may be exposed to multiple sources of information including alternative medicine publications that may interpret the same data differently. Thus, achieving a public health impact for research on supplements may be particularly challenging in light of their current regulatory status and popular appeal.

Our approach has limitations. Temporal associations from sales trends cannot establish causation. At best our approach represents a quasi-experimental design in which the exposure was research publication. Such an approach threatens the internal validity of making accurate causal inferences related to the exposure.²⁸ We did not directly measure whether media coverage actually reached the consumers of interest, nor did we measure public attitudes toward the research results. And it is difficult to say a priori how much individual studies ought to change supplement consumption. It may not be reasonable to expect every individual study to show demonstrable market impact. In fact, it often takes a long-term clinical research investment in order for the weight of the evidence to reach a compelling “tipping point”, as seems to be the case with vitamin E. Thus, achieving a significant public health impact in supplement research may require a longer, more sustained research time horizon than some had initially anticipated.

CONCLUSION

Achieving public health impact should be a goal for all clinical research. Examining the potential public impact of clinical research publication on supplement consumption illustrates some of the challenges in translating research evidence into public health impact. Studies showing harm may more dramatically influence public behavior than studies only showing lack of efficacy. The impact of major studies indicating benefit remains unknown. To enhance the translational impact of supplement research, it may be fruitful to invest a small portion of supplement research funding to better understand and address the factors involved in public responsiveness to evidence from clinical research.