Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Oct 12.
Published in final edited form as: J Sex Med. 2019 Apr 26;16(6):843–852. doi: 10.1016/j.jsxm.2019.03.269

An Analysis of Popular Online Erectile Dysfunction Supplements

Adithya Balasubramanian 1, Nannan Thirumavalavan 2,3, Ashwin Srivatsav 1, Justin Yu 1, James M Hotaling 4, Larry I Lipshultz 2,3, Alexander W Pastuszak 4
PMCID: PMC9555224  NIHMSID: NIHMS1526021  PMID: 31036522

Abstract

Introduction:

Erectile dysfunction supplements (ED-Ss) are featured on online marketplaces like Amazon.com, with dedicated pages and claims that they naturally treat ED. However, their efficacy and safety are largely unknown, limiting the ability to counsel patients regarding their use.

Aim:

To evaluate the highest rated and most frequently reviewed ED-Ss on Amazon.com to facilitate patient counseling regarding marketing myths, ingredient profiles, and evidence for product efficacy and safety.

Methods:

The Amazon marketplace was queried using the key term “erectile dysfunction” with default search settings and ranking items based on relevance. The top 6 ED-S products identified on September 29, 2018, were reviewed based on price, ratings, reviews, manufacturer, and ingredients. Consumer reviews were categorized using subtopics within the International Index of Erectile Function (IIEF) questionnaire to better understand ED-S efficacy and then reanalyzed following filtration of untrustworthy comments using ReviewMeta.com, a proprietary Amazon review analysis software.

Outcomes:

Quantitative and qualitative evaluation of ED-S products sold on Amazon.com.

Results:

The top 6 ED-Ss had an average of 2,121 ± 1,282 reviews and a mean rating of 3.92 ± 0.42 stars. A total of 21 ingredients were identified in these ED-Ss. Ginseng, horny goat weed, L-arginine, and tongkat ali were the most popular ingredients included in the analyzed products. Our literature review identified 413 studies involving the 21 identified ingredients, of which 59 (16%) involved human subjects. Among these 69 human studies, only 12 (17%) investigated supplement ingredients individually and reported improvement in ED. Analysis of top-ranked customer reviews from the first 2 pages of reviews for each supplement revealed differences in IIEF scores before and after ReviewMeta.com filtration. After filtration, we observed a 77% decrease in reviews reporting improved erection strength, an 83% decrease in reviews reporting improved ability to maintain erection, a 90% decrease in reviews reporting increased sexual satisfaction, an 88% decrease in reviews reporting increased enjoyment with intercourse, and an 89% decrease in reviews reporting increased erection confidence.

Strengths & Limitations:

Study strengths include a novel approach to ascertaining consumers’ perceptions and satisfaction with ED-Ss and practical summary information that clinicians can provide to patients. Limitations include selection bias, the small number of supplements analyzed, and the proprietary nature of the Amazon review analysis software.

Conclusions:

Our investigation revealed that human studies evaluating the efficacy of ED-S ingredients are limited and have yielded no definitive findings of the effects on ED. Patients considering ED-S use should receive appropriate counseling, given the prevalence of disingenuous reviews and the ready availability of Food and Drug Administration–approved drug therapies.

Keywords: Erectile Dysfunction, Supplements, Nutraceutical, Internet

INTRODUCTION

Erectile dysfunction (ED) is defined as the inability to achieve or maintain an erection sufficient for satisfactory sexual performance.1 ED is highly prevalent, affecting upward of 52% of men age 40–70 years in the United States alone.2 The pharmacologic management of ED continues to evolve.3 Following the approval of sildenafil in 1998, 2 other oral phosphodiesterase-5 (PDE-5) inhibitor medications, tadalafil and vardenafil, were introduced in 2003.

Despite this considerable pharmacologic innovation, patients seeking treatment for ED continue to face notable barriers to care. Frederick et al4 found that ED is undertreated in the United States in light of patients’ reticence in discussing the condition with physicians, high medication costs, inadequate insurance coverage, and failure to respond to PDE-5 inhibitor therapy. Le et al5 corroborated these findings, noting that reimbursement and insurance policies for ED lack transparency and impede patients from pursuing appropriate care for the condition. The authors concluded that the combination of psychosocial and fiscal barriers were encouraging patients to turn to the online marketplace to research and acquire alternative medical therapies for ED.5

Cui et al6 recently reported that patients are using herbal and nutrient supplements to address various men’s health issues, including ED. Erectile dysfunction supplements (ED-Ss) are composed of a heterogeneous array of ingredients, with popular ingredients including ginseng, maca, and epimedium, among others.7 Pursuant to the Dietary Supplement Health and Education Act of 1994, the US Food and Drug Administration (FDA) does not regulate ED-S products or other supplements.8 This policy has permitted ED-S products to gain traction without the scrutiny given to conventional pharmacotherapies. ED-S marketing materials often claim that these products can naturally improve erectile function as effectively as FDA-approved therapies, such as sildenafil.9 While ED-S products are proliferating, investigations of these supplements have shown that these products are often mislabeled or adulterated with therapeutic doses of conventional PDE-5 inhibitors.9–13

Online marketplaces like Amazon.com are increasingly popular sources for acquiring health and nutritional supplements.14,15 The popularity of electronic commerce (e-commerce) is projected to continue to grow owing to increasing consumer preference for the online marketplace over traditional physical retail stores.14,16 ED-S products are prominently featured on Amazon, with numerous dedicated pages and claims that they can “naturally” improve ED. The presence of ED-Ss in the Amazon marketplace ensures their considerable visibility and ready consumer accessibility.14

Customers acquiring products online, including ED-Ss, often consult user reviews featured on product pages to validate product efficacy. This consumer behavior is promoted by the broader online phenomenon of social commerce, which encourages consumers to post online product reviews. These product reviews, often featured on Amazon, have come under increasing scrutiny owing to disingenuous behavior by vendors, including fake reviews generated by “collusive spammers,” as well as paid reviews.17–24 The impact of fake product reviews is notable given consumers’ general high levels of trust in health-related information available from online portals.25

Physicians should be aware of the most popular ED-Ss, owing to these products’ easy obtainability, the high consumer interest in these products, and frequent questions that arise during patient consultations. In the present study, we evaluated the most highly rated and frequently reviewed ED-S on Amazon.com to facilitate patient counseling regarding marketing myths, ED-S formulations, and evidence of efficacy of these supplements. Specifically, we aimed to evaluate the underlying evidence of the efficacy of ED-S ingredients and to determine whether ED-S product reviews are from actual users or as part of a digital marketing strategy.

METHODS

Identification of the Top 6 ED-S Products

ED-Ss were identified by querying the Amazon.com website using the key term “erectile dysfunction” (Figure 1). The 6 highest-ranked supplements on Amazon, using default search settings that sort and present featured items, on September 29, 2018, were selected for further examination. Supplement information, ingredients, and comments from the initial 2 available pages of reviews were collected and stored offline to ensure consistency in data analysis, given the dynamic and routinely updated online marketplace. Amazon allows consumers to validate reviews via a “helpful” button under each submitted product review. Comments that receive the highest number of “helpful” clicks are presented on the first 2 pages of reviews. Reviews from these 2 pages were selected for analysis in this study, because other consumers have vouched for these comments. In addition, comments from the first 2 pages are automatically presented to consumers who browse product pages. 6 supplements were chosen to provide a holistic survey of popular ED-S products available in the Amazon marketplace.

Figure 1.

Figure 1.

Open in a new tab

Overview of the investigative method.

Assurance of Organic Search Results

A new Amazon account was created to minimize bias from user profiling techniques, including third-party tracking and browser cookies.26 Bias was further mitigated by using browser-based private mode features for all online data acquisition. Additional efforts to ensure the integrity of search results involved installing uBlock Origin (https://github.com/gorhill/uBlock), a third-party software that blocks advertising and query tracking.

Analysis of ED-S Ingredients

Supplement ingredients were aggregated and cataloged. Ingredients were classified into active ingredients versus filler or inactive ingredients formally known as excipients.27 A PubMed search was performed for all active ingredients in conjunction with the key term “erectile dysfunction.” A systematic review was undertaken to classify articles based on study population, including human studies, animal studies, in vitro studies, and other studies (eg, case reports, review articles). Human studies were further stratified into investigations that were not supplementation-oriented and studies in which the ingredient was combined with multiple other ingredients. The remaining identified studies that used the International Index of Erectile Function (IIEF) questionnaire were reviewed to assess the impact of each ingredient on erectile function.

Analysis of User Comments

All available Amazon reviews from the first 2 product review pages were aggregated for each product. The IIEF questionnaire (Supplementary Figure 1), a validated tool for identifying and classifying ED symptoms and assessing response to ED therapies, was used to analyze reviews for ED-S efficacy.28 Two raters (A.B. and A.S.) coded all aggregated comments using the IIEF questionnaire. An qualitative analysis of an Amazon comment using the IIEF is diagrammedin Figure 2.The IIEFquestionnairewas usedas a checklist to grade each comment. As the comment was read, each reviewer analyzed whether the comment’s content was relevant to a particular IIEF domain. If, for example, the comment contained content related to erection frequency or erection strength, then the reviewer independently noted that the comment was pertinent to that IIEF domain. Interrater reliability was determined using the k statistic.29 A third rater (N.T.) determined the final IIEF scoring of a comment if a difference between raters was observed.

Figure 2.

Figure 2.

Open in a new tab

Breakdown of qualitative analysis of Amazon reviews using the International Index of Erectile Function (IIEF) questionnaire.

ReviewMeta Screen

A proprietary web-based online review analyzing service, ReviewMeta, was used to analyze the integrity of review comments for each supplement. ReviewMeta analyzes products hosted on the Amazon marketplace using 11 filters: suspicious reviewers, reviewer ease, rating trend, unverified purchases, word count comparison, phrase repetition, overlapping review history, reviewer participation, brand repeats, incentivized reviews, and deleted reviews. ReviewMeta identifies the percentage of overall comments that are considered suspect. The ReviewMeta algorithm also assigns a score of “pass,” “warn,” or “fail” under each of the 11 domains to provide additional insight into patterns underlying suspicious reviews. Once the filtering process is complete, ReviewMeta produces a ranked list of the most trustworthy comments. After ReviewMeta filtering, the number of failed categories for each supplement was correlated with the percentage of eliminated untrustworthy comments using Pearson correlation. All trustworthy comments returned via ReviewMeta analysis were then analyzed using the IIEF questionnaire as detailed above.

RESULTS

ED-S Ingredients

The top 6 ED-S products sold on Amazon on September 29, 2018, included Korean Panax Ginseng (NutraChamps, Toronto, ON, Canada), Leyzene with Royal Jelly (Natural Subsistence, Wadsworth, OH, USA), Horny Goat Weed Extract (Zhou Nutrition, Park City, UT, USA), Boost Elite (Zhou Nutrition), Extra Strength L-Arginine (Havasu Nutrition, Sunrise, FL, USA), and IncrediBULL (eSupplements, Vineyard, UT, USA). These supplements, including manufacturers, cost per unit, and ingredient profiles, are listed in Table 1. The top 6 ED-S products had an average of 2,121 ± 1,282 reviews (range 37–3,728), and an average rating of 3.92 ± 0.42 stars (range 3.4–4.5 stars).

Table 1.

Cost and ingredient profile of the top 6 ED-S products

Parameter Korean Panax Ginseng (NutraChamps) (NutraChamps) Leyzene with Royal Jelly (Natural Subsistence) Horny Goat Weed Extract (Zhou Nutrition) Boost Elite (Zhou Nutrition) Extra Strength L Arginine (Havasu Nutrition) IncrediBULL (eSupplements)
Cost per pill, $ 0.18 2.2 0.25 0.35 0.3 3.45
Active ingredients
 AAKG x
 Piperine x
 Capsicum annuum x x
 Diindolylmethane x
 Epimedium x
 Fenugreek seed x
 Ginseng (panax ginseng, ginsenosides) x x x x x
 Gluconolactone x
 Horny goat weed x x x
 L-arginine x x x
 L-citrulline malate x
 Maca root x x
 Muira puama x x
 Polypodium vulgare x
 Rehmannia root x
 Royal jelly x
 Saw palmetto berry x
 Tongkat ali x x x
 Tribulus terrestris x x
 Yohimbe bark x
 Zinc x
Inactive ingredients
 Gelatin x
 Magnesium stearate x
 Rice flour X x x x
 Silica x x
 Vegetable cellulose x x x
 Vegetable magnesium stearate x x x
Total ingredients 2 8 13 13 7 6
Active ingredients 1 6 10 9 4 5
Inactive ingredients 1 2 3 4 3 1
Open in a new tab

AAKG = (L-arginine-alpha-ketoglutarate).

All 6 ED-Ss were sold by 5 unique US-based manufacturers, and 3 were identified as being produced in “FDA registered” facilities. The average cost per bottle was $23.83 ± $7.56 (range $15.17 to $34.50). The average suggested dose was 1.8 pills (range 1–3 pills), with an average cost per pill of $1.12 ± $1.40 (range $0.18–$3.45). One of the 6 supplements was designated as an Amazon Choice purchase. The packaging of all 6 ED-Ss reported quantities of individual ingredients. A total of 21 active and 6 inactive ingredients were identified across the 6 supplements, and the supplements had an average of 8.2 ± 4.3 ingredients (range 2–13). Ginseng, L-arginine, tongkat ali, and horny goat weed were the 4 most commonly included active ingredients.

The 6 ED-Ss contained variable amounts of these ingredients per dose. Ginseng was present at an average of 254 ± 420 mg per dose, with a range of 20–1,000 mg across the 6 supplements. A similar degree of variability was seen for the other 3 most popular ingredients (Table 2).

Table 2.

Summary of systematic PubMed literature review for the 10 most common ED-S ingredients

Inclusion in PubMed Searchable Studies
 Impact on Erectile Dysfunction in Human Studies
Ingredients Number of supplements included Human studies Animal studies In vitro studies Other Total Not supplementation oriented Combined with other ingredients Studies using non-IIEF outcomes Pre-/post-intervention IIEF score measured Positive effect Negative effect No effect Indeterminate
Ginseng (panax ginseng, ginsenosides) 5 6 9 2 24 41 0 1 1 5 5 0 0 0
L-arginine 3 36 144 37 39 256 15 14 3 4 4 0 0 0
Tongkat ali 3 0 3 1 6 10 0 0 0 0 0 0 0 0
Horny goat weed 3 0 1 1 1 3 0 0 0 0 0 0 0 0
Tribulus terrestris 2 7 5 1 11 24 0 3 0 4 2 0 1 1
Maca root 2 1 0 1 5 7 0 0 0 1 1 0 0 0
Muira puama 2 1 2 0 0 3 0 1 0 0 0 0 0 0
Capsicum annuum 2 0 0 0 0 0 0 0 0 0 0 0 0 0
Zinc 1 8 6 1 17 32 2 1 5 0 0 0 0 0
Epimedium 1 2 8 1 6 17 0 2 0 0 0 0 0 0
Saw palmetto berry 1 5 3 0 7 15 1 3 0 1 0 0 1 0
Fenugreek seed 1 1 0 0 1 2 0 0 1 0 0 0 0 0
Yohimbe bark 1 0 0 0 2 2 0 0 0 0 0 0 0 0
Diindolylmethane 1 1 0 0 0 1 0 0 0 1 0 0 1 0
AAKG 1 0 0 0 0 0 0 0 0 0 0 0 0 0
Piperine 1 0 0 0 0 0 0 0 0 0 0 0 0 0
Gluconolactone 1 0 0 0 0 0 0 0 0 0 0 0 0 0
L-citrulline malate 1 0 0 0 0 0 0 0 0 0 0 0 0 0
Polypodium vulgare 1 0 0 0 0 0 0 0 0 0 0 0 0 0
Rehmannia root 1 0 0 0 0 0 0 0 0 0 0 0 0 0
Royal jelly 1 0 0 0 0 0 0 0 0 0 0 0 0 0
Total 69 181 45 119 413 18 25 10 16 12 0 3 1
Open in a new tab

AAKG = (L-arginine-alpha-ketoglutarate).

Ingredient Analysis

A PubMed analysis as detailed in Methods was undertaken for the 21 active ingredients used across all supplements (Table 2). The number of studies examining each of these ingredients was determined, and studies were grouped based on in vivo human and animal studies, in vitro studies, and other types of studies. The PubMed search, which included the ingredients as well as the term “erectile dysfunction,” yielded 413 studies mentioning these active ingredients, of which 68 (16%) involved human subjects, 181 (44%) involved animal models, 45 (11%) were in vitro studies, and 119 (29%) were other types of studies (eg, case reports, review articles). There was no uniform distribution of articles among the most common ingredients. An average of 19.7 ± 55.0 articles were identified per ingredient (range 0–256), with no published studies for 8 of the 21 ingredients (38%). The most extensively studied ingredient was L-arginine, with 256 articles, including 36 studies in humans. Ginseng had 41 human studies, and zinc had 32 human studies. Among the analyzed ingredients with a minimum of 5 associated studies, saw palmetto berry had the highest ratio of human studies to total studies (33%; 5 of 15).

Among the 69 human studies, only 26 (38%) investigated ingredients individually, and of these, 16 (62%) used pre- and post-IIEF measurements to monitor response to ingredients. Among these studies, 12 (75%) observed an improvement in ED, none reported a negative effect, 3 (19%) reported no effect, and 1 (6%) had indeterminate data. Accurate comparisons among the 68 human studies were not possible, given the nonuniform dosing and sampling timelines. The study designs, demographics, and inclusion criteria of these studies are summarized in Supplementary Table 1.

Analysis of Supplement Comments Before ReviewMeta Filtration

User comments on Amazon were evaluated to gain insight into the benefits and efficacy of the ED-S products. All reviews from the top 2 pages of comments were collected for each supplement, resulting in a total of 90 reviews, and analyzed using the IIEF questionnaire as described in Methods. The k statistic for interrater reliability was 0.94, indicating strong agreement among our raters before a third rater addressed any discrepancies.30 Results from the qualitative analysis of comments via IIEF from the first 2 product pages before ReviewMeta filtration are summarized in Table 3 and graphically presented in Figure 3. We found that many reviewers reported improved erection strength (36%), improved ability to maintain an erection overall (29%), increased sexual satisfaction (32%), increased enjoyment with sexual intercourse (27%), increased satisfaction with sex life (27%), and improved erection confidence (31%). 53 of 90 comments (58%) from the first 2 pages of reviews did not contain content relevant to any IIEF questions or domains.

Table 3.

Summary of qualitative analysis of comments via the International Index of Erectile Function (IIEF) before and after ReviewMeta.com filtration

IIEF category Comments before ReviewMeta filtration, % (n/N) Comments after ReviewMeta filtration, % (n/N) Difference between pre- and post-ReviewMeta filtration, %
Improvement in erection frequency 11.11 (10/90) 1.67 (1/60) 85.0
Improvement in erection strength 35.56 (32/90) 8.33 (5/60) 76.6
Improvement in ability to penetrate partner 2.22 (2/90) 1.67 (1/60) 24.8
Improvement in ability to maintain erection after penetration 5.56 (5/90) 3.33 (2/60) 40.1
Improvement in ability to maintain erection overall 28.89 (26/90) 5 (3/60) 82.7
Number of intercourse attempts 2.22 (2/90) 1.67 (1/60) 24.8
Increased sexual satisfaction 32.22 (29/90) 3.33 (2/60) 89.7
Increased enjoyment with sexual intercourse 26.67 (24/90) 3.33 (2/60) 87.5
Improved ability to ejaculate on stimulation or intercourse 1.11 (1/90) 0 (0/60) 100.0
Improved ability to orgasm on stimulation or intercourse 2.22 (2/90) 0 (0/60) 100.0
Increase in sexual desire (libido) 23.33 (21/90) 1.67 (1/60) 92.8
Increased satisfaction with sex life 26.67 (24/90) 3.33 (2/60) 87.5
Increased satisfaction with relationship with partner 22.22 (20/90) 5 (3/60) 77.5
Increased erection confidence 31.11 (28/90) 3.33 (2/60) 89.3
Open in a new tab

Figure 3.

Figure 3.

Open in a new tab

Comparison of the International Index of Erectile Function (IIEF) comments between pre- and post-ReviewMeta filter analysis.

Analysis of Supplement Comments Following ReviewMeta Filtration

All supplements were filtered using the proprietary ReviewMeta service as detailed in Methods to analyze malicious review trends and eliminate untrustworthy comments. As described previously, the algorithm uses 11 categories as described in Methods and determines whether reviews of a product “pass,” “warn,” or “fail” under each domain. We correlated the number of “fail” results with the number of comments that were removed and found a positive correlation (R = 0.8484; P < .05), indicating that the more “fail” results a supplement had, the greater the likelihood that it had questionable reviews (Supplementary Figure 2). There were a total of 12,840 reviews for the 6 ED-S products, of which 48% were considered untrustworthy by ReviewMeta filtration and excluded from our analysis. 37 of 60 comments (61%) following ReviewMeta filtration did not contain content relevant to any IIEF questions or domains.

After filtration, 60 user comments were reevaluated using the IIEF questionnaire to semiquantitatively determine the effects of individual ED-S products on quality of life. The k statistic was 0.96,indicating very high interrater agreement before resolution of disagreements by a third rater.30 Results from the qualitative analysis of comments filtered by ReviewMeta via IIEF are summarized in Table 3 and graphically presented in Figure 3. After filtering supplement reviews using the ReviewMeta service, considerable differences in both overall and the popular IIEF categories identified before filtration were observed. We identified a 77% decrease in users reporting improvements in erection strength and an 83% decrease in users reporting improvements in ability to maintain an erection overall. Similarly, we observed a 90% decrease in users reporting increased sexual satisfaction and an 88% decrease in users noting increased enjoyment with sexual intercourse. Furthermore, we observed an 88% decrease in users noting increased satisfaction with their sex life and an 89% decrease in users reporting increased erection confidence. 21 of the 60 filtered comments (35%) explicitly mentioned negative side effects associated with the ED-S products, most commonly a lack of response (86%; n = 18), headache (14%; n = 3), increased blood pressure (9.5%; n = 2), nausea (9.5%; n = 2), dizziness (5%; n = 1), anxiety (5%; n = 1), and heartburn (5%; n = 1).

DISCUSSION

The psychosocial burdens of ED can lead men to experience feelings of shame, guilt, or stigma.31,32 These associated stressors may contribute to patients’ reluctance to discuss ED symptoms and management with providers.4,33 Although efforts such as the “process of care” model are routinely updated in an effort to standardize ED diagnosis and management, similar initiatives have not been undertaken for Internet content focused on ED.34 This lack of oversight is critical, given that people are increasingly receiving health care information via the Internet.35 Globally, Zhang et al36 reported that younger Chinese patients are using the Internet as their default location to learn about ED specifically. Read et al37 and Amber et al38 have demonstrated that online information about ED is often biased. As e-commerce becomes the most popular avenue for obtaining goods for American consumers, investigations of the information presented on ED-S product pages are warranted, given the generally high level of trust in online information.14,16,25 The increasing influence of the Internet, along with the ready availability of supplements oriented toward ED, underscore the importance of understanding ED-S products sold on Amazon.com.

In this study, the top 6 ED-S products listed on Amazon.com on September 29, 2018, were identified and found to be manufactured by 5 unique US manufacturers. Of note, quantities of each ingredient were transparently indicated on the labels of each supplement despite flexible reporting regulations.39 This information enabled us to determine that each ED-S had a heterogeneous ingredient composition. The variability in doses of popular ingredients, such as ginseng, L-arginine, horny goat weed, and tongkat ali extract, raises the question of why specific combinations of ingredients were chosen for each supplement.

We also found that only one-half of these supplements were marketed as being assembled in “FDA-registered” facilities. Although this labeling should be viewed with caution, given that the FDA does not formally review supplements, the lack of any statement regarding standardized manufacturing practices in the remaining ED-S products raises concerns about actual supplement composition. This is particularly significant given a string of recent investigations revealing that some “natural” supplements contain active pharmaceutical ingredients.13,40–43 Patients should also be counseled that these ingredient concerns are not unique to herbal or natural supplements targeted at ED. Chiang et al10 reported that even compounded PDE-5 inhibitors have been counterfeited, even though they are subject to more stringent regulations related to drug manufacturing. Taken together, these findings support the importance of healthcare practitioners counseling patients to obtain ED treatment from reputable sources.

A review of the literature examining ED-S highlights the limited amount of available efficacy data. We observed that only 69 of the 413 studies that we identified (17%) were undertaken in human subjects. Of these 69 human studies, only 26 (38%) investigated these active ingredients individually, whereas 25 (36%) evaluated a combination of active ingredients. Patients should be counseled that the current body of evidence examining the efficacy of ED-S product ingredients is derived from studies using heterogeneous supplementation protocols. The lack of uniformity across these studies highlights concerns about the variable ingredient profiles and online customer reviews claiming that these supplements can dramatically improve ED symptoms.

Nearly 50% of the reviews of the top 6 ED-S products were excluded following filtration using the ReviewMeta.com software. The removal of nearly one-half of all reviews associated with these products raises concerns about the veracity of comments linked to these products. The positive linear correlation that we identified between the number of fails per category and untrustworthy comments as determined by ReviewMeta.com further suggests that product reviews hosted on online marketplaces like Amazon.com should be interpreted with caution, and that product reviews might artificially overinflate supplement efficacy.

We observed that many reviews did not explicitly state the supplementation regimen associated with ED-S products. The lack of specificity regarding dosages is important, given the variable ingredient compositions of the supplements. We also found that many comments did not include any mention of lifestyle modifications or patient comorbidities. Emerging research linking ED to systemic conditions, such as cardiovascular disease and diabetes, underscores the importance of a thorough evaluation before using any supplement or drug.44,45

After filtration by ReviewMeta.com, the online comments were more tempered in their praise for these supplements. Although there were significant reductions in touted benefits across all IIEF categories, 8.3% of product reviews continued to report improvements in erection strength after filtering. This finding can help healthcare practitioners understand patients’ subjective assessment of product efficacy and tailor future counseling efforts about the true impact of these ingredients on erection strength. Previous ED studies have also demonstrated that treatment of ED with oral placebos can elicit clinical effects and improve ED symptoms.46 The positive placebo effect on erectile function and quality of erection should also be considered when interpreting purported benefits of ED-S products.

This study has several limitations that should be acknowledged. First, this analysis was limited to 6 ED-S products hosted on Amazon.com that were yielded the top results from a single search query on a single day. Although many other ED-Ss are available in the Amazon marketplace, these products did not rank sufficiently high to be considered for the present study. Even though these other ED-Ss are likely relevant in a discussion of ED-Ss on Amazon, we believe that our supplement selection is warranted, given that customers often consider only the most highly rated products. In addition, we believe that the findings gleaned from the products included in this study are applicable for other, less highly ranked products as well. Second, we acknowledge that the proprietary nature of the ReviewMeta algorithm limits the transparency with which we and others can report results. Although validation of the review results obtained via the ReviewMeta tool is desirable, no other comparable services are available. Third, we recognize that we have limited information about whether users submitting reviews on these product pages have ED or are using ED-S for self enhancement. Furthermore, we acknowledge that for users who actually did have ED, we had no information on the severity of their ED symptoms to evaluate whether ED-S use resulted in clinically significant improvements.

CONCLUSION

ED-S products are readily accessible from online marketplaces such as Amazon.com and are used by numerous patients with ED. Physicians must be aware of the ingredients in these supplements to better counsel patients about the efficacy of these supplements. Although consumer reviews hosted on ED-S online product pages prominently tout product efficacy, primary evidence supporting positive effects of these products on ED symptoms is lacking. The present study revealed a lack of consistency in supplementation protocols used in human studies evaluating the efficacy of ED-S ingredients. In the absence of more definitive human data, patients should be cautioned before considering ED-S use, particularly in light of the availability of highly effective FDA-approved drugs and increasingly affordable therapeutic options.

Supplementary Material

S-F1
S- T1
S-F2

Funding:

A.W.P. is a National Institutes of Health K08 Scholar supported by a Mentored Career Development Award (K08DK115835-01) from the National Institute of Diabetes and Digestive and Kidney Diseases. This work is supported in part through a Urology Care Foundation Rising Stars in Urology Award (to A.W.P.) and National Institutes of Health Grant K12 DK0083014 and a Multidisciplinary K12 Urologic Research Career Development Program from the National Institute of Kidney and Digestive Diseases (to D.J.L.). N.T. is a K12 Scholar. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Conflicts of interest:

Dr. Pastuszak: Endo Pharmaceuticals: advisor, consultant, speaker; Boston Scientific, advisor; Antares Pharamaceuticals, advisor. Dr. Lipshultz: American Medical Systems, speaker; AbbVie, consultant; Lipocine, consultant; Aytu Bioscience, consultant; Endo Pharmaceuticals, speaker/consultant. Dr. Hotaling: Endo pharmaceuticals, educational/research grant; Boston Scientific, educational/research grant; Streamdx; Andro360; Nanonc-Equity.

Footnotes

STATEMENT OF AUTHORSHIP

Category 1

(a) Conception and Design

Adithya Balasubramanian; Nannan Thirumavalavan; Alexander W. Pastuszak

(b) Acquisition of Data

Adithya Balasubramanian; Nannan Thirumavalavan; Ashwin Srivatsav; Justin Yu; Alexander W. Pastuszak

(c) Analysis and Interpretation of Data

Adithya Balasubramanian; Nannan Thirumavalavan; Ashwin Srivatsav; Justin Yu; James M. Hotaling; Larry I. Lipshultz

Category 2

(a) Drafting the Article

Adithya Balasubramanian; Nannan Thirumavalavan; Ashwin Srivatsav; Justin Yu; James M. Hotaling; Larry I. Lipshultz; Alexander W. Pastuszak

(b) Revising It for Intellectual Content

Adithya Balasubramanian; Nannan Thirumavalavan; Ashwin Srivatsav; Justin Yu; James M. Hotaling; Larry I. Lipshultz

Category 3

(a) Final Approval of the Completed Article

Adithya Balasubramanian; Nannan Thirumavalavan; Ashwin Srivatsav; Justin Yu; James M. Hotaling; Larry I. Lipshultz; Alexander W. Pastuszak

SUPPLEMENTARY DATA

Supplementary data related to this article can be found at https://doi.org/10.1016/j.jsxm.2018.10.004.

REFERENCES

  • 1.Lizza EF, Rosen RC. Definition and classification of erectile dysfunction: Report of the Nomenclature Committee of the International Society of Impotence Research. Int J Impot Res 1999;11:141–143. [DOI] [PubMed] [Google Scholar]
  • 2.Yafi FA, Jenkins L, Albersen M, et al. Erectile dysfunction. Nat Rev Dis Prim 2016;2:16003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Mobley DF, Khera M, Baum N. Recent advances in the treatment of erectile dysfunction. Postgrad Med J 2017;93:679–685. [DOI] [PubMed] [Google Scholar]
  • 4.Frederick LR, Cakir OO, Arora H, et al. Undertreatment oferectile dysfunction: Claims analysis of 6.2 million patients. J Sex Med 2014;11:2546–2553. [DOI] [PubMed] [Google Scholar]
  • 5.Le B, McAchran S, Paolone D, et al. Assessing the variability in insurance coverage transparency for male sexual health conditions in the United States. Urology 2017;102:126–129. [DOI] [PubMed] [Google Scholar]
  • 6.Cui T, Kovell RC, Brooks DC, et al. A urologist’s guide to ingredients found in top-selling nutraceuticals for men’s sexual health. J Sex Med 2015;12:2105–2117. [DOI] [PubMed] [Google Scholar]
  • 7.ElAmrawy F, ElAgouri G, Elnoweam O, et al. Adulterated and counterfeit male enhancement nutraceuticals and dietary supplements pose a real threat to the management of erectile dysfunction: A global perspective. J Diet Suppl 2016;13:660–693. [DOI] [PubMed] [Google Scholar]
  • 8.Bailey RL. Current regulatory guidelines and resources to support research of dietary supplements in the United States. Crit Rev Food Sci Nutr 2018;1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Venhuis BJ, de Kaste D. Towards a decade of detecting new analogues of sildenafil, tadalafil and vardenafil in food supplements: A history, analytical aspects and health risks. J Pharm Biomed Anal 2012;69:196–208. [DOI] [PubMed] [Google Scholar]
  • 10.Chiang J, Yafi FA, Dorsey PJ, et al. The dangers of sexual enhancement supplements and counterfeit drugs to “treat” erectile dysfunction. Transl Androl Urol 2017;6:12–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Campbell N, Clark JP, Stecher VJ, et al. Adulteration of purported herbal and natural sexual performance enhancement dietary supplements with synthetic phosphodiesterase type 5 inhibitors. J Sex Med 2013;10:1842–1849. [DOI] [PubMed] [Google Scholar]
  • 12.Nissan R, Poperno A, Stein GY, et al. A case of hepatotoxicity induced by adulterated “Tiger King,” a Chinese herbal medicine containing sildenafil. Curr Drug Saf 2016;11:184–188. [DOI] [PubMed] [Google Scholar]
  • 13.Mandava S, Ganganna B, Hwang J, et al. Synthesis andstructure revision of dimeric tadalafil analogue adulterants in dietary supplements. Chem Pharm Bull (Tokyo) 2017; 65:498–503. [DOI] [PubMed] [Google Scholar]
  • 14.Meeker M.Internet trends 2018. Kleiner Perkins, May 30,2018. Available at: https://www.kleinerperkins.com/perspectives/internet-trends-report-2018/. Accessed April 4, 2019.
  • 15.Lunden I.Amazon ‘ hare of the U e- commerce market i now 49 %, or 5 % of all retail pend 2018:1e10. Available at: https://techcrunch.com/2018/07/13/amazons-share-of-the-us-e-commerce-market-is-now-49-or-5-of-all-retail-spend/. Accessed April 18, 2018.
  • 16.Smith A, Anderson M. Online shopping and E-commerce. Pew Research Center December 19, 2016. Available at: https://www.pewinternet.org/2016/12/19/online-shopping-and-e-commerce/. Accessed April 4, 2019.
  • 17.Lau RYK, Liao SS, Kwok RCW, et al. Text mining and probabilistic language modeling for online review spam detection. ACM Trans Manag Inf Syst 2011;2:1–30. [Google Scholar]
  • 18.Feng S, Xing LF, Gogar A, et al. Distributional footprints of deceptive product reviews. Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media; June 4–7, 2012; Dublin, Ireland. [Google Scholar]
  • 19.Mukherjee A, Liu B, Wang J, et al. Detecting group review spam. Proceedings of the 20th International Conference Companion on World Wide Web; March 28-April 1, 2011; Hyderabad, India. [Google Scholar]
  • 20.Ott M, Choi Y, Cardie C, et al. Finding deceptive opinion spamby any stretch of the imagination. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics; June 19e24, 2011; Portland, OR. [Google Scholar]
  • 21.Lappas T.Fake reviews: The malicious perspective. In: Bouma G, Métais Ittoo A, et al. , eds. Natural language processing and information systems. NLDB 2012. Lecture Notes in Computer Science, Vol 7337. Berlin: Springer; 2012. [Google Scholar]
  • 22.Malbon J.Taking fake online consumer reviews seriously. J Consumer Policy 2013;36:139–157. [Google Scholar]
  • 23.Salehan M, Kim DJ. Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decis Support Syst 2016;81:30–40. [Google Scholar]
  • 24.Xu C, Zhang J, Chang K, et al. Uncovering collusive spammers in Chinese review websites. In: He Q, Iyengar A, Nejdl W, eds. Proceedings of the 22nd ACM Conference on Information and Knowledge Management (CIKM 2013). New York: ACM Press; 2013. p. 979–988. [Google Scholar]
  • 25.de Freitas J, Falls BA, Haque OS, et al. Vulnerabilities to misinformation in online pharmaceutical marketing. J R Soc Med 2013;106:184–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Bender JL, Cyr AB, Arbuckle L, et al. Ethics and privacy implications of using the internet and social media to recruit participants for health research: A privacy-by-design framework for online recruitment. J Med Internet Res 2017; 19:e104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Vasconcelos T, Marques S, Sarmento B. The biopharmaceutical classification system of excipients. Ther Deliv 2017;8: 65–78. [DOI] [PubMed] [Google Scholar]
  • 28.Rosen RC, Cappelleri JC, Smith MD, et al. Development and evaluation of an abridged, 5-item version of the International Index of Erectile Function (IIEF-5) as a diagnostic tool for erectile dysfunction. Int J Impot Res 1999;11:319–326. [DOI] [PubMed] [Google Scholar]
  • 29.Sim J, Wright CC. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther 2005;85:257–268. [PubMed] [Google Scholar]
  • 30.Viera AJ, Garrett JM. Understanding interobserver agreement: The kappa statistic. Fam Med 2005;37:360–363. [PubMed] [Google Scholar]
  • 31.Wincze JP. Psychosocial aspects of ejaculatory dysfunction and male reproduction. Fertil Steril 2015;104:1089–1094. [DOI] [PubMed] [Google Scholar]
  • 32.Peate I.Breaking the silence: Helping men with erectile dysfunction. Br J Community Nurs 2012;17; 310, 312, 314–310, 312, 317. [DOI] [PubMed] [Google Scholar]
  • 33.Baldwin K, Ginsberg P, Harkaway RC. Under-reporting oferectile dysfunction among men with unrelated urologic conditions. Int J Impot Res 2003;15:87–89. [DOI] [PubMed] [Google Scholar]
  • 34.Mulhall JP, Giraldi A, Hackett G, et al. The 2018 revision to the process of care model for management of erectile dysfunction. J Sex Med 2018;15:1434–1445. [DOI] [PubMed] [Google Scholar]
  • 35.LaValley SA, Kiviniemi MT, Gage-Bouchard EA. Where people look for online health information. Health Info Libr J 2017; 34:146–155. [DOI] [PubMed] [Google Scholar]
  • 36.Zhang K, Yu W, He ZJ, et al. Help-seeking behavior for erectile dysfunction: A clinic-based survey in China. Asian J Androl 2014;16:131–135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Read J, Mati E. Erectile dysfunction and the internet: Drug company manipulation of public and professional opinion. J Sex Marital Ther 2013;39:541–559. [DOI] [PubMed] [Google Scholar]
  • 38.Amber KT, Dhiman G, Goodman KW. Conflict of interest in online point-of-care clinical support websites. J Med Ethics 2014;40:578–580. [DOI] [PubMed] [Google Scholar]
  • 39.Brown AC. An overview of herb and dietary supplement efficacy, safety and government regulations in the United States with suggested improvements. Part 1 of 5 series. Food Chem Toxicol 2017;107(Pt A):449–471. [DOI] [PubMed] [Google Scholar]
  • 40.Lee JH, Mandava S, Baek SY, et al. Identification and structural elucidation of three new tadalafil analogues found in a dietary supplement. J Pharm Biomed Anal 2016;123:1–9. [DOI] [PubMed] [Google Scholar]
  • 41.Xu Y, Kee CL, Ge X, et al. Isolation and characterization of atadalafil analogue, N-cyclopentyl nortadalafil in health supplement. J Pharm Biomed Anal 2016;118:235–241. [DOI] [PubMed] [Google Scholar]
  • 42.Huang YC, Lee HC, Lin YL, et al. Separation and identification of a novel tadalafil analogue adulterant in a dietary supplement. Food Addit Contam Part A Chem Anal Control Expo Risk Assess 2016;33:179–185. [DOI] [PubMed] [Google Scholar]
  • 43.Kern SE, Nickum EA, Flurer RA, et al. Isolation and structural characterization of a new tadalafil analog (2-hydroxyethylnortadalafil) found in a dietary supplement. J Pharm Biomed Anal 2015;103:99–103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Gandaglia G, Briganti A, Jackson G, et al. A systematic review of the association between erectile dysfunction and cardiovascular disease. Eur Urol 2014;65:968–978. [DOI] [PubMed] [Google Scholar]
  • 45.Kouidrat Y, Pizzol D, Cosco T, et al. High prevalence of erectile dysfunction in diabetes: A systematic review and meta-analysis of 145 studies. Diabet Med 2017;34:1185–1192. [DOI] [PubMed] [Google Scholar]
  • 46.de Araujo AC, da Silva FG, Salvi F, et al. The management of erectile dysfunction with placebo only: Does it work? J Sex Med 2009;6:3440–3448. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S-F1
NIHMS1526021-supplement-S-F1.jpg (689.6KB, jpg)
S- T1
NIHMS1526021-supplement-S-_T1.docx (13.5KB, docx)
S-F2
NIHMS1526021-supplement-S-F2.jpg (256.7KB, jpg)

RESOURCES