Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2018 Jul 18;13(7):e0199558. doi: 10.1371/journal.pone.0199558

Hypotheses and evidence related to intense sweeteners and effects on appetite and body weight changes: A scoping review of reviews

Annhild Mosdøl 1,*,#, Gunn Elisabeth Vist 1,, Camilla Svendsen 2,, Hubert Dirven 2,, Inger Therese Laugsand Lillegaard 3,, Gro Haarklou Mathisen 3,, Trine Husøy 2,#
Editor: C Mary Schooling4
PMCID: PMC6051566  PMID: 30020966

Abstract

Observed associations between consumption of diet foods and obesity have sparked controversy over whether intense sweeteners may promote weight gain, despite their negligible energy contribution. We conducted a scoping review of reviews, to obtain an overview of hypotheses, research approaches and features of the evidence on intense sweeteners’ potential relationships to appetite and weight changes. We searched for reviews of the scientific literature published from 2006 to May 2017. Two reviewers independently assessed title and abstracts, and full text publications. Arksey and O’Malley’s framework for scoping reviews guided the process. We extracted and charted data on characteristics of the reviews and the evidence presented. The 40 included reviews present hypotheses both on how intense sweeteners can reduce or maintain body weight and on how these can promote weight gain. We classified only five publications as systematic reviews; another nine presented some systematic approaches, while 26 reviews did not describe criteria for selecting or assessing the primary studies. Evidence was often presented for intense sweeteners as a group or unspecified, and against several comparators (e.g. sugar, water, placebo, intake levels) with limited discussion on the interpretation of different combinations. Apart from the observational studies, the presented primary evidence in humans is dominated by small studies with short follow-up—considered insufficient to assess weight change. Systematic reviews of animal studies are lacking in this topic area. The systematic evidence only partly explore forwarded hypotheses found in the literature. Primary studies in humans seem to be available for systematic exploration of some hypotheses, but long-term experimental studies in humans appear sparse. With few exceptions, the reviews on intense sweeteners and weight change underuse systematic methodology, and thus, the available evidence. Further studies and systematic reviews should be explicit about the hypothesis explored and elucidate possible underlying mechanisms.

Introduction

Associations seen in observational studies between the consumption of diet products, predominantly diet soda, and prevalence of obesity has fuelled a debate over whether intense sweeteners can promote weight gain [1, 2]. A group of non-calorie sweeteners, the intense sweeteners, are up to 700 times sweeter than sucrose and can provide sweet taste while contributing negligible to people’s energy intake. Examples of the intense sweeteners are saccharin, aspartame, sucralose, cyclamate and steviol glycoside. Explanations for this contradiction have been suggested, but it remains unclear if the observed associations are causal relationships. Some intense sweeteners have been used extensively for decades, particularly in beverages. Increasingly, intense sweetener consumption comes through foods [3]. The new World Health Organization (WHO) guideline recommend added sugars to be below 10%, and preferably 5%, of the total energy intake [4]. Due to this recommendation and planned taxes on sugared drinks in many countries, sugar is likely to be replaced with intense sweeteners in more foods and beverages ahead [5].

All permitted food additives are considered safe for human health within specified intake levels (acceptable daily intake; ADI) as evaluated by National or International food safety regulatory agencies. Standard safety assessments determine an ADI based on comprehensive toxicological tests considering numerous endpoints, largely based on animal studies [6]. These include reports on weight changes in animals, but such controlled feeding studies may not be relevant to normal consumption patterns in humans. Weight losses in study animals, for instance, may be due to reduced palatability of feed when testing very high doses of intense sweeteners. In a recent re-evaluation of aspartame, a panel from European Food Safety Authority (EFSA) also considered that potential effects of intense sweeteners on eating behaviour was outside the remit of risk assessment [7].

Given the current obesity epidemic and serious associated health risks [8], it is timely to clarify if intense sweeteners are effective tools to lower sugar consumption and maintain a healthy weight or, on the contrary, if these compounds promote weight gain. Published reviews can provide an overview of topics, hypotheses and research approaches in the research field. The aim of this scoping review is to determine the extent and type of summarized evidence published the last 10 years regarding the potential effects of intense sweeteners on appetite and weight change. We particularly intended to identify gaps where new systematic reviews or primary research are needed, including which hypotheses, types of intense sweeteners and outcomes that need further assessment.

Materials and methods

We performed a scoping review using a pre-defined protocol, guided by the framework for scoping reviews proposed by Arksey and O’Malley [9] with suggested enhancements [1012] and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) [13]. In scoping reviews, the researchers systematically explore the literature on a specified topic, for instance to examine the extent and nature of available research, the value of undertaking a systematic review, or identify research gaps [9].

Systematic search and inclusion criteria

An experienced research librarian conducted the literature search and the search strategy was peer reviewed. We searched without language restrictions for all types of reviews published from 2006 to May 18th 2017 in the following databases: MEDLINE (Ovid) and PubMed [sb], Embase (Ovid), Cochrane Library (CDSR, DARE, HTA), Epistemonikos and Web of Science. The literature search included MESH term Sweetening agents, free text words describing the group (i.e. intense sweeten*, artificial sweeten* etc.) and names of specific intense sweeteners, combined with MESH terms and free text words related to overweight/obesity, weight change and appetite. A filter for reviews limited the search and adjustments made to search terms and structure to suit different databases (see S1 Appendix for search strategy). In addition, we searched for relevant reports published by EFSA, Joint FAO/WHO Expert Committee on Food Additives (JECFA), U.S. Food and Drug Administration (FDA), National Institutes of Health (NIH, the U.S.), European Medicines Agency (EMA), The Federal Institute for Risk Assessment (BfR, Germany), Institut National de l’Environnement Industriel et des Risques (INERIS, France), Rijksinstituut voor Volksgezondheid en Milieu (RIVM, the Netherlands) and The EU Joint Research Centre (Italy).

Two reviewers independently screened title and abstracts, and considered potentially relevant references in full text. We distributed the work among pairs of authors, and resolved discrepancies with a third author. Inclusion criteria were any reviews of the literature whose primary focus was the effect of consuming intense sweeteners (either as a group of sweeteners or for specific sweeteners) on weight change, appetite or related outcomes. To focus the evidence on intense sweeteners, appetite and weight specifically, non-systematic reviews that discussed several related topics (e.g. exposure to other dietary components or outcomes such as diabetes, cardiovascular diseases or cancer) were excluded. Systematic reviews of several exposures were included if the publication presented separate, systematic analyses for intense sweeteners and our defined outcomes. Eligible reviews presented results from empirical studies of any quantitative study design in humans and animal models on intense sweeteners or specified for Acesulfam K, E 950, Aspartame, E 951, Cyclamat, E 952, Saccharin, E 954, Sucralose, E 955, Neohesperidin DC, E 959, Steviol glycoside, E 960, Neotame, E 96. We included reviews discussing any comparator with outcome relevant to appetite and body weight change, including effects through gut microbiota.

Data extraction and analyses

All authors contributed to data extraction using an extraction form and another author validated the extraction. Data included characteristics of the review, its objectives, components and analysis, types of included study designs/experimental models, study populations, intense sweeteners examined (including dose and form of intake), comparisons and outcomes with follow-up time, and hypotheses. We characterized publications as systematic reviews based our simplified version of the definition used by the Cochrane collaboration [14]. Publications were systematic reviews if they had described or presented 1) a systematic literature search, 2) clear criteria for relevant studies to include, and 3) quality assessment of the included studies (for instance Cochrane Risk of Bias assessment for RCTs or other study design specific quality assessment tools). We noted if reviews had data synthesis as meta-analyses.

For systematic reviews and reviews with some description of systematic approach to study selection (related to criteria 1 and 2 above), we presented characteristics of the review. For other reviews, we charted such information in a simplified format. We searched all reviews for explicit or implicit references to hypotheses on how intense sweeteners influence appetite and body weight change. Formulated hypotheses, suggested mechanisms, explanations for an effect or similar statements were noted for each review. Based on our interpretation of their content, we sorted topics across publications. The authors discussed the propositions found and combined these into main hypotheses with similar content. The experimental evidence from systematic reviews/reviews with some systematic approaches to study selection were mapped according to which main hypothesis they provided insight into. This categorization was based on our judgement of the aim, research question or hypotheses presented for each review, study inclusion criteria and analyses presented. Observational studies were excluded from this mapping, as we consider these inadequate to elucidate the hypotheses further. Based on the reviews that provided characteristics on all included primary studies, we extracted and charted the number of participants and longest follow-up time in categories defined by intense sweeteners versus comparator studied, for observational and experimental studies respectively.

Results

The search returned 991 references that we processed as shown in Fig 1. In addition, we considered 11 references from relevant agencies. Of 102 provisionally eligible references, we excluded 62 based on full-text screening (S1 Table). Two of these were protocols for planned Cochrane systematic reviews that will address the scope of our review [15, 16]. We included 40 reviews. The number of reviews published on this topic increased over time, with 31 of the reviews published from 2012 onwards, and nine in 2016 only.

Fig 1. Literature search and selection process.

Fig 1

Systematic reviews and reviews with a systematic approach

Among the included publications, we categorized five [1721] as systematic reviews. Another nine publications [2230] had some description of a literature search strategy for studies, although with varying level of detail on criteria for eligible study designs, exposures, comparisons and outcomes. However, these nine did not describe any quality assessment of the included studies—criteria No 3 for being a systematic review. Characteristics of the fourteen reviews with a systematic approach to study selection are presented in Table 1.

Table 1. Characteristics of included systematic reviews and reviews with some description of systematic approach to study selection.

First author, published Literature search (end search) Inclusion criteria (if not pre-specified, description of what was included. Authors’ terms presented) Quality assessment of studies Data synthesis Included studies
Types of studies and populations Intense sweeteners considered Comparison Outcomes and follow-up period
Wiebe, 2011 [17] Medline, Embase, CENTRAL, CAB Global (to January 2011) RCTs. Humans ≥ 16 years. Healthy, overweight or diabetic. Any non-caloric sweetener. Any study comparing against another sweetener, caloric or non-caloric. Weight change, energy intake, clinical outcomes Follow-up: > 1 week 4 RCT-specific domains for risk of bias Narrative, network meta-analysis 6a
Reid, 2016 [18] Medline, Embase, Cochrane Library, reference list, grey literature (to July 2015) Prospective cohorts and RCTs. Humans, pregnant women, infants and children <12 years only. All non-nutritive sweeteners. Nutritive sweeteners, placebo; regular diet Primary outcomes: BMI/BMI Z-score. Secondary outcomes: Birth weight, growth velocity, adiposity, clinical outcomes. Follow-up: > 6 months Cochrane Risk of Bias tool (RCTs), Newcastle-Ottawa Scale (Cohorts). Narrative, meta-analysis 8
Rogers, 2016 [19] Medline, Embase, Web of Science (to February 2015) Several study designs. Animal, human observational, short and sustained intervention studies analyzed separately. Low-energy sweeteners (intense sweeteners and sugar alcohols). Not pre-specified. Included sugar, water, unsweetened, nothing, placebo Energy intake, body weight, body mass index. Follow-up: open Cochrane Risk of Bias tool used for sustained human intervention studies Narrative, meta-analysis 147
Ruanpeng, 2017 [20] Medline, Embase, Cochrane databases (to May 2015) RCTs, observational studies (cohort, case-control, cross-sectional). Humans, adults. Sugar-sweetened or artificially sweetened sodas No soda consumption Obesity, weight gain Follow-up: Not pre-specified. Included only cross-sectional.* Newcastle-Ottawa quality scale. Meta-analysis 3 a
Santos, 2017 [21] Cochrane, LILACS, PubMed, Scopus, Web of Science, grey literature (to April 2016) Controlled trials. Humans, adults. Aspartame. Control (water, placebo, nothing) or sucrose. Primary outcomes: blood glucose, obesity. Secondary outcomes: glycemic control, overweight, energy intake, clinical measures. Follow-up: Not pre-specified and unclear. Single dose to two years included in same analyses. Cochrane Risk of Bias tool Narrative, meta-analysis 29
de la Hunty, 2006 [22] No search strategy. “All studies […] were identified”. RCTs. Human, adults only. Aspartame, alone or in combination with other intense sweeteners. Primary comparison sucrose, secondary water and “other”. Energy intake, Body weight Follow-up: > 24 hours No. Meta-analysis 16
Brown, 2010 [23] PubMed, Web of Science, EMBASE (end search unclear) Not pre-specified: Mix of study designs. Humans, children 0–18 years. Artificial sweeteners Not pre-specified. Included sugar, water, placebo, different intake levels. Food intake, weight change, clinical outcomes Follow-up: Not pre-specified. Included 20 min to 10 years. No. Narrative 18
Gardner 2012 [24] Pubmed, Evidence Analysis Library of the American Dietetic Association, hand search (from 2000, unclear end date) Prospective cohorts and controlled trials. Systematic reviews. Humans. Animal studies presented. FDA approved non-nutritive sweeteners (aspartame, acesulfame-K, neotame, saccharin sucralose, stevia) Caloric sweeteners. Sucrose. Partly pre-specified. Included energy intake compensation, appetite, hunger, body weight. Follow-up: Unclear. Included short up to 5 years. No. Narrative 32
Pereira, 2013a [25] Medline (to September 2011), reference lists Unclear study design criteria. Humans. Unclear: Artificially sweetened beverages. Not pre-specified and unclear. Included multiple comparisons. Body weight, obesity risk, clinical outcomes. Follow-up: Not pre-specified. Included cross-sectional up to six years. No. Narrative 14
Pereira, 2013b [26] Medline (from 1990 to May 2013) Prospective cohorts and RCTs. Humans. Unclear: Artificially sweetened beverages. Not pre-specified and unclear. Included multiple comparisons. Body weight, body fat, clinical outcomes. Follow-up: RCTs not restricted, ≥ 6 months for cohorts No. Narrative 17
Shankar 2013 [27] Medline, PubMed, websites (from 1987 to 2012) Not pre-specified. Included human and animal studies, various designs. Not pre-specified. Included non-nutritive sweeteners combined, saccharin, aspartame, acesulfame-K, tagalose, sucralose, stevia. Not pre-specified and unclear. Not pre-specified and unclear. Related to weight, obesity and energy compensation. No. Narrative 15
Miller, 2014 [28] Medline, reference lists (to September 2013) Prospective cohorts and RCTs. Healthy populations. Humans. Any low-calorie sweetener (nonnutritive sweetener or polyol). Not pre-specified. Included sugar, lactose capsules, usual diet, different intake levels. Any measure of body weight or composition. Follow-up: ≥ 3 weeks for RCTs, ≥ 6 months for cohorts No. Meta-analysis 23
Pereira, 2014 [29] Medline, reference lists (from 1946 to March 2012) Prospective cohorts and RCTs. Humans. Unclear: Sugar-Sweetened and artificially sweetened beverages, presented separately. Not pre-specified and unclear. Included intake levels, sugar. Body weight, body fat. Follow-up: RCTs not restricted, ≥ 6 months for cohorts No Narrative 18
Romo-Romo, 2016 [30] PubMed, The Cochrane library, Trip Database, hand search (to April 2015) Observational studies. Clinical trials. Humans. Non-nutritive sweeteners. Six specific intense sweeteners presented in trials. Saccharin, aspartame, acesulfame-K Not pre-specified. Included water, simple saccharides, placebo, corn flour, milk. Outcomes related to glucose metabolism, appetite regulating hormones and hunger. Follow-up: Not pre-specified. Included 7–8 years for cohorts, single dose to 18 weeks in trials* No. Narrative 28a

RCTs: Randomized controlled trials,

aRelevant to the aim of this scoping review (The publication also presents separate analyses of other comparisons or outcomes)

The evidence in these reviews was predominantly from human studies. The review by Rogers et al. [19] had a systematic search and separate analyses of animal studies, while two publications presents some evidence from animal studies [24, 27]. Four reviews [17, 21, 22, 30] restricted their analyses to controlled trials in humans. The remaining ten reviews presented both observational and experimental evidence. Two reviews had a specific focus on aspartame [21, 22]. Otherwise, most of these reviews included several intense sweeteners, usually examining the effect of intense sweeteners as one specified or unspecified group. As a whole, these reviews examined the effect of intense sweetener consumption against multiple comparators: no intake, water, sucrose, other caloric sweetener, unsweetened drinks or placebo, or by comparing different intake levels of intense sweeteners in observational studies. All fourteen reviews considered multiple outcomes, primarily related to weight change or body composition. The reviews also varied greatly in how they handled length of follow-up, both in the study selection process and in the analyses. Seven [20, 21, 23, 24, 26, 27, 30] had no mention of follow-up time in their inclusion criteria. When defined, eligibly regarding follow-up varied from being open [19] to including studies with minimum six months follow-up [18, 25, 28, 29]. Seven of these reviews presented results as meta-analyses based on a judgement if the studies were similar enough to combine [1722, 28], sometimes combined with narrative synthesis of the evidence.

We did not extract information on doses or exposure of intense sweeteners, the baseline energy intake or differences in energy intake of study arms as intended. Few review papers reported information on the dose or exposure to intense sweeteners in a systematic way. Unless change in energy intake was an outcome, information on energy intake in study arms was generally lacking in the reviews.

Narrative reviews without description of study selection

We identified another 26 reviews with primary focus on intense sweeteners’ effect on appetite or weight change [2, 3, 3154] (Fig 2), but these had no description of how the evidence had been selected. All had narrative syntheses of the findings. Compared to the reviews in Table 1, these reviews referred to a wider range of evidence: Observational human studies (nearly all reviews), experimental studies, both human and animal (19 of 26 reviews) and findings from experiments in cell cultures (three reviews). Often, the reviews presented evidence from only one or a few studies within each category. In addition to lack of clarity about how the studies had been selected, the reviews provided limited or no information concerning the characteristics of the studies, study designs, number of participants/experimental animals, comparisons made, effect sizes and quality of the primary studies. Ten reviews discussed findings from other reviews as part of their evidence synthesis.

Fig 2. Features of included reviews without description of study selection, all narrative analyses.

Fig 2

Similar to the reviews in Table 1, all publications in Fig 2 referred to effects of unspecified intense sweeteners or these as a group. Six of the 26 reviews discussed findings without specifying any intense sweeteners, but most reviews also provided evidence on one or several named compounds. Another characteristic of these reviews is that the effects were examined against several different comparators (e.g. compared to intake of sugar, water, different intake levels, placebo), often in the same line of reasoning. Sometimes the comparator for a stated effect was unclear from the text [33, 35, 36, 39, 42, 44, 48, 53]. The only exception was the scientific statement paper from EFSA 2011, which only discussed intense sweeteners in comparison to intake of glucose [32]. The most common outcome discussed in these reviews was measures of body weight, but the reviews in Fig 2 referred to a greater range of outcome measures, such as appetite and hunger scores, hormone secretion, intestinal glucose absorption and intestinal microbiome.

Main hypothesis mapped against available systematic evidence

We searched through all 40 reviews included in Table 1 and Fig 2 for descriptions of hypotheses on how intense sweeteners influence appetite and body weight changes. Explicit or implicit hypotheses were identified in the papers’ introduction, as part of the aims, in narrative evidence syntheses, and particularly in the discussion sections as part of suggested mechanisms or interpretation of findings. Although these hypotheses were formulated in different ways, we considered that all could be summarized in five main hypotheses presented in Fig 3. The first two hypothesis assume that use of intense sweeteners may lower sugar consumption and subsequently total energy intake to reduce or maintain body weight. The other three hypotheses consider how intense sweeteners may promote weight gain.

Fig 3. Main hypotheses identified from all included reviews mapped against the evidence found in systematic reviews and reviews with systematic approach to study selection.

Fig 3

a Data extracted from systematic reviews and reviews with description of systematic approach (from Table 1). Categorisation is based on our judgement of the aim, research question or hypotheses presented, study inclusion criteria and analyses presented in each review against the hypothesis. Evidence summaries of observational studies were not considered for this presentation. A: Animal studies.

We mapped experimental evidence from the systematic reviews and reviews with a systematic approach (from Table 1) against whether they provided evidence to explore these five hypotheses. Eleven reviews [1719, 2125, 2729] had evidence of relevance to the main hypothesis No. 1; that substitution of sugar with intense sweeteners in beverages and foods will reduce the total energy intake, and subsequently affect body weight. Only two of these reviews [18] had separate analyses for studies with longer-term follow-up (> 6 months). Four reviews [21, 23, 24, 27] had pooled all follow-up periods together, from single dose exposure to longer-term follow-up, or had no description of the follow-up. We did not find any systematic evidence specific regarding hypothesis No.2: Beverages with intense sweeteners are as protective, or more protective, than water in prevention of obesity when compared to consumption of sugar-sweetened beverages. However, single studies addressing this hypothesis were embedded in analyses related to main hypothesis No. 1.

Seven reviews provided some systematic evidence to explore main hypothesis No.4: Intense sweeteners affect sweet taste receptors, hormonal signalling, reward systems and learned energy sensing perceptions in a manner that interferes with nutrient absorption, appetite control or other weight regulation mechanisms. However, this evidence was scattered and covering only some relevant outcomes. None of these reviews considered aspects of follow-up time specifically. We did not find any systematic evidence syntheses relating to main hypothesis No. 3 “Deliberate consumption of beverages and foods with intense sweeteners is associated with weight gain through mechanisms of giving psychological justification to overeat, being part of an unhealthy dietary pattern or dieting history”; nor hypothesis No. 5 “Intense sweeteners alter gut microbiota with possible secondary effects involving changes in glucose tolerance, appetite-regulating hormones or other factors that alter risk of obesity development”.

Characteristics of underlying primary studies

To map the human primary evidence used systematically, we extracted information from reviews that described core characteristics on all their included primary studies, including number of participants, follow-up time, exposure and comparator. Studies were sorted according to the combination of exposure (including intense sweeteners as a group) and comparator. Charts show the number of participants and follow-up time within categories defined by intense sweeteners versus comparator studied. Seven reviews [1820, 25, 2830] provided such characteristics on observational studies (presented in Fig 4) and nine reviews [1719, 2123, 2830] provided characteristics on human experimental studies (presented in Fig 5). These reviews presented findings from in total 24 observational studies, five analysed as cross-sectional studies and 19 as cohort studies. The number of participants in the observational studies varied greatly, from under 100 up to 53000 participants. Many had several years follow-up time, including nine studies with follow-up from 4 years up to 20 years (Fig 4). Among the 118 human experimental studies found in these reviews’ data synthesis (Fig 5), the most studied sweetener was aspartame with results reported from 96 studies, most often commonly compared against consumption of sucrose. A high number of primary studies were also reported results for sucralose (n = 30) and mixture of intense sweeteners (n = 31), while few experimental primary studies were reported for the intense sweeteners acesulfame K, cyclamate, saccharine and stevia. A notable finding seen in Fig 5 is that the majority of the studies had very short follow-up time; many measured the outcome after hours or days with exposure to intense sweeteners. Only twelve of the 118 experimental studies in the reviews’ reference lists of had a study duration of six months for longer.

Fig 4. Number of participants and longest follow-up timea in cited observational studies (n = 24).

Fig 4

Data from the reviews in Table 1 that presented characteristics on all included primary studies. All studies presented results on diet soda consumption with unspecified type of intense sweetener used. a For studies analysed in multiple primary publications, the longest follow-up is presented.

Fig 5. Number of participants and longest follow-up time in cited human experimental studies (n = 118) in categories defined by intense sweeteners versus comparator studied.

Fig 5

Data from the reviews in Table 1 that presented characteristics on all included primary studies. a One study with 641 participants and 78 weeks follow-up outside chart area. b One study with 414 participants and 43 weeks follow-up outside chart area.

Discussion

In this scoping review of reviews, we examined hypotheses, research approaches and features of the evidence regarding intense sweeteners effect on appetite and weight change. The rising number of reviews published over time shows an increased interest in these questions, but, based on our findings, it appears as if many health claims and discussions are based on limited systematic review of the evidence. Furthermore, the available systematic evidence only partly explore the width of forwarded hypotheses. Largely, intense sweeteners are treated as a group in these reviews, rather than specific compounds with different properties, with limited discussions on relevant comparators and control for total energy intake when assessing their effects. The animal studies in this topic area are underused for systematic examination. Few studies appears to be designed to explore longer-term weight change in humans.

A strength of this scoping review is the systematic approach including a systematic literature search in several databases, using predefined inclusion and exclusion criteria, and with screening by two people independently of each other. Providing an overview of the available summarized evidence is valuable to avoid duplication of efforts. This scoping review gives an overview of where new primary studies and/or systematic reviews appear to be needed. It is possible that some relevant reviews were missed, but it is more likely that the newest reviews, published just prior to or after our literature search, are lacking. We are aware of two forthcoming Cochrane systematic reviews that may provide findings for main hypotheses one and two [15, 16]. This scoping review is assumed up to date as of May 2017.

A limitation of our scoping review is that we describe primary studies based on the review authors’ accounts, rather than mapping all relevant primary studies available. The studies presented in Figs 4 and 5 are based on information from the 11 reviews [1723, 25, 2830] that provided study characteristics for all included studies. We considered that a full mapping of all referenced studies, by retrieving the missing information through full text publications, was beyond the scope of our study. Wang et al. [55] have created an evidence-map database with studies of low-calorie sweeteners effects on similar types of outcomes as in this scoping review, and found 225 potentially relevant primary studies (by June 2014). Although Figs 4 and 5 do not represent the entire pool of potentially relevant studies, they do represent the available evidence that has been summarised in a systematic manner. We also assume that these plots at least indicate possible research gaps, in particular the very limited number of experimental studies with of long-term follow up of participants.

We assessed that only five of 40 review papers were systematic reviews and another nine had systematic approaches. While non-systematic reviews are important contributions in scientific debate, their findings are limited for drawing conclusions on causal relationships. Risk of bias in the primary studies are largely unaccounted for in these reviews, which may lead that low quality and high quality studies are presented as being equally important. A systematic literature search to find all relevant studies is crucial to reduce the risk of bias in the evidence base. A methodological quality assessment of these reviews, for instance using the AMSTAR checklist [56, 57], would have involved judgement on comprehensiveness of the literature search. Methodological quality assessment of the included reviews was beyond our aim, but for some reviews, selection of evidence is apparent. Runanpeng et al. [20] included three cross-sectional studies in analyses of artificial sweetened soda and obesity, despite a clearly wider base of available studies (Fig 4). We found no systematic reviews of animal studies. Rogers et al. [19] searched thoroughly for animal studies and categorized the results in compulsory consumption, voluntary consumption and learning studies, but did not quality assess this set of studies. Systematic reviews of animal studies may meet particular methodological challenges [58], but are important to explore physiological mechanisms.

A notable finding is that effects of intense sweeteners are often presented grouped, rather than for individual compounds. With only a few exceptions, all the observational studies examined intake of beverages containing unspecified type and amount of intense sweeteners, mainly based on food frequency questionnaires (FFQ). In many cases, real-life exposure even from single foods or drinks too are mixtures of intense sweeteners and exact doses are rarely declared. Thus, we agree with others’ [29] claim that further evidence from observational studies, with their crude exposure assessment and methodological problems with reverse causality, residual confounding and lack of control with total energy intake, will not make significant insights into these research questions.

The experimental evidence is more specific on type and dose of intense sweeteners studied. Still, many of these reviews combine results for multiple compounds in the evidence syntheses, as well as omit considerations of doses used. Analyses of intense sweeteners as a group can be warranted if intense sweeteners affects appetite and weight control through mechanisms related to the sweet sensors in the mouth and the gut. Otherwise, compound specific analyses appear more appropriate, as based on current knowledge of different properties of each intense sweetener. For instance, aspartame is metabolized to the constituent amino acids prior to absorption, cyclamate can be metabolized by bacteria in the gastro intestinal tract and toxicity is related to its metabolites, while saccharin is excreted un-metabolized.

One concern identified in this scoping review, was that many of the reviews lacked consideration and justification of appropriate length of follow-up in relation to the hypothesis studied. For instance, Santos et al. [21] pooled all results on weight changes regardless if the participants were followed up only a few hours or up to two years. It is difficult to estimate an appropriate length of follow up, but body weight is an outcome that changes quite slowly over time. A recent systematic review on the effects of energy intake from fat on body weight in people not aiming to lose weight only included studies with minimum 6 months follow up [59]. Twelve of the human experimental studies presented in Fig 5 had 6 months of intervention or longer, but the majority of these studies had short follow-up time—often only hours or days. Short-term experimental studies may give insight into immediate metabolic changes and energy intake after exposure to intense sweeteners to understand immediate mechanisms related to activation of sweet taste receptors, hormonal signalling and reward systems. However, the relevance of such findings to long-term weight changes are limited. We therefore consider that most of these studies have too short follow-up to answer hypotheses related to weight change, including any possible effects through appetite and the microbiome. With the limited number of participants seen in many studies, we also expect several to have too low statistical power to answer the study objective.

Furthermore, this body of evidence contains little discussion on the relevant comparator in interpreting the effects of intense sweeteners and how this relates to overall changes in the total diet, particularly total energy intake. Substitution of high sucrose intake with intense sweeteners is essentially different from effects of intense sweeteners compared to water or nothing. De la Hunty et al. [22] comment that soft drinks were the most common vehicle of exposure in the studies, with intakes up to 2 L of sugar-sweetened soft drinks daily compared with aspartame-sweetened soft drinks—a 3.5 MJ/day difference in total energy intake. When energy intake is an outcome in several of these reviews [17, 19, 2124], the experiment take into account that humans can compensate an initial difference in energy intake over time. This ability to compensate for a difference in the total energy intake, partly or fully, may be dependent on size of the energy difference, mode of delivery (liquid, solids, and products), palatability or overall nutrient composition in the diet. For this reason, we did not treat hypothesis No.2 relative effect of intense sweeteners compared to water as substitutes for sugar) as subordinate No.1 (intense sweeteners comparted to sugar) as the energy difference between the compared treatments is dissimilar, thus possible relevant mechanisms.

A flaw in our thinking about intense sweeteners may also be the notion that these will substitute consumption of sugar. Gartner et al. [24] presents US dietary assessments data indicating that people consume products with intense sweeteners in addition to, rather instead of, high sugar products. In real life, people often choose low-calorie products for a reason, including a developing or existing weight problem and with differential success for weight control, i.e. the phenomenon relevant to reverse causation. Individual behavioural patterns related to dieting, indulging or restrained eating might be important to understand the associations seen between diet soda consumption and obesity seen in many observational studies (e.g. [18]).

Furthermore, the deliberate choice of low energy products may lead to a possible “licensing effect” (main hypothesis No.3), described as “employing justifications that allow violations of the goal they endorse” [60]. An example of the licensing effect occurs when people who have consumed a low-energy drink allow themselves to indulge in foods high in energy. Thus, the licensing effect can be described as a cognitive consideration and “reward to oneself”, and therefore different from a possible physiological sweet sensation rewarding effect as implicated in hypothesis No.4. A recent systematic review examined whether labelling of food, beverages, and tobacco products as “low”, “light”, “diet”, “reduced” or “zero” may lead to changes in consumption patterns and behaviours, but found only few studies exploring this topic [61]. A possible licensing effect of products with intense sweeteners may hypothetically be strong enough to counteract their use for weight control in individuals and populations. This also implies that systematic reviews should examine whether study participants were informed or naïve to the presence of intense sweeteners.

In conclusion, the included reviews present several hypotheses on how intense sweeteners may be associated with weight changes. However, hardly any of the identified publications specifies a hypothesis for their review and discusses whether the included studies are suitable to illuminate this. Apart from the observational studies, the evidence is dominated by small studies with short follow-up. Systematic reviews of animal studies are lacking in this area. We particularly see a need to disentangle the direct physiological effects of each intense sweetener from the possible behavioural aspects related to their everyday use in the population. All hypotheses presented in Fig 3 can be addressed in carefully designed trials and subsequently systematic reviews, but a more systematic approach is needed to explore specific hypotheses and elucidate possible mechanisms through appropriate study designs and follow-up.

Supporting information

S1 Appendix. Search strategy.

(PDF)

S1 Table. Excluded studies from full text.

(PDF)

S2 Table. PRISMA checklist.

(DOC)

Acknowledgments

All authors contributed to the conception, screening, data extraction and analyses. AM wrote the manuscript and all authors commented. No conflict of interests.

Data Availability

The data underlying this study are collected from published and publicly available reviews. The authors did not have special access privileges. Information related to the reviews is included in S1 Table. All other relevant data are within the paper and its Supporting Information files.

Funding Statement

The author(s) received no specific funding for this work.

References

  • 1.Ludwig DS. Artificially sweetened beverages: cause for concern. JAMA. 2009;302:2477–8. 10.1001/jama.2009.1822 [DOI] [PubMed] [Google Scholar]
  • 2.Yang Q. Gain weight by "going diet?" Artificial sweeteners and the neurobiology of sugar cravings: Neuroscience 2010. Yale J Biol Med. 2010;83:101–8. [PMC free article] [PubMed] [Google Scholar]
  • 3.Mattes RD, Popkin BM. Nonnutritive sweetener consumption in humans: effects on appetite and food intake and their putative mechanisms. Am J Clin Nutr. 2009;89:1–14. 10.3945/ajcn.2008.26792 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Guideline: Sugars Intake for Adults and Children. Geneva: World Health Organization; 2015. [PubMed] [Google Scholar]
  • 5.Briggs ADM, Mytton OT, Kehlbacher A, Tiffin R, Elhussein A, Rayner M, et al. Health impact assessment of the UK soft drinks industry levy: a comparative risk assessment modelling study. The Lancet Public Health. 2017;2:e15–e22. 10.1016/S2468-2667(16)30037-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.EFSA Panel on Food Additives and Nutrient Sources added to Food (ANS). Guidance for submission for food additive evaluations. EFSA Journal 2012;10:2760. [Google Scholar]
  • 7.EFSA ANS Panel (EFSA Panel on Food Additives and Nutrient Sources added to Food). Scientific Opinion on the re-evaluation of aspartame (E 951) as a food additive. EFSA Journal. 2013;11:3496. [Google Scholar]
  • 8.Obesity and overweight Fact sheet. Geneva: World Health Organization; 2016. [http://www.who.int/mediacentre/factsheets/fs311/en/. [Google Scholar]
  • 9.Arksey H, O’Malley L. Scoping studies: Towards a methodological framework. International Journal of Social Research Methodologies. 2005;8:19–32. [Google Scholar]
  • 10.Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69 10.1186/1748-5908-5-69 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.The Joanna Briggs Institute. Joanna Briggs Institute Reviewer’s Manual: 2015 edition/Supplement. Methodology for JBI Scoping Reviews. Adelaide: The Joanna Briggs Institute; 2015.
  • 12.Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. 2013;13:48 10.1186/1471-2288-13-48 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535 10.1136/bmj.b2535 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]: The Cochrane Collaboration; 2011. [Google Scholar]
  • 15.Burls A, Price AI, Cabello JB, Roberts NW. Drinking extra water or other non-caloric beverages for promoting weight loss or preventing weight gain. Cochrane Database of Systematic Reviews. 2016;2016 (5) (no pagination). [Google Scholar]
  • 16.Laviada-Molina H, Molina-Seguí F, Arjona-Villicana RD, Morales-Gual M, Cuello-García CA, Pérez-Gaxiola G. Non-nutritive sweeteners for the prevention or treatment of being overweight or obesity. Cochrane Database of Systematic Reviews [Internet]. 2016. [cited Oppdateringsøk; (8). [Google Scholar]
  • 17.Wiebe N, Padwal R, Field C, Marks S, Jacobs R, Tonelli M. A systematic review on the effect of sweeteners on glycemic response and clinically relevant outcomes. BMC Med. 2011;9:123 10.1186/1741-7015-9-123 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Reid AE, Chauhan BF, Rabbani R, Lys J, Copstein L, Mann A, et al. Early exposure to nonnutritive sweeteners and long-term metabolic health: A systematic review. Pediatrics. 2016;137 (3) (no pagination). [DOI] [PubMed] [Google Scholar]
  • 19.Rogers PJ, Hogenkamp PS, De Graaf C, Higgs S, Lluch A, Ness AR, et al. Does low-energy sweetener consumption affect energy intake and body weight? A systematic review, including meta-analyses, of the evidence from human and animal studies. Int J Obes. 2016;40:381–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Ruanpeng D, Thongprayoon C, Cheungpasitporn W, Sangtian J, Harindhanavudhi T. Sugar and artificially sweetened soda linked to obesity: A systematic review and meta-analysis. Endocr Rev. 2017:1–8. [DOI] [PubMed] [Google Scholar]
  • 21.Santos NC, Araujo LM, De Luca Canto G, Guerra EN, Coelho MS, Borin MF. Metabolic effects of aspartame in adulthood: a systematic review and meta-analysis of randomized clinical trials. Crit Rev Food Sci Nutr. 2017:0. [DOI] [PubMed] [Google Scholar]
  • 22.de la Hunty A, Gibson S, Ashwell M. A review of the effectiveness of aspartame in helping with weight control. Nutr Bull. 2006;31:115–28. [Google Scholar]
  • 23.Brown RJ, de Banate MA, Rother KI. Artificial sweeteners: a systematic review of metabolic effects in youth. International journal of pediatric obesity: IJPO: an official journal of the International Association for the Study of Obesity. 2010;5:305–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Gardner C, Wylie-Rosett J, Gidding SS, Steffen FLM, Johnson FRK, Reader D, et al. Nonnutritive sweeteners: Current use and health perspectives—A scientific statement from the American Heart Association and the American Diabetes Association. Diabetes Care. 2012;35:1798–808. 10.2337/dc12-9002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Pereira MA. Diet beverages and the risk of obesity, diabetes, and cardiovascular disease: a review of the evidence. Nutr Rev. 2013;71:433–40. 10.1111/nure.12038 [DOI] [PubMed] [Google Scholar]
  • 26.Pereira MA, Odegaard AO. Artificially sweetened beverages—do they influence cardiometabolic risk? Current atherosclerosis reports. 2013;15:375 10.1007/s11883-013-0375-z [DOI] [PubMed] [Google Scholar]
  • 27.Shankar P, Ahuja S, Sriram K. Non-nutritive sweeteners: Review and update. Nutrition. 2013;29:1293–9. 10.1016/j.nut.2013.03.024 [DOI] [PubMed] [Google Scholar]
  • 28.Miller PE, Perez V. Low-calorie sweeteners and body weight and composition: A meta-analysis of randomized controlled trials and prospective cohort studies. Am J Clin Nutr. 2014;100:765–77. 10.3945/ajcn.113.082826 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Pereira MA. Sugar-sweetened and artificially-sweetened beverages in relation to obesity risk. Advances in nutrition (Bethesda, Md). 2014;5:797–808. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Romo-Romo A, Aguilar-Salinas CA, Brito-Cordova GX, Diaz RAG, Valentin DV, Almeda-Valdes P. Effects of the non-nutritive sweeteners on glucose metabolism and appetite regulating hormones: Systematic review of observational prospective studies and clinical Trials. PLoS One. 2016;11 (8) (no pagination). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Bellisle F, Drewnowski A. Intense sweeteners, energy intake and the control of body weight. Eur J Clin Nutr. 2007;61:691–700. 10.1038/sj.ejcn.1602649 [DOI] [PubMed] [Google Scholar]
  • 32.EFSA Panel on Dietetic Products, Nutrition and Allergies (NDS). Scientific Opinion on the substantiation of health claims related to intense sweeteners and contribution to the maintenance or achievement of a normal body weight (ID 1136, 1444, 4299), reduction of post-prandial glycaemic responses (ID 4298), maintenance of normal blood glucose concentrations (ID 1221, 4298), and maintenance of tooth mineralisation by decreasing tooth demineralisation (ID 1134, 1167, 1283) pursuant to Article 13(1) of Regulation (EC) No 1924/2006. EFSA Journal. 2011;9:2229S. [Google Scholar]
  • 33.Pepino MY, Bourne C. Non-nutritive sweeteners, energy balance, and glucose homeostasis. Curr Opin Clin Nutr Metab Care. 2011;14:391–5. 10.1097/MCO.0b013e3283468e7e [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Sylvetsky A, Rother KI, Brown R. Artificial sweetener use among children: Epidemiology, recommendations, metabolic outcomes, and future directions. Pediatr Clin North Am. 2011;58:1467–80. 10.1016/j.pcl.2011.09.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Anderson GH, Foreyt J, Sigman-Grant M, Allison DB. The use of low-calorie sweeteners by adults: Impact on weight management. J Nutr. 2012;142:1163S–9S. 10.3945/jn.111.149617 [DOI] [PubMed] [Google Scholar]
  • 36.Brown RJ, Rother KI. Non-nutritive sweeteners and their role in the gastrointestinal tract. J Clin Endocrinol Metab. 2012;97:2597–605. 10.1210/jc.2012-1475 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Raben A, Richelsen B. Artificial sweeteners: a place in the field of functional foods? Focus on obesity and related metabolic disorders. Curr Opin Clin Nutr Metab Care. 2012;15:597–604. 10.1097/MCO.0b013e328359678a [DOI] [PubMed] [Google Scholar]
  • 38.Swithers SE. Artificial sweeteners produce the counterintuitive effect of inducing metabolic derangements. Trends Endocrinol Metab. 2013;24:431–41. 10.1016/j.tem.2013.05.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Araujo JR, Martel F, Keating E. Exposure to non-nutritive sweeteners during pregnancy and lactation: Impact in programming of metabolic diseases in the progeny later in life. Reprod Toxicol. 2014;49:196–201. 10.1016/j.reprotox.2014.09.007 [DOI] [PubMed] [Google Scholar]
  • 40.Ferreira AV, Generoso SV, Teixeira AL. Do low-calorie drinks ‘cheat’ the enteral-brain axis? Curr Opin Clin Nutr Metab Care. 2014;17:465–70. 10.1097/MCO.0000000000000082 [DOI] [PubMed] [Google Scholar]
  • 41.Freswick PN. Artificial Sweetened Beverages and Pediatric Obesity: The Controversy Continues. Children (Basel). 2014;1:31–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Gardner C. Non-nutritive sweeteners: evidence for benefit vs. risk. Curr Opin Lipidol. 2014;25:80–4. 10.1097/MOL.0000000000000034 [DOI] [PubMed] [Google Scholar]
  • 43.Bellisle F. Intense Sweeteners, Appetite for the Sweet Taste, and Relationship to Weight Management. Curr Obes Rep. 2015;4:106–10. 10.1007/s13679-014-0133-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Burke MV, Small DM. Physiological mechanisms by which non-nutritive sweeteners may impact body weight and metabolism. Physiol Behav. 2015;152:381–8. 10.1016/j.physbeh.2015.05.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Fernstrom JD. Non-nutritive sweeteners and obesity. Annu Rev Food Sci Technol. 2015;6:119–36. 10.1146/annurev-food-022814-015635 [DOI] [PubMed] [Google Scholar]
  • 46.Pepino MY. Metabolic effects of non-nutritive sweeteners. Physiol Behav. 2015;152:450–5. 10.1016/j.physbeh.2015.06.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Roberts JR. The paradox of artificial sweeteners in managing obesity. Curr Gastroenterol Rep. 2015;17:423 10.1007/s11894-014-0423-z [DOI] [PubMed] [Google Scholar]
  • 48.Swithers SE. Artificial sweeteners are not the answer to childhood obesity. Appetite. 2015;93:85–90. 10.1016/j.appet.2015.03.027 [DOI] [PubMed] [Google Scholar]
  • 49.Fowler SP. Low-calorie sweetener use and energy balance: Results from experimental studies in animals, and large-scale prospective studies in humans. Physiol Behav. 2016;164:517–23. 10.1016/j.physbeh.2016.04.047 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Glendinning JI. Do low-calorie sweeteners promote weight gain in rodents? Physiol Behav. 2016;164:509–13. 10.1016/j.physbeh.2016.01.043 [DOI] [PubMed] [Google Scholar]
  • 51.Nettleton JE, Reimer RA, Shearer J. Reshaping the gut microbiota: Impact of low calorie sweeteners and the link to insulin resistance? Physiol Behav. 2016;164:488–93. 10.1016/j.physbeh.2016.04.029 [DOI] [PubMed] [Google Scholar]
  • 52.Peters JC, Beck J. Low Calorie Sweetener (LCS) use and energy balance. Physiol Behav. 2016;164:524–8. 10.1016/j.physbeh.2016.03.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Shearer J, Swithers SE. Artificial sweeteners and metabolic dysregulation: Lessons learned from agriculture and the laboratory. Reviews in Endocrine and Metabolic Disorders. 2016;17:179–86. 10.1007/s11154-016-9372-1 [DOI] [PubMed] [Google Scholar]
  • 54.Swithers SE. Not-so-healthy sugar substitutes? Curr Opin Behav Sci. 2016;9:106–10. 10.1016/j.cobeha.2016.03.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Wang DD, Shams-White M, Bright OJ, Parrott JS, Chung M. Creating a literature database of low-calorie sweeteners and health studies: evidence mapping. BMC Med Res Methodol. 2016;16:1 10.1186/s12874-015-0105-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10 10.1186/1471-2288-7-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–20. 10.1016/j.jclinepi.2008.10.009 [DOI] [PubMed] [Google Scholar]
  • 58.Leenaars M, Hooijmans CR, van Veggel N, ter Riet G, Leeflang M, Hooft L, et al. A step-by-step guide to systematically identify all relevant animal studies. Lab Anim. 2012;46:24–31. 10.1258/la.2011.011087 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Hooper L, Abdelhamid A, Bunn D, Brown T, Summerbell CD, Skeaff CM. Effects of total fat intake on body weight. Cochrane Database Syst Rev. 2015:Cd011834 10.1002/14651858.CD011834 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.De Witt Huberts JC, Evers C, De Ridder DT. "Because I am worth it": a theoretical framework and empirical review of a justification-based account of self-regulation failure. Pers Soc Psychol Rev. 2014;18:119–38. 10.1177/1088868313507533 [DOI] [PubMed] [Google Scholar]
  • 61.Shemilt I, Hendry V, Marteau TM. What do we know about the effects of exposure to ‘Low alcohol’ and equivalent product labelling on the amounts of alcohol, food and tobacco people select and consume? A systematic review. BMC Public Health. 2017;17:29 10.1186/s12889-016-3956-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 Appendix. Search strategy.

(PDF)

S1 Table. Excluded studies from full text.

(PDF)

S2 Table. PRISMA checklist.

(DOC)

Data Availability Statement

The data underlying this study are collected from published and publicly available reviews. The authors did not have special access privileges. Information related to the reviews is included in S1 Table. All other relevant data are within the paper and its Supporting Information files.


Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES