Probiotic Therapy of the Irritable Bowel Syndrome: Why Is the Evidence Still Poor and What Can Be Done About It?

Abstract

Background/Aims

Despite numerous randomized clinical trials and meta-analyses, there is no increased evidence for the efficacy of probiotics in the treatment of irritable bowel syndrome (IBS). We review this evidence, identify and analyse the reasons for this lack of evidence and propose methodological improvements for future studies.

Methods

Based on a literature search, we identified 56 papers that matched the purpose of our analyses. Twenty-seven studies used multi-species bacterial preparations and 29 used single-strain probiotics. They were analysed regarding patients included, treatment duration, probiotic dosage, and outcome measures.

Results

Trials in both groups suffered from heterogeneity with respect to probiotic concentration, duration of treatment, and other methodological issues (crossover design and underpowered studies). This heterogeneity did not allow the application of a meta-analytic approach and a systematic review was therefore performed instead. Multi-strain preparations combined 2 to 8 different bacterial subspecies, mostly lactobacilli or bifidobacteria, and used variable lengths of treatments. Overall, more than 50% of trials presented negative outcomes. The majority of the single-strain probiotic trials employing lactobacilli or Saccharomyces were negative, whereas trials employing bifidobacteria showed positive results.

Conclusions

The heterogeneity of the studies of probiotics in IBS questions the value of meta-analyses. The use of different bacterial strains and different mixtures of these strains, as well as different dosages, are the main contributors to this heterogeneity. Current data provides limited evidence for the efficacy of a small number of single-strain probiotics in IBS (mostly bifidobacteria) and sound studies following strict trial guidelines (Food and Drug Administration and European Medicines Agency guidelines for clinical trials) are needed. We summarised and proposed some methodological issues for future studies in the field.

Introduction

“Probiotic” therapy–in contrast to antibiotic therapy–defines the use of single bacterial strains or combinations of bacterial strains to influence the commensal gut microbiota, despite the fact that the term “probiotic” was not used before the 1950s.^1,2 Parker³ first established a definition in 1974. Fuller⁴ defined probiotic as a “living microbial food supplement” that improves the host by improving its intestinal bacterial balance. His definition is still valid today. This also resembles the position of the WHO/FAS (2001) which state that probiotics are living organisms that, when ingested in sufficient amounts, may be beneficial for the host.⁵ This definition avoids any speculation about the presumed mechanism of probiotics’ action. A 2014 update of the definition by the International Scientific Association for Probiotics and Prebiotics (ISAPP) confirmed⁶ the latter definition. In addition, bacterial yoghurt cultures usually consisting of Lactobacillus delbrueckii subsp. bulgaricus and Streptococcus salivarius subsp. thermophilus, which were not previously regarded as probiotics, are now included in this group.

The use of probiotics in the treatment of gastrointestinal (GI) tract disorders increased during the 1990s. Among the disorders that have been shown to benefit from probiotic treatment in placebo-controlled double-blinded studies are antibiotic-associated diarrhea,⁷ traveller’s diarrhea,⁸ rotavirus induced diarrhea,⁹ and Clostridium difficile-associated diarrhea.¹⁰ Beneficial effects have also been shown in diarrhea in children.¹¹ Other bacterial strains showed efficacy in chronic constipation.¹² Another potential indication was reported recently in the remission phase of ulcerative colitis,¹³ whereas randomized clinical trials (RCTs) in Crohn’s disease mostly revealed negative outcomes.¹⁴ Testing of probiotic therapy and targeting functional GI symptoms such as defecation problems (diarrhea and constipation) which are, however, not the predominant feature of diseases such as irritable bowel syndrome (IBS) were a significant step forward.

However, clinical trials prior to the formation of the international Rome consensus for functional bowel disorders in 1988 mostly lack credibility because a homogenous disease definition was lacking.¹⁵ The consequence was that in some trials, patients with or without abdominal pain in association with altered bowel habits were classified as IBS sufferers (or irritable colon, mucous colitis, or otherwise).¹⁶ In studies based on the Rome consensus IBS definition,¹⁷ efficacy was often based on improvement of individual symptoms including abdominal pain, diarrhea, and/or constipation.

A number of systematic reviews and meta-analyses^18–26 analysed the efficacy of probiotics in the management of IBS and these publications are frequently mentioned and cited in consensus papers.²⁷

Over the years general assessments have remained rather similar, presuming an overall positive effect whilst claiming a lack of high-quality data: “Probiotics may improve symptoms of irritable bowel syndrome and can be used as supplement to standard therapy.”¹⁹ “While our analyses suggest that probiotic use may be associated with improvement in IBS symptoms compared to placebo, these results should be interpreted with caution given the methodological limitations of contributing studies. Probiotics warrant further study as a potential therapy for IBS”²⁰: “Probiotics may have a role in alleviating some of the symptoms of IBS; however further research should focus on the type, optimal dose of probiotics and the subgroups of patients who are likely to benefit the most”²³: “Six of the eight diseases (...irritable bowel syndrome...) showed positive significant effects.... Across all diseases and probiotic species, positive significant effects of probiotics were observed for all age groups, single vs multiple species, and treatment lengths.”²⁵

Among these meta-analyses, the largest (including 35 trials) found a significant effect of probiotics on global IBS symptom rating for all 24 mono-strain preparations pooled²⁶; however neither lactobacilli nor bifidobacteria was effective when tested separately. There was also an overall benefit of the 15 combination trial; however, when specific combinations were tested, neither yielded a significant benefit over placebo. A significant effect of probiotics on individual IBS symptoms was only found for bloating and flatulence. A later meta-analysis covering data published between 2007 and 2013, which included only 14 RCTs, found probiotics to have a positive effect on abdominal pain only.²⁸

The initial idea of our project was a meta-analysis on the efficacy of probiotics in the management of IBS. However, after the initial search and data extraction (see Methods below), we concluded that the current data did not allow a methodologically sound meta-analysis to be performed. We therefore performed a new literature search with a changed scope: to critically review the body of evidence and to identify and assess the reasons for poor evidence in the current literature, as reflected in recent meta-analyses. We also proposed ways to improve methods for studies in the future. Our new analysis was conducted in 4-steps: (1) literature search, (2) identification of systematic reviews and meta-analyses, (3) comparison of lists from Step 1 and 2, and lastly (4) sorting the final list according to specific criteria.

Methods

In our initial search early in 2014, we performed a PubMed literature search using the search terms “(probiotic OR prebiotic) AND (irritable-bowel-syndrome OR IBS OR functional-bowel-disorder) AND trial” to retrieve randomized trials comparing a probiotic or prebiotic to a control treatment in functional bowel disorders such as IBS.

The abstracts of retrieved papers were screened to match the following inclusion criteria:

1. The study including IBS patients according to pre-defined criteria (Manning, Kruis, and Rome);

2. The study only including adult patients;

3. The study design including a placebo arm;

4. The study being described as randomized;

5. The study with attempted blinding of patient assignment;

6. The study describing the prebiotic or the probiotic bacterial strain in sufficient detail;

7. The study assessing either global IBS measures or single symptoms or QOL;

8. The full paper being available and written in English.

If only an abstract was available, we excluded the study. After exclusion of non-relevant studies, the remaining articles were screened for the following criteria:

1. Whether the study population was IBS, IBS-C or IBS-D, and IBS-M;

2. The number of patients included in the study;

3. Whether the probiotic was single-strain or multiple-strain;

4. In the case of multi-strain products, the strains included;

5. Whether it was a pharmacological or nutritional probiotic;

6. Whether the design was a parallel-group design or a cross-over design;

7. The duration of treatment.

All screenings were performed by 2 authors (E.B. and M.S.). In case of disagreement, a third author (N.M.) was consulted for a final decision. We also assessed the Jadad score for each selected study in order to assess the methodological quality of the clinical trials.

After the first review of the retrieved data and the decision to change the purpose and methodology of the project, we implemented a 4-step approach that helped us achieve our goals.

In Step 1 we performed a new literature search using the same key words– “(probiotic OR prebiotic) AND (irritable-bowel-syndrome OR IBS OR functional-bowel-disorder) AND trial” –which resulted in the generation of new literature set. The last update was performed on April 20, 2015. Retrieved papers were screened again using the inclusion and exclusion criteria described above.

In Step 2 we identified systematic reviews and meta-analyses in our literature search published between 2006 and 2015, which summarise the respective actual state of knowledge. They were reviewed for the included studies as well their major findings across all studies. A complete list of trials included in the reviewed meta-analyses and systematic reviews was generated independently from the literature list from Step 1.

In Step 3 we compared both retrieved literature lists from Step 1 and from Step 2 and identified papers included in one or both lists as well as those that were excluded by us, but which were included in one or more of the performed meta-analyses. For each discrepancy found, we discussed the respective paper among all authors to reach a final decision about it’s inclusion or exclusion using the same criteria described in Step 1.

Step 4: the final list of included studies was sorted according to the following criteria:

1. Single-strain or multi-strain probiotic;

2. Manufacturer of the preparation;

3. If multi-strain, which bacterial strains were present;

4. The dosage of each strain;

5. The form of preparation either as a nutritional product or a drug;

6. If single-strain, the bacterial strain used, and at which dosage;

7. Trial length and patient groups included;

8. Study outcome as defined by the authors–positive or negative.

Based on our critical review of the current literature, we finally discussed the potential cornerstones of studies to investigate the efficacy of probiotics in IBS in the future.

Results

Step 1: Figure 1 shows the results of the literature search, last performed on April 20, 2015. After exclusion of systematic reviews and meta-analyses (n = 9), uncontrolled trials (n = 20), narrative reviews, editorial, letters and comments (n = 78), animal research (n = 2), studies of children and respective reviews (n = 10), non-English papers and studies (n = 8), and others (n = 8), 60 studies remained. Data published in abstract form only was not included.

Figure 1.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) scheme of retrieved literature.

Following a full-paper review, we excluded 8 further studies due to various reasons: only 1 study reported data from prebiotic treatment,²⁹ 1 study did not use a probiotic in a strict sense (no living bacteria),³⁰ 2 were microbial investigations of earlier studies,^31,32 2 were single-blinded,^33,34 and 2 were uncontrolled interventions.^35,36 Based on hand-searches of the published meta-analyses, we added 3 more studies,^37–39 resulting in a total of 55 articles.

Step 2: we identified 10 meta-analyses and systematic reviews (published between 2008 and 2015) related to probiotic trials in IBS,^18–26,28 reporting a total of 44 RCTs with variable number of trials in each of them that increased over the years. One “systematic review” of complementary and alternative therapies in IBS was not comprehensive at all for IBS probiotic trials¹⁸; it included only 4 studies that were also analysed and discussed in all other reviews. Another review²¹ summarized previous systematic reviews and meta-analyses,^{19,22–24,40,41} but did not provide the full list of citations of papers and was therefore of no further help for the purpose of this review.

Step 3: we then compared the literature lists from Step 1 and Step 2 to identify the degree of overlap between each analysis. Of the 45 papers listed in Step 2, we excluded 14 for different reasons.

Six citations were available as abstracts only^42–47 and were thus of no value for our study. One of those⁴³ has been published as a full paper later⁴⁸ and was included in our sample; another⁴⁴ was also published as full paper³⁰ but was excluded since it was not a probiotic in a strict sense; 2^45,46 had only an English abstract however the full papers were written in Korean and Chinese each.

Two frequently included papers reported studies in children.^49,50 Two were placebo controlled but single-blinded studies,^33,51 one combined acupuncture with probiotic treatment,⁴⁶ and one³² reported microbiota changes from a probiotic trial published before⁵² which was already included in our list.

One study was not able to be verified⁵³ but was, by title, identical to another article by the same authors published in French⁵⁴ It was excluded because it was published too long before the first standardized diagnostic criteria for IBS to become available.⁵⁵

One more study⁵⁶ from Step 2 was added to our list of Step 1 studies, resulting in 56 studies available for our analysis.

Taking into account only the studies on our list and in at least one of the other analyses, overall agreement was low, as can be seen in Figure 2: only 5 RCTs from all 56 were included in all 7 previous analyses (unless the analyses restricted their timeframe, as in²⁸) as well as in ours. Seven RCTs were exclusively analysed by the largest study²⁶ and our analysis added another 20 studies to the list.

Figure 2.

Schematic overlap of 56 randomized clinical trials or probiotics in irritable bowel syndrome therapy, as they were included or excluded into 10 systematic reviews and meta-analyses published between 2008 and 2015. Numbers refer to the numbering in the reference list.

Step 4: of the 56 RCTs, 27 reported data from multi-strain probiotics, and 29 from single-strain probiotic products. Such a distinction was not made in the majority of the meta-analyses. Except in 2 studies,^24,26 all others pooled data and provided overall statistics and relative risk calculations.^{19,20,23,25,28} One²² did perform a systematic review instead of a meta-analysis, but nevertheless mixed both groups.

Multi-strain Probiotics

The 27 included studies using multi-strain probiotics are listed in Table 1.^{38,39,48,57–80} Despite this common character, the studies were heterogeneous in many other aspects.

Table 1.

Multi-strain Randomized Placebo-controlled Probiotics Trials in Irritable Bowel Syndrome

Author	Ref	IBS-type	N	Dur	Global result on IBS symptoms
Shavakhi 2014	57	All	132	2 wk	Negative
Jafari 2014	58	All bloat	108	4 wk	Positive
Lorenzo-Zúñiga 2014	59	IBS-D	84	6 wk	Negative exc QoL
Sisson 2014	60	All	186	12 wk	Positive
Ludidi 2014	61	All	40	6 wk	Negative
Ko 2013	62	IBS-D	53	8 wk	Negative
Begtrup 2013	63	All	131	6 mo	Negative
Yoon 2013	64	All	49	4 wk	Positive
Roberts 2013	65	IBS-C	179	12 wk	Negative
Cui 2012	39	All	60	4 wk	Positive
Min 2012	66	All	130	7 day	Positive
Capello 2012	67	All	64	4 wk	Negative exc bloating
Cha 2012	68	IBS-D	50	10 wk	Positive
Michail 2011	69	IBS-D	24	8 wk	Negative exc satiety
Hong 2011	70	All	74	8 wk	Negative
Sondergaard 2011	71	All	64	8 wk	Negative
Ringel-Kulka 2011	72	FGD	60	8 wk	Positive
Simren 2010	48	All	74	8 wk	Negative exc week 1
Hong 2009	38	All	70	8 wk	Positive
Williams 2009	73	All	52	8 wk	Positive
Kajander 2008	74	All	86	5 mo	Positive
Drouault 2008	75	All	100	4 wk	Negative
Kajander 2005	76	All	103	6 mo	Positive
Kim 2005	77	All	48	4–8 wk	Negative exc bloating
Bittner 2005	78	All	25	2 wk	Positive
Saggioro 2004	79	All	50	4 wk	Positive
Kim 2003	80	IBS-D	25	8 wk	Negative

Ref, number in reference list; N, number of patients in study; Dur, duration of treatment; IBS, irritable bowel syndrome; bloat, bloating; IBS-D, diarrhea predominant IBS; exc, excluding/except; QoL, quality of life; IBS-C, constipation predominant IBS, FGD, functional gastrointestinal disease.

Study characteristics

The number of patients included ranged between 24 and 186 (median = 90). One study included patients with functional gastrointestinal disorders, ie, with diarrhea but not constipation and not necessarily suffering from abdominal pain,⁷² one was restricted to constipation predominant IBS (IBS-C) patients⁶⁵ and 4 included only diarrhea predominant IBS (IBS-D) patients.^62,68,69,80 All studies claimed the use of Rome (II or III) criteria, nevertheless, a more detailed description of the patient population (the number of IBS-D, IBS-C, and mixed IBS [IBS-M] patients) was lacking.

Treatment duration varied from 7 days to 6 months, thus not fulfilling Rome criteria for treatment duration in many cases,¹⁷ with the majority of studies assessing treatment results after 8 weeks. A common outcome-reporting period (4 weeks for example) was not available for comparison across all studies and neither were the primary nor the secondary outcome variables according to the FDA/EMA or Rome criteria in most studies.

Bacterial strain combinations

The largest heterogeneity was noted with respect to the bacterial strain combinations used (Table 2). Most of the multi-strain preparations were nutritional supplements from different suppliers worldwide. The same preparation (VSL#3) was used in only 3 studies.^69,77,80 Up to 8 different bacterial strains are used in some preparations and the combinations are numerous, since in many cases subspecies of specific strains were used that are only available to (and eventually patented by) individual companies. Two studies were performed by the same research group with the same preparation.^74,76

Table 2.

Composition of Multi-strain Probiotic Preparations

Author	Ref	Company	S. thermophilus	L. bulgaricus	Dosage (cfu)	L. casei	L. salivarius	L. lactus	L. acidophilus	L. rhamnosus	L. plantagus	L. longum	B. lactus	B. brevis	B. lonum	B. bifidum	B. infantis
Shavakhi 2014	57	Protexin	x	x	1 × 108	x			x	x				x	x
Jafari 2014	58	Pro-Biotec	x	x	4 × 109a				x				x
Lorenzo-Zúñiga 2014	59	AB-Biotics	?	?	3 × 1010						x x
Sisson 2014	60	Symprove	?	?	1 × 1010				x	x	x
Ludidi 2014	61	Winclove	?	?	5 × 109	x	x	x	x	x			x
Ko 2013	62	Cell Biotech	x	?	5 × 109				x		x	x	x	x	x
Begtrups 2013	63	Arla	?	?	1.3 × 1010	x			x							x
Yoon 2013	64	Cell Biotech	x	?	5 × 109				x	x			x		x	x
Roberts 2013	65	Danone	x	x	1.25 × 1010								x
Cui 2012	39	Sine Pharm	?	?	NR				x			x
Min 2012	66	Namyang	x	?	1 × 1011				x				x
Capello 2012	67	CaDi Group	x	?	1.4 × 1010	x	x		x		x	x					x
Cha 2012	68	Cell Biotech	x	?	1 × 1010			x	x	x	x			x	x
Michail 2011	69	VSL#3	x	x	9 × 1011	x			x		x			x	x		x
Hong 2011	70	Yakult	?	x	4 × 109										x
Sondergaard 2011	71	Arla	x	x	7.5 × 1010	x			x				x
Ringel-Kulka 2011	72	NR	?	?	1× 1011				x				x
Simren 2010	48	Cultura	x	x	1 × 1010	x			x				x
Hong 2009	38	Bifido Co	?	?	2 × 1010	x			x				x			x
Williams 2009	73	Cultech	?	?	2.5 × 1010				x				x			x
Kajander 2008	74	Valio Ltd	?	?	4.8 × 109					x x						x
Drouault 2008	75	NR	x	?	1× 1010			x	x						x
Kajander 2005	76	Valio Ltd	?	?	8–9 × 109					x x						x
Kim 2005	77	VSL#3	x	x	4.5 × 1011	x			x		x			x	x		x
Bittner 2005	78	Prescript	?	?	NR				29 soil-based benign organisms plus prebiotics
Saggioro 2004	79	Probial	?	?	1 × 1010				x		x			x
Kim 2003	80	VSL#3	x	x	4.5 × 1011	x			x		x			x	x		x

^aIndicates dosage included Streptococcus thermophiles and Lactobacillus bulgaricus.

Ref, number in reference list cfu, colony-forming units; NR, not reported.

An “x” indicated use of this bacteria, an “x x” indicates use of 2 different bacteria of the same strain.

The use of Streptococcus thermophilus and Lactobacillus bulgaricus as starting cultures for yoghurt production is occasionally noted; in the remaining cases it remains unclear whether these strains were included. They were not regarded as probiotics until 2014, when the ISAPP consented that they are probiotics by definition.⁶

While some strains (such as Lactobacillus acidophilus) are used in almost all studies, many if not most of the strains are used exclusively in a few combinations. The table lists only bacterial strains that were used at least twice among the included studies.

The concentration of bacteria in these studies was not always reported^57,78 and varied substantially, ranging from 10⁸ (Shavakhi et al⁵⁷) to 10¹¹ (Michail et al⁶⁹), providing that the concentrations listed in the papers are correct and that (specifically for dairy products) the cooling chain had not been interrupted. Only one study compared a high dose (3 × 10¹⁰ colony forming unit [cfu]) with a low dose (6 × 10⁹ cfu) and found both equally ineffective compared to placebo.⁵⁹

Due to the heterogeneity of the different probiotic compositions, it appeared inappropriate to perform a meta-analysis; except for the 3 VLS#3 studies which have been reported to provide no benefit over placebo in IBS.²⁶

Global outcome

Most studies included here reported their outcome in global terms. We used these explicit summaries of the efficacy of the respective probiotic treatment to retrieve a globally rated “positive” or “negative” IBS treatment outcome. However, we are aware of the fact that this may be biased by the respective authors that tend to underestimate or undervalue negative results, for instance in comparison to registered trial reports.^81,82

We found that 14 of the 27 studies reported negative outcome on global symptoms, however individual symptoms (bloating^67,77 and satiety⁶⁹) were occasionally reported as being responsive to treatment. Quality of life (QoL) improved in one study but symptomatic improvement was not different from placebo.⁵⁹ Thirteen studies were positive on global symptom reports. While the size of the study populations has increased, this did not affect the (positive or negative) outcome of the studies, as more patients were included in the studies reporting negative outcomes than in the studies with positive outcomes (Table 1).

In summary, the balance of positive and negative studies indicates an arbitrary and random result rather than an effective outcome of treatment attempts in IBS using multi-strain probiotics. This contrasts with the summary of Ford et al²⁶ that found an overall positive effect of multi-strain probiotics in IBS when all included studies were pooled, but not of individual combinations.

Single-strain Probiotics

Study characteristics

The 29 single-strain probiotic intervention studies exhibited a large variability with respect to various design features, as shown illustrated in Table 3.^{37,52,56,83–108} Four studies employed a cross-over design,^52,56,90,105 which is frequently used for motivational purposes (patients are easier to recruit when they are offered an effective treatment, at least for a part of the trial). However, they are difficult to evaluate since the assumption that the data in both periods is equal and can be merged may be distorted by unblinding and conditioning effects.¹⁰⁹

Table 3.

Randomized Clinical Trials With Mono-strain Probiotics in Irritable Bowel Syndrome

Author	Ref	Strain	Drug or Nutr	Daily dosage (cfu)	IBS-type	N	IBS def	Therapy duration	Results
de Chambrun 2014	83	S. cerevisiae	Nutri	4 × 109	All	179	R III	8 wk	Negative exc pain
Abbas 2014	84	S. boulardi	NR	750 mg	IBS-D	72	R III	6 wk	Negative exc for QoL
Stevenson 2014	85	L. plantagus	Drug	1 × 1010	IBS-C/D	81	R II	8 wk	Negative
Rhoga 2014	86	Bac coag + preb	Nutri	4.5 × 108	All	85	R II	12 wk	Positive
Urgesi 2014	87	Bac coag + simet	Drug	NR	All	52	R III	4 wk	Positive
Charbonneau 2013	88	B. infantes	NR	1 × 109	All	76	Rome	8 wk	Negative
Ducrotte 2012	89	L. plantagus	Drug	1 × 1010	All	214	R III	4 wk	Positive
Murakami 2012	90	L. brevi	Drug	1 × 1010	All	35	R III	2 × 4 wk	Negative
Dapoigny 2012	91	L. casei	Drug	6 × 108	All	50	R III	4 wk	Negative
Kruis 2012	92	E. coli Nissle	Drug	5 × 109	All	120	R II	12 wk	Negative exc PI-IBS
Kabir 2011	93	S. boulardi	NR	500 mg	All	70	R II	4 wk	Negative
Guglielmetti 2011	94	B. bifidum	Nutri	1 × 109	All	122	R III	6 wk	Positive
Choi 2011	95	S. boulardi	Drug	8 × 1011	IBD-D/M	67	R II	4 wk	Negative exc for QoL
Ligaarden 2010	52	L. plantagus	Drug	1 × 1010	All	16	R II	2 × 3 wk	Negative
Dolin 2009	96	B. coagulans	Drug	2 × 109	IBS-D	55	R III	8 wk	Positive
Hun 2009	97	B. coagulans	Drug	8 × 108	IBS-D	44	R II	8 wk	Positive
Enck 2009	98	E. coli	Drug	> 1 × 108	All	298	Pre R	8 wk	Positive
Agarwal 2008	99	B. lactis	Nutri	1.25 × 1010	IBS-C	41	R III	4 wk	Positive
Sinn 2008	100	L. acidophilus	Nutri	2 × 109	All	40	R III	4 wk	Positive
Guyonnet 2007	101	B. animalis	Nutri	2.5 × 1010	IBS-C	274	R II	6 wk	Negative exc for QoL
Whorwell 2006	102	B. infantes	Nutri	1 × 108	All (female)	362	R II	4 wk	Positive
Niv 2005	103	L. reuteri	Nutri	2 × 108	All	54	R II	6 mo	Negative
O’Mahony 2005	104	Bifidubacterium vsLactobacillus	Drug	1 × 1010	All	77	R II	8 wk	Negative for Lactobacillus
Sen 2002	105	L. plantagus	Nutri	6.25 × 109	All	12	Rome	2 × 4 wk	Negative
Niedzielin 2001	106	L. plantagus	Nutri	2 × 1010	All	40	Pre R	4 wk	Positive
O’Sullivan 2000	107	L. rhamnosus	Drug	1 × 1010	All	24	Rome	2 × 8	Negative
Nobaek 2000	108	L. plantagus	Nutri	2 × 1010	All	60	R I	4 wk	Negative exc bloating
Halpern 1996	56	L. acidophilus	Nutri	2 × 1010	All	18	Pre R?	2 × 6 wk	Positive
Gade 1989	37	St. faecium	Nutri	8 × 106	All	44	Pre R	2 wk	Positive

Ref, number in reference list; Nutr, nutritional supplement; cfu, colony forming units; IBS, irritable bowel syndrome; N, number of patients in study; RI/II/III, Rome definition of IBS; NR, not reported; 2 × 3 or 2 × 4 or 2 × 6, 2 consecutive treatment periods of 3, 4, or 6 weeks (in a cross-over design); exc, except; IBS-D, diarrhea predominant IBS; IBS-C, constipation predominant IBS; IBS-M, mixed IBS; QoL, quality of life; PI-IBS, postinfectious IBS; S., Saccharomyces; L., Lactobacillus; B., Bifidobacterium; B., Bacillus; St., Streptococcus; coag + preb, Bacillus coagulans+ Prebiotikum; coag + simet, Bacillus coagulans+ simethicone. Pre R was used in a case when other than Rome criteria (eg, Manning) were used to define IBS or the type of criteria was not reported in the paper (“Pre R?”).

Compared to multi-strain studies, more investigations (15/29) were limited to 4 weeks (or 2 × 4 weeks in a cross-over design) and some extended up to 8 weeks or beyond (11/29). Similarly, the range of included IBS patients ranged from 12 to 362. Two trials included IBS-C^99,101 and 3 trials only IBS-D patients,^84,96,97 while 2 studies stratified the patients into IBS-C and IBS-D⁸⁵ or into IBS-D and IBS-M.⁹⁵ Only one study addressed another subgroup of patients who may benefit from probiotics: patients with post-infectious IBS.¹¹⁰ Unfortunately, the researchers did this post-hoc rather than prospectively (see below).⁹²

Inclusion criteria also varied. Four studies did not specify which criteria were used defining IBS patients,^{56,88,105,107} which may indicate that data was collected before Rome criteria became effective and in 2 studies, this was expressed in the article (pre-Rome^98,106). Whilst male patients were included, females were dominant in all trials; one study exclusively recruited female patients.¹⁰²

Bacterial strains

With respect to the bacterial strains studied, the heterogeneity continued: 6 studies used Lactobacillus plantagus,^{52,85,89,105,106,108} however of different origin and subspecies; some were nutritional supplements and some were developed as drugs. Other lactobacilli strains were used only in single trials, such as Lactobacillus brevi,⁹⁰ L. acidophilus,^56,100 Lactobacillus reuteri,¹⁰³ Lactobacillus rhamnosus GG,¹⁰⁷ and Lactobacillus casei (CLR35).⁹¹

A similar picture emerged with bifidobacteria. Individual trials have used Bifidobacterium bifidum (MIMBb75),⁹⁴ Bifidobacterium lactis,⁹⁹ Bifidobacterum animalis,¹⁰¹ and Bifidobacterium infants.^88,102

A number of studies have used the yeast Saccharomyces boulardi;^84,93,95 however, it is unclear whether they used the same strain, produced and/or distributed by different companies. A novel study used a variant, Saccharomyces cerevisiae.⁸³ Four studies investigated the efficacy of Bacillus coagulans.^86,87,96,97 again from different companies, in 2 instances combined with a prebiotic⁸⁶ or simethicone,⁸⁷ which makes direct comparison difficult. One study used a Streptococcus faecium strain³⁷ and 2 used different Escherichia coli strains, E. coli Nissle,⁹² and E. coli.⁹⁸

Only one study compared 2 completely different probiotics and found that B. infantes was superior to Lactobacillus salivarius, which was not superior to placebo.¹⁰⁴ One study¹⁰² compared 3 different putatively effective doses (1 × 10⁶, 1 × 10⁸, and 1 × 10¹⁰ cfu) of a single strain, B. infantis and found the lowest and the highest dose to be ineffective. The authors claimed a technical issue as being responsible for the inefficacy of the highest dose (1 × 10¹⁰ cfu).

As with multi-strain probiotics, the concentration of bacteria in these studies was either not reported,^84,87,93 or the daily dosage varied by almost a factor of 1000, ranging from 1 × 10⁸ (Whorwell et al¹⁰²) to 8 × 10¹¹ (Choi et al⁹⁵).

Global outcome

Again, we used the authors’ evaluation of their results and separated positive from negative studies. Studies that noted an improvement of QoL without changes in abdominal symptoms (pain, stool frequency, and bloating),^84,95,101 were labelled as “negative except QoL,” since it is against all recommendations (Rome criteria and FDA/EMA) to use QoL as a primary endpoint in IBS treatment studies.

Three more studies were identified as negative, contrary to the evaluation of the respective authors: one found a significant effect on abdominal pain, but not on other IBS symptoms. Upon visual inspection of the data (ibid., see Figure 3 in Pineton de Chambrun et al⁸³), the effect occurred during weeks 6 to 8 with a sudden change in pain severity, mimicking a “recruitment bias” of unknown origin; however, it lacks any rational discussion and explanation. A similarly surprising and unexplained change occurred in the responder data of another study (ibid., see Figure 2 in Kruis et al),⁹² during weeks 9 to 12, which may explain for the post-hoc significance in a subgroup of patients with post-infectious IBS. Finally, one study¹⁰⁸ found significant changes in bloating scores, but not for any other core symptoms of IBS.

Balancing all studies revealed that 15 of the 28 studies (excluding the study comparing lactobacilli against bifidobacteria¹⁰⁴) yielded a negative or at least partly negative outcome. Taking all patients in positive and negative studies into account, more patients had a benefit from probiotics than those who did not. This distribution is even more skewed if we allow the studies with limited positive outcome (see above) to count as “positive.” In this case, only 9 of the 29 studies had a clearly negative outcome.

With respect to individual bacterial strains, the ratio of positive to negative studies with Lactobacillus plantagum was 3:3 and for all other lactobacilli studies 2:5 (including the negative results for Lactobacillus compared to placebo in the comparator study).¹⁰⁴ In contrast, the number of positive to negative studies with bifidobacteria was 4:2 (including O’Mahony et al¹⁰⁴). Studies using bifidobacteria included the largest samples,^101,102 indicating that bifidobacteria may be a clinically relevant treatment option in IBS.

The 4 studies using the yeast Saccharomyces^83,84,93,95 demonstrated no or only limited efficacy in IBS. In contrast, all 4 trials with Bacillus coagulans^86,87,96,97 were positive; however, both groups were conducted with rather small groups of IBS patients and require larger studies. The (positive or negative) single studies using E. coli,⁹⁸ E. coli Nissle,⁹² and Streptococcus faecium³⁷ need independent replication before a preliminary conclusion can be drawn.

Discussion

Our systematic review of 56 published RCTs of probiotics in IBS and of 10 systematic reviews and meta-analyses, including most of these trials, identified major flaws in the RCTs, obscuring the evidence for the efficacy of probiotics as a treatment option in IBS. Consequently, less evidence was produced with increasing numbers of RCTs added to meta-analyses. The latest and largest of these meta-analyses²⁶ concluded that while across all studies moderate evidence exists for efficacy of probiotics in general (though restricted to single-strain preparations) on global symptoms, neither individual bacterial strain reached a sufficient level of evidence, nor are individual core IBS symptoms (except bloating) effectively treated by any of the tested bacteria. Multistrain probiotics appear to be of no value at all. Narrative reviews on the efficacy of probiotic treatment in IBS usually follow these systematic reviews and meta-analyses.¹¹¹

In an attempt to understand why the picture has not improved but has rather developed into the opposite in recent years, we analysed all available studies for homogeneity in the applied drugs (or food supplements), in design and trial features and global outcomes and found that heterogeneity rather than homogeneity has increased over the years. Patient recruitment and selection especially, treatment duration, probiotic dosages and the choice of primary and secondary endpoints of the study have not followed the same route that pharmacology of GI drugs have paved through the FDA,¹¹² EMA,¹¹³ and international consensus parties such as the Rome group.¹⁷ Whilst in GI pharmacology placebo rates have dropped and efficacy of drugs over placebo has increased,¹¹⁴ the development of probiotics has not followed the same trend. One reason for this might be that, investments in clinical research and trials are still the exception rather than the rule for the nutritional (dairy) industry, not least because of lower profit rates and higher turn-around of investments into marketing of products.

Instead of rigorously questioning the value of previously tested probiotics in IBS, we finally attempted to outline the requirements for future trials to overcome these limitations. Many of these are already clinical standards that have been set during the last 25 years in GI drug development, by drug companies, approval authorities, and expert consensus parties. These have not been adopted yet by the nutritional industry that is responsible for most of the compounds that have been put to the market. However, clearly there cannot be different standards for drug and nutrient testing, and the future common policy of EMA and European Food Safety Authority in Europe¹¹⁵ has underlined this.

How to Study Efficacy of Probiotics in Irritable Bowel Syndrome

Since GI health claims for probiotics have to assess efficacy with respect to patients with functional bowel disorders of IBS-type,¹¹⁵ they have to be measured with standards of drug testing in the same area, ie, according to FDA and EMA rules^112,113 and the consensus of the gastroenterology community.¹⁷ The following provides a summary of putative paradigms to guide future trials.

First and foremost, any clinical study should be registered and it should be registered before the study start and not after the data evaluation.^57,86,92 Only 6 of the 27 multi-strain probiotic studies were registered. Even more surprisingly, only 3 of the 29 studies with single-strain probiotics were registered despite the fact that the single-strain probiotics were used not as nutritional supplement, but as drug by the majority. Registration prevents (silent) deviations from the proposed protocol and statistical evaluation strategy and diminishes doubts as to whether the reported efficacy data was a re-interpretation of missing efficacy on the primary endpoints chosen (see above).

Secondly, the study would need to be adequately powered. As we have repeatedly discussed,¹¹⁴ studies with samples size lower than 100 patients are at increased risk of producing high and variable placebo response rates and therefore would need strong effects on the primary study endpoint to achieve significance. However, this cannot be expected with nutritional interventions (even when provided as approved drugs) which is confounded by the variability of daily nutrition including consumption of preand probiotics.¹¹⁶ In this respect, 19 of the 27 multi-strain and 21 of the 29 single-strain probiotic studies had a priori low chance of finding significant effects; however, to our knowledge only a few were based on a power calculation.

Thirdly, a probiotic study in IBS patients would preferentially not use a cross-over design; such designs have almost completely been abandoned in gastrointestinal pharmacology and elsewhere for obvious reasons. They carry the risk of unblinding,¹⁰⁹ of cross-over effects when wash-out periods are too short,¹¹⁶ and of conditioning effects in the second (placebo) phase when the initial treatment is the probiotic.¹¹⁷ Consequently, the statistical evaluation should be restricted to the first treatment phase; however, these studies are usually not powered for such “quasi parallel design.” Only 4 of the single-strain studies and none of the others installed a cross-over design.

Furthermore, inclusion and exclusion criteria should closely follow the EMA and FDA guidelines for clinical trials in IBS,^112,113 including the definition of minimal severity for inclusion, global primary endpoints, and adequate secondary endpoints (pain, bloating, and a clinically meaningful responder definition). It should incorporate at least 8 weeks of treatment, an adequate follow-up interval and restriction to one of the different IBS subtypes. Documentation of nutritional habits as well as stool and pain diaries should be mandatory.

Finally, either a single-strain probiotic should be tested against placebo, or 2 different probiotics in a comparative effectiveness research design¹¹⁸ and the compounds chosen should be based on the available evidence. This excludes multi-strain probiotics, dairy products that require maintenance of the cooling chain and probiotics which have not been able to demonstrate superiority over placebo despite frequent trials, as documented in our list above. Comparative effectiveness research trials need larger samples (for demonstration of non-inferiority and the choice of the comparator is essential.¹¹⁸ A 3-arm trial (such as O’Mahony et al¹⁰⁴) testing 2 different probiotics against placebo is certainly an elegant alternative. Comparing probiotic treatment to other dietary or non-dietary therapies (prebiotics⁸⁶ and simethi-cone⁸⁷) or adding probiotics to other treatments (acupuncture⁴⁶) may help in recruiting patients but does not support the search for evidence.

We are aware of the limitations of our own analysis. While we included as many published trials as possible into this systematic review, we excluded trials reported as abstracts only and those not published in English. For a meta-analysis of overall efficacy, this may create a publication bias: however, for a critical review of methodological flaws this may not be as relevant since detailed data needed for critical analyses is usually not included into congress abstracts. We also ignored differences in primary endpoints (and their statistical characteristics, continuous or binary) reported in these studies and referred to reported overall efficacy data as “true” –assuming that the authors have reported the best evidence they could gain from their own data, irrespective of what the initial evaluation plan was. Otherwise, the overall evidence may have been even poorer. Finally, since most of the multi-strain preparations were nutritional supplements from different suppliers worldwide, the same product may have been available in different countries under different brand names with different labelling and approved for different indications due to differences in national regulations for approval of nutritional supplements. We were not able to resolve this issue.

When evaluating the efficacy of the FODMAP diet in IBS on behalf of the Rome Foundation, Yao et al¹¹⁶ thoroughly discussed the methodological difficulties of dietary trials, with an emphasis on functional bowel disorders. Most importantly, dietary intervention trials need to address the collinearity between food, nutrients, and bioactive components which could obscure the relationship between food and their effects in the gut. The authors provided recommendations for such trials, ranging from assessment and documentation of baseline nutrients intake, via adequate masking of study food, to the design of adequate sham controls; however, the authors insist that for food trials, the randomized, double-blind and placebo-controlled study remain as the gold standard.

In summary we conclude, that the heterogeneity of the studies of probiotics in IBS questions the value of meta-analyses. The use of different bacterial strains and different mixtures of these strains, as well as different dosages, are the main contributors to this heterogeneity. Current data provides limited evidence for the efficacy of a small number of single-strain probiotics in IBS (mostly bifidobacteria) and sound studies following strict trial guidelines (FDA and EMA guidelines for clinical trials) are needed.