Guide to designing, conducting, publishing and communicating results of clinical studies involving probiotic applications in human participants

Abstract

The heterogeneity of human clinical trials to assess the effectiveness of probiotics presents challenges regarding interpretation and comparison. Evidence obtained from clinical trials among a population with a disease or specific risk factors may not be generalizable to healthy individuals. The evaluation of interventions in healthy persons requires careful selection of outcomes due to the absence of health indicators and the low incidence of preventable conditions. Given the tremendous resources invested in such trials, development of consistent approaches to assessing the effectiveness of probiotics would be beneficial. Furthermore, the reporting, presentation and communication of results may also affect the validity of the scientific evidence obtained from a trial. This review outlines the challenges associated with the design, implementation, data analysis and interpretation of clinical trials in humans involving probiotics. Best practices related to their design are offered along with recommendations for enhanced collaboration to advance research in this emerging field.

Introduction and Scope

The seventh Annual Conference of the International Scientific Association for Probiotics and Prebiotics (ISAPP) was held from November 4–5, 2009 in Irvine, California. ISAPP is an international collaboration of scientists and representatives from industry who exchange views and discuss topics related to scientific application of probiotics and prebiotics. The discussions of the 16 member work group titled, “Parameters for Designing, Publishing and Communicating Human Studies on Human Probiotics” formed the basis of this review.

The growing interest in the application of probiotics to the maintenance and improvement of human health is evidenced by a recent National Library of Medicine database search using the keywords “probiotic, human, clinical trial” spanning the dates 1992 to 2010. Over 1,186 peer-reviewed publications identified; of which 338 were review articles. Due to the interest in clinical research studies in humans involving probiotics, the ISAPP working group set out to identify the challenges of designing, conducting and interpreting results from human probiotic studies and to develop strategies to serve as standards for future endeavors.

This review will discuss study design, target populations, selection of placebo and probiotic microorganism(s), duration of follow up, outcome and endpoint measurements, safety assessments and regulatory considerations. The issues associated with each of these topics and their contribution to clinical trials in probiotics in human participants will be detailed. Recommendations for optimizing clinical research involving probiotics will be addressed.

Study Design

Although other types of experimental and observational studies may contribute high quality scientific evidence to support claimed relationships between the use of probiotics and health and safety outcomes, well-performed randomized, controlled trials provide the strongest causal evidence. The latter are necessary to establish significant scientific agreement with the causal claims evaluated by government regulators. Considerations that influence key elements of the trial's design include the hypotheses and specific aims of the investigation, the resources available for the study and human participant concerns including the protection, safety, privacy and autonomy of study participants. Trials should be designed and conducted using sound scientific principles and methods to minimize bias in the assessment of intervention effects. Main sources of bias can arise from systematic differences between comparison groups, how treatments are randomized and allocated to trial participants, how care is delivered and how outcomes are assessed, as well as differences in study completion rates.¹ Well-designed, randomized-controlled trials use a variety of methods to minimize these biases, including randomization of subjects to study interventions, blinding of participants, caregivers and assessors and rigorous statistical comparisons between groups to estimate intervention effects.

Components that should be specified during the design phase of the study include: (1) the target population of interest, (2) the specific intervention(s) under study, (3) the control groups, (4) key safety and efficacy outcomes, including the designation of primary outcomes, (5) research hypotheses concerning the effects of experimental and comparison treatments on these outcomes and (6) the type and size of study for addressing hypotheses and producing evidence that generalizes to the target population.^2,3 Although on occasion, key safety and efficacy outcomes and research hypotheses are finalized before beginning the design phase, more often than not, these components are developed and refined iteratively during the design phase. A tentative specification of key study outcomes and hypotheses is followed by the development of a study design to address them.

Each of these components is then refined as necessary, given feasibility constraints. For example, if a researcher is interested in studying probiotics for preventing ulcerative colitis, a condition for which effective interventions exist, a placebo-controlled, superiority trial may not be ethical. However, an active-controlled non-inferiority trial that compares an investigational probiotic to previously recognized treatments would satisfy the principles of good clinical practices.⁴

Important caveats regarding study design pertain to the impact and validity of the outcome measurement(s). These should be selected based on their clinical relevance; if surrogate disease measures will be used, their relevance must be proven.⁵ To justify health claims, a specific measurable disease or health-related condition must be identified and assessed by measuring the incidence, associated morbidity or scientifically validated surrogate endpoints for the specified disease or condition.

Target population for study.

The results of a study should be generalizable to the consumers of the product. For study acceptance, The European Food Safety Authority (EFSA) requires that a study population be representative of the target population. Similarly, the United States Food and Drug Administration (FDA) requires that studies cited in support of health claims for foods be “scientifically appropriate” to extrapolate from the study sample to the population that is the subject of the health claim.⁶ While institutions and clinics with academic affiliations and mass media advertisements are popular locations and methods of recruitment, unless these sources provide the population of interest, the external validity of the study may be greatly reduced. As the claims for probiotic products are applied to broader populations, it is important to ensure that a variety of subjects participate in such trials to ensure validity. To address these issues, researchers may take advantage of “grassroots efforts” including community-networking, targeted media advertisements, individual community practitioners and practice-based research networks.

Comparison group.

The selection of a comparison group is a challenging issue in any study, especially when a probiotic is delivered in a food. When ethical and feasible, placebo agents provide an ideal comparison group. A placebo should be as similar to the active intervention as possible, with the exception of the presence of the probiotic under investigation. A report of a joint FAO/WHO expert consultation team recommends: “For the testing of probiotic foods the placebo would be comprised of the food carrier devoid of the test probiotic.”⁷ However, food carriers may be comprised of multiple ingredients, blurring the differentiation between the study product and the food carrier. For example, when testing efficacy of a yogurt supplemented with a probiotic, several different placebos could be imagined. The placebo might be acidified milk with no active cultures, heat-treated probiotic yogurt containing fermentation end products and bacterial cell components or yogurt without the probiotic but retaining live, active starter bacteria. The choice depends on the research question being asked. However, regardless of the placebo chosen, it is imperative that the rationale for the decision is clearly described in the study protocol.

Length of follow up.

There is no a priori preferred length of follow-up for subjects in clinical studies involving probiotics. The selection of an appropriate follow-up duration depends on the study question, outcome of interest and resources. For example, mortality outcomes require follow-up periods of many years, while morbidity outcomes such as duration of diarrhea may require follow-up for a few days. Typically, the power of a study for detecting important clinical effects on morbidity and mortality outcomes is enhanced by longer follow-up periods. If outcomes have seasonal patterns, (i.e., colds and influenza-like illness), follow-up for an entire year may be necessary. However, limited study resources and the urgency to complete studies as rapidly as possible argue for shorter follow-up periods, necessitating a suitable compromise that should be justified in study descriptions. One rationale is to extend follow up at least as long as persistence of the probiotic in vivo—typically less than four weeks.

Sample size.

A characteristic of a well-designed clinical trial is that it includes a sufficiently large sample size to provide a reliable assessment of study questions. Furthermore, it should have sufficient power to detect clinically significant effects vis a vis the primary outcome. A common misconception about sample size is that the larger the sample size, the better the study. It is more important that the study be appropriately powered at the design stage to detect realistic and meaningful effects on clinically relevant and reliably measured outcomes. The planned study design and analysis should account for the impact of loss to follow up and imperfect adherence on the effective sample size of the study. Two or three small studies with comparable conclusions may provide more robust data than one extremely large study, regardless of the statistical significance or p-value in the single large study. Additionally, effect size and thus sample size will be determined by the outcome of interest. For example, in a study of necrotizing enterocolitis (NEC), a low-cost and well-tolerated probiotic supplement may offer clinically meaningful effects, even if it only reduces the incidence of this serious condition by 4 or 5 percentage points. For detecting such small effects, larger sample sizes are required; however smaller sample sizes are often chosen due to financial constraints. This practice increases the chance that a smaller but clinically meaningful association will be missed, a Type II error.

The CONSORT standards of reporting trials.

To standardize the reporting of results of randomized controlled trials (RCTs), with specific emphasis on describing key determinants of evidentiary quality, a standardized 22-item checklist and flowchart was devised by leading medical journal editors. The Consolidated Standards of Reporting Trials (CONSORT) Statement facilitates assessment of the design, analysis and interpretation of trials by readers. A companion flow diagram enables readers to track the progress of all participants through the trial. The CONSORT checklist has been translated into 10 languages, in addition to English, and has become a de facto standard for reporting of clinical trials.^8,9

As CONSORT items reflect widely accepted markers of study quality, they serve as a helpful reference for investigators in the design, conduct, analysis and reporting of clinical trials. They further provide readers with information on patient flow, attrition and other sources of bias. For example, an enumerated item in the CONSORT statement concerns allocation concealment. Defined as the process of preventing foreknowledge of a potential enrollee's randomized treatment assignment, this blinding is crucial for minimizing bias in randomized trials. A clear statement that allocation concealment was used, is a widely used indicator of study quality for randomized controlled trials.^10,11 Similarly, intent-to-treat analyses offer protection against bias arising from selection effects and differential attrition. This approach, comparing study outcomes based on assigned treatment is preferred by many evaluators.^5,12,13

Registration of clinical trials.

The highest quality scientific evidence results when experiments are used to evaluate hypotheses that were declared prior to embarking on the experiment and when results of the study, whether positive or negative, are publically accessible. Concerns for the quality and availability of safety and efficacy data from industry-sponsored clinical trials led to the establishment of numerous online registries of clinical trials. Mandatory clinical trial registration is a standard of many regulatory bodies as well as journals and provides transparency for stakeholders. Additionally, these publically available databases of information may serve as a source for collaboration and partnership. To fulfill the objectives of good clinical practices, it is imperative that all trials be registered through a publicly accessible online clinical trial registry, accounting for all primary and secondary outcomes before subject enrollment.

Baseline Data

Randomization does not always guarantee equal distribution of patient characteristics across all groups. As a result, it is important to inspect data to assess for differences in the treatment groups on baseline (pretreatment) measures. After randomization, the groups can be compared to determine if there are important differences (e.g., randomization was not effective) between the groups. In general, this comparison is made by graphical and formal statistical methods to detect differences that may have occurred by chance. If major differences exist between the groups, specific techniques (e.g., stratification, multiple regression) can be used to mitigate these differences during the analysis stage. A clear statement of the intention to use such methods should be provided in the formal statistical analysis plan to protect against bias.²

Intervention

Identification and classification of therapeutic micro-organ isms.

A number of probiotic species and strains used in trials involving human participants with a variety of conditions have demonstrated efficacy.^14–20 However, a key concern is the lack of both standardization and microbial characterization in the manufacturing of products being tested. To examine this issue, several recent studies have assessed commercially available probiotic supplements and probiotic-enriched foods to evaluate the accuracy of label claims. Discrepancies between reported and actual bacterial numbers present,²¹ misidentification of starter cultures used for industrial-scale production,²² absence of claimed probiotic strains and presence of non-reported strains^21,23 have been described. Inaccurate declaration of identity generally results from a lack of rigor in using newly available molecular tools as standard practice. Given the reported discordance between product content claims and actual content, it is reasonable to question the composition and potency of products used in some clinical trials. This highlights the importance of careful product characterization and potency assessment and reporting in clinical trials. Furthermore, this situation may account for some reported conflicting results.

The most frequently used bacterial probiotic species include members of the Lactobacillus and Bifidobacterium. Historically, species identification relied on phenotypic attributes, such as metabolite production and fermentation capability, to identify and classify bacteria, particularly the Lactobacillaceae. The advent of genotypic characterization, largely based around sequence analysis of the 16S rRNA gene has led to reliable methods for species identification and for most species comprise the best approach to species determination.^24–27 This approach involves amplification of the target gene and phylogenetic assignment based on the sequence of specific variable regions within the gene. While many bacterial genes represent good targets for this approach, the 16S rRNA gene has been used most extensively and as a result the largest database of reference sequences are available for this genetic region. 16S rRNA sequence-based classification is a useful approach for identification of species for some organisms. In addition, probiotic efficacy is typically predicated on strain-specific characteristics and while a number of discriminatory loci have recently been described for Lactobacillus species-level differentiation.^28,29 Specific regions of the genome that distinguish bacteria at the strain level have not been determined. Therefore, identification of lactobacilli using 16S rRNA sequencing approaches exclusively likely will not provide the resolution necessary to discriminate specific probiotic strains. Polyphasic identification and classification³⁰ that utilizes a combination of genotypic and phenotypic approaches is currently the gold standard for accurate identification and classification of closely related probiotic species.³¹

More recently, PCR-based fingerprinting tools such as random amplified polymorphic DNA (RAPD), enterobacterial repetitive intergenic consensus (ERIC) and amplified fragment length polymorphism (AFLP) have been successfully used to type members of the Lactobacillaceae.^32–34 A unique RAPD PCR product has recently been shown to be specific to Lactobacillus rhamnosus GG, and has been used to design quantitative PCR primers specific to this strain.³⁵

To provide a more global analysis of strain capability, independent research groups have recently developed novel lactic acid bacteria-specific and bifidobacteria-specific microarrays to profile functional gene expression of these species using multiple genes involved in carbohydrate, pyruvate and amino acid metabolism, antibiotic resistance and bacteriocin production.^36–38 Such functional profiling under standardized conditions, may also provide a more comprehensive approach for identification and classification of probiotic strains. “Macroarray” approaches have also been designed to identify species using whole genome DNA-DNA hybridization. The advantage of this approach is that the genome from specific strains may be used as probes and that a large numbers of probes, generated from a variety of strains, may be used in parallel in a single assay.³⁹ These molecular approaches in development represent portable, rapid typing tools and provide the sensitivity and specificity necessary for accurate identification and classification of probiotic strain.

Probiotic viability.

A unique aspect of probiotics is their viable nature. Maintaining probiotics in a live and robust state during the course of a study is essential. Assessment of the viable count of the probiotic product used at initiation and conclusion of a study is necessary to be certain that the potency is maintained. For a long-term study covering several months or more, assessment at one or more mid-points is advisable. Reporting of actual viable count and target viable count of the probiotic test article is recommended.

Probiotic viability is a function of numerous factors, summarized in Figure 1.⁴⁰ Factors that potentially influence probiotic viability and physiological state include production parameters, product parameters and features of the host. The extent to which these factors influence probiotic physiology should be expected to be strain-specific. The amount of active probiotic organisms recovered when isolated from different sites or products will depend on methods used for enumeration.^40–42

Figure 1.

Factors that potentially impact probiotic viability and physiological state. Factors include production parameters, product parameters and features of the host. The extent to which these factors influence probiotic physiology should be expected to be strain-specific. The amount of active probiotic organisms recovered when isolated from different sites or products will depend on methods used for enumeration. Reprinted with permission from Sanders and Marco, 2010.

Probiotic administration, delivery and host localization.

Probiotic administration is typically via the oral,^43–45 rectal,^46,47 or vaginal^48,49 routes. Probiotic strains targeted at intestinal sites are often selected for their ability to withstand transit through the harsh environment of specific niches in the gastrointestinal tract. However, the materials chosen for microencapsulation of live microbial supplements can further enhance survival. Use of natural biodegradable polymers or those minimally engineered to enhance delivery of encapsulated microbial cells to the lower gastrointestinal tract, particularly the colon, are the focus of substantial current research.⁵⁰ While our understanding of the bacterial species that promote host health is dramatically increasing, parallel development of improved delivery systems represent a crucial component for efficacious probiotic therapeutics.

Successful transit of sufficient microbial numbers to target sites also appears to be a key aspect of probiotic efficacy. Several animal studies have demonstrated the localization of supplemented organisms at specific sites along the gastrointestinal tract, such as the Peyer's patches and in the mesenteric lymph nodes.^51,52 These studies also demonstrated that gastrointestinal colonization can be variable, even between isogenic animals,⁵³ a key finding given the added variability introduced when studying human populations.

Endpoint Selection

The characteristics of the trial endpoints will dictate to a large degree the design of a clinical trial. Several characteristics can be considered when selecting an endpoint. First, an endpoint should be feasible to measure, both technically and from a human participants' perspective. Second, an endpoint should be sensitive to the desired effect of the intervention being studied. Third, an endpoint should be a measure of important benefit to the population being studied and fourth, an endpoint should be an objective measurement.⁵⁴

Surrogate endpoints.

A surrogate endpoint is a marker that substitutes for a clinical endpoint. Often these markers are laboratory values or imaging results that are believed to be involved in the causal pathway of a particular condition. The use of surrogate endpoints is appealing as it may facilitate the conduct of studies with a smaller sample size and a shorter follow-up than would otherwise be possible. However, this approach has pitfalls. For example, interventions that decreased cardiac arrhythmias (the surrogate endpoint) were widely marketed under the assumption that they would also decrease mortality (clinical endpoint). However, post-marketing review showed that they actually increased mortality.⁵⁵ To avoid a situation such as this, a selected surrogate marker should fulfill stringent criteria. Firstly, a surrogate marker should have a good positive predictive value. For example, if a surrogate marker predicts that there is benefit to an intervention, it will have the same effect on a clinical endpoint. Conversely, if a surrogate marker predicts that an intervention is not beneficial, the intervention should also not have an effect on the clinical endpoint. Secondly, there should be an understanding of the role of the surrogate marker in the causal chain of events and this should be a validated measure in both the surrogate endpoint and the clinical endpoint.⁵⁶ The use of surrogate endpoints that do not meet these criteria in clinical trials should be avoided, as the scientific evidence generated might not be sufficiently correlated with the clinical endpoint.⁵⁷

Laboratory endpoints.

Endpoints measured in the laboratory can be of value for understanding physiological processes or for documenting potential safety parameters that may otherwise remain undetected. As alterations in laboratory values often precede clinical symptom onset, the former may serve as an early indicator of dysfunction.⁵⁸ The same considerations should be applied to the selection of laboratory and clinical endpoints; they should be feasible, sensitive, a measue of an important benefit, and objective. Other important considerations are the variability and timing of laboratory measurements. For example, since allergen-specific immunoglobulin E (IgE) has a half-life of 6 months to 2 years, it would not be a useful surrogate marker in a study with a follow up period of a shorter duration.

Participant reported endpoints.

Endpoints directly reported by the participant are a valuable contribution to a clinical trial as these subjective assessments may provide a measurement of the impact of an intervention on lifestyle. Participant-reported endpoints can be designed to target a relatively simple endpoint (e.g., headache), a more complex endpoint (e.g., ability to perform daily activities) or even an extremely complex endpoint (e.g., quality of life). Quality of life assessments may consist of multiple domains that consider the effects of the physical, emotional and social components, however quality of life should not be limited to solely an assessment of impairment or simply the presence of symptoms.⁵⁹ Several standardized and validated endpoints have been compiled into condition-specific instruments such as the IBS-QOL, specifically designed for a population with irritable bowel syndrome,⁶⁰ and more generic quality of life instruments such as the SF-36.⁵⁹ Instruments should be evaluated for reliability (consistent results with repeated testing), validity (ability to measure stated concepts) and ability to detect change (does score change when condition changes) before being applied to the clinical trial setting.⁶¹

Adherence

Subject adherence may be defined as the extent to which persons comply with or follow a prescribed regimen. Although clinical trials volunteers with underlying health conditions may be seen as motivated and compliant, data from clinical trials suggest that patient adherence rates typically range between 43–78%.⁶²

Trials that examine the efficacy or effectiveness of probiotic supplements generally stipulate routine consumption of the probiotic supplement by the study subject. The longer or more complicated the study regimen, the lower the likelihood of subject adherence.⁶³ The level of subject adherence may affect outcomes, as well as how clinical trial results are interpreted. Therefore, the assessment of adherence is an important component in the conduct of probiotic clinical trials in human participants, especially if the intended use of a probiotic is on a daily basis.

Clinicians may tend to over-estimate patient adherence and are unable to consistently identify which specific patients are non-adherent.⁶⁴ To address this issue, several methods have been developed to measure adherence. Each method has advantages and disadvantages as listed in Table 1.

Table 1.

Comparison of methods for measuring adherence in probiotic protocols

Method	Advantages	Disadvantages
Directly observed therapy (DOT)	Objective; tends to be the most accurate
	May be more feasible in an inpatient setting for short-term protocols	Expensive
		Easier to apply in an inpatient setting
Measurement of probiotic in stool	Objective	May unblind investigator
	Allows assessment probiotic exposure in “control” subjects	Only measures recent adherence
		Probiotic may still be detected with infrequent adherence
Electronic medication monitor		Expensive; requires downloading data and specific equipment
	Objective	Not applicable for patients who use separate pill boxes
	Precise	Data can be altered
Pill count	Objective	Can be easily altered
Patient questionnaire	Simple to use	Error secondary to recall or distortion by patient
Patient diary	Can assist patient with recall of adherence	Poor adherence
		Can be manipulated

(Modified from L. Osterberg et al., 2005)⁶²

Methods to assess adherence to probiotic protocols.

Directly observed therapy (DOT) involves witnessing a participant ingest a study product. Although this technique is the most accurate, its expense may make DOT impractical, especially in the outpatient setting. Detection of the probiotic supplement in the stool of clinical trial subjects has the advantage of being an objective measure of ingestion (for probiotic strains with this capability), as well as serving as a surrogate assessment for probiotic exposure of subjects who receive active product versus those who receive placebo. However, the measurement of probiotic organisms in the stool is a marker of probiotic organism survival, and does not imply activity or benefit. In addition to the cost, additional steps may need to be taken to ensure that assessment of survival in the stool does not unblind the investigators during the trial. One option might be to store stool samples and assess for probiotic organism content after unblinding has occurred to avoid this risk.

Pill counts have traditionally been used in clinical trials to measure adherence to dosing; however these assessments can be manipulated by subjects. Electronic medication monitors can offer additional data regarding how often patients open their medication container.⁶⁵ Subject questionnaires or symptom diaries can also be used to assess adherence. The most commonly used method includes patient self-report through surveys. Specific techniques to improve self-reported adherence includes ‘normalizing’ non-adherence, asking about adherence over a period of time, and using multiple questions to create an index (e.g., difficulty taking medication on time, frequency of missed doses and time since most recent missed dose).⁶⁵

Safety Assessments

When safety assessments are detailed, many studies neglect to report harms in a standardized and comprehensive way.⁶⁶ The extension of the CONSORT statement for harms provides guidance for reporting adverse events for RCT evaluations.⁶⁷ There are also established systems that allow a classification and grading of harms.⁶⁸ Despite the system used, adverse events should be classified according to operational definitions specified before the initiation of the trial.⁶⁹ Commonly, these include a scale of severity and a likelihood of association with the intervention based on time correlation and other potential causes. As specified in the CONSORT guidelines for the reporting of clinical trials, authors should provide estimates of the main adverse events, reasons for treatment discontinuation for each intervention group, and the methods and timing for assessing and recording safety information, including operational definitions.⁷⁰

The Agency for Healthcare Research and Quality (AHRQ) in conjunction with the Office of Dietary Supplements (ODS), the National Center for Complementary and Alternative Medicine (NCCAM) and the FDA has commissioned an evidence report on the safety of probiotics summarizing the existing evidence on health for human participants.⁷¹ Priority outcomes for these reviews are symptomatic, rather than intermediate outcomes; for example, harms that are important to decision makers and users of interventions.⁷² This report will include an evaluation of the overall body of evidence available for the key research questions pertaining the safety of probiotics. The greater need for reporting of safety information may be accomplished as journal editors provide accessible online space for reporting of safety assessments that are unable to be included in the body of a manuscript. The publication of the AHRQ report will greatly enhance the evidence base for safety, in concert with effectiveness evidence, for evaluation of human probiotic applications.

Using human trials to assess the safety of probiotics.

For most chemical substances, most of the burden of evaluating safety falls on tests performed on well-understood animal models. For the safety-related endpoints important in the assessment of probiotics, validated animal models do not exist and, as a result, the determination of safety rests primarily on human studies. Consequently, it would be of value to incorporate safety-related design elements and endpoints into the planning, the execution, and the reporting of human studies. Guidance is available from the guidelines developed by the Organisation for Economic Co-operation and Development (OECD) for oral toxicity studies on animals.

One key design element is the use of multiple doses—normally three levels in addition to at least one ‘zero-dose’ control group. There are numerous reasons why single-dose studies are rarely acceptable, but two key reasons are listed below. First, multiple doses are needed in order to elucidate the presence of dose-dependence and to characterize the dose/response relationship. Secondly, multiple doses assist in distinguishing real effects from random variability. If there is only one dose group, and the subjects in this group have a statistically significantly higher incidence of diarrhea, for example, is this an adverse effect of the probiotic or just chance? With multiple doses, it is possible to discern dose-dependence for adverse effects. If it is not present— that is, if the mid-dose group rather than the high-dose group has the highest incidence of diarrhea—then one can more confidently ascribe the effect to chance.

A second design element we can acquire from clinical studies is the need for complete characterization of the test article and the vehicle, including repeated testing throughout the study to assure that concentration, purity and potency remain “as advertised.”

Safety-related endpoints should be considered during the trial design stage. These might include hematology, clinical chemistry and immune parameters as well as clinical observations and self-reporting. Clinical observations should begin prior to dosing, should be frequent and regular, systematic and not haphazard, and that they should be included in appropriate detail in published reports of the study. Many human studies of probiotics do not mention adverse effects at all or they contain a meaningless observation that “the treatment was generally well tolerated” or perhaps that “no adverse effects were noted.” In order to be of any real value in evaluating safety, the efforts to observe any adverse effects must be determined during trial planning. Measures of the primary endpoint and the published report must describe these methods along with the results. In short, the safety component needs to be as thoroughly thought out, executed, analyzed and reported as the efficacy component.

Regulatory Considerations

Recently, regulatory issues are a focus for those involved in the development and marketing of probiotic products. The European Food Safety Authority (EFSA) recently determined that none of the claims for specific probiotic strains submitted to date were adequately substantiated by the scientific data that were provided as evidence of support.⁷³ Health Canada recently provided industry guidance on probiotic-containing foods⁷⁴ and is developing guidance for probiotic natural health products. The Indian Council of Medical Research is developing guidelines for India that would require probiotic strains to be backed by clinical trials—preferably conducted in local populations—if they are to be marketed.⁷⁵ Other activities are underway in countries across the globe to address regulatory issues pertaining to probiotics. Regulatory issues relevant to probiotics have been reviewed previously^76–78 and many of the details from these papers regarding allowable claims, target populations for claims, and safety standards are summarized in Table 2.

Table 2.

Distinctions among products to be used as drugs, conventional foods or dietary supplements¹

Product parameter	Drug	Conventional food	Dietary supplement
Investigational new drug application filed with FDA	Required	Not required	Not required
Study outcomes	Treatment, prevention, mitigation or cure of disease	Impact on normal structure or function of the human body; or reduction of the risk of disease1	Impact on normal structure or function of the human body; or reduction of the risk of disease2
Study populations	Healthy at-risk or patients	Healthy or healthy at-risk	Healthy or healthy at-risk
Clinical trial registration	Advisable	Advisable	Advisable
Acceptable claims for products	Drug claim (FDA approval required)	Health Claim (FDA approval required), which characterizes the relationship of the food to a disease or health-related condition; Structure Function Claim (no FDA approval required), which describes the mechanism of or the effect on the structure or function of the human body or general well-being	Health Claim (FDA approval required) Structure Function Claim (no FDA approval required)
Safety standard	Risk/benefit analysis	Reasonable certainty of no harm	Unsafe if significant or unreasonable risk of illness or injury under the conditions of the recommended use
Burden of proof of safety	Rests with manufacturer	FDA must prove product is unsafe	FDA must prove product is unsafe
Approach to safety assessment	Manufacturer must submit evidence of safety to FDA	GRAS process (with or without FDA notification) commonly used	GRAS or New Dietary Ingredient process
Route of administration	No restrictions	Orally consumed	Orally consumed

¹Medical foods are not considered here as these products are specially formulated and intended for the specific dietary management of a disease condition for which distinctive nutritional requirements are medically established and intended for use under medical supervision. This is not the case for probiotics. Medical foods are not intended to cure, treat, mitigate or prevent the underlying disease condition.

²FDA currently views the appropriate target to be chronic, diet-related disease, not acute disease.

Although the focus of this review is on the design and conduct of clinical trials on probiotics, mindfulness of the regulatory framework is important at the outset and throughout development of a probiotic product—from concept to market. Regulatory issues prior to commercialization pertain to product characterization, safety and efficacy. Once commercialized, regulatory impact expands to include issues such as good manufacturing practices, product labeling, advertising and post-market surveillance. This discussion will be limited to regulatory impact on design of human studies for safety and efficacy.

Proper and thorough characterization of the probiotic to be commercialized is essential. It is worth noting that the vast majority of probiotic claims evaluated by EFSA were rejected based on improper product characterization. With the availability of inexpensive genomic sequencing services, that complete sequencing and annotation of commercial strains is a reasonable and advisable expectation prior to probiotic commercialization.

Perhaps least appreciated is that the design of human studies involving probiotics should reflect the intended path to commercialization of the product. The intended use of the probiotic (drug, food, dietary supplement) will impose different requirements for filings with the FDA, study populations and study outcomes. Determination of the regulatory category is dependent on the intent for use, according to the FDA. Furthermore, use as indicated in the stated outcomes for a study dictates the product category. A product studied for its ability to treat, prevent, mitigate or cure a disease is a drug, even if the study product is in the form of a food.

One consequence of this situation is that products marketed for the treatment, prevention, mitigation or cure of a disease (drug use) may trigger a request for an Investigational New Drug (IND) application to be filed with the FDA prior to commencing with human research. In fact, although the probiotic products currently on the United States market are in the form of foods or dietary supplements, many of the studies documenting health effects have evaluated endpoints that would classify them as drugs. For example, treatment of symptoms of irritable bowel syndrome, prevention of antibiotic associated diarrhea, treatment of acute diarrhea in children and extension of remission of pouchitis would all be considered drug trials by the FDA. In addition, a researcher must consider the value of research on a drug endpoint in an effort to substantiate an efficacy claim for a food or dietary supplement. For example, research that targets a population with a medical condition will likely not be relevant to a healthy population. To complicate matters further, in the United States, if the probiotic under study has not previously been sold as a food or dietary supplement ingredient, conducting clinical trials under an IND may permanently bar the studied probiotic from ever being used in foods or as a dietary supplement. Evaluation of a probiotic under an IND would label it as a biotheraputic, and limit other intended uses.

Additional considerations regarding regulatory impact on the design of clinical trials include the fact that regulatory authorities will evaluate the totality of evidence, published and unpublished. It is worth considering that an underpowered study that fails to show a relationship between the probiotic and a health endpoint will contribute to the conclusion that the evidence is inconsistent and therefore not convincing. It may be better not to conduct an underpowered study. Furthermore, it is preferable to match the product dose and delivery format to what is anticipated to be marketed. Proof-of-concept studies early in the efficacy assessment may use higher doses or laboratory-scale product formulations. If so, confirmatory studies should be conducted validating the efficacious dose and delivery method intending to be marketed.

In addition to realistic doses, dosing schedules and carriers, evidence to substantiate an efficacy claim must be obtained using endpoints and target populations stipulated or implied by the claim. Requirements will differ for different categories of claims. To substantiate a drug claim, evidence must causally support the value of the drug to cure, treat, prevent or mitigate disease. The target population might be currently healthy, at-risk or diseased. FDA interprets a health claim for a food or dietary supplement to be limited to reduction in the risk of incurring a diet-related, chronic disease or condition by the currently healthy population. The target population must be the currently healthy or at-risk population or a subgroup from which results can be extrapolated. The endpoint must be the target disease itself or a biological endpoint accepted as a valid marker of development or manifestation of the disease.⁷⁹ Many chronic diseases take years or decades to develop. Further, even common chronic diseases are relatively rare events requiring extremely large study populations in order to have a high probability of accruing adequate numbers of individuals with the disease. This combination of factors may place the costs of an adequately powered trial beyond reach. Trials with a relatively quick-response, valid biomarker are more plausible. Unfortunately, there are few biomarkers universally accepted as valid indicators, and those that exist (e.g., LDL cholesterol as a marker for coronary heart disease, bone density for osteoporosis) are not likely endpoints for probiotic effects. An efficient application of probiotic research funds would be a concerted effort to validate markers of disease endpoints that would be amenable to probiotic risk reduction.

To substantiate a structure/function claim, the evidence must support the role of a nutrient or dietary ingredient to affect the structure or function of some aspect of human physiology or metabolism, characterize a documented mechanism by which a nutrient or dietary ingredient acts to maintain such structure or function or describe general well-being from consumption of a nutrient or dietary ingredient. For example, the FDA has stated that use of a probiotic to help maintain intestinal microbial balance during antibiotic therapy is intended to produce a drug effect, not to affect the structure or function. The FDA has provided guidance on types of evidence it considers appropriate for structure/function claims⁸⁰ and health claims.⁶

Conclusions

This review highlights some of the challenges associated with the design, implementation, data analysis and interpretation of clinical trials in humans involving probiotics. From these challenges, best practices to address these challenges are suggested. Consideration of outcome measurements and assessment of safety issues are paramount to conducting reproducible assessments. Ensuring identity and potency of probiotic products utilizing molecular techniques is vital to ensure that the product(s) being evaluated are the intended ones. Even if sophisticated molecular techniques are not available to researchers, detailed description(s) of the probiotic genus, species and strain designation(s) provides key information. Issues of effectiveness including adherence and generalizability should be accounted for in the trial planning stage. Finally, the concerns of regulatory agencies should be considered in the trial design and conception stage. Enhanced collaboration among clinicians, microbiologists and representatives from industry and regulatory agencies is vital to the conduct of quality clinical trials, to advance research in the field, and to improve human health. The use of international standards for probiotic trials on human health may facilitate the comparison of results from different probiotic products. In addition, common standards can allow meta-analyses and systematic reviews to augment the power of such studies. Given the costs of large, well-done clinical trials, it is important to leverage the resources used for clinical trials to ensure their widest application to current therapy. In addition, the establishment of an international human probiotic clinical trials network would facilitate such discussion as well as bring together a group of individuals who could collaborate to design and conduct quality human clinical trials with probiotics.

Abbreviations

AFLP

amplified fragment length polymorphism

AHRQ

Agency for Healthcare Research and Quality

CONSORT

consolidated standards of reporting trials

DOT

directly observed therapy

EFSA

European Food Safety Authority

ERIC

enterobacterial repetitive intergenic consensus

FDA

Food and Drug Administration

IBS-QOL

irritable bowel syndrome-quality of life

IgE

immunoglobulin E

ILSI

international life sciences institute

IND

investigational new drug

ISAPP

International Scientific Association for Probiotics and Prebiotics

NCCAM

National Center for Complementary and Alternative Medicine

NEC

necrotizing enterocolitis

NIH

National Institutes of Health

ODS

Office of Dietary Supplements

OECD

Organisation for Economic Co-operation and Development

PCR

polymerase chain reaction

RAPD

random amplified polymorphic DNA

RCT

randomized controlled trial

SF-36

short form 36