Improving Natural Product Research Translation: from Source to Clinical Trial

Abstract

While great interest in health effects of natural product (NP) including dietary supplements and foods persists, promising preclinical NP research is not consistently translating into actionable clinical trial (CT) outcomes. Generally considered the gold standard for assessing safety and efficacy, CTs, especially phase III CTs, are costly and require rigorous planning to optimize the value of the information obtained. More effective bridging from NP research to CT was the goal of a September, 2018 transdisciplinary workshop. Participants emphasized that replicability and likelihood of successful translation depend on rigor in experimental design, interpretation, and reporting across the continuum of NP research. Discussions spanned good practices for: NP characterization and quality control; use and interpretation of models (computational through in vivo) with strong clinical predictive validity; controls for experimental artefacts, especially for in vitro interrogation of bioactivity and mechanisms of action; rigorous assessment and interpretation of prior research; transparency in all reporting; and prioritization of research questions. NPCTs prioritized based on rigorous, convergent supporting data and current public health needs are most likely to be informative and ultimately affect public health. Thoughtful, coordinated implementation of these practices should enhance the knowledge gained from future NP research.

INTRODUCTION

Background

Research on health effects of natural products (NPs) has often stemmed from traditional use, or from evidence that higher consumption of plant foods is associated with better health outcomes (see (1) and refs. therein). This research has resulted in the successful development of many active ingredient-based pharmaceuticals from NPs. In contrast to a typical drug development pathway, studies based on traditional uses or epidemiological data rarely begin with a precise understanding of the active molecules or their molecular mechanisms of action. A growing number of recent efforts to elucidate the mechanisms of action of traditionally used NPs have reported finding a complex mixture of bioactive constituents that act at multiple targets (2–5). Perhaps partly as a result of this still incompletely understood complexity, to date many NIH-supported clinical trials (CTs)¹ of complex NPs (NPCTs), in particular the majority of large, randomized, controlled clinical trials (RCTs), have failed to reject the “null hypothesis,” i.e., they detected no significant difference between the intervention and negative controls (6–8). While CTs that fail to reject their null hypothesis can still provide invaluable guidance, the results of such trials are often met with concerns that different design choices – whether of product, dose, timing, participant eligibility criteria, outcomes, etc. – might have yielded a substantially different result. Each interventional trial represents a substantial investment, with well-powered Phase III CTs typically consuming more than $10 million and many person-years (9, 10). Additionally, as time and funds are limited, and participant pools may also be constrained, a decision to invest in a CT often precludes pursuing other research questions (11), which may provide more useful public health-relevant knowledge than an RCT with an insufficient evidence base.

Definitions

Natural products

NPs were defined as including materials such as animals, marine organisms, plants/botanicals/herbals, macroscopic fungi, bacteria, probiotics, and minerals, and materials derived from them, including isolates, extracts, vitamins, and amino acids. NPs may vary in chemical complexity, ranging from crude to extensively purified materials, and were considered to include dietary supplements, certain foods, traditional medicines, and other products derived from multiple natural sources.

Complex NPs

NP preparations which retain a large number of chemical constituents whose relative concentrations and chemistry (metabolome composition) reflect the inherent variability of the sources, as opposed to products consisting of a very limited number of highly purified NP-derived constituents.

Challenges to be addressed

For investigation of traditionally used NP, research design may be complicated by the chemical complexity and inherent variability of the NPs (12–14). The often incomplete and/or conflicting information on mechanisms of action, appropriate uses, and expected health effects poses challenges for formulating hypotheses and designing CT. The ready availability of a variety of preparations of similar NPs and their occurrence in complex matrices such as foods or dietary supplements further increases the challenges for the design, conduct, and interpretation of NPCTs (15–17). The variety of NPs in the marketplace poses challenges for product selection; in addition, participants must be monitored for inadvertent or otherwise unreported exposure to the intervention or components thereof (drop-ins). Here, consistent with the NIH definition, we use the term CT not only for efficacy trials, but also for any study in humans which assesses NP safety, bioavailability, PK, PD, dose-response, mechanisms of action, and biological markers thereof.

Focus and objectives

To discuss good practices for addressing these challenges, a two-day, public workshop on September 13 and 14, 2018, brought together speakers and attendees across disciplines and from academia, government, industry, and private practice at the NIH campus in Bethesda, Maryland. Past efforts to enhance rigor in NP research include the NCCIH Policy on Natural Product Integrity (18) as well as numerous guidance documents addressing identification, reproducibility, study design, and execution, as well as conflict of interest issues (6, 19–27). As was done for a recent workshop focused on fatty acids (28), we assemble here relevant prior publications citing critical or state-of-the-art approaches drawn from a breadth of NP research and related disciplines, and address in greater depth NP- related topics that were specific foci of the workshop.

Reproducibility, replicability, and other critical characteristics of a solid evidential foundation, have been variously defined in different sub-disciplines (29, 30). Consistent with the 2019 NASEM Report on Reproducibility and Replicability in Science (31), this article uses the term reproducibility for “obtaining consistent results using the same input data, computational steps, methods, and code, and conditions of analysis,” and the term replicability for “obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data.”

The workshop was predicated on the premise that adherence to a well-integrated set of good practices along the continuum from ethnobotany and epidemiology through basic biomedical research to CTs should enhance the effectiveness of translation from pre-clinical research and early phase CTs to information usable by consumers, manufacturers, practitioners, research funders, and other stakeholders. The good practices should also help reduce premature initiation of Phase III NPCTs, thus reducing instances producing strong grounds for post-trial concerns about the evidence base for a given trial. This article summarizes and updates the workshop discussions of such practices, which include those that apply to individual sub-disciplines, as well as productive combination of approaches across disciplines. The integration of all of these practices, along with the application of sound inductive reasoning, rigorous, appropriate, and transparent statistical analyses (Table 1) (32–36), as well as targeting an effect size that is significant to clinicians and consumers (37) should increase the replicability and translational relevance of basic and pre-clinical NP research. Ultimately, this will serve to limit resources expended on flawed premises producing irreplicable data (38–40), and to ensure optimal knowledge gained from every NPCT, regardless of whether the results are consistent with benefit, harm, or absence of an effect on the assessed outcome(s).

TABLE 1.

Highlights of broadly applicable good practices for enhanced replicability, transparency, and translational relevance of NP research

1. Models, by definition, differ from the actual subject of interest, however, when carefully chosen to address a clearly defined experimental question they can provide critical insights (236). Sound inductive reasoning, including careful consideration of differences between the model and the actual subject of interest, must be used in interpreting model system outcomes, for example, response in an experimental model may demonstrate a biologically relevant activity, but does not imply clinical efficacy.

2. Quality control. On-going, discipline-appropriate quality control is critical for all experimental assays and research components, from NP, through cell lines, microphysiological systems (58), in silico and whole-organism models, to CT procedures.

3. Replication is critical: within and across distinct model systems, with methods based on orthogonal principles, and in multiple labs. Results that replicate across models, methods, labs, are more likely to replicate in a CT.

4. Rigor. The same rigorous practices used in CT to avoid bias and enhance replicability should be applied in hypothesis-testing experiments that may be used in designing a CT. This includes randomization of wells/dishes/animals, masking/blinding of study personnel, and sample size calculations. Exploratory studies merit equally complete and transparent reporting but may accommodate less stringent procedures.

5. Reporting. Use approaches and venues that support more complete, transparent, and cumulative reporting.

a. Registration: use registered reports and pre-publication registries for preclinical in vivo studies as well as CT, e.g., clinicaltrials.gov and preclinicaltrials.eu. Information needed to replicate research on complex NPs may go substantially beyond the intervention characterization required by these sites, see, e.g., nccih.nih.gov/research/policies/naturalproduct.htm

b. Outlet: publish in journals that do not limit supplemental information.

c. Publish exploratory studies (without forcing them into hypothesis-testing frameworks) and null results of hypothesis-testing studies.

d. Openness: curate key information for scientific research in publicly accessible databases, e.g., curatescience.org.

This review summarizes areas of both consensus and contention that emerged from the workshop discussions. While the workshop focused on approaches to improve the knowledge gained from future randomized, controlled, NPCTs, some attendees questioned the appropriateness of and need for such trials, especially for traditionally used botanical preparations, arguing that beneficial effects of such practices cannot be detected in the context of a conventional CT.

Moving from a narrower to a broader and more applied focus, this review begins with good practices for the application of assays used to elucidate NP bioactive constituents, and continues through the specification and characterization of NP interventions, endogenous and exogenous factors that may influence the translational relevance of preclinical in vivo models, and the strengths and weaknesses of a broad range of state-of-the-art models. Subsequently, the review will address additional factors that influence the likelihood of successful translation to humans, including assessment of the supporting research, and concludes with approaches for evidence-based prioritization of this research. The review is organized into four main sections:

• Good practices for preclinical NP studies

• NP characteristics critical for replicability

• Translational relevance of preclinical NP research

• Good practices for moving to NPCTs

Figure 1 provides a graphical overview of the relationships among the covered topics, and Table 1 highlights broadly applicable good practices. The subsequent tables highlight good practices from each of the four main sections:

Figure 1. Critical information for the design, implementation and interpretation of natural product clinical trials.

Elucidating details (light bulbs) of molecular mechanisms of action prior to initiating a NPCT can help “de-risk” the planned trial by both providing evidence for the hypothesized outcome and its underlying causes, and supporting the development of methods to test that hypothesis, e.g., by measuring in vivo target engagement. It is good practice for researchers to establish standard procedures for product preparation (A), that are optimized for bioactivities and stability, and methods, preferably validated, for thorough characterization of product chemistry and stability. Elucidation of the bioactive NP constituents or metabolites (B) is critical for quality control, for assessing bioavailability, PK and PD, and determining whether minimal effective concentrations of bioactives have been achieved in plasma or at the targets (C). These data can support the development of methods to test mechanistic hypotheses, e.g., by measuring in vivo engagement of the putative targets, proximate bioactivities, or PD, and/or assessing the relationships between these and (D) the primary and secondary NPCT outcomes, and any adverse events.

• Table 2 highlights good practices for the use of in vitro and in silico models,

• Table 3 highlights good practices for NP selection, formulation and characterization,

• Table 4 highlights good practices for the use of preclinical in vivo models,

• Table 5 highlights critical considerations in NPCT design,

• Table 6 highlights issues to consider in NPCT decision-making.

GOOD PRACTICES FOR PRECLINICAL NP STUDIES

Optimizing the use of in vitro, invertebrate, and in silico models and ethnobotanical approaches

Good practices in model systems (in vitro, invertebrate, in silico) and in translating from traditional medical systems² to clinical research can enhance the replicability and translational relevance of preclinical NP studies. To date, the application of invertebrate model systems and some newer in vitro models has been very limited in NP research.

The measurement of activity in in vitro assays is a typical starting point for research on potential clinical application(s) of a NP. Good practices in proceeding from other starting points are discussed below. In vitro assays are also commonly used for monitoring the stability and potency of NPs, as well as for identifying fractions or components responsible for the assayed bioactivities. The replicability of these assays depends on many inputs, beginning with the test article, which often is a chemically complex NP. Good practices to enhance the replicability of vertebrate cell culture-based assays have been described elsewhere (38, 39, 41), as have recommendations for the use of antibodies (42). The following, therefore, focuses on issues more specific to the use of NPs in cell-based assays, and on approaches to enhance the translational relevance of those assays.

Data generated by high-throughput screens (HTS) and in vitro bioactivity assays pose many challenges for interpretation, importantly including their high potential for false discovery. Small molecule NP constituents, their spontaneous breakdown products and/or metabolites (43, 44) have significant potential to generate in vitro assay artefacts via aggregation (45), fluorescence, light absorption, metal chelation, plasma or intracellular membrane disruption (46, 47), protein denaturation, and/or specific and non-specific reactivity with biological nucleophiles (Table 2). For example, HTS and other in vitro studies of extracts and other preparations derived from Curcuma longa (turmeric; often improperly designated as “curcumin”) are susceptible to confounding due to the low stability (44) and high reactivity of the curcuminoids, as well as the frequent lack of clarity as to the actual composition of a given preparation (48, 49). Workshop speakers emphasized the need to design and conduct in vitro experiments in ways that mitigate and/or reveal such artefacts (41, 50, 51), e.g., by time course and chemical analyses, as well as by comprehensive and transparent reporting of experimental parameters.

TABLE 2.

Highlights of good practices for the translational relevance of in vitro models assessing NP bioactivities and mechanisms of action

1. Artefacts. When using in vitro assays of NP activity, test for common types of NP assay interference, e.g., fluorescence or absorption, membrane disruption, protein denaturation, and specific and non-specific reactivity with nucleophiles (48).

2. Bioavailability. If in vitro activity is observed only at concentrations substantially higher than those achievable in vivo in the tissue of interest, then the probability of seeing a robust clinical effect of consuming the NP is likely insufficient to form the basis for a NPCT.

3. Non-specific molecular mechanism(s) of action. Include controls for non-classical mechanisms of action such as metal chelation (87), modulation of DNA modifying enzymes, or intercalation into plasma or subcellular membranes (46, 87).

4. Biomimetic in vitro models. Consider pros and cons of more morphologically detailed “3D” models using human-derived cells. Organoids, “organs-on-a-chip” (OoC), and multi-organ biomimetic models (e.g., human-derived intestine, liver, and kidney OoC coupled by appropriate microfluidics) can yield more translationally relevant high-throughput information than monolayer cell cultures, and are a powerful complement to preclinical in vivo models.

In vitro models

In vitro approaches leverage simplified models of more complex biological systems in the interests of higher throughput and increased ability to decipher causal mechanisms (Table 1). In vitro models that better reflect the biology of an intact organism have been developed, taking advantage of advances including the use of primary, patient-derived cells and stem cells, direct differentiation of induced pluripotent stem cells, more precise methods of gene replacement, and identification and quantification of heterogeneity in vivo and in vitro. In vitro models are also increasingly leveraging advances in tissue engineering, microfluidics, and real-time biosensors of physiological processes. These in vitro models range in the extent to which they mimic in vivo conditions from relatively simple spheroids (52), through patient tissue-derived organoids, self-organizing stem-cell-based organoids (52, 53), single “organs on chips” (OoC) under flow, and integrated, multi-OoC microphysiology systems (MPS)(54–60). While still relatively high-throughput, these models from both healthy and diseased tissues have greater similarity to the chemistry, three-dimensional structure and component localization, environment, and possibly genetic stability of an intact organism than classical cell cultures grown on relatively flat substrates. Moreover, through the use of human-derived cells, these models may partially address the challenge of species differences in responses (Table 2) (61–64). MPS also allow targeted assessment of cell-type- or organ-specific dose responses (Table 4). Some major challenges in the use of MPS and other complex in vitro systems have recently been addressed, including the development and implementation of metrics for replicability and approaches to prevent the artefactual binding of hydrophobic molecules to the devices (59, 65). Models incorporating elements of the human immune system are beginning to be developed (66). However, understanding how to translate results from in vitro systems such as MPS to CT, e.g., into calculated dose ranges remains challenging.

Only those effects observed in vitro at concentrations achievable in vivo following ingestion, or in systems that incorporate upstream components modeling the functions of the gastrointestinal tract (67), are likely to be clinically relevant to oral dosing. Activities observed only at higher concentrations than can be obtained following ingestion should not be over-interpreted in planning CTs of ingested products (Table 2). Notably, many NP constituents show very limited oral bioavailability. Ex vivo models may be used for extensive pharmacodynamic characterization of NP, but they do not fully recapitulate human physiology. For instance, a promising NP candidate in an in vitro cell screening assay may be metabolized by the liver into an ineffective or even toxic byproduct (68). In vivo validation of in vitro results remains a key component of pre-clinical studies.

Speakers described several approaches to manage the sometimes limited or conflicting information on NP biological activities and their underlying mechanisms, including determining which NP components and cellular and molecular targets are required for activity in animals. The application to chemically complex NPs of agnostic (or non-targeted) in vitro assays (69) combining cutting-edge computational approaches, optimized mass spectrometric (MS) data collection and analysis to elucidate chemical composition, and cytological profiling (70, 71) now enables the generation of strong and specific hypotheses regarding bioactive NP components and the mechanisms through which they affect gene expression or cell phenotype (72–76). Such systems have the ability to interrogate multiple chemical components and their combinations simultaneously, without purification of individual constituents. This ability is critical, given accumulating evidence (consistent with tradition) that multiple chemical constituents in even a single botanical extract may contribute to bioactivity via distinct or similar molecular mechanisms (77, 78). These may include effects by constituents on each others’ solubility (or other aspects of bioavailability) or toxicity (5, 79). These novel, non-targeted platforms also have the advantage of generating strong, testable mechanism of action hypotheses without requiring assumptions or prior knowledge regarding those mechanisms.

Invertebrate in vivo models

Invertebrate models such as Caenorhabditis elegans (80, 81) and Drosophila melanogaster are popular in a variety of research areas for their ability to provide relatively high throughput, but in vivo platforms. The conservation of many human genes and signaling networks, combined with the small size and relatively short lifespans of these models facilitate the cost-effective evaluation of a variety of phenotypes (e.g., reproduction, locomotion, cognition) in a large number of individuals (Table 4) (80). This provides the ability to study phenotypes that are challenging to assess in most vertebrate animal models, such as healthspan (the part of an organism’s lifespan during which it is healthy) (82, 83). Invertebrate models can also be used in more focused interventional and mechanistic studies that explore specific aspects of an established phenotype (84–86). For example, tea polyphenols were reported to modulate both stress responses and fertility in D. melanogaster strains (87) via a non-target-ligand mechanism of action (Table 2), by modulating iron homeostasis. Recent advances in omics technology and the use of outbred populations further expand opportunities for using D. melanogaster as a model system for NP pre-clinical studies.

In silico models

When information on NP kinetics, dynamics, bioavailability, clearance, and pharmacologic activities is limited, one useful approach is to apply computational or in silico methods to leverage information from products with known chemistry, mechanisms of action, and activities. Data from more highly characterized products can be integrated into dynamic models, and in vitro to in vivo extrapolation (IVIVE) can be used to predict biological responses (88–92). These approaches, as outlined by the Tox21 Federal collaboration and the U.S. Environmental Protection Agency’s ToxCast program (93–96), can be used to focus high-throughput in vitro and in vivo testing on compounds and concentrations that are more likely to be biologically active and appropriate for each model (97, 98). Such integrated approaches could be further developed to probabilistically model the pharmacological and toxicological kinetics and dynamics of complex NPs (99).

Speakers described several critical, often broadly applicable considerations for utilizing in silico models. First, the purpose of the model must be clearly defined, and the model appropriate and sufficient to fulfill the purpose. Second, as for any research, if experiments that provide the input data are not appropriately designed in the beginning, no amount of computational modeling will yield translatable information. Third, variability and uncertainty in the data that inform the model, as well as correlations between parameters must be considered. Uncertainty and variability in model outcomes are acceptable but must be quantified, and in optimized models they should be similar to real world variability. Preliminary models may be continuously refined, using a “learn and confirm” strategy, or as understanding, e.g., of NP pharmacology increases. Fourth, end users must understand the computational (or any other) model’s limitations, the extent to which it is similar or dissimilar to the biological system of interest, and how the outputs compare with population-derived data, other real-world data, or with data from other models. Finally, computational models and the data used in their development must be made findable, freely accessible, interoperable and reusable (FAIR) (100), just as information for in vitro and preclinical in vivo models must be findable and sufficiently transparent to support replication.

Ethnobotanical approaches

Another typical starting point for translational research is traditional NP use, particularly the use of plants in traditional medical systems. When basing NP research on traditional use, it is essential to consider and thoroughly document how the NP is sourced, identified, prepared, and used in the traditional contexts (Table 3). Intra-species variation and growing conditions strongly influence the metabolic profile of a plant and differences in harvest and preparation will also alter critical product characteristics (101, 102). Many different uses may be described for a given species, and traditional healers have frequently been reported to use only specific subtypes of a given species. Awareness of such practices has the potential to be highly informative for the identification of bioactive components (103). Analogous considerations apply to fungal and animal sources of NP. Good practices for reporting botanical NP sourcing, identification, characterization, vouchering and archiving have been previously described elsewhere (12, 18, 19, 104–106). Comprehensiveness and transparency in reports of biomedical research on traditionally used plants have increased in recent years (107). There are cases, however, where, the available records predate the availability of geolocation data on growing sites, or where the need to protect ecologically or culturally sensitive information may limit what can be published (108, 109). Where a similar use or outcome is reported for the same or closely related species across cultures and geographical regions, those similarities may provide priority clues for translational research.

Translation from traditional to modern practice may be limited by important differences (110) (Table 3). Traditional practices commonly combine multiple NP sources (e.g., multi-botanical) into one intervention and are often associated with specific concurrent behaviors (e.g., rituals, behavior changes, possibly including dietary changes). Moreover, if documented traditional practices and the NP outcomes to be studied differ substantially, it may be inappropriate to claim the traditional or typical use as supporting data; the differences between modern medical diagnoses and practices and traditional principles of diagnosis and practice may be too divergent to base claims for modern application on traditional use. Moreover, where a planned NPCT is expected to use a highly processed (single) botanical, information from traditional use may also lack relevance, as the bioactivity and/or the ADME properties of the highly processed product may differ substantially from those of a (multi-component) traditional preparation. Where the NP and the study population are close to historical use and users, traditional practice may provide initial parameters for dose range to be used in research. In instances where no clear typical usage exists, for example with green tea extract capsules, speakers recommended that preclinical studies prioritize the determination of a therapeutic window and administer the lowest dose to achieve concentrations required for the biological and/or health effect of interest (9).

TABLE 3.

Highlights of good practices for NP integrity including standardization, formulation, characterization, and quality control

1. Quality control. Identity and purity of all materials used, regardless of the supplier, should be verified (independent of supplier) prior to use. Appropriate storage and handling conditions should be maintained and monitored by stability testing at least throughout the duration of the study (consult, e.g., the NCCIH Natural Product Integrity Policy (18) and the NSF Stability Testing Guidelines for Dietary Supplements). To the extent feasible, comprehensive chemical analysis of product should be performed; minor product components or impurities can be responsible for or affect measured activities.

2. Product specifications:

a. Chemistry. Identification of product components required for the bioactivities studied provides critical information for studies of bioavailability, pharmacokinetics (PK), and pharmacodynamics (PD) and should enhance the replicability and clinical relevance.

b. Formulation. Consider the effects of NP formulation and matrix (e.g., capsule, liquid, food) on bio-accessibility, bioavailability, PK, PD, and activity. These may differ significantly between an isolated NP constituent and the same compound in the context of a complex food matrix.

c. Sensory characteristics. Sensory acceptability must be considered for all in vivo research, and effectiveness of masking of placebos or active comparators must be considered in CT.

3. Translation from ethnomedicine. If basing an intervention on ethnobotanical observations, consider replicating traditional practice as closely as is feasible. E.g., highly processed extracts will likely have different activities than a multi-botanical decoction (see 2a and 2b). Contextual differences (environment, behavior, genetics) may also affect the outcomes.

NP CHARACTERISTICS CRITICAL FOR REPLICABILITY

NP specifications and integrity

The ultimate biological effect of an ingested, complex NP results from a dynamic interplay of factors: NP chemical composition, physical characteristics (111, 112), bioaccessibility, bioavailability, metabolism, and target population characteristics including sex, age, genetics, background diet, and other environmental exposures (113). This subsection begins by summarizing issues inherent to the integrity of the NP intervention and subsequently addresses target-related sources of variability.

Replicability of outcomes depends on correct and clearly reported identification and archiving of source materials (12–14, 17) and consistent preparation and activity of the NP (17, 22, 26, 114). Some attendees felt that since the effects of complex NP preparations may not be due to only the identifiable components, transparent reporting of methods, including consistent product preparation and quality control via an in vitro assay relevant to bioactivity, is sufficient for replicable NP research. Others argued that in many research context it is also critical to understand which NP components are responsible for the bioactivities of interest, as this information can inform NP quality control, as well as being utilized in assessments of bioavailability, pharmacokinetics and to inform comparisons of different studies (115) (Table 4).

TABLE 4.

Highlights of good practices for preclinical in vivo research on NP

1. Product translational relevance and replicability:

a. Equivalent product formulation. NPs used in preclinical studies should use similar or analogous formulation(s) to the intended human studies.

b. Biotransformation. Track the actual bioactives, which may be generated from precursors in the NP by the host or the host’s gut microbiota. Either or both may transform NP constituents to active or inactive compounds. These transformations may differ between models and humans, and amongst humans, and affect replicability of results.

c. Dose-ranging. Doses used in models should be comparable to anticipated human dosage. Initial dose ranges may be based on traditional use or MPS dose-response studies, derived using PBPK modeling, or from preclinical in vivo studies using appropriate scaling approaches. A clinical study is still required to establish dose-response, assess safety, and determine PK and PD in the target population.

2. Model translational relevance:

a. Model selection. Consider the translational relevance for the specific research question of different species and strains, humanized, outbred, wildling, and/or genetically modified models. Heterogeneity in the model organisms may contribute to the ability to predict responders vs. non-responders.

b. Documentation. Document as much as possible. Endogenous (e.g., age, sex, physiological/health status, genetic), and external factors (e.g., environment, including background diet, exposure history, enclosure size and enrichment, circadian timing, season, duration and timing of exposure relative to outcome assessment, handlers and handling methods) may affect outcomes. Make these records findable, accessible, interoperable, reusable (FAIR) (100).

3. Validated methods for de-risking the NPCT (see also Fig. 1): Develop and use validated assays to assess achievement of go/no-go criteria based on as many components of the mechanistic model as possible (see Fig. 1), e.g, achievement of effective concentrations of the bioactive component(s) at in vivo targets, target occupancy, and modulation of relevant biological networks.

4. Specimen collection, analysis, archiving. Collect and appropriately document and store specimens needed for assays described in point 3, and as applicable, fecal or other relevant samples to facilitate assessment of any association of the gut microbiota with differential responses. In analyses, include approaches to control for known artefacts, such as reagent contamination that can strongly affect outcomes in low biomass microbiome specimens (195).

Minor product components, impurities, instability, and chemical changes due to instability, resulting in the occurrence of degradation products can be responsible for some or even all measured activity. Instability has been demonstrated for a variety of prominent bioactive botanical marker compounds. These include ligustilide (a constituent of dong quai), which is very unstable and prone to redox reactions and photo-induced dimerization (43, 116, 117); desmethylxanthohumol, a chalcone found in Humulus lupulus, or hops, that forms the phytoestrogen, 8-prenylnaringenin, via Michael addition (118); and curcumin, found in turmeric, which is known to form a myriad of degradation products and metabolites (44). These chemical changes can modify bioactivity through a variety of mechanisms (97, 119, 120) and represent factors that appear to be minor (Residual Complexity, see go.uic.edu/residualcomplexity), but may significantly alter biological activities and therefore, require monitoring. A number of organizations have published general guidelines for stability testing, which are relevant for dietary supplements and other NPs, including the NCCIH Policy on Natural Product Integrity (18) and guidelines published by the International Council for Harmonisation (ICH; ich.org).

Variations in growing conditions generate variations in relative levels of NP constituents; differences in constituent levels are also seen between closely related species interchangeably used in similar contexts NP (121). For example, the licorice source plants, Glycyrrhiza glabra L. and G. uralensis Fisch. ex DC differ significantly in the ratio of liquiritigenin to isoliquiritigenin, as well as in congeneric constituents, and are therefore expected to have different bioactivities (122, 123). Characterization sufficient for the detection of such minor components, known or unknown, is therefore critical for quality control and for monitoring batch-to-batch variation and stability (15, 124). Identity and purity of all materials used, regardless of the supplier, should be verified prior to use (14, 124). Descriptions of approaches to NP selection and characterization have been published recently (17, 22, 114, 125).

Chemical and metabolic factors

Absorption, distribution, metabolism, and excretion (ADME) is a multistep, dynamic biological process that may be modified by product characteristics as well as by host characteristics (115, 126–129) including physiology and behavior. NP processing methods and final formulation can alter the chemical profile, stability, bioaccessibility, bioavailability, and metabolism of a NP (111, 130). For example, soy germ isoflavone glucosides incorporated into pasta can be hydrolyzed to aglycones by wheat β-glucosidase (131).

In contrast to most preclinical models, humans are generally diverse in genotypes, diets, other behaviors, and environmental exposures. Moreover, host genetics and cultural milieu profoundly affect flavor perception (132). If the NP will be delivered in or as part of a food, liquid, capsule or tablet, acceptability of flavor, texture, and or tablet size are essential. For both CT and generally available commercial products, consumer burden is another critical factor; this may include the number of capsules required, ease of swallowing capsules or tablets, cost, and portability. Thus, sensory testing and burden, as well as stability of the NP preparation are critical for successful translation of an intervention (Table 3).

Human microbiota are generally much more varied than those of lab animal models, reflective of both genetic diversity (133) and differences in past exposures (134). Run-in periods prior to pharmacokinetic and metabolic assessments can increase the relevance of preclinical animal models to long-term dietary patterns and provide important information for comparison with post-intervention metabolite profiles. A critical consideration is that, even where preclinical in vivo studies are designed to better model clinical trials, the models usually differ from humans in their greater uniformity of background diet, control of prior exposures to the relevant NP, dietary requirements, and metabolic capacities (Table 4). For example, taurine is an essential nutrient for cats (135), but synthesized by humans and rodents, while ascorbic acid is essential for humans, but synthesized by most other animals (136).

Human genetic differences and variations in gut microbiota cannot be considered to be independent, as both direct and indirect effects of host genetics on gastrointestinal ecology have been reported (133, 134, 137, 138). Both background diet and bioactive NPs affect host physiology directly, as well as through modulation of the microbiota, and the microbiota in turn contribute to modulation of NP processing (139, 140). Non-absorbed polymeric polyphenols may, for example, modulate absorption of polyphenol monomers in humans, such that history of exposure significantly influences pharmacokinetics (141). Effects on the host may in turn modulate microbial ecology and metabolism, while changes in microbial populations or gene expression may modulate host physiology, including ADME, collectively resulting in dynamic host-microbiota interactions that modulate or even generate the observed responses to ingested interventions (141–146).

Gastrointestinal microbiota have been shown in many cases to be required for the generation of bioactive metabolites of NPs. For example, gut microbiota can convert grape constituents to phenolic acids such as 3-hydroxy benzoic acid and dihydrocaffeic acid, which have been reported to inhibit β-amyloid and α-synuclein oligomerization, respectively, and epigenetically modulate gene expression (3, 147). Speakers stressed the importance of a strong, though not necessarily complete, understanding of the pharmacokinetics and metabolism of a NP intervention for optimizing the design of large-scale CT, although opinions differed on whether such knowledge was necessary before pursuing early-phase trials.

Presenters highlighted cases where the biological activities of enantiomeric NPs differ significantly. For example, a cancer preventive effect of soy in rodents could be replicated using the R-(+) enantiomer of the isoflavone metabolite equol, but not the S-(−)isomer produced in vivo by the gut microbiota (148–152). The microorganisms responsible for this conversion are widely present in rodents, but only found in only 20–30% of adult humans, a difference that may contribute to discrepancies between observations in rodents and humans. This finding emphasizes the importance of considering, characterizing, and fully reporting the chirality of NPs and their metabolites, as well as of understanding the role of the gut microbiota. It was pointed out that translational relevance of in vitro HTS assays might be substantially improved through the integrated use of replicable model systems able to mimic the generation of translationally relevant metabolites by the target animal or its gut microbiota.

TRANSLATIONAL RELEVANCE OF PRECLINICAL NP RESEARCH

General considerations

Preclinical in vivo models are widely used in biomedical research, and are often considered a prerequisite for CT. These models provide critical information on the biological response to a product that in silico and in vitro models cannot currently provide in a reliable fashion, including on bioavailability, metabolic activation and deactivation, PD and toxicology. In vivo testing also provides a basis for choosing dose ranges to be tested in CT.

In selecting any in vivo preclinical model system, investigators must keep in mind ways in which the differences between the model and the clinical application of interest are likely to affect inference from the study (20). The model selected should be appropriate for the experimental question based on the organism characteristics considered most critical to the target biology. The likelihood of translational relevance of a preclinical model to a clinical or other experimental question can be considered in terms of different types of model validity, including face validity (similar phenotype, lower probability of successful translation), construct validity (similar mechanism and phenotype, better probability of successful translation), and clinical validity (demonstrated clinical predictive success) (153–156) (Table 4). For example, short-lived invertebrate models may be particularly advantageous to understand NP effects on healthspan, as these species conserve the target-of-rapamycin, insulin/IGF-1-like signaling, and sirtuin pathways, which have been shown to modulate healthspan in organisms ranging from yeast through C. elegans and D. melanogaster to mammals (82).

Many factors that differ between preclinical in vivo models and people may influence the biological effects of a NP. Background diet and dietary history can influence NP absorption and metabolism. For example, the absorption and PK of flavan-3-ols are altered by exposure history, as well as by host factors such as obesity and diabetes (112, 141, 157, 158). This demonstrates the critical importance in research on humans of considering, assessing, and reporting background diet and history, as well as standard biometric parameters. Assessing correlations among human genomic, transcriptomic, proteomic, or metabolomic data, and between these and outcomes following a standardized exposure may elucidate mechanisms that underlie differences in the absorption or generation of, or exposure to, bioactive constituents or bioactive metabolites (128), and allow prediction of responders versus non-responders. These approaches can enhance translation between epidemiological, pre-clinical, and clinical studies.

Selection of a mouse model

Mice are frequently the model species of choice for in vivo preclinical research, due to their relatively small size and short life spans, and the large number of strains and large body of literature available. Nevertheless, mouse and other in vivo preclinical models differ from humans in many known and unknown details, both intrinsic (e.g., genetic, immune (159–164), surface area: volume ratio, lifespan (165)) and extrinsic (e.g., captivity, diet, history), which, depending on the research question, may substantially affect translational relevance (166).

It is critical to remember that each distinct inbred strain may respond differently to an intervention (126, 167), that the same strain supplied by a different vendor (or with different gut microbiota) may respond differently (168), and that sex differences in response may appear only in specific species or strains (169–172).

A wide variety of approaches are used to increase the predictive validity of mouse models. Transgenic mice are commonly employed to enhance construct validity; guidelines for rigor and transparency in research using mouse mutants have been published (173). Transgenic constructs allow experimentation on specific human pathways and targets of interest. Where the genetic mechanism of interest is unknown, for example in toxicity testing (166, 174) or research on neuropathic pain (175), the genetic heterogeneity of outbred mice(174) may provide important information that is unavailable from inbred strains without otherwise affecting outcome variability (176). Many different approaches to “humanizing” mice have been utilized to enhance predictive validity for specific research disciplines or questions. Employing the severe combined immune deficiency mouse (SCID) is one such approach, whereby human peripheral blood cells, and cells of other human fetal immunological tissues such as bone marrow, thymus, or liver are introduced (usually an NOD-SCID-γ mouse) (177). Another approach involves developing transgenic and chimeric mice in which specific human genes are expressed. Various approaches have also been utilized to generate mice with “humanized” gut microbiota, which have been found to have significant clinical predictive validity in developing anti-infective, autoimmune disease, cancer, and transplant therapeutics (178–180). Inbred mice reconstituted with microbiota from wild mice have also been reported to have increased translational value (181, 182). In their natural environment mice, like humans, are exposed to microbes, including pathogens, and thus should be regarded as metaorganisms, with their own specific microbiota (including bacteria, fungi, and viruses). Returning natural microbiota to standard laboratory mice was found to significantly improve the model’s ability to predict human immune responses in two preclinical studies (183).

Model- and context-related considerations

Optimization of preclinical research for relevance to future NPCTs as well as general human use requires not only careful consideration of the choice of models, but also of intervention dose (184), composition, and delivery (Table 4). As discussed above, when research and intervention are founded on a traditional use, available ethnobotanical and ethnopharmacological information may be used for guidance (Table 3). Clinical and preclinical interventions should be as similar or, where more appropriate, as equivalent as is feasible, bearing in mind species differences (185), as well as any age-, sex-, size-, or health status-related differences which may affect behavior, absorption, metabolism, and/or toxicity. Allometric or mechanistic modeling approaches can be used to scale to human dosing from preclinical in vivo data, and vice versa (151, 184, 186).

Background diet, as well as other environmental exposures, including medications, alcohol, smoking, and air quality may also have important effects on NP bioavailability, metabolism, and biological effects in both humans and animal models. In the context of NP-diet interactions, it is critical to consider species-specific dietary requirements and optimal diets (e.g., differences in ascorbic acid requirements (187), optimal amino acid or lipid compositions) (185), as well as species differences in biosynthetic abilities (188) and digestive anatomies (189). When comparing results across studies, it should be noted that differences in local regulation of vertebrate animal research (e.g., the European Union prohibition on feeding animal-derived ingredients to herbivores) result in substantial differences in commonly used basal diets or chows. Researchers must carefully consider, for a given research question, whether background diets should be the same for different species, or at similar levels of optimization for each, or whether both possibilities need to be included as controls. As discussed below, physiologically based pharmacokinetic models may be helpful in making dose adjustments for different subpopulations.

Diet and circadian rhythm interact in their effects on health, gut microbiota, and metabolism (190, 191). These effects are modulated by sex in organisms ranging from fish and mice through humans (192–194). Through their effects on gut microbiota, NPs may have important effects on their own metabolism and on the generation of active metabolites which, in turn, may significantly affect both PK and PD. In addition, changes in the gut microbiota may exert separate direct or indirect effects on animal biology and health (168). Good practices for characterization of the gut microbiota have been discussed elsewhere (195).

Preclinical in vivo models are often used in environments that are optimized for the investigators and different from the species’ natural habitat. This context by itself is likely to affect experimental outcomes and may also modulate the effects of an experimental intervention. In either case, the experimental environment is likely to affect translational relevance. Investigators need to acknowledge these often unavoidable factors and consider them when inferring the translational relevance of studies (156). In general, given the range of interrelated factors that may modulate the effects of a dietary intervention, it is critical that supporting data include experiments rigorously designed to falsify the hypotheses tested, e.g., to test whether the proposed bioactive constituents or metabolite and/or proximate targets are indispensable and inducing for the effect studied (Figure 1) (35, 196).

The robustness of a result is optimally assessed by attempting to replicate the study not only in the same model, by those originally reporting the result and others, but also across models based on a range of species, genotypes, and phenotypes. Recording as much information as possible about the model and the environmental conditions under which it was used may help others to replicate the studies or facilitate exploring why a finding failed to replicate, or to translate to another model or species.

Genetic diversity and gene-NP interactions

One critical difference between most preclinical in vivo models and humans is that the former are usually inbred, except as discussed above. Where the bioactive compounds are known, and population differences in health response have been reported, interrogating those differences may lead to the elucidation of underlying gene-environment interactions. For example, common genetic variants in two fatty acid desaturase genes (FADS1 and FADS2) appear to modulate the response to dietary polyunsaturated fatty acids (PUFA), resulting in differences in resulting levels of pro- and anti-inflammatory lipid mediators and differences in risk for inflammation-associated health outcomes such as asthma, rheumatoid arthritis, and cancer (197). Tracking the effects of prevalence of these variants might shed light on inter-trial differences in NPCT outcomes with PUFA interventions (197, 198). Similarly, a variety of factors including biometrics and genetic polymorphisms appear to be responsible for observed differences between black and white American women in associations between serum 25-hydroxy vitamin D levels and bone fractures (197–199).

In silico modeling approaches

Quantitative or computational modeling and simulation are beginning to show their utility in optimizing NPCT design. Quantitative systems pharmacology (QSP) encompasses computational methods including statistics, pharmacometrics, PK and PD modeling, mechanistic or physiologically based pharmacokinetic models (PBPK), in vitro-in vivo extrapolation, and machine learning approaches. QSP is often employed in drug development to integrate data, optimize experimental dosing regimens, identify sources of inter- and intra-individual variability, predict efficacy or toxicity, or estimate appropriate doses for different sub-populations (200–202). PBPK models incorporate physicochemical data (e.g., molecular weight, pKa, lipophilicity), in vitro or preclinical data (e.g., on metabolism and transport), and population-specific data (e.g., organ volumes, tissue composition, blood flow) to model ADME. PBPK can be used to predict equivalent exposures for humans and other species, for both translational or reverse-translational studies (Table 4). Sensitivity analyses can allow comparison of the magnitudes of the effects of different parameters on the outcomes. These models may also provide insight into the interplay between metabolizing enzymes and transporters, inter-individual variability due to intrinsic (e.g., pharmacogenomics, age, sex) and extrinsic differences (e.g., drug, food, or other environmental exposures). Perhaps most importantly, these computational models can highlight the most significant gaps in the data (203–205), as well as support evidence-based selection of initial dose ranges for CT.

The application of quantitative pharmacology to complex NPs is in its infancy. In addition to the inherent chemical complexity of NPs, understanding of the pharmacology and metabolism of the individual chemical components is mostly very limited. Nevertheless, application to NPs of modeling and simulation approaches that incorporate uncertainty and variability can provide estimates of differences among populations, inform dose selection and go/no-go decisions, and provide early estimates of the probability of rejection of the null hypothesis.

GOOD PRACTICES FOR MOVING TO A NPCT

As outlined above in the section “NP Characteristics Critical for Replicability”, speakers emphasized the importance of having rigorous and replicable data on the chemical components of the NPs required for the biological activities of interest, as well as evidence for the activity of interest itself from at least two distinct in vivo preclinical models. This information is critical for both NPCT planning and for post-trial interpretation of results (Figs. 1, 2).

Figure 2. Good practices for actionable results in future translation of Natural Products research to Clinical Trials (NP2CT).

Translation is most informative and cost-effective when CT Phase (I vs. II) and design (e.g. RCT, adaptive) decisions are built on a solid and rigorous evidence base (pyramid and tabletop). The choice to proceed to an NPCT and the determination of the most appropriate trial type (examples on sphere) depends on the extent and strength of supporting data available (table legs and segments). If multiple supporting data elements are missing or weak, an earlier phase CT may be more informative and cost-effective than a Phase III CT. If a substantial number of CT design choices or parameters lack supporting evidence, a more useful next step may be research addressing those gaps via either preclinical research or a mechanistic or early phase NPCT.

NPCT planning

The foundational information should rigorously support a clear and specific definition of the question to be addressed by the NPCT, framed in quantitative terms, and matched to the study population, the evaluation period, and the primary and secondary outcomes (206, 207). When strong evidence supports the requirement for specific bioactive components and specific in vivo targets, this information can be used to develop approaches for de-risking a project, i.e., assessing whether the accumulating data are consistent with the hypotheses and support continuation. This information enriches the ability to design a NPCT that will allow evidence-based interpretation and planning of next steps regardless of the completed outcome, as shown schematically in Figure 1, by providing the ability to:

• define minimal effective concentrations in plasma or at the targets;

• develop methods to assess in vivo engagement of putative targets, especially methods which may be translationally relevant, such as non-invasive methods;

• use those methods to assess the relationship between target engagement and outcomes;

• elucidate causal cellular or molecular mechanisms mediating the generation of the biological or behavioral responses of interest (i.e., PD biomarkers).

This mechanistic information will be critical for clinical dose-ranging and dose optimization studies, as well as for detecting potential “drop-ins”: given the broad availability of many NPs, placebo group participants sometimes present with anomalously high levels of apparent off-protocol exposure to the intervention. Off-protocol exposures, whether accidental or intentional, may hamper the ability to interpret the results. Knowing which compounds are indispensable and inducing for the bioactivities of interest also allows investigators to ensure that sufficient time is allowed for washout between inventions in cross-over designs (26). Equally important, knowledge, e.g., of minimal effective concentrations and targets, or of metabolites indispensable for activity or genotypes required to generate those metabolites, can be used to generate minimal go/no-go criteria, which help identify CTs that are less likely to be informative, and ensure that only treatments with strong supporting evidence move forward to larger, confirmatory CT (Figure 2). Elucidation of such causal, molecular mechanisms of action may also be useful to predict which participants are likely to respond well or poorly to a specific intervention. Similarly, understanding the molecular mechanisms between an initial target and the final, assessed outcome should enable not only enhanced post-trial assessment of the validity of the pre-trial mechanistic hypotheses, but also contribute to the ability to better predict individual responses to the intervention.

Safety and dose selection

Safety issues that arise in preclinical testing must be carefully considered prior to CT, and safety outcomes must be included in all NPCTs (208). Safety in the context of a NPCT cannot be inferred from traditional nor contemporary use if the product, dose, route, schedule, and context of use differ (Table 3). Dosing in larger NPCT should be based on dose-response data from Phase I/II studies with safety data. Dose ranging in Phase I NPCT may use allometric or mechanistic models to build from in vivo preclinical studies (Table 4), as described above, or, where appropriate, may be based on current clinical practice or traditional use.

As described above and summarized schematically in Figure 2, a variety of rigorous foundational information provides critical input for translation to later-phase efficacy CT, with much of this information requiring input from other types of NIH-defined CT. This foundational information includes:

• safety of the product to be used in the NPCT, in the population to be studied;

• product specifications (for interventions and any controls), including chemistry, stability, consumer acceptability (including sensory qualities), packaging;

• bioavailability, PK, PD of the product to be used in the population to be studied;

• dose-response properties.

Post-trial interpretation of results

In the absence of the foundational information described above, and without meeting the resulting go/no-go criteria, it will be more challenging to interpret a failure to reject the null hypothesis, as lack of effect could be due to many factors (Figure 1). These include, but are not limited to an incorrect hypothesis regarding the relationship between target engagement and outcome, or less interestingly, failure to achieve sufficient levels of bioactive compounds or target engagement. As the following variables could have an important influence on the results, obtaining information on these for each participant can also be critical to the interpretation of NPCT outcomes:

• Background diet – habitual and during the NPCT (the latter is potentially controllable).

• Age, sex, race/ethnicity, body size and composition, health status (especially with respect to outcomes of interest), co-medications, smoking, geography.

• Genetic and epigenetic data, including characterization of microbiota structure and gene expression, especially as they relate to ADME of the NP (134, 168, 195, 209). When known, relevant genetic variants exist, e.g., in metabolism, specimens for genetic analyses should at least be collected and saved.

• Sensory characteristics (texture, flavor, odor; pill, capsule, etc., number and characteristics) of products to be used in a NPCT must be acceptable to trial participants, who should not be able to distinguish amongst intervention(s) and controls. Information on participants’ pre-trial beliefs about product efficacy, and post-trial conclusions with respect to the intervention they received should be collected as this may critically inform interpretation (Tables 3 and 5).

Addressing expectancy effects

Human responses in NPCTs are also influenced by expectations, often resulting in either placebo or nocebo effects. These effects may vary substantially for different populations and interventions (e.g., color, size, number of capsules, estimated cost)(210). Through conscious and subconscious processes, people (researchers as well as participants) can be biased when assessing and reporting symptoms, functional abilities, and other patient-oriented outcomes. Whenever feasible, blinding/masking and other approaches should be employed to minimize such biases, including in preclinical studies (Tables 1 and 5).

Conscious and subconscious mind-body processes can also lead to real positive health outcomes, including in apparently objective, quantitative outcomes such as blood pressure (211). Because blinded “placebo” comparison groups are likely to achieve real health benefits through these processes, the magnitude of benefit estimated by comparing conventionally blinded placebo and treatment (intervention) groups is likely to underestimate the amount of benefit that would be observed when comparing non-blinded treatment to no treatment (212). Therefore, when feasible, no treatment and open-label treatment comparison groups should be included in RCT design, so as to arrive at best estimates of real-world effect sizes, as well as effect size from conventional blinded comparison.

Sample and effect size

Sample size calculations must reflect simultaneous consideration of trial phase and design, estimated and targeted magnitude of treatment effects, and statistical power (207). The targeted effect size should be biologically plausible, and based on the degree of benefit that patients, consumers, or practitioners would consider sufficiently beneficial to be worthwhile, given other considerations such as cost, inconvenience, side effects, and potential for serious adverse effects ((213); Table 5). Effect sizes should not be based on estimates from the literature as a whole, which almost always leads to biased estimates (214), inasmuch as the published literature contains a biased selection of all empirical research that is performed. Meta-analytic evaluations of the literature have consistently found that studies yielding null or mixed results are substantially less likely to appear in the published scientific record (215, 216) than those studies with strong statistical support for the tested hypothesis. This occurs even where the amount of collected data renders these “negative” studies highly informative. As a result, statistically significant published results often provide inflated effect size estimates (217). Increased use of registered reports, e.g., as provided by preclinicalresearch.eu, may begin to address this issue and decrease the bias in the scientific literature.

TABLE 5.

Highlights of major considerations for clinical trial design

1. Statistical power. Base sample sizes on the smallest effect size of interest, in the context of existing options and consumer burden. Use best current meta-analytic approaches to assess bias in the literature, and to correct for bias in reported effect sizes (214).

2. Transparency and statistical analysis plans. Clearly describe and document statistical analysis plans before beginning the trial.

3. Comparison groups and expectancy/placebo. Base controls and interventions on the supporting data and the resulting hypotheses. The inclusion of no-treatment, and open-label treatment groups, as well as conventional blinded comparison groups, in RCTs provides best estimates of real-world effect sizes. Development of inert but effectively masked placebos for NPs with strong color or odor (e.g., with similar or masking odor or color) may be challenging. Assessing participant perceptions of product efficacy pre-trial, and of intervention allocation post-trial may provide critical information for interpretation of trial results.

4. Optimal clinical trial paradigm. Adaptive design (AD) NPCTs may be a more efficient approach to optimizing, and testing the efficacy of an intervention, if the evidence base for some CT parameters is lacking. AD CTs require statistical rigor to avoid bias in the interpretation of the results. ADs may be a good choice for Phase II trials.

Analysis of the literature

Beyond publication bias, two additional factors that challenge the translational relevance of the published literature have been widely agreed upon: analytic flexibility during the data analysis (218), and low statistical power (40, 218, 219). All of these factors need to be taken into account when building on published research. Nevertheless, it should be kept in mind that a substantial portion of the scientific literature does consist of reliable and replicable findings. An important challenge for researchers navigating this intrinsically biased body of literature is to determine which research is reliable enough to build on, and which research is more likely to not be replicable (220). Because it is practically impossible to determine whether a finding will replicate based on a single study (221), meta-analytic techniques have been developed to aggregate evidence across studies. However, analytic flexibility and selection for significant effects in small studies lead to upwardly biased effect size estimates. Novel bias detection techniques can identify different types of biases: P-curve analysis (222) can identify lines of research that consist of false positives, and under some circumstances PET-PEESE meta-regression (222, 223) can calculate bias-corrected effect size estimates.

Clinical trial design

In the design of a later Phase NPCT, it is important to recognize that estimated treatment effects from pilot studies are necessarily imprecise. Pilot studies may under- or overestimate the true effect size (but, as noted above, publication bias generally selects for overestimates). Effect size estimates and sample size calculations should, therefore, be conservative and based on prospective meta-analyses (https://methods.cochrane.org/pma/what-pma-0) or other rigorous meta-analyses, i.e., those which include all eligible studies designed and performed, or those which pass rigorous bias detection tests, such as the p-curve analysis described above, with PET-PEESE meta-regression used in estimating effect sizes. Results of pre-registered, rigorously performed and analyzed pilot studies may also be used to estimate effect sizes when convergent, corroborating, or similar results are obtained in independent research from separate groups, using approaches based on different assumptions (triangulation, or orthogonal approaches), such that any artefacts or sources of bias are likely to differ for the different approaches (224, 225). Adaptive designs (see below) and sequential analyses (https://methods.cochrane.org/pma/what-pma-0) can be used to design informative studies when uncertainty about the effect size estimate remains. An argument can be made that later Phase NPCT intended to test whether an intervention is beneficial should be undertaken only where the estimated effect size from rigorous methods such as those described above is at least as large as the estimated smallest important difference or smallest worthwhile effect (213) for the intervention and outcome to be tested. This would not apply if the hypothesis to be tested is that the intervention has no effect, for example, on an adverse outcome.

Where the goal is to determine the maximum potential of an intervention, Phase II studies should enroll those subpopulations predicted to have strong responses, a practice called study “enrichment.” While enrichment, and the inclusion of multiple endpoints are appropriate for Phase II NPCT, it is important to recognize that these designs tend to bias the results of Phase II studies towards the conclusion that moving forward is worthwhile, although positive findings could be due to chance.

Because NPCT are costly, time consuming (226), and dependent on voluntary human participation (11), approaches that can support making trial design and prioritization decisions which optimize acquisition of replicable, actionable results (Tables 5 and 6) are in high demand. Multilevel modeling can be used in conjunction with existing data to identify parameters most strongly associated with the outcomes of interest (227). Simulation modeling can also be used to identify trials that are unlikely to reject the null hypothesis (228).

TABLE 6.

Highlights of major issues to consider in clinical trial decision-making

1. Trial focus. Is the proposed trial well-justified? Are the evidence for and magnitude of the effect, the size of the relevant population, and the need for the knowledge to be generated commensurate with the anticipated cost of a CT?

2. Likelihood of actionable information. What CT paradigm or other next step is most likely to provide actionable information?

a. Next step: Is the most informative next step likely to be a pilot trial, a larger CT, clinical research other than an efficacy study, or research other than a CT?

b. NPCT protocol: Have trial/protocol design decisions been sufficiently de-risked? Is there sufficient understanding of the path(s) from initial target(s) engagement to outcome, and availability of non-invasive assessments of these, to support evidence-based decisions regarding the next test of activity whether or not the null hypothesis is rejected upon trial completion?

c. Bioavailability: Are assays of bioavailability at the immediate in vivo target available and translationally relevant?

d. Open ends: Can any remaining questions be sufficiently addressed via use of a rigorous, adaptive trial design?

Adaptive designs (AD) can provide an efficient yet rigorous approach to honing the parameters of a NPCT while obtaining preliminary data on clinical effect of the intervention(s). An AD uses accumulating information to implement changes to the design during the course of the trial (229)(230). Changes in the trial must be based on pre-specified decision rules. Changes made on an ad hoc basis may not preserve the type I error rate (false positives), and may lead to bias in parameter estimates or other problems with statistical properties. The many variations of AD include early-stage AD, which may focus on dosing, exposure, heterogeneous responses, or biomarker responses. As it is assumed that AD are exploratory studies, there is less concern about the control of type I error. AD methods are more accepted in the learning (exploratory) stages of CT. Some key variations of AD include:

• Adaptive enrichment designs, which target therapies to participant subgroups most likely to benefit. In such designs, a trial initially enrolls a broad population. The initial study period (discovery phase) reveals participant groups most likely to benefit from treatment. In the second phase of the trial, only those subgroups most likely to benefit from treatment are randomized, providing increased power to detect an effect.

• Sample size re-estimation designs refer to AD that allow for adjustment of sample size based on an interim review of the data. Sample size re-estimation based on revised estimates of treatment effects has been controversial for the reasons cited above. However, internal pilot designs that re-estimate only “nuisance” parameters, such as the variance of control group event rate, are more generally accepted.

• Platform trials are useful for studying multiple targeted therapies in the context of a single outcome, with therapies allowed to enter or leave the platform on the basis of pre-specified stopping boundaries (229). This approach could be used to choose between several different products in a NPCT, e.g., different preparations of NP material from the same source, such as whole dried plant, different extracts, and/or material enriched for one or more putative bioactives (231). Such studies could be done under a master protocol (232), defined as a protocol encompassing multiple substudies, which may have different objectives. A master protocol supports coordinated efforts to evaluate one or more investigational interventions in one or more subpopulations within the overall trial structure.

Adaptive seamless designs combine objectives traditionally addressed in separate trials into a single CT. First-stage results inform the second stage, but final analyses use data from both stages. For example, the first stage may establish preferred dosage, while the second stage can assess the efficacy of that dose. To avoid bias in the interpretation of results, such designs require clear, comprehensive, pre-specified criteria and raise tricky statistical issues, as data from both stages must be combined in a way that preserves key statistical operating characteristics.

In general, most AD require slightly larger sample sizes than a corresponding, conventional Phase II CT, but provide increased opportunity to detect an effect, and can also lead to timelier identification of ineffective (or clearly more efficacious) interventions, improving efficiency by allowing early discontinuation of trials for less effective or ineffective products.

Value of information

Whether and how to pursue a NPCT depends on the value of the information that the CT can be expected to generate. Given the high costs of conducting a CT, it is of critical importance to prioritize the questions to be addressed (11). Value of Information (VOI) analysis provides a framework for generating quantitative estimates of the value of further research in a particular area. VOI estimates can serve as a guide for prioritizing research resources (233).

The basic framework for estimating the VOI is the product of several factors: (1) the value of the information to be gained for individuals who would be affected by the findings, including both changes in health status and changes in costs; (2) the number of affected individuals in the population; (3) the extent to which the information generated would be adopted or implemented; and (4) the degree to which the value of the knowledge gained will depreciate over time as additional knowledge becomes available. This estimated quantity is typically summed over the relevant time horizon, adjusted by an appropriate discount factor reflecting the preference for current over future improvements in health.

Generating estimated values for the elements of the VOI calculation can itself be a costly and complex process. Depending on the NPs under consideration and the effects to be tested, different levels of VOI analysis may be undertaken. An initial level of conceptual VOI can assess whether any of the factors in the product are close enough to zero that the overall VOI would be very small, indicating that substantial research investments would not be justified. Depending on other characteristics of the product under investigation, the health outcomes involved, and the specific data collection challenges to be addressed, progressively more detailed approaches to VOI exist, termed minimal modeling, full modelling, and maximal modeling, that provide more thorough estimates of the value of the information to be gained from pursuing additional research (233, 234).

The use of VOI analysis to support decisions regarding which NPCT to pursue (Table 6) depends significantly on the available research foundation. Assessing the VOI to be gained to a representative affected individual requires definition of the outcome(s), and understanding of the effect sizes to be achieved. Defining and estimating the size of the relevant affected population requires understanding of the dietary, environmental, and genetic context of the effects. Use of VOI in priority setting is still relatively new and will benefit from further exploration and input from researchers and decision-makers. In particular, VOI should be assessed to understand the extent to which it changes decision-making relative to conventional approaches, and the health impact and monetary value that result from such changes.

CONCLUSIONS

The multi-disciplinary approach

Rigorous, evidence-based decision-making with respect to the design and prioritization NPCTs ultimately requires bringing together myriad research sub-disciplines. These disciplines span from ethnobotany through NP chemistry, food science, a range of modeling approaches, biomedical specialties including CT design and statistics, to value of information research. Notably, researchers who specialize in one sub-discipline often do not have ready access to collaborators from other fields, or are unaware of relevant state-of-the-art approaches from these areas. However, the likelihood that an NPCT will be optimally informative depends not only on the rigor of the information contributed by each sub-discipline, but also the rigor with which the different threads are brought together, achieving collaborative synergy. Deploying and synergizing the good practices discussed by attendees at the September 2018 workshop, across the full range of this disciplinary continuum and from the earliest phases of developing data that may inform a future NPCT, should help focus translational NP research on objectives that are most likely to contribute to the design of, and increase the knowledge gained from future NPCTs (Figure 2).

Controversies

Two major areas of controversy at the workshop related to translation from traditional health practices. The first of these areas of disagreement was the appropriateness of the RCT approach for the study of traditional practices, with several attendees arguing that RCT methodology is not well suited to studying them. Most attendees, however, supported the need for NPCTs to assess the safety and efficacy of tradition-based approaches in a 21^st century context, with the many associated differences, including in the interventions, and in individuals’ histories, environments, and behaviors. A second area of disagreement was the extent to which it is necessary to elucidate causal, molecular mechanisms of action, including determination of bioactive substituents and their in-animal targets, in order to support the design of an optimally informative NPCT. A case was made that, in some contexts, adherence to current good manufacturing practices, including rigorous adherence to standardized growing and processing procedures, and in conjunction with an appropriate assay of systemic exposure, and, in some cases, bioactivity, is sufficient to ensure NP replicability. However, many attendees felt that product specifications, PK assessments, eligibility criteria, and outcome assessments should, to the extent feasible, be based on strong mechanism of action evidence, preferably derived from conceptually independent (orthogonal, or triangulated) scientific approaches. Attendees considered such evidence to be critical both for optimal trial design and for the ability to interpret results after trial completion. Information on the mechanism of action, including compound(s) indispensable for inducing effects, as well as biological target(s) and response(s) indispensable for outcomes, may critically inform the design and/or interpretation of an NPCT, from product quality control through understanding the bases for individual differences in response, and may support the ability to develop biomarkers of proximate effects, or biomarkers useful to distinguish likely responders and non-responders.

NCPT good practices

In preclinical research aimed at informing the understanding of purported, causal, molecular mechanisms of action, good practices include the use of biologically relevant concentrations of biologically relevant compounds and/or their in vivo metabolites, in one or more relevant models. Participants also emphasized the importance of including appropriate controls for common artefacts encountered in in vitro assays for the activities of phytochemicals. Further approaches that were highlighted as good practices for strengthening the foundational data for NPCTs included: (a) high-throughput, non-targeted approaches that can generate strong, specific hypotheses regarding molecular mechanisms of action of chemically complex NP; (b) pre-registration of experiments at the hypothesis-testing stage, rigorous adherence to experimental protocols, randomization, and masking, which collectively minimize the risk that unconscious biases influence the results; (c) thoughtful consideration, when selecting model systems, of translational relevance and clinical validity for a specific research question; and (d) comprehensive and transparent reporting, combined with meaningful and FAIR data-sharing. Participants discussed these issues for models ranging from in silico through the burgeoning variety of cell-based models, from organoids through microphysiological systems, through in vivo models from invertebrates to transgenic, outbred, wildling, and humanized mouse strains.

Good practices for NPCT decision-making based on the available foundational data were extensively discussed and gave rise to the third major area of controversy at the workshop. Some attendees felt that n-of-1 trials (235) based on traditional use and/or current practitioner experience provide sufficient evidence for safety and efficacy, and are better suited to tests of these approaches than RCTs. Others felt that RCTs are critical, but that NPCTs should be undertaken only when there is a strong likelihood that the outcome will influence practice. The foundation for such an assessment would come from public health needs and a rigorous, evidence-based effect size estimate, that predicts an effect magnitude considered worthwhile by practitioners and consumers, collectively supported by multiple, distinct lines of evidence (225). The evidence base for such assessments is strongest when it includes experiments designed to falsify the hypotheses tested, such as chemical, genetic, or other knockdown studies (11, 35).

Outlook

While many of the practices discussed at the workshop are already employed by numerous NP researchers, few, if any reported NPCTs have combined the full breadth of state-of-the-art approaches when investigating the effect of a NP on a specific health outcome. Integrating these practices in the development, implementation and interpretation of NPCTs will require research teams to collaborate synergistically across disciplines. The anticipated benefit of combining the rigorous and trans-disciplinary approaches referenced herein increases the likelihood that, in the many instances where the outcome of a NPCT does not signal a change in practice, the trial will nevertheless have provided actionable new information, for example, about NP metabolism or the target biology.

ABBREVIATIONS

AD

adaptive (clinical trial) design

ADME

absorption, distribution, metabolism, excretion

CT

clinical trial

FADS

fatty acid desaturase

FAIR

findable, accessible, interoperable, reusable (desirable characteristics for archived data)

HTS

high-throughput screening

MPS

microphysiological system

NOD

non-obese, diabetic

NP

natural product

NPCT

natural product clinical trial

PBPK

physiologically based pharmacokinetic (a type of in silico model)

PD

pharmacodynamics

PK

pharmacokinetics

PUFA

poly-unsaturated fatty acid

QSP

quantitative systems pharmacology

RCT

randomized clinical trial

SCID

severe combined immune-deficiency

VOI

value of information