Good Practices in Model‐Informed Drug Discovery and Development: Practice, Application, and Documentation

Abstract

This document was developed to enable greater consistency in the practice, application, and documentation of Model‐Informed Drug Discovery and Development (MID3) across the pharmaceutical industry. A collection of “good practice” recommendations are assembled here in order to minimize the heterogeneity in both the quality and content of MID3 implementation and documentation. The three major objectives of this white paper are to: i) inform company decision makers how the strategic integration of MID3 can benefit R&D efficiency; ii) provide MID3 analysts with sufficient material to enhance the planning, rigor, and consistency of the application of MID3; and iii) provide regulatory authorities with substrate to develop MID3 related and/or MID3 enabled guidelines.

Abbreviations: ADME, Absorption distribution metabolism and elimination; BLQ, Below limit of quantification; BOS, Break‐out session; CE, Comparative effectiveness; COPD, Chronic obstructive pulmonary disease; CRCL, Creatinine clearance; CSR, Clinical study report; CTD, Common technical document; CTS, Clinical trial simulation; DDMoRe, Drug Disease Model Resource http://www.ddmore.eu/; ECG, Electrocardiogram; eCTD, Electronic Common Technical Document; EFPIA, European Federation of Pharmaceutical Industries and Associations; EMA, European Medicines Agency; Emax, Maximum effect; FDA, Food and Drug Administration (United States); FEV1, Forced expiratory volume in 1 second; FIP, First‐in‐patient; GCP, Good Clinical Practice; GnRH, Gonadotropin‐releasing hormone; HbA1c, Glycated hemoglobin; HTA, Health technology assessment; HV, Healthy volunteer; ICH, International Conference on Harmonisation; IGI, Integrated glucose insulin; IGRH, Glucose‐red blood cell‐HbA1c; IMI, Innovative Medicines Initiative; iNOS, Inducible nitric oxide synthase; LDL‐C, Low density lipoprotein cholesterol; LME, Line mixed effect; M&S, Modeling and simulation; MABEL, Minimum anticipated biological effect level; MBDD, Model‐based drug development; MBDDx, Model‐based drug discovery; MBMA, Model‐based meta‐analysis; MCPMod, Multiple comparison and modeling; MIDD, Model‐informed drug development; MID3, Model‐informed drug discovery and development; MPG, Mean plasma glucose; MSWG, Modeling and Simulation Working Group; NGF, Nerve growth factor; NLME, Nonlinear Mixed Effects; PBPK, Physiologically‐based pharmacokinetics; PD, Pharmacodynamics; PK, Pharmacokinetics; PoC, Proof of concept; QA, Quality assurance; QC, Quality control; QTc, Heart rate correct QT interval of the ECG; R&D, Research and development; ROI, Return on investment; SAR, Structure affinity relationships; TrKA, Tropomyosin receptor kinase A

1. BACKGROUND

The European Medicines Agency (EMA)/European Federation of Pharmaceutical Industries and Associations (EFPIA) Modeling and Simulation (M&S) joint workshop held in 20111 assembled scientists from the Pharmaceutical Industry, academia and regulatory authorities from across Europe, the USA, and Japan to consider the (then) current and future role of M&S in drug development and regulatory assessment. The output of this workshop has been summarized extensively.2, 3, 4, 5, 6

The current EFPIA workgroup evolved from the 2011 EFPIA workshop committee in order to continue a close working relationship with the nascent EMA Modeling and Simulation Working Group (MSWG) on matters of mutual interest. One of the EFPIA groups' commitment to EMA was to generate a “good practice” manuscript covering aspects of planning, conduct and documentation of a variety of quantitative approaches. In return one of the EMA commitments was to organize a workshop on methods and impact of dose finding and dose‐exposure‐response information on approval, labelling and post‐approval studies, which took place in London in December 2014.7

2. DOCUMENT PURPOSE

The document introduces the term Model Informed Drug Discovery and Development (MID3). We define MID3 as a “quantitative framework for prediction and extrapolation, centered on knowledge and inference generated from integrated models of compound, mechanism and disease level data and aimed at improving the quality, efficiency and cost effectiveness of decision making”. The concept that R&D decisions are “informed” rather than “based” on model derived outputs is a central tenet to the views expressed herein.

There are three principal sections (3–5) of the document, each of which is targeted towards a distinct, but complementary, audience. The first section provides (company) decision makers an overview as to “why” MID3 is important based on current practice and expressed value to‐date. This is followed by a systematic assessment of over 100 published applications across drug discovery, development, and life‐cycle management. The second section addresses “what” MID3 means for practitioners who need to understand the MID3 premise and adopt the methodologies and strategies. This section also highlights some of the challenges and opportunities in MID3 implementation. We propose a categorization scheme to display MID3 business impact and we apply this scheme to the examples introduced in the first section. We believe that this categorization scheme will permit both historical and future business value elucidation. The third and final section addresses “how” MID3 approaches should be conducted with a particular emphasis on the nature and extent of documentation. This includes a systematic approach to evaluate the potential impact of necessary assumptions and associated quality control (QC) and quality assurance (QA) procedures. An important aim of this section is to enable the generation of content that will enhance the contribution of MID3 within the regulatory submission and review cycle.

3. “WHY” MID3 IS IMPORTANT FOR DECISION MAKERS

3.1. MID3: the established business case and value to R&D

The benefits of integrated M&S to core business activities are widely accepted across a diverse range of industries, from aerospace to finance. Approaches to estimating the overall value of M&S application has been investigated (e.g., defence,8 general and software design9) and, in some cases, quantified (e.g., meteorology10 and molecular modeling11). In the field of engineering and technology, it is utilized to facilitate decision‐making across all levels of design, manufacture, quality control, supply, and distribution.

Despite being a relatively late adopter, the pharmaceutical industry continues to grow its utilization of M&S across a broad range of applications, from the assessment of structure‐affinity relationships12 through to the prediction of cost‐effectiveness of new medicines in the health technology assessment process.13 The benefit of M&S application in these two areas can be demonstrated through the establishment and evolution of a variety of good practices that incorporate routine usage of M&S.12, 14 These two areas can be considered as the current “bounds” for MID3 utilization and adoption. Within these bounds, “intermediate” elements encompassed by regulatory guidelines15, 16, 17 and industry good practice recommendations18, 19, 20, 21 advanced and/or advocated M&S application for some time. The concept of “learn and confirm” emerged in the late 1990s and emphasized the need for early development activities to more effectively inform later stage development.22 A fundamental component of these activities was the development of mathematical models to characterize emerging data (and the underlying system) in order to enable prediction to a new situation (“set of conditions”) thereby informing decision‐making. In this respect, MID3 is no different and we will return to this tenet in section 4. The need for greater adoption and integration of many of the components of the modern term “MID3” was popularized by industry analysts as a solution to the decline in productivity around the turn of the century.23, 24 In addition, these approaches were identified as a strategic component for the Critical Path Initiative25 and the current Innovative Medicines Initiative, particularly the Drug Disease Model Resources (DDMoRe). Furthermore, the value of MID3 application has received encouragement and support from within the regulatory domain.7, 26, 27, 28

Although the “business case” for MID3 has become more established, an important question to address is whether “business value” has been delivered as a result of the growth in deployment. Various summaries of the value of MID3 approaches to improved R&D efficiency have been documented (e.g., Reigner et al.29 1997; Chein et al.30 2005 [Lilly]; Milligan et al.31 2013 [Pfizer]; and Visser et al.32 2014 [Merck&Co/MSD]). Direct assessment of a return on investment (ROI) can be difficult because of the multifactorial nature of decision‐making and the level of information that pharmaceutical companies are willing to place in the public domain. However, there is both direct and indirect evidence from Pfizer that these approaches enabled a reduction in the annual clinical trial budget of $100 million and increased late‐stage clinical study success rates.31 Additionally, Merck&Co/MSD has reported significant cost savings ($0.5 billion) through MID3 impact on decision‐making.33

Going beyond internal decision‐making, MID3 has been shown to support regulatory assessment and decision‐making with respect to trial design, dose and schedule selection, and extrapolation to special populations and label claims. The US Food and Drug Administration (FDA) collated examples of impacts, where MID3 analyses enabled approval of unstudied dose regimens, provided confirmatory evidence of effectiveness, and utilization of primary endpoints derived from model‐based approaches.34 The EMA Modeling and Simulation Working Group has collated and published its own activities in this area.35

In summary, the business case for the adoption of MID3 has been established within the pharmaceutical industry. However, the ROI has not been fully determined and the breadth and degree of impact across R&D has not been clearly articulated. In order to address the last point, in the next section, we will provide a collection of examples highlighting both the breadth and depth of MID3 applications obtained from the scientific literature.

3.2. Examples of MID3 application and value to R&D

The value of MID3 approaches in enabling model‐informed decision‐making across drug discovery, development, commercialization, and life‐cycle management is evidenced by a large number of case studies in the public domain. In the Supplemental Materials that accompany this paper (Supplemental Table S1), we have compiled a list of ∼100 case studies sourced from the literature and the 2011 EMA/EFPIA M&S workshop.1 It is recognized that the collated examples are illustrative rather than exhaustive in terms of the nature and extent of current practice levels within the pharmaceutical industry. Nonetheless, our aim was to highlight the variety of application and impact on decision‐making.

Figure 1 illustrates eight application types, spanning a range of discovery and development phases and the associated shift from internal decision support to regulatory assessment and life‐cycle management.

Figure 1.

Range of application types spanning discovery, development, regulatory assessment, and life‐cycle management. The Shading indicates areas for which we have presented case studies which can be found in the supplemental material.

For each example, the specific (motivating) question to be addressed is shown, with a brief description of the work performed, and, where available, the resultant internal and regulatory impact (refer to the APPENDIX for a summary table describing the examples and the Supplemental Table S1 for the full set and the associated references). The examples are categorized by drug development phase, application type, and the MID3 approach used. Further details on the specific MID3 approaches used in these examples: empirical pharmacokinetic/pharmacodynamic (PK/PD), semi‐mechanistic PK/PD, empirical dose/time analysis, model‐based meta‐analysis (MBMA), and systems pharmacology, can be obtained in section 4. A high‐level textual summary of some of the key examples by application type are provided below.

1. Target authorization (increase confidence in target) and mechanistic understanding

MID3 can bring value to early discovery in the target identification and validation phase through developing and applying in silico approaches that integrate experimental data from multiple sources. These modeling approaches have been reported to increase the confidence in the role of the target in the disease or to add to the mechanistic understanding of target modulation. Such in silico pathway modeling approaches do not necessarily only need to occur at the stage of target selection and validation, but could also be applied to enable decision‐making at later stages of drug development.
Identify new targets	Benson Ex3a highlighted the use of a systems biology model for the identification of the most sensitive drug targets in the nerve growth factor pathway using a sensitivity analysis in a sophisticated systems model of the nerve growth factor pathway. The model indicated that after nerve growth factor itself, tropomyosin receptor kinase A was one of the most sensitive drug targets. Further exploration identified characteristics for a successful hypothetical tropomyosin receptor kinase A inhibitor.
Characterization of target mechanism	Jauslin Ex7 described the application of an integrated glucose‐insulin model, which was able to identify the dual mechanism of action of a glucokinase activator through quantitative evaluation of four possible mechanisms of actions. Simulations with this model enabled the selection of doses for dose‐finding trials, gaining more insight into a drug candidate's mechanism of action.

^aFor each example (Ex number) in this and subsequent sections, please refer to corresponding entry in Supplemental Table S1.

2. Candidate optimization, selection, human dose prediction

Model‐informed approaches can enable the quantitative assessment of PK, desired (PD), and undesired (safety) properties of novel molecules to support the benchmarking and selection of drug candidates for clinical development based on increased confidence in the projected efficacious dose and regimen before the first dose in humans.
Early projection of efficacious dose to select clinical candidates	Visser Ex11 and Bueters Ex10 have highlighted cases where a good quantitative understanding of the relation between in vitro and in vivo potency has been used to characterize and select new drug candidates based on in vitro data. In these cases, only the most promising candidate is progressed to in vivo testing. This can lead to more efficient progression of discovery programs through a reduced need for costly and time‐consuming in vivo experimentation, and enabling early projection of the human efficacious dose to assess viability of clinical candidates.
Determination of the minimum anticipated biological effect level	Yu Ex15 illustrated a mechanism based modeling approach to estimate the minimum anticipated biological effect level for a first‐in‐human study based on pharmacokinetics, receptor occupancy, and the dynamics of target cell depletion and recovery in cynomolgus monkeys for a novel biologic. Such predictions of human response and first‐in‐human dose for biologics can enable clear guidance on decisions to go into humans (Lowe Ex18).

3. Study design optimization

Model‐informed study design optimization is applicable, and has been demonstrated across drug discovery and development. The aim is to increase the efficiency and reduce costs of trials by establishing appropriate sample sizing and collecting relevant data at optimal time points to generate knowledge. Through simulations, candidate trial designs can be explored in order to select the best fit‐for‐purpose trial design, considering key assumptions and mitigating against uncertain factors at the same time.
Increase preclinical efficiency	Viberg Ex24 illustrated an improved preclinical design of behavioral pharmacology studies by combining three goals (dose‐finding, duration of effect, and tolerance‐development) into a single study facilitated by quantitative approaches in design and analysis. Additional benefits included substantial savings in animal lives, time, and cost while still delivering a good quality and precise description of the PK/PD relationship.
Increase early clinical efficiency	In early clinical development, Kretsos Ex28 showed that a model‐informed study design and adaptive trial execution resulted in faster, cheaper, and more informative clinical study. As a result, a precise and full characterization of the PK/PD profile of a novel drug in a disease‐relevant population could be done with a minimal number of patients.
Increase the probability of clinical trial success	Dodds Ex31 illustrated how MBMA can inform the clinical trial simulation used to explore the impact of design on the probability of making the correct decision in small first‐in‐patient trials. Simulations based on MBMA across available compounds, developed for psoriasis, showed that dose‐response designs were almost as efficient as maximum tolerated dose designs in determining the correct go/no‐go decision. Whereas the former also provides dose‐response information to help in future design.
Cost reductions for clinical trials	In a review, Milligan et al.31 demonstrated clear cost savings with regard to clinical trials for multiple therapeutic areas where model‐informed approaches utilizing integration of knowledge from previous trials resulted in substantial fewer patients and/or shorter duration of clinical trials.

4. Predicting and characterizing absorption, distribution, metabolism, and excretion, including intrinsic and extrinsic factors impacting PK variability (e.g., impairment of eliminating organs, DDIs, formulation changes)

In the characterization and prediction of human absorption, distribution, metabolism, and excretion (ADME) and understanding PK variability, multiple modeling approaches can be applied. These methods help identify significant covariates that determine expected exposure and the need for dose adjustment in specific subpopulations.
Extrapolation from HV to patients	Dansirikul Ex43 investigated the comparability of the PK and PK/PD relationship of dabigatran between healthy subjects and patients with atrial fibrillation or undergoing orthopedic surgery. The analysis demonstrated that only renal function has a clinical relevant impact on the PK of dabigatran and that the PK/PD relationship is not affected by any factor tested (age, sex, renal function, and concomitant aspirin treatment) in a clinical relevant way. This analysis facilitated the use of data across multiple indications in submission and label documents.
Prediction of in vivo performance of oral dosage forms	Kostewicz Ex 33 reviewed various physiologically based pharmacokinetic (PBPK) modeling approaches that provide an approach to predict the human plasma concentration–time profiles from preclinical in vitro and in vivo data and can thus provide a valuable resource to support decisions at various stages of the drug development process. Such an approach was exemplified by Malmborg Ex 35, in which human PK predictions were made for prodrug conversion into active drugs utilizing PBPK combined with IVIVC, thereby improving decision‐making on prodrug candidates.
Integrated IVIVc development to avoid BE studies	Another application is the use of modeling to demonstrate bioequivalence without actually performing the confirmatory clinical study. Soto Ex48 illustrated that an IVIVC could be established based on data from one immediate‐release and three slow‐release formulations. Validation was done with a fourth slow‐release formulation. Such IVIVC has the potential to avoid bioequivalence studies in situations defined in the FDA guidelines for scale‐up and post‐approval changes (FDA SUPAC‐MR Guideline).
Prediction of clinical equivalence	Vargo Ex44 demonstrated through an MBMA that low‐density lipoprotein‐cholesterol reduction was more correlated to dose than peak statin exposure (peak plasma concentration [Cmax]). This allowed prediction that a near miss on bioequivalence for Cmax for ezetimibe+ atorvastatin fixed‐dose combination FDC vs. coadministration would not impact low‐density lipoprotein‐cholesterol efficacy. Although the work did not negate the need for a comparative effectiveness trial, it did enable the optimal design of the comparative effectiveness trial and an accurate prediction of the outcome.

5. Risk/benefit characterization and outcome prediction from early clinical responses

Characterization of benefit and risk is an ongoing process that occurs all along the compound development path. Although a compound's viability can directly be affected by the emergence of critical data (e.g., a clear safety signal or lack of efficacy at proof of concept), often emerging evidence is integrated into existing knowledge to inform or optimize risk/benefit at the next development stage. MID3 approaches are used here to predict clinical outcome measures based on modeling of preclinical and early clinical data, to: optimize a dosing strategy, estimate a therapeutic window by integrating related safety and efficacy endpoints, provide de‐risking alternative development approaches, and support the selection of optimal candidate compounds. Although most often the design at subsequent stages is impacted, the primary endpoint or the integration of benefit risk may itself be primarily model informed.
Assessing QT liability	A potential QT liability poses a constant threat to many development programs, given the risk it can pose to patients. A modeling approach to integrate the QTc signaling data from preclinical and early clinical studies helps to characterize the extent of the effect and therefore the potential risk. In the examples from Parkinson Ex51 and Chain Ex53, a range of QTc empirical modeling approaches were pursued to quantify a compound's QTc liability by translating and predicting a clinical effect from preclinical experiments performed in dogs. Both examples highlight that a model‐informed understanding of the risk for QTc prolongation can support early go/no‐go decisions.
Predicting clinical outcome for safety and/or efficacy markers	A semimechanistic approach was explored in Soto Ex57 to predict neutropenia for drug combinations based on results from monotherapy studies. The predicted additive neutropenic effects were confirmed in further clinical investigations, thereby encouraging the use of the model‐based approach to optimize the design of drug combination therapy studies in oncology. Kjellsson Ex60 presented an integrated glucose, insulin (IGI) model, and glucose‐red blood cell‐HbA1c (IGRH) model that allows prediction of the formation of HbA1c from average glucose concentrations. These results showed that a 12‐week HbA1c status in patients with diabetes could be predicted from 1‐week studies on glucose and insulin measures. Høivik Ex61 took the use of predictive models for outcome one step further in their proof of concept trial. This used an adaptive longitudinal logistic model for the primary migraine attack endpoint. The trial tested whether the predicted selective inducible nitric oxide synthase inhibition from a preclinical biomarker would lead to a reduction in migraine events. Because the trial failed to show a treatment difference to placebo, the development program was discontinued.
Decisions on risk/benefit predictions	Burghaus Ex62 described a physiology‐based modeling approach that allowed therapeutic window predictions from a clotting cascade model to assess risk (bleeding) vs. benefit (thrombosis prevention) for rivaroxaban. Doses selected for late clinical trials were predicted to fall within the computer‐simulated therapeutic windows, and their risk/benefit was subsequently confirmed in phase III programs in the treatment of venous thromboembolism, supporting the current label. Milligan et al.31 presented two case studies in which a model‐based risk/benefit assessment was done. In an example on endometriosis, a systems pharmacology model of the female menstrual cycle (benefit) and a multiscale systems model of calcium homeostasis (risk) indicated that targeting the gonadotropin‐releasing hormone pathway to achieve the desired range of serum estrogen levels with minimal impact on bone mineral density would be difficult to achieve; therefore, the research program was halted before any compound entered the clinic (see also Riggs Ex56). In another example, a prospective modeling approach was used to select the appropriate tofacitinib dose based on efficacy and safety using probability of technical success as a common metric allowed demonstration of a positive benefit/risk profile with the desired product attributes. This proposed dose was approved by the FDA in 2012.

6. Dose and schedule selection and label recommendations (including drug combinations)

Successfully determining the recommended dose and dose regimen, which provides optimal treatment benefit for the majority of patients, and establishing the supportive evidence sufficient to meet the requirements for all internal and external stakeholders is a challenging aspect of drug development. Although it is accepted that MID3 approaches are useful in predicting alternative doses or regimens, which will be tested and eventually confirmed in later trials, emphasis and regulatory interest has most recently shifted to the ability of using MID3 to interpolate or extrapolate from tested doses and dose regimens to a more optimal choice without the need to retest. This is especially important when additional studies are difficult to conduct or time spent in confirmatory trials could significantly delay access to important new treatments.
Dose and schedule selection in early development	In the early stages of drug development, modeling approaches can be used to determine early development dosing strategies. For example, Higgins Ex66 reported a case study on using a modeling approach to determine the feasibility of intermittent dosing to guide the selection of initial phase I scheduling regimens for a MDM2 antagonist in antitumor therapy with high selectivity and potency. Intermittent regimens instead of chronic administration were suggested for the initial clinical testing to circumvent tolerability issues. Similarly, guiding of dose‐selection was illustrated by Hoeben Ex69 for a D2‐receptor antagonist. A population PK/PD model, including relevant covariates, was developed on early clinical trials, which was then used to simulate D2‐receptor occupancy in support of dose selection for subsequent studies.
Dose and schedule selection in late development	The assessment of impact of covariates on dose regimens can often be better assessed during later stage development. Roy Ex72 reported the model‐informed evaluation of a body weight–tiered dosing regimen for abatacept used for the treatment of rheumatoid arthritis. Simulations demonstrated that body weight–tiered dosing was desirable to ensure consistent steady‐state abatacept exposure and optimal IL‐6 suppression. Hsu Ex68 highlighted the use of M&S in developing robust and sensible trial designs and analytical approaches during development. In this case study, an MID3 approach was applied with the main objective of selecting a dose for phase III studies using data obtained on a chronic obstructive pulmonary disease drug in phase II and using forced expiratory volume in 1 second as the primary endpoint.
Informing the label	Sahota Ex70 evaluated, through modeling, the impact of body weight on the overall systemic exposure to pyrazinamide for treatment of tuberculosis and demonstrated that the use of a fixed dose of pyrazinamide in co‐formulation or combination with novel therapeutic agents would still ensure target exposure and safety across the overall patient population.Cosson Ex71 illustrated, through two examples, the value of MID3 in supporting the approval of non‐tested dosing regimens. In both cases, the knowledge of the exposure‐response relationship was considered sufficient to rely on M&S approaches to investigate new doses. Thanks to well‐characterized exposure‐response relationships, it was possible to demonstrate using trial simulations that the proposed new dosing schemes ensure good efficacy and manageable safety risk. The need to run additional confirmatory trials was avoided.

7. Comparator/standard of care differentiation and commercialization strategies

MID3 approaches to quantify comparator and standard of care therapeutics can be exploited to understand the potential for differentiation of a new chemical entity and its clinical and commercial viability. In the discovery phase, clinical information on comparator and standard of care can be useful in the translation of preclinical effects of novel compounds acting through related mechanisms. During clinical development, comparator study results can enable prediction of long‐term study outcome based on short‐term studies. Often, the MID3 approaches provide a good basis to optimize late‐stage clinical development and support trial designs. In addition, model‐based meta‐analyses can provide essential evidence on the viability of a candidate or can help relate pivotal trial results to clinical practice.
Preclinical target setting through benchmarking of clinical comparator effects	Olsen Ex81 illustrated that the implementation of PK/PD studies in preclinical drug development may serve to accelerate the overall process by enabling earlier identification of suitable doses for phase I and II studies. A correlation was demonstrated between dopamine D2 receptor occupancy levels providing 50% response in conditioned avoidance response tests in rats and the dopamine D2 receptor occupancy levels reported from responding patients with schizophrenia treated with antipsychotics. This approach enabled the prediction of therapeutic effective steady‐state plasma levels in patients based on preclinical PK/PD results in behavioral animal models, under the assumption that there are no interspecies differences in the plasma/brain ratio.
Predicting long‐term outcome from early clinical studies	Møller Ex89 developed a model describing the dynamic relationship between mean plasma glucose and HbA1c after various antidiabetic treatments based on 12‐week data with the aim to predict HbA1c at end‐of‐trial (24–28 weeks) with high accuracy. Numerous phase III and IV trials of already approved drugs were used to qualify the model. Furthermore, HbA1c reductions relative across comparators were accurately predicted providing a good basis to optimize late‐stage clinical development for novel treatments within diabetes.
No‐go due to limited commercial viability	In an example from Demin Ex88, a quantitative assessment of the longitudinal time course of the clinical efficacy for a set of therapeutic agents for rheumatoid arthritis was to support decisions in the development of a novel biologic, canakinumab, during early clinical development. This MBMA integrated in‐house clinical data of canakinumab, and published clinical data from marketed treatments. The analysis results showed that, although it showed positive results according to the primary statistical analysis, the candidate could not provide an improved benefit to patients compared with the existing therapeutic option and thus could support discontinuation.
Head‐to‐head comparison without a trial	Mandema Ex86 studied the therapeutic benefit of eletriptan compared to sumatriptan for the acute relief of migraine pain and the implications of encapsulation of sumatriptan on the time‐course of the effect through an MBMA. The summary data from 19 randomized placebo controlled trials was used to analyze the data across compound dose and time. Although confirming the benefit of 40 mg eletriptan over sumatriptan, the analysis did not detect an impact of encapsulation on the treatment effect, which answered a key scientific and commercial question on the validity of the phase 3 head‐to‐head trial results where sumatriptan was encapsulated.

8. Patient population selection and bridging between populations (such as pediatrics, elderly, and obese patients)

Over the years, evidence has been built on the utility of MID3 approaches to bridge PK, efficacy, and safety to special populations and provide recommendations for subpopulations identified from larger populations (e.g., renally or hepatically impaired, elderly, obese, subgroups) based on genetics or phenotyping. In these cases, knowledge captured in the form of models can be applied by simulation and prediction to substitute for additional data (where there is low risk and significant time required to generate confirmatory data) or where substantive data is considered impossible to generate. In pediatrics, this led to an acceptance of using MID3 approaches, in which the evidence in adults is bridged to children. There are now an abundant number of pediatric development plans that utilize MID3 approaches to different degrees. Similarly for rare diseases, when potential of collecting clinical trial data as the primary source of evidence for risk/benefit is limited, the integration of all existing knowledge through MID3 approaches is the only way to de‐risk the initial development program. In such a case, given the medical need and limited patient population, there is increased emphasis on efficiency in evaluating the potential of a new treatment to facilitate both internal and regulatory decision‐making.
Pediatric dose recommendation	Willmann Ex92 used a PBPK model involving anthropometric and physiological information to predict age‐dependent clearance and protein binding changes to justify pediatric dosing for rivaroxaban, resulting in a dose recommendation for children larger than an mg/kg based dosing would have suggested. The pediatric PBPK model developed informed the dosing regimen for a clinical study in pediatric patients.
Building supporting evidence for pediatric dose‐response characterization	The Frey Ex95 and Harnisch Ex94 examples were discussed as part of the EMA workshop of adult and pediatric data integration in which the provision and analyses of the combined evidence became essential in regulatory decision‐making. In Frey's example, the prediction of a lower clearance led to a different dose in children, which was subsequently tested as part of the phase III program for tocilizumab leading to a higher than pre‐model based planned pediatric dose, as per the European and the US label. In Harnisch's example, the integration of adult and pediatric patient data across PK, hemodynamic, and clinical endpoints led to a model‐based dose recommendation, adopted by the EMA for the treatment of pulmonary arterial hypertension with sildenafil. In both examples, data from pediatrics alone would have been insufficient to characterize the dose response, but an MID3 approach provided regulatory agencies sufficient evidence to approve dose recommendations.
Dose recommendation for renally impaired patients	In the examples from Lehr Ex103 and Kleijn Ex100, similar approaches were pursued to integrate a complex database into empirical PKPD models to predict dose regimens. In the former, severely renally impaired patients treated with a reduced dose were predicted to have exposure data largely within the concentration range proven to be safe and effective in patients with atrial fibrillation and creatinine clearance above 30 mL/min. In the Kleijn example, the interaction with sugammadex and the reversal of the neuromuscular effects of rocuronium in the pediatric and elderly population was modeled. The combined model provided evidence that age, but not renal function, influenced reversal time (most important PD parameter), but not to a clinically relevant extent).

9. Summary of application and value

In summary, the collated examples demonstrate how MID3 has been applied across R&D to increase the confidence in the compound, mechanism, or disease rationales; provide support to internal “go/no‐go decisions,” dose finding, dose adjustments for patient subgroups; support labelling, benefit‐risk, and increasing confidence in next‐stage investment. More examples and details can be found in the supplemental material. In general, the additional information via these approaches provides an “evidence base” to help make decision‐making informed and efficient. However, there remains a need to more succinctly capture the internal impact of MID3 applications in the future. This is discussed along with current MID3 challenges and associated opportunities in section 4.

4. “WHAT” MID3 PREMISE AND APPLICATION FOR PRACTITIONERS

4.1. MID3: genesis of terminology

In this document, we use MID3 as a holistic term to characterize a variety of quantitative approaches used to improve the quality, efficiency, and cost‐effectiveness of decision‐making through “fit‐for‐purpose” data analysis and prediction. Several terminologies have been used in the scientific literature to describe many of these quantitative approaches and in this section we detail the evolution of MID3 and its relationship to these existing terminologies (Figure 2).

Figure 2.

Model Informed Drug Discovery and Development (MID3): genesis of terminology.

M&S is the most established umbrella term to capture a wide range of quantitative approaches and it has been used extensively for decades. A major limitation, however, is that it is too general a term to be informative. The use of M&S makes it explicit that a quantitative act has taken place but it is not clear what particular approach was adopted and what was its context of use.

In order to address this limitation, refinement of M&S into two, potentially more informative, quantitative method subsets (1) systems pharmacology and (2) pharmacometrics, has occurred: (1) systems pharmacology aims to inform therapeutic interventions based on detailed structural knowledge of biological systems.36 A feature of systems pharmacology is that it focuses on the interplay between pharmacology and the underlying system, allowing predictions of the efficacy and safety of compounds to be based on known or possible mechanisms of action. This approach can enable forward and backward integration of information across R&D programs (including backup projects) and rationalize compound and target selection;1, 37, 38 (2) pharmacometrics has been defined as “the intersection of PK models, PD models, PD–biomarker–(clinical) outcome models, data visualization, statistics, stochastic simulation, and computer programming.”39 Mathematical models are developed and applied to characterize, explain, and predict PK and PD behavior of therapeutic agents. These are combined with statistical models to quantify the source and the extent of variability and uncertainty in the underlying responses. The ongoing evolution of pharmacometrics into a distinct multifaceted field has previously been described.40

One remaining limitation is, whereas the terms system pharmacology and pharmacometrics provide more information to the analyst, they do not help to raise awareness amongst “decision‐makers” as to their appropriateness (in terms of inherent strengths and weaknesses) for use within a particular context. This lack of decision‐maker awareness can act as a disabler for their use and can reinforce both conservatism and empiricism in the analysis and data integration methods adopted. In order to address this limitation, over the last decade, the use of M&S within the drug development setting has become synonymous with model‐based drug development31, 41 and recently model‐based drug discovery “Ex 11” in which the context of use is explicit.

The most recent terminology evolution attempted to address any potential negative connotations associated with “model based” activities. An unintended implication was that the model dominated the decision‐making process, disenfranchising the decision‐maker. The more appropriate term “model‐informed drug development” mitigates this potential misunderstanding.42, 43, 44

However, it is our belief that in order to reflect the “true” potential of M&S to inform the typical decisions associated with discovery, development, regulatory domains (and also by extension real‐world effectiveness, pharmacoeconomic and health technology applications), an enhancement of model‐informed drug development to become MID3 is fully encompassing and more accurate.

The addition of “discovery” to become MID3 specifically highlights the growing application of particular quantitative approaches in the assessment of pathways considered relevant (for physiology and/or pathology), in particular, indications and the comparative properties of both existing and new drug candidates that interact with and modulate these pathways.

4.2. Premise for the use of MID3

MID3 in its simplest form embodies using “fit‐for‐purpose” mathematical models, implemented according to good practices, in order to enhance the extraction of inference from both existing information and data emanating from ongoing experiments. As the underpinning foundations for MID3 are based on robust scientific principles derived from pharmacological, physiological, and pathological processes (the domain sciences), MID3 can more effectively support translation across, and extrapolation beyond, the direct inference obtained from standard descriptive methods applied to experimental data. The MID3 “quantitative framework” is summarized diagrammatically in Figure 3 and detailed further in the sections below.

Figure 3.

MID3: a quantitative framework for prediction and extrapolation centered on knowledge and inference generated from integrated models of compound, mechanism, and disease level data aimed at improving the quality, efficiency, and cost‐effectiveness of decision‐making. The colored boxes represent essential components of the “Learn and Confirm Cycle”. The arrows represent processes that link these components.

The colored boxes represent essential components of the “Learn and Confirm Cycle.” The arrows represent processes that link these components.

4.2.1. Learning and confirming paradigm

The strategic value of MID3 within R&D is its support to decision‐makers at portfolio, project, and study levels. This is achieved by operating in a manner that corresponds to the learning and confirming paradigm introduced by Lewis Sheiner.22 This paradigm provides the foundation of this section and is the central tenet of the MID3 approach, as outlined in Figure 3. It is implicit that the establishment and maintenance of the quantitative framework is iterative in nature with inconsistencies between predictions and subsequent observations/outcomes triggering further model development/refinement and/or modified assumptions and evaluation. In this process, any new data or assumptions introduced into the framework can be evaluated, “confirmed,” or at least assessed in terms of their impact on the resultant inference.

4.2.2. The “data” step

Determination of the specific questions or knowledge gaps to be addressed is essential before identifying the relevant data and information to be acquired and aggregated. Depending on the context of use, asset, mechanism, or disease level data could be sourced. Examples of some specific questions that commonly arise within discovery and development and a consideration of effective planning and the integration of multilevel activities are detailed in “MID3 challenges and opportunities.”

4.2.3. The “knowledge” step

In MID3, models are used to represent the relationship between independent variables (e.g., time and dose), endpoints (dependent variables) and covariates and (e.g., demographics, genetic factors, phenotypic factors or concomitant medications) utilizing mathematical equations and statistical principles. The nature of these relationships (i.e., their structure, variability, and uncertainty), may be informed by: (a) prior knowledge and established assumptions based on the known pathology, physiology, and pharmacology; and (b) additional new biological (or statistical)‐based assumptions postulated in the absence of precedence or due to limitations in the ability to generate sufficient evidence from available data.

The modeling process necessitates an assessment of the consistency and the “fit‐for‐purpose” nature of any new (and therefore) untested assumptions. In model evaluation, the inability of a given model to adequately characterize existing data, generate insight into limitations in the current level of understanding. Inconsistencies can also trigger the consideration of alternative models, assumptions, and experimental efforts in an iterative manner to rationalize and mitigate the nature and extent of these limitations.

This process increases R&D efficiency as any new experiments or trials will be more systematically informed by current knowledge derived from relevant pharmacology, pathology, and physiology. Furthermore, greater incorporation of these domain sciences into the MID3 quantitative framework can provide a more comprehensive and precise characterization of a compound's benefit/risk assessment required during the life cycle of any medicinal product.

In addition to any ongoing model development evaluation, judging the model's performance against preset criteria is imperative in establishing that it is fit for the intended purpose. Qualification of the model is part of both the model development and evaluation process and the inference step, as it requires logical conclusions to be made about the adequacy of the model with respect to the current data (internal qualification) or other appropriate data not used in the model building (external qualification).

4.2.4. The “inference” step

As outlined in “Examples of MID3 application and value to R&D,” the R&D process is composed of a succession of distinct but related phases and associated transitions from target identification to lead generation and optimization, from lead generation and optimization to preclinical development, etc.

Integrated models, within a quantitative framework, based on robust physiological, pathological, and pharmacological principles (i.e., MID3) can best support these transition decisions as they enable appropriate inferences to be drawn in the light of all available evidence thereby increasing the likelihood of a “correct” transition decision. As discussed in “MID3 challenges and opportunities,” these decisions also have the highest business impact, as they have profound resourcing implications (cost of opportunity).

Conversion of the current knowledge captured within the “fit‐for‐purpose” model into inference requires a prediction based either directly on particular model parameter estimates or utilizing values generated through simulation. Predictions can either be interpolative or extrapolative with respect to available evidence and the intended purpose. As discussed previously, extrapolations beyond current experience often provide the greatest value to pharmaceutical companies. The recent EMA concept paper on The Extrapolation of Efficacy and Safety Data in Medicine Development7 identified the approaches utilized in MID3 as part of the extrapolation concept and for inclusion in an extrapolation plan. The use of extrapolations emanating from an appropriate quantitative framework to bridge efficacy and safety in special populations was discussed in the 2011 joint workshop¹ with some of the resultant proposals subsequently published in greater detail.5 Replacement of direct experimental evidence (including all or part of a clinical trial) in a development program is conceptually “permissible” but considered to be of “high regulatory impact”2 necessitating substantive a priori discussions with the regulatory agencies to characterize the context of use for any resultant extrapolations.

4.3. MID3: IMPLEMENTATION STRATEGY

For MID3 to inform the variety of decisions taken during the course of R&D, it is essential that a strategic plan is first established after appropriate multidisciplinary and stakeholder discussions outlining objectives, methods, assumptions, deliverables, resources, and timelines. The MID3 strategic plan captures the important knowledge management and MID3‐related activities undertaken in parallel, or in sequence, with the study‐related activities performed for each drug discovery and development milestone. The MID3 plan must be complementary to appropriate product concepts and target profiles and any additional relevant indication‐specific elements, such as endpoints, time points, standard of care, and competitor attributes, etc. It is recommended that objectives for any particular analysis or set of analyses should consider the range of activities necessary across compound, mechanism, and disease levels and not be restricted to solely compound level activities (this aspect introduced in “Premise for the use of MID3” will be discussed further in this section).

The MID3 strategic plan should be fully integrated with the related “general” drug discovery and clinical development plan. The MID3 plan should also identify knowledge gaps that could be adequately filled by alternative sources (scientific literature, clinical practice database, precompetitive collaborations, etc.), thereby avoiding or mitigating the need for “new” data generation. The MID3 strategic plan needs to consider the balance and purpose of any new data generated. On the one hand, there will be a need to generate sufficient empirical evidence derived from appropriate study designs, whereas, on the other hand, there will be a need for emerging data to enable a thorough evaluation of influential assumptions used to fill important knowledge gaps pertinent to MID3. Only fully integrated plans can serve to highlight the interdependence between experimental conditions, data generation, evidence base generation, and MID3. For example, MID3 may provide the important justification for the development of particular biomarkers or may highlight the benefit of adaptive design to more effectively fill the knowledge gap.

An important goal for these integrated plans is to consider the trade‐offs in time and expense of additional data generation to facilitate MID3 vs. the potential downstream costs of relying on untested and potentially flawed assumptions or indeed vs. an overreliance on inferences obtained principally through direct experimentation.

To that end, we propose that an MID3 strategic plan should be constructed through consideration of pertinent R&D questions that commonly arise with respect to some prominent themes, as illustrated in Table 1:

• Medical need/commercial viability – R&D questions related to the understanding of medical need and areas of potential differentiation from the standard of care for a particular disease/indication. These can inform the likelihood of a particular compound achieving the important aspects of a product profile at each development stage.

• Efficacy – R&D questions related to the characterization the dose‐exposure–response relationship for important efficacy outcomes.

• Safety/tolerability – R&D questions related to the characterization the dose‐exposure–response relationship for important safety/tolerability outcomes.

• Pharmacokinetics – R&D questions related to the characterization and extrapolation of the pharmacokinetic properties of a drug across species and patient populations, the general expected impact of a progressive disease state, intrinsic (e.g., age, organ impairment), or extrinsic factors (e.g., coadministered drugs), influence of formulation, or administration method on drug exposure, etc.

• Benefit/risk – R&D questions related to the definition and quantification of the relative trade‐offs between important efficacy and safety outcomes in order to determine optimal dose regimens that are sufficiently effective and safe.

• Clinical viability – R&D questions related to the assessment of potential development programs for a particular indication, considering options with respect to populations, subpopulations, inclusion/exclusion criteria, etc.

• Study design – R&D questions related to the optimum design of the subsequent studies, balancing the cost and time of the current study vs anticipated future risk given the predicted confidence in achieving the required product profile.

Table 1.

Good practice grid with example questions and activities to aid MID3 strategic plan development

Key themes	Disease level		Compound level		Additional mechanism‐level considerations
	Relates to questions and activities that can be answered in advance of development of any particular compound.		Relates to questions and activities that focus on the data associated with a compound of interest.		Relates to compound‐level or disease‐level questions and activities where there is a focus on the MoA (or in the case of PK ADME properties) and knowledge gained from other compounds with a similar MoA (ADME).
	Example questions or required knowledge?	Examples of proposed activities	Example questions or required knowledge?	Examples of proposed activities	Example questions or required knowledge?	Examples of proposed activities
Medical need/commercial viability	How do we quantify the medical need and turn this into a target for future development? What gains in the risk–benefit profile would be required to drive cost‐effectiveness given expected treatment costs vs standard of care?	Develop quantitative product concept utilizing above disease‐level models, MBMA of SOC compounds, knowledge management and commercial/regulatory insight. Define the minimum improvements (target values over SOC) required to meet medical needs Develop HTA model that integrates trial outcome with real‐life data and conduct sensitivity analysis to determine likely improvements required to drive a gain in QALYS vs. expected cost.	What is our confidence in this compound at this stage in development vs. quantitative product concept?	Deterministic and stochastic simulation utilizing the efficacy and safety models (including competitor data) to determine probability of meeting product profile based on associated target values.	How could we optimize the probability of achieving the target product profile with this MoA?	Simulate impact of changes to population/background therapy to commercial viability vs. quantitative product profile optimized to MoA. Consider differentiation characteristics across compounds with same MoA and impact of changing route of administration/combination and add‐on approaches.
Efficacy	What is the dose (exposure)–response relationship for compounds used to treat this disease? What is the dropout rate in the target population for known compounds/placebo and how does it impact efficacy? What is the rate of progression of disease and how do current agents/placebo alter this rate (symptomatic (offset) vs. disease modifying)? What is the linkage between short‐term endpoints and long‐term outcome? What is the optimal point in this pathway to inhibit/activate? Based on current knowledge of the pharmacological target, can we expect genetic variation in response (nonresponders)?	Develop a database of literature for known compounds/placebo used in the disease/indication of interest. Undertake exploration and MBMA analysis. Include information on dropout and imputation in the meta‐analysis database and utilize as covariate in efficacy analysis. Develop a disease‐progression model based on prior literature and preferably individual‐level data. Quantify linkage between mechanism‐specific biomarkers to clinical outcome and early‐stage outcome to late‐stage outcome. Develop system pharmacology model focused on pathway of interest within the wider context of the disease process. Utilize systems pharmacology model to investigate impact of known polymorphisms leading to changes in expression of key proteins in the pathway of interest.	What is the exposure–response relationship for the key endpoint(s) over time? What is the optimal dose/regimen combination that can achieve a desirable exposure profile? Can we predict long‐term efficacy from short‐term endpoints? What is the link between the marker of target engagement and efficacy?	Exposure–response modeling. Exploratory and formal covariate analyses. Combine compound level exposure response model with disease level model based on prior compounds. Incorporate sufficient mechanism into the linkage between exposure and outcome. Consider multiscale modeling approach depending on complexity of system and response.	What is the level of target inhibition/modulation needed to demonstrate efficacy? How is duration and intensity of the inhibition/activation related to dosing interval to achieve adequate efficacy?	Develop a database of literature for known compounds with similar or related mechanisms used in the disease/indication of interest. Develop a mechanistic or semimechanistic PK/PD model using biomarkers and known physiology.
Safety/tolerability	What was the onset and time course of this emerging AE profile for previous compounds/placebo in this indication? Also questions posed for efficacy above apply here but with respect to safety and toleration.	Develop a database of literature for known compounds/placebo with safety or toleration signals of interest. Undertake exploratory analysis and MBMA analysis. Activities for efficacy above apply here but with respect to safety and toleration.	What is the exposure–response relationship for the emerging safety signal?	Exposure–response modeling. Exploratory and formal covariate analyses.	Would dose titration improve the toleration profile?	Development of mechanistic or semimechanistic PK/PD model using safety biomarkers. Quantify linkage between mechanism‐specific safety biomarkers to outcome for compounds with a similar mechanism.
PK	What is the impact of disease on ADME processes?	PBPK model (as described under compound‐level activities) needs to include the pathophysiological link to ADME in order to be useful at the disease level or be dropped from the plan if not feasible.	What is the predicted PK profile in man based on preclinical data? What is the predicted PK in the target population based on HV human data? What is the magnitude of the underlying covariate influences on drug exposure?	PBPK model (based on knowledge of physiochemical properties, in vitro metabolism, and emerging metabolite profile) or semimechanistic model needs to include patho‐physiological properties and translational ADME. Pop PK analysis and/or meta‐analysis of NCA‐parameter estimates factoring in impact of disease. Exploratory and formal covariate analysis.	Can the expected nonlinear PK be predicted or explained? Based on prior compounds with a similar ADME profile, what DDI or genetic variations do we expect?	Develop mechanistic PK model that includes system components, which may be particularly important given knowledge of a compound's likely disposition characteristics (e.g., include saturable component for absorption for compound that is a Pgp substrate or include TMD component for monoclonal antibody etc. Using PBPK or semimechanistic analysis to investigate potential for DDI based on prior in vitro data and translation to DDI for other compounds with similar ADME profiles.
Risk/benefit	How do we balance safety and efficacy for this disease?	Develop clinical utility index to weight risk/benefit for disease state given clinical profile. Consider differences in regulatory, patient, and prescriber weighting factors.	What is the therapeutic window for this compound across the dose range?	Deterministic and stochastic simulations utilizing the above models to understand risk/benefit across exposure range and consider strategies to optimize it.	Does our compound differentiate with other compounds with a similar MoA?	Look at comparative risk/benefit vs. compounds with similar mechanism considering influential covariates utilizing established models.
Clinical viability	What is the best generic development program for this disease/indication/product concept? What population/subpopulation could be targeted?	Explore development program options optimized to de‐risk uncertainty in efficacy, safety, and toleration in disease of interest for typical compound. Use simulation to estimate optimal approach to de‐risk compound early.	What is the best development program for this compound in this disease/indication from its current stage given available data?	Estimate PTS at the next point in the development plan using simulations based on models for compound using target values and consider alternative development paths.	How could we optimize this probability of program success for this MoA?	Consider impact of inclusion/exclusion criteria given MoA on PTS.
Study design	What would be considered as success in the next trial to be designed? How can we reduce trial size based on prior knowledge of placebo and comparators? What is the best target population for this trial at this milestone?	Develop detailed trial‐level decision criteria for the next trial based on target values. Based on MBMA work, consider development of informative priors for active and placebo arms. Optimize inclusion criteria (baseline value, demographic, and disease factors, including pharmacogenetics). Consider need for trial success vs. need to make a correct decision with regard to future use of the drug in a wider population.	What is the optimal dose, regimen/schedule and sample size for this trial given understanding and uncertainty in risk/benefit across the dose range? What is the value of more advanced trial designs, titration, adaptive design given risk/benefit and associated uncertainty?	Estimate probability of making the correct decision and trial success given compound characteristics, decision criteria, and study design. Optimize by iteration. Estimate probability of making the correct decision and trial success given compound characteristics, decision criteria, and study design.	How could we optimize this probability of trial success for this MoA?	Consider impact of inclusion/exclusion criteria given MoA on probability of making the correct decision for current and future trials (e.g., impact of enrichment based on biomarker that is related to the MoA).

ADME, absorption, distribution, metabolism, and excretion; AE, adverse event; DDI, drug‐drug interaction; HTA, health technology assessment; MBMA, model‐based meta‐analysis; MoA, mechanism of action; NCA, non‐compartmental analysis; PBPK, physiologically based pharmacokinetic; Pgp, P‐glycoprotein; PK, pharmacokinetic; PK/PD, pharmacokinetic/pharmacodynamic; PTS, probability of technical success; QALYs, quality‐adjusted life years; SOC, standard of care; TMD, target mediated disposition.

As an aid to MID3 strategic plan development, in Table 1 we provide some example R&D questions with respect to these themes at the compound, mechanism, and disease/indication level in order to promote effective dialog during plan creation and enable adoption of good practices. The ordering of the columns is intended to support development of strategies initially at the disease/indication level. Similarly, the ordering of the rows reflects the common situation in which generic product concepts are first established before identification of pathways, targets, and specific compounds. However, MID3 strategic plans can also be initiated at alternative levels (e.g., additional indications for an established drug may consider mechanism and clinical viability‐related questions, initially).

In developing the MID3 strategic plan, the nature, extent, and priority of these R&D questions should guide the choice of quantitative approach for a particular activity, the necessary compound, mechanism, disease/indication level activity, and the preferred sequence of all these activities. The span of MID3 covers a wide variety of quantitative approaches. A high‐level description of some of the most common quantitative approaches along with some standard literature references, our view of the current status of their use in different types of decision‐making, and links to some associated examples are provided in Table 2.

Table 2.

Comparison of different MID3 modeling approaches

	Empirical dose/time analysis	Empirical PK/PD	MBMA	Semimechanistic PK/PD	Systems pharmacology modeling and PBPK
General description	Data‐driven statistical models that integrate PD data across doses (or average exposures) and/or time. These are established using empirical functions models with no or limited assumptions related to underlying pathology, physiology, or pharmacology.	Standard PK/PD modeling where models are established based on available data. General PK and PD principles are utilized in model development.	Estimation of underlying efficacy and/or safety effects through combination of direct or indirect treatment comparisons of summary statistics taking into account the impact of treatment, patient population, and trial characteristics. This type of analysis can help to estimate the probability that a drug is superior to its competitors in the same drug class or across drug classes. Use for the assessment of the comparative risk benefit of compounds of interest.	PK/PD modeling where models are established based on “known mechanistic understanding” of biology and pharmacology. Most often this will utilize data from different sources (e.g., separate clinical and preclinical studies with different endpoints). Knowledge from one data source will be used to add mechanistic understanding in the interpretation of another data source. Although the model structure and some parameters are derived based on mechanistic understanding, the model is fitted to available data.	Physiologically based or multiscale models that are established based on wide variety of data sources. “Parameters” from these data are extracted via separate analyses and combined together in a mechanistic framework. Multiscale models link target to outcome via modelling that scales from target level interaction to cellular and whole body processes utilizing the understanding of the biology, physiology, PK, pharmacology, and pathology.
Tutorial/review/perspectives/white papers	EMA. Qualification opinion and comments on MCP‐Mod as an efficient statistical methodology for model‐based design and analysis of phase II dose‐finding studies under model uncertainty (2013).45 Bornkamp et al.46 Innovative approaches for designing and analyzing adaptive dose‐ranging trials (2007). Bretz et al.47 Dose finding ‐ a challenge in statistics (2008). ICH Topic E4: Dose‐response information to support drug registration (1994).48 Holford et al.50 Disease progression models (2006). Mould.49 Models for disease progression: new approaches and uses (2012). Overgaard et al.51 Establishing good practices for exposure–response analysis of clinical endpoints in drug development (2015).	Holford et al.52 Pharmacokinetic and pharmacodynamic modeling in vivo (1981). Meibohm et al.53 Basic concept of PK/PD modeling (1997). Holford et al.54 Understanding the concentration‐effect relationship: clinical application of pharmacokinetic‐pharmacodynamic models (1981). FDA Guidance for Industry Population Pharmacokinetics: Center for Drug Evaluation and Research (CDER) & Center for Biologics Evaluation and Research (CBER) (1999).15 EMA Guideline on reporting the results of population pharmacokinetic analysis CHMP (2007).17 Mould et al.55 Basic concepts in population modeling, simulation, and model‐based drug development (2012). Mould et al.56 Basic concepts in population modeling, simulation, and model‐based drug development‐part 2: introduction to pharmacokinetic modeling methods (2013). Upton et al.57 Basic concepts in population modeling, simulation, and model‐based drug development: part 3‐introduction to pharmacodynamic modeling methods (2014).	Mandema et al.58 Model‐based meta‐analysis for comparative efficacy and safety: application in drug development and beyond (2011). Mould.59 Models for disease progression: new approaches and uses (2012). Boucher & Bennetts.60 The many flavours of model based meta‐analysis: Part I ‐ Introduction and Landmark data (2015).	van der Graaf et al.61 Analysis of drug‐receptor interactions in vivo: a new approach in pharmacokinetic‐pharmacodynamic modelling (1997). Krzyzanski & Jusko.62 Mathematical formalism for the properties of four basic models of indirect pharmacodynamic responses (1997). Bender et al.63 Population pharmacokinetic–pharmacodynamic modelling in oncology: a tool for predicting clinical response (2015).	Iyengar et al.64 Merging systems biology with pharmacodynamics (2012). Vicini et al.37 Systems pharmacology for drug discovery and development: paradigm shift or flash in the pan? (2013). NIH.39 Quantitative and systems pharmacology in the post‐genomic era: new approaches to discovering drugs and understanding therapeutic mechanisms (2011). Zhao et al.65 Best practice in the use of physiologically based pharmacokinetic modeling and simulation to address clinical pharmacology regulatory questions (2012). Rostami‐Hodjegan.66 Physiologically based pharmacokinetics joined with in vitro‐in vivo extrapolation of ADME: a marriage under the arch of systems pharmacology (2012). Kostewicz et al.67 PBPK models for the prediction of in vivo performance of oral dosage forms (2014). Sager et al.68 Physiologically based pharmacokinetic (PBPK) modeling and simulation approaches: a systematic review of published models, applications and model verification (2015). Jones and Rowland‐Yeo.69 Basic concepts in physiologically based pharmacokinetic modeling in drug discovery and development (2015).
Model description	Models focus on the statistical properties of the underlying data and generally use simple direct empirical functions models linking dose and time to response. Equations of dose‐response models use generally simple mathematical functions, such as linear, hyperbolic (Emax), exponential, quadratic functions but also Markov models/time to event/logistic regression models depending on the type of variable (continuous or non‐continuous) to be modeled. Both LME and NLME models can be used to account for between and with subject variations. Probabilistic models can be used to address missing data (e.g., due to informative dropout).	Compartmental PK/empirical PD models, pharmacologically realistic drug effect (i.e., from linear to sigmoidal equations) and disease models. Differential equations or analytical solutions, simple equations as slope, hyperbolic function (Emax) or sigmoid. These models often utilize statistical models to describe variabilities between, studies, subjects, and occasions (NLME). Probabilistic models can be used to address missing data (e.g., due to informative dropout).	Synergy between statistical approaches and pharmacologically principled models. Using principles of pharmacology to combine data across dose, time, mechanism, and patient populations (studies). The modeling approaches are similar to those used for empirical PKPD. NLME models are generally used to account for between trial and with trial variations.	Similar to empirical PK/PD model but incorporates system and drug dependent parameters. Physiological/pharmacological knowledge (prior) from a different knowledge source within part of the model. Mechanistic PK/PD model combining drug concentration, biomarker data, and clinical outcome data utilizing supportive in vitro potency estimates. These models can integrate statistical models to describe variabilities between studies, subjects, and occasions (NLME). Probabilistic models can also be used to address missing data (e.g., due to informative dropout).	System pharmacology modeling use systems of differential equations describing the drug behavior and the impact of drug from target interaction through pathway manipulation to observable pharmacology. Formal mathematical and computational models that incorporate data at several temporal and spatial scales; these models focus on interactions among multiple elements (biomolecule, cells, tissues, etc.) as a means to understand and predict therapeutic and toxic effects of drug. PBPK models where parameters can be estimated within physiological framework. Systems of differential equations describing the drug behavior in term of absorption, distribution of drug to tissues, and taking into account the effects of emunctory organs (i.e., kidney, liver). These models can combine physicochemical properties and in vitro data to perform IVIVE. Individual parameters or components of the full systems pharmacology model may be established using LME/NLME. However, the full model is normally constructed to be a simulation model.
Role in decision‐making
Use in internal decision‐making	Established (e.g., Younis Ex98 a)	Established (e.g., Cosson Ex71, Booth Ex73, Frey Ex95, Lehr Ex103)	Established (e.g., Dodds Ex31, Vargo Ex44, Mandema Ex86, Demin Ex88, Gross Ex90)	Established (e.g., Visser Ex11, Stroh Ex55, Soto Ex67, Kretsos Ex28)	Established (e.g., Riggs Ex56, Benson Ex4, Schaller Ex6, Benson Ex58, Johnson Ex102)
Use in regulatory assessment	Established (e.g., Younis Ex98, ICH E4 199447)	Established (e.g., Cosson Ex71, Booth Ex73, Frey Ex95, Lehr Ex103)	Emerging (e.g., Vargo Ex44, Harnisch Ex94)	Established (e.g., Lowe Ex18, Yu Ex15)	Emerging (e.g., Burghaus Ex62, Willmann Ex92, Jones et al.70)
Used to provide prespecified primary inferential analysis of emerging data	Established (e.g., Cohen Ex29)	Established (e.g., Høivik Ex61, Manolis et al.2)	Potentially as part of disease progression modeling (e.g., Mould et al.59)	Has potential	Systems pharmacology limited potential PBPK Emerging especially for special populations and DDIs
Can provide the ability to extrapolate beyond the available data	Limited potential	Limited potential	Potentially as part of disease progression modeling or in estimating scaling factors (e.g., Mould,59 Harnisch Ex94)	Established (e.g., Visser Ex11, Stroh Ex55, Kretsos Ex28, Lowe Ex18)	Established (e.g., Riggs Ex56, Benson Ex4, Schaller Ex6, Benson Ex58, Burghaus Ex62, Johnson Ex102)

^aFor each example (Ex number) in this and subsequent sections, please refer to corresponding entry in Supplemental Table S1.

CHMP, Committee for Medicinal Products for Human Use; DDIs, drug‐drug interactions; EMA, European Medicines Agency; E_max, maximum effect; FDA, US Food and Drug Administration; ICH, International Conference on Harmonisation; IVIVE, in vitro‐in vivo extrapolation; LME, line mixed effect; MBMA, model‐based meta‐analysis; MCP‐Mod, Multiple comparison‐modeling; NLME, nonlinear mixed effect; PBPK, physiologically based pharmacokinetic; PD, pharmacodynamic; PK, pharmacokinetic; PK/PD, pharmacokinetic/pharmacodynamic.

In some instances, the quantitative approach is directly related to the activity; for example, characterization of the exposure‐response relationships of existing treatments to define the competitive landscape will involve a meta‐analysis integrating compound, mechanism, and disease/indication level data. In other instances, outputs from multiple quantitative approaches for a compound of interest require integration; for example, in the determination of an optimized dosing regimen within a specific therapeutic window.

Although empirical models derive robust characterization and description of dose‐efficacy–safety relationships (which can enable specific product labelling statements) at compound level, the more mechanistic approaches will be necessary to effectively incorporate mechanism and disease/indication level data into a broader quantitative framework.

The timing of these MID3 activities is very important as deliverables must be available to inform important R&D decisions. The MID3 strategic plan should be a “living document” that is updated after compound, mechanism, and disease/indication level decision points. After a decision to advance a compound, at least the compound‐specific components of the MID3 plan should be refreshed in order to prepare for the next decision point. Accordingly, after a decision not to advance a compound, the disease and mechanism components of the MID3 plan should be refreshed in order to prepare for future compounds pursed in the particular indication.

4.4. MID3 CHALLENGES AND OPPORTUNITIES

In order to foster an increase in consistency of MID3 utilization and impact across the sector, we should to try to consider the principle challenges to, and opportunities for, change. The following elements seem to be associated with the extent to which individual organizations implement and utilize MID3. The list is certainly not exhaustive, and by necessity it is high level. However, these elements may be useful when considering how to enhance the level of MID3 activities within a particular organization. For any implementation to be successful, the unique set of direct and indirect enablers and disablers that are influential and germane to a particular organization should be acknowledged and addressed.

4.4.1. Pharmaceutical industry sector level influence

Challenge

Despite the well‐documented sector, associated pressures, and productivity challenges, there is a large inconsistency in terms of MID3 utilization across the pharmaceutical industry. This is most likely a consequence of the ROI for MID3 not being well understood within the higher echelons of the pharmaceutical industry and health authorities. Furthermore, although the current MID3 practice levels have been valuable and influential to a (variable) degree, it is unlikely that the overall sector‐associated pressures can be meaningfully impacted if the status quo is maintained.

Opportunity

Generating greater awareness among industry and regulatory decision‐makers as to why and how MID3 can be beneficial is essential. It is important for increased implementation to shift the balance from the technical advocates “pushing” MID3 to the decision‐makers “pulling” MID3. In order to support determination of the ROI for MID3, sections 3.1 and 3.2 illustrated the nature and extent of impacts for a variety of quantitative approaches across a range of relevant R&D scenarios. In addition, in section 4.5 of this document we will introduce an “EFPIA categorization of MID3 value for internal decision‐making” that will enable determination of the business value obtained via MID3.

4.4.2. Organizational level influence

Challenges

MID3 aims to enhance the extraction of inference from both existing information and data emanating from ongoing experiments. MID3 is an integrative approach that can effectively support translation across, and extrapolation beyond, a given set of experimental conditions. An inherent consequence of this integrative approach is that the necessary source data, information, expertise will reside across a broad range of personnel, departments, and institutions. However, an unintended consequence can be the potential for “narrower” concerns emanating from infringements of actual or perceived “control,” “ownership,” “territory,” and “domain” diminishing the likelihood that the “broader” organization can derive the greatest benefit.

Another “integration” challenge surrounds how well MID3 can fit within R&D planning, processes, and timelines that are designed to service the needs of study level activities. For the potential for MID3 to be realized, the associated outputs need to be available for specific R&D decisions and be of a quality consistent with appropriate industry and regulatory standards. This can present a significant challenge for what are predominately iterative modeling approaches and can also curtail the choice of approach taken because of both data and time constraints.

Opportunities

The desired technical methodologies for MID3 are established, and companies have colleagues with the appropriate skills to deliver them. However, most companies do not fully capitalize on this opportunity as a result of organizational and/or cultural impediments. Mitigating or removing these organizational impediments has been shown to improve late‐stage clinical development productivity.31, 33

In addition, the adoption of a more realistic assessment of the merits of any given compound can bring productivity dividends to the organization. A cultural shift toward “truth‐seeking” and away from “progression‐seeking” across individuals, teams, and governance71 can mitigate many if not all of the principle challenges for enhanced MID3 implementation.

4.4.3. Plan level influence

Challenges

Clinical development plans emanating from a rigid “confirmatory mind‐set,” which can lead to an overreliance on empirical evidence to address clinically important questions. This can be the result of a narrow interpretation of the validity of alternative approaches to the randomized clinical trial in generating a robust evidence base from which to draw clinical inference.

Opportunities

There should be more effective dialog during plan creation. Consideration should be made of the following qualifying questions in order to gain greater alignment and consensus across stakeholders and the multiple disciplines contributing to the plan construction and implementation.

• Determine what information will be generated and how will these activities inform the decision(s).

• Check that the proposed activity can provide answers to the identified questions and that the questions are pertinent.

• Check that there is an efficient balance between study‐based activities vs. broader compound, mechanism, and disease‐based activities.

• Determine what are the technical and resource interdependencies and time sequences between each activity in the MID3 plan.

• Determine what the most impactful assumptions are.

• Determine what the most likely limitations are.

• Determine what will be done to ensure availability of deliverables in sufficient time to inform the decision.

• Determine what would be the impact of not performing these activities.

Section 4.3 discusses a variety of approaches to obtain and implement robust MID3 plans.

4.4.4. Activity level influence

Challenges

Approaches that are precedented and conservative can be considered in some quarters to be the most robust and trustworthy. In line with good clinical practice and International Conference on Harmonisation guidelines, primary data analyses are defined in clinical study protocols and statistical analysis plans in advance of study initiation and conduct. Although this approach fulfils a requirement for both statistical rigor and integrity, which is of particular importance within a confirmatory setting, prespecification will often lead to the adoption of more straightforward “assumption‐light” analysis methods. Within a confirmatory setting, MID3 approaches are often conducted as secondary or exploratory analyses after the primary analysis has been conducted and communicated, limiting the potential for MID3 approaches to effectively influence R&D decision‐making and regulatory assessment.

Another challenge is the degree of inconsistency in the format and detail of MID3 application that exist across the range of reports submitted to regulators (e.g., clinical overview or SmPC,72 both across companies29, 73 and between regulatory reviewers).74, 75 Important elements, such as questions to be addressed by particular analyses, resultant conclusions, and recommendations, are not always clearly and effectively communicated. There is a tendency for sponsors and regulators to focus on the technical aspects of MID3, such as individual parameter uncertainty and variability rather than on the “bigger picture” joint/integrated uncertainties manifest in, for example, an exposure‐response curve and the resultant influence on dosing and labeling recommendations. From the perspective of the regulatory assessor, there is often insufficient information provided to be able to effectively judge the appropriateness of the model for estimation and/or prediction/simulation.76 The 2011 EMA/EFPIA M&S workshop¹ identified two common limitations of analysis documentation submitted for regulatory review: a lack of transparent description of influential assumptions, and an ineffective evaluation or reporting of the impact of potentially erroneous assumptions (i.e., sensitivity analyses) on the resultant conclusions and recommendations. These limitations were considered to be an unequivocal barrier to the wider acceptance of MID3 approaches within the regulatory agencies (and industry).

Opportunities

There is a growth in the use of analysis methodologies, such as systematic reviews to complement, and in many aspects enhance, the level of information derived from traditional randomized controlled studies. There is a greater recognition that fit‐for‐purpose approaches derived from a broader set of analysis methods can efficiently inform clinical and regulatory decision‐making. Of particular importance to MID3 capability and capacity will be the Drug Disease Model Resources consortium. This Innovative Medicines Initiative call will enable a continuous integration of available information related to a drug or disease into constantly evolving mathematical models. The models will be capable of describing and predicting the behavior of studied systems to address the questions of researchers, regulators, and public health care bodies. This will be achieved through the generation of a common definition language for data, models, and workflows, along with an ontology‐based standard for storage and transfer of models. All drug and disease model libraries developed will be made available as a public resource and an open‐source interoperability framework will be the backbone for the integration of modeling applications.

With respect to the second challenge, this document aims to establish de facto standards and good practices. As stated earlier, greater transparency with respect to the chosen data, methodologies used, key assumptions, model assessment, and prespecification (when appropriate) have been highlighted as important issues to address. The EFPIA workgroup proposal with respect to good practices and documentation for MID3 is addressed in section 5.

4.5. Determination of MID3 business value

As discussed in the section 4.4.1, there is a need to generate greater awareness among industry and regulatory decision‐makers as to why and how MID3 can be beneficial. In this document, we have provided section 3.1 and 3.2 to illustrate the nature and extent of impact for a variety of quantitative approaches across a range of relevant R&D scenarios.

In this section, we introduce an “EFPIA Categorization of MID3 value for internal decision‐making,” which will enable determination of the derived business value (an important component in any assessment of ROI). A parallel can be drawn to the framework introduced by the EMA to categorize the impact of M&S (and associated inference) on the regulatory decision‐making process where the impacts were partitioned into three categories: descriptive, justify (decision informative), and evidence substitution.42

The aims of our proposed “EFPIA Categorization of MID3 value for internal decision making” (Table 3) are twofold. We wish to enable determination of the likely impact of the examples provided in the section 3 (and the APPENDIX) and by doing so initiate a dialog on the potential usefulness of such an approach for both retrospective and prospective assessments. We are aligned with the EMA framework as our proposed categories are associated with the degree to which the MID3 approach informs a particular decision‐making process (in this case, within the pharmaceutical industry domain). Of the examples provided, a number would be considered high category impacts, as they informed the decisions to discontinue a compound (Høivik Ex61), seek and gain approval for an untested dosing scheme (Cosson Ex71), and recommend particular doses for severely renal‐impaired patients (Lehr Ex103). These examples realize high business value as, had the decision outcome been different than in the first example, there would have been continued expenditure on a compound that would ultimately fail, in the second example, additional studies would increase the project cost and time, and for the last example, the treatment would have not been available for severely renal‐impaired patients.

Table 3.

EFPIA categorization of MID3 value for internal decision‐making

LOW CATEGORY IMPACT – describe ‐ MID3 approach provides inference which has limited impact on internal decisions

• MID3 inference is consistent with standard reporting methods in response to strategic questions. • MID3 inference has limited impact at the next decision point. • Applications include: descriptive population PK and PK/PD, “hypothesis generating” MID3 analyses which are not investigated in subsequent experiments/trials.  ° These applications may provide value in supporting regulatory interactions and/or have higher level impact against (additional) future strategic questions.

MEDIUM CATEGORY IMPACT – inform ‐ MID3 approach provides inference which informs internal decisions

• MID3 inference extends beyond standard reporting methods in response to strategic questions. • MID3 inference is a major component at the next decision point and will be supplemented by evidence generated by future experimental or trial data. • Applications include: selection of optimal dosing, target population or sample size, designs for future trials or identification and selection of pathways, targets, mechanisms, compounds for future investigation prior to major investment.

HIGH CATEGORY IMPACT –– replace ‐ MID3 approach provides inference which informs internal decisions without requiring additional experimental or trial data to be generated

• MID3 inference only can effectively address strategic questions. • MID3 inference is a vital component at the next decision point and may not be verified by evidence generated by future experimental or trial data. • Applications include: strategic decisions at the portfolio level (e.g., starting, stopping, delaying, and accelerating compounds or mechanisms), replication of effectiveness providing evidence of efficacy/safety in lieu of clinical data for regulatory submissions and labelling.

MID3, model‐informed drug discovery and development; PK, pharmacokinetic; PK/PD, pharmacokinetic/pharmacodynamic.

We consider optimization of study designs or choice of phase III dosing regimen to represent a medium category impact, as the decision taken will require further clinical studies to provide supplemental evidence to determine the decision appropriateness. Clearly, the business impact of an incorrect or inaccurate decision relating to study designs or phase III dosing regimen choices can be considerable.

Determination of a high, medium, or low category impact will be based on the particular strategic question under consideration. MID3 activities have the intrinsic quality to possess different levels of impact for different strategic questions. For this reason, we emphasize the need to take a “fit‐for‐purpose” approach in order to determine which particular activity can be most informative. Furthermore, MID3 activities can enhance their business value as they enable and integrate learning/confirming across a range compounds, mechanisms, and disease levels. We believe that ongoing development of a larger more comprehensive quantitative framework will increase the likelihood of realizing high category impacts.

5. “HOW” SHOULD MID3 BE PERFORMED? PLANNING, CONDUCT, AND DOCUMENTATION OF MID3 ANALYSES

5.1. Overview of good documentation practice for MID3

Regulatory guidelines exist from the FDA and the EMA describing good practices for model building, evaluation, and documentation.15, 16, 17, 72, 77 As discussed at the EFPIA/EMA M&S workshop and reported by Manolis,2 this is a developing field and updates to these guidelines and development of more comprehensive guidelines will be important for greater adoption and acceptance of MID3. In addition, an assortment of scientific articles introducing and developing practice recommendations in model building and documentation have also been published (see tutorial/review/perspectives/white papers section of Table 2).18, 20, 21, 22, 39, 51, 78, 79, 80, 81, 82, 83, 84, 85, 86

In the following sections, the EFPIA workgroup position regarding documentation of planning, conduct, and reporting of MID3 analyses will be presented. Although the section was largely developed from the documents listed above and our own experiences with the population pharmacokinetic modeling approach, we consider the recommendations to be relevant to a variety of other modeling approaches. An aim of this section is to promote consensus on good practices with EMA Modeling and Simulation Working Group, which, in turn, may serve as a starting point for future regulatory guidance development in this area.

In order to facilitate greater adoption of MID3 within an R&D or regulatory context, we consider the following documentation attributes to be essential:

• Clear analysis objectives

• Transparency on analysis assumptions and their impacts

• Adequate communication of key findings and recommendations to stakeholders

• Sufficient materials provided to enable complete reproduction of the analysis

In light of the above, we wish to provide some guiding principles regarding an appropriate and acceptable level of documentation (fit‐for‐purpose), inclusion of outputs within regulatory documentation, and quality assurance/control of MID3 analyses.

5.1.1. Appropriate and acceptable level of documentation

For MID3 activities performed to support internal decision‐making (such as go/no‐go decisions), the nature and extent of documentation will ultimately be determined by the particular company. However, in order to be consistent with our good practice recommendations, we consider the following to represent a minimum level of documentation: (1) a short analysis and/or simulation plan or a memo to gain stakeholders agreement on objectives, data plan, assumptions (including their evaluation and impact), and deliverables; and (2) a memo/abbreviated report or slide presentation documenting results.

At some future point in time, MID3 activities performed to support internal decision‐making may be used for related compounds or for interaction with regulators that will necessitate an “upgrade” to the available documentation according to its revised intended purpose. For that reason, irrespective of the formats selected, we recommend that links are established between data, data transformations and manipulation, final model/simulation code, and conclusions in order to facilitate traceability.

For MID3 analyses which are either intended to be used for interaction with regulators or submitted for registration and are not reported as part of the standard clinical study report, it is proposed that more structured MID3 documentation proposals are adopted. There are three principal components of MID3 documentation: analysis plan, simulation plan, and report. These documents have a high‐level structure shown in Table 4. Additional specific considerations relevant for each component are provided in sections 5.3, 5.4, and 5.5, respectively.

Table 4.

The common general structure of documents describing MID3 analyses

Analysis plan	Simulation plan	Report
• Introduction • Objectives • Data plan • Data exploration • Methods • Assumptions	• Introduction • Objectives • Additional information • Methods • Assumptions	• Synopsis • Introduction • Objectives • Data • Methods • Assumptions • Results • Applications (prediction/simulation) • Discussion • Conclusions • Appendix

The report should provide sufficient detail to enable an independent reviewer to assess precisely what activities have been completed and the appropriateness of the results/conclusions.

5.1.2. Inclusion of analysis outputs within regulatory documentation

Stand‐alone reports, as well as MID3 document components included as part of a standard clinical study report, are by default part of a regulatory submission for marketing approval. Their main purpose is to support dosing recommendations, claims, and address a variety of drug and indication‐related strategic questions. Output from population PK, as well as exposure‐response (biomarkers/efficacy/safety) and PBPK analyses (single study as well as across studies analyses), should be included in the section “Summary of Clinical Pharmacology Studies” (Module 2.7.2) of the Electronic Common Technical Document.87, 88

The important high‐level results and conclusions from MID3 activities together with a brief outline of the methodology and source data should be provided in “Background and Overview” (Module 2.7.2.1). A fuller account of the objectives, data, methods, key assumptions, and results reside in section “Summary of Results of Individual Studies” (Module 2.7.2.2) and then integrated with other results in section “Comparison and Analyses of Results Across Studies” (Module 2.7.2.3). The original synopses of the individual MID3 activities should be provided in the “Synopsis of Individual Studies” (Module 2.7.6).

Studies designed with dose/exposure‐efficacy or safety analyses as primary or key secondary objectives, should be described in “Summary of Efficacy” (Module 2.7.3) or “Summary of Safety” (Module 2.7.4) rather than in the “Summary of Clinical Pharmacology Studies” (Module 2.7.2).

In addition to the promotion of key messages (e.g., use of exposure response analysis to adjust dosing for special populations) to the “Overview of Clinical pharmacology” (Module 2.5.3), relevant exposure efficacy, safety, or benefit‐risk analyses along with associated utility (e.g., justification of dosing regimen, adjustment of dosing regimen for subpopulations) should be summarized in the related “Clinical Overview” section(s) of the Electronic Common Technical Document (Module 2.5) and cross‐linked to the more detailed description in Modules 2.7.2/.3/.4.

When addressing questions during the review stage of registration, a memo‐type response to the question together with an appendix providing greater technical details is recommended.

5.1.3. Quality assurance, control, and verification

There are currently no broadly adopted good practices with respect to Quality assurance, control and verification.82 The following aims to ensure that the analyses appropriately address the scientific question, are technically correct; and the data, analysis code, and outputs are stored and documented in a manner that allows reproduction by a third party:

1. Quality assurance: to assure the integrity of the data, processes and/or technical solutions should be in place that allow tracing back from the results in the final documentation to the original data/database used (technical audit trail or process description). The person(s) performing the analysis should have been trained in the use of the software(s) utilized. The software used should be “qualified” for its intended purpose.

2. Scientific review: to further assure the quality of the analysis, an independent peer review (competent person not involved in the analysis) is recommended. Here, the focus is to critically evaluate the appropriateness of the analysis, and its results, with regard to the stated objectives.

3. QC/verification: these are measures that should be performed for each analysis with regard to generation of the initial and final input dataset, model description, model code, and key simulation code. Some activities may not require QC/verification (e.g., for low category impact activities, or an analysis for internal decision‐making). To assist in this determination, a risk‐based approach is suggested:

   • Probability (low, medium, and high) of a possible error during creation.
   • Probability (low, medium, and high) of errors remaining undetected.
   • Possible impact of such errors on the analysis (low, medium, and high).

To illustrate this approach further, the code for standard goodness‐of‐fit plots provides a low‐risk example across these three elements. The code for a stratified, prediction‐corrected visual predictive check or a final model of a covariate analysis provides a medium‐risk example across these three elements. The simulation code for a dosing schedule outside the investigated dose range provides a high‐risk example across these three elements.

The respective QC/verification measures will range from basic testing (e.g., script finishes without error messages), code review, and, at the highest level, independent programming of key code parts. The QC of the documentation should consist of a check to ensure that any numerical values that appear in tables or figures match the numerical values obtained from the software outputs.

5.2. Assumption setting, evaluation, impact assessment, and documentation

Transparency in the setting and evaluation of assumptions that may impact model application is of great importance in the planning and documentation of any MID3 activity. The set of assumptions that are considered important or impactful should be a collaborative decision among the quantitative analyst, the broader project team, and domain experts (disease area, methodologists, etc.). The purpose of this section is to make explicit the range of assumptions commonly adopted, provide some recommendations for terminology to describe these assumptions, and illustrate how assumptions may be captured and displayed in dedicated sections of the MID3 documentation.

In setting assumptions, their utilization and evaluation should be subject to a series of “learn and confirm cycles” such that their application within model building and simulation/prediction can be confirmed in future experiments (Figure 3 ). We consider it good practice that at transition/decision points in the R&D cycle (Figure 1 ) there should be particular emphasis on the degree of testing, acceptance, and mitigation of risk should any important assumptions be erroneous.

Assumptions can be defined by their current degree of acceptance (established vs. new). They can then be divided into those that are testable based on the data used for developing the model (or otherwise available data) or based on data that will be obtained in future studies and those that are not testable. In the case of a testable assumption, the proposed approach used to test and substantiate the assumption can be detailed. In the case of non‐testable assumptions, the impact of an erroneous supposition on the model results/predictions (and therefore the validity of the model with respect to its intended application) should be evaluated in an appropriate sensitivity analysis.

It is important to emphasize that it is not necessary to list all assumptions utilized during model development. They are, by definition, part of the model‐building process, and will be reported with appropriate diagnostic evaluations in the report. It is therefore recommended that consideration is made of the important testable and non‐testable assumptions that may impact the conclusions derived from the final model. This subset of assumptions should be identified and ideally be prespecified in the analysis or simulation plan.

We provide in Table 5 some details of these important assumptions that are typically derived from five principal areas: (1) pharmacological (including compound level)‐related assumptions; (2) physiological‐related assumptions; (3) disease‐related assumptions; (4) data assumptions; and (5) mathematical or statistical assumptions of the model.

Table 5.

Documentation of important assumptions

Examples of important assumptions	Justification	New/established	Testable/not‐testable	Test/approach to assess impact	Evaluation (assuming a particular case scenario for illustration)
Pharmacological assumption: Emax model fixed to 100% is a more physiological description of the data compared to a linear model.	Emax model is not better than linear model; however, for this drug class, Emax of 100% is more realistic.	New	Testable with a wider range of concentrations (external/future study).	Comparison of simulated metrics of interest between the two competing models.	To achieve a 90% response (assumed to be clinically meaningful) requires a twofold higher dose using the Emax model compared to the linear model. → Test doses suggested by Emax model in Phase 2.
Physiological assumption: No difference in clearance between healthy subjects and patients.	Patients with major depression disorders are considered as healthy subjects (in regard of ADME/PK features) once age and weight are taken into account.	Established	Testable by pooling healthy subjects and patient data, assuming that all other qualities across the pooled trials are exchangeable.	Combined analysis with healthy subjects and patients.	Combined analysis found only a 10% lower clearance in patients. → No dose adjustment necessary for PK reasons.
Disease assumption: Linear progression of disease with a slope of X/year.	Cannot be estimated directly from the dataset, but supported by literature review.	Established	Not testable with the present dataset.	Sensitivity analysis changing the value of the slope for disease progression from X to Y.	Varying the slope by between X and Y will not change the selected dose for Phase 3. → Selected dose for Phase 3 can be implemented. Or Varying the slope by between X and Y will change the selected dose for Phase 3 drastically. → Three different doses should be tested in Phase 3.
Data assumption: Data BLQ have no impact on analysis results.	There are less than 20% of concentrations after treatment BLQ.	New	Testable	Run final model with BLQ using M3 method (Beal89) and compare to model without BLQ.	Negligible changes in parameter estimates. → Final model excluding BLQ observations selected.
Mathematical and/or statistical assumption: Similar variability in clearance between adults and children.	Physiological and PK knowledge.	New	Not testable at the stage of predictions but can be evaluated with data from children.	Sensitivity analysis on the variance value of clearance.	If variance is twofold, children would still be within the safety range established for adults. → Suggested dosing can be used in children.

ADME, absorption, distribution, metabolism, and excretion; BLQ, below limit of quantification; E_max, maximum effect; PK, pharmacokinetic.

→ Resulting recommendation.

The first five columns of Table 5 can largely be described in the analysis plan and in the final report; the last column can be added to detail the results obtained for the testable assumptions and the potential impact of the remaining non‐testable assumptions. Our emphasis on assumption detailing in the analysis plan stage does not preclude any addition, substitution, deletion, etc. of assumptions occurring during the conduct of the analysis; however, any deviation will need to be described and justified in the final report.

5.3. Analysis plan components and considerations

As discussed in section 4.3, it is recommended that individual MID3 activities are derived from a high‐level MID3 strategic plan, which emanate from a set of specific questions and associated MID3 activities and milestones. The MID3 activities in turn also require an appropriate and acceptable level of additional documentation (section 5.1.1).

5.3.1. Objectives

The objectives should be clearly stated and aligned with specific development questions identified in the MID3 strategic plan and/or subsequent regulatory questions.

5.3.2. Data plan

The data plan should define the data to be included in any exploratory investigation and analysis. It should describe the data selection rationale, data‐assembly strategy, and data formatting for both dependent and independent variables, especially covariates.

Data selection

The process of identifying relevant data and information to be acquired and aggregated to fulfil the objectives of the particular analysis (section 4.2.2) should also include the rationale for excluding potentially relevant data sources.

Data assembly strategy

For individual data from studies, this covers data cleaning for any draft data, and additional data cleaning steps post‐availability of final data. For data obtained from different sources (e.g., across a variety of internal studies, literature, or external databases), this will be a need to detail variable harmonization, derivation of equivalent variables, etc. Any underlying assumptions associated with this step should be captured as described in section 5.2.

Data formatting

This covers data transformation with respect to units, definition of derived variables, rules around time order of events, and determination of time windows and handling of missing data, including data‐imputation and exclusion‐based (whenever possible) on prespecified criteria, identification, and handling of outliers.

The detailed description of the data specification can be provided in a separate document and only a brief description provided in the analysis plan.

5.3.3. Data exploration

Descriptive and graphical data exploration is an essential initial step. There are generally three aims for this activity: (1) assessment of the quality of the data as part of the quality assurance, control, and verification process (section 5.2); (2) identification of similarities or gaps with respect to prior knowledge/historical data and the objectives of the analysis; and (3) investigation of patterns and relationships in order to guide model development.

Typical outputs would consist of graphical and tabular summaries and descriptive statistics. Although data exploration is essentially (analysis) dataset driven, it is recommended that a high‐level description of the plan based on previous knowledge and the objectives of the analysis is provided in the analysis plan.

5.3.4. Methods

All methods for model building, selection,90, 91 identifiability,92, 93 evaluation, and model qualification94, 95, 96, 97 should be described. The methods utilized will vary based on the available data, the objectives of the analysis. If the analysis is complex, we recommended outlining the MID3 analysis workflow with a visual representation (e.g., the relationship between various methods and data components used to address the proposed questions).

Model evaluation, as part of the learning and knowledge creation step (also described in section 4.2.3), should explore how well the model represents the system under consideration, through comparison of simulations and predictions to the available (integrated) data. The more formal qualification procedures can utilize precise criteria to assess the quality of the model with respect to its intended purpose.

Finally, the planned approach to display important results and related label claims should be described. The display can be graphical (e.g., forest plots to illustrate covariate effects, predicted dose‐response relationships for efficacy, and safety across key subpopulations, including uncertainty) or tabular in nature, with the latter being preferable where exact numbers are required. A description of any simulations to illustrate important results and label claims should be included if not covered in a separate simulation plan.

5.3.5. Assumptions

As detailed in Table 5, it is important to define important or impactful assumptions and how they will be evaluated and tested. Sensitivity analyses are recommended to assess the robustness of the model to strong informative priors (or fixed) parameter estimates where the underlying assumptions are not established or cannot be tested. Sensitivity analyses should be undertaken to explore the impact of data removed during data cleaning (if possible) or as outliers during the model‐building process.

5.4. Simulation and prediction plan: components and considerations

In common “modeling language”, both “prediction” and “simulation” are often used independently as a synonym for each other or to describe the joint “Simulation and Prediction” process (for formal definitions see Glossary). We therefore link the formal and common usage in the following definitions:

• Prediction without simulation (in common modeling language this is often called a “deterministic” approach) contains no degree of randomness, and is conditioned on the estimated parameters for the mean of a population, a set of individual(s) or set of fixed parameters and covariates for a system.

• Prediction with simulations (in common modeling language this is often called a “stochastic” approach or “predictions with variability and uncertainty”) includes variability (e.g., between and within subject variability, measurement error), ranging from encompassing at least one level of random effect to variability on all parameters. Depending on the objectives, uncertainty on the structural parameters of the model is taken into account. Similarly, covariates may be simulated from a multivariate distribution, or re‐sampled from an empirical distribution.

As described in section 4.2, prediction (with/without simulation) from MID3 models can be used to support internal drug‐development decisions and regulatory queries and label claims. Typical outputs can include forecasting the probability of success of the next experiment/trial or the probability of technical success or defining optimal posology. Depending on the objectives, this may utilize mathematical models under different conditions from those used during model development (e.g., a different range of covariates), or a clinical trial simulation (CTS) of various complexity.98, 99, 100, 101, 102 It should be noted that optimization techniques that derive an analytical solution, rather than being simulation and prediction based, may also inform trial design efficiency.103

A separate simulation and prediction plan, in addition to the analysis plan, is recommended when the detail and complexity merit separate documentation. The plan should define the simulation and prediction strategy and, where appropriate, provide prespecified criteria against which the results can be assessed.

5.4.1. Objectives

The objective of the simulation and prediction is to provide responses to “what if” questions. As discussed above, there are a wide range of these types of questions from the simple exploration of the impact of some covariates to underpinning extrapolation, as defined in the EMA concept article (discussed in section 4.2). Once again, it is important that the objectives should be clearly stated in advance.

5.4.2. Additional information

In this section, additional information required to perform the simulation and prediction should be described (e.g., covariate distributions).

5.4.3. Methods

The theoretical aspects of CTS have been described elsewhere100, 101, 103 and any Methods section should provide the description of models for simulation and prediction (i.e., the input‐output model, the covariate distribution model, and the trial execution model), the statistical analysis on which interferences will be drawn.

Qualification of simulation and prediction ensures that they are “fit‐for‐purpose.” The degree of qualification will depend on the use and their importance to the subsequent decision. The boundaries, within which a simulation can be considered valid, should be defined in advance. For example, the simulated data may over or underrepresent the possible transitions or fluctuations observed in the individual subject level (observed) data because of an assumed correlation structure; which may be of concern, depending on the objectives. Sensitivity analysis may be proposed as part of the plan to help further explore the impact of limitations on the results. As in the analysis plan, the planned display of important simulations and predictions should be detailed.

5.4.4. Assumptions

As highlighted by Holford et al.101 in their review, assumption‐making is an inherent feature of CTS, it is, therefore, important that they are adequately described in the simulation and prediction plan. The general assumptions are related to the models used for the simulation and prediction, the input‐output model, and the covariate distribution model. In addition to general assumptions for simulations, there are technical assumptions specific to CTS that are related to the execution model and the statistical analysis methods. The execution model intends to make the CTS realistic as it deals with real world settings (e.g., compliance, (informative) drop‐out, between‐trial differences due to external factors (e.g., season effect on disease). All of these assumptions should be described and categorized according to Table 5 in section 5.2.

5.5. Report components and considerations

For activities related to a single clinical trial, the opportunity exists for the results to be reported in conjunction with the standard statistical analysis, as described in the clinical study report. In other situations where data is obtained from a variety of studies or sources an analysis report should be a standalone report that complements the clinical study report(s). As highlighted in section 4.4.4, improved communication and display of important conclusions, recommendations, and assumptions in analysis reports are required to ensure that these conclusions are easily communicated and utilized during clinical review.72

Details of the purpose, audience and content for each section is provided in Table 6. Whilst we acknowledge that the content recommendations provided in Tables 4 and 6 reflect the more “established” modeling practices; we do believe that these are also relevant to the broader range of quantitative activities encompassed by MID3.

Table 6.

Recommended sections of an analysis report

Title	Purpose	Audience	Important elements of content
Synopsis	• Summary of analysis process • Key results and conclusions	All readers	• Objectives of the analysis • High‐level methodology (focus on data sources) • Key results • Conclusions from the analysis
Introduction	Context for the analysis in the development program	All readers	Background to place the analysis in the context of the development program
Objectives	Statement of the analysis objectives	All readers	Precise objectives that answer important development question(s)
Data	Description of data used in the analysis	Technical	Dataset description focusing on the important elements relevant to the objectives of the analysis
Methods	Documentation of methods used in the analysis • Model building, evaluation, and qualification strategy • Applications (simulations/predictions)	Technical	• Mainly refer to analysis plan in the appendix • High‐level description of model building and qualification strategy • Changes to the analysis plan should be described
Assumptions	Main assumptions for whole analysis including justification	All readers	• Describe the important assumptions that may impact conclusions drawn from the final model, separated by: • Pharmacological (including compound level)‐related assumptions • Physiological‐related assumptions • Disease‐related assumptions • Data assumptions • Mathematical or statistical assumptions (of the model) • Best listed in form of a table (see Table 5)
Results	Description of analysis results	Technical	• Summary of the data and data exploration with adequate graphical and tabular displays • Description of the best model • Description of evaluation/qualification results • Description of the evaluation of assumptions (see Table 5)
Applications/simulations	Description of results of the model application/simulation	All readers	Description of model application/simulation results with an adequate display, (i.e. illustrative plots for clinical interpretation or to support a label claim)
Discussion	Explanation of the relevance of the results	All readers	• Place results in technical and clinical context • Discuss assumptions of the model (see Table 5) and limitations of the data • Discuss clinical relevance of the analysis
Conclusions	Present main findings focused on clinical relevance	All readers	Preferably presented as a bullet point list
Appendix	Documents/scripts and code that allow reproduction of the analysis	Technical	• Analysis plan • Dataset specification (including missing data rules) • Scripts/code and output files of key models, complementary plots and any additional information for comprehensive documentation but not included in the main body of the report • Run summaries listing key modeling steps

It is possible that during data exploration assumptions that were not foreseen (e.g., need to use prior knowledge in determining a maximum effect [E_max]). Similarly, the model‐building process and the model qualification may lead to changes in the methodology compared to that originally envisaged. It is therefore important to appropriately document deviations from the analysis plan and capture the evaluation of the emerging assumptions in the report.

6. SUMMARY

This white paper provides a variety of Model Informed Drug Discovery and Development (MID3) good practice recommendations, with the aim of enabling more efficient and robust R&D and Regulatory decision making. It will be necessary for decision‐making bodies' to embrace MID3 across many levels, from planning and resourcing to assumption assessment and impact assessment across commercial viability, PTS (Probability of Technical Success), dose selection and trial design evaluation. The business case for MID3 (Section 3.1) and over 100 R&D case studies (Section 3.2) have been provided to motivate this step change. We provide means to enable effective planning and terminology to categorise activities at the organisation level (Section 4 Table 1 & 2). We also provide a basis to capture MID3 impact and value (Table 3). In order to achieve the stated aim, greater consistency in the application of the MID3 quantitative framework across the Pharmaceutical Sector is required (Section 4.4). We believe that the Drug and Disease Model Resources (DDMoRe) will have a substantial role to play in driving future quality, efficiency and cost effectiveness of MID3 implementation.

While the first two (“Why” & “What”) sections of the document address the needs of decision makers and practitioners, the final (“How”) section focuses on MID3 related documentation. This is in accordance with the EMA request for EFPIA to produce a “good practice” manuscript covering the many aspects of planning, conduct and documentation of the variety of quantitative approaches discussed at the 2011 Modeling and Simulation (M&S) joint workshop. We provide a systematic approach to assumption evaluation and the quality control (QC) and quality assurance (QA) procedures necessary for MID3 implementation. We outline the necessary MID3 components to support low, medium and high impact regulatory decisions. These elements have been the focus of a continued dialogue between EMA and EFPIA since 2011. In this regard we believe that this white paper on Good Practice in MID3 provides a means to foster a broader dialogue between the Pharmaceutical Industry and Regulatory Agencies in order to enable more efficient and robust R&D and Regulatory decision making.

7. ACKNOWLEDGMENTS

The EFPIA MID3 work group acknowledges the personal contribution of Efthymios Manolis and Terry Shepard for their ongoing support in the development of this guideline including, but not limited to, their insightful comments and advice during face‐to‐face planning meetings and regular review meetings. We also acknowledge the personal contribution of the EMA Modeling and Simulation Working Group colleagues and many other company colleagues who have provided review comments and assisted with the compilation of the application examples via their EFPIA MID3 workgroup representatives. Based on a detailed scientific draft provided by the authors, Gardiner‐Caldwell Communications provided medical writing support with respect to formatting, styling, and document administration. We would also like to thank Julia Bray for administrative and document support.

8. CONFLICT OF INTEREST

All authors are employees of the stated companies and represent these companies as part of their participation in the EFPIA MID3 Workgroup.

9. AUTHOR CONTRIBUTIONS

All authors were involved in the writing and review of the manuscript.

10. Glossary

Key steps in MID3
Data	This is a selected database of relevant data associated with and generated from the system of interest. It is proposed as good practice that Compound, Mechanism, and Disease level data should be assimilated. The determination of the possible pertinent data is the first step in development of the data plan component of an analysis plan.
Knowledge	This is the outcome from cycles of Model Development and Model Evaluation. The parameter estimates, associated uncertainty, and their interrelationship represent the gain in understanding of the underlying pharmacological, physiological, and pathological properties that describe the system of interest.
Inference	The process of drawing logical conclusions with respect to future experiments/studies or to inform decision‐making using the current best model. This is achieved either directly from the parameter estimates or following simulation/prediction. Simulations/predictions would utilize the underlying model and inputs (e.g., time, dose, covariate distributions, and design information) representing the scenario of interest.
Key process in MID3
Modeling (develop and evaluate)	The process of interpreting multifactorial (disease, mechanism, and compound level) data generated and their interplay in the form of mathematical relationships, based on a set of assumptions (either established or new). This will involve cycles of model development and model evaluation to arrive at the current best model. In this process, data is converted into knowledge about the system of interest.
Application of current best model	The general process of using the knowledge captured in the current best model either directly based on the parameter estimates or via simulation/prediction to determine how the system will perform in other situations. In this process, the knowledge gained about the system is utilized to make inference with respect to the next experiment/trial or inform decision‐making. The sensitivity of any inference to the new assumptions can be explored.
Generation of new experimental/trial data	The process of producing data from optimized experiments or trials, designed based on the inferences established from the current best model. One of the goals of the future experiments and trials will be to provide key data to allow assumptions to be tested. In this process, new data that fills gaps in knowledge of the system are generated.
Key MID3 terminology1
Evaluation of model	The process of determining the degree to which a model is an accurate representation of the system from the perspective of the integrated data.
Prediction (interpolative and extrapolative)	This is a forecast of how the system will perform under a specified set of inputs in order to make inference about a system in a new situation. Calculations are performed using the model, the estimated or parameter values simulated from an estimated distribution, and specified inputs (e.g., time, dose, covariate distributions, and design information).
QC/verification of final data	This is the process of checking the manipulations used in creating the final dataset from the source data are consistently and correctly applied and documented. The nature (e.g., clinical trial database) and status (e.g., final and locked) of the source data should also be appropriately captured.
QC/verification of model code	This is the process of confirming that the algebra is consistent with the model description (e.g., two compartment models with zero order absorption and mixed linear and nonlinear elimination from the central compartment) and that it is translated correctly into the software code. Similar QC/verification would apply to the ensuring used software settings (estimation approach, priors, stating estimates, and termination criteria/messages are correctly described in the documentation).
Qualification of model	Model evaluation may lead to assessment of the model against set criteria used to judge the model's quality with respect to its intended purpose. This comparison is used to indicate whether the model is “qualified” for its intended purpose.
Qualification procedure that maybe applied to a model/method	The CHMP can issue an opinion on the acceptability of a specific use of a method, such as the use of a novel methodology or an imaging method (which can extend to a model) in the context of research and development. The method can apply to nonclinical or to clinical studies, such as the use of a novel biomarker. The opinion is based on the assessment of data submitted to the agency.
Simulation (computer simulation)	The execution of a computer code that mimics a system in order to generate “pseudo” data that are exchangeable with “real” data. Where inputs (e.g., time, dose, covariate distributions, and design information) and/or parameter values are altered (often using a stochastic process with respect to underlying variability or uncertainty) in order to learn about the system itself and how it would perform under a new situation (unstudied scenarios including future experiments and trials).
System	A general term for a set of interacting or interdependent components. Most often the system of interest is a patient with the disease of interest. Where the system includes the impact of the disease, but also the knowledge of how the asset (PK and mechanism of action) impacts the temporal aspects of the disease. Knowledge of the system is gained not only from the patients but also from healthy volunteers, animal models of the disease, and in vitro experiments.

¹ Terminology has been developed with reference to terminology specified in the National Research Council (2012) publication on “Assessing the Reliability of Complex Models: Mathematical and Statistical Foundations of Verification, Validation, and Uncertainty Quantification”104 and several online reference sources (e.g., Oxford dictionary http://www.oxforddictionaries.com, Merriam‐Webster http://www.merriam-webster.com, Wolfram Mathworld http://mathworld.wolfram.com/, but adapted where necessary to fit with the common usage and understanding of terms across the community of practitioners authoring this document.

Supporting information

Supporting Information

Supporting Information

The Table below summarises the compiled list of ∼100 case studies sourced from the literature and the EMA/EFPIA M&S Workshop 2011. Some of these examples are highlighted in the section 3.2.

The case studies are codified with respect to the specific modeling approaches utilized (Supplemental Table S1), what key question was addressed, whether this most relates to filling knowledge gaps at the compound, disease, and/or mechanism (Table 1). A brief description why this case study is illustrative for a specific application type and an assessment of internal impact (where known) is provided. Based on the authors’ interpretation, an assessment is made with regard to the impact from both an EMA and EFPIA perspective. These full details and references are provided in the supplemental materials that accompany this paper.

Appendix Table 1 Summary of number of papers for each of eight identified application types across the drug development phases

		A	B	C	D	E	F	G	Total
	Application Type (below) /Development Phase (right)	Target selection and validation	Lead Generation and Optimization	Preclinical Development	Early Clinical Development	Late Clinical Development	Approval Phase	Life Cycle Management & Therapeutic use
1	Target authorization and mechanistic understanding	3	2		3	1			9
2	Candidate comparison, selection, human PK and dose prediction		7	6					13
3	Study design optimization		2		4	4			10
4	Predicting and characterizing ADME including intrinsic and extrinsic factors impacting PK variability		3	2	2	9			16
5	Risk/Benefit characterization, and outcome prediction from early clinical responses		2	3	8	2			15
6	Dose and schedule selection and label recommendations (including drug combinations)			1	3	3	3	6	16
7	Comparator / Standard‐of‐Care differentiation and commercialization strategies		2	1	1	7		1	12
8	Patient population selection and bridging between populations (pediatrics, elderly, obese )				1	5	1	5	12
Total		3	18	13	22	31	4	12	103