Verifying and Validating Quantitative Systems Pharmacology and In Silico Models in Drug Development: Current Needs, Gaps, and Challenges

The added value of in silico models (including quantitative systems pharmacology models) for drug development is now unanimously recognized. It is, therefore, important that the standards used are commonly acknowledged by all the parties involved. On April 25 and 26, 2019, a multistakeholder workshop on the validation challenges for in silico models in drug development was organized in Belgium. As an outcome, a White Paper is foreseen in 2020 on standards for in silico model verification and validation.

CURRENT STATUS, GAPS, AND CHALLENGES IN ASSESSMENT OF MODELS FOR REGULATORY SUBMISSIONS

Drug research, design, and development has a long‐standing tradition in the use of in silico methodologies. In the context of clinical drug development Quantitative Structure‐Property Relationship models in general and Quantitative Structure‐Activity Relationship (QSAR) methods in particular, as well as pharmacometric approaches like population pharmacokinetics, pharmacokinetics (PKs)/pharmacodynamics, exposure‐response, and physiology‐based pharmacokinetics (PBPK) models are well‐known. However, the in silico toolbox is rapidly expanding beyond these traditional/historical modeling technologies and new ones have emerged the last decades, including multiphysics simulations, the so‐called systems medicine/pharmacology models (QSP) and clinical trial simulation tools (in silico clinical trials). In the remainder of this document, the term in silico models will be used to describe the collection of all the aforementioned modeling technologies.

The added value of in silico models for drug development is now unanimously recognized by the scientific community. ¹ , ² Irrespective of the model used and the concerned part of the drug development pipeline, the evidence generated from these models, also called digital evidence, might eventually be included in regulatory submissions. In that case, the incorporation of digital evidence needs to follow standards of data/evidence generation, analysis, and reporting to enable the regulatory bodies to efficiently perform an adequate assessment of the submitted material.

It is, therefore, of utmost importance that the standards to be considered are commonly acknowledged by all the involved parties (regulators, health technology assessment (HTA) agencies, academia, industry, regulators, and patients) and are relevant for all the types of models that can be included in regulatory submissions. The endorsement of these standards by regulators is particularly valuable because regulators generally provide guidance for data generation and reporting back to sponsors (industry or academia) thereby accelerating the uptake of the standards in the entire community and in the healthcare systems.

Guidance documents have been published for QSAR models, ³ population PK models, ⁴ PK/pharmacodynamic or exposure‐response models, ⁵ , ⁶ and more recently PBPK models, both by the European Medicines Agency (EMA) and the US Food and Drug Administration (FDA). ⁷ , ⁸ However, these guidelines are not fully applicable to all emerging in silico models without some adaptation or extrapolation. The underlying reasons are multifactorial including (but not limited to) the following:

1. Traditional pharmacometrics models are simpler from a mathematical and numerical point of view as compared with the newly emerging mechanistic models;

2. traditional pharmacometrics models aim at predicting the average behavior of a population of patients rather than the behavior of an individual (virtual) patient predicted by in silico models;

3. newly emerging mechanistic models, depending on their nature, might require more (retrospective and prospective) data to validate their predictions;

4. predictive error is driven by different considerations for the different types of models.

Specific guidance documents on the reporting, verification, and validation of in silico models (including QSP models) for drug development/approval are, therefore, currently an unmet growing need. One of the prerequisites for the development of such regulatory guidance documents, in addition to some skills and experience from the concerned assessors, is the agreement on standards among relevant aforementioned stakeholders and further described hereafter.

Of interest is the standard recently published by the American Society of Mechanical Engineers (ASME) on assessment of credibility of computational modeling through Verification and Validation, applied to medical devices (V&V40). ⁹ The application of this framework to PBPK modeling was published the same year. ¹⁰ In the current situation, a similar initiative oriented to drug development exceeding PBPK would be of great value.

Lessons learned from regulatory guidelines on QSAR and traditional pharmacometric models reflect the general philosophy that model evaluation starts with the regulatory impact assessment closely related to the context of use. ⁵ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ Two important points should be considered: (i) what the impact is of the model prediction on the identification of the appropriate research and development strategy and (ii) what the impact is of the research and development strategy in the regulatory submission. If both impacts are rated as high (e.g., model predictions used to replace a therapeutic study for extension of an indication in children), the requirements regarding overall quality of the model and related data are much more stringent than if both impacts are rated low (e.g., population pharmacokinetic model to describe data from a well‐designed phase I PK study). Moreover, for in silico models, good tracking and adequate reporting of knowledge and data sources, analytical and statistical tools, as well as decision criteria to move to the next step/component, or to assess the whole model should also be part of such guidance documents. In view of the currently unmet need for specific guidance and complexity of the task, tackling the validation challenge of the growing amount of digital evidence is, therefore, not something that any stakeholder should be left alone with, be it the regulators, academia, industry, patients, payers, HTA agencies, or healthcare professionals.

On April 25 and 26, 2019, a workshop was organized in Belgium gathering regulators, academics, and industry to start working on tackling the validation challenge for in silico models in drug development. This successful meeting clearly showed common interest of the participating stakeholders. In a next phase, started September 2019, the initiative was extended to a larger number of stakeholders from the entire European Union (as detailed below) interested in this transdisciplinary inter‐stakeholder project, aiming to provide a roadmap document (White Paper) on standards for assessment of in silico models dedicated to regulatory submission. This White Paper will discuss in detail all the gaps and challenges for in silico models verification and validation as well as the proposed approaches for moving forward illustrated by examples.

STAKEHOLDERS

Regulators

They act as policymakers regarding drug assessment and need to ensure not only that suboptimal models are not being used for decision making but also that good and innovative models are not disregarded, all in the interest of public health. Adequate standards are, therefore, needed by regulators to make proper and consistent assessments in order to play their roles as both gatekeepers and enablers. Given the rapidly evolving field, the training of regulatory experts is made easier when clear standards and related up‐to‐date guidance documents are available.

HTA agencies

HTA agencies, as regulators, need clear standards and related up‐to‐date guidance documents to ensure a correct assessment of the novel drugs developed with the support of in silico models.

Academia

One of the main drivers of innovation, academia, is regrettably not visible enough in the current scene of drug development or evaluation, if not under the umbrella of industry (as external consultants) or regulatory agencies (as external experts). By being part of the reflection on adequate standards for in silico modeling and by adopting these rules (and related terminology), it can be expected that the distance between academia and industry/regulators/patients can be narrowed. Furthermore, academia is the main producer of the data and knowledge on which knowledge‐based models are built. The quality of this production needs to be improved, as shown by the reproducibility crisis, ¹⁰ and verified if incorporated in a model. Without hampering innovation and flexibility inherent to academic research, the developed set of guidelines for verification and validation can also be applied to research models published by academia. Altogether, this will increase the robustness and repeatability of the published body of work and will align methodologies among all the stakeholders of tomorrow.

Industry

Being the current key players for data and related model generation for drug development, it is essential that the industry is involved in the reflection to ensure that the proposed standards are realistic and implementable in practice. The transparency on the criteria and standards on which the produced models would be assessed by the regulator will permit better design and conduct of in‐house modeling related activities and ultimately saving time and resources toward marketing of drugs.

Patients

Having verification and validation guidelines means that in silico models can more readily be used by sponsors, and, per se, this will allow quicker and safer delivery of products to patients. Specifically, for niche populations (pediatrics and rare diseases), in silico might be the only way to obtain sufficient evidence to make rational decisions. In all domains, it should result in less patients enrolled in failed development as well as in successful ones.

CONCLUSION

Despite the unanimous recognition of the added value of the in silico models for drug development, including systems medicine/pharmacology models and clinical trial simulations tools, the availability of specific guidance documents related to these models is currently an unmet growing need.

There is an ongoing initiative in the European Union space, bringing together relevant stakeholders (academia, industry, and regulators) to agree on standards for assessment of these in silico models that will be considered as a premise of dedicated regulatory guidelines. A White Paper is planned for early 2020.

Funding

F.T. Musuamba received funding from European commission: CSA EUSTAND4PM.

Conflict of Interest

The authors declared no competing interests for this work.