Skip to main content
PMC home page
British Journal of Clinical Pharmacology logoLink to British Journal of Clinical Pharmacology
. 2015 Jul 29;80(5):979–991. doi: 10.1111/bcp.12674

Integration of PKPD relationships into benefit–risk analysis

Francesco Bellanti 1, Rob C van Wijk 1, Meindert Danhof 1, Oscar Della Pasqua 1,2,3
PMCID: PMC4631171  PMID: 25940398

Abstract

Aim

Despite the continuous endeavour to achieve high standards in medical care through effectiveness measures, a quantitative framework for the assessment of the benefit–risk balance of new medicines is lacking prior to regulatory approval. The aim of this short review is to summarise the approaches currently available for benefit–risk assessment. In addition, we propose the use of pharmacokinetic–pharmacodynamic (PKPD) modelling as the pharmacological basis for evidence synthesis and evaluation of novel therapeutic agents.

Methods

A comprehensive literature search has been performed using MESH terms in PubMed, in which articles describing benefit–risk assessment and modelling and simulation were identified. In parallel, a critical review of multi-criteria decision analysis (MCDA) is presented as a tool for characterising a drug's safety and efficacy profile.

Results

A definition of benefits and risks has been proposed by the European Medicines Agency (EMA), in which qualitative and quantitative elements are included. However, in spite of the value of MCDA as a quantitative method, decisions about benefit–risk balance continue to rely on subjective expert opinion. By contrast, a model-informed approach offers the opportunity for a more comprehensive evaluation of benefit–risk balance before extensive evidence is generated in clinical practice.

Conclusions

Benefit–risk balance should be an integral part of the risk management plan and as such considered before marketing authorisation. Modelling and simulation can be incorporated into MCDA to support the evidence synthesis as well evidence generation taking into account the underlying correlations between favourable and unfavourable effects. In addition, it represents a valuable tool for the optimization of protocol design in effectiveness trials.

Keywords: benefit–risk assessment, European Medicines Agency, multi criteria decision analysis, not-in-trial simulations, paediatric drug development, pharmacokinetic–pharmacodynamic modelling

Benefit–risk analysis: the current situation

Despite the recognized implications of unmet medical needs and challenges in dealing with new diseases, the current regulatory framework in the European Union has made drug approval a demanding task. This situation is compounded by emerging safety findings, which have led to post-approval withdrawals of more than a dozen products with high therapeutic potential in the past decade 1,2. Such a landscape places regulators, clinical scientists and drug developers with yet another dilemma: how to balance rapid access to new drugs vs. gathering relevant data on efficacy and safety 3? Currently, regulators make these decisions in an isolated, fragmented and, to a large extent, subjective manner.

In principle, the decision to approve a new medicinal product is based on the assumption that a systematic review of all available data provides an accurate, unbiased picture of a drug's efficacy and safety. This assumption may, however, not be true for the large majority of drugs. The evidence generated to support regulatory filing does not always account for the overall heterogeneity of the target population, nor incorporates the impact of treatment on disease progression or external confounding factors on treatment response. Moreover, one needs to acknowledge that the information gathered in the context of pivotal clinical trials may not provide evidence that dose selection, dosing regimen and treatment duration are truly optimal. In fact, poor dose rationale has been a common denominator in numerous publications describing safety issues and attrition in phase II and III trials.

Undoubtedly, efficient gathering and use of data are required to answer the clinical questions that arise with new drugs or therapeutic interventions. Among other things one needs to distinguish effectiveness from clinical response. In addition, it is crucial to understand whether there is added value, as compared with other treatments. These are multidimensional questions which require clear understanding of how data will be generated and how benefit and risk will be quantified. Whereas different theoretical considerations and techniques have been used by health technology assessment agencies, a clear framework for benefit–risk assessment is still lacking during drug development and subsequently at the time of regulatory approval. Consequently, decision making at important milestones in R&D and at submission remains empirical, inconsistent and more often than not, non-transparent 1,4–8.

In the past years awareness about the aforementioned issues has increased significantly. Several projects 9–13 have been funded to evaluate some of the available methodologies and better understand the requirements for a more systematic approach to benefit–risk analysis. In this context, the work of the Committee for Medicinal Products for Human Use (CHMP) of the European Medicines Agency (EMA) is particularly relevant. Starting in 2006, a working group was installed to examine the issue and provide recommendations about ways to improve benefit–risk assessment, including aspects such as transparency, consistency and communication between stakeholders 9. Among the techniques evaluated by the working group, quality-adjusted life years (QALYs) and number needed to treat (NNT) were found to be the most used concepts in clinical practice, very likely due to their simplicity 9,14. However, these methods are imprecise in nature and as such lack some important features that allow one to make appropriate inferences about quantitative differences, especially when comparing treatment options. For example, preventing a stroke is of greater value than preventing a headache. Thus, treatments with dramatically different overall benefit may have similar NNTs. Also, the NNT does not incorporate the type of treatment or treatment adverse effects. A different statistic, the ‘number needed to harm’ (NNH) must be calculated to capture the risk of side-effects. There is a clear need for more quantitative methodologies, which enable better integration of data and facilitate the evaluation of complex clinical scenarios that arise in real life.

Most of these complexities seem to have been addressed by the development of multi-criteria decision analysis (MCDA), an integrative approach that has gained interest from the scientific and clinical community over the last few years. From 2009 to 2011, data can be found for nine products which have been evaluated by MCDA alone, or in combination with simulation, decision trees or Markov modelling 15.

In this review, a brief overview of different techniques for the evaluation of benefit and risk is presented, with especial focus on the contribution of quantitative methodologies to the development and approval of novel medicines. Two main topics are discussed initially. First, the definition of benefit–risk balance and the impact of qualitative and quantitative methodologies on the assessment of benefit and risk during the drug development process 7. In addition, we consider further refinement of the approaches used to characterise benefit–risk balance by integrating it with pharmacokinetic–pharmacodynamic (PKPD) modelling. It is envisaged that modelling and simulation may account for correlations between therapeutic response and adverse events, providing a biologically plausible basis for the analysis of benefit and risk data. The availability of such an integrated approach may enable better choices regarding treatment selection and dose rationale, in particular when dealing with special populations or conditions involving small numbers of patients such as rare diseases.

Methods

Initially, an exploratory literature search was performed to retrieve relevant publications about the use of quantitative approaches for benefit–risk assessment and their impact on decision making in drug development. Seven publications 1,4,14,16–19 were available prior to the exploratory phase, which contributed to the identification and selection of 58 articles, books and reports. From a pool of 65 publications 21 were found which focused on quantitative methodologies (see Tables1). These data were used as a basis for a systematic literature search within PubMed, in which the name of the methodology was combined with relevant terms, such as benefit–risk assessment or analysis. A total of 253 publications from 1990 to 2014 were identified, of which 231 were excluded based on title and abstract information. The resulting 22 publications, together with the previous 65 publications from the exploratory search were then reviewed. During the implementation of the proposal, 23 publications were included based on external advice. Altogether, 110 publications were reviewed. The steps related to the literature search and data abstraction are summarised in the diagram in Figure1.

Figure 1.

Figure 1

Open in a new tab

Flow diagram describing the literature search strategy

Definition of benefit and risk

An important aspect of any benefit–risk analysis is the definition of both terms, and more importantly, how to measure or quantify them. Benefit is usually described as a potential effect that moves the condition of the patient from disease towards health, within a given (pre-defined) context 20–23. Risk is the opposite, a potential effect that moves the condition of the patient from health towards disease, also within a pre-defined context (Table1). To measure both possibilities, at least two concepts play an important role: the magnitude or severity of the effect and its incidence or frequency. Benefit or risk is then estimated by the product of these concepts, possibly multiplied by the duration 21 or the reliability of the data 23.

Table 1.

Glossary of terms

Term Definition
ADE Adverse drug effects
Bayesian statistics Probability-based statistics, concerning parameter values derived from distributions
Benefit Favourable effect, accounting for the uncertainty of that effect (as defined by the EMA)
BILAG-index British Isles Lupus Assessment Group, a measure for severity of SLE
BRAT Benefit Risk Action Team, operating under PhRMA
CHMP Committee for Medicinal Products in Human Use, operating under the EMA
Decision tree Method to aid decision making by visualizing different scenarios as a series of events, and by calculating outcome based on assigned probabilities of the events
DSD Death or serious disabled, measure of estimated outcome in the swine flu case study
EMA European Medicines Agency
FDA Food and Drug Administration [USA]
H1N1 Influenza virus categorized by surface proteins hemagglutinin and neuraminidase (in this case swine flu)
In silico Experiment in a computer, including estimation and prediction of virtual experimental conditions or scenarios
In vitro Experiments in cell culture, tissue or organ preparation
In vivo Experiment in animals (preclinical)
IPRED Individual prediction, possible outcome of PKPD modelling prediction variables and parameter values of an individual patient
Markov model Quantitative method based on parameters describing states and transitions between states
MCDA Multi criteria decision analysis, quantitative method analysing single weighted components of a problem before reassembling it to aid a final decision
NDA New drug application, to be submitted to the FDA for approval before market access
NNH Number needed to harm, measure of the number of patients that has to be treated to yield a single adverse effect
NNT Number needed to treat, measure of the number of patients that has to be treated to prevent a single occurrence
PhRMA Pharmaceutical Research and Manufacturers of America
PKPD Pharmacokinetics and pharmacodynamics, two disciplines within pharmacology concerning what the body does to the drug and what the drug does to the body, respectively
PrOACT-URL Qualitative framework by Hammond, Keeney and Raiffa, consisting of Problem, Objective, Alternatives, Consequences, Trade-offs, Uncertainty, Risk tolerance and Linked decisions
QALY Quality-adjusted life year, measuring the outcome of therapy by the adjustment of a quality life year, in which the patient can fully function (economically)
Risk Unfavourable effect, accounting for the uncertainty of that effect [as defined by the EMA]
RV-NNT Relative value adjusted number needed to treat, a type of NNT accounting for the patient preference as value function
SLE Systemic lupus erythematosus, an autoimmune disease
SLEDAI DLE Disease Activity Index, a measure of severity of SLE
TURBO Transparent Uniform Risk–Benefit Overview
Open in a new tab

Currently a slightly different definition of benefit and risk has been adopted by the EMA. They are defined, respectively, as favourable and unfavourable effects and are at the same time coupled to the uncertainty of both effects (Figure2) 14. Whereas the reasoning seems intuitive, this situation represents a mathematical challenge, i.e. integrating terms or factors that are measured in incommensurable units and in different time scales. Any reliable product of these factors imposes data manipulation or transformation to ensure that all terms are expressed in the same unit and time scale. However, the illusion of this mathematical precision tends to hide another important conceptual challenge: what is acceptable 22? This depends on the perception and values of the stakeholders, i.e. the regulator, the clinical experts and the patients. Procedures have been devised to ensure that perceived benefits and risks are quantified in a systematic manner. This process is known as prior elicitation and involves expert judgment. It is aimed at making subjective opinions more consistent, comprehensive and transparent 20,23,24.

Figure 2.

Figure 2

Open in a new tab

EMA's definition of benefit and risk, where favourable effects are beneficial to the population and unfavourable effects are undesirable for the population. Uncertainty is caused by variation, biased data, limitations of data or methodology etc. Based on 14

Current approaches

The assessment of benefit and risk has evolved in a rather empirical manner and still relies on subjective criteria, in that perceived benefits and risks depend on the context in which the treatment is used, i.e. which standard of care is set as reference and whether short and long term consequences of the intervention are considered against the progression of disease and any correlated co-morbidities or complications. Irrespective of the lack of consensus on how to assess and weight any measures associated with benefit and risk, one needs to consider two different dimensions of the problem. First, a qualitative approach can be used that provides explicit contextualization of the problem. It is crucial to understand fully the main issues before any quantitative analysis starts, i.e. to identify the factors that contribute and/or determine benefit and risk as well as capture the views and differences of opinion from different stakeholders, especially with regard to the perception of risk, in terms of its incidence, severity, chronicity and reversibility.

Whilst quantitative methods can be applied independently from any previous qualitative evaluation, the use of such a stepwise approach may support the identification of suitable measures or criteria in terms of their clinical relevance. However, the use of quantitative methods imposes the availability of sufficient data for those endpoints and measures which have higher weights. It also imposes clear understanding of the trade-offs between benefit and risk, especially of the correlations between outcomes. Whereas these requirements seem obvious, little attention has been paid to the biological or pharmacological basis that determines treatment outcome, i.e. how exposure–response (PKPD) relationships underpin favourable and unfavourable events.

The next paragraphs will provide an overview of the available techniques, including recent examples in which benefit and risk have been evaluated in the context of regulatory approval and treatment optimization. Additional details of the methodologies can be found in the supplementary material (Supplementary Figuress1 and s2 and Supplementary Tabless1 and s2).

Qualitative approaches

A qualitative framework is essential for characterising benefit and risk, as it structures the problem and its context, before any measurement is actually performed. It provides clarity about the possible outcomes as well as the input and the process in between, for example, by defining which decision criteria are to be used. This framework ensures that no alternative measures or trade-offs are overlooked during the subsequent steps, i.e. during which quantitative methods are applied.

The Pharmaceutical Research and Manufacturers of America (PhRMA)

PhRMA assigned a Benefit Risk Action Team (BRAT) to create a decision framework. Their framework consists of six steps which are developed and implemented prior to regulatory approval. Before phase III, focus is given to the definition of a decision frame, identification of relevant outcomes, identification of the data sources and customization of the framework for benefit–risk analysis. At the time of filing and New Drug Application review, attention is paid to the outcome itself as well as to the quantification and interpretation of key benefit–risk metrics 13,14. It should be noted that this framework seems to end with the decision and defence after which a drug is approved or rejected. It does not involve post marketing data, which are known to potentially change benefit–risk balance.

EMA PrOACT

The qualitative framework suggested by the EMA is based on Hammond's, Keeney's & Raiffa's PrOACT approach 25, combined with the less known addition of the so-called URL: problem, objective, alternatives, consequences, trade-offs, uncertainty, risk tolerance and linked decisions. In this way, the problem is clearly structured and information can be gathered in a consistent way to assist the decision-making process 14. Despite its general nature, the use of PrOACT-URL has proven its success since 1999. In contrast to PhRMA BRAT, the inclusion of uncertainty paves the way for a more statistically sound implementation of the approach.

Quantitative approaches

The use of a qualitative framework for assessing benefit and risk may be sufficient when complexity is minimal. This is however not the case in drug development where very complex scenarios arise. To include all data and present a sound overview of all alternatives, consequences and trade-offs, as well as differentiate between objectives otherwise considered comparable, one or more quantitative techniques are required 1,4,11,14,16,17,21,23. A qualitative framework will still be essential to define the problem and the objectives of the analysis and as such will precede the implementation of a quantitative benefit–risk analysis.

In the past decades several methodologies have been developed and used to evaluate the benefit–risk balance of a number of drugs. These methodologies present completely different features and their use has been tailored for very specific cases, contributing to an increase in the number of options available when starting an analysis. These specificities have however made them unsuitable for subsequent application in a general benefit–risk framework. An overview of these methods 1,4,16–19,21,23,26–107, including advantages and limitations, is provided in Supplementary Table S1. By contrast, multi-criteria decision analysis (MCDA) in combination with decision trees has been suggested as a plausible quantitative approach that embeds the needed features for a generalized and structured framework for benefit–risk evaluation.

MCDA presents several advantages compared with other methodologies. The main one is the simplification of a complex problem by breaking it into smaller pieces and making them comparable by weighting their scores on a single scale. Normalizing the different criteria allows comparison on the same ground. In addition, the uncertainty carried by the subjective component, is further reduced by the possibility of performing a sensitivity analysis, in which the model provides different outcomes depending on weights variation. There are, however, still limitations. Given the complexity of the scenarios analysed, it is often expected to observe correlations between the endpoints considered. This is not yet taken into account within the methodology, where each endpoint is analysed in an independent manner. In the systemic lupus erythematosus (SLE) case, which is discussed in the supplementary material, the immunosuppressive effect of Benlysta and the incidence of infection might very well be correlated in a non-linear way. This might influence the outcome, leading to biased results.

Furthermore, it is a matter of concern how the input data for the decision model is provided. This is not a direct limitation of the methodology, but of how the analysis is implemented. Many quantitative methods are limited by statistics and inclusion of uncertainty, confounding factors or limited data. The latter concerns both the experimental data, as well as preference values of different stakeholders required for the weighting criteria 1. MCDA offers a statistically sound method, where probability and uncertainty are combined with preference. Its limitation lies in the complexity of the data required, which are often unavailable, as well as in the subjective judgement that is required and the dependence on risk perception differences. Besides, sequential decisions require data gathering over a longer time period, especially in case of conditional approval 108.

Despite the aforementioned advantages, MCDA, like any other quantitative method, still relies on subjectivity. This is partly overcome by structuring the analysis in a transparent, consistent manner and by incorporating communication with different stakeholders as a critical step 14,15. In fact, communication with different stakeholders is also accounted for in NNT/NNH. Although applicability of the former to benefit–risk assessment in general is very limited because of the lack of preference data, as well as limited statistical power 59,60, it highlights an important issue in communication. For instance, individual patients seem unable to estimate objectively their own chances. In a distribution of 1 out of 20, all 20 patients expect to be the exception, when it comes to a beneficial effect, but not in case of an adverse effect. As a result, the magnitude of risk is misperceived, as the chances of common consequences are underestimated and those of rare consequences are overestimated 8. This problem of risk perception is essential when considering including different stakeholders. Although MCDA does present data in a transparent and consistent way, it is not a technical process, but an effective design of the social processes required for subjective weighting (see Supplementary FigureS3) 43.

Integration of PKPD modelling into benefit–risk analysis

Modelling and simulation techniques represent an invaluable resource for drug development. Of relevance for benefit–risk analysis is the opportunity that PKPD modelling offers in terms of describing variability in a parametric manner. This allows the characterisation and prediction of the time course of treatment response at an individual level under physiological and pathological conditions 109,110. The current emphasis on mechanism-based modelling has also the advantages of increased understanding about drug-specific and system-specific properties such as target site distribution, binding, pharmacokinetic interactions, pharmacodynamic interactions, homeostatic feedback, tolerance and disease progression 111–113. In addition, model-based simulations can provide insight into conditions that may not have been tested experimentally, unravelling patterns or responses that may represent clinically relevant changes in the benefit–risk balance.

From a technical, scientific point of view, modelling and simulation allows for the integration of data and knowledge in a continuous, objective and reproducible manner, thereby enhancing the quality of decision making 110. Over the last decade, the regulatory perception of the role of modelling and simulation in drug development has changed. Its relevance in clinical development has been acknowledged and processes are in place to support a more structured assessment of the use of modelling and simulation in the regulatory approval process 111,114.

In the next paragraphs we describe how the integration of modelling and simulation can be advantageous to further improve the existing framework for the evaluation of benefit–risk balance, as suggested by the EMA. To this purpose, we consider three main aspects, namely, 1) the optimization of evidence that is generated by clinical trials, 2) the evaluation of virtual scenarios and 3) mechanism-based multivariate analysis. The optimization of the input data available for decision making entails not only the integration of data from different trials, but also the use of optimality concepts for the design of prospective clinical studies. The availability of an integrated model allows for the creation of virtual experiments, which provide a more coherent, biologically plausible basis for performing interpolations and extrapolations. In contrast to current practice, multivariate modelling allows one to establish correlations between therapeutic and adverse events of interest, which are often linked by the very pharmacological nature of the treatment. This overview is complemented by a brief discussion of the issues associated with prior elicitation, which could be better guided by the use of models, rather than empirical distributions. As such, a model-based approach could provide somewhat less subjective weighting and preferences.

Optimizing input data

Modelling and simulation techniques can be used to optimize the input data available for the benefit–risk analysis. PKPD modelling allows the creation of a framework that can be refined and improved throughout the development process, by integrating data from different sources as well as by pooling the information gathered across different phases of development. This iterative process allows one to understand and distinguish drug-specific from system-specific properties. Most importantly, it enables the identification of sources of variation and clinical implications thereof. Among other things, benefit–risk analysis could be performed with and without the residual variability or in by inclusion of variability in a stepwise manner. In other words, these procedures increase the value of data whilst decreasing uncertainty 111. On the other hand, modelling and simulation can also be used to optimize the design of prospective clinical trials. The quality of the information collected can be considerably improved through optimal design 115–117, enabling the generation of more informative data input for the decision analysis. This is particularly important in special populations where limited evidence is generated, such as in the case of paediatric and rare diseases 111,118,119. The assumptions about the informative value of data obtained from randomized clinical trials are often overlooked. It is assumed that the output or results from a trial are a consequence of the drug treatment, rather than a consequence of the interaction between drug properties, disease processes, patient characteristics and experimental protocol.

Evidence from virtual scenarios

A second aspect that could be beneficial for benefit–risk assessment is the use of PKPD modelling for simulation purposes. The availability of a qualified or validated model may provide the opportunity to perform virtual experiments. This allows one to explore scenarios that have not been evaluated during clinical development. Not only efficacy and safety data can be considered, but also the influence of covariates such as disease severity, co-medications, co-morbidities and drug compliance can be evaluated. By inter- or extrapolating, new input data can be generated for a different population or different dosing regimens. The results from these simulations can be subsequently used as input for benefit–risk analysis. As mentioned previously, scenario analysis by modelling and simulation may have an even larger impact when considering special populations and rare diseases 119–122.

Correlating multiple endpoints

Thus far we have highlighted the fact that PKPD modelling may reduce the uncertainty in a benefit–risk analysis by optimizing the information used as input. Modelling and simulation techniques may overcome another important limitation of benefit–risk methodologies, namely the assumption that favourable and unfavourable events are clinically, pharmacologically and statistically independent from each other. This assumption violates our current understanding of the nature and cause of adverse events. Hence, any analysis involving multiple endpoints in a multidimensional system will have to account for the correlations between them. Moreover, we believe that these correlations are often non-linear, requiring some advanced statistical techniques to ensure that interactions between variables and covariate factors are captured accordingly. Multidimensional models can be used to assess quantitatively how endpoints are linked together and how response changes with changes in drug exposure 26.

Advantages from the integration of modelling and simulation techniques to benefit–risk analysis are not only conceptual. From a technical perspective, PKPD models may contribute to bias reduction during prior elicitation. In addition, it may provide a stronger basis for sensitivity analysis. Although weighting is a subjective procedure, expert opinions can be modelled using prior elicitation. Moreover, if the uncertainty associated with the weights is assessed, it is possible to factor in the impact of each expert's opinion on the overall analysis. Other possibilities exist to weight the experts input, by scaling their precision based on training and experience, or by assigning them to groups of thought that are more or less representative of the common opinion 28,64. PKPD models describing the underlying disease processes as well as the impact of treatment over time through virtual scenarios may facilitate prior elicitation, providing systematic, consistent input for the evaluation of weights and uncertainties.

An example of the impact of modelling and simulation concepts on benefit–risk analysis is given in Table2.

Table 2.

The impact of modelling and simulation on the MCDA approach is summarised and further elucidated by the example of Benlysta. Emphasis is given to the first steps of the methodology; namely, the input data and evaluation for correlations between parameters and outcomes

MCDA Reference figure Modelling and simulation Example: Benlysta
Step 0: Input data gathering Step 0.1: Explore and refine the informative contents of data, accounting for variability and uncertainty. Step 0.1: Distinguish between-subject variability in relevant parameters from residual error.
Step 0.2: Incorporation of virtual measurements (samples), by evidence generation through simulations. Step 0.2: Evaluate parameter uncertainty by exploring the implications of different experimental protocol conditions.
Step 1: Defining decision context Step 1.1: Prioritizing elements which affect variability and /or uncertainty.
Step 2: Identifying options Step 2.1: Inference by extrapolation, e.g. an additional arm that has not been tested clinically. Step 2.1: Assess treatment response for alternative dosing regimens than the actual treatment arms in the trial (i.e. 1 and 10 mg doses)
Step 3: Identifying objectives and criteria Inline graphic Step 3.1: Assess outcomes taking into account the correlation between events. Step 3.1: The correlation between immunosuppressive effects and incidence of infection can be incorporated into the model, enabling accurate evaluation of the impact of different dose levels on outcome.
Step 4: Scoring Inline graphic Step 4.1: Estimation of the correlation between events in a parametric manner, thereby avoiding biased scoring of the data. Step 4.1: Estimation of the parameters describing the nonlinear relationship between immunosuppressive effects and incidence of infection in patients undergoing long term treatment.
Step 5: Weighting factors for differences of opinion Step 5.1: Prior elicitation of expert opinion can be translated into consistent weighting, including distributions describing differences of opinion (e.g. priors in parameter distributions). Step 5.1: Simulate outcomes for Benlysta-treated patients taking into account different weighting factors.
Step 6: Combining data Step 6.1: Simulated scenarios increase the quality of the data and therefore the quality of the overall value, by increasing granularity. Step 6.1: Simulation of different treatment arms to explore the implications of dose selection.
Step 7: Examining data Inline graphic Step 6.1: Outcome evaluation is not limited to the data, but to evidence arising from virtual clinical trials, including patients who belong to risk groups (e.g. those who meet exclusion criteria)
Step 8: Sensitivity analysis Step 8.1: Irrespective of the decision criteria, model parameters on which the data are based can also be analysed. Step 8.1: The PKPD model of Benlysta is evaluated by sensitivity analysis.
Open in a new tab

Discussion and conclusion

In this short review, an overview was given of the methodologies currently used for the evaluation of benefit–risk balance. Growing consensus suggests that a combined approach involving qualitative and quantitative methods is required to ensure meaningful evaluation and interpretation of benefit and risk data. In fact, this is recommended by the EMA, which suggests the use of PrOACT-URL and MCDA.

Even though a more structured approach is still lacking for benefit–risk analysis, MCDA seems to address the need for a multidimensional characterisation of the scenarios that arise in drug development and in the clinical practice. One of its limitations is the way uncertainty is handled. There is a need to reduce further the uncertainty or preferably to capture it accordingly. Attempts have been made to construct stochastic multi-attribute models, also known as stochastic multi-criteria acceptability analysis (SMAA), which incorporates uncertainty regarding the criteria measurements. Among other things, SMAA provides the possibility to include the sampling variation. It also allows the characterisation of typical trade-offs supporting a drug benefit–risk profile without eliciting the (exact numerical) preferences beforehand 123. An analysis without preference information is valuable when preferences cannot be elicited or when the potential benefits of a drug have to be assessed across a wide range of preferences. This latter situation occurs, for example, when different subgroups of patients are considered. However, stochastic methods do not eliminate discrepancies between perceived risk or benefit and their biological and pharmacological plausibility. Undoubtedly, integration of mechanism-based modelling to multi-criteria decision methods will enhance our ability to characterise benefit–risk balance. It will provide indirect evidence from virtual scenarios in a more effective manner than sensitivity analysis and other statistical techniques have allowed for. Such an integrated approach will also represent an advancement for the field of modelling and simulation, which is often restricted to single endpoints, facilitating the assessment of causality and correlation between favourable and unfavourable events 124.

Unfortunately, there are very few examples in the published literature that present in a clear manner the concepts discussed throughout this manuscript. It is worth mentioning two publications which provide an excellent illustration of these concepts. The work carried out by Bender et al. 125 shows how exposure-response relationships determined by modelling of multiple endpoints can be used to explore and assess benefit–risk across different dosing regimens in the context of oncology trials. In the same way, the work by Pink et al. 126 shows the feasibility of integrating modelling and simulation with pharmacoeconomic analysis to inform decision making throughout the whole drug development process and possibly achieve personalized evaluations. Both examples support the fact that PKPD relationships are crucial in the assessment of a drug's efficacy and safety profile and should not be omitted when performing a benefit–risk appraisal.

In addition, we propose here the use of PKPD modelling as the pharmacological basis for evidence synthesis and evaluation of novel therapeutic agents at the time of regulatory approval. Whilst network meta-analysis (NMA) may be invaluable in the post-marketing authorisation phase, PKPD becomes crucial for a comprehensive benefit–risk evaluation when limited data are available. Various methodologies can be used for the purpose of evidence synthesis, and among them NMA has been widely used in benefit–risk analyses to combine all available evidence 127,128. These approaches rely, however, on a very large amount of information and as discussed previously depend only on the evidence generated. Moreover, in contrast to a model-based approach, they do not warrant understanding of the underlying PKPD mechanisms and therefore are not suitable for prospective evaluation of virtual scenarios through Clinical Trial Simulations and/or Not-in-trial Simulations 129.

In conclusion, it should be highlighted that models do not make decisions, people do. Ultimately, patients, clinicians, drug developers and regulators need to acknowledge that decisions are better made when data are presented and communicated in a clear, systematic manner. PKPD modelling can complement evidence generation by providing stakeholders the opportunity to explore conditions that have not been experimentally tested at the time of the benefit–risk analysis. Regardless of the limitations models and simulation scenarios may have, model-based evaluation is likely to outperform gut feeling, which often prevails in clinical decision-making.

Competing Interests

All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare F.B. had financial support from the DEEP consortium (sponsored by the European Union). There are no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.

This contribution is part of the DEferiprone Evaluation in Paediatrics (DEEP) consortium, supported by the FP7 Framework Research Programme ‘HEALTH- 2010.4.2-1: Off-patent medicines for children’.

Supporting Information

Additional Supporting Information may be found in the online version of this article at the publisher's web-site:

Figure S1 Example of the decision tree for the approval of the swine influenza vaccine in 2009

bcp0080-0979-sd1.pdf (75.3KB, pdf)

Figure S2 Outcomes tree based on the criteria used for the Benlysta example

bcp0080-0979-sd2.pdf (56.3KB, pdf)

Figure S3 Different data presentations to evaluate benefit risk balance

bcp0080-0979-sd3.pdf (84.1KB, pdf)

Table S1 Overview of the available quantitative methodologies to assess benefit-risk balance

bcp0080-0979-sd4.pdf (38.4KB, pdf)

Table S2 Scoring of Benlysta according to the different criteria, as visualised in the decision tree

bcp0080-0979-sd5.pdf (66.9KB, pdf)

References

  1. Guo JJ, Pandey S, Doyle J, Bian B, Lis Y, Raisch DW. A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. Value Health. 2010;13:657–66. doi: 10.1111/j.1524-4733.2010.00725.x. [DOI] [PubMed] [Google Scholar]
  2. Busch M, Walderhaug M, Custer B, Allain J-P, Reddy R, McDonough B. Risk assessment and cost-effectiveness/utility analysis. Biol J Int Assoc Biol Stand. 2009;37:78–87. doi: 10.1016/j.biologicals.2009.01.007. [DOI] [PubMed] [Google Scholar]
  3. Eichler H-G, Pignatti F, Flamion B, Leufkens H, Breckenridge A. Balancing early market access to new drugs with the need for benefit/risk data: a mounting dilemma. Nat Rev Drug Discov. 2008;7:818–26. doi: 10.1038/nrd2664. [DOI] [PubMed] [Google Scholar]
  4. Holden WL. Benefit-risk analysis : a brief review and proposed quantitative approaches. Drug Saf. 2003;26:853–62. doi: 10.2165/00002018-200326120-00002. [DOI] [PubMed] [Google Scholar]
  5. Bjornson D. Interpretation of drug risk and benefit: individual and population perspectives. Ann Pharmacother. 2004;38:694–9. doi: 10.1345/aph.1D401. [DOI] [PubMed] [Google Scholar]
  6. Okie S. Safety in numbers - monitoring risk in approved drugs. N Engl J Med. 2005;352:1173–6. doi: 10.1056/NEJMp058029. [DOI] [PubMed] [Google Scholar]
  7. Califf R. Benefit assessment of therapeutic products: the Centers for Education and Research on Therapeutics. Pharmacoepidemiol Drug Saf. 2007;16:5–16. doi: 10.1002/pds.1215. [DOI] [PubMed] [Google Scholar]
  8. Cohen J, Neumann P. What's more dangerous, your aspirin or your car? Thinking rationally about drug risks (and benefits) Heal Aff. 2007;26:636–46. doi: 10.1377/hlthaff.26.3.636. [DOI] [PubMed] [Google Scholar]
  9. EMA. 2009. Benefit-risk methodology project - Project description.
  10. Liberti B, McAuslane N, Walker S. Progress on the development of a benefit/risk framework for evaluating medicines. Regul Focus. 2010;15:32–7. [Google Scholar]
  11. FDA. 2013. Structured Approach to Benefit-Risk Assessment in Drug Regulatory Decision-Making - Draft PDUFA V Implementation Plan.
  12. Walker S, McAuslane N, Liberti L, Salek S. Measuring benefit and balancing risk: strategies for the benefit-risk assessment of new medicines in a risk-averse environment. Clin Pharmacol Ther. 2009;85:241–6. doi: 10.1038/clpt.2008.277. [DOI] [PubMed] [Google Scholar]
  13. Coplan P, Noel R, Levitan B, Ferguson J, Mussen F. Development of a framework for enhancing the transparency, reproducibility and communication of the benefit–risk balance of medicines. Clin Pharmacol Ther. 2011;89:312–5. doi: 10.1038/clpt.2010.291. [DOI] [PubMed] [Google Scholar]
  14. EMA. 2011. pp. 1–33. Benefit-risk methodology project - work package 2 report: applicability of current tools and processes for regulatory benefit-risk assessment.
  15. Phillips L. 2013. The European Medicines Agency Benefit-Risk Project (presentation)
  16. Curtin F, Schulz P. Assessing the benefit: risk ratio of a drug - randomized naturalistic evidence. Dialogues Clin Neurosci. 2011;13:183–90. doi: 10.31887/DCNS.2011.13.2/fcurtin. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Puhan MA, Singh S, Weiss CO, Varadhan R, Boyd CM. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol. 2012;12:173. doi: 10.1186/1471-2288-12-173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Inn G, Washington DC, Philips L, Cone M. 2006. Benefit-risk assessment model for medicines : developing a structured approach to decision making. Report of the Workshop organised by the CMR International Institute for Regulatory Science Stuart Walker,
  19. Ouellet D. Benefit-risk assessment: the use of clinical utility index. Expert Opin Drug Saf. 2010;9:289–300. doi: 10.1517/14740330903499265. [DOI] [PubMed] [Google Scholar]
  20. Lane D, Hutchinson T. The notion of ‘Acceptable Risk’: The role of utility in drug management. J Chronic Dis. 1987;40:621–5. doi: 10.1016/0021-9681(87)90023-3. [DOI] [PubMed] [Google Scholar]
  21. Edwards IR, Wiholm BEMC. Concepts in risk-benefit assessment. A simple merit analysis of a medicine? Drug Saf. 1996;15:1–7. doi: 10.2165/00002018-199615010-00001. [DOI] [PubMed] [Google Scholar]
  22. Ernst E, Resch K. Risk-benefit ratio or risk-benefit nonsense? J Clin Epidemiol. 1996;49:1203–4. doi: 10.1016/0895-4356(96)00187-4. [DOI] [PubMed] [Google Scholar]
  23. Beckmann J. Basic aspects of risk-benefit analysis. Semin Thromb Hemost. 1999;25:89–95. doi: 10.1055/s-2007-996430. [DOI] [PubMed] [Google Scholar]
  24. Morgan G, Henrion M. Uncertainty - A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge, United Kingdom: Cambridge University Press; 1990. [Google Scholar]
  25. Hammond J, Keeney R, Raiffa H. Smart Choices. Boston, USA: Harvard Business School Press; 1999. [Google Scholar]
  26. Ahn JE, French JL. Longitudinal aggregate data model-based meta-analysis with NONMEM: approaches to handling within treatment arm correlation. J Pharmacokinet Pharmacodyn. 2010;37:179–201. doi: 10.1007/s10928-010-9152-6. [DOI] [PubMed] [Google Scholar]
  27. Airoldi M, Morton A. Adjusting life for quality or disability: stylistic difference or substantial dispute? Health Econ. 2009;18:1237–47. doi: 10.1002/hec.1424. [DOI] [PubMed] [Google Scholar]
  28. Albert I, Donnet S, Guihenneuc-Jouyaux C, Low-Choy S, Mengersen K, Rousseau J. Combining expert opinions in prior elicitation. Bayesian Anal. 2012;7:503–32. [Google Scholar]
  29. Ashby D, Smith AF. Evidence-based medicine as Bayesian decision-making. Stat Med. 2000;19:3291–305. doi: 10.1002/1097-0258(20001215)19:23<3291::aid-sim627>3.0.co;2-t. [DOI] [PubMed] [Google Scholar]
  30. Barrett J, Skolnik J, Jayaraman B, Patel D, Adamson P. Discrete event simulation applied to pediatric phase I oncology designs. Clin Pharmacol Ther. 2008;84:729–33. doi: 10.1038/clpt.2008.193. [DOI] [PubMed] [Google Scholar]
  31. Berchialla P, Scarinzi C, Snidero S, Gregori D. Comparing models for quantitative risk assessment: an application to the European Registry of foreign body injuries in children. Stat Methods Med Res. 2013 doi: 10.1177/0962280213476167. . DOI: 10.1177/0962280213476167. [DOI] [PubMed] [Google Scholar]
  32. Brass EP, Lofstedt R, Renn O. Improving the decision-making process for nonprescription drugs: a framework for benefit-risk assessment. Clin Pharmacol Ther. 2011;90:791–803. doi: 10.1038/clpt.2011.231. [DOI] [PubMed] [Google Scholar]
  33. Janssen M, Over J, van der Poel C, Cuijpers H, van Hout B. A probabilistic model for analyzing viral risks of plasma-derived medicinal products. Transfusion. 2008;48:153–62. doi: 10.1111/j.1537-2995.2007.01493.x. [DOI] [PubMed] [Google Scholar]
  34. Chauvin P, Josselin J-M, Heresbach D. Incremental net benefit and acceptability of alternative health policies: a case study of mass screening for colorectal cancer. Eur J Health Econ. 2012;13:237–50. doi: 10.1007/s10198-011-0300-8. [DOI] [PubMed] [Google Scholar]
  35. Chawla A, Mytelka D, McBride S, Nellesen D, Elkins B, Ball D, Kalsekar A, Towse A, Garrison LP. Estimating the incremental net health benefit of requirements for cardiovascular risk evaluation for diabetes therapies. Pharmacoepidemiol Drug Saf. 2014;23:268–77. doi: 10.1002/pds.3559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Chuang-Stein C. A new proposal for benefit-less-risk analysis in clinical trials. Control Clin Trials. 1994;15:30–43. doi: 10.1016/0197-2456(94)90026-4. [DOI] [PubMed] [Google Scholar]
  37. CIOMS working group I. 1998. Benefit-Risk Balance for Marketed Drugs: Evaluating Safety Signals. Report of CIOMS Working Group IV,
  38. Citrome L, Ketter TA, Cucchiaro J, Loebel A. Clinical assessment of lurasidone benefit and risk in the treatment of bipolar I depression using number needed to treat, number needed to harm, and likelihood to be helped or harmed. J Affect Disord. 2014;155:20–7. doi: 10.1016/j.jad.2013.10.040. . Elsevier. [DOI] [PubMed] [Google Scholar]
  39. Corey-Lisle PK, Peck R, Mukhopadhyay P, Orsini L, Safikhani S, Bell JA, Hortobagyi G, Roche H, Conte P, Revicki DA. Q-TWiST analysis of ixabepilone in combination with capecitabine on quality of life in patients with metastatic breast cancer. Cancer. 2012;118:461–8. doi: 10.1002/cncr.26213. [DOI] [PubMed] [Google Scholar]
  40. Corso PS, Hammitt JK, Graham JD, Dicker RC, Goldie SJ. Assessing preferences for prevention versus treatment using willingness to pay. Med Decis Mak. 2002;22(5 suppl):S92–101. doi: 10.1177/027298902237713. [DOI] [PubMed] [Google Scholar]
  41. DeCosse JJ, Cennerazzo WJ. A quality-adjusted time without symptoms or toxicity (Q-TWiST) analysis of adjuvant radiation therapy and chemotherapy for resectable rectal cancer. J Natl Cancer Inst. 1996;88:1686. doi: 10.1093/jnci/88.22.1686. [DOI] [PubMed] [Google Scholar]
  42. Djulbegovic B, Hozo II, Fields K, Sullivan D. High-dose chemotherapy in the adjuvant treatment of breast cancer: benefit/risk analysis. Cancer Control. 1998;5:394–405. doi: 10.1177/107327489800500502. [DOI] [PubMed] [Google Scholar]
  43. Dogson JS. Multi-criteria analysis: a manual. 2009. Wetherby, United Kingdom Communities and Local Government Publications.
  44. Donovan AK, Smith KJ, Ragni MV. Anticoagulation duration in heterozygous factor V Leiden: a decision analysis. Thromb Res. 2013;132:724–8. doi: 10.1016/j.thromres.2013.09.012. [DOI] [PubMed] [Google Scholar]
  45. Earnshaw SR, Brogan AP, McDade CL. Model-based cost-effectiveness analyses for prostate cancer chemoprevention: a review and summary of challenges. Pharmacoeconomics. 2013;31:289–304. doi: 10.1007/s40273-013-0037-6. [DOI] [PubMed] [Google Scholar]
  46. EMA. 2012. pp. 1–20. Benefit-risk methodology project Work package 4 report: benefit-risk tools and processes.
  47. Eriksen S, Keller LR. A multiattribute-utility-function approach to weighing the risks and benefits of pharmaceutical agents. Med Decis Mak. 1993;13:118–25. doi: 10.1177/0272989X9301300205. [DOI] [PubMed] [Google Scholar]
  48. Fahey T, Griffiths S, Peters TJ. Evidence based purchasing: understanding results of clinical trials and systematic reviews. BMJ. 1995;311:1056–60. doi: 10.1136/bmj.311.7012.1056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Felli JC, Noel RA, Cavazzoni PA. A multiattribute model for evaluating the benefit-risk profiles of treatment alternatives. Med Decis Making. 2009;29:104–15. doi: 10.1177/0272989X08323299. [DOI] [PubMed] [Google Scholar]
  50. Francis GS. Importance of benefit-to-risk assessment for disease-modifying drugs used to treat MS. J Neurol. 2004;251(Suppl):v42–9. doi: 10.1007/s00415-004-1507-8. [DOI] [PubMed] [Google Scholar]
  51. Gelber RD, Gelman RS, Goldhirsch A. A quality-of-life-oriented endpoint for comparing therapies. Biometrics. 1989;45:781–95. [PubMed] [Google Scholar]
  52. Gelber RD, Cole BF, Gelber S, Goldhirsch A. Comparing treatments using quality-adjusted survival: The Q-TWiST method. Am Stat. 2014;49:161–9. [Google Scholar]
  53. Geva A, Gray J. A quantitative analysis of optimal treatment capacity for perinatal asphyxia. Med Decis Making. 2012;32:266–72. doi: 10.1177/0272989X11421527. [DOI] [PubMed] [Google Scholar]
  54. Greving JP, Buskens E, Koffijberg H, Algra A. Cost-effectiveness of aspirin treatment in the primary prevention of cardiovascular disease events in subgroups based on age, gender, and varying cardiovascular risk. Circulation. 2008;117:2875–83. doi: 10.1161/CIRCULATIONAHA.107.735340. [DOI] [PubMed] [Google Scholar]
  55. Gupta SK. Use of Bayesian statistics in drug development: advantages and challenges. Int J Appl Basic Med Res. 2012;2:3–6. doi: 10.4103/2229-516X.96789. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Hauser JR. 2002. Conjoint analysis, related modeling, and applications. Advances in marketing research: Progress and Prospects,
  57. Hirsch GB, Edelstein BL, Frosh M, Anselmo T. A simulation model for designing effective interventions in early childhood caries. Prev Chronic Dis. 2012;9:E66. doi: 10.5888/pcd9.110219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Hodder SC, Edwards MJ, Brickley MR, Shepherd JP. Multiattribute utility assessment of outcomes of treatment for head and neck cancer. Br J Cancer. 1997;75:898–902. doi: 10.1038/bjc.1997.158. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Holden WL, Juhaeri J, Dai W. Benefit-risk analysis: examples using quantitative methods. Pharmacoepidemiol Drug Saf. 2003;12:693–7. doi: 10.1002/pds.794. [DOI] [PubMed] [Google Scholar]
  60. Holden WL, Juhaeri J, Dai W. Benefit-risk analysis: a proposal using quantitative methods. Pharmacoepidemiol Drug Saf. 2003;12:611–6. doi: 10.1002/pds.887. [DOI] [PubMed] [Google Scholar]
  61. Hynes N, Sultan S. A prospective clinical, economic, and quality-of-life analysis comparing endovascular aneurysm repair (EVAR), open repair, and best medical treatment in high-risk patients with abdominal aortic aneurysms suitable for EVAR: the Irish patient trial. J Endovasc Ther. 2007;14:763–76. doi: 10.1583/07-2194.1. [DOI] [PubMed] [Google Scholar]
  62. Ijzerman MJ, van Til JA, Bridges JFP. A comparison of analytic hierarchy process and conjoint analysis methods in assessing treatment alternatives for stroke rehabilitation. Patient. 2012;5:45–56. doi: 10.2165/11587140-000000000-00000. [DOI] [PubMed] [Google Scholar]
  63. Khan AA, Perlstein I, Krishna R. The use of clinical utility assessments in early clinical development. AAPS J. 2009;11:33–8. doi: 10.1208/s12248-008-9074-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Kinnersley N, Day S. Structured approach to the elicitation of expert beliefs for a Bayesian-designed clinical trial: a case study. Pharm Stat. 2013;12:104–13. doi: 10.1002/pst.1552. [DOI] [PubMed] [Google Scholar]
  65. Kopylev L, Chen C, White P. Towards quantitative uncertainty assessment for cancer risks: central estimates and probability distributions of risk in dose-response modeling. Regul Toxicol Pharmacol. 2007;49:203–7. doi: 10.1016/j.yrtph.2007.08.002. [DOI] [PubMed] [Google Scholar]
  66. Korsan B, Dykstra K, Pullman W. A clinical utility index ( CUI ) openly evaluates a product's attributes - and chance. Pharm Exec. 2005 . Available at http://www.pharmexec.com/transparent-trade-offs (last accessed on 14 January 2015) [Google Scholar]
  67. Krishna R. Model-based benefit-risk assessment: can Archimedes help? Clin Pharmacol Ther. 2009;85:239–40. doi: 10.1038/clpt.2008.240. [DOI] [PubMed] [Google Scholar]
  68. Langley RG, Strober BE, Gu Y, Rozzo SJ, Okun MM. Benefit-risk assessment of tumour necrosis factor antagonists in the treatment of psoriasis. Br J Dermatol. 2010;162:1349–58. doi: 10.1111/j.1365-2133.2010.09707.x. [DOI] [PubMed] [Google Scholar]
  69. Leil TA, Feng Y, Zhang L, Paccaly A, Mohan P, Pfister M. Quantification of apixaban's therapeutic utility in prevention of venous thromboembolism: selection of phase III trial dose. Clin Pharmacol Ther. 2010;88:375–82. doi: 10.1038/clpt.2010.106. [DOI] [PubMed] [Google Scholar]
  70. Levitan B, Yuan Z, Turpie AGG, Friedman RJ, Homering M, Berlin JA, Berkowitz SD, Weinstein RB, DiBattiste PM. Benefit-risk assessment of rivaroxaban versus enoxaparin for the prevention of venous thromboembolism after total hip or knee arthroplasty. Vasc Health Risk Manag. 2014;10:157–67. doi: 10.2147/VHRM.S54714. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Loftus EV, Johnson SJ, Wang S-T, Wu E, Mulani PM, Chao J. Risk-benefit analysis of adalimumab versus traditional non-biologic therapies for patients with Crohn's disease. Inflamm Bowel Dis. 2011;17:127–40. doi: 10.1002/ibd.21341. [DOI] [PubMed] [Google Scholar]
  72. Lynd LD, Marra CA, Najafzadeh M, Sadatsafavi M. A quantitative evaluation of the regulatory assessment of the benefits and risks of rofecoxib relative to naproxen: an application of the incremental net-benefit framework y. Pharmacoepidemiol Drug Saf. 2010;19:1172–80. doi: 10.1002/pds.1994. [DOI] [PubMed] [Google Scholar]
  73. Lynd LD, Najafzadeh M, Colley L, Byrne MF, Willan AR, Sculpher MJ, Johnson FR, Hauber AB. Using the incremental net benefit framework for quantitative benefit-risk analysis in regulatory decision-making--a case study of alosetron in irritable bowel syndrome. Value Health. 2010;13:411–7. doi: 10.1111/j.1524-4733.2009.00595.x. [DOI] [PubMed] [Google Scholar]
  74. Lynd LD, O'brien BJ. Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. J Clin Epidemiol. 2004;57:795–803. doi: 10.1016/j.jclinepi.2003.12.012. [DOI] [PubMed] [Google Scholar]
  75. Mann J, Ellis S, Waternaux C, Liu X, Oquendo M, Malone K. Classification trees distinguish suicide attempters in major psychiatric disorders: a model of clinical decision making. J Clin Psychiatry. 2013;69:23–31. doi: 10.4088/jcp.v69n0104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  76. Marino P, Roché H, Moatti J-P. High-dose chemotherapy for patients with high-risk breast cancer: a clinical and economic assessment using a quality-adjusted survival analysis. Am J Clin Oncol. 2008;31:117–24. doi: 10.1097/COC.0b013e3181573e83. [DOI] [PubMed] [Google Scholar]
  77. Meyboom R, Egberts A. Comparing therapeutic benefit and risk. Therapie. 1999;54:29–34. [PubMed] [Google Scholar]
  78. Mounier N, Haioun C, Cole BF, Gisselbrecht C, Sebban C, Morel P, Marit G, Bouabdallah R, Ravoet C, Salle G, Reyes F, Lepage E. Quality of life-adjusted survival analysis of high-dose therapy with autologous bone marrow transplantation versus sequential chemotherapy for patients with aggressive lymphoma in first complete remission. Groupe d'Etude les Lymphomes de l'Adulte (GELA) Blood. 2000;95:3687–92. [PubMed] [Google Scholar]
  79. Mudd PN, Groenendaal H, Bush MA, Schmith VD. Probabilistic risk analysis: improving early drug development decision making. Clin Pharmacol Ther. 2010;88:871–5. doi: 10.1038/clpt.2010.231. [DOI] [PubMed] [Google Scholar]
  80. Mussen F, Salek S, Walker S. 2009. Benefit-risk appraisal of medicines. A systematic approach to decision-making,
  81. Mussen F, Salek S, Walker S. A quantitative approach to benefit-risk assessment of medicines – part 1: the development of a new model using multi-criteria decision analysis y. Pharmacoepidemiol Drug Saf. 2007;1:S2–S15. doi: 10.1002/pds.1435. ; 16 Suppl. [DOI] [PubMed] [Google Scholar]
  82. Nixon RM, Hagan AO, Oakley J, Madan J, Stevens JW, Bansback N, Brennan A. The Rheumatoid Arthritis Drug Development Model: a case study in Bayesian clinical trial simulation. Pharm Stat. 2009;8:371–89. doi: 10.1002/pst.368. [DOI] [PubMed] [Google Scholar]
  83. Ohlssen D, Price KL, Xia HA, Hong H, Kerman J, Fu H, Quartey G, Heilmann CR, Ma H, Carlin BP. Guidance on the implementation and reporting of a drug safety Bayesian network meta-analysis. Pharm Stat. 2014;13:55–70. doi: 10.1002/pst.1592. [DOI] [PubMed] [Google Scholar]
  84. Ouellet D, Werth J, Parekh N, Feltner D, McCarthy B, Lalonde RL. The use of a clinical utility index to compare insomnia compounds: a quantitative basis for benefit-risk assessment. Clin Pharmacol Ther. 2009;85:277–82. doi: 10.1038/clpt.2008.235. [DOI] [PubMed] [Google Scholar]
  85. Pfohl M, Schädlich PK, Dippel F-W, Koltermann KC. Health economic evaluation of insulin glargine vs NPH insulin in intensified conventional therapy for type 1 diabetes in Germany. J Med Econ. 2012;15(Suppl 2):14–27. doi: 10.3111/13696998.2012.713879. [DOI] [PubMed] [Google Scholar]
  86. Phillips LD, Fasolo B, Zafiropoulous N, Eichler H-G, Ehmann F, Jekerle V, Kramarz P, Nicoll A, Lönngren T. Modelling the risk-benefit impact of H1N1 influenza vaccines. Eur J Public Health. 2013;23:674–8. doi: 10.1093/eurpub/ckt006. [DOI] [PubMed] [Google Scholar]
  87. Poland B, Hodge FL, Khan A, Clemen RT, Wagner JA, Dykstra K, Krishna R. The clinical utility index as a practical multiattribute approach to drug development decisions. Clin Pharmacol Ther. 2009;86:105–8. doi: 10.1038/clpt.2009.71. [DOI] [PubMed] [Google Scholar]
  88. Green P, Rao V. Conjoint Measurement for Data Quantifying Judgmental. J Mark Res. 2014;8:355–63. [Google Scholar]
  89. Sabin T, Matcham J, Bray S, Copas A, Parmar MKB. A quantitative process for enhancing end of phase 2 decisions. Stat Biopharm Res. 2014;6:67–77. doi: 10.1080/19466315.2013.852617. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Sadatsafavi M, Marra C, Marra F, Moran O, FitzGerald JM, Lynd L. A quantitative benefit-risk analysis of isoniazid for treatment of latent tuberculosis infection using incremental benefit framework. Value Health. 2013;16:66–75. doi: 10.1016/j.jval.2012.09.006. [DOI] [PubMed] [Google Scholar]
  91. Shaffer ML, Watterberg KL. Joint distribution approaches to simultaneously quantifying benefit and risk. BMC Med Res Methodol. 2006;6:48. doi: 10.1186/1471-2288-6-48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Shepherd J, Brodin H, Cave C, Waugh N, Price A, Gabbay J. Pegylated interferon a-2a and -2b in combination with raibavirin in the treatment of chronic hepatitis C: a systematic review and economic evaluation. Health Technol Assess (Rockv) 2004:1–125. doi: 10.3310/hta8390. [DOI] [PubMed] [Google Scholar]
  93. Sherrill B, Amonkar MM, Stein S, Walker M, Geyer C, Cameron D. Q-TWiST analysis of lapatinib combined with capecitabine for the treatment of metastatic breast cancer. Br J Cancer. 2008;99:711–5. doi: 10.1038/sj.bjc.6604501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  94. Sherrill B, Wang J, Kotapati S, Chin K. Q-TWiST analysis comparing ipilimumab/dacarbazine vs placebo/dacarbazine for patients with stage III/IV melanoma. Br J Cancer. 2013;109:8–13. doi: 10.1038/bjc.2013.298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  95. Smart A. A multi-dimensional model of clinical utility. Int J Qual Health Care. 2006;18:377–82. doi: 10.1093/intqhc/mzl034. [DOI] [PubMed] [Google Scholar]
  96. Stojadinovic A, Nissan A, Eberhardt J, Chua TC, Pelz JO, Esquivel J. Development of a Bayesian belief network model for personalized prognostic risk assessment in colon carcinomatosis. Am Surg. 2011;77:221–30. [PubMed] [Google Scholar]
  97. Stonebraker JS. How Bayer Makes Decisions to Develop New Drugs. Interfaces (Providence) 2002;32:77–90. [Google Scholar]
  98. Thompson JP, Noyes K, Dorsey ER, Schwid SR, Holloway RG. Quantitative risk-benefit analysis of natalizumab. Neurology. 2008;71:357–64. doi: 10.1212/01.wnl.0000319648.65173.7a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  99. Tomeczkowski J, Stern S, Müller A, von Heymann C. Potential cost saving of Epoetin alfa in elective hip or knee surgery due to reduction in blood transfusions and their side effects: a discrete-event simulation model. PLoS One. 2013;8:e72949. doi: 10.1371/journal.pone.0072949. [DOI] [PMC free article] [PubMed] [Google Scholar]
  100. Verduijn M, Grootendorst DC, Dekker FW, Jager KJ, le Cessie S. The analysis of competing events like cause-specific mortality - beware of the Kaplan–Meier method. Nephrol Dial Transplant. 2011;26:56–61. doi: 10.1093/ndt/gfq661. [DOI] [PubMed] [Google Scholar]
  101. Weinstein L, Radano TA, Jack T, Kalina P, Eberhardt JS. Application of multivariate probabilistic (Bayesian) networks to substance use disorder risk stratification and cost estimation. Perspect Health Inf Manag. 2009;6:1–27. [PMC free article] [PubMed] [Google Scholar]
  102. Keeney RL, Raiffa H. Decisions with Multiple Objectives. Cambridge: Cambridge University Press; 1993. [Google Scholar]
  103. Phillips L. 2010. Improving the process of balancing benefits and risks in approving drugs (presentation). EMA Decision Analysis Affinity Group Workshop,
  104. Phillips L. 2012. Benefit-risk modelling of pharmaceuticals: where are we now? (presentation). EFSPI-FMS-DSBS Benefit Risk Assessment Methodology Workshop 7,
  105. Lam G, Petri M. Assessment of systemic lupus erythematosus. Clin Exp Rheumatol. 2005;23:S120–32. [PubMed] [Google Scholar]
  106. French S, Xu D. Comparison study of multi attribute decision analytic software. J Multi Criteria Decis Anal. 2005;13:65–80. [Google Scholar]
  107. EMA. 2013. EPAR Benlysta: summary of product characteristics.
  108. Walker S, Cone M. 2004. The Development of a Model for Benefit-Risk Assessment of Medicines Based on Multi-Criteria Decision Analysis (Summary Report)
  109. Breimer D. PK/PD modelling and beyond: impact on drug development. Pharm Res. 2008;25:2720–2. doi: 10.1007/s11095-008-9717-x. [DOI] [PubMed] [Google Scholar]
  110. Breimer D, Danhof M. Relevance of the application of pharmacokinetic-pharmacodynamic modelling concepts in drug development - the wooden shoe paradigm. Clin Pharmacokinet. 1997;32:259–67. doi: 10.2165/00003088-199732040-00001. [DOI] [PubMed] [Google Scholar]
  111. Zineh I, Woodcock J. Clinical pharmacology and the catalysis of regulatory science: opportunities for the advancement of drug development and evaluation. Clin Pharmacol Ther. 2013;93:515–25. doi: 10.1038/clpt.2013.32. [DOI] [PubMed] [Google Scholar]
  112. Jonker D, Visser S, van der Graaf P, Voskuyl R, Danhof M. Towards a mechanism-based analysis of pharmacodynamic drug–drug interactions in vivo. Pharmacol Ther. 2005;106:1–18. doi: 10.1016/j.pharmthera.2004.10.014. [DOI] [PubMed] [Google Scholar]
  113. Danhof M, de Jongh J, De Lange E, Della Pasqua O, Ploeger B, Voskuyl R. Mechanism-based pharmacokinetic-pharmacodynamic modeling: biophase distribution, receptor theory, and dynamical systems analysis. Annu Rev Pharmacol Toxicol. 2007;47:357–400. doi: 10.1146/annurev.pharmtox.47.120505.105154. [DOI] [PubMed] [Google Scholar]
  114. Machado S, Miller R, Hu C. A regulatory perspective on pharmacokinetic/pharmacodynamic modelling. Stat Methods Med Res. 1999;8:217–45. doi: 10.1177/096228029900800304. [DOI] [PubMed] [Google Scholar]
  115. Sjörgen E, Nyberg J, Magnusson M, Lennernäs H, Hooker A, Bredber U. Optimal experimental design for assessment of enzyme kinetics in a drug discovery screening environment. Drug Metab Dispos. 2011;39:858–63. doi: 10.1124/dmd.110.037309. [DOI] [PubMed] [Google Scholar]
  116. Yousef A, Melhem M, Xue B, Arafat T, Reynolds D, Van Wart S. Population pharmacokinetic analysis of clopidogrel in healthy Jordanian subjects with emphasis optimal sampling strategy. Biopharm Drug Dispos. 2013;226:215–26. doi: 10.1002/bdd.1839. [DOI] [PubMed] [Google Scholar]
  117. Lim H, Kim S, Noh Y, Lee B, Jin S, Park H, Kim S, Jang IJ, Kim SE. Exploration of optimal dosing regimens of haloperidol, a D2 antagonist, via modeling and simulation analysis in a D2 receptor occupancy study. Pharm Res. 2013;30:683–93. doi: 10.1007/s11095-012-0906-2. [DOI] [PubMed] [Google Scholar]
  118. Sammons H, Starkey E. Ethical issues of clinical trials in children. Paediatr Child Heal. 2012;22:47–50. [Google Scholar]
  119. Bellanti F, Della PasquaO. Modelling and simulation as research tools in paediatric drug development. Eur J Clin Pharmacol. 2011;67(Suppl 1):75–86. doi: 10.1007/s00228-010-0974-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  120. Cella M, Zhao W, Jacqz-Aigrain E, Burger D, Danhof M, Della Pasqua O. Paediatric drug development: are population models predictive of pharmacokinetics across paediatric populations? Br J Clin Pharmacol. 2011;72:454–64. doi: 10.1111/j.1365-2125.2011.03992.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  121. Manolis E, Osman T, Herold R, Koenig F, Tomasi P, Vamvakas S, Saint RA. Role of modeling and simulation in pediatric investigation plans. Paediatr Anaesth. 2011;21:214–21. doi: 10.1111/j.1460-9592.2011.03523.x. [DOI] [PubMed] [Google Scholar]
  122. Manolis E, Pons G. Proposals for model-based paediatric medicinal development within the current European Union regulatory framework. Br J Clin Pharmacol. 2009;68:493–501. doi: 10.1111/j.1365-2125.2009.03484.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  123. Tervonen T, Van Valkenhoef G, Buskens E, Hillege HL, Postmus D. A stochastic multi-criteria model for evidence-based decision making in drug benefit-risk analysis. Stat Med. 2011;30:1419–28. doi: 10.1002/sim.4194. [DOI] [PubMed] [Google Scholar]
  124. EMA. 2012. Benefit-risk methodology project - Report on risk perception study module.
  125. Bender BC, Schindler E, Friberg LE. Population pharmacokinetic pharmacodynamic modelling in oncology: a tool for predicting clinical response. Br J Clin Pharmacol. 2013;79:56–71. doi: 10.1111/bcp.12258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  126. Pink J, Lane S, Hughes DA. Mechanism-based approach to the economic evaluation of pharmaceuticals. Pharmacoeconomics. 2012;30:413–29. doi: 10.2165/11591540-000000000-00000. [DOI] [PubMed] [Google Scholar]
  127. Naci H, van Valkenhoef G, Higgins JPT, Fleurence R, Ades AE. Evidence-based prescribing: combining network meta-analysis with multicriteria decision analysis to choose among multiple drugs. Circ Cardiovasc Qual Outcomes. 2014;7:787–92. doi: 10.1161/CIRCOUTCOMES.114.000825. [DOI] [PubMed] [Google Scholar]
  128. Van Valkenhoef G, Tervonen T, Zhao J, De Brock B, Hillege HL, Postmus D. Multicriteria benefit-risk assessment using network meta-analysis. J Clin Epidemiol. 2012;65:394–403. doi: 10.1016/j.jclinepi.2011.09.005. [DOI] [PubMed] [Google Scholar]
  129. Chain ASY, Dieleman JP, van Noord CHA, Stricker BHC, Danhof M, Sturkenboom MCJM, Della PasquaO. Not-in-trial simulation I: bridging cardiovascular risk from clinical trials to real-life conditions. Br J Clin Pharmacol. 2013;76:964–72. doi: 10.1111/bcp.12151. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Figure S1 Example of the decision tree for the approval of the swine influenza vaccine in 2009

bcp0080-0979-sd1.pdf (75.3KB, pdf)

Figure S2 Outcomes tree based on the criteria used for the Benlysta example

bcp0080-0979-sd2.pdf (56.3KB, pdf)

Figure S3 Different data presentations to evaluate benefit risk balance

bcp0080-0979-sd3.pdf (84.1KB, pdf)

Table S1 Overview of the available quantitative methodologies to assess benefit-risk balance

bcp0080-0979-sd4.pdf (38.4KB, pdf)

Table S2 Scoring of Benlysta according to the different criteria, as visualised in the decision tree

bcp0080-0979-sd5.pdf (66.9KB, pdf)

Articles from British Journal of Clinical Pharmacology are provided here courtesy of British Pharmacological Society

RESOURCES