Value of evidence in drug development
Integrated evidence plans (IEPs) are used in the pharmaceutical industry to address evidence gaps across all functions, the product life cycle and geographies, aiming to meet the needs of both external and internal stakeholders as well as to result in considerable efficiency gains [1,2]. They represent a fundamental shift in the way evidence is proposed and generated in the pharmaceutical industry, promoting a consistent, efficient and collaborative approach to evidence generation. They also incorporate a wide array of data types and methodologies, and offer a detailed, reliable and practical perspective on complex matters. IEPs capture all existing and planned evidence, identify unmet evidence requirements and seek to prioritize and generate the necessary evidence in a structured and collaborative manner. This strategic approach aligns R&D, medical, regulatory, HEOR, market access and commercial objectives for pharmaceutical products, aiming to optimize evidence generation and maximize its value, enhancing the quality and effectiveness of both internal and external decision-making processes.
An important question that arises with evidence generation is, what is its value and why would one change from the existing evidence generation models? A common answer to the question was that more evidence must be better so there was a push to generate all the evidence that the budget could afford and at the time the gap was identified. A look into the ‘value of evidence’ from decision theory suggests that it is not always better to have additional information and can depend on anticipated failures of rationality and the issue of cost [3]. The idea being that not all recipients will use the evidence rationally and that additional evidence might be good if it were free. Another perception of the value of evidence comes to us from the legal world where evidence has a ‘probative value’, meaning the weight or persuasive value assigned to the evidence [4]. The probative value of the evidence is weighed by the stakeholders (judge, jury, etc.) to help make a decision. The notion that evidence can be ‘persuasive’ to enable decision-makers such as health authorities, payers, HTA bodies, physicians and patients to reach decisions will be discussed below as value drivers in product development pharma.
Value drivers for drug development
We consider two ways of thinking about value when developing a new product; the number of patients reached along with associated costs (R&D and commercial) or the estimated sales along with the associated costs, the former resonating more with clinical development and medical functions and the latter more with commercial functions. For illustration, both the sales and number of patients reached can be interchangeably represented on the same y-axis. To increase value, one simply needs to shift from the solid curve to the dashed curve in Figure 1. The area under the curve represents the overall value. A push on any of the value levers will increase the area under the curve and, hence, overall value. These concepts of value become important tools in the context of evidence generation planning.
Figure 1. . Illustration of value and value drivers across product development.
The value drivers in Figure 1 are indicative of inflection points on the curve and give an idea of what is meant but they have been defined a bit more precisely in pharma jargon in Table 1.
Table 1. . Value driver definitions.
| Value driver | Definition |
|---|---|
| Faster | Time at which the product is available. ‘Faster’ translates to product being available earlier |
| Steeper | Speed of adoption of a new product in a healthcare system. ‘Steeper’ refers to quicker adoption |
| More | Number of patients receiving a new product. ‘More’ means that more patients will benefit from the product |
| Longer | Time at which product loses patent protection. ‘Longer’ refers to a longer period of patent protection |
| Cheaper | Costs associated with the evidence generation. An ideal evidence generation plan would ideally be ‘Cheaper’ but is often more expensive, requiring it to be offset by the other value drivers |
| Probability of success | Probability of achieving regulatory approval and/or reimbursement |
Value-based decision-making
To match the idea behind integrated evidence, the decision-making process has to change from the share of voice approach, in which the loudest voice(s) wins, and the budget-holder approach, in which the holder of the budget decides, to a process that is based on the value that the evidence brings to the product as agreed upon by all relevant internal stakeholders and based on a common metric. In addition, silo structure has led to generating evidence that is important for the sponsoring function because of perceived importance to that function, with little regard to how the evidence will resonate with the multitude of external stakeholders that are addressed in an IEP. Many metrics have been used in the past, from number of publications and number of congresses, to input from key opinion leaders, payers and patients. However, these did not comprehensively represent the whole product lifecycle, all stakeholder needs, or necessarily focusing on the real value or impact. On the other hand, standard metrics used in the financial world and in pharma itself include the expected net present value (eNPV) and return on investment (ROI) offer a promise for a holistic value estimation. Both of these metrics can easily be calculated by following an evaluation of the impact of an IEP on the value drivers above and allow for the use of a common metric to be used as an aid to value-based decision-making [5].
How could this work in practice?
The integrated evidence process brings together all functions that generate evidence and/or have a direct interest in the evidence that is generated and typically includes: clinical, safety, biostatistics, epidemiology, medical, HEOR, RWE, data science, clinical pharmacology, biomarker development and precision medicine, access and commercial (and, perhaps others). Equipped with knowledge of the value drivers and the detailed product forecast or a simplified patient funnel, teams can begin to look at addressing value gaps rather than data or evidence gaps. This process fits well with the IEP 2.0 that has been proposed to define an upfront enterprise-wide perspective to evidence planning and generation [6].
To illustrate the process, consider the patient funnel chart for malaria [7] in Figure 2. The value levers presented earlier are represented here by the bars with an estimate of the level of achievement. Only 60% of suspected malaria patients are accessing clinics with artemisinin-based combination therapy (ACT). Could this be improved? If so, with what evidence? If evidence is generated, would the percent increase to 80% or only to 65%? This can then be entered into the flow chart to see the impact on the number of patients reached. If needed, the discussions could be extended to look at this anticipated benefit and the cost of the evidence to make calculations of ROI and/or eNPV. The importance to the product is explored in the team discussions on which lever to focus on, which ones can be influenced with evidence, what evidence could influence and to what degree.
Figure 2. . Patient funnel chart for malaria.
ACT: Artemisinin-based combination therapy.
Patient funnels are a useful tool to investigate value gaps due to their simplicity and general understanding among all functions in the pharmaceutical industry. They mainly focus on the first three value drivers related to faster availability, steeper adoption and more patients reached. Thus, additional tools, such as the forecast model and clinical study designs, should also be employed to assess the impact on all the value drivers. The type of evidence used to influence any of the value drivers can vary widely and could include (not exhaustive), literature reviews, communication plans, RWE studies, implementation science studies [8], modelling, use of advanced analytics, machine learning and AI or Target Population Outputs [9] and must be tailored to the specific need in the disease area of interest.
Recommendations
The simple tool of value drivers can be used to change the way evidence is planned and generated in the pharmaceutical industry that:
Focuses on value gaps, and
Conducts inclusive team discussions.
therefore, leading to integrated evidence plans that contribute to planning for more successful pharmaceutical products. Using this framework of value and value drivers, it is possible for the pharmaceutical industry to be much more efficient in their development of evidence enabling better informed decision-making about pharmaceutical products and ultimately better outcomes for patients. Pharmaceutical companies should also ensure organizational structure, processes, clarity on roles and responsibilities as well as technology and data infrastructure are supportive to this new way of evidence generation.
Acknowledgments
The authors are grateful for valuable input on the financial assessment topics raised in this editorial from Joseph DiMasi and Kenneth Getz of the Tufts Center for the Study of Drug Development, Tufts University.
Footnotes
Author contributions
All authors have fulfilled the authorship criteria and accept accountability for all content. Each author has significantly contributed to, reviewed, and approved the editorial.
Financial disclosure
Melvin Olson consults on this subject matter with Merck Healthcare KGaA and Boehringer Ingelheim GmbH. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Competing interests disclosure
The authors have no competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
Writing disclosure
No writing assistance was utilized in the production of this manuscript.
Open access
This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
References
Papers of special note have been highlighted as: • of interest
- 1.Olson M. Developing an integrated strategy for evidence generation. J. Comp. Eff. Res. 7(1), 5–9 (2017). [DOI] [PubMed] [Google Scholar]
- 2.Olson M. Can real-world evidence save pharma US$1 billion per year? A framework for an integrated evidence generation strategy. J. Comp. Eff. Res. 9(2), 79–82 (2019). [DOI] [PubMed] [Google Scholar]; • Outlines a framework for integrated evidence strategies.
- 3.Salow B. The Value of Evidence. (2019). Available at: https://users.ox.ac.uk/~shug2406/ValueOfEvidence.pdf (Accessed: 21 February 2024). [Google Scholar]
- 4.Chapter 3: What You Need to Know About Evidence. In: Introduction to Criminal Investigation: Processes, Practices, and Thinking. Gehl R, Plecas D. (Eds). BCcampus Open Education, Pressbooks, British Columbia, Canada: (2017). [Google Scholar]
- 5.DiMasi JA, Olson M, Smith Z, Getz KA, Capkun G. Assessing the value of integrated evidence approaches in drug development. Available at: 10.21203/rs.3.rs-4321838/v1 [DOI] [Google Scholar]
- 6.Grueger J, Andre N, Vaidyanathan S. The future of integrated evidence planning in biopharma. BCG, (2023). Available at: https://www.bcg.com/publications/2023/the-future-of-integrated-evidence-planning-in-biopharma (Accessed: 8 February 2024). [Google Scholar]
- 7.TDR and WHO. Implementation Research Toolkit (Second Edition). Geneva, Switzerland: 22 (2018). [Google Scholar]; • Provides an illustration of a patient funnel and explains how and why patients are lost at each successive step.
- 8.Olson M, Rootkin L. The triple win – implementation science benefits patients, healthcare systems and industry alike. J. Comp. Eff. Res. 11(9), 639–642 (2019). [DOI] [PubMed] [Google Scholar]
- 9.Atun R. Target Population Outputs (TPOs): Creating High-Value Health Systems Through Strategic Public-Private Partnerships. Health Systems Innovation Lab, Harvard University, MA, USA: (2023).Available at: https://www.hsph.harvard.edu/health-systems-innovation-lab/tpos/ (Accessed: 21 February 2024). [Google Scholar]