Recognizing that Evidence is Made, not Born

Abstract

Therapeutic product development, licensing and reimbursement may seem a well‐oiled machine, but continuing high attrition rates, regulatory refusals, and patients’ access issues suggest otherwise; despite serious efforts, gaps persist between stakeholders’ stated evidence requirements and actual evidence supplied. Evidentiary deficiencies and/or human tendencies resulting in avoidable inefficiencies might be further reduced with fresh institutional cultures/mindsets, combined with a context‐adaptable practices framework that integrates emerging innovations. Here, Structured Evidence Planning, Production, and Evaluation (SEPPE) posits that evidence be treated as something produced, much like other manufactured goods, for which “built‐in quality” (i.e., “people” and “process”) approaches have been successfully implemented globally. Incorporating proactive, iterative feedback‐and‐adjust loops involving key decision‐makers at critical points could curtail avoidable evidence quality and decision hazards—pulling needed therapeutic products with high quality evidence of beneficial performance through to approvals. Critical for success, however, is dedicated, long‐term commitment to systemic transformation.

The case for change

Therapeutic product research and development (R&D) still yields too many late stage failures, with negative impacts on individual health, general public health and the economic health of companies, healthcare systems, and countries. The large majority of assets entering clinical development continue to fail, with persistently high attrition rates even at late‐stage phase III.1, 2 Patient clinical study participation efforts, institutional resources, and time can be wasted; development costs3, 4 spiral upward; health system coherence and efficiencies and patient access and outcomes are suboptimal.5 In response, visions for the future of healthcare systems and R&D6, 7, 8 are rapidly driving a wealth of innovations, but uncertainties about ultimate outcomes remain.

Inadequate product performance is the obvious but not sole reason accounting for development and marketing/reimbursement failures; inadequate evidence quality is also a problem. Many rejected applications exhibit insufficient evidence of efficacy and/or safety—including clinical relevance and/or methodological deficiencies—with requirements for new clinical studies to resolve outstanding critical uncertainties.9, 10, 11 Regulators’, health technology assessors’ (HTAs’), and payers’ challenges in adequately assessing product performance and value due to evidence deficiencies thwart positive marketing authorization and access decisions while fueling stakeholder frustration—especially if hindsight shows that critical problems were avoidable.

Evidence generation insufficiencies particularly threaten “transformative” product candidates that might otherwise be eligible for expedited regulatory and access pathways. Numerous applications seeking designation for an expedited pathway reveal evidence inadequacies in data, trial design, analyses, and/or value.^12,13 Thus, situations of serious unmet medical need—in which a less uncertain but more supportive path for transformative therapies development and decision making is vital—remain challenging to navigate, despite good intentions and numerous concrete, evidence‐enabling efforts of health system partners and decision‐makers thus far, which include:

• conceptual and practical innovations and paradigm shifts in methodological and analytical approaches to therapeutic product development;14, 15, 16, 17, 18, 19, 20

• highly strategic and structured research approaches;21

• structured regulatory “benefit‐risk” and uncertainty assessment and management;22, 23, 24

• structured decision‐making approaches;25

• patient‐focused product development;26, 27, 28, 29, 30

• precision and personalized medicine strategies;31, 32

• structured HTA/payer‐led scientific advice and value‐based frameworks and agreements;33, 34, 35, 36, 37, 38

• seamless, “efficacy‐to‐effectiveness” models;39, 40

• real‐world data and evidence innovations across product life‐cycle;41, 42, 43, 44, 45

• collaborative, facilitated pathways, and platforms for therapeutic product development, regulation, and reimbursement.46, 47, 48, 49, 50, 51

Superior outcomes remain elusive, however, due to the diversity and complexity of the advances, remarkable as they each are, and the suboptimal coordination between them. Moreover, although early consultative efforts to understand stakeholders’ evidence needs upfront have successfully progressed stakeholders’ openness and understanding,52 the problem can now be one of gaps between stated evidence desires and actual evidence ultimately delivered.

Costly late‐stage development failures can also result from regrettable “go/no‐go” decisions and/or other unfortunate strategic choices to advance experimental therapeutic products through development. Development decision risks are plentiful and include analytical gaps and/or discordances (e.g., in investigational strategies between sequenced trials in an evidence plan), cognitive disconnects (e.g., between what the stakeholders and/or the evidence are saying and the strategic choices subsequently taken) and uninterpretable data.53, 54 Overall, the risks of failure escalate with accumulating decision flaws.

Well‐established solutions to avert high failure rates

In the food and therapeutic products manufacturing realms and beyond, analogous derailments are mitigated in the modern era through whole‐system mindset transformation and reframing, proactively “building in quality” throughout the design and production cycle; waiting to the end of the process to find out if the product is fit for purpose is no longer an option. Several decades ago, cultural and procedural innovations catalyzed a wholesale “shift left” toward early identification and preventive management of potential issues.55, 56 Two established “total quality” or “built‐in quality”‐type models, which embody this systematic and anticipatory thinking and are particularly relevant, are the “Hazard Analysis Critical Control Point” system,57, 58, 59 used by food product manufacturers and regulators, and “Quality by Design,”60 applied during manufacture (and regulation of manufacturing quality) of therapeutic products. The former emerged first and was developed by the National Aeronautics and Space Administration and Pillsbury to design and manufacture consistently safe food products for the first space flights using systematic, preventive, and optimizing practices.61 Subsequently, Quality by Design was developed to improve customer satisfaction and advance product, process and innovation in manufacturing activities, including physical therapeutic product design and manufacturing.62

From the earliest stages of product manufacturing planning and onward, these quality models embrace the principles of proactive minimization of “hazards” (i.e., potential sources of negative and undesirable effects) and product optimization, as well as understanding and control of both product and process; for example, the goals of Quality by Design include:62

1. achieving meaningful product quality specifications that are based on clinical performance:

2. increasing process capability and reducing product variability and defects by enhancing product and process design, understanding, and control;

3. enhancing root cause analysis of defects and change management;

4. increasing product development and manufacturing efficiencies.

Systematic quality design and production, based upon these principles and practices (which also encompass “hazard pre‐identification” approaches and overlap with “quality management” approaches) are now integral to the regulation of manufacturing of food and drug products worldwide. The food manufacturing model, for example, is incorporated as part of the international standard for food safety of the International Organization for Standardization (ISO),63 promoting international trade agreements for foods.64

A different approach for therapeutic product safety and efficacy r&d is possible: “structured evidence planning, production, and evaluation” (seppe)

Similar to the “shift left” with food and drug manufacturing, “upstream” communities involved in therapeutic product research and evidence development could also adopt formal “built‐in” quality mindsets and procedures. R&D activities and products, like those for manufacturing, are also amenable to quality improvement, from earliest planning stages through the entire pre‐authorization and post‐authorization product lifespan. After all, much of R&D evidence on product performance and value is “made” de novo, through a series of deliberations, activities and choices/decisions during evidence planning and production, not born from random processes and activities. Asset selection and continuing development (e.g., at “go/no‐go,” development phase transitions) also result from deliberations, activities, and choices/decisions. “Quality” is defined as “the degree to which a set of inherent characteristics of an object fulfills requirements.”65 If the evidence itself were also held explicitly to this standard through prescribed mechanisms, then truly meaningful improvements could be achieved—above and beyond the other optimizing initiatives.

The “quality products” to be overtly specified and targeted through systematic and coordinated built‐in quality activities throughout development could be redefined, from the current one‐dimensional paradigm (i.e., the manufactured, physical therapeutic product) to a three‐dimensional one: the manufactured product, plus the evidence, plus the “benefit‐risk”/value of the product. In practical terms, the following features would be applied across the three dimensions:

• proactive hazards identification and avoidance/correction planning; and

• real‐time monitoring and correction; with

• multistakeholder involvement in setting quality specifications and in dynamic evaluations and adaptations; using

• comprehensive and strategic thinking across stakeholders, activities, and development phases.

Seppe compared with the existing paradigm

Beyond therapeutic product manufacturing Quality by Design, various instruments, and/or approaches striving for quality currently exist in R&D and evaluation, such as: Good Laboratory Practice (GLP)66, 67; Good Clinical Practice (GCP)68; other International Conference on Harmonization (ICH) guidelines69; regulators’ jurisdiction‐specific development regulatory guidances70, 71, 72; traditional and next generation scientific advice73, 74, 75, 76, 77, 78; research site regulatory inspections/audits79, 80, 81; and structured “benefit‐risk,” uncertainty and value assessment and management practices.82, 83, 84, 85, 86 These underqualify as total quality management, however, because: a) the span of planning, production and evaluation components is covered only intermittently, with limited coordination between them; and b) the scope, codification and/or obligations under each of these, influencing the degree, fidelity and consistency in implementation, are highly variable. Of note, only GCP (in its latest revision) specifically mentions quality management approaches, with this simply as allusions to future implementation.

There is at present no comprehensive, systems‐based ability to nip development problems in the bud: (a) explicit, up‐front hazard identification and avoidance planning is missing altogether; (b) real‐time, iterative, multilateral practices are limited; and (c) evaluations regarding product and evidence performance by regulators, HTAs and/or payers occur mostly only late in the development cycle, in peri‐authorization and postauthorization stages. Current practices and cultures, therefore, reflect, at best, an appreciation for quality management approaches, but are disjointed, differ in how best to achieve quality, and generally fall short in terms of practical execution. In addition, where such features are absent (e.g., during much of actual planning of evidence), siloed, intuitive guesswork and potentially suboptimal practices in decision making and conduct of practices/processes are the only option.

Indeed, when the data do not seem to be what is wanted, the default “plan,” now, can be to “keep trying and maybe something better will happen”—an action practice that would be anathema in a food or pharmaceutical manufacturing context. Thus emerges a problematic cycle driven by human nature, in which people are so “bought‐in” that they simply cannot make truly “quality‐based” decisions any longer. Thus, the process can rapidly become ad hoc and post hoc and prospective planning then falls apart, unlike in true built‐in quality systems. In this situation, post‐authorization evidence generation becomes a salvage operation.

What would change under structured evidence planning, production, and evaluation?

Movement toward a mindset and culture for comprehensive thinking within and between all points

The foundational premise of SEPPE is that problems and potential failures are inevitable, especially during and across siloed activities sited within the complex earlier developmental and postlicensing worlds. Thus, SEPPE would overcome current limitations facing each actor and decision‐maker in the current linear, sequential stage‐gate approach (e.g., preclinical investigations; successive phases of clinical trials; initial authorization; reimbursement; and product use), in which each activity has been habitually approached by its particular participants as an all‐or‐nothing proposition, and the therapeutic product either fails or goes on to the next activity. SEPPE would invite comprehensive, anticipatory/forward thinking across each and all steps and activities, together, while prompting agile feedback and continual adjustment strategies, to maximize the chance to get to the long‐term goal: making sure patients have fit‐for‐purpose, high quality therapeutic products and information for their use, rather than letting products fail because avoidable problems were not addressed up‐front and were not managed to avert collateral damage at other points in development.

Better awareness, framing, and reframing of product and evidence needs of key stakeholders (“users”)

Another SEPPE premise, building upon existing adaptive/flexible regulatory/reimbursement strategies, is that product and evidence outputs which are comprehensively “user‐friendly,” have the best chance of success; moreover, that well‐informed, intentional approaches are the most efficient way to achieve these. SEPPE would, thus, intensify situation‐specific and rule‐based,87 highly inclusive engagement and input from stakeholders regarding both product88, 89, 90, 91 and evidence needs, before heavy investments take place on all sides. Moreover, it would prompt stakeholders to do so at each critical point along the development pathway, incorporating planned opportunities to deliberately refine those needs (e.g., to larger or smaller patient populations), as evidence accrues and product performance (i.e., positive and negative effects) becomes apparent, during critical evidence generating points before marketing, as well as in the post‐marketing phase. SEPPE would thus continue to shift the system away from the “shot‐in‐the dark” approach, in which research institutions and industry routinely develop products and/or evidence based almost entirely on their own, internal perspectives. SEPPE's advance here would be to stipulate the setting and management of stakeholder expectations all along the way.

Cooperative risk‐anticipating, monitoring, and corrective actions are built in all along the way

SEPPE would also set aside the limiting and expensive practice whereby, if something goes wrong with the performance of product or evidence, post hoc appraisal and potential scientific or regulatory intervention are deployed to try to save the day. Because SEPPE takes for granted that flaws and problems are bound to occur throughout, a third premise is that explicit, active, preventive and more distributed risk‐managing and risk‐communicating efforts—understanding when and why they need to be taken—must become the norm to minimize the occurrence, frequency and impact of these events. SEPPE would compel the structuring of specific, comprehensive activities in anticipation of the things that could go wrong, in the form of overtly described and well‐controlled monitoring, feedback‐and‐adjustment loops; moreover, these would be iterative in both smaller and larger frames. Key decision‐makers would, thus, be enabled to contribute to—and be comfortable with—initial development strategies and evidence designs, as well as possible continuing adaptations prompted by the evolving evidence and other contextual considerations.

In a nutshell, each participant in planning, production, and evaluation would be responsible for neither making, nor accepting, nor passing along a defect92 (i.e., avoidable product, decision, and/or evidence problem/uncertainty). A resulting feature of SEPPE‐generated products (i.e., quality evidence and product deliverables) is that these would be explicitly acknowledged by stakeholder partners as “best quality possible”—a decided improvement over the current situation.

What would seppe look like in practice?

The five global domains of activities that constitute the full cycle of development and assessment following molecule discovery would be retained, but SEPPE would change how each of these is approached, to minimize defects (Figure 1):

Figure 1.

Structured Evidence Planning, Production and Evaluation (SEPPE) Principles. The existing five main research and development (R&D) activity blocks would be approached with end‐to‐end, built‐in quality mindsets and explicit procedures: (1) understand stakeholder product needs; (2) understand stakeholder evidence needs; (3) plan the evidence; (4) produce the evidence; and (5) evaluate the totality of the evidence and product performance, and within applicable contexts. Collaborative approaches would be taken within and between each block, incorporating: early and iterating feedback‐loops and adjust loops; avoiding anticipatable problems and promptly rejecting problems if they do occur90; optimizing and adapting stakeholders’ decision‐making throughout planning, production, and evaluation.

1. understand stakeholders’ product needs;

2. understand stakeholders’ evidence needs;

3. plan the evidence generation;

4. produce the evidence;

5. evaluate the therapeutic product, at predefined time points, based on the evidence and contexts.

Existing best practices elements12, 13, 69, 93 would be upheld and incorporated into the larger SEPPE framework and specific new approaches and concrete activities would be introduced. Within and between each of the domains would be iterating practices for collaborative stakeholder deliberations and choices, adapting and adding to structured methodologies for: (1) stakeholder engagement94; (2) decision making using “benefit‐risk” and value frameworks95; and (3) flexible/expediting regulatory/reimbursement approaches.46, 47, 48, 50, 51 Modern, practical, and user‐friendly tools like fit‐for‐purpose software programs incorporating all these best practices in a highly granular way would guide developers and other participants in building optimal development plans. Software could be modeled after total quality‐based programs being implemented for food defense and safety.97, 98 Other electronic tools using visual representations, such as heat maps, could be developed to permit readily discernible tracking of evidence development progress against stakeholders’ needs.99 Together, next generation electronic tools would enable continuous, transparent documentation of activities, decisions and outcomes, with monitoring throughout.

Understand stakeholder product needs

Following molecule discovery, structured, widely‐inclusive and interactive input from key decision‐makers (i.e., regulators, HTAs, payers, prescribers and patients), in conjunction with industry, would provide realistic establishment and clear definition of need at the very outset, as is currently done with adaptive/flexible approaches.

Stakeholders would first define disease management gaps and then identify the types of treatments needed, in light of those gaps. For example, where there is a treatment gap in management of chronic pain (including palliative, non‐palliative and neuropathic), specific characteristics of desired products would be identified among stakeholders in the larger healthcare context—such as nonopioid/non‐addictive product(s) best suited to the specific patient population(s) in need.95

Next to be delineated would be the characteristics of the developed product that would be desired; for example: (1) specific positive attributes that the putative product should have, as well as the particular unwanted negative effects it should avoid; (2) its performance regarding overall effectiveness; (3) its value to patients and to the health system; and (4) any specific delivery requirements with respect to the healthcare system. These attributes would be set by stakeholders based on considerations of the disease/condition, the subgroups of patients with the disease/condition, stakeholders’ experiences with available therapies and regimen optimization, if any. Larger, societal considerations and practical, context considerations would also be included in these discussions, so that realistic goals and expectations could be set regarding product indication scope to be developed, and performance and “effect size” with respect to alternative therapies if available. This is also the appropriate time to try to resolve any outstanding differences between stakeholders in their perceptions of product need and requisite attributes. Considerations, expectations and goals would be taken in anticipation of, and in context with, the activity domains to come, the next of which is to tackle stakeholders’ evidence needs.

Understand stakeholder evidence needs

As with flexible/adaptive development models, evidence needs of the health system before starting evidence development would be set up‐front, to avoid unnecessary or off‐target research and to prevent avoidable delays in obtaining research answers. SEPPE would also compel the evidence to be collaboratively designed by requiring key stakeholders to formulate their critical research questions,21 to discuss these as a group to identify commonalities and departures, and to determine the collective's final research questions array. Stakeholders would also collaboratively map at which points in the developmental lifespan answers to these questions would be necessary. Managing unrealistic expectations for “early, infinite evidence” would be important at this stage.

Plan the evidence

Given the practicalities and highly technical nature of this phase, narrower groups of technically and/or practically expert stakeholders (including expert patients, for example) would collaborate to explore and validate the research approach options and challenges to meet the stated product indication targeting and evidence requirements. Participants would each “do their homework,” collecting together rich and comprehensive starting materials for designing both the individual studies and the overall evidence strategy (i.e., linkages between the individual evidence components). State‐of‐the‐art understandings and appreciation of the complexities of the disease/condition, the subpopulations of patients, and the opportunities and challenges around data, study designs, potential biomarkers, and analytics would be critical.

Designing each evidence product.

Structured as an explicit and proactive deliberation, the complementary tasks of identification of evidence quality attributes vs. hazards would be carried out for each of the proposed studies, to overtly avoid introduction of pre‐identifiable evidence hazards, as well as explicitly strive for optimized evidence quality during individual studies design. Evidence quality attributes would include meaningfulness, validity, timeliness, and transparency,41 as well as overall interpretability. On the other hand, pre‐identifiable evidence hazards that could be introduced into the data/evidence at design and subsequent planning and production stages, would include bias and noise, as well as irrelevancy, conduct mismanagement, and missing information.

As a correlative task, participants would map critical protocol design control points, and identify which stakeholders would be involved at these monitoring and correction points for each study. For example, in composing a study protocol, identifying and assigning up‐front the points at which to correct any design flaws (e.g., confirming and re‐confirming that the protocol's design would address the specific research question), would anticipate the requirement for a very early amendment.

Designing the evidence process.

Participants would also strategize how each planned study and its outcomes could fit into the totality and synchronization of the evidence. Additionally, possible alternative research design strategies (“contingency plans”) would be identified for each critical evidence inflection point, given that as the evidence accrues and knowledge builds, adaptations in the individual studies and the overall evidence plan may be necessary.15

Pre‐identification and strategic mapping of the critical studies (i.e., studies whose outcomes could dramatically influence the choice of follow‐on studies/strategies) would also occur. The specific stakeholders who would be involved in monitoring the outcomes of these studies at these points would also be identified. Potential corrective actions would be spelled out to prevent evidence planning choices from inadvertently directing production and outputs away from stakeholders’ needs.

Produce the evidence

Monitoring, feeding back, and adjusting or correcting within each action step of each study—that is, in conduct, documentation, analysis, and reporting—as well as in evidence synthesis across studies, would take place to continue to minimize introduction of evidence flaws. SEPPE would continue to shift industry away from working in isolation as an institution as a whole, as well as break up siloes internal to a company, during each of these evidence production activities. Earlier pre‐specified evidence monitoring points, activities, and corrections would now be implemented as a structured, mandatory, incremental, “rolling” development assessments by regulators (and possibly others) to assess accruing evidence outcomes in terms of both evidence quality and product performance.

Evaluate the totality (i.e., evidence performance, product performance, contexts)

Iterative evaluations of evidence quality, product performance, and contexts would now be spread strategically across R&D critical choice points and activities, including at the major inflection points, such as go/no‐go, development phase transitions, submission, licensing, reimbursement, and use.

Of note, at peri‐authorization and reimbursement evaluation points, SEPPE would bring a significant refocusing in the evaluative activities by reducing the efforts now spent on problems with the interpretability, sufficiency, and meaningfulness of evidence, or concerns about the actual needs for the drug. This would result in less conflated product performance evaluations at these later stages, streamlined licensing and reimbursement decision‐making considerations, and an unencumbered focus on truly inescapable next steps regarding any on‐market restrictions and ongoing gathering and evaluation of on‐market evidence.

Structured and coordinated negotiation of “real‐world” system stakeholders’ roles and responsibilities regarding the marketed product would anticipate and risk‐manage the licensing and reimbursement point. This would help prevent wayward, or discordant, health system behaviors and outcomes, such as are reflected in the opioid‐pain crises.100, 101

Seppe's limitations and countervailing opportunities

We acknowledge that even if optimally implemented, SEPPE could do nothing to resolve the unavoidable, inherent uncertainties of scientific research, in which even clear answers to intelligent research questions inevitably lead to follow‐on uncertainties and questions and, moreover, the answers themselves may be disappointing. Given the considerable proportion of extensively developed products that fail due to inadequacies in evidence, however, incremental effects to tip potential failures into successes could be sizeable. Although a therapeutic product's performance outcomes cannot be guaranteed at the outset, the quality of the overall plan and individual studies to investigate that performance, should be.

Nor could SEPPE rescue extensively developed therapeutic products that demonstrate unacceptable harms and/or lack of effectiveness/value. More careful, real‐time handling at critical decision points for products that are not achieving their promise, as these become apparent with the accumulating evidence in context with the set product performance parameters,51 however, has real potential to curtail avoidable, flawed, and inherently expensive late‐stage development. Redistributing stakeholders’ time and resources, and advancing these in the development and evaluation cycles could obtain greatest impact and highest efficiency.

Resistance to large scale change, such as SEPPE proposes, could be anticipated due to typical “human nature” implementation barriers, such as overwhelm, conflicting, and/or diverging views and expectations, challenges translating abstract principles into concrete practice, or even basic force of habit.102 Moreover, this resistance may be magnified by the long‐term investment and commitment across all levels of organizations necessary to develop and implement SEPPE, in what is already a challenging environment replete with rapidly evolving, complex, and expensive change. A notable caveat is that decades of experience with quality approaches in other domains has made clear that “quality is a mindset that impacts everyone and that it is more than a process or programme to be implemented by workers on an assembly line…It is not enough to fix problems after we notice them—the real challenge is to catch them before they happen.”103 In addition, we concede that SEPPE would be unachievable in situations of irretrievably entrenched cultures, where management and staff do not acknowledge the existing weaknesses; introduced quality processes only become part of the problem.103

However, initial negative perceptions and reactions might be overcome if the genuine potential to mitigate the current risks of complex system overload could be made clear. SEPPE, by design, neither competes nor substitutes for any of the existing change drivers; instead, it should play well with and enhance them, providing a needed organizing scaffold. By prompting the systematic consideration of each—singly or in combination—and supporting their strategic and coordinated use, SEPPE offers an opportunity to clarify and accelerate, rather than complicate, adoption of the myriad technical, scientific, and socially driven advances in therapeutic product development. Although SEPPE would need to be customized for each development program, it would not be unique in this requirement: very few “off‐the‐shelf” solutions exist for any R&D components. It also would enable current obligatory transparency initiatives.104, 105, 106 Moreover, ultimately, once in operation, the core aspects of “built‐in quality” and “people‐critical” attitude would be neither complex nor expensive.

In the end, SEPPE could not eliminate the risks and uncertainties inherent in medicine or device development, but it could provide: (1) a set of shared expectations; (2) concrete practices to anticipate and avoid evidence and decision quality problems that are preventable; and (3) transparent ways of working as data emerges.

Could seppe satisfy the current needs of the health system and stakeholders?

New paradigms for development, regulation, and reimbursement must be: practical and feasible, as well as be able to address scientific and other uncertainties that lead to most clinical development failures;107 address the needs of many stakeholders; and enable the wider stakeholder community to embrace explicit trade‐offs that need to be made.108 Interoperability with current and emerging innovations and initiatives is also critical in the current environment. SEPPE's Quality by Design elements could provide a unique strategy to meet each one of these criteria:

• quality‐based systems have been long demonstrated to be both practical and feasible, including within the therapeutic product industry;

• the sequencing and shared decision‐making design of SEPPE's built‐in quality interventions (i.e., early and often throughout life‐cycle R&D) promotes the ability to address uncertainties and hazards of go/no‐go decision making in a timely and explicit fashion to reduce development failures;

• SEPPE's multi‐stakeholder platform, which builds upon existing adaptive‐type regulatory and reimbursement paradigms, is designed to address and progress the deliberative system to meet multiple needs;

• SEPPE's practical routines would assist stakeholders to side‐step anticipatable problems and, additionally, confront and reconcile explicit trade‐offs—not only between evidence generation and access at major inflection decision points, but also between evidence methodological approaches during evidence planning and production. Careful selection and preparation of appropriate stakeholders to exchange information and make timely decisions would be necessary to reap the full benefits.

Considering practical aspects to develop and implement seppe

The timing to consider SEPPE development and implementation is optimal, given other innovations, such as in information technology (discussed below), which could be harnessed. Indeed, given the current evolutions within therapeutic product development, such as the latest GCP revisions, moving to the three‐dimensional total quality management approach may be inevitable.68

Consulting with those involved in existing initiatives for evidence/quality improvements to be adapted and integrated in the SEPPE framework (for example, food and therapeutic product manufacturing models of built‐in quality, structured, and parallel scientific advice and decision models, Provocative Questions Initiative, flexible/expediting regulatory and reimbursement pathways, among others) would be vital to understand in detail and incorporate valuable “lessons learned” into development of the fit‐for‐purpose framework that SEPPE envisages.

Employing information technology innovations may be critical to enabling SEPPE. For example, digital health technologies could support evidence production processes, such as in recruiting patients, following patients’ treatment compliance, and/or for collecting data in real‐world settings. Emerging advances in communications and social media technologies should also help make SEPPE's structured broad stakeholder decision deliberations a more tractable endeavor, as well as enable adaptability of involved organizations.109 Artificial intelligence approaches could also be leveraged to help routinize the signaling of defects during planning and production of the evidence.110 In addition, blockchain technology (in which a continuously growing list of records, called blocks, are linked and secured using cryptography111) could provide the operating platform to minimize wayward health system behaviors around marketed products, given the widely distributed transparency that blockchain embeds in all participants’ transactions.112 Indeed, unlike the current less structured paradigm, SEPPE's highly structured activities and processes would facilitate the application of blockchain approaches.

Although orchestrating electronic meta‐processes for decision‐making and prevention of avoidable problems may well be formidable tasks, the collective of health ecosystem stakeholders would benefit greatly in the longer run. Taken into context with current wastage in general, and the human and financial costs of health system crises driven by network‐spanning disconnections (e.g., opioids‐pain) in particular, a technology‐enabled SEPPE framework could be a good way to manage complexity and resource costs sustainably.

Implementing SEPPE incrementally, adapting over time (as was done with implementation of the food manufacturing built‐in quality framework61), is perhaps the most viable road to success. Initially, applying SEPPE in a selective, focused fashion—for example, to only potentially transformative products where there is greatest individual and/or societal need—could facilitate SEPPE's cost‐effective and resource‐effective introduction, especially if used as part of emerging, more efficient, multitreatment, disease platform‐based R&D approaches.20 Such targeted application, involving focused numbers of individuals in participating organizations, would also have the greatest chance for effective “change mindset” in early phases of SEPPE implementation.

Setting up the appropriate interactions would be one of the first challenges. It would be critical for representatives across each stakeholder group—industry, regulatory, HTA, payer, prescriber, and patient/caregiver—to form a coalition of the willing. Careful consideration of how best to facilitate/accelerate this type of collaboration would be necessary; for example, to incentivize sponsors to nominate products, and regulatory and other authorities to devote time to participate. Moreover, these participants would need to be familiar with the product and/or indication(s) under consideration and the critical issues. They also should be empowered, on behalf of their respective stakeholder organization, to make commitments regarding stakeholder‐specific decisions, roles, and responsibilities, toward a mutually recognized and temporally evolving set of roles and responsibilities spanning the processes of development, regulation, reimbursement, provision, and use, with the need to remain engaged, and, where appropriate, confidential, throughout the whole development process. The entire process may be best governed by context‐sensitive and context‐specific collaborative platforms46, 48, 113, 114, 115, 116 with information on decisions to be stored, referenced, and retrieved, and decisions obligated or amended transparently within the stakeholder coalition.99, 117

Ensuring strong and sustained leadership

“Cathedral thinking”—that is, a long‐term commitment and mindset—although necessary across all stakeholders, would be critical in the executive leadership communities to sustain development and implementation.118 “Change champions” would be needed from within their respective organizations. The movement could grow organically and horizontally, or be imposed in a top‐down approach—or a combination of the two approaches (i.e., “deliberate vs. emergent strategies”102, 119) depending on evolving contexts and preferences. A pilot, similar to those for adaptive‐type pathways,47 to “test” the approach might be one way a product sponsor chooses to initiate SEPPE. Previous examples of transformational change described earlier, such as in implementation of foundational quality and collaborative models, in the uptake and use of structured “benefit‐risk” frameworks, as well as forward thinking initiatives to “pull” needed therapeutic products,91, 120 could shed light on options for possible paths forward. Moreover, understanding current efforts to successfully implement collaborative innovation platforms and/or collective leadership models, from practical/scientific/technical, business, policy, and governmental perspectives, would greatly increase the likelihood of success.109, 113, 114, 115, 116

Seppe's effects

If developed and operated well, SEPPE would bring the sciences of R&D, evaluation, and decision making closer to the realities of society today. It could provide a means to routinely and transparently align goals and incentives across stakeholder communities, improving health ecosystem connections and behaviors. Patients, prescribers, and payers would have dependable opportunities for their voices to be heard before it is too late in the development cycle, maximizing the chance of pulling products through development, which are really needed and of high value. Patient‐focused critical questions seeking answers about patient “responders” (i.e., those most likely to benefit, or benefit the most) and “reacters” (i.e., those most susceptible to harm, or to worst harms), could be tackled earlier, accelerating research. Patients’ and researchers would be assured that participation in SEPPE clinical studies would be worth their time and effort.

SEPPE would catalyze and integrate scientific regulatory, HTA, and payer developments, streamlining the efforts of those stakeholders. Given SEPPE's distribution of resources and activities across planning, development, evaluation, and reimbursement processes, a log‐jam of evaluations downstream could be avoided. Industry could anticipate better development predictability, clearer understanding of social and technical needs for their products, higher probability of success, and reduced investment/development waste and costs. SEPPE could also reduce costs to the health system as a whole, because critical decisions by all stakeholders depend on the quality of the accumulating evidence: collateral damage and expenditures arising from nonvalue‐added activities in development, regulation, reimbursement, and/or use could be minimized.

SEPPE should also help leverage best use of real‐world data and evidence approaches and resources throughout product life‐span and avoid narrower, more negatively focused, down‐the‐line evidence recovery operations. SEPPE's documentation requirements regarding evidence attributes and hazards, building upon each SEPPE‐based product development cycle, could also provide a reference platform for generalizable and open‐source learnings to fast‐track R&D learning across the communities for continuous improvement and advancing efficiencies. Ultimately, SEPPE could help make more sense out of an increasingly complicated and shifting environment.

Seppe next steps

We offer the SEPPE concept to stimulate discussions and questions: is it worth exploring further?

To develop SEPPE beyond the concept stage, many practical, collaborative next steps can be envisaged, including the following:

1. explore in detail the originating initiatives (e.g., food and therapeutic product manufacturing models of built‐in quality; structured and parallel scientific advice and decision models; Provocative Questions Initiative) to identify best approaches for policy and scientific adaptations to SEPPE's built‐in quality context. It would be wise to also thoroughly understand, from sectors in which built‐in quality approaches have been adopted, the financial, human, and time costs vs. opportunities of adoption, implementation, and operation of built‐in quality approaches compared with previously operating models and outcomes, as well as the ultimate, real‐world impact on product quality. Examination of emerging developments in collaborative/open innovation platforms, as well as of deliberate vs. emergent strategies for implementation would also be valuable.102, 113, 114, 115, 116, 119

2. construct libraries of case studies of successful and unsuccessful therapeutic product development outcomes, to build further generalizable learnings and lessons for incorporation into SEPPE tools; for example, (1) reference datasets of evidence hazards and decision hazards, and, conversely, of evidence quality attributes and best decision practices; (2) maps of critical control and decision points; and (3) effective (and ineffective) corrective strategies. Given that no products have yet been developed under SEPPE, case studies could instead include products referred to external advisory committees (often reflecting problematic evidence or development strategies) and those involved in flexible/expedited regulatory pathways initiatives (which apply certain SEPPE elements, such as stakeholder engagement in defining evidence needs upfront with enhanced scientific advice).52, 121

   • Additionally, examining exceptional evidence development success stories, such as the I‐SPY platforms for patient‐focused breast cancer therapeutic product research,122, 123 which exemplify best thinking, strategic choices, and practices, would contribute to articulating explicitly the principles, evidence quality attributes, and practices necessary for SEPPE. On the other hand, much could, and indeed, has already been learned from instances of widely‐recognized evidence planning and development miscarriages and compilations of the numerous specific types of hazards from these would also contribute to the initial reference datasets for SEPPE.124 Other existing real‐world success stories, and hypothetical failure cases compiled to illustrate problematic elements from real‐world filings, would also provide rich material for study.12

3. examine comprehensive lessons learned from flexible/expediting regulatory and reimbursement pathways and innovative evidence development initiatives, for strategies and rules, for example, to pick relevant drug candidates, determine core outcome measures, generate the evidence, manage entries/exits, and/or engagement.12, 13, 52, 87, 93, 125

This foundational research would support evidence‐based policy and scientific/technical, as well as software development for an SEPPE framework. This would entail:

• mapping and establishing stakeholders’ roles and responsibilities; NB: It would be critical for practical feasibility of implementation of SEPPE to (a) delineate context‐adapted rules for governance, leadership, stewardship, facilitation, and engagement (e.g., who would facilitate SEPPE for each drug product?; would there be a designated third party agent?), and (b) identify practical solutions to coordinate and track deliberations, decisions, and activities over time. Adapting and coordinating these roles and responsibilities in the context of additional emerging innovation, such as disease‐focused (rather than product‐focused) master protocols would be another consideration;

• creating the SEPPE reference datasets of attributes/hazards, which could be amassed in an open‐source SEPPE database. As noted previously, food manufacturing‐related hazards reference databases, in conjunction with recent food safety software developments, could serve as practical starting points to integrate these into SEPPE software;

• plotting critical control (monitoring, feedback, and adjust) points for product and evidence attributes and hazards, across product life‐cycle:

  • ○chart systematic/common leverage points for incorporation of recent social, scientific, and technical innovations to maximize opportunities for R&D to be patient‐focused and make best use of real‐world data and evidence;
  • ○explore application of existing structured and dynamic engagement, decision, and documentation tools.96, 99

These elements would also be embedded in the SEPPE software tools;

• identifying reconsideration, adjustment, and stopping rules;

• considering and addressing intellectual property issues;

• developing risk‐sharing strategies;

• considering approaches for strategic, step‐wise, efficient, and effective SEPPE implementation, given the human and financial resources required;

  • ○prioritize the most obvious and/or highest needs areas for initial, focused SEPPE practical application (e.g., develop further the revised GCP68); apply to expedited/flexible pathways applications with greatest patient and societal needs;
  • ○explore information technology opportunities to enable SEPPE operation109, 110, 111, 112;

• designing and conducting SEPPE development and implementation pilots.

Clearly, there would be much work to be done—over the short term and long term—but the continuing development of a more reliable and efficient system to reduce preventable mishaps in therapeutic product development, regulation, reimbursement, and use, should be well worth the effort (as was the case, earlier, for the Hazard Analysis Critical Control Point model, as well as for Quality by Design). Shifting the burden of avoidable problems, uncertainties, and waste down the line is no longer a viable option. Fortunately, the load would be lightened by building upon work already done by consortia leading adaptive/facilitating pathways.

The view may exist that we have already achieved the upstream “Quality by Design” proposed here, given that certain quality improving elements are occurring now in R&D and regulatory science, as described earlier. This view may also reflect the recognition of the value that SEPPE approach could provide, as well as the fact that this is within our grasp to attain. SEPPE's synergistic blend of “tried‐and‐true” built‐in quality approaches, “next generation” strategies for progressing existing adaptive and collaborative strategies, and organizing scaffold for scientific/technological innovations offers a navigable path forward, which is in plain sight. Taking the next steps, for broad deliberations and transparency across stakeholders in real time, combined with the obligation for us to take a harder look at evidence earlier and periodically throughout the process—and with clearly articulated corrective measures—could result in consistently better and more predictable development programs and higher later‐phase success rates.

There will always be medicines that rise and medicines that fall; but they should do so for the right reasons.

Funding

No funding was received for this work.

Conflict of Interest

The authors declared no competing interests for this work.

Disclaimer

The views expressed in this article reflect the authors’ personal views and not necessarily those of the institutions they represent.