Enhancing clinical evidence by proactively building quality into clinical trials

Abstract

Background:

Stakeholders across the clinical trial enterprise have expressed concern that the current clinical trial enterprise is unsustainable. The cost and complexity of trials have continued to increase, threatening our ability to generate reliable evidence essential for making appropriate decisions concerning the benefits and harms associated with clinical interventions. Overcoming this inefficiency rests on improving protocol design, trial planning, and quality oversight.

Methods:

The Clinical Trials Transformation Initiative convened a project to evaluate methods to prospectively build quality into the scientific and operational design of clinical trials (“quality-by-design”), such that trials are feasible to conduct and important errors are prevented rather than remediated. A working group evaluated aspects of trial design and oversight and developed the Clinical Trials Transformation Initiative quality-by-design principles document, outlining a series of factors generally relevant to the reliability of trial conclusions and to patient safety. These principles were then applied and further refined during a series of hands-on workshops to evaluate their utility in facilitating proactive, cross-functional dialogue, and decision-making about trial design and planning. Following these workshops, independent qualitative interviews were conducted with 19 workshop attendees to explore the potential challenges for implementing a quality-by-design approach to clinical trials. The Clinical Trials Transformation Initiative project team subsequently developed recommendations and an online resource guide to support implementation of this approach.

Conclusion:

The Clinical Trials Transformation Initiative quality-by-design principles provide a framework for assuring that clinical trials adequately safeguard participants and provide reliable information on which to make decisions on the effects of treatments. The quality-by-design workshops highlighted the value of active discussions incorporating the different perspectives within and external to an organization (e.g. clinical investigators, research site staff, and trial participants) in improving trial design. Workshop participants also recognized the value of focusing oversight on those aspects of the trial where errors would have a major impact on participant safety and reliability of results. Applying the Clinical Trials Transformation Initiative quality-by-design recommendations and principles should enable organizations to prioritize the most critical determinants of a trial’s quality, identify non-essential activities that can be eliminated to streamline trial conduct and oversight, and formulate appropriate plans to define, avoid, mitigate, monitor, and address important errors.

Reliable evidence from clinical trials is essential for making appropriate decisions concerning the benefits and harms associated with clinical interventions.¹ Decisions made in the absence of reliable evidence (i.e. relevant trials have never been performed or were inadequately designed, conducted, analyzed, or reported) may harm individual patients and public health.^2–4

Stakeholders across the clinical trial enterprise have expressed concern that the current model for conducting trials is unaffordable and unsustainable and potentially impedes the generation of reliable evidence.^5,6 Overcoming this inefficiency rests on improving protocol design, trial planning, and quality oversight.^7,8 An approach emphasizing error prevention, rather than remediation, should be the norm. Protocol design and trial planning should include appropriate attention to making the study feasible and incorporating methods that help avoid important errors. For example, incorporating simplified but well-justified eligibility criteria, flexible visit schedules, a variety of data capture methods, and streamlined collection of relevant clinical events with centralized review of critical safety and efficacy information, can all facilitate efficient, effective, and feasible trials. Quality oversight can also be enhanced by focusing on data and activities that are critical to trial participants’ safety and the reliability of trial results, rather than on monitoring the accuracy of each individual data point.^9,10 Prospectively focusing on critical trial aspects also removes unnecessary burden on trial participants, clinicians, and investigators.

The Clinical Trials Transformation Initiative (CTTI) has recently reemphasized the importance of prospectively building quality into the scientific and operational design of clinical trials (“quality-by-design”; see Figure 1), rather than relying only on retrospective monitoring, inspection, or scientific review.^11,12 This approach evolved from concepts used in pharmaceutical manufacturing. It adapts the idea of prospectively designing quality into manufacturing processes, rather than testing for it at the end, to the clinical research environment. This adaptation is compatible with recent clinical trial guidelines from major regulatory agencies in the United States and Europe.^13–15 Understanding the processes involved in conduct of the study, assessing whether they are essential and if so, determining when deviation from pre-defined quality tolerance limits is acceptable, is at the core of clinical quality-by-design, as it is in the manufacturing realm. In contrast to manufacturing, clinical trial processes must take place in an environment that is rarely under the complete control of those designing or conducting the study. Additionally, many clinical activities required to meet the healthcare needs of participants will necessarily run in parallel.

Figure 1.

Clinical Trials Transformation Initiative quality-by-design recommendations.

In the context of clinical trials, “quality” can be generally defined as the absence of errors that matter to decision-making—that is, errors which have a meaningful impact on the safety of trial participants or the credibility of the results (and thereby the care of future patients). This definition puts the patient at the center of the process: those aspects most likely to impact trial participants negatively are identified, avoided, mitigated, and acted upon, and information about new interventions is generated in a timely and efficient manner. Views about what errors matter most may vary among the parties involved in clinical trials. Thus, a full understanding of trial quality must incorporate these various perspectives, with the ultimate goal to focus finite time and resources on the reliable prevention of errors that matter.

CTTI has developed a set of quality-by-design principles that facilitates proactive, cross-stakeholder dialogue and decision-making about trial design and planning that can be adopted and systematically applied across the clinical trial enterprise (see Supplementary Appendix).¹⁶ The principles are described in broad categories of feasibility, protocol design, participant safety, study conduct, reporting, and oversight. Each of these categories encompasses multiple, potential “critical to quality” factors that are generally relevant to the safety of study participants (e.g. ensuring that the consent form succinctly describes the key elements of the study so participants understand the risks and the importance of adherence to study procedures) and the reliability of the trial results (e.g. facilitating the collection of essential safety and efficacy data even if participants prematurely stop the assigned study treatment). Potential considerations about the relative importance of each factor are suggested, and examples of issues to consider are given. The quality-by-design principles were not developed to serve as an operational checklist but to promote critical thinking about trial design: to prospectively identify how significant errors might occur, prevent these through trial design where feasible, mitigate remaining risks, and address any that arise (see Figure 2).

Figure 2.

Application of quality-by-design in the trial life cycle.

Utilizing such an approach should aid in identifying and reducing avoidable errors that may raise questions about trial credibility and decisions about whether a medical product or intervention is safe and effective. While rare, denial of approval to market a product for reasons related to trial quality represents a tragic waste, particularly for trial participants who have dedicated their time to furthering generalizable knowledge. A review of new drug applications not approved by the US Food and Drug Administration over a 10-year period found that 10% failed to recruit a patient population appropriate to the intended use.¹⁷ The quality-by-design principles would have focused explicit attention on the intended population and use, prompted dialogue on whether the eligibility criteria were appropriate for enrolling the population, and supported development of unique quality measures to monitor and respond rapidly to a shift in the type of patients enrolled. Similarly, 3% of applications were ultimately not approved due to missing critical data. Application of a quality-by-design approach may have prompted discussion about which data were critical as well as the potential causes and impact of missing these data. Ultimately, strategies to limit missing data could have been implemented. Finally, a recent report noted that a quarter of study procedures in phase 3 trials are not relevant to the assessment of primary endpoints.¹⁸ A quality-by-design approach would encourage study teams to examine and remove non-essential procedures, thereby reducing overall trial cost, complexity, and burden to study participants.

The CTTI quality-by-design principles were developed collaboratively by industry, regulators, and academia and were refined during a series of hands-on workshops involving over 200 attendees with a range of perspectives (including patient advocates, institutional review boards, academic trialists, clinical investigators, clinical research organizations, pharmaceutical and medical device companies, regulatory reviewers, and inspectors). Small, cross-disciplinary groups applied the principles to mock clinical trial protocol synopses covering various development phases, disease conditions product types. Independent qualitative interviews were conducted with 19 attendees to explore the challenges of implementing a quality-by-design approach to clinical trials within academic and commercial organizations.

Workshop attendees initially found it challenging to prioritize factors that were critical to quality. For example, some were reluctant to deprioritize conventional Good Clinical Practice factors for which errors are generally well controlled and infrequent (e.g. checking that every page of every consent form is initialed and dated or verifying every data point against hospital records) and instead focus on those with potentially serious effects such as lack of adherence to randomization, incomplete follow-up, or failure to provide trial participants with timely information about important new safety data for the investigational product. This dilemma reflects the difficulty that an organization may face in seeking to shift from a retrospective “checking” culture (e.g. “Was the consent form signed?”) to one focused on overall quality and error prevention (e.g. “Can we be confident that the nature of the trial has been explained adequately to the participant and that any questions they may have are answered appropriately?”). One tactic that helped attendees focus on the most critical factors was to answer the question, “If I only had $100, what would I spend it on?” Attendees recognized that distributing the money to eliminate every possible error would be far less effective than focusing the limited funds on those aspects of the trial where errors would have a major impact on participant safety and reliability of results.

The workshops highlighted the value of active discussions incorporating different perspectives from the sponsor functions responsible for trial design, operations, monitoring, and analysis. The workshop also demonstrated that including clinical investigators, research site staff, and trial participants in this dialogue can improve trial quality by, for example, affirming relevance of trial design, endpoints, and feasibility. Taking the time to design quality into the trial avoids the all-too-common practices of designing protocols that try to capture all parties’ wish lists or attempting to fix quality issues once a trial is up and running. It also facilitates the development of strategies to prevent, mitigate, monitor, and address any problems that may affect critical data and procedures (see Figure 2). Systemic issues (e.g. consistent exceeding of pre-defined quality tolerance limits) may be identified early through ongoing monitoring and review. This awareness permits timely action, which may include correcting the design through protocol amendment in order to prevent further issues, reviewing and potentially refining mitigation strategies to better control risks, and applying lessons learned to other trials as continuous improvement.

For trial sponsors, the primary challenge in moving quality-by-design from concept to routine practice is cultural: the need to overcome organizational inertia, change behavior of individuals or functional groups, and allay concerns that implementing quality-by-design would significantly extend timelines. Senior management sponsors and well-placed champions are needed to build awareness and enthusiasm for quality-by-design at all levels within the organization, develop staff skilled in its application, address obstacles and barriers, and develop sustainable processes for integrating quality-by-design principles across clinical development. CTTI has developed an online resource with varied tools to support trial sponsors in each of these areas.¹⁹

Trialists and sponsor organizations remain concerned that they will be criticized by regulators for errors that are not critical to quality. Indeed, if regulators place undue emphasis on areas that are not critical to quality, confidence in the approach may be undermined. It is thus important that regulators encourage the adoption of quality-by-design principles and practice in highly visible ways, both in their policy (e.g. guidance documents) and practices (e.g. application reviews and inspections). For example, an addendum to the International Council for Harmonization (ICH) E6 Good Clinical Practice guidance has recently been developed and includes recommendations intended to facilitate and encourage implementation of improved and more effective and efficient approaches to clinical trial design, conduct, oversight, recording, and reporting.²⁰ Ethics committees are also an important stakeholder in the adoption of quality-by-design practices. In particular, ethics committees may benefit from an awareness of how quality-by-design principles have been applied in developing a study and how important errors will be prevented and/or mitigated. This information may help facilitate the review process.

Finally, the ultimate consumers of clinical research—trial participants and future patients—are placing their trust in industry, academia, regulators, and investigators to design, execute, and oversee meaningful research. They rely on the clinical trial protocol and procedures as well as the regulatory, ethical, and governance framework to be appropriate and proportionate to support their needs. A person’s voluntary participation in research represents an invaluable contribution; the corollary is that a clinical trial failing to answer a question because of poor design or execution presents a tragic waste. Nonprofit patient advocacy groups are well-placed to champion the application of quality principles to clinical trials. Increasingly, experienced patient advocates are empaneled in policy discussions alongside other stakeholders, and these patient representatives play an important role in the design of individual clinical trials (e.g. feasibility of recruitment, appropriateness of safety monitoring procedures, and relevance of the endpoints, comparators, and outcomes). Patient advocates understand the need to focus on errors that matter rather than spreading resources thinly across all potential errors.

The CTTI quality-by-design principles provide a framework for assuring that clinical trials adequately safeguard participants and provide reliable information. This approach covers the entire life cycle of quality management (plan, do, check, and act),²¹ combines broad engagement of the many stakeholders involved, and recommends a risk-informed approach for identifying and managing the critical to quality factors. The principles are intended to stimulate discussion and prioritization of the most critical determinants of a trial’s quality and formulation of an appropriate plan to define, avoid, mitigate, monitor, and address important errors.