Abstract
Background
Data Monitoring Committees (DMCs) have important roles in safeguarding patient interests and enhancing trial integrity and credibility. To effectively fulfill their responsibilities, DMCs should be independent of study sponsors, study investigators and caregivers managing study participants. Unfortunately, in real world settings where DMCs are in place, there are some practices that threaten to diminish the level of independence of these committees.
Purpose
Our goal is to inform those who have responsibilities in implementing the DMC process, or who are relying upon or participating as members, about some evolving issues that can meaningfully impact the DMC’s ability to remain sufficiently independent to be able to effectively address its mission.
Methods
We identified specific issues that are likely to be very important, both now and in the near future, with regard to the actual level of independence of DMCs. We provide insights into how these issues have emerged and to their importance, and provide recommendations for approaches to effectively address them.
Results
Among the recommendations: a DMC Charter should outline the roles and responsibilities of the DMC without appearing to be a legal contract; the meetings of the DMC should be led by its chair, ideally with a meeting format that ensures independence from the investigators and sponsor; the DMC and those having leadership roles in the monitoring process should have adequate training and experience; procedures should be in place to enable the DMC to have access to interim safety and efficacy data that are accurate, current and comprehensive; these data should be presented to the DMC unblinded by treatment group, while being kept confidential from all others; DMC recommendations should be developed through consensus development rather than by casting votes; creative approaches are needed for the engagement of DMC members to increase the transparency that they are neither employees of, nor consultants to, the sponsor of the trial; meaningful conflicts of interest should be identified and addressed; finally, members of DMCs should have adequate indemnification that provides effective protection.
Conclusions
The independence of DMCs is of integral importance to their ability to effectively carry out their responsibilities. We need wider recognition of the influence of some practices that could diminish the independence of DMCs, and a commitment to identify and implement approaches to enhance their independence.
Keywords: DMC Charter, confidentiality, conflict of interest, indemnification
Large scale randomized trials that are properly designed, conducted and analyzed provide the most reliable method to evaluate benefits and risks of interventions, especially since the most plausible clinically meaningful effects are small to moderate in magnitude (1). Each year, results from hundreds of randomized trials are published. While the clinical research community and healthcare providers are aware of these findings, they likely are much less aware about how often these trials had in depth oversight provided by a Data Monitoring Committee (DMC). There are evolving issues that can meaningfully adversely impact the DMC’s ability to effectively carry out that oversight (2). In this article, we briefly discuss the function and structure as well as other salient features of DMCs, and then discuss issues that should be addressed to protect their independence and thereby their ability to enhance the integrity of the trials they monitor.
Structure and Functions of DMCs
According to a mandate of the United States (US) Department of Health and Human Services (DHHS) that has been adopted by the US Food and Drug Administration (FDA), a monitoring plan, often involving an independent DMC, is a necessary component of all DHHS-sponsored clinical trials. (3) The primary roles of the DMC are to safeguard the interests of study subjects, and to enhance the integrity and credibility of these trials. These goals are accomplished by monitoring emerging data, unblinded by treatment group, for measures of safety, efficacy and quality of trial conduct. To fulfill these roles, DMCs should have expertise in relevant disease areas and in clinical trials methodology. DMC members also should be independent. Specifically, they should not have meaningful personal or professional conflicts of interest, such as those created by being involved in sponsoring a trial, in the development of the investigational product or clinical development plan for that product, or in managing any patient in the clinical trial.
As adopted by the US National Institutes of Health (NIH) and industry, the DMC model includes DMC members, an independent Statistical Data Analysis Center (SDAC) that includes the ‘presenting’ or ‘independent’ statistician who is the liaison between the DMC and the data base, the sponsor, whether federal or industry, a Steering Committee (SC) which includes academic investigators and sponsors, Institutional Review Boards and regulatory agencies (2,4).
Since the DMC’s independence is of integral importance to its ability to effectively carry out its responsibilities, there is need to more widely recognize the influence of some current practices that could diminish that independence, including some common myths that are presented in Table 1. There also is need to prevent any deleterious consequences of those practices by implementing proper approaches that are already known or are creatively identified. In the remainder of this article, these issues will be discussed in greater detail.
Table #1.
A Dozen of the Common Myths that can Adversely Impact DMC Independence
|
DMC Charter
The DMC Charter outlines roles and responsibilities of the DMC, SDAC, SC and sponsor. Hence, all these parties should participate in the Charter’s development and revisions. The DMC, however, should have the primary responsibility for the Charter’s final approval. The components of a DMC Charter should include the main features of the trial, membership of the DMC, format for meetings, procedures for communication while ensuring confidentiality, an outline of the content of DMC reports, reporting requirements and statistical guidelines for interim analyses. DMC's have primary responsibilities to trial subjects, investigators and sponsors in that order and the DMC Charter should reflect that hierarchy.
Collectively, the DMC Charter should provide a set of guidelines, not rules, for the process the DMC uses in their review of evolving and admittedly incomplete information and in the formulation of their recommendations. Thus, contrary to the sense conveyed by some sponsors, the DMC Charter is neither an instruction manual nor a legal contract. Unfortunately, these documents seem to be getting progressively more detailed, with DMC members being required to approve and sign new revisions on a regular basis throughout the trial. Regarding the DMC Charter’s specification of the DMC reports’ format and content, these should be viewed to be sufficiently flexible that important unanticipated as well as anticipated issues can be addressed rapidly, accurately, and efficiently by the collective expertise of the DMC. Also, the DMC Charter should properly indicate the DMC’s general and specific recommendations are formulated through development of consensus, rather than having misleading language indicating the DMC is a voting body.
DMC Meeting Format
The format for the DMC meeting can meaningfully impact the independence and integrity of the DMC (4). Originally, many DMC meetings were conducted as a single Closed Session not attended by the SC, other investigators or product sponsors. This format increased the independence of the DMC and enhanced maintaining confidentiality of safety and efficacy data, yet was very restrictive regarding the ability of the DMC to gain insights beyond those provided by the reports they received. In the 1980’s in some NIH sponsored trials (4), an Open Session was inserted in the middle of DMC meetings. In this session, DMC members were joined by some members of the SC and sponsor. This provided the DMC additional insights about progress of the trial and evidence from other relevant completed or ongoing trials, obtained in a manner that maintained confidentiality of the emerging safety and efficacy data. By beginning and ending the meeting with Closed Sessions, the DMC chair and members maintained proper control of the meeting and its agenda. Beginning with an initial Closed Session also allowed the DMC to develop insights and agreements about whether and how to raise issues in the Open Session that would enhance their insights without compromising confidentiality of interim data.
Since industry-sponsored trials began to include DMCs in the early 1990s, a format evolved where meetings often begin with an initial Open Session. With this format, increased efforts are needed to protect the leadership and independence of the DMC. In particular, members of the sponsor, a Contract Research Organization or SDAC should not lead or control the Open Session or have undue influence on the agenda of the meeting.
The sponsor, whether industry or federal, and SC members should not attend the Closed Session and, in some circumstances, should not even be aware of any ad hoc Closed Sessions called by the DMC, so as to preserve proper blinding and to protect the trial from potential biases. A final Open or Debriefing Session may be held with the SC and sponsor. Usually, this should be a brief and non-interactive session where the DMC chair simply provides an oral summary of the general as well as specific DMC recommendations that will be provided soon thereafter in writing.
Statistical Data Analysis Center (SDAC), and Currentness of DMC Reports
The independent SDAC generates DMC reports that should provide an adequately comprehensive understanding about emerging evidence from the trial. As discussed below, this requires a knowledgeable SDAC that has the academic training and experience necessary to have an in depth understanding of the scientific rationale of the trial, as well as the ability to prepare proper reports and to effectively discuss their content with the DMC.
Regulatory agencies have recommended that SDACs be independent of the investigators and sponsor (3). Such independence enhances the ability of SDACs to provide an unbiased presentation of emerging evidence, to maintain confidentiality of that evidence, and to have sufficient flexibility to interact with the DMC regarding the presentation of additional analyses requested by the DMC, in a manner without alerting the sponsor or SC to any potential signals from the emerging data.
The DMC reports generated by the SDAC in some trials present information based on a ‘passive’ approach of taking a snapshot of the database well before the date of the DMC meeting. To enhance the ability of the DMC to address its mission, ‘active’ approaches are needed for data capture to ensure, at the very least, that data are as current as feasible for those key safety and efficacy measures that could meaningfully influence the benefit-to-risk profile of the intervention being evaluated. When planning for a DMC meeting, a ‘clinical cut date’ should be chosen. Information in the DMC reports about patient visits and trial safety and efficacy outcomes should be as complete and accurate as possible through this pre-specified date that typically should be 6–9 weeks before the DMC meeting. The ‘data lock date’ should also be pre-specified, as the calendar date on which the statistical database should be locked and statistical analysis files should be generated and provided to the statisticians at the SDAC responsible for generating analyses and the Open and Closed Reports. This typically should be 2–3 weeks before the DMC meeting. Thus, between the ‘clinical cut date’ and ‘data lock date’, all reasonable efforts should be made to achieve follow-up on key outcome measures that is current at least through the ‘clinical cut date’.
Adequate Training/Experience in the DMC Process
The scientific rationale for the data monitoring process for randomized trials is well developed. While not everyone involved in the process needs to be a seasoned expert, such expertise is required of those in leadership roles or in situations where it is not readily possible to rely on the expertise of others. Those who do need considerable depth of expertise and experience in the DMC process include the DMC chair and DMC statistician, the SDAC’s ‘presenting’ or ‘independent’ statistician, and the sponsor-appointed DMC meeting coordinator. It is unacceptable for the DMC chair to not realize it is their responsibility to lead the Open as well as Closed Sessions of DMC meetings, or to begin a Closed Session by simply asking if anyone has identified “any problems”. Rather, the DMC chair needs to set the agenda and manage the proceedings in the Open Session and should ensure the DMC is led through the key findings in the DMC Closed Report. It is unacceptable for the presenting statistician, through inexperience, to generate an Open Report that has the exact format and content of the Closed Report, except that the actual treatment group for each patient is randomly permuted. Such an approach, recently taken by an inexperienced ‘presenting’ statistician, led some DMC members to mistake the Open Report to be the Closed Report, and led to unblinding all attendees in the Open Session to the pooled data on all key efficacy and safety measures. It also is unacceptable for the sponsor to appoint an inexperienced DMC meeting organizer who creates their own DMC processes and terminology. Fortunately, during the past several decades, standard terminology for DMC processes has been established, and this enhances clarity of communication regarding purpose and function. Having standardized terminology matters. For example, currentness of DMC reports often has been adversely impacted due to failure to understand the distinction between the ‘clinical cut date’ and the ‘data lock date’.
Scientific insights about the DMC process can be obtained from a review of the literature that includes textbooks and a broad collection of articles. Short courses also are available, although some are presented by those with limited DMC experience. It is particularly valuable for DMC members to have had broad experiences on DMCs. Implementation of an apprentice-like approach would increase the breadth of those with the desired level of experience. For example, to increase the pool of biostatisticians with DMC experience while ensuring that each DMC has an experienced biostatistician, it is encouraged that some have two biostatisticians, only one of whom has the required level of experience.
Confidentiality of Interim Data
There is considerable evidence establishing that the DMC and the SDAC’s ‘presenting’ or ‘independent’ statistician should have sole access to emerging efficacy and safety data (4, 5). This reduces the risk of prejudgment that can adversely impact factors such as recruitment, treatment adherence, and retention. While the importance of maintaining confidentiality of emerging data is widely recognized, creative approaches are needed to achieve this in some particularly challenging settings. Such a setting arises when, by trial design, interim data may be released for regulatory review and action, even though the trial would be continued to address its primary hypothesis. An illustration is provided by a recently emerging setting in type 2 diabetes mellitus where cardiovascular safety trials are designed to assess, using the trial’s pre-marketing results, whether large relative increases in morbidity and mortality can be ruled out as a condition for regulatory action, and then are continued to assess, using pre- and post-marketing study results, the trial’s primary hypothesis regarding whether smaller relative increases can be ruled out. Those who gain access to such unblinded efficacy and safety results during trial conduct, including regulatory authorities, should identify and implement approaches that avoid the risk of substantive bias that would arise if interim data informative about the trial’s primary hypothesis are released while the trial continues to collect data to address that hypothesis and others of interest.
While blinding others is important to trial integrity, providing the DMC full access to unblinded data on safety and efficacy during the closed session is of considerable importance to the integrity of the DMC process. Such unfettered access to unblinded data gives enhanced and timely insights about complex patterns in the safety and efficacy data, as well as about inconsistencies due to irregularities in trial conduct or errors in how the reports are generated.
Reducing Conflicts of Interest, and Creating Independent Relationships
To protect the integrity of the DMC, as noted earlier, its members should not have meaningful personal or professional conflicts of interest. DMC members should not be in the position to receive financial or academic gains that are dependent on the outcome of the trial (4). However, perceived conflict of interest cannot be avoided completely, and these considerations should not have an unintended consequence of excluding from DMC membership all those with the requisite knowledge and experience to fulfill the mission of the DMC.
Proper approaches are needed in the process to select and engage DMC members. DMC members should be viewed as independent scientists whose principal role is safeguarding interests of study participants. DMC members are not working for the sponsor so they should not be required to sign contracts structured in the same manner as for consultants hired to advise sponsors about the development of their products. Serving on a DMC should not be viewed as creating a conflict of interest that precludes a DMC member from serving on a committee that develops treatment guidelines, or on an FDA Advisory Committee that is reviewing a related product. It is more likely that the independence of the DMC from the sponsor would be recognized and respected if there were an entity independent from the sponsor that engages the DMC members and also pays their fees and travel expenses.
The process for providing logistical support for DMC meetings also should be guided by conflict of interest considerations. Frequently, sponsors will engage an external group to provide this support, including making the travel arrangements. The key criterion in selecting the location for an in-person DMC meeting should be maximizing the ability of DMC members to attend, such as choosing the meeting site to be at a mutually convenient location, often in association with a professional meeting. To avoid even the perception of conflict of interest, DMC meetings should not be held in a resort type location having vacation amenities, and there should not be social events at the time of DMC meetings, including dinners held the previous evening that bring the DMC members together with the investigators or sponsor.
Indemnification
DMC members assume major responsibilities when monitoring interim data on safety and efficacy. Their responsibilities to minimize risk to participating subjects should be viewed in the context of avoiding risk of early termination based on emerging but not yet reliable data. The principle terms of reference should be to minimize risk and maximize benefit, not total avoidance of risk. Hence, the process leading to DMC recommendations about continuation, alteration or early termination of a trial is complex (4). This complexity and the ability of their recommendations to meaningfully impact patients in and outside the clinical trial have led some DMC members to be subpoenaed and to become defendants in litigation. The DMC cannot function properly if its members fear that statements or actions, or lack thereof, even if based on an objective and well informed review process, might result in litigation for which they are not indemnified. For example, fears of litigation should not influence a DMC to recommend early termination for some adverse event that has a nominally significant p-value even though, with more complete data, there is considerable likelihood the intervention may be proven effective with an acceptable and better quantified risk/benefit profile.
Sponsors should be guided by recent recommendations for indemnification of DMC members (6). The DMC Charter, as well as the independent scientist agreements with all DMC members, should have wording concerning indemnification. The indemnification should not be compromised by the insertion of “negligence” as an exclusion for protection. In this regard, most lawyers would not accept such an exclusion in their own automobile insurance policies. There have been recommendations for legislation that would require all sponsors to indemnify DMC members and, in addition, that DMC members should be empowered to select and retain their own independent counsel (7).
Conclusions
DMC's frequently have been an integral component of monitoring plans for properly designed, conducted and analyzed clinical trials evaluating benefits and risks of drugs, biologics, devices, procedures or other interventions. As noted by FDA and others (2, 3, 4, 8), the DMC has the responsibility to safeguard the interests of subjects while enhancing the credibility and integrity of the trial.
To address this responsibility, DMCs need independence that, in turn, requires that we recognize and then effectively address the increasingly challenging issues discussed in this article. First, it is important to turn the tide on making the DMC process more complicated than it needs to be. Implementing rigid and legalistic procedures isn’t consistent with the proper objective of protecting the integrity of independent oversight. Second, enhanced training and experience are needed, in particular for DMC chairs, DMC statisticians, sponsor designated DMC meeting coordinators, and SDACs supporting DMCs. Textbooks, articles and courses are readily available, and an increased implementation of an apprentice approach would enable a much wider group to become experienced in the DMC process while maintaining necessary expertise on each DMC. Third, the identification of creative approaches and the implementation of these as well as of current best practices are needed to enhance the independence of DMCs. For example, consideration should be given to beginning DMC meetings with Closed Sessions; the DMC chair should lead the Open Sessions; DMC members should be engaged by independent entities; conflict of interest should be reduced when planning meeting venues and events; DMC members should have proper indemnification; active rather than passive approaches are needed for data capture and adjudication of endpoints to ensure currentness; the DMC should be unblinded in its review of Closed Reports; the need for a process of consensus development rather than voting on recommendations should be recognized; regulators should ensure confidentiality of interim data on primary endpoints if these data are from ongoing clinical trials; and regulators should provide co-leadership in ensuring necessary steps are implemented to protect DMC independence.
Ensuring that trials are properly monitored by independent DMCs can provide assurance to study subjects, clinicians, Institutional Review Boards and regulatory authorities that the trials will achieve a degree of integrity that is difficult to attain using alternative strategies.
Acknowledgments
This research was partially supported by funding provided by an NIH/NIAID grant entitled "Statistical Issues in AIDS Research" (R37 AI 29168), and by funding provided by the Wisconsin Alumni Research Foundation (WARF) as part of a named professorship award.
Footnotes
The authors do not have commercial or other associations that might pose a conflict of interest.
Contributor Information
Thomas R. Fleming, Department of Biostatistics, University of Washington.
Charles H. Hennekens, Charles E. Schmidt College of Medicine, Florida Atlantic University.
Marc A. Pfeffer, Brigham and Women’s Hospital, Harvard Medical School.
David L. DeMets, Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison.
REFERENCES
- 1.Hennekens CH, DeMets D. The need for large scale randomized evidence without undue emphasis on small trials, meta-analyses or subgroup analyses. JAMA. 2009;302:2361–2362. doi: 10.1001/jama.2009.1756. [DOI] [PubMed] [Google Scholar]
- 2.Hennekens CH, DeMets D. Data and safety monitoring boards of randomized trials: emerging principles and practical suggestions. Clin Invest. 2011;1:1–5. [Google Scholar]
- 3.US Food and Drug Administration. Guidance for clinical trial sponsors: establishment and operation of clinical trial data monitoring committees. 2005 http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM127073.pdf.
- 4.Ellenberg S, Fleming T, DeMets D. Data Monitoring Committees in Clinical Trials: A Practical Perspective. West Sussex, England: John Wiley & Sons, Ltd; 2002. [Google Scholar]
- 5.Fleming TR, Sharples K, McCall J, Moore A, Rodgers A, Stewart R. Maintaining confidentiality of interim data to enhance trial integrity and credibility. Clinical Trials. 2008;5:157–167. doi: 10.1177/1740774508089459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.DeMets DL, Fleming TR, Rockhold R, et al. Liability issues for data monitoring committee members. Clinical Trials. 2004;1:525–531. doi: 10.1191/1740774504cn54oa. [DOI] [PubMed] [Google Scholar]
- 7.Tereskerz P. Data safety monitoring boards; Taylor and Francis Group LLC. Accountability in Research. 2010;17:30–50. doi: 10.1080/08989620903520313. 6. [DOI] [PubMed] [Google Scholar]
- 8.Shalala D. Protecting research subjects – what must be done. N Engl J Med. 2000;343:808–810. doi: 10.1056/NEJM200009143431112. [DOI] [PubMed] [Google Scholar]