Independent Data Monitoring Committees: An Update and Overview

Abstract

An Independent Data Monitoring Committee (IDMC) duty is to ensure that the interests of the patients entered on the trial are being well-served (i.e., that the risk-benefit ratio is appropriate) and that the scientific integrity of the trial is maintained during the interim between trial initiation and trial completion. Industry sponsors form IDMCs to ensure an independent assessment to assure that the study participants are not exposed to unnecessary or unreasonable risks as a consequence of their trial participation, and to ensure that the study is being conducted according to highest scientific and ethical standards. IDMCs are needed to analyze interim data for large randomized studies, in particular those that involve multiple sites and important clinical endpoints such as survival or disease progression. Ethical principles mandate that clinical trials begin with uncertainty as to which treatment is better (clinical equipoise). This uncertainty should be maintained during study conduct and analysis unless there is compelling data that emerges during the conduct of the trial. Group sequential statistical designs offer a mechanism to consider terminating a trial early and the results made public if the interim data become sufficiently compelling. Interim monitoring of safety and efficacy data is an integral part of modern clinical trials.

Introduction, History, and Context

The history of Data Monitoring Committees (DMCs) or Independent Data Monitoring Committees (IDMCs) can conceptually be traced back to the “Greenberg Report”. This report was made from the Heart Special Project Committee to the National Advisory Heart Council (part of the National Heart Institute) in 1967 and was prepared by an expert committee, headed by Dr. Bernard Greenberg, a statistician from the University of North Carolina. The report was designed to address the management of complex multi-institutional clinical trials and specifically addressed the need for an independent advisory committee that could help to manage large complex clinical trial endeavors funded by the National Heart Institute. The report was not intended for publication and it was not actually published in a citable form until 1988 (1). Even today that report encapsulates many of the relevant concerns in the organization and execution of large clinical trials.

In 1979 the National Institute of Health (NIH) issued a policy developed by the NIH Clinical Trials Committee (2) and made note that “every clinical trial should have prevision for data and safety monitoring”. What was novel then is now accepted as part of the normal conduct of large complex multi-institutional trials. A workshop convened by the U.S. Food and Drug Administration in 1992 (2) reviewed operational aspects of these committees and policy was further developed by a 1994 report by the NIH Office of External Research which established a committee on clinical trials monitoring. At that time it was generally agreed that monitoring should be proportional to risk and that risk associated with participation and research should make very attempt to maximize the opportunity for benefit while minimizing the risk to the participants and future participants. In 1998 the National Institute of Health issued an updated policy for data and safety monitoring committees (3) and noted that data safety monitoring committees were required for multi-site clinical trials involving interventions that entailed potential risk to the participants.

In 2006 the Food and Drug Administration issued a guidance document for clinical trial industry sponsors on the establishment and operation of clinical trial data monitoring committees. The current FDA guidance was initially issued in draft form in November 2011 and a guidance entitled “Guidance for clinical trials sponsors: On the establishment and operation of clinical trial data monitoring committees” is available on their current web site (5) and this document (or subsequent ones should this version be updated), are essential for sponsors to review prior to initiating a trial oriented toward regulatory approval.

Today a variety of federal regulations are specific with regard to the sponsors of new drugs requiring an IDMC when evaluating new drugs, biologics, or devices. These federal regulations regulate a variety of factors around trial conduct that extend beyond IDMC issues. For instance, federal regulations 21 CFR 56.103, 21 CFR 312.66, 21 CFR 812.40, and 21 CFR 812.150(a) govern how the sponsors or individuals conducting a trial are responsible for informing the various Institutional Review Boards (IRBs) regarding significant new information that arises during the it’s conduct. Information such as the IDMC recommendations after interim data reviews should be communicated to the various IRBs responsible for managing the risks and benefits of the trial at individual sites.

When to Engage an IDMC

The primary purposes of the IDMC are to assure that: the interests of patients entered on the trial are being well-served (i.e., that the risk-benefit ratio is appropriate) and the scientific integrity of the trial is maintained during the period of interim analysis. Virtually all clinical trials potentially pose some risk to patients under treatment. Given that sponsors have vested interests in trial results, it is generally agreed that IDMCs are needed for randomized studies, in particular those that involve multiple sites and endpoints such as survival or other critically important health outcomes. If there are particular concerns about risks because the treatment may involve toxicity, or there is a relative lack of experience with an agent making assessments somewhat unpredictable, then these issues need to be taken into consideration when contemplating whether or not an IDMC is needed.

The clearest reasons to establish an IDMC is to enhance the safety of trial participants where safety concerns may be unusually high. In this case, regular analysis of the interim data is critical. For any interim analyses outside the SAP, the trial sponsor should be fully blinded. For SAP related intermediate endpoints, the sponsor may or may not have access to binary (yes/no) outcomes of the analyses. The trial then should continue in accordance with the original design and SAP.

It is not just the agent and experience with the intervention that determine risks, certain populations are more fragile than others including children, pregnant women, or the elderly and extra measures of IDMC protection may be necessary regardless of the perceived risks of the drug under study.

The Members of the IDMC

The IDMC is typically comprised of 3–5 individuals with extensive clinical experience both in the disease under study, in the management of large complex clinical trials that represent different expertise and points of view (e.g. patient advocate). Each trial has an IDMC appointed by the trial sponsor and each trial should have a distinct IDMC. Larger committees have been suggested by some but the necessity for in depth discussions and the practicalities of ensuring availability suggest that a small group of committed individuals is best. Two clearly designated positions are typically present, these positions being the Chair and the Statistician. The Chair is expected to lead the IDMC in deliberations (especially in “closed sessions”), sign the official minutes (after review by all IDMC members), and be responsible for communications to the sponsor. He/She should have considerable experience both in serving on IDMCs and in the disease under investigation. Given the fact that many adverse events in cancer patients are not due to drugs, but rather due to the underlying disease, and that adverse events are typically reviewed in the interim without causal attributions (to drug or disease), experienced clinicians familiar with the disease under study are critical for appropriate decision making. A statistician expert in the disease under study might or might not be available but having an expert with both statistical expertise and disease expertise is clearly optimal. Individuals without experience in the disease under study can be a clear liability during deliberations.

Expert statistical input is an absolute requirement for optimal IDMC function and it is best if that statistician has experience in the nuances of the disease being addressed. In all instances, for large registrational-type trials, some prior IDMC experience should be a requirement for all IDMC members given the often complex decision making that can occur during the interim data analysis.

Group Sequential Monitoring

The type I error rate increases with repeated testing of a hypothesis performed on sequential data from a clinical trial (8). The type I error is defined as the probability of rejecting a null hypothesis when it is true. In other words, multiple sequential testing of data always increases the probability of a false positive result if no adjustment is made on the type I error rate. To illustrate this concept, suppose that an investigator would like to analyze the clinical trial data at four different time points that tests the hypothesis that a new treatment prolongs overall survival compared to a standard therapy. Without adjustment the type I error rate is 0.14. As a result, the probability of committing a type I error rate has increased from 0.05 to 0.14, as opposed to the original design (type I error rate of 0.05). Historically many National Institute of Health (NIH) funded trials were designed as fixed sample size (that is does not allow for interim analyses) and it was not until the development of group sequential statistical methods, such as the Pocock (6) and the O’Brien-Fleming (7), that sponsors implemented these methods in randomized trials.

Most randomized clinical trials include strategies for terminating the trial early if a combination of treatments is found to be either effective or harmful to the patients. Statistical methodology termed group sequential design (GSD) has been developed for interim monitoring (and potential termination) of clinical trials to minimize the role of subjective judgment. GSDs are both an aid in monitoring and serve as a tool throughout the trial. GSD methodologies and interim monitoring guidelines are now considered part of standard statistical practice. The monitoring rules are used so that the overall type I error rate is preserved at the nominal level (usually a one-sided level of 0.025) and the power is maintained at the required level with the target sample size. When applied for superiority testing, the trial may be terminated early for evidence of benefit if the test statistic crossed the one-sided upper boundary which indicates that statistical significance has been reached. Pampallona and Tsiatis (9) provide a formulation for a family of one-sided and two-sided designs that includes both the Pocock (6) and O’Brien-Fleming (OBF) approaches (7). This class of boundaries requires that both the number and timing of the planned analysis be prospectively specified at the design stage. The OBF approach is commonly used in phase III trials and it strongly protects early on in the trial with small fraction of information (that is, number of deaths) against making an error in the decision that an experimental arm is superior to a control or a standard therapy when in fact it is not. Thus, the OBF is a conservative procedure, as there is a small probability of making an incorrect decision in rejecting the null hypothesis.

A more practical approach that overcomes the limitation of the OBF approach is to use the alpha spending function (how to allocate the type I error rate based on proportion of information for the interim analysis). This approach does not require that both the number and timing of the planned analysis be known in advance and eliminates the need to specify this information. The alpha spending function allows calculation of an interim boundary time point of test given the information fractions of previous interim analyses and the present information fraction.¹⁰

We illustrate an example of the Lan-Demets analog of the OBF sequential boundary that the IDMC may have used in monitoring the COU302 trial. The COU302 trial was designed with two co-primary endpoints: radiographic progression-free survival and overall survival with 85% power to detect a hazard ratio of 0.80. The type I error rate was allocated mostly to the overall survival endpoint (two-sided type I error rate=0.04) with the target number of deaths of 773. Figure 1 presents the OBF boundary as a function of the fraction information. The COU302 trial was designed with four analysis at about 15% (116/773), 40% (311/773 deaths), 55% (425/773 deaths) and 100% (773/deaths) of the total information. The Y axis is the standardized log-rank test whereas the X-axis depicts the proportion of observed deaths at the interim analysis out of the total expected deaths at the end of the trial. As you can note, there is an inverse relationship between the fraction information which is defined as the ratio of observed deaths out of the total expected deaths at the end of the trial. At the beginning of the trial, the boundary should be very high with low number of observed deaths but the OBF boundaries will decrease as the proportion of information increases. A high value of the standardized log-rank test will correspond to a very small p-. At the end of the trial, one would expect the p-value for the last boundary to be around the nominal level. If the value of the standardized log-rank test is above the boundary then there is a strong evidence for the efficacy of the drug.

Figure 1.

The Lan-Demets analog of the O’Brien-Fleming Sequential Boundary for Testing the Superiority of a New Therapy vs. a Standard Therapy (one-sided type I error rate =0.02)

In the second analysis of the COU302 trial, which was based on 333 deaths (0.43 of the total information), the stratified log-rank p-value was 0.0097 which is greater than the O’Brien-Fleming p-value=0.0008). In other words, there was no statistically significant difference observed in overall survival time by the two treatment arms. Furthermore, the observed hazard ratio was 0.75, for patients randomized to abiraterone acetate plus prednisone compared to patients randomized to placebo plus prednisone.

There are several different methodologies to testing for lack of benefit (futility or abandoning a lost cause). One common approach is stochastic curtailment which uses the concept of conditional power. The IDMC may consider to stop a trial early due for futility if the conditional power drops below some specified level (i.e., 20%).

The decision to stop a trial early is complex, requiring a combination of both statistical and clinical judgment. Stopping a trial too late means exposing patients to an ineffective treatment, or potentially delaying some participants from receiving efficacious treatments. On the other hand, stopping a trial too early may fail to persuade others to change medical practice. Early stopping of a trial comes at a cost in terms of potential biased estimates of treatment effects, decreased precision from the original study plan and/or inability to investigate other important secondary objectives.

Purposes and Actions the IDMC

The primary responsibilities are to periodically review and evaluate the interim data with an emphasis on the safety of the patients enrolled in the trial, the overall conduct of the study, progress toward the trial’s goals, and (under certain circumstances), the efficacy of the protocol defined intervention. The protocol should include a statistical section describing the proposed plan for formal interim analysis. The plan should specify the endpoint to be monitored, the timing of all interim analyses, the expected target of deaths (for a time-to-event endpoint) and the decision rules that will be used at each analysis that will guide early termination of the trial. The GSD methodology should not be used as the sole basis for terminating a trial, but provides a framework for IDMC decision making. Based on these reviews, the IDMC would make recommendation to the industry sponsor regarding whether or not the trial should continue, be modified, or be terminated. Considerations include not only the data derived from the study, but also the relevant disease landscape and/or the population under study. It is conceivable for instance that a long running trial could become irrelevant or unethical due to changes in the standard of care.

The integrity of the study is also an important part of the IDMC purview. Such elements as data completeness and quality, compliance with the protocol, tracking the frequency and type of protocol deviations, performance at individual centers or within certain geographic regions, and compliance with recruiting goals in various populations (i.e. minorities, gender issues, etc.) are elements to consider. The IDMC should also track factors regarding the treatment landscape such as important therapeutic developments that may impact the ethics of the protocol.

Both the quality and timeliness of the data reviewed by the IDMC are always of concern. Given that both participants and sponsors may have considerable risks during the conduct of the trial, untimely reviews or poor quality data at the time of IDMC review is unacceptable. At times, additional data may be required for proper risk/benefit assessments to occur. Again, timely high quality data are essential for appropriate IDMC decision making.

The Charter

The relationship between the IDMC and the sponsors should be carefully documented and agreed upon in a “Charter” that is approved both by the IDMC and the trial sponsor before data has begun to accumulate. In particular, the IDMC recommendation to terminate a study early or make major changes to the protocol need to have a clearly defined pathway from the IDMC to the sponsor. These communications are typically conveyed directly to a senior sponsor representative given the import of such a recommendation. In some cases, the IDMC will also report to the trial’s principal investigators or the trial steering committee. Safety concerns require rapid action on the part of the sponsor and trial leadership. Safety concerns also require notification of both IRBs as well as potential modifications of informed consents. For previously consented patients, a re-consent with a modified consent form would be appropriate under these circumstances. If the trial should simply continue as planned, the communications with the sponsor can be very brief. The sponsor should not be informed of the committee’s deliberations and the ensuring that the trial results remain blinded is of paramount importance.

The IDMC conducts business in “closed sessions”. These are only attended by members of the statistical team charged with managing the data by treatment arm, and the IDMC members. If is not uncommon to have an “open session” preceding the closed meetings, as the sponsor may take that time to provide the IDMC with updates and their own perspective on the trial’s progress.

If a major modification of the trial based on interim data is suggested by the IDMC, the sponsor team will usually confer and render an opinion. The IDMC serves in an advisory role and does not itself have the capacity to stop or alter trials. Based on both considerations of the IDMC recommendation and advice, and their own assessment of the data, the sponsor will make a final decision regarding that recommendation. Sponsor disagreements with a formal IDMC recommendation are extremely rare to non-existent in our experience. At times there may extensive discussions to ensure that the recommendations are appropriately communicated and understood.

Institutional Review Boards (IRBs) have a responsibility for the safety of patients but there is a clear inability of individual IRBs to track all side effects in a comprehensive fashion in a multi-center trial. On the other hand, the IDMC has access to all the trial data on an ongoing basis. There may be many IRBs but there is only one IDMC. The relationship between IRBs and IDMCS are now well established and it is typical that IRBs will receive reports regarding the IDMC’s actions.

The Charter will typically pre-specify but the IDMC will ultimately decide on the frequency of meetings based on risk assessments and timing of data availability. Review of pre-specified interim results will typically require a planned meeting for data review and risk benefit assessment. If pre-specified endpoints per the SAP are reached, then the trial is typically terminated and the DMC discharged from duty. Most IDMC meetings may be held by teleconference. In rare cases a face to face meeting may be required if decisions are particularly critical and debate is anticipated.

The usual reason for the termination of IDMC duties is because the trial itself has terminated and there are no more interim data. If no more patients are being treated on study and only followup is being performed, then the IDMC may be disbanded if there are no safety concerns. Many sponsors request a final DMC meeting after the trial has been terminated to review the committee’s thoughts and actions during the course of the trial.

The Need for Confidentiality and Minimization of Conflicts of Interest

Trial integrity requires that members of the IDMC maintain confidentiality and not to discuss the details of the meeting elsewhere. The IDMC should maintain confidentiality around the internal discussions and should take care not to seek to inform the sponsor of interim findings that might influence the trial blinding or jeopardize its conduct. Given the highly sensitive nature of interim data, complete confidentiality is a requirement given both financial and regulatory consequences of large clinical trials in today’s environment. Those making investment decisions using confidential clinical trial data are subject to regulations and penalties by various bodies including the Security and Exchange Commission (8). All minutes kept by the IDMC during the course of the trial are confidential and not released until the trial is terminated.

The committee members should be free of both real and apparent conflicts of interest (COI). It is generally accepted that IDMC members will be compensated for travel and for time spent on the project at “usual rates”. Annual COI reporting is typically adopted for most committees to ensure that any interim COI is detected and managed.

Data Management

Results of interim analysis are provided to the IDMC typically via an independent third party statistical consultant charged both with maintaining data confidentially and with the communication of reports to the IDMC. Typically these reports aggregate both appropriate adverse event summaries and individual patient data listings and are made available prior to the scheduled IDMC meetings. The data formats are critical and “table shells” are agreed upon by the IDMC members and the independent statisticians in advance. Such “shells” will typically contain basic demographic information on screened patients, enrolled patients, protocol deviations, summaries of baseline inclusion and exclusion laboratories, interim laboratories, adverse events by grade, concomitant medications, tables and listing of serious adverse events (SAEs), time on study, reasons for the patient being removed from the study, tables for deaths on studies and an attribution of these deaths, listings of deaths on study, and the like. Graphical representations of the data and/or statistical analyses are often helpful to the IDMC in their deliberations. It is generally agreed that safety data is analyzed based on treatment received, contrary to the efficacy data. For instance, the adjustment for multiple testing (such as the Bonferroni correction) on the adverse events are not performed. Different drugs may require different matters of interest and different and data configurations.

Though IDMCS have the ability to review efficacy data at pre-specified intervals that have been planned at the design stage of the trial and are stated in the protocol and the SAP. Information typically available for discussion in open meetings (with the sponsor) includes recruitment status, as well as summary patient data (i.e., patient baseline characteristics, protocol violations/deviations, a summary pf serious adverse events, adverse events and abnormal laboratory findings, and status of patient follow-up). Unlike the sponsor, the IDMC is able to view data by randomization arm even though this is typically displayed as arm A or B rather than specified treatments in the report. Assessment of risk to the subjects may require unblinding data sets, but this is rarely the case. The IDMC may request additional data if such data are deemed necessary to assess risk and benefit or to determine whether or is not the clinical trial should continue. On occasions it may be necessary to have additional clinical expertise, perhaps in cardiology or neurology. It is best in the charter to have such arrangements established well in advance if such a need to prevent the sponsor from being aware of data that is subjected to blinding.

Practical Considerations and Early Terminations in Recent Trials

In metastatic castrate-resistant prostate cancer, IDMCs have been particularly active in terminating clinical trials after interim instead of final analyses during the last several years. Three trials^12–14 have been terminated because positive interim results; the COU-302 trial with abiraterone/prednisone, the PREVAIL trial with enzalutamide, and the ALSYMPCA trial with radium-223. Both the ALSYMPCA and PREVAIL trials were terminated because SAP pre-defined end points were reached for overall survival and the OBF boundary was crossed. The COU-302 trial was controversial because the boundary for overall survival was not crossed, but the IDMC terminated the trial earlier based on the totality of evidence for clinical benefit. The FDA carefully examined these data and rendered a favorable judgment for regulatory approval, based on a variety of positive clinical data, but in part because the drug had been previously proven to have survival benefit in patients with more advanced disease.¹⁵ With more follow-up, statistical significance was achieved for the primary endpoint overall survival in the COU 302 trial. As expected, the observed hazard ratio at the final analysis was higher (hazard ratio=0.80 for patients randomized to abiraterone acetate plus prednisone compared with patients randomized to placebo plus prednisone) than what was reported in the second interim analysis (hazard ratio=0.75). ¹⁶ In addition, several trials have been terminated early because survival was compromised including the trial with docetaxel/revlimid, the trial with docetaxel/GVAX, and the trial with docetaxel/vitamin D.^17–20

Summary

DMCs should be utilized for all randomized trials, especially trials that utilize important clinical endpoints such as survival or cancer progression. The safety of the patients and integrity of the trial are the primary focal points during interim data collection. Access to confidential data must be carefully managed by experienced and non-conflicted IDMC members whose expertise spans both clinical and statistical issues. Guidance for sponsors on these issues has been established by respected agencies, including the FDA.