Developing training for Data Safety Monitoring Board members: A National Institute of Allergy and Infectious Diseases case study

Abstract

Background

Data Safety Monitoring Boards primarily review accumulating data on clinical trials and provide recommendations to sponsors on whether a protocol should continue as planned, be modified, or be terminated. Data Safety Monitoring Boards often provide their recommendations based upon accumulating data to which only their members are given access. Despite the substantial responsibilities assumed by Data Safety Monitoring Board members, there is limited information in the literature about the unique knowledge they must possess and, consequently, the training content needs that are required in order for them to fulfill their obligations.

Purpose

This article describes how the National Institute of Allergy and Infectious Diseases identified the knowledge that Data Safety Monitoring Board members should acquire and the computer-based training they developed to address the learning needs of the National Institute of Allergy and Infectious Diseases assembled Data Safety Monitoring Board members.

Methods

The National Institute of Allergy and Infectious Diseases conducted a comprehensive literature search and interviewed Data Safety Monitoring Board subject matter experts, including Data Safety Monitoring Board members and chairs from academic institutions, pharmaceutical companies, and the National Institutes of Health to (1) assess whether Data Safety Monitoring Board training is an identified need, (2) evaluate whether Data Safety Monitoring Board training has been developed, and (3) formulate suitable learning objectives. Data Safety Monitoring Board training modules were developed based on the identified learning objectives identified from the interviews.

Results

Three Data Safety Monitoring Board training modules were developed and formatted for web-based access, which is free of charge to the public at https://dsmblearningcenter.niaid.nih.gov. The modules include the following: an introduction to the objectives and purpose of Data Safety Monitoring Boards, the organization and responsibilities of Data Safety Monitoring Boards, and a review of statistical topics.

Limitations

The complex concepts that Data Safety Monitoring Board members must apply to their deliberations and decisions require practice and application that come through hands-on experience. To build competency in the Data Safety Monitoring Board member role, not only does a member need to understand these complex concepts but also the member must have the opportunity to practice and apply this knowledge to real-life situations. Additional resources to facilitate practice and application of the complex issues that Data Safety Monitoring Boards deal with should be considered. The computer-based training is targeted to new and inexperienced Data Safety Monitoring Board members. Ongoing learning opportunities should be developed for experienced Data Safety Monitoring Board members. Non-English training must also be considered.

Conclusion

The National Institute of Allergy and Infectious Diseases identified that training is not widely available for Data Safety Monitoring Board members to build the unique knowledge and skills necessary to serve on Data Safety Monitoring Boards. Consequently, National Institute of Allergy and Infectious Diseases developed publicly available web-based Data Safety Monitoring Board training modules for new or inexperienced members. Additional tools and resources are needed to help Data Safety Monitoring Board members acquire the knowledge and skills to serve their critical function in clinical research and to maximize research participant protections.

Background

In the spring of 2008, a group of National Institute of Allergy and Infectious Diseases (NIAID) subject matter experts on data and safety monitoring from all NIAID divisions assembled to prepare a seminar titled “Sharing Best Practices: Data and Safety Monitoring.” During the preparation of this seminar, the subject matter experts raised the following issues: qualified Data Safety Monitoring Board (DSMB) members were difficult to identify¹ and recruit; no training on how to be a member or chair of a DSMB had been offered to NIAID's current DSMB members. Other than relevant experience in the conduct of clinical research,² absence of conflicts of interest,^1–3 and appropriate representation (i.e. reflective of NIAID's commitment to diversity, inclusive of professional expertise from applicable disciplines and a biostatistician), no specific knowledge sets or qualifications¹ have been identified for DSMB members.

Informal literature searches and surveys of subject matter experts in other settings such as DSMB members and chairs in other institutes within the National Institutes of Health (NIH) and the pharmaceutical industry did not yield information on any available DSMB trainings or content. The experts concluded that computer-based training for new DSMB members or chairs would facilitate DSMBs in fulfilling their important roles in safety monitoring of clinical research trials. Computer-based training was specifically identified because NIAID DSMB members are located throughout the world, travel costs are prohibitive, and participants would be unlikely to be available for travel to classroom training sessions. The development of a computer-based training for new DSMB members and chairs was proposed to both the NIAID Clinical Research Subcommittee and the NIAID Executive Committee where the concept received overwhelming support.

Methods

A training needs assessment was performed. Its primary objective was to identify the learning objectives that would enhance NIAID DSMB members’ abilities to meaningfully contribute to DSMB participation and deliberations. This project was performed through a contract with Westat (Rockville, MD). The contract had three planned components: (1) a literature review, (2) a series of semi-structured interviews with both NIAID and non-NIAID DSMB chairs, and (3) a web-based survey with NIAID DSMB chairs and members. Based on the project's results, NIAID developed a training curriculum designed for new DSMB members.

The purpose of the literature review was to determine the availability of any existing training content in the private and public sectors designed to prepare DSMB members with the indicated knowledge, skills, and abilities for role responsibilities. Articles and other references were obtained from several sources, including three bibliographies that had been created before the commencement of this literature review and a new search strategy specifically developed for this project. This strategy involved searching on terms for data safety monitoring committees, training, knowledge, competence, and qualifications. The bibliographies and database searches resulted in 278 unique records published in English. After an abstract review of these citations, 88 were selected for full text review. Information obtained was grouped into 10 primary categories of interest: (1) DSMB purpose and objective definitions;^1–5 (2) DSMB structure, function, and responsibilities;^1–5 (3) DSMB processes and procedures;^1–5 (4) general and role-specific DSMB competencies; (5) clinical concerns; (6) statistical methods and approaches;² (7) ethical and philosophical issues;² (8) regulatory considerations;² (9) legal issues;⁵ and (10) DSMB training.^1,2,5

We conducted 15 semi-structured interviews with subject matter experts who have served on DSMBs as members and/or chairs. The participants included both clinical research experts and statisticians, in addition to individuals experienced with DSMBs chartered by the US government, academic institutions, and pharmaceutical companies.

Results

Our literature search revealed that current DSMB training is typically accomplished via an informal, on-the-job mentoring method. In fact, only two formal trainings were found in our extensive literature and web searches. They included the face-to-face workshop at Johns Hopkins University in 2008 and a web-based training for DSMB members found at http://www.statistics.com/ClinicalMonitoring, targeted toward statisticians as part of an online certificate program in advanced statistical study.

Our interviews with DSMB members and chairs, as well as our findings in the literature, showed interest and support for the creation of more formal training for DSMB members. As Herson⁶ argues, formal training would allow younger professionals to serve and increase the diversity of committee membership offering “many advantages over a smaller aging power elite of [data monitoring committee] members.” Armstrong and Califf¹ suggest that formal training about DSMBs should be part of academic clinical research training programs and should include apprenticeships in which potential DSMB candidates attend meetings as non-voting members and case-based training sessions in which real and hypothetical trials are made available for study.

In an article articulating the need for a DSMB educational curriculum, authors Johnson and Milewicz note that the confidential and sensitive nature of the way DSMB tasks are conducted has resulted in a “degree of secrecy” where individuals are unfamiliar with the roles and responsibilities of a DSMB and are unaware of the working differences between a DSMB and an Institutional Review Board or a Committee for the Protection of Human Research Subjects. The authors suggest development of a DSMB program that is similar to web-based certification programs such as those used for Human Subject Protection. They propose that such a curriculum include case studies and tools suitable for group discussion with new and experienced DSMB members. Such discussions would facilitate practical insights for the conceptual topics taught and reviewed.⁷ Adding modular assessments to the training, especially via web-based/on-line certification, would serve to reinforce the instruction.

Development of curriculum content

NIAID created a web-based training program that consists of three modules.

Module 1: DSMB purpose and objectives

The first module describes the purpose and objectives of a DSMB. At the conclusion of this module, the user should be able to

• Explain the overall purpose of a DSMB;

• Describe the history of DSMB monitoring of clinical trials;

• Understand the types of clinical trials for which establishment of a DSMB is recommended;

• Describe the DSMB's scope of authority and obligations to the study sponsors and participants;

• Explain the importance of avoiding conflict of interests;

• Understand NIH and NIAID DSMB policies.

Module 2: DSMB organization and responsibilities—administrative and clinical

This module describes the organizational structure and administrative and clinical responsibilities of a DSMB. At the conclusion of this module, the user should be able to

• Describe how a DSMB is formed;

• Understand the qualifications of DSMB members;

• Explain the confidentiality requirements of DSMB members;

• Understand the components of a DSMB charter and the responsibilities of DSMB members;

• Explain the policies and procedures that a DSMB should follow when reviewing interim data;

• Understand the importance of receiving accurate and up-to-date data presentations;

• Differentiate among the various types of DSMB meeting sessions;

• Know the various topics that should be understood at the initial orientation meeting of the DSMB;

• Describe the components that should be present in a DSMB member's appointment agreement;

• Understand the various groups that may interact with the DSMB;

• Comprehend common clinical terms that relate to safety monitoring of a clinical trial;

• Understand their clinical responsibilities.

Module 3: statistical topics

The third module, intended for non-statisticians, describes common statistical discussions that may arise during a DSMB meeting. At the conclusion of this module, the user should be able to

• Understand the statistical procedures that may be presented during a DSMB meeting;

• Explain the general approach to applying statistical stopping rules;

• List three criteria that should be considered when a negative early trend in the data raises questions about stopping the trial early;

• Comprehend items that influence judgments based on short-term versus long-term effects;

• Explain the problems that can be encountered when making multiple statistical comparisons;

• Know the possible sources of bias and understand the issues associated with bias;

• Differentiate between statistical and clinical significance.

Limitations

The computer-based training offered in the NIAID DSMB Training Center is targeted to new and inexperienced DSMB members. This is consistent with the findings noted in the report from the Office of the Inspector General entitled “Data and Safety Monitoring Boards in NIH Clinical Trials: Meeting Guidance, But Facing Some Issues,” where it is stated that “many DSMB members responding to our survey noted that formal training would have benefitted them most only at the outset of their DSMB experiences to create a common understanding of the role of a DSMB.”⁸ The Office of the Inspector General report also highlights the limitations of this training, as it indicates that stakeholders noted that the best form of training is on-the-job experience. The online training does not offer practice or hands-on experience with the complex concepts that DSMB members must apply to their deliberations and decisions.

Conclusion

The web-based training modules developed by NIAID are publically available. The learning objectives are targeted to new or inexperienced DSMB members to create a common understanding of the role of a DSMB. Consistent with the Office of the Inspector General's recommendations, an online forum for DSMB members to share information and network and the inclusion of a non-voting apprentice member on DSMBs are two methods to further enhance DSMB training opportunities. These would mitigate the limitations of the NIAID DSMB training and expand the pool of experienced DSMB members. Two notable topics that should also be considered in the next iteration of the online training include the unique role and responsibilities of the DSMB chair and potential risk factors associated with DSMB membership, such as liability and potential litigation. The use of additional scenarios within the computer-based training and/or teaching scenarios made available to DSMB Chairs for use in face-to-face discussions should also be considered as additional training modalities.