An Automated Standardized System for Managing Adverse Events in Clinical Research Networks

Abstract

Multi-site clinical protocols and clinical research networks require tools to manage and monitor adverse events (AEs). To be successful, these tools must be designed to comply with applicable regulatory requirements, reflect current data standards, international directives and advances in pharmacovigilance, and be convenient and adaptable to multiple needs. We describe an Adverse Event Data Management System (AEDAMS) that is used across multiple study designs in the various clinical research networks and multi-site studies for which we provide data and technological support. Investigators enter AE data using a standardized and structured web-based data collection form. The automated AEDAMS forwards the AE information to individuals in designated roles (investigators, sponsors, Data Safety and Monitoring Boards) and manages subsequent communications in real time, as the entire reporting, review and notification is done by automatically generated emails. The system was designed to adhere to timelines and data requirements in compliance with Good Clinical Practice (International Conference on Harmonisation E6) reporting standards and US federal regulations, and can be configured to support AE management for many types of study designs and adhere to various domestic or international reporting requirements. This tool allows AEs to be collected in a standard way by multiple distributed users, facilitates accurate and timely AE reporting and reviews, and allows the centralized management of AEs. Our design justification and experience with the system are described.

1. Introduction

1.1. Collection of Adverse Events (AEs) in Clinical Research

The need to monitor and track the prompt dissemination of adverse event (AE) reports is a fundamental component of clinical research.^[1] In addition to protecting the safety of clinical research subjects, AE information drives changes in research protocols and informed consent documents, along with influencing dose administration regimens for certain clinical trial designs. Many research organizations have tried to improve upon current paper-based facsimile (FAX) reporting processes by developing automated tools for AE reporting and discovery in clinical research^[2–7] and in the broader healthcare delivery arena.^[8,9] AE reporting systems in clinical research collect reported AE information, often via the Internet, in standardized ways.^{[2,4,5,9–11]} Examples include the Vaccine Adverse Event Reporting System^[12–15] and the drug AE portions of the National Electronic Injury Surveillance System.^[16] Also, the National Institutes of Health (NIH) Genetic Modification Clinical Research Information System (GeM-CRIS), which is an information resource for human gene transfer research,^[6,7,17] includes a standardized web-based AE reporting component that some speculate could be a future standard for NIH-sponsored research. The importance of collecting AEs over multi-site protocols is underscored by the presence of dozens of commercial and internally developed^[18,19] AE reporting systems. Adverse drug event detection systems in healthcare delivery settings examine extant data for indicators of AEs,^[20–32] typically using medical chart review as a gold standard, but generally do not circulate and update information, or manage communications and updates of individual events. AE detection and reporting systems differ from AE management systems in that they generally do not circulate reported AE information to decision-makers, nor do they facilitate communication between individuals with roles and responsibilities for protocol management and patient safety.

The National Cancer Institute’s (NCI) Adverse Event Expedited Reporting System (AdEERS),^[5] used by all NCI-sponsored clinical research, is the most advanced and longest used electronic AE reporting system in clinical research. AdEERS was an early adopter of data standards, and enforced their use by requiring the coding of AEs at time of data entry by the data submitter. Unlike other (free-standing) AE reporting systems, including AdEERS, the system we describe operates within the context of automated protocol management tools that are used to collect data in a variety of study designs and disease areas, including diabetes mellitus,^[33–37] cystic fibrosis and multiple rare diseases.^[38,39] Additionally, our system design was governed by the modelling requirements that emerged from analysis of relevant system (electronic record) and AE reporting regulations and best practices.^[40–43]

Automated tools offer benefits over manual paper and FAX reporting systems, including opportunities for faster data transmission to multiple parties simultaneously, data standardization and enforcement of completeness and integrity of data with quality checks at time of entry.^[44–46] AE management system tools that facilitate electronic communication between medical reviewers, Data Safety and Monitoring Boards (DSMBs) and study investigators can increase the efficiency and value of monitoring processes. Although several electronic, Internet-based AE reporting systems exist, various local requirements, design and data elements have led to a multiplicity of systems that cannot be linked together – often within a single institution. For example, the University of Missouri conducted an assessment of patient safety activities in its campus healthcare system, and found at least six separate data systems for reporting AEs, with multiple conflicting paper reports. Authors concluded that the disparate nature of these systems drastically limited opportunities for systemic prevention activities.^[11]

Certainly, all academic medical centres have the same challenge of trying to monitor AEs on multiple investigator-initiated studies, many of which are multi-site, in different disease domains and study populations, and with different sponsors and reporting responsibilities. Electronic AE reporting systems have been built in response to the need for standardized and rapid AE reporting within an organization. One such system, the Temple (University) Adverse Event Reporting system, was developed in-house and has been in use for several years.^[19] This system collects AE data from biomedical research projects within the institution and stores these data in a central repository, which can be queried periodically for Institutional Review Board (IRB) review. The system, because it is electronic, has the ability to customize required data entry based on whether the AE is internal or external (for internal AEs, i.e. those experienced by subjects under purview of Temple IRB, more data are collected than for external AEs, i.e. those AEs occurring at other sites on a multi-site protocol). The University of Connecticut Health Center also developed and uses a web-based AE reporting system for their research protocols.^[18] The system employs data-integrity checks and enables rapid and standardized reporting of AEs across all protocols. As with the Temple system, the system must be actively queried for AE reviews, and does not have automated delivery of reported AEs nor does it manage related communications.

There are several commercial systems designed to be compliant with US FDA reporting guidelines. The four major commercial systems in terms of market share are: the Aris system from Aris Global (Stamford, CT, USA), Argus Safety from Relsys, Inc. (Irvine, CA, USA), Clintrace and Electronic Case Submission Module (ECSM) from Phase Forward (Waltham, MA, USA) and Oracle AERS from Oracle (Redwood Shores, CA, USA). All meet the FDA technical requirements for Title 21 Code of Federal Regulations (CFR) Part 11, e.g. electronic signatures, electronic records and audit trails. Many of these and other commercial systems are web-based. Some offer linkage to proprietary protocol management software but are most commonly used as free-standing AE reporting systems either by trial sponsors or multicentre clinical trial coordinating centres. Because of the variation (across sponsors and regulators) in data and timeline requirements, all of these commercial AE reporting systems are flexible in allowing additional data items for specific business requirements. Because there are no universally accepted comprehensive AE data standards at this time,^[47] existing commercial systems tend to support a variety of medical coding dictionaries, including Medical Dictionary for Regulatory Activities (MedDRA) and the WHO Drug Dictionary, as well as proprietary dictionaries.

We have developed an automated AE management system to support existing federal regulations, best pharmacovigilance practices and data standards, so that the system can be utilized to manage AEs in a standardized way in an array of settings and study designs. In this descriptive paper, we describe the regulatory, data and system requirements and final features of our automated Adverse Event Data Management System (AEDAMS), and present our experience using the system to collect and monitor communications surrounding reported AEs in a variety of interventional and non-interventional studies. While this is of primary interest to readers from academic medical institutions, we feel that the presentation of our design requirements and subsequent system description should appeal to those from industry and government, as well as those from larger academic settings.

1.2. Regulatory Requirements for AE Reporting and Electronic Records

The International Conference on Harmonisation’s (ICH) Guideline for Good Clinical Practice E6 is an international ethical and scientific quality standard for designing, conducting, recording and reporting trials that involve human subjects.^[40] The CFR Title 45 is the US regulation for all human subject research; CFR Title 21 is the codification of rules for research involving agents under the purview of the FDA.

At each stage of AE reporting (clinical site to sponsor, sponsor to all participating centres, all participating centres to local IRB/Research Ethics Committees [RECs], sponsor to FDA/The European Agency for the Evaluation of Medicinal Products [EMEA], sponsor to other applicable agencies), there are best practices and regulations that govern the expected time frame in which AEs – based on severity, expectedness, and relatedness to intervention – must be disseminated. For example, for clinical investigations conducted under an FDA investigational new drug (IND) application where the Study Chair is the sponsor (as in the phase I, II and III IND trials managed by our centre), the sponsor must report any adverse experience associated with the use of the drug, both serious and unexpected, to the FDA (and all participating investigators) within 15 calendar days of initial receipt of the information (21CFR312.32[c]). Recently, the FDA posted an online form to enable voluntary AE reporting (AEs associated with post-marketed drugs and devices) of AEs over the Internet, but holders of INDs must complete the paper form and FAX to the FDA. However, the FDA is moving toward electronic reporting for all AEs, and has developed guidelines for systems developers to become approved for automated reporting from existing information systems.

Investigations conducted within the scope of the FDA utilizing electronic records must adhere to controls, which include processes for validation of electronically captured data accuracy and reliability. The system must also possess the ability to: produce comprehensive copies of the data, in both hard copy and electronic format, for inspection and review by the FDA; provide time-stamped audit trail of all data entry, updates, modifications and deletions; and protect the records from unauthorized access (21CFR11.10[a]-[g]). Automated AE management systems can also function as electronic records: electronic case report forms (e-CRFs) can be used to securely maintain audit trails for all data entry, communications and data updates.

Although international sites do not require the reporting of AE data on observational research trials involving human subjects to research ethics boards, the US CFR does (45CFR46.103[b]^[5]). Additionally, any human subject research utilizing US Federal Funds conducted internationally must adhere to the AE reporting requirements detailed in the US CFR (45CFR46.101[a]).

For interventional clinical trials (investigational/novel agents as well as off-label research), the concept of pharmacovigilance, which is defined as the systematic review and procedures regarding the identification, evaluation and prevention of AEs or other potential drug-related problems, is of paramount importance both domestically and internationally.^[48,49] The FDA Center for Drug Evaluation and Research Good Reporting Practice encourages the use of trained healthcare practitioners (acting as Medical Monitors) to actively query researchers for AEs.^[49] Internationally and within the US, required reporting to applicable agencies is seldom done in real-time. For example, in Germany, the majority of reporting on adverse drug reactions to EudraVigilance (both through the EudraVigilance Post-Authorisation module [EVPM] and the EudraVigilance Clinical Trial module [EVCTM] as required by Directive 2001/20/EC of the EMEA) is done by the pharmaceutical industry, not healthcare professionals or site principal investigators; therefore, the German government is in the process of establishing national pharmacovigilance centres by 2010 to actively look for adverse drug reactions in the community by reviewing incidence of hospitalizations and serious diseases.^[50] However, the delays associated with collecting and reporting AEs retrospectively is not ideal.

The CIOMS, in conjunction with the WHO, has been working toward defining in greater detail methodology and data collection standards for AE management.^[51] It is the goal of CIOMS and the WHO to address the timeliness, technical and technological issues surrounding the methodology for identifying, reviewing and evaluating product safety collected during clinical trials. However, like ICH E2B and the FDA MedWatch programme, recent directives are targeted toward expedited reporting initiatives and the collection/dissemination of Suspected Unexpected Serious Adverse Reactions (SUSARs) rather than comprehensive AE data collection across all study designs.

1.3. System Requirements for AE Reporting and Management

Current regulations (21CFR Part 11) affect both the system design and drive the requirement for centralized management and procedures that monitor access to the system. In addition to allowing retrieval of items for later verification (usually by manual audit), the system design must include the saving of all data entry or change in order to enable views of changes to a given data record over time. The system itself should include security features, such as password and log-in, to identify individuals viewing, entering or altering data. The AE management system implementation procedures should include mechanisms for verifying identity and credentials of individuals before granting access to the system.

The AE data collected must be able to meet regulatory requirements in terms of content and representation. Although not all research protocols are required to report AE data to the FDA or EudraVigilance, a flexible system should use the strictest available data requirements as a minimum standard for the system. A comprehensive AE management system should also capture sufficient information about each reported AE to determine applicable reporting timelines (e.g. the FDA, ICH-E6 and the NCI use the seriousness, relatedness to intervention, and expectedness of an AE, relative to the specific protocol, as criteria for defining expected reporting timelines and managing reportable AEs). Different study designs have different AE reporting needs and timelines that the design of AE management systems should support, e.g. device versus drug studies. The system should be customized to support the notification of reported AEs to different parties within different timeframes as required by individual protocols.

A set of minimum data requirements and functionalities, regulation-driven, drove the design of our AEDAMS. General requirements included ease of reporting and the protection of subject data,^[52] and that the system incorporate and facilitate the use of existing data standards. The system should also grant varying levels of access for viewing and action permissions based on roles within the study, and be able to easily accommodate changes in staff and roles. Functional requirements include information for audits and tracking and verification of entered data, enforcement of regulatory timelines, communication and follow-up of events as per current regulations.

1.4. Data Standards for AE Reports

The FDA has asserted that data standards for AEs will increase the effectiveness of the FDA response in the interest of public safety^[53] and actively participates in the Clinical Data Interchange Standards Consortium (CDISC) and Health Level 7 (HL7) standards organizations. The FDA mandates certain required data elements for mandatory reporting (form FDA 3500A), does not mandate a standard coding system for submission of serious adverse events, but, like ICH GCP-E6, endorses the Med-DRA terminology for codifying AEs after they are reported to the agency. The FDA recently named Systematized Nomenclature of Medicine – Clinical Terms (SNOMED CT) as the standard terminology for parts of structured product labelling, and it is not clear at this time whether the FDA will name SNOMED CT as a standard for AEs or other reported safety data.^[47] The NCI, a major sponsor of clinical research in the US, requires investigators to use the Common Terminology Criteria for Adverse Events (CTCAE) classification (version 3.0), which is mapped to MedDRA codes via the National Library of Medicine’s Unified Medical Language System^[54] and by an NCI-Maintenance Support Services Organisation collaborative mapping effort, posted on the NCI’s website.^[55]

The concepts of drug safety and patient safety are undoubtedly linked. Though the FDA views AEs primarily from a drug safety point of view, FDA Good Pharmacovigilance Practices guidance for industry encourages sponsors to conduct select case history level review.^[49] A management system for reporting AE data that is already linked electronically with the affected patient’s case history (e-CRF) allows stakeholders (sponsors, investigators, review committees) to easily assess in real-time, as the FDA Pharmacovigilance guidance suggests, the occurrence of the AE in relation to the subject’s current status and clinical course. Additionally, the linkage of AE reporting tools and patient data collection can simplify the currently arduous and highly manual process of ‘AE reconciliation’, by facilitating the use of patient data from the protocol management system as ‘case histories’ for the AE reporting system.

2. System Design

2.1. Overview

The AEDAMS consists of three different components: administration and configuration of the system, AE reporting and AE review. The system can be configured by a system administrator to allow the reporting of AEs by approved research personnel and to circulate information on reported AEs to customized groups of individuals based upon the protocol, i.e. the system supports not only data collection for AEs but also the workflow associated with their tracking, management and communication. Upon the occurrence of a reportable AE, information is entered onto a standardized web-based data collection form. The severity and the expectedness (relative to each protocol or intervention) of the event determine the handling and communication of AE information (figure 1). The automated AEDAMS forwards reported AE information to a medical monitor reviewer (copying in the study chair or protocol principal investigator and other designated researchers), who will request additional information if necessary, determine causality and possibly recommend changes to the protocol or consent form as a consequence of the AE. Designated individuals are notified of the AE after the review and final causality assessment. The system automatically creates aggregate reports of all reported AEs, which can be reviewed by research staff on demand, and can be forwarded to IRBs, RECs, DSMBs and external regulatory groups as needed.

Fig. 1.

Flowchart of adverse events (AEs) handling in Adverse Event Data Management System (AEDAMS) applied to a clinical research network. DSMB = Data Safety and Monitoring Board; DTCC = Data and Technology Coordinating Center; PI = Principal Investigator.

2.2. Administration and Configuration of the System

The system is centrally administered, and users are only granted access after verification of their role on the study by the study chair. The major roles in the system are AE administrator, AE reporter and AE reviewer. Other roles determine who receives reported or reviewed AE information and when (e.g. SUSAR notification, serious AE notification and notification after review). Customized notification roles can be configured in the system to support specific AE management needs for specific protocols. Examples of specific notification roles include NIH notification and the NCI Cancer Therapy Evaluation Program (CTEP) notification, both of which can include science and project officers of sponsoring institutions. The administrator role enables the assignment of system and notification roles to other research staff and the subsequent permissions for accessing data from AE reports and reviews, based upon the requirements of various protocols. The reporter role allows those responsible research staff to report and update AE information in the system. Those designated with the reviewer role for a particular study are notified of reported AEs that they are assigned to review, able to review the reported and updated data, and able to comment on the causality and recommend changes to study procedures or termination of the study (figure 1). Those designated with the AE reviewer role essentially function as medical monitors for the study. Since the system is designed to collect and circulate AE information using these basic roles, the system can be reused across many study designs simply by the assignment of roles to individuals associated with a given protocol, and changing the roles as individual responsibilities change on the study or if a person leaves the study.

Permissions to enter, edit and view data are also controlled by the user’s role on a given study or studies. The roles, which determine who can report, edit, receive electronic notifications and view data, are assigned to research staff by the AE system administrator based upon the responsibilities specified in a Site Delegation of Authority Log approved by each study site’s principal investigator. The system includes an interface to facilitate an administrator to configure the system for individual protocols, to assign appropriate roles to research staff, and to monitor the timeliness and completeness of review and follow-up. The system automatically assigns AEs to reviewers as they are reported, but an administrator can manually assign AEs to different reviewers or change the settings for reviewers based upon their availability and schedules. The system allows the administrator to view the status of all reported and reviewed AEs, and any delays in review beyond a specified period of time can be assigned to a secondary reviewer. The administrator can easily monitor all communications related to an AE as the AEDAMS tracks the receipt of messages and replies.

2.3. AE Reporting

At the occurrence of an AE, the reporter logs into the secure system and accesses a patient’s study data. A link for reporting AEs can be followed to a standardized reporting form (figure 2). The user completes the required information and selects a ‘submit’ button and the information is forwarded. The user can also save information and can log back into the system to edit data before submitting. The form is customized to facilitate the use of the required AE data elements and to control for common data entry errors. The standard AE reporting form enables the use of CTCAE classification data standard (by including three data fields to represent the selection of a CTCAE: AE category, AE and severity). When the user selects a given CTCAE’s category in the AE category field, a customized drop-down list of AE terms specific to that category is dynamically generated for the AE event field. Similarly, the severity rating drop-down list is dynamically populated to reflect valid severity ratings and clinical descriptions specific to the AE selected. Help buttons and hyperlinks are embedded within the reporting form to link the reporter to outside CTCAE resources (from the NCI CTEP) if needed.

Fig. 2.

Adverse event reporting form.

The standardized AE report form allows users to link multiple AEs they suspect are related, to facilitate concurrent review. Reported AEs can be identified as primary or secondary, and if secondary, reporters reference the primary event. Additionally, users can update a previously entered AE by submitting a follow-up report. The follow-up report is an online form pre-populated with the data from the original AE report, and users complete or edit only those data fields requiring updates. A data field on the follow-up report captures the reason for the update: additional information as requested by reviewer or medical monitor; correction of initial report; or new information (e.g. progression of event). When editing a previously reported event, the original report is maintained, and users can view a copy of the report and make changes/updates/additions to any data field. The original data and all data changes are stored. Users can view original and updated AE reports, populated with original data and data changes for each respective update. Therefore, they can easily see the latest information on a given AE report, and visualize all changes that have been made to the record.

2.4. AE Monitoring

A distinct set of interfaces facilitates the review of AEs and is accessible only to individuals assigned the role of reviewer on a given protocol. Our system uses seriousness and expectedness of reported AEs as criteria for review as do others.^[56] Because AEs meeting specified criteria are reported by research staff, the automated AEDAMS notifies the designated reviewer via an email message with embedded hyperlinks to view the event information. All reported events for a given patient are presented together. The status of all reported AEs is clearly indicated (figure 3). Possible status categories for reported AEs include: open events (not assigned a reviewer); events awaiting review (assigned to a reviewer but not yet reviewed); and closed (event reviewed). All AEs in each of the status categories can be viewed by selecting a separate tab button. A designated medical reviewer views the event online, and completes a web-based adverse event review form (figure 4). The AEDAMS reviewer interface facilitates the medical reviewer to perform several actions: close the AE case (i.e. assign causality); request further/follow-up information; or request further discussion with DSMB members or study investigators. The reviewer’s complete review, including the assignment of causality, is immediately sent to individuals who have been designated with the ‘notification after review’ role (usually the Study Chair, site investigators, data coordinating centre, study sponsor and any others designated by the protocol or study chair). The database underlying AEDAMS maintains audit trails by storing all data and communications related to each reported AE.

Fig. 3.

Adverse event review system. The list of reported adverse events for a protocol can be viewed in three ways: open events (shown), events awaiting review, closed events.

Fig. 4.

Adverse event review form. The medical reviewer can view reported event information (top of form) and request changes to protocol or consent (bottom of form).

2.5. Notifications

Different roles on a research protocol imply different information needs. Customized notification lists provide recipients with the information that they need based upon their roles on the study. Using roles, groups of people can be automatically and immediately notified of any serious or non-serious AE reported for a given protocol, or the disposition of such events after review. Additionally, ‘special notifications’ can be generated for additional external groups, such as NIH project and science officers or pharmaceutical company representatives, as specified by a research protocol. Designated individuals can be notified of AEs as they are reported (entered into AEDAMS) and/or at the time the reviewer completes his/her AE review. These notifications can be customized for protocols with additional external reporting requirements. For example, in one protocol with joint NCI funding, automated notifications are sent from our system to the NCI CTEP expedited AE report system (AdEERS) each time an AE is reported, updated or reviewed via our system. Other external notification lists and rules can be created for new protocols as needed when AEDAMS is customized to report to other sponsoring organizations or regulatory groups (e.g. the NIH Office of Biotechnology Activities). Notifications are sent by email, and then can be forwarded to any cellular wireless data communication device (e.g. pager, PDA).

2.6. Functionalities

The features of our AEDAMS emerged from requirements related to the anticipated final data and functional needs as well as regulations governing electronic records and electronic signatures (21CFR part 11). Specific functional features of our AE reporting and management system include the following.

1. Reporters have the option to report multiple related events that are associated with a single episode (e.g. hospitalization), yet the system manages these events separately, because, for example, one hospitalization might have several associated clinical events, of which only a subset are actually treatment related. Although the data can reflect the linkage between related events, each AE can be sent to reviewers for expedited review based upon its seriousness and subsequently have a unique causality assignment. Additionally, these can be counted accordingly for periodic safety monitoring reports.

2. The study chair, site principal investigators, DSMB and/or medical reviewers specified in a research protocol are all automatically notified of reported AEs. The type and speed of notification vary based upon whether the event is serious or expected.

3. There is a mechanism to obtain more information about reported events as they emerge. Medical reviewers are able to request additional information from reporters/study investigators, and reporters are able to provide data updates (e.g. mortality status, event resolution) or corrections to the initial report. A specific field captures the reason for information changes (e.g. update, correction).

4. The system monitors all communication patterns, information updates and data changes. The system is able to track and log the receipt of messages and replies. Furthermore, the system maintains ‘views’ (in the form of multiple populated AE report forms) of original and updated data on all reported events.

5. The system can transmit reported AE information to specific parties (e.g. IRBs, DSMBs, NIH review committees).

6. AE notifications can be customized by study role (e.g. AE reviewer, study chair, site investigators) or by characteristics of the event (severity, causality).

7. The system allows the AE system administrator to assign reported AEs to designated medical reviewers. The AE system administrator also has the ability to monitor that reported AEs are reviewed in a timely manner; if necessary, events can be re-assigned to back-up reviewers.

8. The system allows viewing of reported AEs in aggregate, organized by patient, research site and other key AE descriptors, such as clinical features or aetiology, severity, seriousness or causality.

9. The system allows for safety reports, consisting of tabulated AEs, to be generated for DSMBs and other regulatory agencies as needed. These reports are automatically updated and posted to secure website monthly, and can be submitted to committees (IRB and FDA for annual reports; DSMB and internal institutional committees for progress/safety review) as required.

10. The AE reporting and management system is linked to a protocol management and data collection system, which allows protocol-specific and patient-specific data to be used by the AEDAMS without re-entry. The system is linked to the participant’s e-CRFs. The reporter or the medical monitor can easily view the participant’s case history and current clinical course.

11. The system allows the rapid configuration for new protocols, and is customizable for different studies. Additionally, user permissions based upon changes in research staff or roles can be updated in the system configuration very quickly (~minutes).

3. Utilization

The AEDAMS has been in use since 2000 in over 40 epidemiological and interventional research protocols (collectively representing over 150 sites in more than 20 countries worldwide) in diabetes, oncology, cystic fibrosis and rare diseases. To date, over 200 research staff and a dozen medical reviewers have been trained on the AEDAMS. The average length of training for using the system, including discussion of the CTCAE data standard and regulatory requirements, is 30 minutes. As of June 2007, the AEDAMS has been used to manage over 300 AEs for >1300 individuals enrolled in studies from the Rare Diseases Clinical Research Network (RDCRN)^[29,30] [the network studies opened to enrolment in 2005]. Of the 365 AEs reported to the RDCRN, the majority (75.9%) were initial reports; only 24.1% represented updated information in the form of a follow-up report. A total of 95.1% of AEs managed by the AEDAMS for the RDCRN were considered primary events, and only 4.9% were considered secondary events. (We define secondary AEs as adverse experiences that are derivative symptoms or manifestations of a primary event but which are not a progression of the primary event. Our capture of the distinction between primary and secondary nature of the events aligns with a conceptual model of incident reporting by Pronovost et al.,^[57] who note that this distinction has implications on both workflow and response.)

Of the 365 AEs reported to the AEDAMS for RDCRN studies, 13.2% (n = 48) were serious AEs and were immediately sent to medical monitors for review. For most of the serious AEs that were reviewed, the final disposition of the events, determined by medical reviewers, were that they were not related to the RDCRN protocol procedures or investigational agents (60.4%). For some (4.9%) of the reported serious AEs, the medical reviewer requested more information from the reporter. This information was reported into the AEDAMS and is maintained with the record of the AE.

The medical review officer (MRO) can recommend changes to the informed consent form or protocol documents; to date, only one MRO has recommended a change to the ICF or protocol. The site was notified that changes needed to be made within 24 hours of the AE review. On one occasion, the reporting of a particularly severe serious AE (blind-ness) prompted the MRO to assemble an immediate meeting of the full DSMB for a study, and this meeting was conducted within the week of the report date.

Although the majority of AEs reported to AEDAMS were not reviewed in real-time, all the data reported to AEDAMS are used to generate reports for DSMBs to use for study monitoring. In several protocols, the DSMB has recommended changes to protocol/ICF based on data from AEDAMS. For example, for one interventional trial, multiple reports of non-serious AEs triggered the addition of insomnia as a possible risk factor in the ICF and protocol.

Because current RDCRN protocols include a large proportion of observational studies, we see a low volume of AEs and serious AEs, although the system is scalable to support larger numbers of events that are expected as more interventional studies open to accrual in the network. The system has handled over 14 000 events (most non-serious) in a large multi-national trial in childhood diabetes of 2000 enrolled participants.^[37]

The data entered into AEDAMS are also routinely used on site audits to verify that reports have been submitted by the site in a timely manner (compared against source documents). Since September 2006, audit site visits have been successfully completed for 35 studies at 22 clinical research sites. During these audit visits, case reviews for approximately 200 research participants have been conducted. We can quickly reconcile the AEs found during the case reviews to the AEs reported in the AEDAMS and easily identify when AE data are missing or have been reported inappropriately. Our findings thus far reveal that nearly all AEs have been reported into AEDAMS.

One important consequence of AEDAMS is that all clinical research sites collect the same data in the same format. The online reporting and review forms control data entry by specifying the fields and answer formats that need to be completed. Answer sets are controlled by allowing users to select from a given set of choices in a drop-down menu. The automated AEDAMS we describe exploits data quality features^[51] of web-based data collection forms including: range and relational checks for data consistency; logical checks for data integrity; required data elements; data standards incorporated into form design; and browser tools that ease the use of data standards by enabling researchers to search CTCAE for clinical findings. The structured form can be expanded to include additional data items for the reporter to complete for protocols with additional reporting requirements. This use of structured data entry forms enables high quality and consistent data, and eliminates the need for data cleaning. Although all key variables are collected in a controlled fashion, a free-text comment box does allow reporters to specify additional information regarding the event. A major impact of AEDAMS has been the standardized reporting and coordinated management of AEs within our centre for a variety of different research study designs. Using a standardized system for AE management allows the re-use of the system, as well as related documentation and training materials.

4. Discussion

Our system design was driven to support the ICH’s E2B Data Elements for Transmission of Individual Case Safety Reports,^[41] GCP-E6^[40] and CFR Title 45, CFR Title 46 and CFR Title 21, CFR11, 50, 56, 312, 812. The system can be configured to support varying timelines and notifications and documentation as required by current regulations in human subject research and clinical trials. The system supports checks and controls on data integrity, including controlled system access and verification of study role. The use of structured entry forms ensures consistent data collection and reduces the need for data cleaning after entry. This allows the automated handing of AE information immediately after report or review, and limits the resource burden for supporting multiple protocols and growing subject enrolments.

Current regulations imply systems requirements at the data and process levels. Automated systems such as AEDAMS can enforce data requirements and improve data integrity. Additionally, we have seen that our automated system can enforce compliance to regulations regarding the scope and timelines for adverse reporting and institutional patient safety response.

Because the flow of reported and reviewed AE information is automated, and error-prone manual steps are replaced by consistent, timely and reliable routing and notification, no person needs to be involved in the day-to-day management of AEs, making the AEDAMS scalable. In our centre, the system supports over 40 protocols with just a few minutes of configuration time for each one. The cost and time for adapting the system to support the management of AEs for new protocols is minimal.

The research designs supported by the AEDAMS are varied, and include epidemiological and observational studies as well as phase I–III clinical trials. The handling of reported AEs by severity and expectedness allows the core system functionality to accommodate the basic AE management needs for different study designs, although the criteria for review and required timelines can be customized for different studies. Research protocols have varying AE reporting requirements to Scientific Review Committees (SRCs), DSMBs, local IRBs and sponsoring agencies such as the NIH. Phase I and II trials have additional regulatory reporting and system requirements (e.g. FDA, The EMEA’s EudraVigilance, GCP standards), which any multi-site or network-wide AE reporting system and underlying standards should address. The system we describe was initially designed to support a large multi-national study in diabetes, but has proven to be flexible enough to meet the needs of the variety of study designs and settings encountered in the RDCRN and other network studies.

The AEDAMS has reduced the burden of multiple AE reporting on investigators, as AEDAMS-produced reports and customized notification lists can be used by investigators to address the (various) reporting requirements of local institutions. The flexibility of AEDAMS is that it allows customization to occur on top of our base functionality as needed to address unique protocol needs, without changing the core design and mechanics of the system. Designing a flexible system that meets broad AE reporting and management needs, and which addresses applicable regulations and data standards, increases the likelihood for the use of the system for multiple studies, as well as the re-use of training and analysis resources, which arguably reduces development and training time for new protocols or research staff.

The use of the AEDAMS to coordinate standardized AE reporting enhances the ability to monitor AEs across distributed research sites. All research sites conform to AE reporting timelines specified in their protocol, and standardized reports allow clinical centre performance to be evaluated at site audits. Quality controls are embedded into the standard adverse events report form, and validity and logical checks improve data quality. Additionally, the online data entry forms facilitate the use of data standards. Systems such as AEDAMS illustrate the potential benefit of information technology applications on the efficiency of the clinical research process.^[44,58,59]

The automated nature of the AEDAMS ensures that AEs are quickly reported to multiple parties responsible for participant safety, including investigators and DSMBs. In clinical research networks and multi-site protocols, a standard approach to AE reporting and management allows AEs to be monitored and examined at a network or organizational level, which might increase the understanding of the safety of agents used across protocols. An automated approach impacts the speed and efficiency of the system and is scalable. Efficient, scalable and flexible systems for AE monitoring can impact patient safety and streamline the clinical research process to enable faster application of investigational treatments to the community. Future research should include formal evaluation of this and other AE management systems, quantitatively measuring their benefits and performance in terms efficiency, reliability, accuracy and any reduction of patient morbidity and mortality resulting from enhanced communication and decision making.